Clinical registry sampling inaccurate for measuring hospital efficiency

The use of clinical data registries to measure hospital performance is inaccurate, particularly because in most cases, such tools determine efficiency via sampling rather than looking at 100 percent of cases, according to new research published this week in JAMA Surgery.

The study, led by surgeons from the University of Michigan Health System in Ann Arbor, Mich., analyzed complication and mortality data for six common surgical procedures reported to the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) in 2009. It involved data on 55,466 patients who underwent colon resection, pancreatic resection, laparoscopic gastric bypass, ventral hernia repair, abdominal aortic aneurysm repair or lower extremity bypass.

The program found that few hospitals met data thresholds for reliability, which it defined as "performance variation explained by true quality differences." Much of that could be attributed to low caseload in these areas.

To that end, the researchers urged the elimination of sampling, and instead called for collecting information on all cases to gain a more complete picture.

Groups such as Healthgrades, which last year analyzed Medicare outcomes data for 40 million patient records, tout the statistical backing of their rankings. Yet, Texas County Memorial Hospital, for one, has called out ratings from Leapfrog Group, saying it used incomplete and incorrect data from inappropriate sources.

Meanwhile, Medicare's Hospital Compare website is designed to help consumers compare hospitals, based on some of the 100 quality measures it collects.

A Government Accountability Office report published in December determined that the U.S. Department of Health & Human Services needs to require more evidence of improved quality and efficiency of care for Medicare patients in implementing its program for clinical data registries. the report advised drawing on expert judgment to monitor qualified CDRs and include, if feasible, key data elements needed by qualified CDRs in its requirements under the EHR incentive programs.

The American College of Physicians last September listed clinical quality measures reporting among the reasons Meaningful Use Stage 2 will be burdensome for physicians.

To learn more:
- find the article