Don't rely on National Surgical Quality Improvement Program (NSQIP) data to compare hospitals. The surgical outcomes don't necessarily reveal hospital quality, according to a new study published in JAMA Surgery.
Researchers, led by Robert W. Krell, M.D., of University of Michigan Health System in Ann Arbor, conducted a retrospective cohort study using 2009 data on common surgical procedures from the American College of Surgeons' NSQIP. It reviewed data on patients who underwent pancreatic resection, laparoscopic gastric bypass, lower extremity bypass, colon resection, abdominal aortic aneurysm repair and ventral hernia repair patients.
"Quality improvement platforms commonly use risk-adjusted morbidity and mortality to profile hospital performance," Krell and his colleagues wrote. "However, given small hospital caseloads and low event rates for some procedures, it is unclear whether these outcomes reliably reflect hospital performance."
Krell and his team assessed reliability for several outcomes, including severe morbidity, mortality and overall morbidity. They found that outcome reliability depended largely on how frequently an event occurred, with higher reliability for the more common outcomes.
"The combination of low caseload and low outcome rates reduces the ability of many outcomes to distinguish true quality differences among providers, which results in low reliability," the authors wrote. "The findings are cautionary to ranking systems that use observed to expected ratios as a surrogate for surgical quality," Kim F. Rhoads, M.D., of Stanford (Calif.) University School of Medicine added in an accompanying commentary.
In an era of value-based care, unreliable outcome measures can paint a hospital's outcomes quality picture inaccurately, the authors wrote. To remedy this, they suggest using 100 percent of patient information in data registries rather than data sampling.
"Eliminating sampling to achieve the highest possible caseloads, adjusting for reliability, and using advanced modeling strategies (e.g., hierarchical modeling) are necessary for clinical registries to increase their benchmarking reliability," the authors wrote.
An earlier study found the NSQIP's patient risk calculator can predict patient risk of post-surgical complications, FierceHealthcare previously reported.
To learn more:
- here's the study abstract
- here's the commentary