Accuracy of Data in Quality Measurements
A number of quality initiatives are in place that attempt to improve quality by not paying for poor outcomes; one obvious example being health problems that an individual attains in the course of receiving health care, for example, urinary tract infections associated with catheter placement and use. Research into the effect of a policy of not paying for such infections made an interesting side finding; that there are likely serious data quality issues that hinder accurate measurement. (Annals Article) Beginning in 2008, CMS no longer would pay for costs to treat hospital-acquired complications, including catheter-related infections, and complication rates are reported publicly. The authors looked at an all-payer database in Michigan to determine if the rates of catheter-related infections reported by hospitals on claims data was consistent with the rates identified through other means. The study design was a comparison of rates before and after the policy change as well.
The administrative claims data suggest that hospitals rarely code urinary tract infections as either hospital-acquired or catheter-associated and therefore rarely were denied payment on the basis of claims data and the CMS policy, even though other surveillance data sets suggests that there are a much larger number of such infections. As the authors put it, “the accuracy of reporting from the (claims) data set is suspect”. They further conclude that the current data set is not accurate for either the financial penalties program or for public reporting and comparison of hospital quality performance. And no change appears to have occurred in the actual rate of hospital-acquired infections, which was the whole point of the program, probably because hospitals know that their coding practices will insulate them from non-payment. Similar claims data and coding approaches are used for a number of CMS’ quality initiatives and the data is likely of similar poor quality. Assuming that these programs could have the benefit they are intended to have, if better data is not used, that benefit can’t occur.