New Research on Performance Measurement

Few Primary Care Practices Large Enough for Reliable Measurement of Quality, Costs

February 23, 2010 04:50 pm News Staff

Although the federal government and other health care stakeholders have expressed interest in moving the nation's health care system toward a value-based system that relies on measuring the quality of health care delivered against the dollars spent to provide that care, new research suggests that this may not work well for primary care practices.

The study, "Relationship of Primary Care Physicians' Patient Caseload With Measurement of Quality and Cost Performance(jama.ama-assn.org)," was funded by grants from the Commonwealth Fund and the National Institute on Aging. The study suggests that few primary care practices are large enough to reliably measure quality and cost performance measures among fee-for-service Medicare patients.

"It is unlikely that individual primary care physicians annually see a sufficient number of eligible patients to produce statistically reliable performance measurements on common quality and cost measures, calling into question whether their performance can be differentiated with respect to national benchmarks," say the study authors.

The study, which appears in the Dec. 9 issue of the Journal of the American Medical Association, is significant in light of the fact that CMS has invested in programs such as the Physician Quality Reporting Initiative and numerous pay-for-performance demonstration projects around the country.

Study Specifics

For the study, researchers looked only at primary care physicians -- defined as internists, family physicians, general practitioners or geriatricians. Of special interest were study findings that showed how a practice's size affected its ability to show a desired 10 percent relative difference in ambulatory care costs and/or specific quality measures.

The researchers calculated the national mean Medicare costs for ambulatory care using reimbursement data from a 2005 Part B sample. Process measures of quality were quality indicators identified in 2005 Part B claims data -- mammography for women ages 65-69 and hemoglobin A1c testing for individuals with diabetes ages 65-75. Outcome measures of quality used also were drawn from 2005 claims data -- preventable hospitalizations associated with any of 13 adult ambulatory care-sensitive conditions, as well as hospital readmission for any reason within 30 days of discharge for congestive heart failure.

The study authors note that they chose the 10 percent marker because they thought it represented a meaningful difference and an appropriate starting point for measuring costs and quality.

After analyzing one year of caseload samplings, the researchers found that

  • virtually no practices with fewer than six physicians had enough patients to detect the desired 10 percent relative difference in costs or in any quality measure;
  • about 9 percent of practices with six to 10 physicians had a sufficient number of patients to detect a 10 percent relative difference in costs;
  • less than 3 percent of any primary care physician practices could detect 10 percent relative differences in mammography or hemoglobin A1c testing.

In addition, although nearly 50 percent of practices with 11 to 20 physicians could detect a relative 10 percent difference in costs, fewer than 30 percent of primary care practices could do so for any quality measure.

Researchers also found that more than half of practices with 21-50 physicians and all practices with more than 50 physicians had enough patients to detect 10 percent relative differences in costs, mammography and hemoglobin A1c testing. However, no physician practices had large enough caseloads to detect relative differences for preventable hospitalization or 30-day readmission after discharge for congestive heart failure.

Implications for Value-Based Purchasing

The authors conclude that about 65 percent of their 71,980 primary care physician sampling -- all Medicare participating physicians -- had insufficient numbers of beneficiaries to reliably differentiate their practices' performance from national quality and cost benchmarks.

"Only the largest primary care physician practices, which are also the most uncommon," would have sufficient numbers of patients to measure significant differences in performances, say the authors.

Without a majority of physicians with caseloads large enough to detect the 10 percent relative differences, the authors suggest formulating new approaches to performance measurement in ambulatory care.

For example, patients could be pooled from all payer sources or pooled across a variety of measures rather than for each single measure. Another suggestion is to measure performance during a two- to three-year period.

Researchers note that the medical home recently has been identified as a model of care that works well in terms of identifying an accountable entity responsible for delivering a patient's care. However, the authors note, their findings "suggest that it would be extremely difficult to surmount the limitations of performance measurement for a medical home the size of the typical primary care physician practice."

In addition, because the study indicates that practices need at least 50 or more primary care physicians to be able to make reliable measurements, the authors question whether smaller practices could be aggregated into "virtual groups of networks" of more than 50 physicians.

Finally, the researchers conclude that their study results "call into question the wisdom of pay-for-performance programs and quality reporting initiatives that focus on differentiating the value of care delivery to the Medicare population by primary care physicians."


please wait Processing