This Article Submit a related Letter to the Editor Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Request Copyright Permission Citing Articles Citing Articles via Google Scholar Google Scholar Articles by Gregg, E. W. Articles by Narayan, K. M. V. Search for Related Content PubMed PubMed Citation Articles by Gregg, E. W. Articles by Narayan, K. M. V.

Evaluating Diabetes Health Services Interventions: True Effects, Changing Tides, or Moving Targets?

Michael M. Engelgau Division of Diabetes Translation National Center for Chronic Disease Prevention and Health Promotion Centers for Disease Control and Prevention Atlanta, Georgia 30341

Rubin et al. (1) recently reported improved quality of diabetes care and reduced health care utilization and costs after a comprehensive diabetes management program was implemented in managed care settings. Considering the ongoing structural changes in American health care and the acknowledged deficiencies and high costs of diabetes care, it is refreshing to see evaluation of system-level interventions such as the "NetCare" system, which involves complex coordination of health care providers, patients, and health services. It is also noteworthy that this study evaluated both the clinical effectiveness and the economic consequences of such an intervention.

However, a more cautious interpretation of the results of this study is warranted, and the "take-home message" of this report—that the NetCare system improves outcomes and reduces costs—is not justified by the study design and data presented. In particular, the absence of an appropriate control population of people with diabetes or historical (prebaseline) trend data makes it difficult to separate effects of the intervention from those due to secular changes in clinical practice and health care utilization. In fact, the reported decrease in inpatient costs and average length of stay mirror 5-yr downward trends in average length of stay and hospital discharges, increases in physician contacts, and major shifts from inpatient to outpatient services (2, 3). Previous articles have pointed out that even short-term intervention effects may be highly variable and confounded by secular trends (4). Thus, health care costs, which are largely driven by inpatient utilization, may have been decreasing for several years, and much of the favorable results of the NetCare evaluation may have coincided with these trends. Presentation of historical data, ideally with an application of formal time series analyses, would help separate already occurring trends from those induced by this intervention. The authors do present a comparison of relative cost changes for nondiabetic members. However, this comparison group likely uses a vastly different range of services and absolute dollar cost than the diabetic population, making this comparison relatively uninformative and perhaps misleading.

The lack of inclusion of NetCare-related costs also makes it difficult to interpret the true changes in costs associated with this program. As the authors point out, an advantage of a coordinated case-management driven system is that the nurse case manager is able to help influence risk factors and manage resource use, potentially averting unnecessary inpatient and outpatient utilization. In essence, the nurse case manager becomes a component of the health care system, which is "utilized" in place of other services. Thus, excluding NetCare-related costs from the total utilization costs essentially results in a less complete accounting of utilization at follow-up than at baseline, which naturally leads to less costs.

It is less likely that secular trends would have accounted for the impressive improvements in the percent obtaining foot, eye, and glycosylated hemoglobin exams observed in this study. However, the particularly low baseline levels (2% receiving foot exams; 23% receiving eye exams) relative to national estimates (61% for foot and eye exams, followed by dramatic increases, raise questions about the reliability of measurement of these parameters (5). Could the NetCare system itself have led to more reliable recording of these tests by health care providers? If true, this would suggest an advantage of electronic medical records, but would also represent a measurement bias wherein part of the improvement in these outcomes was simply due to better accounting. A greater concern is that the authors seem to have used person-time as the denominator for computing the proportion of diabetic subjects receiving preventive services [see Fig. 1, Rubin et al. (1)]. If true, is it possible that the 42% increase in member years from baseline to follow-up means that a longer time period is available for patients to receive appropriate treatment during follow-up than during baseline, making comparison of these time periods invalid for these outcomes? If the increase in member years is due instead primarily to an increase in number of members rather than months of follow-up, adding so many new members raises questions about whether there were population-related differences (e.g., age, demographics, case-mix, and comorbidity) that may explain some of the improvements in quality indicators.

The problems we have raised are not easily overcome. It is often impractical to find appropriate comparison populations for broad and complex system-level interventions such as this. Such evaluations are also complicated by the diverse set of health care services used, variations in population characteristics, and the difficulty in reliably measuring quality of care using electronic data. Ultimately, the goal of this evaluation—to determine whether a coordinated system of care works better than former approaches—may be achieved through enhanced collaboration among HMOs with differing systems and by careful examination of the populations and trends in diabetes care. Meanwhile, we urge careful consideration of the issues we have raised when interpreting findings from Rubin et al. and from other observational studies of diabetes health services and utilization.

Footnotes

Received September 29, 1998. Address correspondence to: Edward W. Gregg, Division of Diabetes Translation, Centers for Disease Control and Prevention, Mailstop K-68, 4770 Buford Highway NE, Atlanta, GA, 30341.

References

1. Rubin RJ, Dietrich KA, Hawk AD. 1998 Clinical and economic impact of implementing a comprehensive diabetes management program in managed care. J Clin Endocrinol Metab. 83:2635–2642.[Abstract/Free Full Text]
2. Centers for Disease Control and Prevention. 1997 Diabetes Surveillance 1997. Atlanta, Georgia, U.S. Department of Health and Human Services.
3. American Diabetes Association. 1998 Economic consequences of diabetes mellitus in the U.S. in 1997. Diabetes Care. 21:296–309.[Abstract]
4. Soumerai SB, Ross-Degnan D, Fortess EE, Abelson J. 1993 A critical analysis of studies of state drug reimbursement policies: research in need of discipline. Milbank Q. 71:217–252.[CrossRef][Medline]
5. Beckles GLA, Engelgau MM, Narayan KMV, Herman WH, Aubert RE, Williamson DF. 1998 Population-based assessment of the level of care among adults with diabetes in the U.S. Diabetes Care. 21:1432–1438.[Abstract]

This Article Submit a related Letter to the Editor Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Request Copyright Permission Citing Articles Citing Articles via Google Scholar Google Scholar Articles by Gregg, E. W. Articles by Narayan, K. M. V. Search for Related Content PubMed PubMed Citation Articles by Gregg, E. W. Articles by Narayan, K. M. V.