Healthy Skepticism Library item: 8002

Warning: This library includes all items relevant to health product marketing that we are aware of regardless of quality. Often we do not agree with all or part of the contents.

Publication type: Journal Article

Stukel TA, Fisher ES, Wennberg DE, Alter DA, Gottlieb DJ, Vermeulen MJ.
Analysis of Observational Studies in the Presence of Treatment Selection Bias
JAMA 2007 Jan 17; 297:278-285
http://jama.ama-assn.org/cgi/content/full/297/3/278?maxtoshow=&HITS=10&hits=10&RESULTFORMAT=&fulltext=observational+studies&searchid=1&FIRSTINDEX=0&resourcetype=HWCIT

Abstract:

Context Comparisons of outcomes between patients treated and untreated in observational studies may be biased due to differences in patient prognosis between groups, often because of unobserved treatment selection biases.

Objective To compare 4 analytic methods for removing the effects of selection bias in observational studies: multivariable model risk adjustment, propensity score risk adjustment, propensity-based matching, and instrumental variable analysis.

Design, Setting, and Patients A national cohort of 122 124 patients who were elderly (aged 65-84 years), receiving Medicare, and hospitalized with acute myocardial infarction (AMI) in 1994-1995, and who were eligible for cardiac catheterization. Baseline chart reviews were taken from the Cooperative Cardiovascular Project and linked to Medicare health administrative data to provide a rich set of prognostic variables. Patients were followed up for 7 years through December 31, 2001, to assess the association between long-term survival and cardiac catheterization within 30 days of hospital admission.

Main Outcome Measure Risk-adjusted relative mortality rate using each of the analytic methods.

Results Patients who received cardiac catheterization (n = 73 238) were younger and had lower AMI severity than those who did not. After adjustment for prognostic factors by using standard statistical risk-adjustment methods, cardiac catheterization was associated with a 50% relative decrease in mortality (for multivariable model risk adjustment: adjusted relative risk [RR], 0.51; 95% confidence interval [CI], 0.50-0.52; for propensity score risk adjustment: adjusted RR, 0.54; 95% CI, 0.53-0.55; and for propensity-based matching: adjusted RR, 0.54; 95% CI, 0.52-0.56). Using regional catheterization rate as an instrument, instrumental variable analysis showed a 16% relative decrease in mortality (adjusted RR, 0.84; 95% CI, 0.79-0.90). The survival benefits of routine invasive care from randomized clinical trials are between 8% and 21%.

Conclusions Estimates of the observational association of cardiac catheterization with long-term AMI mortality are highly sensitive to analytic method. All standard risk-adjustment methods have the same limitations regarding removal of unmeasured treatment selection biases. Compared with standard modeling, instrumental variable analysis may produce less biased estimates of treatment effects, but is more suited to answering policy questions than specific clinical questions.

Author Affiliations: Institute for Clinical Evaluative Sciences, Toronto, Ontario (Drs Stukel and Alter, and Ms Vermeulen); Center for the Evaluative Clinical Sciences, Dartmouth Medical School, Hanover, NH (Drs Stukel and Fisher, and Mr Gottlieb); Department of Health Policy, Management, and Evaluation, University of Toronto, and Clinical Epidemiology and Health Care Research Program, Sunnybrook Health Sciences Centre, Toronto, Ontario (Drs Stukel and Alter); Veterans Administration Outcomes Group, White River Junction, Vt (Dr Fisher); Center for Outcomes Research and Evaluation, Maine Medical Center, Portland (Dr Wennberg); and Division of Cardiology and the Li Ka Shing Knowledge Institute of St Michael’s Hospital, Toronto Rehabilitation Institute, and Department of Medicine, University of Toronto, Toronto, Ontario (Dr Alter).