Research Methods
The study consisted of a qualitative analysis of a set of case studies selected to help illustrate how proven, cost-effective interventions have been successfully deployed and brought to scale with dramatic results in low- and middle-income countries in Africa, Asia, and Latin America and the Caribbean. We examined evidence culled from interviews, peer-reviewed articles published in journals, and official project evaluations and attempted to organize this information in a way that would allow us to reach tentative conclusions about the most significant elements associated with the interventions' success.
The study thus followed one of Mill's (1843) five methods of experimental reasoning: the method of agreement. Such a method postulates that "if two or more instances of the phenomenon (A)have only one circumstance (B) in common, the circumstance (B) in which alone all the instances agree is the cause (or effect) of the given phenomenon (A)." For this study, the phenomenon (A) is represented by success. Cases that qualified as successes had to conform to the following five criteria:
-
Scale. All cases selected for study involved a national, regional, or global scale. Pilot projects or interventions implemented on a subnational scale were not considered.
-
Importance. Selected cases addressed a problem of major public health significance that could be expressed, at the program's inception, in terms of disability-adjusted life years, a composite measure of mortality and morbidity caused by the disease.
-
Health impact. Selected cases had documented evidence of a clear and measurable effect on the health of the population targeted by the intervention. Process indicators, including immunization coverage rates, were not considered an acceptable substitute for health impact data.
-
Duration. All cases selected for study had a life span of at least five consecutive years.
-
Cost-effectiveness. Selected cases relied on interventions that had been proven to be cost-effective at a threshold of approximately US$100 per disability-adjusted life year saved.
The unabashed focus on success meant that the study ignored potentially important information about factors that may be associated with programmatic failures or not-so-successful cases that did not meet the strict criteria described above. However, the inclusion of less-than-successful cases was not an option in light of time, resource constraints, and paucity of available documentation. Thus, a significant limitation of our study results from the lack of variance in the outcome observed. This type of selection bias is a common problem that may result from the nonrandom selection of cases in qualitative research.
Although bias cannot be eliminated without expanding the study to include unsuccessful examples, working skillfully with the presumption of bias to increase the level of confidence in the findings is possible. First, counterfactual examples, even if purely speculative (what would have happened if?), can be used to further substantiate the hypothesis that circumstance (B) has directly contributed to the observed phenomenon (A). Second, theorizing in a constructive way about what the direction of the bias might be and, therefore, minimizing its impact on the results of the study are also possible. For example, any potential bias more than likely results from overdetermining causality rather than overlooking or ignoring key factors related to success. In a related point, the study design makes discerning the relative contributions of the various factors difficult, because weights cannot be assigned easily.
Remarkably few rigorous studies of this sort attempt to track the implementation of proven, cost-effective interventions in the field. Although we have a good understanding of the efficacy of the available arsenal of interventions for treating and preventing diseases specific to low-income countries, we often know little about the range of programmatic and policy options that are needed to support these interventions in the real world. This study represents the first major contribution toward the development of a body of knowledge in that area, and the preliminary conclusions reached should be understood in that context (Collier and Brady 2004).
