More and Less Comprehensive Data and Analysis
Several authors (Drummond and others 1997; Gold and others 1996; Sloan 1995) provide similar guidance and recommendations for relatively comprehensive economic evaluation in general or for specific medical procedures. This volume aims at estimating cost-effectiveness for interventions against many different problems in all low- and middle-income regions, for which varying amounts and quality of information are available. It has therefore not always been possible to conduct as complete an analysis as would be desirable. Some degree of modeling is usually inescapable (Buxton and others 1997).
More complete analysis starts by characterizing, in each regional setting where an intervention is relevant (where the health problem causes some measurable burden and the intervention appears feasible), estimates of the quantities of inputs required (Q), the unit costs of those inputs (P), and the effectiveness or health gain (E). Authors were provided regional estimates of unit costs for the major inputs—salaries, facility costs, fuel and vehicle operation, drugs, representative equipment, diagnostic tests, and buildings (Mulligan and others 2003). The total cost of delivery is the sum of the input costs PQ, which is compared with effectiveness E and the CER calculated from the total costs and total effects of the proposed intervention or from the changes in those costs and outcomes compared with current practice.
The data on unit costs, quantities, and outcomes may all derive from published literature; what is original is how that information is combined to calculate cost-effectiveness rather than taking the ratios from existing studies. Estimates are built up using prices and physical inputs in the chapters on tuberculosis (chapter 16), vaccine-preventable diseases (chapter 20), malaria (chapter 21), cancers (chapter 29), psychiatric disorders (chapter 31), neurological disorders (chapter 32), cardiovascular disease (chapter 33), hemoglobin disorders (chapter 34), water and sanitation (chapter 41), indoor air pollution (chapter 42), tobacco (chapter 46), alcohol (chapter 47), community programs (chapter 56), family planning (chapter 57), surgery (chapter 67), emergency care (chapter 68), and complementary medicine (chapter 69). Several chapters analyze some interventions more fully and others less fully, depending on the available information.
As indicated in box 15.2, expanding or contracting the scale of an intervention may change the CER because of difficulty in reaching more of the population. The ratio may also vary because of the cost of identifying who would benefit most from the intervention—for example, whether to screen all newborns for sickle cell disease or only those of African origin (chapter 34). And expansion may change the cost-effectiveness because it would require considerable fixed investment. The costs of expanding capacity to deliver an intervention, including physical capital and training of human resources, should be amortized over a reasonable interval (10 years is the standard in this volume) and included in the total costs. Ideally, one would know the complete production function of the intervention, including the possibilities of substituting one input for another to minimize costs in response to differences in prices. However, analysis of this level of complexity is difficult to achieve, so most chapters assume fixed input proportions. Q, then, does not depend on P, and the CER varies (at most) only with coverage, prices, and outcomes. This result could be an underestimate of the true cost-effectiveness if much substitution is possible (see chapter 16 on tuberculosis).
Approximations are required when the average and incremental CERs have to be taken directly from the literature and when key parameter values are not easily available. Existing estimates of total cost or effectiveness may or may not incorporate the standard assumptions about discounting, disability weights, and life expectancy. Authors then need to judge how to adjust the available estimates for a more consistent analysis.
Local cost and outcome estimates that have not been constructed transparently from inputs and prices provide a less complete basis for secondary analysis. Where such estimates are used, information about how costs are constructed or how results vary with the scale of the intervention is usually not available, but the data may explicitly show regional differences in one or both elements, thereby permitting regionally differentiated recommendations (or may show differences so small that recommendations need not differ regionally). If costs and results refer to only one moment or are specified year by year, they can be discounted at 3 percent. Published analyses often use higher constant rates of 5, 6, or even 10 percent and may specify only total costs and outcomes rather than the respective streams through time. In that case, both costs and health gains occurring in the future are valued less, but conversion to a CER based on 3 percent discounting may be impossible and is at best only approximate. Some published analyses discount costs but not health outcomes, which makes interventions look more cost-effective when costs are spread over long intervals (for examples, see chapter 29). Imported estimates of cost and effect—that is, estimates from other regions, commonly from high-income countries—are often all that is available. Sometimes data on costs and outcomes derive from the same source; in other cases, they come from different sources and even different regions and are difficult to compare directly. More appropriate adjustments to total costs are possible with information on quantities of inputs. In the absence of data on quantities of resources used, differences in average cost can sometimes be calculated using estimates of input proportions. Such an approximation characterizes the analysis for diabetes (chapter 30), in which proportions were known in one region and assumed to be the same elsewhere and costs were estimated from regional cost ratios.
Variation of Results and Uncertainty of Estimates
Variation and uncertainty are two different aspects of cost-effectiveness estimates that also need to be accounted for in setting priorities. Because costs of inputs differ among regions, intervention costs vary even if effectiveness does not—and there are often reasons why the same intervention is more effective in one place than another. Such variation means that a single estimate of incremental or average cost-effectiveness of an intervention is not universally applicable. All estimates should ideally be local, and regional values capture only part of the real variation. For example, the average cost per DALY of chemotherapy for active or contagious tuberculosis, in the absence of HIV and AIDS, is US$15, but that figure varies from US$6 to US$31 across regions, and such wide variation is common in many chapters. Whenever the estimates of cost-effectiveness in different chapters use the same input prices, their results are comparable within a given region. Analyses that draw on published estimates for price or unit cost information are necessarily less comparable across interventions and introduce another element of variation, even in the same locale. Still more variation arises when costs or outcomes are extrapolated from one country or region to another.
Because the CER depends on many parameters and variables, of which only the discount rate and the disability weights are uniform, good analytic practice calls for sensitivity analysis to see how the ACERs and ICERs change with plausible variation in one or more parameters. Many chapters (such as chapter 26) provide such analyses, varying one value at a time, to sketch the likely range of estimates. This method is one way of dealing with uncertainty (which differs from real, known variation) about the true values of the data and seeing whether the ranking of interventions changes when those values change. Such analyses do not indicate the probability that the true CER falls in a particular interval, only under what input values it would do so. Estimating such probabilities requires knowing or assuming the statistical distributions of the parameters in question and using that information to derive confidence intervals around the point estimates. Guides to CEA recommend these approaches (Gold and others 1996), and the National Institute for Clinical Excellence requires probabilistic sensitivity analysis before approving medical treatments in the United Kingdom (NICE 2004).
Data for estimating probability distributions around mean parameter estimates are seldom available in low- and middle-income countries. Simply having available several different estimates of a parameter is inadequate for deriving a distribution, because the differences may be caused by variation in regional costs or expected life years rather than uncertainty. However, assumptions about the shape of distributions can be applied within modeling exercises to give an indication of the likely distribution of ICERs. Only a few chapters, therefore, include confidence intervals. The analyses for tuberculosis (chapter 16) and malaria (chapter 21) do, but ranges associated with most cost-effectiveness estimates (see chapter 2) reflect other causes of variation, not statistical accuracy.
Although calculations are often reported to several significant digits, such precision is not really feasible given the uncertainties in the original data: "economics is a one- or at most a two-digit science" (Morgenstern 1963). However, even crude findings can be valuable, either as guides to value for money if inaccuracies do not affect the relative order of magnitude of the results or for understanding and exploring the sources of variation and their effect on priorities as well as indicating future research needs (Claxton, Sculpher, and Drummond 2002). These issues arise, for example, when considering whether to expand the EPI or to add new antigens (chapter 20), how far to extend screening procedures (chapters 29 and 34), and when to change drugs in response to vector or parasite resistance (see chapters 21 and 23).
The quality and relevance of evidence can vary considerably, depending on whether information comes from randomized controlled trials or systematic overviews, nonrandomized studies with multivariate analyses and well-defined endpoints, or case studies or expert opinion. For these analyses, the quality of evidence also depends on geographic coverage, as distinguished in chapter 2:
-
literature review of one cost-effectiveness study, in one country
-
literature review of several studies in different countries in different regions
-
literature review of several studies in different countries in the same region
-
original analyses starting with price and quantity data in one country
-
original analyses starting with price and quantity in one or more regions.
The first three categories differ in how representative published findings are; the latter two categories differ according to the data used in constructing total effects and total costs.
Besides the quality of the evidence at its source, how the results will apply to other settings matters, particularly when the data are limited to high-income countries. The more that outcomes depend on underlying biology, the more the findings will apply to low- and middle-income countries. Outcomes depending more on cultural or environmental factors are less readily transferred and require judgment and evidence as to their applicability elsewhere. Sometimes the only detailed studies refer to high-income countries, as for abuse of substances other than alcohol and tobacco (chapter 48). At the other extreme, in a few cases all or nearly all the information comes from low- and middle-income countries, and there is no need to extrapolate, as for nutritional interventions (chapter 28) and community health and nutrition programs (chapter 56).
