Methods of Assessing the Value of Health Research Projects
Although the aggregate value of health research has unquestionably been large, not all health research projects are sufficiently valuable to justify their costs. Some of these latter projects are inevitable because the outcome of research is intrinsically unpredictable. However, even if one abstracts from such uncertainty by considering the expected returns from a project before one embarks on it, research projects clearly vary in their likely value relative to their cost. Such assessments are obviously difficult to make, yet they are routinely done as part of the process of deciding the allocation of research funds. These assessments have traditionally been done rather informally, even in the most closely structured settings in which research funding is sought, such as the NIH study sections in the United States that review research proposals for NIH funding.
An active discussion over the past decade has helped to identify valuable strategies for priority setting in global health research and determine their implications for research priorities in the field. The 2004 report on health research of the GFHR summarized the major contributions to this discussion (GFHR 2003). The approaches that are described date back to as early as 1990 with the work of the Commission on Health Research for Development. They include the essential national research approach developed by the Council on Health Research for Development and approaches developed by the Ad Hoc Committee on Health Research, the Advisory Committee on Health Research, and the GFHR's own Global Forum combined matrix approach. The 2004 report compares these approaches along several dimensions that include the following:
-
Objective of priority setting
-
Focus at global or national level
-
Strategies or principles (especially relating to the process for participation by stakeholders)
-
Criteria for priority setting:
-
Burden of disease
-
Analysis of determinants of disease burden
-
Cost-effectiveness of interventions (resulting from proposed research)
-
Effect on equity and social justice
-
Ethical, political, social, and cultural acceptability
-
Probability of finding a solution
-
Scientific quality of research proposed
-
Feasibility (availability of human resources, funding, facilities)
-
-
Contribution to capacity strengthening.
In addition to describing the dimensions considered in these approaches to priority setting, the 2004 report also summarizes several efforts to apply these approaches in individual countries or for individual diseases or disease areas (see, for example, Remme and others 2002). The priorities identified include many of those identified in chapter 4 in this volume.
Examining these dimensions of priority setting, one finds that some are intrinsically qualitative in nature but quantitative analysis also sometimes plays an important role. For example, several approaches, including the GFHR's combined matrix approach, use DALYs to quantify the number of healthy years of life lost to the disease toward which the research is directed.
Looking forward, one may find additional quantitative approaches helpful in assessing the value of research. For example, new methods based on value-of-information (VOI) techniques have begun to be proposed to better inform such assessments. In essence, VOI techniques model the uncertainty in the outcomes of research and the value of research contingent on that uncertainty in order to assess the expected value of the research. For example, a research project that has a 5 percent chance of success that it would produce health gains worth US$100 million and a 95 percent chance of failure, would have an expected value of (0.05 xUS$100 million) + (0.95 xUS$0) = US$5 million. Although these techniques have rich roots in statistical and economic theory, they are often difficult to apply well because of the difficulty in valuing health outcomes and modeling the uncertain outcomes of research. As a result, these methods often can provide only loose bounds on the value of a research project (Meltzer 2001). Indeed, in the worst-case scenarios, in which the available information about the likely outcomes of research is essentially uninformative, VOI techniques only reproduce information on the burden and costs of illness. Nevertheless, in settings where more structure can be put into the problem, these techniques have begun to be applied successfully to specific research questions, such as the prioritization of research in Alzheimer's disease (Claxton and others 2001). In general, the more remote the connection of the research to health outcomes, the harder it is to gain meaningful information from a VOI approach. Thus VOI approaches are far better at providing information on the value of applied research than they are at providing information on the value of basic research.
For lower-income countries, this ability to illustrate the value of applied work may be especially valuable because often a technology has already been developed and the question is how best to apply it in a given setting. With the limited funds that have traditionally been available for research in lower- income countries and the high value that appears to be placed on improvements in health in these countries, VOI studies, as discussed previously, seem likely to frequently show that the benefits of research in lower-income countries far exceed the costs. It is hoped that such findings may be used to increase the pool of funds available for research in these settings. However, even if VOI techniques do not lead to increases in the total funding available for research in low-income countries, they may be helpful in informing resource allocation decisions, given available resources. In the context of a fixed research budget in which some research projects whose expected benefit exceeds their costs could still not be funded, rigorous data on the high rates of return on projects that did not make the funding line would then serve to highlight the returns to greater research in these settings. Because VOI calculations need not be complex (see box 7.1), the most important barriers to their useful application to research in developing countries are the lack of awareness and acceptance of these methods on the part of researchers and funders focused on these settings. Organizations concerned with international health can exercise valuable international leadership by exposing these researchers and funders to these methods and encouraging their use.
[Box 7.1]
