Implementation of Programs: Lessons of Experience
The IMCI strategy was thoroughly evaluated from its onset. Only two years after the first health worker training course took place, the MCE was launched (Bryce and others 2004). MCE researchers visited 12 countries and carried out in-depth studies in five of those. The MCE and the more recent Analytic Review of IMCI (DFID and others 2003), which included visits to six countries, provide the background for this section.
Institutions and Programs
IMCI introduction was highly successful. As of December 2002, WHO's global monitoring team reported that IMCI had been introduced in 109 countries (see http://www.who.int/child-adolescent-health/overview/child_health/map12_02.jpg). Twelve countries were included in the introduction phase, in which the strategy was officially endorsed, a national IMCI coordination group was appointed, and key ministry of health staff members were trained in the IMCI clinical guidelines. Another 50 countries were in the early implementation phase, which included development of a national plan, selection of initial districts for implementation, adaptation of the IMCI clinical guidelines and materials, training of course facilitators, and planning at the district level. Finally, 47 countries were in the expansion phase, which included scaling up IMCI activities in districts already covered and expanding to cover additional districts (WHO and UNICEF 1999).
The fact that a country has adopted the IMCI strategy, however, does not mean that a high population coverage has been reached. The best available estimates of IMCI coverage are provided by the percentage of health workers who underwent IMCI training and are managing sick children. For example, the Brazil MCE (Amaral and others 2004) has shown that IMCI is being implemented in all 27 states, but in some of those states only a few health professionals were trained. In the three states selected for the evaluation because of reportedly strong IMCI implementation, there was at least one IMCI-trained health worker in 239 out of 443 municipalities (54.0 percent), but only 23 municipalities (5.2 percent) had at least 50 percent of health workers trained after three years. In Peru, also a leading country in IMCI training, approximately 10 percent of all doctors and nurses providing child care were trained after seven years of IMCI implementation (Huicho and others 2005). Therefore, levels of training coverage in most countries appear to be low.
Lessons about Implementation Success and Failure
Both the MCE (Bryce and others 2004) and the Analytic Review of IMCI that were carried out in 2002-3 (DFID and others 2003) confirm that IMCI has been highly successful in motivating managers and health workers. The training program is highly regarded, and trainees are pleased with its logical, consistent approach to child health problems. Innovative clinical skills, such as the use of palmar pallor to diagnose anemia and the use of breathing rate for pneumonia, are often praised. Nutritional counseling, an area in which most health workers receive little formal training in school, is also greatly appreciated. When asked about the limitations of IMCI, health workers often mention the increased time required for a consultation and the difficulty of following the IMCI guidelines when there is a high patient load.
Several studies have shown that health workers trained in IMCI do perform better than those not trained. Health facility surveys carried out in Tanzania (Schellenberg and others 2003), Brazil (Amaral and others 2004), and Uganda show that IMCI training substantially improves health worker performance in assessing and managing sick children, and in counseling their caretakers. Box 63.2 summarizes MCE findings on antibiotic prescribing patterns, a critical area for managing sick children (Gouws and others 2004).
Important constraints to IMCI implementation were also identified through visits to 17 countries by the MCE and Analytic Review teams. Using the framework developed by the Commission on Macroeconomics and Health (Hanson and others 2001), the teams described shortcomings in three areas: community and household issues, health service delivery issues, and issues related to health sector policy and strategic management.
Community and Household Issues
Coverage levels for effective interventions to improve child survival are remarkably low in most developing countries. A review of the 42 countries that account for 90 percent of global child deaths showed that only two out of nine key interventions reached more than half of all children (Bryce and others 2003). This finding agrees with those of the MCE, showing that the third component of IMCI—improving family and community practices—was poorly implemented. At the global level, UNICEF was primarily in charge of developing this component (see http://www.childinfo.org/eddb/imci/practices.htm), and at the country level, UNICEF often acted through nongovernmental organizations (NGOs). Coverage with these community-based programs tended to be patchy. In Peru (Huicho and others 2005) and Tanzania, the districts that were selected for implementation of the community component were not the same as those prioritized for health worker training, which were chosen by the ministry of health with WHO support. This precluded any possible synergy at the district level between improved quality of care in health facilities and community interventions, including those aimed at improving care seeking and compliance with health workers' advice.
All the countries that were visited, however, have a number of programs and projects that deliver child survival interventions at the community level. Many of these interventions are part of the key IMCI family practices, but they are being delivered in an uncoordinated manner by national, international, and nongovernmental organizations in limited geographical areas. The low population coverage of these projects makes it unlikely that they will ever result in a substantial effect on a larger scale. The notable exceptions are the EPI programs, which, despite some recent evidence of falling coverage (Bryce and others 2003), still reach the vast majority of children in developing countries.
On the basis of the experience obtained in these countries, it appears that those key family practices that are most likely to be synergistic with facility-based IMCI—improved care seeking, home management of disease, and compliance with health worker advice—are among those least likely to be supported by existing programs. Existing programs seem to favor biological interventions such as vaccines, micronutrient supplementation, and insecticide-treated materials.
The present criticism of community IMCI should not be extrapolated to community-level child health interventions in general, which can often be highly successful. These interventions are covered in chapter 56.
Health Service Delivery Issues
Given the difficulties in implementing the community component, IMCI was largely restricted in nearly all countries to training health workers in the improved management of care for young children. Even there, some difficulties were apparent.
In countries such as Peru, Brazil, and Uganda, after an initial sharp increase in the number of health workers who were trained, budgetary and other restrictions led to a decrease in the number of training courses being offered. In Peru, about 10 percent of all eligible health workers in the public sector were trained after seven years of implementation. At the current rate of training, several decades will be needed before full coverage is reached (Huicho and others 2005). Similar results were observed in Uganda (J. Nsungwa-Sabiti, personal communication).
Staff turnover is also a major problem. In Peru, between 1996 and 2001, 43 percent of IMCI-trained health workers had already been rotated since their training (Huicho and others 2005). In Tanzania, where staffing patterns appear to be quite stable in comparison with the situation in other countries, 23 percent of trained staff had moved within three years of initial training (C. Mbuyia, personal communication). Problems with turnover were also observed in Bolivia, Brazil, and Niger. These health workers did not necessarily leave government employment, but high rotation means that IMCI may not be continually delivered to the same target population over time.
Another relevant issue mentioned in several countries was that of low staff motivation, which was often associated with low salary levels. In Uganda, the performance of health workers fell dramatically in 2001 after the government discontinued cost-sharing schemes that were used to supplement drug supplies and health worker salaries at the facility level (Burnham and others 2004). In Cambodia and Tanzania, salary levels are so low that health workers need other sources of income to maintain their families. Issues related to human resources are addressed in greater detail in chapter 71.
Poor supervision was a major issue in all countries that were visited. IMCI recommends regular supervisory visits that should include systematic observation and feedback on case management. In Peru, the average number of supervisory visits was 0.19 per facility per year (Huicho and others 2005). In Bangladesh, a baseline (pre-IMCI) health facility survey conducted in 2000 found that none of the facilities in the study area had received a supervisory visit, including observation of case management, within the previous six months (S. E. Arifeen, personal communication). Common reasons given by health workers for erratic supervision activities are shortages of vehicles, fuel, and staff members.
Problems with referral were also common. The Urgent Referral category in figure 63.1 requires immediate referral to a hospital. In several countries (Bangladesh, Cambodia, Niger, Uganda, and Tanzania), it was reported that children in this category are often not taken to a hospital because of distance or lack of funds for travel and hospitalization-related expenses. For example, in a Tanzania survey, only 5 of 13 children who had been referred were actually taken to a hospital (Schellenberg and others 2003). Also, in some countries hospital staff members who had not been trained in IMCI were reluctant to admit children with danger signs identified through the IMCI algorithm. This situation highlights the need for reinforcing training of referral-level health workers using IMCI guidelines.
Another important limitation, observed in Bangladesh, Cambodia, Niger, and Uganda, is the low use of public sector health care for a variety of reasons (accessibility, official or under-the-table user fees, perceived poor quality, lack of drugs, and so on). For example, using Ministry of Health documents in Niger, the authors estimated that the average annual number of attendances by children under age five was 0.5. In Bangladesh, only 8 percent of children who were ill were taken to a qualified provider (S. E. Arifeen, personal communication). In the presence of such low utilization rates, it is unlikely that health worker training can have an effect on mortality rates, unless simultaneous community activities improve care-seeking practices.
Although equipment and vaccines that are needed for IMCI delivery were available in most countries visited, availability of drug supplies varied from country to country. Shortages were reported in Cambodia and Zambia, and other countries, such as Peru and Tanzania, reported that essential IMCI drugs were mostly available.
Health Sector Policy and Strategic Management Issues
Several of the problems described in the preceding section are directly related to health sector issues. In addition, issues related to higher-level policy and management may also represent constraints to successful IMCI implementation.
In some countries, IMCI was not fully institutionalized at national or subnational levels. For example, a national coordinator was not appointed or was appointed on a part-time basis. In Peru, IMCI was implemented side by side with CDD and ARI programs, which it was expected to replace, and in several districts the ARI coordinator's tasks were expanded to also encompass IMCI. In several countries, IMCI activities did not have a separate budget line, or they were not included in district health plans, or neither. A report on the Analytic Review of IMCI (DFID and others 2003, 39) states, "IMCI was generally introduced as a strategy, not as a program. If this was not a barrier in the pilot phase, it seemed to generate problems for rapid scaling up. In five of the six Analytic Review countries, IMCI focal persons did not have the rank or the responsibility of previous disease specific program managers within their Ministry of Health, and IMCI did not have a budget line and a strong management structure." The report also argued that decentralization, as part of health sector reform, reduced managerial capacity at the central level and had, at least in the short term, a negative effect on IMCI implementation.
Conflict between IMCI guidelines and existing policies and regulations was present in some countries, particularly the former Soviet republics of Kazakhstan and the Kyrgyz Republic, where policies for hospital admission—requiring, for example, that all children with diarrhea be hospitalized—were in conflict with IMCI guidelines. Another regrettable example comes from Brazil, where both doctors and nurses were being trained in IMCI until medical associations threatened legal action to prevent nurses from being trained in using antibiotics for life-threatening conditions. This obstruction succeeded despite an MCE health facility survey that showed that IMCI-trained nurses performed as well as doctors in managing sick children (Amaral and others 2004). In Morocco, IMCI-trained nurses are also unable to prescribe antibiotics because of central regulations.
A particular challenge came to light when the MCE team visited Cambodia and Niger and the Analytic Review team visited Mali. These countries have high levels of under-five mortality and thus the greatest need for IMCI. They also have weak health systems and low utilization rates and are therefore having difficulty implementing IMCI successfully. Just as for individuals, the inverse care law—which suggests that those who most need high quality care are the least likely to get it—seems to also apply to countries (Hart 1971).
However, there is a possibility that IMCI (or other approaches to managing sick children) might help strengthen selected health system functions through specific approaches, as the Analytic Review team observed in regard to drug and commodity availability, service management, and health worker motivation through IMCI in the Arab Republic of Egypt (DFID and others 2003).
Implications for Health System Development
The first component of IMCI, which involves training of health care workers, has been implemented in many developing countries and has resulted in important improvements in the quality of care delivered to children in first-level facilities in limited geographic areas. The potential population-level effect of IMCI case-management training has not been realized, however, for three reasons:
-
Sufficient resources were not available for full implementation.
-
Few health systems in low-income countries are capable of providing the policy, personnel, and managerial support needed to expand and sustain high levels of IMCI training coverage.
-
At the time of this writing, not one country had succeeded in mounting a behavior-change program capable of improving care seeking, home management of illness, and nutrition-related practices to coverage levels that will result in population-level changes in service utilization or health status.
One implication for health system development is that support should be continued and expanded for integrated case management in first-level facilities as an essential component of an effective child survival strategy. A second implication, however, is that greatly expanded efforts must be directed simultaneously to the development of new and innovative approaches to strengthening health systems and to reaching families and communities with known and affordable child survival interventions.
An important distinction can be made between interventions and delivery strategies (Bryce and others 2003). The same intervention (vitamin A capsules, perhaps) can be delivered through different strategies—for example, to children attending health facilities, on National Immunization Days, or directly at the household level through community networks. In spite of its community component—which in most countries has not been operational anyway—IMCI, as implemented to mid 2004, relies on health facilities as its key delivery strategy.
The first component of the IMCI strategy—a focus on improving health worker skills—was innovative to the extent that it provided clear technical guidelines and yet required country-level adaptation. Similar levels of technical clarity and country-level flexibility did not exist for the second and third components of the IMCI strategy, which focused on improving health systems to support IMCI and improving family and community practices. Within these two components, the IMCI strategy has been criticized for attempting to become a uniform global strategy, with guidelines for implementation that do not allow room for country-level modifications, especially the incremental approaches to implementation needed by weak health systems (Bryce and others 2003). The first component of IMCI can serve as a model for the types of development work that must now move forward in the health systems and family practice areas; however, in these areas, key decisions about how best to deliver interventions will need to be made at the country level and below.
One example of progress is that WHO and its partners have now developed a process for assessing country-level opportunities and requirements for achieving population-level behavior change in relation to key family practices and for developing feasible and collaborative work plans for effectively implementing child health activities at the community level (A. Bartlett, personal communication).
As the MCE and Analytic Review have shown, IMCI requires a functional health system with managerial capacity; an ability to train health workers and to keep them on the job; an efficient means of supplying drugs, vaccines, and equipment; and the capacity to maintain regular supervisory activities. It also requires appropriate care-seeking practices, leading to a reasonable level of health services utilization by children under age five. In most countries, appropriate health services utilization is unlikely to be achieved without strong family and household-level interventions such as those promoted by community IMCI.
These problems, however, are not specific to IMCI; they affect every other delivery strategy that relies heavily on health facilities, including the predecessors of IMCI, namely the CDD and ARI programs. In fact, at least in theory, the efficiency gains represented by the integration promoted by IMCI should make it easier for developing countries to implement as the key child health strategy, so a return to vertical programs is not the answer.
Given these difficulties, however, there may be a temptation to bypass health services altogether in the poorest countries by promoting the delivery of child health interventions directly to families and households. There are successful examples of such community delivery schemes—for example, projects dispensing antimalarials (Pagnoni and others 1997; Kidane and Morrow 2000) and antibiotics for pneumonia (Sazawal and Black 1992). This approach may, in fact, be the most viable short-term solution for countries with weak health services, but it should not be forgotten that most success stories represent small-scale pilot projects with strong managerial backup. In countries with weak systems, the managerial support for implementing and sustaining high-quality, community-based interventions is also likely to be lacking, so it may be naive to assume that such programs will have the effects that health services have failed to deliver. Also, just as first-level health facility care depends on referral services for backup, community delivery schemes will require operational first-level health facilities to handle complications and treatment failures.
There is no substitute for strengthening health systems in the poorest countries. In the long run, strengthening these systems will be the key intervention for reducing child mortality as well as for promoting healthy growth and development. Delivery strategies that reach communities either directly or through other mechanisms are needed in the short term, but the long-term goal of improving health services is paramount.
