AGRA: Still failing Africa’s farmers

Donor-funded evaluation shows “AGRA did not meet its headline goal” to reduce hunger

On February 28, 2022, the Alliance for a Green Revolution in Africa (AGRA) released an evaluation of its recently concluded five-year strategy. The media outlet Devex, which covers international development issues, reported on the evaluation under the headline, “AGRA has failed to improve Africa's food security, report finds.” While noting the range of “mixed results” reported in the evaluation, the article highlighted the key finding of importance to all who want to understand the impacts of this billion-dollar initiative on Africa’s small-scale farmers: “AGRA did not meet its headline goal of increased incomes and food security for 9 million smallholders.”

While the evaluation is limited in its scope and its access to reliable data, that finding confirms earlier studies that have shown that there is little evidence that this 15-year investment in promoting market-driven development based on expanded use of commercial seeds and fertilizers has improved farmers yields, incomes or food security. My own 2020 study documented that yields were increasing slowly and only for a few supported crops, farmers’ incomes were not rising significantly and the number of hungry people in AGRA’s 13 focus countries had increased 31%.

That study also documented the lack of rigorous evaluations of AGRA’s work by either AGRA itself or its lead donors. The lack of accountability, combined with the poor outcomes, prompted African civil society organizations in 2021 to call on donors to withdraw support for AGRA and shift funding to more promising and effective farmer-based strategies such as agroecology.

The new evaluation is a welcome, if limited, contribution to our understanding of AGRA’s performance. Undertaken by the consulting firm Mathematica, the evaluation examines only the last five years of AGRA’s activity, which was carried out under the banner, Partnership for Inclusive Agricultural Transformation in Africa (PIATA). The evaluation was funded by AGRA’s lead donor, the Bill and Melinda Gates Foundation (BMGF) on behalf of all five PIATA donors, including the U.K. Foreign, Commonwealth & Development Office; the Rockefeller Foundation; the U.S. Agency for International Development (USAID); and Germany’s Federal Ministry for Economic Cooperation and Development. It includes a summary of findings, a statistical appendix, and AGRA’s formal responses to the findings, all available publicly.

Since the 2020 publication of my own evaluation of AGRA and the Green Revolution strategy for Africa, I have closely monitored efforts by AGRA and its donors to provide better data and analysis of its impacts. Last year I assessed AGRA’s 2020 Outcome Monitoring reports, published following a public records request to USAID, and AGRA’s subsequent 2020 Annual Report, published alongside an “Emerging Results” report offering additional impact monitoring. I found those monitoring and reporting efforts to be deeply flawed, pointing out that they largely failed to show accountability to AGRA’s original goals to double yields and incomes for 30 million smallholder families while cutting food insecurity in half.

While the new evaluation is quite narrow in scope, it at least assesses AGRA’s performance in improving yields and welfare for Africa’s small-scale farming households. Here, I examine those findings in detail, focusing on the parts of the evaluation that relate to farmer outcomes. AGRA, under its 2017-21 PIATA strategy, has focused on creating an “enabling environment” for agricultural transformation, including private sector development, government capacity-building and policy support, agricultural extension to supplement underfunded government efforts, and other initiatives.

As Devex summarizes in its coverage, “Researchers from Mathematica… found that ‘PIATA was successful in developing key policy reforms, mobilizing flagships and partnerships, and reaching farmers with extension and seeds.’ The strategy also helped ‘incentivize private sector engagement in the production and delivery of improved seeds’ in some countries.”

These are all intermediate objectives that need to be evaluated for their impacts on the larger goals of improving farmers productivity, incomes and food security. Here, I focus on those larger goals. If the intermediate objectives are being accomplished but are failing to produce the desired outcomes for farmers, after 15 years of AGRA programming, it may well be an indication that what AGRA calls its “theory of change” is fundamentally flawed.

That is indeed what the new evaluation suggests: the Green Revolution strategy is failing to achieve AGRA’s stated goal to “catalyze a farming revolution in Africa.” There is little evidence that such transformative change is taking place, which is why donors should reconsider their continued support for AGRA.

“AGRA did not meet its headline goal”

The evaluation is quite clear that when it comes to impacts on African farmers, “AGRA did not meet its headline goal of increased incomes and food security for 9 million smallholders.” Here is the evaluators’ summary statement on the disappointing contributions of AGRA’s current PIATA programs to farmers’ productivity, incomes and food security:

“PIATA’s impact on farmer-level outcomes—adoption of inputs, yields, sales, food security, and resilience—was mixed. Notably, PIATA improved maize yields in Ethiopia, Ghana, and Nigeria, but not in Tanzania, Burkina Faso, or Kenya. Across these six countries, only farmers in Burkina Faso experienced improved maize sales as a result of PIATA. These mixed results likely reflect remaining farmer constraints in access to affordable inputs and output markets, as well as low per-farmer investment levels. These findings suggest that AGRA did not meet its headline goal of increased incomes and food security for 9 million smallholders, despite reaching over 10 million smallholders through its systems development work.” (p. 2)

Let us look at those areas of weak performance one by one. Together, they highlight the continued failures of the entire Green Revolution model.

“PIATA improved maize yields in Ethiopia, Ghana, and Nigeria, but not in Tanzania, Burkina Faso, or Kenya.”

Maize is one of the crops most heavily supported by AGRA, and the Green Revolution package of seeds and fertilizers is additionally supported with generous government subsidies in all those countries. The failure to register any yield growth in half of the six countries, in the crop AGRA has most heavily supported, should be alarming to donors.

Elsewhere the data suggests maize yield increases in only two of the six countries. The evaluation is often unclear on the magnitude of the yield increases in the countries that registered improvements. In part that is because the evaluators were left to use inconsistent data from AGRA’s poor monitoring over time. Their methodology involved drawing data from three different sources: country-level U.N. data, AGRA’s Outcome Monitoring reports from 2019, and phone surveys with stakeholders which AGRA began conducting in recent years and which were the basis for its 2021 “Emerging Results” report. These represent poor sources of data for evaluators to draw on for the following reasons:

• They fail to include all AGRA countries. The new evaluation refers to only six of AGRA’s current 11 focus countries: Ethiopia, Ghana, Tanzania, Burkina Faso, Kenya and Nigeria. That leaves out Malawi, Mali, Uganda, Rwanda and Mozambique, as well as Zambia, which was part of AGRA for many years, left, but is now rejoining. Those six are not all included in the full analysis in the evaluation, leaving confusing references, such as the one above, to maize yields rising in three or two of the six countries.
• With such spotty and inconsistent coverage, the evaluation fails to present an aggregate picture of AGRA’s impact. As such, it offers no response to our findings using national-level data that AGRA has failed to catalyze a productivity revolution.
• Data focuses almost entirely on maize, with passing references to data on rice. But AGRA’s mission was to improve food crop productivity overall, not just maize and rice productivity. The evaluation thus replicates one of AGRA’s key flaws: its narrow and destructive focus on two cereal crops at the expense of other important crops such as millet, sorghum, cassava and legumes. My Staple Yield Index presented an aggregate measure of yield growth across a range of important crops and showed that AGRA’s claimed improvements in maize and rice yields were undercut by stagnant or declining yields in other crops. Overall, yield growth for that basket of staple crops was just 18% over 12 years, roughly the same as before AGRA began.
•  Nearly all of the data is from too short a time period — the last five years — to draw meaningful conclusions about yield or income trends, as I explained in an assessment of AGRA’s 2020 Annual Report. Neither AGRA nor its donors has yet provided any evaluation of the initiative’s entire history of work since 2006. That is why we did our own analysis using national data. (In fact, the Gates Foundation in the acknowledgment to the new evaluation notes the value of the 2016 evaluation of AGRA’s first 10 years commissioned by the foundation, but it has refused to make the document publicly available or share it with researchers. Donors, not just AGRA, must be faulted for the lack of transparency and accountability.)
• Evaluators are left to use a range of methodological tools to try to extract meaningful measures of AGRA’s impacts, but the results are unconvincing because they fail to assess growth over time. They show, for example, that higher maize yields and sales correlate with better access to inputs and extension services in two countries (Vol. 2, p. E32), but they do not have the data to show that this was because of AGRA’s interventions or that the correlation is not simply the product of those farmers being wealthier to begin with.
• Most evaluator mentions of yield increases fail to note the magnitude of those increases. This reflects AGRA’s current limited ambition, simply to increase yields with no clear target for how much it intends to raise productivity. Given such limited goals, it is striking that evaluators still find such scant evidence of impact.

In the end, evaluators relied on only a narrow set of AGRA-generated data, much of it flawed. The list of references used in the evaluation includes just eight documents, five AGRA studies and three methodological texts. (Vol. 1, p. 51) (They fail to list AGRA’s 2020 Outcome Monitoring country reports, though they rely on some of them in their analysis.)

To their credit, the evaluators note in the recommendations that AGRA’s mechanisms for monitoring and evaluation do not allow for meaningful impact assessment, a finding that for donors should be an important red flag. It should also be a red flag that AGRA no longer has its Outcome Monitoring reports available to the public on its website, which AGRA posted in 2021 only after a public-records request to the U.S. government revealed them.

Returning to the evaluators’ summary statement:

“Across these six countries, only farmers in Burkina Faso experienced improved maize sales as a result of PIATA.”

This means that AGRA’s core theory of change, its entire Green Revolution rationale, proved false even in two countries in which maize yields increased. In other words, rising yields failed to translate into rising incomes for farmers. And improved maize sales alone are not proof that farmer incomes have increased, since the Green Revolution package of seeds and fertilizers is more expensive for farmers.

The reasons are clear from the data presented in the evaluation. As evaluators acknowledge, even for maize, “PIATA’s farmer-facing interventions had modest impacts on farmers’ adoption of improved inputs… Overall, PIATA had a positive impact on only two of eight measures of farmers’ adoption of improved inputs across the four countries for which data were available.” (p. 40)

In other words, Green Revolution programs and subsidies failed to convince farmers to use the promoted seeds and fertilizers. As a result, according to the evaluators, “PIATA’s farmer-facing interventions led to increased productivity in two of six countries: Ethiopia and Nigeria.” (p. 40)

Using a different data source, evaluators suggest even worse outcomes. “AGRA’s farmer outcome surveys suggest that AGRA increased targeted farmers’ maize productivity in Nigeria but had no impact on maize yields among targeted farmers in Burkina Faso and Ghana, even though targeted farmers in all three countries report consistently higher access to extension relative to comparison farmers.” (p. 40)

Note, again, that there is no claim of particularly dynamic productivity growth where yields increased, an indication of just how low the bar is now set by AGRA and its donors. Where they initially set out to double productivity by 2020, and not just for maize, they now struggle to find instances where yields increased at all as a result of their interventions. Note, too, that the evaluation leaves out nearly half of AGRA’s focus countries, so donors are getting a very limited picture of the impacts of their millions of dollars in support. And that picture looks bleak for Africa’s farmers.

Returning again to evaluators’ summary conclusion:

“These findings suggest that AGRA did not meet its headline goal of increased incomes and food security for 9 million smallholders, despite reaching over 10 million smallholders through its systems development work.” (p. 2)

It is telling that in its published response to the evaluation, AGRA states: “This finding is an expected outcome and a true reflection of the realities that farmers, AGRA, and other institutions that support farmers today live with daily.”

An “expected outcome?” AGRA failed in its core mission to increase incomes and food security for small-scale farming households. And it failed to do so despite reaching one million more farmers than it had set as its goal. In fact, AGRA President Agnes Kalibata has claimed that AGRA surpassed its original goal of reaching 30 million smallholder families, directly and indirectly, reaching 44 million over AGRA’s 15-year history. Clearly, AGRA’s programs are failing to produce the kinds of outcomes it claims they will produce. In part that may be because their “expected outcomes” are so poor.

In fact, the evaluation provides scant evidence on food security in any of the countries included in the study. This reflects the limited importance given such a crucial goal by AGRA and its donors. As we documented in our 2020 assessment of AGRA, the number of severely undernourished people in AGRA’s original 13 countries increased by 31%, and that was even before the pandemic added to the suffering. We know now from United Nations data that the number of severely hungry people in Sub-Saharan Africa as a whole has increased 50% since AGRA was founded in 2006.

Recommendations identify but do not address failures

The evaluators’ recommendations include some hard-hitting critiques of AGRA’s prevailing practices, but they fail to address the underlying flaws in the Green Revolution model itself. To examine a few of the recommendations:

“Tackle critical issues facing agriculture in sub-Saharan Africa.” (p. 6) — This was supposed to be AGRA’s core mission, but 15 years on it seems to be something AGRA needs to be reminded of. Consider each of the recommendations under this point:

• “AGRA already promotes climate-smart and drought-resistant crop varieties, which is highly commendable. However, its systems development work does not appear to fully account for farmers’ poor access to irrigation and growing exposure to drought and other severe weather conditions. AGRA’s next strategy should articulate these acute challenges and make more explicit investments in improving farmers’ water use efficiency and climate resiliency.” Irrigation, more than seeds and fertilizers, was at the heart of yield gains in the first Green Revolution in India and elsewhere. Sustainable water management and climate resilience are achieved through intercropping, water catchment and carefully managed farms, not monocultures of maize and rice.
• “AGRA’s next strategy could also formally recognize that agricultural technologies and practices—such as fertilizer use and rice cultivation—can negatively impact environmental conditions and greenhouse gas (GHG) emissions.” It is alarming that in 2022 AGRA’s evaluators are just now alerting it to the negative environmental impacts of fertilizer-intensive monocrop cultivation of maize and rice.
• “In contrast, other agricultural technologies can help to reduce GHG emissions and store carbon. Cognizant of these linkages, AGRA could make more explicit investments in eco-friendly technologies and practices among smallholders.” Such “agricultural technologies” include: intercropping, cover cropping, organic fertilizer use, agro-forestry, permaculture, all consistent with agroecology, which is at odds with the Green Revolution model and inconsistent with AGRA’s approach.
• “As it develops the next set of country strategies, AGRA could also assess the environmental impact of its fertilizer recommendations, as well as the alignment of its full set of proposed investments with national pathways to net zero.” My institute has shown that the Gates Foundation and AGRA’s goal for nitrogen fertilizer use would have the equivalent climate impacts of deforesting half a million hectares of Amazon rainforest (about 1.2 million acres) every year. By contrast, the Alliance for Food Sovereignty in Africa (AFSA) has proposed the adoption of agroecology as part of African governments’ National Adaptation Plans under the climate accords.

AGRA’s response, in its formal document after the evaluation, is weak: “We must therefore rethink our models and focus our support, and that of our partners, on building resilience and adaptation specifically for smallholder farmers.”

Rethinking the Green Revolution model is indeed what AGRA’s poor results over 15 years call for. There is no indication that AGRA is prepared to stop promoting this deeply flawed agricultural development model. It remains to be seen if AGRA’s donors will take more decisive action.

Wealthier male farmers seem to be the main beneficiaries — As the evaluators note, “Farmers who adopted improved inputs and experienced yield increases were typically younger, male, and relatively wealthier. In particular, male farmers with larger dwellings, access to electricity, greater total landholdings, and lower rates of disability were more likely to adopt improved maize varieties and inorganic fertilizer, and engage with extension services.” In addition, “productivity and income gains were also concentrated among these relatively high-resource farmers.” (p. 40-41)

This finding flies in the face of AGRA’s repeated claims that it is prioritizing poor women farmers with its programs. In their recommendations, the evaluators stress this, calling on AGRA to “Expand gender and youth inclusion efforts. Only a small portion of AGRA’s portfolio features intentional diversity, equity, and inclusion programming.” (p. 6)

This finding also directly contradicts the stated goals of USAID and other bilateral donors to ensure that their assistance programs benefit and empower women.

Unlikelihood that AGRA initiatives can be sustained — Evaluators were clear that they did not see evidence that many of the initiatives AGRA launched would be self-sustaining over time without continued donor support. Donors should take this concern particularly seriously. The entire premise of the Green Revolution strategy is that yields will increase to the point that rising incomes from sales of surplus crops will not require continued government subsidies and support because farmers will have the money to invest in ongoing productivity improvements on their farms. That is not happening. Similarly, AGRA’s extension programs, based on its funded Village Based Advisors (VBAs), are identified in this and previous evaluations as unlikely to continue when donor support runs out.

As evaluators write of AGRA’s “flagship” program, one of its key partnership initiatives, “this evaluation found no evidence that governments would continue to design and execute the next generation of agriculture flagships without AGRA support.” (p. 4)

Poor monitoring and evaluation, still — AGRA’s poor track record in monitoring and evaluating its work and impacts was supposed to be addressed by the PIATA initiative in 2017. Evaluators still find deep flaws in AGRA’s systems for collecting and analyzing data. “AGRA surveys are currently not suited for rigorous impact analysis,” according to the evaluation, and they fail to cover all AGRA countries. (pp. 46-47)

Donors should pay particular attention to the flaws identified by the evaluators:

• “AGRA relies on a series of in- person farmer outcome surveys and Geopoll phone surveys to estimate the impact of its work on farmers. However, farmer outcome surveys sampled both treatment and comparison farmers in only three focus countries from 2017 to 2021…”
• “… and outcome surveys commonly feature no baseline data collection—only endline. These two factors greatly limit the utility of AGRA farmer outcome surveys for rigorous impact analysis.”
• AGRA also relies on a Geopoll phone survey to assess farmer, agro-dealer, and extension activities, needs, and outcomes. This is likely to produce unreliable data biased in favor of AGRA, according to evaluators. “The phone survey asks farmers to recall activities and outcomes from several years prior to the survey date. The Geopoll’s absence of a valid comparison group—combined with its strong reliance on farmers’ recall from previous years—is likely to produce biased estimates of AGRA’s impact.” (pp. 46-47)

“More deeply engage with and empower civil society and smallholders. AGRA should engage with civil society and smallholders more deeply and even earlier in the policymaking and flagship development process.” (p. 6) The exclusion of African farmer and civil society voices by AGRA and its donors is an endemic part of the institution’s lack of accountability. Consider:

• This evaluation, commissioned by the Gates Foundation on behalf of its partners at the Rockefeller Foundation, and U.S., U.K. and German aid agencies, failed to consult with AFSA or any of the 200 organizations that signed its open letter to AGRA donors in September 2021 calling on donors to reconsider their support for AGRA and other Green Revolution programs.
• Both AFSA and African church leaders wrote directly to the Gates Foundation in June 2021 demanding changes in these policies. Church leaders still have received no reply. AFSA got a response from the Gates Foundation in early January of this year expressing an interest in discussing AFSA’s concerns, but only after making a $40 million December grant to AGRA.
• AGRA officials in August 2020 shut off communications with African civil society groups when they demanded evidence of AGRA’s effectiveness. Even after the new evaluation, those questions remain unanswered.

Time to change course away from a failing model

AGRA gets more positive marks from evaluators for its partnerships, consortia, and support for state capacity and policy development. But success in forming partnerships, working with the private sector and supporting African governments in designing and implementing policies is only beneficial if those activities are moving things in a positive direction. By failing to examine the content of those policies and interventions, evaluators ignore the connections between AGRA’s failing outcomes for farmers and the policies that contribute to those failures.

Evaluators, for example, lauded AGRA’s “outsized role in improving the policy environment and stimulating private investment, particularly seed system reforms in Rwanda, Ghana, and Nigeria.” (p. 2) Such policy reforms, which pave the way for Green Revolution programs, have been widely criticized by farmer organizations in AGRA countries. They are actually part of the failing model, not something that necessarily deserves praise from evaluators.

As AFSA’s Million Belay wrote last year, African food producers are not in favor of these and many of the other policies being advocated by AGRA and its public and private-sector partners:

“The strategy has indebted our farmers, ruined our environment, harmed our health and undermined our seeds and culture. We object to the flurry of initiatives to amend our seed laws, biosafety standards, and institutionalize fertilizer rules and regulations that seek to entrench Africa’s overreliance on corporate agriculture.”

Donors should demand better

The poor farmer outcomes evidenced in the recent commissioned evaluation of AGRA’s recent five-year strategy should be a wake-up call for donors that have failed for too long to hold AGRA accountable to its goals. AGRA is making no significant progress toward its original goals of doubling yields and incomes for 30 million small-scale farm families while halving food insecurity by 2020. It is even failing to achieve its scaled-back goals “to increase incomes and improve food security for 30 million smallholder farm households in 11 African countries by 2021.”

The evaluation if anything confirms rather than refutes the findings from our comprehensive review of AGRA’s progress toward its goals using national-level data:

• Yields for a basket of staple crops grew just 18% over 12 years through 2018, far below the goal of doubling productivity, which would be a 100% increase. Yield growth was barely higher than before AGRA.
• There was no sign of significant increases in farmer incomes thanks to rising yields and marketable surpluses. Overall, poverty remained endemic in most AGRA countries.
• The attention to favored crops such as maize, supported by government subsidies for the purchase of Green Revolution inputs, resulted in a decline in the land and resources devoted to key staples such as millet, sorghum and sweet potato. This had negative impacts on soil fertility, as well as nutritional diversity.
• Excessive support to maize and rice contributed to unsustainable and destructive land use rather than productivity improvements, as I show in more detail in a recent study.
• Hunger rose dramatically, with the number of undernourished people increasing 31% across AGRA countries, not decreasing 50% as promised by AGRA.

AGRA’s donors should reconsider their support for such an unsuccessful and unaccountable initiative. They should shift their funding to agroecology and other low-cost, low-input systems. These approaches have shown far better results, raising yields across a range of food crops, increasing productivity over time as soil fertility improves, increasing incomes and reducing risk for farmers by cutting input costs, and improving food security and nutrition from a diverse array of crops.