Internal evaluations carried out by the Alliance for a Green Revolution in Africa (AGRA), recently obtained by the U.S. Right to Know (USRTK) campaign through Freedom of Information Act (FOIA) requests, show no new evidence that AGRA is making progress improving yields, incomes and food security for African farmers. In fact, the reports, which AGRA had previously declined to share with researchers, suggest that AGRA is only now beginning to track progress for its current programming period while providing no evidence from its first 10 years of work.
AGRA was founded in 2006 with the goals of doubling yields and incomes for 30 million small-scale farmer households while halving food insecurity by 2020. Our 2020 report, “False Promises,” documented limited progress across AGRA’s 13 focus countries in achieving those goals. For AGRA countries as a group, yield gains for just 18% for staple crops from 2006-2018, well below the goal of doubling productivity (a 100% increase). We found few signs of income gains, and the number of undernourished people increased 30%.
In the course of that research, AGRA first promised then refused to provide data from its own progress evaluations of impacts on its beneficiaries. After the July 2020 publication of our report, AGRA refused to respond to my requests for evidence to refute our findings and declined an explicit request from three African organizations, two of which were involved in the research, to provide evidence of impacts. (See my October 2020 update for details.) They received a courteous but not very informative reply, one that offered very little data. Anne Maina of the Biodiversity and Biosafety Organization of Kenya, speaking for the three organizations, responded:
“In our letter, we posed a concrete set of questions about impacts and invited you to present evidence to counter our report's findings that yield growth is slow and uneven across staple crops, that farmer incomes are not rising much as a result, and that food security has not improved and may have gotten worse with the decline in crop and diet diversity. We would be interested in engaging in a public, evidence-based dialogue on these issues if AGRA is willing to present evidence. Perhaps a good place to start would be the ‘outcomes panel surveys’ you refer to and the mid-term evaluation of your 2017-2021 strategy.”
AGRA refused to provide those reports. USRTK obtained the documents last month through FOIA requests to the U.S. Agency for International Development, one of AGRA’s key funders. The mid-term evaluation and the 11 country outcomes surveys are now available from USRTK. (AGRA recently published these documents, though at the time of this publication AGRA’s web pages for the mid-term evaluation and the 11 country outcomes surveys were not responsive.)
As the lead researcher who provided the background research for the “False Promises” report, I was asked by USRTK to review and assess the documents to determine if they provide new data on AGRA’s progress in improving yields, incomes and food security. Unfortunately, they do not. In a cumulative 1,365 pages of reports, these commissioned evaluations:
- focus only on AGRA’s current 2017-2021 programs, providing no data on the first 10 years of AGRA’s work;
- provide some data on beneficiaries but only as a “baseline” against which to compare data from two future surveys;
- as such, they provide no data at all on AGRA’s impacts over time on farmers’ yields, incomes or food security.
Following are brief summaries of the limited information on AGRA’s top-line goals in the Outcome Monitoring Reports and the Mid-Term Evaluation.
Outcomes Monitoring Reports
This series of 11 reports has the publication date of April 30, 2020, though they were posted only in the last three months. Each covers one of AGRA’s current focus countries using a consistent methodology, carried out by consultants from KIT Royal Tropical Institute in the Netherlands. The evaluations were carried out in 2018 and mostly refer to the 2018 cropping year. They vary in length from 58 to 171 pages. Four of the 11 (Uganda, Kenya, Ethiopia and Malawi) do not include household surveys of producers. Seven of the 11 (Ghana, Nigeria, Tanzania, Rwanda, Burkina Faso, Mozambique and Mali) include two extensive household surveys of roughly 1,000 AGRA beneficiary farmers growing an AGRA priority crop of maize, rice, soybeans or cowpeas. Each report includes an evaluation of progress on Small and Medium Enterprise (SME) development, an AGRA priority. Each also includes a “Qualitative Systems Analysis” of AGRA’s initiatives in promoting policy reform and market development to expand farmer access to input markets, training in input use, access to markets and reducing post-harvest losses.
Because our interest is assessing AGRA’s progress in achieving its top-line goals of improving yields, incomes and food security, I focused my review mainly on the seven country reports with household surveys of beneficiaries, looking in those surveys for evidence of progress. I did not examine closely the assessments of systems or SME development. These are the limited results I found:
No evidence of impacts: In 1,210 pages of documentation I could not find a single piece of evidence showing the impacts AGRA’s interventions had on its beneficiary farmers’ yields, incomes or food security. That is because the surveys make no reference to AGRA’s first 10 years of programming, report no outcomes from that earlier period and state that the current household surveys will establish a baseline from which to evaluate progress. The absence of such baselines and the failure, in 2020 as AGRA reached its self-declared deadline for progress, to report any progress at all on its top-line goals is the most striking finding from reviewing these reports. One of the main findings of our “False Promises” report was the lack of accountability from AGRA and its funders, and these reports confirm that failing.
Continued lack of accountability: AGRA claims it is simply reporting on progress in implementation of its current 2017-21 strategy, which shifted the organization’s operational structures if not its primary objectives of promoting the adoption of Green Revolution seeds, fertilizers and pesticides in expanding market interactions. This would be justified if AGRA or its donors separately documented progress in its first 10 years of work in increasing yields, incomes and food security. Remarkably, they have not, though the mid-term evaluation makes reference to a comprehensive evaluation of AGRA commissioned in 2016 by the Bill and Melinda Gates Foundation. To date, neither AGRA nor BMGF has agreed to make the document available.
Poor yields even with high technology adoption: For maize farmers, AGRA often documents relatively high levels of adoption of promoted Green Revolution practices, yet reported maize yields remain quite poor. In no country did maize yields exceed 3 tons/hectare. For example:
- In Ghana, beneficiary farmers had good land endowments (3.5 ha on average), relatively high levels of technology adoption (58%) and used moderately large quantities of fertilizer (31 kg/ha). Yet average maize yields were just 0.58 tons/ha. This suggests that Green Revolution technologies are not producing the promised yield impacts. Household revenues from maize sales were just $35 on average for the year.
- In Tanzania, moderately well-off farmers with decent land endowments showed 82% technology adoption, used on average 52 kg/ha of fertilizer — above the African Union target of 50 kg/ha — but got mediocre yields of just 1.3 tons/ha on average, with the vast majority of AGRA farmers getting well below 1.0 tons/ha.
Limited income gains from sales of crops: AGRA’s “theory of change” is that small-scale farming households will increase crop yields through adoption of Green Revolution inputs, which will lead to increased sales and increased incomes. Incomes from sales of maize appear to be very low, according to the surveys. For example, in Tanzania evaluators estimated the average income from maize sales at $77 per household for the year. How low is that?
- Tanzania’s "basic needs" poverty line is about $190 per person per year, approximately $950 for a household of five people. For a poor Tanzanian farming household at the poverty line, that $77 is just 8% of a poverty income. On a per capita basis, it is just $.04/day from maize sales. Not from increased maize sales, from total maize sales.
- The metric does not account for the added costs of inputs, which can easily exceed the added income from sales.
- Because AGRA failed to establish prior baselines for yields or incomes, the report promises to estimate “incremental income from maize sales due to AGRA” but then can offer only total revenues from sales.
AGRA impacts often limited compared to other interventions: As we noted in the “False Promises” report, AGRA contributions are dwarfed by other programs, particularly input-subsidy schemes funded by African governments. This is evident in the Outcome Monitoring Reports. For example, the report on Ethiopia, which in our report showed some of the strongest yield gains, acknowledges that “The support of AGRA is considered relatively small, which does not give it much clout to trigger major sector transformation and reform.” Similarly, the Kenya report states: “The volume of grassroots activities by AGRA in Kenya, which are directly aiming at productivity increase among farming households, is limited.”
Flawed methodology for gauging impacts: AGRA’s current plan is to carry out three such surveys by the end of 2021. Those results can be compared to the baseline studies reported here to assess progress. But four years is far too short a time period to gauge impacts, with seasonal variations making it difficult to draw reliable conclusions from the data. In my research for “False Promises,” I observed the common practice of smoothing seasonal variations by using three-year averages instead of a single year of data. Our pre-AGRA baseline was 2004-6, our end point was 2016-18, the latest data available. If AGRA fails to include the results from its first 10 years of work, its promised 2021 impact report will cover too short a time period and be too distorted by seasonal variations to be meaningful.
To summarize and reiterate: the main finding from reviewing AGRA’s Outcome Monitoring Reports is that they fail to report any progress at all on the initiative’s top-line goals of increasing yields, incomes and food security.
Mid-term Evaluation of AGRA’s 2017-21 Strategy Implementation
Carried out by ITAD consultants for AGRA’s donors, the January 2020 evaluation offers a mid-course assessment of AGRA’s implementation of the strategy it adopted for the 2017-21 period. As with the Outcomes Monitoring Reports, it is largely silent on AGRA’s progress in achieving its top-line goals. It limits its scope to implementation efforts under the new strategy with no accounting for work under AGRA’s first 10 years. In fact, the evaluators do not list any documents in their references that reports on that earlier period. They include a summary of the BMGF-commissioned institutional evaluation of AGRA, but the document is not available for public review. They draw on the Outcomes Surveys and come to similarly limited conclusions.
That said, they take a more evaluative approach to their task, which is perhaps unintentionally revealing. They suggest that AGRA is:
- failing to monitor its progress effectively on top-line goals like yield; in fact, the word appears only eight times in the entire 155-page document;
- failing to reach as many farmers as it claims; for example, “AGRA has only reached 2 million out of 4.6 million (44%) of the 2019 target for the ‘number of farmers and value chain actors participating in AGRA supported extension services’”;
- the change in strategy isn't producing much of a change in outcomes;
- progress is not sustainable as it is dependent on AGRA funding and subsidies: “Most of the farmer-level results achieved to date appear to be unsustainable as they result from either the direct delivery of products and services to farmers or models with questionable incentive structures.”
Overall, the evaluators seem quite critical, suggesting that AGRA is failing to show that its interventions can or will lead to yield or income gains. The evaluators state that this may indicate a potentially flawed theory of change. In the section of their conclusion in which they separate their own observations from those agreed with AGRA staff, they write:
“AGRA would benefit from reconsidering and strengthening its intervention logic…. This includes interrogating the assumptions and evidence that their inputs and outputs will result in expected outcomes and impacts such as inclusive agricultural transformation, given the risks and opportunities within their operating contexts.”
As our “False Promises” report suggested, AGRA should indeed question its “theory of change,” recognizing that promoting the Green Revolution technology package to Africa’s small-scale farmers will not achieve “inclusive agricultural transformation” that significantly increases yields, incomes and food security.
Unfortunately for Africa’s farmers, the one area in which evaluators praised AGRA was work with governments to change policies and regulations. An appendix lists these policy initiatives, many of which relate to reforming seed policies to increase the access of commercial seed providers to African markets. In “False Promises,” the authors were clear that AGRA’s policy initiatives may indeed be its most important — and pernicious — interventions.
