An independent lens to gauge impact
Learn about ourExternal Evaluation
For all AgResults projects, the External Evaluator determines on a scientific basis if each Pay-for-Results (PfR) prize competition achieves its objectives. Abt Associates and IDinsight, which jointly serve in this role, apply the following approaches in their work:
Abt Associates and IDinsight split evaluation responsibilities for the different AgResults projects: While Abt Associates leads evaluation for the Brucellosis and Tanzania Dairy Challenge Projects, IDinsight leads evaluation for the FMD Vaccine and Senegal Crop Storage Finance Challenge Projects. Abt Associates also served as the External Evaluator for the now-closed projects in Kenya, Nigeria, Uganda, Vietnam, and Zambia.
AgResults uses incentives to overcome market failures for specific agricultural technologies. The prize competition model, which is a sub-class of Pay-for-Results approaches, incentivizes the private sector to work towards pre-defined results through rewards that are paid out only if the results are achieved. AgResults projects use prize competition incentives to encourage key private sector actors’ involvement in the market for a technology, either by developing the technology or encouraging its adoption by poor households.
The failure of a market to develop can be caused by low supply, low demand, a weak policy environment, or a combination of these. This diagram shows some examples:
The Evaluator is charged with assessing whether the prize competition (what was previously called pull mechanism) incentives address the key underlying market failures and engage the private sector in the market while leading to smallholder farmer impact as implied in the evaluation questions.
The Evaluator employs a common evaluation across all AgResults prize competitions, applying economic theory to examine how each incentivizes the private sector to overcome the market failure and improve smallholder farmers' welfare. It also identifies behaviors of market actors that might impede or aid development impact on smallholders farmers.
The common evaluation framework allows for comparison across AgResults' prize competitions and generalization of results and lessons learned. In all AgResults project evaluations, the Evaluator applies rigorous qualitative and quantitative data gathering and analytic methods to test economic theory-driven hypotheses. The diagram below displays how the Evaluator answers all evaluation questions in its charge.
To inform each Challenge project-specific evaluation design, conduct an initial qualitative assessment to assess the project context, form a hypothesis of expected impact, examine the expected heterogeneity of impacts, identify external factors that might inhibit or boost impact, and meet with key stakeholders to secure their cooperation and buy-in.
During the baseline stage, gather information on market for the technology, its adoption, initial engagement of potential competitors, as well as smallholder farmers' livelihoods and agricultural practices.
During the endline stage, gather information on competitor activities, development of the technology market, and smallholder farmer adoption and income gains.
At the end of AgResults, synthesize the findings across Challenge projects to draw lessons on the design and implementation of prize competitions. Assess AgResults’ cost effectiveness in achieving key outcomes across projects and also as compared to traditional approaches.
For key projects, evaluate the sustainability of AgResults’ impact by assessing private sector engagement in technology provision and the continuation of favorable development outcomes two years after the project ends.
Find out more about the Learning Library.
