Pop Quiz: What do a treadle pump and an aspirin have in common? If you are like most normal people, you probably answered “not much” (although an enterprising sales rep offering pumps on credit might tell a potential customer, “Take two and call me in the morning”).
But for a group of researchers gathered in Washington DC last week for the Agricultural Technology Adoption Initiative (ATAI), we will not make real headway with the introduction of pumps or other agricultural technologies until we test them as rigorously as we would a medical treatment: something like an aspirin for poverty.
On January 21, I joined researchers (from Berkeley, MIT and Yale), practitioners (KickStart, Technoserve and other like-minded organizations) and funders (Bill & Melinda Gates Foundation, USAID among others) at the ATAI meeting to discuss key questions facing groups working to increase the adoption of agricultural technologies – and how we might get better answers through rigorous research. The discussion focused on conducting randomized control trials to test various approaches.
The randomized control trial has formed the backbone of medical research for decades. If you really want to know that a particular therapy works, you need to give one group of randomly selected patients the drug and compare the results with another randomly selected group (the “controls”). Without the random selection, you don’t really know if there was something special about the group of drug takers that made them more likely to get better anyway.
We face the same challenges. We can compare a group of treadle pump users to a group of non users. But what if the people who buy treadle pumps are more entrepreneurial – and better farmers in general – than those who do not buy pumps. Differences between the two groups might then be simply because one group (the pump buyers) happens to be better at business and farming than the other group – and not because of anything to do with the pump.
In order to address this, the researchers at ATAI want to conduct randomized control trials in agriculture. One plenary presentation highlighted work already being done to test Technoserve’s agronomic support to coffee farmers in Tanzania using randomization. (For more about the use of randomized control trials to test social interventions in developing countries, watch this TED Talk by researcher Esther Duflo from the Jameel Poverty Action Lab at MIT)
What are the potential benefits of this approach?
1. Credibility: In many circles, randomized control trials are fast becoming viewed as the ‘gold standard’ for establishing the effectiveness of a given intervention. If you want to prove to the world that your program is truly effective, there is more and more pressure to demonstrate that using a randomized trial. Being able to point to randomized control trial results would be a huge boost to our credibility in the eyes of many influential groups.
2. Insight: The approach does have the potential to challenge our assumptions and generate new insights for our program. We stand to learn new things about what works and what does not work – and we may be surprised by the results. In general, the approach works best where you have a readily identifiable and well-defined offering (or ‘treatment’ in the language of the researchers – a nod to the method’s medical roots).
There are several challenges. First, randomization does not stand on the sidelines and observe the play. Instead, we need to construct the intervention to ensure that the key offerings of interest (training, marketing, etc.) are randomly assigned. This means telling field staff that they are to go into villages A, B and C to conduct their demonstrations, but that they must steer clear of control villages X, Y and Z for the time being to avoid ‘contaminating’ the results. For country directors facing stiff targets, any such restriction on activities is a tall order (and would likely require us to either ask the donor to relax the targets for the sake of the research or increase the funding to allow us to expand our operating area and thus hit our targets despite the randomization). The payoff of course is insight that allows us to be more effective and efficient in future.
In my view, we are much more open to the idea of randomization where we have
1) a reasonable level of uncertainty about the effectiveness of an offering (e.g. is finance package A better than finance package B?)
and/or
2) limited funding to fully roll out an offering (e.g. where we only have enough resources to finance 50% of eligible applicants, we could randomly assign finance to a subset of villages or individuals, since we can’t cover everyone anyhow). Also, we are much more open to the idea of randomizing the timing of an offering (group A gets financing this year and group B next year) versus the eligibility for an offering (group A gets financing and group B not at all).
In partnership with a group from the Institute for Fiscal Studies in London UK, we are currently exploring a randomization in Ethiopia to test the effectiveness of finance packages and approaches to technology demonstration. Watch this space for updates on this project and our evolving experience with a randomized control trial.
And if you get a headache just thinking about this?
Take two aspirin and call me in the morning.
Like this:
Tags: agricultural technology, ATAI, IDE, Institute for Fiscal Studies, Poverty Action Lab, randomized control trial, RCT
