Despite the hype, randomized control trials don’t help much in framing policies to redress poverty

The Nobel Prize for economics this year has been awarded to Abhijit Banerjee, Esther Duflo and Michael Kremer for “their experimental approach to alleviating global poverty". The economists advocated the use of randomized control trials (RCTs) to understand the impact of micro-interventions on poverty alleviation. This has reignited the debate on the use of evidence in policymaking.

Drawing from medical sciences, RCTs have been in use in economics for more than two decades. The debate on them revolved largely around three aspects: their approach, methodology and impact on policy. The approach of using RCTs to address pressing development problems is perhaps the most contested. This involves “treating" a section of population with some micro-interventions and then examining the impact against a control group, which is similar in all aspects, except treatment. By design, these experiments are only as good as the nature of the interventions, and they typically examine micro aspects of behaviour—although not all aspects. Such interventions may be a useful tool for policymakers to understand the efficacy of a particular step, but are unlikely to provide solutions to some of the vexed issues of poverty, which is caused by multiple factors rooted in social, political and economic structures. Any claim, therefore, that these can solve global poverty not only exaggerates, but also shows an inaccurate understanding of why poverty persists and how societies that manage to reduce it do so. The approach holds—though not so explicitly—individuals or households responsible for their poverty, rather than the socio-economic and political context.

Similarly, the methodological superiority of the evidence collected is limited to drawing an inference on the benefits of the treatment. Further, the control groups are rarely similar on social, political and institutional aspects, despite the best possible design and implementation. It also isn’t possible to assess the impact of such interventions on parameters of individual welfare, since some of these may not be quantifiable. In addition, replication of an experiment in another context may produce completely different results, limiting their generalization.

But the real issue is the impact of such experiments on policy. While there is certainly an argument for the use of better evidence in policymaking, what is also important is the very nature of the intervention. Some policy instruments may be amenable to experiments, while a large majority may not; particularly those that seek to alter the structure of production and distribution that drive growth, and, in turn, help reduce poverty. Macroeconomic policies, which have contributed to large-scale poverty reduction in countries such as China and India, have hardly been influenced by any RCT. Even large-scale social protection interventions such as the National Rural Employment Guarantee programme or the Mid-Day Meal scheme have emerged from political struggles, rather than RCT-based evidence. But despite the exaggerated claims on their usefulness to policy, RCTs may have some relevance in improving the design of some schemes.

RCTs have helped start debates on some policy interventions. The evidence generated through such experiments on micro-finance, the quality of education and access to health services does suggest the need for better informed policies. The mad rush to view micro-finance as a panacea for poverty, for example, needs to be nuanced with evidence, which shows no impact on the poverty or income of recipients.

The need for better information and evidence for policy is regardless of which instrument is used. Perhaps multiple instruments and methodologies ought to be adopted. In fact, there is a long history in India of using large-scale secondary data as well as micro studies—such as village surveys conducted by agro-economic research centres—to guide government policy. A good example of this is the debate that led up to the passage of the National Food Security Act, in which large-scale National Sample Surveys were used together with micro experiments for a better design and approach to food security. Also, some of the insights came from evidence based on the experience of state governments and grassroot activists.

At a time when India’s economy is slowing down and all evidence points to an increase in deprivation, unemployment and hunger, data and evidence are crucial in not just understanding the extent of poverty, but also finding the pathway to lift millions out of it.

Himanshu is associate professor at Jawaharlal Nehru University and visiting fellow at the Centre de Sciences Humaines, New Delhi

Log in to our website to save your bookmarks. It'll just take a moment.

Your session has expired, please login again.

You are now subscribed to our newsletters. In case you can’t find any email from our side, please check the spam folder.