This document discusses the use of randomized controlled trials (RCTs) to evaluate development policies and programs. It notes that RCTs are useful for determining the internal validity and causal effects of specific interventions. However, RCTs also have limitations, including issues with external validity since results from one population may not generalize to others, difficulty assessing long-term effects once programs are scaled-up, and the fact that RCTs typically only evaluate one intervention at a time rather than comparing alternatives. The document concludes that while RCTs leave little room for questioning internal validity, some of their limitations regarding external validity, long-term impacts, and alternative comparisons could be addressed through better trial design.
This document discusses the use of randomized controlled trials (RCTs) to evaluate development policies and programs. It notes that RCTs are useful for determining the internal validity and causal effects of specific interventions. However, RCTs also have limitations, including issues with external validity since results from one population may not generalize to others, difficulty assessing long-term effects once programs are scaled-up, and the fact that RCTs typically only evaluate one intervention at a time rather than comparing alternatives. The document concludes that while RCTs leave little room for questioning internal validity, some of their limitations regarding external validity, long-term impacts, and alternative comparisons could be addressed through better trial design.
This document discusses the use of randomized controlled trials (RCTs) to evaluate development policies and programs. It notes that RCTs are useful for determining the internal validity and causal effects of specific interventions. However, RCTs also have limitations, including issues with external validity since results from one population may not generalize to others, difficulty assessing long-term effects once programs are scaled-up, and the fact that RCTs typically only evaluate one intervention at a time rather than comparing alternatives. The document concludes that while RCTs leave little room for questioning internal validity, some of their limitations regarding external validity, long-term impacts, and alternative comparisons could be addressed through better trial design.
This document discusses the use of randomized controlled trials (RCTs) to evaluate development policies and programs. It notes that RCTs are useful for determining the internal validity and causal effects of specific interventions. However, RCTs also have limitations, including issues with external validity since results from one population may not generalize to others, difficulty assessing long-term effects once programs are scaled-up, and the fact that RCTs typically only evaluate one intervention at a time rather than comparing alternatives. The document concludes that while RCTs leave little room for questioning internal validity, some of their limitations regarding external validity, long-term impacts, and alternative comparisons could be addressed through better trial design.
evidence on the effectiveness of polices that induce increased schooling demand or
increase healthiness. Evidence on the former may be more difficult to obtain com- pared with evaluating specific interventions that raise the quality of schooling, attract students and teachers to school, or eradicate disease. 8. We learn from many different types of evidence. Development policy progresses through the updating of priors on what works, how, and where. These chapters demonstrate that there are multitudes of ways in which our priors can be updated. For example, economists have exploited historical events—for example, colonization and techno- logical inventions; natural events—for example, twinning and disease spikes; and policy variation to assess the impacts of institutions and population and health policies on income growth. Economists are also directly inducing intervention ran- domness, rather than achieving it through imposing structure on existing data. Ran- domized controlled trials (RCTs), in which randomly selected subpopulations are selected for an intervention and then outcomes are compared across the treated and untreated populations, have been used to evaluate the causal effects of specific programs (e.g., cash transfers, subsidies to medical inputs), delivery mechanisms (e.g., kinds of financial products), and, less pervasively, to obtain evidence on fun- damental behavioral assumptions that underlie models used to justify policy—for example, adverse selection. As we learn from the chapters on aid, microfinance, health, and education, RCTs are extremely useful in some settings, because, unlike many other empirical techniques, they leave little room for questioning the internal validity of the results. But this method has a number of important limitations. First, in common with other empirical methods, the findings obtained from one population may not be generalizable to others (problems of external validity). This, of course, is just the same principle that applies to development policies—we need to know where inter- ventions work, and why. Second, in common with RCTs applied to medicines but not to other methods, if an intervention is deemed successful in the short run it is almost always then provided to the control population, and if the intervention is deemed ineffective in the short run it is abandoned, thus making it difficult to assess long-term effects of any randomized trial. For example, the original Mexican Pro- gresa program, which restricted cash transfers to a randomly chosen subpopulation, was deemed to be a success and then immediately extended to all eligible house- holds. Third, in common with the medical use of RCTs and many other methods, interventions are evaluated one at a time—there are few comparisons of alternative interventions that could achieve the same outcome within a given experiment. Some of these limitations are remediable—multiple interventions can be rando- mized, treated and nontreated populations can be followed over time, and the same set of interventions can be tried in different populations to assess external validity.