This post starts a series of notes from guest bloggers about papers presented at the CSAE conference 2012. Tobias Lechtenfeld (University of Göttingen) summarizes the findings of a joint paper with Stephan Klasen, Kristina Meier and Johannes Rieckmann about the evaluation of a program that provides piped water in Yemen.
While the world is still celebrating the achievement of the MDG on access to save water, very little is known about whether piped water schemes really lead to improved health outcomes. We don't want to spoil the MDG celebrations, but the health impacts seem limited. Take the World Bank's in-house evaluation department IEG: after reviewing hundreds of water projects worth tens of billions of USD from the past decade, IEG concludes that "evidence of improved water quality is rare, as are indications of the improved health of project beneficiaries".
We have been working on a large scale project in Yemen that provides household connections to piped water and sewerage in provincial towns in different regions of the country. Yemen is a very dry country where water resources are systematically overused - just like most other countries in the Middle East and North Africa.What came as a big surprise is that diarrhea and other health outcomes actually increase in the treatment group, especially among up to 5 year old children. This negative health impact appears to be caused by frequent interruptions of water supply. During interruptions pollutions builds up in the pipes, and the reduction of water pressure leads to an intrusion of pollutants. In addition, some households also begin using traditional water sources (although mixing alone does not explain why the treatment group is worse off than the control group).
The project was implemented without choice among beneficiaries, meaning that project areas are fully connected. That makes it a lot easier to use quasi-experimental methods, since we don’t need to worry about self-selection. At the same time, progress reports from the time of construction revealed selection effects that occurred during project implementation. We argue that these are exogenous to health outcomes and use them as instruments.
Construction always started in the city center, so we use ‘Distance to Center’ as first instrument. The ancient Old City was also given preference, so we include ‘Age of House’ as second instrument. In addition, the labor intensive construction avoided digging trenches in streets built on hard rock, which yields our third instrument, ‘rocky ground’. Although the results are robust to excluding the second and/or third instrument, we also provide matching estimates, which confirm the overall picture. In addition, we use baseline data from a small household survey that was conducted as part of the feasibility study of the project to calculate the Diff-in-Diff effect. While all these methods come with some caveats that are discuss in the paper, we hope that the mix of methods provides convincing evidence on the robustness of these health impacts.
So what goes wrong? We derive a number of hypotheses that provide alternative explanations of water pollution that are not directly related to piped water. We also hired a laboratory to conduct some 10,000 water quality tests in wells, pipes, water storage tanks and drinking cups. The tests include hard indicators, such as e.coli, but also rather soft measures, such as total dissolved solids (TDS). We show that much of the pollution occurs in the pipes. Nevertheless, a substantial part of water pollution also happens within households.
Several policy conclusions emerge, which we also discussed in several workshops with experts from donor agencies before the release of the full evaluation report . Most importantly, investments in piped water supply should not be made when reliable water supply cannot be guaranteed. In such cases engaging with existing networks of trucked water vendors (or designing public standpipes with chlorination devices ) might generate better health outcomes at lower costs than piped house connection. Second, water purification at point-of-use is crucial when we want access to save water to translate into better health outcomes.
Tobias Lechtenfeld is a research associate at the Chair of Development Economics, University of Göttingen, Germany.
Klasen, Stephan, Tobias Lechtenfeld, Kristina Meier and Johannes Rieckmann (2012). Benefits trickling away: The health impact of extending access to piped water and sanitation in urban Yemen. Courant Research Center Discussion Paper No. 110, University of Göttingen, Germany