Value is a critical concept in economics and philosophy. Economics is a discipline concerned with resource allocation informed by the value placed on alternative uses of those resources. In welfarist economics, value is taken as the strength of preference for a good or service (Brouwer et al., 2008). In that paradigm, strength of preference can be measured as the amount of money people are willing to pay (or accept) to have (or forgo) something. In other words, value is the benefit provided to an individual by something. Alternatives approaches to welfare economics exist, such as the capability approach (Sen, 1980), which focuses on individuals being able to achieve the kind of lives they “have reason to value”. In all approaches however, value is a relative concept in which some things are more valuable/important than others.
Whichever welfare economic paradigm is used, the concept of value is different to how it is used in philosophy to mean a principle or moral standard (Baker et al., 2021). In this post, I draw on the distinction between intrinsic and instrumental value in normative ethics. Something has instrumental value if it a means to an end (i.e. to achieve something else), and intrinsic value if it is desirable in and of itself (Weber, 1921).
Health has intrinsic value in that having less than full health brings disutility (e.g. being in pain, being depressed), but also in that “being healthy” is an important part of a good life regardless of utility.* However, health also has instrumental value in that it enables productive work or full enjoyment of family life (Table 1). Health therefore has both intrinsic and instrumental value. Health care, however, is a service (or commodity) which has only instrumental value through its ability to improve health. So demand for primary care (GP) appointments is a derived demand for health itself (Grossman, 1972).**
Instrumental value for:
Intrinsic value in:
enabling work, participation in family life
being healthy (mobile, not in pain/depressed etc.)
enabling time savings (for school/work/leisure) and prevented disease
being water secure (feeling safe / not worrying about water)
enabling time savings (for school/work/leisure) and prevented disease
being sanitation secure (feeling safe / not worrying about sanitation)
Table 1: instrumental and intrinsic aspects of value of health, water, and sanitation
I think water brings both intrinsic and instrumental value from the household perspective. Its instrumental aspects are more often emphasised, e.g. in preventing disease and enabling time savings. However, “being water secure” has intrinsic value in that since water is necessary for life, being water secure is part of being human. In addition, worrying about having enough water, or feeling unsafe in water collection, bring disutility (Table 1). Water supply is analogous to health care in being a service/commodity only of instrumental value through how it supports water security. The same thinking applies to sanitation (Jain and Subramanian, 2018). Sanitation services have instrumental value in the same way as water (Table 1), but sanitation security has intrinsic value (Caruso et al., 2017; O’Reilly, 2016).
In benefit-cost analysis (BCA) of sanitation and water interventions, it is usually the benefits of instrumental value which are quantified (e.g. time savings, avoided morbidity/mortality). In health BCAs, however, the value of health is regularly quantified in monetary terms, e.g. US$ (Robinson et al., 2019). For example, willingness to pay for a quality-adjusted life year (QALY), a regularly-used measure of the value of health, can be estimated as through methods such as contingent valuation (Bobinac et al., 2010). A review identified 24 QALY monetary valuation studies with a trimmed median of 24,000 Euros in 2010 prices (Ryen and Svensson, 2015). Such monetary valuations can be summed with other benefits in BCA, just as disability-adjusted life years (DALYs) have been for some water supply BCAs (Whittington et al., 2017). I have a pre-print in which I make the case for using a new “water-adjusted person year” to quantify the value of water for people’s quality of life (Ross, 2022). I think that capturing the monetary value of water security in such a way could better reflect the quality of life gains from water supply interventions in BCA, just as is done with the monetary value of QALYs.
*To illustrate, it is worth quoting Brouwer et al. (2008) in full: “Health is pursued and valued by policy makers for its own sake (and possibly because of its impact on productivity) rather than because it yields utility or merely to the extent that it yields utility. Although good health certainly also contributes to welfare and, for that matter, to opportunity for welfare, it is valuable in itself as an important characteristic of human beings. Indeed, especially in the context of health it has been claimed that utility is an unsuitable guide to policy, if only because a person may adjust his expectations to his condition.”
**Of course, health care may have intrinsic value for a small minority of people who appreciate their problems being listened to, regardless of health consequences (Ball et al., 2018).
Summary: the severity of climate impacts on WASH services is uncertain. “Low-regrets” investments or interventions are those which generate net economic benefits under a range of the most plausible scenarios of climate impact severity. The concept is explored in Figure 1, which illustrates relationships between net benefits and the severity of climate impacts for different types of high/low/no-regrets options. It is also important to explore non-climate uncertainty, ideally in a probabilistic way.
Uncertainty is when we have imperfect information about variables in the present or the future. Even though the effects of climate change are increasingly upon us already, the scale and nature of their economic impacts remain uncertain (Burke et al., 2015). The further into the future the projection, the more this uncertainty increases (IPCC, 2022), because: (i) many variables interact in determining climate impacts; (ii) we can (and must) reduce greenhouse gas emissions to mitigate the worst impacts, and any effect of those actions is also uncertain.
The higher levels of water and sanitation services sought by SDG 6 are characterised by infrastructure assets with useful lives of 20-50 years or more (Hutton and Varughese, 2016). It is particularly important to characterise the vulnerability of such long-lived infrastructure to climate risks, especially since retro-fitting can be more expensive than designing for uncertainty upfront (Chester et al., 2020). I go into some of the climate risks to WASH services in this post.
Benefit-cost analysis (BCA) is the most commonly-used economic evaluation method for appraising WASH investments [a short introductory paragraph on BCA is below this post]*. In appraisal of interventions for adaptation or resilience, “no regrets” interventions are those which generate net benefits under all future climate/impact scenarios (Heltberg et al., 2009). A more achievable principle, endorsed by the IPCC (2012), may be aiming at least for “low”-regrets interventions. These are interventions which generate net benefits under a range of the most plausible scenarios. However, they also account for the risk that we might “regret” additional investment in adaptation/resilience if climate impacts are not as bad as expected. No-regrets options would be first choice, and often they will be available. However, low-regrets options may be important if adaptation/resilience increases costs substantially in relation to benefits in a “no climate change” scenario.
Low-regrets thinking has been applied in identifying opportunities for short/medium-term climate risk reduction within development interventions (Conway and Schipper, 2011). Identifying low-regrets options can also help reduce the risk of maladaptation (Barnett and O’Neill, 2010). This line of thinking can be applied whether what is being evaluated is a whole new investment in WASH services, or options for adapting/upgrading existing WASH services.
A few years ago, I was part of a three-country study looking at risk assessment and economic appraisal for adaptation to climate change in WASH (Oates et al., 2014). In making the economic arguments, we used a diagram which I’ve simplified here (Figure 1), and which I think Kit Nicholson came up with. The x-axis plots the severity of climate impacts (broadly defined) as an uncertain continuous variable. The y-axis plots the benefit-cost ratio (BCR) [see explanation at bottom]* of intervention options. The threshold where benefits equal costs on average over the time horizon (e.g. 20 years) is shown as “1”. In simple terms, we want to be above the green band, but we don’t know where we’ll be on the x-axis.
Plenty of WASH infrastructure constructed in recent decades might be climate risky (blue line A), i.e. in the absence of climate impacts it looks economically attractive, but as climate impacts worsen then BCR<1. Designing for the worst-case scenario may result in investments which are high regrets (orange line B), i.e. over-designed such that climate impacts have to be very severe before BCR>1. No-regrets options (both green lines C) are any interventions for which BCR>1 regardless the severity of climate impacts. Low-regrets options (yellow line D) may have BCR slightly below 1 when climate impacts are small, but gradually appear more attractive as climate impacts worsen. Low-regrets options need not necessarily have BCR<1 in the case of “no climate change”, but at least they would need to have lower BCR in that scenario than an option without investment in adaptation/resilience. While BCAs often present decisions as “doing something” versus “doing nothing”, this framework aims to account for the fact that in the real world there are usually multiple options under consideration.
A simplified WASH example can help illustrate. A team is planning a piped water supply with a treatment plant fed by a river intake, and the risk is identified that turbulent flows resulting from an extreme weather event may damage the intake. A “climate-risky” option might be to design the intake to withstand a flood of a given height with a 25-year return period, which is fairly likely to be exceeded within the useful life of the infrastructure. A “high-regrets” option might be to design for a 200-year return period, which would be more expensive, but increasingly worth doing as the probability of climate change-induced floods increases (Figure 1). A low-regrets option might be somewhere in-between. The reality is more complex than this, and there are many specific options within this scenario, related to, e.g. overflows, intake design, floating booms, early warning systems, etc. (Howard and Bartram, 2010).
There are some qualifications to make regarding this way of framing adaptation options. First, this framework does not make value judgements, e.g. high-regrets options are not necessarily a bad idea. However, since all investments have an opportunity cost (i.e. resources are scarce), high-regrets options may be less desirable from an equity perspective, because more people in a given year could be provided with WASH services under a low-regrets option. Second, while I often refer to these interventions as “adaptation options”, many might comprise what we should be doing anyway given existing climate variability, and the need to be resilient to risks other than the climate.
Third, many non-climate parameters in BCAs are also uncertain (e.g. costs, health effects, uptake, maintenance etc.), but this framework puts the focus on uncertainty about climate impacts. Bands incorporating uncertainty of many other parameters may therefore be more appropriate than lines. The low-regrets option from Figure 1 could be assessed in a probabilistic sensitivity analysis (PSA) per climate scenario. Such a PSA would posit plausible probability distributions for key parameters (Briggs, 2000), then run a Monte Carlo simulation with (say) 1,000 iterations. An uncertainty interval could then be posited by graphing the range of the middle 95% of iterations within a band, such as in Figure 2. This line of thinking is the main thing that is new in this post, as compared to the 2014 work (Oates et al., 2014).
Fourth, one challenge in undertaking such analyses is that, due to “deep uncertainty” in the context of climate change, it is hard to ascribe probabilities to many key variables (Hallegatte et al., 2012). Nonetheless, a Bayesian approach to uncertainty requires that the analyst makes their best estimate at the shapes of probability distributions (Briggs, 1999). Simply leaving variables out of the analysis, or not doing a PSA at all, is the same as assuming they are known with certainty. Assuming a uniform distribution for a given parameter only makes sense if the aim is to explore possible heterogeneity across settings, rather than estimating a realistic mean and uncertainty interval to inform a specific decision in a given setting. Expert opinion, tested in scenario analysis alongside the PSA, is therefore likely to play an important role. Fifth, in the real world, the “severity of climate impacts” is not a single continuous variable as in Figures 1 and 2. The IPCC provides multiple projections, and practically it would make sense to undertake scenario analysis using those.
In conclusion, I suggest that appraisal of investments in WASH infrastructure adaptation or resilience can be informed by a “regrets” perspective focused on climate uncertainty (Figure 1), but also taking account of uncertainty of non-climate parameters (Figure 2). Low-regrets options are those which generate net economic benefits under a range of the most plausible scenarios of climate impact severity.
*BCA combines all the consequences of an intervention (e.g. saved time, reduced disease, quality of life gained) and places a monetary (e.g. US$) value on them. These monetised benefits are then compared to the costs of an intervention over time, with discounting. Metrics for comparing options include the net present value (=benefits–costs) or the benefit-cost ratio (=benefits/costs). The benefit-cost ratio (BCR) is often communicated in terms of US$ X economic returns on US$ 1 invested. If the BCR is greater than 1 (the clearing rate or threshold) then the intervention has net benefits, and if less than 1 it does not. Benefit-cost ratios of different intervention options can be compared to assess their relative efficiency, although other factors should be taken into consideration (equity, feasibility, relative size of net benefits, etc.)
I’ve previously written about defining hygiene. In this post, I discuss what counts as a “handwashing facility” (HWF) for global monitoring purposes. Points arising that may not be obvious include: (i) an on-plot water point can be considered a HWF if that’s where handwashing is most often practised (e.g. tubewell with handpump, yard tap); (ii) if the HWF most often used is off-plot, that counts as “no service”.
The SDG6 hygiene indicator is the % of people with access to a basic HWF. “Basic” is defined according to the below figure from the JMP 2021 report for households: “availability of a handwashing facility with soap and water at home”. Of greater importance for this post is the note below the figure, which emphasises that HWFs may be (i) located within the dwelling, yard or plot; (ii) fixed or mobile.
How the question is asked and coded in the latest DHS household questionnaire is below. Key aspects of the question are: (i) if there are multiple places, it’s the one used “most often”, and no specific critical time (after using the toilet, before preparing food) is referred to; (ii) the most-used HWF must be observed – if no handwashing facility is observed or it is off-plot, the household is counted as having “no HWF at home”. If the household does not give permission, or it is not possible to observe for other reasons, it is counted as ‘no permission to see/other’ in survey reports and is excluded from the denominator for JMP service ladders. The DHS interviewer’s manual (p.46) recommends probing to observe mobile HWFs.
In the note below the JMP 2021 figure above, examples provided of HWFs are a sink with tap water, buckets with taps, tippy-taps, and jugs or basins designated for handwashing. However, it is common in many countries and settings to have yard taps from a utility supply, and in some countries (especially in South Asia) to have an on-plot tubewell with handpump. If one of those water points is the place where household members most often wash their hands, then it is natural to interpret that as a fixed HWF. I have clarified with the JMP team that this interpretation is correct.
However, households with water accessible on premises may not necessarily have a “basic” hygiene service, because soap and water must be available at the time of survey (see “limited” category in the figure at the top). In Bolivia for example, 86% of people have improved on-plot water but only 27% have “basic” hygiene, with availability of soap being the limiting factor (see JMP figure below). In analysis of 46 least developed countries for our recent costing study (under review), we established from JMP data that amongst the population with limited services, “missing soap only” was the reason for 47% on average, “missing water only” was 10%, and “missing both” was 43%. So households with a “limited” service are missing soap in 90% of cases.
Humanity is emitting too much CO2 and using ever-increasing amounts of energy and water. The human population is set to swell for the foreseeable, requiring both more food and the water to grow it. At the same time, climate change is threatening progress across the board.
These trends have spurred a new way of thinking about the health of humans and our world: planetary health. First articulated in a Lancet commentary in 2014, the concept was expanded upon in a 2015 Lancet Commission, which defined planetary health as “the health of human civilisation and the state of the natural systems on which it depends”. This emphasises the interconnectedness of societies and ecosystems. Climate change understandably dominates planetary health discourse, but its other major themes include soil/food, water resources, and biodiversity.
How does planetary health relate to WASH?
WASH services, particularly sanitation, contribute to pollution of water and greenhouse gas (GHG) emissions, predominantly methane. On the other side of the coin, WASH services are likely to be harmed by climate change and other environmental degradation. I tried and failed to find an existing conceptual framework combining this “cause and consequence” perspective, so made my own (Figure 1).* Click here for a higher resolution version.
All of the consequences are likely to have knock-on impacts on human health, quality of life and livelihoods, which is part of the point of planetary health. Another angle is that better WASH services, including having multiple options available, can make people more resilient to shocks. However, in this post I focus on the consequences for WASH services to keep things manageable – this paper provides a useful review including some of the climate-related health consequences. I have seen little work explicitly applying the planetary health concept to WASH, though reams of it touches on the issues without using the term. Notable exceptions are this report on sanitation and planetary health, and this feature on planetary health approaches for dry cities.
Taking the climate aspect of planetary health first, sanitation is an important contributor to GHG emissions via methane (and nitrous oxide) from wastewater treatment and pit latrines. Methane emissions contribute to about a third of today’s anthropogenic warming, because it is about 30 times better at trapping heat in the atmosphere than CO2. Wastewater contributes about a tenth of all anthropogenic methane emissions globally. This is likely to grow (emissions more than doubled over 15 years in China), but the relative contribution of methane emissions from pit latrines and septic tanks remains unclear.
In considering the impact of WASH services on climate, recall that infrastructure development of any kind currently uses fossil fuels and concrete. Operating water supply and sanitation assets also uses energy. Turning to other aspects of planetary health, human urine and faeces also contain phosphates and nitrates which contribute to eutrophication if discharged untreated, potentially harming biodiversity.
Most planetary health consequences for WASH services are felt through water resources. The relationship between climate and water resources is reviewed in the World Water Development Report 2020, and this paper provides a useful WASH-specific review. Impacts are likely to vary by geography, but will be felt via changes in: (i) levels and intensity of precipitation (with snow melt also being crucial in some regions, e.g. South Asia); (ii) extent of aquifer recharge and surface run-off; and (iii) sea levels. These factors will interact on a local basis and may not reflect the standard newspaper image of dried-out mud. For example, groundwater is less vulnerable to changes in rainfall than surface water, and availability may be resilient in some areas, depending on precipitation-recharge relationships.
There are three major WASH-relevant areas of impact from these changes, taken in turn below.
Availability and quality of groundwater and surface water
First is the availability and quality of groundwater and surface water, and thence the amount of usable source water for water supply. Considering piped water systems, the further that source water is from being of drinkable quality, the more extensive the treatment processes needed. The less the surplus of water production potential over demand, the higher the risk of “day zero” (see Cape Town and Chennai). Users relying on point sources (e.g. wells, springs) may see changes in water quality or quantity making their lives much more difficult. The lower the water availability, the less likely handwashing becomes. The sanitation consequences of such fluctuations are discussed below.
Considering other areas of planetary health, the ways in which agricultural practices impact on water and soil can in turn affect the availability and quality of groundwater and surface water (e.g. contamination with fertilisers, impact of soil degradation on run-off and suspended solids). Less obvious is that greater biodiversity (e.g. in algae) can be beneficial for water quality. Also, despite most planetary health consequences for WASH being felt through water resources, climate can have a direct impact, for example through higher air and water temperatures affecting treatment processes.
Floods, droughts and storms
The second area of impact is through increases in extreme events like storms, floods, and droughts. The high flows and carried debris resulting from storms and associated abnormal rainfall can damage infrastructure, particularly for sanitation and drainage but also water supply. Storms can also interrupt power supply and in turn the operation of infrastructure. The onset of flooding can be rapid or slow, but either kind can contaminate water supplies and/or increase loading of suspended solids to levels exceeding the capacity of water treatment plants. Floods and pit latrines are not a good mix – contents can be inundated from above, or rising water tables can flood them from below. Droughts have obvious consequences for water supply, discussed above, but can also negatively affect sanitation, for example if there are insufficient water flows for sewerage operation. More detail on sanitation is provided in a WHO discussion paper.
Sea level rise
The third area of impact via water resources is sea level rise. This can affect source water availability and quality by saline intrusion. Longer-term sea level rises threaten to overwhelm existing water and sanitation infrastructure in coastal regions. Consider that the majority of Lagos, Nigeria, is only a few metres above sea level and the mega-city already suffers from flooding.
We live in the anthropocene, the epoch in which humans are putting huge pressure on the planet, threatening our own health and survival. WASH services are part of the problem, to the extent that wastewater treatment and pit latrines contribute to GHG emissions. However, better WASH services are part of the solution in that they make people more resilient to shocks. Climate change is important but it is not everything. Other aspects of planetary health such as biodiversity have important interactions with WASH services both as a cause and consequence. Furthermore, in many of the poorest countries, there is likely to be more strain on water resources and WASH services from population growth, urbanisation and economic development than from climate change. Regardless of climate change, people’s lives will be improved if we make WASH services more resilient, whether to existing variability in water availability and quality, or to existing extreme events. Taking a planetary health perspective can help with this.
*In this post I focus on the implications of planetary health for WASH services and vice versa, taking the impacts on health as given. I have also left out of the discussion (and diagram) how water demand might change in response to the trends described, e.g. through patterns of migration, urbanisation, population growth etc. Demand will also be affected by changes on the supply-side, including those influenced by planetary health factors. The diagram does not claim to be a complete picture of the relationships at play, just to capture the main ones. I also don’t consider this an area of much personal expertise. The main work I’ve done in this area was a three-country study with ODI focused on risk and options appraisal for climate adaptation in WASH. Therefore, I would welcome corrections or recommendations of things to read!
In labour economics, human capital is a worker’s stock of knowledge and skills which contributes to their productivity and earnings. Human capital accumulation is a process of developing skills within and beyond cognitive domains, in which the first 1,000 days of a child’s life are crucial.
In a note on this topic available here, I present a conceptual model (shown above) for the relationship between improvements in WASH services and increased human capital. Three pathways are proposed: early childhood development; all-age health capital; and school.
The early childhood development pathway is likely to be most important, due to its far-reaching and long-lasting implications for human capital. I also review some recent evidence linking sanitation and early childhood cognitive development published since the last systematic review on this topic.
[I’ve started writing longer “notes” like this on certain topics, which are not quite working papers but longer than blogposts. Feedback/critique is most welcome!]
I’ve been doing a fair amount of work on hand hygiene since November, e.g. this piece on the economics of hygiene for the Hygiene Hub and some costing work for WHO/UNICEF. It bothered me that definitions were not clear, so I put together this Venn diagram. It aims to be a broad means of categorisation, rather than a comprehensive listing of all possible aspects of hygiene. I had domestic settings in mind, and other factors may be more important in other settings, particularly health care facilities. Please flag other resources on this definitional question in the comments below, if you know of them!
Within and beyond the WASH sector, “hygiene” has often been taken to be synonymous with handwashing. However, it is really a broader concept than this, but how broad? At the extreme end of the spectrum, hygiene is “the practice of keeping oneself and one’s surroundings clean, especially in order to prevent illness or the spread of diseases” (Boot and Cairncross, 1993). This aligns with the adjective hygieinos in Ancient Greek, meaning good for health (Liddell and Scott, 1889).
Taking the broad Boot & Cairncross definition, many behaviours and practices would fall within the scope of hygiene, including: (i) excreta disposal; (ii) use and protection of water sources; (iii) personal hygiene (e.g. hand hygiene, menstrual hygiene management); (iv) food hygiene (e.g. handling, preparation and storage); and (v) environmental hygiene (e.g. surface wiping, solid waste disposal, animal management).
Personally, I think including sanitation and water within hygiene is so broad as to be unhelpful. It is more typical to take a narrower approach. For example, UNICEF and WHO (2019) define hygiene as comprising “a range of behaviours that help to maintain health and prevent the spread of diseases, including handwashing, menstrual hygiene management and food hygiene”. Some studies also note face hygiene and bathing (Prüss-Ustün et al., 2019). You could argue for separating out other things like surface cleaning, toy cleaning etc. The various aspects are discussed in more detail by Curtis et al (2001) This and a suggestion from my colleague Karin Gallandat led me to separate out personal, domestic and food hygiene in the above diagram, but I am sure there are many other ways to cut it. More environmental aspects might be included, but then overlap with environmental sanitation (also ill-defined) would become more problematic. Another argument might be to include disease prevention behaviours such as wearing a face mask when exhibiting respiratory symptoms.
Zooming in on hand hygiene
The final definitional twist to note is that hand hygiene comprises not only handwashing with soap and water, but also handrubbing with alcohol-based hand rub (not technically “washing”). These WHO guidelines have lots of definitions along these lines. Ash can also be used as a last resort. For all your questions on rubs vs. soaps, see the Hygiene Hub, especially this piece. In short, soap is just as effective, relatively cheap, and more widely available by comparison to rubs – it is also more gentle on hands. However, the calculus is likely to be different in health care facilities where rubs are often considered more appropriate, for various reasons.
From an infectious disease perspective, focusing on clean hands (vs. other hygiene behaviours in the venn diagram) is warranted. Hand hygiene is likely the hygiene behaviour that makes the most important contribution to preventing faecal-oral disease and, annually, 165,000 deaths from diarrhoea are attributable to inadequate hand hygiene behaviours (Prüss-Ustün et al., 2019). However, food produce may be an important exposure pathway in many settings, and more evidence is needed on this (WHO and UNICEF, 2019).
Hand hygiene can prevent faecal-oral diseases by removing pathogens after fingers touch faeces (or things which have touched faeces) and before those fingers touch food, fluids, or the new host’s mouth. See the famous F-diagram below (Wagner and Lanoix, 1958). Human faeces might touch hands directly, before entering the environment (e.g. after defecation or child faeces management). Crucially, however, they are also transmitted indirectly once pathogens are already in the environment (e.g. surfaces, other people’s hands, animals and their faeces). Therefore, even with good water supply and sanitation services, pathogens can still circulate and hand hygiene is necessary to reap the full benefits of WASH.
[note – this post was updated after helpful suggestions in replies to this tweet – thanks!]
This post summarises a note I have drafted on the definition of “transformative WASH”, available here, with references.
There has been a lot of talk about “transformative” WASH since the WASH-B, SHINE and MapSan results came out. I have previously written about those results here. The argument runs that “basic” or “elementary” WASH services do not reduce environmental faecal contamination to a sufficient degree to see health impact, so we need transformative interventions. However, authors typically provide little detail on how they see “transformative WASH”, which is unsurprising for an emerging concept.
In the note, I summarise what has been written about transformative WASH, adding reflections and an economic evaluation perspective. What emerges from existing publications is an idea of transformative WASH as “safely-managed” levels of water and sanitation service in combination with basic hygiene, better housing conditions (e.g. sealed floors / play spaces) and better management of animals, all with a view to reduced faecal contamination and health gains. Most authors also agree that the WASH system functions such as policy, planning, M&E and regulation are important enabling factors for transformative WASH.
I have five points to make:
1. I tend to think of “transformative” as a level of ambition, rather than a level of service. Being transformative, then, is about achieving incremental changes which are large rather than small, regardless of the starting point or end point.
2. I also tend to think about transformation in terms of outcomes, as opposed to interventions or environmental contamination (Figure 1). By outcomes I mean not only infectious disease and nutrition, but also quality of life, because users value WASH services for many reasons.
3. As and when a definition emerges, it does not follow that only interventions which are “transformative” in disease risk terms should be funded. The most economically efficient interventions are not necessarily those which are most effective at improving health. The costs of a highly effective intervention may be substantially higher than the next best option, to the extent that it is much less cost-beneficial. Choosing the most efficient interventions permits extending services to a larger number of people within a given budget constraint, maximising net benefits to society.
4. The equity principle demands that a substantial proportion of collective effort is placed on extending services to those with the worst levels of service at present.
5. To avoid wasted investments, services must be sustained. Recurrent costs typically increase with level of service, and must be covered. Therefore, levels of service offered should be aligned with households’ willingness and ability to pay those recurrent costs. Without this principle, services may fall into disrepair or low levels of uptake will make schemes inefficient to run.
This post summarises a note (available here), which summarises what has been written about transformative WASH, adding reflections and an economic perspective. I summary, I see “transformative” as a level of ambition, rather than a level of service, best conceptualised at the level of outcomes. Being transformative is then about achieving incremental changes which are large rather than small, regardless of the starting point or end point. The most economically efficient interventions are not necessarily those which are most effective in health terms.
Yesterday I was on a panel at the UNC Water and Health conference side-event entitled “What is “Quality” Sanitation? Investigating Service Standards and User Experience in Rural and Urban Settings”. You can watch back the event here (if registered for the conference, which is free). Below follows some of my views on the measurement of quality, covering three aspects: (i) quality of what? (ii) quality from whose perspective? (iii) quality for what purpose?
1. Measuring the quality of what?
As background to “quality of what”, here’s three blogposts I wrote last year:
This post explains the concept of sanitation-related quality of life (SanQoL)
This post discusses differences between “quality of life” and “quality of service”
This post discusses subjectivity / objectivity, and generalises to the service chain beyond containment
In summary, measures of quality of life (QoL) focus on subjective user-perceived outcomes (e.g. how safe the user feels). By contrast, measures of quality of service (QoS) focus on infrastructure attributes (e.g. presence of an inside lock) or level of service attributes (e.g. number of households sharing). Both QoS and QoL measures provide useful information and can be complementary, but it is important to be clear on whose perspective they provide, to inform their intended purpose (Table 1).
An example of a QoL measure is SanQoL, and I hope to publish a paper on its development, validity and reliability soon. An example of a QoS measure is the “sanitation quality index” (SQI) presented in the side-event (link at top) by EAWAG and others. Both SanQoL and SQI focus on the containment stage, but QoL and QoS have applications for emptying, treatment etc.
Table 1: Quality of what?
2. Measuring quality from whose perspective?
There are three aspects to this: (i) user or decision-maker focus, (ii) what goes in the measure, and (iii) who is evaluating/scaling its attributes.
First, the two main focuses for evaluating quality are typically the user or the decision-maker. For users, this might be direct (by asking users, e.g. SanQoL) or by proxy through evaluating infrastructure characteristics and inferring user-perceived quality indirectly (e.g. SQI). For decision-makers, sanitation regulation, for example, may give explicit or implicit guidance on minimum quality standards. In some cities, there are debates as to whether container-based sanitation should be permitted. Furthermore, downstream implications of quality for broader impacts are also important to decision-makers. The most obvious example here is health of the broader public via externalities (e.g. when septic tanks discharge to drains, illegal dumping of sludge, leaking sewers). This is a broader point raised in the first two presentations in the side-event. In summary, if you ask users and decision-makers about what is important for sanitation quality, you are likely to get slightly different answers.
Second, on what goes into a measure of quality, QoL measures generally follow the principle that the inclusion of attributes/items should be based on the priorities of the target population. That means consulting users rather than “expert opinion”. The qualitative study for the SanQoL measure is currently undergoing peer review. The same should go for QoS measures, and qualitative work underlying the SQI was recently published here (leading them to include things like material of floor, roof, solid waste, insects etc.)
Third, the question of “whose perspective” during fieldwork is also important (Table 1). For assessing infrastructure attributes (slab, walls, lock on door) there are many options, e.g. an enumerator can observe, or ask the user, or both. In theory, such indicators should be objective. However, there will always be an element of subjectivity when an enumerator is evaluating presence/absence of faeces, or the material of the walls is (e.g. how large do holes in corrugated metal sheets have to be until privacy is unlikely?). When it comes to less observable technical aspects (e.g. lining, septic tank or holding tank?) this can be very hard to verify regardless of who is responding. However, level of service attributes which are part of QoS (e.g. number of users, sharing status), cannot be observed and users must be asked.
When it comes to QoL (e.g. user perception of safety, privacy), only the user is in a position to respond, and each individual is different. This highlights an important distinction between QoS and QoL measures. Results for QoS measures should be identical or near-identical when asking two users of the same toilet (i.e. very high inter-rater reliability). For QoL measures, however, there is no reason to expect this. There should probably be correlation, but two users might experience the same toilet very differently (see example in this post) and so have different QoL outcomes. In the QoL case then, it is not a question of inter-rater reliability, though inter-rater methods could be used to assess the level of correlation.
3. Measuring quality for what purpose?
The purpose of informing regulation or defining standards was flagged above. The SQI has its roots in WSUP wanting to push back on the SDG sanitation ladder categorising shared sanitation as “limited”. Other possible purposes of measuring quality are routine M&E (e.g. by municipalities or NGOs), impact evaluation aspiring to causal inference, or economic evaluation. SanQoL could be used in impact evaluation – it would be useful to compare multiple intervention options in the extent to which they improve QoL. However, the ultimate intended purpose of SanQoL is in economic evaluation, that is, cost-benefit analysis (CBA) or cost-effectiveness analysis (CEA). I have developed a measure of the value of sanitation, weighted using SanQoL, which could be used in CBA or CEA. More on that in due course… The broad point here is that the purpose of measuring quality should inform the design of any measure. Economic evaluation frameworks are quite restrictive in terms of what methods should be used to infer value, for example. Routine M&E may be restricted by what data are feasible to collect quickly, and implementers may be more interested in basic aspects of QoS from a quality control perspective.
When measuring the quality of sanitation, it is important to be clear on three things: the quality of what, quality from whose perspective, and the purpose which the measurement aims to inform. To illustrate these points, I have distinguished between quality of service (QoS) and quality of life (QoL), and I continue to believe both are complementary. Some purposes require QoS data and others QoL data. If the ultimate aim is to understand user experience, it is probably important to collect both, in order to explore QoS variables as influences on QoL, as the Clean Team study presented in the side-event aimed to do.
My colleague Seungman Cha has a paper out this week, which I co-authored with him and others. It’s a trial-based cost-benefit analysis (CBA) of a community-led total sanitation (CLTS) intervention in rural south-western Ethiopia. We estimated intervention delivery costs from financial records and recurrent costs from the trial’s surveys. All outcome data are from the trial (health, time savings, cost of illness) – the trial effect paper is accepted pending revisions at AJTMH, but the protocol is here. Avoided mortality comprised ~60% of benefits, and the base case benefit–cost ratio (BCR) was 3.7. In probabilistic sensitivity analysis, 95% of estimates of the BCR were within a range of 1.9–5.4.
nb. the paper was already under review at IJERPH when the recent controversy about prioritising open access waivers for high-income (!) countries came out. I won’t be submitting more papers to MDPI journals, or reviewing for them, until they report results of the apparently ongoing internal investigation and ensure nothing like that happens again.
Obviously I think you should read the paper. In case you need some persuading, here’s three important things about it. One is a broad conclusion about the intervention, and the other two are about (the lack of) ex post and/or trial-based economic evaluations in the sanitation sector.
1. Upgrading low-quality latrines
The story of this intervention in Ethiopia is about upgrading poor-quality latrines, rather than ending OD. More detail is in the effects paper when it comes out. OD was already <5% at baseline, so not your typical setting for CLTS. About 75% of intervention group households used private latrines, of which almost all were unimproved. The other 25% used neighbours’ or communal latrines. What the intervention achieved (Figure 1 in the paper) was (i) ~100% coverage of private latrines of varying quality, and (ii) substantial upgrading from unimproved pits to improved (and “partially improved”) pits. Definitions are in the paper, but “partially” in this study essentially meets the JMP definition of improved . Achieving ~70% coverage of ~improved latrines, as opposed to the unimproved latrines typically achieved under CLTS, was probably a key factor in seeing the effect on longitudinal prevalence of child diarrhoea. High baseline usage of latrines of one type or another is also why the value of time savings comprised <30% of total benefits, which is small compared to many such studies.
2. First fully trial-based economic evaluation of a sanitation intervention
Our paper reports a “single study” economic evaluation , meaning that all key* parameter values come from the specific setting rather than being assumptions from the literature. I know of only one other sanitation study which does this: a cost-effectiveness analysis based on a case-control study of a latrine intervention in Kabul, Afghanistan, in the late 90s (Meddings et al., 2004). Our paper is therefore the only fully trial-based economic evaluation of a sanitation intervention, despite such study designs being pretty common in public health. Trial-based economic evaluations are valuable because they show the economic performance of an intervention in real conditions, with high internal validity. They are also not hard to bolt onto existing trials. In my opinion, many or most impact evaluations (RCTs or otherwise) should include an accompanying economic evaluation, if they are to influence investment decisions. It is surprising that researchers do not do them and funders do not demand them, as others have argued recently (Whittington et al., 2020). It is not enough to know whether interventions are effective – we also need to know whether their benefits justify their costs. More importantly, we need to compare the relative economic performance of competing WASH intervention options.
3. Very few ex post economic evaluations of real interventions more broadly
Our Ethiopia study presents an ex post CBA of a specific sanitation intervention – that is, the intervention actually happened. It is quite surprising just how many studies in this literature are of hypothetical interventions. There are only four other examples of ex post economic evaluations of sanitation interventions I know of, in addition to the Afghanistan study above. Two studies in India combine primary cost data from the setting with health impact estimates from secondary sources (Hutton et al (2020); Dickinson et al, 2014). The East Asian studies synthesised by Hutton et al. (2014) also combine primary cost data with secondary outcomes (and are immensely detailed in the country-level reports), though they focus primarily on technologies rather than interventions. Finally, a further Indian study by Spears (2013) combines secondary data on both costs and outcomes.
Hypothetical studies can be very informative, such as a recent one which explored how the extent of uptake (and other factors) influences the economic performance of CLTS (Radin et al., 2020). However, to be able to make investment decisions about which sanitation interventions are most efficient, we need more studies that evaluate interventions which actually happened! Interestingly, the coverage increase for improved latrines achieved by the intervention (~35%) in our Ethiopian study was the same the “high-uptake” scenario in the Radin et al., 2020 hypothetical study, and our headline result is almost identical. However, note the discussion regarding definitions above – the intervention increased coverage of “JMP-improved” latrines by ~60%. That’s quite a lot of upgrading, and a fair amount of new construction as well.
In conclusion, read our paper and reflect on toilet upgrading in rural areas! But more importantly, if you’re currently running or planning an impact evaluation, strongly consider adding a cost-effectiveness or cost-benefit analysis to the protocol. The incremental effort of collecting good-quality cost data is very low compared to the overall research cost of your study. As is the incremental effort in carrying out a cost-effectiveness or cost-benefit analysis. Effectiveness estimates only tell us so much – economic evaluations help us make decisions about investing scarce resources. If the intervention “works”, one of the first things you’ll be asked is how much it costs…
* OK, the case fatality rates come from the Global Burden of Disease study, but very few WASH trials are powered to have mortality as an outcome. Likewise the estimate for value of a statistical life (VSL) is secondary – there are precious few VSL studies in LMICs, let alone undertaken as part of a trial such as this. My point is that the key sources of data for benefits are primary (health effect, value of time, and cost of illness).
I’ve just arrived at the UNC Water and Health conference 2019 this week (verbal abstracts book here). I have three verbal presentations, a poster, and am involved in a side-event. Short summaries are below. Further below, I highlight others’ economics-related presentations/sessions that I’m looking forward to seeing.
All my stuff is on Thursday… but please talk to me about SanQoL at any other time! If you have time for just one of my bits and pieces, please come to the verbal on cost-effectiveness, Thurs 1600 in Azalea. It’s the piece of work I’d most like critique on, as it’s the headline output of my PhD.
My verbal presentations:
Title: “Cost-effectiveness Analysis of a Sanitation Intervention with a Quality of Life Measure as the Outcome” (Thurs 1600 in Azalea, 1st up)
One-sentence abstract: An overview of ‘sanitation-related quality of life’ (SanQoL), and an empirical study in Mozambique of how SanQoL can be used to weight ‘quality-adjusted service years’ (QASYs), for better economic evaluation of sanitation programmes.
Title: “Human Waste of Time—Valuing Open Defecation Time Savings” (Thurs 1600 in Azalea, 2nd up)
One-sentence abstract: An exploration of how we should value ‘time saved’ when people switch from OD to household toilets, using data from the 2013 SQUAT survey in India, because how we do this makes a big difference to results of cost-benefit analysis.
Title: “Three-Quarters of People in Port-au-Prince, Haiti, Get Their Drinking Water from Private Providers” (Thurs 1430 in Redbud, 2nd up)
One-sentence abstract: Demonstrating a “water flow diagram” for visualisation of flows of water and money in a city-wide water market, through a mixed-methods study of private water providers in Port-au-Prince.
Title:“How does sanitation contribute to a good life? Qualitative research in urban Mozambique informing quantitative measure development” (Thurs 1700)
One-sentence abstract: Results of the qualitative study (yes, economists do proper qual…) that informed the development of the SanQoL measure, using focus groups and in-depth interviews in low-income areas of Mozambique.
Side-event I’m involved in
Title: “An Agenda Setting Workshop for “Limited” (Shared) Sanitation: User Experiences, Measurement, and Improvement Approaches” (Thurs 1030 in Redbud)
One-sentence abstract: starting with a “quick fire” format (presenters have 1 slide / 5 mins each), the meat of the session will be focused on the creation of a research agenda for the role of shared sanitation in bringing safely managed sanitation to all.
Others’ WASH economics verbals and side-events
The best thing about the UNC conference is that, when I look at the agenda, I want to go to almost everything. If you take a broad definition of WASH economics (as I do here) then a lot of the sessions/papers at UNC will touch on it. So here’s just a few of the other WASH economics verbals/sessions I’m looking forward to. There’s some interesting-looking posters too.
“An Evidence-driven Approach to Establishing Prices for Pit Emptying Services by Vacuum Truck Operators” Bernard Elegbe, ABMS/PSI – looks like a cool application of “mystery shopper” techniques to FSM services. Mon 1430, Dogwood
“Using Public Subsidy to Unlock Household Finance: Evidence from the Field” Lesley Pories, Water.org – using RCT data to explore how households used sanitation microfinance loans. Mon 1430, Dogwood
“Findings from the Implementation of the First Cross-culturally Validated Household Water Insecurity Experiences (HWISE) Scale in Zambia and Democratic Republic of Congo” Sera Young, Northwestern University – I’ve read a lot about HWISE so looking forward to seeing its empirical use. It’s exciting to see a user-reported measure for water that does a lot of what I’m trying to do with SanQoL. Tues 1430, Redbud
“Time, Economic and Health Benefits of the Transition to Rural Piped Water Systems in Southern Zambia” James Winter, Stanford University – interesting approaches to measuring the benefits of water supply interventions using matched controls. Weds 1600, Dogwood
“Supply and Demand: Assessing Costs and Willingness-to-pay for Urban Sanitation in Bangladesh, Ghana, and Kenya” Rachel Peletz, Aquaya Institute – have been looking forward to seeing these results for a while, which represent one of the several important initiatives underway on urban sanitation costing. Thurs 1430, Redbud (and helpfully just before my Haiti verbal)
“Revisiting Subsidies for Water Supply and Sanitation Services” – the recent World Bank flagship report on subsidies, incl. debates around allocation of subsidies between service types and between ‘new’ access vs. consumption. Tues 1030, Bellflower
“Understanding Demand for WaSH Services: How Much are Consumers Willing to Pay?” – should be a good debate on WTP methods with example of vouchers, auctions, hedonic pricing, contingent valuation, and discrete choice experiments. Weds 0830, Dogwood
“The Maputo Sanitation (MapSan) Trial: Measuring Health, Environmental, and Social Impacts of an Urban Sanitation Intervention in Mozambique” – long-awaited results of the MapSan trial, the largest controlled health impact trial of an urban sanitation intervention to date (OK it’s not strictly speaking econ, but it’s the study my PhD is linked to, and incredibly interesting…). Weds 1030, Redbud
There has been a lot of debate about the well-designed and well-conducted WASH-B (Kenya, Bangladesh) and SHINE (Zimbabwe) trials of rural WASH interventions in the past year or so. Most recently, researchers active in WASH epidemiology published a consensus paper. Many funders may not read it, which would be a shame, as it is easy to misinterpret the WASH-B and SHINE findings as “WASH doesn’t work”. If funders make this misinterpretation, people will miss out on life-changing WASH services. In this post, I discuss the incremental changes the interventions delivered, how epidemiologists have interpreted the results, and how funders should interpret them.
In summary, I think that funders should read the consensus paper, and draw two conclusions:
Most WASH programmes make small incremental improvements in service level which may not bring substantial health gains in the short-term, but do have a good chance of doing so.
Funders should continue to support “basic” services and the systems that sustain them, in the knowledge that later incremental improvements can build on those investments to achieve “safely-managed” services further down the line.
This is not an argument that “transformative” WASH services are not achievable or desirable. In some settings they will be. Rather, I mean to point out that sustaining safely-managed services may cost more than rural households in LMICs are able and willing to pay in the medium-term. Small incremental improvements are more realistic and affordable, both for users and funders (in which I include LMIC governments).
1. What were the interventions and results?
In understanding the WASH-B and SHINE results, two things are often overlooked: the baseline conditions, and what incremental improvement the interventions made upon them. This is a point that is similar to, but different from, discussions about external validity (which I don’t plan to get into here).
Table 1 at the bottom of this post summarises baseline conditions and interventions for each WASH component in the three trials. [I haven’t seen this analysis anywhere else, but let me know if someone else did it in more depth already]. Also at the bottom, and for completeness, Table 2 summarises the trials’ main results, though most readers will already be aware of them (no impact in the WASH arms on any key outcomes, except on diarrhoea in WASH-B Bangladesh).
The first point I want to make is that, given the baseline conditions and small incremental improvements achieved (a term I differentiate from “marginal” here – incremental improvements can be large), these interventions did not constitute a typical “WASH” programme. Let’s explore this a little.
Baseline conditions: the highest priorities under SDGs 6.1 and 6.2 should be achieving basic services for all first (as I have previously argued), by ending open defecation (OD) and ending the drinking of water from unimproved and/or far-away sources. See definitions of basic for sanitation and water. The WASH-B settings in particular were already much better than what one might think of as “poor WASH conditions”. For example, there was ~95% latrine access in both Kenya (though mostly unimproved) and Bangladesh. In Kenya there was ~75% improved water, which the time-to-source data suggests fulfilled the “basic” SDG criterion for most. In Bangladesh, it appears most people used on-plot tubewells. DHS 2014 shows that >95% of rural Bangladeshi had “basic” SDG water (and 74% on-plot water). Baseline sanitation conditions were worse in SHINE (~50% OD). Overall, then, WASH conditions in these settings were far from terrible, particularly in Bangladesh.
Incremental changes: the incremental changes to fairly good baseline service levels were small in most cases. Take sanitation: the WASH-B Kenya intervention was to replace an unimproved latrine slab with a lidded plastic one. This is an improvement, to be sure, and probably valued by the user. But not a big change – it is still a pit latrine without a water seal. In WASH-B Bangladesh, ~95% of people used an improved latrine with a concrete slab (and ~30% a water seal), so the change was the switching to a pour-flush in most cases. Again, an important but small improvement. I don’t say this to criticise the interventions, but to argue that maybe we shouldn’t expect health gains with such minor improvements. Many WASH interventions make more substantial incremental changes in WASH service levels. Turning to water, interventions in all three trials were chlorination. The most recent systematic review shows that chlorination interventions have no effect on diarrhoea once non-blinding is taken into account. So, it was not surprising that the water arms had no effect. In none of the three trials was any change made to the water supply itself, whether increasing its proximity or reducing its intermittency.
Taking these two points together, baseline conditions were already at or close to “basic”, and the intervention brought them up to “basic”. This is not the kind of incremental change one might expect to have a substantial effect – the systematic review shows that diarrhoea impacts are sensitive to the change in level of service (my comment here). I am not arguing that small incremental changes are unimportant – on the contrary. Cumulatively over time, they can have a big impact. My point is that the kind of incremental changes studied in these trials (i.e. chlorination, toilet interface upgrade) are not the kinds which most WASH programmes currently focus on, or should focus on. More important transitions to characterise as “WASH” would be OD–>basic sanitation (which SHINE did, on the whole) and off-plot unimproved or faraway water –> basic water (i.e. water quantity interventions, which none of these studies did). In short, small incremental changes may be beneficial, but evidently not the very small ones assessed in WASH-B (and SHINE, to some extent) .
2. How have epidemiologists interpreted the results?
The recent consensus paper is the most important contribution to the debate around WASH-B and SHINE. Table 3 shows its five main messages, with my appended reflections. I would summarise their argument as follows: the theory behind the impact of WASH is still sound – we might just need bigger service improvements over longer periods of time to see a measurable health effect. Don’t give up on WASH because, when we change conditions enough to see an effect, history suggests it will be a big one.
What I see as missing in these debates is the value that people get from WASH beyond health. These benefits (e.g. time savings, quality of life and psychosocial factors) are substantial contributors to the overall economic benefits of WASH interventions. We are still very bad at measuring and valuing these benefits. I am biased because measuring quality of life gains from sanitation interventions is what I work on currently… but the point stands.
Table 3: Interpretation of consensus paper messages
Five consensus messages
1. “Despite high compliance, the evaluated WASH interventions – as delivered in these settings – had no effect on linear growth, and mixed effects on diarrhea”
Unlike some previous trials, we can’t explain away these results with interventions that didn’t do what they were supposed to. We have to face up to them.
2. “The biological plausibility of WASH interventions as public health interventions is not challenged by these findings”
Finding significant health effects in a trial depends on many environmental and social factors. Trials are important, but the underlying theory supporting an intervention is also important, especially when interventions are particularly hard to test with trials.
3. “Historically, large, population-level gains in child health have not been achieved without significant improvements in WASH services”
Big health gains will take many big cumulative environmental changes. This will take longer than you can assess in a trial, and is likely to come in several incremental steps.
4. “Current evidence suggests that basic WASH services alone are unlikely to have a large impact on childhood stunting”
OK, but basic WASH should deliver important economic benefits (time savings, quality of life). These are likely worth paying for, even in the absence of health gains, and cost-benefit analysis can help answer that question.
5. “The results of these trials do not undermine the new and ambitious SDG target of safely managed services for all”
Achieving “safely-managed” services (which these interventions did not) may be more likely to deliver health impact. However, delivering and sustaining safely-managed services may cost more than rural households in LMICs are willing and able to pay in the medium-term.
3. How should funders interpret the results?
I worry that funders (in which I include LMIC governments) may read the abstracts of these papers, or hear second-hand readings of them, and misinterpret the findings. It is easy to infer that “WASH doesn’t work”, which would be a mistake. It depends on how we define “WASH” and “work”, and on how long we are willing to wait to see cumulative gains.
Most WASH programmes make small incremental improvements in service level which may not bring substantial health gains in the short-term, but do have a good chance of doing so.
“Transformative” WASH might be more effective, but it would also be very expensive, both for me and for the users who need to sustain the service.
Users value the non-health benefits of WASH services too, such as time savings and quality of life benefits, and these have an economic benefit.
In terms of their actions, funders who consider the bigger picture should:
Continue to support “basic” services and the systems that sustain them, in the knowledge that later incremental improvements can build on those investments to achieve “safely-managed” services further down the line
Continue to have a broader impact by funding the sector-wide systems that are necessary for further extending and sustaining services for all.
This is not an argument that “transformative” WASH services are not achievable or desirable. Rather, I mean to say that sustaining safely-managed services may cost more than rural households in LMICs are able and willing to pay in the medium-term. In many settings, small incremental improvements will continue to be more realistic and affordable, both for users and funders.
Table 1: baseline situations and interventions in the WASH-B and SHINE trial settings, and their incremental changes
Note. this is a summary – full details are in the papers and their supplementary info. Approximate baseline conditions are based on my eye-balling across arms in the studies’ baseline tables – SHINE had 4 arms and WASH-B had 7.
This post explores ways of breaking down the “field” of WASH economics. On the one hand, one can argue that WASH economics doesn’t exist as a coherent field. After all, most people actually working on WASH economics questions are in the field of engineering and/or public health. There are fairly few people with an economics MSc or PhD who are working on WASH. I doubt many people self-identify as “WASH economists” in the way that health economists or environmental economists do.
On the other hand, one can argue that disciplinary boundaries are unhelpful in relation to a subject like WASH, which defies being put in a disciplinary box. Getting WASH services right often involves building stuff well, proper consideration of the underlying resources, reliable prevention of pathogen transmission, changing people’s behaviour, and doing all that with the right incentives, prices and funding/financing model. So WASH economics exists in the same way that public health engineering exists.
Personally, I would argue that WASH economics does exist as a field. However, it requires a little more shaping and communicating in the coming years to make it into a coherent one. Who cares what disciplinary background someone has as long as they’re solving a problem by (i) asking and answering the right question, (ii) using the right method, and (iii) doing a good job of it?
If an engineer is looking at preferences for different water services, and using economic methods to do so, then they are doing WASH economics. I can point to many recent papers that go in this box. Many are good or excellent, though definitely we can raise the bar by interacting more with each other.
How to divide up WASH economics sub-fields?
I already explored definitions of WASH economics in this post, going with “the study of how people make decisions about the allocation of scarce resources in the delivery and use of WASH services.” Within this there are many sub-fields. Below is my current way of sub-dividing it, drawing strongly on how my colleagues at the Centre for Health Economics in London (CHIL) at LSHTM divide up our work into themes.
Economic evaluation – including work on resource use and allocation (e.g. costing, cost-effectiveness analysis, cost benefit-analysis), the valuation of outcomes, and broader priority-setting questions such as how to include equity in decisions.
Economics of service providers and systems – including work on funding and financing, the role of government in providing WASH services, and the regulation of markets for WASH services. This is a broad area that would include questions of production and pricing (e.g. tariffs and subsidies), as well as the interaction of demand and supply at both the provider and sector level. Or, as is de rigeur, looking at the ‘WASH system’ as a whole.
Policy evaluation – including work on attributing impact (of any kind) to WASH interventions that use economic-based methods. There is a lot of cross-over with epidemiology / public health and “implementation science” here, but some methods/questions are more economic than others.
Choice and behaviour – including work on understanding the choices people make between different behaviours, products and services. This would include studies of demand drivers, preferences and willingness to pay. It may encompass work using behavioural economics or discrete choice experiments.
I hope to write a post exploring each of these in more detail another time.
An aside on disciplines
WASH research is not unique in often requiring multiple disciplinary inputs to be successful. Many of those working on health economics, for example, have backgrounds in statistics, medicine, mathematical modelling and epidemiology. Often these other disciplines are in fact required in order to answer questions properly. It is practically impossible to do many types of economic evaluation of a vaccine or a tuberculosis intervention without involving modellers, for example. Likewise, it can be hard to know whether a change in a health outcome is clinically relevant in a given population, without asking a medical doctor.
The same would apply to doing an economic evaluation of a complex sewer intervention without involving people who know how the system works. Some decisions that look odd to an economist may make absolute sense to an engineer.
While there are trained economists who work consistently on WASH, there are others who delve in and out of many sectors. Some of these, perhaps, do not truly attempt to understand WASH before employing their standard methods toolbox (which may not be appropriate to the question at hand). They might answer the wrong question entirely. Worse, they might answer a question that public health people already answered 5 years ago but they didn’t bother to read the literature properly.
But these people are few. My point is that it would be helpful to have far more trained economists deciding to work consistently on WASH over a long period. Economists who’ve entered WASH like Britta Augsburg, Molly Lipscomb and Dean Spears (and many of the r.i.c.e. team) have added lots of value to the sector. Likewise there are those with a ‘harder’ science background like Meera Mehta (architecture) and Marc Jeuland (engineering) who have all but switched to economics. We need more of both these kinds of people.
Lots of people work on WASH economics without considering themselves economists. This is great and to be encouraged. While it is something of a cliché, we do need to break down boundaries between fields and borrow methods from each other. We also need to improve the quality and quantity of work on WASH economics, both by those who are economists and those who are not. For this, it would be helpful to have (i) more trained economists deciding to work consistently on WASH, and (ii) a more coherent dialogue around WASH economics.
In a previous post, I proposed a working definition of WASH finance as “the study of how WASH services are paid for, including who pays, how and when”. There is also the huge question of what is paid for (i.e. level of service, technology, paying to cover capital or recurrent costs, how equitable it is, etc.). To simplify, let’s take that investment option as given. We could be thinking about a shared pit latrine, a gravity-fed network of rural standposts or a sewer network.
The “who, how and when” are important in the following ways:
Who pays: It matters who is fronting the money, not least because it will give them a say in what is done. Service providers should be accountable to users paying tariffs. Banks offering loans will set conditions. Money might come from the public sector, private firms, individuals/households, donors/NGOs, etc. Some or all of that money might be borrowed (i.e. repayable), so there are effectively multiple payers sharing risk.
How: There are many ways to provide money, whether through cash, debt, equity, etc. Money can be “cheap” or “expensive” if it is borrowed, or “free” if it is not. In-kind payment and unpaid labour can be important but would technically not cover a financial cost (see this post).
When: Timing is everything. Spreading costs can make things more affordable to those without deep pockets, but sometimes interest payments can be crippling. Possible scenarios include immediate payment in full, instalments spaced over time periods (with or without interest), instalments on delivery of contracted outputs etc. Capital itself has an opportunity cost, i.e. it could make a return if invested in something else, like government bonds.
In summary, finance is about how financial costs are covered. The above three dimensions are just an intuitive way I like to think about it – comprehensive frameworks exist for conceptualising WASH finance, discussed below. This post aims to scratch the surface of those in fairly simple terms.
Categorising different types and sources
“Finance” is a useful catch-all term, but it should really be split into funding and financing, as I mentioned via a quote in this post. In short:
Funding means providing money which is not expected to be repaid.
Financing means providing money on expectation that it will be returned in full, plus interest or dividends, so perhaps better framed as repayable financing.
More on the sub-categories underlying these are as follows. In the WASH context, funding usually comes from three “sources”, together known as the “3Ts” framework popularised by the OECD (2009).
Tariffs, meant in the usual sense as ‘fee for service’ but, conceptually, this bucket also includes self-supply expenditure (e.g. household-funded toilet construction) or user charges such as connection fees.
Tax revenue, which might be collected by different levels of government (local, municipal, state, national)
Transfers, e.g. from donors, NGOs, foundations or remittances
The best explanation I’ve seen of this is the table in the TrackFin guidance doc p.49, which shows how an accounting perspective (“types”) overlays with the 3Ts perspective (“sources”).
Repayable financing fits broadly within two categories:
Debt, which could be a concessional or non-concessional loan, bonds or loan guarantees, etc. – on which the principal must be repaid and interest can be paid
Equity, which could be a formal stake (or share) in a company, a PPP-based “contribution” to capital costs, etc. – on which dividends may be paid and the stake can be withdrawn.
The GLAAS 2017 report graphic at the top of this post shows how different countries use different mixes of funding and financing sources in their sectors. However, only the countries asterisked in the figure have done full TrackFin studies. I would take data for other countries with a pinch of salt (i.e. they are likely ‘best guesses’ by someone), because detailed information on flows is rarely available at the sector level. That is the whole reason why TrackFin was originated by WHO and is so important (but I’m biased as I was involved in the Tunisia TrackFin study and am fully bought into the approach ).
When is WASH finance cheap or expensive?
Funding is “free” in the sense that it has no cost of capital (though capital may be allocated an opportunity cost in economic analysis). Repayable financing, meanwhile, comes at a price. That price can be “cheap” or “expensive” depending on the terms and implied interest rate. Concessional loans from development banks (such as the African Development Bank or World Bank) are generally the “cheapest” type of repayable financing for WASH services. They allow borrowing at below-market rates but come with various conditions. Non-concessional loans can be secured from commercial banks which probably comes with fewer strings, but at full capital market rates.
In practice in low-income countries, service providers are unlikely to be able to secure repayable financing beyond these options. Most will simply not have the track record of revenue generation and loan repayment (and subsequent credit rating) to be able to issue bonds, for example. A graphic from IRC shows how this can work. With equity meanwhile, allowing private stakes in service providers may not be legally straightforward in many low-income countries.
There is increasing interest in the idea of “blended finance”, whereby public funds (or concessional loans) and commercial repayable financing are combined in a synergistic way. The objective is usually to make an overall package cheaper than the commercial market could provide, but still crowding in private capital rather than crowding it out. Sophie Trémolet gives a clear explanation in this episode of WASH Talk or there’s this World Bank report.
All sources of funding and repayable financing have a role to play in expanding and sustaining WASH services. However, some are more appropriate than others for (i) different purposes, (ii) different providers, and (iii) at different stages of a WASH sector’s development.
In a previous post, I wrote about how I see measures of quality of service (QoS) and quality of life (QoL) capturing different things which are both important. In this post, I expand on that, proposing a generalised way of thinking about this at different stages of the service chain. First, though, a few words on user satisfaction and objectivity.
Distinction between QoL and user satisfaction
In the last post I skipped over user satisfaction, but I think it’s important to describe the subtle difference between user satisfaction and QoL. In marketing, theoretical groundings of customer satisfaction are focused on the product. For example, one influential marketing theory focuses on expectations about a product’s “ideal”, “expected minimum”, “tolerable”, and “desirable” performance. For sure, this involves psychological processes, but it is focused on the product not the person.
To illustrate this, consider these two questions asking about a similar concept:
How satisfied are you with the cleanliness of the bathroom?
Can you feel clean while carrying out your sanitation practices?
It’s likely that responses for these two questions would be highly correlated. However, there is an important conceptual difference, and it’s rooted in whether the question asks about an attribute of the service/infrastructure or about an attribute of the person/QoL.
Subjectivity and objectivity
The distinction between subjectivity and objectivity in measurement is clearly important. Most measurements are unlikely to be objective in the philosophical sense. However, achieving objectivity in the scientific sense is often an implicit goal in QoS measures. To be classed as objective, the value for a QoS variable should theoretically be the same regardless of the rater. The mean queuing time at a public toilet should be more or less the same for all (by gender and time of day, at least).
Subjective measures, meanwhile, may differ a priori between people using the same service (I gave an example for SanQoL for users of the same toilet in the last post). Inter-rater reliability, i.e. whether the person asking the questions makes a difference, is nonetheless a concern in some measures. Of course, many sanitation QoS measures that are theoretically objective require some element of subjective judgement by the rater. For example, two different enumerators amy have different thresholds for answering yes/no to “are faeces present on the slab?”. So subjectivity is really a continuum not a binary attribute.
A matrix incorporating subjectivity and objectivity
I applied these ideas in a 2×2 matrix on QoS and QoL with respect to toilets – see below. Most should be obvious from this post and the previous, except for quadrant C on proxies. On the one hand, it may not be possible for anyone other than the individual themselves to scale their QoL adequately. However, out of necessity, health economists have developed proxy measures for people who are too young or incapacitated to rate health-related QoL themselves. It is not too hard to imagine the same for SanQoL, though hardly a priority right now.
This touches on the issue of “who is the rater?”. At the containment stage of the sanitation service chain, one can imagine many types of rater. For QoL it would ideally be the user themselves (though cf. proxy discussion above). For QoS it could be the user, a non-specialist observer (e.g. typical enumerator) or a sanitation specialist. The main difference between the specialist and non-specialist is that only the former would be qualified to inspect or evaluate sub-surface infrastructure. One example is the verification that what the user calls a “septic tank” really is a septic tank not a holding tank or soak pit (see bottom of p.36 in this Indonesian study re: discussion of “cubluk”).
Generalising to the sanitation service chain
Here’s a more generalised framework for QoS and QoL along the sanitation chain. Taking QoL first, the measure of SanQoL I’m working on is assessing QoL attributes of the user at the containment stage. At the emptying & transport stage, however, it would be more appropriate to assess QoL of workers. Same for the treatment & reuse stages. Some people are working on measuring sanitation workers’ conditions, for example, which would fall into that category.
QoL could be “objectively” proxied by a caregiver or observer, though with many caveats on the reliability of that data – better to ask the person themselves unless impossible to do so. As with the earlier 2×2 matrix, I would put user satisfaction in the box of subjective QoS metrics. This can be measured for containment services or emptying services.
The basket of “objective” QoS measures, meanwhile, is very large. It could range from infrastructure observations by an enumerator, through to measurements from technical instruments (e.g. for treatment efficiency) or financial and economic performance metrics.
Certainly there is plenty of room for better measurement of outcomes affected by sanitation interventions. In this post, I’ve explored the concepts of subjectivity and objectivity with respect to measuring QoS and QoL for sanitation services. Lots of people have developed measures in this space and one thing I want to do is map this out across the above generalised matrix.
[I was chuffed to be joint winner in the ESRC ‘Better Lives’ writing competition with the below piece intended for a general audience. The Guardian published an abridged version. Deadlines are 80% of writing, for me at least… so I would recommend competitions like this for forcing oneself to write to a hard deadline for public engagement, not something researchers often have to do. A more researcher-facing post on SanQoL is here.]
Imagine not having somewhere safe to go to the toilet. Really imagine it – leaving your house and defecating behind a bush or a building. It’s hard to bend your mind to consider that, if you’ve had access to a clean, comfortable bathroom since you were a child. However, around the world, 900 million people have no option but to defecate in the open. A further 1.4 billion use a toilet that doesn’t meet World Health Organisation standards for ‘basic’ toilets, meaning that it could still be a direct source of disease.
Fortunately, lots of investment is being made in sanitation in poorer countries – many billions of pounds in fact. Right now, somewhere, a municipal official is drafting their budget and a charity worker is writing a funding proposal. There are hundreds of ways that money could be spent. However, we don’t know enough about whether money is being spent on the right programmes. Inefficient choices are certainly being made.
How can I know this? Aren’t there established economic techniques for comparing ways to spend the money? There are, but they predominantly focus on health, alongside some consideration of time savings and avoided costs. This is a problem because health is not the thing people value most about sanitation. When researchers ask people the reasons why they invested in a toilet, health is usually far down the list. Concerns about privacy, safety, or pride are usually at the top. Together, we can call these improvements in quality of life in general. Excluding them from economic comparisons is a glaring omission.
Economists are very concerned with what people value in their lives. We think that the highest valued changes are the most important when deciding between project A and project B. So why don’t we just measure privacy, safety, etc., and plug that into the economic models alongside health? The challenge is that these things are not easily measured – they are subjective perceptions and vary from person to person. There’s also the problem of how to select the different elements, and then the problem of weighting them. These problems are not insurmountable – such quality of life measures exist for comparing programmes in health or social care. For example, the NHS in the UK makes huge funding decisions based on ‘quality-adjusted life years’. This measure takes account of how people value changes in health, by weighting life years with a ‘health-related quality of life’ scale. However, nobody has yet developed one for sanitation programmes.
That’s where my research comes in. I’m working on a measure for sanitation-related quality of life, building on the experience of health. The challenge is to measure this by asking people less than 10 questions, so it is manageable for regular use. The questions need to reflect what people value most about having a toilet. My work is based in Mozambique which is one of the poorest countries in the world. In collaboration with a local research team, we started by interviewing people living in slum settlements in the capital city, Maputo, both on their own and in groups. These were ordinary men and women, young and old. They all used different kinds of toilets, some good, some terrible.
First we asked them what was important for a good life – people often mentioned having enough food, having a good house, and having happy children. Then we asked about how a good toilet or bad toilet contributed to each of those things. Many interesting and important stories emerged, some happy, some sad. One young woman talked about privacy: “Anyone that passes can peep at you if you are going to urinate or defecate.” An older man talked about no longer feeling embarrassment when entertaining relatives: “When visitors come to see us now, they won’t feel bad when entering the toilet”. By systematically analysing these conversations, I identified a list of issues that kept coming up. The list of issues was too long, so I narrowed it down by showing people sets of three options and asking them to choose which was the most important. By doing this lots of times with many people, it is possible to exclude the least important items, and use statistical techniques to develop weights for those that are left.
So how will this improve people’s lives? My measure of sanitation-related quality of life can be used to compare different investment options. It can be used by the municipal planners deciding where the billions of pounds are spent. Through using it, they can know which types of investments improve quality of life the most, by focusing on what people value about sanitation. Considered alongside data on costs, health and engineering considerations, this can make for more efficient use of public funds. That means more people using a toilet which makes them proud, safe and comfortable.
This research is part of the MapSan Trial supported by the Bill and Melinda Gates Foundation. Ian is funded by an ESRC doctoral studentship.
In a previous post I explored the concept of sanitation-related quality of life (SanQoL). In that post I distinguished between “quality of service” (and/or infrastructure) and quality of life. The present post expands a bit on that distinction.
1. Different ways of measuring “quality”
Consider these three questions for assessing the concept of privacy:
[observe] Does the bathroom* lock from the inside?
[ask respondent] Does the bathroom you usually use lock from the inside?
[ask respondent] Can you use the bathroom without people interrupting you?
Here’s a different angle on privacy:
[observe] What is the main material of the bathroom’s walls?
[ask respondent] What is the main material of the walls of the bathroom you usually use?
[ask respondent] Can you use the bathroom in private, without being seen?
All these are like ones in surveys I’ve been involved in over the years. In each case, the ‘A’ questions involve an enumerator (who is not a user of the bathroom) observing infrastructure and making a judgement. In the ‘B’ and ‘C’ questions, the enumerator asks a respondent user, who goes through a cognitive process to respond.
The main distinction between ‘B’ and ‘C’ is that ‘B’ questions are asking about the infrastructure / service, and ‘C’ questions are asking about the person’s experience in using the bathroom. I would argue that this is at the heart of a distinction between ‘quality of service’ (QoS) and ‘quality of life’ (QoL). I would argue that, ‘A’ and ‘B’ are measuring QoS and ‘C’ is measuring QoL, and I expand on this below. For ‘C’ to be measuring QoL, we need to hypothesise (i) that things like privacy, safety, dignity etc. affect QoL, and (ii) that sanitation affects those concepts. However, there are plenty of studies that support these hypotheses.
2. Infrastructure only tells you so much
There are also some attributes of SanQoL which cannot be proxied by questions or observations about presence or quality of infrastructure. These include shame, pride, safety, all of which are about perception rather than infrastructure.
For example, consider these ‘C’-type questions:
Can you use the bathroom without feeling ashamed for any reason?
When visitors come, do you ever feel embarrassed providing this bathroom for them to use?
Are you able to feel safe while using the bathroom?
How often do you use a bucket at night for due to fear of using the bathroom?
I’m not arguing that QoS approaches are not useful. Infrastructure is important, and observations can be rapid and useful. I’m just trying to show how QoS and QoL approaches measure different things. Of course, there are also other aspects of QoS which are about users not infrastructure (number of users, distance from house to toilet etc.). And some composite QoS measures may incorporate items which measure QoL.
There is also the issue of questions about user satisfaction with the service, which go in a slightly different box unless they directly measure QoL attributes. For example, you can ask “how satisfied are you with the cleanliness of the toilet”, which is arguably still about infrastructure rather than QoL. Where the distinction is less clear is with questions like: “how satisfied are you with the level of privacy?”. More on this another time. In short, I would argue that to be measuring QoL the question has to specifically be measuring functionings (as with the widely-used EQ-5D , SF-36 or WHOQOL-Bref) or capabilities (as with the ICECAP-A)
3. Are QoS and QoL distinct?
Above, I wrote that the ‘A’ and ‘B’ questions are measuring QoS and the ‘C’ questions are measuring QoL. But how distinct are these concepts really? It is possible to argue that the ‘C’ questions are really part of QoS as well. However, I think it makes more sense to see them mostly as separate things, as what is being measured is fundamentally different.
‘C’ questions are not really measuring the service itself – they are measuring a person’s perception of how that service affects them. More specifically, given the capability-based formulation of the questions, they are measuring what a person is able to do with respect to their sanitation practices.
Here’s an example. Imagine a pit latrine with slab shared between 4 households, with no roof and some fairly dodgy walls, that is not kept particularly clean. It is easy to imagine that service being experienced completely differently by different people (e.g. a 30-year old man as opposed to a 16-year old woman or an infirm 80-year old of either sex). As a consequence, while QoS measures might give the same result, each individual’s SanQoL might be markedly different.
Therefore, both toilet and user characteristics matter for SanQoL. However, these are far from its only determinants. Population density, levels of community security, environmental conditions, trust between neighbours etc. will all affect SanQoL but not most measures of QoS.
This post has just scratched the surface of a complex issue, I am characterising QoS measures in a certain way, when they are certainly many and various. My main point is that QoL focuses on outcomes in user’s lives, rather than infrastructure or levels of service.
* As a Brit it pains me to use such an Americanism… but it is for good reason. SanQoL does not distinguish between sanitation practices that people carry out, in the given place they call the bathroom, toilet, casa de banho, sintina, choo, charpi etc. As it is conceivable that people bathe in that space (as they do in the Maputo setting I’m working in), then “bathroom” is more appropriate than “toilet”.
Since investment options are always compared under a budget constraint, economic evaluation aims to inform unavoidable decisions and support allocative efficiency. Various economic evaluation methods (such as cost-effectiveness analysis and cost-benefit analysis) compare costs and consequences of alternative interventions.
Improvements in sanitation can impact on health, and it is typically health outcomes such as averted deaths and diarrhoea cases which have been valued previous cost-effectiveness analyses. Cost-benefit analyses have gone further, also valuing time savings, productivity and other economic benefits.
However, none of these economic evaluations have incorporated the broader “quality of life” (QoL) benefits of sanitation, related to privacy, safety, pride, dignity etc.* Some studies call these “intangible”. However, these broader QoL benefits are often the primary demand-side drivers of sanitation investment by poor households.
Therefore, economic evaluations which overlook these outcomes may lead to in misallocated resources. At the very least, they will not fully reflect the value placed on sanitation by users.
Where my PhD research comes in
Can anything be done about this? If there was a quantitative measure of ‘Sanitation-related quality of life’ (SanQoL) that represented people’s valuation of QoL benefits, it could be used as an outcome in cost-effectiveness analysis. Further down the line, a SanQoL measure could potentially facilitate monetary valuation of QoL outcomes in cost-benefit analysis.
Taking the first steps in developing and applying such a measure is 50-60% of my PhD. I’ve kept fairly quiet about it so far while doing lots of reading and initial qualitative work. Now, however, the ideas are clear enough to share, for reflection and critique. I can’t write about interim results, unfortunately. Hopefully I’ll get some abstracts into academic conferences later in the next year, and then comes papers… but I can write about ideas at least.
This post sets out a bit of my rationale and thinking – in the rest of this post there’s space for two issues, (i) why qualitative grounding is important, and (ii) what distinguishes a psychometric approach from an ‘observer’ approach. How I’m defining and framing QoL itself in all this will have to wait for another post. In short, I’m using Sen’s capability approach. However, I’m also building on the concept of ‘health-related quality of life’ which is widely used in health economics for valuation of health states in support of quality-adjusted life years.
What do people value about sanitation? The importance of qualitative work
Value is a key concept in economics. Demand for a service in any sector is a derived demand for a stream of benefits valued by the user. For example, demand for healthcare services is a derived demand for health itself. So, demand for household toilets is a derived demand for the benefits of sanitation. These are many and various (see above) so any measure of SanQoL would likely need to be multi-dimensional.
A measure of SanQoL would ideally capture all QoL-related outcomes perceived by as important by users. What are those outcomes? First port of call is looking into the literature, but best practice is to ask people directly in the setting of interest, in support of validity (amongst other reasons)
The setting in which I plan to test the quantitative measure is low-income settlements of Maputo, Mozambique. Accordingly, I did some qualitative interviews and focus groups with users of different types of sanitation services in 2018. To identify the relative value attached to different ‘attributes’ of SanQoL, I also included some ranking and triadic comparison exercises. More on that once I get the qual. paper written…
Measuring SanQoL quantitatively
Having a rich qualitative description of how people think sanitation relates to ‘a good life’ is important and helpful. However, in order to be practically used in economic evaluation, it requires translation into a quantitative measure. For example, it would be helpful to be able to say “this intervention cost US$ 1,000 per incremental point on the SanQoL scale, as compared to that intervention.” You can’t do this with qualitative data.
Such a SanQoL measure would need to be experiential or psychometric, in that it would be “imposing measurement and number upon operations of the mind”. For a given dimension of SanQoL (privacy, for example) this involves asking people to scale their own level on that dimension, by recalling their experience (e.g. variations on “do you feel like you have privacy”). The non-psychometric way to do this would be an external observer looking at a toilet and scaling the level of privacy provided (e.g. variations on “does this toilet provide adequate privacy”). This ‘observer approach’ measures something fundamentally different, that is, quality of service (or infrastructure) rather than quality of life. Individuals themselves are best-placed to scale their quality of life.
I have developed some questionnaire items on the basis of the qualitative work and will be running a quantitative survey in April, including piloting and cognitive interviews beforehand to make sure the items are well-understood. Of course, even if that work is successful, the measure will not necessarily be valid in other settings. More on all this another time.
I hope that useful insight will come out of this work to inform how sanitation-related quality of life can be measured. My primary purpose is in economic evaluation, but there are many other potential uses of such a measure if it can be validated across settings. I hope to write occasional blogposts on other aspects of SanQoL in due course. Please contact me if you are working in this field as well and want to discuss.
*At least, none have done so quantitatively – Hutton et al.’s East Asia phase 2 studies did so qualitatively.
Today I attended not one, but two, seminars on CLTS. The first was Britta Augsburg presenting results of a recent cRCT of a WaterAid CLTS intervention in Nigeria (at LSHTM). The second was Dale Whittington reflecting on CLTS trials in the last few years and his recent CBA paper incorporating their results (at Oxford). A key theme from both is targeting. Britta’s analysis shows that interventions should be targeted at the settings in which they are most likely to be most effective. Dale’s analysis shows that resources should be targeted to places where net benefits can be maximized.
1. Britta’s presentation
Britta presented the results of a cRCT reported in this IFS working paper (soon to be updated) which evaluated a 2014 CLTS intervention by WaterAid Nigeria. They found that the intervention achieved only a 3 percentage point decrease in OD (at the very low end of recent experience). That doesn’t necessarily reflect well on the quality of implementation, but it’s not the most important result. More intriguing is the sub-group analysis they did for community-level wealth measures. Looking separately at ‘richer’ and ‘poorer’ communities (as defined by asset wealth), they found a 9pp decrease in OD in the poorer communities and no significant decrease in the richer communities. It’s a carefully done piece of work – they went to great pains to show that this key result is robust to different SES measures and not driven by baseline differences in sanitation coverage (see table 5, p.19 in the paper).
Most importantly for external validity, they reanalyse data from other trials and find that impacts on OD are stronger in poorer contexts in those settings too – see figure below (from the PPT, updated since the paper). This is an important insight. Read the left and right axes first – in short, it is plotting treatment effects by study and the average ‘night light index’ over the areas of study at baseline (a proxy for economic activity). The figure shows that the higher the night light index, the more likely the study was to report statistically insignificant reductions in OD. In other words, “heterogeneity in CLTS impacts across studies can be rationalised by differences in the average area wealth”.
The analysis is notable for looking at wealth effects at the community level rather than the household level.
The wealth effect is an appealing idea, but if correct, why does community wealth affect CLTS effectiveness? A lot of the avenues they tested for this (community social interactions, public infrastructure, leader characteristics) were not statistically significant as drivers of CLTS effectiveness (nb. this was in the PPT but not the working paper – assume it will be in new version). So, the wealth measure is picking up something, but it’s not clear what it is?
Britta argued that the effect is strong enough to be used to target CLTS in the right places, especially since asset data is widely available in DHS. In other words, secondary data can be used to target CLTS intervention in poor areas with high OD, where it is more likely to be effective. These are probably areas of greater need, too, so it is not a hard sell (except that implementers may well face higher costs in poorer areas as they may be harder to reach).
Note that this intervention was pretty light-touch, only 1 day of CLTS activities. Almost all triggerings did defecation mapping and action planning, but almost none did transect walks, graphic exercises, and medical expenses calculations. Follow-up was also light-touch (see bottom of p.5). Would the effect have been greater (and the wealth effect more or less visible) if the intervention had been more intensive?
2. Dale’s presentation
Dale summarised the rationale for CLTS, reflected on the 10 or so recent sanitation trials, mostly funded by BMGF to the tune of c.$100m. He set out that a lot of these trials showed weak evidence for the impact of CLTS, with limited impacts even on reducing OD in many cases. This is not news, but his argument was a more subtle economic one: CLTS interventions are essentially risky investments. Despite this, uncertainty around costs, and especially benefits, is not much considered by sanitation advocates. He argued this with reference to a cost-benefit model of a hypothetical intervention, set out in this working paper. That model further elaborates earlier versions he (with co-authors) has been applying since 2008, and in an important version in 2012.
The main conclusions from the model are that while CLTS has a positive NPV only about two thirds of the time in their probabilistic sensitivity analysis. The BCR is rarely close to the high values of 5-7 seen in global studies like Hutton’s in 2012 and 2015. Rather, it is 1.5 on average in the base case (ranging from 0.6 – 2.3 depending on the level of uptake). In summary, his is a more skeptical (or perhaps, realistic) take on the economic performance of sanitation. The argument was not that we shouldn’t do sanitation programmes / CLTS, but that we need to be much more careful about selecting where we do them, and that ‘global silver bullet’ interventions shouldn’t be blindly applied in very diverse contexts.
He ended by pondering whether, in dirty environments with multiple exposure pathways for pathogens, making marginal tweaks to toilets or similar is going to change very much. Substantial gains in public health may only be made by upgrading all of housing, roads, drainage, WASH etc. Timing and sequencing are likely to matter. As asides, he also noted (i) the hubris of many of the RCT teams in flagging the importance of non-health benefits in their conclusions but mostly neglecting to measure them, (ii) the fact that almost none of these trials seem to have collected data on costs, or other data that would allow economic evaluations.
The basic argument and results in the new paper are not so different from the 2012 paper. However, the new model makes substantial innovations, such as:
Synthesising effectiveness evidence on the most recent RCTs on CLTS
Presenting results with and without a “sanitation externality” effect and, crucially, showing that it doesn’t make that much difference to the economic performance.
modelling at the level of a district of 100,000 people, thereby allowing for heterogeneous effects (high / medium / low uptake) in different villages.
These innovations are important because they make the conclusions even more convincing that previous iterations. In the first year of my PhD I read pretty much every sanitation economic evaluation ever done (at least, all the CEAs and CBAs on this list), and this new model is now the benchmark for comprehensiveness.
His concluding point about needing to change multiple environmental conditions to see substantial public health gains reminded me of this Gambian natural experiment published a few months ago. It supports the argument that very high environmental conditions are needed to close the stunting gap.*
The point about people not bothering to measure non-health effects is close to my heart. Half my PhD is about developing a measure of ‘sanitation-related quality of life’ (SanQoL) that would aim to make measuring these outcomes more routine in trials and in general M&E. More on that in due course…
That so many well-funded WASH trials not including any economic component is close to scandalous. In the UK, for example, it is more or less the norm for health trials funded by the NIHR to include an economic evaluation component. Of course, my health economist colleagues still manage to gripe about not being given enough time or not being brought onto the team at the design stage. It is similar in global health trials, but at least the work is done and having an economics component is the norm. In WASH we seem to completely ignore economics in most trials (e.g. costing, cost-effectiveness, cost-benefit) , as if decisions will be made on effectiveness alone.
In conclusion, given recent trials, the benefits of CLTS are probably lower on average than we thought. Given the Crocker (2017) costing study the costs of CLTS are probably higher on average than we thought. However, the whole point of both Britta and Dale’s presentations is that the average is not that helpful. Interventions should be targeted at the settings in which they are most likely to be most effective (and not used at all in places where they aren’t likely to be). Furthermore, resources should be targeted to places where net benefits can be maximized. For CLTS, if Britta’s analysis is correct, these places are fortunately also likely to be those that most need support.
* this isn’t the place to discuss the Gambia study in more detail, but it has some limitations: (i) very small sample, (ii) the key analysis, involving splitting the 1A/1B groups, was not planned ex ante (leaving it open to accusations of p-hacking), (iii) we don’t know why the 1A group lived on the MRC compound but 1B didn’t, (iv) we would need to know more about hygiene behaviours, toilet type and animal contacts to drawn stronger conclusions. This doesn’t take away from that fact that it’s a very interesting study, and the differences between 1A and 1B are substantial.
The JMP’s online analysis tool allows water supply data to be cut by service level (safely managed, basic etc.) or facility type (piped, non-piped), as set out in their 2018 methods doc. “Piped”, W2 in their indicator list (p.4) includes all tap water classifications (p.9), i.e. both on-plot piped and off-plot kiosks or public taps. Non-piped improved, W3 in their indicator list, includes all other improved services, namely:
Groundwater point sources (protected springs, wells and boreholes)
Rainwater (e.g. from house roof)
Packaged and delivered water (e.g. bottles, tanker trucks)
Since categories 2 and 3 form a very small proportion in rural areas in most countries, the vast majority of “non-piped improved” comes from groundwater. A lot of “piped” may do too, but that depends on the network’s sources and the data cannot be collected via household surveys [let me know if you know of data on the relative split of surface/ground water in rural African piped systems]. A shift towards on-plot piped supplies in rural areas will hopefully take place, and this may or may not involve more reliance on treated surface water.
The analysis of JMP data below came out of ideas kicked about by Richard Carter and myself as a follow-up to our 2016 paper on functionality. One day we might manage to finish the follow-up paper, but in the meantime here’s some analysis of some JMP data, followed by a few thoughts.
First, Table 1 below shows data from the JMP “world file” available here. I used that file to extract the proportion of the rural population in 2015 using different service levels and facility types for drinking* water, for the five most populous Sub-Saharan African countries. This mainly serves to bring clarity on what is included in each category in adding up to 100%.
Table 1: Access to rural water supply for drinking water in five African countries in 2015
Second, I calculated the number of the rural population in 2000 and 2015 using each of ‘piped improved’ and ‘non-piped improved’ for the same five countries (Figure 1) for Sub-Saharan Africa as a whole (Figure 2). Note that populations using an unimproved point source or surface water are not shown. The white-filled bars indicate projected total rural population in 2030 from the UN World Urbanisation Prospects 2018 database – if we achieve universal basic access by 2030 these bars would be filled with one of the two colours (remember that “basic” also implies round-trip time <30mins).
Figure 1: Rural populations in five African countries using piped and non-piped improved water.
Figure 2: Rural population in Sub-Saharan Africa using piped and non-piped improved water.
With the slight exception of Ethiopia, the absolute increase in the number of people using ‘piped improved’ supplies in rural areas has been relatively small (Figure 1).
This comes in contrast to the huge increases in all five countries in non-piped improved, also visible for Sub-Saharan Africa as a whole (Figure 2).
The increases in population using non-piped improved (as opposed to piped on-plot) was approximately 2x higher in DRC, Ethiopia and Kenya, 4x higher in Tanzania, and 18x higher in Nigeria. It was 2.9x higher for Sub-Saharan Africa as a whole.
As a group of facility types, “non-piped improved” has seen astonishing growth in the 2000-2015 period in these large countries and in Sub-Saharan Africa as a whole. The majority of these service types (i.e. all except rainwater and packaged/delivered water) are based on groundwater. The numerical (Figures 1 & 2) and proportional (Table 1) significance of these groundwater-based supplies, and their likely future importance, are difficult to ignore.
While it would be ideal to have piped on-plot supplies for all by 2030, this appears unrealistic given current access trends (not to mention other constraints). In other words, it is hard to see the blank 2030 bar being filled with blue, and don’t forget that the blue category includes off-plot taps/kiosks. Meanwhile, “at least basic” off-plot services for all in rural areas by 2030 appears an achievable goal (noting that “non-piped improved” is not the same as “basic” – see p.16 of the JMP methods doc). Nonetheless, achieving piped on-plot** services should be targeted for as many as people as possible, where incomes and willingness to pay are sufficient to fund operational costs to keep piped systems running.
These data show that point-source groundwater supplies are likely to remain important to 2030 and beyond. In a recent book chapter, Calow et al. (2018) show the nuance of what this means for equity in access. We would do well to note their ‘three priorities’:
Invest in water resource assessment and monitoring
Recognise that degrading water quality poses at least as great a risk to drinking water as over-exploitation
Engage in the wider conversation about water resources management – who gets what as pressures on resources increase and climate change accelerates.
In conclusion, this post makes three points: (i) rural water supply access gains over 2000-2015 in African countries relied predominantly on point-source groundwater supplies, (ii) point-source groundwater supplies are likely to remain important to 2030 and beyond, (iii) a shift towards on-plot piped supplies in rural areas will hopefully take place, which may or may not involve more reliance on treated surface water. However, the shift is unlikely to be fast enough such that universal access to this type of service is achievable by 2030 in rural areas of African countries.
*Surveys which underlie JMP ask about the primary source of water for drinking. Other studies have established that people use multiple water points (improved and unimproved) if available, especially for purposes other than drinking.
** Note that in some countries, a substantial proportion of non-piped improved sources are on-plot, see figure on p.29 of this JMP report on safely managed drinking water.