WatSan.eu Feeds

🔒
❌ About FreshRSS
There are new articles available, click to refresh the page.
☑ ☆ ✇ WASHeconomics Blog

Quality of what? From whose perspective? Thoughts on measuring sanitation quality

By: IanRoss

Yesterday I was on a panel at the UNC Water and Health conference side-event entitled “What is “Quality” Sanitation? Investigating Service Standards and User Experience in Rural and Urban Settings”. You can watch back the event here (if registered for the conference, which is free). Below follows some of my views on the measurement of quality, covering three aspects: (i) quality of what? (ii) quality from whose perspective? (iii) quality for what purpose?

1. Measuring the quality of what?

As background to “quality of what”, here’s three blogposts I wrote last year:

  • This post explains the concept of sanitation-related quality of life (SanQoL)
  • This post discusses differences between “quality of life” and “quality of service”
  • This post discusses subjectivity / objectivity, and generalises to the service chain beyond containment

In summary, measures of quality of life (QoL) focus on subjective user-perceived outcomes (e.g. how safe the user feels). By contrast, measures of quality of service (QoS) focus on infrastructure attributes (e.g. presence of an inside lock) or level of service attributes (e.g. number of households sharing). Both QoS and QoL measures provide useful information and can be complementary, but it is important to be clear on whose perspective they provide, to inform their intended purpose (Table 1).

An example of a QoL measure is SanQoL, and I hope to publish a paper on its development, validity and reliability soon. An example of a QoS measure is the “sanitation quality index” (SQI) presented in the side-event (link at top) by EAWAG and others. Both SanQoL and SQI focus on the containment stage, but QoL and QoS have applications for emptying, treatment etc.

Table 1: Quality of what?

2. Measuring quality from whose perspective?

There are three aspects to this: (i) user or decision-maker focus, (ii) what goes in the measure, and (iii) who is evaluating/scaling its attributes.

First, the two main focuses for evaluating quality are typically the user or the decision-maker. For users, this might be direct (by asking users, e.g. SanQoL) or by proxy through evaluating infrastructure characteristics and inferring user-perceived quality indirectly (e.g. SQI). For decision-makers, sanitation regulation, for example, may give explicit or implicit guidance on minimum quality standards. In some cities, there are debates as to whether container-based sanitation should be permitted. Furthermore, downstream implications of quality for broader impacts are also important to decision-makers. The most obvious example here is health of the broader public via externalities (e.g. when septic tanks discharge to drains, illegal dumping of sludge, leaking sewers). This is a broader point raised in the first two presentations in the side-event. In summary, if you ask users and decision-makers about what is important for sanitation quality, you are likely to get slightly different answers.

Second, on what goes into a measure of quality, QoL measures generally follow the principle that the inclusion of attributes/items should be based on the priorities of the target population. That means consulting users rather than “expert opinion”. The qualitative study for the SanQoL measure is currently undergoing peer review. The same should go for QoS measures, and qualitative work underlying the SQI was recently published here (leading them to include things like material of floor, roof, solid waste, insects etc.)

Third, the question of “whose perspective” during fieldwork is also important (Table 1). For assessing infrastructure attributes (slab, walls, lock on door) there are many options, e.g. an enumerator can observe, or ask the user, or both. In theory, such indicators should be objective. However, there will always be an element of subjectivity when an enumerator is evaluating presence/absence of faeces, or the material of the walls is (e.g. how large do holes in corrugated metal sheets have to be until privacy is unlikely?). When it comes to less observable technical aspects (e.g. lining, septic tank or holding tank?) this can be very hard to verify regardless of who is responding. However, level of service attributes which are part of QoS (e.g. number of users, sharing status), cannot be observed and users must be asked.

When it comes to QoL (e.g. user perception of safety, privacy), only the user is in a position to respond, and each individual is different. This highlights an important distinction between QoS and QoL measures. Results for QoS measures should be identical or near-identical when asking two users of the same toilet (i.e. very high inter-rater reliability). For QoL measures, however, there is no reason to expect this. There should probably be correlation, but two users might experience the same toilet very differently (see example in this post) and so have different QoL outcomes. In the QoL case then, it is not a question of inter-rater reliability, though inter-rater methods could be used to assess the level of correlation.

3. Measuring quality for what purpose?

The purpose of informing regulation or defining standards was flagged above. The SQI has its roots in WSUP wanting to push back on the SDG sanitation ladder categorising shared sanitation as “limited”. Other possible purposes of measuring quality are routine M&E (e.g. by municipalities or NGOs), impact evaluation aspiring to causal inference, or economic evaluation. SanQoL could be used in impact evaluation – it would be useful to compare multiple intervention options in the extent to which they improve QoL. However, the ultimate intended purpose of SanQoL is in economic evaluation, that is, cost-benefit analysis (CBA) or cost-effectiveness analysis (CEA). I have developed a measure of the value of sanitation, weighted using SanQoL, which could be used in CBA or CEA. More on that in due course… The broad point here is that the purpose of measuring quality should inform the design of any measure. Economic evaluation frameworks are quite restrictive in terms of what methods should be used to infer value, for example. Routine M&E may be restricted by what data are feasible to collect quickly, and implementers may be more interested in basic aspects of QoS from a quality control perspective.

4. Conclusion

When measuring the quality of sanitation, it is important to be clear on three things: the quality of what, quality from whose perspective, and the purpose which the measurement aims to inform. To illustrate these points, I have distinguished between quality of service (QoS) and quality of life (QoL), and I continue to believe both are complementary. Some purposes require QoS data and others QoL data. If the ultimate aim is to understand user experience, it is probably important to collect both, in order to explore QoS variables as influences on QoL, as the Clean Team study presented in the side-event aimed to do.

washeconomics

☑ ☆ ✇ WASHeconomics Blog

New paper – trial-based cost-benefit analysis of a CLTS intervention in Ethiopia

By: IanRoss

title

My colleague Seungman Cha has a paper out this week, which I co-authored with him and others. It’s a trial-based cost-benefit analysis (CBA) of a community-led total sanitation (CLTS) intervention in rural south-western Ethiopia. We estimated intervention delivery costs from financial records and recurrent costs from the trial’s surveys. All outcome data are from the trial (health, time savings, cost of illness) – the trial effect paper is accepted pending revisions at AJTMH, but the protocol is here. Avoided mortality comprised ~60% of benefits, and the base case benefit–cost ratio (BCR) was 3.7. In probabilistic sensitivity analysis, 95% of estimates of the BCR were within a range of 1.9–5.4.

nb. the paper was already under review at IJERPH when the recent controversy about prioritising open access waivers for high-income (!) countries came out. I won’t be submitting more papers to MDPI journals, or reviewing for them, until they report results of the apparently ongoing internal investigation and ensure nothing like that happens again.

Obviously I think you should read the paper. In case you need some persuading, here’s  three important things about it. One is a broad conclusion about the intervention, and the other two are about (the lack of) ex post and/or trial-based economic evaluations in the sanitation sector.

1. Upgrading low-quality latrines

The story of this intervention in Ethiopia is about upgrading poor-quality latrines, rather than ending OD. More detail is in the effects paper when it comes out. OD was already <5% at baseline, so not your typical setting for CLTS. About 75% of intervention group households used private latrines, of which almost all were unimproved. The other 25% used neighbours’ or communal latrines. What the intervention achieved (Figure 1 in the paper) was (i) ~100% coverage of private latrines of varying quality, and (ii) substantial upgrading from unimproved pits to improved (and “partially improved”) pits. Definitions are in the paper, but “partially” in this study essentially meets the JMP definition of improved . Achieving ~70% coverage of ~improved latrines, as opposed to the unimproved latrines typically achieved under CLTS, was probably a key factor in seeing the effect on longitudinal prevalence of child diarrhoea. High baseline usage of latrines of one type or another is also why the value of time savings comprised <30% of total benefits, which is small compared to many such studies.

2. First fully trial-based economic evaluation of a sanitation intervention

Our paper reports a “single study” economic evaluation , meaning that all key* parameter values come from the specific setting rather than being assumptions from the literature. I know of only one other sanitation study which does this: a cost-effectiveness analysis based on a case-control study of a latrine intervention in Kabul, Afghanistan, in the late 90s (Meddings et al., 2004). Our paper is therefore the only fully trial-based economic evaluation of a sanitation intervention, despite such study designs being pretty common in public health. Trial-based economic evaluations are valuable because they show the economic performance of an intervention in real conditions, with high internal validity. They are also not hard to bolt onto existing trials. In my opinion, many or most impact evaluations (RCTs or otherwise) should include an accompanying economic evaluation, if they are to influence investment decisions. It is surprising that researchers do not do them and funders do not demand them, as others have argued recently (Whittington et al., 2020). It is not enough to know whether interventions are effective – we also need to know whether their benefits justify their costs. More importantly, we need to compare the relative economic performance of competing WASH intervention options.

3. Very few ex post economic evaluations of real interventions more broadly

Our Ethiopia study presents an ex post CBA of a specific sanitation intervention – that is, the intervention actually happened. It is quite surprising just how many studies in this literature are of hypothetical interventions. There are only four other examples of ex post economic evaluations of sanitation interventions I know of, in addition to the Afghanistan study above. Two studies in India combine primary cost data from the setting with health impact estimates from secondary sources (Hutton et al (2020); Dickinson et al, 2014). The East Asian studies synthesised by Hutton et al. (2014) also combine primary cost data with secondary outcomes (and are immensely detailed in the country-level reports), though they focus primarily on technologies rather than interventions. Finally, a further Indian study by Spears (2013) combines secondary data on both costs and outcomes.

Hypothetical studies can be very informative, such as a recent one which explored how the extent of uptake (and other factors) influences the economic performance of CLTS (Radin et al., 2020). However, to be able to make investment decisions about which sanitation interventions are most efficient, we need more studies that evaluate interventions which actually happened! Interestingly, the coverage increase for improved latrines achieved by the intervention (~35%) in our Ethiopian study was the same the “high-uptake” scenario in the Radin et al., 2020 hypothetical study, and our headline result is almost identical. However, note the discussion regarding definitions above – the intervention increased coverage of “JMP-improved” latrines by ~60%. That’s quite a lot of upgrading, and a fair amount of new construction as well.

Conclusion

In conclusion, read our paper and reflect on toilet upgrading in rural areas! But more importantly, if you’re currently running or planning an impact evaluation, strongly consider adding a cost-effectiveness or cost-benefit analysis to the protocol. The incremental effort of collecting good-quality cost data is very low compared to the overall research cost of your study. As is the incremental effort in carrying out a cost-effectiveness or cost-benefit analysis. Effectiveness estimates only tell us so much – economic evaluations help us make decisions about investing scarce resources. If the intervention “works”, one of the first things you’ll be asked is how much it costs…

======

* OK, the case fatality rates come from the Global Burden of Disease study, but very few WASH trials are powered to have mortality as an outcome. Likewise the estimate for value of a statistical life (VSL) is secondary – there are precious few VSL studies in LMICs, let alone undertaken as part of a trial such as this. My point is that the key sources of data for benefits are primary (health effect, value of time, and cost of illness).

washeconomics

title

☑ ☆ ✇ WASHeconomics Blog

My and others’ WASH economics sessions at #UNCwaterandhealth 2019

By: IanRoss

unc

I’ve just arrived at the UNC Water and Health conference 2019 this week (verbal abstracts book here). I have three verbal presentations, a poster, and am involved in a side-event. Short summaries are below. Further below, I highlight others’ economics-related presentations/sessions that I’m looking forward to seeing.

All my stuff is on Thursday… but please talk to me about SanQoL at any other time!  If you have time for just one of my bits and pieces, please come to the verbal on cost-effectiveness, Thurs 1600 in Azalea. It’s the piece of work I’d most like critique on, as it’s the headline output of my PhD.

My verbal presentations:

Title: “Cost-effectiveness Analysis of a Sanitation Intervention with a Quality of Life Measure as the Outcome” (Thurs 1600 in Azalea, 1st up)

One-sentence abstract: An overview of ‘sanitation-related quality of life’ (SanQoL), and an empirical study in Mozambique of how SanQoL can be used to weight ‘quality-adjusted service years’ (QASYs), for better economic evaluation of sanitation programmes.

Title: “Human Waste of Time—Valuing Open Defecation Time Savings” (Thurs 1600 in Azalea, 2nd up)

One-sentence abstract: An exploration of how we should value ‘time saved’ when people switch from OD to household toilets, using data from the 2013 SQUAT survey in India, because how we do this makes a big difference to results of cost-benefit analysis.

Title: “Three-Quarters of People in Port-au-Prince, Haiti, Get Their Drinking Water from Private Providers” (Thurs 1430 in Redbud, 2nd up)

One-sentence abstract: Demonstrating a “water flow diagram” for visualisation of flows of water and money in a city-wide water market, through a mixed-methods study of private water providers in Port-au-Prince.

My poster

Title: “How does sanitation contribute to a good life? Qualitative research in urban Mozambique informing quantitative measure development” (Thurs 1700)

One-sentence abstract: Results of the qualitative study (yes, economists do proper qual…) that informed the development of the SanQoL measure, using focus groups and in-depth interviews in low-income areas of Mozambique.

Side-event I’m involved in

Title: “An Agenda Setting Workshop for “Limited” (Shared) Sanitation: User Experiences, Measurement, and Improvement Approaches” (Thurs 1030 in Redbud)

One-sentence abstract: starting with a “quick fire” format (presenters have 1 slide / 5 mins each), the meat of the session will be focused on the creation of a research agenda for the role of shared sanitation in bringing safely managed sanitation to all.

Others’ WASH economics verbals and side-events

The best thing about the UNC conference is that, when I look at the agenda, I want to go to almost everything. If you take a broad definition of WASH economics (as I do here) then a lot of the sessions/papers at UNC will touch on it. So here’s just a few of the other WASH economics verbals/sessions I’m looking forward to. There’s some interesting-looking posters too.

Verbals

  • “An Evidence-driven Approach to Establishing Prices for Pit Emptying Services by Vacuum Truck Operators” Bernard Elegbe, ABMS/PSI – looks like a cool application of “mystery shopper” techniques to FSM services. Mon 1430, Dogwood
  • “Using Public Subsidy to Unlock Household Finance: Evidence from the Field” Lesley Pories, Water.org – using RCT data to explore how households used sanitation microfinance loans. Mon 1430, Dogwood
  • “Findings from the Implementation of the First Cross-culturally Validated Household Water Insecurity Experiences (HWISE) Scale in Zambia and Democratic Republic of Congo” Sera Young, Northwestern University – I’ve read a lot about HWISE so looking forward to seeing its empirical use. It’s exciting to see a user-reported measure for water that does a lot of what I’m trying to do with SanQoL. Tues 1430, Redbud
  • “Time, Economic and Health Benefits of the Transition to Rural Piped Water Systems in Southern Zambia” James Winter, Stanford University – interesting approaches to measuring the benefits of water supply interventions using matched controls. Weds 1600, Dogwood
  • “Supply and Demand: Assessing Costs and Willingness-to-pay for Urban Sanitation in Bangladesh, Ghana, and Kenya” Rachel Peletz, Aquaya Institute – have been looking forward to seeing these results for a while, which represent one of the several important initiatives underway on urban sanitation costing. Thurs 1430, Redbud (and helpfully just before my Haiti verbal)

Side-events

  • “Revisiting Subsidies for Water Supply and Sanitation Services” – the recent World Bank flagship report on subsidies, incl.  debates around allocation of subsidies between service types and between ‘new’ access vs. consumption. Tues 1030, Bellflower
  • “Understanding Demand for WaSH Services: How Much are Consumers Willing to Pay?” – should be a good debate on WTP methods with example of vouchers, auctions, hedonic pricing, contingent valuation, and discrete choice experiments. Weds 0830, Dogwood
  • “The Maputo Sanitation (MapSan) Trial: Measuring Health, Environmental, and Social Impacts of an Urban Sanitation Intervention in Mozambique” – long-awaited results of the MapSan trial, the largest controlled health impact trial of an urban sanitation intervention to date (OK it’s not strictly speaking econ, but it’s the study my PhD is linked to, and incredibly interesting…). Weds 1030, Redbud

 

washeconomics

unc

☑ ☆ ✇ WASHeconomics Blog

Funders shouldn’t misinterpret WASH-B and SHINE results as “WASH doesn’t work”

By: IanRoss

change
We shouldn’t give up on incremental WASH improvements

There has been a lot of debate about the well-designed and well-conducted WASH-B (Kenya, Bangladesh) and SHINE (Zimbabwe) trials of rural WASH interventions in the past year or so. Most recently, the big names in WASH epidemiology published a consensus paper. Many funders may not read it, which would be a shame, as it is easy to misinterpret the WASH-B and SHINE findings as “WASH doesn’t work”. If funders make this misinterpretation, people will miss out on life-changing WASH services. In this post, I discuss the incremental changes the interventions delivered, how epidemiologists have interpreted the results, and how funders should interpret them.

In summary, I think that thoughtful funders should read the consensus paper, and draw two conclusions:

  1. “I understand that most WASH programmes make incremental  improvements in service level which are unlikely to bring substantial health gains in the short-term.”
  2. “I see the bigger picture and will continue to fund “basic” services and the systems that sustain them, in the knowledge that others will further increase that level of service to be “safely-managed” in 10 or 15 years, bringing the desired health gains.

This is not an argument that “transformative” WASH services are not achievable or desirable. In some settings they will be. Rather, I mean to point out that sustaining safely-managed services may cost more than rural households in LMICs are able and willing to pay in the medium-term. Incremental improvements are more realistic and affordable, both for users and funders (in which I include LMIC governments).

1. What were the interventions and results?

In understanding the WASH-B and SHINE results, two things are often overlooked: the baseline conditions, and what improvement the interventions made upon them. This is a point that is similar to, but different from, discussions about external validity (which I don’t plan to get into here).

Table 1 at the bottom of this post summarises baseline conditions and interventions for each WASH component in the three trials. [I haven’t seen this analysis anywhere else, but let me know if someone else did it in more depth already]. Also at the bottom, and for completeness, Table 2 summarises the trials’ main results, though most readers will already be aware of them (no impact of “WASH” on any key outcomes, except on diarrhoea in WASH-B Bangladesh).

The first point I want to make is that, given the baseline conditions and small incremental improvements achieved (a term I differentiate from “marginal” here), these interventions did not constitute a typical “WASH” programme. Let’s explore this a little.

Baseline conditions: the highest priorities under SDGs 6.1 and 6.2 should be achieving basic services for all first (as I have previously argued), by ending open defecation (OD) and ending the drinking of water from unimproved and/or far-away sources. See definitions of basic for sanitation and water. The WASH-B settings in particular were already much better than what one might think of as “poor WASH conditions”. For example, there was ~95% latrine access in both Kenya (though mostly unimproved) and Bangladesh. In Kenya there was ~75% improved water, which the time-to-source data suggests fulfilled the “basic” SDG criterion for most. In Bangladesh, it appears most people used on-plot tubewells. DHS 2014 shows that >95% of rural Bangladeshi had “basic” SDG water (and 74% on-plot water). Baseline sanitation conditions were worse in SHINE (~50% OD). Overall, then, WASH conditions in these settings were far from terrible, particularly in Bangladesh.

Incremental changes: the incremental changes to fairly good baseline service levels were small in most cases. Take sanitation: the WASH-B Kenya intervention was to replace an unimproved latrine slab with a lidded plastic one. This is an improvement, to be sure, possibly valued by the user. But not a big change – it is still a pit latrine without a water seal. In WASH-B Bangladesh, ~95% of people used an improved latrine with a concrete slab (and ~30% a water seal), so the change was the switching to a pour-flush in most cases. Again, an important but small improvement. I don’t say this to criticise the interventions, but to argue that maybe we shouldn’t expect health gains with such minor improvements. Many WASH interventions make more substantial incremental changes in WASH service levels. Turning to water, interventions in all three trials were chlorination. The most recent systematic review shows that chlorination interventions have no effect on diarrhoea once non-blinding is taken into account. So, it was not surprising that the water arms had no effect. In none of the three trials was any change made to the water supply itself, whether increasing its proximity or reducing its intermittency.

Taking these two points together, baseline conditions were already at or close to “basic”, and the intervention brought them up to “basic”. This is not the kind of incremental change one might expect to have a substantial effect – the systematic review shows that diarrhoea impacts are sensitive to the change in level of service (my comment here). I am not arguing that small incremental changes are unimportant – on the contrary. Cumulatively over time, they can have a big impact. My point is that the kind of incremental changes studied in these trials (i.e. chlorination, toilet interface upgrade) is not the kind most WASH programmes focus on, or should focus on. More important transitions to characterise as “WASH” would be OD–>basic sanitation (which SHINE did, on the whole) and off-plot unimproved or faraway water –> basic water (i.e. water quantity interventions, which none of these studies did). In short, incremental changes are good, but perhaps bigger ones than WASH-B (and SHINE, to some extent) studied.

2. How have epidemiologists interpreted the results?

The recent consensus paper is the most important contribution to the debate around WASH-B and SHINE. Table 3 shows its five main messages, with my appended reflections. I would summarise their argument as follows: the theory behind the impact of WASH is still sound – we might just need bigger service improvements over longer periods of time to see a measurable health effect. Don’t give up on WASH because, when we change conditions enough to see an effect, history suggests it will be a big one.

What I see as missing in these debates is the value that people get from WASH beyond health. These benefits (e.g. time savings, quality of life and psychosocial factors) are substantial contributors to the overall economic benefits of WASH interventions. We are still very bad at measuring and valuing these benefits. I am biased because measuring quality of life gains from sanitation interventions is what I work on currently… but the point stands.

Table 3: Interpretation of consensus paper messages

Five consensus messages My reflection
1. “Despite high compliance, the evaluated WASH interventions – as delivered in these settings – had no effect on linear growth, and mixed effects on diarrhea” Unlike some previous trials, we can’t explain away these results with interventions that didn’t do what they were supposed to. We have to face up to them.
2. “The biological plausibility of WASH interventions as public health interventions is not challenged by these findings” Finding significant health effects in a trial depends on many environmental and social factors. Trials are important, but the underlying theory supporting an intervention is also important, especially when interventions are particularly hard to test with trials.
3. “Historically, large, population-level gains in child health have not been achieved without significant improvements in WASH services” Big health gains will take many big cumulative environmental changes. This will take longer than you can assess in a trial, and is likely to come in several incremental steps.
4. “Current evidence suggests that basic WASH services alone are unlikely to have a large impact on childhood stunting” OK, but basic WASH should deliver important economic benefits (time savings, quality of life). These are likely worth paying for, even in the absence of health gains.
5. “The results of these trials do not undermine the new and ambitious SDG target of safely managed services for all” Achieving “safely-managed” services (which these interventions did not) may be more likely to deliver health impact. However, delivering and sustaining safely-managed services may cost more than rural households in LMICs are willing and able to pay in the medium-term.

3. How should funders interpret the results?

I worry that funders may read the abstracts of these papers, or hear second-hand readings of them, and misinterpret the findings. It is easy to infer that “WASH doesn’t work”, which would be a mistake. It depends on what we mean by “work”, and how long we are willing to wait to see cumulative gains.

A thoughtful funder would read the consensus paper, and conclude:

1. As an enlightened funder, I understand that:

  • Most WASH programmes make small changes in service level, and this is unlikely to bring substantial health gains in the short-term.
  • “Transformative” WASH might be more effective, but it would also be very expensive, both for me and for the users who need to sustain the service.
  • Users value the non-health benefits of WASH services too, such as time savings and quality of life benefits, and these have an economic benefit.

2. Therefore, being an enlightened funder who considers the bigger picture:

  • I will continue to fund incremental “basic” improvements in WASH services, in the knowledge that someone else (e.g. the users or the government) will further increase that level of service to be “safely-managed” in 10 years or so. This may bring more substantial gains, which I will have contributed to.
  • I will also continue to have a broader impact by funding the sector-wide systems that are necessary for further extending and sustaining services for all.

This is not an argument that “transformative” WASH services are not achievable or desirable. Rather, I mean to say that sustaining safely-managed services may cost more than rural households in LMICs are able and willing to pay in the medium-term. Incremental improvements are more realistic and affordable, both for users and funders (in which I include LMIC governments).

================ 

 Table 1: baseline situations and interventions in the WASH-B and SHINE trial settings, and their incremental changes

ints

Note. this is a summary – full details are in the papers and their supplementary info. Approximate baseline conditions are based on my eye-balling across arms in the studies’ baseline tables – SHINE had 4 arms and WASH-B had 7.

 

Table 2: results of WASH-B and SHINE trials

effects

 

 

washeconomics

change

ints

effects

☑ ☆ ✇ WASHeconomics Blog

Economist-engineers and public health economists: is WASH economics a “field”?

By: IanRoss

Is WASH economics a ‘field’?

This post explores ways of breaking down the “field” of WASH economics. On the one hand, one can argue that WASH economics doesn’t exist as a coherent field. After all, most people actually working on WASH economics questions are in the field of engineering and/or public health. There are fairly few people with an economics MSc or PhD who are working on WASH. I doubt many people self-identify as “WASH economists” in the way that health economists or environmental economists do.

On the other hand, one can argue that disciplinary boundaries are unhelpful in relation to a subject like WASH, which defies being put in a disciplinary box. Getting WASH services right often involves building stuff well, proper consideration of the underlying resources, reliable prevention of pathogen transmission, changing people’s behaviour, and doing all that with the right incentives, prices and funding/financing model. So WASH economics exists in the same way that public health engineering exists.

Personally, I would argue that WASH economics does exist as a field. However, it requires a little more shaping and communicating in the coming years to make it into a coherent one. Who cares what disciplinary background someone has as long as they’re solving a problem by (i) asking and answering the right question, (ii) using the right method, and (iii) doing a good job of it?

If an engineer is looking at preferences for different water services, and using economic methods to do so, then they are doing WASH economics. I can point to many recent papers that go in this box. Many are good or excellent, though definitely we can raise the bar by interacting more with each other.

How to divide up WASH economics sub-fields?

I already explored definitions of WASH economics in this post, going with “the study of how people make decisions about the allocation of scarce resources in the delivery and use of WASH services.” Within this there are many sub-fields. Below is my current way of sub-dividing it, drawing strongly on how my colleagues at the Centre for Health Economics in London (CHIL) at LSHTM divide up our work into themes.

  1. Economic evaluation – including work on resource use and allocation (e.g. costing, cost-effectiveness analysis, cost benefit-analysis), the valuation of outcomes, and broader priority-setting questions such as how to include equity in decisions.
  2. Economics of service providers and systems – including work on funding and financing, the role of government in providing WASH services, and the regulation of markets for WASH services. This is a broad area that would include questions of production and pricing (e.g. tariffs and subsidies), as well as the interaction of demand and supply at both the provider and sector level. Or, as is de rigeur, looking at the ‘WASH system’ as a whole.
  3. Policy evaluation – including work on attributing impact (of any kind) to WASH interventions that use economic-based methods. There is a lot of cross-over with epidemiology / public health and “implementation science” here, but some methods/questions are more economic than others.
  4. Choice and behaviour – including work on understanding the choices people make between different behaviours, products and services. This would include studies of demand drivers, preferences and willingness to pay. It may encompass work using behavioural economics or discrete choice experiments.

I hope to write a post exploring each of these in more detail another time.

toilet
What are her preferences for toilet options? Are options cost-effective? How much is she willing to pay? 

An aside on disciplines

WASH research is not unique in often requiring multiple disciplinary inputs to be successful. Many of those working on health economics, for example, have backgrounds in statistics, medicine, mathematical modelling and epidemiology. Often these other disciplines are in fact required in order to answer questions properly. It is practically impossible to do many types of economic evaluation of a vaccine or a tuberculosis intervention without involving modellers, for example. Likewise, it can be hard to know whether a change in a health outcome is clinically relevant in a given population, without asking a medical doctor.

The same would apply to doing an economic evaluation of a complex sewer intervention without involving people who know how the system works. Some decisions that look odd to an economist may make absolute sense to an engineer.

While there are trained economists who work consistently on WASH, there are others who delve in and out of many sectors. Some of these, perhaps, do not truly attempt to understand WASH before employing their standard methods toolbox (which may not be appropriate to the question at hand). They might answer the wrong question entirely. Worse, they might answer a question that public health people already answered 5 years ago but they didn’t bother to read the literature properly.

But these people are few. My point is that it would be helpful to have far more trained economists deciding to work consistently on WASH over a long period. Economists who’ve entered WASH like Britta Augsburg, Molly Lipscomb and Dean Spears (and many of the r.i.c.e. team) have added lots of value to the sector. Likewise there are those with a ‘harder’ science background like Meera Mehta (architecture) and Marc Jeuland (engineering) who have all but switched to economics. We need more of both these kinds of people.

Conclusion

Lots of people work on WASH economics without considering themselves economists. This is great and to be encouraged. While it is something of a cliché, we do need to break down boundaries between fields and borrow methods from each other. We also need to improve the quality and quantity of work on WASH economics, both by those who are economists and those who are not. For this, it would be helpful to have (i) more trained economists deciding to work consistently on WASH, and (ii) a more coherent dialogue around WASH economics.

washeconomics

toilet

☑ ☆ ✇ WASHeconomics Blog

What is WASH finance?

By: IanRoss

glaas
A 2017 GLAAS report graphic on the WASH finance mix in a set of countries

In a previous post, I proposed a working definition of WASH finance as “the study of how WASH services are paid for, including who pays, how and when”. There is also the huge question of what is paid for (i.e. level of service, technology, paying to cover capital or recurrent costs, how equitable it is, etc.). To simplify, let’s take that investment option as given. We could be thinking about a shared pit latrine, a gravity-fed network of rural standposts or a sewer network.

The “who, how and when” are important in the following ways:

  1. Who pays: It matters who is fronting the money, not least because it will give them a say in what is done. Service providers should be accountable to users paying tariffs. Banks offering loans will set conditions. Money might come from the public sector, private firms, individuals/households, donors/NGOs, etc. Some or all of that money might be borrowed (i.e. repayable), so there are effectively multiple payers sharing risk.
  2. How: There are many ways to provide money, whether through cash, debt, equity, etc. Money can be “cheap” or “expensive” if it is borrowed, or “free” if it is not. In-kind payment and unpaid labour can be important but would technically not cover a financial cost (see this post).
  3. When: Timing is everything. Spreading costs can make things more affordable to those without deep pockets, but sometimes interest payments can be crippling. Possible scenarios include immediate payment in full, instalments spaced over time periods (with or without interest), instalments on delivery of contracted outputs etc. Capital itself has an opportunity cost, i.e. it could make a return if invested in something else, like government bonds.

In summary, finance is about how financial costs are covered. The above three dimensions are just an intuitive way I like to think about it – comprehensive frameworks exist for conceptualising WASH finance, discussed below. This post aims to scratch the surface of those in fairly simple terms.

Categorising different types and sources

“Finance” is a useful catch-all term, but it should really be split into funding and financing, as I mentioned via a quote in this post. In short:

  • Funding means providing money which is not expected to be repaid.
  • Financing means providing money on expectation that it will be returned in full, plus interest or dividends, so perhaps better framed as repayable financing.

More on the sub-categories underlying these are as follows. In the WASH context, funding usually comes from three “sources”, together known as the “3Ts” framework popularised by the OECD (2009).

  1. Tariffs, meant in the usual sense as ‘fee for service’ but, conceptually, this bucket also includes self-supply expenditure (e.g. household-funded toilet construction) or user charges such as connection fees.
  2. Tax revenue, which might be collected by different levels of government (local, municipal, state, national)
  3. Transfers, e.g. from donors, NGOs, foundations or remittances

The best explanation I’ve seen of this is the table in the TrackFin guidance doc p.49, which shows how an accounting perspective (“types”) overlays with the 3Ts perspective (“sources”).

Repayable financing fits broadly within two categories:

  1. Debt, which could be a concessional or non-concessional loan, bonds or loan guarantees, etc. – on which the principal must be repaid and interest can be paid
  2. Equity, which could be a formal stake (or share) in a company, a PPP-based “contribution” to capital costs, etc. – on which dividends may be paid and the stake can be withdrawn.

The GLAAS 2017 report graphic at the top of this post shows how different countries use different mixes of funding and financing sources in their sectors. However, only the countries asterisked in the figure have done full TrackFin studies. I would take data for other countries with a pinch of salt (i.e. they are likely ‘best guesses’ by someone),  because detailed information on flows is rarely available at the sector level. That is the whole reason why TrackFin was originated by WHO and is so important (but I’m biased as I was involved in the Tunisia TrackFin study and am fully bought into the approach ).

When is WASH finance cheap or expensive?

Funding is “free” in the sense that it has no cost of capital (though capital may be allocated an opportunity cost in economic analysis). Repayable financing, meanwhile, comes at a price. That price can be “cheap” or “expensive” depending on the terms and implied interest rate. Concessional loans from development banks (such as the African Development Bank or World Bank) are generally the “cheapest” type of repayable financing for WASH services. They allow borrowing at below-market rates but come with various conditions. Non-concessional loans can be secured from commercial banks which probably comes with fewer strings, but at full capital market rates.

In practice in low-income countries, service providers are unlikely to be able to secure repayable financing beyond these options. Most will simply not have the track record of revenue generation and loan repayment (and subsequent credit rating) to be able to issue bonds, for example. A graphic from IRC shows how this can work. With equity meanwhile, allowing private stakes in service providers may not be legally straightforward in many low-income countries.

There is increasing interest in the idea of “blended finance”, whereby public funds (or concessional loans) and commercial repayable financing are combined in a synergistic way. The objective is usually to make an overall package cheaper than the commercial market could provide, but still crowding in private capital rather than crowding it out. Sophie Trémolet gives a clear explanation in this episode of WASH Talk or there’s this World Bank report.

Conclusion

All sources of funding and repayable financing have a role to play in expanding and sustaining WASH services. However, some are more appropriate than others for (i) different purposes, (ii) different providers, and (iii) at different stages of a WASH sector’s development.

washeconomics

glaas

☑ ☆ ✇ WASHeconomics Blog

A generalised way of thinking about sanitation quality of service and quality of life

By: IanRoss

In a previous post, I wrote about how I see measures of quality of service (QoS) and quality of life (QoL) capturing different things which are both important. In this post, I expand on that, proposing a generalised way of thinking about this at different stages of the service chain. First, though, a few words on user satisfaction and objectivity.

Distinction between QoL and user satisfaction

In the last post I skipped over user satisfaction, but I think it’s important to describe the subtle difference between user satisfaction and QoL. In marketing, theoretical groundings of customer satisfaction are focused on the product. For example, one influential marketing theory focuses on expectations about a product’s “ideal”, “expected minimum”, “tolerable”, and “desirable” performance. For sure, this involves psychological processes, but it is focused on the product not the person.

To illustrate this, consider these two questions asking about a similar concept:

  1. How satisfied are you with the cleanliness of the bathroom?
  2. Can you feel clean while carrying out your sanitation practices?

It’s likely that responses for these two questions would be highly correlated. However, there is an important conceptual difference, and it’s rooted in whether the question asks about an attribute of the service/infrastructure or about an attribute of the person/QoL.

Subjectivity and objectivity

The distinction between subjectivity and objectivity in measurement is clearly important. Most measurements are unlikely to be objective in the philosophical sense. However,  achieving objectivity in the scientific sense is often an implicit goal in QoS measures. To be classed as objective, the value for a QoS variable should theoretically be the same regardless of the rater. The mean queuing time at a public toilet should be more or less the same for all (by gender and time of day, at least).

Subjective measures, meanwhile, may differ a priori between people using the same service (I gave an example for SanQoL for users of the same toilet in the last post). Inter-rater reliability, i.e. whether the person asking the questions makes a difference, is nonetheless a concern in some measures. Of course, many sanitation QoS measures that are theoretically objective require some element of subjective judgement by the rater. For example, two different enumerators amy have different thresholds for answering yes/no to “are faeces present on the slab?”. So subjectivity is really a continuum not a binary attribute.

A matrix incorporating subjectivity and objectivity

I applied these ideas in a 2×2 matrix on QoS and QoL with respect to toilets – see below. Most should be obvious from this post and the previous, except for quadrant C on proxies. On the one hand, it may not be possible for anyone other than the individual themselves to scale their QoL adequately. However, out of necessity, health economists have developed proxy measures for people who are too young or incapacitated to rate health-related QoL themselves. It is not too hard to imagine the same for SanQoL, though hardly a priority right now.

tab1.jpg
Figure 1: types of QoS and QoL measures for toilets, by objectivity

This touches on the issue of “who is the rater?”. At the containment stage of the sanitation service chain, one can imagine many types of rater. For QoL it would ideally be the user themselves (though cf. proxy discussion above). For QoS it could be the user, a non-specialist observer (e.g. typical enumerator) or a sanitation specialist. The main difference between the specialist and non-specialist is that only the former would be qualified to inspect or evaluate sub-surface infrastructure. One example is the verification that what the user calls a “septic tank” really is a septic tank not a holding tank or soak pit (see bottom of p.36 in this Indonesian study re: discussion of “cubluk”).

Generalising to the sanitation service chain

Here’s a more generalised framework for QoS and QoL along the sanitation chain. Taking QoL first, the measure of SanQoL I’m working on is assessing QoL attributes of the user at the containment stage. At the emptying & transport stage, however, it would be more appropriate to assess QoL of workers. Same for the treatment & reuse stages. Some people are working on measuring sanitation workers’ conditions, for example, which would fall into that category.

tab2
Figure 2: generalisation of QoS and QoL measures across the sanitation service chain

QoL could be “objectively” proxied by a caregiver or observer, though with many caveats on the reliability of that data – better to ask the person themselves unless impossible to do so. As with the earlier 2×2 matrix, I would put user satisfaction in the box of subjective QoS metrics. This can be measured for containment services or emptying services.

The basket of “objective” QoS measures, meanwhile, is very large. It could range from infrastructure observations by an enumerator, through to measurements from technical instruments (e.g. for treatment efficiency) or financial and economic performance metrics.

Conclusion

Certainly there is plenty of room for better measurement of outcomes affected by sanitation interventions. In this post, I’ve explored the concepts of subjectivity and objectivity with respect to measuring QoS and QoL for sanitation services. Lots of people have developed measures in this space and one thing I want to do is map this out across the above generalised matrix.

washeconomics

tab1.jpg

tab2

☑ ☆ ✇ WASHeconomics Blog

Better lives with better toilets – SanQoL translated for public engagement

By: IanRoss

[I was chuffed to be joint winner in the ESRC ‘Better Lives’ writing competition with the below piece intended for a general audience. The Guardian published an abridged version. Deadlines are 80% of writing, for me at least… so I would recommend competitions like this for forcing oneself to write to a hard deadline for public engagement, not something researchers often have to do. A more researcher-facing post on SanQoL is here.]

Imagine not having somewhere safe to go to the toilet. Really imagine it – leaving your house and defecating behind a bush or a building. It’s hard to bend your mind to consider that, if you’ve had access to a clean, comfortable bathroom since you were a child. However, around the world, 900 million people have no option but to defecate in the open. A further 1.4 billion use a toilet that doesn’t meet World Health Organisation standards for ‘basic’ toilets, meaning that it could still be a direct source of disease.

Fortunately, lots of investment is being made in sanitation in poorer countries – many billions of pounds in fact. Right now, somewhere, a municipal official is drafting their budget and a charity worker is writing a funding proposal. There are hundreds of ways that money could be spent. However, we don’t know enough about whether money is being spent on the right programmes. Inefficient choices are certainly being made.

How can I know this? Aren’t there established economic techniques for comparing ways to spend the money? There are, but they predominantly focus on health, alongside some consideration of time savings and avoided costs. This is a problem because health is not the thing people value most about sanitation. When researchers ask people the reasons why they invested in a toilet, health is usually far down the list. Concerns about privacy, safety, or pride are usually at the top. Together, we can call these improvements in quality of life in general. Excluding them from economic comparisons is a glaring omission.

Economists are very concerned with what people value in their lives. We think that the highest valued changes are the most important when deciding between project A and project B. So why don’t we just measure privacy, safety, etc., and plug that into the economic models alongside health? The challenge is that these things are not easily measured – they are subjective perceptions and vary from person to person. There’s also the problem of how to select the different elements, and then the problem of weighting them. These problems are not insurmountable – such quality of life measures exist for comparing programmes in health or social care. For example, the NHS in the UK makes huge funding decisions based on ‘quality-adjusted life years’. This measure takes account of how people value changes in health, by weighting life years with a ‘health-related quality of life’ scale. However, nobody has yet developed one for sanitation programmes.

That’s where my research comes in. I’m working on a measure for sanitation-related quality of life, building on the experience of health. The challenge is to measure this by asking people less than 10 questions, so it is manageable for regular use. The questions need to reflect what people value most about having a toilet. My work is based in Mozambique which is one of the poorest countries in the world. In collaboration with a local research team, we started by interviewing people living in slum settlements in the capital city, Maputo, both on their own and in groups. These were ordinary men and women, young and old. They all used different kinds of toilets, some good, some terrible.

MapSan
An unimproved shared toilet in Maputo, Mozambique

First we asked them what was important for a good life – people often mentioned having enough food, having a good house, and having happy children. Then we asked about how a good toilet or bad toilet contributed to each of those things. Many interesting and important stories emerged, some happy, some sad. One young woman talked about privacy: “Anyone that passes can peep at you if you are going to urinate or defecate.” An older man talked about no longer feeling embarrassment when entertaining relatives: “When visitors come to see us now, they won’t feel bad when entering the toilet”. By systematically analysing these conversations, I identified a list of issues that kept coming up. The list of issues was too long, so I narrowed it down by showing people sets of three options and asking them to choose which was the most important. By doing this lots of times with many people, it is possible to exclude the least important items, and use statistical techniques to develop weights for those that are left.

So how will this improve people’s lives? My measure of sanitation-related quality of life can be used to compare different investment options. It can be used by the municipal planners deciding where the billions of pounds are spent. Through using it, they can know which types of investments improve quality of life the most, by focusing on what people value about sanitation. Considered alongside data on costs, health and engineering considerations, this can make for more efficient use of public funds. That means more people using a toilet which makes them proud, safe and comfortable.

This research is part of the MapSan Trial supported by the Bill and Melinda Gates Foundation. Ian is funded by an ESRC doctoral studentship.

 

washeconomics

MapSan

☑ ☆ ✇ WASHeconomics Blog

Quality of Service vs. Quality of Life – ways of measuring sanitation outcomes

By: IanRoss

In a previous post I explored the concept of sanitation-related quality of life (SanQoL). In that post I distinguished between “quality of service” (and/or infrastructure) and quality of life. The present post expands a bit on that distinction.

1. Different ways of measuring “quality”

Consider these three questions for assessing the concept of privacy:

  1. [observe] Does the bathroom* lock from the inside?
  2. [ask respondent] Does the bathroom you usually use lock from the inside?
  3. [ask respondent] Can you use the bathroom without people interrupting you?

Here’s a different angle on privacy:

  1. [observe] What is the main material of the bathroom’s walls?
  2. [ask respondent] What is the main material of the walls of the bathroom you usually use?
  3. [ask respondent] Can you use the bathroom in private, without being seen?

All these are like ones in surveys I’ve been involved in over the years. In each case, the ‘A’ questions involve an enumerator (who is not a user of the bathroom) observing infrastructure and making a judgement. In the ‘B’ and ‘C’ questions, the enumerator asks a respondent user, who goes through a cognitive process to respond.

The main distinction between ‘B’ and ‘C’ is that ‘B’ questions are asking about the infrastructure / service, and ‘C’ questions are asking about the person’s experience in using the bathroom. I would argue that this is at the heart of a distinction between ‘quality of service’ (QoS) and ‘quality of life’ (QoL). I would argue that, ‘A’ and ‘B’ are measuring QoS and ‘C’ is measuring QoL, and I expand on this below. For ‘C’ to be measuring QoL, we need to hypothesise (i) that things like privacy, safety, dignity etc.  affect QoL, and (ii) that sanitation affects those concepts. However, there are plenty of studies that support these hypotheses.

2. Infrastructure only tells you so much

There are also some attributes of SanQoL which cannot be proxied by questions or observations about presence or quality of infrastructure. These include shame, pride, safety, all of which are about perception rather than infrastructure.

For example, consider these ‘C’-type questions:

  • Can you use the bathroom without feeling ashamed for any reason?
  • When visitors come, do you ever feel embarrassed providing this bathroom for them to use?
  • Are you able to feel safe while using the bathroom?
  • How often do you use a bucket at night for due to fear of using the bathroom?

I’m not arguing that QoS approaches are not useful. Infrastructure is important, and observations can be rapid and useful. I’m just trying to show how QoS and QoL approaches measure different things. Of course, there are also other aspects of QoS which are about users not infrastructure (number of users, distance from house to toilet etc.). And some composite QoS measures may incorporate items which measure QoL.

There is also the issue of questions about user satisfaction with the service, which go in a slightly different box unless they directly measure QoL attributes. For example, you can ask “how satisfied are you with the cleanliness of the toilet”, which is arguably still about infrastructure rather than QoL. Where the distinction is less clear is with questions like: “how satisfied are you with the level of privacy?”. More on this another time. In short, I would argue that to be measuring QoL the question has to specifically be measuring functionings (as with the widely-used EQ-5D , SF-36 or WHOQOL-Bref) or capabilities (as with the ICECAP-A)

3. Are QoS and QoL distinct?

Above, I wrote that the ‘A’ and ‘B’ questions are measuring QoS and the ‘C’ questions are measuring QoL. But how distinct are these concepts really? It is possible to argue that the ‘C’ questions are really part of QoS as well. However, I think it makes more sense to see them mostly as separate things, as what is being measured is fundamentally different.

‘C’ questions are not really measuring the service itself – they are measuring a person’s perception of how that service affects them. More specifically, given the capability-based formulation of the questions, they are measuring what a person is able to do with respect to their sanitation practices.

Here’s an example. Imagine a pit latrine with slab shared between 4 households, with no roof and some fairly dodgy walls, that is not kept particularly clean. It is easy to imagine that service being experienced completely differently by different people (e.g. a 30-year old man as opposed to a 16-year old woman or an infirm 80-year old of either sex). As a consequence, while QoS measures might give the same result, each individual’s SanQoL might be markedly different.

Therefore, both toilet and user characteristics matter for SanQoL. However, these are far from its only determinants. Population density, levels of community security, environmental conditions, trust between neighbours etc. will all affect SanQoL but not most measures of QoS.

4. Conclusion

This post has just scratched the surface of a complex issue, I am characterising QoS measures in a certain way, when they are certainly many and various. My main point is that QoL focuses on outcomes in user’s lives, rather than infrastructure or levels of service.

* As a Brit it pains me to use such an Americanism… but it is for good reason. SanQoL does not distinguish between sanitation practices that people carry out, in the given place they call the bathroom, toilet, casa de banho, sintina, choo, charpi etc. As it is conceivable that people bathe in that space (as they do in the Maputo setting I’m working in), then “bathroom” is more appropriate than “toilet”.

washeconomics

☑ ☆ ✇ WASHeconomics Blog

Using the concept of ‘sanitation-related quality of life’ (SanQoL) to measure what is valued by users

By: IanRoss

SanQoL
This post introduces the concept of ‘sanitation-related quality of life’ (SanQoL), which is part of my PhD thesis

Introduction

Since investment options are always compared under a budget constraint, economic evaluation aims to inform unavoidable decisions and support allocative efficiency. Various economic evaluation methods (such as cost-effectiveness analysis and cost-benefit analysis) compare costs and consequences of alternative interventions.

Improvements in sanitation can impact on health, and it is typically health outcomes such as averted deaths and diarrhoea cases which have been valued previous cost-effectiveness analyses. Cost-benefit analyses have gone further, also valuing time savings, productivity and other economic benefits.

The problem

However, none of these economic evaluations have incorporated the broader “quality of life” (QoL) benefits of sanitation, related to privacy, safety, pride, dignity etc.* Some studies call these “intangible”. However, these broader QoL benefits are often the primary demand-side drivers of sanitation investment by poor households.

Therefore, economic evaluations which overlook these outcomes may lead to in misallocated resources. At the very least, they will not fully reflect the value placed on sanitation by users.

Where my PhD research comes in

Can anything be done about this? If there was a quantitative measure of ‘Sanitation-related quality of life’ (SanQoL) that represented people’s valuation of QoL benefits, it could be used as an outcome in cost-effectiveness analysis. Further down the line, a SanQoL measure could potentially facilitate monetary valuation of QoL outcomes in cost-benefit analysis.

Taking the first steps in developing and applying such a measure is 50-60% of my PhD. I’ve kept fairly quiet about it so far while doing lots of reading and initial qualitative work. Now, however, the ideas are clear enough to share, for reflection and critique. I can’t write about interim results, unfortunately. Hopefully I’ll get some abstracts into  academic conferences later in the next year, and then comes papers… but I can write about ideas at least.

This post sets out a bit of my rationale and thinking – in the rest of this post there’s space for two issues, (i) why qualitative grounding is important, and (ii) what distinguishes a psychometric approach from an ‘observer’ approach. How I’m defining and framing QoL itself in all this will have to wait for another post. In short, I’m using Sen’s capability approach. However, I’m  also building on the concept of ‘health-related quality of life’ which is widely used in health economics for valuation of health states in support of quality-adjusted life years.

What do people value about sanitation? The importance of qualitative work

Value is a key concept in economics. Demand for a service in any sector is a derived demand for a stream of benefits valued by the user. For example, demand for healthcare services is a derived demand for health itself. So, demand for household toilets is a derived demand for the benefits of sanitation. These are many and various (see above) so any measure of SanQoL would likely need to be multi-dimensional.

A measure of SanQoL would ideally capture all QoL-related outcomes perceived by as important by users. What are those outcomes? First port of call is looking into the literature, but best practice is to ask people directly in the setting of interest, in support of  validity (amongst other reasons)

The setting in which I plan to test the quantitative measure is low-income settlements of Maputo, Mozambique. Accordingly, I did some qualitative interviews and focus groups with users of different types of sanitation services in 2018. To identify the relative value attached to different ‘attributes’ of SanQoL, I also included some ranking and triadic comparison exercises. More on that once I get the qual. paper written…

Measuring SanQoL quantitatively

Having a rich qualitative description of how people think sanitation relates to ‘a good life’ is important and helpful. However, in order to be practically used in economic evaluation, it requires translation into a quantitative measure. For example, it would be helpful to be able to say “this intervention cost US$ 1,000 per incremental point on the SanQoL scale, as compared to that intervention.” You can’t do this with qualitative data.

Such a SanQoL measure would need to be experiential or psychometric, in that it would be “imposing measurement and number upon operations of the mind”. For a given dimension of SanQoL (privacy, for example) this involves asking people to scale their own level on that dimension, by recalling their experience (e.g. variations on “do you feel like you have privacy”). The non-psychometric way to do this would be an external observer looking at a toilet and scaling the level of privacy provided (e.g. variations on “does this toilet provide adequate privacy”). This ‘observer approach’ measures something fundamentally different, that is, quality of service (or infrastructure) rather than quality of life. Individuals themselves are best-placed to scale their quality of life.

I have developed some questionnaire items on the basis of the qualitative work and will be running a quantitative survey in April, including piloting and cognitive interviews beforehand to make sure the items are well-understood. Of course, even if that work is successful, the measure will not necessarily be valid in other settings. More on all this another time.

Conclusion

I hope that useful insight will come out of this work to inform how sanitation-related quality of life can be measured. My primary purpose is in economic evaluation, but there are many other potential uses of such a measure if it can be validated across settings. I hope to write occasional blogposts on other aspects of SanQoL in due course. Please contact me if you are working in this field as well and want to discuss.

*At least, none have done so quantitatively – Hutton et al.’s East Asia phase 2 studies did so qualitatively.

washeconomics

SanQoL

☑ ☆ ✇ WASHeconomics Blog

CLTS, targeting and economic performance – reflections on two seminars

By: IanRoss

Today I attended not one, but two, seminars on CLTS. The first was Britta Augsburg presenting results of a recent cRCT of a WaterAid CLTS intervention in Nigeria (at LSHTM). The second was Dale Whittington reflecting on CLTS trials in the last few years and his recent CBA paper incorporating their results (at Oxford). A key theme from both is targeting. Britta’s analysis shows that interventions should be targeted at the settings in which they are most likely to be most effective. Dale’s analysis shows that resources should be targeted to places where net benefits can be maximized.

1. Britta’s presentation

Brief overview

Britta presented the results of a cRCT reported in this IFS working paper (soon to be updated) which evaluated a 2014 CLTS intervention by WaterAid Nigeria. They found that the intervention achieved only a 3 percentage point decrease in OD (at the very low end of recent experience). That doesn’t necessarily reflect well on the quality of implementation, but it’s not the most important result. More intriguing is the sub-group analysis they did for community-level wealth measures. Looking separately at ‘richer’ and ‘poorer’ communities (as defined by asset wealth), they found a 9pp decrease in OD in the poorer communities and no significant decrease in the richer communities. It’s a carefully done piece of work – they went to great pains to show that this key result is robust to different SES measures and not driven by baseline differences in sanitation coverage (see table 5, p.19 in the paper).

Most importantly for external validity, they reanalyse data from other trials and find that impacts on OD are stronger in poorer contexts in those settings too – see figure below (from the PPT, updated since the paper). This is an important insight. Read the left and right axes first – in short, it is plotting treatment effects by study and the average ‘night light index’ over the areas of study at baseline (a proxy for economic activity). The figure shows that the higher the night light index, the more likely the study was to report statistically insignificant reductions in OD. In other words, “heterogeneity in CLTS impacts across studies can be rationalised by differences in the average area wealth”.

britta

My reflections

  1. The analysis is notable for looking at wealth effects at the community level rather than the household level.
  2. The wealth effect is an appealing idea, but if correct, why does community wealth affect CLTS effectiveness? A lot of the avenues they tested for this (community social interactions, public infrastructure, leader characteristics) were not statistically significant as drivers of CLTS effectiveness (nb. this was in the PPT but not the working paper – assume it will be in new version). So, the wealth measure is picking up something, but it’s not clear what it is?
  3. Britta argued that the effect is strong enough to be used to target CLTS in the right places, especially since asset data is widely available in DHS. In other words, secondary data can be used to target CLTS intervention in poor areas with high OD, where it is more likely to be effective. These are probably areas of greater need, too, so it is not a hard sell (except that implementers may well face higher costs in poorer areas as they may be harder to reach).
  4. Note that this intervention was pretty light-touch, only 1 day of CLTS activities. Almost all triggerings did defecation mapping and action planning, but almost none did transect walks, graphic exercises, and medical expenses calculations. Follow-up was also light-touch (see bottom of p.5). Would the effect have been greater (and the wealth effect more or less visible) if the intervention had been more intensive?

2. Dale’s presentation

Brief overview

Dale summarised the rationale for CLTS, reflected on the 10 or so recent sanitation trials, mostly funded by BMGF to the tune of c.$100m. He set out that a lot of these trials showed weak evidence for the impact of CLTS, with limited impacts even on reducing OD in many cases. This is not news, but his argument was a more subtle economic one: CLTS interventions are essentially risky investments. Despite this, uncertainty around costs, and especially benefits, is not much considered by sanitation advocates. He argued this with reference to a cost-benefit model of a hypothetical intervention, set out in this working paper. That model further elaborates earlier versions he (with co-authors) has been applying since 2008, and in an important version in 2012.

The main conclusions from the model are that while CLTS has a positive NPV only about two thirds of the time in their probabilistic sensitivity analysis. The BCR is rarely close to the high values of 5-7 seen in global studies like Hutton’s in 2012 and 2015. Rather, it is 1.5 on average in the base case (ranging from 0.6 – 2.3 depending on the level of uptake). In summary, his is a more skeptical (or perhaps, realistic) take on the economic performance of sanitation. The argument was not that we shouldn’t do sanitation programmes / CLTS, but that we need to be much more careful about selecting where we do them, and that ‘global silver bullet’ interventions shouldn’t be blindly applied in very diverse contexts.

He ended by pondering whether, in dirty environments with multiple exposure pathways for pathogens, making marginal tweaks to toilets or similar is going to change very much. Substantial gains in public health may only be made by upgrading all of housing, roads, drainage, WASH etc. Timing and sequencing are likely to matter. As asides, he also noted (i) the hubris of many of the RCT teams in flagging the importance of non-health benefits in their conclusions but mostly neglecting to measure them, (ii) the fact that almost none of these trials seem to have collected data on costs, or other data that would allow economic evaluations.

My reflections

  1. The basic argument and results in the new paper are not so different from the 2012 paper. However, the new model makes substantial innovations, such as:
    • Synthesising effectiveness evidence on the most recent RCTs on CLTS
    • Presenting results with and without a “sanitation externality” effect and, crucially, showing that it doesn’t make that much difference to the economic performance.
    • modelling at the level of  a district of 100,000 people, thereby allowing for heterogeneous effects (high / medium / low uptake) in different villages.
  2. These innovations are important because they make the conclusions even more convincing that previous iterations. In the first year of my PhD I read pretty much every sanitation economic evaluation ever done (at least, all the CEAs and CBAs on this list), and this new model is now the benchmark for comprehensiveness.
  3. His concluding point about needing to change multiple environmental conditions to see substantial public health gains reminded me of this Gambian natural experiment published a few months ago. It supports the argument that very high environmental conditions are needed to close the stunting gap.*
  4. The point about people not bothering to measure non-health effects is close to my heart. Half my PhD is about developing a measure of ‘sanitation-related quality of life’ (SanQoL) that would aim to make measuring these outcomes more routine in trials and in general M&E. More on that in due course…
  5. That so many well-funded WASH trials not including any economic component is close to scandalous. In the UK, for example, it is more or less the norm for health trials funded by the NIHR to include an economic evaluation component. Of course, my health economist colleagues still manage to gripe about not being given enough time or not being brought onto the team at the design stage. It is similar in global health trials, but at least the work is done and having an economics component is the norm. In WASH we seem to completely ignore economics in most trials (e.g. costing, cost-effectiveness, cost-benefit) , as if decisions will be made on effectiveness alone.

3. Conclusion

In conclusion, given recent trials, the benefits of CLTS are probably lower on average than we thought. Given the Crocker (2017) costing study the costs of CLTS are probably higher on average than we thought. However, the whole point of both Britta and Dale’s presentations is that the average is not that helpful. Interventions should be targeted at the settings in which they are most likely to be most effective (and not used at all in places where they aren’t likely to be). Furthermore, resources should be targeted to places where net benefits can be maximized. For CLTS, if Britta’s analysis is correct, these places are fortunately also likely to be those that most need support.

 

* this isn’t the place to discuss the Gambia study in more detail, but it has some limitations: (i) very small sample, (ii) the key analysis, involving splitting the 1A/1B groups, was not planned ex ante (leaving it open to accusations of p-hacking), (iii) we don’t know why the 1A group lived on the MRC compound but 1B didn’t, (iv) we would need to know more about hygiene behaviours, toilet type and animal contacts to drawn stronger conclusions. This doesn’t take away from that fact that it’s a very interesting study, and the differences between 1A and 1B are substantial.

washeconomics

britta

☑ ☆ ✇ WASHeconomics Blog

Recent increases in rural water access in African countries relied predominantly on groundwater

By: IanRoss

The JMP’s online analysis tool allows water supply data to be cut by service level (safely managed, basic etc.) or facility type (piped, non-piped), as set out in their 2018 methods doc. “Piped”, W2 in their indicator list (p.4) includes all tap water classifications (p.9), i.e. both on-plot piped and off-plot kiosks or public taps. Non-piped improved, W3 in their indicator list, includes all other improved services, namely:

  1. Groundwater point sources (protected springs, wells and boreholes)
  2. Rainwater (e.g. from house roof)
  3. Packaged and delivered water (e.g. bottles, tanker trucks)

Since categories 2 and 3 form a very small proportion in rural areas in most countries, the vast majority of “non-piped improved” comes from groundwater. A lot of “piped” may do too, but that depends on the network’s sources and the data cannot be collected via household surveys [let me know if you know of data on the relative split of surface/ground water in rural African piped systems]. A shift towards on-plot piped supplies in rural areas will hopefully take place, and this may or may not involve more reliance on treated surface water.

The analysis of JMP data below came out of ideas kicked about by Richard Carter and myself as a follow-up to our 2016 paper on functionality. One day we might manage to finish the follow-up paper, but in the meantime here’s some analysis of some JMP data, followed by a few thoughts.

First, Table 1 below shows data from the JMP “world file” available here. I used that file to extract the proportion of the rural population in 2015 using different service levels and facility types for drinking* water, for the five most populous Sub-Saharan African countries. This mainly serves to bring clarity on what is included in each category in adding up to 100%.

Table 1: Access to rural water supply for drinking water in five African countries in 2015

table

Second, I calculated the number of the rural population in 2000 and 2015 using each of ‘piped improved’ and ‘non-piped improved’ for the same five countries (Figure 1) for Sub-Saharan Africa as a whole (Figure 2). Note that populations using an unimproved point source or surface water are not shown. The white-filled bars indicate projected total rural population in 2030 from the UN World Urbanisation Prospects 2018 database – if we achieve universal basic access by 2030 these bars would be filled with one of the two colours (remember that “basic” also implies round-trip time <30mins).

Figure 1: Rural populations in five African countries using piped and non-piped improved water.

1Figure 2: Rural population in Sub-Saharan Africa using piped and non-piped improved water.

2

Some analysis:

  • With the slight exception of Ethiopia, the absolute increase in the number of people using ‘piped improved’ supplies in rural areas has been relatively small (Figure 1).
  • This comes in contrast to the huge increases in all five countries in non-piped improved, also visible for Sub-Saharan Africa as a whole (Figure 2).
  • The increases in population using non-piped improved (as opposed to piped on-plot) was approximately 2x higher in DRC, Ethiopia and Kenya, 4x higher in Tanzania, and 18x higher in Nigeria. It was 2.9x higher for Sub-Saharan Africa as a whole.

As a group of facility types, “non-piped improved” has seen astonishing growth in the 2000-2015 period in these large countries and in Sub-Saharan Africa as a whole. The majority of these service types (i.e. all except rainwater and packaged/delivered water) are based on groundwater. The numerical (Figures 1 & 2) and proportional (Table 1) significance of these groundwater-based supplies, and their likely future importance, are difficult to ignore.

While it would be ideal to have piped on-plot supplies for all by 2030, this appears unrealistic given current access trends (not to mention other constraints). In other words, it is hard to see the blank 2030 bar being filled with blue, and don’t forget that the blue category includes off-plot taps/kiosks. Meanwhile, “at least basic” off-plot services for all in rural areas by 2030 appears an achievable goal (noting that “non-piped improved” is not the same as “basic” – see p.16 of the JMP methods doc). Nonetheless, achieving piped on-plot** services should be targeted for as many as people as possible, where incomes and willingness to pay are sufficient to fund operational costs to keep piped systems running.

These data show that point-source groundwater supplies are likely to remain important to 2030 and beyond. In a recent book chapter, Calow et al. (2018) show the nuance of what this means for equity in access. We would do well to note their ‘three priorities’:

  1. Invest in water resource assessment and monitoring
  2. Recognise that degrading water quality poses at least as great a risk to drinking water as over-exploitation
  3. Engage in the wider conversation about water resources management – who gets what as pressures on resources increase and climate change accelerates.

Conclusion

In conclusion, this post makes three points: (i) rural water supply access gains over 2000-2015 in African countries relied predominantly on point-source groundwater supplies, (ii) point-source groundwater supplies are likely to remain important to 2030 and beyond, (iii) a shift towards on-plot piped supplies in rural areas will hopefully take place, which  may or may not involve more reliance on treated surface water. However, the shift is unlikely to be fast enough such that universal access to this type of service is achievable by 2030 in rural areas of African countries.

*Surveys which underlie JMP ask about the primary source of water for drinking. Other studies have established that people use multiple water points (improved and unimproved) if available, especially for purposes other than drinking.

** Note that in some countries, a substantial proportion of non-piped improved sources are on-plot, see figure on p.29 of this JMP report on safely managed drinking water.

 

washeconomics

table

1

2

☑ ☆ ✇ WASHeconomics Blog

The economics of antimicrobial resistance and the role of water and sanitation services

By: IanRoss

 

wwtp.jpg
Wastewater treatment plants – hotspots for resistant genes and bacteria?

Seeing a paper published a few weeks ago in Nature Communications (more on that below) reminded me of some reading I did last year on WASH and antimicrobial resistance (AMR), and got me thinking about the economics of this.

What is AMR?

Antimicrobial resistance (AMR) occurs when microorganisms adapt after exposure to antimicrobial drugs (e.g. antibiotics, antivirals).  Microorganisms include bacteria, viruses and parasites. While AMR is not only about antibiotics, that is the area which has garnered the most concern. Why? Because it is potentially terrifying. Imagine if most hospital operations became too risky due to the chance of infection, or if people died from basic diseases once easily overcome.

As Margaret Chan of WHO put it, “AMR threatens the very core of modern medicine and the sustainability of an effective, global public health response to the enduring threat from infectious diseases”. And from 1980s-2000s, there was a 90% decline in the approval of new antibiotics.

Economic analysis of AMR scenarios

For economic analysis of this kind of thing, the problem is uncertainty about costs and consequences. It is hard to assign probabilities to different AMR scenarios, and hard to adequately cost up the potential economic damage. Prominent health economists Smith & Coast found in a 2012 UK-focused review that estimates of the economic burden were surprisingly modest – from £5 to £20,000 in additional hospital costs per patient per episode (driven by methods for incorporating productivity losses) but argued that these had been too simplistic. At a societal level, estimates were far lower than things like cancer and heart disease in the US and UK, so AMR wasn’t prioritised.

In an accompanying BMJ editorial, Smith & Coast suggested that existing cost estimates did not take into account the fact that antimicrobials are integral to modern healthcare. Noting that accurate cost forecasting is near-impossible, they argue that we should view greater investment in preventing AMR as an “insurance policy”.

This reminded me of working on the economic appraisal of WASH in the context of climate change, and the debates about no-regrets / low-regrets adaptation (see p.32 of this). In a another paper I co-authored with Julian Doczi, we considered appraisal options based on the fact that the ‘deep’ uncertainty surrounding future climate scenarios severely weakens the theoretical foundations of cost-benefit analysis. This DFID topic guide on ‘decision-making under uncertainty’ is good at explaining why. It’s the same problem for AMR – there is the potential for catastrophic outcomes with unknown but not-that-small probability. Standard econ. appraisal methods don’t like this at all.

How does WASH affect AMR?

But what has WASH got to do with AMR? This WHO briefing note provides a good overview of the issue. Essentially, water, wastewater and faeces play a key role in the carriage of microorganisms and their genetic material. Water can act as a reservoir of resistant bacteria and exposure routes to humans (and animals).

So there are many ways in which WASH, and water more broadly, could affect AMR. However, there is evidence that wastewater treatment plants (WWTPs) are hotspots for AMR genes and bacteria. This is unsurprising as they bring multiple sources of waste into one place in huge volumes. Landfills and food crops (cf. wastewater irrigation) are also a concern. WHO state that many faecal indicator bacteria, such as E. coli and enterococci, are now resistant to some antibiotics, with some evidence of higher morbidity and mortality as a result.

The study published a few weeks ago tackles an important question in all this. That is, whether increased levels of antibiotic-resistant bacteria in sewage and river water are more a consequence of (A) on-site selection from antibiotic residues in the environment, or (B) contamination by fecal bacteria that just tend to be more resistant than other bacteria. Explanation A would be “worse”, in the sense that it would confirm fears about increasing AMR. Fortunately for us, by looking a specific virus as a marker, they find:

“the presence of resistance genes can largely be explained by fecal pollution, with no clear signs of selection in the environment, with the exception of environments polluted by very high levels of antibiotics from [pharmaceutical] manufacturing, where selection is evident.”

Phew, up to a point. Nonetheless, as one of the co-authors noted, their results do not exclude the possibility that there is gene selection going in in parallel. And it’s only one study. The potential risks linking AMR and WASH remain.

What’s this got to do with WASH economics?

The Smith & Coast work discussed above focuses on the economic burden / damage costing, i.e. what are the costs of inaction. These fall mainly on people’s health and in healthcare services. What would be more important for WASH is an intervention cost perspective. That is, what does a WASH-related bit of the Smith & Coast ‘insurance policy’ look like? What should we be spending money on, in the WASH sector, to reduce risk of catastrophic AMR?

The answer appears to be that we don’t know yet. The WHO briefing note sets out recommendations for risk assessment and risk management and at the policy level, as well as identifying research needs. One thing they recommend we should be doing is identifying and quantifying relevant bacteria as part of existing microbial monitoring. But there’s still clearly a lot we don’t know. It seems less obvious what should be done in the WASH sector, as compared to the health sector. That makes sense given the health sector is the source of antibiotic use while WASH systems and ecosystems are vectors for genes and bacteria. This is one area of water risk that I’m sure we’ll be hearing more about in the years to come.

Once we do know more, and are in a position to be recommending specific investments, my three questions that would need to be asked of any WASH-related insurance policy would be:

  1. Is there is anything the WASH sector could be doing to reduce risk of AMR, beyond fully implementing existing guidelines for water and wastewater management?
  2. What would those things cost?
  3. Would doing them be a good use of resources compared to alternatives, and how uncertain are we about that? [it will be very hard to say because, as discussed above, the damage costs of AMR are highly uncertain].

washeconomics

wwtp.jpg

☑ ☆ ✇ WASHeconomics Blog

The difference between economic and financial analysis for WASH services

By: IanRoss

The distinction between economic analysis and financial analysis is not always straightforward. In this post I try to clarify this.

Definitions

I have previously defined WASH economics as “the study of how people make decisions about the allocation of scarce resources in the delivery and use of WASH services.” See that post for more discussion of definitions.

In turn, my working definition of WASH finance is “the study of how WASH services are paid for, including who pays, how and when”. More on this definition another time. Within the realm of “finance” it is important to distinguish between funding and financing, as is now becoming the norm. In a recent book chapter on equality in WASH funding and financing I co-authored with Richard Franceys, we explain this as follows:

“Funding is broadly defined as providing money which is not expected to be repaid. In the WASH context, funding usually comes from three sources: tariffs (including self-supply expenditure or user charges such as connection fees), government tax revenue, and donor transfers. Together, these are known as the “3Ts” framework, popularised by the OECD. In contrast, financing, is defined as providing money as a loan or equity in the expectation that it will be returned in full and with interest, in the case of debt, or dividends from profits, in the case of equity. In other words, funding is the provision of non-repayable money and financing is the provision of money which is repayable to the financiers.”

Turning to economic and financial analysis, I would characterise that as technical work which uses the perspective and methods of those two disciplines to appraise plans, projects and investments. The reason I emphasised the phrase “paid for” in the above definition of WASH finance is that it brings out the difference between economic and financial costs.

Economics costs vs. financial costs

Economic costs are the opportunity cost of resources (i.e. the value of the highest-value alternative use). Financial costs, meanwhile, are resources that are “paid for” (a turn of phrase borrowed from the health sector).

Not all resources used in the delivery of WASH interventions and programmes are paid for. Consider unpaid household time in programme participation or toilet construction, and the use of an asset that is donated to the programme, such as a vacuum truck. An estimate of the value of each of these resources would be excluded from a financial analysis but included in an economic analysis. This is because economic analyses should assess opportunity costs (defined above).

Underlying this is issue of valuation, i.e. what are things worth? Theories of value have been debated in economics since the discipline began. Financial costs are normally straightforwardly valued at the price paid. The complicated part is how to spread them over time – the financial cost of a programme in a given year is rarely the same as programme expenditure in that year.

Valuation of economic costs, however, is more tricky. There can be many competing ways to value an opportunity cost. For example, the opportunity cost of a person’s unpaid time in undertaking unskilled labour might be taken as (i) the minimum wage rate in that country for unskilled labour, (ii) 50% of that (reflecting the fact that the time may not have been allocated to income‐generating activity), or (iii) some other assumption based on another wage rate local to the setting (if the minimum wage is not a good reflection of market wages). The opportunity cost of a donated vacuum truck might be its estimated resale value in the open market. So, the total economic cost of a programme or intervention is the value forgone of all resources used.

Financial analysis implies the perspective of a given payer, whereas economic analysis usually (but not always) implies a societal perspective. So, economic evaluations (such as those employing cost-benefit or cost-effectiveness analysis) usually take a societal perspective. Planning and budgeting exercises, meanwhile, usually take the perspective of the institution that will pay for the programme of service. For example, the budget for an NGO’s rural water programme would only include the financial costs that would pass through their books. It would not include financial costs borne other stakeholders partners (such as local governments or households) covered from other revenue sources.

Types of economic and financial analysis 

There are many types of economic and financial analysis. All require cost analysis. Going into them is beyond the scope of this post. In brief, economic analysis is primarily concerned with efficiency (whether technical, productive or allocative) so includes things like economic evaluation (cost-benefit, cost-effectiveness), damage cost assessment, etc. but also assessment of economy and input/output relationships. Altogether, most of these things are part of Value for Money analysis (see diagram on p.5 of this). Financial analysis, meanwhile, includes things like funding gap analysis, cashflow analysis, willingness to pay assessment etc. – note the focus on covering costs, rather than on assessing efficiency.

The table below uses a few examples to illustrate some important of the purposes, and how the purpose drives the analytical perspective and type of cost used. More on this another time.

blogpost_-_fin_v_econ

Conclusion

My short (and perhaps flippant) summary of the difference between economics and finance is that “economics is about valuing stuff” and “finance is about who pays, how and when”. When considering an investment, both economic and financial analysis are important. In addition to knowing whether a project will have net benefits (i.e. is worth doing), it is important to know that it will be financially sustainable (i.e. the life-cycle costs can be covered).

 

washeconomics

blogpost_-_fin_v_econ

☑ ☆ ✇ WASHeconomics Blog

Recall bias and cost data

By: IanRoss

I’ve been working on costing a few programmes recently where the intervention happened between 3-10 years ago. Both used household surveys asking people what they spent (in cash and in kind) towards the original infrastructure output (CapEx), towards regular operational and maintenance (OpEx) and irregular capital maintenance (CapManEx). It’s got me thinking about the various recall bias issues involved.

Look at the graph below, which is completely hypothetical. Let’s assume it’s for a sanitation intervention amongst 1,000 households which happened in 2008. The y-axis is some measure of ‘data quality’ when you ask 1,000 households about expenditure. If you asked in 2008 how much they spent (cash and kind) to construct a toilet (CapEx – blue line), they’ll probably still have a good idea because it was very recent. However, as time goes by, they’ll forget exactly what they spent, so the blue line drops. Data quality will drop fast at first, but then plateau after a while because people are likely to remember the order of magnitude of  what they paid.

recall bias

For OpEx (orange line), which is by definition recurrent expenditure which occurs with a regularity of one year or less, it’s a different problem. If you ask people in 2009 about their OpEx, they won’t have that good an idea because they only have 1 year’s experience of using the toilet. Maybe they’ve cleaned it regularly but not spent much more money so far. Over time, they build up more experience of how much they tend to spend on OpEx and data quality becomes good.

CapManEx (grey line) is the hardest. It is recurrent expenditure occurring less than once a year (e.g. pit or tank emptying costs). So any time you ask people about it they’re less likely to have experienced it recently than with OpEx. Stuff normally works well when it’s new, so people are unlikely to experience CapManEx for a fair few years. With a toilet, for example, you’re only likely to need to empty the pit or septic tank 3-8 years after installation depending on numerous factors. So you only start getting likelihood of good data on CapManEx many years after the intervention, but even then it’s never going to be brilliant because many people may not have incurred it recently, and you have the same recall bias issues.

So when should you do your cost data collection? It depends on your objectives. If you’re most interested in CapEx, then do it ASAP. But if the lifecycle element is more important to your study, then it’s probably best to wait at least 3 years, maybe even 6-8 years if it’s the CapManEx you’re most interested in. You can always impute the CapEx from other sources or ask a different sample of people who constructed more recently. Of course the best case would be to get regular data with the same panel of households at intervals, but who is going to fund that…

washeconomics

recall bias

☑ ☆ ✇ WASHeconomics Blog

Preferences and constraints – when does container-based sanitation address the binding constraint to uptake?

By: IanRoss

In welfare economics, “preferences” denote which alternative goods or services someone would choose, based on the relative “utility” provided by each (more on utility another time). For example, when presented with a box of chocolates, my first choice is always a praline (P). But if only marzipan fruits (M) and brazil nut caramels (B) were left, I would choose the latter. This allows my preferences over a “set” of chocolates to be written as:

P B M

However, if I liked B and M equally, I would be “indifferent” towards them, written as:

B ~ M

Enough about chocolate. Let’s assume that I interviewed 1,000 people in an urban setting. I asked them about their preferences over various household sanitation options if there were no constraints like space or money, and I found that:

  1. People’s first choice would be a private household toilet if it were possible
  2. Preferences for other types of sanitation followed the order of categories of sanitation I set out in this post, giving the result:

Household private Household shared Communal Public Open defecation

This is called stated preference (what they told us) as opposed to revealed preference (what we observe in people’s actual choices).

Household shared was preferred to communal because of not having to leave the plot. Communal was preferred to public on the assumption it would be closer, cleaner and cheaper on average. The other preference relationships are obvious, though may not hold in all urban settings, e.g. there is a revealed preference for open defecation in some parts of India.

The interesting question is, if people aren’t able to fulfil a stated preference for a private household toilet, which constraints prevent them from doing so? Furthermore, which of those is binding? Interventions addressing other constraints will not increase demand unless they address the binding constraint.

I’ve been thinking about this every time container-based sanitation (CBS) is considered as an option in a city I’m working on. CBS comprises systems in which toilets collect excreta in sealable, removable containers for transport to treatment. Below is one example of a CBS service chain from SOIL in Haiti.

SOIL_cycle

Where would CBS fit into the above preference set? The answer would depend on which constraint the household was facing towards fulfilling their preference.

So, what are common constraints to uptake of private toilets in urban areas? Three factors I’ve seen most reported are:

  1. limited space – nowhere to build a toilet on the plot
  2. limited ability to pay (ATP) – not enough access to cash or finance to purchase
  3. limited willingness to pay (WTP) – investment not perceived as worth its opportunity cost

Land tenure and tenancy status are key, and closely related to WTP. People who own their home but are squatting on land may have ATP but not WTP, because they fear making sunk investments which would be lost in the event of eviction. Landlords may not have WTP for private toilets for tenants, since they will not benefit personally (though in theory they could increase rents – evidence on this is mixed). Tenants may be reluctant or unable to invest in a landlord’s property. The ATP constraint relates not only to inability to fund the absolute CapEx cost, but also to inability to spread it over time. There are more factors besides.

To my mind, CBS seems most appropriate where space and/or ATP are the binding constraints.

  • Space: CBS overcomes this by placing the toilet in an existing room rather than a new structure
  • ATP: CBS overcomes this, partially, by making sanitation a service with a small regular fee (like water), rather than an upfront capital investment.

CBS may also have its place in some settings where WTP is the binding constraint, but then its value proposition would have to be better than the alternative. From a tenancy/tenure perspective, a CBS investment is not a sunk cost.

Where does this leave CBS in the hypothetical preference set, then? Of course it would depend on the household e.g. what their alternatives are, whether they have a suitable room, their relative preference for leaving the household building / plot to use a toilet (which could be gendered). On average, I think it might look like this:

private CBS  shared communal  public open defecation

The symbol ≽ denotes  “weak” preference (“better than or equal in value to”), as opposed to “strict” preference ( “better than”). A CBS toilet is essentially a private toilet which is not in a room constructed for the purpose. So it offers many of the advantages of private toilets over shared ones (privacy, security, convenience etc.). I suggest weak preference above (≽) because whether CBS is preferred to an on-plot shared arrangement will very much depend on the setting. The same is true for the other relationships but probably less so. Whether a private toilet is preferred to CBS will depend on other factors too – a household with space and ATP would probably prefer a specific structure not taking up an existing room (allowing those in adjacent rooms to hear and smell…).

CBS is not a silver bullet for all urban sanitation challenges. However, it does have potential in some settings, especially informal settlements where space, ATP (cost-spreading) or tenancy/tenure are the binding constraint to uptake of improved sanitation.

washeconomics

SOIL_cycle

☑ ☆ ✇ WASHeconomics Blog

Categorisation of shared sanitation – some city-wide data using one approach

By: IanRoss

There has been a fair amount of debate on the role of shared sanitation in urban settings recently, see e.g. this comment piece from various stakeholders, this paper (plus others) from Sheillah Simiyu and this one from Marieke Heijnen. Also, WSUP recently issued an RFP for multi-country research on shared sanitation. In my own little corner of the sector, I’m working on a costing and cost-effectiveness study of the WSUP shared sanitation intervention in Maputo.

There are many types of shared sanitation – a toilet (or toilet block) can be shared amongst 2-3 families on a compound, among 20-30 identified families in a small area, or among all-comers willing to pay the entrance fee per use. A few years ago Adrien Mazeau did some work on ways of categorising the many different ways we can cut shared sanitation, ending up with a detailed typology (p.23). The categorisations are by location, access to whom, relationship of users, ownership, management, operation and payments.

A recent conversation on this issue made me look back at the data we collected under the World Bank five-city FSM study in 2014-16 (summary report here). Some useful data tables didn’t make it into the city reports (they were already 120pp long…). Below is a graph that didn’t make the cut. It shows sample survey data which are city-wide representative, collected in 2014-15 – more on the sampling in each city report. The variable shown is the latrine “usually used”, based on a simple sharing typology more or less the same as that suggested by the 2017 editorial referenced above:

  1. Household private (on-plot)
  2. Household shared (on-plot)
  3. Communal – pay per period (off-plot)
  4. Public – pay per use (off-plot)

Therefore, this sets aside issues of improved/unimproved (let alone safely managed), to zoom in on whether the toilet is on/off plot and the payment arrangement.

santype5city

* In Dhaka, there is also data representative of ‘slum’ areas (as defined by the Bangladesh Centre of Urban Studies). All information on sampling is in the city reports.

In the city-wide data, the graph shows that private household sanitation is used by the majority of households across these cities, with a sizeable proportion also using on-plot household shared. Only in Hawassa, Ethiopia, does communal sanitation play a role, for around 6% of households. In the data for low-income areas, it is clear that sharing plays a far larger role. In the Dhaka slums, less than 20% have private household toilet, and communal sanitation comprises almost 40% of toilet use. Pay-per-use public toilets were almost never used as the primary sanitation option in these cities, but this does not mean they don’t play an important role in providing sanitation options when people are out and about. nb. there was no open defecation reported as primary option in any city.

These categories were not derived from a single household survey question, but a series of questions about the attributes of the categories (questionnaire here). The difference between “household shared” and “communal” is that the former involves households sharing a toilet on their plot, with whatever cost-sharing arrangement they decide together, if any. The latter is normally a bigger “toilet block” type of arrangement where households can pay per month or similar for access whenever needed, and they have to leave their plot to get there (see this paper). Public toilets can be used by anyone, either for a small fee for each use or for free.

I’m not suggesting that the above categories are the best way to go. More work is needed on which categorisations are most important for policy/planning. Definitely it will be more than one set, as you can’t get all the relevant information (imp./unimp., on/off-plot, who can access, payment etc.) into one variable without it becoming unmanageable. The ways in which we categorise sanitation options and frame household survey questions and response categories are crucial for a good understanding of what is going on. Any given study or monitoring regime will have its own priorities. Whatever is done, it is key is that categories are well-defined, and enumerators (who are not sanitation specialists if a data collection firm is being used) understand the difference between different categories.

The JMP is still revising its “core questions” for household sanitation, but the latest JMP-endorsed questions are the module in the latest round of MICS here. The ones applicable to sharing are pasted below. So next time you’re doing an urban survey, best to use the below questions, unless you have a good reason not to! That way we’re all working towards consistent and comparable data on shared sanitation in the future.

MICS_san

washeconomics

santype5city

MICS_san

☑ ☆ ✇ WASHeconomics Blog

Sanitation’s share of water sector aid is falling

By: IanRoss

I went to an interesting event at LSHTM last night run by Countdown 2030, on tracking aid flows to track global aid flows to reproductive, maternal, newborn and child health (RMNCH). Their dataset is here. Yet another reminder that the health sector is way ahead of the WASH sector on so many analytical questions, but also that they face many of the same problems.

For example, the four (!) different methods for tracking RMNCH flows all use, at best, the “long description” on the OECD’s creditor reporting system (CRS), in which donors report aid flows. In my experience these can often have as illuminating descriptions as “the WASH project”, even for multi-million dollar disbursements. Delving deeper into project documents underlying the headline figures, in order to allocate between sub-sectors / areas, is only possible for national-level analyses, and is tedious and hard to automate.

Last night’s event took me back to work I did at WaterAid in 2008 as part of advocacy towards establishing what is now Sanitation and Water for All (SWA). At that time, I became rather too acquainted with the CRS, being the lowly RA crunching the numbers for this report. Many of our observations still stand for WASH, but also for RMNCH. It was depressing to hear research presented last night that showed aid effectiveness has actually gone backwards on many counts since 2010/11. Five countries had >30 donors providing aid to RMNCH – such fragmentation involves huge duplication and unnecessary administration for government ministries.

Going back to WASH, at that time of our 2008 analysis, the OECD didn’t have separate reporting codes for sanitation and water, so it was not possible to see what was going on for sanitation specifically. One results of the advocacy around SWA and the “international year of sanitation” that year was that the OECD instituted separate codes for reporting from 2010 onwards.

So, on the bus journey back to Oxford I decided to check back what had happened since, within the water sector as a whole. See the graph below, which necessitated a few methods assumptions summarised below this post. For transparency, here’s the XLS. The results show that the overall share of water sector aid allocated to sanitation has been slowly and steadily falling over the past 5-6 years, while that for water supply has been increasing.

washaid

Other categories have stayed more or less the same. Sanitation’s share has been falling from around the high 20s in 2011 to the low 20s in 2016. It would take some more detailed analysis to look at what is causing this (which donors, which recipients). I did google for this analysis but nobody seems to have done it that I can find (though see re: GLAAS below) – if I missed something, please point it out in the comments.

Note that this is in constant US$ / real terms, i.e. inflation is accounted for in the figures, and that these are disbursement data from all donors to “developing countries” as defined by the OECD. They are also for ODA, so excludes anything that isn’t within the DAC definition (e.g. Gates foundation, which is sizeable).

The 2017 GLAAS report (p.30) has a figure related to this, pasted below. The right-hand panel has a 65/35 split between water/sanitation (when I do the 2015 split for the sanitation and water codes I get 68/32, a minor niggle probably resulting from which donors they included or some minor methods difference). GLAAS 2017 was on financing, so also has a lot more analysis on different aspects of aid flows and government WASH funding. I would argue that it’s only when more countries are producing National WASH Accounts, using the TrackFin methodology or similar, that we’re really going to know what’s going on with financial flows at the national level, from all sources, which is far more important.

glaas_san_aid

In conclusion, the fact that sanitation’s share of water sector ODA is falling should be cause for concern. We don’t want the momentum built up after the international year of sanitation to be lost. Someone with more time than me should look into what’s behind this trend, and which donors/recipients are driving it, so that advocates can try to ensure that the share doesn’t fall further.
Methods points:

DAC purpose codes for 2016 are here. In the figure, the “WASH” category up to 2009 comprises six codes: two which are WASH combined (‘WASH – large systems’, ‘WASH – basic’,) and four which are separated by sector (‘Water – large systems’, ‘Sanitation – large systems’, ‘Water – basic’ and ‘Sanitation – basic’). Under “WRM” I grouped the codes for water resources conservation and river basin development. “Other” comprises codes for waste management and WASH education & training.

The key assumption is as follows: up to 2009, all water supply and sanitation aid is grouped under WASH (grey in the figure). From 2010 I used the newly-disaggregated codes to separate out sanitation (light blue) from water (yellow). However, even in 2016 there was still more aid under the combined WASH codes than under the separate sanitation and water codes. So the key assumption is that, where data are disaggregated between sanitation and water, the proportion of those values in that year can be applied to the WASH codes which are not disaggregated.

washeconomics

washaid

glaas_san_aid

☑ ☆ ✇ WASHeconomics Blog

Incremental benefits from increases in sanitation service level

By: IanRoss

The Indus valley civilisation (c.2,000 BCE) coupled on-plot water supply from wells with the first known sewers. However, it was the Minoans (also c.2,000 BCE) who were the first to have piped water systems – I marvelled at the clay pipes and stone sewers at Knossos on Crete. The Minoans understood that piped water on demand provided a better service than carrying it in jars. Their piped systems are likely to have cost more than alternatives, especially in a slave-owning society where labour was “cheap”/free. But the richer households of Knossos were willing to pay for that higher level of service.

Turning to modern day sanitation, a high level of service such as a sewer connection is going to cost more than an unimproved pit latrine, but also provide more benefits. By extension, each movement up the rungs of the sanitation ladder has incremental costs and incremental benefits. Note that ‘incremental’ is different from ‘marginal’ – in welfare economics marginal benefit is strictly speaking the additional satisfaction or “utility” we receive from an additional unit of a good or service (e.g. from an additional litre of water). Incremental benefit, however, can refer to any change in the output of interest.

It was thinking about this, and playing with the cost data in Hutton & Varughese’s 2016 report on SDG costs, which led me to produce the below chart. It aims to visualise which incremental benefits are associated with the incremental cost of an increase in sanitation service level. For example, the movement from open defecation to a private but unimproved pit latrine is associated with time savings and ideally some privacy and security too (depending on the superstructure). This movement has a fairly small annualised life-cycle cost per household, which is even lower if the latrine is shared with other households. Achieving such a service level increase might be the objective of many CLTS programmes.

The bars are ‘annualised life-cycle costs per household’ of that option (comprising hardware/software CapEx, OpEx and CapManEx). The coloured text qualitatively describes possible incremental benefits of moving up to that rung on the ladder, from the previous.

Figure 1: Incremental benefits of moving up the sanitation ladder, alongside costs of different levels of sanitation service (average for Sub-Saharan Africa)

costs_per_service_level

A similar logic applies to the other increases in service level. Moving from an unimproved pit to an improved-but-shared system (“limited” in SDG terms) can bring health benefits in the right circumstances, as well as some ‘wellbeing benefits beyond health’ such as privacy, dignity, security and comfort. However, many factors will determine whether these benefits are realised, including consistency of use, cleanliness of the facility, the sanitation practices of the rest of the community, and many more. For the move to ‘basic’ services, there is evidence for higher benefits over ‘limited’ services but it is mixed – no space to go into that here. Finally, the move to safely managed services (whether non-networked with FSM or networked sewerage) is where significant health benefits community-wide should be seen, through the removal of negative externalities once a high enough proportion of people are at that service level.

The cost data comes from Hutton & Varughese 2016 – the World Bank has helpfully published the dataset here. I used their raw data for urban areas for four technology options, reported in annualised per capita life-cycle costs: (i) cost of any pit latrine, (ii) cost of a septic tank system, (iii) incremental cost of septic tank system with FSM, and (iv) incremental cost of sewerage with treatment. Since the latter two are incremental costs, I added them to the cost of a septic system to get the total cost. I calculated the average for Sub-Saharan African countries, and then used assumptions as follows: I assumed a household size of 5 to get to per household costs, and an assumption of 3 households sharing to get to the shared estimates. Finally, an unimproved pit was assumed to cost 25% of an improved pit.

The figure above represents a simplification of reality, since all benefits rely on contextual factors – note the ‘likelihood’ framing in the figure. Around 1,000 people building and using improved pit latrines is likely to have a bigger health effect in a village of 1,200 people than in a city of 1 million, depending on the baseline situation. Similarly, a new borehole is likely to  have more benefits in a village where everybody drinks from the river, than in a village where most people already have piped water.

Furthermore, there are other economic benefits from different levels of service, such as avoided healthcare costs and time wasted in sickness or caregiving, or the potential value of resource reuse. Nonetheless, I think the figure represents a useful way to think about what we get for our money when we invest in higher levels of service.

Has someone else visualised incremental costs/benefits before, like this or in a different way (I couldn’t find anything)?

What would you improve about the figure? Do comment below.

 

washeconomics

costs_per_service_level

❌