Would you believe someone who said they had just had one drink? Or would you prefer to ask someone else who had seen them in the bar? I know who I would believe.
So who would you trust more to tell you about the state of public services: the government department responsible for delivering those services, or someone independent?
In both cases, independent monitoring, (or at least independent verification), is likely to produce much more trustworthy results.
Which is why some new data on water supplies in Dar es Salaam from a World Bank-financed survey is very interesting - part of the Mobile Phone Public Services Monitoring Survey, previously under Twaweza. It contradicts official data from the Ministry of Water and DAWASCO in several ways. Analysts in the sector had always assumed that this official data was unreliable, but had always used it anyway because there was no alternative, better, independent data to work with.
So what does the new data tell us:
- 40% of Dar residents collect water from a well - see chart.
- Only 16% have a tap in their own house or yard - see chart.
- Only 12% report having a public kiosk or standpipe providing water in their area
All these figures contradict official data, and paint a picture of access to clean and safe water in Dar es Salaam that's worse than the Ministry of Water will tell you. Let's look at some of these differences:
- Ministry of Water (Budget Speech, 2011): 55% Dar residents collect water from an improved source (i.e. not including wells). Including the four "tap" options above as "improved", the independent data puts this figure at 48%.
- Ministry of Water (2011): 35% Dar residents live in areas served by public kiosk or standpipe. The independent data says 12%.
So why the differences?
Well, the biggest reason is likely to be that the Ministry of Water, and the people responsible for collecting and reporting data within the Ministry, all have an interest in making things look better. If you are responsible for increasing access to clean and safe water, you will want to show that improvements are being made. So even if you know what the real situation is, still you will find a way of making the numbers look better. You might ignore population growth, for example, or assume that some broken-down pumps and pipes are still working. Or you might even just make up the numbers. And if people collecting data keep making it look like there are small improvements each year, after a few years the data becomes wildly inaccurate.
In rural water supply (which I am more familiar with) the situation is very similar. Official data from the Ministry of Water has shown a steady increase in access to clean and safe water in rural areas of 1-2% per year, for the past 15 years. The increase was steady even in years when almost no money was spent on new water projects, and even in years of very low rainfall. In the same time period, household survey data collected by the National Bureau of Statistics (a more independent source), found that access to clean and safe water in rural areas had been decreasing.
I had a conversation a couple of years ago about this with a statistician within the Ministry, who quite openly told me that there were two sets of data kept in the Ministry - the real data that they themselves believe to be accurate, and the official data that they present to their political masters, the donors and the public.
This isn't just a Tanzanian phenomenon. In the UK, independently collected survey data on crime is trusted much more by politicians (and even by senior police officers) than official data collected by the police.
Which is why independent data like this new data from the Mobile Phone Public Services Monitoring Survey is so valuable. It gives us a much more accurate picture of the state of public services than official data can.
I don't trust the drinker to tell me what they've drunk. I want to hear from someone who saw their glass.
- - - - -