by Sasha Alyson

Follow @TrojanAid

The UNESCO Institute for Statistics handles U.N. data about education, so I regularly look at UIS reports. Again and again, I’m stunned at what I find. Here are two examples I’ve reported previously:

One report used a line chart to connect education levels for various regions, suggesting a non-existent trend. The mistake would be understandable for a high school student. Here, it demonstrated that UIS had no grasp of basic statistics. Story.

By feeding low-quality data through complex formulas, UIS produces statistics with 2- and 3 significant digits – suggesting a high level of accuracy for numbers that are really junk. Story.

Here’s an interesting chart from UIS. It shows, by region, levels of gender equity regarding how many children have achieved the Minimum Proficiency Levels in reading and math.

Western NGOs tell us that when it comes to education, girls in developing regions face great discrimination. But it looks like Western countries have, by far, the greatest inequities. Sub-Saharan Africa is closest to equal. So what are these NGOs talking about?(1)

I looked at the data UNESCO keeps for each country. Here’s what I found for one of the relevant statistics:

Apologies to mobile phone users; I needed a big image to capture the breathtaking lack of data. Here’s what it shows: For 47 countries in sub-Saharan Africa, from the years 2006 to 2020, UIS had just four datapoints: Mauritius in 2009, Uganda in 2014, Senegal and Zambia in 2015. The other 701 spaces are blank. In short, there is no data at all for 91% of the region’s countries, and 93% of its population.(2)

I thought I must be looking in the wrong place, or interpreting something wrong. The online UIS database gives an email address for questions, so I wrote. No reply. I wrote again. No reply. I wrote to another UNESCO information address. No reply.

Here’s what UIS shows for “Oceania”:

There’s data for 2 of 13 countries – the two wealthy ones — with about 73% of the population. But these two are enormously different from the other 11. There is no reason to group them all together.(3) Do the regional averages for “Oceania” represent only the two Western countries? Or do they include “imputed” data from the other eleven? UNESCO doesn’t tell.

Back to sub-Saharan Africa: Did UIS really calculate a regional average despite having data (and not even current data) on only 4 of 47 countries? That’s hard to believe, but after two days of digging I found this statement from UIS:

“When data are not available for all countries in the region, the UIS imputes national data for the sole purpose of calculating regional averages. These imputed data are not published.”(4)

What does “impute” mean? It means the data you need is not available, so you wing it however you can.

Had I read this “UIS imputes” statement a year ago, I’d have assumed that in a 47-country region, data might be missing for three or four countries, so UNESCO filled in some small gaps. That’s common whenever you collect data for a large area. It would never have occurred to me that more than 90% of the numbers going into UIS regional averages were “imputed” from thin air.(5)

Why does UNESCO Institute for Statistics do this?

UIS itself says these numbers not suitable for planning; they are used to evaluate progress on the Sustainable Development Goals. But nobody who knows where they came from, will think they’re much good for that, either.

So why does UIS go through all this trouble? My answer is presented in another story: Why do UN agencies fabricate data?

Comments from Twitter

We announced this story on Twitter, where readers made these comments.]

Z3dWrit3r, @shasho_m: Reading this makes me realise we actually have to save ourselves. They manufacture the problem and provide the “cure” while we on the ground wonder what is going on. Truly sad from an organisation we are intended to rely on

QSehrose, @QSehrose: This means that NGOs etc. benefit greatly from it. The educational mandate serves as a pretext. So it is impossible to work out accurate statistics. Why isn’t it surprising to hear this?

Kayima, @Bukenmd: And I can tell you with authority, the Ugandan data is manipulated and very very inaccurate

Obsidian Geek, @JohnTembo1982: This is why we always take W.H.O data, and really data and reports from any UN organisation, with a pinch of salt. Read further into the numbers. Check to see how the data was collected. For a lot of African countries data is extrapolated.

Notes and Sources

1. More Than One-Half of Children and Adolescents Are Not Learning Worldwide, by UNESCO Institute for Statistics, September 2017, page 9. One region, Oceania (for which most or all of the data is from Australia and New Zealand), is not in the chart, although the relevant numbers do appear in the report. Here’s how it would have looked with Oceania:

As for why this chart shows a very different story than what we usually hear, I can propose two reasons. First, in the West it is boys who are disadvantaged in education. Aid agencies have no interest in this; they’ve trained donors to believe that funding for girls’ education is all that matters. Second, this inequity is in Western countries. The U.N. and the big NGOs aren’t going to waggle a finger at the USA, Europe, Australia, and New Zealand and say, “You discriminate!” They won’t bite the hand that feeds them.

2. Source of both charts: The UIS database: http://data.uis.unesco.org/ On the left, move down through several menu layers to SDG, Sustainable Development Goals 1, Target 4.1.1, then make another selection the the drop-down menu at the top. The charts I’ve displayed show “Proportion of students at the end of lower secondary education achieving at least a minimum proficiency level in reading, female (%).” I used this, rather than primary level, because data can be available for either of two primary levels and UIS doesn’t say which it used. I used reading, not math, because I consider that the more important fundamental skill. To calculate gender parity, which is shown in the UIS chart, involves calculating a ratio of male and female learning levels. You must have both. I chose the chart for females, just because it came first; the chart for males shows data only for the same four countries and years, though the males are usually doing much worse. (Uganda shows no significant difference. The others show much worse outcomes for males. In Zambia, 13 females achieved minimum levels for every 7 males who did so. Mauritius and Senegal are in between.) From a spreadsheet of all countries, I pulled out just those in sub-Saharan Africa for display here.

3. UNESCO states that it is U.N. policy to group the Pacific Islands with Australia and New Zealand for purposes of SDG statistics. That is not true. The U.N.’s SDG department says (quite logically) that these are separate and different, see UN SDG Statistics Regions.

4. UIS Frequently Asked Questions, Education Statistics, September 2015, page 6.

5. When I say data was pulled from thin air, I do not mean that numbers were entirely random, but that the process was so subject to error, from many causes, that the results are meaningless, and different analysts, working from the same limited data, could easily have produced wildly different results. In various sources, UIS gives a general sense of how its “imputation” occurs. It may involve taking similar data from other locations, or sort-of-similar data from the same location, and extrapolating. But in this case, the other data available was sparse, and of dubious quality. If UIS believed it had done a good job, it could publish the underlying numbers and tell how they were derived.

Related stories

Other stories of interest