by Sasha Alyson

Follow @TrojanAid

A few weeks back, I tweeted the map shown above (without the bulletholes), which displayed data published by UNESCO.

I was making a point about hypocrisy in the aid industry: The U.N. and Western agencies want us to believe that girls in developing regions face terrible discrimination in education, and that their intervention is needed to save the day. But even according to U.N. data, it’s Western countries that have the greatest gender inequality in one education outcome which UNESCO chose to highlight.

The map drew many irate responses from Australia and New Zealand. So before going further, let’s review a few points:

It’s boys who are disadvantaged in Oceania. Most people assumed the map said that girls faced a disadvantage in Australia and New Zealand. (Australia has about 61% of Oceania’s total population; Papua New Guinea has 21%; New Zealand has 11%.) They were indignant. Wrong assumption, but this time, for the tweet, I added an explanation. And there’s no doubt that boys are disadvantaged in the school systems of Australia and New Zealand — and also many other Western countries. Considerable data shows it. Many people tweeted their own observations to this effect. Here’s from @BarbaraMcK42, a Kiwi: “Totally consistent with NZ education policies that, with Australia, it has the highest gender gap in education – the disadvantaged being boys.”

Many people were less upset when they learned it was boys who are disadvantaged. So it appears we do still have gender prejudice. Some blamed the boys: “They’re lazy!” Clearly they’ve never seen 6-year-old boys in their natural environment. Children are eager to learn, if the adults in control will make it a priority to provide the right environment. But school environments are bad for many children, and are especially disastrous for many boys. How small does a mind have to be, to think that if not all children thrive in an unfriendly situation which lazy adults allow to continue, we should blame the children rather than the adults who run it?

Education quality and gender-outcome equality are very different things. Many critics protested that the quality of education in Australia and New Zealand is far better than in Africa. I’m sure that’s true. But that’s not what the map is about.

Schooling and education are very different things. In parts of sub-Saharan Africa, enrollment rates for boys are higher than for girls. When the aid industry boasts about promoting “girls’ education,” it nearly always means it is getting these girls into school. But even the U.N. has finally admitted that a lot of them “do not learn” in school. It’s cruel and stupid to continue focusing on enrollment rather than learning. But that keeps the donor money flowing. So maybe it’s not stupid. It’s just cruel.

---

But that’s not the end of the story. While constructing the map, I noticed a few anomalies. As I did more research, things got curiouser and curiouser….

It appears that UNESCO got some numbers wrong.

I don’t know which ones. But something’s fishy. The chart I used, produced by the UNESCO Institute for Statistics (UIS), gave 4 gender-parity statistics for each of 6 regions — 24 numbers in all. Here’s their chart. (MPLs are Minimum Proficiency Levels. Lines going to the left show that boys are disadvantaged; to the right, girls are disadvantaged. If a line ends in the gray area, that’s considered reasonably close to gender parity.)

Originally I had planned to publish that chart as my main graphic. But it was confusing to take in at a glance, so I converted the 4 datapoints per region into a single average, and created the map.

In the same report, the same regions and categories were also reported in numeric tables. Here I encountered curiosity #1: The numeric tables listed 7 regions, thus 28 numbers, rather than just 6. Even so, 24 datapoints appeared in both places, and the numbers should have been the same.(1)

In 16 spots, they were the same; in 8, they were different. And not just a trifling difference; in one case the chart showed a value of 0.88 and the table showed it as 1.00. (2)

The inconsistencies all happen in reading, not math. It appears that to make the chart, somebody at UNESCO copied numbers by hand, from a table. (Why weren’t they imported digitally from the same database without manual recopying? That’s the kind of thing computers are good at.) My best theory – merely a theory! – is that whoever made the table copied the math data in the morning, then did reading after a three-martini lunch.

UNESCO’s data actually tells us nothing about Australia and New Zealand.

UNESCO lumps together statistics for Australia, New Zealand, Papua New Guinea and many other smaller island nations. This is absurd; these countries are enormously different. It is possible, though unlikely, that Australia-NZ have perfect gender equality in this respect, and the Oceania average is dragged down by extreme inequality in Papua New Guinea and other countries.

On the other hand, it’s quite possible that boys in Australia-New Zealand are even more disadvantaged than the map suggests, and Papua New Guinea is pulling up the region’s average. Other Western countries may have even greater inequities but it’s hidden because they are buried in much larger regions. From the consolidated data UNESCO provides, we have no way to know…. although with further research, I found the answer.

Not only in this extreme case is it absurd to lump together different countries. Spain and France may be quite different; Kenya and Uganda. A region might appear to have perfect equality, whereas the average hides great inequalities which favor boys in one country, girls in another. UNESCO endlessly tells us that data is crucial for development; but it deliberately blurs the data. Why?

UNESCO dropped Oceania from its bar chart.

Used well, bar charts make numbers come alive. But while UNESCO produced data for 7 regions, the bar chart shows only 6. It leaves out Oceania. Why?

Here’s what the chart would have looked like, had they included all 7 regions:

Oceania kind of pops out at you, doesn’t it? Was UNESCO trying to avoid offense?

Former Australian prime minister Julia Gillard now chairs the Global Partnership for Education, which has burned through some $7 billion advocating, especially in Africa, for greater attention to girls’ education. Someone looking at this chart, which shows boys getting the short end of the stick in Australia, just might be inclined to turn to Gillard and say, “Oh, shut up and go home.” She wouldn’t like that, and she’s got a lot of well-connected friends.

Why 7 regions, not 8?

To report on its Sustainable Development Goals, the U.N. groups the world into 8 regions. Australia and New Zealand are one region; Oceania is a different region. But UNESCO combines them into 7 regions – and then claims that this is how the SDG does it.

Do these people communicate with each other? Here are their maps:(3)

What’s going on? Why does UNESCO go counter to U.N. policy, and also misrepresent it, just for the purpose of merging data from some of the world’s poorer countries with data from two of the richest?

I thought martinis might play a role again, but as I did more research, I came up with a better explanation.

Tricky statistics

To answer some of these questions, I looked inside UNESCO’s online database.(4)

There’s no relevant data at all for Papua New Guinea. That’s a fifth of the region; if we don’t have those numbers, we shouldn’t be claiming 2- or 3-digit accuracy for the region as a whole. Nor could I find data for any of the 12 other, much smaller islands. Unless it’s got more numbers that it’s hiding, UNESCO’s relevant data for Oceania consists only of Australia and New Zealand. Now we know why it chose to merge these two countries with the small island nations of Oceania! But it should have been honest with us.

And now some news that puts Australia and New Zealand in a different light. Yes, your gender inequity ratings are high when we measure children not achieving MPLs. But this offers an example of how statistics can be misleading, even when accurate.

Suppose your family, with 3 boys, is relocating and you’ve got a choice of two countries. In country A, boys are severely disadvantaged. Calculated by the criteria UNESCO has used here, the gender parity score is a miserable 0.57. In country B, boys get almost equal outcomes; the score is 0.97. Knowing nothing else, which will you choose?

Now you learn more. In country A, 77% of boys achieve minimum levels, compared with 87% of girls. That’s a 0.57 score. (13 girls fall short for each 23 boys who do so; 13/23=.57). In country B, 24% of boys compared with 26% of girls. That’s near equality; a 0.97 score. But you probably won’t chose to move there if education is a priority.

This offers a new explanation of why Oceania got left off the chart. UNESCO decided to publish statistics about how many children are “not learning” and then, too late, discovered that it had opened a can of worms: The computer produced a chart which makes two important, “developed” countries look off-the-charts bad – and unfairly so. What do you do? In this case UNESCO effectively swept it under the rug. Had the chart reflected badly, and unfairly, on Rwanda and Zambia, I doubt those countries would have received the same consideration.

UNESCO chose a bad indicator here — a ratio that creates misleading impressions when comparing countries with different levels of education. The data still makes an important point: That when it comes to society’s most disadvantaged children, Western countries put boys at extra disadvantage, and these countries have no moral high ground from which to preach their policies to the rest of the world. Understanding the statistics better now, if I were doing it over I’d caution against using one criteria to compare countries, and I’d try to show a bigger picture. That might be difficult. The issue is mostly ignored, because it gets in the way of conventional orthodoxy that focuses entirely (and profitably) on getting more girls into school. Western countries do put boys at a disadvantage; at this writing I haven’t found good data to compare these regions with the rest of the world.

And what’s the point?

Now let’s go back to where we started: A UNESCO Institute for Statistics report about education. But UIS was missing a lot of data, it scrambled some of what it did have, it swept an awkward chart under the rug when it would have embarrassed important countries, and by the time it consolidated country-by-country data into regions, the result was uselessly vague for anyone who might actually be working to improve education quality.

It’s not as pointless as it may seem.

First, the thousands of reports published by the U.N. each year create an impression that it knows what’s going on, and thus can be trusted to shape policy for all the countries which are assumed incapable of running their own affairs without U.N. advice.

Second, this work creates jobs. Well-paid, respectable U.N. jobs with generous benefits, for people to massage the data, hold meetings copy data, make charts, write reports. And, of course, fly off to conferences where they can give presentations, swap business cards, and line up connections for a different job a few years down the road.

Finally, these staff, who exert so much influence on public policy in countries they may never have even visited, are in turn easily controlled by big-money interests in the West. For a clear example, please read the story linked below, about the UNESCO-Nokia alliance.

In short, reports like this one won’t make education any better. But they benefit lots of people who will do their best to be sure the system continues.

Notes and Sources

1. Chart and data are from More Than One-Half of Children and Adolescents Are Not Learning Worldwide, UNESCO, and UNESCO Institute for Statistics, Fact Sheet No. 46, September 2017 (UIS/FS/2017/ED/46). Chart is on page 9. (I made small adjustments to the layout, but did not change data.) Reading table is on page 3, Math is at the end.

2. This table shows datapoints as reported in the UNESCO chart, and tables, both from the report listed above.

3. The 8-region SDG map is from the U.N.’s Sustainable Development Goals Report 2017. The 7-region is from the UNESCO report cited above.

4. Database for the UNESCO Institute for Statistics: http://data.uis.unesco.org/ Given the amount of data and categories, UIS has done a reasonably good job of structuring this, but it can still be confusing. You’ll need to burrow down through multiple layers, in the lists on the left, to find the data category you want; then more choices are offered in a dropdown menu at the top. They offer a user guide, if you need more help. Note that the database defines gender parity rates based on the male-female ratios of those in school who did achieve a certain level. For this report, UIS used ratios of those who did not achieve it, and attempts to report for all children, including those out-of-school.

Related stories

Other stories of interest