Cooking the Numbers about “Learning Poverty”

Childhood illiteracy is already a pressing concern in the global South. But U.N. agencies, the Gates foundation and USAID have greatly exaggerated it. Why?

by Sasha Alyson

A recent UNICEF announcement caught my eye:

70% of 10-year-olds in low- and middle-income countries cannot read and understand a simple text.”

“As a result of the worst shock to education and learning in recorded history, learning poverty has increased by a third in low- and middle-income countries, with an estimated 70% of 10-year-olds unable to understand a simple written text…. This rate was 57% before the pandemic, but now the learning crisis has deepened.”(1)

And in sub-Saharan Africa, it says that even before Covid-19, 86% of these children couldn’t read. I’ve researched this topic a lot, I expected the number to be high – but not nearly that high. Something seemed off.

I downloaded the full Global Learning Poverty report and looked up Laos, where I live. It said that 97.7% of 10-year-olds in Laos could not read and understand a simple text.

Now I knew I was looking at garbage. Global Learning Poverty was published by six foreign aid giants: The World Bank, UNESCO, UNICEF, the British office handling aid (FCDO), USAID, and the Gates Foundation – a group which I’ll hereafter call WB+5. These six agencies are headed by four Americans, one British politician and one French politician. And I felt, with a moral certainty, that as weak as schools are here, these giants had gotten their data completely wrong.

* * *

To draw its conclusions about literacy, WB+5 compiled results from various standardized tests. TIMSS and PIRLS are often used in Western countries. For Africa, the most common test is PASEC.

The PASEC 2019 report defines these levels of literacy:

Below Level 1: Reads little or not at all.

Level 1: May read words or very short sentences, but often has difficulty with a short, simple sentence.

Level 2: Can read short, simple sentences and “paraphrase explicit information from a text.”

(A red bar appears between Levels 2 and 3, with the words “Minimum proficiency level”, apparently marking Level 2 as minimum proficiency.)

Level 3: Can read a range of longer texts and make simple inferences.

Level 4: Can read longer and harder texts, can “combine and interpret multiple implicit ideas.”

In Global Learning Poverty, WB+5 say that 86% of 10-year-olds in sub-Saharan African fell short of Level 2 in 2019. But it pulled the wrong data, and published numbers for the hardest group (Level 4). In other words, a child who could read longer texts and make simple inferences, was said to be unable to read a simple text.

On top of that, WB+5 assumed that every child not in school cannot read. So what was already an inflated illiteracy rate, was further inflated. And wrongly so. UNESCO itself noted last year that “3 in 10 of all young adults who have not been to school… could read.”(2)

The tables below show data from the PASEC report, and how it was used in Global Learning Poverty:

The PASEC chart, showing what percentage of students scored in each category, with <1 being non-readers, and 4 being the most skilled. Level 2 can read a simple text. The Learning Poverty report used data from Level 4, subtracting it from 100% to get the “Learning Deprivation” figure shown in the table below. But this tells how many students did not reach the highest, hardest level, not how many were unable to read a simple text.
From The State of Global Learning Poverty 2022 Update. I have extracted only the countries of Africa, and condensed the chart and shortened some names to make it easier to read on small screens. Columns are:
LP: Learning Poverty. The percentage of all 10-year-olds who (says the report) cannot read and comprehend a simple text. This is the headline number. It is derived from the next two numbers.
LD: Learning Deprivation. The percentage of those tested who failed to reach the designated level.
SD: Schooling Deprivation: The percentage not in school. WB+5 assumes they cannot read.
Assessment: Year, and name of assessment.

PASEC was mostly used in smaller countries. What about the others?  WB+5 tell us that the age-10 illiteracy rate for the whole region of sub-Saharan Africa was 86% in 2019. But they’ve got no data for Nigeria, Kenya, Tanzania, and other countries that account for about half the population. How did they produce a regional average?

The U.N. has explained (though it wasn’t easy to find) that if data is missing for a country, the agency conjures up a number “based on the data of neighbouring countries or of countries with similar levels of income,” then uses that to produce a regional average. It publishes the average, but not the conjured-up numbers.(3)

For six countries of southeast Asia, WB+5 used the 2019 SEA-PLM report, co-published by UNICEF. Again, they took data from the very hardest level rather than the level which showed who could not read a simple text.

That explains the bewilderingly low number for Laos. According to SEA-PLM, 50% of Lao children can read a simple text; WB+5 said only 2.3% could do so. In the six participating Asian countries, the tests showed that 84% could read a simple text; WB+5 reported that only 37% could.

And the rest of the globe? WB+5 used the PIRLS evaluation for 53 mostly-wealthier countries including Australia, France, the U.K., and U.S.. But for these, it used the right data: The easiest level. Thus we’re comparing data which claims to measure the same thing for various countries, but it does not. Here’s an example:

10-year-olds in school who could not read a simple text, according to WB+5’s Global Learning Poverty:
Senegal: 58.9%
France: 6.3%

10-year-olds in school who could not read a simple text, according to the original reports from which WB+5 took its data:
Senegal: 7%
France: 6.3%

Overall, the PASEC countries range from Gabon with an impressively low 2.6% rate (lower than the U.S., U.K., Germany, or Australia), to Chad at 51%. Nobody comes close to the WB+5 scare number of 86% non-readers.

For 2022, WB+5 estimates that only 11% of 10-year-olds in sub-Saharan Africa can read a simple text. I’ll do what they do not: I’ll admit that neither I nor anyone else knows the figure with much precision. But looking at all available data, I’d estimate it to be around 60-70%. Six times their number.

* * *

Several questions arise from all this.

Didn’t anybody notice?

Three Lao colleagues and I have been doing literacy work in Laos together since 2006. All four of us, when we saw the WB+5 statistic for Laos of 97.7% not reading, were stunned. We knew it was wrong. One colleague contorted his face with a look of pure scorn that I didn’t know he was capable of.

Out of six global agencies, who present themselves as experts, did no one suspect or know that this data was wrong? Or, as numerous U.N. whistleblowers have said, did many people know but they were afraid to say? How much will the developing world have to suffer, because well-paid functionaries blithely sacrifice the ideals they may have once held, just to stay on pleasant terms with their boss and keep the paychecks coming?

Why has the UN changed its tune?

There is indeed a learning crisis, although you wouldn’t have known it from United Nations press releases in 2015. At that time, it announced great success on the education front, because more children than ever were in school. Were they learning anything? The U.N. in 2015 did not even address that question.

What changed? Motive. In 2015, the U.N. was eager to portray its Millennium Development Goals (the MDGs, which ran from 2000 to 2015) as a spectacular success so as to get unquestioning approval, and then funding, for its new Sustainable Development Goals (SDGs, running 2016 to 2030).

Approval was duly granted by the General Assembly. With that secured, the very next year UNICEF announced a “learning crisis” in east and south Africa, and in 2017 the World Bank joined it to tell of a “staggering” crisis. Now they all wanted funds to address the newly-discovered crisis.(4)

How did it happen?

How did such bad numbers get into the report? I can propose two or three possible theories.

Pure sloppiness. I’ve written before about the high levels of carelessness among U.N. agencies, and especially UNESCO. (Large INGOs also produce bad numbers, but usually with a clear goal of deceit. With U.N. agencies, it’s often just pure cluelessness.) They are accountable to no one except their donors, and while donor motivations vary, they’re not in this because they expect good data from the U.N.

Willful deceit. The Learning Poverty report gives the aid industry what it wanted: A big crisis which justifies intervention, and more funding. But was this a conscious, deliberate plan? There’s no way to know.

Some of each. I find this the most plausible explanation. When the sloppy data handling produced numbers that met their needs, they ran with that and never looked back. If they didn’t like the result, they changed a few variables and tried again.

Why?

Why publish junk data? Why the urge to exaggerate illiteracy, when it’s already far too high? Look at this unretouched screenshot and try to guess one reason.

Institutions are driven to find money, every bit as strongly as animals are driven to find food.

On an individual level, this creates jobs. Somebody’s got to look up test scores in Niger, copy them into the empty spot for Nigeria, calculate regional averages, then erase the false Nigeria data. This money also boosts the travel budget, so they can fly to a conference and discuss how the North will solve the South’s problems, make good contacts, and sit by the pool.

But the desk jockeys, and even the agency directors, aren’t running the show. They work for the donors – Western money. It does not want children in the global South to fully develop as individuals … and then to help their countries develop … and then claim a right to use their country’s natural resources for their own benefit.

Nobody has to consciously say, “We want to crush those kids.” They merely have to keep funding a system which makes that happen.(5)

Now what?

As a solution, WB+5 propose a five-part “RAPID framework” which really boils down to three parts:

Get every single child into school. They’ve just described most schools as being essentially disaster zones, and they don’t explain why they want more kids to attend. There’s nobody to insist that they explain themselves.

Get more data. “We have committed to solving the learning data crisis, because countries cannot improve what they do not measure,” writes WB+5. Rubbish. Societies were educating their young long before anybody made spreadsheets about it. The “learning crisis” and U.N. databanks have mushroomed together. UNESCO (which handles the data) has made hash of the data it does have. How about we start letting children play and learn, instead of taking tests, and let teachers teach, instead of disaggregating data by age and gender, and let their communities decide how its going, rather than have plans handed down by Western aid professionals who wouldn’t know garbage data if it jumped out of the spreadsheet and bit them on the nose?

Improve school quality. Good idea! But if they know how to do this, why have they been achieving the opposite?

Meanwhile, two things are missing from the WB+5 analysis.

First, it focuses on certain types of academic achievement. We want our children to develop in far more dimensions than simply reading and math. Critical thinking, cooperation, initiative, planning, arts and physical activity, and much more. They need the opportunity for diverse experiences. Instead, this puts the focus whatever standardized tests will measure. Then children are pushed into rote-based classrooms that stunt their mental development.

Second, WB+5 never ask why schools have gotten worse. The only “why” question they want to discuss is “why some children have not returned to school.” I believe the U.N. led the world into this crisis, with a focus on enrollment rather than learning. Along the way, it got plenty of help from governments that were corrupted by aid money, or thought they could trust U.N. advice, or both. Other people may have other explanations. But just bailing water out of the boat isn’t enough. We have to find the leak.

That, however, is the last thing that the World Bank, UNESCO, UNICEF, the UK’s FCDO, USAID, and the Gates Foundation want to discuss. They’ve produced their scary headlines, that propels them right to their real objective: Control. They don’t mince words:

“These interventions should start immediately…. decisive action should not wait until countries have strong systems, because the cost of short-term inaction would be too high.”

In short, the World Bank, UNESCO, UNICEF, USA and UK aid, and the Gates Foundation have created fake news, to give themselves permission to go in and push through the systems they want, in the global South.

Sources

I invited the six agencies that published Global Learning Poverty to comment on the issues I raise here. Only one – Benjamin Piper at the Gates foundation – has responded to the question, a few others sent auto-replies. One referred me to someone else, who hasn’t answered. Mostly my inquiry was ignored.

Anyone skeptical of my account can easily check the facts in the original documents:

The State of Global Learning Poverty 2022 Update, published by the World Bank, UNESCO, UNICEF, the British aid office (FCDO), USAID, and the Gates Foundation. (Download from the “Related” link at the bottom of the page.) “Learning poverty” is defined on page 5; country data begins on page 66.

PASEC 2019 study report in English. Reading levels are defined on page 70; country data is on page 279. PASEC is for 14 French-speaking countries. Global Learning Poverty did not have data for some of the larger English-speaking countries. I have taken PASEC’s Level 2 as the level that meets the WB+5 “simple text” threshold. This states that a child is “able to paraphrase explicit information from a text,” so clearly they understand it.

PIRLS 2016 Report. Benchmarks are described on pages 52-53; country data is on page 58.

SEA-PLM 2019 Main Regional Report: Children’s learning in 6 Southeast Asian countries, by UNICEF and Southeast Asia Primary Learning Metrics (SEA-PLM). Levels defined on page 42, data chart on page 44, numbers on page 140. I used Band 3 as meeting the “simple text” threshold used by WB+5. This says “children can read a range of everyday texts… and begin to engage with their meaning.” Had WB+5 used Band 4, which says “children can understand simple texts that contain some ideas and information that are partly outside of their personal experience,” that might be defensible. But WB+5 used Band 6, the hardest of all, and then these numbers appear alongside children who reached the easiest levels in Europe and the USA. They are not comparable.

Other major assessments used by The State of Global Learning Poverty

For 19 countries, WB+5 used data from the TIMSS evaluation. But TIMSS measures math and science achievement, not reading. Most data used is from 2019 but some is from 2015, 2011, even 2007 – yet it appears in a table titled “Detailed 2019 country learning poverty data.” I have not looked at it further. Math and science scores do not tell us how many children can read a simple text.

For much of South Asia (including India and Pakistan), also Uganda, Ethiopia, and a few other countries, WB+5 gives “LAN” as the data source. But it gives no clue what LAN is, and so far I’ve been unable to track it down.

Data for nearly all of Spanish-speaking Latin America was drawn from UNESCO’s LLECES  assessment. I could not find results broken down by country, and it tests grades 3 and 6, so data would not be comparable with grades 4 and 5.

Notes

The collage at the top shows the directors of the six agencies that produced this report.

1. I count three errors in UNICEF’S brief statement. (1) UNICEF has just said that learning poverty went from 57% to 70%. That’s a 23% increase, not 33%. The “increased by a third” claim was apparently based on a previous estimate of 53%, although the newer 57% estimate is given in the same paragraph. I’ve referred to sloppiness at U.N. agencies; this is an example. (2) Whatever had already driven the number to 57% would seem like worse shock than a pandemic that pushed it another 13 points. I believe the U.N.’s disastrous push to increase enrollment and ignore learning was the primary cause, so it’s understandable that they want a scapegoat. (3) The 70% figure is wrong, this story explains why. Source: “70 per cent of 10-year-olds in ‘learning poverty’, unable to read and understand a simple text”, UNICEF, 23 June 2022.

2. Global Education Monitoring Report, 2021-22, UNESCO.

3. Yes, UN agencies do invent numbers, though they euphemistically call it “imputation.” I’ve explained here, with sources: Why do UN agencies fabricate data? and FOBOS: Fear Of Blank On Spreadsheet.

4. See, for example, the BBC report “In School But Learning Nothing,” 3 October 2017.

5. This is the essence of karma colonialism. The U.N. looks virtuous for supporting “education,” while in fact weakening school systems, mentally stunting children – and nations. I’ve described this process more fully in The Pan-African Review.

Related stories

Other stories of interest