THE SCHOOL

Feature

EXPERIMENT
COVID-19 caused the largest disruption to education
in history. But research has identified effective ways
to help children catch up. By Helen Pearson

B
y October last year, Meg Brydon loss,” says Margarete Sachs-Israel, who leads
could see the terrible toll the the Inclusive Quality Education Section at
pandemic had taken on children at UNESCO in Bangkok.
her school. Brydon was a teacher Now, governments and schools need to
at Ashwood High School, in the know the best approach to help children
suburbs of Melbourne, Australia — catch up — and research could show the
the city that has spent more time in way. Over the past 20–30 years, researchers
COVID-19 lockdowns than any other in education, economics and international
in the world. The school had been closed, on development have built substantial bodies
and off, for about seven months. of evidence, including banks of randomized
Before the pandemic, around 10% of chil- controlled trials, showing strategies that are
dren who joined Ashwood at the age of 12 effective at boosting school attendance and
would be below the expected national stand- learning. They reveal, for example, that tutor-
ard. But in the latest cohort, Brydon could ing is one of the most cost-effective ways to
see that a shocking 30% of them were behind. help children to make up lost ground. And
And the damage ran even deeper. So many some countries are drawing on this evidence
children had behavioural or psychological in their COVID-19 responses, putting a focus
problems after lockdowns that some were on tutoring and other programmes that edu-
getting violent, and the school hired a full-time cational studies have shown to be effective. some transformative changes in education
psychologist to help. “The number of referrals But experts point to a number of concerns. — ones that both improve practices and reach
to her was astronomical,” Brydon says. The true extent of learning losses in the pan- more students, researchers say. “I do think it
Similar scenarios have played out in class- demic is not yet clear; educational research has thrown into the air many of the assump-
rooms around the world. By February this year, rarely provides simple answers about what to tions that we make about education,” says Lee
schools globally had been closed because of do; and nations might not use this opportunity Elliot Major, who studies social mobility at the
COVID-19 for an average of 4.5 months, affect- to make much-needed systemic change. “Every University of Exeter, UK.
ing an estimated 1.6  billion students and single time there’s been a calamity in the world,
creating what the United Nations has called the we’ve rushed back to the old normal fast,” says Tough sell
largest disruption to education in history. Even John Hattie, an educational researcher at the The concept of using research in education has
2 years into the pandemic, 48 countries had University of Melbourne. “The biggest travesty been a long, tough sell. “The fundamental issue
not yet fully reopened their schools, according of COVID is if we learn nothing.” is that many practitioners do not believe it will
to the UN cultural organization UNESCO. What’s more, the scale of the task ahead ever be a science,” says Andreas Schleicher,
The consequences of these closures follow is immense. Researchers and education who heads the directorate for education
a sad but predictable course. In rich countries, experts are concerned that the amounts being and skills at the Organisation for Economic
disadvantaged and vulnerable children have invested are laughably insufficient, given the Co-operation and Development (OECD) in
fallen behind the most. Those in poorer coun- number of students who need help. “It’s a real Paris. Teachers are not expected to browse
tries have been the hardest hit, and millions test for the global community,” says Kenneth academic journals, and educational policies
will never go back to school at all. UNESCO Russell, an education specialist at the UN chil- are often set by the ideology of bureaucrats
estimates that today’s generation of students dren’s charity UNICEF in New York. “And I don’t rather than by research showing what actually
could lose US$17 trillion in lifetime earnings at think the magnitude of the response matches works. “Many of them use evidence to confirm
current values because of missed learning and the magnitude of the need.” what they want to do,” Schleicher says.
skills. “We’re really talking about a generational Even so, the pandemic could eventually drive Some researchers and educators have been

608 | Nature | Vol 605 | 26 May 2022

©
2
0
2
2
S
p
r
i
n
g
e
r
N
a
t
u
r
e
L
i
m
i
t
e
d
.
A
l
l
r
i
g
h
t
s
r
e
s
e
r
v
e
d
.
 one, for changing the conversation about
evidence in education,” says Nancy Madden, a
psychologist and researcher at Johns Hopkins
University School of Education in Baltimore,
Maryland. “People want something that works,
they aren’t just doing business as usual.”

Dismantling dogma
The crown jewel at the EEF is its Teaching and
Learning Toolkit, which is based on system-
atic reviews and meta-analyses of studies,
such as randomized controlled trials, that
have tested 30 educational approaches. The
toolkit translates findings into an easy-to-un-
derstand metric: the number of months of
additional progress achieved over a year, on
average, by children who receive an inter-
vention, compared with similar children who
do not. It also displays the strength of the
underlying evidence and the intervention’s
cost (see ‘Which educational techniques get
top grades?’ and go.nature.com/3nbhdzm).
The toolkit dismantles many common beliefs
by showing that modest reductions in class size
(from 30 to 20 students, for example), wear-
ing school uniforms and grouping children
according to attainment level have little if any
effect, on the basis of the evidence so far. The
most effective strategies include ones that help
children to understand what they read; giving
them meaningful feedback; and approaches
that improve meta-cognition — the ability of
students to think about, plan and evaluate their
own learning. These each give children six or
seven months of progress, on average.
More than 70% of secondary-school leaders
in England now use the toolkit when making
decisions about how to spend funding. The
EEF has partnered with groups to adapt it for
use in Australia and parts of Latin America, the
Students in India in March; schools there were closed for months earlier in the pandemic. Middle East and Africa.
Long before the pandemic, it was clear that
trying to change that view for decades. They trials. The investment in the EEF “had a ripple one of the most cost-effective approaches
SAQIB MAJEED/SOPA IMAGES/LIGHTROCKET/GETTY

want education to operate more like medicine, effect around the world”, says Annette Boaz, is tutoring, either in small groups or one-to-
where a drug typically has to be proven effec- who studies evidence and policy at the London one. The toolkit says this can buy four to five
tive in randomized controlled trials before it’s School of Hygiene & Tropical Medicine. months of additional progress at relatively low
used. Advocates of evidence-informed educa- Other databases of educational research cost. And, unlike some other effective meth-
tion argue that teaching and learning methods have flowered, too. Hattie led an early, pio- ods, tutoring programmes can be ramped up
should also be shown to work by research — neering project to synthesize evidence from and implemented quickly. So, in 2020, the EEF
rather than being used because of tradition, around the globe on what influences learning1. rapidly reviewed evidence on the possible
opinion or the latest fad. But they acknowledge And, the US Department of Education’s Insti- impacts of the United Kingdom’s nationwide
that testing whether a method improves edu- tute of Education Sciences in Washington DC school closures2 and highlighted that tutoring
cational outcomes is often more complex than maintains the What Works Clearinghouse, a was likely to be a particularly effective way to
testing whether a drug improves health. source of information on educational pro- help children to catch up. At the time, “tutoring
In late 2010, evidence-informed education grammes that have been shown to be effective seemed such a plausible response”, says Becky
got one of its biggest boosts when the UK gov- through rigorous research. Hattie argues that Francis, an education researcher who is chief
ernment invested £125 million (US$156 million) with databases such as these, the field doesn’t executive of the EEF. The recommendation
to raise standards in schools. This gave rise to need more evidence — the challenge lies in get- “landed in a void at the time and was seized
the Education Endowment Foundation (EEF), ting the information used by governments and upon eagerly by policymakers”, she says.
a non-profit organization in London that has schools. “We’re hopeless at that,” he says. In June 2020, the UK government announced
since become a leader in educational research. The pandemic could, in theory, help to bridge a £350-million National Tutoring Programme
It has funded at least 160 randomized con- that gap. Countries worldwide want to know as part of its wider £1-billion catch-up funding
trolled trials in education, probably more than the best way to invest in educational recovery, for children. (The EEF was one of several part-
any other organization in the world. Around and billions of dollars are already pouring into ners that ran the programme for the first year;
half of English schools have taken part in these schools. “This moment in time really is a unique the Dutch company Randstad took it over in

Nature | Vol 605 | 26 May 2022 | 609

©
2
0
2
2
S
p
r
i
n
g
e
r
N
a
t
u
r
e
L
i
m
i
t
e
d
.
A
l
l
r
i
g
h
t
s
r
e
s
e
r
v
e
d
.
Feature
the second year.) But the tutoring programme to develop the training. still closed, the obvious top priority, say educa-
has been widely criticized for drastically fail- The United States, too, has put some tion specialists, is for classes to reopen so that
ing to reach enough children, in particular emphasis on evidence in its recovery plans. In children can return — even if COVID-19 cases
those who stand to gain most from it. “I think 2021, a giant stimulus bill channelled $122 bil- start rising again. Sachs-Israel says schools
it hasn’t targeted the most disadvantaged lion to schools. The law requires that at least have to be welcoming and safe, and need to
pupils properly. It hasn’t won over teachers,” 20% of funds received by districts must be used overcome any fears that parents, teachers
says Elliot Major. “And partly that’s because on evidence-based measures to help students’ and children might have about infection risks.
there’s some scepticism about variation in the academic, social and emotional needs. In prac- According to a 2020 report3 from an inter-
quality of the tutors.” tice, however, it’s hard to know how this money national group called the Global Education
This March, the government ended Rand- is being used, says Mike Petrilli, president of the Evidence Advisory Panel, one cost-effective
stad’s contract and announced that funding Thomas B. Fordham Institute, an educational approach for schools is to target teaching to
for tutoring would go directly to schools in the a child’s learning level rather than to their age.
2022–23 academic year. The National Founda- “It’s not just hiring some And education researchers say that schools
tion for Educational Research in Slough, UK, should assess each returning student.
is conducting independent evaluations of
people that call themselves This is the strategy behind an evidence-based
the tutoring programme’s impact on student tutors … you can waste a lot programme called Teaching at the Right Level,
attainment. of money that way.” run by the learning organization Pratham in
Both the EEF and Randstad say they are New Delhi. The organization’s chief executive,
proud of what they achieved with the tutor- Rukmini Banerji, says it is working with several
ing programme. In statements to Nature, the foundation in Washington DC. “Based on past state governments in India and other countries,
EEF said that 60% of secondary schools had experience, we should expect that much of the and has observed that children are making pro-
accessed tutoring by July 2021, and Randstad money will not be spent in the best way.” gress in basic literacy and numeracy in just a
said it had tripled the number of students in Another complication is that tutoring comes few weeks. “We feel that is what is really needed
the tutoring programme. in many styles: one-to-one or small groups; across the world,” she says.
online or in person; delivered by human teach-
Case studies ers or digital ones. There is no guarantee that a Questioning the evidence
Another evidence-backed programme has been particular programme will be effective, or that Even with all the support for use of evidence in
widely, and less controversially, put in place in it will be successful in a particular school or for education, there have been some long-standing
England. The Nuffield Early Language Interven- a certain child. “It’s not just hiring some people concerns about how reliable some of that evi-
tion (NELI) has been shown in randomized con- that call themselves tutors and putting them in dence is. In 2019, a pair of researchers exam-
trolled trials to boost language skills in children the room with some kids — you can waste a lot ined 141 large randomized controlled trials
aged 4–5 through a series of teaching sessions of money that way,” Madden says. commissioned by the EEF and the US-based
in small groups (see go.nature.com/39xtgsk). In Melbourne, Brydon saw the challenges National Center for Educational Evaluation
NELI is now being used in two-thirds of English of putting a tutoring programme in place. and Regional Assistance. They concluded that
primary schools to help make up for learning Her school was able to place an extra teacher 40% of the trials were uninformative because
missed during the pandemic, and its results in some classrooms to help children who have their effects were small or imprecise4. “So at
are being independently evaluated. “Although fallen behind, using money it received as part of the beginning, you didn’t know whether the
it’s had a tremendous reach, it’s flown almost a catch-up programme from the government. intervention works or not. But at the end, we’re
entirely under the radar,” says Francis. But the school is struggling to find teachers still unsure whether it works,” says study author
Some researchers point to the Nether- to fill positions, she says, because exhausted Hugo Lortie-Forgues, who studies mathemat-
lands as having taken an exemplary approach colleagues are quitting their jobs. “We need ics education at Loughborough University,
to education recovery based on evidence. upwards of ten substitute teachers every day UK. This could be because early, promising
There, the government handed €4.2 billion just to keep the school running,” she says. research on an approach turned out to be mis-
(US$4.4 billion) of funding to schools to sup- leading, a method was hard to scale up or the
port students, and required that they spend it Global problems trial was poorly designed, he says.
by picking from a ‘menu card’ of evidence-based Things are looking even grimmer elsewhere This was no big surprise to researchers who
approaches largely based on the EEF’s toolkit. in the world. UNESCO estimates that, by April conduct such studies. Just as most new drugs
“We want to make sure as much as possible that 2020, more than 1.2 billion children in the prove ineffective in large clinical trials, most
schools will base their decisions on knowledge highly populous Asia Pacific region had been bright ideas for improving learning show
that’s available on effective approaches,” says affected by school closures. And, whereas little effect when they are put to the test. And
Femke Bink, senior adviser in the Department schools closed in Japan and Singapore for whereas in medicine, physicians start with
for Secondary Education at the Ministry of only a month or so, those in Bangladesh and someone who is ill and try to make them meas-
Education, Culture and Science in The Hague. the Philippines have experienced some of the urably better, in education, many countries
And in Panama, where schools were fully worst disruptions in the world, with schools are starting with a fairly healthy education
closed for more than a year, the Ministry of fully shut for more than 13 months. system — so any new method is likely to pro-
Education in April launched resources and Even before COVID-19, there was a learning duce only marginal gains. “It’s perhaps a little
training for teachers showing how to imple- crisis in the region, Sachs-Israel says, because naive to assume that teachers haven’t discov-
ment evidence-based practices, including so many children did not achieve expected ered, over time, some of the approaches that
feedback to students. “Teachers are tired proficiency levels at school. An estimated are more likely to be successful,” says Steve
and stressed, so we’re trying to say to them, 10 million children in the Asia Pacific region Higgins at Durham University, UK, who has
‘we want to channel your efforts into what will not go back to school, and the expecta- led work on the EEF’s toolkit.
really works’,” says Javier González, director tion is that early or forced marriages and child With data still rolling in, there are some sug-
of SUMMA in Santiago, Chile. SUMMA aims to labour are expected to soar. gestions that school closures might have had a
improve education systems in Latin America The scale of this problem is not one that extra smaller impact on some children’s achievement
and the Caribbean using research, and helped tutoring alone can address. With many schools than many doom-laden headlines suggest — or

610 | Nature | Vol 605 | 26 May 2022

©
2
0
2
2
S
p
r
i
n
g
e
r
N
a
t
u
r
e
L
i
m
i
t
e
d
.
A
l
l
r
i
g
h
t
s
r
e
s
e
r
v
e
d
.
 that students might bounce back quickly. When WHICH EDUCATIONAL TECHNIQUES GET TOP GRADES?
Hattie examined the effects of school closures The Education Endowment Foundation, a UK charity, has systematically reviewed evidence supporting many
educational approaches. Its Teaching and Learning Toolkit for schools rates each technique on the basis of its cost,
in Victoria, Australia, where schools (includ-
how much it improves student achievement and the strength of evidence supporting it.
ing Brydon’s) had been closed for extended
periods, he concluded that it was surprising Impact
that learning trajectories had only marginally Evidence Better Worse
High impact, extensive Low impact, limited
decreased (see go.nature.com/3mtxucq). One Cost evidence, low cost evidence, high cost
possible reason is that some students work-
See below for definitions.
ing alone were able to be more efficient than
at school. Schleicher adds that technology
also became more accepted, teachers rallied Metacognition and Oral language Reading
self-regulation interventions comprehension strategies
to support children socially and emotionally,
Learning how to learn, such as Focus on speaking Improving understanding
and parents became more involved in their through planning and evaluation and verbal interaction of written text
children’s education. Looking at the overall
impacts of the pandemic on education, he says, 6 6 6
“the balance sheet has pluses and minuses”.
4 4 4
In the longer term, a key way to get research
2 2 2
used in education more routinely will be to
weave it into teachers’ training and contin- 0 0 0
uing professional development. One model
comes from Japan, where teachers have for
decades conducted ‘lesson study’. This is a
form of research in which they develop a goal —
to improve understanding of fractions, say Feedback Phonics One-to-one tutoring
— then write a detailed lesson plan, observe Providing meaningful Knowledge of the Intensive individual
information about a relationship between support for pupils
the lesson in action and discuss what they student’s performance written symbols and sounds
learnt. Schools draw on external research and
often consult an academic in the process. This 6 6 6
type of ongoing professional development is
4 4 4
unusual, says mathematics education special-
ist Toshiakira Fujii at Tokyo Gakugei Univer- 2 2 2

sity. Teachers develop a deep understanding 0 0 0

of teaching materials “but more importantly
they learn how to learn as a teacher”.
Other countries are starting to integrate
evidence into teacher training, too. The EEF
and SUMMA are working with the University Extending Within-class Reducing class size
of West Indies at Cave Hill, Barbados, to train school time attainment grouping Lowering the student:teacher
teachers in evidence-based practices such as Adding hours or Organizing students ratio; most studies looked at
days of learning by level in a class cuts of 8–10 students
giving effective feedback to students. And
starting this year, all 650 students enrolled in
6 6 6
the master’s in education at Harvard Graduate
School of Education in Cambridge, Massachu- 4 4 4

setts, will have to take a course on evidence, 2 2 2

says Carrie Conaway, who is a senior lecturer 0 0 0
there. “The idea is that we have a generation
of leaders who understand the value of this as
part of their decision-making,” she says.
Brydon says she was taught almost noth-
ing about using research evidence during
her training — “you get exposed to a couple Impact: Number of months of progress made over a year, on average, by children who received the intervention,
compared to similar children who did not.
of major theorists and then that’s really it”.
But she is now part of Q Project, an effort in Evidence: The robustness of the evidence, based on the number of studies supporting each intervention and their
rigour. 1 = at least 10 studies (very limited evidence); 2 = 11–24 (limited); 3 = 25–44 (moderate); 4 = 45–69 (extensive);
Australia to improve the use of evidence in
SOURCE: EEF

5 = 70 or more (very extensive).

schools. She thinks that the biggest barrier, Cost: Estimated cost of each intervention per student per year. 1 = less than £80 (US$99); 2 = up to £200;
however, is a lack of time. “We’re so swamped, 3 = up to £720; 4 = up to £1,200, 5 = more than £1,200 (see go.nature.com/3nbhdzm for full details).
and when you have to decide between getting
your year-12 essays marked or reading some she’d tell them that teaching is the greatest 1. Hattie, J. Visible Learning: A Synthesis of Over 800 Meta-
research evidence, I know which one I’m going job in the world. But now, for the first time, she Analyses Relating to Achievement (Routledge, 2008).
to choose every day of the week.” has a different response. “There are some parts 2. Education Endowment Foundation. Impact of School
Closures on the Attainment Gap: Rapid Evidence
Right now, Brydon and her colleagues are that I really love,” she says, “but other parts that Assessment (EEF, 2020).
still battling to help children to catch up, amid are making it really hard to do the job.” 3. Global Education Evidence Advisory Panel. Cost-Effective
Approaches to Improve Global Learning (World Bank,
simmering concerns that the next coronavirus
2020).
variant could shut schools all over again. When Helen Pearson is an editor at Nature in 4. Lortie-Forgues, H. & Inglis, M. Educ. Res. 8, 158–166
people used to ask Brydon about her work, London. (2019).

Nature | Vol 605 | 26 May 2022 | 611

©
2
0
2
2
S
p
r
i
n
g
e
r
N
a
t
u
r
e
L
i
m
i
t
e
d
.
A
l
l
r
i
g
h
t
s
r
e
s
e
r
v
e
d
.