Vol 20 No 3 March 2021

IJLTER.
ORG
International Journal
of
Learning, Teaching
And
Educational Research
p-ISSN: 1694-2493
e-ISSN: 1694-2116
Vol.20 No.3
International Journal of Learning, Teaching and Educational Research
(IJLTER)
Vol. 20, No. 3 (March 2021)
Print version: 1694-2493
Online version: 1694-2116
IJLTER
International Journal of Learning, Teaching and Educational Research (IJLTER)
Vol. 20, No. 3
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part
of the material is concerned, specifically those of translation, reprinting, re-use of illustrations,
broadcasting, reproduction by photocopying machines or similar means, and storage in data banks.
Society for Research and Knowledge Management

The International Journal of Learning, Teaching and Educational

Research is a peer-reviewed open-access journal which has been
established for the dissemination of state-of-the-art knowledge in the
fields of learning, teaching and educational research.
Aims and Objectives
The main objective of this journal is to provide a platform for educators,

teachers, trainers, academicians, scientists and researchers from over the
world to present the results of their research activities in the following
fields: innovative methodologies in learning, teaching and assessment;
multimedia in digital learning; e-learning; m-learning; e-education;
knowledge management; infrastructure support for online learning;
virtual learning environments; open education; ICT and education;
digital classrooms; blended learning; social networks and education; e-
tutoring: learning management systems; educational portals, classroom
management issues, educational case studies, etc.
Indexing and Abstracting

Research is indexed in Scopus since 2018. The Journal is also indexed in
Google Scholar and CNKI. All articles published in IJLTER are assigned
a unique DOI number.
Foreword
We are very happy to publish this issue of the International Journal of

Learning, Teaching and Educational Research.

Research is a peer-reviewed open-access journal committed to
publishing high-quality articles in the field of education. Submissions
may include full-length articles, case studies and innovative solutions to
problems faced by students, educators and directors of educational
organisations. To learn more about this journal, please visit the website
http://www.ijlter.org.
We are grateful to the editor-in-chief, members of the Editorial Board

and the reviewers for accepting only high quality articles in this issue.
We seize this opportunity to thank them for their great collaboration.
The Editorial Board is composed of renowned people from across the

world. Each paper is reviewed by at least two blind reviewers.
We will endeavour to ensure the reputation and quality of this journal

with this issue.
Editors of the March 2021 Issue

VOLUME 20 NUMBER 3 March 2021
Table of Contents
Effective Social Studies Pedagogy: Effect of Simulation Games and Brainstorming Strategies on Students
Learning Outcome .................................................................................................................................................................. 1
Sunday Obro, Clifford Edore Ogheneakoke, Williams P. Akpochafo
Innovative Teaching: A Qualitative Review of Flipped Classrooms ............................................................................. 18

Kevin Fuchs
Elementary Teachers’ Thoughts about Distance Education and Learning 21st-Century Skills during COVID
Pandemic................................................................................................................................................................................ 33
Amani K H. Alghamdi, Naimah Ahmad Al-Ghamdi
English as Foreign Language Teaching in High Schools: A Chilean Case Study ......................................................... 51
Andrew Philominraj, Ranjeeva Ranjan, Rodrigo Arrellano Saavedra, Claudio Andrés Cerón Urzúa
The Practicum in Times of Covid-19: Knowledge Developed by Future Physical Education Teachers in Virtual
Modality ................................................................................................................................................................................. 68
Alejandro Almonacid-Fierro, Ricardo Souza de Carvalho, Franklin Castillo-Retamal, Manuel Almonacid Fierro
Challenges to Online Engineering Education during the Covid-19 Pandemic in Eastern Visayas, Philippines...... 84
Perante Wenceslao, Gomba Felisa
Success Indicators of Mathematical Problem-Solving Performance Among Malaysian Matriculation Students .... 97
Suriati Abu Bakar, Nur Raidah Salim, Ahmad Fauzi Mohd Ayub, Kathiresan Gopal
What do Malaysian ESL Teachers Think About Flipped Classroom? ......................................................................... 117
Ngo Hui Kiang, Melor Md Yunus
Initial Teacher Education of Primary English and CLIL Teachers: An Analysis of the Training Curricula in the
Universities of the Madrid Autonomous Community (Spain) ..................................................................................... 132
Alfonso López-Hernández
The Attitudes of Pupils towards using Flipgrid in Learning English Speaking Skills .............................................. 151
Joan Lim Ker Shin, Melor Md Yunus
Effect of Accounting Lecturer Behavior on the Level of Online Learning Outcomes Achievement ....................... 169
Alwan Sri Kustono, Wahyu Agus Winarno, Ardhya Yudistira Adi Nanggala
Challenges of Virtual Education during the COVID-19 Pandemic: Experiences of Mexican University Professors
and Students ........................................................................................................................................................................ 188
Claudia Patricia Contreras, David Picazo, Aixchel Cordero-Hidalgo, Paola Margarita Chaparro-Medina
Advancing Preservice Physics Teachers’ Critical Thinking through Active and Passive Microteaching Lesson
Study..................................................................................................................................................................................... 205
Billy A. Danday
Ecuadorian Teachers’ Perceptions of Teaching English: Challenges in the Public Education Sector ...................... 229
Agnes Orosz, Mirdelio Monzón, Paola Velasco
Constructivist Learning Amid the COVID-19 Pandemic: Investigating Students’ Perceptions of Biology Self-
Learning Modules ............................................................................................................................................................... 250
Aaron A. Funa, Frederick T. Talaue
The COVID-19 Pandemic and the Challenges of E-Assessment of Calculus Courses in Higher Education: A Case
Study in Saudi Arabia ........................................................................................................................................................ 265
Fatima Azmi, Heba Bakr Khoshaim
The Effects of Physical, Emotional, and Cognitive Demands on Academic Leaders’ Performance in Malaysian
Research Universities ......................................................................................................................................................... 282
Mayadah Graizi, Kenny S. L. Cheah, Kazi Enamul Hoque
1

Vol. 20, No. 3, pp. 1-17, March 2021
https://doi.org/10.26803/ijlter.20.3.1
Effective Social Studies Pedagogy:

Effect of Simulation Games and Brainstorming
Strategies on Students’ Learning Outcome
Sunday Obro, Clifford Edore Ogheneakoke and Williams P. Akpochafo

Delta State University, Nigeria
https://orcid.org/0000-0002-7675-9166
https://orcid.org/0000-0001-8545-6274
https://orcid.org/0000-0002-8632-3323
Abstract. This study explored the effects of game simulation and

brainstorming pedagogy strategies on students’ learning outcomes. In
addition, the study compared the effects of three strategies on students’
learning outcomes. This study tested pedagogical strategies for effects
on learning outcomes. The quasi-experimental research involved 180
students. The results reliably indicate that a simulation-game
pedagogical strategy boosted students’ learning outcomes, while a
brainstorming pedagogical strategy was effective on students’ learning
outcomes. When equated with brainstorming and the lecture strategies
in enhancing students’ learning outcomes, the superiority of the
simulation-game pedagogical strategy was also observed. These
findings indicate that innovative and student-centred pedagogical
strategies such as simulation-game and brainstorming strategies
improve students’ learning outcomes. In-service teachers should be
appropriately trained through seminars and conferences on modern
pedagogical strategies such as brainstorming and simulation games for
better Social Studies pedagogical strategies.
Keywords: brainstorming; pedagogy; Social Studies; simulation game;

students’ learning outcomes
1. Introduction
In the face of challenging experiences of the present time, one needs to advance
and create diverse and innovative solutions for what seems like a problematic
life task. In Nigeria, Social Studies is a mandatory discipline at the Basic
Education level. Irrespective of the ethnic, racial and cultural differences, Social
Studies is seen as a tool for building and creating a robust Nigerian nation. With
recent modifications or changes in the Upper Basic Social Studies curriculum,
what still dominates Social Studies classrooms is the lecture strategy with no
quality or lasting learning outcomes (Essien et al., 2015). Teachers generally rely
©Authors
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0
International License (CC BY-NC-ND 4.0).
2
on the lecture strategy for imparting Social Studies knowledge and skills. This
lack of an inventive, creative teaching strategy and critical progression in Social
Studies lessons and classrooms can be linked to teachers’ disinclination to learn
and use novel and creative teaching strategies. Also, the dictate of high-stakes
experimentation and execution involves students’ churning out factoids to grasp
the content. Thus, there is a lack of real motivation for teachers or students to
learn more than a particular fact. Any challenging of the subject matter is
regarded as unnecessary and unimportant. According to Wood (2011), the
typical Social Studies classroom inhibits critical and creative thinking and
strengthens the idea that facts and information are unchangeable and not free for
criticism or interpretation. Students stuck in this type of Social Studies classroom
or environment quickly discover that they are bored and helpless, having been
taught from the standpoint that Social Studies is an assemblage of useless
inconsequential knowledge.
Simulation games challenge students’ understanding to comprehend difficult

tasks by means other than the traditional lecture strategy. In other words, they
have the potential for helping students achieve more than the conventional
teaching method (Nja et al., 2019). On the other hand, a brainstorming strategy
enhances the productivity of ideas and explores solutions. It helps learners to
discover better solutions or answers to problems (Malkawi & Smadi, 2018).
In the expository strategy, learners are passive and collect information that can
be reclaimed when the teacher requires it from them (Tarman & Kuran, 2015).
However, evidence has shown that knowledge gained through an active
discussion strategy is generally retained better than knowledge gained through
a lecture strategy. Furthermore, according to Jack and Kyado (2017), students
more often favour active participation in discussion than being inactive or
passive in a lecture. Through meaningful or fruitful learning, Social Studies
learning need no longer be a matter of memorisation facts and principles that
cannot be applied to novel problem-solving situations (Shear, 2016). Students are
given the means and the opportunity to participate actively in the teaching and
learning situation in activity-based learning, unlike in the conventional
pedagogical strategy.
Study outcomes of other research have shown that brainstorming and

simulation-game pedagogical strategies aided students' learning outcomes.
Customarily, students' inculcation of knowledge using a lecture strategy is
inefficient to grasp curriculum contents. There are various Social Studies
pedagogical strategies. Some of these strategies are thought to be more valuable
and successful than others. The question is which strategy is more productive
and beneficial for Social Studies teaching, specifically at the Upper Basic level.
Hence, this study compares and establishes the effect of the selected strategies
(simulation games and brainstorming strategies) on Social Studies students’
learning outcome. This study is intended to find the answers to the following
propositions: a simulation-games pedagogical strategy will not enhance
students’ learning outcomes; students instructed by means of a brainstorming
pedagogical strategy may not increase their learning outcomes; and differences
http://ijlter.org/index.php/ijlter
3
will not be found among brainstorming, simulation-games, and lecture

pedagogical strategies on Social Studies students’ learning outcomes.
The hypotheses listed below were formulated and tested:

i) Students’ involvement in simulation-game strategies will not improve their
learning outcomes.
ii) Students’ involvement in brainstorming strategies will not improve their
learning outcomes.
iii)There is no statistically significant difference among brainstorming,
simulation-games and lecture pedagogical strategies regarding students’
learning outcomes.
1.1 Practical gaps

The study could provide teachers with the desired information to design and
adopt the right teaching strategies to suit varied learners and enhance students’
learning outcomes in Social Studies. Similarly, the study could provide
researchers in Social Studies areas with future research in instructional
strategies. It could also help authors and publishers in their presentation of
content to readers. Finally, the results obtained may lead to further research in
other subject areas.
2. Literature Review
2.1 Simulation-Game Pedagogical Strategy
As indicated by Mozelius et al. (2017), a simulation game denotes a board game,
or those various teacher-made games for teaching and learning purposes. The
focus of these pedagogical games is usually on the socioeconomic, religious,
political and aspects of society. A variety of games is accessible which cover
substantial areas of the Social Studies programme. The varieties include, among
others, chess, tug of war, Diplomacy, Monopoly, Risk, Die Macher, Scrabble, and
Hacienda, among others (Nja et al., 2019). Ochoyi (2018) opined that simulation-
assisted learning merges distinctive characteristics that make it appropriate to
situations where the emphasis is on interactive or cooperative learning. It
produces excitement, enhances learning and almost mirrors the real-life world.
They refer to simulation games as contrived or artificial activities which match
some facet of reality. A simulation game is a representation of a real social or
physical situation reduced to manageable sizes to serve a particular function or
purpose. It is any environment or game among challengers functioning under
rules towards achieving a goal such as winning, or a victory. It has two
features, namely overt rivalry or competition, and rules.
Simulation games give students the understanding to comprehend difficult

tasks. In other words, they have the potential for helping students achieve more
than they would by means of the conventional teaching method (Nja et al.,
2019). On the other hand, a brainstorming strategy enhances the productivity of
ideas and explores solutions. It helps learners to discover better solutions or
answers to problems (Malkawi & Smadi, 2018).
4
According to Ezeudu and Ezinwanne (2013), a simulation game is often referred

to as an activity that is based, partially or wholly, on the judgment and
decisions of the players. It is an excellent enhancement to the standard
traditional lecture. It is a didactic tool whereby students learn through the
application of decision-making and theories to a simulated real-life scenario
(Folta, 2010). Antunes et al. (2012) contend that simulation games are applied in
the teaching-learning process because they provide fascinating challenges to
learners and add interest, activity and novelty to the lesson. According to
Mozelius et al. (2017), it increases their enthusiasm and leads to learning
outcomes.
However, despite the several positive educational benefits or advantages, it still

has some shortcomings. Guy and Lownes-Jackson (2015) pointed out that
teachers using the simulation game strategy must be aware that it takes much
time, and students are likely to be very noisy, unmanageable and might at times
be uncontrollable. Students should therefore be organised and educated on how
to behave during simulation activities. Simulation games demand adequate
preparation and coordination from both the teacher and the institution.
2.2 Brainstorming as a Pedagogical Strategy

Hashempour et al. (2015) described the brainstorming pedagogical strategy as
a group or individual creative strategy by which members attempt to solve a
particular problem or issue by collecting a list of instinctive ideas contributed by
its member(s). Ashammari (2015) asserted that it is called brainstorming because
it involves a situation where individuals generate as many fresh ideas as
possible around a particular concern or problem using guidelines which
eliminate shyness and produce creative thinking and novel solutions and ideas.
In that way they, they come up with several new ideas and answers. The
participants or contributors shout out ideas as they come to mind and
subsequently build on the ideas suggested by others. All views or opinions are
recorded but not criticised or evaluated. It is only when the brainstorming
session is over that students’ ideas are evaluated. Rowan (2014) defined
brainstorming as a creative individual or a groupactivity in which efforts are
made to ascertain a definite solution or conclusion for a particular problem by
gathering facts in the form of a list of opinions and ideas instinctively
contributed by the members. When used as a Social Studies strategy for
teaching, particularly in introducing and laying out new facts and ideas, a
brainstorming pedagogical strategy is simple, less complicated, and very useful.
Jack and Kyado (2017) opined that a brainstorming strategy is an instructional

strategy that emphasises students’ participation, dialogue, input and two-way
interaction. The fundamental purpose of a brainstorming pedagogical strategy is
to build and boost communication andinteraction skills, enhance thinking and
decision-making or judgement skills, and simultaneously foster different
opinions. It is effective for the reason that it stimulates the students’ background
knowledge and raises their interest. The teacher can ascertain whether the
students have sufficient background experience and knowledge to go ahead
5
with the study during the learning process (Rashtchi & Beiki, 2015).
Brainstorming can be utilised in all relevant facets of learning.
However, in spite of all the usefulness and qualities of a brainstorming strategy,

weaknesses or inadequacies abound: the brainstorming strategy has its
shortcomings. The major shortcoming is that brainstorming groups’
contributions and productivity may be inhibited by fear of critical evaluation.
Students may desire to follow the prevalent practice and pattern of idea
generation. For Owo et al. (2016), brainstorming is generally not appropriate at
the primary school level because of the level of reasoning required to work out
and achieve known objectives. Simultaneously, the teacher must be equipped to
guide and be of assistance as necessary, bearing in mind the class environment
as such considerations often determine the outcomes. As mentioned earlier,
these barriers can lower levels of enthusiasm and effort when individuals work
cooperatively in the classroom. When these barriers are present, the individual
gives up on the group, and interaction and cooperation are reduced. Moreover,
the kind and amount of time dedicated to assigned tasks may also affect
individuals’ decisions to leave the group (Owo et al., (2016).
2.3 Studies Related to Simulation Games and Students’ Learning Outcome

Balasubramanian and Brent (2010) explored the challenges and opportunities
offered by simulation and games to improve learning with students’ ethnicity
and gender as factors. They found that students from all groups exhibited
significant learning outcomes through the employment of simulation and
games. However, Hsu et al. (2011) found no effect of games on students’
learning outcomes. Ezeudu and Ezinwanne (2013) examined the effects of
simulation games on chemistry students’ learning outcomes. They deduced that
simulation games brought about improved performance in mathematics. Ahmad
et al. (2013) examined the computer games’ effects on students’ interest and
achievement in geometry. The study found that students taught using games did
better than those instructed by means of the traditional strategy. The study of
Beuk (2015) looked into the effect of sales’ simulation games on students’
learning. The research established that the academic learning of those students
who were exposed to simulation games improved.. This result means that
students subjected to games teaching methods did better than students in the
control condition. Carenys and Moya (2016) explored students’ learning
outcomes in digital game-based business and accounting education. The study
found that digital game-based pedagogical strategies significantly improved
students’ learning in business and accounting education.
2.4 Studies Related to Brainstorming Strategy and Students’ Learning

Outcome
Owo et al. (2016) examined whether the brainstorming method would improve
students’ knowledge in chemistry. The study established that the brainstorming
method did not improve their performance. In addition, Hashempour et al.
(2015) examined the usefulness of a brainstorming strategy on students’
learning. The study found that a brainstorming strategy failed to enhance
students’ learning outcomes. Also, students did not differ due to gender. Owo
6
et al. (2016) explored the brainstorming strategy efficacy on students’ previous

knowledge and learning outcomes in chemistry. Their finding proved that the
brainstorming strategy did not boost students’ learning outcomes. However,
Jack and Kyado (2017) concluded that a brainstorming pedagogical strategy
resulted in students’ better learning outcomes in electro-chemistry. In addition,
Malkawi and Smadi (2018) examined the effect of the brainstorming method on
students’ learning outcomes in English grammar in Jordan. The study concluded
that the brainstorming pedagogy method improved students’ learning
outcomes.
3. Theoretical Background/Framework
The present study is grounded on Albert Bandura’s (1999) social cognitive
learning. The theory accentuates the importance of observation and modelling in
the actions, attitude, and emotional reactions or responses of others. Therefore, it
centres on learning by modelling and observation. Social cognitive learning
theory explains how both cognitive and environmental factors interact to affect
human learning and conduct. Its emphasis is on learning within a social setting
or framework. As indicated by Bandura (1999), people learn from each other’s
ideas through observational learning, imitation, and modelling. This theory is
pertinent to this study because its propositions are traditionally considered
necessary ingredients required for activity-based teaching.
The theory provided the theoretical foundation for organising simulation-game

environments and developing brainstorming, which can be utilised as practical
teaching strategies. The theory has shown why teachers must promote the most
creative and helpful strategies by moving away from ineffective practices and
moving towards more supportive learning strategies for all students. The theory
is robustly associated with this study because students will boost their learning
outcomes as they actively build their knowledge through imitation, modelling,
observation, and interactions with different simulation games and
brainstorming exercises that Social Studies teachers provide. Furthermore, the
social cognitive theory is vital to this study because it is used to examine this
study's cause and effect. It is additionally suitable and relevant to highlight and
relate it to pedagogical strategies (simulation games and brainstorming) on
learning outcomes.
Simulation
Games
Social Studies
Learning Outcome
Brainstorming
Test (SSLOT)
Control
Condition
Figure 1: The conceptual framework
7
4. Methodology
4.1 Study Design
The research design was quasi-experimental. The design encompassed three
groups, namely two experimental groups (EGs) and one control group (CG). The
pedagogical strategies include simulation games (SGs) and brainstorming as
treatment or intervention, and the traditional lecture strategy was utilised for the
control group. The study design signifies the following:
Table 1: Research blueprint

Groups Pretest Treatment Posttest
Experimental Group 1 O1 X O2
Experimental Group 1 O3 X O4
Control Group O5 O6
The codes are explained as follows:

O1 = Pretest measurement for Experimental Group I
X1 = Treatment for Experimental Group I (Simulation games)
O2 = Posttest extent for Experimental Group I
O3 = Pretest extent for Experimental Group II
X2 = Treatment for Experimental Group II (Brainstorming)
O4 = Posttest measurement for Experimental Group II
O5 = Pretest measurement for Control Group
O6 = Posttest measurement for Control Group (Rogers & Révész, 2020).
As detailed previously, an intervention was introduced to the two experimental

groups, while there was no such intervention or no difference in treatment with
the control group. Accordingly, changes observed in the posttest learning
outcomes were ascribed to the intervention or treatment effect.
4.2 Participants
Participants are all Upper Basic Education 2 students of the public schools in
Delta and Edo States, Nigeria. The study sample consisted of 180 Basic 2 (Upper)
students who constitute 0.22% of the total population as the study was an
experimental study. The multistage sampling method at four levels through the
balloting method was utilised to select the study sample. The first level of
sampling was the senatorial districts which were used as the sampling units. For
the second level of sampling, a local government area was randomly selected. In
the third level of sampling, a school was chosen from the local government areas
by means of a balloting method. The judgemental approach was employed in
selecting all the students from the six (6) schools. Furthermore, a class of Upper
Basic level eight was sampled as the fourth sampling level from each school.
All the students in that class from the six (6) government secondary schools were
the experimental study subjects. In selecting the schools for the study, only
mixed schools were considered as appropriate for the research as gender was a
variable that was investigated. The ballot method was used to assign these
schools to either the experimental or control groups. The schools selected were
sufficiently far off from each other, and no school had double treatment to
prevent interference. Specifically, the topics were not taught at any school before
8
the start of the experiment. The classes used in the research were carefully
chosen using a die.
4.3 Instrument
The instrument employed for the study was a test instrument titled “Social
Studies Learning Outcome Test” (SSLOT) (see Appendix 1). The SSLOT
contained fifty (50) items (multiple choice) which were Social Studies topics
taught in Upper Basic level 2 during the period of experimentation. The test
items were spread to cover the following topics: Drugs abuse, Harmful
substances and Drug trafficking. In constructing the test, a specification table
was worked out. It was a two-dimensional table showing the test objectives and
the content to be tested. In drawing up the SSLOT, the researcher took
cognisance of the taxonomy of objectives in the cognitive domain using three
cognitive reasoning skills: Remembering, Understanding and Thinking (RUT).
The items were shared around the three levels of Remembering (25%),
Understanding (50) and Thinking (25), all totalling 100%.
Table 2: Test blueprint for Social Studies learning outcomes (SSLOT)

REMEMBERING 25%
UNDERSTANDING
THINKING 25 %
SYLLABUS SECTIONS
50%
% TOTAL
Drug abuse 28% 3 6 3 12
Drug trafficking 32% 4 9 4 17
Harmful substances 40% 6 10 5 21
Total 100% 13 25 12 50
Instrument reliability was confirmed by using 30 students in a test-retest

process. Using the Pearson coefficient (r), a value of 0.74 was obtained. Thus, the
instrument was deemed reliable and therefore suitable for the study.
4.4 Research Procedure

The experiment took six (6) weeks. Students were assigned after selection to
intervention groups and control conditions. The SSLOT was administered as a
pretest. This was followed by the intervention (experimentation) through the
subject matter or content instruction as presented in the curriculum using the
selected instructional conditionstrategies. Two teachers were employed to
conduct the experiment or treatment (that is, two treatment administrators).
The students selected for the experiment (experimental group) were taught
Social Studies content three days per week with each lesson lasting 40 minutes
per period, making a total of 120 minutes a week. Students were pretested with
the SSLOT to establish their learning outcomes level prior to experimentation.
9
After six weeks of experimentation or treatment, a posttest was administered to

determine students' learning outcome levels.
The control groups were instructed using only the lecture strategy and both a
pretest and posttest were administered. The teaching and test administration
were done simultaneously in the six schools.
5. Results
RQ 1
Will a simulation-game pedagogical strategy lead to enhanced students’ learning
outcomes?
Table 3: Students’ pretest and posttest learning outcome scores of simulation-game

pedagogical strategy
Pretest Posttest Learning
Strategy/ Standard Standard outcome
Treatment No Mean Deviation No Mean Deviation Gain
Simulation game 58 53.34 12.50 58 70.78 11.85 17.44
Table 3 shows that students taught using a simulation-game pedagogical

strategy had a mean score of 53.34 in the pretest and a mean score of 70.78 in the
posttest, making a pretest-posttest learning outcome gain to be 17.44. The result
proved that students instructed using a simulation-game pedagogical strategy
had a better learning outcome in the posttest than in the pretest.
Ho1
Students’ involvement in simulation-game strategies will not improve their
learning outcomes.
Table 4: Results of ANCOVA of learning outcome test according to simulation game

strategy
Sum of Mean
Source Square DF Squares F value Sig. of F
Corrected/Adjusted
5476.124 1 5476.124 35.254 .000
Model
Intercept 682325.124 1 682325.124 4320.159 .000
Simulation 5476.124 1 5476.124 35.254 .000
Error 28558.197 178 158.893
Total 755678.001 180
Corrected/Adjusted
33834.121 179
Total
Table 4 shows that students involved in a simulation-game pedagogical strategy

had improved learning outcomes. Data in the table revealed that simulation
games' effect on students' learning outcomes was significant (F (1,178) = 35.254,
p = 0.000). Consequently, the hypothesis that students’ involvement in
simulation-game strategies will not improve their learning outcomes was
rejected.
10
RQ 2
Will students instructed by means of a brainstorming pedagogical strategy
improve their learning outcomes?
Table 5: Students’ retest and posttest learning outcome scores of brainstorming

strategy
Pretest Posttest Learning
Strategy/ Standard Standard outcome
Treatment No Mean Deviation No Mean Deviation Gain
Brainstorming 64 50.77 12.28 64 64.46 13.14 13.69
Table 5 shows that students instructed using brainstorming had a mean score of
50.77 and a standard deviation of 12.28 in the pretest and a mean score of 64.46
and standard deviation of 13.14 in the posttest, making a pretest-posttest
learning outcome gain to be 13.69. The result showed that students instructed
using a brainstorming pedagogical strategy had better learning outcomes in the
posttest than in the pretest.
Ho2
Students’ involvement in brainstorming conditions will not improve their
learning outcomes.
Table 6: Results of ANCOVA of learning outcomes test according to brainstorming

strategy
Sum of Mean Sig. of
Source Square DF Squares F value F
Corrected/Adjusted
115.824 1 115.824 .610 .043
Model
Intercept 630084.358 1 630084.358 3316.401 .000
Brainstorming 115.824 1 115.824 .610 .043
Error 33818.287 178 189.990
Total 754668.000 180
Corrected/Adjusted
33935.112 179
Total
As shown in Table 6, students involved in the brainstorming conditions

improved their learning outcomes. This is confirmed by the value (F (1,178)
=.610, p=0.043). This demonstrates that the brainstorming strategy boosted
students’ learning outcomes. The implication is that the effect on students’
learning outcomes due to treatment or teaching strategy was significant. Thus,
the hypothesis was rejected.
RQ 3
Will there be a difference amongst brainstorming, simulation games, and lecture
strategies on students' learning outcomes in Social Studies?
11
Table 7: Pretest/Posttest of students’ learning outcomes by strategies

Strategies Pretest Posttest Learning
Standard Standard outcome
No Mean Deviation No Mean Deviation Gain
Simulation game 58 53.34 12.50 58 70.78 11.85 17.44
Brainstorming 64 50.77 12.28 64 64.46 13.14 13.69
Lecture strategy
(control) 58 45.60 12.09 58 55.37 10.91 9.77
Total 180 49.90 12.29 180 63.54 11.97 13.63
Table 7 shows that at pretest, students' mean score when exposed to simulation
games was 53.34, which was better than the pre-test total mean of 49.90.
Brainstorming had a mean score of 50.77, which was also better than the pretest
total mean (49.90), while the lecture method pretest mean score was 45.60 which
was lower than the total mean of 49.90. However, at the posttest, the simulation
games had a mean score of 70.78, which was better than the total mean of 63.54
and a learning outcome gain of 17.44 which was better than the grand mean gain
of 13.63. The brainstorming strategy mean score at the post-test was 64.46, which
was also better than the total mean of 63.54 while the learning outcome gain of
13.69 was slightly better than the learning outcome gain of 13.63.
Simultaneously, the control groups had an overall mean score of 55.37 that was
less than the total mean of 63.54 and a learning outcome gain of 9.77, which was
lower than the total learning outcome gain. Thus, the table's results indicate that
students exposed to brainstorming and simulation strategies attained a better
score than the control group. In effect, simulation games proved to be superior
to both brainstorming and lecture strategies in enhancing students' learning
outcomes. On the other hand, the brainstorming strategy proved to be better
than the lecture strategy in improving students' learning outcomes.
Ho3
There is no statistically significant difference among simulation game,
brainstorming and lecture strategies on students’ learning outcomes.
Table 8: ANCOVA summary of the posttest according to strategies

Sum of Mean
Source Square DF Squares F value Sig. of F
Corrected/Adjusted model 8509.920 2 4254.960 59.245 .000
Intercept 717131.541 1 717131.541 4992.579 .000
Strategies 8509.920 2 4254.960 59.245 .000
Error 25424.191 177 143.639
Total 754669.010 180
Corrected/Adjusted Total 33935.112 179
Table 8 displays a statistically significant difference among simulation-game,

brainstorming and lecture (control) pedagogical methods on learning outcomes
of students’ (F (1, 177) = 59.245, p= .000). This implies that the hypothesis stating
there is no significant statistical difference among simulation-game,
brainstorming and lecture pedagogical methods on students’ learning outcomes
12
was rejected. To prove the difference among the groups, Scheffe’s posthoc was
used. The outcome is presented in Table 9.
Table 9: Scheffe’s posthoc analysis by strategies

Subset
STRATEGIES No 1 2 3
Control 59 54.2667
Brainstorming 64 64.4561
Simulation 57 70.7937
Sig. 1.000 1.000 1.000
Table 9 indicates that significant differences existed between the posttest mean
scores among the different groups. According to the results of Scheffe’s posthoc
analysis, there is a significant difference among the groups of simulation (70.79),
brainstorming (64.45) and control (54.27). From the result, the simulation
pedagogical strategy was superior to both the brainstorming pedagogical
strategy and the lecture pedagogical strategy as it obtained the highest mean
score. However, brainstorming also proved to increase learning outcomes more
significantly than the lecture strategy did.
The posthoc scores proved that the experimental groups differ significantly from
the control or lecture group. These pairs contributed to the observed significant
differences among the three strategies on students’ learning outcomes. Thus, the
hypothesis which stated there is no significant statistical difference among the
three pedagogical strategies or methods on students’ learning outcomes was
rejected.
6. Discussion
Simulation games and brainstorming pedagogical strategies have been proved
to increase and boost learning outcomes more significantly than the lecture
strategy. Students instructed using a simulation-game pedagogical strategy
improved more than students tutored by means of the lecture strategy.
Similarly, students taught with a brainstorming pedagogical strategy had
significantly better learning outcomes than those instructed using the lecture
pedagogical strategy. The result supports the views of Balasubramanian and
Brent (2010), Ezeudu and Ezinwanne (2013), Ahmad et al. (2013), Beuk (2015),
Rashtchi and Beiki (2015), Owo et al. (2016) and Dankbaar et al. (2016), who had
earlier testified that students instructed using a simulation-game pedagogical
strategy demonstrated better learning outcomes than those students instructed
by means of the lecture strategy. However, this finding is in contrast with that
of Hsu et al. (2011) who proved that simulation games did not improve students'
learning outcomes. Furthrmore, this study’s results also confirmed the findings
of Mehr et al. (2016) and Jack and Kyado (2017), namely that the use of a
brainstorming pedagogical strategy enhanced students’ learning outcomes more
than the lecture strategy did. However, this finding disagreed with those of
Hashempour et al. (2015) and Owo et al. (2016).
A simulation game pedagogical strategy is significantly superior to

brainstorming in enhancing students’ learning outcomes. This superiority of a
13
simulation-game pedagogical strategy in improving students’ learning outcomes

is consistent with the views of Kikot et al. (2013), Ranchhod et al. (2014), Lu et al.
(2014), and Carenys and Moya (2016). They reported that a simulation-game
pedagogical strategy motivates and enhances students’ learning outcomes.
The limitations of students’ learning outcomes are that they may be given
greater importance than they deserve. They may be treated as sacrosanct,
whereas learning outcomes are merely the end product of a value judgement on
the teachers’ part. It may lead to turning out students who are undoubtedly
well-trained in particular areas but are inadequate in a broad range of skills,
desirable attitudes and abilities associated with a comprehensive education.
7. Study Limitations
The study was conducted using Social Studies teachers; however, their
personalities, experience and attitudes were not considered, which may have
affected the study results. The content used was also limited to what is in the
school syllabus. It is believed that the application of more units of instruction
might make for a better generalisation of the study results.
8. Conclusions
The aims of the study were established. This study proved the effectiveness of
simulation games and brainstorming pedagogical strategies as well as the
superiority of simulation game strategy to brainstorming and the lecture
strategies in enhancing students’ learning outcomes. It was concluded in the
study that if Social Studies teachers embrace simulation games, students will
achieve better Social Studies learning outcomes. Thus, rather than limiting
students at the upper basic education level to conventional pedagogical strategy,
introducing modern pedagogical strategies for teaching such as simulation-
game and pedagogical brainstorming strategies will help students improve their
learning outcomes.
The study could provide teachers with the desired information to design and
adopt the right teaching strategies to suit varied learners and enhance students’
learning outcomes in Social Studies. Similarly, the study could provide Social
Studies researchers with areas for future research in instructional strategies. It
could also help authors and publishers in their presentation of content to
readers.
9. Implication of the study

This study’s observable implication rests on confirming that activity-based
strategies such as simulation games and brainstorming are superior to the
lecture strategy in boosting students’ learning outcomes. Teachers could attract
and sustain students’ interest and make learning permanent through the use of
these strategies. The strategies include numerous activities that will encourage
and enable both male and female students from different environments and
experiences to assimilate and internalise Social Studies skills and knowledge
effectively.
14
10. Recommendations
i) In-service teachers should be appropriately trained through seminars and
conferences on modern pedagogical strategies such as brainstorming and
simulation games for better Social Studies pedagogical strategies.
ii) Educational institutions charged with training teachers responsiblyshould
restructure the methodology course to include simulation games and
brainstorming pedagogical strategies. This will ensure that Social Studies
teachers are effectively trained in employing these Social Studies teaching
strategies. iii) Social Studies textbook writers should include explicit instructions
and illustrations in their textbooks for applying these strategies to enable
teachers to utilise in teaching.
Acknowledgements
The authors would like to thank the participants in this study.
11. References
Ahmad, S. M. S., Fauzi, N. F. M., Hashim, A. A., & Zainon, W. M. N. W. (2013). A study
on the effectiveness of computer games in teaching and learning. Journal of
Advanced Studies in Computers, Science and Engineering, 2(1), 1-8.
https://www.academia.edu.
/8177173/A_Study_on_the_Effectiveness_of_Computer_Games_in_Teaching_and
_Learning_1
Antunes, M., Pacheco, M. A. R., & Giovanela, M. (2012). Design and implementation of
an educational game for teaching chemistry in hgher education. Journal of Chemical
Educational, 89, 577–521. https://doi.org/10.1021/ed2003077
Ashammari, M. K. (2015). Effective brainstorming in teaching social studies for
elementary schools in Saudi Arabia. Journal of Education & Social Policy, 2(3), 70–75.
http://jes pnet.com/journals/Vol_2_ No_3_September_ 2015/8. pdf
Balasubramanian, N., & Brent, G. W. (2010). Games and simulations. Instructional
Technology Forum (ITFORUM). http://itech1.coe.uga.edu/itforum/paper73/paper
73.html
Bandura, A. (1999). Social cognitive theory: An agentic perspec tive. Asian
Journal of Social Psychology, 2, 21-41. https://doi.org/10.1111/1467-839X.00 024
Beuk, F. (2015). Sales simulation games student and instructor perceptions. Journal of
Marketing Education, 38(3), 170-182. https://doi.org/10.1177/0273475315604686
Carenys, J., & Moya, S. (2016). Digital game based learning in accounting and business
education. Accounting Education, 25(6), 598-651.
https://doi.org/10.1080/09639284.2016. 1241951
Dankbaar, M. E. W., Alsma, J., Jansen, E. E. H., Van Merrienboer, J. J. G., Van Saase, J. L.
C. M., & Schuit, S. C. E. (2016). An experimental study on the effects of a
simulation game on students’ clinical cognitive skills and motivation. Advances in
Health Sciences Education, 21(3), 505–521. https://doi.org/10.1007/s10459-015-
9641-x
Essien, E. E., Akpan, O. E., & Obot, I. M. (2015). Students’ interest in social studies and
academic achievement in tertiary institutions in Cross Rivers State, Nigeria.
European Journal of Training and Development Studies, 2(2), 35-40.
Ezeudu, F. O., & Ezinwanne, O. P. (2013). Effects of simulations on students’
achievement in senior secondary school chemistry in Enugu east local
government area of Enugu State, Nigeria. Journal of Edu & Practice, 4(19), 58-66.
15
Folta, E, E. (2010). Investigating the impact on student learning and outdoor science interest
through modular serious educational games: A design-based research. [Doctoral
dissertation, North Carolina State University].
Guy, R. S., & Lownes-Jackson, M. (2015). The use of computer simulation to compare
student performance in traditional versus distance learning environment. Issues
in Informing Science and Information Technology, 12, 95–109.
Hashempour, Z., Rostampour, M., & Behjat, F. (2015). The effect of brainstorming as a
pre-writing strategy on EFL advanced learner’s writing ability. Journal of Applied
Linguistics and Language Research, 2(1), 86-99.
Hsu, C., Tsai, C., & Liang, J. (2011). Facilitating preschoolers’ scientific knowledge
construction via computer games regarding light and shadow: The effect of the
prediction observation explanation (POE) strategy. Journal of Science Education and
Technology, 20(5), 482-493. https://doi.org/10.1007/s10956-011-9298-z
Jack, G., & Kyado, J. (2017). Effectiveness of brain-based learning strategy on students’
academic achievement, attitude, motivation and knowledge retention in
electrochemistry. Journal of Education, Society and Behavioral Science, 21(3), 1-13.
https://doi.org/10.9734/ JESBS/2017/34266
Kikot, T., Costa, G., Magalhães, R., & Fernandes, S. (2013). Simulation games as tools for
integrative dynamic learning: The case of the management course at the
University of Algarve. Procedia Technology, 9, 11–21.
https://doi.org/10.1016/j.protcy.2013.12.002
Lu, J., Hallinger, P., & Showanasai, P. (2014). Simulation-based learning in management
education: A longitudinal quasi-experimental evaluation of instructional
effectiveness. Journal of Management Development, 33(3), 218–244.
https://doi.org/10.1108/J MD-11-2011-0115
Mehr, M. M., Aziz-Malayeri, F., & Bayat, A. (2016). The effects of brainstorming as a
prewriting activity on Iranian EFL learners prompted expository
writing. International Journal of Educational Investigations, 3(3), 85-93.
Malkawi, N. A. M., & Smadi, M. (2018). The effectiveness of using brainstorming
strategy in the development of academic achievement of sixth grade students in
english grammar at public schools in Jordan. International Education Studies, 11(3),
92-100. https://doi.org/10.5539/ies.v11n3p92
Mozelius, P., Hernandez, W., Sällström, J., & Hellerstedt, A. (2017). Teacher attitudes
toward game-based learning in history education. ICTE Journal, 6(4), 27-35.
https://doi.org/10.1515/ijicte-2017-0017
Nja, O. C., Cornelius-Ukpepi, B. U., & Orim, E. R. (2019). Effect of simulation
instructional method on undergraduate chemistry education student’s academic
performance in sodium reactions. European Journal of Scientific Research, 155(1), 6-
12.
Ochoyi, U. E. (2018). Effects of computer simulated games and power point on senior secondary
students’ attitude and achievement in geometry in Adamawa State. [Ph.D. thesis,
Benue State Univeristy, Markudi, Nigeria].
Owo, W. J., Idode, V. O., & Ikwut, E. F. (2016). Validity of brainstorming strategy on
students’ prior knowledge and academic performance in chemistry in selected
secondary schools in South-South Nigeria. American Scientific Research Journal for
Engineering, Technology, and Sciences, 24(1), 113-130.
Ranchhod, A., Gurău, C., Loukis, E., & Trivedi, R. (2014). Evaluating the educational
effectiveness of simulation games: A value generation model. Information Sciences,
264, 75–90. https://doi.org/10.1016/j.ins.2013.09.008
Rashtchi, M., & Beiki, M. (2015). The effect of teachers’ generated cooperative
brainstorming versus learner-generated cooperative brainstorming on activating
16
EFL learners’ background knowledge in essay writing classes. Indian Journal

of Fundamental and Applied Life Sciences, 5(s2), 1218-1227.
Rowan, K. (2014). Glossary of instructional strategies. Beesburg.com.
http://www.beesburg.com/edtools/glossary.html
Shear, S. B. (2016). Does critical theory have a place in social studies research? A
commentary. Journal of Social Studies Research, 40(3), 229-231.
https://doi.org/10.1016/j.jssr.2016. 03.009
Tarman, B., & Kuran, B. (2015). Examination of the cognitive level of questions in social
studies textbooks and the views of teachers based on Bloom Taxonomy.
Educational Sciences: Theory & Practice, 15(1), 1-10.
https://doi.org/10.12738/estp.2015. 1.2625
Wood, K. R. (2011). Simulation video games as learning tools: An investigation of instructor
guided reflection on cognitive outcomes. [Dissertation, Georgia State University].
Appendix 1
Social Studies Learning Outcome (SSLOT) Instrument
Time: 1hr
INSTRUCTIONS: ATTEMPT ALL QUESTIONS
Choose from the options lettered A-E the one that best answers each of the
following questions and write out in your answer sheet the correct letter only.
Give only one answer to each question.
1. One of these is NOT a symptom of drug abuse: A. Depression; B. Violent
behaviour tendencies; C. Impaired vision; D. Lack of sleep; E. Good health.
2. ……. is the name of the agency responsible for the control and prevention of
harmful substances. A. NECO; B. EFCC; C. NURTW; D. NAFDAC; E.
NDLEA
3. ……… is one of the ways to prevent trafficking in drugs. A. Education;
B. Conflict; C. War; D. Disturbance; E. Greed.
4. All of the following ways would help to discourage drug abuse EXCEPT........
A. strict penalties on drug offenders;
B. education. C. constructive use of time;
D. choosing good friends. E. belonging to cult.
5. The following are factors responsible for drug trafficking EXCEPT ……
A. bad nation economy; B. education; C. greed; D. poverty;
E. unemployment.
6. The agency responsible for controlling and preventing drug abuse and drug
trafficking is called …. A. NDLEA; B. ICPC; C. EFCC;
D. FRSC; E. JAMB.
7. The following are the effects of harmful substances EXCEPT……………. A.
vomiting; B. death; C. good health; D. ill health; E. frequent stooling.
8. The following are the consequences of drug abuse to the individuals
EXCEPT…
A. mental disorder; B. poor attitude to work; C. good health; D. brain fatigue;
E. long disease.
9. The process whereby a person prescribes drugs for him- or herself is called….
A. acceptance;
B. drug abuse; C. discipline; D. protection; E. injection.
17
10. These are the negative impacts of drug trafficking on a country’s economy
EXCEPT …. A. currencies are buried underground; B. leads to death of
victims; C. increases development; D. increases crime rate; E. increases
money laundry.
11. One of the following is correct about harmful substances EXCEPT ….. A. can
make people sick; B. can lead to diabetes; C. healthy growth; D. can be
destructive to the body; E. damage to internal organs.
12. Which of the following is NOT a consequence of drug abuse on the
individual? A. good nutrition; B. death;
C. mental illness; D. depression; E. leads to crime.
13. A powerful person in an organisation that deals in illegal drugs is called…
A. Drug baron; B. Distributor; C. Drug officer; D. Drug master; E. Drug
seller.
14. ……… is the misuse or excessive consumption of drug. A. Drug trafficking;
B. Medication; C. Drug abuse; D. Treatment; E. Operation.
15. Food that has been exposed to insects is called….. A. stale food; B. good
food; C. infested food; D. expired food; E. rotten food.
16. The following are some of the causes of drug abuse EXCEPT… A. emotional
disturbance; B. broken homes; C. desire to feel high; D. education; E.
curiosity/ experiment.
17. …… food’s life span has been outlived. A. Expired; B. Rotten; C. Stale; D.
Immature; E. Infested.
18. One who sells illegal drugs is called …… A. drug baron; B. drug dealer; C.
drug carrier; D. drug runner; E. drug addict .
19. The following are the consequences of drug abuse on the community
EXCEPT…. A. development of gangsters; B. insecurity of lives and
properties.
C. increase in crime; D. destruction of the youths in the community;
E. growth and development of the community.
20. The following are consequences of drug trafficking EXCEPT ……
A. Bad image for the country; B. shame and disgrace;
C. improved education; D. Imprisonment;
E. Death penalty.
18

Vol. 20, No. 3, pp. 18-32, March 2021
Innovative Teaching: A Qualitative Review of

Flipped Classrooms
Kevin Fuchs
Prince of Songkla University, Phuket, Thailand
https://orcid.org/0000-0003-3253-5133
Abstract. The extent and importance of web-based learning in higher

education have increased tremendously in the last decade, triggered by
new educational technologies and pedagogical approaches. Higher
education lecturing has traditionally followed a teacher-centered
approach, with lecturers giving classes in the classroom and students
performing out-of-class activities. Under this traditional approach, the
main actor in the teaching-learning process is the lecturer, while
students play a passive role. The flipped classroom emphasizes the role
of the student in the center of the learning environment and facilities an
active learning pedagogy. This paper reviews the most recent case
studies related to the flipped classroom approach in order to provide
educators guidelines on the best practices, traits, and merits of the
flipped classroom. A total of 22 case studies were included in this
qualitative review of the flipped classroom. The methodological inquiry
followed the PRISMA flow diagram that identified an initial pool of
3,764 articles. Upon identification of relevant case studies (n=22), a five-
point Likert-type sentiment rating was assigned as the basis to structure
the discussion. The rating was based on the students’ perceptions of the
flipped classrooms as investigated in the articles. The assessment
confirms that the majority of students have a positive perception of this
learning pedagogy. However, there are concerns about increased
workload for students, ambiguity in expected learning outcomes, and an
initial barrier to actively contribute; these are the primary implications
of this review.
Keywords: flipped classroom; inverted learning; active learning;

constructivism; TEFC
1. Introduction
Both the importance and extent of web-based or online learning in higher
education have increased vastly in the last decade. As a result, this trend has
been further accelerated by new educational technologies and pedagogical
approaches while educators continue to debate the best practices and
approaches for their classrooms. The variety of courses have continued to
increase in scale and scope and the sudden COVID-19 crisis in early 2020 has
©Author
19
further fast-tracked universities’ implementation of online teaching (Tang et al.,

2020). Flexibility with regard to the workload and allowing the students to make
decisions about when and what they learn are two of the primary advantages of
technology-enhanced learning (TeL). Besides, research has indicated that online
learning is pedagogically promising because it encourages deeper learning via
its self-paced and student-centered approach (Tang et al., 2020). The flipped
classroom is a versatile and well-researched pedagogical approach that
facilitates active learning with the power of peer learning (Crews & Butterfield,
2014; Thai, De Wever & Valcke, 2017). It is the objective of this paper to share a
review of the existing literature and provide comprehensive insights by
outlining the most important traits of the flipped classroom, merits with regard
to student development, and shortcoming of this pedagogical approach. It is not
within the scope of this review to examine the perspective of the educator or the
institution, but to provide a holistic overview from the perspective of students.
The important findings of the most recent case studies in 2019 and 2020
contribute to the body of knowledge and close an identified research gap.
Through the findings of this review paper, educators will be able to obtain
practical guidance which will help them implement the most effective methods
for their flipped classroom courses. The next chapter will provide insights into
the conceptual design of a flipped classroom.
1.1. Conceptual design of a flipped classroom

The design and implementation of flipped classrooms largely depends on the
educators, yet there is no consensus with regard to a universal design of a
flipped classroom (Thai, De Wever & Valcke, 2017). However, a recurring claim
regarding the design of a flipped classroom is that there are three particular
consecutive phases (Figure 1). Crews and Butterfield (2014) noted that the pre-
class, during-class, and post-class stages are essential elements when
implementing a flipped classroom. There is an agreement in the literature that
the first phase of a flipped classroom takes place as a prerequisite for students to
acquire fundamental knowledge (Thai, De Wever and Valcke, 2017). During this
phase, the students are expected to work individually in anticipation of the
second phase. This stage is typically administered through a Learning
Management System (LMS) with online tutorials, lecture notes, video clips
prepared by the teacher, or other course material (Crews & Butterfield, 2014).
The second phase is considered the most influential phase in the flipped
classroom design and determines the impact on the students’ learning within the
paradigm of active learning (Thai, De Wever & Valcke, 2017). This student-
centered approach places emphasis on discussion amongst peers, wherein the
teacher acts as a facilitator of the discussion (Fuchs, 2021).
20
Figure 1: A simplified design of a flipped classroom (adapted from Fuchs, 2021)
The post-class phase is designed to allow the students to reflect on the content of
the pre-class phase in combination with the peer-discussion in the during-class
phase (Lo, Lie & Hew; 2018). This last phase is oftentimes accompanied by a
personal learning assignment to reflect on the newly constructed knowledge
(Crews & Butterfield, 2014; Thai, De Wever & Valcke, 2017).
2. Methods
A comprehensive review of the literature was conducted using the PRISMA flow
diagram in order to cross-examine case studies and identify which
characteristics of flipped classrooms were most commonly researched, as well as
what additional calls for future research had been stated. Moreover, upon
identifying the 22 case studies that were reviewed, a sentiment score was
assigned to each paper.
2.1. The PRISMA flow diagram

The Preferred Reporting Items for Systematic Reviews flow diagram, or
PRISMA, was developed by a delegation of 29 review authors, medical editors,
consumers, clinicians, and methodologists (Liberati et al., 2009). In 2005, the
focus group attended a three-day meeting and participated in extensive post-
meeting digital correspondence. A consensus was reached concerning the
optimal process. Henceforth, a 27-item checklist and a four-phase flow diagram
was developed (Liberati et al., 2009; Stovold et al., 2014). Liberati et al. (2009)
further noted, “PRISMA focuses on ways in which authors can ensure the
transparent and complete reporting of systematic reviews and meta-analyses”
(Liberati et al., 2009, p. 120).
2.2. The search process

A search in the abstract and citation database Scopus was conducted and yielded
3,752 records in January 2021 (Figure 2). The inquiry was guided by the
keywords "flipped classroom" and "flipped learning". A large volume of
literature was retrieved and further criteria were applied to refine the results.
The following five limits were set:
• articles dated in 2019 and 2020
• a focus on social science and business as the subject area
21
• publication stage: final

• publication language limited to English
• source type was set to journal and conference proceeding
This exact search syntax was used:

“TITLE-ABS-KEY ("flipped classroom") AND ( LIMIT-TO (
PUBSTAGE,"final" ) ) AND ( LIMIT-TO ( LANGUAGE,"English" )
) AND ( LIMIT-TO ( SRCTYPE,"j" ) OR LIMIT-TO ( SRCTYPE,"p"
) ) AND ( LIMIT-TO ( PUBYEAR,2020) OR LIMIT-TO (
PUBYEAR,2019) )”
An additional 12 publications that were not Scopus-indexed, but the content of

which were familiar to the author, were included in this stage of the process
based on their merits. The identification stage was concluded with a total of
1,187 items that qualified for further screening.
The first stage of the screening process included reviewing each title of the 1,187
records. In this stage, a total of 752 records were excluded. In the second stage of
the screening process, another 317 records were excluded based on reviewing
their abstracts. A total of 118 articles were deemed eligible for full-text analysis.
In this final step, 96 articles were excluded wherein 79 were out of scope, 13 had
insufficient information about the methodology, and 4 were not adaptable to the
scope of this review. Eventually, 22 relevant case studies were selected for in-
depth analysis.
2.3. Appointing a sentiment score

A Likert-type rating system was utilized to assign a descriptive score to each
article that was included in the review. The following descriptive scores were
based on the Likert-type scale. The highest descriptive score corresponded to the
label “Positive”, the second-highest “Slightly Positive”, followed by “Neutral,
“Slightly Negative”, and finally “Negative”. Every article included in this paper
(n=22) was assigned a descriptive score by the reviewer (the author of this
report). The assessment for each score was based on the conclusion of the article
in regard to the participants’ perceptions of the flipped classroom. The purpose
of the sentiment score was to classify the articles into different categories and to
generate a variable for further analysis. After each article was assigned a
descriptive score, the author asked a colleague for an informal peer-review to
validate the results. Any opposing views were discussed until a consensus was
reached.
22
Figure 2: PRISMA flow diagram for a review of the literature
3. Flipping Classes: Where does all the hype come from?

The 22 articles that were included (Table 1) as the result of the PRISMA
procedure were reports of quantitative, qualitative, or mixed methods inquiries.
Within the criteria that were used to retrieve the literature, the majority of
articles had conducted quantitative research (n=10), followed by mixed methods
(n=8) and with only four qualitative studies (n=4) reviewed. The most common
means of analysis for quantitative research was descriptive analysis as opposed
to thematic analysis. The sample size for the quantitative studies ranged from
the smallest sample of 52 participants [No. 10] to 495 participants [18]. The
mixed-methods approach utilized sample sizes ranging from 42 [No. 05] up to
the largest sample of 152 participants [No. 02]. Naturally, the sample size for
purely qualitative research was smaller and ranged from 13 participants for the
smallest study included [No. 08] to 25 participants in the largest qualitative
study [No. 15]. To collect versatile and comprehensive information on previous
studies about flipped classrooms, there was no limitation placed on the
geographical location of the sampling. This allowed for the generation of a broad
perspective and analysis of results from different learning environments and
learning cultures. While no specific limitations were applied to the origin of the
study, it can be noted that four (4) studies originated from Spain, three (3) from
Australia, and the remainder were single studies, each representing one of 15
countries (Table 1).
23
Table 1: Summary of articles included in this review

No. Author/s (Year) Type of Inquiry Sample Origin Country3
Murillo-Zamorano et al. Quantitative Method N = 160
[01] Spain
(2019)
[02] Castedo et al. (2019) Mixed Methods N = 152 Spain
[03] Kay et al. (2019) Quantitative Method N = 103 Canada
[04] Lopes et al. (2019) Quantitative Method N = 152 Portugal
[05] Goedhart et al.(2019) Mixed Methods N = 42 Netherlands
[06] Cai et al. (2019) Quantitative Method N = 111 China
[07] Tomas et al. (2019) Mixed Methods N = 171 Australia
[08] Ha et al. (2019) Qualitative Method N = 13 Hong Kong
[09] Cilliers & Pylman (2020) Quantitative Method N = 82 South Africa
[10] Stöhr et al. (2020) Quantitative Method N = 52 Sweden
Martínez-Jiménez et al. Quantitative Method N = 58
[11] Spain
(2020)
[12] Abdekhoda et al. (2020) Quantitative Method N = 110 Iran
Colomo-Magaña et al. Quantitative Method N = 123
[13] Spain
(2020)
[14] Alamri (2019) Mixed Methods N = 52 Saudi Arabia
[15] Wilson (2020) Qualitative Method N = 25 Australia
[16] Alebrahim & Ku (2020) Qualitative Method N = 14 United States
Aprianto & Purwati Qualitative Method N = 15 Indonesia
[17]
(2020)
[18] Aljaraideh (2019) Quantitative Method N = 495 Jordan
[19] Almisad (2019) Mixed Methods N = 27 Kuwait
[20] Bhat et al. (2020) Mixed Methods N = 32 India
[21] Awidi & Paynter (2019) Mixed Methods N = 117 Australia
[22] Bicen & Beheshti (2019) Mixed Methods N = 130 Cyprus
3.1. The polarizing effect of flipped classrooms

Due to emerging educational technology and pedagogical methods, the extent
and value of online learning in higher education has increased tremendously in
the last decade. Teaching in higher education has conventionally followed a
teacher-centered method, with lecturers providing classroom courses and
students conducting out-of-class tasks. In this conventional method, students
play a passive role and the instructor is the key player in the teaching-learning
process (Murillo-Zamorano et al., 2019). A new viewpoint is provided by the
flipped classroom, in which education is student-centered and the lecturer has
shifted into a passive role.
“Flipping the classroom means that the traditional classroom becomes inverted.”
(Murillo-Zamorano et al., 2019)
Students are expected to regularly collect information by reading, watching

recorded lectures, or listening to podcasts outside of class time. This relates to
the two bottom levels of Bloom's Taxonomy, which allow students to remember
and process the class information given. The student is then required to apply
the knowledge they have acquired to problem-solving activities in groups
through peer instruction during classroom time. These tasks may include case
study research, analyzing different scenarios, and developing new problem
24
solutions that conform to Bloom's Taxonomy's higher-order level. When

students have a more extensive and comprehensive understanding of the
content, this makes for a better learning experience (Cilliers & Pylman, 2020).
Lopes et al. (2019) indicated that students are required to dedicate themselves to
being self-directed in their learning. This is particularly pertinent in the
classroom's e-learning aspect, which implies a cultural shift from an academic
point of view that can be pursued persistently, mainly in the context of higher
education (Lopes et al., 2019). The flipped classroom has been conceived as a
student-centered pedagogical method aiming to enhance the performance of
students. The literature recognized the flipped classroom's potential over
conventional methods for teaching and learning (Murillo-Zamorano et al., 2019).
Besides, to achieve comparable pre-knowledge and comprehension levels, the
self-regulated pre-class learning combination through prescribed reading and
video materials was considered to have merit, particularly in a diverse student
population alongside in-class activities integrating collaborative learning
activities which promote deeper learning (Goedhart et al., 2019; Lopes et al.,
2019).
Following the flipped approach to the classroom and to achieve a successful

learning outcome, the results of Murillo-Zamorano et al. (2019) trigger the
following essentials to be taken into account: firstly, there should be two-way
input (instructor to students and vice versa) as an efficient link between in-class
and out-of-class activities. Secondly, technology is used to move the lecture
outside (Murillo-Zamorano et al., 2019). Even though the overall experience was
pleasant, it was disappointing to see that some of the students claimed that the
flipped classroom did not contribute to their learning outcomes (Goedhart et al.,
2019). To effectively promote student participation and constructive learning,
teachers are often expected to incorporate several in-class pedagogical
modifications (Cai et al., 2019).
Moreover, the author performed an informal assessment of the 22 articles that

were included in the review and assigned an individual sentiment rating to each
article. A peer validated the rating by consensus with the reviewer and it ranged
through positive, slightly positive, neutral, slightly negative, and finally
negative on a five-point Likert-type scale. This approach was adapted from Qiu
et al. (2018), who used a similar technique to categorize articles based on the
sentiment expressed in the conclusion. In this context, the author assigned the
sentiment rating based on how the flipped classroom pedagogy was perceived
by the university students in the 22 articles. Furthermore, the author modified
the model from Qui et al. (2018) and sliced the results based on their type of
inquiry (Table 2). This aimed to identify whether there was a pattern between
the students’ perceptions and the strategy of how the data were collected.
25
Table 2: Cross-examination by sentiment and method

Sentiment Type of Inquiry Case Study No.
Quantitative Method [01], [04], [09], [11], [12], [13], [18]
Positive Mixed Methods [14], [19], [20], [22]
Qualitative Method [17]
Slightly Positive Mixed Methods [02], [05], [21]
Quantitative Method [03], [06]
Neutral Qualitative Method [16]
Mixed Methods [07]
Slightly Negative Qualitative Method [08]
Negative Quantitative Method [10], [15]
It can be noted that a simple majority of the studies were assigned a positive
rating. Out of the total 22 articles, 12 received a positive sentiment rating and, at
the other end of the spectrum, two (2) received a rating indicating a negative
perception towards the flipped classroom by the students. The remaining eight
(8) case studies fall in between, wherein (3) of them were rated slightly positive,
four (4) neutral, and one (1) slightly negative. The author did not identify any
pattern or correlation between the strategy of inquiry and the sentiment rating of
the study. While some of the flipped classroom case studies are associated with a
negative perception by the participating students, it can be stated that a large
majority (68%) had, at the minimum, a positive attitude towards this new active
learning approach. The sentiment rating helped to determine the general tone
amongst the selected case studies (Al-Natour & Turetken, 2020). In the
following, a more in-depth examination into both the challenges and positive
aspects of the flipped classroom pedagogy will be provided.
3.2. Flipped Classrooms: A remedy to achieve inclusive education?

As a consequence of the flipped classroom approach, Wilson (2020) identified a
significant problem in the students' workload. Also, it is recommended that a
lower workload could be tested to determine whether the same instructional
result could still be obtained. The paper highlights major concerns related to
flipped learning pedagogy due to the increased workload (Wilson, 2020).
Furthermore, Stöhr et al. (2020) were unable to confirm the flipped classroom's
reported positive effect in higher education in a quantitative assessment that
contrasted the flipped classroom with conventional lecture-based learning. Ha et
al. (2019) discussed beneficial outcomes of the flipped approach to the
classroom; in their case, the researchers indicated that students had adapted to a
“spoon-fed” system and, thus, initially failed to respond to the flipped model of
the classroom (Ha et al., 2019). A related observation was reported by Castedo et
al. (2019), in which students initially failed to respond to the latest method. Kay
et al. (2019) as well as Alebrahim and Ku (2020) defined positive and negative
perceptions; both studies concluded that there is a need for clarification,
flexibility, timely instruction, and feedback in the implementation of a flipped
classroom. Besides, it was observed that the transfer of materials from the
conventional face-to-face model to the computer-mediated format requires
specialized expertise, time, and dedication (Cai et al., 2019). Moreover, the usage
of efficient technology must be reviewed beforehand because this helps facilitate
outside-class engagement amongst students (Aprianto & Purwati, 2020). These
26
suggestions shared by Cai et al. (2019) and Aprianto and Purwati (2020)
demonstrate that there is still uncertainty among students when it comes to
technology-enhanced learning (TEL).
Table 3: Empirical findings of the selected case studies

No. Empirical findings
FC has positive effects on students’ knowledge, skills, and
[01]
engagement.
Initial reluctance towards the FC methodology, but finally a higher
[02]
attendance than traditional learning
Identified a need for clarity, flexibility, timely guidance and feedback,
[03]
and cognitive engagement
FC requires a high level of teacher work and creativity in developing
[04]
content and then properly organizing it
The combination of self-regulated pre-class learning in combination
[05]
with in-class activities facilitates deeper learning
The migration of materials from traditional face-to-face format to
[06] computer-mediated format is a process that requires specific skills, as
well as a significant amount of time and effort
Respondents’ opinions regarding whether a flipped classroom
[07] motivated them to learn, or was more engaging than traditional
instruction, were divided
Students that were used to a “spoon-fed” approach initially struggled
[08]
to adapt to the flipped classroom model
The activities done in the classroom provided real-life experience that
[09]
could be used for problem-solving when they enter the workplace
Results do not confirm the documented positive effect of the flipped
[10]
classroom in STEM education
The academic results have been better in which the flipped learning
[11]
methodology was implemented
An approach that encourages active learning and improves student
[12]
performance; meanwhile decreasing the failure rate in education
Sample responded positively to the initiative to implement the flipped
[13]
classroom and appreciated its potential to promote learning
Results demonstrated improved student academic achievement and
[14]
satisfaction, compared with the traditional lecture-based approach
This study highlights important problems associated with Flipped
Learning pedagogy and suggests future research in divergent
[15] pedagogies attend to the critical issues of student workload, and to
consider if the same educational outcome could be achieved equally as
well by a lower workload pedagogy.
Positive and negative opinions depending on how they implemented
[16] the flipped classroom including the engagement of students and the
environment
The use of efficient technology must be reviewed prior to use because
[17]
it helps facilitate outside-class engagement amongst students
Generally a high satisfaction with flipped classroom, wherein
[18] difference were detected regarding the gender variable in favor of
male students
The results showed that the majority of the students were ready and
[19]
able to this active learning method
[20] Students prefer operating in the flipped mode as it provides them the
27
flexibility to acquire new insights on their own time while getting an

opportunity to interact with peers and teachers during the lecture time
We have found positive signs that a flipped classroom approach can
[21]
enhance the student learning experience and outcomes
The outcome of the study revealed that technology integrated learning
[22]
environments have positive impacts on students’ learning
Goedhart et al. (2019) indicated, contrary to the drawbacks described, that self-
regulated pre-class learning in combination with in-class activities encourages
deeper learning. This was confirmed by the research analysis of Awidi and
Paynters (2019), who noticed positive signs that a flipped-classroom approach
could increase the experience and results of student learning. The most
prominent positive outcomes from higher education case studies relating to
flipped classrooms confirmed positive results on students' understanding,
expertise, and participation (Alamri, 2019; Murillo-Zamorano et al., 2019;
Abdekhoda et al., 2020; Bhat et al., 2020). The fact that students adopted active
learning in this method is another prominent discovery and is corroborated by
research studies (Aljaraideh, 2019; Almisad, 2019; Cilliers & Pylman, 2020). Also,
the chance to develop new perspectives in their own time and the provision of
the opportunity to interact with peers and teachers were discussed as beneficial
factors during the lecture period (Bhat et al., 2020). This main observation
underlines the basic idea behind Constructivism's pedagogical ideology, in
which knowledge is built in a passive role through peer interaction rather than
by lecturing to the student. Cilliers and Pylman (2020) suggested that another
positive aspect identified by students was real-life exposure through
conversations with peers, as these could be used for problem-solving as they
enter the workplace (Table 3).
4. Conclusion
To conclude, among various learning cultures, the flipped classroom design and
its implementation were identified as challenges that led to negative experiences
of the flipped classroom pedagogy. Furthermore, the review identified that case
studies that attested a positive sentiment towards the flipped classroom stated a
common perception regarding the merits. Amongst the most commonly noted
characteristics were improved student engagement, better academic success, and
higher learning outcomes. The review contributed an outline of possible focus
areas for future research. Educators would benefit from a practical guide about
the best practices in designing their flipped classroom. More importantly, a
qualitative inquiry to identify why the flipped classroom was successful in some
cultures but failed in others would generate additional scientific value. This
could be further strengthened by a meta-analysis that aims to identify a
correlation between learning cultures and the perception of flipped classrooms.
The purpose of the study was not to generate a versatile stakeholder perspective,
but instead to identify areas of improvement through an analysis of perceptions
from students in higher education. The paper validated that the majority of the
samples have a positive perception towards this learning pedagogy. However,
there remain concerns about increased student workload, the ambiguity of
expected learning outcomes, and an initial barrier to actively contributing –
these were identified as the primary implications of the review.
28
5. Acknowledgements
The author declares having no potential conflicts of interest concerning the
research, authorship, and/or publication of this article.
6. Notes
1Authors’ ORCID: https://orcid.org/0000-0003-3253-5133.
2TITLE-ABS-KEY is referring to the syntax Title-Abstract-Keyword.
3Origin Country refers to the geographical location from where the sample was
collected.
4Mr. Kevin Fuchs is currently a lecturer in the Faculty of Hospitality and
Tourism, Prince of Songkla University, in Phuket, Thailand. He graduated with

a Master of Science in Informatics from Linnaeus University in Sweden and has
a keen research interest in topics that revolve around Technology-enhanced
Learning (TeL), Connectivism, Contemporary Educational Theories, and the
Technology-Enhanced Flipped Classroom (TEFL).
7. References
Abdekhoda, M., Maserat, E., & Ranjbaran, F. (2020). A conceptual model of flipped
classroom adoption in medical higher education. Interactive Technology and Smart
Education, 17(4), 393-401. doi:10.1108/ITSE-09-2019-0058
Alamri, M. M. (2019). Students' academic achievement performance and satisfaction in a
flipped classroom in Saudi Arabia. International Journal of Technology Enhanced
Learning, 11(1), 103-119. doi:10.1504/IJTEL.2019.096786
Alebrahim, F., & Ku, H. Y. (2020). Perceptions of student engagement in the flipped
classroom: a case study. Educational Media International, 57(2), 128-147.
doi:10.1080/09523987.2020.1786776
Aljaraideh, Y. (2019). Students' perception of flipped classroom: A case study for private
universities in Jordan. JOTSE: Journal of Technology and Science Education, 9(3),
368-377. doi:10.3926/jotse.648
Almisad, B. (2019). Perceptions of the'flipped classroom': a case study from a developing
country. International Journal of Learning Technology, 14(1), 78-98.
doi:10.1504/IJLT.2019.100617
Al-Natour, S., & Turetken, O. (2020). A comparative assessment of sentiment analysis
and star ratings for consumer reviews. International Journal of Information
Management, 54, 102132. doi:10.1016/j.ijinfomgt.2020.102132
Aprianto, E., & Purwati, O. (2020). Multimedia-Assisted Learning in a Flipped
Classroom: A Case Study of Autonomous Learning on EFL University Students.
International Journal of Emerging Technologies in Learning (IJET), 15(24), 114-127.
doi:10.3991/ijet.v15i24.14017
Awidi, I. T., & Paynter, M. (2019). The impact of a flipped classroom approach on
student learning experience. Computers & Education, 128, 269-283.
doi:10.1016/j.compedu.2018.09.013
Bicen, H., & Beheshti, M. (2019). Assessing perceptions and evaluating achievements of
ESL students with the usage of infographics in a flipped classroom learning
environment. Interactive Learning Environments, 1-29.
doi:10.1080/10494820.2019.1666285
Bhat, S., Raju, R., Bhat, S., & D’Souza, R. (2020). Redefining quality in engineering
education through the flipped classroom model. Procedia Computer Science, 172,
906-914. doi:10.1016/j.procs.2020.05.131
29
Cai, J., Yang, H. H., Gong, D., MacLeod, J., & Zhu, S. (2019). Understanding the
continued use of flipped classroom instruction: a personal beliefs model in
Chinese higher education. Journal of Computing in Higher Education, 31(1), 137-
155. doi:10.1007/s12528-018-9196-y
Castedo, R., López, L. M., Chiquito, M., Navarro, J., Cabrera, J. D., & Ortega, M. F. (2019).
Flipped classroom—comparative case study in engineering higher education.
Computer Applications in Engineering Education, 27(1), 206-216.
doi:10.1002/cae.22069
Cilliers, L., & Pylman, J. (2020). South African students’ perceptions of the flipped
classroom: A case study of higher education. Innovations in Education and
Teaching International, 1-10. doi:10.1080/14703297.2020.1853588
Colomo-Magaña, E., Soto-Varela, R., Ruiz-Palmero, J., & Gómez-García, M. (2020).
University students’ perception of the usefulness of the flipped classroom
methodology. Education Sciences, 10(10), 275. doi:10.3390/educsci10100275
Crews, T., & Butterfield, J. (2014). Data for Flipped Classroom Design: Using Student
Feedback to Identify the Best Components from Online and Face-to-Face
Classes. Higher Education Studies, 4(3), 38-47. doi:10.5539/hes.v4n3p38
Fuchs, K. (2021). Evaluating The Technology-Enhanced Flipped Classroom Through The
Students’ Eye: A Case Study. Proceedings of The 3rd International Conference on
Research in Education, 2021, Vol. 1, 25-33. doi:10.6084/m9.figshare.14173622
Goedhart, N. S., Blignaut-van Westrhenen, N., Moser, C., & Zweekhorst, M. B. M. (2019).
The flipped classroom: supporting a diverse group of students in their learning.
Learning Environments Research, 22(2), 297-310. doi:10.1007/s10984-019-09281-2
Ha, A. S., O’Reilly, J., Ng, J. Y., Zhang, J. H., & Serpa, S. (2019). Evaluating the flipped
classroom approach in Asian higher education: Perspectives from students and
teachers. Cogent Education, 6(1), 1638147. doi:10.1080/2331186X.2019.1638147
Kay, R., MacDonald, T., & DiGiuseppe, M. (2019). A comparison of lecture-based, active,
and flipped classroom teaching approaches in higher education. Journal of
Computing in Higher Education, 31(3), 449-471. doi:10.1007/s12528-018-9197-x
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P., &
Moher, D. (2009). The PRISMA statement for reporting systematic reviews and
meta-analyses of studies that evaluate health care interventions: explanation and
elaboration. Journal of clinical epidemiology, 62(10), e1-e34.
doi:10.1016/j.jclinepi.2009.06.006
Lo, C. K., Lie, C. W., & Hew, K. F. (2018). Applying “First Principles of Instruction” as a
design theory of the flipped classroom: Findings from a collective study of four
secondary school subjects. Computers & Education, 118, 150-165.
doi:10.1016/j.compedu.2017.12.003
Lopes, S. F. S. F., Gouveia, L. M. B., & da Cunha Reis, P. A. (2019). The Flipped
Classroom and Higher Education-Experiences with Computer Science Students.
International Journal of Advanced Engineering Research and Science, 6(10).
doi:10.22161/ijaers.610.3
Martínez-Jiménez, R., & Ruiz-Jiménez, M. C. (2020). Improving students’ satisfaction and
learning performance using flipped classroom. The International Journal of
Management Education, 18(3), 100422. doi:10.1016/j.ijme.2020.100422
Murillo-Zamorano, L. R., Sánchez, J. Á. L., & Godoy-Caballero, A. L. (2019). How the
flipped classroom affects knowledge, skills, and engagement in higher
education: Effects on students' satisfaction. Computers & Education, 141, 103608.
doi:10.1016/j.compedu.2019.103608
30
Qiu, J., Liu, C., Li, Y., & Lin, Z. (2018). Leveraging sentiment analysis at the aspects level
to predict ratings of reviews. Information Sciences, 451, 295-309.
doi:10.1016/j.ins.2018.04.009
Stovold, E., Beecher, D., Foxlee, R., & Noel-Storr, A. (2014). Study flow diagrams in
Cochrane systematic review updates: an adapted PRISMA flow diagram.
Systematic reviews, 3(1), 1-5. doi:10.1186/2046-4053-3-54
Stöhr, C., Demazière, C., & Adawi, T. (2020). The polarizing effect of the online flipped
classroom. Computers & Education, 147, 103789.
doi:10.1016/j.compedu.2019.103789
Tang, T., Abuhmaid, A. M., Olaimat, M., Oudat, D. M., Aldhaeebi, M., & Bamanger, E.
(2020). Efficiency of flipped classroom with online-based teaching under
COVID-19. Interactive Learning Environments, 1-12.
doi:10.1080/10494820.2020.1817761
Thai, N. T. T., De Wever, B., & Valcke, M. (2017). The impact of a flipped classroom
design on learning performance in higher education: Looking for the best
“blend” of lectures and guiding questions with feedback. Computers & Education,
107, 113-126. doi:10.1016/j.compedu.2017.01.003
Tomas, L., Doyle, T. and Skamp, K., 2019. Are first year students ready for a flipped
classroom? A case for a flipped learning continuum. International Journal of
Educational Technology in Higher Education, 16(1), p.5. doi:10.1186/s41239-019-
0135-4
Wilson, K. (2020). What does it mean to do teaching? A qualitative study of resistance to
Flipped Learning in a higher education context. Teaching in Higher Education, 1-
14. doi:10.1080/13562517.2020.1822312
Appendix 1
Table 4: Summary of empirical key findings with sentiment rating

Author/s Type of Inquiry
No. Empirical Findings Sentiment Country
(Year) (Sample Size)
Murillo- Quantitative FC has positive effects on Positive Spain

[01] Zamorano et Method students’ knowledge,
al. (2019) skills, and engagement.
(n=160)
Castedo et al. Mixed Methods Initial reluctance towards Slightly Spain

(2019) the FC methodology, but Positive
[02] (n=152) finally a higher attendance
than traditional learning
Kay et al. Quantitative Identified a need for Neutral Canada

(2019) Method clarity, flexibility, timely
[03]
guidance and feedback,
(n=103) and cognitive engagement
Lopes et al. Quantitative FC requires a high level of Positive Portugal

(2019) Method teacher work and
[04] creativity in developing
(n=152) content and then properly
organizing it
Goedhart et Mixed Methods The combination of self- Slightly The

al.(2019) regulated pre-class Positive Netherlan
[05] (n=42) learning in combination ds
with in-class activities
facilitates deeper learning
[06] Cai et al. Quantitative The migration of materials Neutral China

from traditional face-to-
31
(2019) Method face format to computer-

mediated format is a
(n=111) process that requires
specific skills, as well as a
significant amount of time
and effort
Tomas et al. Mixed Methods Respondents’ opinions Neutral Australia

(2019) regarding whether a
(n=171) flipped classroom
[07] motivated them to learn,
or was more engaging
than traditional
instruction, were divided
Ha et al. (2019) Qualitative Method Students that were used to Slightly Hong
a “spoon-fed” approach negative Kong
[08] (n=13) initially struggled to adapt
to the flipped classroom
model
Cilliers & Quantitative The activities done in the Positive South

Pylman (2020) Method classroom provided real- Africa
life experience that could
[09] (n=82) be used for problem-
solving when they enter
the workplace
Stöhr et al. Quantitative Results do not confirm the Negative Sweden

(2020) Method documented positive
[10] effect of the flipped
(n=52) classroom in STEM
education
Martínez- Quantitative The academic results have Positive Spain

Jiménez & Method been better in which the
[11] Ruiz-Jiménez flipped learning
(2020) (n=58) methodology was
implemented
Abdekhoda et Quantitative An approach that Positive Iran

al. (2020) Method encourages active learning
and improves student
[12] (n=110) performance; meanwhile
decreasing the failure rate
in education
Colomo- Quantitative Sample responded Positive Spain

Magaña et al. Method positively to the initiative
(2020) to implement the flipped
[13] (n=123) classroom and appreciated
its potential to promote
learning
Alamri (2019) Mixed Methods Results demonstrated Positive Saudi

improved student Arabia
(n=52) academic achievement
[14]
and satisfaction, compared
with the traditional
lecture-based approach
Wilson (2020) Qualitative Method This study highlights Negative Australia

important problems
(n=25) associated with Flipped
Learning pedagogy and
[15] suggests future research in
divergent pedagogies
attend to the critical issues
of student workload, and
to consider if the same
32
educational outcome
could be achieved equally
as well by a lower
workload pedagogy.
Alebrahim & Qualitative Method Positive and negative Neutral United

Ku (2020) opinions depending on States
(n=14) how they implemented
[16] the flipped classroom
including the engagement
of students and the
environment
Aprianto & Qualitative Method The use of efficient Slightly Indonesia

Purwati (2020) technology must be Positive
(n=15) reviewed prior to use
[17]
because it helps facilitate
outside-class engagement
amongst students
Aljaraideh Quantitative Generally a high Positive Jordan

(2019) Method satisfaction with flipped
classroom, wherein
[18] (n=495) difference were detected
regarding the gender
variable in favor of male
students
Almisad Mixed Methods The results showed that Positive Kuwait

(2019) the majority of the
[19] (n=27) students were ready and
able to this active learning
method
Bhat et al. Mixed Methods Students prefer operating Positive India

(2020) in the flipped mode as it
(n=32) provides them the
flexibility to acquire new
[20] insights on their own time
while getting an
opportunity to interact
with peers and teachers
during the lecture time
Awidi & Mixed Methods We have found positive Slightly Australia

Paynter (2019) signs that a flipped Positive
(n=117) classroom approach can
[21]
enhance the student
learning experience and
outcomes
Bicen & Mixed Methods The outcome of the study Positive Cyprus
Beheshti revealed that technology
(2019) (n=130) integrated learning
[22]
environments have
positive impacts on
students’ learning
33

Vol. 20, No. 3, pp. 33-50, March 2021
Elementary Teachers’ Thoughts about Distance

Education and Learning 21st-Century Skills
during COVID Pandemic
Amani K. H. Alghamdi
Imam Abdulrahman bin Faisal University, Dammam, Saudi Arabia
http://orcid.org/0000-0002-8500-0266
Naimah Ahmad Al-Ghamdi

Imam Abdulrahman bin Faisal University, Dammam, Saudi Arabia
https://orcid.org/0000-0002-8851-0456
Abstract. There was a dearth of research documenting the Arab World’s

educational response to the COVID-19 pandemic. In particular, female
Saudi elementary teachers’ views on distance education during those
unprecedented times were under-explored in the research literature. In
the wake of the outbreak in Saudi Arabia, concerns have been raised
about education disruption at the elementary school level and its impact
on instilling and developing 21st-century skills. Conducted at a college of
education in Saudi’s Eastern Province, a qualitative exploratory study
examined 20 Saudi postgraduate female students’ written responses to a
journal question concerning teaching elementary pupils online about the
pandemic while 21st-century skills were being inculcated. A critical
finding was that the participants expressed a sense of ample scope for
personal and pedagogical transformation despite the enormity of the
challenge. Suggestions for future research, pre-service, and in-service
teacher training, and professional development were recommended
around pedagogical adjustments for teaching elementary learners online
while ensuring they still learn and develop their 21st-century skills.
Keywords: COVID-19 pandemic; elementary students; distance learning;

21st-century skills; Saudi Arabia
1. Introduction
The COVID-19 pandemic provided an opportunity to explore the subject of
schooling in a new context. At present, attempts to contain the crisis encompass
an adaptive response, which includes a transition to hybrid education (adopting
a blend of both online and face-to-face learning strategies in both learning
environments (Gagnon et al., 2020). There were concerns about whether such a
new approach will strengthen learning and whether, or how, teaching will resume
©Authors
34
at the end of the de-escalation (Al-Roubaie & Alaali, 2020; Khatun & Saadat, 2020;
Kovačević, 2020).
This exploratory study was based in the Kingdom of Saudi Arabia (KSA), which
is transitioning to a knowledge-based and information-age economy necessitating
students’ learning of 21st-century skills. In addition, the main objective of this
research was to disclose female Saudi elementary teachers’ views on distance
education during this unusual time. Such knowledge was beneficial and much-
needed. For clarification, these skills constituted critical thinking, creativity,
collaboration, communication, several types of literacy (information, media,
technology), flexibility, leadership, initiative, productivity, social skills, and local
and global connections (Abualrob, 2019; Stauffer, 2020). The most useful
pedagogical approach for teaching these skills was a student-centered, self-
directed, active learning and experiential learning pedagogy (Van Gompel, 2019).
Prior to the pandemic, the Saudi educational system was challenged because its
teaching philosophy was detached from students’ experiences and needs. A
report on new models of education for the fourth industrial revolution argued
that in both developing and developed countries, many educational systems still
depend heavily on passively taught and memorized learning rather than
interactive, critical, and self-directed methods (Basham et al., 2020; Jung, 2020;
Sekiyama, 2020). Despite the strain caused by the pandemic, the Saudi school
system was pushed to make significant changes. That is to say, any resultant
school reconfiguration must respond and adapt to these complex times through
approaches such as online or blended learning (Campa, 2020; Khatun & Saadat,
2020; Voskoglou, 2020).Gagnon et al. (2020) defined blended learning as a
synchronous online learning, which was used to enhance student learning
between face-to-face sessions. Online instruction does NOT replace face-to-face
time.
At the outbreak of the pandemic in spring 2020, virtually all nations opted for
immediate school closure; about 1.6 billion pupils from elementary school to
postsecondary level were prohibited from attending face-to-face classes. Instead,
pupils were instructed to study online so that learning would not be disrupted
because of the severe public health concerns. The COVID-19 crisis had
emphasized the need to build both connected school communities and a stronger
educational relationship between homes and schools. These unanticipated and/or
not well-formed connections amount to a mix of positive and negative perceptions
among students, teachers, and parents of the merits of distance learning (Burgess
& Sievertsen, 2020; Douglas et al., 2020).
Of relevance to this paper is that this prolonged crisis presented a remarkable

challenge to schools and it was further exacerbated by the widespread lack of
access to high-speed internet or interactive media. This digital divide among the
student population affected their perceptions of their learning experiences during
the COVID-19 pandemic (Esposito & Principi, 2020; Lee, 2020; Viner et al., 2020;
Wang et al., 2020). In particular, the number of Saudi Arabian (SA) young people
and teens who are disadvantaged by a lack of technology was and still steadily
35
growing (Alaboudi & Alharbi, 2020; Almutairi et al., 2020; Alshammari, 2020).
Anchored in this and other concerns of the pandemic, SA researchers should
explore the experiences of elementary school children and their teachers in online
learning during the pandemic. By doing so, resultant deeper understandings can
be used to address attendant challenges so students can pursue their studies at
home without being academically disadvantaged. This study focused herein on
teachers who were responsible for ensuring that the learning environment run
well (König et al., 2020; Rapanta et al., 2020).
2. Significance of Research and Research Question

The topic of teaching 21st-century skills in elementary school was under-
researched (Van Gompel, 2019). This coupled with the fact that one third of Saudi
citizens are aged from 0 to 14 (young learners) (Algaissi et al., 2020) and more than
half of Saudi elementary teachers (52.3%) are women (UNESCO Institute of
Statistics, 2019). Such a fact led the researchers to serendipitously focus not only
on soliciting opinions from formally trained Saudi female elementary teachers,
but also on the notion of integrating 21st-century skills into the Saudi elementary
curriculum during the pandemic. The study participants, therefore, included
those who had returned to university on a part-time basis to obtain a master’s
degree in education.
It was thought that gainfully employed elementary teachers attending the

graduate school for intellectual enrichment would provide a fertile avenue for
soliciting authentic data, because they could draw on lived experiences before and
during the pandemic. Hence, the research question guiding this inquiry was put
forward:
RQ1: What are the thoughts of Saudi female postgraduates (who study
elementary education) about how to develop and deliver a distance education
lesson about COVID-19 the best while ensuring elementary students’ 21st-century
skills development?”
3.1. Saudi Context and Global Pandemic Response
Saudi Arabia is the largest country in the Arab world having a land mass of 21.5
million km2. Its population is approximately 35 million with the bulk aged 15 and
64 years. Those between 0 and 14 years accounted for 32.4% of the population
with only 2.8% aged +65 years (Algaissi et al., 2020). Saudi Arabia also attracted
many foreign workers with approximately 12 million overseas nationals that
covered 37% of the country’s residents. It is a key destination for investors,
representatives of new industries and, increasingly, tourists. KSA is also of
tremendous religious importance for the almost 2 billion Muslims around the
world (Hackett & McClendon, 2017). Each year, more than 2.5 million pilgrims
visit the country, primarily the Makkah region, for both the five-day annual
pilgrimage known as Hajj and the Islamic rite known as Umrah. With this influx
of visitors, the KSA has been identified as a potential pandemic hotspot.
The country has a comprehensive medical system with free medical insurance
available to all citizens and a network of hospitals and clinics, both public and
36
private. Although there are currently 2.2 beds per 1,000 people (Alazmy et al.,
2020; Samra et al., 2020), there are calls for significant structural changes in the
healthcare sector to satisfy the growing needs of the country. Despite the aim of
KSA to safeguard the well-being, protection, and welfare of both citizens and
pilgrims, the spread of infectious diseases worldwide is always extremely
dangerous (Al Saidi et al., 2020; Aldarhami et al., 2020); the implications of a
pandemic could be particularly severe.
Bearing this in mind, on March 2, 2020, before the first case of COVID-19 was
detected in Saudi Arabia (Alyami et al., 2020), the nation became one of the first
countries to implement early and unparalleled security to deter or alleviate the
spread of the disease on its soil. This included school closures and mandatory
online distance education. Many studies around the world have documented
school systems’ responses to the COVID-19 crisis, including Armitage and
Nellums (2020), Auger et al. (2020), Hilppö et al. (2020), Loima (2020), Poletti and
Raballo (2020), and Sintema (2020) among others. Triggered by the novel Corona
virus, school closure was imposed in different countries such as the US, Finland,
and Sweden affecting school children’s prospect of continuing their education and
social life. However, few studies have been conducted within the Saudi context
making this Arab-world study a valuable contribution to this bourgeoning
knowledge base. In particular, Saudi female elementary teachers’ outlooks
towards teaching 21st-century skills online during the pandemic were obscure.
Regarding other jurisdictions, using a rapid systematic review of the literature,

Viner et al. (2020) assessed the success of school suspensions, closures and
management practices during Corona virus outbreaks including COVID-19. Per
the latter, studies revealed two countries and a major city in the world: China,
Hong Kong, and Korea that imposed school suspensions swiftly. However, school
closures alone eliminated only 2%to 4% of virus transmissions relative to social
distancing strategies (Bayrakdar & Guveli, 2020). Viner et al. (2020) cautioned
policymakers to be aware of misguided evidence urging them to give due
consideration to combinations of social distancing measures and school closures.
Stringent social distancing policies were implemented over lengthy stretches of
time including measures to be taken within classrooms. According to Burgess and
Sievertsen (2020), the COVID-19 pandemic is essentially a safety problem. Most
countries have rightly decided to close schools, universities, and colleges.
Politicians, however, faced a dilemma merely in closing schools to reduce contact
and potentially save lives or leave them open allowing employees to do their jobs,
and keeping the economy functioning and not disrupting learning. Keeping in
mind that the latter required changes in the nations’ education delivery systems.
3.2. Importance of Elementary Education

Indeed, the COVID-19 pandemic is creating an opportunity to explore the subject
of schooling in a new context. The kneejerk reaction to minimize the learning
continuation crisis was to impose distance learning or a hybrid version of learning
(online and face-to-face) with negligible preparedness or readiness for all
concerned parties. Perceptions and thoughts of both Saudi students and teachers
about this policy decision must be explored giving the dearth of pertinent
37
literature about this topic in the KSA. An investigation of this nature will reveal
pedagogical implications for all levels of education with a special interest herein
about the elementary curriculum provided to Saudi pupils aged six to 12 (from
Grades One through Grade Six equivalent).
This age group matters because elementary students starting school now are
future workers and leaders in a digital world that will demand new skills and new
ways of thinking. Based on this, elementary educators must be able to teach
elementary pupils effectively, in particular, in the face of a global crisis, such as a
pandemic. As a matter of fact, elementary school education is considered as a
ladder for child’s future development. It shapes predispositions for lifelong
learning and informs cognitive and social development of young generations. It
is viewed as “an essential building block for a child’s future success” (Walden
University, 2020). Mead (2016) concurred that the quality and success of
elementary education was a powerful predictor of a child’s life outcomes. In the
same line of thought, Mishrif and Alabduljabbar (2018) pointed out how
important the primary and secondary education in Saudi Arabia are, because they
are meant to help pupils and students adopt a positive attitude towards work and
employment achieving the goals set in Vision 2030.
3.3. Teaching 21st-Century Skills in Elementary School

Regarding nation building, special attention is warranted for ensuring Saudi
elementary students’ learning of 21st-century skills is an effective one so that the
future generations excel in a knowledge-based economy and in the modern
Information Age. Young people comprised one third of the Saudi population
(Algaissi et al., 2020) and will eventually assume the mantle of carrying the KSA
into the future per Vision 2030, its most recent national development plan (KSA,
2016). Vision 2030 heavily depends on education to ensure its three goals:
ambitious nation, thriving economy and vibrant society. All levels of education
matter. In a powerful statement and as asserted by Abualrob (2019), “elementary
school systems... should work to provide pupils with the dexterity they need to
become engaged thinkers, resilient and resourceful learners, creative problem
solvers and active members of their communities” (pp. 108-109).
Van Gompel (2019) has recently affirmed, however, that “research about effective
pedagogical frameworks to support [teaching of 21st century] skills in an
elementary context is limited” (pp. 6). Yet, educators agreed that elementary
students must be oriented to this skill set. It is never too young to start with this
aspect of learning (Abualrob, 2019). To illustrate, Van Gompel (2019) explained
that even for this age cohort, “creativity is required for critical thought [and] oral
communication and collaborative skills are important together in order for
[elementary] students to express thought and share ideas” (p. 29).
Unfortunately, like many teachers around the world, Saudi elementary educators
have experienced all sorts of difficulties in delivering online lessons to young
learners during the pandemic. Despite Saudi schools usually offering teachers
access to teaching resources, and access to the internet and information
technology (IT) connection, challenges and disruptions occurred during online
38
lesson delivery and lesson preparation training. With the prospect of national
lockdowns and the eventual reopening of schools in several areas of the country,
it is critically important to assess and evaluate the possible future of schooling and
curricula delivery in the KSA. This study focused on Saudi elementary educators’
attitudes about teaching elementary pupils via distance education during the
pandemic while ensuring students’ attainment of 21st-century skills.
Results from surveying the internet showed some educational websites for school
children that have enumerated extended lists of the 21st-century skills and
personal qualities. It emphasized that new generations need to learn and build at
a young age those skills. Just to name a few, they included empathy, curiosity,
adaptability, resilience, self-directed skills, collaboration/teamwork, active
listening, information seeking, collaboration, lifelong learning, communication,
creativity, and critical thinking (Nana, 2020; Simply Learning Tuition &
Consultancy, 2021).
4. Method
The study employed a qualitative exploratory research design (journaling).
Exploratory studies tentatively probe or investigate a phenomenon especially
when little is known about it in a given context. Rather than providing conclusive
and final answers to the research questions and using smaller samples and
structured answers, researchers strived to provide broad and initial
understandings to lay the groundwork for more conclusive studies (Dudovskiy,
2016; McGregor, 2018).
4.1. Sample Frame and Participants

Education in Saudi Arabia is segregated. Female teachers are trained to teach
female pupils, although this is slowly changing (Obaid, 2019). Using convenience
sampling (i.e., the lead author was teaching the study participants as postgraduate
students at an Eastern Province university), the final sample frame comprised of
20 female participants studying for their master’s degree in education with 14
gainfully employed and six unemployed. Participants ranged in age between 26
to 45 years old. Most of them had a BA in Education or Arts, and a few had a BSc
in Mathematics, Chemistry, Biology or Physics. Their teaching experience ranged
between 6 and 11 years (averaging 8.5 years), and all worked at female schools.
Within this sample frame, many participants were also parents with school-age
children.
4.2. Data Collection

Data in the form of journal entries were collected in May2020. Participants were
expected to offer their views and reflections about delivering an online class
during the pandemic to elementary school pupils in the KSA. Specifically, the lead
author emailed participants asking them to provide written answers and
reflections on the following question: “If you were asked to prepare a distance
learning-based educational lesson directed to the elementary stage on the dangers
of the emerging Corona virus pandemic, how would you accomplish that?. They
were further asked to write clear steps in line with teaching of 21st-century skills.
39
Participants were required to return their responses via email within one week
after receiving the given task. The lead author translated the journal entries from
Arabic to English, and the co-authors checked the translation. Ethical
considerations were observed, all the 20 participants provided signed consent (on
submission of their assignments) for their journals to be used for research
purposes only. One student in the class opted out of the study. All participants’
names presented in this study were pseudonyms: AB, AL, CH, DM, EH, EM, ET,
FG, HD, HN, MN, MS, NJ1, NJ2, QS, RH, RW, SD, ZN, and ZR. In addition, the
two analysts, both internal and external, who worked on the data set, did not have
a direct involvement with the study participants to maintain the objectivity in
handling the data.
4.3. Data Analysis

Upon reading the data set relative to the research question, the researchers
decided to present the major findings using mostly extracts (longer passages or
paragraphs) rather than shorter one-to-two sentence quotes (Kirklees Council,
n.d.). The whole research team read the entire data set, identified extracts deemed
especially powerful and insightful, by considering the outcomes of the content
analysis. To clarify, participants’ contributions were considered powerful if they
covered innovative ideas that reflected how they would teach their students in
such difficult times, and if they alluded to 21st-century skills. Decisions on which
extracts to present were made “typically on a case by case basis” (Corden &
Sainsbury, 2006, p. 14), with assurances that their “selection reflected strong
patterns in the data” (Lingard, 2019, p. 362).
The investigators consciously avoided presenting long extracts without

analyzing, interpreting and discussing them (Anderson, 2010). Sometimes
however, long extracts are exactly what is needed. The length of extracts staves
off the need for readers’ inference, because longer extracts can best express the
participants’ full idea (Lingard, 2019). To dissect the data set, a content analysis
was also made. It revealed how the participants structured the content of their
responses in these 10 categories: lesson preparation, digital set up,
goals/objectives, student background knowledge, learning outcomes, lesson
introduction, lesson development, consolidation, evaluation and conclusion.
Erlingsson & Brysiewicz (2017, 23) affirmed that “content analysis” is a reflective
process “of working and re-working your data that reveals connections and
relationships”.
40
Figure 1. Capturing the essence of the collected data
Figure 1 above showed the analyst-generated matrix that was used to capture and
analyse the data teasing out important details about digital resources, teaching
techniques, learning activities, sets of 21st-century skills and evaluation (Row 2).
Indeed, all what have been contributed against the categories (Column A) were
used in the content analysis and will be illustrated in the following section.
5. Findings
The content analysis of the qualitative data categories into 10 areas according to
the requirements of the journal task, i.e., writing steps in teaching 21st-century
skills, were as follows: (1) lesson preparation, (2) digital set up, (3)
goals/objectives, (4) student background knowledge, (5) learning outcomes, (6)
lesson introduction, (7) lesson development, (8) consolidation, (9) evaluation and
(10) conclusion. Table 1 illustrated what categories the participants had
addressed. It also revealed some quantitative outcomes from the data analysis.
Table 1. Responses under different categories of the content analysis

Partici- Categories #
Row pants 1 2 3 4 5 6 7 8 9 10 Clear
Steps
1 AB Yes Yes  Yes  Yes Yes  Yes Yes 8
2 AL           0
3 CH  Yes  Yes  Yes Yes  Yes  4
4 DM Yes Yes  Yes   Yes    0
5 EH    Yes  Yes Yes    4
6 EM Yes   Yes  Yes Yes  Yes  5
7 ET  Yes    Yes Yes    5
8 FG      Yes Yes    7
9 HD Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes 12
10 HN Yes Yes Yes Yes Yes Yes Yes Yes Yes  6
41
11 MS Yes Yes Yes Yes  Yes Yes Yes Yes Yes 6

12 MN    Yes Yes Yes Yes Yes  Yes 5
13 NJ1 Yes Yes    Yes Yes    10
14 NJ2  Yes  Yes  Yes Yes Yes Yes  7
15 QS Yes Yes Yes Yes  Yes Yes Yes Yes Yes 8
16 RW Yes  Yes Yes Yes Yes Yes Yes   5
17 RH Yes Yes Yes Yes Yes Yes Yes   Yes 7
18 SD Yes Yes Yes Yes x Yes Yes Yes Yes  4
19 ZN Yes Yes Yes Yes Yes Yes Yes Yes Yes  9
20 ZR Yes  Yes  Yes Yes Yes  Yes  6
COUNT *13 *13 9 *15 7 *18 *19 9 11 6 μ=
= 6.0
Statistically, each participant wrote six steps in average (μ = 6.0) in showing how
to teach 21st-century skills online. In fact, two students (AL and DM) either talked
about theories or wrote a long description as opposed to showing steps.
Conversely, HD illustrated 12 steps in incorporating 21st-century skills into her
lessons. Row 21 showed the frequency count (COUNT=) revealing that
participants collectively focused on Lesson Preparation, Digital Set Up, Students’
Background Knowledge, Lesson Introduction, and Development (see the asterisks
*).
To begin, MS pensively observed that history will remember the powerful

disruption that the Corona virus pandemic has brought to the world, and peoples’
memories will be passed down to the coming generations. The impact it has had
in the minds of one and all―pathological, psychological, and economical―and
what it has been associated with―isolation, and halt of daily-life activities―is
unpresented. The world needs time to get past this and determine what we, as
educators and teachers, are supposed to do in relation to the community and our
students.
In more depth, HN saw the importance of collaborating with parents, including

goal-setting, learning outcomes, technologies for lesson delivery, parental
support, deadlines for assignment submission, and answering parents’ queries, to
guarantee the success of distance learning. She said, “Communicating with
parents via phone or social media platforms to clarify the purpose of the lesson
will introduce to them its mechanism, and make them aware of the importance of
this in the lives of girls in the primary stage”. When planning lessons, HN pointed
out that students’ experience must be considered. The teacher needs to “make
clear the values of great loyalty to the Kingdom of Saudi Arabia and conclude the
lesson by directing students to thank God first for the existence of a rational
government that cares about us, praying for all patients, and directing them to
thank their families for offering this opportunity to meet online”.
EH affirmed the need for student-centered instruction related to the COVID-19

pandemic, the symptoms of the disease and the ways of prevention. According to
her, “at the outset, instruction would be achieved by briefly talking about the
pandemic. That is to say, the teacher will explain in details to students the
symptoms of the pandemic, and the ways of prevention by means of a video
42
concerning the right way of washing hands in order to avoid contracting the
disease, and also another video presentation on the importance of prevention
against the Corona virus. Then, I would put forward the problem to students and
try to elicit a number of solutions from them with substantiating evidence (using
the skills of problem solving, critical thinking, and communication). After this, I
would share with them the number of positive cases of the virus [in the world and
the KSA], and I would ask the question: How can we minimize these figures? I
would listen to their responses”.
AL drew on a specific theoretical framework to frame a journal entry: Sternberg’s

(1988) theory of successful intelligence, which stated that intelligent human
behaviour arose from a balance of analytical, creative and practical abilities
thereby enabling people to be successful within given socio-cultural contexts. In
this respect, she said, “The use of Sternberg’s theory would help students to know
themselves, identify their own strengths and weaknesses, rectify limitations, and
utilize their own strengths in performance improvement. According to this
theory, everyone needs the ability to think, analyze, and innovate in a scientific
way in order to develop useful 21st-century skills”.
EM, who is in favour of distance learning, described what she had done in an
online class to teach elementary students with a special focus on 21st-century
skills. She said, “At the outset, I made clear, through a video, the story of this
disease. What is it? Where did it begin? What are the best ways to avoid it? And
how can it be prevented? Here, I stimulated the students’ thinking skills through
viewing, clarifying the problem, and starting to think about solving the problem,
thus combining and linking information and then analyzing and interpreting
results”.
As far as the lesson development is concerned, EM said, “I then displayed images,

for example, ways of washing hands, mixing with people without caution,
antiseptics, gloves, masks and the like. I then divided the students into groups
and let them communicate and express their thoughts efficiently, using oral and
written communication skills among themselves through their cell phones,
because each student has her own cell phone.
For class activities, NJ1 said, “Divide the students in light of this topic into some
categories (doctor, nurse, rescuer, vaccine manufacturer, teacher, official, security
man). The following step is to listen to their ideas and what will be presented to
reduce this pandemic and address it through own profession”. ET said, “To
achieve sharing and communication among students, giving them a word to
search for its meaning on the internet to collect some information about it is
considered a wise move. To achieve self-learning, it is significant to show some
pictures that contain correct and wrong behaviours and ask pupils to critique
them to achieve critical thinking. Certainly, technology here is the main tool and
has been relied upon”.
In addition, ZR said, “Interacting through technical media with people from other
cultures benefits our pupils. Together they can identify how to reduce the
43
incidence of disease, knowing what other countries are achieving, respecting the
world’s views on dealing with diseases, and accepting differences in cultures and
new ideas to combat the disease”.
Finally, the researchers established a list of the 21st-century skills from the content
analysis that the study participants had aimed to teach their pupils. Accordingly,
the top 11 skills were: collaboration & communication, critical thinking, creative
thinking, problem solving, information sharing, presentation skills, leadership
skills & responsibility taking, research skills, respecting others’ ideas, digital
skills, and self-directed learning skills.
6. Discussion and Implications

Recent research affirmed that the abrupt shift from traditional in-person, class-
based learning to distance technology-based instruction has prompted teachers to
adapt in their role as 21st-century educators (Colaneri et al., 2020; Ferdig, 2020;
Hishan et al., 2020; Mulenga & Marbán, 2020). The same seemed to hold in Saudi
Arabia as far as the postgraduate elementary teachers in this sample frame are
concerned. A critical finding was that despite the enormity of the challenges,
participants expressed a sense of ample scope for personal and pedagogical
transformation. An analysis of the collection of powerful extracts pulled from the
overall data set revealed four common ideas around this finding.
First, threaded throughout the data was a collection of principles the participants
wanted to instil in their students. They amounted to an esprit de core or common
spirit that must be respected to sustain long-term morale, enthusiasm and
eventual success in dealing with COVID-19. Participants wanted elementary
students to embrace the idea that the country is depending on everyone’s
cooperation. Students have a duty to the nation, a responsibility to their
community and to themselves, and with the latter meaning they must take
responsibility for their own learning while learning to cooperate and value other’s
contributions.
Being the torchbearers responsible for the nation’s future is a heady aspiration for
elementary students at age 5 to 12, but it resonates with the core message of Vision
2030 (KSA, 2016, p. 72). Participants’ esprit de core seems to reflect the sentiment
that “we are each personally responsible for our own futures. As such, we will
develop ourselves and will work to become independent and active members of
society. ... We will remember our lifelong obligations to our families” (KSA, 2016,
p. 72).
Mishrif and Alabduljabbar (2018) maintained that elementary education is very

important because it helps pave the way for a nation’s success. Elementary
students must be taught how to bring out their inner talents to share responsibility
for each other and their home country. The appearance of this idea in this
exploratory study warranted further investigation especially on how pervasive it
is among all Saudi elementary teachers, and whether they are in the graduate
school or not. At first glance, it appeared that it might not be realistic for the study
participants to expect their pupils to take on such a grave responsibility in shaping
44
their nations’ future considering their tender age. Others, nonetheless, thought
this is reasonable. Abualrob (2019) claimed it is never too early to start learning
21st-century skills, which are essential for nation building. Mead (2016) also
asserted that elementary education is a powerful predictor of a child’s life
outcomes and contributions.
Second, as would be expected, participants commented on technical issues

wherein, when preparing their online COVID-19 lesson, they intended to orient
themselves and learners to the digital learning site where future learning would
take place (e.g., Zoom, Collaborate, etc.). In this concern, future research should
explore whether other Saudi female elementary educators would feel obligated to
ensure a well-run virtual classroom, or whether this was unique to this small
sample frame. Several researchers have documented the technical-related
challenges teachers have faced switching to online learning and advocated for
preparedness (O’Brien et al., 2020; König et al., 2020). It is the instructor’s
responsibility to ensure that the learning environment operated well (König et al.,
2020; Rapanta et al., 2020).
Some participants also tried to affirm for themselves that distance education is
beneficial and that students can learn this way. Other studies have recognized
similar concerns (Al-Roubaie & Alaali, 2020; Khatun & Saadat, 2020; Kovačević,
2020). Given that this shift to distance learning was immediate and imposed, it
therefore made sense that some participants sought validation of its merit and
tried to ease their misgivings. Lederman (2020) concurred that doubts about its
efficacy are natural and widespread with most educators rapidly embracing it.
Participants in this Arab-world study appeared to be following this trend.
Study participants (enrolled part-time in a master’s degree program) averaged

eight years of teaching experience and seemed to be receptive to distance learning.
Other researchers have reported that more experienced teachers are most
receptive (Lapada et al., 2020). Future studies could discern how receptive other
Saudi female elementary educators are to online learning (with a focus on work
experience differences), and whether they express doubts about its efficacy to help
very young children learn. Results from such research could inform pre-service,
postgraduate and in-service professional development initiatives.
Third, most participants commented on specific instructional strategies they

would or had already used to teach an online COVID-19 lesson. Most of them
indicated that before getting students to engage with the topic, they, as the
teachers, would prepare and share (via PowerPoints and videos) background
material on COVID-19. With this scaffolding, different instructional strategies
were shared. Van Gompel (2019) observed that self-directed, active learning and
experiential learning strategies are important. The development of student-
centered learning strategies in an unconventional learning environment such as
an online platform will need to be further examined. University teacher-training
programs, either at bachelors or masters degree level, should purposefully focus
on these instructional strategies and how to implement them via distance
45
education platforms. Since this paper placed focus on the teachers themselves,
future investigations should be made through the lens of the pupils.
Several study participants specifically commented on the need to pay attention to

evaluation strategies and to regularly communicate this aspect of the course with
students. Per MS’s comment, they favoured “student evaluation based on
participation.”In this way, we will need to understand the shift in the new
assessment method, which is unconventional in Saudi Arabia, and tailor both pre-
service teacher teaching and in-service professional development to this
methodological shift. Due to the pandemic, study participants were veering away
from the traditional rote learning pedagogy (Basham et al., 2020; Jung 2020;
Sekiyama, 2020) towards more student-centered, self-directed learning with
authentic assessment strategies (i.e., students demonstrate what they are learning
rather than just recall facts and information) and concerted efforts to communicate
this active learner participation to students. EM commented that “I have kept my
students informed… making clear the evaluation policy and its strategies.”
Fourth, several participants made a point to say their approach to teaching had
changed upon shifting to distance learning mode. EM explicitly said, “since this
crisis, I have changed my teaching methods.” Participants saw themselves as
facilitators who actively ensured consensus building amongst learners, and
reinforced the power of teamwork and collaborative, self-directed learning – in
effect, student-centered learning (Van Gompel, 2019). With this pedagogical shift,
many participants elaborated on changes in their pedagogy. Comments varied
but included: (a) a deeper respect for different learning styles; (b) the merit of the
problem-solving approach with substantiated evidence; (c) being mindful of
questions posed to students using ones that made them think with the
requirement that the teacher listen to their answers; and (d) ensuring students’
chance to learn about themselves (strengths and weaknesses) and innovate.
Further research should determine if the findings from this exploratory study
hold true for the majority of Saudi elementary teachers who also had to shift
pedagogical gears so rapidly. Further investigation, therefore, will need to show
whether Saudi elementary educators were all open to such a pedagogical change
and whether the Saudi female elementary teachers were skilfully shifting
pedagogical approaches in such a short time frame.
Many study participants reiterated the importance of elementary students being

able to learn 21st-century skills with the most frequently mentioned, including:
critical thinking, problem solving, communicating, creative thinking, innovation,
and analyzing (Abualrob, 2019; Stauffer, 2020; Van Gompel, 2019). Granted, part
of the research design protocol specifically requested participants to ensure that
elementary students can develop 21st-century skills while learning via distance
education. One cannot assume safely that teachers outside the scope of this study
will link this imperative with their efforts to teach using distance learning
platforms. Thus, it would be needed to draw their attention explicitly to such an
imperative shift. Without further investigations, it would be impossible to a
certain extent to ascertain whether learning 21st-century skills is on the radar of
46
Saudi female elementary educators other those who were in this postgraduate
sample frame.
7. Limitations
First, this exploratory study was undertaken with only 20 female postgraduate
students at one Saudi university. Future studies should draw from other Saudi
universities across the nation and solicit data from male elementary teachers as
well who comprise nearly half of Saudi educators teaching young children.
Second, the study participants were either gainfully employed or seeking
employment as elementary teachers. Future studies should focus on both years of
work experience and employment status of the participants to determine if these
variables would impact their thoughts about what is involved in preparing
lessons for online learning that ensure elementary students’ learning of 21st-
century skills. Third, there is a need for broader research using both qualitative
and quantitative research designs to best represent and cross-check teachers’
experiences in such situations.
8. Conclusion
Findings suggested that the COVID-19 pandemic has had a significant influence
on and changed the educational process for these Saudi postgraduate students
who were also gainfully employed or seeking positions as elementary educators.
The results were very encouraging. That is, despite the pandemic and national
lockdown, participants were empowered because the shift to online education
enabled them to continue their pursuit of their postgraduate studies in elementary
education. They could also apply what they had learned from the university
program into their online teaching incorporating 21st-century skills for their
pupils. The way they put pedagogical theories and teaching methodologies into
practice would benefit the younger generations who possessed adequate digital
skills. No doubt, their formative years have been indelibly shaped by a global
pandemic that has changed the world. Nevertheless, their teachers have stepped
up to the challenge taking on the mission of educating and equipping youngsters
with new, most-needed skills to thrive in the future.
If other Saudi elementary teachers had also found a similar comfort level using
online learning platforms to instil 21st-century skills, the future of elementary
education in Saudi Arabia looked promising. Participants’ comments were
thought-provoking and optimistic, reflecting their openness to using alternative
modes of learning that still meet the needs of elementary pupils. The study
participants contributed an array of digital resources, teaching techniques,
learning activities, and evaluation methods. Subsequently, a list of the 21st-
century skills that their pupils could acquire during the lesson has emerged, as a
result. This research has offered the international audience deeper insights into
the teaching practices of elementary school teachers in Saudi Arabia during the
pandemic.
47
9. References
Abualrob, M. (2019). Determinants of building 21st-century skills in Palestinian
elementary schools. Higher Education Studies Journal, 9(2), 108-116.
https://doi.org/10.5539/hes.v9n2p108
Alaboudi, A., & Alharbi, A. S. (2020). Impact of digital technology on Saudi
students. International Journal of Information Technology.
https://doi.org/10.1007/s41870-020-00451-7
Alazmy, W., Samarkandi, O., & Williams, B. (2020). The history of emergency medical
services response to mass casualty incidents in disasters, Saudi Arabia. Journal of
Emergency Medicine, Trauma and Acute Care, 2020(1), 3012.
https://doi.org/10.5339/jemtac.2020.3
Aldarhami, A., Bazaid, A. S., Althomali, O. W., & Binsaleh, N. K. (2020). Public perceptions
and commitment to social distancing during the COVID-19 pandemic: A national survey
in Saudi Arabia. https://www.preprints.org/manuscript/202005.0517/v1
Algaissi, A. A., Alharbi, N. K., Hassanain, M., & Hashem, A. M. (2020). Preparedness and
response to COVID-19 in Saudi Arabia: Building on MERS experience. Journal of
Infection and Public Health, 13(6), 1–14. https://doi.org/10.1016/j.jiph.2020.04.016
Almutairi, S. M., Gutub, A. A. A., & Al-Juaid, N. A. (2020). Motivating teachers to use
information technology in educational process within Saudi Arabia. International
Journal of Technology Enhanced Learning, 12(2), 200–217.
https://doi.org/10.1504/IJTEL.2020.106286
Al-Roubaie, A., & Alaali, M. (2020). The fourth industrial revolution: Challenges and
opportunities for the MENA Region. In M. G. Leitch & C. J. Rushton (Eds.), Joint
European-US workshop on applications of invariance in computer vision (pp. 672–682).
Springer.
Al Saidi, A. M. O., Nur, F. A., Al-Mandhari, A. S., El Rabbat, M., Hafeez, A., & Abubakar,
A. (2020). Decisive leadership is a necessity in the COVID-19 response. The Lancet,
396(10247), 295–298. https://doi.org/10.1016/S0140-6736(20)31493-8
Alshammari, R. (2020). The current use of mobile devices among students and faculty in
EFL teaching in a Saudi Arabian context. Turkish Online Journal of Educational
Technology, 19(2), 34–51.
Alyami, H. S., Orabi, M. A., Aldhabbah, F. M., Alturki, H. N., Aburas, W. I., Alfayez, A. I.,
Alharbi; A., Almasuood; R. A., & Alsuhaibani, N. A. (2020). Knowledge about
COVID-19 and patients’ beliefs about and use of herbal products during the COVID-19
pandemic: A cross-sectional study in Saudi Arabia. https://www.x-
mol.com/paper/1275880461310652416
Anderson, C. (2010). Presenting and evaluating qualitative research. American Journal of
Pharmaceutical Education, 74(8). https://doi.org/10.5688/aj7408141
Armitage, R., & Nellums, L. B. (2020). Considering inequalities in the school closure
response to COVID-19. The Lancet Global Health, 8(5).
https://doi.org/10.1016/S2214-109X(20)30116-9
Auger, K. A., Shah, S. S., Richardson, T., Hartley, D., Hall, M., Warniment, A., Timmons,
K., Bosse, D., Ferris, S. A., Brady, P. W., Schondelmeyer, A. C., & Thomson, J. E.
(2020). Association between statewide school closure and COVID-19 incidence
and mortality in the US. JAMA, 324(9), 859-870.
https://doi.org/10.1001/jama.2020.14348
Basham, J. D., Han, K., Zhang, L., & Yang, S. (2020). Considering the fourth industrial
revolution in the preparation of learners with and without disabilities. In Careers
for students with special educational needs: Perspectives on development and transitions
from the Asia-Pacific region (pp. 31-48). Springer.
48
Bayrakdar, S., & Guveli, A. (2020). Inequalities in home learning and schools’ provision of
distance teaching during school closure of COVID-19 lockdown in the UK. Institute for
Social and Economic Research Working Paper Series.
https://www.iser.essex.ac.uk/research/publications/working-
papers/iser/2020-09.pdf
Burgess, S., & Sievertsen, H. H. (2020). Schools, skills, and learning: The impact of COVID-19
on education. https://voxeu.org/article/impact-COVID-19-education
Campa, R. (2020). Fourth industrial revolution and emotional intelligence: A conceptual
and scientometric analysis. Changing Societies & Personalities, 4(1), 8-30.
http://hdl.handle.net/10995/82236
Colaneri, M., Sacchi, P., Zuccaro, V., Biscarini, S., Sachs, M., Roda, S., 1 Chiara, T.,
Valsecchi, P., Piralla, A., Seminari, E., Matteo, A. S., Maiocchi, L., Pagnucco, L.,
Tirani, M., Baldanti, F., Mojoli, F., Perlini, F., Bruno, R., & Di Matteo, A. (2020).
Clinical characteristics of corona virus disease (COVID-19): Early findings from a
teaching hospital in Pavia, North Italy. Eurosurveillance, 25(16).
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7189652/
Corden, A., & Sainsbury, R. (2006). Using verbatim quotations in reporting qualitative social
research: Researchers’ views. York, England: University of York, Social Policy
Research Unit.
https://www.york.ac.uk/inst/spru/pubs/pdf/verbquotresearch.pdf
Douglas, M., Katikireddi, S. V., Taulbut, M., McKee, M., & McCartney, G. (2020).
Mitigating the wider health effects of COVID-19 pandemic response.
https://doi.org/10.1136/bmj.m1557
Dudovskiy, J. (2016). The ultimate guide to writing a dissertation in business studies. Research
Methodology. http://research-methodology.net/about-us/ebook/
Erlingsson, C., & Brysiewics, P. (2017). A hands-on guide to doing content analysis. African
Journal of Emergency Medicine, 7(3), 93-99.
https://doi.org/10.1016/j.afjem.2017.08.001
Esposito, S., & Principi, N. (2020, May 13). School closure during the coronavirus disease
2019 (COVID-19) pandemic: An effective intervention at the global level?. JAMA
Pediatrics, 174(10), 921-922. https://doi.org/10.1001/jamapediatrics.2020.1892
Ferdig, R. E., Baumgartner, E., Hartshorne, R., Kaplan-Rakowski, R., & Mouza, C. (2020).
Teaching, technology, and teacher education during the COVID-19 pandemic: Stories
from the field. Association for the Advancement of Computing in Education.
https://www.learntechlib.org/p/216903/
Gagnon, K., Young, B., Bachman, T., Longbottom, T., Severin, R., & Walker, M. J. (2020).
Doctor of physical therapy education in a hybrid learning environment:
Reimagining the possibilities and navigating a “new normal”. Physical Therapy and
Rehabilitation Journal, 100(8), 1268-1277. https://doi.org/10.1093/ptj/pzaa096
Hackett, C., & McClendon, D. (2017). Christians remain world’s largest religious group, but
they are declining in Europe. https://www.pewresearch.org/fact-
tank/2017/04/05/christians-remain-worlds-largest-religious-group-but-they-
are-declining-in-europe/
Hilppö, J., Rainio, A., Rajala, A., & Lipponen, L. (2020). Children and the COVID-19
lockdown: From child perspectives to children’s perspectives. Cultural Praxis,
20(6), 24-32.
Hishan, S. S., Ramakrishnan, S., Qureshi, M. I., Khan, N., & Al-Kumaim, N. H. S. (2020).
Pandemic thoughts, civil infrastructure and sustainable development: Five
insights from COVID-19 across travel lenses. Journal of Talent Development and
Excellence, 12(2), 1690–1696.
49
Jung, J. (2020). The fourth industrial revolution, knowledge production and higher
education in South Korea. Journal of Higher Education Policy and Management, 42(2),
134–156. https://doi.org/10.1080/1360080X.2019.1660047
Khatun, F., & Saadat, S. Y. (2020). Fourth industrial revolution, technological advancement
and youth employment: A South Asian perspective. South Asia Economic
Journal, 21(1), 58–75. https://doi.org/10.1177/1391561420914187
Kingdom of Saudi Arabia. (2016). Vision 2030.
https://vision2030.gov.sa/sites/default/files/report/Saudi_Vision2030_EN_20
17.pdf
Kirklees Council. (n.d.). Involve: Analyzing and reporting qualitative data.
https://www.kirklees.gov.uk/involve/document/AnalysingQualitativeData.p
df
König, J., Jäger-Biela, D. J., & Glutsch, N. (2020). Adapting to online teaching during
COVID-19 school closure: Teacher education and teacher competence effects
among early career teachers in Germany. European Journal of Teacher Education,
43(4), 608-622. https://doi.org/10.1080/02619768.2020.1809650
Kovačević, A. (2020). Creativity and innovation in educating engineers and product
designers of the 21st century for the fourth industrial revolution. In Handbook of
research on integrating industry 4.0 in business and manufacturing (pp. 568–586). IGI
Global.
Lapada, A. A., Miguel, F. F., Roldan, R. D. A., & Alam, Z. F. (2020). Teachers’ Covid-19
awareness, distance learning education experiences and perceptions towards
institutional readiness and challenges. International Journal of Learning, Teaching
and Educational Research, 19(6), 127-144. https://doi.org/10.26803/ijlter.19.6.8
Lederman, D. (2020). Will shift to remote teaching be boon or bane for online learning?.
https://www.insidehighered.com/digital-learning/article/2020/03/18/most-
teaching-going-remote-will-help-or-hurt-online-learning
Lee, J. (2020). Mental health effects of school closures during COVID-19. The Lancet: Child
& Adolescent Health, 4(6). https://doi.org/10.1016/S2352-4642(20)30109-7
Lingard, L. (2019). Beyond the default colon: Effective use of quotes in qualitative research.
Perspectives on Medical Education, 8(6), 360-364. https://doi.org/10.1007/s40037-
019-00550-7
Loima, J. (2020). Socio-educational policies and Covid-19: A case study on Finland and
Sweden in spring 2020. International Journal of Education and Literacy Studies, 8(3),
59–75.
McGregor, S. L. T. (2018). Understanding and evaluating research. SAGE.
Mead, S. (2016). Don’t forget the early elementary years.
https://www.usnews.com/opinion/articles/2016-10-06/early-elementary-
education-years-are-important-for-public-policy
Mishrif, A., & Alabduljabbar, A. (2018). Quality of education and labour market in Saudi
Arabia. In A. Mishrif & Y. Al Balushi (Eds.), Economic diversification in the Gulf
Region, Volume I (pp. 97-115). Palgrave Macmillan. https://doi.org/10.1007/978-
981-10-5783-0_5
Mulenga, E. M., & Marbán, J. M. (2020). Prospective teachers’ online learning mathematics
activities in the age of COVID-19: A cluster analysis approach. EURASIA Journal
of Mathematics, Science and Technology Education, 16(9).
https://doi.org/10.29333/ejmste/8345
Nana (2020, November 10). 21st century skills for kids & rethinking how students learn.
SKIDOS. https://skidos.com/blog/21st-century-skills-for-kids-students/
Obaid, R. (2019). Women teach young boys for the first time in Saudi public schools.
https://www.arabnews.com/node/1548366/saudi-arabia
50
O’Brien, W., Adamakis, M., O’Brien, N., Onofre, M., Martins, J., Dania, A., & Costa, J.
(2020). Implications for European Physical Education Teacher Education during
the COVID-19 pandemic: A cross-institutional SWOT analysis. European Journal of
Teacher Education, 43(4), 503-522. https://doi.org/10.1080/02619768.2020.1823963
Poletti, M., & Raballo, A. (2020). Letter to the editor: Evidence on school closure and
children’s social contact: Useful for corona virus disease (COVID-19)?
Eurosurveillance, 25(17), 200-223. https://doi.org/10.2807/1560-
7917.ES.2020.25.17.2000758
Rapanta, C., Botturi, L., Goodyear, P., Guàrdia, L., & Koole, M. (2020). Online university
teaching during and after the Covid-19 crisis: Refocusing teacher presence and
learning activity. Postdigital Science and Education, 2, 923-945.
https://doi.org/10.1007/s42438-020-00155-y
Samra, H., Li, A., Soh, B., & Zain, M. A. (2020). Utilisation of hospital information systems
for medical research in Saudi Arabia: A mixed-method exploration of the views
of healthcare and IT professionals involved in hospital database management
systems. Health Information Management Journal, 49(2–3), 117–126.
https://doi.org/10.1177/1833358319847120
Sekiyama, T. (2020). The impact of the fourth industrial revolution on student mobility
from the perspective of education economics. Creative Education, 11(4), 435–447.
https://doi.org/10.4236/ce.2020.114031
Simply Learning Tuition & Consultancy. (2021). 21st century skills for children.
https://www.simplylearningtuition.co.uk/advice-for-parents/21st-century-
skills-for-children/
Sintema, E. J. (2020). E-learning and smart revision portal for Zambian primary and
secondary school learners: A digitalized virtual classroom in the COVID-19 era
and beyond. Aquademia, 4(2). https://doi.org/10.29333/aquademia/8253
Stauffer, B. (2020). Ultimate guide to teaching 21st-century skills in secondary schools.
https://info.aeseducation.com/ultimate-guide-teach-21st-century-skills
Sternberg, R. J. (1988). The triarchic mind: A new theory of human intelligence. New York:
Viking.
UNESCO Institute for Statistics. (2019). Primary education, teachers (% female) – Saudi Arabia.
The World Bank Data.
https://data.worldbank.org/indicator/SE.PRM.TCHR.FE.ZS?locations=SA
Van Gompel, K. (2019). Cultivating 21st-century skills: An exploratory case study of design
thinking as a pedagogical strategy for elementary classrooms [Doctoral dissertation,
Pepperdine University]. ProQuest Dissertations and Theses global.
Viner, R. M., Russell, S. J., Croker, H., Packer, J., Ward, J., Stansfield, & Booy, R. (2020).
School closure and management practices during corona virus outbreaks
including COVID-19: A rapid systematic review. The Lancet: Child & Adolescent
Health, 4(5), 397-404. https://doi.org/10.1016/S2352-4642(20)30095-X
Voskoglou, M. (2020). New challenges for education in the forthcoming era of the fourth
industrial revolution. In Inclusive growth in the fourth industrial revolution (pp. 98–
117). IGI Global.
Walden University. (2020). What’s the importance of early childhood education?.
https://www.waldenu.edu/online-masters-programs/ms-in-early-childhood-
studies/resource/what-is-the-importance-of-early-childhood-education
Wang, G., Zhang, Y., Zhao, J., Zhang, J., & Jiang, F. (2020). Mitigating the effects of home
confinement on children during the COVID-19 outbreak. The Lancet, 395(10228),
945–947. https://doi.org/10.1016/S0140-6736(20)30547-X
51

Vol. 20, No. 3, pp. 51-67, March 2021
English as Foreign Language Teaching in High

Schools: A Chilean Case Study
Andrew Philominraj, Ranjeeva Ranjan,

Rodrigo Arellano Saavedra and Claudio Andrés Cerón Urzúa
Universidad Católica del Maule, Talca, Chile
https://orcid.org/0000-0003-2251-7150
https://orcid.org/0000-0002-7667-2191
https://orcid.org/0000-0001-8671-3623
https://orcid.org/0000-0001-6063-7642
Abstract. In Chile, English is taught as a foreign language and its poor

performance, as per the English Proficiency Index report, is a disturbing
factor now that Chile is an official member of the Organisation for
Economic Cooperation and Development (OECD). This study seeks to
understand the dynamics of English language teaching by focusing on
the didactics, methodologies, evaluative tools, techniques and resources
commonly used in the municipal schools in the city of Talca. This case
study involves a quantitative method within the characteristics of a
descriptive, interpretative and correlational framework. The
purposefully selected participants were fifteen teachers from four
municipal schools in the city of Talca to whom an open-ended
questionnaire was distributed to collect the primary data. The secondary
data consisted of the Sistema de Medición de la Calidad de la educación
(education quality measurement system) (SIMCE) results from 2010
to 2014 and the National Study results of the English language testing in
2017. The results show that the teaching of English as a foreign language
is a difficult task because the didactics and methodologies used by the
teachers are not adequate to favour students’ reaching the level of
proficiency established by the Chilean Ministry of Education. The
findings suggest a systematic teacher training programme and
continuous focus on the teaching of the English language to enhance
learners’ language proficiency.
Keywords: didactics; English as a foreign language; methodology;

SIMCE
1. Introduction
English as a foreign language has been taught in Latin America for more than a
hundred years, mainly because of economic communication with the globalised
world (Glas, 2013; Cronquist & Fiszbein, 2017). Chile is no exception to this
reality. The substantial growth of Chile’s economy in the 1990s encouraged the
©2021 The authors and IJLTER.ORG. All rights reserved.

52
use of the English language as a tool for its competitiveness in the international
market (Castro, 2011; Albuja & Merino, 2017). By the end of the decade it had
become a compulsory subject to be taught at schools from the 5th grade as
established by the Ministry of Education (Mineduc, 2012). The aspiration of
Chile to transform the nation to the highest international standards is
challenged, especially since only 2% of working adults speak a moderate level of
English. Moreover, the results from the education quality measurement system,
namely the SIMCE regarding the English language indicate the poor quality of
English teaching in the municipal schools. One of the hypotheses to explain
these results is that the didactics and methodologies currently used are not
adequate, or that the teachers have failed to adapt them to the Chilean context.
Chile, as per the English Proficiency Index report (EF EPI), stands 37th out of 100
countries evaluated (EF EPI, 2020). This is relatively a good position in terms of
the Latin American region and indicates progress but not so encouraging at the
global level. Despite the apparent progress of Chileans in English language
proficiency, an analysis of the English SIMCE results from 2010 to 2014, the
National Study results of the English language tests of 2017, and the EF EPI
report reveal that there are still important challenges ahead. The SIMCE and the
National Study are Chilean evaluation programmes which are highly relevant
and referred to by the teachers to enhance their teaching practices. These
programmes provide essential information on language proficiency and are
carried out in a systematic process of collecting, dissecting, and interpreting
information (Loi & Hang, 2021). Given this importance, the data from the above-
mentioned years were considered since there has been no other English
proficiency assessment carried out by the Chilean Ministry of Education. The
proficiency level established by the Chilean government based on the Common
European Framework of Reference for Languages (Council of Europe, 2001) by
the end of high school ranges from A2 to B1. The Common European
Framework of Reference for Languages (CEFR) is an international standard for
describing language ability. It defines language ability on a six-point scale from
A1 for beginners, up to C2 for those who have mastered a language. The results
show that more than half of the students in Chile were below level A1,
indicating the lowest level of proficiency in the English language (Agencia de
Calidad de la Eduación, 2017).
Given such results of the English language performance, it becomes relevant to

understand the underlying concept of education as a process of the development
and empowerment of individuals, and as the enabler of efficient responses to the
new conditions imposed by globalisation. In the development of any nation,
education is a key success, and the introduction of language teaching becomes
even more important (Philominraj et al., 2020). In Chile, despite the
implementation of the English language education for so many years, the
proficiency in this language as established by the Ministry of Education has not
yet been achieved by the learners. English language proficiency enhances the
capacities and skills of learners, leading to integral development and enabling
success in language learning. Proficient learners are believed to possess a “good
command of the language”, i.e., they have the ability to comprehend the

53
language without difficulty, communicate various ideas distinctly, and interact

with other language users easily (Renandya et al., 2018).
Therefore, educational institutions must develop a new organisational culture

and innovative strategies, and design a curriculum according to the context. It is
necessary to revise the teaching practices in terms of methodology, techniques of
learning and reading comprehension, elements on which the Chilean school
curriculum is based. It is for this purpose the literature review of the study
covers educational reform, basic skills, and the didactics, methodologies, and
strategies used for teaching English for a better understanding of the whole
pedagogical process. Hence, the focus of the present research is to examine the
didactic strategies, methodological approaches, evaluative tools, techniques, and
resources employed by the teachers in urban schools in their English language
teaching process. This was carried out by surveying fifteen urban municipal
schoolteachers on their teaching practices that could approximate the reality of
the teaching of a foreign language as that of the English language in the Chilean
context (Ávila et al., 2010).
2. Educational Reform
In the past, the educational reform in Chile proposed a restructuring that
considered the main elements such as equity, equality, and quality education,
characteristics that are essential to all schools (Bellei et al., 2018). Currently the
need for learning English as a second language is on the increase, given the fact
that it is a language of international communication (McKay & Brown, 2016).
English as an international language has led ministries to formulate new
national policies for language curricula and language teacher education (Garton,
2020). The Chilean national curriculum clearly states that the English language is
considered a tool that allows global communication and at the same time
permits wider access to a knowledge of information and technology. This in turn
enables the country to face the demands of the global society (Barahona, 2015).
The first reform to the English curriculum carried out in 1998 (McKay, 2003) was
oriented towards the receptive skills of the language. Indeed, it was believed
that reinforcing receptive abilities was going to facilitate access to the
information network and thus, to the global economy. Therefore, productive
skills such as speaking and writing were relegated to a secondary role
(Barahona, 2016). Nonetheless, Ormeño (2009) states that during the 1980s,
English teachers were asked to change the traditionally known grammar-
translation method, giving more emphasis to the use and production of the
language rather than replicating it perfectly.
In 2009, and later in 2012, the English curriculum was adjusted to give more
emphasis to productive skills, ensuring the development of the four skills of the
language (Barahona, 2016). These modifications made by the Ministry of
Education in 2012 led to aligning the objectives of the curriculum according to
the CEFR to uplift the English curriculum to international standards.
The educational reforms carried out by the Ministry of Education laid the
foundation for the individual competencies towards English language

54
communication and thus considered these individualities in developing the

language learning skills. However, the development of competence is due to a
cognitive mediation that points to different competencies (Iglesias Ortuño, 2019).
3. Basic Skills
Competence is defined as the development of a set of linguistic, sociolinguistic,
pragmatic, strategic, and discursive processes and knowledge that the speaker-
listener-reader-writer must put into action to produce or understand discourses
appropriate to each context (Garcia & Fierro, 2017). Although the Chilean
educational reform has promoted some of these in its plans and programmes, it
has not been sufficient enough to achieve success and optimal management of
the same (Bellei et al., 2018; Barahona, 2015). There are different competencies
such as communicative, linguistic, literary, and metric; and the development of
each one of these has its own different strategy.
In communicative competence, for example, it is necessary to form an idiolect

with knowledge and information of pragmatics. Linguistic competence demands
the mediation of knowledge of linguistics, essentially in the text and rules of
contextualisation. To develop these competencies new learning strategies are
required. Concerning the comprehension of a text, many activities follow after
the identification of the words, such as constructing propositions and ordering
the ideas of the text, extracting the global meaning, and interrelating the ideas
globally. These operations give rise to one of the dimensions of understanding:
the textual representation or base of the text. However, the understanding also
has another dimension, namely situational representation, which entails the
construction of a model about the world or situation described by the text,
where the readers’ previous knowledge becomes useful to accommodate the
present information (Richmond & Zacks, 2017).
Understanding a text requires understanding the meaning of the text and at the
same time building a context-based model (Figueiredo, 2010). Subsequently, to
understand each sentence within the text and construct the structure to integrate
it with the knowledge that one already has means the process is divided into
two blocks; on one hand, identifying the written text, a low level process and on
the other hand the comprehension of the text, a high level process (García, 2006).
It is important to underline that reading is something artificial that does not
develop spontaneously as occurs with oral language.
Speaking and reading are the two processes with clear differences, since the
latter, besides needing to understand what is written to decode phonemes, also
requires knowing the written word and contextualising it. The faster
identification of the words results in a better understanding of the text. Reading
is related to the development of phonological awareness and socioeconomic and
cultural factors, which imply school performance. This proposition would
explain the differences in students from a low economic stratum, those who fail
to follow the reading process of the formal school programmes due to the lack of
access and are also disadvantaged in cognitive and verbal scope (Libertad &
Desarrollo, 2010). Students who have greater linguistic, verbal, and emotional

55
stimuli at an age before they enter basic education have a greater prediction of
success in the reading and writing process (Hoyos & Gallego, 2017).
Writing is the graphic representation of oral language since everything that is

spoken must appear in the written form in the process of teaching. Therefore,
the importance of written language should be emphasised as it favours the
development of the teaching process, both individually and collectively
(Davidson, 2019). The communicative activity in the classroom must produce a
dialogue between the teacher and the student that explains situations
meaningfully, both in the mother tongue and in the foreign language. In this
way, the student and the teacher can share with others their affective world,
emotions, expectations, and feelings. Thus, the learning of English becomes
relevant, emerging as an easy, pleasant, useful, and effective task (Ramírez Ortiz
& Artunduaga, 2018).
On the one hand teaching is a communicative process and, on the other, it has
the function of transmitting and developing communicative faculties. Therefore,
the teaching process requires that the English teacher should encourage
communication and avoid a monologue in which the teacher is the only one who
knows and who passively transmits knowledge without taking into account the
cognitive ability of the students with their preconceptions and concepts
(Monsalve & Correal, 2006). In order to be successful, the English teacher should
communicate in and out of the classroom, have command of the language, know
how to interpret information and ideas, as well as understanding what his or her
student wishes to communicate in another language efficiently.
The teacher should show respect for what the students say, fostering
understanding and tolerance before frustration so that the students can
participate actively. The teacher must generate a communicative environment of
familiarity which favours independence and creativity, and facilitates a dynamic
positive group. The use of appropriate verbal language is essential; hence, the
need for vocabulary and precision in ideas to express oneself effectively. The
teacher could be an excellent facilitator of learning if s/he has a good command
of oral language since s/he could very easily convert complicated information
into something simple (Ramírez Ortiz & Artunduaga, 2018).
To sum up, it could be said that facility of communication in English is vital for
success in learning and teaching the language. This means the teacher should be
capable of keeping the student motivated throughout the lesson by speaking
only in English, in an easy, entertaining, and meaningful way. This could
encourage the students to communicate in another language and ultimately
make them feel capable of expressing themselves spontaneously both in and out
of the classroom.
4. Didactics, methodologies and strategies used for teaching English

In the process of teaching-learning, curricular disciplines and content
organisation that stimulates the capacity to learn are introduced into the
educational practice. This process involves methodology, didactic strategies,

56
evaluation, techniques, and resources. The methods form the sequence of

teacher and student actions aimed at achieving the objectives of teaching. It is
essentially the method which makes the process dynamic.
To be consistent with the teaching didactic idea that is used, the methods must
be characterised by being essentially productive carried out with expository
methods; guaranteeing the active and interactive participation of students in
their search for knowledge, and helping them to find solutions to their problems.
At the same time, it should be directed towards promoting group work in
harmony with the individual as well as teaching students to learn through
developing skills of guidance, planning, supervision or control, and evaluation
(Bravo, 2011).
In the trajectory of the teaching of English as a foreign language, methods that

are of importance and which have been frequently used are the grammar
translation method, direct method, audio-lingual method, cognitive approach,
natural approach, and communicative approach.
The strategy of teaching is a description of learning environments; therefore, the

organisation of a classroom can be considered as a strategy as it would be
supported by cooperative learning. Moreover, the means and didactic resources
can be considered as products of didactic strategies, such as texts or printed
materials, without losing their category of being mediation tools (Muñoz, 2010).
Strictly speaking, a strategy is a process of execution of the subject’s ability to
learn, for which skills are chosen, coordinated, and applied. These sequences of
action are strongly oriented toward the final objective to be achieved. On the
other hand, Ruiz (2001) as cited in Urgellés and Hernández (2018) understands
that every strategy transits through a phase of obtaining information and a
phase of evolution. Strategies are designed to solve practical problems and
optimise time and resources. They also involve a planning process in which
target-oriented action sequences are established. Depending on the context in
which the strategies are targeted, they can be pedagogical, educational, didactic,
and methodological.
The didactic strategy is a result of a system of actions carried out in the short,
medium and long term that allows the transformation of the teaching-learning
process in an assignment, level, or institution based on the components of the
same and that allows the achievement of the proposed objectives at a specific
time (Vergara, 2015). The term ‘didactic’ comes from the Greek didaskein
meaning ’teach’ and teckne meaning ’art’. It is a pedagogical discipline of a
practical and normative nature the specific objective of which is the teaching
technique. This technique stimulates and guides students effectively in their
learning (Amos, 1998). In any teaching, the use of pedagogy is in the foreground,
taking into account the social aims and adapting these to new programmes and
methods of teaching. The purpose of teaching is to provide the student with the
resources of expression, comprehension, and reflection on linguistic uses
(Vergara, 2015).

57
In terms of didactic order, the starting point to define the learning process is the
theory of communicative learning, based on conflict solution, derived from the
interactions of individuals with their environment. It becomes a process that
evolves from the concept of transmission of knowledge to the concept of
generating active learners through social interactions (Martinez, 2005). From
this perspective, it is assumed that the teaching of a second or a foreign language
must favour the development of a communicative learning strategy, understood
as an active and constructive process. In addition, communicative learning is
significant in regulating, developing, and systematising (Martínez Lagares,
2015).
The following terms – methodology, didactic strategies, evaluation, techniques,

and resources – are interrelated in the exercise of educational practice. As a
whole, these items seek a quality result in the teaching-learning process in the
geographical, social, political, historical, and cultural reality of the student,
identifying the learner as an integral part of a particular social group. In the
Chilean context, there has been a proposal to strengthen the English language
teaching through several educational reforms, as seen in the introductory
section, which is also highlighted in this literature review. These aspects
discussed above indicate the lacuna, which is currently far from being addressed
in the teaching of English as a foreign language in Chilean classrooms.
5. Methodology
This case study involves a quantitative method within the characteristics of a
descriptive, interpretative, and correlational framework.
Participants
The participants were fifteen teachers from four municipal schools in the city of
Talca. The selection of the participants for this research was directed or
purposeful (Otzen & Manterola, 2017). This was done taking into account the
teachers from Marta Donoso Espejo School, Abate Molina School, Carlos
Condell School, and Diego Portales School, who were teaching English as a
foreign language in these schools. These schools are public schools managed by
the municipality and are categorised as schools of excellence by the Agency of
Quality of Education under the Ministry of Education.
Research instrument
In the current study, an open-ended questionnaire was used to collect the
primary data. This instrument was distributed to the teachers of the above-
mentioned schools. It was constructed based on the categories of the Chilean
National Curriculum (Mineduc, 2012), such as methodology, strategies,
techniques, evaluation, and resources used in the teaching of English as a
foreign language. The questionnaire was validated through a panel of experts in
the field of English language teaching methodology. The secondary data
consisted of the SIMCE results from 2010 to 2014 and the National Study English
language results in 2017.

58
Data analysis
The responses of the open-ended questionnaire were coded in an Excel file and
were categorized thematically. These themes were the independent variables
representing different categories such as methodology, strategies and
techniques. The results from the SIMCE and the National Study of English
constituted dependent variables with which the independent variables from the
survey were correlated and the data were analysed through means of the
Statistical Package for the Social Sciences (SPSS).
The objective of this study was to analyse the didactics and methodologies of the
English language teaching used by the teachers to understand the dynamics of
teaching and consequently its impact on students’ achievement in the learning of
English as a foreign language. Hence, the study aimed to describe and analyse
the meaning of teachers’ actions related to the didactic sequences (Denardi,
2017).
5. Results and Discussions

The independent variables and their percentages used by the fifteen sample
teachers are presented in the following graphs.
Figure 1: Frequency of methodology
On comparing the most used methodologies, Figure 1 indicates that with 26.7%,
the reading or oral method enjoys priority. The second most used are shared by
communicative act centred on the student, meaningful learning, and the
inductive methodology, all three with 20%. Lastly, the deduction with 13.3%
refers to exercises and examples of daily acts that require logical reasoning on
the part of the students.

59
Figure 2: Frequency of commonly used strategies
In analysing the most used strategies, the use of books, which are distributed
free to the students by the Ministry of Education, is employed by 40%of the
teachers. Group work is favoured by 26.7%, followed by 13.3% who rely on
research and dissertations.
Figure 3: Frequency of techniques
In analysing the most used techniques, 46.7% of the teachers use books for
reading purposes. These are commonly distributed to students by the Ministry
of Education for free. The technique of extraction or simplification of content
taught being used in the teaching process is favoured by 20%, while inference
and transcoding are used by 13.3% of the teachers.

60
Figure 4: Frequency of instruments of evaluation
The written test is the assessment instrument most frequently used by 53.3% of
the teachers of the municipal schools in the city of Talca. This is followed by
dissertations, written texts in the English language, and interrogation, each
employed by 13.3% of the teachers.
Figure 5: Frequency of resources
When considering the resource materials used by the English teachers from the
participating high schools, there is a tendency to use textbooks among 33.3%,
followed by multimedia projectors and the use of laptops with lectures and
techniques such as PowerPoint being used by 20% of the teachers.
The results of the English test collected from the four schools as published by the
SIMCE are shown in Figure 6:

61
SIMCE Results by School

110 109
Average SIMCE score in English 105
100
95 94
90
85
80 81
79
75
70
Liceo Abate Liceo Marta Liceo Diego Liceo Carlos
Molina Donoso Espejo Portales Condell
Series1 109 94 81 79
Figure 6: SIMCE results by school
With regard to the results obtained between dependent variables (results

obtained from the SIMCE test) and their correlations with the major relevant
independent variables of this study based on the Chilean National Curriculum
are presented in Figure 7 below.
Figure 7: Average of SIMCE English test by methodologies used
In terms of methodologies, the significant learning and the learning centred on

students indicate the highest results in the SIMCE test. The reading or oral
method implemented by the teachers (27%) in these schools has the least
relevance in the SIMCE results as evident in Figure 7. This is because there may
be a mismatch between the methodology used by the teachers for enhancing oral
and communicative production and the evaluative criteria used by SIMCE.

62
Figure 8: Average of SIMCE English test by strategies used
On comparing the results of the SIMCE test with the strategies reported by the
teachers, research was found to be statistically significant and to have the most
influence leading to good results. It is also important to note that the use of only
books in the classroom determines the lowest score in this measurement.
Table 1: Average of SIMCE tests on listening and reading to evaluation instrument

Instruments of
Evaluation Written Tests 53.3
Test Results Listening result 42.75
Reading result 47.75
On comparing the results of listening and reading skills obtained from the the
SIMCE test with one of the items from the list of evaluative instruments based
on the national curriculum, the written tests were found to be the most used
instrument by 53.3% of the teachers at school. The SIMCE test measures
listening and reading skills to determine the achievement of the English
language in students. The SIMCE does not include oral assessment which
reflects a gap in their evaluative criteria. This seems to be contrary to the
instrument of evaluation predominantly used by the teachers, which clearly
indicates the incoherence between these two types of evaluative activities,
namely the one used by the teachers to foster English language teaching and the
other employed by the SIMCE to verify the achievement of the same in students.
This proves the hypothesis of this study, namely that the teachers are not using
appropriate methodologies and strategies. They have also failed to adapt these
to the Chilean context.
This study indicates that textbooks continue to dominate as the didactic resource
most used by the high school teachers of English. In three of the five categories
taken from the national curriculum, all of three highlighted the use of textbooks
as the main source, whether for methodology, strategies, or evaluation. It is
because this tool helps the teachers to maintain order in the classroom; on the

63
other hand, it is a resource to which all have access since it is facilitated to all
students of municipal education by the Ministry of Education in Chile. Of all the
strategies used by the teachers, one could derive that communicative
competence is the least used, contrary to the literature, which states that it is one
of the most effective strategies to learn a language such as English (Ramírez
Ortiz & Artunduaga, 2018). This is because many of the teachers who teach in
the classrooms of the municipal schools do not have advanced levels of
competence in the language they teach (Mineduc, 2019). This is an impediment
in terms of correction and oral stimulation. The data on the resources used by
the teachers surveyed show the use of some technological tools such as the
multimedia projector and laptop. However, there was a lack of language
laboratories to consolidate students’ reading and listening skills.
The secondary data, which consisted of the SIMCE results from 2010-2014 and
the National Study results of the English language test of 2017, were compared
and are presented in Figure 9 and Table 2:
Figure 9: Comparison of number of students evaluated, certified and uncertified
Table 2: Percentage of certified and uncertified students

Test* Yea No. of students % of certified % of uncertified
r evaluated students students
SIMCE 2010 208,038 11.00 89.00
SIMCE 2012 186,385 17.80 82.20
SIMCE 2014 154,097 24.50 75.50
National Test 2017 7,340 32.00 68.00
* The data in the above table have been compiled from the various reports from the
Chilean Ministry of Education (Mineduc, 2010; Agencia de Calidad de la Educación,
2012, 2014, 2017).
The results of the English test carried out by the Agency for Quality of the
School Education (2017) revealed that 68% of the students evaluated were below
level A2 as defined by the CEFR. On the one hand, data from 2010 to 2017 is

64
promising, showing an increase in terms of the percentage of students obtaining

certification to prove their proficiency in the English language; on the other
hand, the same data also indicates a decline in the number of students
participating in the certification process, as highlighted in Table 2 and Figure 9
respectively. Based on the CEFR definitions of the level of proficiency, this data
clearly indicates that students in Chile are not able to understand frequently
used phrases and expressions related to areas of experience that are especially
relevant to them. Students in level A2 should be able to communicate when
carrying out simple daily tasks that require only simple and direct exchanges of
information on issues that are common to them. In addition, they should be able
to describe in simple terms aspects pertaining to their past experiences as well as
issues related to their immediate needs.
To sum up, the analysis of the above results shows that the teaching of English
as a foreign language is regarded as a difficult task because the didactics and
methodologies used by the teachers are not conducive to students’ reaching the
level of proficiency established by the Chilean Ministry of Education.
6. Conclusion
The data of this study revealed that the methodologies, which are student
centred and provide meaningful learning, are correlated with language
achievement (SIMCE results) ensuring greater success as shown in Figure 1. In
terms of strategies, research and dissertations favoured students’ achievement of
language as evident in Figure 2. Concerning evaluative tools, there is a need to
include evaluation of listening and reading skills, which were lacking in the
teachers’ responses. This may be because oral assessment is not included in the
SIMCE test, which the present study aims to highlight. Referring to resources,
the study found it was necessary to have language laboratories at schools to
strengthen the reading and listening skills, as well as oral production in
students. The data enable the conclusion that the teachers do not do specific
planning for each class; they rather use the available resources to carry out their
job as best as they can, even when this implies not having sufficient
comprehension of the didactic concept and methodology for better teaching. The
data analysis also indicates that there is a need to systematise mediation and
design planning according to the pedagogical model or the time invested. It also
highlights the necessity to create a synergy between teaching and learning
strategies, balancing the most appropriate techniques and resources to develop
activities for students that will stimulate their thinking and their
conceptualisation of thought appropriate to the learning of English as a foreign
language. Despite all the efforts made by the teachers, the research highlighted
the challenges that Chile needs to tackle in the teaching of English as a foreign
language.
7. Recommendations and limitations

The present study was limited to municipal schools in the city of Talca, and the
inclusion of private and subsidised schools for further study in other cities is
proposed. Teachers of English were the only participants and perhaps the
responses from students learning English as a foreign language could have

65
strengthened the findings of this study. The findings of the current study
recommend a systematic teacher-training programme and continuous
development in the teaching of the English language to enhance learners’
language proficiency.
8. References
Agencia de Calidad de la Educación. (2012). Informe Nacional de Resultados SIMCE.
http://archivos.agenciaeducacion.cl/biblioteca_digital_historica/resultados/20
12/result_2012.pdf
Agencia de Calidad de la Educación. (2014). Síntesis Resultados de Aprendizaje.
http://archivos.agenciaeducacion.cl/resultados-
2014/Sintesis_Resultados_IIIM_2014.pdf
Agencia de Calidad de la Educación. (2017). Informe de Resultados Estudio Nacional de
Inglés de III medio.
http://archivos.agenciaeducacion.cl/Informe_Estudio_Nacional_Ingles_III.pdf
Albuja, J.A., & Merino, L. J. (2017). Analysis of the competitive factors of businesses in
Chile. Revista Espacios, 38(29), 11.
https://www.revistaespacios.com/a17v38n29/a17v38n29p11.pdf
Amos, J. (1998). Didáctica magna. Editorial Porrúa: Mexico.
http://www.pensamientopenal.com.ar/system/files/2014/12/doctrina38864.p
df
Ávila, R., Rivero, M., & Domínguez, P. (2010). Metodología de investigación en didáctica de
las ciencias sociales. Zaragoza IFC-AUPDCS.
Barahona, M. (2015). English language teacher education in Chile: A cultural historical activity
theory perspective. Abingdon: Routledge. https://www.routledge.com/English-
Language-Teacher-Education-in-Chile-A-cultural-historical-
activity/Barahona/p/book/9780815357483
Barahona, M. (2016). Challenges and accomplishments of ELT at primary level in Chile:
Towards the aspiration of becoming a bilingual country. Education Policy
Analysis Archives, 24(82). doi.org/10.14507/epaa.24.2448
Bellei, C., Muñoz, G., Rubio, X., Alcaíno, M., Donoso, M.P., Martínez, J., De la Fuente, L.,
Del Pozo, F., & Díaz, R. (2018). Nueva educación pública. Contexto, contenidos y
perspectivas de la desmunicipalización. Santiago: Universidad de Chile.
https://www.educacionpublica.cl/wp-content/uploads/2018/11/Nueva-
Educaci%C3%B3n-P%C3%BAblica.-Contexto-contenidos-y-perspectivas-de-la-
desmunicipalizaci%C3%B3n.pdf
Bravo, L. (2011). Lenguaje escrito y dislexias: Enfoque cognitivo del retardo lector (5ªed.).
Santiago Chile: Ediciones Universidad Católica
Castro, M. (2011). Antidialogical and dialogical actions: A Freiren analysis of Chilean
educational reform and the English Opens Doors program. Paper presented at the
Conversations VII: Social and Political Philosophy, Cave Hill Philosophy
Symposium.
http://www.cavehill.uwi.edu/fhe/histphil/philosophy/chips/2011/papers/ca
stro2011.pdf
Council of Europe. (2001). The common European framework of reference for languages:
Learning, teaching and assessment. Cambridge: Cambridge University Press.
http://www.coe.int/t/dgg/linguistic/source/manual/revision-proofread-
final_en.pdf
Cronquist, K., & Fiszbein, A. (2017). English language learning in Latin America. The
dialogue leadership for the Americas. https://www.thedialogue.org/wp-

66
content/uploads/2017/09/English-Language-Learning-in-Latin-America-Final-
1.pdf.
Davidson, A. (2019). Writing: the re-construction of language. Language Sciences, 72, 134-
149. http://doi.org/10.1016/j.langsci.2018.09.004
Denardi, D. A. C. (2017). Didactic sequence: A dialetic mechanism for language teaching
and learning. Revista Brasileira de Linguística Aplicada, 17(1), 163-184.
http://doi.org/10.1590/1984-6398201610012
EF EPI 2020–Chile. (n.d.). https://www.ef.com/cl/epi/regions/latin-america/chile/
Figueiredo, D. (2010). Context, register and genre: Implications for language education.
Revista Signos. Número especial, 1, 119-141. Retrieved from
https://scielo.conicyt.cl/pdf/signos/v43s1/a08.pdf . García, I. D. (2006). Un
modelo didáctico para la orientación del proceso de construcción de textos
escritos. Varona, (42), 68-74.
https://www.redalyc.org/articulo.oa?id=360635561012
García, I. D. (2006). Un modelo didáctico para la orientación del proceso de
construcción de textos escritos. Varona, (42), 68-74.
https://www.redalyc.org/articulo.oa?id=360635561012
García Caballero, A. M. & Fierro Chong, B. M. (2017). The analytical-textual/discourse
competence: A permanent learning. Revista Atenas, 1(37).
https://www.redalyc.org/jatsRepo/4780/478055147003/html/index.html
Garton, S. (2020). “This is why students feel lost when they go into teaching practice”:
English language teachers’ views on their initial teacher education. International
Journal of Learning, Teaching and Educational Research, 19(10), 317-387.
http://doi.org/10.26803/ijlter.19.10.21
Glas, K. (2013). Teaching English in Chile: A study of teacher perceptions of their professional
identity, student motivation and pertinent learning contents. Frankfurt am Main,
Germany: Peter Lang GmbH.
Hoyos, A. & Gallego, T. (2017). Desarrollo de habilidades de comprensión lectora en
niños y niñas de la básica primaria. Revista Virtual Universidad Católica del Norte,
51, 23-45.
http://revistavirtual.ucn.edu.co/index.php/RevistaUCN/article/view/841/13
59
Iglesias Ortuño, E. d. l. Á. (2019). Diseño educativo de una actividad profesional
emergente desde las competencias cognitivas: Mediación de conflictos sociales.
Revista Educación, 43(2), 394-409. http:// doi.org/10.15517/revedu.v43i2.33063
Loi, N.V., & Hang, C.T.T. (2021). English proficiency gain and mediating factors in
training: A self-evaluation of Vietnamese pre-service EFL teachers. International
Journal of Learning, Teaching and Educational Research, 20(1), 22-47.
http://doi.org/10.26803/ijlter.20.1.2
Libertad y Desarrollo. (2010). Los proyectos de ley de la teforma educacional.
https://archivos.lyd.org/other/files_mf/tp998losproyectosdeleydelareformaed
ucacional03122010.pdf
Martínez Lagares, M. D. (2015). From the intercultural communication to the
intercultural learning in Spanish class. marcoELE Revista de didáctica ELE, 21.
https://marcoele.com/descargas/21/martinez-aprendizaje_intercultural.pdf
Martínez, M. (2005). Integración de lengua y cultura en el aula de ELE: Hacia un enfoque
intercultural. Red electrónica de didáctica del español como lengua extranjera 4.
http://www.educacionyfp.gob.es/dam/jcr:722c9fb1-87d5-4beb-a830-
6efda0a54c8a/2005-bv-04-06martinez-lafuente-pdf.pdf
McKay, S. (2003). Teaching English as an international language: The Chilean
context. ELT Journal, 57(2), 139-148. https://dx.doi.org/10.1093/elt/57.2.139

67
McKay, S.L., & Brown, J.D. (2016). Teaching and assessing EIL in local contexts around the
world. New York: Routledge.
Mineduc. (2010). Resultados Nacionales SIMCE. http://mapeal.cippec.org/wp-
content/uploads/2014/06/Informe-Nacional-Simce-2010.pdf
Mineduc. (2012). Bases curriculares. Idioma extranjero inglés. Educación básica. Santiago,
Gobierno de Chile.
http://archivos.agenciaeducacion.cl/biblioteca_digital_historica/orientacion/2
012/bases_curricularesbasica_2012.pdf
Mineduc. (2019). Agencia de Calidad entregó los resultados del Estudio Nacional de Inglés.
https://www.mineduc.cl/resultados-del-estudio-nacional-de-ingles/
Monsalve, S., & Correal, A. (2006). Children’s oral communication in English class
activities: An exploratory study. Profile Issues in Teachers` Professional
Development, 7, 131-146.
http://www.scielo.org.co/scielo.php?script=sci_arttext&pid=S1657-
07902006000100010&lng=en&tlng=en
Muñoz Oyola, J. E. (2010). Instructional materials: A platform to enhance cognitive skills
and writing development. Colombian Applied Linguistics Journal, 12(1), 27-53.
http://www.scielo.org.co/scielo.php?script=sci_arttext&pid=S0123-
46412010000100003&lng=en&tlng=en
Ormeño, V. (2009). Metacognitive awareness-raising in EFL teacher education with special
reference to a Chilean context. (Unpublished doctoral thesis, University of
Birmingham, Birmingham).
https://www.researchgate.net/publication/283961137_Metacognitive_awarene
ssraising_in_EFL_teacher_education_with_special_reference_to_a_Chilean_cont
ext?channel=doi&linkId=564a6d9908ae127ff986a233&showFulltext=true
Otzen, T., & Manterola, C. (2017). Técnicas de muestreo sobre una población a estudio.
International Journal of Morphology, 35(1), 227-232. http://doi.org/10.4067/S0717-
95022017000100037
Philominraj, A., Ranjan, R., & Diaz, M.Y. (2020). Paulo Freire y el cambio de paradigma
en la enseñanza y el aprendizaje del idioma: Un estudio de caso sobre chile-
India. Brazilian Journal of Development, 6(5), 31601-31614.
http://doi.org/10.34117/bjdv6n5-567
Ramírez Ortiz, S. M., & Artunduaga Cuéllar, M. T. (2018). Authentic tasks to foster oral
production among English as a foreign language learners. HOW Journal, 25(1),
51-68. http://doi.org/10.19183/how.25.1.362
Richmond, L.L., & Zacks, M. J. (2017). Constructing experience: Event models from
perception to action. Trends in Cognitive Sciences, 21(12), 962-980.
http://doi.org/10.1016/j.tics.2017.08.005
Renandya, W. A., Hamied, F.A., & Joko Nurkamto, J. (2018). English language
proficiency in Indonesia: Issues and prospects. The Journal of Asia TEFL, 15(3),
618-629. http://doi.org/10.18823/asiatefl.2018.15.3.4.618
Urgellés, T. H., & Hernández, K. T. (2018). Estrategia educativa para favorecer el
conocimiento de la historia local de “Montería” a través de las actividades
coordinadas y de recreación sana de la biblioteca en los alumnos de la escuela
primaria de “Ciro Frías”. Revista Atlante: Cuadernos de Educación y Desarrollo.
https://www.eumed.net/rev/atlante/2018/06/estrategia-educativa-
conocimiento.html
Vergara, M.G. (2015). Didactics, temporality and teacher education. Revista Brasileira de
Educação, 20(62), 595-617. http://doi.org/10.1590/S1413-24782015206203

68

Vol. 20, No. 3, pp. 68-83, March 2021
The Practicum in Times of Covid-19: Knowledge

Developed by Future Physical Education
Teachers in Virtual Modality
Alejandro Almonacid-Fierro, Ricardo Souza de Carvalho

and Franklin Castillo-Retamal
Universidad Católica del Maule, Talca, Chile
https://orcid.org/0000-0002-8328-017X
https://orcid.org/0000-0003-1715-9213
https://orcid.org/0000-0001-9118-4340
Manuel Almonacid Fierro

Universidad Autónoma de Chile, Talca, Chile
https://orcid.org/0000-0001-5186-4974
Abstract. The Covid-19 pandemic has put enormous pressure on the

educational system in all its teaching and learning activities, and in the
specific case of the practical training of teachers. This study seeks to
investigate through qualitative research with a comprehensive
interpretive approach, the training of the practicum of Physical Education
teachers in Chile, considering the knowledge and experiences, in the
context of virtual classrooms during the Covid-19 pandemic. Semi-
structured interviews and focus groups were used to collect the data; and
were analyzed with the content analysis technique. As result, it can be
observed that the students manifest diverse experiences in their
practicum process, with difficulties inherent to the remote work
condition, such as difficulty in communication with students and school
teachers, problems in didactic and evaluation processes, and adaptation
to context. It is necessary to discuss the access of students and teachers to
technologies, as well as the training of physical education teachers in
areas of didactic and assessment in contexts where face-to-face is difficult.
Keywords: physical education; teacher training; Covid-19, practicum
1. Introduction
On December 31, 2019, cases of pneumonia occurred in Wuhan City, Hubei
Province, People's Republic of China. At that time, the World Health Organization
(WHO) was notified to verify the recurrence of the cases. Then the transmitting
agent was identified, which is a new corona virus: SARS-CoV-2, which can cause
acute respiratory syndrome, hospitalization, and death. As of March 12, 2020, the
©Authors
69
global outbreak of SARS-CoV-2 was declared a pandemic (WHO, 2020). The

WHO indicated that, to effectively combat Covid-19 and preserve lives, countries
should isolate the population, avoid crowds, and intensify healthy hygiene habits,
such as proper handwashing, social distancing, and then confinement (Cruz et al.,
2020). Today's world is witnessing a new form of social behavior, a product of the
pandemic, since the ways of relating, consuming, work strategies, and teaching
tasks were impacted.
The scenario of the pandemic has led people to make different reflections and
review attitudes, ways of being and of relating socially, among others. One of
these reflections refers to the comfort of the classrooms and the inseparable
teacher-student relationship, since now the classes are remote or virtual (Daniel,
2020; Ferdig et al., 2020). The teaching and learning process needs, therefore, to
adapt to the new reality, and, in this sense, the initial teacher training (ITT) is of
vital importance, especially at the training instance called practicum, at which
point future teachers are deeply inserted into the school system for a whole
semester (Fazio et al., 2010). In this context, students have strong pedagogical,
emotional, and social support from collaborating teachers, trainers, and peers, in
order to achieve the implementation of new strategies in the classrooms (Hume &
Berry, 2013; Chaliès et al., 2008; Medina & Bohórquez, 2020; Nash, 2010).
Physical Education (PE) as a curricular component of education must assume the

task of introducing and integrating the student into the movement culture;
preparing the citizen to produce, reproduce and transform it. The PE teacher’s
goal is that people enjoy the game, sport, rhythmic activities, and dance,
gymnastics and physical activity, benefitting the quality of life (Bailey et al., 2009;
Kim & Ko, 2016; Lander et al., 2017). On the other hand, if the student learns the
technical and tactical fundamentals of a collective sport, he must also learn to
organize himself socially to practice it, and must understand the rules as an
element that makes the game possible. Physical Education should lead the student
to discover the motives and meanings in bodily practices, encourage the
development of positive attitudes toward them, leading to the learning of
behaviors appropriate to their practice and to the knowledge, understanding and
analysis of body culture of the movement phenomena (Benites et al., 2008).
The practicum is considered one of the most critical components of Initial Teacher
Training with a great impact on the quality of teachers (Zeichner, 2010). Despite
the value of this instance, the practicum represents a complex experience for
students, since it is experienced as an emotionally, physically, and interpersonally
demanding process with significant consequences in the development of the
teacher and in the permanence in the profession (Melville et al., 2014). Likewise, it
is possible to understand the practicum as a point of disagreement between
universities and schools, in a relationship that is often problematic and tense, that
places students in a complex scenario in which they must perform (Starkie, 2007).
During the practicum, the teacher in training is totally immersed in the reality of
the school, doing a one-semester internship in primary or secondary school. It is
the moment when the idea of the teacher as an observer, interpreter and decision-
maker, the reflective teacher, and critical reflection based on the identification of
70
the theories or assumptions that underlie teaching practice is best worked on

(Canning, 2011). The proposal is to work on the teaching projects in the school,
according to the challenges that the class can offer; therefore, the work is carried
out in agreement with the school teacher and, during the semester, the students
are required to systematize the reflections made during the process (Fazio et al.,
2010).
The teachers in initial training who attend the practicum find in this period a
meaning to the teaching profession, awakening in them a reflective and critical
attitude about what they do and how they do it and, therefore, shaping their
teaching identity, according to what was reported by Cohen et al. (2013) and
White and Forgasz (2016). In this sense, initial physical education teacher training
must be committed to the subjects that make up the school environment, since it
is where the future teacher acquires the scientific, and pedagogical knowledge
and skills necessary to face a teaching career (Castillo-Retamal et al., 2019; Rezer
& Fensterseifer, 2008; Ward & Ayvazo, 2016).
However, this training cannot be understood as offering finished products, but,

on the contrary, it is an initial phase of professional development. Among all the
components that are interconnected during the initial training of Physical
Education teachers, the process of practicum stands out as one of the fundamental
elements of this process. This curricular component is recognized as a
determining factor in the process of identity construction as a future professional
in the Physical Education area (Iaochite & Costa Filho, 2016; Zhu & liu, 2020).
In Chile, the pedagogical practices at the ITT have been the focus of discussion in
the universities, given that these training spaces have not managed to produce the
necessary changes in the training of future teachers (Vanegas & Fuentealba, 2019).
Practical training is perceived as any activity that puts the future teacher in contact
with an educational center (Canning, 2011; Fazio et al., 2010). Several studies point
out the relevance of considering that students are strongly influenced by their
biographies and by their own education, which act as frames of reference that
shape their disposition toward new teaching and learning paradigms. Teacher
training programs need to actively take responsibility for the tension that is
generated between the accumulated experience of students versus the current
demands of teaching and learning, through reflection (Melville et al., 2014; Rees
et al., 2012; Rezer & Fensterseifer, 2008; Zuljan, et al., 2011).
According to the foregoing, investigating the strategies that students use in the
formative instance of an online practicum, as a learning resource during a non-
presence situation, is extremely relevant, understanding that Physical Education
is one of the most affected subjects, as it is preferably an experiential content
(Lander et al., 2017). Physical Education in times of pandemic is a challenge, to
which teachers and future teachers must adapt and generate strategies that allow
students to learn and, consequently, remain physically active (Chen et al., 2020;
Isidori, 2020). The main objective of this study is to analyze the learning
experienced by Physical Education students in their practicum period, within the
context of the Covid-19 pandemic.
71
2. Methods
This study is ascribed to the interpretive-comprehensive paradigm (Given, 2008),
since its objective points to the construction of meanings through the
interpretation and understanding of the subjectivities of the knowledge and
experiences that Physical Education students reveal in the period of the practicum
in times of Covid-19. In general terms, qualitative research follows an
interpretative tradition because it starts from the assumption that people act in
accordance with their beliefs, perceptions, feelings, and values (Flick, 2009;
Moraes et al., 2019). The background collection techniques were the focus group
and the semi-structured interview and were chosen for their characteristics which
promote more participative, flexible and close dialogues with the participants. In
this line, Bailey and Bailey (2017) and Miles et al. (2014), point out that these
techniques favor oral expression and dialogue between collaborators and the
researcher, and with the researched, since they give the possibility of reflecting
and sharing their life experiences. The script used for the focus groups and the
interviews was validated by five academic experts, who, through an exhaustive
review, contributed to making the questions specific and focused on the objectives
set.
Three focus groups were held, consisting of a minimum of six and a maximum of
ten participants. Subsequently, six interviews were conducted, for which key
informants were called, who were selected from each of the focus groups (two per
focus group), the selection criteria being participatory, disposition and richness of
opinions. In total, 34 Physical Education students, in period of practicum, from
two universities in the Maule-Chile region participated. The inclusion criteria
were: final year Physical Education student, students who are in the period of
professional practice (practicum) and who did the practicum during the first
semester of 2020. As exclusion criterion, those students who did not finish the
period of the practicum. To proceed with the analysis of the collected antecedents,
the interviews were transcribed.
For data treatment, NVivo 10 software was used, likewise, the content analysis
method was used to extract codes and categories inductively. These were reduced
through the constant comparison method, which is limited to grounded theory
(Glaser & Strauss, 1967). It should be noted that the dimensions were obtained
from the review of the theoretical framework (Moreira & Costa, 2016).
Consequently, these fragments were brought of the data and gathered to create
categories that were defined based on some property. Later they were grouped
together with the previous categories through open coding, an issue that allows
the emergence of primary categories, giving way for a first moment descriptive
interpretive instance (Strauss & Corbin, 1990). The data are grouped under these
dimensions, categories, and codes obtained from the systematic and repeated
reading of the participants' stories. In this way, it was possible to access the
subjective experiences and the meanings attributed by Physical Education
students to the academic period of the practicum in times of the Covid-19
pandemic. All the interviews were made in Spanish, and the stories contained in
this study were translated to English.
72
3. Results
Next (Table 1) the dimensions, category, descriptions, and codes that emerged
from the analysis and interpretation of the collected antecedents are presented,
from the three focus groups and six interviews carried out with students of the
Physical Education programs from two universities in the Maule region, Chile.
Table 1. Data Systematization Matrix

Mega Previous
Description Primary Category
Category Category
Learning Category that refers to the Strategy implemented
at
Learning impact of the pandemic on Process perception
Initial Teacher’s
perception Training learning. Communication with teachers
in times of Learning and Implemented methodologies
Covid-19 Category addressing classroom
Physical Confinement and obstacles
learning in times of pandemic.
Education Class Verification of learning
Source: Own Elaboration.
The description and interpretation of the categories that were obtained from the
analysis of the interviews are presented, as well as the units of analysis (which
were extracted from the reports), to account for the category obtained. Each story
presented has a code that expresses the following nomenclature: Interview,
paragraph (para), and page (p. N). Then, to refer to them within the analysis and
in relation to the stories, it will be as follows: "Textual account extracted from the
interview".
3.1. Strategy implemented

The Covid-19 pandemic has generated a profound change in education systems
during 2020, impacting the teacher training processes in tertiary education. In the
following reports, the Physical Education students give an account of their
experiences during the practicum in terms of the strategy implemented by the
university and the educational establishments where they were assigned:
“During the Covid-19 pandemic, at the school where I did my internship
they proposed a strategy, sending homework, then a month passed and
they changed again, now sending videos, and last time, they tried to
connect with the students by Zoom. So, in the end, stress is being
generated both for the teachers and for us students in practice.” (Interview
1; para.1, p. 5).
“Personally, I did not agree from the beginning with this practice, it
seemed to me that the strategy was not adequate on the part of the
university. I felt that it was deficient in terms of the support that the
university gave us to be able to face something new, such as an online
practice. Especially in our case that we have always been used to face-to-
face, interacting with students doing activities, so it didn't seem quite
right to me. In the end I finished, but I didn't learn much.”(Interview 2;
para. 2, p. 5).
“As for the strategy that was addressed at the university, due to the fact
that the schools were in confinement, they informed us that the practice
73
would be virtual. At first, we were full of doubts and uncertainty because

my practices were always face-to-face. Now in a few weeks everything
changed and when we got to school it was the same. I never visited it. I
did not meet my students, my contact was only with my guidance teacher,
so on the one hand it was fine, we got out of the problem, but on the other
there is a feeling of emptiness ”(Interview 4; para.63; p. 13).
According to the report of the participating subjects in the focus groups and in the
individual interviews, the strategy implemented by both the university and the
educational establishment generated a good deal of uncertainty. The university
decided that the practicum would be carried out virtually, an issue that radically
changed the experiential model that was applied until 2019 in the Faculty of
Education. For their part, the guiding teacher of the educational establishments
chose to send paper homework and video capsules at the end of the semester. The
option was the development of the classes via video conference platforms.
Consequently, the strategy implemented had adverse effects on the learning of
future teachers, since students conceive the discipline as an essentially practical
area in which contact with students through bodily practices is essential. In this
scenario, the pandemic forced them to assume a different task, without presence,
without contact, without corporality, to which they had to adapt; however, they
had some objections in terms of their learning.
3.2. Process Perception.

The students of Physical Education describe their perception of the process
experienced in the first semester with respect to their insertion into the school
system through the practicum, a question that is reflected in the following stories:
“I had to adapt to the online mode, so it was difficult for everyone, for
some more than others, it was in all the college programs, not only
Physical Education. I always tried to use the best tools and everything
that the university had given me and, even so, there were many
deficiencies in the school and in the teaching procedures; at the beginning,
there was a lot of improvisation.”(Interview 14; para.13; p. 6).
“I had a similar experience to that of my classmates, I had to request

feedback and if I did not do so, they would not give it to me. I could not
understand myself with the teacher of the establishment, there was little
communication and I lacked feedback, in short, support. I'm still in the
learning stage, so I don't have a good perception of the process.”
(Interview 13; para.17; p. 7).
“Virtual education in times of pandemic does not favor learning, focusing

on the area of Physical Education. I feel that it impacts in many ways and
it also impacted us by taking classes virtually, since it does not favor us
so much, because our discipline is practical, we need to be on the patio or
in the gym,” (Interview 10; para.14; p. 6).
The interviewed students indicated that it was a difficult experience at the

beginning, because the university and the school itself were just adapting to the
virtual teaching modality; therefore, a time of improvisation was generated in the
74
administrative-pedagogical aspects, the support and feedback from the guiding

teacher was only tangential. On the other hand, and perhaps the most relevant, is
that the students expected to meet with the children in the Physical Education
classroom, on the playground and the gym, to develop the practical aspects of the
discipline, an issue that they could never experience and, consequently, their
perception of the process is not favorable.
3.3. Communication with Teachers.

As a result of the Covid-19 pandemic, a lockdown was decreed at the country
level, in such a way that the state of alarm and lockdown measures in Chile began
on March 18, 2020. On the other hand, at the beginning of April the students began
to carry out the practicum in a virtual way, in which they had the support of the
tutor (university professor) and the guiding teacher of the educational
establishment, with whom they developed different communication instances:
“My professor tutor at the university left a lot to be desired, that’s the
truth, he had never worked with the professor. I feel that he was very
absent during the process with us, the meetings we had that were counted,
arose out of our need to clarify issues related to teaching in times of
pandemic, which we did not know how to deal with” (Interview 14;
para.8; p. 6).
“From my point of view, as a result of this pandemic and remote teaching,

the initiative is essential; many teachers were unable to support us
correctly. I expected 100% of the school's guiding teacher and the truth
is, I didn't have it, because perhaps he had other things to do, so at times
I felt adrift, I did not communicate with my teacher guide and less with
the students.” (Interview 14; para.12; p. 6).
“With the guiding teacher of the establishment, it was clear with

meetings, he would summon me every two or three weeks to a meeting,
where we talked about the issues and the ways to address the content
resulting from the pandemic. It was meetings through platforms and there
he was informed about the process, the difficulties, positive aspects and
suggestions; sometimes I felt that he had as many doubts as I did, because
it was all new.” (GF 3; para. 18; p. 7).
According to the report of the students who participated in the focus groups and
in the individual interviews, communication with the tutor teacher and the
guiding teacher was sporadic and unsystematic, since it was a product of the
initial uncertainty, in terms of the duration of the period. From the state of alarm
and confinement, the schools reacted belatedly with pedagogical proposals that
incorporated all the students and, consequently, the teachers of the establishments
that were in charge of students in training were somehow separated from their
formative role. The same happened with the university tutors, who faced a new
scenario and their reaction did not measure up to the circumstances, in such a way
that the students felt themselves without pedagogical support at a fundamental
moment of their learning to become teachers.
75
3.4. Implemented Methodologies.

One of the characteristics of the educational process developed in a pandemic is
that it reconfigured the traditional way of teaching, that is, teachers had to
implement new strategies to address the contents of the national curriculum,
which were later prioritized and limited by the Ministry of Education. In this
scenario, the investigated subjects report the different methodologies that they
implemented in the Physical Education discipline:
“In my case, the school used two methodologies; a monthly guide was left
that the parents had to collect from the school all the subjects. Physical
Education had a minimum of four pages and a daily video was added to
that; the video was delivered via WhatsApp, which is more accessible, the
majority had free social networks.” (Interview 1; para.41; p. 10).
“The school acquired a platform that is a virtual classroom in the

background, and what the child does is enter this virtual classroom, and
the teacher gives him a 45-minute class on the subject. My job in terms of
the methodological in Physical Education was to deliver a guide and a
feedback video of the content we were addressing, later we would send a
video of some exercises. ”(Interview 3; para.43; p. 10).
“In my case, the school where I did my internship is quite poor, they have
internet connection problems; it is not rural but it has very limited
resources. From the outset, when we arrived, they were giving homework
guides to the parents, who were going to look for guides to school. At the
end, that was not giving results, so there we spoke with the principal, who
is a Physical Education teacher, and we looked for a way to attract more
attention from the boys, which is when we started using social
networks.“(Interview 5; para.44; p. 10).
According to the testimonies previously described, the methodology that was

used at the beginning in the preferably public and low-income schools consisted
of sending work guides, which were normally collected from the establishments
themselves and sometimes, the teachers would drop off at the children's houses.
This modality did not have the expected effect; therefore, the schools opted for
teaching through platforms such as Zoom or Google Meet, which additionally
entailed the technological problems of not having a PC or the problems of internet
connection, which were the most frequent. The students in training reported that,
during the practicum, they used social networks to address the content through
video capsules, which were sent to students by this means.
3.5. Confinement and Obstacles.

The Covid-19 pandemic revealed different obstacles; issues that normally go
unnoticed when society works with its usual way and study rhythms. In the case
of students, the school occupies a relevant space-time in the lives of families, In
general, children spend around eight hours in their schools and suddenly, the
confinement product of the need for social isolation changed this whole routine,
as shown in the following testimonies:
“The biggest obstacle that I had is that many of those who study at the
school where I did my internship are from rural areas. They travel to
76
connect with classes, and, in this case, that is why they did not attend
class sessions. Many said that they did not have the internet or a computer
where they could see their classes and do the tasks that the school asked
them to do.” (GF 2; para.82; p. 15).
“As an obstacle I mention connectivity, because the establishment

required that there be a formative evaluation. The student was required to
send the material and when we had the answers there was very low
participation, so I consider that, in the family, there is no support for
students. Because parents work or simply do not want to take care of their
children and in the absence of face-to-face school, the children are
harmed.” (Interview 19; para.87; p. 15).
“It was really difficult, quite difficult, the issue of Covid-19, and how it
affected our subject. It is something more physical, more interaction, and
with confinement everything was virtual, nothing experiential, the truth
is that I could not know if my school students did the proposed exercises,
and in what physical condition they were.”(Interview 8; para.90; p. 15).
“The confinement evidenced the enormous economic gaps in our country.

Those with greater resources can access all the facilities for a good
education from home and those who do not have to search for them to be
able to access it. Without internet at home, without personal computers,
without family support, in short, too many social differences, which makes
the social gap bigger.” (Interview 19; para.92; p. 21).
According to the report of the investigated subjects, the confinement makes

visible the great social gap that exists in Chilean society. According to the
standards of the OECD countries, Chile is one of the most unequal countries in
the world, where the difference between the poorest 10% and the richest 10% of
the population is greater than 30 times. Consequently, confinement has an impact
on the quality of education that children receive, due to the different economic
and social conditions of families, which has prevented a significant percentage of
school children in Chile from accessing virtual education and teaching,
monitoring and feedback processes, as expressed by teachers in training.
3.6. Verification of Learning.

In the framework of the previous category, Learning and Physical Education
Classroom, students in the process of professional practice reported that one of
the greatest difficulties during the pandemic was the possibility of verifying the
learning of their students, since they encountered situations such as that exposed
next:
“The teacher told us that the videos were very useful and that he gave
them to the students. However, you do not know how it came to them, or
how it worked for them, what you did was always upload. In my case, I
think that the method of planning the class through the video seemed
unnecessary since one had to pay attention to what the teacher asked only
and according to the guidance he gave us.”(Interview 11; para.15; p. 7).
77
“It was difficult to know if they were learning. I could not answer the
students' doubts, how they were receiving what I sent them and they are
things that one thinks, like the work that is being done is questioned. At
the end of this semester, I did not know how they were learning or if they
had any doubts. Those who had the possibility of uploading the video to a
platform would upload it. At the school, they had a possibility every 15
days to go to look for the material. It generates sadness because with the
material you make, you want the children to learn and clarify their doubts,
but in a pandemic, not knowing what was happening with them creates a
problem.” (GF 2; para.100; p. 17).
“No verification of learning, it was only as a mere delivery of information,

despite the fact that, in each document, in each activity, one put their
email to resolve doubts or be able to establish a closeness. The truth is I
could not verify if they learned or not and it is conditioned by the context;
if the student, for example, does not have a camera and I want to do an
exercise and for him to see it and then demonstrate it to me, it depends on
whether or not he has a telephone or a computer or internet So, from my
perspective, I think that not all learning is carried out.” (Interview 12;
para.106; p. 17).
The virtual teaching modality became widespread in the world, and in Chile as of
March, as a result of the expansion of the pandemic and the alarming numbers of
infected and deceased people. However, the educational systems were not
prepared for a task of such scale. In developing countries like Chile, which has
deficits in terms of infrastructure, equipment, teaching staff, and students who
preferably receive public education, which represents about 80% of enrollment.
Along these lines, the interviewees reported that they did not have the possibility
of contact with their students, that the response was scarce and late, that they did
not receive feedback regarding the material they prepared, an issue that generated
a feeling of disenchantment with the process.
4. Discussion
During the confinement period, the implementation of government policies
asking for the adoption of teaching strategies meant for face-to-face classes to
online and distance learning is not a guarantee that the teaching and learning
process is developed effectively. It was presented by the subjects investigated in
the previous category, "Learning at the ITT", in which they addressed the
deficiencies of the process experienced during the first semester of 2020, since the
Physical Education discipline has a strong procedural and practical exercise
connotation, as reported in the literature (Bailey et al., 2009; Fávaro et al., 2006;
Lander et al., 2017; Rezer & Fensterseifer, 2008; Voitovska & Tolochko, 2018; Ward
et al., 2015). In this line, virtual teaching for future Physical Education teachers
generated some frustration and uncertainty about the future. It was not possible
to develop a space of presence and contact with their students, an important
characteristic of the Physical Education subject. That question correlates with the
research findings of Varea & González-Calvo (2020), who determined in their
study that Physical Education teachers during the Covid-19 pandemic question
78
the purpose and identity of the discipline, and, consequently, their professional
subjectivities as Physical Education teachers.
They also expressed how much they miss the physical and direct contact with the
students and their concerns regarding the teaching of a practical subject such as
Physical Education through virtual teaching. This question generates certain
doubts and uncertainty, since, as university students, they have had to adapt to
education and learning processes with technological support (Tugun et al., 2020).
On the other hand, the contact and relationship with the university tutors and
with the guiding teachers of the respective schools was not what was required in
the circumstances of the pandemic, since when the teachers in training needed
support, guidance or even just being listened to and given feedback, the
conditions were not in place for that. The foregoing is contradictory to Chaliès et
al. (2008) Nash, (2010), Remy (2015), Medina and Bohórquez (2020) and Qadhi et
al. (2020), by highlighting the relevance of the task of the tutors and guides of
educational establishments when developing the mentoring work, in the
understanding that students in practicum, and new teachers especially, require
systematic support during their insertion in the school system.
On the other hand, and by virtue of the previous category, Learning and Physical
Education Classroom, it can be pointed out that, in recent months, the educational
situation of future educators has taken a turn due to the health crisis caused by
the Covid-19 pandemic. By losing the presence in the classrooms and with it the
essence of the practices, the formative potential of the practicum and the
professional identity of the future teachers have been altered during these
months, generating consequences that could affect their future professional
practice (González-Calvo et al., 2020; Maffei, 2014). Education on the planet was
affected as a result of Covid-19, so much so that worldwide education was
interrupted to avoid large-scale infections. In this scenario, many teachers have
had to continue with the teaching process while this was happening, creating
different ways of doing it throughout these months (Allen et al., 2020; Fernandes
et al., 2020; Kanbul et al., 2020; Mukuna & Aloka, 2020). Therefore, we find
ourselves in a context in which future teachers have to learn to navigate in times
of uncertainty and in which their teaching identity is stressed by this condition
that the pandemic has generated in the educational-training field, as presented by
the stories of the students who participated in the investigation.
In this scenario, the methodologies used by the future teachers were based on the
elaboration of guides, the creation of videos, the use of social networks and
sometimes the use of virtual platforms, due to the socioeconomic conditions of the
establishments. It is a question that is reported in the literature, by the COLEF
studies research (Basilaia & Kvavadze, 2020; Crawford et al., 2020; Ferdig et al.,
2020; Langford & Damsa, 2020; Moorhouse, 2020) the findings of which account
for the complexity of the teaching profession in times of the Covid-19 pandemic
and the various strategies implemented by countries in order to address learning
in the classroom, mainly using media teaching. Clearly, in the Physical Education
discipline, there are few possibilities to verify the learning and progress of the
students. On the other hand, the confinement consequently meant that not all
79
schoolchildren in Chile had access to the same educational conditions, evidencing

the social and economic gap in the country, harming the most vulnerable groups
in society in most cases. In this line, the works of Drane et al. (2020) and Martin &
Sorensen, (2020) point out that the current situation is exceptional; therefore, some
students will be deprived of the right to a quality education. Beyond the
seriousness of this fact, it is necessary to take the debate beyond the question of
access to resources, since, as has been pointed out, distance education (even with
all the technological, environmental and emotional resources) cannot replace face-
to-face education in the school setting by teachers. In this sense, it is worth
highlighting how the measures adopted to continue school activities at a distance
may be contributing to deprive a part of the students of the right to education,
since not all students have effective conditions to carry out school activities in a
non-face-to-face way.
5. Conclusions
As a way of maintaining the continuity of education in the most regular way
possible, distance or online education has become one of the strategies most
adopted by education systems globally. In response to the pandemic, many
schools are now contemplating a technology-based distance learning mode that
allows teachers and students to continue their classes using tools such as
videoconferencing meeting applications. UNESCO itself (2020) recommended the
use of open educational programs, platforms and applications for schools and
teachers to use to reach students remotely, while the Organization for Economic
Cooperation and Development (OECD) created a framework to guide an
educational response to the pandemic through distance learning (OECD, 2020).
The Covid-19 pandemic has brought humanity new learning: the urgency for
everyone to adapt to information and communication technologies (ICT). The
unexpected situation caused the abrupt interruption of classes, demanding quick
decisions and establishing options that are still in doubt of success. Given the new
reality imposed by Covid-19, it is necessary to question not only access to
technologies, but also the possibility of offering teachers and students conditions
to promote student learning, particularly in a discipline such as Physical
Education. In this scenario, the students who attended the practicum during the
first semester of 2020 will have to reconfigure their experience, to build learning
that allows them to face the challenge of the profession once the Covid-19
pandemic is just a bad memory.
6. Study limitations
Among the limitations presented in the study is the need to increase the sample
size, since the participants belonged to two regional universities, and it is
necessary to include students from other regions of the country. Another
important limitation is the impossibility of accompanying the subjects in the
moments of training of the practicum, synchronous classes, meetings with
teachers and supervisors among other situations
7. References
Allen, J., Rowan, L., & Singh, P. (2020). Teaching and teacher education in the time of
COVID-19. Asia-Pacific Journal of Teacher Education, 48(3), 233-236.
https://doi.org/10.1080/1359866X.2020.1752051
80
Bailey, R., Armour, K., Kirk, D., Jess, M., Pickup, I., & Sandford, R., (2009). The educational
benefits claimed for physical education and school sport: an academic review.
Research papers in education, 24(1), 1-27.
https://doi.org/10.1080/02671520701809817
Bailey, C. R., & Bailey, C. A. (2017). A guide to qualitative field research. Sage Publications.
Basilaia, G., & Kvavadze, D. (2020). Transition to online education in schools during a
SARS-CoV-2 coronavirus (COVID-19) pandemic in Georgia. Pedagogical Research,
5(4), 1-9. https://doi.org/10.29333/pr/7937
Benites, L., Souza Neto, S., & Hunger, D. (2008). O processo de constituição histórica das
diretrizes curriculares na formação de professores de Educação Física. Educação e
Pesquisa, 34(2), 343–360. https://doi.org/10.1590/S1517-97022008000200009
Canning, R. (2011). Reflecting on the Reflective Practitioner: Vocational Initial Teacher
Education in Scotland. Journal of Vocational Education and Training, 63(4), 609-617.
https://doi.org/10.1080/13636820.2011.560391
Castillo-Retamal, F., Oliveira, AAB, Souza de Carvalho, R., Castillo-Retamal, M., &
Faúndez-Casanova, C. (2019). Competency based Physical Education teacher: a
case of a chilean university. Journal of Physical Education, 30(1), e3003.
https://doi.org/10.4025/jphyseduc.v30i1.3003
Cohen, E., Hoz, R., & Kaplan, H. (2013). The practicum in preservice teacher education: a
review of empirical studies. Teaching Education, 24(4), 345-380.
https://doi.org/10.1080/10476210.2012.711815
COLEF, C. (2020). Recomendaciones docentes para una educación física escolar segura y
responsable ante la “nueva normalidad”. Minimización de riesgos de contagio de
la COVID-19 en las clases de EF para el curso 2020-2021. Revista Española de
Educación Física y Deportes. (429), 81-93.
http://www.reefd.es/index.php/reefd/article/viewFile/902/748.
Chaliès, S., Bertone, S., Flavier, E., & Durand, M. (2008). Effects of collaborative mentoring
on the articulation of training and classroom situations: A case study in the French
school system. Teaching and teacher education, 24(3), 550-563.
https://doi.org/10.1016/j.tate.2007.09.010
Chen P., Mao, L., Nassis, G. P., Harmer, P., Ainsworth, B., & Li, F. (2020). Coronavirus
disease (COVID-19): the need to maintain regular physical activity while taking
precautions. J Sport Health Sci. 9(2), 103-104.
https://doi.org/10.1016/j.jshs.2020.02.001
Crawford, J., Butler-Henderson, K., Rudolph, J., Malkawi, B., Glowatz, M., Burton, R., &
Lam, S. (2020). COVID-19: 20 countries' higher education intra-period digital
pedagogy responses. Journal of Applied Learning & Teaching, 3(1), 1-20.
https://doi.org/10.37074/jalt.2020.3.1.7
Cruz, M. P., Santos, E., Cervantes, M. V., & Juárez, M. L. (2020). COVID-19, una
emergencia de salud pública mundial. Revista Clínica Española, 221(1), 55-61.
https://doi.org/10.1016/j.rce.2020.03.001
Daniel, S. (2020). Education and the COVID-19 pandemic. Prospects, 49, 91–96 (2020).
https://doi.org/10.1007/s11125-020-09464-3
Drane, C., Vernon, L., & O’Shea, S. (2020). The impact of ‘learning at home’ on the educational
outcomes of vulnerable children in Australia during the COVID-19 pandemic. Literature
Review prepared by the National Centre for Student Equity in Higher Education,
Curtin University, Australia. https://www.ncsehe.edu.au/wp-
content/uploads/2020/04/NCSEHE_V2_Final_literaturereview-
learningathome-covid19-final_30042020.pdf
Fazio, X., Melville, W., & Bartley, A. (2010). The Problematic Nature of the Practicum: A
Key Determinant of Pre-service Teachers’ Emerging Inquiry-Based Science
81
Practices. J Sci Teacher Educ., 21, 665–681. https://doi.org/10.1007/s10972-010-

9209-9
Fávaro, P., Nascimento, G., & Soriano, J. (2006). O conteúdo da intervenção profissional
em educação física: o ponto de vista de docentes de um curso de formação
profissional. Movimento, 12(2), 199–221. https://doi.org/10.22456/1982-8918.2901
Ferdig, R. E., Baumgartner, E., Hartshorne, R., Kaplan-Rakowski, R., & Mouza, C.
(2020). Teaching, Technology, and Teacher Education during the COVID-19 Pandemic:
Stories from the Field. Association for the Advancement of Computing in Education
(AACE). https://www.learntechlib.org/p/216903/
Fernandes, S. M., Henn, L. G., & Kist, L. B. (2020). O ensino a distância: alguns
apontamentos. Research, Society and Development. 9(1).
https//doi.org/10.33448/rsd-v9i1.1551
Flick, U. (2009). An introduction to qualitative research (4th Ed.). Sage Publications.
Given, L. M. (Ed.). (2008). The Sage encyclopedia of qualitative research methods. Sage
Publications.
Glaser, B., & Strauss, A. (1967). The discovery of grounded theory: strategies for qualitative
research. Piscataway, NJ: Aldine Transaction.
González-Calvo, G., Barba-Martín, R.A., Bores-García, D., & Gallego-Lema, V. (2020).
Aprender a Ser Docente Sin Estar en las Aulas. La COVID-19 Como Amenaza al
Desarrollo Profesional del Futuro Profesorado. International and Multidisciplinary
Journal of Social Sciences, 2(9), 46-71. https://doi.org/10.17583/rimcis.2020.5783
Hume, A., & Berry, A. (2013). Enhancing the Practicum Experience for Pre-service
Chemistry Teachers Through Collaborative CoRe Design with Mentor Teachers.
Res Sci Educ., 43: 21-36. https://doi.org/10.1007/s11165-012-9346-6
Iaochite, R. T., & Costa Filho, R. A. D. (2016). Teacher efficacy beliefs during the practicum
experiences in physical education classes. Motriz: Revista de Educação Física, 22(3),
183-189. https://doi.org/10.1590/S1980-6574201600030009
Isidori, E. (2020). Sports Pedagogy at the Time of COVID-19. Cultura, Ciencia y Deporte,
15(44), 145-146. http://dx.doi.org/10.12800/ccd.v15i44.1480
Kanbul, S., Zaitseva, N., Ikonnikov, A., Kalugina, O., Savina, T., & Evgrafova, O. (2020).
Determining Expert Opinions of the Faculty of Education on the Development of
Distance Learning Course. International Journal of Emerging Technologies in Learning
(iJET), 15(23), 52-62. https://doi.org/10.3991/ijet.v15i23.18783
Kim, I., & Ko, B. (2016). Developing elementary content knowledge in physical education
teacher education. International Journal of Human Movement and Sports Sciences,
4(2), 13-19. https://doi.org/10.13189/saj.2016.040201
Langford, M., & Damsa, C. (2020). Online Teaching in the Time of COVID-19: Academic
Teachers’ Experience in Norway. Centre for Experiential Legal Learning (CELL),
University of Oslo.
Lander, N., Eather, N., Morgan, P. J., Salmon, J., & Barnett, L. M. (2017). Characteristics of
teacher training in school-based physical education interventions to improve
fundamental movement skills and/or physical activity: A systematic review.
Sports Medicine, 47(1), 135-161. https://doi.org/10.1007/s40279-016-0561-6
Maffei, W. (2014). Prática como componente curricular e estágio supervisionado na
formação de professores de educação física. Motrivivência, 26(43), 229–244.
https://doi.org/10.5007/2175-8042.2014v26n43p229
Mukuna, K. R., & Aloka, P. J. (2020). Exploring Educators’ Challenges of Online Learning
in Covid-19 at a Rural School, South Africa. International Journal of Learning,
Teaching and Educational Research, 19(10). https://doi.org/10.26803/ijlter.19.10.8
Martin, E. G., & Sorensen, L. C. (2020). Protecting the health of vulnerable children and
adolescents during COVID-19–related K-12 school closures in the US. In JAMA
82
Health Forum (Vol. 1, No. 6, pp. e200724-e200724). American Medical

Association.
Medina, S. X. B. & Bohórquez, Y. S. (2020). Modern and Postmodern Views of Education
that Shape EFL Mentoring in the Teaching Practicum. Colombian Applied
Linguistics Journal, 22(1), 55-68. http://dx.doi.org/10.14483/22487085.14576
Melville, W., Campbell, T., Fazio, X., Stefanile, A., & Tkaczyk, N. (2014). Problematizing
the Practicum to Integrate Practical Knowledge. Res Sci Educ. 44, 751–775.
https://doi.org/10.1007/s11165-014-9404-3
Miles, M. B., Huberman, A. M., & Saldana, J. (2014). Qualitative data analysis: A methods
sourcebook.
Moorhouse, B. L. (2020). Adaptations to a face-to-face initial teacher education course
‘forced’online due to the COVID-19 pandemic. Journal of Education for Teaching, 1-
3. https://doi.org/10.1080/02607476.2020.1755205
Moraes, I. Amaral, C. Arid, P., & Bastos, F. (2019). The use of qualitative research in Sport
Management: asystematic review. Journal of Physical Education and Sport, 19, 1468-
1475. https://doi.org/10.7752/jpes.2019.s4213
Moreira, A., & Costa, A. P. (2016). Introduction: Qualitative Analysis: Quantifying Quality
and Qualifying Quantity. The Qualitative Report, 21(13), 1-5.
https://nsuworks.nova.edu/tqr/vol21/iss13/1
Nash, R. (Ed.). (2010). The active mentor: Practical strategies for supporting new teachers.
Corwin Press.
OCDE. (2020). A framework to guide an education response to the COVID-19 Pandemic
of 2020. https://www.aforges.org/wp-
content/uploads/2020/04/framework.pdf
Qadhi, S., Hendawi, M., Mohammad, E. G., Ghazi, I., Al-Dosari, N., & Du, X. (2020). The
Impact of a Teacher Preparation Programs on Professional Teaching
Competencies–Female Novice Teachers’ Perspectives. International Journal of
Learning, Teaching and Educational Research, 19(1), 118-135.
Rees, C., Pardo, R., & Parker, J. (2012). Steps to Opening Scientific Inquiry: Pre-Service
Teachers’ Practicum Experiences with a New Support Framework. J Sci Teacher
Educ., 24, 475–496. https://doi.org/10.1007/s10972-012-9315-y
Remy, P. (2015). Tutoring and Mentoring: An ARC Model for Future Teachers: Affective,
Reflexive and Cognitive Orientation to Self-regulated Learning. Practice and
Theory in Systems of Education, 10(4), 323-328. https://doi.org/10.1515/ptse-2015-
0031
Rezer, R., & Fensterseifer, P. (2008). Docência em Educação Física: reflexões acerca de sua
complexidade. Pensar a prática, 11(3), 319–329.
https://doi.org/10.5216/rpp.v11i3.4960
Starkie, E. G. (2007). The practicum: An example of changes in the teaching and learning
process in the European higher education space. Odgojne Znanosti- Educational
Sciences, 9(1), 119- 134. https://hrcak.srce.hr/20832
Strauss, A., & Corbin, J. M. (1990). Basics of qualitative research: Grounded theory procedures
and techniques. SAGE Publications.
Tugun, V., Bayanova, A., Erdyneeva, K., Mashkin, N., Sakhipova, Z. & Zasova, L. (2020).
The Opinions of Technology Supported Education of University Students.
International Journal of Emerging Technologies in Learning (iJET), 15(23), 4-14.
https://doi.org/10.3991/ijet.v15i23.18779
UNESCO. (2020 UNESCO Covid-19 Education Response Education Sector Issue Notes.
UNESCO. https://en.unesco.org/covid19/educationresponse.
83
Vanegas, C., & Fuentealba, A. (2019). Identidad profesional docente, reflexión y práctica
pedagógica: Consideraciones claves para la formación de profesores. Perspectiva
Educacional, 58(1), 115-138. http://dx.doi.org/10.4151/07189729-vol.58-iss.1-
art.780
Varea, V., & González-Calvo, G. (2020). Touchless classes and absent bodies: teaching
physical education in times of Covid-19. Sport, Education and Society. 1-15.
https://doi.org/10.1080/13573322.2020.1791814
Voitovska, O., & Tolochko, S. (2018). Physical Education Teachers' Perspectives in a
Changing World: From Future Studies to New Physical Culture. Philosophy and
Cosmology, 20, 139-145. http://doi.org/10.29202/phil-cosm/20/13
Ward, P., Kim, I, Ko, B., & Li, W. (2015). Effects of Improving Teachers’ Content
Knowledge on Teaching and Student Learning in Physical Education. Research
Quarterly for Exercise and Sport, 86(2), 130–139.
https://doi.org/10.1080/02701367.2014.987908
Ward, P., & Ayvazo, S. (2016). Pedagogical content knowledge: Conceptions and findings
in physical education. Journal of Teaching in Physical Education, 35(3), 194-207.
https://doi.org/10.1123/jtpe.2016-0037
White, S., & Forgasz, R. (2016). The Practicum: The Place of Experience? In: Loughran
J., Hamilton M. (eds) International Handbook of Teacher Education, (231-266).
Springer. https://doi.org/10.1007/978-981-10-0366-0_6
World Health Organization. (2020). Coronavirus disease (COVID-19) situation report–102.
(2020). https://www.who.int/docs/default-
source/coronaviruse/situationreports/
Zeichner, K. (2010). Rethinking the connections between campus courses and field
experiences in college and university-based teacher education programs. Journal
of Teacher Education, 89(11), 89-99. https://doi.org/10.1177/0022487109347671
Zhu, X., & Liu, J. (2020). Education in and After Covid-19: Immediate Responses and Long-
Term Visions. Postdigital Science and Education, 1-5.
https://doi.org/10.1007/s42438-020-00126-3
Zuljan, M. V., Zuljan, D., & Pavlin, S. (2011). Towards improvements in teachers’
professional development through the reflective learning paradigm–the case of
Slovenia. Hacettepe Üniversitesi Eğitim Fakültesi Dergisi, 41(41), 485-497.
https://dergipark.org.tr/en/download/article-file/87421
84

Vol. 20, No. 3, pp. 84-96, March 2021
Challenges to Online Engineering Education

during the Covid-19 Pandemic in Eastern
Visayas, Philippines
Perante Wenceslao
Eastern Visayas State University, Philippines
https://orcid.org/0000-0002-1511-2430
Gomba Felisa
Samar State University, Philippines
https://orcid.org/0000-0001-8744-148X
Abstract. The Covid-19 pandemic halted the normal operations of schools

and universities around the world. Various educational institutions,
including the Philippines Commission on Higher Education (CHED),
adapted to the limitations caused by the pandemic and recommended
online education to reach students and learners remotely. This research
investigated the challenges to online engineering education in higher
educational institutions (HEIs) (n=4) in Eastern Visayas, Philippines.
Challenges and problems experienced by faculty (n=25) and students
(n=421) were gathered through an online survey on December 2020 using
Google Forms. Findings indicate that 98% of the respondents were
gadget ready for online education for the first semester of SY 2020-2021,
and the primary type implemented is Online Education Only (OEO)
(n=369). The majority of the respondents (94%) believed that the quality
of education suffered from the sudden shift to online education and 64%
believed it is not as effective as the traditional face-to-face classroom
interaction. Post Covid-19, 60% of the faculty prefers Blended Education
(BE); while the students (65%) prefer the traditional classroom face-to-
face interaction. The challenges faced by the respondents during the
Covid-19 analysed through qualitative content analysis can be
categorised into Personal Challenges, Limited Social Interaction,
Technology Difficulties, Assessment Issues, and Concerns on Learning
Materials and Methods.
Keywords: coronavirus pandemic; Covid-19; online engineering

education challenges; Philippines
1. Introduction
Covid-19 was first reported in Wuhan, China in late 2019 (Huang et al., 2020;
Mackenzie & Smith, 2020; Wu, Yi et al., 2020) and since then it has spread all over
©Authors
85
the world. On December 1, 2020 cases of Covid-19 had cumulative numbers of

over 61.8 million reported cases and 1.4 million deaths globally. In the Philippines,
as of December 7, 2020, there have been 439,834 confirmed cases with 8,554 deaths
(WHO, 2020).
The pandemic disrupted the way of life all over the world. To contain the spread
of the virus, policies such as sheltering, social distancing, washing of hands,
wearing a face mask and quarantine procedures were recommended as a standard
practice (Adedoyin & Soykan, 2020; Radha et al., 2020; Rajab et al., 2020;
Suryaman et al., 2020); further, limited allowable groupings made traditional face-
to-face education not feasible. This resulted in the transformation of the way
education was delivered, from the classroom in-person interaction to online
education through the internet (Machado et al, 2020; Radha et al., 2020; Suryaman
et al., 2020). Educational institutions all over the world, including the Philippines
Commission on Higher Education (CHED, 2020), adapted to the limitations
prompted by the pandemic and recommended online learning education to reach
students and learners safely and remotely. Online learning during the pandemic
had been supported and implemented in many countries around the world
(Goldschmidt, 2020).
Online learning is the use of the internet and related technologies and devices to
develop, deliver and manage education programmes (Fry, 2001; Means et al.,
2009). The advances in technological innovation and the improvement in internet
speed and accessibility have increased the initiatives and programs for online
learning in the past couple of decades (Tallent-Runnels et al., 2006). The use of
hardware like desktops, laptops, smartphones and various software applications
such as online videos, email, video conferencing, chatrooms and learning
management systems are some of the primary tools used in online education.
Online education is not new, it has been a part of educational tools in many
educational institutions in many countries (Allen & Seaman, 2008; Larreamendy-
Joerns & Leinhardt, 2006), but the Covid-19 pandemic accelerated its widespread,
if not obligatory use for lack of other options. Many colleges and universities had
been implementing a transition from traditional face-to-face teaching to online
learning, and to a combination of online and traditional education called blended
learning before Covid-19 (Bonk & Graham, 2012).
Online learning has its advantages, such as flexibility in study time and location
(O'Donoghue et al., 2004; Smedley, 2010), interactivity (Leszczyński et al., 2018;
Wagner et al., 2008), self-pacing (Palaigeorgiou & Papadopoulou, 2018),
accessibility and cost savings ((O'Donoghue et al., 2004). It also has many issues,
concerns, and challenges (Dumford & Miller, 2018) that have been made apparent
during the Covid-19 pandemic, for example, there are concerns about the quality
of education resulting from the sudden shift to online education (Hodges et al.,
2020). The adequately planned online education and learning that is characterised
with quality could be different from the courses presented online as a response to
a crisis, i.e., Covid-19 “emergency remote teaching” (Hodges et al., 2020).
86
Another concern is the unsuitability of certain courses for online education

(Machado et al., 2020). The compatibility of online learning with social science and
humanities has been proved effective (Leszczyński et al., 2018); however, other
researchers have disputed its compatibility with sports sciences, engineering and
medical sciences; this is because in-person practical experiences and hands-on
instructional activities are a required and essential part for these courses
(Leszczyński et al., 2018; Machado et al., 2020).
This study was undertaken to investigate the concerns and challenges faced by
students and faculty in online engineering education in Eastern Visayas,
Philippines.
2. Methodology
This study was conducted in Eastern Visayas Philippines (Figure 1). Eastern
Visayas consists of three main islands: Samar, Leyte and Biliran, and has a
population of 4,440,150 inhabitants as of 2015 (PSA, 2015). Eastern Visayas is home
to several state universities: University of Eastern Philippines (UEP), Visayas State
University (VSU), Eastern Visayas State University (EVSU), Southern Leyte State
University (SLSU), Naval State University (NSU), Eastern Samar State University
(ESSU), and Samar State University (SSU).
Figure 1. Location of the HEIs in Eastern Visayas, Philippines that participated in the
survey: EVSU (n=308), ESSU (n=80), SSU (n=40), and NSU (n=18).
The researchers developed an online survey questionnaire using Google Forms®

(Google LLC, Mountain View, CA). The questions were composed of three parts:
(1) general information of the respondents, (2) pre-Covid-19 readiness for online
education, and (3) experiences and challenges during the online education
87
programme implemented during the first semester (September to December 2020)

of the school year 2020-2021. The questions were a combination of closed and
open-ended queries to provide flexibility and to generate descriptive answers
from respondents. The questionnaire was validated by inviting three faculty and
three students to provide feedback and comments on the survey questions. After
the comments were incorporated into the questionnaire; it was pilot tested to
prospective respondents (students, n=21; faculty, n=10) to determine the
questionnaire’s internal consistency and the calculated Cronbach’s alpha value
was 0.776.
The questionnaire administration to respondents was conducted online and to

obtain a large number of responses, an email introducing the research and the web
address of the Google Forms survey was sent to HEIs with engineering courses in
Eastern Visayas inviting them to participate in the survey. Before the respondents
filled the survey properly, they were first informed of its purpose and how their
responses will be treated with confidentiality. The survey questionnaire can be
accessed through this link: https://forms.gle/X6L1yahVhksBru419
Data generated from the survey were processed into categories based on
profession: student or faculty; and according to time frame: pre and during Covid-
19. The results of the open-ended questions were processed and analysed by
coding using quantitative content analysis (Züll, 2016). Data analysis employed
descriptive statistics using the software SPSS Version 21.0; while the map was
created using QGIS Desktop 3.16.5.
3. Results
3.1 Respondents Profile
The survey generated 446 responses from December 1 to 20, 2020. The faculty
respondents were composed of females (n=9) and males (n=16); having a mean
age of 35.88 years old, with the oldest at 59 and the youngest at 24 years old; the
mean teaching experience was 8.12 years, with the highest at 32 years and lowest
at one year. Conversely, the students’ respondents were composed of females
(n=187) and males (n=234), with a mean age of 20.3 years old (Table 1). Further,
most of the student respondents were Bachelor of Science in Civil Engineering
(BSCE) students (n=379), Bachelor of Science in Electrical Engineering (n=21), and
other Engineering Courses (n=21). The respondents’ distribution in terms of HEIs
were EVSU (n=308), ESSU (n=80), SSU (n=40, and NSU (n=18); while the year
level is fairly distributed as follows: 1st year (n=119), 2nd year (n=123), 3rd year
(n=105), and 4th year (n=74).
3.2 Gadget and Internet Connection

The majority (98%) of the respondents owned a smartphone, and 32.3 % have a
laptop computer, 2.9% owned a Tablet, while the others have a desktop (n=7).
Before Covid-19, 35.3% had a home internet connection, while 61.4% had no home
internet connection. However, this changed during the pandemic, home internet
connection increased to 70.1%, while 29.9% use mobile data to connect to the
internet.
88
Table 1: The respondents’ demographic profile

Demographic Characteristic Frequency (n=446) Percent
Gender Male 250 56.2%
Female 196 43.8%
Profession Faculty 25 5.6%
PhD. 3 1%
Masters 15 4%
MS Student 7 2%
Student 421 94.4%
BSCE 379 85%
BSEE 21 5%
Others 21 5%
HEI EVSU 308 69%
ESSU 80 18%
SSU 40 9%
NSU 18 4%
Age Mean: 21.2 years old 446 100%
< 19 years old 166 37%
20 to 21 years old 180 40%
22 to 30 years old 82 18%
> 31 years old 18 4%
Student Year Level 1st Year 119 27%
2nd Year 123 28%
3rd Year 105 24%
4th Year 74 17%
3.3 Respondents’ Perceptions on Online Education

The majority of the respondents (99%) reported that their HEI is implementing
online education for the first semester of the school year 2020-2021. Most of the
respondents (82.6%) were taking Online Education Only (OEO) (no personal face-
to-face interaction); while 8.7% of the respondents reported Blended Education
(BE) (a combination of online and personal face-to face-interaction); and the rest
(n=35) reported Modular Online Education (MOE) (a combination of printed
modules and online).
Table 2: The respondents’ perceptions of online education (OE) and preference of

education delivery post Covid-19.
Preference of OE Delivery
n Online only Blended Face-to-Face only
Post Covid-19
Faculty 25 8% 60% 32%
Students 421 3% 32% 65%
Consequence of OE n Yes No Somewhat
Education Quality Declined? 444 94% 4% 2%
Is Online Education Effective? 441 13% 64% 22%
OE Effect on Poor Fair Good Very Good
Quality of Teaching (n=25) 4% 12% 72% 12%
Quality of Learning (n=421) 12% 39% 42% 6%
Most of the respondents (94%) believed that the quality of education declined due
to the sudden shift to online education and 64% believed it is not as effective as
the traditional face-to-face interaction (Table 2).
89
Table 3: The challenges, problems, and difficulties faced by the respondents in online
education during the Covid-19 Pandemic in Eastern Visayas, Philippines.
Category Sub-Category Responses Percent
(n=446) Response
Time-Management 338 76%
Personal Pandemic Related Health Issues: Anxiety, 300 67%
Challenges Stress, Backpain
Lack of Self-Discipline 165 37%
Home Environment & Distractions 140 31%
Lack of Motivation 119 26%
Financial Limitations (Money for Internet 33 7%
Connection)
Limited Social Lack of Face-to-Face Interaction 231 52%
Interaction Lack of Student Community or Campus 67 15%
Environment
Lack of Student Group Activity & Sharing 49 11%
Limited or Lack of Instant Feedback from 36 8%
Teachers
Technology Slow Internet Speed 411 92%
Difficulties Communication Problems (Audio & Video) 271 60%
Unfamiliarity with New Technology or New 238 53%
Software Applications
Learning Curve Too High 147 33%
Lagging Connection or Lack of Signal 141 32%
Coverage
Power Outage 32 7%
HEIs Poor Technology Infrastructure 29 6%
Assessment Student Assessment Limited to Multiple 200 45%
Issues Choice and Essay
Easy to Cheat on Online Exam 78 17%
Poor Quality of Student Assessment 33 7%
Concerns on Higher than Normal Assignments, Tasks, 188 42%
Learning and Quizzes
Materials and No Actual Laboratory Activities and 152 34%
Methods Experiments; More on Theory
Lack of Hands-on, Real Life Exposure, Field 148 33%
Experience and Trips
Some Lessons are Difficult to Understand 69 15%
Without In-person Lecture
Limited Online Meetings; Low to No 36 8%
Guidance from the Teachers
3.4 Challenges to Online Engineering Education

The respondents were asked open-ended questions about the challenges,
problems, limitations and difficulties with online education. The responses were
analysed qualitatively by coding the frequencies into categories and sub-
categories. The top challenges identified in each category were as follows: Time-
90
Management (n=338) had the most responses in the category of personal

challenges; Lack of Face-to-Face Interaction (n=231) for the category of Limited
Social Interaction; Slow Internet Speed (n=411), for Technology Difficulties;
Student Assessment-Limited to Multiple Choice and Essay (n=200) for
Assessment Issues; and Higher than Normal Assignments, Tasks, and
Quizzes(n=188) for Concerns on Learning Materials and Methods (Table 3).
4. Discussion
The majority (99%) of respondents were involved in online education for the first
semester of SY 2020-2021 and most (82.6%) are into Online Education Only (OEO),
which is understandable because of the fear of being infected with Covid-19. An
OEO approach may not be the most appropriate given that most of the
respondents (50%) had no online education experience, but, for lack of other
options, perhaps, this is reasonable. Even so, there are concerns about the quality
of education ensuing from this hasty shift to online learning (Hodges et al., 2020).
The sudden shift to online education at a large scale on short notice would
certainly face difficulties and problems. A particular concern in developing
countries like the Philippines is the poor internet infrastructure and the frequency
of power interruption. Likewise, HEIs have many concerns including: the need to
adjust or revise the curriculum to suit to online teaching environment; the need
for appropriate logistics (software, hardware, learning management systems and
others); the need for training of faculty and staff (skills on online teaching
including creation and development of online teaching materials); the need to
revise student assessments, tasks and approaches; and the need to adjust the
length of the school semester to give ample time for preparations for all
stakeholders, to name but a few. These concerns cannot be suddenly addressed
given the limited time and resources of most of the HEIs; nevertheless, the
Philippines started the first semester on August 17, 2020.
Faculty respondents perceived the impact of online education to quality of

teaching as Good (72%). While the students believe the impact of online education
to quality of learning is Fair (39%) and Good (42%). Additionally, the education
delivery preference post Covid-19 for faculty respondents (60%) is Blended
Education (BE); while 65% of the students prefer the traditional classroom face-
to-face interaction (FTFI).
Furthermore, the challenges faced by the respondents during the pandemic (Table
3) can be grouped into five categories: Personal Challenges, Limited Social
Interaction, Technology Difficulties, Assessment Issues, and Concerns on
Learning Materials and Methods. These concerns, issues, and challenges are
discussed below; possible methods on how to address these challenges and how
to improve the current online education are presented.
(1) Personal challenges identified by the respondents are poor time-management,

increased stress, lack of self-discipline, lack of motivation and lack of financial
resources for internet connection. These challenges are also common in
traditional classroom education, but probably a bigger concern in an online
education setting. The OEO provides a higher level of flexibility and autonomy
91
to students and can be characterised as an online self-directed learning approach

where students take responsibility for their time, their learning needs, their
learning goals, their learning strategies, the sequence, pace, and the amount of
content to be learned (Kebritchi et al., 2017) making motivation and self-discipline
vital in the students’ learning process as motivation for learning in an online
setting is critical in the success of learners and in increasing learners’ retention
(Kebritchi et al., 2017; Saade et al, 2017). The HEIs maybe can address these
concerns by offering a programme to help students in developing strong attitudes
and dispositions towards online learning, i.e., improving self-discipline,
cultivating motivation, and learning time-management skills. Also, various forms
of support from families, teachers and schoolmates can be explored to help
maximise the students’ ability to learn through online education (OECD, 2020).
Further, to address the students’ financial concerns, the HEIs and the government
should provide financial aid to students. particularly for internet connection fees
because, if this is not addressed, this could lead to some students being left behind.
(2) Limited social interaction such as lack of face-to-face meetings, lack of campus
atmosphere and group activities, limited or lack of instant feedback from teachers
are the challenges in this category. If the OEO approach is continued for some
time, then maybe HEIs should develop methods and approaches to increase
online interaction through online engagement activities such as the use of
interactive and dynamic multimedia content and games. HEIs should also
conduct training for teachers to effectively use online tools to improve student
engagement and participation, including adopting pedagogical teaching practices
suited to online context; or the use of innovative teaching methods to engage
students and to stimulate their proactive behaviour (Ferri et al., 2020) and help
improve online social interaction between and among teachers and students.
Otherwise, this concern can easily be mitigated by shifting to a BE approach,
which provides a limited face-to face-interaction. This study recognises that there
is no substitute for teacher-student interaction, which is vital in the learning
process of students particularly in the primary and secondary levels; however, to
mitigate problems of inclusion, other studies suggested using a BE approach
(Ferri et al., 2020), whenever possible, particularly at the tertiary level.
(3) Technology difficulties like unfamiliarity with new software applications, slow
internet speed, poor signal coverage, audio and video communication problems,
and HEIs’ poor technology infrastructure are the challenges in this category. The
sudden shift to OEO was a new experience for most teachers and students; and
this caught teachers unprepared and struggling with unfamiliar teaching methods
and software applications, forcing them to employ trial-and-error approaches
resulting in a minimalist usage of what software and online tools are capable of
providing (Do, 2020). However, these challenges can be addressed by HEIs
through online training on new software applications (particularly its various
functionalities and its full potential for use in OE) for faculty and students as it is
important that both have knowledge about using communication technology
platforms, not only for face-to-face teaching, but, importantly, for BE and OEO
(Ogbonnaya et al., 2020). Further, HEIs should allocate a budget for technology
hardware and software to improve its poor technology infrastructure (i.e., limited
92
server capacity, limited upload memory, and Learning Management System

failure to handle heavy traffic during peak hours). HEIs should recognise that,
now, and in the future, Information and Communications Technology (ICT)
infrastructure, as well as its software support system, is going to be an integral
part of teacher-student interaction and thus should be included in its budget
allocations. On the concern of slow internet speed and poor signal coverage, this
is a technological issue and challenges challenge connected with governance and
policies related to economic development and technology adoption, not only in
the Philippines but also in many other countries as well. Thus, governments
together with development agencies and internet service providers should
address the slow internet speed and poor signal coverage to minimise the existing
digital divide.
(4) Assessment challenges such as limited methods of student assessment, easy to

cheat on online exams, and poor quality of assessment are some of the challenges
identified by the respondents. These problems are not surprising since most HEIs
have no clear policy and guidelines on online teaching and more so on online
assessment (Guangul et al., 2020). In HEIs, assessments are used to (1) support
learning, (2) to execute accountability, and (3) to provide certification, progress
and transfer (Capsim, 2020). It is also aimed to determine competence, gaps and
progress so learners can adapt their learning strategies and teachers their teaching
strategies (Guangul et al., 2020). These assessment concerns can be attributed to
the lack of preparation and inexperience in online education by most HEIs,
teachers and students in Eastern Visayas. However, now that both students and
faculty have knowledge and experience on online education, these issues can
easily be addressed through various creative ways, for example, on the issue that
it is easy to cheat on online exams, certain software can be used for remote
assessment that uses webcams to track student activity during an exam and can
warn or assist the teacher whenever suspicious actions are taken by the students
(Rutgers, 2020). If Google Forms are used as an assessment tool, it could be
partnered with a “timer and proctor” add-on that time limits the exam, and
provides proctoring through a camera that records the student's actions while
taking the exam.
(5) Concerns on learning materials and methods included higher than normal
assignments, tasks and quizzes; no actual laboratory activities and experiments;
teaching approach is more on theory and lacks hands-on real-life exposure and
field experience; and minimal to no guidance from the teachers. In another study
(Armstrong-Mensah et al., 2020), students reported that the transition to online
classes increased their academic workload in the form of written reports and
assignments, reflection papers, quizzes and discussion posts. This increase in
student workload could be the teacher’s way to compensate for the lack of face-
to-face interaction and to ensure that the students are learning the subject. To
address the lack of laboratory and hands-on experience, perhaps HEIs should
train teachers to use digital interactive multimedia content, including digital
laboratories and other specific online skills to supplement the need for
experiments and hands-on learning (Ferri et al., 2020). Also, OEO may not be
suitable for courses like engineering where in-person experiences and hands-on
93
instructional lessons are an essential part of learning; the HEIs should consider
this nature of the course.
Further, this study opines that, given the challenges, concerns and issues
discussed above, the best approach to education delivery post Covid-19 is,
perhaps, BE for two main reasons: Firstly, most of the challenges identified above
can be fully, if not, partially addressed through a BE approach. These issues were
all the components of (2) limited social interaction, (4) assessment challenges, and
(5) concerns on learning materials and methods; furthermore, while faculty and
students are transitioning and learning, new software applications and online
tools as identified in (3) as technology difficulties; and while the students are in
the process of learning, motivation development and self-discipline skills, as
identified in (1) are personal challenges; a BE approach can, possibly, supplement
and hasten the learning process of both faculty and students. Secondly, a BE
approach is more efficient, which helps in saving time, money and resources for
all stakeholders compared to an FTFI mode of education delivery. To cite an
example, with reduced FTFI, both faculty and students have more time for other
tasks, possibly resulting in reduced travel (and related expenses) due to a reduced
frequency of travel to and from the campus, not to mention the reduced carbon
footprint related to less frequent FTFI activities.
In summary, this study believes that online education could become a permanent
part of HEIs mode of education delivery in Eastern Visayas. The reasons are: HEIs
have invested in Learning Management Systems; modules and online education
materials have been developed; the respondents have acquired experience, new
skills and knowledge on online education; and the faculty believes that online
education had a good impact on their quality of teaching, thus most faculty
respondents prefer a BE education delivery post Covid-19.
5. Conclusion and Recommendations

The Covid-19 pandemic has frozen the normal operations of schools and
universities in the Philippines. The Commission on Higher Education adapted to
the limitations caused by the pandemic and endorsed online education for the SY
2020-2021; as a result, students and faculty faced various challenges and issues
related to online education (Table 3). This study, thus, recommends the following
to address these issues and challenges: (1) HEIs should offer a programme to help
students develop strong attitudes towards online learning particularly focused on
improving self-discipline, cultivating motivation and developing time-
management skills; (2) HEIs and the government should provide financial aid to
students to cover internet connection fees so as to avoid some students being left
behind; (3) HEIs should develop methods and approaches to increase online
interaction, including the use of interactive and dynamic multimedia content and
games, among other methods; (4) HEIs should conduct trainings for teachers to
effectively use online tools to improve student engagement and participation,
including adopting pedagogical teaching practices suited to online context; (5)
HEIs should provide online training on new software applications used in online
education to faculty and students as it is important that they have knowledge
about using communication technology platforms; (6) HEIs should allocate a
94
budget for technology hardware and software to improve its internet-related

communications infrastructure; (7) The government should encourage the
internet service providers to improve the slow internet speed and poor signal
coverage in many regions of the country; (8) CHED and the HEIs should adjust or
maybe modify the curriculum to suit to the online teaching environment,
including revising student assessment approaches and methodologies.
This research can serve as reference to future studies related to Covid-19 and its
effects on online education. This can also provide policy makers, education
administrators and related agencies and stakeholders guidance on how to address
the common challenges faced by students and faculty with regard to online
education.
6. Limitations
This study was constrained in terms of the number of respondents (4 out of 7 HEIs
in Eastern Visayas had respondents), and the non-random selection of
participants as the study relied on an online survey due to the Covid-19
pandemic-imposed limitations. This study suggests that future researches should
endeavour to cover all the HEIs in Eastern Visayas and should employ a random
selection of survey participants. Additionally, a separate future study may be
needed to validate the perceptions of the respondents on the effectiveness of
online education.
Funding: This research was funded by the Samar State University (SSU-Extension
Program) and Eastern Visayas State University (Research and Development
Extension Programme).
Informed Consent Statement: Informed consent was obtained from all subjects
involved in the study.
Data Availability Statement: Data are not publicly available, though may be made
available on request from the author.
Conflicts of Interest: The author declares no conflict of interest.
7. References
Adedoyin, O. B., & Soykan, E. (2020). Covid-19 pandemic and online learning: the
challenges and opportunities. Interactive Learning Environments, 1-13.
https://doi.org/10.1080/10494820.2020.1813180
Allen, I. E., & Seaman, J. (2008). Staying the course: Online education in the United States, 2008.
Sloan Consortium. https://files.eric.ed.gov/fulltext/ED529698.pdf
Armstrong-Mensah, E., Ramsey-White, K., Yankey, B., & Self-Brown, S. (2020). COVID-19
and Distance Learning: Effects on Georgia State University School of Public
Health Students. Frontiers in Public Health, 8, 547.
https://doi.org/10.3389/fpubh.2020.576227
Bonk, C. J., & Graham, C. R. (2012). The handbook of blended learning: Global perspectives,
local designs. John Wiley & Sons. https://doi.org/10.5465/amle.2008.31413871
Do, J. W. (2020). An investigation of design constraints in the process of converting face-
to-face course into online course. Journal of Education & Culture, 26(2), 153-173.
https://doi.org/10.24159/joec.2020.26.2.153
95
Dumford, A. D., & Miller, A. L. (2018). Online learning in higher education: exploring
advantages and disadvantages for engagement. Journal of Computing in Higher
Education, 30(3), 452-465. https://doi.org/10.1007/s12528-018-9179-z
Ferri, F., Grifoni, P., & Guzzo, T. (2020). Online learning and emergency remote teaching:
Opportunities and challenges in emergency situations. Societies, 10(4), 86.
https://doi.org/10.3390/soc10040086
Fry, K. (2001). E‐learning markets and providers: some issues and prospects. Education+
Training, (4/5), 233-239. https://doi.org/10.1108/EUM0000000005484
Goldschmidt, K. (2020). The COVID-19 pandemic: Technology use to support the
wellbeing of children. Journal of Pediatric Nursing, 53, 88-90.
https://doi.org/10.1016/j.pedn.2020.04.013
Guangul, F. M., Suhail, A. H., Khalit, M. I., & Khidhir, B. A. (2020). Challenges of remote
assessment in higher education in the context of COVID-19: a case study of Middle
East College. Educational Assessment, Evaluation and Accountability, 32, 1-17.
https://doi.org/10.1007/s11092-020-09340-w
Hodges, C., Moore, S., Lockee, B., Trust, T., & Bond, A. (2020). The difference between
emergency remote teaching and online learning. Educause Review, 27, 1-12.
Huang, C., Wang, Y., Li, X., Ren, L., Zhao, J., Hu, Y., ... & Cao, B. (2020). Clinical features
of patients infected with 2019 novel coronavirus in Wuhan, China. The
Lancet, 395(10223), 497-506. https://doi.org/10. 1016/S0140-6736(20)30183-5
Kebritchi, M., Lipschuetz, A., & Santiague, L. (2017). Issues and challenges for teaching
successful online courses in higher education: A literature review. Journal of
Educational Technology Systems, 46(1), 4-29.
https://doi.org/10.1177/0047239516661713
Larreamendy-Joerns, J., & Leinhardt, G. (2006). Going the distance with online
education. Review of Educational Research, 76(4), 567-605.
https://doi.org/10.3102/00346543076004567
Leszczyński, P., Charuta, A., Łaziuk, B., Gałązkowski, R., Wejnarski, A., Roszak, M., &
Kołodziejczak, B. (2018). Multimedia and interactivity in distance learning of
resuscitation guidelines: a randomised controlled trial. Interactive Learning
Environments, 26(2), 151-162. https://doi.org/10.1080/10494820.2017.1337035
Machado, R. A., Bonan, P. R. F., Perez, D. E. D. C., & Martelli Junior, H. (2020). COVID-
19 pandemic and the impact on dental education: discussing current and future
perspectives. Brazilian Oral Research, 34. https://doi.org/10.1590/1807-3107bor-
2020.vol34.0083
Mackenzie, J. S., & Smith, D. W. (2020). COVID-19: a novel zoonotic disease caused by a
coronavirus from China: what we know and what we don’t. Microbiology
Australia, 41(1), 45-50. https://doi.org/10.1071/MA20013
Means, B., Toyama, Y., Murphy, R., Bakia, M., & Jones, K. (2009). Evaluation of evidence-
based practices in online learning: A meta-analysis and review of online learning
studies. https://repository.alt.ac.uk/629/
O'Donoghue, J., Singh, G., & Green, C. (2004). A comparison of the advantages and
disadvantages of IT based education and the implication upon students. Digital
Education Review, (9), 63-76. dialnet.unirioja.es/descarga/articulo/4544753.pdf
OECD Policy Responses to Coronavirus (COVID-19). (2020). Strengthening online learning
when schools are closed: The role of families and teachers in supporting students during
the COVID-19 crisis. https://read.oecd-ilibrary.org/view/?ref=136_136615-
o13x4bkowa&title=Strengthening-online-learning-when-schools-are-closed
Ogbonnaya, U. I., Awoniyi, F. C., & Matabane, M. E. (2020). Move to Online Learning
during COVID-19 Lockdown: Pre-Service Teachers’ Experiences in
96
Ghana. International Journal of Learning, Teaching and Educational Research, 19(10),

286-303. https://doi.org/10.26803/ijlter.19.10.16
Palaigeorgiou, G., & Papadopoulou, A. (2019). Promoting self-paced learning in the
elementary classroom with interactive video, an online course platform and
tablets. Education and Information Technologies, 24(1), 805-823.
https://doi.org/10.1007/s10639-018-9804-5
PSA Census of Population. (2015). Region VIII (Eastern Visayas). Retrieved from
https://www.psa.gov.ph/sites/default/files/attachments/hsd/pressrelease/R
08.xlsx
Radha, R., Mahalakshmi, K., Kumar, V. S., & Saravanakumar, A. R. (2020). E-Learning
during lockdown of Covid-19 pandemic: A global perspective. International
Journal of Control and Automation, 13(4), 1088-1099.
Rajab, M. H., Gazal, A. M., & Alkattan, K. (2020). Challenges to online medical education
during the COVID-19 pandemic. Cureus, 12(7).
https://doi.org/10.7759/cureus.8966
Rutgers. (2020). Remote exams and assessments. Retrieved from
https://sasoue.rutgers.edu/teaching-learning/remote-exams-
assessment#special-advice-for-open-book-assessment-in-quantitative-courses.
Saadé, R. G., He, X., & Kira, D. (2007). Exploring dimensions to online learning. Computers
in Human Behavior, 23(4), 1721-1739. https://doi.org/10.1016/j.chb.2005.10.002
Smedley, J. (2010). Modelling the impact of knowledge management using technology. OR
insight, 23(4), 233-250. https://doi.org/10.1057/ori.2010.11
Suryaman, M., Cahyono, Y., Muliansyah, D., Bustani, O., Suryani, P., Fahlevi, M., ... &
Munthe, A. P. (2020). COVID-19 pandemic and home online learning system:
Does it affect the quality of pharmacy school learning? Systematic Reviews in
Pharmacy, 11(8), 524-530. https://doi.org/10.31838/srp.2020.8.74
Tallent-Runnels, M. K., Thomas, J. A., Lan, W. Y., Cooper, S., Ahern, T. C., Shaw, S. M., &
Liu, X. (2006). Teaching courses online: A review of the research. Review of
Educational Research, 76(1), 93-135. https://doi.org/10.3102/00346543076001093
Wagner, N., Hassanein, K., & Head, M. (2008). Who is responsible for e-learning success
in higher education? A stakeholders' analysis. Journal of Educational Technology &
Society, 11(3), 26-36. https://doi.org/10.2307/jeductechsoci.11.3.26
WHO. (2020). Weekly epidemiological update - 1 December 2020. Retrieved from
www.who.int/publications/m/item/weekly-epidemiological-update---1-
december-2020
Wu, Y. C., Chen, C. S., & Chan, Y. J. (2020). The outbreak of COVID-19: an
overview. Journal of the Chinese Medical Association, 83(3), 217.
https://doi.org/10.1097/JCMA.0000000000000270
Züll, C. (2016). Open-ended questions. GESIS Survey Guidelines, 3.
https://doi.org/10.15465/gesis-sg_en_002
97

Vol. 20, No. 3, pp. 97-116, March 2021
Success Indicators of Mathematical Problem-

Solving Performance Among Malaysian
Matriculation Students
Suriati Abu Bakar, Nur Raidah Salim,

Ahmad Fauzi Mohd Ayub and Kathiresan Gopal
Institute for Mathematical Research, Universiti Putra Malaysia, Malaysia
https://orcid.org/0000-0002-5572-9611
https://orcid.org/0000-0002-7941-1878
https://orcid.org/0000-0002-4313-2922
https://orcid.org/0000-0001-5302-8179
Abstract. This study identifies the success indicators of mathematical

problem-solving performances among Malaysian matriculation students
divided into four indicators: mathematical beliefs, mathematics attitudes,
mathematics self-efficacy and metacognitive skills. For this purpose, 368
matriculation students from three matriculation colleges were selected as
respondents using proportioned stratified sampling. This study utilized
a descriptive correlational design approach. A set of questionnaires and a
mathematics test were used as the instruments. Independent variables
were measured using a questionnaire, while mathematical problem-
solving performance was measured using a mathematics test. The
findings show students had a high level in mathematics beliefs, attitude
towards mathematics, mathematics self-efficacy and metacognitive skills.
Statistical tests to determine success indicators predicting mathematical
problem-solving performance revealed that mathematics self-efficacy
does not contribute significantly to these variables and that metacognitive
skills make the most decisive contribution, followed by mathematics
attitude and mathematics beliefs. Hence, this study suggests that
problem-solving should be included as an essential part of the
mathematics matriculation syllabus to enable students to improve their
problem-solving abilities.
Keywords: mathematics beliefs; mathematics attitude; mathematics self-

efficacy and metacognitive skills; problem-solving performance
1. Introduction
The Malaysian Ministry of Education implemented steps for the transformation
of education through the Malaysia Education Blueprint 2013_2025 to empower
the nation’s human capital with the expertise and skills of the 21st century and be
among the top global players (Ministry of Education Malaysia, 2016). The
©Authors
98
government aims to develop the intelligence, critical thinking skills, leadership

skills, language proficiency, ethics and spirituality and national identity of
Malaysian children to enable them to thrive in the 21st century. In the 21st century,
students will need to equip themselves with good communication, teamwork,
imagination and innovation, critical thinking and problem-solving skills to
achieve success (Warner & Kaur, 2017).
The Malaysia Education Blueprint 2013–2025 (Shift 1) focuses on providing equal

access to the Malaysian education standard to be at par with international
standard. The ministry hopes that Malaysia will produce globally competitive
students by benchmarking and setting the standard criteria for education systems
by focusing on the students’ learning practices and performances. In 2017, the
blueprint was redrafted to introduce the revised Kurikulum Standard Sekolah
Menengah (KSSM) and the revised Kurikulum Standard Sekolah Rendah (KSSR),
which focus on advocating the knowledge and specific skills, such as, creative
and innovative thinking, problem-solving and leadership. The new curriculum is
designed based on international benchmarks to ensure that children born into the
country’s school system have the necessary skills to compete globally (Ministry of
Education Malaysia, 2013). The enforcement will ensure that students will be
capable of portraying the subject content while extrapolating the information and
creatively connecting them to unfamiliar settings. The ministry will reasonably
escalate the percentage of higher-order thinking questions with a minimum of
40% questions in Ujian Pencapaian Sekolah Rendah (UPSR) and 50% in Sijil Pelajaran
Malaysia (SPM) to validate the effectiveness of the enforcement. The changes will
hone the students to foster their creativity in handling the typical routines using
different settings.
As problem-solving focuses on the teaching and learning of mathematics, the

emphasis must not be at the secondary level only but also at the post-secondary
level. The Ministry of Education provides a wide range of post-secondary
education, including Matriculation, which is recognized as a credible foundation
program for undergraduate programs in higher learning institutions in Malaysia
and several overseas universities (Ministry of Education Malaysia, 2016). Hence,
the matriculation mathematics syllabus was designed by considering the topics
covered in secondary schools to strengthen and broaden their knowledge and
skills in Mathematics and enable students to have complete and robust
foundation to pursue programs in science, technology, social science and
management (Matriculation Division, 2006).
Most studies among matriculation students have been conducted by other

scholars (Buzzai et al. l, 2020; Boğar, 2018). For example, Ong and Lim (2014)
identify the factors that influence the matriculation students in mathematical
problem solving, but they used Bayesian Network to analyse the causal
relationships. This study will utilise an inferential research approach to explain
and predict interrelationships between influential domain variables to fill this
gap.
99
The factors underlying this phenomenon, specifically among matriculation

students need to be determined. Convincing the students to solve mathematical
problems cannot be easily accomplished unless we know the critical aspect that
could affect students’ mathematics performance, such as mathematics beliefs,
mathematics attitudes, metacognitive skills and mathematics self-efficacy. This
study will utilize an inferential research approach to explain and predict
interrelationships between influential domain variables to fill these gaps.
The government’s vision is that children will never stop learning throughout their
lives and eventually connect the dots and develop new knowledge (Ministry of
Education Malaysia, 2013). Changing the education system and enhancing the
teaching and learning environment will ideally create students who can master
various essential cognitive abilities, including logical thinking, brainstorming,
creative thinking and innovation. It is believed that the prominent students with
the ability to solve problems will have a better future in their careers and lifetimes
(Khalid, 2017). Thus, this study examined four success indicators that affect
students’ results in mathematical problem-solving significantly.
1.1. Problem-Solving in Mathematics

Problem-solving involves extensive skills in parallel to the curriculum,
particularly in teaching and learning mathematics. Teaching mathematics topics
through problem-solving focuses on problem-solving contexts and inquiry-
oriented environments (Khalid, 2017). Students who participate in problem-
solving activities will experience mathematics’ power by understanding and
applying their knowledge to real-life problems (Tarmizi et al., 2015).
The aspects of human psychology, such as cognitive, affective, metacognitive and

behaviour need to be emphasized from primary to tertiary levels to help students
develop their mathematical problem solving skills (Mason, 2016). Additionally,
the problem-solving process involves steps, such as determining how an issue can
be addressed, strategizing the approach used and how much effort and time-
frame set to obtain a solution (Tambunan, 2019). Students must also apply their
knowledge in solving a problem, a process that would affect the deep
understanding of mathematics and the choice of strategies (Liljedahl et al., 2016;
Khalid, 2017).
Numerous studies have focused on students' characteristics towards their

mathematical performance. These studies include exploring the difficulties that
students faced in mathematics problem solving (Ayop & Tarmizi, 2015). Several
studies have also revealed that success indicators, such as mathematics beliefs
(Rincon et al., 2020; Kamalimoghaddam et al., 2016), mathematics attitude,
metacognitive and mathematics self-efficacy affect problem-solving (Özcan &
Gümüş, 2019; Kahramanoglu & Deniz, 2017).
1.1.1. Mathematics Belief

Mathematics beliefs are defined as an individual’s beliefs on mathematics that
encourage his/her to make decisions regarding actions he/she will pursuit
(Kloosterman et al., 1996). Mathematics beliefs have also been defined as students’
conceptions of mathematics that they hold to be true either implicitly or explicitly
100
and can influence them in their mathematics learning and mathematical problem
solving (Wang et al., 2019). In this study, mathematics beliefs are defined as
students’ assumptions about mathematical problem solving and their
mathematics learning approaches, which consist of five components.
Kloosterman and Stage (1992) suggested these five components as listed in Table
1.
Table 1: Components of Mathematics Belief

Difficult Perceived ability in solving time-consuming mathematics
Problems problems (Abedalaziz and Akmar, 2012).
Steps Students assume that mathematical word problems cannot be
resolved using the procedural method (Barrett, 2016).
Understanding The necessity for students to understand how to obtain solutions
(Prendergast et al., 2018).
Word Problems An essential element of mathematics and deals with mathematics
discipline (Abedalaziz and Akmar, 2012).
Effort Students believe that attempts to study can motivate them to
successfully solve mathematics problems (Prendergast et al.,
2018).
1.1.2. Mathematics Attitude

Mathematics attitude is defined as a positive attitude or personal temperament
towards mathematics (Peteros et al., 2019). Mathematics attitude is also defined
as an individual’s emotional response towards mathematics, mathematics beliefs
and behaviour towards mathematics (Öztürk et al., 2020). In this study,
mathematics attitudes are defined as students’ attitudes towards mathematics
during the classroom learning process, which consist of four dimensions. Tapia
and Marsh (2004) suggested the four components listed in Table 2.
Table 2: Components of Mathematics Attitude

Self-confidence Students’ sense of dread, anxiety and hate in hearing the word
mathematics (Simegn et al., 2018).
Value The usefulness and relatedness of learning mathematics to
student’s life situations (Kasimu & Imoro, 2017).
Enjoyment Students’ feeling of pleasure in solving challenging mathematics
problems, willingness to participate in mathematics discussion and
excitement in the mathematics classroom (Simegn et al., 2018).
Motivation Students’ interest in learning mathematics and willingness to
further their knowledge in mathematics (Ajisuksmo & Saputri,
2017)
1.1.3. Mathematics Self-Efficacy

Mathematics self-efficacy can be defined as the assumptions or expectations of
individuals that he/she can perform a particular task or overcome mathematical
problems (Bandura, 1997). Mathematics self-efficacy is defined by students’
beliefs on their potential to carry out any assignment or tasks needed for learning
(Salim et al., 2018). This study defined mathematics self-efficacy as students’
confidence in their abilities to accomplish various tasks, from understanding
mathematical concepts to solving mathematical problems correctly. Mathematics
self-efficacy was measured with four components as listed in Table 3.
101
Table 3: Components of Mathematics Self-efficacy

Mastery How students view and assess the previous achievement and their
experience judgments are amended according to these views (Doménech-
Betoret et al., 2017).
Vicarious How students feel and observe peer or adult models’ academic
experience skills helps them manipulate their experience (Gopal et al., 2018).
Social Encouraging messages that students get from teachers, parents and
persuasions close friends can empower their confidence in academic
capabilities (Kandemir & Akbaş-Perkmen, 2017).
Physiological Students’ emotional condition (Kandemir & Akbaş-Perkmen,
states 2017).
1.1.4. Metacognitive Skills

Metacognitive skill was defined as an individual’s power to deliberate and this
skill is used to understand, plan, monitor, evaluate learning activities, summarise
and reflect (Veenman & van Cleef, 2019). Metacognitive skills can also be
expressed as an individual’s awareness to periodically monitoring whether
he/she has attained the goal and carrying out different strategies after selecting
them if necessary (O’Neil & Abedi, 1996). In this study, metacognitive skills are
defined as students’ awareness of the mathematical problem-solving process. As
suggested by O’Neil & Abedi, 1996, metacognitive skills consist of four
components: planning, self-checking, cognitive strategy and awareness.
Table 4: Components of Metacognitive Skills

Planning Learning options, target setting, inquisitiveness and way of doing
things (Loh & Lee, 2019).
Self-checking Students monitor their learning process to determine problems and
change strategies to achieve the goal (Hinojosa et al., 2020).
Cognitive Individual cognitive or affective strategies which needed during
strategy the learning process (Hinojosa et al., 2020).
Awareness A process which is someone fully aware of (Loh & Lee, 2019).
1.2. Mathematical Problem-Solving Performance

Mathematical problem solving is presumed to challenge and strengthen student’s
intellectual understanding and encourage the development of mathematics
concepts (NCTM, 2010). Polya (1957) defined mathematical problem solving as
finding a strategy around a hardship and identifying a solution to an unfamiliar
problem. Accordingly, Amalia et al. (2017) discussed in detail the basic procedure
in problem-solving, which include the following:
1. Describe the problem/problem statement
2. Strategize and develop the plan
3. Assess and decide the alternatives
4. Implement and seek for solution
Stylianides and Stylianides (2014) agree that mathematical problem solving

provides an activity for students to think out of the box by relating the actual
situation or actual data with the unknown. Therefore, this study’s mathematical
problem-solving performance is defined as the students’ achievement in finding
102
the best solution to a set of mathematical problem-solving tests. To evaluate this

performance, the Mathematical Problem-Solving Test (MPST) has been adopted.
Five mathematics problems based on the matriculation mathematics curriculum
were filtered to ensure that only cover the Integration topic. This study's
mathematics problems were the types of problems that had a clear structure. The
problems also had a set of possible solutions and can be solved in writing. The
problems selected are also challenging, but the context is still within what
students have learned in their classrooms.
Identifying the success indicators that could predict their performance in

mathematical problem-solving is necessary. Hence, a conceptual framework
shown in Figure 1 was constructed involving four variables to measure the
mathematical problem-solving performance. The conceptual framework was
developed based on a combination of social cognitive theory (SCT), theory of
planned behaviour (TPB) and model of mathematical problem solving, which had
been drawn from the literature review. These variables were combined with
modelling the relationship of students’ affective indicators, metacognitive and
mathematical problem-solving performance.
Figure 1: Conceptual Framework
1.2.1. Success Indicators of Mathematical Problem-solving Performance

Previous studies have validated that mathematics beliefs have a constructive
influence on mathematics performance. A study on factors influencing
mathematical problem-solving performance revealed a significant relationship
between students’ mathematics beliefs and their mathematical problem-solving
performance (Deng et al., 2015). The second indicator, mathematics attitude, has
also shown a significant effect on mathematics performance. In Wolkite Town, 240
students were surveyed using Attitude Towards Mathematics Scales (ATMS),
103
which consists of five components, namely, Value, Male Domain, Enjoyment,

Motivation and Confidence. The study found that mathematics attitude had the
strongest positive significant correlation on mathematics achievement (Simegn &
Asfaw, 2018). Kandemir and Akbaş-Perkmen (2017) found that mastery
experience, a source in mathematics self-efficacy, has a significant and positive
influence on mathematics achievements. Consistent with another previous
studies on mathematics self-efficacy, a study in Iran also showed that the third
indicator has a strong effect on mathematics achievement (Kamalimoghaddam et
al., 2016). Abdullah et al. (2017) focused on the role and effects of metacognitive
skills strategy in mathematical activity found a positive and moderate significant
correlation between metacognitive skills and mathematical problem-solving
performance.
The remaining sections are structured as follows. In section 2, the objective and
significance of this study are described. In Section 3, the details of the
methodology are expounded. The survey results are presented in Section 4 and
discussions are elaborated on in Section 5. Lastly, the concluding remarks are
given in Section 6.
2. Objectives
In general, this study’s motivation and rationale would add to the growing body
of knowledge on mathematics problem-solving. It proposed to investigate the
success indicators predicting mathematical problem-solving performance among
Malaysian matriculation students. This paper provides a survey on the
relationship between all indicators with their pros and cons. The objectives of this
paper are as follows:
(i) To determine the relationship of mathematics beliefs, mathematics
attitudes, mathematics self-efficacy and metacognitive skills on
mathematical problem-solving performance among Malaysian
matriculation students.
(ii) To identify success indicators that predicting mathematical problem-
solving performance among Malaysian matriculation students.
The significance of this study are as follows:

(i) Post-secondary level education in Malaysia, especially the
Matriculation Program, has been less explored than primary,
secondary and tertiary education levels.
(ii) This study’s findings can provide researchers and lecturers with
insights into how beliefs, attitudes, metacognitive skills and self-
efficacy affect how students transfer knowledge.
(iii) Analysing the success indicators that may influence the mathematical
problem-solving performance will enrich information that can be used
in improving mathematics education to researchers, lecturers, or
policymakers.
3. Methodology
This study utilized a descriptive correlational design approach by collecting data
obtained from a questionnaire. The design involved data collection and statistical
104
analysis of numerical data (Ary et al., 2014) to characterise, clarify, anticipate or

control the phenomena of interest (Mills & Gay, 2016). This study also referred to
Kamalimoghad who utilized the correlational design as a research methodology
to determine the factors predicting mathematical problem-solving performance
using various variables, including mathematics beliefs, mathematics attitude and
metacognitive skills. Correlational design is a statistical procedure of correlational
analysis in which researchers measure the degree of association or relationship
among variables in a single group of subjects (Ary et al., 2014).
A set of questionnaires and a mathematics test were used as instruments in this

study. Denscombe (2010) pointed out that questionnaire is suitable for any
research with a large sample size from different locations. Therefore, standardized
similar questions are required from the respondents. Data on mathematics beliefs,
mathematics attitude, mathematics self-efficacy and metacognitive skills were
obtained from students’ ratings on the questionnaire items. Simultaneously,
based on the total marks of a mathematics test, we measured the mathematical
problem-solving performance. The results that adequately indicate significant
relationships among the five variables would serve as strong evidence to carry out
the next objective, i.e., prediction, using correlational analysis.
3.1. Populations
This quantitative correlational study employed a cluster sampling procedure. The
sample for this study comprised of matriculation students under the Ministry of
Education in three zones (North, Central and South). The North zone consists of
four matriculation colleges namely, Kolej Matrikulasi Perlis, Kolej Matrikulasi
Kedah, Kolej Matrikulasi Pulau Pinang and Kolej Matrikulasi Perak. The Central
zone consists of five matriculation colleges namely, Kolej Matrikulasi Selangor,
Kolej Matrikulasi Negeri Sembilan, Kolej Matrikulasi Kelantan, Kolej Matrikulasi
Sarawak and Kolej Matrikulasi Labuan while the South zone has three
matriculation colleges namely, Kolej Matrikulasi Melaka, Kolej Matrikulasi Johor
and Kolej Matrikulasi Pahang.
Based on the total population identified as 18 717 students from the matriculation
program across Malaysia, the sample size calculation was obtained using
Cochran’s (1977) formula as 263. Therefore, the minimum sample size required,
according to Cochran (1977,) is 263. Fraenkel et al. (2016) mentioned that a
minimum sample of 30 should be generally acceptable to establish the existence
of a relationship for a correlational study. However, for educational and social
research studies, the response rate using surveys as the data collection method is
usually below 100% (Barlett et al., 2001). Hence, Salkind (2012) recommended
oversampling by 40% to 50% to account for lost respondents and uncooperative
respondents. Hence, in this study, the sample size was increased by 40% of the
original number of the sample size calculated above. The addition to the sample
was 105 students (40% x 263). Therefore, this study's total sample size was 368 to
ensure that an adequate sample was used in this study.
A total of 368 students from the matriculation program under the Ministry of
Education Malaysia were selected randomly. A list of colleges in the North,
105
Central and South zones was acquired and based on random sampling, three
colleges were chosen for the actual data collection process. The selected colleges
were Kolej Matrikulasi Perak (North zone), Kolej Matrikulasi Negeri Sembilan
(Central zone) and Kolej Matrikulasi Melaka (South zone). The sampling strategy
process is shown in Table 5.
Table 5: Matriculation Colleges in Malaysia

Number of
No. of Percentage Additional
Colleges samples
Students (%) 40%
required
North
Kolej Matrikulasi Perlis 1,007
Kolej Matrikulasi Kedah 1,708
Kolej Matrikulasi Pulau 1,943
Pinang 35 92 129
Kolej Matrikulasi Perak 1,817
Total 6,475
Central
Kolej Matrikulasi 1,718
Selangor
Kolej Matrikulasi Negeri 1,645
Sembilan
Kolej Matrikulasi 1,403 36 95 132
Kelantan
Kolej Matrikulasi 119
Sarawak 1,918
Kolej Matrikulasi Labuan
Total 6,803
South
Kolej Matrikulasi Melaka 1,616
Kolej Matrikulasi Johor
Kolej Matrikulasi Pahang 2,058 29 76 107
1,765
Total 5,439
Overall Total 18,717 100 263 368
3.2. Instrumentation
Instrumentation involves designing the instruments and procedures of data
collection (Fraenkel et al., 2016). This research instrument was a structured
questionnaire composed of three parts, namely, Parts A, B and C. The number of
items and sources for each construct in the questionnaire is listed in Table 6.
In Part B, respondents were given options by using a five-point Likert scale

ranging from Strongly Disagree (1), Disagree (2), Moderately Agree (3), Agree (4)
and Strongly Agree (5) to measure their agreement on the questionnaire
statements. All five items in Part C in section 1 were scored in three categories:
understanding, planning a solution and getting answers. Each item was given six
points with a total score of 30.
106
Table 7 shows the problem-solving marking scheme details to assess

mathematical problem-solving questions in this study. The percentages of total
scores from each step of the marking scheme were used to determine students’
mathematical problem-solving performance.
Table 6: Distribution of Items in Each Section and the Sources

Part/ Section Components No. of Source
Item
Part A
Section 1 Demographic Information 5 Self-
developed
Part B
Section 1 Mathematics Beliefs 28 Kloosterman
• Difficult Problems 6 & Stage (1992)
• Steps 4
• Understanding 6
• Word Problems 6
• Effort 6
Section 2 Mathematics Attitudes 40
• Self-confidence 15 Tapia &
• Value 10 Marsh (2004)
• Enjoyment 10
• Motivation 5
Section 3 Mathematics Self-efficacy 24 Usher &
• Mastery Experience 6 Pajares (2009)
• Vicarious Experience 6
• Social Persuasions 6
• Physiological States 6
PART C
Section 1 Mathematical Problem 5 Self-
Solving Performance developed
Section 2 Metacognitive Skills 20 O’Neil &

• Awareness 5 Abedi (1996)
• Cognitive Strategy 5
• Planning 5
• Self-checking 5
118
Table 7: Marking Scheme Details

Stages Scores Characteristics Description
0 Complete Lack of comprehension problem
misunderstanding Not able to identify important given
Understanding
data.
1 Partial understanding Misunderstood/ misinterpreted part of
the data, partially understand goals and
hidden data.
2 Complete Ability to take information and
understanding translate it in the mathematical model,
fully retrieve given and hidden data.
107
0 No Wrong integration procedure, not able

attempt/inappropriate to put constant of integration.
Planning a solution plan
1 Partially correct plan Accurate interpretation up to a certain
point but fail to strategize
appropriately.
2 Plan led to a correct Execute the plan, translate the plan into
solution series of appropriate mathematical
actions, successful findings.
0 No answer/ wrong Cannot execute integration steps.
answer
answers
Getting
1
Copying error; Mathematical/ computational error.
computational error
2 The correct answer, Solution complete, no error in response.
the correct label
Adapted rubric scores for mathematical problem solving based on Analytic Scoring
Scale (Charles et al., 1987)
Table 8 shows the distribution of the students’ scores and their test performance
level. In Section 2, 25 items were measured on a five-point Likert scale. The scales
for items in this section were Never (1), Rarely (2), Sometimes (3), Often (4) and
Always (5).
Table 8: Distribution of Students Score and Performance Level

Scores Level
0.00 to 36.72 Very Low
36.73 to 46.68 Low
46.69 to 56.75 Medium
56.76 to 66.86 High
66.87 to 100.00 Very High
Adapted from: (Lee et al., 2013)
3.3. Reliability
An instrument’s reliability is the degree of coherence or consistency, which can
measure the same variables even if used in different samples, times and places
(Ary et al., 2014). In this pilot study, the reliability measurement was used to
assess the internal coherence. Internal coherence refers to the items that comprise
the scale by measuring the same bottom-line attributes (Pallant, 2020). The
reliability analysis was performed on an instrument using Cronbach’s Alpha
coefficient in the IBM SPSS Statistics software. All constructs tested were
considered reliable because the Cronbach’s Alpha value was more significant than
0.50, which is congruent with the acceptable value for reliability as indicated by
Hinton et al. (2014). The Cronbach’s Alpha reliability coefficients for the pilot and
actual studies are summarized as in Table 9.
108
Table 9: Cronbach’s Alpha Coefficient

Part Variable No. of No. of Pilot Actual
Items Items after Study Study
before Pilot Pilot (n=99) (n=312)
Study Study
Part A
Section 1 Demographic 5 5 - -
Information
Part B
Section 1 Mathematics Beliefs 30 28 .615 .652
Section 2 Mathematics Attitudes 40 40 .952 .959
Section 3 Mathematics Self- 24 24 .827 .928

efficacy
PART C
Section 1 Mathematical Problem- 5 5 .839 .772
Solving Performance
Section 2 Metacognitive Skills 20 20 .898 .922
124 122 .826 .965
4. Analysis and Results

The relationship between the four success indicators for mathematical problem
solving and mathematics achievement was calculated utilising the Pearson
product-moment correlation technique. Then, multiple regression analysis was
performed to identify the best predictors of the four indicators on the
mathematical problem-solving in mathematics achievements. Based on Hair et al.
(2010), the normality, linearity, homoscedasticity assumptions had been met.
4.1. Demographic Information

The respondents’ general demographic information is given in Table 10. A total
of 368 matriculation students participated in this study, composed of 125 (34.0%)
male students and 243 (66.0%) female students. Therefore, a significant majority
of the respondents were females. Ethnic groups were categorized into four sub-
categories including Malay respondents (322 respondents, 87.5%), 30 Indian
respondents (8.1%), 15 Chinese respondents (4.2%) and only one respondent from
others. The majority were Malay respondents because the Ministry of Education’s
Matriculation Programme is a preparatory program for Bumiputera students with
only 10% non-Bumiputera students.
In terms of mathematics grades in PSPM1, 120 respondents (32.7%) earned an A

and 15.7% (58 respondents) earned A- , about 3.5% (13 respondents) and 0.6%
(two respondents) earned C and C- for mathematics, respectively. As for the
number of revision hours for the mathematics subject, most of the respondents
(178 respondents, 48.4%) had less than an hour of mathematics revision per day.
Moreover, only 18 respondents (4.8%) had four to five hours of mathematics
revision per day.
109
Table 10: Demographic Information

Demography Category Number of Percentage
Respondents (%)
Gender Male 125 34.0
Female 243 66.0
Race Malay 322 87.5
Chinese 15 4.2
Indian 30 8.1
Others 1 0.2
Mathematics A 120 32.7
Grade A- 58 15.7
B+ 65 17.6
B 60 16.3
B- 26 7.1
C+ 24 6.4
C 13 3.5
C- 2 0.6
Hours of 0–1 hour/day 178 48.4
Revision
2–3 hours/day 172 46.8
4–5 hours/day 18 4.8
4.2. Statistical Analysis

The means of all variable items were summed up and averaged to obtain the
overall mean for each variable. The mean value of less than 3.00 was categorized
as low and the value that fell between 3.01 to 5.00 was classified as high. Table 11
shows the means (M) and standard deviations (SD) of the respondents’ responses
on the variables investigated in this study. The average mean for the responses on
all variables was 3.83 (SD=0.49). The highest mean rating was attained on
mathematics attitude (M=3.91, SD=0.57), followed by metacognitive skills
(M=3.84, SD=0.52), then by mathematics beliefs (M=3.80, SD=0.28) and lastly,
mathematics self-efficacy (M=3.75, SD=0.59). The results implied that
matriculation students perceived themselves as demonstrating high mathematics
attitude, metacognitive skills, mathematics beliefs and mathematics self-efficacy
towards mathematical problem-solving performance.
Table 11: Mean and Standard Deviation

Variable Mean SD Category
Mathematics Beliefs 3.80 0.28 High
Mathematics Attitude 3.91 0.57 High
Mathematics Self-efficacy 3.75 0.59 High
Metacognitive Skills 3.84 0.52 High
Overall Mean and High
3.83 0.49
Standard Deviation
The Pearson correlation analysis was performed to determine any significant

relationships among mathematics beliefs, mathematics attitude, metacognitive
skills, mathematics self-efficacy and mathematical problem-solving performance.
The results of Pearson’s correlation analysis are presented in Table 12.
110
The results showed significant relationships at the 0.01 level of significance

between the independent and dependent variables. Significant relationships were
observed between mathematics beliefs (r=0.448**, p=0.000), mathematics attitude
(r=0.489**, p=0.000), metacognitive skills (r=0.455**, p=0.000), mathematics self-
efficacy (r=0.484**, p=0.000) and mathematical problem solving performance. All
the relationships were of moderate strengths (Lodico et al., 2010).
Table 12: Results of Pearson Correlation Coefficients

Mathematics Mathematics Metacognitive Mathematics
Beliefs Attitude Skills Self-efficacy
Mathematics r 0.448** 0.489** 0.455** 0.484**
Problem Sig. 0.000 0.000 0.000 0.000
Solving (2-
Performance tailed)
**Correlation is significant at the 0.01 level
The following model summary table (Table 13) shows Malaysia’s multiple
correlation coefficient was 0.558 and the R2 value was 0.311. The R2 value indicated
how much the model disclosed the variance in the dependent variable. The R2
value obtained was 0.311, indicating that 31.1% of the mathematical problem-
solving performance variance could be explained by the model containing the
combination of the independent variables.
Table 13: Model Summary

R R Square Adjusted R Square Std. Error of the Estimate
0.558 0.311 0.305 1.45160
Table 14 presents the Analysis of Variance (ANOVA) for the multiple regression
model. The F-test in the ANOVA table test identified whether the regression
model fits the data. The F-test results showed that the independent variables
significantly predicted the dependent variable [F (3, 311) = 46.402, p =0.000] at a
significance level of 0.05.
Table 14: ANOVA

Model Sum of Squares df Mean Square F Sig.
Regression 293.325 3 97.775 46.402 0.000
Residual 648.995 308 2.107
Total 942.320 311
The result of the multiple linear regression shown in Table 15 revealed that
mathematics beliefs (t=2.304, p<0.05), mathematics attitude (t=3.916, p<0.05) and
metacognitive skills (t=4.849, p<0.05) had unique and statistically significant
contribution to the mathematical problem-solving performance. However,
mathematics self-efficacy did not contribute significantly and was excluded from
the regression, which might be because mathematics self-efficacy overlapped with
other independent variables in the model (Pallant, 2020).
The data in Table 15 show that the beta (β) value for metacognitive skills (β=0.284)
111
was the most significant coefficient followed by mathematics attitude (β=0.251)

and mathematics beliefs (β=0.132). These implied metacognitive skills were the
most vital unique contributors in explaining mathematical problem-solving
performance. The model indicated with each increase of one standard deviation
in metacognitive skills and mathematical problem-solving performance would
lead to an increment by 0.284 standard deviation units. With an increase of one
standard deviation in mathematics attitude, mathematical problem-solving
understanding would increase by 0.251 standard deviation units. Likewise, with
every increase of one standard deviation in mathematics beliefs, mathematical
problem-solving performance would lead to an increment of 0.132 standard
deviation units.
Table 15: Results of Regression Analysis

Unstandardized Standardized
Coefficients Coefficients
Independent Variable B Std. Error Beta t Sig.
(Constant) -7.282 1.237 -5.886 0.000
Mathematics Beliefs 0.906 0.393 0.132 2.304 0.022
Mathematics Attitude 0.763 0.195 0.251 3.916 0.000
Metacognitive Skills 0.946 0.195 0.284 4.849 0.000
Mathematics Self- 0.088 0.897 0.370
efficacy
5. Discussion
This study aimed to identify the relationship between students’ mathematics
beliefs, mathematics attitudes, mathematics self-efficacy and metacognitive skills
towards mathematical problem-solving performance among Malaysian
matriculation students.
Mathematics beliefs referred to students’ thoughts about mathematics that

encourage them to make decisions regarding their actions. It is an essential
component of students’ participation in mathematics learning activities and
mathematics performance. The findings of this study indicated that the
respondents had a high level of mathematics beliefs. However, among the five
components, respondents believe that efforts can enhance their mathematics
ability and succeed in mathematics subjects. They agreed that studying hard
could improve their mathematics ability and enable them to become more
competent in solving mathematics problems. Respondents perceived that
understanding why a particular procedure was correct is essential rather than
getting the right answer. Conjointly, the respondents also believed they would
not surrender easily while solving the time-consuming mathematics problem.
In this study, mathematics attitude refers to students’ attitude towards

mathematics during the learning process in the classroom. The four components
used to evaluate mathematics attitude were self-confidence, value, enjoyment and
motivation. The finding of this study revealed that the respondents concurred that
mathematics was one of the remarkably crucial subjects to learn and that studying
112
mathematics at an advanced level was appealing because of the challenge of

solving mathematics questions. They also view mathematics as an exciting subject
and they usually enjoyed learning mathematics in college because they gained a
sense of satisfaction after solving mathematics problems. The students tended to
believe they had self-confidence and a positive self-concept on their mathematics
performance.
Mathematics self-efficacy is outlined as personal persuasions or impressions that

an individual has the power to perform a specific task or solving mathematics
problems. The four components used to evaluate mathematics self-efficacy were
mastery experience, vicarious experience, social persuasions and physiological
states. A high level of mathematics self-efficacy was identified in this study
because respondents perceived that seeing friends do better than them when
solving mathematics problems could motivate them to do better. Moreover,
seeing adults perform well in mathematics could also encourage them to do
better.
Metacognitive skills refers to students’ ability to plan, monitor, evaluate and

control their cognitive strategies during mathematical problem-solving activity.
Various components are used to assess students’ metacognitive skills in
mathematical problem solving, including planning, self-checking, mental
approach and awareness. The results showed that students’ planning skills were
at its highest level compared to the self-checking, awareness and cognitive
strategy. The highest mean in planning skills suggested that respondents
preferred to use planning skills compared to other metacognitive skills and
indicated that would try to comprehend the mathematics problems before
attempting to solve them fully. They also checked their working solutions, which
caused them to correct the mistakes they made.
This study has several limitations. The study was limited and focused on students
of the Matriculation Division, Ministry of Education Malaysia. Students'
mathematical problem-solving in matriculation colleges may not relate to other
subjects and another level as primary and tertiary education. Hence, the
generalization of the results obtained may not be appropriate.
because it involves a survey, the data are self-reported and gathered using
questionnaires and tests set. The findings are based on respondents’ self-reports,
which is an obvious limitation to the study. Moreover, the present study
determined students' mathematics beliefs, mathematics attitudes, metacognitive
skills and mathematics self-efficacy on students' mathematical problem-solving
performance. Other factors that may affect students’ mathematical problem-
solving performances such as gender, parents’ education and socioeconomic
context, were not explored. Hence, only certain parts of topics for post-secondary
education level, especially from the matriculation syllabus of mathematics, were
studied.
Mathematical problem-solving refers to students’ activity in finding a

mathematics problem and their ability to relate the unknown with the actual
113
situation or data. In this study, mathematical problem-solving performance

referred to students’ performance in solving mathematics problems. Based on the
students’ performance level analysis, most students’ performance level in
mathematical problem solving are very low. The inadequate level in mathematical
problem-solving performance suggested that the students are very poor in solving
the mathematics problems, which might be due to a lack of problem-solving
practice for the students in the classroom. Therefore, this study suggested that
problem-solving should be included as an essential part of the mathematics
matriculation syllabus to provide students with the space to upgrade their
problem-solving abilities. By teaching through a problem-solving learning
environment, students are expected to have greater mathematics conceptual
understanding that can help them become good problem solvers.
6. Conclusion
Students’ mathematics beliefs, attitude towards mathematics, mathematics self-
efficacy and metacognitive skills in mathematical problem solving were
categorized as high. However, mathematical problem-solving performance was
classified as low. Correlation analysis was used to identify the relationship
between mathematics beliefs, attitude towards mathematics, mathematics self-
efficacy and metacognitive skills towards mathematical problem-solving
performance. The findings indicated a significant positive correlation between
mathematics beliefs, attitude towards mathematics, mathematics self-efficacy and
metacognitive skills towards mathematical problem-solving performance.
Multiple regression analysis was used to identify the best predictor of
mathematical problem-solving performance. The findings revealed that the
metacognitive abilities had the most substantial contribution, followed by
mathematics attitude and mathematics beliefs. However, among these variables,
mathematics self-efficacy did not contribute significantly. These results identified
that the final model with three predictor variables explained 31.1% of the
mathematical problem-solving performance variation.
7. References
Abdullah, A. H., Rahman, S. N. S. A., & Hamzah, M. H. (2017). Metacognitive skills of
Malaysian students in non-routine mathematical problem-solving. Bolema: Boletim
de Educação Matemática, 31(57), 310-322. https://doi.org/10.1590/1980-
4415v31n57a15
Abedalaziz, N., & Akmar, S. N. (2012). Epistemology beliefs about mathematical problem
solving among Malaysian students. OIDA International Journal of Sustainable
Development, 5(1), 59–74.
Ajisuksmo, C., & Saputri, G. (2017) The Influence of Attitudes towards Mathematics, and
Metacognitive Awareness on Mathematics Achievements. Creative Education, 8,
486-497. http://doi.org/10.4236/ce.2017.83037
Amalia, E., Surya, E., & Syahputra, E. (2017). The effectiveness of using problem-based
learning (PBL) in mathematics problem-solving ability for junior high school
students. International Journal of Advance Research and Innovative Ideas in
Education, 3(2), 3402-3406.
Ary, D., Jacobs, L. C., Sorensen, C. K., & Walker, D. (2014). Introduction to research in
education (9th ed.). Belmont, CA: Wadsworth, Cengage Learning.
114
Ayop, R. A., & Ahmad Tarmizi, R. (2015). Pre-university Students’ Difficulties in Problem
Solving by Using Q-methodology. Jurnal Pendidikan Sains dan Matematik Malaysia,
5(2), 1–11.
Bandura, A. (1997). Self-efficacy: The exercise of control. New York: W.H. Freeman.
Barrett, L. L. (2016). Development and Validation of the Mathematical Problem-solving
Dispositions and Beliefs Scale. [PhD Dissertations, Kennesaw State University].
Barlett, J. E., Kotrlik, J. W., & Higgins, C. C. (2001). Organizational research: Determining
appropriate sample size in survey research. Information Technology, Learning, and
Performance Journal, 19(1), 43-50.
Buzzai, C., Filippello, P., Puglisi, B., Mafodda, A. V., & Sorrenti, L. (2020). The relationship
between mathematical achievement, mathematical anxiety, perfectionism, and
metacognitive abilities in Italian students. Mediterranean Journal of Clinical
Psychology, 8(3).1-18
Boğar, Y. (2018). Exploring 6th grade students’scientific epistemological beliefs and metacognitive
awareness regarding achievement level [Doctoral dissertation, Middle East Technical
University].
Deng, F., Tay, E. G., Toh, T. L., Leong, Y. H., Quek, K. S., Toh, P. C., ... & Ho, F. H. (2015).
Enhancing Student Beliefs about Mathematical Problem Solving: Effects of a
Problem-Solving based Intervention. Research in Mathematical Education, 19(1), 19-
41.
Denscombe, M. (2010). The Good Research Guide for Small-Scale Social Research Project (4th
ed.). New York: Open University Press.
Doménech-Betoret, F., Abellán-Roselló, L., & Gómez-Artiga, A. (2017). Self-efficacy,
Satisfaction, and Academic Achievement: The Mediator Role of Students’
Expectancy-Value Beliefs. Frontiers in Psychology, 8(2), 1–12.
https://doi.org/10.3389/fpsyg.2017.01193
Fraenkel, J. R., Wallen, N. E., & Hyun, H. H. (2016). How to design and evaluate research in
education (9th ed.). New York: McGraw Hill Higher Education.
Gopal, K., Salim, N. R., & Ayub, A. F. M. (2018, November). Influence of Self-efficacy and
Attitudes Towards Statistics on Undergraduates’ Statistics Engagement in A
Malaysian Public University. In Journal of Physics: Conference Series (Vol. 1132, No.
1, p. 012042).
Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2010). Multivariate Data Analysis
(7th ed.). New Jersey, USA: Pearson Prentice Hall.
Hinojosa, L. M. M., Rodriguez, M. C., & Paez, C. A. O. (2020). Measurement of
Metacognition: Adaptation of Metacognitive State Inventory in Spanish to
Mexican University Students. European Journal of Educational Research, 9(1), 413-
421.
Hinton, P. R., McMurray, I., & Brownlow, C. (2014). SPSS Explained (2nd ed.). New York:
Routledge.
Kahramanoglu, R., & Deniz, T. (2017). An investigation of The Relationship Between
Middle School Students’ Metacognitive Skills, Mathematics Self-Efficacy, and
Mathematics Achievement. Inonu University Journal of the Faculty of
Education, 18(3), 189-200. https://doi.org/10.17679/inuefd.334285
Kamalimoghaddam, H., Ahmad Tarmizi, R., Ayub, A. F. M., & Jaafar, W. M. W. (2016).
The Influences of Mathematics Beliefs on Mathematics Achievement Through
Mathematics Self-Efficacy: A Structural Equation Model. Communication in
Mathematical Modeling and Applications, 51(1), 44–51.
Kandemir, M. A., & Akbaş-Perkmen, R. (2017). Examining Validity of Sources of
Mathematics Self-Efficacy Scale in Turkey. European Journal of Education Studies.
3(11), 69-88. https://doi.org/10.5281/zenodo.1041845.
115
Kasimu, O., & Imoro, M. (2017). Students’ Attitudes Towards Mathematics: The Case of
Private and Public Junior High Schools in The East Mamprusi District,
Ghana. IOSR Journal of Research & Method in Education (IOSR-JRME), 7(5), 38-43.
https://doi.org/10.9790/7388-0705063843
Khalid, M. (2017). Fostering problem solving and performance assessment among
Malaysian mathematics teachers. Sains Humanika, 9(1–2), 51–55.
https://doi.org/10.11113/sh.v9n1-2.1098
Kloosterman, P., Raymond, A. M., & Emenaker, C. (1996). Students' Beliefs about
Mathematics: A three-year Study. The Elementary School Journal, 97(1), 39-56.
Kloosterman, P., & Stage, F. K. (1992). Measuring Beliefs about Mathematical Problem
Solving. School Science and Mathematics, 92(3), 109-115.
Lee, J. H., Abdullah, F. A. P., & Bunyamin, M. A. H. (2013). Aplikasi konsep fizik dalam
menyelesaikan masalah yang berasaskan STEM bagi pelajar tingkatan enam atas.
In 2nd International Seminar on Quality and Affordable Education (pp. 470–481).
Universiti Teknologi Malaysia, Malaysia.
Liljedahl, P., Santos-Trigo, M., Malaspina, U., & Bruder, R. (2016). Problem Solving in
Mathematics Education. Switzerland: Springer International Publishing.
Lodico, M. G., Spaulding, D. T., & Voegtle, K. H. (2010). Methods in Educational Research:
From theory to practice (Vol. 28). John Wiley & Sons.
Loh, M. Y., & Lee, N. H. (2019). The Impact of Various Methods in Evaluating
Metacognitive Strategies in Mathematical Problem Solving. In Mathematical
Problem Solving (pp. 155-176). Springer, Cham. ttps://doi.org/10.1007/978-3-030-
10472-6_8
Mason, J. (2016). Part 1 Reaction: Problem Posing and Solving Today. In P. Felmer, E.
Pehkonen, & J. Kilpatrick (Eds.), Posing and Solving Mathematical Problems (pp.
109–113). Switzerland: Springer. https://doi.org/10.1007/978-3-319-28023-3_7
Mills, G. E., & Gay, L. R. (2016). Educational Research: Competencies for Analysis and
Applications (11th. ed.). Boston: Pearson Education.
Ministry of Education Malaysia. (2013). Malaysia Education Blueprint 2013-2025 (Preschool
to Post-Secondary Education). Education. Putrajaya: Ministry of Education
Malaysia.
Ministry of Education Malaysia. (2016). Malaysian Education Blueprint (2013-2015) Annual
Report 2015. Putrajaya: Ministry of Education Malaysia.
NCTM. (2010). Why is teaching with problem solving necessary to student learning?
National Council of Teachers of Mathematics, 13(12), 1–6.
https://doi.org/10.1016/S2213-8587(14)70016-6
Ong, H. C., & Lim, J. S. (2014). Identifying factors influencing mathematical problem
solving among matriculation students in Penang. Pertanika Journal of Social Science
and Humanities, 22(1), 393–408.
O’Neil, H. F., & Abedi, J. (1996). Reliability and Validity of A State Metacognitive
Inventory: Potential for Alternative Assessment. Journal of Educational Research,
89(4), 234–245. https://doi.org/10.1080/00220671.1996.9941208
Özcan, Z. Ç., & Gümüş, A. E. (2019). A modeling study to explain mathematical problem-
solving performance through metacognition, self-efficacy, motivation, and
anxiety. Australian Journal of Education, 63(1), 116-134.
https://doi.org/10.1177/0004944119840073
Öztürk, M., Akkan, Y., & Kaplan, A. (2020). Reading comprehension, Mathematics self-
efficacy perception, and Mathematics attitude as correlates of students’ non-
routine Mathematics problem-solving skills in Turkey. International Journal of
Mathematical Education in Science and Technology, 51(7), 1042-1058.
https://doi.org/10.1080/0020739X.2019.1648893
116
Pallant, J. (2020). SPSS survival manual: A step-by-step guide to data analysis using IBM SPSS.
Routledge.
Peteros, E., Columna, D., Etcuban, J. O., Almerino, Jr., P., & Almerino, J. G. (2019). Attitude
and Academic Achievement of High School Students in Mathematics Under the
Conditional Cash Transfer Program. International Electronic Journal of Mathematics
Education, 14(3), 583-597. https://doi.org/10.29333/iejme/5770
Polya, G. (1957). How to solve it: A new aspect of mathematical method (2nd. ed.). Princeton, NJ:
Princeton University Press.
Prendergast, M., Breen, C., Bray, A., Faulkner, F., Carroll, B., Quinn, D., & Carr, M. (2018).
Investigating secondary students beliefs about mathematical problem-
solving. International Journal of Mathematical Education in Science and
Technology, 49(8), 1203-1218. https://doi.org/10.1080/0020739X.2018.1440325
Rincon, G. A., Cézar, R. F., & Hernandez, C. F. (2020, March). Beliefs about mathematics
and academic performance: A descriptive-correlational analysis. In Journal of
Physics: Conference Series (Vol. 1514, No. 1, p. 012021). IOP Publishing.
https://doi.org/10.1088/1742-6596/1514/1/012021
Salkind, N. J. (2012). Exploring Research (8th. ed.). New Jersey: Pearson Education, Inc.
Salim, N. R., Gopal, K., & Ayub, A. F. M. (2018, June). The influence of statistics self-
efficacy towards statistics engagement among undergraduate students. In AIP
Conference Proceedings (Vol. 1974, No. 1, p. 050005). AIP Publishing LLC.
https://doi.org/10.1063/1.5041705
Simegn, E. M., & Asfaw, Z. G. (2018). Assessing the Influence of Attitude Towards
Mathematics on Achievement of Grade 10 and 12 Female Students in Comparison
with Their Male Counterparts: Wolkite, Ethiopia. International Journal of Secondary
Education, 5(5), 56. https://doi.org/10.11648/j.ijsedu.20170505.11
Stylianides, A. J., & Stylianides, G. J. (2014). Impacting positively on students’
mathematical problem solving beliefs: An instructional intervention of short
duration. Journal of Mathematical Behavior, 33(1), 8–29.
https://doi.org/10.1016/j.jmathb.2013.08.005
Tambunan, H. (2019). The Effectiveness of the Problem Solving Strategy and the Scientific
Approach to Students' Mathematical Capabilities in High Order Thinking
Skills. International Electronic Journal of Mathematics Education, 14(2), 293-302.
https://doi.org/10.29333/iejme/5715
Tarmizi, R. A., Kargar, M., & Saadati, F. (2015). Correlates of Mathematics Attitude and
Problem Solving Behaviour among Teheran, Republic of Iran Year 10 Students.
Jurnal Pendidikan Sains Dan Matematik Malaysia, 5(1), 18–34.
Tapia, M., & Marsh, G. E. I. (2004). An Instrument to Measure Mathematics Attitudes.
Academic Exchange Quarterly, 8(2), 16–21.
Usher, E. L., & Pajares, F. (2009). Sources of Self-efficacy in Mathematics: A validation
study. Contemporary Educational Psychology, 34(1), 89–101.
Veenman, M. V., & van Cleef, D. (2019). Measuring Metacognitive Skills for Mathematics:
Students’ Self-reports Versus Online Assessment Methods. ZDM, 51(4), 691-701.
https://doi.org/10.1007/s11858-018-1006-5
Wang, G., Zhang, S., & Cai, J. (2019). Chinese High School Students’ Mathematics-Related
Beliefs and Their Perceived Mathematics Achievement: A Focus on Teachers’
Praise. EURASIA Journal of Mathematics, Science and Technology Education, 15(7), 1-
12. https://doi.org/10.29333/ejmste/105875
Warner, S., & Kaur, A. (2017). The perceptions of teachers and students on a 21st Century
mathematics instructional model. International Electronic Journal of Mathematics
Education, 12(2), 193-215.
117

Vol. 20, No. 3, pp. 117-131, March 2021
What do Malaysian ESL Teachers Think About

Flipped Classroom?
Ngo Hui Kiang and Melor Md Yunus

Universiti Kebangsaan Malaysia, Bangi, Selangor, Malaysia
https://orcid.org/0000-0002-8398-0731
https://orcid.org/0000-0001-7504-7143
Abstract. Student-centred teaching and learning methods are favoured

for promoting active and collaborative learning. In the flipped classroom
approach, student-centred teaching and learning are the focus in the
learning process. The approach has been gaining attention from educators
at all levels worldwide. The present study was aimed at exploring
Malaysian primary ESL (English as a Second Language) teachers’
perceptions of the implementation of flipped classroom. The study used
the mixed method design, a Likert-scale questionnaire and an open-
ended question. This research employed a whole population sample,
which consisted of primary ESL teachers in a rural area in Malaysia. The
findings showed that the teachers have positive perceptions of the flipped
classroom approach as well as some concerns regarding its
implementation, such as the availability of ICT (information and
communications technology) gadgets and Internet connection in rural
areas. Teachers’ perceptions of flipped classrooms can be used to make
suggestions or give ideas to the authorities for improving the
infrastructure in rural areas to help teachers and students. It is
recommended that future studies be carried out with a larger and more
diverse population to gather a more detailed picture of Malaysian
teachers’ perceptions of the flipped classroom approach.
Keywords: flipped classroom; primary ESL teachers; teachers’

perceptions; challenges; education
1. Introduction
In this era, the teaching and learning of the English language require the
integration of technology. Students find traditional teaching methods less
interesting; they prefer modern styles of teaching. Hence, teachers need to be
innovative and creative by integrating modern teaching approaches to attract
students and ensure better learning outcomes. Student-centred teaching and
learning methods are preferred for promoting collaborative learning among
students. Flipped classroom is a teaching approach that emphasises student-
centred learning in the classroom, and it is gaining popularity globally. In a
©Authors
118
flipped classroom, students are given the chance to grow as independent learners
(Fauzan & Ngabut, 2018; Sharma, 2018; Yang & Chen, 2020; Zainuddin & Halili,
2016).
In this new class setting, the content is delivered outside the classroom, and the
in-class time is employed for more student-centred activities (Jeong et al., 2018).
The students are required to study the materials before coming to the class for the
lesson. Hence, the classroom time is dedicated to more meaningful learning such
as discussions regarding the topic given to the students. The rapid development
of multimedia technology has contributed to the rising trend of using the digital
environment in learning, thereby promoting new learning models (Wang, 2020).
Flipped classroom’s popularity has been growing in many countries, and

therefore, many researchers have explored teachers’ and students’ perceptions of
and attitudes towards this model (Khaled, 2019). Several studies have been
conducted on teachers’ perceptions of the implementation of flipped classroom
(Ansori & Nafi’, 2019; Gough et al., 2017; Khaled, 2019; Yang, 2017; Yang & Chen,
2020). However, according to Rahman et al. (2020), the studies conducted on
flipped learning in Malaysia are fewer in number compared to the studies
conducted around the globe.
Most of the past studies focused on students or teachers from high schools,
universities, colleges or other higher education institutions rather than rural
primary schools, especially in the Malaysian context. The students and teachers
of rural schools are often left out due to their lack of technologies and facilities.
Hence, this study was aimed at exploring Malaysian ESL teachers’ general
perceptions of the implementation of the flipped classroom approach in their
daily teaching processes; this was accomplished through a survey and an open-
ended question, and teachers from rural areas were particularly focused on as the
studies conducted involving them are still few (Rahman et al., 2020). As the focus
was on schools in rural areas, problems such as lack of ICT and infrastructure
were expected to be the challenges. It is hoped that through the results of this
study, the authorities are able to have a clearer picture of the real conditions in
rural areas and, hence, provide better opportunities to the students. So, the
research question of this study is as follows: What are the Malaysian primary ESL
teachers’ general perceptions of the implementation of the flipped classroom
approach?
2.1 The Theory of the Flipped Classroom Approach
According to Eppard and Rochdi (2017), the revised version of Bloom’s taxonomy
is relevant to the concept of flipped learning, which involves independent
learning and assimilation of learning. The independent learning occurs outside of
the classroom, while the assimilation of information, which needs greater critical
thinking and reasoning, occurs in the classroom, under the guidance of teachers.
Regarding the implementation of the flipped classroom approach, Zainuddin and

Halili (2016) claimed that the lower levels of the cognitive domain are normally
119
engaged outside the classroom, where students study the materials prepared by
the teachers, and the reading is completed independently by the students; the
higher cognitive levels are engaged inside the classroom, during collaborative
discussions with teachers and peers. A flipped classroom allows the students to
have more time and opportunities to enhance their performance in higher-level
cognitive activities.
Zainuddin and Halili (2016) also compared conventional and flipped classrooms
in terms of students’ achievement of the higher-order thinking skills (HOTS)
mentioned in Bloom’s revised taxonomy. This comparison is shown in Table 1.
Table 1: A comparison between traditional classroom approach and flipped classroom

approach by Zainuddin and Halili (2016)
Levels of Traditional Classroom Flipped Classroom
Learning Tools Tools
Remembering Face-to-face lecture Watching recorded lecture, reading
materials and online materials
independently
Understanding Question and answer Reflection, peer discussion and

(Q&A) collaboration
Analysing Homework Classroom activities such as collaborative
discussion
Applying Homework or not Projects, presentations, peer evaluation
Evaluating applicable and instructor evaluation
Creating
It is noticeable that the flipped classroom approach is able to help students use
HOTS in the classroom through peer discussion and collaboration. Furthermore,
Sharma (2018) stated that the traditional method of teaching always focuses on
the lower level of Bloom’s taxonomy in the classroom, and the students are left to
work independently on the higher levels’ skills with homework or additional
exercises.
2.2 Concepts of the Flipped Classroom Approach

Flipped classroom is a type of blended learning in which the teachers choose and
prepare pre-class work, such as online materials, and the in-class time is devoted
to collaborative tasks among students (Ho, 2020; Unal & Unal, 2017). It is well
documented in the literature that interaction among learners helps learning take
place (McCallum et al., 2015; Murray et al., 2015; Nguyen, B., et al., 2016).
Moreover, Chun and Sathappan (2020) defined flipped classroom in terms of the
shifted roles of learners and teachers: the learners are responsible for their own
learning, and the teachers provide guidance and assistance to the learners. In a
flipped classroom, different learning activities and platforms are used to teach the
lessons (Yousufi, 2020). Discussion and collaboration among learners are the focus
throughout the lessons.
In a flipped classroom setting, the teaching content and activities are provided
online to students in advance so that they may read the materials before the next
lesson (Jones, 2016; Yousufi, 2020). Hence, the students are familiar with the
120
content of the lesson and the activities planned. Fauzan and Ngabut (2018) stated
that a flipped classroom helps avoid teacher-centred lessons as teachers have the
chance to use the classroom time for countless student-centred activities. Al-
Ibrahim (2019) asserted that the flipped classroom approach has positive effects
on students’ learning as it increases their motivation to learn and keeps them
engaged in student-centred activities, which help in developing collaborative,
research and inquiry skills. To implement flipped learning effectively, technical
infrastructure, particularly high-speed Internet, should be reinforced, and
students should be encouraged and motivated to study using various sources and
course contents offered online (Say & Yildirim, 2020).
2.3 Past Studies on Flipped Learning

Su Ping et al. (2020) conducted a study on low-proficiency students’ experience of
and reflections on a flipped writing programme. The students believed that by
viewing lecture videos and reflecting upon their content before the lessons, they
could be better prepared, which contributed to better engagement and
participation in classroom activities in the flipped classroom compared to a
conventional classroom. Immediate feedback from their teachers and peers also
helped the low-proficiency students engage in discussions.
A study analysing the implementation of blended learning was conducted by

Mahalli et al. (2019). The advantages of adopting blended learning models are that
they can make it easier for students to learn anytime and anywhere and enable
them to learn independently, at their own pace. In addition, they will prepare
students for what will be learnt in class and stimulate their curiosity and interest
regarding their classroom experiences.
Nguyen, H. A. V., et al. (2019) conducted a study investigating the advantages of

flipped classroom in the teaching of grammar. The results of the study showed
that the approach enables students to be self-paced in learning, creates
opportunities for them to express their opinions, saves time for in-class
communicative activities and facilitates the learning of English grammar in a
communicative manner.
Abdelrahman et al. (2017) conducted a study involving Sudanese students and

focused on the effects of the flipped classroom approach on students’ writing
proficiency and their satisfaction with the use of the approach. The students
agreed that the approach helped them improve their writing. The online video
lessons prepared by the teacher could be watched anytime and anywhere, and
they enabled students to have a better understanding. The students also stated
that their teacher was more attentive towards them. Discussions and more
interactive group work replaced the traditional process of the teacher delivering
lessons. The researchers proclaimed that thanks to the flipped classroom
approach, even shy students responded to the questions in the online form. Hence,
this approach may help in encouraging collaborative work among students.
Goedhart et al. (2019) studied the outcomes of a flipped classroom experiment

performed during a master's course at the Netherlands’ Vrije Universiteit
121
Amsterdam. The positive reviews from both the teacher and the students
indicated that the experiment was a success. Deeper learning was enabled
through a mixture of personalised pre-class learning and peer-learning classroom
activities. Fathi and Rahimi (2020) conducted a study in an Iranian university.
They found that the EFL students in the flipped classroom greatly outperformed
those in the non-flipped classroom in terms of global writing performance and
writing fluency.
Ali et al. (2021) explored the use of flipped classroom in teaching English as a
second language to 5th-grade learners in a Pakistani school. The students who
were taught using the flipped classroom approach as a teaching method scored
higher in a test compared to those who were taught using the traditional method.
The authors also stated that flipped classroom is able to upgrade basic reasoning
and encourage communication among and participation of learners (Ali et al.,
2021). Kurt (2017) conducted a study at a higher education institution in Turkey.
It was found that pre-service teachers perceived themselves to be more motivated
in a flipped classroom as they were allowed to participate actively in the
classroom and because they could watch the videos as many times as they needed
before the lesson, at their own pace.
2.4 Past Studies on Teachers’ Perceptions of the Flipped Classroom Approach

A flipped classroom allows students to be more active in their learning and
provides them with more opportunities to collaborate with their friends in the
classroom, thereby building their confidence in using the language. Gough et al.
(2017) found that for teachers, the greatest advantages of the flipped classroom
approach are the benefits it provides to absent students and struggling students,
as they can re-watch the recorded lessons, the increased interaction it enables
between students and teachers and the increased time it allows for various
learning activities. In addition, Gough et al. (2017) discussed the challenges faced
by the teachers. The teachers who participated in the study agreed that accessing
technology outside of school could be an issue for some learners.
In a study, some teachers considered flipped classroom creative and felt that it
may be useful in teaching English grammar (Yang, 2017); however, they also
believed that it may only be practical for more ambitious learners, and the extra
job of making pre-lesson videos was their main concern. Based on their study,
Ansori and Nafi’ (2019) claimed that this approach facilitates active learning,
enhances teamwork among learners and encourages autonomous learning. The
participants in their study mentioned that the challenges in the implementation
of this approach include lack of supporting facilities, technical and technological
problems, and the difficulty of producing flipped learning materials. As for the
advantages, in-class activities involving discussion and presentation can increase
classroom communication (Ansori & Nafi’, 2019; Jaramillo, 2019).
Khaled (2019) conducted a study to investigate EFL teachers’ perceptions of the

flipped classroom model in the Palestinian context. The data collected showed the
positive impacts of the model. In interviews, the teachers stated that it made the
students more responsible for their learning outside the classroom. Furthermore,
this model gave the students more freedom to search for new information related
122
to the lesson before coming to the classroom. Some teachers felt that the model
would help them in planning, teaching and communicating effectively with
students using technology.
Flipped classroom is a model that enables students to focus and interact during
their learning process. It facilitates flexible and independent learning as students
can learn anytime and anywhere and individually, at their own pace (Gilboy et
al., 2015; Kurt, 2017; Mahalli et al., 2019; McCallum et al., 2015). In addition, this
approach can save a lot of the time that is spent on teaching and ensure that more
class time is used for student practice; it makes the learners more responsible for
their learning and further enhances their independent learning skills (Yang &
Chen, 2020). Hence, we can see that the past studies on teachers’ perceptions of
the flipped classroom approach revealed positive responses and some concerns
and challenges related to the implementation of the approach.
3. Methodology
3.1 Research Design
This mixed method research data was collected via a 5-point Likert-scale
questionnaire as a Google form and an open-ended question. The questionnaire
consisted of 20 questions, with the answers ranging from “Strongly Agree” to
“Strongly Disagree”.
3.2 Research Respondents

For this study, the research participants were selected via the whole population
sampling technique. The selected population consisted of English language
teachers from the primary schools of a district in Malaysia; the total population
was 114. Of them, only 78 responded to the online questionnaire; thus, the
response rate was 68%, which is acceptable according to Creswell (2014), who
stated that a response rate of 50% is acceptable in survey research.
3.3 Research Instrument

Two research instruments were used in this research: a survey questionnaire and
an open-ended question. The questionnaire used in the survey was based on
previous studies by Khaled (2019) and Alsowat (2016). A total of 20 items, with a
5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree), were
formulated, and the questionnaire was distributed online. The questionnaire was
piloted on 31 teachers with similar teaching backgrounds. The reliability of the
questionnaire was verified using the software IBM SPSS (Statistical Package for
the Social Sciences) Version 25, and it obtained the Cronbach’s alpha reliability
coefficient with the value of 0.95. Vaske et al. (2017) stated that a value more than
or equal to 0.8 (≥ 0.8) is an excellent indication of good internal consistency. The
open-ended question in the survey was created to explore the teachers’ perception
of a flipped classroom in primary school settings, particularly involving primary
school students and English language lessons.
3.4 Data Collection Procedure

The survey was conducted by providing the participants with the Google form’s
link. It was shared with all the English language teachers at the research site
123
through the online social application WhatsApp. The data collected through the
form was analysed descriptively and thematically.
3.5 Data Analysis Procedure

The quantitative data collected through the questionnaire was analysed using the
software IBM SPSS Version 25. The means, frequencies and percentages of the
Likert-scale questionnaire data were used for the descriptive statistical analysis.
To analyse the teachers’ perceptions of the flipped classroom approach and better
understand their responses, the researchers used a scale developed by Ibrahim et
al. (2015), which is shown in Table 2. The qualitative data was analysed
thematically through its categorisation into themes and sub-themes.
Table 2: Interpretation of the mean score, as proposed by Ibrahim et al. (2015)

Mean Score (M) Interpretation of Mean Score
1.00–2.00 Low
2.01–3.00 Moderately Low
3.01–4.00 Moderately High
4.01–5.00 High
4. Findings
This study was aimed at identifying primary ESL teachers’ perceptions of the
implementation of the flipped classroom approach. To achieve this aim, a mixed
method research design was applied. The results of the quantitative data collected
were related to the research question, “What are the Malaysian Primary ESL
teachers’ general perceptions of the implementation of the flipped classroom
approach?” A total of 78 teachers (68% of the total population) answered the
Likert-scale questionnaire. The researchers tabulated the frequencies, percentages
and means to better illustrate the data; this can be seen in Table 3 and Table 4. As
for the qualitative data, the teachers’ responses were coded and analysed
thematically, as shown in Table 5.
Table 3. Frequencies, percentages and means of teachers’ perceptions of the

advantages of flipped classroom approach with regard to the teacher’s role
Mean (M)
Disagree
Disagree
Strongly
Strongly
Not sure
Agree
Agree
Interpretation of
Item
Mean Score
I have knowledge of the flipped 3 3 20 41 11 3.69 Moderately High

classroom methodology. (4%) (4%) (26%) (52%) (14%)
I practise the flipped classroom 6 13 33 21 5 3.08 Moderately High
approach. (8%) (17%) (42%) (27%) (6%)
I would love to practise flipped 0 0 15 42 21 4.08 High
classroom approach. (0%) (0%) (19%) (54%) (27%)
In flipped classroom approach, 0 0 6 45 27 4.27 High
teachers become organizers, mentors, (0%) (0%) (8%) (58%) (34%)
and facilitators.
In flipped classroom approach, the 0 2 20 32 24 4.00 High
teacher can communicate a lot with (0%) (2%) (26%) (41%) (31%)
students.
Flipped classroom approach allows 0 0 9 38 31 4.28 High
teachers to broaden and deepen (0%) (0%) (11%) (49%) (40%)
124
students’ learning within limited

class hours.
Flipped classroom approach 1 0 5 43 29 4.27 High
involves less lecturing and more (1%) (0%) (7%) (55%) (37%)
collaborative activity in the
classroom.
Flipped classroom approach helps 0 1 7 42 28 4.24 High
teachers to maximise the lesson (0%) (1%) (9%) (54%) (36%)
content.
Flipped classroom approach allows 0 0 10 40 28 4.23 High
class time to be more productive for (0%) (0%) (13%) (51%) (36%)
in-depth discussion.
provides a solution to the problem (0%) (1%) (12%) (50%) (37%)
of lack of time, which hinders
teachers in the classroom.
Overall mean score 4.04 High
As seen in Table 3, the overall mean score of teachers’ perceptions of the

advantages related to the teacher’s role was 4.04, which is considered as a high
level of perception. The means of the items in the questionnaire ranged from 3.08
to 4.28, which is moderately high to high according to Ibrahim et al. (2015).
Table 4. Frequencies, percentages and means of teachers’ perceptions of the

advantages of flipped classroom approach with regard to students’ learning
Mean (M)
Disagree
Disagree
Strongly
Strongly
Not sure
Agree
Agree
Interpretation of
Item
Mean Score

recorded lessons in the form of videos (0%) (0%) (8%) (51%) (41%)
help students because they can re-
watch the part of lessons that they do
not understand.
In flipped classroom approach, the 0 0 6 42 30 4.31 High
materials given before class help (0%) (0%) (8%) (54%) (38%)
students practise knowledge with
their friends.
Flipped classroom approach makes 0 0 8 44 26 4.23 High
each student responsible for coming (0%) (0%) (10%) (57%) (33%)
to class with a basic understanding of
the subject.
Flipped classroom approach enables 0 0 9 40 29 4.26 High
students to construct knowledge (0%) (0%) (12%) (51%) (37%)
outside of class.
Flipped classroom approach reverses 0 0 10 39 29 4.24 High
the role of the students from passive (0%) (0%) (13%) (50%) (37%)
observers to active participants.
strengthens students’ preparedness (0%) (0%) (8%) (55%) (37%)
before the class.
Flipped classroom approach has the 0 0 7 41 30 4.29 High
potential to facilitate active learning (0%) (0%) (9%) (53%) (38%)
during the lesson.
125
In flipped classroom approach, more 0 0 8 44 26 4.23 High

learning can be done outside the (0%) (0%) (10%) (57%) (33%)
classroom.
students have a sense of responsibility (0%) (1%) (6%) (58%) (35%)
for their learning.
students have more flexible learning (0%) (0%) (5%) (59%) (36%)
time.
Overall mean score 4.28 High
As seen in Table 4, the overall mean score of teachers’ perceptions of the

advantages related to students’ learning was 4.28, which is weighed as a high level
of perception. All the items were measured as high, as they ranged from 4.23 to
4.33, as per the scale of Ibrahim et al. (2015).
Table 5: Selected excerpts from the open-ended question

Question: What do you think about flipped classroom approach?
Themes Sub-themes Selected excerpts
Perceived Prepare the students (a) Learners could activate their schemata before the
advantages for lesson teachers convey the lesson. They will be more alert, and
(mentioned by 15 it would enhance learning readiness. [T41]
participants, 19.2%) (b) Flipped classroom approach serves as another
alternative to preparing the pupils with prior knowledge
on the subject matter that will be further discussed in
their formal classroom-based lesson beforehand. [T47]
Promote active (c) Flipped classroom approach is relevant in the 21st-
learning century learning era with pupils being the active
(mentioned by 10 participant during the learning process. [T46]
participants, 12.8%)
Promote independent (d) A good method to encourage the pupil to take on their
learning learning while given a task by teachers. [T4]
(mentioned by 8 (e) It’s an independent self-learning in which they are
participants, 10.3%) responsible for their own learning. [T18]
Maximise learning (f) A lesson that allows learners to understand in depth
(mentioned by 8 without time constraints. [T27]
participants, 10.3%) (g) It helps teacher to maximise their teaching and learning
contents outside of the classroom. [T29]
(h) It’s a very good way to maximise learning in limits of the
teaching periods especially in SJKC (Chinese medium
national school) schools where English periods are lesser
compared to SK (national school) schools. [T68]
Promote student- (i) It promotes student-centred learning. [T5]
centred learning (j) It's good because it promotes student-centred learning.
(mentioned by 7 [T7]
participants, 9.0%)
Good and worth (k) Basically, I think that flipped classroom approach is one
implementing of the methods or techniques of teaching that should be
(mentioned by 6 implemented in this globalised era so as to compare to
participants, 7.7%) the traditional and dull teaching method. [T72]
Perceived Not applicable to (l) I think it’s suitable for some students in town areas/parts
challenges rural area of suburban areas only. It’s because not all the students
(mentioned by 7 have the facilities at home. Especially in my area. [T15]
participants, 9%)
126
Perceived Lack of ICT gadgets (m) … not all the students have the facilities at home.
challenges and facilities Especially in my area. [T15]
(mentioned by 6 (n) Cannot be carried out if pupils didn’t have the
participants, 7.7%) required tools. [T32]
The participants who answered the questionnaire also responded to the open-
ended question. The selected excerpts were coded and analysed thematically and
then categorised into themes and sub-themes. The end of every excerpt is attached
with a [Tn], where n indicates the participant. Thus, as seen in Table 5, through
qualitative thematic analysis of the open-ended question, two major themes were
identified in relation to the perceptions of the flipped classroom. The themes were
perceived advantages and perceived challenges. ‘Perceived advantages’ is the most
significant perception; the sub-theme ‘Prepare the students for lesson’ was
mentioned most frequently by the participants. The theme ‘Perceived challenges’
had two sub-themes: ‘Not applicable to rural area’ and ‘Lack of ICT gadgets and
facilities’.
5. Discussions
As shown in Table 3, most teachers agreed with ‘Flipped classroom approach allows
teachers to broaden and deepen students’ learning within limited class hours’ (M = 4.28),
which is probably because they are aware that they can collaborate and discuss
more with the students while implementing this approach. The data is supported
by the participants’ responses to the open-ended question, under the sub-theme
‘Maximise learning’. T68 wrote, “It’s a very good way to maximise learning in limits of
the teaching periods especially in SJKC (Chinese medium national school) schools where
English periods are lesser compared to SK (national school) schools.” This is supported
by the study by Yang and Chen (2020), which also showed that this approach
helps increase the teaching and learning effectiveness within the restricted class
hours. The students in a flipped classroom have the opportunity to adjust their
study times outside the classroom (Fauzan & Ngabut, 2018).
The item ‘I practice flipped classroom approach’ (Table 3) had the lowest mean score
(M = 3.08); 67% teachers do not implement the method, and this is considered as
a moderately high level of perception. This is probably due to Internet connection
issues in the rural area and unavailability of gadgets at school or home. This is
supported by the qualitative data from the open-ended question, under the sub-
themes ‘Not applicable to rural area’ and ‘Lack of ICT gadgets and facilities’. T15 stated,
“… not all the students have the facilities at home. Especially in my area.” This is
supported by past studies (Ansori & Nafi’, 2019; Gough et al., 2017; Say &
Yildirim, 2020). This is one of the challenges perceived by the teachers in
implementing the flipped classroom approach.
Table 4 shows that most teachers agreed with the item ‘In flipped classroom
approach, recorded lessons in the form of videos help students because they can re-watch
the part of lessons that they do not understand’ (M = 4.33). Students may need to re-
watch the entire videos or any parts of the lessons to develop a better
understanding of the concepts taught. This data is supported by the responses to
the open-ended question, under the sub-theme ‘Maximise learning’, which was
mentioned by eight participants. T27 claimed, “A lesson that allows learners to learn
127
in depth without time constraints”. The data is supported by past studies (Gough et
al., 2017; Su Ping et al., 2020; Sharma, 2018), which stated that struggling learners
are able to re-watch the recorded lessons prepared by the teachers.
As shown in Table 4, ‘Flipped classroom approach makes each student responsible for
coming to class with a basic understanding of the subject’ had the lowest mean score
of all the items (M = 4.23). This is probably because some students have less
awareness about their responsibilities with regard to learning. Although it is the
lowest mean score, it is regarded as a high level of perception (of the flipped
classroom approach). The data is further supported by the responses to the open-
ended question, under the sub-theme ‘Prepare the students for lesson’; this theme
was mentioned by 15 participants (19.2%), as seen in Table 5. One of the teachers,
T47, stated, “Flipped classroom approach serves as another alternative of learning to
preparing the pupils with prior knowledge on the subject matter that will be further
discussed in their formal classroom-based lesson beforehand.”
In addition, T41 (see Table 5) mentioned, “Learners could activate their schemata
before the teachers convey the lesson. They will be more alerted and enhance learning
readiness.” This data aligns with the findings of past studies (Jones, 2016; Khaled,
2019; Mahalli et al., 2019; B. Nguyen, B., et al., 2016; Nguyen, H. A. V., et al., 2019;
Yang & Chen, 2020). The study by Khaled (2019) indicated that students feel more
confident and responsible for their learning when they prepare their own learning
material before coming for the lessons.
As shown in Table 4, ‘In flipped classroom approach, more learning can be done outside
the classroom’ also has the lowest mean score (M = 4.23), which is indicated as high
perception. A study by Fauzan and Ngabut (2018) showed that students can
explore the learning materials further as this model allows them to study the
materials at home and continue their learning in the classroom. Students get
trained to be independent learners when they are left to explore the learning by
themselves; this is supported by past studies that saw students grow as
independent learners (Fauzan & Ngabut, 2018; Nguyen, B., et al., 2016; Yang &
Chen, 2020; Zainuddin & Halili, 2016). The findings of these past studies confirm
the importance of shifting from the traditional teaching methods to more effective
teaching practices such as the flipped classroom approach, which directly
involves the students in the learning process. This is further supported by the sub-
theme ‘Maximise learning’ (see Table 5). T29 stated, “It helps teacher to maximise their
teaching and learning contents outside of the classroom.” Generally, teachers are aware
of the advantages related to students’ learning. The findings of previous
researches revealed that teachers in general are satisfied and agree on the
advantages of the flipped classroom approach (Fauzan & Ngabut, 2018; Gough et
al., 2017; Jones, 2016; Khaled, 2019; Yang & Chen, 2020).
Another sub-theme that often came up in the responses to the open-ended

question was ‘Promote active learning’, mentioned by 10 participants (12.8%), as
shown in Table 5. This was probably because after reviewing the materials online,
students have a better understanding and are able to participate actively during
class. T46 highlighted, “Flipped classroom approach is relevant in 21st century learning
era with pupils being the active participant during the learning process.” Past studies
128
(Abdelrahman et al., 2017; Ali et al., 2021; Kurt, 2017; Nguyen, B., et al., 2016;
Sharma, 2018) have claimed that flipped classroom approach is able to promote
active learning in the classroom. The other perceived advantages’ sub-themes are
‘Promote independent learning’, ‘Promote student-centred learning’ and ‘Good and
worth implementing’. In general, the teachers had positive perceptions of the
implementation of flipped classroom approach, but they also stated some
concerns, as shown in Table 5.
6. Conclusion and Implications

The present study was aimed at investigating Malaysian primary ESL teachers’
perceptions of the implementation of the flipped classroom approach. The
quantitative results reported a high level of perception. The excerpts from the
participants signified the benefits of the approach, such as preparation of students
for the lessons, promotion of active learning, independent learning and student-
centred learning and maximisation of learning, as compared to the traditional
methods of teaching. However, the teachers implementing the approach face
some challenges: non-applicability to rural areas and lack of ICT gadgets and
facilities. The flipped classroom approach is still worth implementing in the
classroom as it has advantages for the students and teachers. The obstacles to its
implementation should be overcome by the relevant parties. Teachers’
perceptions of flipped classrooms can be used as a basis for giving suggestions or
ideas to the authorities for improving the infrastructure in rural areas to help
teachers and students. With a more robust and modern infrastructure, the
teachers and students in the rural areas will have the opportunity to follow in the
footsteps of the schools in the urban areas. A limitation of this study is that it only
focused on primary ESL teachers. It is suggested that future studies involve a
larger and more diverse population of teachers in order to get a more detailed
picture of Malaysian teachers’ perceptions of the flipped classroom approach.
Students’ perceptions can also be taken into account to know and understand the
advantages or challenges of this method.
7. References
Abdelrahman, L. A. M., Dewitt, D., Alias, N., & Rahman, M. N. A. (2017). Flipped learning
for ESL writing in a Sudanese school. Turkish Online Journal of Educational
Technology, 16(3), 60–70.
Ali, M. M., College, G., & Road, W. (2021). The application of flipped classroom approach
on the academic performance of Pakistani ESL learners. Elementary Education
Online, 20(5), 2552–2561. https://doi.org/10.17051/ilkonline.2021.05.276
Al-Ibrahim, A. (2019). Deaf and hard of hearing students’ perceptions of the flipped
classroom strategy in an undergraduate education course. European Journal of
Educational Research, 8(1), 325–336. https://doi.org/10.12973/eu-jer.8.1.325
Alsowat, H. (2016). An EFL flipped classroom teaching model: Effects on English language
higher-order thinking skills, student engagement and satisfaction. Journal of
Education and Practice, 7(9), 108–121.
Ansori, M., & Nafi’, N. N. (2019). English teachers’ perceived benefits and challenges of
flipped classroom implementation. Journal of English Education and Linguistics
Studies, 5(2), 211–228. https://doi.org/10.30762/jeels.v5i2.820
Chun, T. W., & Sathappan, R. (2020). Using flipped classroom approach to teach adjectives
for Malaysian year 4 Chinese ESL learners 41. The English Teacher, 49(1), 41–54.
129
Creswell, J. W. (2014). Educational research: Planning, conducting and evaluating quantitative

and qualitative research (5th ed.). Lincoln, United States: Pearson.
Eppard, J., & Rochdi, A. (2017). A framework for flipped learning. In Proceedings of the 13th
international conference on mobile learning (pp. 33–40).
Fathi, J., & Rahimi, M. (2020). Examining the impact of flipped classroom on writing
complexity, accuracy, and fluency: A case of EFL students. Computer Assisted
Language Learning, 1–39. https://doi.org/10.1080/09588221.2020.1825097
Fauzan, A., & Ngabut, M. N. (2018). EFL students’ perception on flipped learning in
writing class. Journal on English as a Foreign Language, 8(2), 115.
https://doi.org/10.23971/jefl.v8i2.792
Gilboy, M. B., Heinerichs, S., & Pazzaglia, G. (2015). Enhancing student engagement using
the flipped classroom. Journal of Nutrition Education and Behavior, 47(1), 109–114.
https://doi.org/10.1016/j.jneb.2014.08.008
Goedhart, N. S., Blignaut-van Westrhenen, N., Moser, C., & Zweekhorst, M. B. M. (2019).
The flipped classroom: Supporting a diverse group of students in their learning.
Learning Environments Research, 22(2), 297–310. https://doi.org/10.1007/s10984-
019-09281-2
Gough, E., DeJong, D., Grundmeyer, T., & Baron, M. (2017). K-12 teacher perceptions
regarding the flipped classroom model for teaching and learning. Journal of
Educational Technology Systems, 45(3), 390–423.
https://doi.org/10.1177/0047239516658444
Ho, J. (2020). Gamifying the flipped classroom: How to motivate Chinese ESL learners?
Innovation in Language Learning and Teaching, 14(5), 421–435.
https://doi.org/10.1080/17501229.2019.1614185
Ibrahim, W. N. A., Bakar, A. R., Asimiran, S., Mohamed, S., & Zakaria, N. S. (2015). Impact
of entrepreneurship education on the entrepreneurial intentions of students in
technical and vocational education and training institutions (TVET) in Malaysia.
International Education Studies, 8(12), 141. https://doi.org/10.5539/ies.v8n12p141
Jaramillo, N. V. (2019). Evaluating a flipped intermediate Spanish course through students
and instructor’s perceptions. JALT CALL Journal, 15(2), 41–62.
https://doi.org/10.29140/jaltcall.v15n2.172
Jeong, J. S., Cañada-Cañada, F., & González-Gómez, D. (2018). The study of flipped-
classroom for pre-service science teachers. Education Sciences, 8(4), 1–11.
https://doi.org/10.3390/educsci8040163
Jones, M. (2016). A case study of blended learning in higher education in Malaysia:
Flipped, flopped or forgotten? Teaching Education, (June), 132-137.
https://doi.org/10.20472/tec.2016.003.012
Khaled, A. D. A. A. A. R. A.-N. (2019). English as a foreign language (EFL) teachers’
perceptions of flipped classroom model in Palestine. Journal of Educational and
Psychological Sciences, 27(03), 29–47.
Kurt, G. (2017). Implementing the flipped classroom in teacher education: Evidence from
Turkey. Educational Technology and Society, 20(1), 211–221.
Mahalli, Nurkamto, J., Mujiyanto, J., & Yuliasri, I. (2019). The implementation of station
rotation and flipped classroom models of blended learning in EFL learning.
English Language Teaching, 12(12), 23. https://doi.org/10.5539/elt.v12n12p23
McCallum, S., Schultz, J., Sellke, K., & Spartz, J. (2015). An examination of the flipped
classroom approach on college student academic involvement. International
Journal of Teaching and Learning in Higher Education, 27(1), 42–55.
http://www.isetl.org/ijtlhe/
Murray, D., Koziniec, T., & McGill, T. (2015). Student perceptions of flipped learning.
Conferences in Research and Practice in Information Technology Series, 160(2013), 57–
130
62. https://www.semanticscholar.org/paper/Student-Perceptions-of-Flipped-
Learning-Murray-Koziniec/7c6f51595edb20ac396ce80b0309b86f3ab538c1
Nguyen, B., Yu, X., Japutra, A., & Chen, C. H. S. (2016). Reverse teaching: Exploring
student perceptions of “flip teaching.” Active Learning in Higher Education, 17(1),
51–61. https://doi.org/10.1177/1469787415616727
Nguyen, H. A. V., Tan, C. K., & Lee, K. W. (2019). The affordances of the flipped classroom
approach in English grammar instruction. International Journal of Education,
Psychology and Counseling, 4(33), 95–106. https://doi.org/10.35631/ijepc.433008
Rahman, S. F. A., Yunus, M. M., & Hashim, H. (2020). The uniqueness of flipped learning
approach. International Journal of Education and Practice, 8(3), 394–404.
https://doi.org/10.18488/journal.61.2020.83.394.404
Say, F. S., & Yildirim, F. S. (2020). Flipped classroom implementation in science teaching.
International Online Journal of Education and Teaching, 7(2), 606–620.
http://iojet.org/index.php/IOJET/article/view/759
Sharma, P. (2018). Flipped classroom: A constructivist approach. International Journal of
Research in Engineering, IT and Social Sciences, 08(08), 164–169.
http://indusedu.org/pdfs/IJREISS/IJREISS_2139_41011.pdf
Su Ping, R. L., Verezub, E., Adi Badiozaman, I. F. bt, & Chen, W. S. (2020). Tracing EFL
students’ flipped classroom journey in a writing class: Lessons from Malaysia.
Innovations in Education and Teaching International, 57(3), 305–316.
https://doi.org/10.1080/14703297.2019.1574597
Unal, Z., & Unal, A. (2017). Comparison of student performance, student perception, and
teacher satisfaction with traditional versus flipped classroom models. International
Journal of Instruction, 10(4), 145–164. https://doi.org/10.12973/iji.2017.1049a
Vaske, J. J., Beaman, J., & Sponarski, C. C. (2017). Rethinking internal consistency in
Cronbach’s alpha. Leisure Sciences, 39(2), 163–173.
/doi.org/10.1080/01490400.2015.1127189
Wang, T. (2020). Technology-enhanced multimodality in encouraging student interaction:
A paradigm of a flipped second language classroom. International Conference on
Education Science and Economic Development (ICESED2019), Malaysia, 116, 293–298.
https://doi.org/10.2991/icesed-19.2020.54
Yang, C. C. R. (2017). An investigation of the use of the “flipped classroom” pedagogy in
secondary English language classrooms. Journal of Information Technology
Education: Innovations in Practice, 16(1), 1–20.
https://scholars.cityu.edu.hk/en/publications/publication(e7062db7-ca0d-
425f-a1ae-2be4522cde74).html
Yang, C. C. R., & Chen, Y. (2020). Implementing the flipped classroom approach in
primary English classrooms in China. Education and Information Technologies, 25(2),
1217–1235. https://doi.org/10.1007/s10639-019-10012-6
Yousufi, U. (2020). An integrative review of flipped classroom model. American Journal of
Educational Research, 8(2), 90–97. https://doi.org/10.12691/education-8-2-4
Zainuddin, Z., & Halili, S. H. (2016). Flipped classroom research and trends from different
fields of study. International Review of Research in Open and Distance Learning, 17(3),
313–340. https://doi.org/10.19173/irrodl.v17i3.2274
131
Appendix A: (Questionnaire & Open-ended Questions)
Title: The ESL Teachers’ Perceptions on Flipped Classroom

This survey is divided into Part A and B.
Part A: Teachers’ perceptions about the advantages of Flipped Classroom on teachers.
Part B: Teachers’ perceptions about the advantages of Flipped Classroom on students.
Please read the following statements and choose the most appropriate response for each
item. = strongly disagree 2 = disagree 3 = not sure 4 = agree 5 = strongly agree
Part A: Teachers’ Perceived Advantages of Flipped Classroom on Teachers

No. Statements Scale
1 2 3 4 5
1 I have knowledge of the Flipped Classroom methodology.
2 I practice the Flipped Classroom.
3 I would love to practice the Flipped Classroom.
4 In Flipped Classroom, teachers become organizers, mentors, and
facilitators.
5 In Flipped Classroom, teachers can communicate a lot with
students.
6 Flipped Classroom allows teachers to broaden and deepen
students' learning within limited class hours.
7 Flipped Classroom involves less lecturing and more
collaborative activity in the classroom.
8 Flipped Classroom helps to maximise the lesson content by
allowing students to prepare the lesson before the class.
9 Flipped Classroom allows class time to be more productive for
in-depth discussion.
10 Flipped Classroom provides a solution to the lack of time that
hinders teachers in the classroom.
Part B: Teachers’ Perceived Advantages of Flipped Classroom on Students

No. Statements Scale
1 2 3 4 5
1 In Flipped Classroom, recorded lessons in the form of videos
help students because they can re-watch the part of lessons that
they do not understand.
2 In Flipped Classroom, the materials given before class helps
students practice knowledge with their friends.
3 Flipped Classroom makes each student responsible for coming
to class with a basic understanding of the subject.
4 Flipped Classroom enables students to construct knowledge
outside of class.
5 Flipped Classroom reverses the role of the students from passive
observers to active participants.
6 Flipped Classroom strengthens students' preparedness before
the class.
7 Flipped Classroom has the potential to facilitate active learning
during the lesson.
8 In Flipped Classroom, more learning can be done outside the
classroom.
9 In Flipped Classroom, students have a sense of responsibility for
their learning.
10 In Flipped Classroom, students have more flexible learning time.
Open-ended Question
1. What do you think about Flipped Classroom?
132

Vol. 20, No. 3, pp. 132-150, March 2021
Initial Teacher Education of Primary English and

CLIL Teachers: An Analysis of the Training
Curricula in the Universities of the Madrid
Autonomous Community (Spain)
Alfonso López-Hernández
Universidad Pontificia Comillas, Madrid, Spain
https://orcid.org/0000-0001-8174-595X
Abstract. The Madrid Autonomous Community (Spain) offers a

significant case of extensive bilingual (English) education provision in
primary and secondary school, with over 50% of students following a
Content and Language Integrated Learning (CLIL) programme.
However, there are doubts about the ability of the current pre-service
teacher education provision to meet the demands of the rapid growth of
bilingual education in the region. This study analyses the initial teacher
education (ITE) of English as a Foreign Language (EFL) and CLIL teachers
in the Madrid community, by providing a systematic comparison of the
14 Primary Education teacher training degrees offered and, specifically,
their EFL specialist itineraries. In particular, it assesses the balance
between the development of linguistic and pedagogical competences, the
linguistic requirements to enter the EFL track, and the inclusion of
dedicated CLIL modules as part of the training curriculum. Results show
that there is a great heterogeneity in the design of such programmes,
especially regarding the value given to a language improvement
component. Furthermore, it was found that the training in CLIL is
insufficient, both in the core programmes and in the EFL itinerary.
Beyond the Spanish context, the questions raised in this study can offer
insight into the effective design of initial teacher education programmes
for educational systems that have committed to mainstream bilingual
education as a tool for enhancing foreign language competence.
Keywords: pre-service teacher education; initial teacher education, CLIL;

EFL; higher education; bilingual education; Spain
1. Introduction
The aim of this study is to provide a critical overview of the initial teacher
education (ITE) of primary school English teachers in the Madrid Autonomous
Community (Spain), by analysing the role of English in general, and the EFL
formative itinerary in particular, in the teacher training programmes offered by
©Authors
133
the numerous public and private universities found in a region with over 6.5
million residents.
The comparison is a timely one, as it comes ten years after the creation of the new
teacher training degrees in Spain, following the national educational reform
(Order 3857/2007, 27th of December) that aligned Spanish teacher training
degrees with the European Higher Education Area (EHEA), in what became
known as the Bologna Process. Amongst other targets, the reform aimed at
offering teacher trainees a more generalist training in the subjects of primary
education, somewhat neglecting the former specialist degrees (in TEFL, Physical
Education., Speech and Language Therapy, Special Needs Education, etc.) of the
year 2000 educational law. In the new EHEA degrees, trainees willing to specialise
may choose from a number of formative itineraries or tracks, called menciones,
which offer significantly fewer modules and credits than the former specialised
degrees, and tend to concentrate them in the last two years of students’ training.
Paradoxically, such a turn away from specialisation has coincided with a

heightened social and professional awareness of the role of English as the world’s
lingua franca, resulting in greater pressure on educational systems to provide
more extensive and effective foreign language training (Eurydice, 2017),
especially considering that the average competence in EFL among Spaniards is
still poor compared to most other European countries (English First, 2020;
European Commission, 2012a, 2012b). Interestingly, regional educational
authorities have tried to meet this demand, not so much through an increase in
hours of English taught in school, but by committing to offering Content and
Language Integrated Learning programmes, in which English is also used as a
vehicle of instruction. Indeed, most autonomous communities in Spain have
developed some form of CLIL (Scott & Beadle, 2014), to the point that Spain can
be considered to be the European spearhead of CLIL research and practice (Perez-
Cañado, 2016). In sum, in a time when primary teachers are required to teach more
English and other subjects in English, the number of credits devoted to foreign
language teacher education in ITE is significantly lower than before the Bologna
reform (Pérez Murillo & Steele, 2017).
In this general context, the Madrid Autonomous Community stands out

particularly as one of Europe’s most extensive and ambitious cases of CLIL
provision in primary and secondary education. Indeed, about 50% of students of
public and concertado (semi-private) schools follow a CLIL English programme
(Comunidad de Madrid, 2019) and most other students of semi-private and
private schools receive some form of EMI or English reinforcement component, in
the framework of programmes such as BEDA or UCETAM.1 As a result, it is fair
1
The Bilingual English Development and Assessment (BEDA) programme
(https://www.ecmadrid.org/en/programs/beda-program) is run by the association of Roman
Catholic schools of Madrid in partnership with Cambridge English Language Assessment. In turn,
UCETAM, an association of private and semi-private schools, awards the CBC (cooperative
bilingual school certification) o some of its schools. (See
https://colegiosbilinguescooperativos.com/cbc/) Both programmes are compatible with the
regional authority’s extensive bilingual education programme for public and publicly funded
schools.
134
to say that many - if not most - teacher trainees in the Madrid region can expect to
be in close contact with English as soon as they begin their teaching careers:
whether as English specialist teachers, teachers of other subjects in English, or
teachers of classes taught in Spanish who work in a CLIL school.
There are a number of studies that discuss the training needs of pre-service
English and CLIL teachers in the Spanish context (e.g., Cabezuelo & Fernández,
2014; Fernández-Viciana & Fernández-Costales, 2017; Gutiérrez Gamboa &
Custodio Espinar, 2021; Pavón et al., 2020). As to the curricular design of ITE
programmes, the main study is that of De la Maya Retamar and Luengo González
(2015), in which 60 primary education teacher training programmes at the
national level were surveyed. However, as suggested above, the special bilingual
educational scenario found in the Madrid Autonomous Community suggests the
need of a closer look at the regional ITE provision, as a case study that can help to
shed light on the specific challenges of ITE curricular design in countries and
regions that require to effectively train large numbers of non-native foreign
language and CLIL practitioners.
2. Research questions
This study analyses the responses provided by public and private higher
education institutions to the increasing need for qualified EFL and CLIL primary
teachers in the Madrid region. To do so, it addresses the following research
questions:
Research question 1: To what extent does the curricular design of the generalist
EHEA ITE programmes in Madrid universities reflect the rise of English as a
lingua franca, as well as the higher demand for qualified English and CLIL
primary teachers in the region?
Research question 2: What, if any, are the linguistic requirements for trainees who
wish to enter the EFL formative itinerary at the universities?
Research question 3: What is the balance between English language improvement,

foreign language pedagogy and other areas in the different universities’ curricular
design of the EFL formative itinerary?
Research question 4: What is the role of training in CLIL in the different ITE
programmes, both inside and outside the specialist EFL itinerary?
Before analysing the university curricula, the first part of this paper discusses the
training needs of primary EFL and CLIL teachers in the Madrid region, and
provides an overview of the changes in the Spanish higher education framework
that help to explain the nature and current role of the menciones [formative
itineraries] in the teacher training degrees.
3. Training demands on pre-service teacher education

The first question that needs to be addressed concerns the specific demands made
by the educational system on ITE, especially in the area of EFL.
135
3.1. The role of language improvement

In line with the competency-based paradigm that guided the reform of the
Spanish higher education degrees, the Spanish Agency for Quality in Education
(ANECA, 2005, p. 113 ff.) provided a list of specific professional competencies that
teacher trainees should develop in order to be able to teach the school curricula
effectively. They were divided into three types: disciplinary knowledge
(knowing), professional (knowing how to) and trans-national competencies
(ANECA 2005; Madrid Fernández, 2014).
In the case of foreign language teaching, it is interesting to note the emphasis

given to “full communicative competence in the target language”, as the first
disciplinary content. The inclusion of such linguistic competence reflects the
predominance of the communicative language teaching (CLT) paradigm at the
time, but also the important fact that, in the Spanish context, which is defined by
a generally low mastery of English as a foreign language (English First, 2020;
European Commission, 2012a, 2012b), linguistic competence in the target
language should not be taken for granted, even among trainees who choose to
specialise in the EFL subject.
In fact, several studies conducted on Spanish teacher trainees studying the EFL
specialist itinerary have shown that language improvement is perceived as a
crucial component of their training. Amengual-Pizarro (2007) found that
prospective language teachers in the Balearic Islands considered that there should
be a balanced combination between language improvement and methodology in
their pre-service training, considering that they will be expected to teach English
to young learners following a communicative approach. In a more recent study
with trainees following different formative itineraries (not just EFL), the same
author found that students feel that the current primary education degree does
not provide the necessary training for them to teach English effectively to
children, and identified the lack of English language proficiency, especially oral,
as their main shortcoming as future English teachers (Amengual-Pizarro, 2013).
In line with Amengual-Pizarro’s findings, another study, conducted with

prospective EFL teachers in a different university (Fernández-Viciana &
Fernández-Costales, 2017), concluded that a perceived low competence in English
is one of the factors that most negatively affects their self-efficacy as foreign
language teachers. As a result, the participants demanded more language
improvement, especially focused on language related to their profession
(teaching). This latter request is consistent with voices such as Freeman (2016) and
Richards (2017), who argue for a functional, “ESP” type approach in ITE
programmes, especially in countries where average general proficiency in English
is low.
Overall, the voice of Spanish student teachers supports the view, shared by many
other non-native English teachers across different continents (Richards, 2017), that
a poor proficiency in English will negatively affect not only the teachers’
performance in class, but also their sense of self-efficacy and even their
identification with the ELT community (Young et al., 2014). Hence, when looking
136
at specific teacher training programmes in the Spanish context, it will be

important to consider to what extent this need is addressed.
3.2. Training for CLIL

As Flores (2011) claims, ‘understanding the nature and purpose of teacher
education implies an analysis of the conceptual and epistemological assumptions
underpinning its models and organisational systems, but it also entails the
consideration of the social, political, cultural and economic context in which it is
embedded.’; In the case of the Madrid Autonomous Community, the educational
context of the last years has been crucially affected by the rapid growth of
bilingual education programmes and, most notably, the community’s bilingual
programme for public and semi-private schools. As of 2019, there were 399
bilingual public primary schools, 181 public secondary schools and 216 semi-
private schools offering the bilingual programme in primary or primary and
secondary school. Furthermore, the programme was recently expanded to infant
education (ages 3-6), both in public and semi-private schools. In total, close to
200,000 students participate in this government-funded programme, accounting
for approximately 50% of the student population in compulsory stages of
education in publicly funded schools (Comunidad de Madrid, 2019).
Without discussing this programme in detail, it is worth noting that it claims to

offer instruction of subjects in English following a CLIL approach. Content and
Language Integrated Learning has been used as an umbrella term for the different
context-bound varieties of bilingual education offered across Europe, that
nevertheless share ‘a dual-focused educational approach in which an additional
language is used for the learning and teaching of both content and language’
(Coyle et al., 2010, p.1). Moreover, there is a wide consensus among CLIL
researchers and practitioners that CLIL is much more than translating instruction
into an additional language; rather, it must seek to promote a strong cognitive
element, as well as develop inter-cultural competence in students, to mention but
two of CLIL’s central tenets. To put CLIL into practice effectively, there are a
number of key competences that CLIL practitioners must possess, and which have
been identified by recent research: linguistic and pedagogical, of course, but also
scientific, organisational, interpersonal and collaborative, and reflective and
developmental, to use Pérez-Cañado’s (2018) CLIL teacher profile.
In what relates to the scope of this paper, there are serious doubts as to the ability
of teacher education provision to adapt to the needs of CLIL. Pérez-Cañado (2018,
p. 213) believes that ‘the demands placed by the implementation of this approach
have been largely overlooked.’ Moreover, existing ITE programmes are criticised
for not offering sufficient methodological training in CLIL (De la Maya Retamar
& Luengo González, 2015), and generally not being in line with the new teacher
demands (Cabezuelo Gutiérrez & Fernández Fernández, 2014). To meet them,
recommendations include pre-service and in-service training modules (Madrid
Manrique & Madrid Fernández, 2014; Pavón et al., 2020; Pérez-Cañado, 2018), as
well as offering EMI (English as Medium of Instruction) teacher training
programmes at both the bachelor and master’s levels (Pérez-Cañado, 2018).
137
The need for an enhanced role of CLIL in ITE is even more urgent in the Madrid
Autonomous Community, owing to changes in the requirements to obtain
linguistic capability certification (habilitación lingüística, in Spanish), which
qualifies teachers to teach subjects in English. Before 2010, there was an Initial
Training Plan that combined language improvement and methodological training
(in CLIL), but, since 2010-11, certification is granted following purely linguistic
criteria, requiring a level of English of C1 (CEFR), which may be demonstrated by
different means (Cabezuelo Gutiérrez & Fernández Fernández, 2014). On top of
that, in-service training in CLIL, although offered, is voluntary and not tied to the
aforementioned process of certification. The current scenario offers the interesting
paradox, outlined by Custodio Espinar (2020), that, at a time when teachers are in
dire need of methodological training in CLIL, there is no effective guarantee that
they receive it either prior to, or during, their careers as CLIL practitioners.
Therefore, an assessment of ITE in the Madrid region must also consider the role
of CLIL in the different programmes: whether a specific CLIL module is taught
inside or outside the EFL formative itinerary, and whether some form of EMI
teacher education programme is offered.
4. The pre-service training of English and CLIL teachers in Spain:

generalist degrees and formative itineraries
We now turn to a brief discussion of the legal and academic framework in which
universities and other tertiary level institutions plan their curricula in order to
address the training demands outlined in the previous section.
As was suggested in the introduction, the current menciones or formative

itineraries in Spanish teacher training degrees have resulted from the tension
created by two conflicting demands in ITE: providing a more generalist training
in primary education, on the one hand, and the need to train specialist teachers in
such subjects as Physical Education or EFL. It must be noted that, in the Spanish
tradition, foreign languages in primary education have been taught by specialist,
not generalist or class teachers (Eurydice, 2017) as is the case in some countries
with a longer tradition in foreign language education, such as Scandinavian ones
(Jover et al., 2016). Interestingly, the discussions leading to the EHEA educational
reform did envisage the possibility of training all student teachers in FL education,
but decided to postpone this scenario until a moment in time that was ‘difficult to
predict’ (ANECA, 2005).
In this generalist-oriented framework, universities were allowed to offer specialist

formative itineraries, but these were not meant to replace the former specialist
degrees. On the contrary, it was assumed that Education graduates would
specialise through post-graduate studies (García Jiménez & Lorente García, 2014).
This assumption was probably unrealistic given the resources involved -
especially in the context of economic downturn and budget cuts - and so the
Ministry of Education eventually established that EFL specialist teachers with an
EHEA bachelor’s degree would be required the EFL itinerary as well as a certified
English language competence of B2 according to the Common European
Framework of Reference (CEFR) (Royal Decree 1594/2011, November 4th).
138
As a result, some itineraries, such as Physical Education or English, have,

accidentally, come to replace the former specialist qualifications. Albeit, as recent
studies point out (Amengual-Pizarro, 2013; De la Maya Retamar & Luengo
González, 2015), the new formative itineraries have significantly reduced the
number of credits allocated to training student teachers in the specialist subject.
To cite but one example, the pre-EHEA EFL degree offered at Autonomous
University, one of the largest public universities in the region, offered close to 60
credits in EFL-related contents, amounting to almost twice the number of similar
credits taught today (Resolution, Autonomous University of Madrid, 26th June,
2000).
An analysis of the universities’ respective programmes should, therefore,

consider (a) whether they have attempted to compensate this lack of training in
EFL outside of the specialist itinerary (i.e., in the core modules of the primary
education degree) as well as (b) what choices have been made in order to make
the best possible use of the reduced credit load devoted to foreign language
teacher education.
5. Method
This study focuses on the ITE of primary teachers who are likely to pursue
teaching careers in the Madrid Autonomous Community. Hence, although based
in Madrid, online universities such as UNIR or UNED have been disregarded, as
their student intake comes from all over the country, as well as from abroad. As a
result, a total of 14 training programmes taught at 14 universities in the academic
year 2019-2020 have been surveyed.
This research can be considered a case of comparative programme evaluation

(Frey, 2018), and employs a qualitative methodology that blends document
analysis and personal interviews (Owen, 2014). Data on the different programmes
were obtained from their institutional websites, and then systematically classified
and tabulated. Furthermore, department heads or programme coordinators were
contacted in order to confirm or clarify some of the data, thus ensuring their
credibility. In particular, they provided useful details that were not available in
published documentation, including the required language level to enter the
itinerary, the methodological training received by faculty teaching in EMI
programmes, and the attention given to CLIL in those programmes that lack a
dedicated bilingual education module.
The data obtained were then analysed from both statistical and interpretive
viewpoints (Nunan, 2013), resulting in the graphic representation and discussion
presented below.
Finally, when comparing the structure of the specialist itineraries, credits allotted
to practicum and end-of-degree dissertations (TFG, in its Spanish acronym) have
been disregarded. Although such credits can offer a significant contribution to
students’ training, the great heterogeneity in their implementation makes them
unsuitable for cross-programme comparison.
139
6. Results and discussion

6.1. Overview of the ITE programmes in the Madrid Autonomous Community
In all, there are 14 universities or colleges based in the Madrid Autonomous
Community that offer the primary education teacher training bachelor’s degree.
Of them, four are public universities, five are private, and the remaining five are
private colleges affiliated with a public institution, called centros adscritos.2 Eight
of these institutions offer traditional on-site tuition only, five provide both face-
to-face and distance learning programmes, and only one university (UDIMA)
teaches the degree exclusively online.
Tables 1-3 provide an overview of the teacher training courses in the three kinds
of institutions discussed. The tables also specify the type of learning offered,
which formative itineraries can be studied, and whether there is an EMI stream
that can be followed.
Table 1: Overview of primary education ITE programmes in the Madrid Autonomous

Community. Public universities.
University Type of EMI Formative itineraries
learning programme
(Yes/No)
Complutense On-site Y EFL, Speech and Language
University of Therapy, Physical Education,
Madrid (UCM) Music
Autonomous On-site N EFL, Arts, ICT, Music,
University of Therapeutic pedagogy, Speech
Madrid (UAM) and Language Therapy, Physical
Education.
University of On-site Y EFL, Arts, Special Needs

Alcalá (UAH) Education, Spanish Language
and Literature, Physical
Education, Natural Science.
Rey Juan Carlos On-site, Y EFL, Physical Education, Music.

University distance
(URJC)
2
These affiliated institutions are private; however, their degrees are awarded by the public
institution they are affiliated with. Note that the curricula of their degrees they offer are identical
or very similar to the ones of the equivalent degrees at the public university.
140

Community. Private colleges affiliated with public universities (in brackets).
learning programme
(Yes/No)
CES Don Bosco On-site Y EFL, Speech and language
(UCM) therapy, Physical Education,
Music, Therapeutic Pedagogy
ESCUNI (UCM) On-site N EFL, Music, Physical Education,

Therapeutic Pedagogy
CES Villanueva On-site N EFL, Physical Education,

(UCM) Therapeutic Pedagogy
CSEU Lasalle On-site, N EFL, Therapeutic Pedagogy,

(UAM) distance Speech and Language Therapy
Cardenal On-site, Y EFL, Physical Education, Special

Cisneros (UAH) distance Needs Education

Community. Private universities.
learning programme
(Yes/No)
Comillas On-site N EFL, Therapeutic Pedagogy
Pontifical
University
Francisco de On-site N EFL, Physical Education, Music,
Vitoria Therapeutic Pedagogy.
University
(UFV)
Camilo José On-site, Y EFL, Physical Education,
Cela University distance, Therapeutic Pedagogy, Speech
(UCJC) blended and Language Therapy, Music,
ICT in Education.
Madrid Open Distance N EFL, Therapeutic Pedagogy,

University Speech and Language Therapy,
(UDIMA) ICT in education
Antonio de On-site, Y EFL, Therapeutic Pedagogy,

Nebrija distance Physical Education, French,
University Speech and Language Therapy
141
Considering the overall picture, the number of new programmes that have been
created following the EHEA reform is striking. Indeed, out of the 14 institutions
that currently offer the primary education teacher training degree, only 10 of them
taught the former, pre-EHEA teacher training programmes, and one (Universidad
Francisco de Vitoria) did not offer the EFL specialist degree (Comunidad de
Madrid, 2006). It is also telling that four of these new programmes have been
implemented by private universities, which seem to have found in teacher
training a fruitful domain to increase their student intake.
Private institutions also lead the way as regard online and blended learning, with
only one public university (URJC) offering a primary education teacher training
programme online – which does not include the EFL formative itinerary. This can
be explained by two reasons. Firstly, the greater need of private institutions to
attract students in order to become profitable or at least survive, given the fact
that they receive little or no direct government funding. Second, the fact that, at
least traditionally, the role of providing distance education has been assigned to
UNED, the Spanish Open University, with other public institutions focusing on
on-site training as well as research.
6.2. English in the core primary ITE curricula

The fact that all of the teacher training faculties and departments offer the EFL
formative itinerary attests to the high demand of qualified EFL and CLIL teachers
in the region. The EFL one is among the most popular, which is consistent with
research conducted in other regions (García Jiménez & Lorente García, 2014)
which found that trainees tend to enrol in formative itineraries that lead to a
professional qualification, such as Physical Education or EFL. The offer of EMI
degrees or streams in as many as half of the universities also reflects the
popularity of EMI programmes in the region (Jeffrey et al. 2019; Ramos García,
2015) as well the perceived suitability of EMI instruction to train future CLIL
teachers (Madrid Manrique & Madrid Fernández, 2014; Pérez-Cañado, 2018).
However, it is also worth looking at the number of English Language, Teaching

English as a Foreign Language (TEFL) and CLIL credits that are offered outside
of the formative itinerary. Given the overall increase in degree credit load, and the
reduction of the relative weight of specialist training, it would be expected that
EHEA degrees would include a greater component of EFL-related core credits
than in the past. This is found to be the case, although again there are significant
differences between programmes. If, in the pre-EHEA courses, non-specialist
primary trainees would study only 4.5 credits (that is, 45 hours of class time) of a
subject named “English and its didactics” (See, for example, Resolution,
Autonomous University of Madrid, 26th June, 2000), in the current degree courses
the minimum number of ECTS credits is 10 (UCM and affiliated colleges), and the
most common number is 12, which amounts to two semester-long courses,
approximately 100-120 hours of class time (See Figure 1).
These results also compare favourably with numbers at the national level. In their
2015 study, De la Maya Retamar and Luengo González found that half of the
primary degrees sampled included only six credits of foreign language study, and
142
only 18.3% offered 12 or more ECTS which, as they see it, would be the minimum
required for a trainee entering with a CEFR B1 level to move on to B2, the
minimum level required to obtain English teacher qualified status (De la Maya
Retamar & Luengo González, 2015).
On the other hand, some private universities offer a significantly higher number
of English credits to their non-EFL specialist trainees, as many as 24 (Nebrija,
UFV) and 17 (UCJC). Again, it would seem that private institutions are more
responsive to labour market demands than public ones.
Figure 1: University curricula: ECTS of EFL-related coursework outside the EFL

formative itinerary
Despite this encouraging overall picture, there are two significant pitfalls which
must be addressed. Firstly, in six of the courses, including two of the largest
public universities (UAM and UAH) and their affiliated colleges, students receive
no explicit training in foreign language teaching methodology, as the credits are
devoted to language improvement only (See Figure 1). This is consistent with the
picture across the country, where over a third of the programmes offer no EFL
methodology credits in the trainee’s core training (De la Maya Retamar & Luengo
González, 2015), and reveals the worrying fact that a significant number of
teachers will have completed their ITE without even an elementary
understanding of the basic principles of one of the compulsory areas of the
primary education curriculum.
Second, no less strikingly, none of the surveyed universities seems to train non-
specialist primary trainees in the principles, strategies and resources of Content
and Language Integrated Learning, which, as we have seen, is one of the main
thrusts of educational innovation in the region. Here, it could be argued that a
number of universities are catering to this need by offering degrees that are
partially taught in English, following strategies of EMI (Jover et al., 2016). Indeed,
143
half of the analysed courses offer EMI streams that typically offer between 30%
and 50% of the credit load in English.
Yet, there is a significant caveat that must be considered. While EMI instruction
may serve to complement language improvement in significant ways, such as
developing the Cognitive Academic Language Proficiency required from CLIL
practitioners (Pérez Cañado, 2018), there is no guarantee that exposure to EMI will
translate into a training in CLIL. Indeed, for student teachers to learn through
their EMI experience, their lecturers must also apply CLIL instructional principles
and strategies, which, in most cases, they have not been trained for. In this respect,
programmes that offer their EMI lecturers methodological training, such as UAH,
UCM, CES Don Bosco (Sierra Macarrón & López Hernández, 2015) and,
especially, Cisneros (Fernández & Johnson, 2016), seem to be pointing in the right
direction.
6.3. The EFL formative itinerary: linguistic entry requirements

The following sections compare the different EFL formative itineraries, by
focusing on the minimum linguistic entry requirements, the balance of language
improvement and TEFL in their curricular design and, finally, whether and how
they provide training in CLIL.
The required levels of English language competence to enter the EFL itinerary are
surprisingly heterogeneous, ranging from A2 (Comillas) to B2 (UAH, UCM,
UCM-Villanueva). Overall, as can be seen in Figure 2, most courses settle on either
the B1 or the B2 level, according to the CEFR.
Figure 2: English language level, according to the CEFR, to enter the EFL itinerary
The high number of programmes (5) that require a B1 level appears to be

surprising. This may be explained, of course, by non-pedagogical reasons, such as
the desire to encourage a high enrolment in a formative itinerary that features
among the most demanded by the labour market. In this respect, setting too high
a level would probably deter many potential students from studying the EFL
144
subject. However, it is worth noting that programmes that require an initial level
of CEFR B1 or below also seem to provide the means for language improvement
to take place. Indeed, they typically offer a relatively high number of credits
devoted to mostly general English, taught normally through several years of the
programme. This is the case, for instance, of Comillas, that, despite accepting
trainees with an A2 level, offers 18 ECTS of language improvement in the first two
years of the degree.
6.4. Curricular design of the EFL itinerary

Turning to the structure of the formative itinerary in the different universities and
colleges, the first feature that can be noticed is that, compared to the former pre-
EHEA specialist courses, the current programmes have streamlined their contents
and, overall, devote less weight, if at all, to more philological contents such as
morphosyntaxis, grammar, phonetics, culture or literature, that appeared as
module names in the former specialist EFL teacher training degrees (see, for
instance, the one taught at UAM [Resolution, Autonomous University of Madrid,
26th June, 2000]). Indeed, as can be seen in Figure 3, most curricula prioritise a
balance of language improvement and TEFL, with many of them offering a course
on CLIL or bilingual education.
In what concerns the offer of modules outside language or methodology, the most
popular choices are children’s literature in English (taught in four programmes),
culture and/or literature of English-speaking countries (3), and specific focus on
teaching resources such as games (2), phonics or CALL. Phonetics and phonology,
popular in the former specialist degrees, only appears in one of the surveyed
formative itineraries. Lastly, there are no references to literacy, which has been
hailed as the most effective approach for teaching English in CLIL contexts
(Halbach, 2020).
Figure 3: University curricula: distribution of coursework ECTS inside the EFL

formative itinerary (excluding practicum and end-of-degree dissertation)
145
However, here again, a closer look at the distribution of credits reveals a high
heterogeneity in the balance of language, methodology and other contents that
are offered as part of the itinerary.
In terms of the language improvement component, a clear distinction emerges

between courses that rely heavily on it and those that do not. In the former group,
we find the cases of Comillas, UAM, Lasalle, Alcalá, Rey Juan Carlos and UDIMA,
which add more English credits to the ones already offered as a mandatory
component of the primary education degree (See Figure 3). The other universities
and colleges have left EFL coursework out of the itinerary and place the focus on
methodological training in the form of modules devoted to TEFL, skills
development, teaching resources, and so on.
Considering the sum total of language improvement credits at a given university

- the aggregate of core and specialist credits - the range is astonishing, from 24
ECTS at UDIMA and Alcalá, to only six ECTS at Complutense and its affiliated
colleges (Villanueva, Don Bosco, ESCUNI). Conversely, courses such as Alcala’s
and Lasalle’s (UAM) devote a scanty six ECTS to training in TEFL, in contrast to
22 at UCM and affiliated colleges or a very high 27 at Camilo José Cela. It is worth
noting that, in this regard, there is no evident line that can be drawn between the
choices made by private and public universities.
Hence, not all the programmes are in line with the view that a language
improvement component is crucial to enhance trainee’s self-efficacy to teach
English following a communicative approach (Amengual-Pizarro, 2007;
Fernández-Viciana & Fernández-Costales, 2017). In half of the programmes,
language competence is either taken for granted, or its development left to EMI
courses.
6.5. Training in CLIL

The last key feature of the formative itineraries that must be discussed is the role
of Content and Language Integrated Learning in the formative itineraries’ design.
Whereas, as was seen before, CLIL plays no role in the core ITE syllabus of the
different programmes, most formative itineraries - 9 out of 14 - offer some training
in CLIL, normally as a dedicated module that is typically taught in the last year of
the degree. In those cases where there is no explicit reference to CLIL, programme
coordinators mentioned that CLIL is discussed in TEFL modules; or, in the case
of Cardenal Cisneros, that training in CLIL is integrated into the EMI programme
as a whole.
Still, the role of CLIL in EFL formative itineraries may reveal a number of
questionable assumptions about the relationship of EFL and CLIL teacher
education. The first one is that many or most future CLIL teachers will choose the
EFL itinerary as part of their ITE. This belief is probably founded on the idea that
trainees who enter their initial training with a high level of English will be inclined
to pursue a career in teaching EFL. However, this is not necessarily the case.
Firstly, because future Physical Education specialists - one of the most popular
146
CLIL profiles - will need the P.E., not EFL, formative itinerary in order to work as
CLIL teachers. Secondly, because many trainees who start their degrees with a
high level of English may wish to follow an EMI programme, while not
necessarily choosing the EFL itinerary. According to programme coordinators at
Nebrija, Cardenal Cisneros, UAH and UCM, all of which offer EMI programmes,
this is indeed the case to a lesser or greater degree.
The second assumption is that training in CLIL is, in many ways, an appendix of
broader training in TEFL. This can be seen in the fact that CLIL modules are taught
only as part of the EFL itinerary and, perhaps more tellingly, at the end of it. Such
a belief could be warranted in cases of more language-driven CLIL programmes,
the so-called ‘soft’ CLIL, in which CLIL is mainly taught as part of the FL
curriculum, and by language teachers (Ball et al., 2016). However, this is clearly
not the case in the Madrid region, where CLIL takes place mainly in the content
subjects (Science, P.E., Arts and Crafts, Music), and is taught by subject, not
foreign language, specialists. As has already been discussed, one of the practical
implications of this assumption is that, in most cases, CLIL is not included in the
core ITE curricula, thus failing to provide training to a significant number of
future CLIL practitioners. An additional one is that, even in the context of the EFL
itinerary, this training is offered too late in the degree for trainees to properly
master CLIL pedagogy, methods and resources, and connect them both to the
subject-specialist training they have received (Social Science, Arts & Crafts, etc.)
and, perhaps more importantly, to their teaching internships. Such shortcomings
in the treatment of CLIL in the training curricula may well help explain the finding
that the new EHEA degrees are not properly training CLIL practitioners who
work in the Madrid Autonomous Community (Custodio Espinar, 2020).
7. Conclusions
This study set out to investigate the role of English language improvement,
teaching methodology, and CLIL in the design of the current ITE programmes in
the Madrid Autonomous Community. Answering research question 1, the
respective designs of the 14 EHEA 4-year long ITE programmes in the Madrid
universities do reflect the rise of English as Europe's unquestionable lingua franca,
as well as the higher demand of qualified EFL and CLIL primary teachers in the
region brought about by its large-scale bilingual education programmes. This is
evident from the increase in credit load devoted to English and foreign language
pedagogy, which is higher than the national average, and which was found to be
particularly noticeable in private universities.
Turning to the EFL formative itinerary, the first interesting finding of this study
is that the minimum entry levels of English required by the different programmes
tend to be low, with about two thirds of the programmes requiring CEFR levels
B1, A2 or no requirement (research question 2). As to their curricular design
(research question 3), the EFL itineraries have moved away from the theoretical,
philological contents that used to be taught in the former specialist degrees, and
now offer a heterogeneous combination of language improvement, language
teaching methodology and, to a lesser extent, TEFL resources and CLIL.
147
Finally, this analysis contributes to explaining, at least in part, the research

findings that claim that, in general, student teachers in Madrid are not being
properly trained in the principles and strategies of Content and Language
Integrated Learning. Overall, the credit load devoted to CLIL appears to be
insufficient and, when offered, inadequately placed at the end of the EFL itinerary
and not as part of the core training of prospective teachers.
8. Recommendations
From the findings presented in this study, it is recommended that primary ITE
programmes offer coursework on both foreign language teaching methodology
and CLIL as part of their core curriculum; that is, outside of the EFL specialist
itinerary. In what concerns language pedagogy, all trainees should develop an
elementary understanding of the teaching pedagogy of one of the most prominent
subjects in the school timetable. Among other benefits, this would enable more
effective cooperation with the English language teachers.
Turning to CLIL, placing dedicated bilingual education modules in the core, non-
specialist training, would ensure that all future CLIL practitioners - who may or
may not have followed the EFL formative itinerary - receive at least some training
in the principles and strategies of the CLIL approach. Moreover, such training
should be offered earlier on in the programme and, whenever possible, be
integrated with other relevant modules (on Natural Science, Social Science, Arts
& Crafts and P.E.), as well as with the different teaching internships, many of
which will take place in schools offering bilingual education.
Beyond the context of this study, these two recommendations may also be
relevant in countries and regions that, like Spain, combine a low average level of
English in teacher trainees with a commitment to a content-driven form of
bilingual education that is largely taught by subject, not EFL specialist, teachers.
9. Research limitations
This study has several limitations that should be addressed by further research.
Firstly, practicum and, to a lesser extent, the end of degree dissertation (TFG)
should also be included in the analysis, in order to assess whether they are used
by the different programmes to enhance their students’ training as future EFL
specialists or CLIL teachers. Secondly, a closer look into the broad area of
language improvement is necessary. Indeed, it would be helpful to identify what
the balance is between general English, English for Academic Purposes (EAP) and
English for Specific Purposes (ESP), and whether the latter effectively targets the
needs of prospective foreign language and CLIL teachers. In addition, EMI
programmes in ITE should be carefully assessed in order to determine to what
extent their students learn CLIL through exposure and practice, as well as
enhancing their subject-specific and academic language skills. Finally, in what
regard to data collection, more details could have been obtained regarding the
treatment of CLIL in programmes lacking dedicated CLIL modules: for instance,
number of hours of instruction and competences developed.
148
10. References
Amengual-Pizarro, M. (2007). How to respond to the demands set by the communicative
approach? New challenges L2 teachers face in the classroom. The European Journal
of Teacher Education, 30(1), 63-73. http://dx.doi.org/10.1080/02619760601120007
Amengual-Pizarro, M. (2013). Primary education degrees in Spain: do they fulfil the
linguistic and pedagogic needs of future teachers? Vigo International Journal of
Applied Linguistics, 10, 9-27. http://webs.uvigo.es/vialjournal/pdf/Vial-2013-
Article1.pdf
ANECA. (2005). Libro blanco. Título de Grado en Magisterio (vol. I and II.) Agencia Nacional
de Evaluación de la Calidad y Acreditación. http://www.aneca.es/Documentos-
y-publicaciones/Libros-Blancos
Ball, P., Clegg, J., & Kelly, K. (2016). Putting CLIL into practice. Oxford University Press.
Cabezuelo, G. P., & Fernández, F. R. (2014). A case study on teacher training needs in the
Madrid bilingual project. Latin American Journal of Content & Language Integrated
Learning, 7(2), 50-70. https://doi.org/10.5294/4220
Comunidad de Madrid. (2006). Estudiar en Madrid: Universidades madrileñas. Curso 2006-
2007. http://www.madrid.org/universidades
Comunidad de Madrid. (2019). Datos y Cifras de la Educación 2018-2019.
https://www.comunidad.madrid/sites/default/files/doc/educacion/sgea_dat
osycifras_2019-20.pdf
Coyle, D., Hood, P., & Marsh, D. (2010). CLIL: Content and language integrated learning.
Cambridge University Press.
Custodio, E. M., & García, R. J. M. (2020). Are Accredited Teachers Equally Trained for
CLIL? The CLIL Teacher Paradox. Porta Linguarum, 33, 9-25.
http://hdl.handle.net/10481/62800
De La Maya Retamar, G., & Luengo González, R. (2015). Teacher training programs and
development of plurilingual competence. In D. Marsh, M. L. Pérez Cañado, & J.
Ráez Padilla (Eds.), CLIL in action: Voices from the classroom (pp. 114–129).
Cambridge Scholars Publishing.
English First. (2020). EF English proficiency index. http://www.ef.com/epi
European Commission. (2012a). Eurobarometre 386: Europeans and their languages.
http://ec.europa.eu/public_opinion/archives/ebs/ebs_386_en.pdf.
European Commission. (2012b). European survey on language competences.
http://ec.europa.eu/languages/library/studies/executive-summary-
eslc_en.pdf.
Eurydice. (2017). Key Data on Teaching Languages at School in Europe - 2017 Edition. Eurydice
Report. Publications Office of the European Union.
https://eacea.ec.europa.eu/national-policies/eurydice/content/key-data-
teaching-languages-school-europe-%E2%80%93-2017-edition_en
Fernández, R., & Johnson, M. (2016). Opinions, perceptions and attitudes of a group of
students taking a Bilingual Infant Teacher Training Degree: an empirical
study. PULSO. Revista de Educación, 39, 59-78.
https://revistas.cardenalcisneros.es/index.php/PULSO/article/view/214
Fernández-Viciana, A., & Fernández-Costales, A. (2017). El pensamiento de los futuros
maestros de inglés en Educación Primaria: creencias sobre su autoeficacia
docente. Bellaterra Journal of Teaching & Learning Language & Literature, 10(1), 42-
60. https://doi.org/10.5565/rev/jtl3.684
Flores, M. A. (2011). Curriculum of initial teacher education in Portugal: new contexts, old
problems. Journal of Education for Teaching, 37(4), 461-470.
https://doi.org/10.1080/02607476.2011.611015
149
Freeman, D. (2016) Educating Second Language Teachers. Oxford University Press.

Frey, B. (2018). The SAGE encyclopedia of educational research, measurement, and
evaluation (Vols. 1-4). SAGE Publications,
Inc. https://dx.doi.org/10.4135/9781506326139.n547
García Jiménez, E., & Lorente García, R. (2014). Grado en maestro de educación primaria:
motivaciones y preferencias en la elección de mención. Aula de encuentro: Revista
de investigación y comunicación de experiencias educativas, 16(1), 103-119.
https://revistaselectronicas.ujaen.es/index.php/ADE/article/view/1297
Gutiérrez Gamboa, M., & Custodio Espinar, M. (2021). CLIL teacher’s initial education: a
study of undergraduate and postgraduate student teachers. Encuentro Journal, 29,
104-119.
http://www3.uah.es/encuentrojournal/index.php/encuentro/article/view/55
Halbach, A. (2020). English Language Teaching Goes CLIL: Fostering Literacy and
Language Development in Secondary School in Spain. In P. Mickan, & I. Wallace
(Eds.), The Routledge Handbook of Language Education Curriculum Design (pp. 175-
189). Routledge.
Jeffrey, S., Rodríguez Melchor, D., & Walsh, A. (2019). Linguistic requirements for
students and staff on EMI undergraduate degrees at universities in the
Community of Madrid. European Journal of Language Policy, 11, 95-107.
https://doi.org/10.3828/ejlp.2019.6
Jiménez, E. G., & García, R. L. (2014). Grado de maestro de educación primaria:
motivaciones y preferencias en la elección de mención. Aula de encuentro: Revista
de investigación y comunicación de experiencias educativas, 16(1), 103-119.
https://revistaselectronicas.ujaen.es/index.php/ADE/article/view/1297
Jover, G., Fleta, T., & González, R. (2016). La formación inicial de los maestros de
educación primaria en el contexto de la enseñanza bilingüe en lengua extranjera.
Bordón. Revista de Pedagogía, 68(2), 121-135.
https://doi.org/10.13042/Bordon.2016.68208
Madrid Fernández, D. (2014). Integrating the European Portfolio in a competency-based
teacher education approach. In J.D. Martínez Agudo (Ed.), English as a foreign
language teacher education. Current perspectives and challenges (pp. 35-58).
Rodopinde.
Madrid Manrique, M., & Madrid Fernández, D. (2014). La formación inicial del profesorado
para la educación bilingüe. Universidad de Granada.
Nunan, D. (2013). Research methods in language learning. Cambridge University Press.
Owen, G. T. (2014). Qualitative Methods in Higher Education Policy Analysis: Using
Interviews and Document Analysis. The Qualitative Report, 19(26), 1-
19. https://doi.org/10.46743/2160-3715/2014.1211
Pavón, V.P., Lancaster, N., & Callejas, C.B. (2020). Key issues in developing teachers’
competences for CLIL in Andalusia: training, mobility and coordination. Language
Learning Journal, 48(1), 81–98. https://doi.org/10.1080/09571736.2019.1642940
Perez-Cañado, M. L. (2016). Are teachers ready for CLIL? Evidence from a European
study. European Journal of Teacher Education, 39(2), 202-221.
https://doi.org/10.1080/02619768.2016.1138104
Pérez Cañado, M. L. (2017). CLIL teacher education: Where do we stand and where do we
need to go? In M.E. Gómez Parra & R. Johnstone (Eds.), Bilingual Education:
Educational Trends and Key Concepts/ Educación Bilingüe: Tendencias Educativas y
Conceptos Clave (pp. 129–144). Spanish Ministry of Education.
https://sede.educacion.gob.es/publiventa/educacion-bilinge-tendencias-
educativas-y-conceptos-claves--bilingual-educational-trends-and-key-
concepts/educacion-investigacion-educativa-lenguas/22107
150
Pérez-Cañado, M. L. (2018). Innovations and Challenges in CLIL Teacher Training. Theory

Into Practice, 57(3), 1-10. https://doi.org/10.1080/00405841.2018.1492238
Pérez Murillo, M. D., & Steele, A. (2017). Initial Teacher Education for CLIL at Primary
Education Level in Madrid: Key Issues and Challenges. In M.E. Gómez Parra M.E.
& R. Johnstone (Eds.), Bilingual Education: Educational Trends and Key Concepts/
Educación Bilingüe: Tendencias Educativas y Conceptos Clave (pp. 221-233). Spanish
Ministry of Education. https://sede.educacion.gob.es/publiventa/educacion-
bilinge-tendencias-educativas-y-conceptos-claves--bilingual-educational-trends-
and-key-concepts/educacion-investigacion-educativa-lenguas/22107
Ramos García, A. M. (2013) Higher Education Bilingual Programmes in Spain. Porta
Linguarum, 19, 101–11. https://doi.org/10.30827/digibug.29632
Richards, J. C. (2017). Teaching English through English: Proficiency, pedagogy and
performance. RELC Journal, 48(1), 7-
30. https://doi.org/10.1177/0033688217690059
Scott, D., & Beadle, S. (2014). Improving the effectiveness of language learning: CLIL and
computer assisted language learning. European Commission.
http://ir.uv.es/pOJh718
Sierra Macarrón, L., & López-Hernández, A. (2015). CLIL en la formación inicial del
profesorado de Educación Infantil y Primaria: la experiencia del CES Don Bosco.
Educación y Futuro, 32, 83-116.
https://cesdonbosco.com/documentos/revistaeyf/EYF_32.pdf
Spain. Order ECI/3857/2007, 27th of December, that establishes the requirements for the
recognition of official university degrees that serve as qualification for working as
a Teacher of Primary Education. Boletín Oficial del Estado (Official State Bulletin),
No. 312, 27th of December 2007.
https://www.boe.es/eli/es/o/2007/12/27/eci3857
Spain. Royal Decree 1594/2011, 9th of November, that establishes the teaching specialisms
in Primary and Infant Education. Boletín Oficial del Estado (Official State Bulletin),
No. 270, 9th of November 2011. https://www.boe.es/buscar/pdf/2011/BOE-A-
2011-17630-consolidado.pdf
Spain. Universities. Resolution, 26th June 2000, of Autonomous University of Madrid,
whereby the Degree structure of the Foreign Language (English) Primary
Education Teaching Degree is published. Boletín Oficial del Estado (Official State
Bulletin), No. 152, 26th of June 2000.
https://www.boe.es/boe/dias/2000/06/26/pdfs/A22573-22585.pdf
Young, J. W., Freeman, D., Hauck, M. C., Gomez, P. G., & Papageorgiou, S. (2014). A
Design Framework for the ELTeach Program Assessments. ETS Research Report
Series, 2014, 1-29. https://doi.org/10.1002/ets2.12036
151

Vol. 20, No. 3, pp. 151-168, March 2021
The Attitudes of Pupils towards using Flipgrid in

Learning English Speaking Skills
Joan Lim Ker Shin and Melor Md Yunus*

Universiti Kebangsaan Malaysia, Selangor, Malaysia
https://orcid.org/0000-0002-0978-9904
https://orcid.org/0000-0001-7504-7143
Abstract. The objective of this research is to conduct an investigation into

the attitudes of primary pupils towards using Flipgrid in learning English
speaking skills in a Year 4 Common European Framework of Reference
for Languages (CEFR) classroom. The research used a mixed-methods
approach using an explanatory sequential design. Data collection was via
a questionnaire and semi-structured interviews with 60 CEFR primary
Year 4 pupils with low English-speaking proficiency. The results of the
questionnaire were analysed descriptively, whereas the data from the
semi-structured interviews was analysed using thematic analysis. The
findings showed that primary pupils in a CEFR classroom had a high
positive attitude towards using Flipgrid to learn English speaking skills.
This research has highlighted the potential use of Flipgrid in both
learning and teaching English speaking skills in a CEFR classroom. More
extensive research into the use of Flipgrid by secondary pupils in a CEFR
classroom would be of further interest. Future research on this topic may
provide greater insight into the different ways in which English educators
can teach English speaking skills to secondary pupils. Moreover, the
results could provide useful information to the Ministry of Education on
the use of the e-learning approach in the learning and teaching of English
speaking skills in the Malaysian CEFR classroom.
Keywords: Flipgrid; speaking; attitude; primary pupils; English language

education
1. Introduction
The industrial revolution has had a major impact on businesses and workplaces
(Coldwell, 2019) and English has become the international lingua-franca of the
business world irrespective of geographical, social, political or religious
differences (Rao, 2019). In Malaysia, the use of the English language is further
enhanced by the issue of the increasing rate of unemployed graduates. At the start
of 2020, the mass media reported that half of young graduates were unemployed
(Welsh & Chang, 2020). Bakar (2018) states that 59% of new graduates do not get
hired because of their poor grasp of the English language. The chief economist at

152
Malaysian Rating Corp Bhd, Zahidi Alias, supports this stating that the reason for
their unemployment is because of their poor English-speaking ability (Free
Malaysia Today, 2017).
The importance of the English language and the increasing need to improve the
mastery of spoken English in a globalised world has led to the introduction of the
Common European Framework of References for languages (CEFR) in the revised
Kurikulum Standard Sekolah Rendah (KSSR) to act as an exemplar of
international standards (Sidhu et al., 2018). Learners use a student-centred
learning method where they are actively involved in the teaching and learning
sessions, whilst teachers assist them (Sabudin, 2020). Reviewing the situation,
Flipgrid has been introduced as one of the more popular and widely used ICT
tools in schools in many countries (Nieves, 2020). Many researchers have shown
great interest in exploring its effectiveness in teaching and learning practices
(Hashim et al., 2018; Johnson & Skarphol, 2018; Stoszkowski, 2018). Although
most researchers have focused on the efficacy of Flipgrid in learning and teaching
speaking skills, little is known about the attitudes of primary pupils towards using
Flipgrid to learn English speaking skills in a Malaysian CEFR primary classroom.
Abdullah and Shah (2014) state that attitude is a key element in determining the
participation of students in language learning. Nijat et al. (2019) concur with this
finding. Their results showed that psychological factors of pupils, including fear
of making mistakes, anxiety, and shyness could hinder them from practising their
speaking skills in the English classroom. This finding concurs with that of Haidara
(2016), whose results indicated that psychological factors could negatively affect
the English-speaking performance of students; students who believe that they are
proficient in English remain anxious when speaking English as they lack
confidence and are fearful of making a mistake. It is also believed that these
problems result from classroom learning and teaching practices. As the new CEFR
aligned curriculum takes into consideration the use of contextual language,
students have a negative attitude towards English speaking practices in the
classroom as they are unprepared to engage in communicative activities (Azman,
2016). They claimed that they were not taught the fundamental skills of the
language before the production of language was embarked upon (Azman, 2016).
Thus, this study was conducted to investigate pupils’ attitude towards the use of
Flipgrid in learning English-speaking skills. The research question is ‘What are
the attitudes of primary pupils towards using Flipgrid to learn English speaking
skills in a Year 4 CEFR classroom?’ The findings will make a great contribution to
society, as being able to speak English plays a significant role in globalisation.
Increased demand for graduates with high English-speaking competency is
another justification for more effective teaching and learning approaches. This
study can therefore shed light on the potential use of Flipgrid so that teachers can
vary their teaching methodologies and utilise Flipgrid in their English-speaking
lessons.

153
English as an international language in the age of globalisation has gained the
attention of many researchers who have conducted research in relation to the
English language in a variety of contexts. Give technology’s rapid advancement,
many studies in the educational field have attempted to explain the integration of
information and communication technology (ICT) in both the learning and
teaching of ESL. Although the literature has covered many contexts, this review
focuses on four major themes which have been found to frequently occur in the
literature: benefits of ICT tools, Flipgrid, attitudes towards using ICT tools in
learning English, and past studies. The theme on attitudes toward using ICT tools
in learning English is reviewed in relation to the technology acceptance model
(TAM).
2.1 Benefits of ICT Tools

Adegbenro et al. (2019) stated that ICT tools include desktop and laptop
computers, printers, scanners, software programs, data projectors, and interactive
teaching boxes. Conversely, Fusic et al. (2020) defined ICT tools as the newest
technologies or devices, and concepts such as the flipped classroom, mobile
applications, and clickers devices are used by both students and teachers. These
ICT tools have been increasingly used in the education system and have had a
major impact on teaching and learning processes. Hashim et al. (2018) found that
mobile learning applications (such as WhatsApp) can provide students with a
way to practise their English in real situations. The results showed that students’
vocabulary noticeably improved and students who were afraid of making
grammatical mistakes had greater confidence. Their interest in learning English
was also increased by the integration of digital learning.
This result is parallel to a study by Marisa and Zaiyana (2018) who found that
using podcasts could improve confidence in using listening and speaking skills
by students. Radzuwan et al. (2017) attempted to determine the usefulness of
interactive audio-based applications in enhancing English language speaking
skills amongst students with less proficiency in English, and found that every
student increased their confidence levels after engaging with the interactive
audio-based application. The students were actively engaged with the teacher in
the feedback sessions, and the application’s interactive nature is believed to be
effective in improving the speaking skills of students.
Evidence also exists which indicates that ICT could facilitate various learning
styles and abilities, therefore making learning more effective because it involves
multiple senses within a multimedia context (Le, 2020). Semenov et al. (2005) also
found that integrating ICT involves multiple senses which allows those with a
social, mental, or physical disadvantage to have greater active involvement in the
learning process. Brewster et al. (2003) further added that younger learners learn
better when using all of their senses and that they tend to need more physical
activities to help them with their learning. This suggests that using ICT makes
learning more effective for younger learners, as ICT involves more senses in a
multimedia context parallel to the younger learners’ learning styles and abilities.

154
2.2 Flipgrid
Today, ICT is very popular in schools, and Flipgrid has also become a popular
classroom tool; researchers are expressing great interest in exploring its
effectiveness in teaching and learning practices. One study into Flipgrid showed
that it reduced anxiety and improved communication skills among 22
undergraduate students. Hashim et al. (2018) found that Flipgrid could help
students improve their confidence level, as they were more comfortable using the
language when they could practise it. Stoszkowski (2018) studied the use of
Flipgrid to develop social learning among undergraduate students and found that
Flipgrid could bring the back row to the front and that everyone became more
participative in the teaching and learning process. Johnson and Skarphol (2018)
studied the effects of Flipgrid and digital portfolios on student communication
and engagement in a connected learning secondary visual art classroom. The
results revealed that Flipgrid could motivate the introverted students to become
more involved in discussions with their classmates. There was an overwhelming
response from the students, showing the less threatening nature of Flipgrid. The
students stated that Flipgrid is a good platform for practising English as they
could receive feedback from the digital platform, which is less intimidating.
Based on the aforementioned studies, it can be seen that many researchers have
started to explore the effectiveness of Flipgrid in improving English-speaking
skills in various educational institutions. However, little is known about the
attitudes of pupils towards using Flipgrid in learning English-speaking skills in a
Malaysian CEFR primary classroom, which is the reason why this study was
conducted.
2.3 Attitude Towards Using ICT Tools in Learning the English Language
Abdullah and Shah (2014) state that an attitude is a set of beliefs which develops
over time and which could determine a person’s behaviour. It is a key element in
determining the participation of students in language learning (Abdullah & Shah,
2014). In a study which revealed students’ attitudes towards using blended
learning, Ikhwan and Widodo (2019) discovered that students’ attitudes correlate
with teachers’ designed model of teaching, implying that this teaching model
could influence the attitudes of students towards using blended learning and
therefore affect their results in English language learning. The study found that
students who have positive attitudes are motivated to participate in blended
learning and obtain better grades in English. This concurs with the work of
Sudiran (2016) who explored students’ attitudes towards using ICT to learn in
English universities. Sudiran (2016) found that students with positive attitudes
towards using ICT as a medium to learn English at the higher education
institutions led to them improving their English.
2.3.1 Technology Acceptance Model (TAM)

It is believed that students’ attitudes towards using ICT in learning and teaching
English could be affected by a few factors. Davis (1989), in the technology
acceptance model (TAM), suggested that an individual’s intention towards using
e-learning tools is the result of perceived ease of use of e-learning tools (PEU) and
perceived usefulness (PU) (Davis et al., 1989). Davis (1989) states that PU is the
extent to which users believe that the use of technology will improve their job

155
performance whereas PEU is the extent to which users believe that using the
system will be effortless. Hong et al. (2009) found that PEU positively affected
users' attitudes and their PU. This is parallel to the research of Wu and Chen
(2017), who investigated continual intentions to use MOOCs, revealing that PEU
is a strong predictor of PU and that students are more likely to perceive MOOCs
as useful if they perceive them as easy to use. As previously mentioned, many
researchers have shown great interest in exploring the intentions of users to use
various e-learning systems and technologies as there has been significant growth
and adoption of technology in education (Al-Ammary et al., 2014; Hsu & Chang,
2013; Sanchez & Hueros, 2010; Wu et al., 2013). Yet, TAM has been insufficiently
used in a Flipgrid context. In this study therefore, TAM is used to investigate the
attitudes of primary pupils towards using Flipgrid to learn English-speaking
skills in the Year 4 CEFR classroom.
2.4 Past Studies

Given the fast pace of technological development and the increasing importance
of speaking skills in today’s globalised world, several studies related to using ICT
in learning and teaching speaking skills have been conducted. Iqbal (2017)
examined mobile phone use and the perceptions of students towards m-learning
and found positive attitudes towards m-learning were mainly influenced by
integrative motivation, instrument motivation and attitudes towards the
classroom and the teacher. Faramarzi et al. (2019) found the same, that learners
also have positive attitudes and perceptions in an e-learning project towards ESL
vodcasting tasks, they saw the experience as significantly positive and
constructive and regarded it as enjoyable. Faramarzi et al. (2019) also determined
that learners were satisfied with the application as it is user-friendly and suited to
teaching speaking skills. It can therefore be concluded that the positive attitudes
of learners towards using ICT tools in language learning can be influenced by the
model of teaching quality, as well the e-learning experiences of learners.
The aim of this literature review was to gain an understanding of the trends in
teaching and learning of ESL and inform readers of the different research aspects
conducted in the learning and teaching of English-speaking skills, in addition to
the influences of attitudes on ESL learning. It can be seen from the reviewed
research that ICT positively impacts on the teaching and learning process. This
implies that more research is required to obtained an improved understanding of
pupils’ attitudes towards ICT integrated English lessons. This field of inquiry is
crucial as it directly impacts the learning of English-speaking skills.
3. Methods
3.1 Research Design
This study used a mixed-methods approach with an explanatory sequential
design. Creswell (2014) explains that this is a simple design which provides a
thorough explanation to the quantitative results obtained using qualitative
methods. Tashakkori and Teddlie (2009) state that this research design provides
better inferences and a greater quantity of divergent views. Methodological bias
can also be avoided by using both quantitative and qualitative approaches, which

156
allows the researcher to obtain an improved understanding of the reality of the

phenomenon being investigated (Subedi, 2016).
In this study, the researcher gathered and interpreted both quantitative and
qualitative data to explore the attitudes of primary pupils towards using Flipgrid
to learn English-speaking skills in a Year 4 CEFR classroom. The aim of integrating
the use of qualitative data into this study was to perform a deeper investigation
of the attitudes of pupils towards using Flipgrid to provide the researcher with a
more comprehensive view of the problem. However, using a quantitative
approach alone is insufficient to properly understand the problem. Therefore, the
research design chosen in this study is suitable for the study’s purpose.
3.2 Research Participants

The participants were 60 CEFR primary Year 4 pupils selected from a population
of 360, as the researcher aimed to only concentrate on CEFR primary Year 4 pupils
with low English-speaking proficiency. Using purposive sampling would better
assist the researcher as it is a non-random technique involving the identification
and selection of individuals with particular characteristics which could help with
providing information relevant to the area of interest (Etikan et al., 2016).
All participants were chosen based on a School Based Oral Assessment (SBOA) in
the first semester of 2020. They were ranked between band 2 and band 3, meaning
that they could communicate basic information in regard to their opinions,
everyday routines, directions and simple predictions with support from a teacher;
some were able to display adequate ability to communicate simple information
based on the criteria of assessment set by the Malaysian Education Ministry. All
also had a minimum of three years of English language learning experience in
schools.
3.3 Research Instruments

The questionnaires used in the research were adapted from a TAM designed by
Davis et al. (1989) to determine the attitudes of primary pupils towards using
Flipgrid to learn English speaking skills in a Year 4 CEFR classroom (refer to
Appendix 1). Pupils responded to four major constructs, which were perceived
ease of use, perceived usefulness, attitudes and behavioural intentions towards
using Flipgrid in learning English-speaking skills in a primary CEFR classroom.
A five-point Likert-scale was used to measure the responses, which ranged from
1- Strongly Disagree, 2- Disagree, 3- Neutral, 4- Agree to 5- Strongly Agree. The
questionnaire’s reliability was measured using Cronbach’s alpha coefficient for
each construct, which revealed that questionnaires achieved acceptable reliability,
α = 0.977 (refer to Appendix 2).
The researcher also used semi-structured interviews to determine the attitudes of

pupils towards using Flipgrid to learn English-speaking skills. Therefore, the
interview protocol was developed which was adapted and adopted from
Ihmeideh and Al-Maadadi (2018) (refer to Appendix 3). The interview protocol
was comprised of six questions and conducted with the participants using the
Google Meet platform. The interview sessions were recorded for data analysis
purposes.

157
3.4 Data Collection Procedure

The study was carried out over the course of one month through four phases. The
four phases were: (1) recruitment of participants, (2) training of the primary
English teacher, (3) the implementation of Flipgrid and (4) post-evaluation.
3.4.1 Phase 1: Recruitment of participants

The researcher requested permission from the selected school before conducting
the study. 60 less proficient CEFR primary Year Four pupils were identified as
participants. They were informed of the purpose of the study and promised that
all information and responses from them would be confidential.
3.4.2 Phase 2: Training of the primary English teacher

In the second phase, the researcher trained a primary English teacher on how to
use Flipgrid to teach speaking skills using an online workshop conducted using
the Google Meet platform.
3.4.3 Phase 3: The implementation of Flipgrid

In the first online meeting with the pupils the primary English teacher introduced
them to Flipgrid. They were taught how to view and complete their assignments
using Flipgrid via online hands-on practice. During the second online meeting
and thereafter, the selected teacher used Flipgrid to teach speaking skills during
every speaking lesson for one month.
3.4.4 Phase 4: Post-evaluation

Phase 4 was the post-evaluation phase where pupils answered questionnaires in
Google Forms through a link attached in their Google Classroom. 20 participants
were randomly chosen to take part in an interview session with the researcher
using the Google Meet platform. The reason was to find out more about their
attitudes towards using Flipgrid to learn English-speaking skills.
3.5 Data Analysis Procedure

The means and standard deviations for all questionnaire items were analysed
using descriptive analysis. To determine the attitudes of pupils towards using
Flipgrid, the score obtained from the questionnaires was compared with the
medium score as the cut-off point. The total score of the questionnaires ranged
from 12 to 60 if no items were left blank, therefore 36 was the medium score
used as the cut-off point. A score above 36 was deemed to indicate a positive
attitude while a score below 36 was the opposite. Furthermore, all the items
were also analysed in four components related to attitudes towards using
Flipgrid, namely perceived ease of use, perceived usefulness, attitudes and
behavioural intentions, to determine which componential variable the
participants were most concerned with. In each of the components, the mean
score of each item indicated the factors influencing the attitudes of pupils
towards using Flipgrid. The data was analysed using the Statistical Package for
Social Scientist (SPSS) Version 23.
Thematic analysis was used to analyse the data obtained from the semi-structured
interviews. The recordings of the semi-structured interview sessions were then
transcribed and the researcher read the transcripts thoroughly to search for

158
emerging data, generate initial codes in relation to the research questions, and
search for emerging themes (Braun & Clarke, 2006; Denscombe, 2007). To
maintain anonymity and confidentiality, the pupils were coded as P1, 2 and 3.
4. Findings and Discussion

4.1 Analysis of Questionnaire Data
The mean and standard deviation of the overall attitudes of primary pupils
towards using Flipgrid in learning English-speaking skills are shown in Table 1.
Table 1: Mean and Standard Deviation of Primary Pupil’s Attitudes Towards Flipgrid
Componential Variable Mean
Perceived Ease of Use 11.26
Perceived Usefulness 10.90
Attitude Towards Use 11.31
Behavioural Intention 10.98
Attitude Towards the Use of Flipgrid
44.46
(Overall)
The results in Table 1 show that the mean score of attitudes of primary pupils
towards using Flipgrid to learn English-speaking skills was 44.46. This score was
above the medium score (36) which was used as the cut-off point. The findings
reveal that primary pupils had positive attitudes towards using Flipgrid in
learning English-speaking skills.
The mean score for attitudes towards use was 11.31. This was the highest among
all the other componential variables. This result therefore shows that primary
pupils were optimistic about using Flipgrid in learning English-speaking skills.
Some positive remarks from the pupils included:
“I feel so happy using Flipgrid to learn English-speaking skill. I can record
my own voice and watch it later on. It is very fun (Sic)” (Pupil 4) and
“I like Flipgrid because it is fun and relaxing (Sic)” (Pupil 12).
The second and third factors which contributed to the positive attitudes of
primary pupils towards using Flipgrid were the pupils’ perceived ease of Flipgrid
usage and their intentional behaviours to use Flipgrid, with a mean score of 11.26
and 10.98, respectively. The result explains that pupils were in favour of using
Flipgrid to learn English-speaking skills, which is supported by a pupil who
commented:
“I want to use it again next time because it is very easy to use and it is
very good also.” (Pupil 6).
The mean score for perceived usefulness was 10.90. This was the lowest of the
scores and indicates that pupils’ perceptions on the usefulness of Flipgrid had
contributed the least to their positive attitudes towards using Flipgrid compared
to all the other componential variables.

159
4.2 Analysis of the Semi-structured Interviews

Semi-structured interviews was also conducted during the research. The main
findings of the interviews are presented based upon the following themes:
4.2.1 Perceived Ease of Use

It was found that primary pupils positively perceived that using Flipgrid was easy.
Most pupils (90%) displayed positive responses towards using Flipgrid. When
requested to provide their responses on the ease of using Flipgrid, 18 pupils stated
that Flipgrid was simple and convenient to use. However, 10% of pupils
commented that they encountered technical issues when using Flipgrid. This may
be because of an unstable internet connection, as mentioned by Pupil 3:
“Sometimes, there is technical error when the Internet line is not good.”
These findings suggest that, in general, pupils perceived the use of Flipgrid as
easy. Some pupils reported a weak internet connection and technical issues as
some of the challenges faced when using Flipgrid, although these problems could
be overcome without many issues. Parents can also play a significant role in
providing pupils with enough ICT facilities so that pupils can benefit from the
learning process.
4.2.2 Perceived Usefulness

In terms of perceived usefulness, the findings indicate that primary pupils highly
perceived Flipgrid as a useful tool to learn English-speaking skills. Over half of
pupils (75%) stated that using Flipgrid could help them to increase their
confidence in speaking English and reduce their speaking anxiety. They stated
that using Flipgrid was relaxing as they could practise speaking at their own pace
without worrying about being judged by others.
Over half of the pupils (55%) stated that using Flipgrid could help them to
improve and develop their English language proficiency. It helped them to master
the language better as they could repeatedly practise. This was demonstrated in
the case of Pupil 15, who reacted very positively towards using Flipgrid to learn
English-speaking skills. She stressed that Flipgrid was a useful tool to learn and
improve her English language as it is about more than simply memorising. It
requires continuous practice to excel in the language.
Seven pupils (35%) also stated that using Flipgrid helped them with speaking
better English as they felt less stressed having practised it many times, indicating
that using Flipgrid allowed them to foster the development of their English-
speaking skills effectively. Three pupils (15%) stated that Flipgrid was a useful
tool as it allowed them to discover more English words as they were given time
to complete the task using Flipgrid at home and could therefore look up new
words to express their thoughts and ideas more effectively. When their
vocabulary increases by using Flipgrid, their English-speaking skills could also
improve.
When asked about the benefits of learning English-speaking skills using Flipgrid,
four pupils (20%) stated that Flipgrid could facilitate peer learning during
English-speaking activities. Meanwhile, two pupils (10%) remarked that Flipgrid

160
is a good platform for pupils to practise their English language skills, claiming
that it provided great opportunities for using the English language in their daily
conversations. This suggests that Flipgrid is a powerful tool for everyone to
practise their English language skills.
4.2.3 Attitude Towards Use

In terms of attitude towards using Flipgrid, all pupils showed very positive
responses. They were pleased that Flipgrid had improved their English-speaking
skills and claimed that Flipgrid had been of great benefit to them. They highly
perceived Flipgrid as a fun learning tool that could help them to develop their
English language, suggesting that primary pupils had a positive attitude towards
using Flipgrid to learn English-speaking skills. They also commented that using
Flipgrid had provided them with a positive student-centred learning environment
which made learning fun. Pupil 15 stated that it was fun to be able to edit his own
video after recording himself using Flipgrid, while Pupil 5 also added that it was
fun to take a photo of herself after the recording. Iqbal (2017) mentioned that the
positive attitudes of pupils is mainly influenced by integrative motivation,
instrument motivation and attitudes towards the classroom and teacher. The
findings reveal similar results to this study, which aims to investigate the attitude
of primary pupils towards using Flipgrid. It can therefore be concluded that
primary pupils are positive towards using Flipgrid to learn English-speaking
skills.
4.2.4 Behavioural Intention

The findings indicate that all pupils (100%) exhibited a high positive intention to
learn English-speaking skills using Flipgrid, stating that it was fun to use and
helped to develop their English language skills. They saw it as an enjoyable and
practical learning tool, suggesting that pupils were positive towards the
experience of using Flipgrid. Based on these findings, the benefits of using
Flipgrid can be seen when learning English-speaking skills, and it had convinced
the pupils to continue using it in the future.
4.3 Discussion on Primary Pupils’ Attitude Towards Using Flipgrid to Learn

English Speaking Skills
From the research findings it is concluded that primary pupils have a positive
attitude towards using Flipgrid. In the study, pupils attributed their positive
attitudes to the fun elements of Flipgrid. Hence, it can be inferred that, because
Flipgrid is innovative, it has aroused the interest of pupils to learn English-
speaking skills as they can experience a new kind of learning in a fun environment.
It also encourages student-centred learning which is parallel to the younger
learners’ learning styles and abilities. This concurs with Brewster et al. (2003) who
stated that younger learners learn best when they use their senses and perform
more physical activities to help them in their learning.
Moreover, primary pupils’ perceived ease of Flipgrid usage and their behavioural
intentions towards using it has added to their positive attitude towards using
Flipgrid. This study has found that Flipgrid was simple and convenient to use,
which was also found by Faramarzi et al. (2019), who stated that learners were
satisfied with the vodcasting application as it is user-friendly and suitable to be

161
used to teach speaking skills. The findings reveal similar results to this study,
which aims to identify the attitudes of primary pupils towards using Flipgrid to
learn English-speaking skills.
The result also indicate that the behavioural intention of pupils to use Flipgrid is
mainly influenced by their positive e-learning experiences. This agrees with the
research of Faramarzi et al. (2019) which concluded that learners had positive
attitudes and perceptions towards ESL vodcasting tasks in an e-learning project.
They saw the experience as significantly positive and constructive and regarded
it as enjoyable. This therefore implies that primary pupils’ e-learning experiences
impact upon their attitudes towards using ICT tools to learn English-speaking
skills.
The findings also reveal that pupils’ perceived usefulness also contributed to their
positive attitude towards using Flipgrid, with pupils being optimistic towards
using Flipgrid as it could help to improve their English-speaking confidence and
reduce their speaking anxiety. Furthermore, pupils stated that Flipgrid was a
useful tool to develop their English-speaking skills, a result which is supported
by Hashim et al. (2018) which revealed that using mobile learning (WhatsApp)
can help to provide students with a platform on which to practise their English
communication skills in real situations. The results revealed that students’
vocabulary had noticably improved and those who were fearful of making
grammatical errors had greater confidence to communicate in English. They also
gained great interest in learning English with the integration of digital learning.
The conclusion is therefore that primary pupils have high positive attitudes
towards using Flipgrid to learn English-speaking skills. This is significant because
Abdullah and Shah (2014) stated that attitude is a set of beliefs developed over a
period of time which determines a person’s behaviour and is a main element in
determining their participation in language learning (Abdullah & Shah, 2014).
Sudiran (2016) noted that students have a positive attitude towards using ICT as
a medium in learning English in higher education institutions and this results in
them advancing their English in their educational field of study. Ikhwan and
Widodo (2019) concluded that students who have positive attitudes are motivated
to participate in blended learning and obtain good scores in English. These studies
reveal that attitudes have a significant role in the learning processes of pupils. As
speaking skill is a required skill to communicate, pupils should search for
alternatives to better engage with the lesson. In this instance, Flipgrid can function
in assisting pupils to learn English-speaking skills.
5. Conclusion and Implications

In conclusion, analysis of the findings reveals that primary pupils have a high
positive attitude towards using Flipgrid to learn English-speaking skills. They
claim that Flipgrid has facilitated an opportunity to experience a new kind of
learning in a fun environment which encourages student-centred learning and
which is parallel to the younger learners’ learning styles and abilities. Flipgrid can
help to improve the pupils’ English-speaking confidence and reduce their
speaking anxiety. Despite all the positive responses, some pupils stated that a

162
weak internet connection and technical issues hindered their use of Flipgrid.
These findings can of benefit to the Minister of Education and headmaster as they
could be made aware of the problems faced by primary pupils in using Flipgrid
for the learning of English-speaking skills. Actions could be taken to help primary
pupils to overcome their problems so as to encourage wider usage of Flipgrid in
ESL learning. One major limitation faced during the study was the time factor.
Pupils who were not familiar with using Flipgrid took up extra time. It is
recommended that more time should be spent on the duration of the study and
hopefully future research would yield a more significant result. In addition,
deeper research into using Flipgrid among secondary pupils in a CEFR classroom
would be beneficial. Future research could provide insight to English educators
on the different alternatives to teaching English-speaking skills among secondary
pupils. The results may be used to provide information to the Ministry of
Education on the use of the e-learning approach in the teaching and learning of
English-speaking skills in the Malaysian CEFR classroom. In general, the study’s
findings have revealed promising evidence that using Flipgrid offers a
worthwhile alternative teaching and learning approach of English-speaking skills.
The suggestions for future research will hopefully yield a more significant result
that could shed light on the role of Flipgrid in the teaching and learning of
English-speaking skills.
6. References
Abdullah, H. I., & Shah, P. M. (2014). Motivation and attitudes towards learning English
among undergraduates in National University of Malaysia (UKM). International
Journal of English and Education, 3(4), 209-227.
Adegbenro, J. B., Gumbo, M. T., & Olugbara, O. O. (2019). Exploring technological knowledge
of office data processing teachers: using factor analytic methods. TPACK, 155-
184. https://doi.org/10.4018/978-1-5225-7918-2.ch007
Al-Ammary, J. H., Al-Sherooqi, A. K., & Al-Sherooqi, H. K. (2014). The acceptance of social
networking as a learning tools at University of Bahrain. International Journal of
Information and Education Technology, 4(2), 208-214.
https://doi.org/10.7763/ijiet.2014.v4.400
Azman, H. (2016). Implementation and challenges of English language education reform
in Malaysian primary schools. 3L: Language, Linguistics, Literature, 22(3), 65-
78. https://doi.org/10.17576/3l-2016-2203-05
Bakar, S. K. A. (2018). Looking for jobs? Learn to speak English first, says MEF.
https://www.freemalaysiatoday.com/category/nation/2018/01/18/looking-
for-jobs-learn-to-speak-english-first-says-mef/
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research
in Psychology, 3(2), 77-101. https://doi.org/10.1191/1478088706qp063oa
Brewster, J., Ellis, G., & Girard, D. (2003). The primary English teacher’s guide. (new ed.).
London: Penguin Books.
Coldwell, D. (2019). Negative influences of the 4th industrial revolution on the workplace:
Towards a theoretical model of entropic citizen behavior in toxic organizations.
International Journal of Environmental Research and Public Health, 16(15),
2670. https://doi.org/10.3390/ijerph16152670
Creswell, J. W. (2014). A concise introduction to mixed methods research. California: SAGE
publications.

163
Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of
information technology. MIS Quarterly, 13(3), 319-340.
https://doi.org/10.2307/249008
Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989). User acceptance of computer
technology: A comparison of two theoretical models. Management Science, 35(8),
982–1003. https://doi.org/10.1287/mnsc.35.8.982
Denscombe, M. (2007). The good research guide for small-scale social research projects (3rd
ed.). United Kingdom: McGraw-Hill.
Etikan, I., Musa, S. A., & Alkassim, R. S. (2016). Comparison of convenience sampling and
purposive sampling. American Journal of Theoretical and Applied Statistics, 5(1), 1-
4. https://doi.org/10.11648/j.ajtas.20160501.11
Faramarzi, S., Tabrizi, H. H., & Chalak, A. (2019). Learners’ perceptions and attitudes
towards vodcasting tasks in an e-learning project. Teaching English with
Technology, 19(3), 3-21.
Free Malaysia Today. (2017). Economist: Poor English, lack of experience costing grads jobs.
https://www.freemalaysiatoday.com/category/nation/2017/11/13/economist
-poor-english-lack-of-experience-costing-grads-jobs/
Fusic, S. J., Anandh, N., & Thangavel, M. (2020). A case study on improving learner
engagement by incorporating ICT tool usage and active learning strategies in engineering
courses. Pennsylvania: IGI Global.
Haidara, Y. (2016). Psychological factor affecting English speaking performance for the
English learners in Indonesia. Universal Journal of Educational Research, 4(7), 1501-
1505. https://doi.org/10.13189/ujer.2016.040701
Hashim, H. U., Yunus, M. M., & Hashim, H. (2018). Bitesize learning: WhatsApp status
for English communication skills. Edu Innovation Proceeding, 8-10.
Hong, J. Y., Suh, E. H., & Kim, S. J. (2009). Context-aware systems: A literature review and
classification. Expert Systems with Applications, 36(4), 8509-
8522. https://doi.org/10.1016/j.eswa.2008.10.071
Hsu, H. H., & Chang, Y. Y. (2013). Extended TAM model: Impacts of convenience on
acceptance and use of Moodle. Online Submission, 3(4), 211-218.
Ihmeideh, F., & Al-Maadadi, F. (2018). Towards improving kindergarten teachers’
practices regarding the integration of ICT into early years settings. The Asia-Pacific
Education Researcher, 27(1), 65-78. https://doi.org/10.1007/s40299-017-0366-x
Ikhwan, E. J. Q., & Widodo, P. (2019). Attitude conception: The role of blended learning
in environmental education. Online Submission, 2(6), 53-62.
Iqbal, S. (2017). Mobile phone usage and students’ perception towards m-learning: A case
of undergraduate students in Pakistan. The Journal of Distance Education, 32(1).
Johnson, M., & Skarphol, M. (2018). The effects of digital portfolios and Flipgrid on student
engagement and communication in a connected learning secondary visual arts classroom.
https://sophia.stkate.edu/maed/270/
Le, T. M. A. I. (2020). Benefits and challenges to integrate ICT in EFL teaching and learning
activities. Journal of Research & Method in Education, 10(3), 46-50.
Marisa, Y., & Zaiyana, P. (2019). Podcast: An alternative way to improve EFL students’
listening and speaking performance. Englisia: Journal of Language, Education and
Humanities, 6(1), 15-26. https://doi.org/10.22373/ej.v6i1.3805
Nieves, K. (2020). 9 new ways to use Flipgrid in the classroom.
https://www.edutopia.org/article/9-new-ways-use-flipgrid-classroom
Nijat, N., Atifnigar, H., Chandran, K., Selvan, S. L. T., & Subramonie, V. (2019).
Psychological factors that affect English speaking performance among
Malaysian primary school pupils. American International Journal of Education and
Linguistics Research, 2(2), 55-68. https://doi.org/10.46545/aijelr.v2i2.117

164
Radzuwan, A. R., Mohamed, S. B., Rahman, M. F. A., & Shamsuddin, S. N. W. (2017).

Developing speaking skills using virtual speaking buddy. International Journal of
Emerging Technologies in Learning, 12(05), 195-
201. https://doi.org/10.3991/ijet.v12i05.6955
Rao, P. S. (2019). The role of English as a global language. Research Journal of English, 4(1),
65-79.
Sabudin, M. S. F. (2020). CEFR: A new initiative to empower English.
https://www.pendidik.com.my/2019/06/11/cefr-a-new-initiative-to-
empower-english/
Sánchez, R. A., & Hueros, A. D. (2010). Motivational factors that influence the acceptance
of Moodle using TAM. Computers in Human Behavior, 26(6), 1632-1640.
https://doi.org/10.1016/j.chb.2010.06.011
Semenov, A., Pereverzev, L., Bulin-Sokolova, E., Anderson, J., Khvilon, E., & Berenfeld, B.
(2005). Information and Communication Technologies in Schools: A Handbook for
Teachers. France: UNESCO.
Sidhu, G. K., Kaur, S., & Chi, L. J. (2018). CEFR-aligned school-based assessment in the
Malaysian primary ESL classroom. Indonesian Journal of Applied Linguistics, 8(2),
452-463. https://doi.org/10.17509/ijal.v8i2.13311
Stoszkowski, J. R. (2018). Using Flipgrid to develop social learning. Compass: Journal of
Learning and Teaching, 11(2). https://doi.org/10.21100/compass.v11i2.786
Subedi, D. (2016). Explanatory sequential mixed method design as the third research
community of knowledge claim. American Journal of Educational Research, 4(7), 570-
577.
Sudiran, S. (2016). Students' attitude and their level of the ICT use as learning English
media. Sino-US English Teaching, 13(5), 315-323. https://doi.org/10.17265/1539-
8072/2016.05.001
Tashakkori, A., & Teddlie, C. (2009). Integrating qualitative and quantitative approaches to
research. The SAGE Handbook of Applied Social Research Methods 2, 283-317.
https://doi.org/10.4135/9781483348858.n9
Welsh, B., & Chang, C. (2020). Malaysia’s youth on the unemployed frontline.
https://www.malaysiakini.com/columns/521283
Wu, B., & Chen, X. (2017). Continuance intention to use MOOCs: Integrating the
technology acceptance model (TAM) and task technology fit (TTF) model.
Computers in Human Behavior, 67, 221-232.
Wu, C., Kuo, Y., & Wu, S. (2013). Investigating the antecedents of university students’
behavioral intention to use iPad for learning. International Journal Of E-Education,
E-Business, E-Management And E-Learning, 3(6), 468-471.
https://doi.org/10.7763/ijeeee.2013.v3.280

165
Appendix 1
Attitudes of Primary Pupils Towards using Flipgrid – Questionnaire
(adapted from the Technology Acceptance Model)
This survey is being conducted to understand the attitudes of pupils towards

using Flipgrid. Please read the following statements carefully and choose from 1
to 5 on the scale below as to how accurately they describe you. Please answer as
sincerely as possible.
1. Gender: Male Female
2. Race: Chinese Malay Indian Others
3. First Language Spoken: Chinese Malay English Tamil Others
4. Number of years spent studying English: <3 >3
5. Age when you first began to study English: 0-2 3-6 7-10
A Perceived Ease of Strongly Disagree Neutral Agree Strongly

Use Disagree Agree
1 2 3 4 5
1 I think using
Flipgrid does not
take too much time.
2 Learning to use
Flipgrid for my class
activities was easy.
3 Interacting with my
teacher and other
members through
Flipgrid was easy
and not stressful.
B Perceived
Usefulness
1 Learning using
Flipgrid helped me
to speak English.
2 I think that using
Flipgrid can help me
to improve my
English-speaking
skill.
3 The comments and
feedback given by
my teacher and
classmates in
Flipgrid were really
useful.

166
C Attitude Towards
Use
1 I like using Flipgrid
to learn how to
speak English.
2 I have a positive
attitude towards
using Flipgrid in
learning how to
speak English.
3 I feel that using
Flipgrid is a good
way to learn how to
speak English.
D Behavioural
Intention
1 I will continue to use
Flipgrid to learn
how to speak
English.
2 Using Flipgrid
helped me to
become more
willing to speak
English.
3 When I use Flipgrid,
I have more self-
confidence to speak
English.

167
Appendix 2
Cronbach’s alpha for primary pupils’ attitudes towards the use of Flipgrid
Questionnaire adapted from the TAM
Reliability Statistics
Cronbach's
Alpha Based
on
Cronbach's Standardized N of
Alpha Items Items
.977 .982 12

168
Appendix 3
SEMI-STRUCTURED INTERVIEW PROTOCOL
(Adapted from Ihmeideh & Al-Maadadi 2018)
This semi-structured interview is used to determine the attitudes of pupils

towards using Flipgrid in the learning of English-speaking skills in the Year 4
CEFR classroom.
1. Do you think Flipgrid is a useful tool for you to learn English-speaking skills?
2. How do you feel when using Flipgrid to learn English-speaking skills?
3. What are the benefits of learning English-speaking skills using Flipgrid?
4. Do you like to use Flipgrid to learn English-speaking skills?
5. Do you find it challenging to use Flipgrid in the process of learning English-
speaking skills?
6. Would you like to use Flipgrid again in the future to learn English-speaking
skills?

169

Vol. 20, No. 3, pp. 169-187, March 2021
Effect of Accounting Lecturer Behavior on the

Level of Online Learning Outcomes Achievement
Alwan Sri Kustono

Accounting Department, University of Jember, Indonesia
https://orcid.org/0000-0002-3180-0735
Wahyu Agus Winarno

Faculty of Economics and Business, University of Jember, Indonesia
https://orcid.org/0000-0003-3124-3278
Ardhya Yudistira Adi Nanggala

Faculty of Economics, Abdurachman Saleh University, Situbondo, Indonesia
https://orcid.org/0000-0001-5133-6266
Abstract. Changes in learning models in reaction to the COVID-19

pandemic have a significant impact on how accounting is taught. The
objective of this study was to compare the differences in learning
outcomes before and during the pandemic. A total of 367 research
participants were collecting and the data were analyzed using the Partial
Least Square – Structural Equation Modelling approach. Additional
testing to control the demographic variable shows that the demographic
variable is not a determinant of learning outcome achievement. The
results showed that anxiety reduces the ease of use, and external control
perception positively affects it. The theoretical implication is that the
online learning outcome increases depending on user behavior variables.
Technology acceptance variables are a mediation between personality
variables and online learning. Other constructions of the TAM model
have been empirically proven. The level of achievement before the
pandemic is higher than during the pandemic. These results indicate that
the implementation of online learning is more effective if it has been
prepared from the beginning. The practical implication is to achieve a
good outcome. A university must reduce anxiety and increase the
positive control of the external perception of each lecturer.
Keywords: online learning playfulness; ease of use; self-efficacy;

usefulness; behavioral intention to use; achievement of learning outcome
1. Introduction
The spread of COVID-19 at the end of 2019 has led to changes in all areas. Some
countries have implemented lockdowns, migration restrictions, and physical
distance policies. The pandemic is also impacting educational institutions. The
©Authors
170
teaching and learning process in Indonesia must change the techniques, methods,
and related learning tools. The teaching and learning process is done from and at
home—the learning method goes from face-to-face in the classroom to distance
learning. The learning model that was once a complementary method before the
pandemic is now the core mode of instruction. Almost all universities in Indonesia
are applying this method. Changing the learning model from the classroom to
online was an emergency learning strategy. The learning design was modified,
but the same learning outcomes are expected from the online model.
Online learning uses online technology to convey material and knowledge

(Nortvig et al., 2018). This method minimizes face-to-face interaction and relies on
technology to form virtual classrooms. Lecturers must operate advanced
technology to ensure that the learning process is running well. The preferences
and skills required differ from classroom learning. Lecturers are required to
handle devices that support online learning, both technology and learning design.
There are two beliefs for determining the behavior of information technology
acceptance. The first belief is ease of use and usefulness drive successful
implementation. A framework based on reasoned action theory is built on
understanding the factors that cause a new technology to be accepted and applied
(Davis et al., 1989). This acceptance model is known as the technology acceptance
model (TAM). This model explains that its actual use and success are influenced
by the ease of use and mediated by usefulness. The second model adds
adjustment and anchors as additional variables. Acceptance of online learning
uses antecedent variables: self-efficacy, perception of external control, online
learning, playfulness, and anxiety (Al Kurdi et al., 2020; Clair, 2015; Igbaria &
Iivari, 1995). This model is expected to make the best contribution in predicting
and explaining the success of online learning. This framework is known as the
computer usage model (CUM). Learning outcomes have not been achieved
optimally due to the emergency learning model, minimal preparation, and initial
learning design due to the COVID-19 pandemic situation. TAM provides
direction on how an information system can be received and delivers optimal
results. CUM provides advice regarding variables that are predictors of ease of
use and usefulness variables. In the context of online learning, CUM predict that
online learning anxiety (anxiety), self-efficacy (SE), perception of external control
(PEC), and system playfulness (PF) are variable which determines the perceived
ease of use (EU) and usefulness (PUS). University management should reduce
anxiety to increasing acceptance. Improving self-efficacy, external control
perception, and online learning playfulness will also encourage success (Achim
& Kassim, 2015; Adetimirin, 2015; Kustono, 2020). Self-efficacy is a belief in
organizing and carrying out a series of activities necessary to obtain planned
performance.
The contributions of this research in the field of accounting education are as

follows: (1) it examines the antecedents of online learning acceptance during the
Covid-19 pandemic, (2) provides evidence of learning achievements before and
during the pandemic, and (3) offers accounting lecturers insights into the
importance of learning design. This article is organized as follows. We first review
171
the literature and develop the hypotheses. Section 3 describes the design of the
research. The results were then discussed, and finally, the conclusions and
suggestions are presented.
2. Literature review
2.1 Achievement of learning outcomes
In colleges, learning needs to be formulated through learning outcomes (Chahine
& Khan, 2015). Learning outcomes (LO) are goals that are to be achieved in the
learning process. One form is a change in knowledge expected to occur in
students after going through a series of learning processes. The LO should show
the characteristics of the disciplines studied. The achievement of learning
outcomes (ALO) is designed using the curriculum (Nambi, 2019). The curriculum
is a map of how students can achieve specific competencies. The learning model
is an integral part of learning design. Performance competencies are designed to
be accommodated gradually in the lecture process. Assignments, evaluations,
discussion processes, and other student portfolios improve their competence
(Ayeni & Akinfolarin, 2014; Chahine & Khan, 2015). The level achieved is then
evaluated on the level of competence that each student has in a particular subject.
Each subject has a suitable learning strategy. The selection of methods takes into
account the material that students must master (Nambi, 2019). If it refers to
Bloom's taxonomy, the LO level also consists of several levels (Hanum, 2013). The
lowest level is low-level thinking skills, and the highest level is high-level
thinking skills.
2.2 Computer self-efficacy

Computer self-efficacy (SE) is an individual’s evaluation of the ability to use
computers. A person's self-efficacy plays an essential role in influencing
motivation and behavior. Some people believe that their ability to use computers
is not related to their experience in carrying out specific tasks (Chahine & Khan,
2015). It shows that belief is an essential factor for completing a task. A person
who has self-efficacy sees complex tasks requiring computer programs as an
opportunity to master various programs. SE is defined as considering any
appropriate person who can take the necessary action in such circumstances. SE
construction refers to beliefs about a person's ability to perform a behavior. SE has
a positive direct effect on ease of use. SE is a concept that can be used to measure
the knowledge of each individual, especially in terms of using accounting
information systems. Students with high SE have an advantage in digital
processing information (Abdullah & Ward, 2016). The research showed that there
are SE factors that determine the success of online learning. SE is a variable that
affects academic performance (Al Kurdi et al., 2020). Their research found that
students with high SE had better academic achievements. They tested SE on the
ease of use and usefulness of online learning systems and their influence on
behavioral intentions on online learning acceptance. Thus, the result showed that
self-efficacy serves as an antecedent to receiving online learning. Similar results
were also delivered by previous research in online learning and technology
acceptance (Achim & Kassim, 2015; Al-Azawei & Lundqvist, 2015; Chahine &
Khan, 2015). Our first hypothesis there is:
H1: Self-efficacy will have a positive effect on perceived ease of use.
172
2.3 Perception of external control

Perception of external control (PEC) is defined as the extent to which one believes
that organizational resources and technical experts support the system's use. A
person feels safe using particular applications if they think infrastructure support
is guaranteed (Pham et al., 2020).
During the pandemic, lecturers were not allowed to teach in the classroom. The
online learning method is one of the instruments that can be used to ensure the
continuation of learning. Lecturers use a wide variety of software to facilitate this
learning model. Often new lecturers use it for the first time. In situations where
there are irregularities, such as during the pandemic, the perception of external
support becomes essential. The choice of using a specific application is also based
on the university's readiness to assist student's needs. PEC refers to an
individual's perception of the extent to which an organization contributes to the
application used. If the lecturer assesses positive support, the lecturer develops a
more positive perception of the application. External control factors include time,
opportunity, and cooperation with others. Support from the organization is
important when assistance is needed to ensure employees' tasks run efficiently
and when they face stressful situations (Rhoades & Eisenberger, 2002).
Previous research discovered the influence between PEC and ease of use
(Adetimirin, 2015; Oturakci & Oturakci, 2018). System users feel that the system
is easy to use to make adjustments according to their needs. The user of the system
wants to have control over the applications used. The system platform provides
users with the opportunity to make adjustments to achieve usage goals. We
hypothesize that:
H2: Perception of external control will have a positive effect on perceived ease of use.
2.4 Online learning anxiety

Another internal factor thought to affect the use of online learning is online
learning anxiety (anxiety). Anxiety is a predictor of usefulness and indicates an
intervening variable between anxiety and reception (Gbongli et al., 2019).
Individuals prefer technology that can produce the expected results. Anxiety
negatively impacts user confidence and performance. Higher anxiety drives lower
performance. Several studies have shown the influence of computer analysis
variables on usefulness perception (PUS) and acceptance (Igbaria & Livari, 1995;
Yuwana & Kustono, 2017).
Researchers concluded that the higher the anxiety, the lower the ease of use (EU).
If a person feels anxious and feels unsuccessful in using the system, it will fail in
implementing the information system. Computer knowledge and skills have a
significant negative relationship with computer anxiety. Anxiety also negatively
impacts the acceptance of the technology (Abdullah & Ward, 2016; Ajmal &
Ahmad, 2019; Clair, 2015). Here we hypothesize that:
H3: Online learning anxiety will have a negative effect on perceived ease of use.
2.5 Online Learning Playfulness (PF)

Online learning Playfulness (PF) is defined as how much one believes that using
an online learning system gives them comfort and satisfaction. PF is the level of
173
satisfaction that the user feels. The more users enjoy the website's online
processes, the higher the chance of re-using it (Chen, 2018). PF explains that the
extent to which activities using a particular system are considered enjoyable,
regardless of the performance consequences resulting from using the system.
TAM discusses the perception of convenience, that one's attitude in using a
technology depends on the individual's playfulness in using the technology. The
more user-friendly the system is, the higher the level of playfulness (Adetimirin,
2015; Chen, 2018; Dumpit & Fernandez, 2017; Mohammadi & Isanejad, 2018). Our
hypothesis is:
H4: Online learning playfulness will have a positive effect on perceived ease of use.
2.6 Ease of use

Ease of use (EU) is the belief that an application is easy to learn, understand, and
operate (Yuwana & Kustono, 2017). If the person has a perception of ease of use,
he will accept it when using a particular application. Individuals do not reject but
rather react positively to the application. Conversely, if an individual believes that
an information system is challenging to understand and operate, he responds
negatively by rejecting the application. Individuals feel the ease of use when
meeting the indicators of easy operation; (1) the application is understandable, (2)
its use is uncomplicated, (3) it can be operated as the user wishes, and (4) it is
flexible. The application means it does not cause problems when implemented.
The context of online learning leads to the use of online learning applications.
Lecturers choose applications that are easy to use and learn. Previous research has
shown that the EU drives perceived usefulness (Abdullah & Ward, 2016; Ho et al.,
2019; Nagy, 2018). We, therefore, hypothesize that:
H5: Perceived ease of use will have a positive effect on perceived usefulness.
H6: Perceived ease of use will have a positive effect on behavioral intention.
2.7 Perceived usefulness

Perceived usefulness (PUS) is the user's belief that the system can get their work
done. The use of the applications helps improve their performance (Kustono &
Nanggala, 2020). The results of using these applications are more satisfying than
not using them. In terms of time, the work is done faster. PUS is related to
decision-making choices. Under normal conditions, if an application has benefits,
then individuals use it. Conversely, if the user finds it less valuable, they don't use
it. PUS construction is related to productivity, work performance, effectiveness,
and task completion (Lai, 2017). Usefulness is an essential control for behavioral
intention and actual usage. In the context of online learning, lecturers choose to
use applications that can help the teaching and learning process (Al Kurdi et al.,
2020). PUS increases an individual's behavioral intention to use technology
(Nanggala, 2020).
Other studies have also found supportive results (Amer et al., 2013; Feriady et al.,
2020; Siegel et al., 2017). They found a positive connection between PUS and
behavioral intention. The usefulness of the information systems is the benefit
obtained or expected by the user in carrying out their duties and work. The level
of use affects the user's system's intent. The proposed hypotheses are as follows:
H7: Perceived usefulness will have a positive effect on behavioral intention.
174
2.8 Behavioral intention to use

Behavioral intention (intention) is an antecedent of individual behavior. Intention
directs a person to behave (Nanggala, 2020). It indicates acceptance or rejection of
someone using a particular application. Intention affects how often individuals
use it. Acceptance means the use of an application to complete work more
frequently than others. More often, this indicates that the individual accepts the
new system and it reveals a high intention. Intention means the individual's
probability of doing or not doing. Action is a consequence of intention. In the
theory of planned behavior, the construct is the prediction of activity. A person
engages in conduct if they have the desire or interest to do so. Behavioral
intentions predict participation in online learning initiatives (Hanif et al., 2018).
Previous research has shown that intention positively affects technology (Al
Kurdi et al., 2020; Amer et al., 2013; Hanif et al., 2018; Nanggala, 2020). They found
a positive relationship between individual interests and the online learning
application. Lecturers’ intention strive to maximize the facilities provided in
online learning software. We therefore hypothesize that:
H8: Behavioral intention will have a positive effect on the degrees of online learning usage.
2.9 Degrees of online learning usage

Learning is focused on achieving the outcome. Learning outcome (LO) can be
fulfilled from knowledge, skills, and attitudes by social, economic, and academic
cultural conditions. Students' abilities and LO are accommodated through several
strategic steps and academic completeness, including courses, final assignments,
presentations, tests, and student portfolios. The achievement of output becomes
the focus of the learning process and improves learning quality (Chahine & Khan,
2015). In ideal conditions with adequate infrastructure preparation, the online
learning system improves students' academic performance (Muchlas, 2013). The
learning model is one of the factors that influence the success of learning activities.
Online learning improves the interaction that occurs between lecturers and
students.
According to the previous findings that different methods will produce different
outcomes (Pablico, 2017). These differences can be in the form of differences in
technique or differences in degrees of use. Effective learning is learning that
utilizes information and communication technology optimally. The online
technology application can improve the achievement of learning outcomes
(ALO). Digital learning has better positive effects on learning motivation than
traditional teaching, and digital learning shows better positive results on learning
outcomes than conventional education (Lin et al., 2017; Nortvig et al., 2018). Our
last hypothesis is:
H9: Degrees of use of online learning will have a positive direct effect on the achievement
of learning outcomes.
3. Methodology
3.1 Sample and data collection
The target population in this study is accounting lecturers in Indonesia. An online
questionnaire was used to measure nine constructs: PEC, SE, anxiety, PF, EU, PUS,
intention, OLU, and ALO. Online questionnaires were shared using Google
175
Forms. All items are measured on a five-point Likert scale, with endpoints 1
(totally disagree) and 5 (totally agree). The SmartPLS path analysis was used to
test the research hypotheses. Partial Least Square (PLS) is chosen over the
covariance-based structural equation modeling approach (CB-SEM) because PLS
is prediction-oriented (Chin, 1998) and, as such, can fulfill our goals well.
Based on these empirical theories and findings, the conceptual framework of this
study is shown in figure 1.
Figure 1. Conceptual framework
Figure 1 illustrates the relationship between endogenous variables (achievement

of learning outcomes) and antecedent variables. Nine hypotheses were tested. The
research framework develops on the alleged influence of playfulness on ease of
use, self-efficacy, ease of use on usefulness, ease of use and usefulness on
behavioral intention, the level of e-learning intentions used, and the level used
achievement.
3.2 Measurements
The instrument used in this study was an online questionnaire containing a
closing statement (see Appendix 1). To measure TAM variables, it using
Venkatesh and Bala's (2008) instruments with certain modifications (Kustono et
al., 2020). Questionnaire to measure ALO with the self-reported percentage of
achievement. The questionnaire for OLU uses seven indicators (Hanum, 2013).
Degrees of use of online learning (OLU) in this study are not proxied by the
frequency of use but the quality of online learning. The quality of online learning
follows. Other comments are statements related to the quality of online learning
being carried out. The higher the value, the more optimal the online learning will
be.
176
4. Results and discussion

Questionnaires were distributed using Google form using channels of
professional organizations and a network of colleagues for dissemination. The
total number of questionnaires returned was 373. Six questionnaires were
incomplete and were therefore ignored. Sampling using stratified method with
geographical considerations in order to obtain a sufficient and representative
composition. Lecturers who are in one area are considered to be homogeneous
characteristics following the research objectives. According to geography, data of
Indonesian accounting lecturers have not been found. The representation of the
population is adjusted to the number of Indonesians. It means that the
composition of the population in Indonesia becomes a reference for the
geographical composition of the lecturers' origin. Indonesia's territories are
grouped into six area, namely (1) Sumatra, (2) Java, (3) Kalimantan, (4) Bali and
Nusa Tenggara, (5) Sulawesi and Maluku, and (6) Papua.
Table 1. Sampling procedures

Area Total Populations % Samples % % Pop - %
(1000) Population Obtained Sample Sample
1 59,196 .8 22% 73 20% 2%
2 151,650 .2 56% 226 62% (5%)
3 15,153 .7 6% 16 4% 1%
4 16,432 .9 6% 20 5% 1%
5 22,790 .7 8% 28 8% 1%
6 43,79 .1 2% 4 1% 1%
Indonesia 269,603 .4 100% 367 100%
As a whole, the participants met geographic characteristics. Table 1 indicates the

number is over the target (Java), and some others are less than the target, reflecting
the number of colleges with accounting lecturers. The amount of difference from
the target sample is not too large, so it is considered that the sample is reasonably
representative of the population.
Table 2. Validity testing- outer loadings

PUS EU SE PEC
Org. T- Org. T- Org. T- Org T-
sample statistic sample statistic sample statistic sample statistic
Item 1 0.56 3.35 0.87 27.63 0.56 18.21 0.59 6.23
Item 2 0.79 7.28 0.91 40.41 0.49 5.40 0.80 15.19
Item 3 0.64 5.40 0.92 35.60 0.79 4.55 0.77 12.09
Item 4 0.82 15.9 0.80 35.67 0.79 13.06 0.85 18.74
Item 5 0.81 18.55 0.81 17.01
ANXIETY PF INTENTION OLU

Item 1 0.93 45.19 0.54 4.81 0.79 21.40 0.82 28.92
Item 2 0.95 73.84 0.78 3.37 0.84 24.78 0.71 17.36
Item 3 0.87 22.58 0.79 15.97 0.80 22.69 0.92 35.02
Item 4 0.63 5.14 0.84 20.88 0.79 18.73
Item 5 0.56 5.51
Item 6 0.50 4.60
Item 7 0.44 4.35
177
Overall the questionnaire items have an outer loading with a t-statistic of more
than 1.96. Each item meets the validity requirements. This study used Cronbach's
alpha technique to ensure the reliability of the questionnaire. The reliability of the
instrument is fulfilled if it shows a score of 0.70.
Table 3. Reliability testing – Cronbach Alpha

Construct Cronbach
Alpha
Anxiety 0.82
Self-efficacy 0.82
Playfulness 0.71
Perception of external control 0.72
Ease of use 0.77
Usefulness 0.78
Behavioral intention to use 0.71
Online learning usage 0.85
All measurement variables meet the requirements of reliability. Each score is

above 0.70 so that they can be used in the analysis.
4.1 Research participant description

Table 4 presents descriptive statistics for each variable used in this study and data
on the characteristics of the research participants. The data used in this study
include the following data: age, gender, education, and teaching experience.
Table 4. Descriptive statistics

Mini- Maxi- Std. Skew-
Variables Mean
mum mum Dev. ness
Perception of external control 16.00 25.00 20.77 1.76 0.84
Self-efficacy 16.00 20.00 17.28 1.64 0.67
Anxiety 5.00 13.00 7.88 1.96 0.58
Ease of use 11.00 20.00 15.59 2.03 -0.03
Usefulness 8.00 20.00 15.60 2.42 -0.22
Playfulness 7.00 15.00 11.55 1.97 -0.44
Behavioral intention to use 3.00 13.00 6.26 3.06 -0.14
Online learning usage 19.00 35.00 28.44 2.93 -0.10
Achievement of LO - During 0.65 0.75 0.70 0.02 0.38
Achievement of LO - Before 0.85 0.95 0.89 0.03 0.76
Based on the analysis results in Table 4, there are no variables that indicate
extreme distribution. It suggests that the distribution blend with the mean value
of the variable scale range for each variable. Skewness indicates good numbers as
well, except for perception of external control (PEC). The skewness for this
variable is 0.84, close to 1 but not yet. The best data distribution is on the EU
variable; that is, the median mean coincides at 15.59 and 16.00, the skewness value
is at a score of -0.03.
178
4.2 Demographics additional test

Additional testing was conducted to find indications that caused ALO differences
based on the research participants' demographics. The test was conducted to
increase the research model's credibility in detecting the antecedent variables for
achieving LO. Research participant demographics include gender, college status,
education level, age, and teaching experience. The demographic data of research
participants shows the following information. The mean of male LO achievement
participants (70.20) is lower than that of female participants (70.35). Privat
universities have better achievement than private universities. The participants'
attainment of the target LO from state universities was (70.50) higher than
participants from private universities (70.02). Participants with a doctoral degree
have a higher ALO (70.36) than LO achievement participants with a master's
degree (70.20). Lecturers who have doctor degrees have better achievement than
master's degrees. These differences need to be tested statistically to ensure that
the differences are significant. Testing was through Levene's test for variables
categorized as a dummy (0, 1) and one-way ANOVA test with a scale of more than
two (0, 1, 2).
Table 5. Levene's test for equality of variances - during a pandemic

Item Item Mean F Sig. T Sig. (2-
tailed)
Male 70.35 0.31 0.58 0.49 0.62
Gender
Female 70.20 0.54 0.59
State 70.50 20.15 0.00 1.77 0.08
College Status
Privat 70.02 1.69 0.10
Doctor 70.36 0.21 0.65 0.52 0.60
Level of Education
Master 70.20 0.57 0.57
The test results in Table 5 show that ALO of male participants was higher. This
difference is not significant because the p-value is 0.62. In other words, gender
differences were not related to ALO. Male and female accounting lecturers did
not have differences in the achievement of the learning outcome. The test results
show that the level of participants' education is different in ALO. This difference
is not significant (p-value is 0.60). There are no different learning outcomes
between accounting lecturers with a doctoral degree and master degree lecturers.
Table 5 shows that the participant college group mean does not show any
difference (0.08). This situation may be due to online learning, both private and
public, facing relatively the same infrastructure readiness problems. Indonesian
accounting lecturers who come from state universities have additional resource
support compared to private universities. In pandemic and emergency
conditions, it is not enough to provide significance for achieving LO. The situation
is an unpredictable force majeure. The best strategy that must be taken is to
prepare the competence of lecturers to use online learning media as soon as
possible. College groups are not a determinant of online learning outcomes in a
pandemic situation. Hypotheses test does not need to consider college status as a
confounding factor. Further testing was conducted using age and experience
demographics with the Analysis of Variance (ANOVA) to find the influence. The
test results are shown in Table 6.
179
Table 6. Anova- during a pandemic

Sum of Mean
Item
Squares Square F Sig.
Between Groups 7.54 3.77 1.23 .29
Age Within Groups 500.88 3.05
Total 508.42
Between Groups 4.72 2.36 .77 .47
Experience Within Groups 503.70 3.07
Total 508.42
Table 6 demonstrates participants grouped into 3 (three), namely, age up to 30

years, 31- 45 years, and 45-70 years. The ANOVA test results showed that the
differences between groups had an F-value of 1.23 with p=0.29. The age did not
show any difference in ALO. Different ages of accounting lecturers in Indonesia
are not related to the level of achievement. Participants with long experience
demographic data are grouped into three categories: age up to 10 years, 10 - 25
years, and more than 25 years. The ANOVA test results showed that the
differences between groups had an F-value of 0.77 with p = 0.47. The length of
experience as lecturers did not show any difference in ALO. Accounting lecturer
experience in teaching is not related to the achievement of learning outcomes.
5. Result and discussion

Additional testing to control for the demographic variable shows that the
demographic variable is not a determinant of LO achievement. The model can
work regardless of participant demographic factors. Hypothesis testing was
conducted by observing the calculation t-value using smartPLS application
version 3.2.2 with 500 subsamples and 300 iterations. The results are shown in
Table 7.
Table 7. Path coefficients

Relationship Entire Mean of Standard T- P-
Between Variables Sample Sub- Error statistic Values
Estimate samples
ANXIETY → EU -0.931 -0.928 0.042 22.393 0.000
PEC→ EU 0.256 0.286 0.065 -3.911 0.000
PF → EU -0.050 -0.069 0.133 0.443 0.658
SE → EU -0.136 -0.128 0.097 1.401 0.162
EU → PUS 0.415 0.420 0.140 2.967 0.003
EU → INTENTION 0.348 0.347 0.077 4.530 0.000
PUS → INTENTION 0.537 0.538 0.061 8.733 0.000
INTENTION → OLU 0.657 0.060 0.088 7.423 0.000
OLU → ALO 0.870 0.869 0.051 17.086 0.000
The test results in Table 7 show that hypotheses 3 and 4 were rejected. The test
results of PF effect on the EU have a negative path coefficient of -0.050 and a p-
value of 0.658. PF does not affect the EU. Hypothesis 3, which states that variable
playfulness positively affects the ease of use, is rejected. The test results of SE's
effect on the EU have a negative path coefficient of -0.136 and a p-value of 0.162.
180
SE does not affect the EU. Hypothesis 4, which states that the self-efficacy variable
positively affects the ease of use, is rejected.
Table 7 shows that except for the third and fourth hypotheses, other hypotheses
failed to be rejected. Hypothesis 1 states that anxiety negatively affects the ease
of use. Anxiety is proven to reduce the EU. The test result shows that anxiety
negatively affects the EU with a coefficient value of -0.931 and a p-value is less
than 0.001. Perceived anxiety reduces the confidence of lecturers in using the
online learning system. Lecturers avoid using applications that feel complicated
and unusable—the desire to use the application decreases with rising anxiety
levels. These results confirm the findings of Majid (2012). Lecturers who are
restless in using online learning find it difficult and feel compelled to use it to not
optimal their performance.
Hypothesis 2 states that the perception of external control positively affects the
ease of use. PCE increases the EU. The test result showed that PCE had a
statistically significant positive effect on the EU with a coefficient of 0.256 and a
p-value is less than 0.001. Thus it fails to reject the second hypothesis.
Lecturers who believe that an institution or environment provides the

infrastructure that supports online learning systems find it easy to use online
learning. If there is a problem with online learning, the lecturer has the confidence
that the institution will help solve it. Lecturers expect to have adequate control
over the e-learning application used. Application customization can be adjusted
according to their needs. That is consistent with Oturakci & Oturakci (2018), who
found a significant influence between PEC and the EU. If lecturers have access to
adequate technology and usage skills, external control will also increase.
The test results of hypothesis 5 testing show that the EU has a statistically
significant positive effect on PUS with a coefficient value of 0.415 and a p-value
of 0.003. It fails to reject the fifth hypothesis. The impact shown by the regression
coefficient is positive, meaning that the higher the EU, the higher the PUS. The
EU explains the extent to which one trusts an information technology system that
is free from physical and mental efforts. An easy-to-operate system will provide
convenience and provide benefits to the users. If the lecturer feels that the EU is
high, then the PUS is also high. Lecturers who believe that online learning is easy
to understand and use will react positively to this learning model.
Online learning is useful if there is an ease of use factor. If the individual feels that
the technology is easy to use and learn, it will encourage them to use the new
technology. Lecturers who find it easy to use can take advantage of online
learning. This result is in line with research conducted by research that proves EU
influence on PUS (Abdullah & Ward, 2016; Ho et al., 2019; Lin et al., 2017). The
test results demonstrate a direct positive relationship between EU and intention
to use online learning. The test result showed that statistically, the EU had a
positive effect on the intention with a coefficient value of 0.348 with a p-value is
less than 0.001. Hypothesis 6 (six) fails to be rejected. Lecturers are interested
in using online learning technology if the technology is easy to do. If the
181
individual thinks the technology has sound capabilities, the user will be interested
in using it.
A person's intention to adopt a particular part of a technology is determined by

one's attitude towards using such technology. EU has been shown to impact
intention. If the online learning system is easy for users to use, it will affect users'
interest in using online learning to carry out their activities. Individuals commit
particular actions because that they have the interest or desire to do so. These
results follow previous studies (Cigdem & Ozturk, 2016; Hanum, 2013). The
lecturer intends to use the e-learning methods to facilitate the teaching and
learning process and achieve the learning objectives. With the e-learning
applications, lecturers’ interest will increase if they find it easy to use. Therefore,
it can be concluded that the EU affects the intention in the online learning system.
The result showed that usefulness has a positive effect on behavioral intention.
PUS has an impact with a coefficient value of 0.537 and a p-value is less than 0.001.
Hypothesis 7 (seven) fails to be rejected. Usefulness is considered a measure of
the user's belief that using information technology can improve work
performance. Instead of using information technology, the work will be more
efficient and effective using information technology. The results of the work will
also be better. The lecturers' attitude in online learning is driven by how much
confidence that online learning can improve their performance.
According to the lecturers, using online learning is easy and does not require
much effort. Lecturers will take full advantage of online learning. Lecturers accept
the use of the online system to complete teaching if the system is easy to use and
completes learning tasks better and effectively. These results are in line with
previous research. Lecturers choose to use online learning applications to help the
teaching and learning process (Al Kurdi et al., 2020). Usefulness increases
individual behavioral intention to use technology (Nanggala, 2020). Other studies
have also found similar results (Amer et al., 2013; Yuwana & Kustono, 2017).
The results showed that intention had a positive effect on the OLU. The test result
indicates that intention has a significant positive impact on OLU, with a coefficient
value of 0.657 and a p-value is less than 0.001. Hypothesis 8 (eight) fails to be
rejected. Their behavioral intentions influence the behavior of lecturers in
teaching. The level of use is affected by level of interest. Expertise in operating
online learning programs includes the ability to implement learning, complete
tasks, and the competencies for using online learning software packages to
influence the degree of interaction between lecturers and students.
This study's results align with previous studies (Cigdem & Ozturk, 2016; Kustono
et al., 2020). They found a positive relationship between an individual's behavioral
intention to use technology. It shows that lecturers who use the online learning
system tend to use online learning software optimally. The learning model will be
interactive by involving students and lecturers in a discussion. This hypothesis
test results are similar to previous research conducted, which found a positive
relationship between intention and OLU variables (Al Kurdi et al., 2020).
182
Hypothesis 9 states that the degrees of use of online learning affect the
achievement of learning outcomes. The test results showed that OLU positively
influences ALO with a coefficient value and a p-value is less than 0.001.
Hypothesis 9 (nine) fails to be rejected. OLU is an actual level of action of
lecturers who use online learning technology. Its quality is showed an interactive
level between students and lecturers. The more interactive the learning pattern is,
it is assumed that the online learning system that is applied is more optimal.
From the results, it can be concluded that OLU affects ALO. It is related to the fact
that the model used was only an online learning model, which influenced the
results. However, lecturers who have transitioned to a fully online model can
improve the quality of learning. On the other hand, online learning is no longer
voluntary but has become mandatory. The use of online learning systems at
various levels determines its effectiveness. Lecturers should use the learning
model at the highest level of online learning for optimal results. These results
align with previous research (Al Kurdi et al., 2020; Hanum, 2013; Lin et al., 2017).
We use a one-sample t-test for Hypothesis 10. Table 8 shows the average learning
designed from the beginning. It shows an LOA rate of 85.77%, but it dropped to
69.76% during the pandemic.
Table 8. One-sample test

Time Dimension Mean T Statistic Sig. (2-tailed)
Difference
During pandemic 69.76 645.05 .00
Before pandemic 85.77 287.57 .00
The design from the beginning has taken into account all aspects that can support
the maximum achievement of LO. The methods implemented during pandemics
were intended as a substitute. Thus, the teaching and learning process was
achieved, even though it is not optimal in its implementation.
Before the pandemic (85.77%), the level of achievement is higher than during the
pandemic (69.76%). These results indicate that online learning is more effective
when the learning design has been prepared from the beginning. The low level of
achievement during pandemic suggests that online learning at the beginning of
the pandemic is an emergency measure. Lecturers and students were not
prepared for the learning model that was applied. Learning outcomes that were
initially designed for face-to-face should be transformed into a complete online
learning system.
The university is expected to prepare an adequate online learning infrastructure.

The use of information technology for online learning continues to evolve as
technology advances. The maximum online application provides better student
LO. Online learning can make it easier for students to improve their knowledge
and skills, and even attitudes towards the learning environment. Students can
learn both individually and collaboratively in a group to complete their learning
tasks.
183
The online learning system's planned use has a positive impact; that is, it can
increase the ALO. Online learning methods developed can affect improving
student's academic achievement. Similar results were found by (Lin et al., 2017;
Nortvig et al., 2018). They concluded that digital learning presents better positive
effects on learning outcomes than traditional teaching does. Computer-based
learning showed promising results (Cigdem & Ozturk, 2016). If planned, online
learning becomes a constructive tool, and it takes facilities the development of
critical thinking.
The practical implication of the results is that the variables that affect online
learning are anxiety and external perceptions. Universities can take corrective
action based on these two variables. University administrators can reduce anxiety
by increasing skills as well as adequate preparation. Increasing external
perceptions can be done through discussion groups, outreach and ad-hoc teams
to support lecturers using online technology.
6. Conclusions
This research aims to identify the determinants of accounting lecturers' online
learning achievements in Indonesia during the COVID-19 pandemic. There are
several antecedent constructions for online learning acceptance that have been
identified. The results showed that, first, the perception of anxiety reduces the
confidence of lecturers in using the online learning system, and the perception of
external control positively affects the ease of use. Second, online learning
playfulness and computer self-efficacy do not affect the ease of use. Lastly, the
degrees of use of online learning affect the achievement of targeted learning
outcomes. These results indicate that online learning is more effective if the
learning design has been prepared from the beginning. Lecturers should use
online learning models to their maximum to ensure optimal results. The results of
the study provide clues to how online learning can be optimized. University
administrators can use two variables: anxiety and perception of external control
by modifying the two antecedent variables' level. The results showed that more
careful preparation from the start resulted in achieving the learning outcome
target. From a theoretical perspective, the results explain that user behavior
influences online learning outcomes. The variables built from the reasoned action
view become a mediation between the lecturer’s behavior and the online learning
outcomes.
Declaration of conflicting interests

The authors declare no potential conflict of interest in this research, authorship,
or publication of this article.
7. References
Abdullah, F., & Ward, R. (2016). Developing a General Extended Technology Acceptance
Model for E-Learning (GETAMEL) by analysing commonly used external factors.
Computers in Human Behavior, 56, 238–256.
Achim, N., & Kassim, A. Al. (2015). Computer Usage: The Impact of Computer Anxiety
and Computer Self-efficacy. Procedia - Social and Behavioral Sciences, 172, 701–708.
https://doi.org/10.1016/j.sbspro.2015.01.422
184
Adetimirin, A. (2015). An empirical study of online discussion forums by library and

information science postgraduate students using technology acceptance model 3.
Journal of Information Technology Education: Research, 14(2015), 257–269.
https://doi.org/10.28945/2269
Ajmal, M., & Ahmad, S. (2019). Exploration of Anxiety Factors among Students of Distance
Learning : A Case Study of Allama Iqbal Open University. Bulletin of Education and
Research, 41(2), 67–78.
Al-Azawei, A., & Lundqvist, K. (2015). Learner differences in perceived satisfaction of an
online learning: An extension to the technology acceptance model in an arabic
sample. Electronic Journal of E-Learning, 13(5), 408–426.
Al Kurdi, B., Alshurideh, M., Salloum, S. A., Obeidat, Z. M., & Al-dweeri, R. M. (2020). An
Empirical Investigation into Examination of Factors Influencing University
Students’ Behavior towards Elearning Acceptance Using SEM Approach.
International Journal of Interactive Mobile Technologies (IJIM), 14(02), 19.
https://doi.org/10.3991/ijim.v14i02.11115
Amer, A.-A., Ahmad, A.-A., & Jo, S. (2013). Exploring Students Acceptance of E-learning
Using Technology Acceptance Model in Jordanian Universities. International
Journal of Education and Development Using Information and Communication
Technology, 9(2), 4–18. http://ijedict.dec.uwi.edu/viewarticle.php?id=1617
Ayeni, A. J., & Akinfolarin, C. A. (2014). Assessing Principals’ Coordinating and
Controlling Strategies for Effective Teaching and Quality Learning Outcome in
Secondary Schools in Ondo State , Nigeria. International Journal of Learning,
Teaching and Educational Research, 7(1), 180–200.
Chahine, I. C., & Khan, F. (2015). Improving the Quality of Learning Outcomes and
Enhancing the Performance of Education Systems : A Case Study in Kuwait.
International Journal of Learning, Teaching and Educational Research, 10(1), 1–21.
Chen, J. K. (2018). The influence of behavioural intention on third-party e-commerce
payment. South African Journal of Economic and Management Sciences, 21(1), 1–9.
https://doi.org/10.4102/sajems.v21i1.2157
Chin, W. W. (1998). Commentary: Issues and opinion on structural equation modeling.
MIS Quarterly, 22(1), 7–15.
Cigdem, H., & Ozturk, M. (2016). Factors affecting students’ behavioral intention to use
LMS at a Turkish post-secondary vocational school. International Review of Research
in Open and Distance Learning, 17(3), 276–295.
https://doi.org/10.19173/irrodl.v17i3.2253
Clair, D. S. (2015). A Simple Suggestion for Reducing First-time Online Student Anxiety.
MERLOT Journal of Online Learning and Teaching, 151(2005), 10–17.
https://doi.org/10.1145/3132847.3132886
Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989). User Acceptance of Computer
Technology: A Comparison of Two Theoretical Models. Management Science, 35(8),
982–1003.
Dumpit, D. Z., & Fernandez, C. J. (2017). Analysis of the use of social media in Higher
Education Institutions (HEIs) using the Technology Acceptance Model.
International Journal of Educational Technology in Higher Education, 14(1).
https://doi.org/10.1186/s41239-017-0045-2
Feriady, M., Nurkhin, A., Mahmud, N., Setiani, R., & Astuti, D. P. (2020). Influence of
organizational suport and digital literacy on lecturer acceptance of e-learning in
indonesia: A modification of technologi acceptance model. International Journal of
Scientific and Technology Research, 9(1), 2229–2233.
Gbongli, K., Xu, Y., & Amedjonekou, K. M. (2019). Extended technology acceptance model
to predict mobile-based money acceptance and sustainability: A multi-analytical
185
structural equation modeling and neural network approach. Sustainability

(Switzerland), 11(13), 1–33. https://doi.org/10.3390/su11133639
Hanif, A., Jamal, F. Q., & Ahmed, N. (2018). Behavioral Intention for Adopting Technology
Enhanced Learning Initiatives in Universities. Journal of Behavioural Sciences, 28(1),
1689–1699. https://doi.org/10.1017/CBO9781107415324.004
Hanum, N. S. (2013). The effectiveness of e-learning as a learning model (evaluation study
in SMK Telkom Sandhy Putra Purwokerto). Jurnal Pendidikan Vokasi, 3(1), 90–102.
https://doi.org/10.21831/jpv.v3i1.1584
Ho, K. F., Ho, C. H., & Chung, M. H. (2019). Theoretical integration of user satisfaction
and technology acceptance of the nursing process information system. PLoS ONE,
14(6), 1–14. https://doi.org/10.1371/journal.pone.0217622
Igbaria, M., & Iivari, J. (1995). The effects of self-efficacy on computer usage. Omega, 23(6),
587–605. https://doi.org/10.1016/0305-0483(95)00035-6
Igbaria, M., & Livari, J. (1995). The Effects of Self-efficacy on Computer Usage. OMEGA -
The International Journal of Management Science, 23(8), 587–605.
Kustono, A. S. (2020). How Total Quality Management Mediates Antecedent Variables of
Employee Performance? The Journal of Asian Finance, Economics and Business, 7(12),
523–534. https://doi.org/10.13106/jafeb.2020.vol7.no12.523
Kustono, A. S., & Nanggala, A. Y. A. (2020). Preparing quality financial statements at
health centers: to explore the role of regional inspectorates and professionalism of
accounting personnel. The Indonesian Accounting Review, 10(1), 1–12.
https://doi.org/10.14414/tiar.v10i1.1878
Kustono, A. S., Nanggala, A. Y. A., & Mas’ud, I. (2020). Determinants of the Use of E-
Wallet for Transaction Payment among College Students. Journal of Economics,
Business, & Accountancy Ventura, 23(1), 85–95.
https://doi.org/10.14414/jebav.v23i1.2245
Lai, P. (2017). the Literature Review of Technology Adoption Models and Theories for the
Novelty Technology. Journal of Information Systems and Technology Management,
14(1), 21–38. https://doi.org/10.4301/s1807-17752017000100002
Lin, M. H., Chen, H. C., & Liu, K. S. (2017). A study of the effects of digital learning on
learning motivation and learning outcome. Eurasia Journal of Mathematics, Science
and Technology Education, 13(7), 3553–3564.
https://doi.org/10.12973/eurasia.2017.00744a
Majid, F. (2012). A Study on the Online Language Learning Anxiety among Adult
Learners. International Journal of E-Education, e-Business, e-Management and e-
Learning, 2(3). https://doi.org/10.7763/ijeeee.2012.v2.106
Mohammadi, S., & Isanejad, O. (2018). Presentation of the extended technology acceptance
model in sports organizations. Annals of Applied Sport Science, 6(1), 75–86.
https://doi.org/10.32410/huj.10079
Muchlas. (2013). Development of Online Learning Models for Digital Engineering Practices in
Higher Education. Universitas Negeri Yogyakarta.
Nagy, J. T. (2018). Evaluation of online video usage and learning satisfaction: An extension
of the technology acceptance model. International Review of Research in Open and
Distance Learning, 19(1), 160–185. https://doi.org/10.19173/irrodl.v19i1.2886
Nambi, R. (2019). Adopting a TEDDIE learning design model to plan blended teaching
activities: Reflections from a teacher trainees’ poetry lecture. International Journal
of Learning, Teaching and Educational Research, 18(3), 164–185.
Nanggala, A. Y. A. (2020). Use of Fintech for Payment : Approach to Technology
Acceptance Model Modified. Journal of Contemporary Information Technology,
Management, and Accounting, 1(1), 1–8.
186
Nortvig, A. M., Petersen, A. K., & Balle, S. H. (2018). A literature review of the factors
influencing e-learning and blended learning in relation to learning outcome,
student satisfaction and engagement. Electronic Journal of E-Learning, 16(1), 45–55.
Oturakci, M., & Oturakci, M. (2018). Developing New Technology Acceptance Model With
Multi-Criteria Decision Technique: An Implementation Study. Engineering
Management Research, 7(2), 43. https://doi.org/10.5539/emr.v7n2p43
Pablico, J. (2017). Differentiated Instruction in the High School Science Classroom :
Qualitative Differentiated Instruction in the High School Science Classroom :
Qualitative and Quantitative Analyses. International Journal of Learning, Teaching
and Educational, 16(7), 30–54.
Pham, C. H., Vu, N. H., & Tran, G. T. H. (2020). The role of e-learning service quality and
e-trust on e-loyalty. Management Science Letters, 10(12), 2741–2750.
https://doi.org/10.5267/j.msl.2020.4.036
Rhoades, L., & Eisenberger, R. (2002). Perceived organizational support: A review of the
literature. In Journal of Applied Psychology (Vol. 87, Issue 4, pp. 698–714). American
Psychological Association. https://doi.org/10.1037/0021-9010.87.4.698
Siegel, D., Acharya, P., & Sivo, S. (2017). Extending the Technology Acceptance Model to
Improve Usage & Decrease Resistance toward a New Technology by Faculty in
Higher Education. Journal of Technology Studies, 43(2), 58–69.
https://doi.org/10.21061/jots.v43i2.a.1
Yuwana, A. M., & Kustono, A. S. (2017). Why is User "s Expertise Important for
Implementation of Regional Asset Information Systems: Case in Indonesia.
International Journal of Science and Research (IJSR), 6(7), 377–379.
https://doi.org/10.21275/art20175027
187
Appendix 1
188

Vol. 20, No. 3, pp. 188-204 March 2021
Challenges of Virtual Education during the

COVID-19 Pandemic: Experiences of Mexican
University Professors and Students
Claudia Patricia Contreras, David Picazo,

Aixchel Cordero-Hidalgo and Paola Margarita Chaparro-Medina
Autonomous University of Chihuahua, Chihuahua, México
https://orcid.org/0000-0003-4162-3055
https://orcid.org/0000-0003-0521-6954
https://orcid.org/0000-0001-9927-0972
https://orcid.org/0000-0002-7270-9903
Abstract. Students’ and university professors’ challenges due to the

COVID-19 worldwide pandemic have been enormous. Without prior
notice, they had to switch from traditional education to virtual teaching.
This study’s objective was to identify the perception of students and
professors of the Bachelor in the English Language from the University of
Chihuahua, Mexico, during the transition to virtual classes due to the
pandemic. Researchers applied a survey of 28 items with a Likert scale to
152 students during the spring semester 2020 and the second instrument
of 51 items to 26 professors. The students reported having complications
in connectivity (21%) and difficulties interacting with professors and
classmates (34%). Many students reported experiencing anxiety (69.1%)
and feeling isolated (62.5%). Most agree that they learn better in face-to-
face classes (85%). In turn, the professors agree that the students had a
lower performance than in a face-to-face semester (69.3%). Despite this,
the professors affirm that they followed the program to the letter (92.3%)
and completed the programmed contents (84.6%). Besides, 92.3% of the
academics are confident to face a new semester in virtual modality, if
necessary. Although we can say that the transition from face-to-face to
virtual classes in the institution was successful, students and professors
encountered difficulties and disadvantages in the use of platforms for
online classes. The benefit of this research´s findings is understanding the
University must have all the necessary elements in place for online
teaching. It must establish a formal regulation of virtual programs that
includes professor´s and student´s profile, as well as the strategies to be
followed. The students’ and professors’ perceptions highlight the crucial
elements that professors need to consider to teach virtual classes and take
advantage of these lessons to face the future better.
Keywords: COVID-19; Virtual Learning Environment; university;

education
©Authors
189
1. Introduction
In 2008 the United Nations Educational, Scientific and Cultural Organization
(UNESCO), decided to implement the Information and Communication
Technology (ICT) Competency Framework for Teachers having three versions:
2008, 2011, and 2018. It anticipates the predominating approach to the relationship
between technology and education and includes recommendations for
developing skills using current technologies.
The UNESCO, since the Qingdao Declaration (2015) affirmed it is crucial to

include the ICT in educational policies and strategies to transform the learning
process. It is necessary to rethink the professor's role and reform how the
educational systems prepare, considering the opportunities that present online
learning as a new pathway toward lifelong learning (UNESCO,
2015).Traditionally there was a list of functions and perceptions of the professor
that has been historically and socially constructed, as shown in Figure 1. Experts
should review this role, leading to a new pedagogical relationship because
students demand professors to participate more and more in teaching with
technologies. That is to say, to consider the technological dimension in the
teaching profession. Technologies have emerged as a challenge for the link
between professors and students, which causes a significant opportunity to
review roles, functions, and holistically understand formal education and their
professional performance (Casablancas, 2017).
Figure 1. The conception of the professor´s role

(García Contador & Gutiérrez Esteban, 2020)
Professors must adapt and train themselves to face the current times and contexts.
This adaptation involves being flexible, dynamic, moving away from the
traditional and tortuous for students, raising new forms of interaction and
scenarios that include technologies for educational purposes (Cabrero & Barroso,
2015). In recent months, the Coronavirus disease pandemic (COVID-19) caused
by the SARS-CoV-2 virus rocked the entire world. On March 11, 2020, the World
Health Organization (WHO) declared COVID-19 as a pandemic; by April 20, the
same organism reports that many countries had reacted by asking their
inhabitants to stay at home and restrict the movement of the population to reduce
the virus transmission (WHO, 2020). WHO reported the first two cases in Mexico
on February 29.
190
According to UNESCO, by April 1 of year 2020, 185 countries had to close higher
education institutions (HEI) activities, affecting 1,542,412,000 students,
representing 89.4% of the total (UNESCO, 2020b). According to the University of
Oxford, 70% of these countries finished the school semester with schools closed,
severely impacting education in general (Blavatnik, 2020). In Latin America and
the Caribbean, the temporary interruption of academic activities due to COVID-
19 affected 23.4 million university students and 1.4 million professors, more than
98% of students and teachers from the region (UNESCO, 2020b).
Countries, organizations, and educational institutions had to double their efforts

to continue distance education and reduce the impact of this school closure,
particularly among vulnerable and disadvantaged communities (UNESCO,
2020a). By resuming activities with virtual classes, and the challenges faced by
HEIs in pedagogical aspects and technical infrastructure to implement the new
way of teaching, universities worldwide have reported negative impacts on
research issues and students’ mobility (Marinoni et al., 2020).
Due to the pandemic, higher education institutions have moved into an emerging
situation of distance education, which will have many impacts, some have not yet
been documented or have not been perceived. There are three main reasons why
this effort of pedagogical continuity may not go well: the first is the technological
aspect since according to data from the International Telecommunications Union
in Latin America, only 52% of households have technological equipment and
broadband connectivity, the second is the distance education coverage of higher
education in 2017 only represented 15.3% of the total and covered 4.3 million
students, in the absence of planning has led students to think that education is
poor, and the third and last is related to teaching skills in distance education by
using scarce technological resources for teaching as if they were still in the
classroom (Pedró, 2020). In 2017 the Ibero-American Observatory of Science,
Technology, and Society (OCTS) reported that university students studying the
distance education modality represented 15.3% of the total enrollment, that is, 4.3
million students (OCTS, 2019). However, the modality was still incipient in Latin
America and was contrasting among the countries due to economic and social
inequalities and the existing digital divide in the region (UNESCO, 2020b).
Ibero-American counties show an upward trend as the enrollment of distance

students pursuing a first degree in higher education increased by 72.9% from 2010
to 2017 and only 27.3% in face-to-face education (OCTS, 2019). Even though a
large majority of the population in Mexico has access to the Internet and a mobile
device; there are considerable challenges in distance education without
considering that this modality represents a valuable tool to increase education
coverage in its population.
In research that used the UNESCO ICT Competency Framework for Teachers 2008
intending to evaluate the management of the ICTS for the application of the class
plan, a population of 922 professors of Mexican public universities whose
employed sample was 432 volunteers, gave the following results: most professors
use email and the Internet in an average of four times a week. The didactic use of
technologies is related to the different scientific universes (hard and soft sciences).
191
Professors who have ICT training possess a greater mastery of developing

teaching support materials. The more proficient they are in ICT, the higher their
score in an educational application, and the younger they are, the more proficient
they are in ICT (Vera Noriega et al., 2014).
For this reason, the abrupt change of modality did not have the best reception by
students and professors. The institutions did not design the contents as distance
courses before, which requires a more significant commitment and discipline
from the student. Something crucial was the disadvantages of some students due
to the lack of adequate technological resources to continue with their studies
successfully.
In this sense, the context of our country is not very favorable. According to the
National Council for the Evaluation of Social Development Policy (CONEVAL),
in 2018, the population living in poverty in Mexico was 41.9%, and 7.4% in
extreme poverty; 29.3% were in conditions of vulnerability due to social
deprivation, and only 21.9% of the population was not in conditions of poverty or
vulnerability (CONEVAL, 2019).
According to the National Institute of Statistics and Geography (INEGI), in

Mexico, there are 80.6 million Internet users aged six years or more (70.1%), 44.4%
have a computer, 56.4% of households have Internet access, whether by fixed or
mobile connection, 95.3% connect to a smartphone, 33.2% to a laptop and 28.9%
to a desktop. However, 91.5% use the Internet for entertainment, 90.7% for
information, 90.6% for communication, 83.8% for education/training 83.8%. The
three most important groups of Internet users are 18 to 24 years old (91.2%), the
second group is 12 to 17 years old (87.8%), and the third group is 25 to 34 years
old (86.9%) (INEGI, 2019).
In the rank called “Measuring the Information Society Report 2017” made by the
International Telecommunication Union (ITU), Mexico was in 87th place of the
world (ITU, 2017).
In this scenario, the considerable consequences of education in general and

university have been evident. Students’ and professors’ challenges worldwide
have been enormous (Almarzooq et al., 2020). Starting with the authorities’
declared health emergency on March 21, 2020, Mexican universities experienced
an unprecedented situation, a massive change from traditional "face-to-face"
teaching to an online education model. The health emergency caused by COVID-
19 pandemic revealed the deficiencies that many of Mexico’s universities had
concerning virtual modalities.
Virtual education requires specific skills in both students and professors; it is

essential to plan contents, activities, and the effective and efficient use of
technological tools in the virtual environment in which the teaching-learning
process will take place. As an immediate response to the situation, the institutions
had to decide which virtual learning environment (VLE) to use, while professors
have had to rethink and innovate their traditional teaching practice to adapt it to
192
new circumstances in virtual mode. Some had training and experience before the
pandemic, while others improvised to get ahead with the course contents.
It was imperative to anticipate integrating the ICTs into the curriculum as part of
educational change and innovation. It was also important to consider that
teacher's constant training is a variable that influences the incorporation of ICTs
into the teaching and learning process (Rangel Baca & Peñalosa Castro, 2013).
Given the need to continue with the January-June 2020 semester, the university
authorities of the Autonomous University of Chihuahua (UACH) gave the
instructions to continue with the development of the course content through the
use of educational platforms and the tools to establish contact with students.
Although the UACH already had extensive experience in virtual education, there
is no doubt that the challenge was enormous, given the need for 100 percent of
the professors to have the appropriate training to continue with their classes. As
in the whole country, in this university, within a short period, more than 29,000
students and 2,935 professors (UACH, 2019) found it necessary to continue their
courses at a distance, connected through the Internet, using available educational
platforms and other communication resources. In the case of the University of
Chihuahua, we chose Moodle because this platform is a learned-oriented (Al-
Ajlan & Zedan, 2008) resource. It allows the teacher to post new items such as
assignments, books, files, web pages, and YouTube videos, parallel to using
WebEx, Zoom, Google Classroom, Skype, or Facebook to hold video conferences
with the students. It is evident that the institution, like all universities in Mexico
and the world, needed to find practical and immediate technological solutions to
optimize the distance learning education process at an accelerated pace.
Therefore, it is unknown whether the abrupt change of modality has had a
significant adverse effect on the teaching-learning process.
It is also essential to asses professors and students as well as institutional response

capacity to know these actors´ experiences about their performance, preferences,
and difficulties they had to face during the semester. In the same way, it is
interesting to evaluate the emotional effects that arose as a consequence of the
days of uncertainty while all the university students were adapting to the new
situation, carrying out, unexpectedly, unplanned activities. The practical
implications of the knowledge that we can obtain from this study are related to
the administration and professors' strategies, especially considering the students'
feelings, the limited access to technological resources and connectivity problems.
The objective of this work is to understand the transition process from classroom
to online classes. It is evident everyone is trying their best to cope with the
challenges they faced in that context. However, a central question arises: What are
the perceptions of the transition to virtual classes among the University of
Chihuahua professors and students given the confinement during the pandemic
caused by COVID-19? To meet the objective of this research, an inductive
theoretical model is necessary, where the information from a sample of
individuals allows us to explain a phenomenon and infer the behavior of the
193
selected population. The survey, Likert scale, statistical analysis, and quantitative
measurements are the basis to obtain answers from the selected population.
2. Materials and methods

Researchers conducted this cross-sectional study during the three-month timeline
of April and June 2020. It is a cross-sectional design since its objective is to identify
the frequency of conditions in students and professors. The researchers did not
perform any intervention (interference), and the variables had only one
measurement in each participating individual (Rodríguez & Mendivelso, 2018).
This work follows a quantitative method research paradigm, with an exploratory,
projective and descriptive character, to understand the phenomena under study,
where human beings and diversity are involved (Pereira Pérez, 2011). The authors
also chose exploratory research because it approaches a little-studied topic such
as virtually in times of COVID-19 and the results may generate data and elements
to identify the context and subsequently conduct descriptive research that is a new
topic of study (Arias, 2012). One public university was the study population:
University of Chihuahua (UACH), located in Chihuahua, Mexico. This higher
education institution selected the B.A. in English Language students and
professors because of the accessibility of the institutional information necessary
to carry out this study. Our institution does not have an ethics committee but
privacy principles; we observed the subjects' confidentiality, anonymity, and
protection.
Drawing from data available through the Strategic System of Academic

Management (SEGA), we identified 230 students of the Bachelor in English
Language Program (undergraduate students) as the institution's population of
interest. The researchers used Cochran’s formula (see details below) to determine
the sample size (145 students), with a confidence interval of 95% and a 5% margin
error; even that 152 had participated, it was a non-probability, for convenience
sampling since the participation was voluntary (Mcmillan & Schumacher, 2005).
𝑁 ∗ 𝑍2 ∗ 𝑝 ∗ 𝑞
𝑛=
(𝑁 − 1) ∗ 𝑒 2 + 𝑍 2 ∗ 𝑝 ∗ 𝑞
Where:
n = sample size.
N = total number of elements that make up the population.
Z = reliability coefficient score.
e = margin of error.
p = proportion of elements exhibiting a certain characteristic.
q = proportion of elements that do not present the characteristic under research.
A total of 152 students answered an instrument (Appendix 1). It contained 28

statements they valued through a five-point Likert Scale that allowed students to
express how much they agreed or disagreed with the statements divided into
194
three categories: advantages, disadvantages, and modality preference of virtual

vs classroom classes, plus four variables to stratify the results.
After selecting the sample of students, we conducted a semi-structured survey

with them, and we used another instrument for the professors and grouped the
questions in six sections (training, teaching-learning process, institutional aspects,
infrastructure, limitations, virtual modality) in a 51-item mixed survey
questionnaire made up of Likert scale questions, yes-no questions, and a multiple-
choice question (Appendix 2). These instruments enabled the collection of
quantitative data that emerged from participants’ practices and experiences with
the issues related to COVID-19.
The data were collected using a Google Forms survey sent to the participants via
email and WhatsApp and analyzed with the statistical software IBM® SPSS
version 20 through descriptive statistical techniques such as frequency
distributions, central tendency measures and dispersion measures.
3. Results and Discussion

a) Students’ survey.
Out of the 152 Bachelor in the English Language students who participated in the
survey, 120 (79%) were women, and 32 (21%) were men; 86% (131) of the students
who answered the survey were in the city of Chihuahua, Mexico, 19 (13%) were
in other towns within the State of Chihuahua, while two (1.3%) were in the State
of Sonora, México.
According to the data shown in Table 1, we may think that students could
maintain good communication with their professors and classmates during the
semester in confinement since a large proportion answered to agree with this
statement (66.4 and 65.1%, respectively). However, the remaining 34% who could
not affirm the same draw attention, especially in a modality where constant
communication is essential to carry out the contents successfully (Table 1).
Table 1. Student’s perceptions of the advantages of taking virtual classes.

Strongly Somewhat Neither agree Somewhat Strongly
Question agree agree nor disagree disagree disagree SD
f % f % f % f % F %
My level of self-confidence has 6 3.9 27 17.8 58 38.2 36 23.7 25 16.4 18.9
increased from virtual classes
I maintain good communication 38 25.0 63 41.4 23 15.1 17 11.2 11 7.2 20.8
with my teachers in the virtual
modality
I maintain good communication 41 27.0 58 38.2 24 15.8 18 11.8 11 7.2 19.0
with my classmates in the
virtual modality
My ability to analyze 10 6.6 35 23.0 53 34.9 33 21.7 21 13.8 16.1
information has improved from
virtual classes
I manage my time efficiently in 22 14.5 28 18.4 39 25.7 29 19.1 34 22.4 6.4
the virtual modality
Online exams offer me more 26 17.1 43 28.3 37 24.3 19 12.5 27 17.8 9.5
advantages than the exams I
take in the classroom
195
The virtual learning process at 11 7.2 35 23.0 63 41.4 19 12.5 24 15.8 20.2
UACH is supported by state-of-
the-art platforms
Online classes allow me to relate 8 5.3 34 22.4 45 29.6 38 25.0 27 17.8 14.1
theory and practice
I have had access to the Internet 95 62.5 26 17.1 13 8.6 9 5.9 9 5.9 36.8
since classes started in the
virtual modality
I have had access to a computer 101 66.4 28 18.4 9 5.9 7 4.6 7 4.6 40.4
since classes started in the
virtual modality
I have more free time for non- 39 25.7 26 17.1 35 23.0 20 13.2 32 21.1 7.5
school related activities since I
began taking virtual classes
My expenses were reduced 61 40.1 49 32.2 28 18.4 6 3.9 8 5.3 24.4
when I started taking virtual
classes
My teachers are using better teaching 11 7.2 21 13.8 45 29.6 44 28.9 31 20.4 14.7
strategies in virtual education
compared to the strategies they use
during face-to-face classes
Total/Mean 469 23.7 473 23.9 472 23.8 295 14.9 267 13.5 19.2
About the students’ perception of the disadvantages of this modality (Table 2),
some of the relevant responses of the students relate to experiencing anxiety with
virtual classes (69.1%) and feeling isolated (62.5%) during the contingency.
Universities should pay attention to this situation since it could affect university
students’ emotional state (Dill et al., 2020), translating into low achievement and
dropout. These results are far above the study of González-Jaimes et al. (2020),
where 32% of 644 university students surveyed from seven states of the Mexican
Republic expressed the presence of symptoms of stress, and 40% anxiety. This
stress is explainable because of the difficulties they experienced through
interactions with professors (86.1%) and classmates (67.1%), as well as the
difficulty in organizing their online activities (62.5%) (Table 2).
Table 2. Student’s perceptions of the disadvantages of taking virtual classes.

f % f % f % f % f %
I don’t understand the 21 13.8 60 39.5 35 23.0 22 14.5 14 9.2 18.2
instructions in my virtual
classes
I experience anxiety taking 54 35.5 51 33.6 15 9.9 13 8.6 19 12.5 20.3
virtual classes
I feel isolated 53 34.9 42 27.6 23 15.1 10 6.6 24 15.8 17.0
The interaction with the 75 49.3 56 36.8 12 7.9 8 5.3 1 0.7 33.0
teacher is affected during the
class due to technical
problems
The interaction with my classmates 63 41.4 39 25.7 27 17.8 13 8.6 10 6.6 21.6
is complicated since it does not
happen spontaneously due to turn-
taking on the microphone
196
I have difficulty organizing 55 36.2 40 26.3 22 14.5 12 7.9 23 15.1 17.0

online homework
I don’t know how many 55 36.2 37 24.3 36 23.7 9 5.9 15 9.9 18.5
hours I should dedicate to the
virtual program
It is difficult for me to access 11 7.2 22 14.5 30 19.7 25 16.4 64 42.1 20.0
the Internet
I have had difficulties 20 13.2 36 23.7 31 20.4 37 24.3 28 18.4 6.9
accessing the contents of the
UACH platform
Total/Mean 407 30 383 28 231 17 149 11 198 14 19.2
We consulted the students on what modality they prefer. Students prefer face-to-
face classes over virtual classes. The vast majority (93%) considered that
interaction with the teacher is better face-to-face than virtually, 85% think that
they learn better in classroom classes, and 78% disagree with the statement that
they would like all their classes to be virtual (Figure 2). Undoubtedly, these results
reflect what Pedró (2020) reports on students' attitudes towards distance
programs, who seek to receive online classes as if they were face-to-face, only
adding the technological aspect.
Students' preferences: classroom vs. virtual classes
Strongly agree Somewhat agree Neither agree nor disagree Somewhat disagree Strongly disagree
90.0% 80.3%
80.0% 68.4%
70.0% 60.5%
60.0%
50.0%
37.5%
40.0% 28.3%
30.0% 20.4% 17.8%
16.4%
20.0% 9.2% 12.5% 12.5%
7.2% 6.6% 5.3% 5.3% 3.9%
10.0% 2.0% 3.9% 0.7% 1.3%
0.0%
I dislike taking virtual classes I learn better in classroom Interactions with teachers are I would like all my classes to
classes better face-to-face than in a be virtual
virtual way
Figure 2. Students’ preferences: classroom vs virtual classes.
Students living in different locations in the State capital had much more trouble
accessing the Internet for their classes. This result may also be a reflection of the
vulnerable situation in which many Mexicans live (CONEVAL, 2019), the lack of
connectivity and limited technological resources (INEGI, 2019), and information
literacy still deficient in a significant proportion of the population in Mexico (ITU,
2017).
Translating the Likert scale to numerical rating, where 1 means Strongly disagree,
2 Somewhat disagree, 3 Neither agree nor disagree, 4 Somewhat agree and, 5
Strongly agree, the mean to the question "I have had access to the Internet since
classes started in the virtual modality" results with an 𝑥̅ = 4.35 for the students
living in the city of Chihuahua and 𝑥̅ = 3.57 for the students of other locations.
The difference in means was significant with the t-test for independent samples
(p < 0.05). Other questions that showed statistically significant differences (p <
197
0.05) for these two groups of students mentioned were: "Online classes offer more
advantages compared to classroom learning". Students from other locations
(usually rural) feel that they have more advantages with this type of online class
modality; however, these same students reported having more difficulty
accessing the Internet (𝑥̅ = 3.05) vs. the students of the state capital (𝑥̅ = 2.16) (See
Table 3).
Table 3. Comparison of significant differences in means of student location.

Chihuahua Other locations of the State
Question Mean N Standard Mean N Standard
deviation deviation
The interaction with my classmates 3.75 131 1.26 4.62 21 0.67
is complicated since it does not
happen spontaneously due to turn-
taking on the microphone
Online classes offer more 2.29 131 1.21 3.10 21 1.04

advantages compared to classroom
learning
I have difficulty organizing online 3.51 131 1.48 4.19 21 0.93

homework
It is difficult for me to access the 2.16 131 1.27 3.05 21 1.50

Internet
By grouping the variables that corresponded to the Advantages and

Disadvantages of virtual education, there is a significant moderate positive
correlation (p < 0.05) between the students who said they prefer face-to-face
classes and the Disadvantages category, with a Pearson correlation coefficient of
0.366.
b) Professors’ survey.
One hundred percent of the professors of the Bachelor of English Language
program answered the survey. Of the 26 professors, 19 (73%) are women, with an
average age of 48.3 years, and 7 (17%) are men, with an average of 43.9 years. Of
the 26 professors surveyed, 16 (62%) are full-tenured, and 10 (38%) are non-
tenured professors; two (7.7%) have a bachelor’s degree, 13 (50%) have a master’s
degree, and 11 (42.3%) have a doctorate. The majority of professors (73%) teach
only in undergraduate programs, while 27%, in addition to undergraduate
programs, attend masters and doctorate courses. Regarding seniority in general,
the professors have 15.1 years of teaching experience, the average seniority of
female professors is 14.8 years, and 16.0 years of the male professors.
Due to the pandemic, the contingency has been an unexpected situation in all
areas of our lives. In the academic aspect, in higher education, professors
suddenly went from classrooms to remote sessions supported by technology and
computer tools to teach classes. Suddenly, we faced a complicated situation that
required professors to have training in digital and communication tools. For
example, many in this group of professors trained previously, but some did not
prepare. This lack of training can be understood because, before this situation,
virtual education in this institution was only an alternative; now, it has become
an obligation. Regarding the experience using virtual learning platforms, 19 (73%)
198
of the professors indicated that they had previously used these tools, while 7
(27%) stated that they had no previous experience. Fourteen of them (53.8%) took
training previously, 9 (34.6%) got training courses during the contingency, and 3
(11.5%) of them mentioned that they had not received any training yet. Three
professors (11.5%) considered that they had a low level of training to teach in
virtual environments; 17 (65.4%) answered that they had an intermediate level,
and 6 (23.1%) stated that they had a high level of training in these tools. Regarding
the institutional support received, this study coincides with that of Sánchez et al.
(2020), who surveyed 788 professors from UNAM, where 60.5% acknowledged
having had institutional support to face the challenges of the pandemic.
In the present study, regarding the university’s training, 50% affirmed that this
training gave them sufficient elements to teach in virtual environments, 19.2%
considered it did not give them sufficient elements to teach, and 30.8% answered
that it partially did. As mentioned before, we agree with other researchers that,
during the contingency, university-level professors’ proficiency of digital skills
takes particular relevance to move forward with the course content (Martínez-
Garcés & Garcés-Fuenmayor, 2020). About the tools that the professors used to
teach during the semester, 17 (65.4%) used Virtual Classrooms, 21 (80.8%) used
social networks, 25 (96.2%) also supported themselves with video conferences,
and 17 (65.4%) used email additionally. Most of the professors (81.8%) used the
Moodle platform for their courses, while 18.2% preferred Google Classroom, and
only one of the professors used Schoology during this contingency.
Although the university was flexible regarding the professors’ tools, Moodle is
the platform that the institution used for the courses this semester. These results
are consistent with those of Sánchez et al. (2020), where most professors trusted
the institutional platform to carry out their courses. Regarding the tools for video
conferences, the professors combined the virtual platforms; most of the professors
(96.2%) used Zoom to teach their classes, 57.7% used WebEx, which is the tool that
the institution provides to professors, 53.8% connected via Google Meet, while
23% used other tools, like Skype, WhatsApp, and Facebook to establish
communication with the students. Although the UNAM professors used almost
the same resources as those of our institution, at the UACH, Zoom’s use was
preferred for distance sessions, while people at the UNAM preferred Google
Hangouts (Sánchez et al., 2020). It should be noted that the UACH reported
frequent failures when using the WebEx tool, which was the recommended
platform by the institution to carry out the sessions.
Table 4. Students’ overall performance during the contingency.

Seriously
Excellent Good Regular Poor
Question deficient SD
f % f % f % f % f %
Motivation during class 3 11.5 12 46.2 10 38.5 1 3.8 0 0.0 5.4
activities
Class attendance 5 19.2 13 50.0 6 23.1 2 7.7 0 0.0 5.0
Task and work fulfillment 8 30.8 14 53.8 4 15.4 0 0.0 0 0.0 5.9
Extra time and attention for 6 23.1 13 50.0 5 19.2 2 7.7 0 0.0 5.0
reviews and study
199
Browsing of materials and 3 11.5 14 53.8 7 26.9 1 3.8 1 3.8 5.5

bibliography
Use of information technologies 7 26.9 15 57.7 3 11.5 1 3.8 0 0.0 6.1
Optimal use of class time 3 11.5 18 69.2 4 15.4 1 3.8 0 0.0 7.3
Total/Mean 35 19 99 54 39 21 8 4.4 1 0.5 5.7
In general, the professors stated that the performance of the students during the
contingency was good. It stands out that 84.6% think that students have an
excellent or a good level using information technologies. The same percentage of
professors consider performance in terms of fulfilling tasks and works between
good and excellent. The results are consistent with an experimental study carried
out with 458 university students in Spain, where they found that the students
improved their performance and ability to develop tasks during confinement
(Gonzalez et al., 2020). In that study, the best performance related to the
assessment processes and the new learning methodology; in the present study,
the professors rated the students’ good level when using information technologies
(Table 4). The aspect with the highest negative evaluation turned out to be that of
motivation during class activities; here, 42.3% considered the performance of the
students to be between fair and poor (Table 4), which is understandable, and it
coincides with the argument of Dill et al. (2020) since the same students reported
having experienced negative emotions such as feeling isolated and experiencing
anxiety during the confinement (Table 2).
Table 5. Teaching-Learning Process.

f % f % f % f % f %
The interaction with students during 8 30.8 16 61.5 1 3.8 1 3.8 0 0.0 6.8
the semester as appropriate
I had to develop new teaching 14 53.8 10 38.5 1 3.8 1 3.8 0 0.0 6.4
materials for my class
I had to locate and adapt teaching 9 34.6 9 34.6 5 19.2 1 3.8 2 7.7 3.8
materials from other authors
I followed to the letter the program 9 34.6 15 57.7 0 0.0 2 7.7 0 0.0 6.6
of the subjects taught
I was able to cover the contents of 9 34.6 13 50.0 0 0.0 4 15.4 0 0.0 5.7
the subjects taught entirely
The students’ academic performance 6 23.1 12 46.2 4 15.4 1 3.8 3 11.5 4.2
during this semester was lower in
comparison to last semester
Total/Mean 55 35.3 75 48.1 11 7.03 10 6.38 5 3.2 5.6
Regarding the teaching-learning process, 92.3% of the professors had to develop

new materials for their classes, and 69.2% said they needed to adapt teaching
materials from other authors. Even though the situation changed in the middle of
the semester, 92.3% of the professors affirmed that they followed the program to
the letter, while 84.6% said they had completely covered the subjects’ contents.
Even though these results were encouraging for the teaching-learning process, it
is striking that a large proportion of professors (69.3%) think that the students’
performance was lower compared to the last semester, where students only had a
200
face-to-face modality. We may think that it is a contradictory finding, but it is not,

since there was a previous discussion about the perception of the student’s
performance, which professors rated as good, but we understand that professors
thought at the same time that it was not as good as in former periods, where they
only had face-to-face classes. An objective way to measure performance would be
comparing grades between semesters, as Gonzalez et al. (2020) did. However, this
objective was outside the scope of this research.
Regarding the experience in virtual mode, the professors are positive, since a large
proportion of them (76.9%) consider that the virtual classes have helped them to
improve their teaching practice; 92.3% stated that the communication with the
students was easy and the same proportion (92.3%), said they feel prepared to
carry out another semester virtually if necessary (Figure 3). In general, professors
are optimistic about taking on the challenge of virtual classes, as Prokopenko &
Berezhna (2020) reported, whose professors agreed that this situation also
motivated them to improve their creativity and skills for a distance teaching-
learning process. Despite this, only 15.4% prefer virtual classes over classroom
classes, and half of them do not agree that students learn better in the virtual
model, which is consistent with the vision of students who do not like virtual
classes because they consider that learning and interaction are better in face-to-
face courses (Figures 2 and 3).
Teachers’ survey regarding the virtual modality
90.0%
76.9%
80.0%
70.0%
60.0% 53.8%
50.0%
50.0% 42.3%
38.5%
40.0%
30.8% 30.8% 30.8%
30.0% 23.1% 23.1%
19.2%
20.0% 15.4% 15.4%
11.5%
7.7% 7.7% 7.7%7.7%
10.0% 3.8% 3.8%
0.0% 0.0% 0.0% 0.0% 0.0%
0.0%
Communication with Virtual lessons have I prefer to teach virtually I consider that students I am prepared to carry
students was easy helped me improve my more than on-site learn better virtually out another virtual
teaching semester if necessary
Strongly agree Somewhat agree Neither agree nor disagree Somewhat disagree Strongly disagree
Figure 3. Professors’ survey regarding the virtual modality.
When analyzing the institutional aspects, 80.8% of the professors answered that
they received support from the institution to communicate with their students;
92.3% received timely communication from the academic area, and 84.6%
answered that the administrative area’s communication was equally timely.
While 84.6% considered that the institution’s contingency face strategies were
appropriate, only 38.5% think that the university had a contingency plan
previously. At a global level, although the COVID-19 pandemic had an immediate
impact on the academic activities of universities, a study with 406 universities
from 109 countries reveals that 91% considered that they were prepared with the
201
necessary infrastructure to maintain good communication with their students and

professors (Marinoni et al., 2020).
Although the challenge of achieving effective communication with students was

great initially, this variable did not represent a significant problem for this
university community since the UACH has come a long way in virtual teaching
and experiences with blended learning schemes. Finally, in this university, 80.8%
of the professors think that the institution is prepared to take another virtual
semester if necessary.
Table 6. Institutional aspects.

f % f % f % f % f %
I received the institution’s 12 46.2 9 34.6 3 11.5 1 3.8 1 3.8 5.0
support to communicate with
my students
I received timely 14 53.8 10 38.5 1 3.8 0 0.0 1 3.8 6.4
communication from the
institution’s academic area
I received timely 11 42.3 11 42.3 1 3.8 2 7.7 1 3.8 5.3
communication from the
institution’s administrative area
I think next semester’s 7 26.9 11 42.3 6 23.1 2 7.7 0 0.0 4.3
enrollment will be lower due to
the circumstances
The institution’s strategies 9 34.6 13 50 2 7.7 1 3.8 1 3.8 5.5
implemented in response to the
contingency were appropriate
I believe the institution already 4 15.4 6 23.1 6 23.1 2 7.7 8 30.8 2.3
had a formal contingency plan
The institution is prepared to 11 42.3 10 38.5 4 15.4 0 0.0 1 3.8 5.1
carry out another virtual
semester if necessary
Total/Mean 68 37.4 70 38.5 23 12.6 8 4.39 13 7.11 4.8
4. Conclusions
We can conclude that most of B.A. in the English Language students’ perceptions
are related to the disadvantages in virtual classes, especially on the problems of
accessing the Internet, difficulties interacting with professors and classmates, and
not understanding clearly all the assignment instructions. During this experience,
the professors used Moodle as an institutional platform. Regarding
communication tools, the vast majority of professors relied on Zoom to conduct
conferences with their students, and many of them combined social media to
maintain contact. Despite the suddenness of the situation, we can affirm that the
transition from face-to-face teaching to the virtual modality was successful,
professors had the institutional support in technological resources, training, and
communication to conduct the courses, and they had the opportunity to conclude
the topics of their programs. However, a large proportion thinks that the
performance of the students was lower than in face-to-face modality. Professors
base this idea on students not using class time optimally, less participation took
202
place, and they were not achieving excellent task and work fulfillment. The
professors feel confident to face a new semester in virtual modality if necessary.
In general, professors agreed that this situation also motivated them to improve
their creativity and skills for a distance teaching-learning process. Even so, the
perception of the students concerning this modality is noteworthy, since the
majority affirmed that they prefer face-to-face classes to the virtual modality; this
is possibly a reflection of the difficulties that students experienced when
interacting with professors and classmates, as well as the difficulty in organizing
their online activities and connectivity problems. The recommendations derived
from this research are in three levels: professors, the education system, and the
government. Professors should consider their students' context and limitations to
make adjustments to their programs to successfully orient education towards
critical thinking. The education system must constantly train professors by
offering free online courses and lectures from experts in educational platforms
and technological tools to enhance their skills. The government must support the
education system by issuing laws to formalize virtual education, giving schools
the necessary tools and resources to meet their needs.
This research reflects upon experiences of professors and students during the
pandemic and the perception of its advantages and disadvantages; however, the
limit is the resistance to migrate to online modality, coupled with the fact that
sometimes there is a disconnection between the teacher and the educational
authorities who often belong to the administrative area and face this challenge
from another angle. The experiences from the pandemic of 2020 will lead to the
development of new platforms and regulations that will allow the education
system to be ready for the challenges to come.
5. References
Al-Ajlan, A., & Zedan, H. (2008). Why Moodle. Proceedings of the 2008 12th IEEE
International Workshop on Future Trends of Distributed Computing Systems, 58–64.
https://doi.org/10.1109/FTDCS.2008.22
Almarzooq, Z., Lopes, M., & Kochar, A. (2020). Virtual Learning During the COVID-19
Pandemic. Journal of the American College of Cardiology, 75(20), 2635–2638.
https://doi.org/10.1016/j.jacc.2020.04.015
Arias, F. G. (2012). El proyecto de Investigación [The research project] (6 Ed.). Editorial
Episteme.
Blavatnik. (2020). OXFORD COVID-19 Government Response Tracker(OxCGRT).
https://data.humdata.org/dataset/oxford-covid-19-government-response-
tracker
Cabrero, J., & Barroso, J. (2015). Nuevos retos en tecnología educativa [New challenges in
educational technology] (1st ed.). Editorial Síntesis.
Casablancas, S. (2017). No es malo perder el rumbo: reconfiguraciones del rol docente en
el contexto digital [It is not bad to lose the way: recofigurations of the teaching
role in the digital context ]. En Educar en la era digital (pp. 17–33). Editorial Pandora,
S.A. de C.V. http://www.pent.org.ar/extras/micrositios/libro-
educar/educar_en_la_era_digital.pdf
CONEVAL. (2019). Medición de pobreza 2008-2018, Estados Unidos Mexicanos[Poverty
Measurement 2008-2018, United Mexican States].
https://www.coneval.org.mx/Medicion/PublishingImages/Pobreza_2018/Seri
203
e_2008-2018.jpg
Dill, E., Fischer, K., McMurtrie, B., & Beckie, S. (2020). As Coronavirus Spreads, Is the First
Step. What’s Next? The Chronicle of Higher Education, 1, 5–7.
García Contador, Y., & Gutiérrez Esteban, P. (2020). El rol docente en la sociedad digital.
[The teacher´s role in the digital era]. Digital Education, 38, 1–22.
https://www.redalyc.org/pdf/274/27447325008.pdf
González-Jaimes, N. L., Tejeda-Alcántara, A. A., Espinosa-Méndez, C. M., & Ontiveros-
Hernández, Z. O. (2020). Impacto psicológico en estudiantes universitarios
mexicanos por confinamiento durante la pandemia por Covid-19. [Psychological
impact on Mexican university students due to confinement during the Covid-19
pandemic]. Preprint, 644(1), 17. https://doi.org/10.1590/SCIELOPREPRINTS.756
Gonzalez, T., de la Rubia, M. A., Hincz, K. P., Comas-Lopez, M., Subirats, L., Fort, S., &
Sacha, G. M. (2020). Influence of COVID-19 confinement in students performance in
higher education. 1–25. https://doi.org/10.35542/osf.io/9zuac
INEGI. (2019). Medición de pobreza 2008-2018, Estados Unidos Mexicanos. [Poverty
Measurement 2008-2018, United Mexican States].
https://www.coneval.org.mx/Medicion/PublishingImages/Pobreza_2018/Seri
e_2008-2018.jpg
ITU. (2017). ICT Develepment Index 2017. Measuring the Information Society Report 2017.
https://www.itu.int/net4/ITU-D/idi/2017/index.html#idi2017economycard-
tab&MEX
Marinoni, G., Land, H. Van, & Jensen, T. (2020). The impact of Covid-19 on higher education
around the world. IAU Global Survey Report (International Association of
Universities (Ed.)). https://www.iau-
aiu.net/IMG/pdf/iau_covid19_and_he_survey_report_final_may_2020.pdf
Martínez-Garcés, J., & Garcés-Fuenmayor, J. (2020). Competencias digitales docentes y el
reto de la educación virtual derivado de la covid-19. [Digital teaching
competencies and the challenge of virtual education derived from covid-191].
Educación y Humanismo, 22(39), 1–16.
https://doi.org/10.17081/eduhum.22.39.4114
Mcmillan, J., & Schumacher, S. (2005). Investigación Educativa. [Educational Research].
At Investigación educativa. Pearson Addison Wesley. https://des-
for.infd.edu.ar/sitio/upload/McMillan_J._H.__Schumacher_S._2005._Investigac
ion_educativa_5_ed..pdf
OCTS. (2019). Informe de coyuntura No 06: Se expande la educación superior a distancia en
Iberoamérica, aunque es todavía incipiente. [Conjuncture Report No 06: Distance
higher education is expanding in Ibero-America, although it is still incipient].
http://octs-oei.org/coyuntura/coyuntura06.pdf
Pedró, F. (2020). Covid-19 y educación superior en América Latina y El Caribe: Efectos,
impactos y recomendaciones políticas. [Covid-19 and higher education in Latin
America and the Caribbean: Effects, impacts and policy recommendations].
Análisis Carolina, 36, 1–15.
https://doi.org/https://doi.org/10.33960/AC_36.2020 La
Pereira Pérez, Z. (2011). Los diseños de método mixto en la investigación en educación:
Una experiencia concreta. [Mixed method designs in educational research: A
concrete experience]. Revista Electrónica Educare, 15(1), 15–29.
http://www.revistas.una.ac.cr/index.php/EDUCARE/article/view/867
Prokopenko, I., & Berezhna, S. (2020). Higher Education Institutions in Ukraine during the
Coronavirus, or COVID-19, Outbreak: New Challenges vs New Opportunities.
Revista Romaneasca pentru Educatie Multidimensionala, 12(1Sup2), 130–135.
https://doi.org/10.18662/rrem/12.1sup2/256
204
Rangel Baca, A., & Peñalosa Castro, E. A. (2013). Alfabetización digital en docentes de
Educación Superior: construcción y prueba empírica de un instrumento de
evaluación. [Digital literacy in Higher Education professors: construction and
empirical testing of an assessment instrument]. Pixel-Bit Revista de Medios y
Educación, 43, 9–23. https://doi.org/10.12795/pixelbit.2013.i43.01
Rodríguez, M., & Mendivelso, F. (2018). Diseño de investigación de Corte Transversal.
[Cross-sectional research design]. Revista Médica Sanitas, 21(3), 141–146.
https://doi.org/10.26852/01234250.20
Sánchez, M., Martínez, A., Torres, R., de Agüero, M., Hernández, A., Benavides, M.,
Rendón, V., & Jaimes, C. (2020). Educational challenges during the covid-19
pandemic: a teachers’ survey at UNAM. Revista Digital Universitaria, 21(3), 1–24.
https://www.revista.unam.mx/wp-content/uploads/a12.pdf
UACH. (2019). Estadística Básica 2019. [Basic Statistics 2019].
https://uach.mx/assets/media/publications/2017/10/143_agenda-
estadistica/Estadistica-2019-2020.pdf
UNESCO. (2015). Quindgao Declaration. (2015). Seize digital opportunities, lead education
transformation. https://unesdoc.unesco.org/ark:/48223/pf0000233352
UNESCO. (2020a). COVID-19 Impact on Education. Education: From disruption to recovery.
https://en.unesco.org/covid19/educationresponse
UNESCO. (2020b). COVID-19 y educación superior: De los efectos inmediatos al día
después. [COVID-19 and higher education: From immediate effects to the day
after]. At Unesco. http://www.iesalc.unesco.org/wp-
content/uploads/2020/04/COVID-19-060420-ES-2.pdf
Vera Noriega, J. A., Torres Moran, L. E., & Martínez García, E. E. (2014). Evaluación de
competencias básicas en TIC en docentes de educación superior en México.
[Evaluation of basic ICT competencies in higher education teachers in
Mexico.]Pixel-Bit. Revista de Medios y Educación, 44, 143–155.
http://www.redalyc.org/articulo.oa?id=36829340010
WHO. (2020). WHO Director-General’s opening remarks at the media briefing on COVID-19 - 1
April 2020 (p. 1). https://www.who.int/dg/speeches/detail/who-director-
general-s-opening-remarks-at-the-media-briefing-on-covid-19---1-april-2020
205

Vol. 20, No. 3, pp. 205-228, March 2021
Advancing Preservice Physics Teachers’ Critical

Thinking through Active and Passive
Microteaching Lesson Study
Billy A. Danday
Leyte Normal University, Tacloban City, Philippines
https://orcid.org/0000-0002-9771-5912
Abstract. A pretest-posttest comparison-group quasi-experimental study

was endeavoured to unravel the effects of the two forms of Microteaching
Lesson Study (MLS), the Active MLS and the Passive MLS, on the critical
thinking of aspiring physics teachers. Eighteen Bachelor of Secondary
Education specializing in Physical Science students participated in the
six-week study. Data were gathered using the Critical Thinking Inventory
in Physics and were analysed utilizing the Wilcoxon Signed Ranks Test,
Mann-Whitney U Test, and descriptive statistics. Results revealed that
both the Active and Passive MLS have positive effects on the overall
critical thinking and on all of the critical thinking sub-skills of the
preservice teachers. Results further showed that the Active MLS is
significantly more effective than the Passive MLS in developing overall
critical thinking and its sub-skills, specifically, inference and
interpretation. The implementation of lessons by the Active MLS group
in microteaching sessions indicated positive affordances on the
development of critical thinking. Further studies involving a greater
number of preservice teachers specializing in different fields of science
are recommended. The integration of the MLS, especially the Active MLS,
in the preservice teaching curricula is deemed a worthwhile engagement.
Keywords: critical thinking; lesson study (LS); microteaching; physics

teaching; preservice teachers
1. Introduction
The world is currently facing insurmountable challenges – climate change,
environmental degradation, political conflicts, depletion of resources, and
pandemics, among others. Humans around the globe are constantly challenged to
act efficiently and effectively for sustainable existence. These, apparently, require
critical thinking considering that effective actions demand critical scrutiny of
problems at hand (Buckley, 2012; Sensibaugh, 2015; Tseng, 2008; Williams, 2005).
Educational institutions are commissioned to proliferate learning; hence,
curricular programmes need to inculcate critical thinking among learners, be it in
the sciences or the arts, if sustainable existence is aspired (Nilson et al., 2013).
©Authors
206
Teaching critical thinking is supposed to emanate from teacher education

institutions. Preservice teachers’ primary interest should be to possess good
critical thinking dispositions and to acquire critical thinking skills for continued
proliferation of analytical and systematic reasoning and problem-solving (Arsal,
2015). Studies, nevertheless, disclose a dismal level of critical thinking among
preservice teachers (Akdere, 2012; Grosser & Nel, 2013; Qing et al., 2010).
Aspiring teachers often hold an inadequate perception of critical thinking and its
fundamental skills, particularly regarding its application in classroom activities
(Gashan, 2015; Meister, 2011). They even were unclear about the associations
between learning objectives and assessment of teaching outcomes (Cavanagh et
al., 2019). This alarming deficiency requires immediate and profound
interventions; hence, scholars launched various initiatives to address this deficit.
However, varied strategies did not generate consistent positive outcomes (Akyuz
& Samsa, 2009; Behar-Horenstein & Niu, 2011; Goyak, 2009; Qing et al., 2010;
Sulaiman, 2013). Therefore, further exploration of didactic approaches that
support the development of critical thinking in future teachers is urgently
required.
The conventional, individualistic instructional approach, which is extensively

employed in preservice teacher education curricula, particularly in the practicum
stage, seems to lack opportunities to scaffold critical thinking skills. Perspectives
from collaborative instructional approaches, like the Lesson Study, are worth
pursuing, due to the various interactions that the instructional approach demands
from participants (Chen & Zhang, 2019; Chew & Lim, 2013; Elipane, 2012). A
microteaching-based Lesson Study framework can be especially engaging,
because microteaching does not only bring the prospective teachers to an actual
classroom setting but it also strengthens their professional competence (Akkus &
Uner, 2017; Remesh, 2013; Sentumbwe, 2018, Zhou & Xu, 2017).
While a few studies have accounted for the effects of Microteaching Lesson Study
on some aspects of student learning, little or nothing is known about the effects of
the active and passive versions of Microteaching Lesson Study on preservice
teachers’ critical thinking. This study has been designed to explore salient
influences of two forms of this instructional approach on overall critical thinking
and its specific components in the domain of physics instruction. Specifically, the
study was aimed at answering the following questions:
(1) Do the active MLS and passive MLS approaches render significantly higher
critical thinking scores in the post-test than in the pre-test?
(2) Is there a significant difference between the critical thinking scores of the
active MLS group and passive MLS group after the intervention?
(3) What are the benefits of active and passive MLS in the development of critical
thinking?
207
2. Related Literature
2.1. Lesson Study
Japan has long been on the upper stratum and continues to be one of the leading
countries in terms of performance in the Trends in Mathematics and Science Study
(TIMSS), alongside with Singapore and other Asian countries. The Japanese
educational system is undeniably one of the better systems in the world that is
worthy of emulation. It is not surprising that one of their most effective
professional development programmes, the jugyokenkyu or the lesson study
(Figure 1), is extensively adopted by educational institutions around the world
(Buchard & Martin, 2017; Cerbin & Kopp, 2006; Chen & Zhang, 2019; Fernandez,
2002; Lewis et al., 2009; Zhou & Xu, 2017).
setting lesson goals
planning the lesson
designing the study
implementing the lesson plan
discussing the implementation
revising the lesson plan
implementing the revised

lesson plan
discussing the final

implementation
documenting and sharing

results
Figure 1: The Lesson Study Process
“Jugyo” is the Japanese term for lesson while “kenkyu” refers to study, hence,
Lesson Study is the literal English translation of the two Japanese words
208
(Fernandez, 2002). Lesson Study is an in-service professional development

approach for teachers that is distinguished due to its collaborative nature. It has
fascinated countless educators of various disciplines, particularly those who have
been oriented according to the individualistic instructional approach.
The Lesson Study (LS) process possesses the following features: investigation,
planning, research lesson, and reflection (Lewis et al., 2009). An extensive
examination of the various adaptations of the Lesson Study shows the following
procedure: (1) setting lesson goals, (2) planning the lesson, (3) devising the study,
(4) executing the lesson plan, (5) deliberating on the implementation, (6)
modifying the lesson plan, (7) executing the revised lesson plan, (8) deliberating
on the implementation of the new version of the lesson plan, and (9) documenting
and sharing results (Cavin, 2007; Cerbin & Kopp, 2006; Chew & Lim, 2013;
Fernandez, 2002). Figure 1 illustrates the complete procedure of the Lesson Study
model as implemented by diverse scholars.
It must be noted that the Lesson Study members are three or more teachers
working together throughout the whole process. External experts are often invited
to observe the lesson implementations and take part in the analyses of the data
gathered (Lewis et al., 2006). Meanwhile, Cerbin and Kopp (2006) point out that
lesson studies are intended to examine how students learn the lessons, not to
assess what students have learned. The analysis of the interactions among
students, teachers, and instructional aids is the heart of the Lesson Study
framework.
Research has revealed various benefits of the Lesson Study to in-service teachers,
preservice teachers, and students. This professional development approach can
enhance teachers’ content knowledge, pedagogical skills, and didactic beliefs
(Danday, 2019; Lewis et al., 2009). Moreover, LS fosters group ownership, collegial
support, instructional expertise, teaching confidence, and a better understanding
of students’ thinking (Barrett et al., 2013; Fernandez, 2002; Kolenda, 2007; Lewis,
2002).
Similar findings have been reported on the integration of Lesson Study into
preservice teacher education. Elipane (2012) and Gurl (2009) reported that LS has
the potential of improving teaching skills, habits, and competencies. Furthermore,
engaging in the Lesson Study can broaden the Nature of Science (NOS)
pedagogical content knowledge of preservice teachers and can successfully
transform it into classroom praxis (McDowell, 2010).
Lesson Study also has shown positive influences on the learners. Barrett et al.
(2013) reported that the students whose teachers had been in the Lesson Study
groups performed significantly better in geometry than the students whose
teachers had been in the traditional setting. Similarly, Lucenario et al. (2016)
reported that a Pedagogical Content Knowledge-Guided Lesson Study (PCKLS)
for chemistry teachers improved students’ problem-solving skills. Teele et al.
(2015) also noted that exposure to Lesson Study-based instruction had a positive
impact on students’ achievement in mathematics.
209
Integrating technology in the Lesson Study framework has offered more

interesting outcomes. Chew and Lim (2013) found that employing Lesson Study
(LS) with the use of Geometer’s Sketchpad (GSP) in teaching mathematics could
enhance the technological pedagogical content knowledge of preservice teachers.
According to Cavin (2007) a Microteaching Lesson Study (MLS) can generate a
positive learning atmosphere, ensuing in the appreciation of technology
integration into learner-centred instruction.
Apart from classroom instruction, lesson study also may serve as an effective
vehicle for curriculum design, implementation, and assessment. For instance,
Gutierez (2015) utilized the Lesson Study model to identify the issues associated
with the implementation of inquiry-based instruction in elementary science in the
Philippines. She emphasised three key issues: (1) scarcity of inquiry-based
resources and lack of professional support; (2) emphasis on content learning,
rather than inquiry-learning; and (3) the time-consuming and laborious nature of
inquiry-based pedagogies.
From the above discussion it is clear that literature provides a satisfactory

elucidation of the Lesson Study framework. However, a gap still exists in
knowledge about characterizing active and passive forms of microteaching lesson
study, and the corresponding effects on preservice teachers’ critical thinking skills
and sub-skills. The author, hence, deemed it imperative to investigate these
domains to cast broader light on this instructional approach in the field of physics
education.
2.2. Critical Thinking

One of the primary goals of global education is for learners to attain critical
thinking skills. The 21st century learning framework establishes the skills that
modern-day learners should possess. Education experts agree that learners
should acquire life and career skills, learning and innovation skills, and
information, media, and technology skills from all corners of educational
institutions. In the learning and innovation skills domain, critical thinking is a
priority goal, along with communication, collaboration, and creativity (Battelle for
Kids, 2019). The curricula for science and all other subject areas, therefore, are
aimed towards the attainment of these goals.
Edward Glaser (1941) identified three facets of critical thinking - the attitude of
insightful thinking, the knowledge of systematic reasoning and analytical
methods, and the skill to employ such methods. The American Philosophical
Association (APA) Delphi Consensus expresses the contention of the critical
thinking experts on the definition of the term. They assert that critical thinking is
“the purposeful, self-regulatory judgment which results in interpretation,
analysis, evaluation, and inference, as well as explanation of the evidential,
conceptual, methodological, criteriological, or contextual considerations upon
which that judgment is based” (Facione, 1998).
210
Figure 2: The Core Critical Thinking Skills and Sub-skills
To place critical thinking in a well-defined context, scholars have posited

interpretation, analysis, inference, evaluation, explanation, and self-regulation as
the core critical thinking skills, with specific competencies indicating each skill.
This opens the construct for observation, measurement, and validation. Figure 2
captures the critical thinking construct at one glance.
The acquisition of satisfactory levels of all the critical thinking skills is the
embodiment of an ideal critical thinker. Experts, however, concur that a critical
thinker may not necessarily possess a high level of all the aforementioned skills.
One may be proficient at some skills but may not be at other skills (Facione, 2020).
Measuring critical thinking has been a crucial step in expanding this body of
knowledge. Experts have developed well-crafted and standardized instruments
to quantify critical thinking in formal/non-formal education and career
assessments. Based on content and features, these instruments have been
categorized into (1) general-content, multi-aspect; (2) general-content, aspect-
specific; and (3) subject-specific, multi-aspect critical thinking tests (Ennis, 2009).
The investigations of critical thinking in higher education have shown a dominant
utilization of the Watson-Glaser Critical Thinking Appraisal (WGCTA), California
Critical Thinking Skills Test (CCTST), and Cornell Critical Thinking Tests (CCTT),
all of which are classified under general-content, multi-aspect instruments (Behar-
Horenstein & Niu, 2011).
Meanwhile, research has shown that the Watson-Glaser Critical Thinking

Appraisal (WGCTA) is aptly applicable in measuring the critical thinking of
211
preservice teachers. Gadzella et al. (2005) used the Short Form (Form-S) of this
measurement instrument in their investigation and confirmed its suitability for
teacher education. The instrument measures five critical-thinking skills, namely
inference, recognition of assumptions, deduction, interpretation, and evaluation
of arguments (Behar-Horenstein & Niu, 2011). With reference to the APA Delphi
Consensus, the WGCTA does not contain explanation, analysis, and self-
regulation as principal critical-thinking skills; however, it includes recognition of
assumptions and deduction. Scholars in various fields, though, recognize its
applicability and effectiveness in assessing the critical thinking of students,
employees, and other adult participants (Behar-Horenstein & Niu, 2011; Ennis,
2009; Grosser & Nel, 2013).
Some scholars argue that the assessment of critical thinking can be done not only
quantitatively, but qualitatively as well; they posit that certain aspects of the
quality of thinking need not be captured by quantitative methods. Facione and
Facione (2011), for example, developed the Holistic Critical Thinking Scoring
Rubric (HCTSR) to evaluate thinking expressed as verbal and written accounts. .
This scoring rubric consists of explicit qualitative descriptions with an associated
numerical value which may be used in various assessments such as in training
programmes and other educational engagements. This contention motivated the
author to utilize both quantitative and qualitative measurement of critical
thinking of the preservice teachers who participated in the study, employing a
physics-oriented critical thinking instrument and a modified HCTSR.
Behar-Horenstein and Niu (2011) established the dominance of quasi-

experimental pretest-posttest design studies on critical thinking in higher
education. They also found that the majority of the studies conducted had been
executed in the field of medical education. Among the instructional strategies that
had been investigated were concept mapping, scenario-based course exercises,
active learning techniques, online instruction, inquiry-based learning, guided
practice, computer-assisted instruction, structured web-based bulletin boards,
and problem-based learning. The results of these studies generated mixed
impressions; no single instructional method was shown to either be always
effective or ineffective.
Few studies have been undertaken investigating critical thinking levels and
pedagogical methods with the potential of effecting the critical thinking of
preservice teachers. One such study (Akdere, 2012) found a ’below average’ level
of critical thinking among Turkish preservice teachers. Such a finding was
unexpected, considering that the same participants had a moderately positive
attitude towards critical thinking (Akdere, 2012). A similar observation was
claimed by Qing et al. (2010). The Chinese preservice teachers who participated in
their study showed a low level of critical thinking prior to their engagement in the
inquiry-based chemical experiment. The same level of performance was observed
even after their exposure to the pedagogical approach.
Tican and Taspinar (2015), likewise, reported that preservice teachers who had
been exposed to reflective thinking-oriented didactic activities had critical
212
thinking skills comparable to those who were exposed to traditional teaching

methods. No significant statistical difference was found between the two groups.
On the other hand, Jatmiko et al. (2018) and Sulaiman (2013) probed the effects of
the Problem-Based Learning (PBL) model on the critical thinking of preservice
physics and preservice science teachers, respectively. Jatmiko et al. employed the
conventional PBL model with 94 participants, while Sulaiman employed the PBL
Online with 41 participants. Jatmiko et al. reported positive effects of the PBL on
the critical thinking of the preservice physics teachers using the Student Critical
Thinking Skills Test. Meanwhile, employing the WGCTA Forms A and B,
Sulaiman found a non-significant difference in the critical thinking of the PBL
group and the traditional group after the intervention, notwithstanding the
significantly better performance of the PBL group than the traditional group in
creativity.
In an attempt to assess the efforts to enhance critical thinking of undergraduate

college students, a recent study has been conducted by investigating 4,000
participants. It was found that the 2,000 randomly selected undergraduate college
students in 2019 scored significantly higher in the California Critical Thinking
Skills Test (CCTST) than the 2,000 randomly selected students at the same level
who took the same test in 2012 (Facione et al., 2020). The result implies that
educational efforts strived for the development of critical thinking among learners
in higher education have paid off. This implies that a sustained fortitude may
eventually build communities of critical thinkers.
It can be assumed that the critical thinking construct in the preservice teacher
education domain remains to be cultivated. A plethora of didactic innovations yet
have to be forged, particularly collaborative and learner-centred pedagogical
approaches. One such approach is the Microteaching Lesson Study that offers
potential advances in various facets of learning. Its effects on preservice teachers’
critical thinking, using both quantitative and qualitative methods, still are
unexplored, especially during active and passive forms of microteaching.
Moreover, little is known about research on critical thinking utilizing content-
based assessment tools, especially in physics education. These motivations
propelled the researcher to undertake the present study.
3. Methodology
3.1. Research Design
A pretest-posttest comparison-group quasi-experimental design was employed in
this study. Two groups were formed from a class of fourth-year level college
students in the Bachelor of Secondary Education (BSED) programme specializing
in Physical Science. Nine participants were randomly chosen for each group - the
Active MLS Group and the Passive MLS Group - through a simple fishbowl
technique. Each of these groups was divided further into three sub-groups with
three members each. Figure 3 exemplifies the formation of the Active MLS and
Passive MLS sub-groups.
213
Microteaching
Lesson Study
Groups (MLS)
(N=18)
Active MLS Passive MLS

Group (n=9) Group (n=9)
Active MLS Active MLS Active MLS Passive MLS Passive MLS Passive MLS
Group 1 (n=3) Group 2 (n=3) Group 3 (n=3) Group 1 (n=3) Group 2 (n=3) Group 3 (n=3)
Figure 3: The formation of the Active MLS and the Passive MLS sub-groups
In adherence to ethical standards set by the university for social science research,
an informed consent form was required to be completed by the research
participants to declare their voluntary participation in the study and free will to
withdraw anytime during the course of the study. Based on Philippine law, none
of the participants were under age since all of them were 18 years old and above.
Moreover, none of them withdrew from participating in the study.
The data collection procedure for this study phase lasted for six weeks, and it was
conducted at a state university during the summer term of the school year 2016-
2017. It commenced with orientating the subject teacher and research participants
to the research procedure. The pre-test for critical thinking was administered a
day before the intervention was conducted. During the succeeding weeks the nine
preservice teachers were exposed to the Active MLS and the other nine to the
Passive MLS. The post-test was administered on the last day of the six-week
intervention.
The researcher guided the subject teacher throughout the data-collection process.
He assisted the teacher in assessing the participants’ instructional plans and
materials. He also took part in the microteaching sessions and post-lesson
discussions. The researcher acted the role of an external expert and facilitator in
the Lesson Study model.
The Active and Passive Microteaching Lesson Study

The research intervention utilized the lesson study (LS) framework through
microteaching sessions. In-service lesson study engagements involve actual
lesson implementations in classes handled by the teachers in the LS group. It must
be noted that in this study the research participants implemented their lesson
plans with their classmates serving as “students”.
It must also be clear that only the active MLS group members implemented their
lesson plans. One group was assigned to teach per microteaching session. The
passive MLS group participated in the microteaching sessions by acting as
“students” together with the non-teaching active MLS group for the session.
214
Nine grade 8 science lessons in physics were covered for the study. The areas
included were dynamics, heat and thermodynamics, electricity and magnetism,
waves, and optics. Three instructional plans and teaching materials had been
prepared by each sub-group, and they shared these with other sub-groups within
their respective group at the end of the research intervention. For instance, the
outputs of Group 1 were shared with the other two sub-groups in the active MLS
group only. The same process was followed by the passive MLS sub-groups.
Table 1: The Active MLS and Passive MLS procedures

1. Identifying learning areas 1. Identifying learning areas
2. Designing the research lesson 2. Designing the research lesson
3. Investigating the learning process 3. Investigating the learning process
4. Implementing the lesson plan 4. Acting as “students” in the
(microteaching) microteaching
5. Discussing the results of the 5. Revisiting the lesson plan
implementation
6. Revising the lesson plan 6. Revising the lesson plan
(optional)
7. Implementing the revised lesson 7. Acting as “students” in the
plan (microteaching) microteaching
8. Discussing the results of the 8. Reassessing the lesson plan
second implementation (optional)
9. Documenting and sharing results 9. Documenting and sharing results
Table 1 differentiates the MLS processes that were undertaken by the active MLS
group and passive MLS group. It can be assumed that all the steps of the Lesson
Study Model described in Figure 1 were carried out by all the sub-groups, except
that the passive MLS group did not implement its lesson plans. They rather took
part in the lesson implementations of the active MLS members by acting as
“students” until all the nine lessons had been implemented (Steps 4 and 7).
Post-lesson discussions among the teaching sub-group members of the active

MLS group, subject teacher, and researcher were conducted after lesson
implementations (Steps 5 and 8). During this stage, the passive MLS sub-groups
reviewed their lesson plan to incorporate possible changes based on their
observations in the microteaching.
Revision of the instructional plan by the teaching active MLS sub-group followed
the post-lesson discussions (Steps 6 and 9). At this time, the passive MLS sub-
groups had the option of revising their lesson plan (Steps 6 and 8). The final step
involved sharing the finalized lesson plan, improved instructional materials, and
detailed observations.
3.2. The Sample

The research participants were 18 students, specializing in physical science in the
Bachelor of Secondary Education (BSED) programme at a state university in
Tacloban City, Philippines, with ages ranging from 18 to 22 years. The mean age
of the active MLS group was 19.00 years, and 19.72 years for the passive MLS
group.
215
A total of nine (50%) male and nine (50%) female preservice teachers participated
in the study. The active MLS group comprised three males, that is, 33.3% of the
group sample and six female (66.7%) participants. Meanwhile, six male or 66.7%
of the group sample and three female (33.3%) participants formed the passive
MLS group.
Initial Comparability in Academic Performance

The comparability of the academic performance of the two groups of research
participants was established by analysing their respective ratings in the two
mathematics and 13 science courses that they had successfully completed prior to
their involvement in the study. The mean and median ratings per group were
determined and were further analysed using the Mann-Whitney U test through
the SPSS Version 23. Results are summarized in Table 2.
Table 2: Data and analysis for the ratings in Science and Mathematics
Subject/Group N Mean Rating Median p
Science
Active MLS 9 1.89 1.94
.863
Passive MLS 9 1.89 1.87
Mathematics
Active MLS 9 1.77 1.80
.489
Passive MLS 9 1.68 1.75
Note: The highest passing mark is 1.00, while the lowest is 3.00 in the university where the study
was conducted. It means that the higher the numerical value, the lower is the rating .
Results show that the mean ratings for science were equal between the active MLS
and passive MLS group (1.89), although a slightly higher median rating was
observed in the passive MLS group (1.87) than in the active MLS group (1.94). A
p-value of p=.863 denotes that the academic performance of the two groups in
science is not significantly different.
Results further show that the active MLS group had lower mean and median
ratings (1.77 and 1.80, respectively) than the passive MLS group (1.68 and 1.75,
respectively). However, the statistical analysis reveals that the difference in group
ratings was not significant. Hence, it can be affirmed that the active MLS and
passive MLS groups are comparable with respect to scholastic performance in the
two major academic fields, the sciences and mathematics.
3.3. The Instruments

The researcher gathered the data using three instruments as described in the
succeeding texts.
A. Preservice Teacher Critical Thinking Inventory in Physics (PTCTI)

The critical thinking of the research participants was measured through the
PTCTI, a physics content-based researcher-made instrument, composed of 60
objective-type and open-ended items. The instrument was modelled on the
Watson-Glaser Critical Thinking Appraisal (WGCTA), where 12 items in the
216
physics context were crafted for each component - Inference, Recognition of

Assumptions, Deduction, Interpretation, and Evaluation of Arguments.
An explanation of the choice of option was required for each objective-type item .
While one point was allotted for each correct response to the objective-type items,
the explanation was scored based on the descriptions provided in The Holistic
Critical Thinking Scoring Rubric by Facione and Facione (2011), which ranged
from zero to three. Few modifications were made to the assessment tool to suit
the research objectives and context.
The instructions, items, and content of the PTCTI were validated by experts in
science and mathematics education whose academic works entail studies into
critical thinking development, as well as the production of research instruments
and instructional materials that promote critical thinking skills. Meanwhile, the
BSED-Physical Science specialists from two state universities were invited to the
pilot-testing of the PTCTI. The overall reliability test of the instrument revealed a
Cronbach’s alpha of .923, as shown in Table 3, signifying its high reliability in
critical thinking assessment. Its sub-tests also recorded acceptable reliability levels
for social science research, which ranged from .701 to .807.
Table 3: The SPSS computation of Cronbach’s alpha of the PTCTI
Cronbach's
Alpha Based
on
Cronbach's Standardized
Alpha Items N of Items
.923 .926 60
B. Interview Protocol
An interview was conducted on the last day of the intervention. A researcher-
compiled interview protocol was utilized to substantiate observation notes, post-
lesson discussions, and quantitative data. The pre-constructed questions revolved
around the participants’ reflections on the instructional approach to which they
were exposed. Follow-up questions were added to illuminate vague ideas or
statements as necessary.
C. Journal
The participants recorded their observations, learning experiences, and insights
in a journal for the entire MLS engagement. The content of the entries was
analysed to deduce themes concerning critical thinking development to support
the quantitative analyses.
3.4. Data Analysis Procedure

The research objectives necessitated the collection of quantitative data that could
be illuminated by qualitative data. Descriptive statistics like mean and median
were used to depict scores of the research participants in the Preservice Teacher
Critical Thinking Inventory in physics (PTCTI). Significant differences between
groups were identified by employing the Mann-Whitney U Test, whereas the
217
Wilcoxon Signed Ranks Test was utilized to analyse within-group differences.

These inferential statistical tests were set at α=.05 level of significance.
The researcher employed the conceptual content analysis technique to analyse the
qualitative data derived from interview responses, journal entries, and
observation notes. Results of the analysis were used to substantiate both
quantitative and qualitative findings.
4. Results
4.1. Critical Thinking Prior to the Study.
A pre-test on the critical thinking of the research participants was administered a
day before conducting the intervention. It must be noted that the highest possible
score for the Critical Thinking test is 300 points and the lowest is zero.
Table 4 indicates that the active MLS group (Mn=124.00, 41.3%) scored lower than
the passive MLS group (Mn=131.44, 43.8%). The same trend was observed when
the median scores were examined.
Table 4: Pre-test results for the critical thinking inventory

Mn
Test/Group N Md (Max.=300) SD
(Max.=300)
Pre-test
Active MLS 9 118 (39.3%) 124.00 (41.3%) 31.2
Passive MLS 9 130 (43.3%) 131.44 (43.8%) 26.2
Total 18 127 (42.3%) 127.72 (42.6%) 28.2
Despite the passive MRLS group scoring higher than the active MRLS group, it
can be gleaned from Table 5 that the difference in the pre-test group scores is not
significant (p=.627), with a low effect size (r=.11). This suggests that the active
MLS group and the passive MLS group were comparable before they participated
in the study.
Table 5: Mann-Whitney U test of the critical thinking inventory pre-test

Mean Mann-
Test/Group N Z p r (ES)
Rank Whitney U
Pre-test
Active MLS 9 8.89
35.0 -.486 .627 .11
Passive MLS 9 10.11
Table 6 exhibits the analysis of the pre-test results of the five components of critical
thinking. The passive MLS group scored slightly higher in the Inference,
Deduction, Interpretation, and Evaluation of Arguments components than the
active MLS group. Conversely, the active MLS group scored slightly higher than
the passive MLS group in the Recognition of Assumptions. It can be observed,
further, that there is no significant difference between the group scores across the
five components, considering that the p-values range from p=.309 to p=.929, all of
which are greater than p=.05. These show that the active MLS group and the
218
passive MLS group were comparable in critical thinking sub-skills prior to their
participation in the study.
Table 6: Mann-Whitney U test per component of the critical thinking inventory pre-test
CT
Mn Mean Mn Mean M-W U Z p
Component
(Max.=60) Rank (Max.=60) Rank
Inference 31.3 (52.2%) 9.22 32.1 (53.5%) 9.78 38.0 -.222 .825
Recognition of 22.9 (38.2%) 9.61 20.8 (34.7%) 9.39 39.5 -.089 .929
Assumptions
Deduction 24.3 (40.5%) 8.67 27.2 (45.3%) 10.33 33.0 -.665 .506
Interpretation 27.7 (46.2%) 9.17 28.7 (47.8%) 9.83 37.5 -.266 .790
Evaluation of 17.8 (29.7%) 8.22 22.7 (37.8%) 10.78 29.0 -1.02 .309
Arguments
4.2. Results of the Critical Thinking Inventory Post-test. The Preservice Teacher
Critical Thinking Inventory in physics was administered again a day after the
intervention as a post-test. Table 7 presents the mean and median group scores
and standard deviations. Interestingly, the active MLS group incurred a higher
overall mean score (Mn=241.8, 80.6%) than the passive MLS group (Mn=214.1,
71.4%). A lower standard deviation was also incurred by the active MLS group
(SD=12.7) than by the passive MLS group (SD=23.4). This signifies that the scores
of the participants exposed to the active MLS were closer than the scores of those
exposed to the passive MLS.
Table 7: Post-test results for the overall critical thinking inventory

Mn
Test/Group N Md (Max.=300) SD
(Max.=300)
Post-test
Active MLS 9 246 (82.0%) 241.8 (80.6%) 12.7
Passive MLS 9 217 (72.3%) 214.1 (71.4%) 23.4
Total 18 239 (79.7%) 227.9 (76.0%)
To determine the effect of the intervention on the group’s overall critical thinking,
the Wilcoxon Signed Ranks test was utilized. The test compares the pre-test and
post-test scores of both groups. It is shown in Table 8 that both the active MLS and
passive MLS groups obtained significantly higher post-test scores (p=.008) than
their pre-test scores. This suggests that both the active MLS and the passive MLS
can significantly improve the critical thinking of the preservice physics teachers.
Table 8: Wilcoxon Signed Ranks Test for the two groups in the overall critical
thinking inventory
Group N Z p
Active MLS 9 -2.67 .008**
Passive MLS 9 -2.67 .008**
Note: *p<.05; **p<.01
Further examination was done to probe significant differences in the effects of the
two interventions on participants’ critical thinking. The Mann-Whitney U test, as
shown in Table 9, revealed that the participants who were exposed to the active
219
MLS obtained a significantly higher mean score than those who were exposed to
the passive MLS, as indicated by a p-value of p=.007. Moreover, a relatively high
effect size of r=.636 had been established by the intervention between the two
groups of participants. These connote that the active MLS is significantly better at
developing the overall critical thinking of the preservice physics teachers than the
passive MLS.
Table 9: Mann-Whitney U test of the critical thinking inventory post-test

Mean Mann-
Test/Group N Z p r (ES)
Rank Whitney U
Post-test
Active MLS 9 12.9
10.00 -2.70 .007** .636
Passive MLS 9 6.11
Note: *p<.05; **p<.01
The post-test mean scores of the participants per component of the Critical
Thinking Inventory are presented in Table 10. It must be noted that the active MLS
group scored higher than the passive MLS group across all components, the score
of which ranged from 75.55% up to 86.85%, as opposed to 68% up to 76.30%. This
points out that the active MLS group performed better than the passive MLS
group in all the sub-tests after the intervention.
Table 10: Post-test mean scores per critical thinking component

Critical Thinking Active MLS Passive MLS
Component Mn (Max.=60) Mn (Max.=60)
Inference 52.11 (86.85%) 45.78 (76.30%)
Recognition of 45.33 (75.55%) 41.00 (68.33%)
Assumptions
Deduction 47.44 (79.07%) 42.11 (70.18%)
Interpretation 49.44 (82.40%) 41.56 (69.27%)
Evaluation of Arguments 47.44 (79.07%) 43.67 (72.78%)
A comparison between the pre-test and post-test scores of each group per
component was established through the Wilcoxon Signed Ranks Test. It can be
seen in Table 11 that the p-values across all components are less than the .05 (i.e.
p=.008) for both groups, implying that both groups obtained significantly higher
scores in the post-test than in the pre-test of the critical thinking sub-tests. Further,
it can be deduced that both the active MLS and the passive MLS are effective
interventions in enhancing the critical thinking sub-skills of the preservice
teachers.
Table 11: Wilcoxon Signed Ranks Test per component of the critical thinking
inventory
Critical Thinking
Group N Z p
Component
Inference Active MLS 9 -2.67 .008**
Passive MLS 9 -2.67 .008**
Recognition of Active MLS 9
-2.67 .008**
Assumptions
Passive MLS 9 -2.67 .008**
220
Deduction Active MLS 9 -2.67 .008**

Passive MLS 9 -2.67 .008**
Interpretation Active MLS 9 -2.67 .008**
Passive MLS 9 -2.67 .008**
Evaluation of Active MLS 9
-2.67 .008**
Arguments
Passive MLS 9 -2.67 .008**
Note: *p<.05; **p<.01
Table 12 exhibits the Mann-Whitney U test analysis between the two groups of
participants per component of the Preservice Teacher Critical Thinking Inventory.
It can be reckoned that the Active MLS group obtained a significantly higher mean
score than the passive MLS group in the Inference (p=.024) and Interpretation
components (p=.003). No significant difference in the group scores was perceived
in the Recognition of Assumptions, Deduction, and Evaluation of Arguments,
since the p-values fall beyond the p=.05 level. These results imply that engaging
in the active MLS can significantly improve the critical thinking skills of the
preservice physics teachers, rather than the passive MLS, specifically on inference-
making and interpreting information.
Table 12: Mann-Whitney U test per critical thinking component for the post-test
Critical Thinking Post-test
Group
Component Mean Rank M-W U p
Inference Active MLS 12.33
15.00 .024*
Passive MLS 6.67
Recognition of Active MLS 7.28
20.50 .076
Assumptions Passive MLS 11.72
Deduction Active MLS 11.61
21.50 .093
Passive MLS 7.39
Interpretation Active MLS 13.17
7.50 .003**
Passive MLS 5.83
Evaluation of Active MLS 11.89
19.00 .056
Arguments Passive MLS 7.11
Note: *p<.05; **p<.01
5. Discussion
Findings of the study indicated the positive effects of both the active and passive
MLS in enhancing overall critical thinking and its sub-skills in preservice physics
teachers. These findings concur with the results of the study of Burroughs and
Luebeck (2010), who proclaimed that Lesson Study allows the preservice teachers
to act as critical thinkers and think like real teachers. It must be noted that the
research participants worked collaboratively in preparing the instructional plans
and materials; the exchange of thoughts and viewpoints is inevitable.
Accordingly, the theory of adult learning asserts that establishing a cooperative
milieu augments learning through varied networks (Knowles, 1973; 1984). In
essence, the active and passive MLS are collaborative learning engagements that
develop social skills (Mewald & Mürwald‐Scheifinger, 2019), and promote a
collective sense of responsibility, powerful social interactions, and academic
discourses (Cajkler et al., 2015; Cerbin & Kopp, 2006; Hixon, 2009; Kolenda, 2007;
Laal & Laal, 2012).
221
To explicate the assertion above, Dianne stated in an interview that working with
her groupmates had helped to improve her instructional skills. She stated:
“…it’s very helpful that I have partners [groupmates] who help me, to
give additional advices [pieces of advice], additional ideas for [my] lesson
and [on] how to deliver the lesson.” (Dianne)
In view of the aforementioned desired results, engaging in the active and passive
MLS might serve as an excellent training ground for accelerating teaching
performance considering that these forms of instructional approach demand
insightful reflections and critical analysis of their thinking and teaching (Arsal,
2015, 2017; Hamzah et al., 2008). Likewise, the assessment of critical thinking
through a content-based instrument such as the Preservice Teacher Critical
Thinking Inventory in Physics permits a close examination of both critical
thinking skills and content knowledge of the participants. Exposure of the
preservice teachers to such an assessment tool brings Physics ideas in a different
perspective, opening for more learning opportunities geared towards conceptual
understanding in Physics (Buabeng et al., 2016).
Results further suggest that the Active MLS is more effective at enhancing overall
critical thinking and its sub-skills, namely, inference and interpretation, than the
Passive MLS. Cerbin and Kopp (2006) and Zhou and Xu (2017) have proclaimed
that the collaborative preparation of instructional plans and materials, lesson
implementations, and post-lesson discussions prompt the Lesson Study members
to clarify inquiries and reservations, explore techniques and procedures, and
reflect on all aspects of instruction. This alludes to the contention that one essential
feature of collaborative learning is a receptive discourse that promotes
articulation, scrutiny, and justification of viewpoints leading the learners to
generate tailored conceptual frameworks (Laal & Laal, 2012).
The higher level of critical thinking of the Active MLS group than the Passive MLS
group may also be attributed to the “internal pressure” incited by the lesson
implementations with their contemporaries serving as “students” and with the
presence of the subject teacher and researcher (Cavin, 2007; Cerbin & Kopp, 2006).
The active MLS members were compelled to keenly examine lesson
implementation procedures, bearing in mind that their audience had comparable
or even greater knowledge, skills, and faculties than they themselves. In an
interview, Robert divulged his anxiety when he implemented his first lesson:
Robert: Yes, Sir. Based on what we experienced as a teacher, we felt
conscious of the presence of our classmates. They already know
the concepts, so we tend to be affected in our demo, Sir. We also
tend to be conscious of what we say. We are intimidated by our
classmates … and by your presence as well, Sir.
Researcher: Okay. So, you get intimidated?
Robert: Yes, Sir. That is why we try our best in our teaching.
Alexa expressed a similar impression in one of her journal entries.

….Two days from now, our group will teach the next topic. We are
nervous because Rina’s group has done great [in their demonstration
222
teaching]. We plan to use computerized [digital] representations because

they [classmates] enjoy the discussion and activities… (Alexa)
The lesson implementations prompted the active MLS group to embark on

technology-based pedagogies such as the incorporation of digital representations.
They utilized diagrams, animations, video clips, and simulations of scientific
concepts, principles, and processes. The interaction with media-aided
representations fosters critical analysis competencies and engagements with
critical thinking (Barlow et al., 2013; Quinnell et al., 2020). The preparation,
implementation, and reflection phases of the active MLS allowed the group
members to scrutinize, assess, and revise the instructional aids that complement
teaching methodologies.
6. Conclusion and Recommendation

The findings of this study suggest that both the active and passive MLS are
effective in developing the critical thinking of the preservice physics teachers. It
further has been confirmed that active MLS is more effective at developing overall
critical thinking and sub-skills inference and interpretation than passive MLS. The
findings of this study suggest that preservice teacher education may be enriched
by integrating active and passive MLS in the teacher education curriculum. The
affordances of these two forms of microteaching lesson study (MLS) can address
the need to scaffold preservice teachers’ instructional competencies (Cavanagh et
al., 2019) which the conventional individualistic instructional approach does not
warrant, particularly the development of critical thinking.
It is, hence, recommended that preservice teachers be exposed to these two

instructional approaches, especially to the active MLS. Conducting similar studies
involving more participants may be endeavoured to reinforce positive influences
of the intervention. Digital instructional aids and tools also may be provided to
these future participants to maximize learning engagements. Likewise, content-
oriented critical thinking assessment tools like the PTCTI may be developed,
evaluated, and utilized in research to supplement existing knowledge on the
critical thinking construct. Finally, the integration of computer-generated
representations in collaborative instructional designs is encouraged not only to
capture interest but also to stimulate critical analysis and higher-order thinking
(Ozmantar et al., 2010).
7. Acknowledgments
The author acknowledges the professional and financial support of the Leyte
Normal University (LNU) through the LNU Faculty and Staff Development
Program (FSDP) and the Dissertation Grant of the Commission on Higher
Education in the completion of this academic endeavour. Profound gratitude is
hereby conveyed.
Likewise, the author expresses earnest appreciation to the academic pillars of the
UP College of Education, UP NISMED, UP Open University, and the National
Institute of Physics for the insightful assistance in scaffolding the integrity of this
paper.
223
8. References
Akdere, N. (2012). Turkish preservice teachers’ critical thinking levels, attitudes and self-
efficacy beliefs in teaching for critical thinking. [Thesis, Middle East Technical
University, Ankara]. http://etd.lib.metu.edu.tr/upload/12614436/index.pdf
Akkus, H., & Üner, S. (2017). The effect of microteaching on preservice chemistry teachers’
teaching experiences. Cukurova University Faculty of Education Journal, 46 (1), 202-
230. https://doi.org10.14812/cuefd.309459
Akyuz, H., & Samsa, S. (2009). Critical thinking skills of preservice teachers in the blended
learning environment. International Journal of Human Sciences, 6(2), 538-550.
Arsal, Z. (2015). The effects of microteaching on the critical thinking dispositions of
preservice teachers. Australian Journal of Teacher Education, 40(3), 140-153.
http://dx.doi.org/10.14221/ajte.2014v40n3.9
Arsal, Z. (2017). The impact of inquiry-based learning on the critical thinking dispositions
of preservice science teachers. International Journal of Science Education, 39(10),
1326–1338.
Battelle for Kids (2019). Framework for 21st century learning: Definitions.
http://static.battelleforkids.org/documents/p21/P21_Framework_DefinitionsB
FK.pdf
Barlow, A. T., McCrory, M. R., & Blessing, S. (2013). Classroom observations and
reflections: Using online streaming video as a tool for overcoming barriers and
engaging in critical thinking. International Journal of Education in Mathematics,
Science and Technology, 1(4), 238-258.
Barrett, D., Riggs, L., & Ray, J. (2013). Teachers’ collaborative use of the lesson study
approach to foster student achievement in geometry. International Journal of Social
Sciences & Education, 3(4), 1188-1192.
Behar-Horenstein, L., & Niu, L. (2011). Teaching critical thinking skills in higher
education: A review of the literature. Journal of College Teaching & Learning, 8(2),
25-41.
Buabeng, I., Conner, L., & Winter, D. (2016). Physics teachers’ views on their initial teacher
education. Australian Journal of Teacher Education, 41(7), 36—55.
Buchard, J., & Martin, D. (2017). Lesson study… and its effects. Hellenic Journal of Research
Education, Special Issue, 1-15. http://dx.doi.org/10.12681/hjre.14809
Buckley, S. (2012). The role of computational thinking and critical thinking in problem
solving in a learning environment. European Conference on e-Learning (pp. 63-70).
Kidmore End, United Kingdom: Academic Conferences International Limited.
Burroughs, E., & Luebeck, J. (2010). Preservice teachers in mathematics lesson study. The
Mathematics Enthusiast, 7(2), 391-400.
Cajkler, W., Wood, P., Norton, J., Pedder, D., & Xu, H. (2015). Teacher perspectives about
lesson study in secondary school departments: A collaborative vehicle for
professional learning and practice development. Research Papers in Education,
30(2), 192-213.
Cavanagh, M., Barr, J., Moloney, R., Lane, R., Hay, I., & Chu, H. (2019). Preservice teachers’
impact on student learning: Planning, teaching, and assessing during professional
practice. Australian Journal of Teacher Education, 44(2), 66-81.
Cavin, R. (2007). Developing technological pedagogical content knowledge in preservice teachers
through microteaching lesson study [Doctoral Dissertation, Florida State University].
Ann Arbor, MI: ProQuest Information and Learning Center/UMI.
224
Cerbin, W., & Kopp, B. (2006). Lesson study as a model for building pedagogical
knowledge and improving teaching. International Journal of Teaching and Learning
in Higher Education, 18(3), 250-257.
Chen, X., & Zhang, Y. (2019). Typical practices of lesson study in East Asia. European
Journal of Education, 54(2), 189–201.
Chew, C., & Lim, C. (2013). Developing preservice teachers' technological pedagogical
content knowledge for teaching mathematics with the geometer's sketchpad
through lesson study. Journal of Education and Learning, 2(1), 1-8.
Danday, B. (2019). Active vs. passive microteaching lesson study: Effects on pre-service
teachers’ technological pedagogical content knowledge. International Journal of
Learning, Teaching, and Educational Research, 18(6). 181-200.
Elipane, L. (2012). Integrating the essential elements of lesson study in preservice
mathematics teacher education [Doctoral dissertation, Copenhagen University].
Ennis, R. (2009). An annotated list of critical thinking tests.
https://www.academia.edu/34209488/AN_ANNOTATED_LIST_OF_CRITICA
L_THINKING_TESTS
Facione, P. (1998). Critical thinking: A statement of expert consensus for purposes of educational
assessment and instruction. Executive Summary “The Delphi Report”.
https://www.insightassessment.com/wp-content/uploads/ia/pdf/DEX-2014-
reprinting.pdf
Facione, P. (2020). Critical thinking: What it is and why it counts (2020 ed.)., Hermosa Beach,
CA: Measured Reasons LLC.
Facione, P., & Facione, N. (2011). The holistic critical thinking scoring rubric – HCTSR: A tool
for developing and evaluating critical thinking. San Jose, CA: The California Academic
Press / Insight Assessment.
Facione, P., Facione, N., & Gittens, C. A. (2020). What the critical thinking data tell us.
Hermosa Beach, CA: Measured Reasons LLC/Insight Assessment.
Fernandez, C. (2002). Learning from Japanese approaches to professional development:
The case of lesson study. Journal of Teacher Education, 53(5), 393-405.
Gadzella, B., Stacks, J., Stephens, R., & Masten, W. (2005). Watson-Glaser critical thinking
appraisal, form-S for education majors. Journal of Instructional Psychology, 32(1), 9-
12.
Gashan, A. (2015). Exploring Saudi preservice teachers' knowledge of critical thinking
skills and their teaching perceptions. International Journal of Education & Literacy
Studies, 3(1), 26-33.
Glaser, E. (1941). An experiment in the development of critical thinking. New York: Teachers
College, Columbia University.
Goyak, A. (2009). The effects of cooperative learning techniques on perceived classroom
environment and critical thinking skills of preservice teachers [Doctoral
dissertation, Liberty University]. Ann Arbor, MI: ProQuest LLC/UMI.
Grosser, M., & Nel, M. (2013). The relationship between the critical thinking skills and the
academic language proficiency of prospective teachers. South African Journal of
Education, 33(2), 1-17.
Gurl, T. (2009). An analysis of an adaptation of lesson study with preservice secondary
mathematics teachers [Doctoral dissertation, Columbia University]. Ann Arbor,
MI: ProQuest LLC/UMI.
Gutierez, S. (2015). Collaborative professional learning through lesson study: Identifying
the challenges of inquiry-based teaching. Issues in Educational Research, 25(2), 118-
134.
225
Hamzah, S. G., Mohamad, H., & Ghorbani, M. R. (2008). Excellent teachers’ thinking
model: Implications for effective teaching. Australian Journal of Teacher Education,
33(4), 11-27. http://dx.doi.org/10.14221/ajte.2008v33n4.2
Hixon, M. (2009). Lesson study: A proposed intervention for professional development and
student achievement [Doctoral dissertation, Walden University]. Ann Arbor, MI:
UMI Dissertation Publishing, ProQuest LLC.
Jatmiko, B., Prahani, B. K., Munasir, Supardi, Z. A. I., Wicaksono, I., Erlina, N., &
Zainuddin. (2018). The comparison of OR-IPA teaching model and problem-
based learning effectiveness to improve critical thinking skills of preservice
physics teachers. Journal of Baltic Science Education, 17(22), 300-319.
Kolenda, R. (2007). Japanese lesson study, staff development, and science education
reform: - The Neshaminy story. Science Educator, 16(1), 29-33.
Knowles, M. (1973). The adult learner: A neglected species. Houston: Gulf Publishing.
Knowles, M. (1984). The adult learner: A neglected species (3rd ed.). Houston, TX: Gulf
Publishing.
Laal, M., & Laal, M. (2012). Collaborative learning: What is it? Procedia - Social and
Behavioral Sciences 31, 491-495.
Lewis, C. (2002). Does lesson study have a future in the United States? Nagoya Journal of
Education and Human Development 1, 1-23.
Lewis, C., Perry, R., & Hurd, J. (2009). Improving mathematics instruction through lesson
study: A theoretical model and North American case. Journal of Mathematics
Teacher Education, 12, 285-304. doi:10.1007/s10857-009-9102-7
Lewis, C., Perry, R., & Murata, A. (2006). How should research contribute to instructional
improvement? The case of lesson study. Educational Researcher, 35, 3–14.
https://doi.org/10.3102/0013189X035003003
Lucenario, J., Yangco, R., Punzalan, A., & Espinosa, A. (2016). Pedagogical content
knowledge-guided lesson study: Effects on teacher competence and students’
achievement in chemistry. Education Research International, 2016, 1-9.
McDowell, A. (2010). Preservice teachers’ use of lesson study in teaching nature of science
[Doctoral dissertation, Georgia State University].
http://scholarworks.gsu.edu/cgi/viewcontent.cgi?article=1064&context=msit_
diss
Meister, D. (2011, February 15). Preservice teachers' impact on student learning: Three-
year study of the impact on student learning project [Paper presentation].
Association of Teacher Educators Annual Conference, Orlando, Florida.
http://files.eric.ed.gov/fulltext/ED524510.pdf
Mewald, C., & Mürwald‐Scheifinger, E. (2019). Lesson study in teacher development: A
paradigm shift from a culture of receiving to a culture of acting and reflecting.
European Journal of Education, 54(2), 218–232.
Nilson, C., Fetherston, C. M., McMurray, A., & Fetherston, T. (2013). Creative arts: An
essential element in the teacher’s toolkit when developing critical thinking in
children. Australian Journal of Teacher Education, 38(7), 1-17.
Ozmantar, M., Akkoc, H., Bingolbali, E., Demir, S., & Ergene, B. (2010). Preservice
mathematics teachers' use of multiple representations in technology-rich
environments. Eurasia Journal of Mathematics, Science & Technology Education, 6(1),
19-36.
Qing, Z., Jing, G., & Yan, W. (2010). Promoting preservice teachers’ critical thinking skills
by inquiry-based chemical experiment. Procedia Social and Behavioral Sciences, 2,
4597–4603.
226
Tican, C., & Taspinar, M. (2015). The effects of reflective thinking-based teaching activities
on preservice teachers’ reflective thinking skills, critical thinking skills,
democratic attitudes, and academic achievement. The Anthropologist 20(1,2), 111-
120. http://dx.doi.org/10.1080/09720073.2015.11891730.
Quinnell, L., Iyer, R. & Ewing, B. (2020). Preservice teachers as document detectives.
Australian Journal of Teacher Education, 45(6).
Remesh, A. (2013). Microteaching: An efficient technique for learning effective teaching.
Journal of Research in Medical Sciences, 18, 158-63.
Sensibaugh, C. (2015). Defining and assessing problem solving across a biochemistry
curriculum [Doctoral dissertation, University of New Mexico]. Ann Arbor, MI:
ProQuest LLC.
Sentumbwe, D. (2018). Student-teachers’ experiences of microteaching on an economics
methods course. African Research Review, 12(2), 101-108.
http://dx.doi.org/10.4314/afrrev.v12i2.10.
Sulaiman, F. (2013). The effectiveness of PBL online on preservice science. Wudpecker
Journal of Educational Research, 2(6), 086 - 096.
Teele, S., Maynard, D. & Marcoulides, G. (2015). The lesson study process: - An effective
intervention to reduce the achievement gap. US-China Education Review, 5(4), 229-
243.
Tseng, M.-H. (2008). Data-driven problem solving and students' critical thinking in a problem-
based learninng environment . Ann Arbor, MI: ProQuest LLC/UMI.
Williams, R. (2005). Targeting critical thinking within teacher education: The potential
impact on society. The Teacher Educator, 40(3), 163-187.
Zhou, G. & Xu, J. (2017). Microteaching lesson study: An approach to prepare teacher
candidates to teach science through inquiry. International Journal of Education in
Mathematics, Science and Technology, 5(3), 235-247. DOI:10.18404/ijemst.296039
227
Appendix 1: Sample Items of the Research Instrument
Preservice Teacher Critical Thinking Inventory in Physics

Name: ____________________________________ Date: ______________________
Course/Year/Section: _________________________ Specialization: _______________
GENERAL INSTRUCTIONS:
There are five components of this test – Inference, Recognition of
Assumptions, Deduction, Interpretation, and Evaluation of Arguments.
Each subtest or component consists of statements with proposed
inferences, assumptions, conclusions or arguments. Specific instructions are
provided in each subtest. Read carefully and follow instructions correctly.
You are REQUIRED to explain or state your reasons for your choice of
answer for each item. Limit your explanation to 1 - 3 sentences only. You
have a maximum of one (1) hour and thirty (30) minutes to finish this test.
STOP!
DO NOT TURN TO THE NEXT PAGE UNLESS YOU ARE TOLD TO DO SO.
228
TEST II: RECOGNITION OF ASSUMPTIONS

Directions: There are five statements or scenarios in this test. Each statement is followed
by suggested assumptions. Encircle ‘YES’ in the corresponding item number on your
answer sheet if you believe that the given assumption is correct. If you think the
assumption is not necessarily correct, encircle ‘NO’. Give a concise explanation of
your answer on the space provided.
Statement II-2: Two charges at rest, q1 and q2, lie behind a rectangular cardboard. An electric field
pattern is observed as shown in the figure below.
Proposed Assumptions:
16. The charges are the same and are equal in magnitude. YES NO
Explain your answer: ________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
17. Charge q1 is positive while charge q2 is negative. YES NO

Explain yourcount
Word answer:
8657________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
229

Vol. 20, No. 3, pp. 229-249, March 2021
Ecuadorian Teachers’ Perceptions of Teaching

English: Challenges in the Public Education
Sector
Agnes Orosz, Mirdelio Monzón and Paola Velasco

Universidad Nacional de Educación (UNAE), Azogues, Ecuador
https://orcid.org/0000-0002-1841-8320
https://orcid.org/0000-0001-7831-0963
https://orcid.org/0000-0001-6587-5260
Abstract. The current English curriculum in Ecuador requires that high-

school students graduate with an intermediate (B1) level. Attaining this
target depends on several factors, such as teaching strategies employed
in class and teacher training opportunities offered. Ecuador is ranked low
for English language proficiency and, therefore, an exploratory approach
has been used as the main data collection method to investigate the
possible reasons for poor performance. In the previous phase of an
overarching research project, round table discussions were held to
explore themes related to English teachers’ pedagogical practices. The
data gathering process in the phase described in the present study
involved conducting in-depth interviews with 10 primary and secondary
school English language teachers. The transcribed interviews were
analyzed by putting the material through the MAXqda qualitative
analysis software. The resulting coded categories allowed the authors to
investigate three specific issues dealt with in the present research study,
namely: teachers’ perceptions of the importance of learning English,
actual classroom teaching practices, and the teachers’ observations on in-
service training options. The findings of the study demonstrate that
English teachers in Ecuadorian public schools recognize the importance
of English and are making a substantial effort to keep up with the
professional demands related to teaching English as a Foreign Language,
but they lack the required knowledge to develop skills, such as critical
thinking and creativity. They are also facing several challenges, such as a
demanding new curriculum and students’ lack of motivation at a time of
apparently dwindling teacher training opportunities.
Keywords: English language teaching; perceptions on English; English

teaching curriculum; classroom teaching strategies; English teacher
training
©Authors
230
1. Introduction
In Ecuador, the importance of learning English as a Foreign Language (EFL) has
been acknowledged in several education policy documents, such as the National
Curriculum Guidelines (Ministerio de Educación, 2013) and English as a Foreign
Language (Ministerio de Educación, 2016a). The recognition of the need for
learning English as a global language has resulted in curricular changes, whose
aim is to build a solid foundation for authentic and culturally relevant language
production as set out in the document titled Currículo de los niveles de Educación
Obligatoria [Curriculum for the Compulsory Levels of Education] (Ministerio de
Educación, 2016b).
Nevertheless, a recent survey carried out by Education First (2020a) shows that
Ecuador ranks the last of the 19 surveyed Latin American countries in English
language proficiency. In fact, in the annual ranking of 100 countries and regions,
Ecuador slid from No. 81 in 2019 to No. 93 in 2020 (Education First, 2019;
Education First, 2020b). Despite the requirements laid down in the curriculum,
most students leave high school without reaching the required B1 level,
meanwhile English teachers appear to be struggling with the application of up-
to-date methodology and language teaching strategies (Halim, 2017). An
important question for inquiry could then be raised in this respect: what might be
the reasons for the low results and the overall ineffectiveness of English teaching
in this Latin American country?
Acosta and Cajas (2018) describe the attempts that Ecuador has been making to
prioritize EFL in the education system, especially as education policy experts
perceive English as an important communication tool for the country to succeed
in a globalized world (British Council, 2015). However, there is growing
realization that successful language policy changes require that policymakers are
aware of the challenges English teachers are facing in their classrooms (Soto et al.,
2020). In Ecuador, the new English teaching curriculum was launched in 2016 with
the aim of introducing up-to-date methodology via promoting the use of
technology and effective teaching-learning strategies throughout the country. The
main tenets of the new curriculum are the inclusion of Content and Language
Integrated Learning (CLIL), the application of Communicative Language
Teaching (CLT), and the use of technology (ICT) for educational purposes
(Ministerio de Educación, 2016a). The curriculum was last updated in 2018.
The most important addition is that high school students now need to graduate
with an intermediate level of English, which corresponds to B1 on the six-point
scale used by the Common European Framework of Reference for Languages
(CEFR). However, since English is not among the subjects of the national
secondary school leaving exam, only few students reach this level, which has a
knock-on effect on those who then pursue tertiary education (Calle et al., 2012;
Halim, 2017). The flagship state university of UNAE (Universidad Nacional de
Educación [National University of Education]), whose researchers carried out the
present research study, is committed to training a new generation of English
teachers and is looking to establish best practices in teacher education. Therefore,
it was deemed important to carry out an initial exploration of how English is being
231
taught in public sector schools and how teaching and learning issues are
perceived in primary and secondary schools by one of the main protagonists,
namely, the teachers themselves.
To explore the strengths and weaknesses of English Language Teaching (ELT) in

the public sector, a two-year research program was carried out from September
2018 to August 2020 at the university mentioned above. The overarching research
project aimed at investigating the factors that foster or hinder the teaching-
learning process of English as a Foreign Language in “Zone 6”, which is
comprised of three of Ecuador’s provinces, namely: Azuay, Morona Santiago, and
Cañar, representing about 10% of the country’s population. As part of this wider
research program, the authors of the present article are focusing on how English
teachers perceive the importance of learning English, what teaching strategies
they employ in their classrooms, and how far they feel they are supported in their
work by in-service teacher training opportunities provided by the state. Overall,
the research study offers further evidence that an exploratory and bottom-up
approach provides rich data from the field which might help policymakers design
interventions that can, in the long run, improve the quality of English language
teaching in Ecuador.
2.1. The Importance of Learning English
English has undeniably emerged as the global language by the beginning of the
21st century. Although it is difficult to establish how many people are fluent or
competent speakers of the world’s first truly global lingua franca, it is assumed that
there are between 1.5 and 2 billion people (roughly one in four of the world’s
population) that speak English as a first, second or foreign language (Crystal,
2003; British Council, 2013; Cronquist & Fiszbein, 2017). English is now
incorporated into the curriculum of almost 140 countries including those of Latin
America (Rojas & Hernández-Fernández, 2018). Policymakers recognize the
power of English (Chan, 2016) and the prospects it provides “ranging from ease
of travelling abroad and participating in the global economy to opportunities to
consume knowledge and media in other languages and to engage in diplomatic
affairs” (Rojas & Hernández-Fernández, 2018, p. 12). Research has shown that
there is a correlation between the economic performance of a country and the
English language proficiency of its population (McCormick, 2013). In addition,
English language skills also provide opportunities at an individual level, and this
is true for people who live in emerging economies such as Brazil and Mexico
(Rojas & Hernández-Fernández, 2018).
Even though the relevance of learning English is widely recognized on the whole
of the South American continent, researchers remind us that there are two specific
challenges in Latin America. One is that besides Spanish and Portuguese a
number of indigenous languages are spoken in these countries. Second, owing to
their colonial history, there might be sensitivity against a language that could be
perceived as a threat to sovereignty (Rojas & Hernández-Fernández, 2018).
However, there is an acknowledgment that public policies should support
societies that are both multilingual and intercultural (Starkey, 2010).
232
2.2. Teaching Strategies

English Language Teaching (ELT) has changed profoundly in the past decades
thanks to the work of applied linguists, the developments in teacher training
methodology, and the creative approaches of classroom practitioners (Richards,
2015; Herrell & Jordan, 2020). Hinkel (2006) described four themes that have
changed policy makers’ and teachers’ perspectives on how English as a Second
Language (ESL) or English as a Foreign Language (EFL) can be taught more
effectively. These themes are listed as the decline of methods, the need to use
bottom-up and top-down skills, the incorporation of new knowledge on the
English language itself, and an acknowledgment that language skills need to be
taught in context and in an integrated manner. Hinkel (2006) emphasized that
“these factors have had a profound influence on classroom instruction and
curriculum development in practically all L2 skills and across learner proficiency
levels” (p. 110).
Another trend that has evolved is the result of the realization that 21st-century
language teaching has to prioritize elements that predispose learners to lifelong
learning and engagement. This implies the integration of the so-called 4Cs
(communication, critical thinking, creativity, and collaboration) into the four
language skills: listening, speaking, reading, and writing (Erdoğan, 2019;
Chiruguru, 2020). The list can be extended by adding digital literacy (Dudeney,
2015), citizenship and student leadership, and personal development (Norris,
2019).
2.3. English Language Teaching Strategies in Ecuador

The 57-page in-depth report prepared by the British Council (2015) examined the
policies, perceptions, and factors that influenced the teaching of English in
Ecuador since the early 1990s. The Ministry of Education and Culture and the
British Council launched the so-called CRADLE project in 1992 to reform the
English curriculum in public and semi-public schools (the latter run by the central
government and religious orders). The report emphasized that CRADLE aimed at
focusing on the four language skills (rather than using the grammar-translation
method exclusively), included the designing of a series of textbooks that were
appropriate in the Ecuadorian context, and provided technical assistance to the
English teachers employed in the public education sector.
English language policy in Ecuador was reformed in 2012 (Macías Mosquera &
Villafuerte Holguin, 2020; Ministerio de Educación, 2013) and then again in 2016
(Ministerio de Educación, 2016a). In their study on English language teaching in
an urban setting, Burgin and Daniel (2017) focused on the language teaching
pedagogy of nine English teachers working at a public secondary school. The
observations identified many positives about the teachers’ instructional practices,
which the authors attributed to “the impact of systemic governmental movements
and expenditures” in the preceding years (Burgin & Daniel, 2017, p. 114).
However, they also pointed out that the instruction at the school was still mainly
teacher-led, and critical thinking paradigms and learner empowerment were not
in evidence.
233
2.4. Recent Relevant Literature

There have been several recent studies conducted in teaching English in the public
sector in Ecuador. In this respect, Calle et al. (2012) and Halim (2017) concluded
that English proficiency levels were low due to English teachers’ traditional
classroom methodology (e.g., grammar-translation). Ortega and Minchala (2019)
also drew attention to the low levels of English performance at secondary level,
both in urban and rural areas. Suasnavas (2018) recommended that the teaching
practices in public schools need to be examined in order to determine the specific
weaknesses of the teaching/learning process, as well as to promote the
implementation of new teaching strategies. The above-mentioned studies also
refer to the challenges that teachers face since they find the implementation of the
latest curriculum very demanding, mainly because the material contained in the
course books cannot be covered owing to the insufficient number of English
classes in primary and secondary schools.
2.5. Teacher Training

In-service teacher training has always been considered an essential element in the
professional development of ELT teachers (Dikilitaş & Erten, 2017; Richards &
Farrell, 2005). From Estonia (Raud & Orehhova, 2017) through to Turkey (Koç,
2016) and Colombia (Buendía & Macías, 2019), there is widespread agreement that
there are common elements that need to be included in the training of teachers,
and some of these are specific to teachers in an EFL setting, such as the
improvement of language proficiency. Based on 25 empirical studies conducted
in Colombia, Buendía and Macías (2019) included a comprehensive list of aspects
to be considered (beyond language proficiency): “research skills and reflective
practice, teachers’ beliefs and identities, an integrated approach to teacher
professional development, pedagogical skills and teaching approaches, and
emerging technologies”(p. 98).
In the Ecuadorian context, relevant literature also points to the need for
continuing professional development (Burgin & Daniel, 2017) leading to new
instructional approaches, such as problem-solving pedagogy and co-designing
“new ways of interacting” (p. 121), especially in multilingual and multicultural
settings – the reality of Ecuador today. Suasnavas (2018) likewise drew attention
to the need for sustained teacher training programs. Having found low levels of
English performance at secondary levels both in urban and rural areas, Ortega
and Minchala (2019) also recommended that regular in-service training programs
should be set up. Meanwhile, Soto et al. (2020) emphasized the need for the
enhancement of student-centered approaches, adding that transformation will
only start in class when teachers have the skills and pedagogical knowledge to
apply language teaching strategies and techniques correctly. Finally, Halim (2017)
stressed the need for improved training courses for pre-service teachers, the
designing of which would require the re-thinking of the language teaching
methodology component of teacher education.
234
3. Methodology
3.1. Research Design
The present exploratory study was carried out in the second phase of a large-scale
research project, which ran from September 2018 to August 2020 with funding
secured through the National University of Education, Ecuador (known as UNAE
by its Spanish acronym). The researchers aimed to carry out “a broad-ranging,
purposive, systematic and prearranged undertaking designed to maximize the
discovery of generalizations leading to description and understanding of social…
life” (Vogt, 1999. p. 105). Translated into the present research project, the intention
was to obtain exploratory qualitative data through in-depth interviews with
teachers from three provinces of Ecuador, namely: Azuay, Cañar, and Morona
Santiago. These three provinces were selected because they belong to the same
administrative region (Zone 6) and the university (UNAE) is situated within this
political and geographical area. The three neighbouring provinces cover a
substantial area of the country, and have a large number of teachers working in
public education institutions. The issues explored in the interviews and contained
in the interview guide were based on the themes that had been identified in
several round-table discussions held with a group of 40 teachers preceding the
recording of the interviews (see Appendix 1).
3.2. Participants
To gain deeper insights, the researchers conducted in-depth interviews with 10
teachers out of the full cohort of 40 that participated in the round-table
discussions. All teachers had a B2 (CEFR) level of English and were teaching in
public schools at the time when the research was carried out. Four of them worked
in primary schools, two of them were employed in secondary schools and four of
them were ‘hybrid’ teachers, who worked at both primary and secondary levels
at the time the interviews were conducted. The teachers participated voluntarily
and were informed that they could withdraw from the project at any point in time.
They each signed a consent form stating that they were participating voluntarily.
Furthermore, the names of the participants and their educational institutions were
kept anonymous (referring to them as T1, T2 … T10).
The two pre-conditions that the researchers had set were that the teachers needed
to have a minimum of two years of teaching experience and they needed to have
participated in the first phase of the study, which meant that they had already
taken part in the round-table discussions. The reason for this requirement was
that the teachers were expected to be familiar with the initial themes arising from
the round-table discussions, following which the researchers aimed to learn more
about the teachers’ feelings, perceptions and beliefs via the in-depth interviews.
The 10 interviewees, who were selected from the 40 participants, exhibited the
defining characteristics that were deemed important for the exploratory part of
the present research. Furthermore, the 10 selected teachers were cooperating
teachers in different programs, such as UNAE’s Practicum and Community
Outreach Programs.
The interviews were conducted in Spanish because this allowed the participants
to discuss complex issues in the mother tongue without the pressure of having to
235
use L2 for the exploration of their ideas. Their contributions were translated from
Spanish into English by the investigators of the present research study. The
translations were prepared independently by two of the three authors and then
compared and matched for accuracy.
3.3. Data Analysis

The in-depth interviews were transcribed and then analyzed by putting the
material through the MAXqda qualitative analysis software. MAXqda is
considered to be a useful tool, suitable both for data processing and the coding of
qualitative data (Godau, 2003). The process resulted in a list of categorized topics,
which allowed the grouping of related ideas together. For the purposes of the
present study, the researchers focused on three main topics:
• The importance of English as perceived by teachers working in primary

and secondary public schools;
• Teaching strategies applied by teachers in order to achieve the goals set
out by the curriculum; and
• The degree to which in-service teacher training is seen as helping teachers
to achieve their goals.
4. Findings
This section discusses the findings related to the above-mentioned three issues:
the importance of learning English, teaching strategies, and in-service teacher
training. These three topics are intertwined and their relationship is further
examined in the discussion section. All quotes below were taken from the
categorized topic grid that was created as the result of the qualitative analysis
accomplished by the MAXqda software.
4.1. Perceived Importance of English in Basic and Secondary Education

Ecuadorian English teachers appear to be acutely aware of the importance of
learning English as a global language. One of the reasons mentioned is its
significance for the tourist sector and foreign trade relations. Another reason is
related to the fact that students need English for their further studies, including
MA and Ph.D. programs, often combined with opportunities to win scholarships.
The potential of higher earnings by professionals who speak English is also
acknowledged. Teachers stressed that English is an international language, which
is also used most extensively on the Internet and social media. Altogether, eight
teachers gave specific reasons why they thought that English is important for their
students. In the interviews with T8 and T10, there was no mention of this aspect,
while the rest of the teachers cited several arguments in favor.
236
Table 1. Why is English Important?

Arguments: Mentioned by:
Economic reasons:
• Tourism
T1, T7, T9
• Foreign trade relations
• Transmission of business
ideas
Academic reasons:
• Essential for further studies
• Useful when doing MAs or
PhDs abroad T1, T2, T5, T6
• Valuable when applying for
scholarships in-country or
abroad
Global communication reasons:
• Access to the Internet and
social media T3, T4
• Enables user to communicate
internationally
Other reasons:
• Better professional jobs
T2, T5
• Higher earnings
4.1.1. Teachers’ Voices

While teachers appeared to be adamant that learning English is paramount for
their students, some interviewees believe that their students do not treat English
as a priority and neither does Ecuadorian society at large:
I think we live in a society that does not attach too much importance to
English and, as a result, our students don’t think it is important, either.
They don’t do their homework, but this is true for other subjects, too. (T2)
Another teacher highlighted the socio-economic circumstances that seem to lie

behind the lack of motivation:
The students in our high school come from dysfunctional families, in
most cases, the parents live in the United States, so these kids are kind
of orphans who live with their grandparents and all they want is to get
their school-leaving exam certificate. (T5)
Overall, teachers do recognize the importance of learning English, but this does
not seem to be reflected in their students’ level of proficiency. To discover the
possible reasons for the apparent mismatch between teacher perceptions and
learning outcomes, it is crucial to take stock of the teaching strategies that teachers
claim they employ in their classrooms.
4.2. Teaching Strategies

The primary and secondary teachers who were interviewed gave an account of an
array of teaching strategies that they use. Table 2 summarizes the approaches and
tools that have been mentioned in the course of the interviews.
237
Table 2. Teaching Strategies Employed in the English Classroom

Teachers’ Mentioned
Strategies Tools
remarks by
Teaching the 4 skills
CDs
T5
Using own mobile and mini-
No Internet,
Listening speakers T7
daily use of CDs
Language lab to watch movies T9
and songs
Active use of language
T7
Speaking (greetings, classroom
instructions)
Students don’t
always
Using the textbook and
Reading understand the T9
provide extra information
text or find it
useful
Dictation in group work,
Writing T6
active practice
Methodology
Introducing new Relying on previous E.g., Discussing

T4
material knowledge Ecuadorian
traditions
T6
Interactive whiteboard
Presentation of new T3
Using visual materials,
material
images, photos
T1
Videos
Systematic revision Last lesson of T3

the week
Personalized
T2
attention
Practice and
Worksheets T1
assessment
Vocabulary
Flashcards T5
expansion
Appealing to
Games T2
cooperative and
Bingo T2
competitive skills
Extracurricular activities
Presentations Singing songs on Open Days T6
Greetings,
Using English in the
numbers, T2
school playground
instructions
Theatrical
performance and T3
singing on the patio
238
Alongside the teaching strategies involving methodological approaches and

specific tools or activities, the teachers also described the challenges that they face
in their daily work:
• Class size: 35-40 students in a group
• Crowded classrooms (often too hot or too cold)
• Class hours: 3-5 hours per week (40-minute classes) - not enough time to
practice and internalize the material
• Lack of exposure to English outside school
• Unrealistic and unachievable curriculum
• Assessment: English is not among the subjects of the national school-
leaving exam
• Overworked and underpaid teachers (low social status)
• Lack of support from parents
• Low levels of knowledge in Spanish (especially in the case of indigenous
students, for whom Spanish is a second language)
• Students’ lack of motivation to learn English (or other subjects)
• Regulations favoring students (leniency)
• General issues with discipline (students playing and fighting with each
other rather than paying attention)

As has been stated before, one of the main tenets of the 2016 curriculum is the
application of what is widely termed as Communicative Language Teaching or
CLT (Ministerio de Educación, 2016a), an approach that also encourages group
work and the use of cooperative learning. Some of the teachers interviewed were
in favour of using the activities arising from the principles of CLT:
I like to use the communicative method, especially role plays, where they
can apply straight away what they have learned… And what I like best is
cooperative learning where the students can help each other… This is
because we have a great disadvantage in public schools: we may have 35
and even 40 students in the classroom and it is impossible to deal with
everyone individually. But when we use cooperative learning, the
students work together. (T4)
The teachers used group work or pair work as well, whereby they utilized the
multi-level groups to their students’ advantage:
When they are using worksheets, the older ones work in pairs or groups,
especially, when we have a double class. But it is the same with the
younger ones, because they can help each other in case one of them did not
understand what I said and the other understood it first time round. (T1)
Many teachers believe that the amount of English input is insufficient to gain an
acceptable level of proficiency because English is a foreign and not a second
language in Ecuador, which means that there are only few opportunities to
practice the language outside class. On average, primary school students have
three English classes of 40 minutes, this increases to five classes of 40 minutes in
Year 8 and 9, but is decreased again to three hours per week in Year 10 when
students are preparing for their final school-leaving exam. A further challenge
239
might be that it is difficult to cater for students who are more advanced than the
majority of the students in a given class:
In my opinion, we should have more English classes … perhaps we could
take away a couple of hours from other subjects. Also, we would need to
do some streaming as it is done in other institutions because at times you
can have an advanced student and a beginner in the same class. (T3)
Apart from not having enough classes, teachers often feel that the material that
they need to cover is far too much:
We are told we have six weeks to finish one unit in the book, but each unit
has six topics and six sets of tasks, so it is practically impossible to finish
one topic in one week. In Ecuador we often do not have six weeks of
teaching without a break, so we do not have enough time for practice…
We then straight away have to start a new unit, which is often not
connected at all to the previous one. (T4)
The following aspect to be looked at is the issue of the set course book and how
far teachers feel they can identify with it. Some of the respondents of the research
study were not thoroughly pleased with the course book. Teachers’ dislike for the
textbook they are required to use may be rooted in the fact that they feel the topics
dealt with are far removed from the local context, as was stated by this secondary
school teacher:
Sometimes the topics are a little complicated and they sound a bit strange,
too, and in actual fact, I don’t like them much at all… (T1)
On the whole, it appears that primary and secondary school English teachers use
a variety of teaching strategies (which are mostly in line with the CLT approach)
as well as a range of activities and tools, while they are facing a number of
challenges that characterize not only Ecuador, but also other countries in Latin
America (Rojas & Hernández-Fernández, 2018).
4.3. In-service Teacher Training

Meaningful and appropriate teacher training is one of the most important
elements that can improve the quality of the teaching-learning process. In-service
training has the advantage of providing support to teachers who already have
class experience and can transfer knowledge into their classrooms without any
delay. While there have been serious efforts to reform how English is being taught
in Ecuador, it seems that teacher training, and especially in-service teacher
training, has been lagging behind. Table 3 demonstrates the various types of
training that the interviewees had access to over the period when the research
study was conducted.
240
Table 3. Types of In-service Training for English Teachers in Public Schools

Providers of teacher
Availability in 2018-
training
2019
in 2016-2017
Provincial English
Language Coordinators No
Training sessions at the
school Publishers (Cambridge
University Press, Oxford Yes
University Press)
Training courses at state University of Cuenca No

universities UNAE Yes
Scholarships Go Teacher No
External sources Peace Corps volunteers Yes
Self-initiated professional Online courses Yes

development

Over the past years, Ecuadorian teachers could see how English as a Foreign
Language (EFL) has gained increasing importance in this Latin American country
(British Council, 2015), and how the official acknowledgment of those changes has
led to a new set of requirements. English is perceived as an important school
subject for the country’s social and economic development and the expectation is
that language teachers should have the right qualifications:
Some time ago anybody could teach English, even if they did not speak the
language well, because nobody took much interest in it as a subject. But
now we are being evaluated all the time and training sessions are also
being set up. (T5)
However, some teachers mentioned that there were no systematic teacher training
sessions in their schools; one of them added that this might have been due to the
fact that s/he was a new teacher at the school. It is worth mentioning that up until
2016, there was another course book series in circulation titled Our World through
English (Rosero et al., 2004), and it appears that at that time teacher training
sessions were organized in a more systematic fashion at provincial level:
We had regular training sessions because there were coordinators in each
province. Ours was very active, he called us every month for a training
session and we got to know all the other English teachers as well. He chose
some topics from the book, but he also added others and taught us
grammar, too. (T5)
Besides the frequent sessions provided by the provincial English Language

Coordinators, the state university of the regional capital, Cuenca, also played an
important role in supplying in-service training. Owing to recent government
policies (leading to budget cuts in education), teacher training programs tapered
off at a time when paid-for courses for professional development are often beyond
241
teachers’ means, especially after the salary cuts introduced in the wake of the
COVID-19 epidemic:
Four-five years ago we had training sessions at the State University of
Cuenca and I always went to the workshops there. With this new
government so much has been lost: now there is no state institution that
carries out this task, and private courses are not available to teachers
because of the high cost. (T9)
Teachers also said that even when there are training sessions, they are often
unable to attend because of the high number of contact hours and the clashes in
their timetables (T10). One of the ten teachers mentioned that s/he benefited a lot
from the so-called Go Teacher program, which began in 2012 as a partnership
between Kansas State University and Ecuador’s governing body of higher
education, the National Secretary of Higher Education, Science, Technology and
Innovation (SENESCYT). The program provided an opportunity for
approximately 1000 Ecuadorian English teachers to improve their language and
teaching skills for seven months at Kansas State University and other partner
institutions in the US:
I was one of the teachers who won a scholarship of the Go Teacher
program, it went well for me, I was able to finish my studies and do all
the tasks and participate in all the projects that were set for us. (T9)
In sum, the respondents’ perception is that while there is a requirement to employ

properly trained English teachers, whose performance is scrutinized more closely
than before, the range and quality of the training sessions may vary. Furthermore,
multiple respondents highlighted the fact that training sessions had been better
organized and more frequent before the severe budget cuts in the education sector
were implemented in recent years.
5. Discussion
English language teaching education has been a central issue for educational
policy in Ecuador for almost a decade (Ministerio de Educación, 2013; 2016a;
2016b). However, recent results suggest that for language policy changes to
succeed, policymakers should take into account the challenges encountered by
English teachers in their daily practice (Acosta & Cajas, 2018).
The present exploratory research study found that many of the challenges that
were identified by recent studies have been confirmed by the participating 10
English teachers. Low proficiency levels were mentioned by several teachers,
corroborating the findings of Calle et al. (2012) and Halim (2017). These authors
concluded that one of the reasons for poor performance is the fact that teachers
use traditional classroom methodology. As opposed to this, the teachers in the
present survey were able to refer to an array of classroom tools and pedagogies
that they were applying in order to make learning more effective. However, the
teaching strategies applied are often limited by a number of challenges in the
classroom, such as lack of motivation, insufficient number of English classes, and
the fact that for indigenous students English is not the first, but the second
additional language.
242
The primary and secondary school teachers interviewed for the present
exploratory study are fully aware of the importance of learning English as a
Foreign Language. They recognize that Ecuadorian students need to be able to use
English proficiently for trade and tourism, further studies abroad, and better-paid
jobs and professional advancement. This, to some extent, is in line with the British
Council report (2015), which surveyed the reasons given by learners as to why
they studied English. While a high percentage of students mentioned that they
studied English because it was compulsory in primary and secondary school (34%
and 52%, respectively), they also mentioned that they needed English for their
university studies or in order to improve their employment prospects, and have
access to information in English.
One can argue that the low levels of language proficiency in the public education
sector in Ecuador are not so much the result of ill feelings about the language itself
(Rojas & Hernández-Fernández, 2018), but can, to some extent, be explained by
the challenges that indigenous students face when studying not a second but a third
language. Rojas and Hernández-Fernández (2018) are clear about the tasks that
Latin American governments need to set themselves so that they can ”... provide
indigenous communities education services that integrate them into society (by
teaching in the official language); and ... respect and preserve their identity by
providing a curriculum adapted to their mother tongue and ethnic customs” (p.
19). In fact, the British Council report (2015) makes a mention of how diversity
affects English learning in Ecuador when it says that being educated in the mother
tongue is enshrined in the Ecuadorian Constitution, however, this means that
education systems can differ from region to region, which “presents a challenge
for the implementation of national education reforms and English language
learning initiatives” (p. 53). Some of the interviewees have indeed mentioned how
these challenges manifested themselves in their classrooms.
The teaching strategies applied by the 10 teachers appear to be varied and in line
with the basic tenets of Communicative Language Teaching, although Cajas and
Acosta (2018) emphasized that in Ecuador, ”the nature, purpose and use of the
CLT as the pedagogical approach of the CEFR is apparently not well understood
and interpreted” (p. 102). This is not a unique phenomenon. For example, Mai
(2007) stresses that even though teachers in Vietnam recognize the importance of
communication skills, they still feel the need to accommodate the local context, in
which more traditional approaches are deemed either useful or are probably
expected. Similar issues, such as the employment of teacher-fronted non-
communicative activities, have been raised in Japan (Sakui, 2004), South Korea
(Beaumont & Chang, 2011), Iran (Sarab et al., 2016), and Thailand (Promtara &
Suwannarak, 2018).
The teachers who were interviewed usually give due emphasis to the four basic
skills of listening, speaking, reading and writing and use the tools that are at their
disposal (CDs, mobiles, language labs), and furthermore, they utilize the
opportunities that extracurricular activities can offer, such as using English in the
school playground or setting up theatrical performances and karaoke singing
contests. However, it seems that they are only incorporating two of the so-called
243
4Cs (communication, collaboration, creativity, and critical thinking), namely,

communication and collaboration, while critical thinking and creativity do not
appear to be integrated into the teaching-learning process. The development of
these two skills might be perceived as a need for improvement, since the 4Cs are
considered to be essential skills (Chiruguru, 2020; Erdoğan, 2019) because the
knowledge of the four basic language skills, namely listening, speaking, reading
and writing is not considered sufficient any longer. As Chiruguru (2020) put it
succinctly when he said “if today’s students want to compete in this global society,
… they must also be proficient communicators, creators, critical thinkers and
collaborators” (p. 2).
The challenges that the teachers face are typically shared in the whole Latin
American region (Carrier, 2018) including: large classes, too few class hours, lack
of exposure to English outside school, limited resources, overworked and
underpaid teachers with low social status. Ecuadorian teachers are aware that
many of their students are socio-economically disadvantaged, but they also
mention a general lack of motivation as well as some discipline issues.
Furthermore, the interviewees often seem to struggle owing to the demands of the
curriculum and the difficulty level of the course books. These challenges are
acknowledged in the British Council report (2015), which additionally highlights
the shortage of properly-trained and experienced English language teachers.
Recent studies (Suasnavas, 2018; Ortega & Minchala, 2019; Soto at al., 2020) have
recurrently pointed out that in order to achieve better learning results, sustained
teacher training programs are required and these should involve training courses
both for pre- and in-service teachers. Based on the teachers’ account, it seems that
in-service training opportunities for teachers have dwindled in the past couple of
years because of a series of austerity measures that have been introduced in
Ecuador since 2018. In 2020, the country’s public debt reached almost 70% of its
GDP, and the government cut public spending for education steeply (USD 142m
in 2019 and 578m in 2020). Teachers’ salaries were reduced by 20% on average
(Gómez Ponce, 2020; Iturralde, 2020). The severe cuts have probably affected the
quantity and quality of opportunities for teachers’ continuing professional
development. Teachers used to participate in training sessions provided by
provincial English Language Coordinators, and Cuenca State University also
offered a range of teacher training courses but these have ceased to exist. These
tasks, to some extent, have been taken over by UNAE (the other state university
of education in Azuay province), however, teachers are not always able to
participate owing to the high number of contact hours and clashes with their
timetables.
Altogether, in-service teacher training is sporadic and less effective than before
2017. The Go Teacher program, which allowed approximately 1000 English
teachers to spend about 7 months in the US studying at several universities
between 2012 and 2016, may have had a beneficial effect on teachers’ effectiveness,
but to the best of the researchers’ knowledge, so far there has been no research
conducted on its success. The British Council report (2015) cites anecdotal
evidence which suggests that “the exposure to native English teachers and culture
244
gained through the programme is invaluable and that teachers have returned with
improved English language and pedagogical skills” (p. 54). However, it is unclear
how these positive outcomes may have been translated into more effective English
teaching, first and foremost, in rural areas. Many of the teachers that took part in
the Go Teacher program, especially those who were selected to study for MA
degrees, have by now moved to tertiary education, and even though this might
have a ripple effect on English teaching in the future, most of these teachers do
not presently operate in low-resource primary and secondary classrooms.
6. Conclusion
The perceived importance of English, classroom strategies, and in-service teacher
training are interrelated issues. The changes introduced in the 2016 English
language curriculum have brought about mixed results so far since Ecuador still
ranks the last out of 19 Latin American countries when it comes to English
language proficiency. The 2016 curriculum may have been well-prepared, but its
implementation has not been an unequivocal success from the point of the EFL
teachers involved in our study. This is because they find that many of the
objectives of the curriculum are unrealistic and unachievable. The participating
primary and secondary school English teachers in the public education sector are
conscious about the importance of English for the advancement of their students.
They are prepared to apply the approaches and tools that they are familiar with,
but they lack the skills and knowledge to apply new instructional approaches
regularly. This implies that some of the 21st-century language skills, such as
creativity and critical thinking, do not have much room in the English classes of
the teachers interviewed. At the same time, teachers are mindful of the need to
improve their skills, which would require effective in-service teacher training
programs. The English teachers of the current study believe that training
opportunities have dwindled in recent years. They would like to see educational
policymakers provide regular and expanding opportunities for their continuing
professional development. The findings of this exploratory study are, therefore,
in line with those of the recent studies discussed above and draw attention to how
exploratory research projects can identify and reiterate systemic issues to be
resolved in order to achieve better performance levels in the Ecuadorian public
education sector.
7. Limitations
This qualitative research study was conducted as a segment of a comprehensive,
two-year project (between September 2018 and August 2020 – now extended),
characterizing the different factors that facilitate or impede English learning in
Ecuadorian public schools and, as such, can throw light only on a limited number
of issues that surfaced in the exploratory phase. Furthermore, the 10 teachers
whose responses were explored above, came from only three neighboring
provinces of Ecuador representing approximately 10% of the country’s
population (Azuay, Morona Santiago, and Cañar). Owing to this, the opinions
expressed by these primary and secondary school educators cannot be taken to
represent the views of all English teachers in the country.
245
A ‘sister study’ examined several other issues, such as the use of L1 in the
classroom, the inclusion of special needs students in the teaching-learning
process, and education policies (Sevy-Biloon et al., 2020). Therefore, the present
study explored three other interrelated issues: the perceived importance of
English, teaching strategies, and teacher training. As a result, the findings of the
present research study cannot be generalized beyond the tentative conclusions
that could be drawn from the interviews with 10 English teachers working in the
public education sector (both in primary and secondary schools).
8. Recommendations and Further Research

The teachers’ candid comments made in the interviews cannot be expected to
provide an all-round picture, but can probably serve as a good starting point for
further research that may aim to identify suggestions and recommendations as
remedies to some of the problems that have been revealed. Regular and improved
in-service teacher training schemes, with special regard to the fact that instruction
has largely switched to emergency remote teaching since March 2020, could
enhance the teaching-learning process of English. Setting up national and regional
teacher associations could promote camaraderie, and professional development
while classroom research, which is considered an accessible research approach for
teacher-practitioners (Smith & Rebolledo, 2018), could enhance teachers’
engagement.
A further area of study might be investigating the effectiveness of the state

scholarship system, especially that of the Go Teacher scheme. Such an inquiry
could establish how far (if at all) it has contributed to improved classroom
practices which, to date, do not appear to have been translated into higher levels
of proficiency among learners of English in Ecuador. Hence, it is hoped that the
findings of the present research study will be perceived as a meaningful
contribution when future policies for EFL instruction in Ecuador are designed and
implemented, and the resulting process will lead to more effective and efficient
ways of teaching and learning English as an international language for the 21st
century.
Acknowledgments
The authors would like to thank all the participants of this research study for
having supplied a wealth of data including their considered opinions on the topics
discussed herewith. They are also grateful for the research and logistical support
provided by the National University of Education, Ecuador (UNAE). The valuable
comments and insights of the editors and reviewers, which significantly improved
the manuscript, are also gratefully acknowledged.
Notes
1- The project, which ran from September 1, 2018 to August 31, 2020 (its
extension beyond two years approved by the university) was entitled
Caracterización de los factores que impiden o facilitan la enseñanza y
aprendizaje de inglés como lengua extranjera en las escuelas de educación
básica y bachillerato de la zona 6: Un estudio exploratorio [A characterization
of the factors that impede or facilitate the teaching and learning of English as
246
a Foreign Language in public schools in Zone 6 of Ecuador: An exploratory

study]. Project code: VIP-UNAE-2017-55.
2- This is a role that has, to some extent, been taken over by the university whose
researchers conducted the present research project.
3- On average, teachers’ salaries were cut by 20%.
4- The Go Teacher program involved about 14% of the 7000 English teachers
working in the public education sector. See at
https://global.k-state.edu/dayofecuador/docs/go-teacher-fact-sheet.pdf
https://www.eltelegrafo.com.ec/noticias/sociedad/1/ecuador-ocupa-el-
puesto-55-en-el-dominio-del-ingles
9. References
Acosta, H., & Cajas, D. (2018). Analysis of teaching resources used in EFL classes in
selected Ecuadorian universities. Indonesian Journal of Applied Linguistics, 8(1),
100-109. https://doi.org/10.17509/ijal.v8i1.11469
Beaumont, M., & Chang, K. (2011). Challenging the traditional/communicative
dichotomy. ELT Journal, 65(3), 291-299. https://doi.org/10.1093/elt/ccq091
British Council. (2013). The English effect. https://www.britishcouncil.org/research-
policy-insight/policy-reports/the-english-effect
British Council. (2015). English in Ecuador: An examination of policy, perceptions and
influencing factors. https://www.teachingenglish.org.uk/sites
/teacheng/files/English%20in%20Ecuador.pdf
Buendía, X. P., & Macías, D. F. (2019). The professional development of English language
teachers in Colombia: A review of the literature. Colombian Applied Linguistics
Journal, 21(1), 98-111. https://doi.org/10.14483/22487085.12966
Burgin, X., & Daniel, M. (2017). Exploring English language teaching in an Ecuadorian
urban secondary institution. GIST Education and Learning Research Journal, 14,
107-134. https://doi.org/10.26817/16925777.364
Calle, A. M., Calle, S., Argudo, J., Moscoso, E., Smith, A., & Cabrera, P. (2012). Los
profesores de inglés y su práctica docente: Un estudio de caso de los colegios fiscales de la
ciudad de Cuenca, Ecuador [English teachers and their teaching practice: A case study of
public secondary schools in Cuenca, Ecuador].
https://doi.org/10.18537/mskn.03.02.01
Carrier, M. (2018). Improving English language proficiency in state schools: Policy and
systemic implementation. In H. Rojas & J. Hernández-Fernández (Eds.), English
public policies in Latin America: Looking for innovation and systemic improvement in
quality English language teaching (pp. 10-29). British Council Mexico.
https://www.teachingenglish.org.uk/sites/teacheng/files/RPD_Publication.pd
f
Chan, K. L. (2016). Power Language Index. Which are the world’s most influential languages?.
http://www.kailchan.ca/wp-content/uploads/2016/12/Kai-Chan_Power-
Language-Index-full-report_2016_v2.pdf
Chiruguru, S. B. (2020). The essential skills of 21st century classroom (4Cs).
https://doi.org/10.13140/RG.2.2.36190.59201
Cronquist, K., & Fiszbein, A. (2017). English language learning in Latin America.
https://www.thedialogue.org/wp-content/uploads/2017/09/English-
Language-Learning-in-Latin-America-Final-1.pdf
Crystal, D. (2003). English as a global language (2nd ed.). Cambridge, UK: Cambridge
University Press. https://doi.org/10.1017/CBO9780511486999
247
Dikilitaş, K., & Erten, I. H. (Eds.). (2017). Facilitating in-service teacher training for
professional development (1st ed.). Hershey, PA: IGI Global.
https://doi.org/10.4018/978-1-5225-1747-4
Dudeney, G. (2015). 21st century skills and digital literacy in action.
https://www.teachingenglish.org.uk/article/gavin-dudeney-21st-century-
skills-digital-literacy-action
Education First. (2019). EF English Proficiency Index. https://www.ef.com/assetscdn
/WIBIwq6RdJvcD9bc8RMd/legacy/__/~/media/centralefcom/epi/download
s/full-reports/v9/ef-epi-2019-english.pdf
Education First. (2020a). EF English Proficiency Index.
https://www.ef.com/ca/epi/regions/latin-america/ecuador/
Education First. (2020b). EF English Proficiency Index. https://www.ef.com/ca/epi
Erdoğan, V. (2019). Integrating 4C skills of 21st century into 4 language skills in EFL
classes. International Journal of Education and Research, 7(11), 113-124.
Godau, R. I. (2003). Qualitative data analysis software: MAXqda. Qualitative Research
Journal, 4(1), 66-72.
https://www.researchgate.net/publication/259185207_Qualitative
_Data_Analysis_Software_MAXqda
Gómez Ponce, L. (2020). Un año de reduccionesen el presupuesto para los sectores sociales.
https://www.gastopublico.org/informes-del-observatorio/2020-un-ano-de-
reducciones-en-el-presupuesto-para-los-sectores-
sociales#:~:text=En%202017%20fue%20de%20USD,anterior%2C%20con%20USD
%2032.530%20millones
Halim, A. O. J. (2017). Ecuador and bilingualism a “mirage or reality”: A critical look at
English as a foreign language. Universidad Politécnica Estatal del Carchi. Ecuador.
https://doi.org/10.32645/13906925.123
Herrell, A. L., & Jordan, M. L. (2020). 50 strategies for teaching English language learners
(6thed.). London, UK: Pearson Education Ltd.
Hinkel, E. (2006). Current perspectives on teaching the four skills. TESOL Quarterly,
40(1), 109-131. https://doi.org/10.2307/40264513
Iturralde, P. J. (2020). Ecuador, COVID-19 and debt.
https://www.eurodad.org/ecuador_covid19_and_debt
Koç, E. M. (2016). A general investigation of the in-service training of English language
teachers at elementary schools in Turkey. International Journal of Elementary
Education, 8(3), 455-466. https://files.eric.ed.gov/fulltext/EJ1096582.pdf
Macías Mosquera, K. Y., & Villafuerte Holguin, J. S. (2020). Teaching English language in
Ecuador: A review from the inclusive educational approach. Journal of Arts and
Humanities, 9(2), 75-90. https://doi.org/10.18533/journal.v9i2.1854
Mai, H. T. N. (2017). Contextual factors affecting the implementation of communicative
language teaching in Vietnam. EFL Journal, 2(2), 103-113.
https://efljournal.org/index.php/efljournal/article/view/40/pdf
McCormick, C. (2013). Countries with better English have better economies.
https://hbr.org/2013/11/countries-with-better-english-have-better-economies
Ministerio de Educación del Ecuador [Ministry of Education, Ecuador]. (2013). National
English curriculum guidelines. https://educacion.gob.ec/wp-
content/uploads/downloads /2013/04/Curriculum_Guidelines-EFL-1.pdf
Ministerio de Educación del Ecuador [Ministry of Education, Ecuador]. (2016a). English
as a Foreign Language. https://educacion.gob.ec/wp-
content/uploads/downloads/2016/03/EFL1.pdf
248
Ministerio de Educación del Ecuador [Ministry of Education, Ecuador]. (2016b).

Currículo de los niveles de Educación Obligatoria [Curriculum for the Compulsory
Levels of Education]. https://educacion.gob.ec/wp-
content/uploads/downloads/2016/03/Curriculo1.pdf
Norris, L. (2019). Promoting 21st century skills.
https://www.teachingenglish.org.uk/sites/teacheng/files/J105_12_Promoting
_21_Skills_web.pdf
Ortega, D. P., & Minchala, B. E. (2019). Explorando las Aulas de Clase de Inglés en
Cañar: Currículo, instrucción y aprendizaje. [Exploring English language
classrooms in Cañar: Curriculum, instruction and learning. CIENCIA UNEMI,
12(30), 57-73.
http://ojs.unemi.edu.ec/index.php/cienciaunemi/article/view/772
Promtara, N., & Suwannarak, K. (2018). Thai students and teachers’ perceptions of
learning and teaching English through the communicative language teaching
approach. NIDA Journal of Language and Communication, 23(33), 23-42.
https://so04.tci-thaijo.org/index.php/NJLC/article/view/135387/101154
Raud, N., & Orehhova, O. (2017). In-service training of teachers of English as a foreign
language in Estonia: Mapping of trends and opportunities. Problems of Education
in the 21st century, 75(2), 194-203.
Richards, J. C. (2015). Key issues in language teaching. Cambridge, UK: Cambridge
University Press.
Richards, J. C., & Farrell, T. S. C. (2005). Professional development for language teachers:
Strategies for teacher learning. Cambridge, UK: Cambridge University Press.
Rojas, J., & Hernández-Fernández, J. (2018). The importance of English as a second
language in Latin America. In H. Rojas & J. Hernández-Fernández (Eds.), English
public policies in Latin America: Looking for innovation and systemic improvement in
quality English language teaching (pp. 10-29). British Council Mexico.
https://www.teachingenglish.org.uk/sites/teacheng/files/RPD_Publication.pd
f
Rosero, S. I., Embleton, L., Rivera, I. M., & Ponce, R. R. (2004). Our world through English.
Quito, Ecuador: Edimpres.
Sakui, K. (2004). Wearing two pairs of shoes: Language teaching in Japan. ELT Journal,
58(2), 155-163. https://doi.org/10.1093/elt/58.2.155
Sarab, M. R. A., Monfared, A., & Safarzadeh, M. M. (2016). Secondary EFL school
teachers’ perceptions of CLT principles and practices: An exploratory survey.
Iranian Journal of Language Teaching Research, 4(3), 109-130.
Sevy-Biloon, J., Recino, U., & Muñoz, C. (2020). Factors affecting English language
teaching in public schools in Ecuador. International Journal of Learning, Teaching
and Educational Research, 19(3), 276-294. doi:10.26803/ijlter.19.3.15
Smith, R., & Rebolledo, P. (2018). A handbook for exploratory action research. London, UK:
British Council.
https://www.teachingenglish.org.uk/sites/teacheng/files/pub_30510_BC
%20Explore%20Actions%20Handbook%20ONLINE%20AW.pdf
Soto, S. T., Encalada, M. A. R., Sarmiento, M. A., Galimberti, J. V., & Cárdenas, S. S. E.
(2020). Understanding EFL students’ learning through classroom research: Experiences
of teacher-researchers. http://repositorio.utmachala.edu.ec/handle/48000/15504
Starkey, H. (2010). Language learning for human rights and democratic citizenship. In C.
Rosi Solé & J. Fenoulhet (Eds.), Mobility and localisation in language learning (pp.
79-106). Oxford, UK: Peter Lang.
249
Suasnavas, K. (2018). The use of technological tools: Movie Maker and weblogs to foster
social interaction in the classroom. Kronos – The Language Teaching Journal, 1(01),
45-52.
Vogt, W. P. (1999). Dictionary of statistics and methodology: A non-technical guide for the
social sciences (2nd ed.). London, UK: Sage Publications.
Appendix 1
Interview questions
1. What level or levels do you teach?

2. What is your most common way to teach an English lesson? Please,
describe and give examples.
3. During your English lessons, what is the time percentage you use to
explain, and what is the time percentage you let your students work
independently?
4. Do you have your students practice in pair and group work? Explain.
5. What are the factors that help or impede the student learning in the
classroom?
6. Do you use Spanish during your lessons? In one teaching hour, how much
Spanish do you use?
7. What advantages and disadvantages have you experienced with the use
of Spanish in the English lessons?
8. To what extent do you think your teaching practice facilitate or impede the
English language learning? Please, describe and give examples.
9. Do you use a textbook? Explain why.
10. To what extent do you think that the current educational policies
encourage or inhibit the English language learning?
11. How much importance is given to English learning at the institution where
you work? Do the students´ parents and the school academic authorities
give importance to the English learning?
12. Have you received any formal training by the Ministry of Education?
13. After the curriculum implementation in 2016, have you received any
training? Have you personally found any extra training for your
performance?
14. Is it important that Elementary and High School level students learn
English? What is the existing perception about this topic in the country?
Support your answer, please.
15. How do you use the existing infrastructure in the institution to teach
English?
16. What is the role of the infrastructure for learning English? Support your
answer and give examples, please.
Vol. 20, No. 3, pp. 250-264, March 2021
Constructivist Learning Amid the COVID-19

Pandemic: Investigating Students’ Perceptions of
Biology Self-Learning Modules
Aaron A. Funa and Frederick T. Talaue

De La Salle University, Manila, Philippines
https://orcid.org/0000-0002-6648-8825
https://orcid.org/0000-0002-6452-0641
Abstract. Modes of teaching and learning have had to rapidly shift amid
the COVID-19 pandemic. As an emergency response, students from
Philippine public schools were provided learning modules based on a
minimized list of essential learning competencies in Biology. Using a
cross-sectional survey method, we investigated students’ perceptions of
the Biology self-learning modules (BSLM) that were designed in print and
digitized formats according to a constructivist learning approach. Senior
high school STEM students from grades 11 (n = 117) and 12 (n = 104)
participated in a survey using a 3-point Likert-scale questionnaire
uploaded online through Google Forms. The survey results indicate that
majority of the students perceived the modules positively, suggesting
that aspects of the modules that were salient to students corresponded to
essential elements of constructivist pedagogies. However, during
interviews, students reported several difficulties in learning with BSLM
as it was constrained by, to name a few, the use of unfamiliar words, lack
of access to supporting resources, slow internet connection, and time
constraints. To address these problems, teachers reported that they gave
deadline extensions, complemented modules with other channels of
support, and used online and offline platforms for reaching out to
students to answer their queries and plan out their schedule for the week.
The findings across the data sources point to the complex demands of
emergency distance education that teachers, as curriculum designers and
enactors, need to bear in mind in order to craft productive pedagogies,
constructivist or otherwise, during this unprecedented time.
Keywords: constructivist learning; COVID-19 pandemic; curriculum

materials; scaffolding; student perception; pandemic pedagogy
1. Introduction
Our lives during this COVID-19 pandemic have become marked by a persistent
sense of uncertainty because of forced lockdowns and necessary quarantines
(Caggiano et al., 2020). The education sector has had to make numerous
©Authors
251
emergency adjustments to existing practices (Talidong & Toquero, 2020;

Williamson et al., 2020) in order to continue the provision of educational services.
The Department of Education (DepEd) in the Philippines offered two alternative
distance learning modalities, namely, pure online and hybrid (online and offline),
which were supplemented with contextualized and standardized self-learning
modules (SLM) crafted by teachers for each region of the country. In addition, the
DepEd generated the Most Essential Learning Competencies (MELC) as a way of
decongesting the curriculum and making schooling during the pandemic less
stressful (DepEd, 2020a). The MELC-aligned SLM is the backbone of distance
learning aimed at making education accessible to students, both online and
offline, most especially to those who live in far-flung rural areas without internet
access (DepEd, 2020b). The distribution of print and digitized SLM would
facilitate distance learning with minimized risks of COVID-19 infection among
students and school personnel. However, despite the trainings given to teachers
and the quality assurance provided for instructional material and module
development, the SLM received negative informal feedback. For example, it was
said that the modules required performance of complicated activities, included
complex vocabularies that needed to be memorized, and resembled passive
lecture formats. With such criticisms, it is easy to imagine the challenges students
experienced in understanding target concepts (Çimer, 2012; Gutierrez, 2014;
Sañosa, 2013). It now becomes imperative to investigate students’ experiences in
using the SLM in order to understand their interactions with new curriculum
materials and inform similar future curriculum development efforts.
In carrying out the K-12 Basic Education Program, the DepEd is mandated to
adhere to sound educational principles including, but not limited to, the
utilization of constructivist pedagogical approaches (DepEd, 2019a). Policy
standards that were set for new modes of learning delivery during the COVID-19
pandemic stipulated that the content of the SLM use constructivist, inquiry-based,
reflective, collaborative, and integrative pedagogical approaches (DepEd, 2019b).
In designing the modules, teachers utilized corresponding policy guidelines that
defined constructivism as a pedagogical approach that aims to develop learners
as active constructors of meaningful knowledge. Such an outcome could be
achieved by designing lessons that engage students in internalization, self-
reflection, and real-life problem solving (Bada & Olusegun, 2015). An appreciation
of students’ knowledge frameworks is integral to designing and developing
constructivism-oriented instructional materials that engender productive
learning (Çimer & Coşkun, 2018).
Constructivist learning theory positions students as active agents in the

construction of knowledge from meaningful interpretations of real-life
experiences (Bada & Olusegun, 2015; Doolittle, 1999; Gross & Gross, 2016). The
consequent constructivist pedagogies have been influenced by three strands of
constructivism—cognitive, radical, and social. While constructivist pedagogies
share core principles with the three strands, their peripheral principles may vary.
What makes a pedagogy constructivist is the assembly and rationale for the
inclusion of particular principles that have been proposed by theorists over time
(Doolittle, 1999; Van Bergen & Parsell, 2019). A consensus based on the general
constructivist theories and practices has resulted to the formulation of principles
252
that are essential to constructivist pedagogy (Bada & Olusegun, 2015; Doolittle,
1999), namely, (1) learning transpires in authentic and real-life settings; (2)
learning involves social negotiation and mediation; (3) content and skills are made
suitable to the learner; (4) content and skills are understood within the framework
of the learner’s prior knowledge; (5) students are assessed formatively, informing
future learning experiences; (6) students are encouraged to become self-regulated,
self-mediated, and self-aware; (7) teachers act as a guide and facilitator of
learning; and (8) teachers provide for and encourage multiple perspectives and
representations of content.
Principles of constructivist pedagogy have been adopted in numerous

educational innovations, including problem-based learning, project-based
learning, inquiry-based approach, and design thinking. Problem-based learning
has a significant and positive effect on students’ academic performance, achieving
higher marks on all assessments than the control group who experienced
traditional approach, according to a meta-analysis of studies, mostly from Asia,
conducted from 2016 to 2020 (Funa & Prudente, in press). Cakir (2017) examined
38 studies from 2006 to 2016 that utilized the 5E inquiry learning model and
showed a positive effect on student achievement. Furthermore, Ayaz and
Sekerci’s (2015) analysis of studies from 2003 to 2014 suggest the effectivenes of
constructivist learning approaches. These results are congruent to other meta-
analyses of constructivist approaches in science education that showed increases
in student academic achievement (Dochy et al., 2003; Toraman & Demir, 2016).
The adoption of constructivist principles in the design of curriculum materials for

use in emergency distance learning is an opportunity offered by the COVID-19
pandemic. The objective of the present study is to investigate secondary students’
experiences with the Biology Self-Learning Modules (referred to as BSLM
hereafter), whose development was informed by constructivist learning theory.
Specifically, we analyzed student’ perceptions of BSLM with respect to its: (1)
content, which pertains to the sufficiency and accuracy of conceptual information;
(2) learning-teaching activities, which refers to the various tasks and exercises
such as problem solving and knowledge application; (3) assessment and
evaluation, which pertains to encouraging and stimulating questions; and (4)
structure, which refers to the systematic and visual representation of the modules
which may be in digital or printed form. We also aimed to understand the context
of students’ and teachers’ use of BSLM in order to draw a more holistic picture of
the enactment of a pandemic pedagogy.
2. Methodology
2.1 Research Design
This study followed a survey descriptive research design (Creswell & Creswell,
2018) to investigate using a constructivist learning framework students’
perspectives on BSLM employed as an instructional material for emergency
distance education during the COVID-19 pandemic. The cross-sectional survey
method was implemented using Google Forms that were made accessible for two
weeks in order to give participants enough time to answer. The choice to use an
online data-gathering platform ensured the respondents’ health safety as they can
complete the survey questionnaire from home. The results from the survey were
253
triangulated with online interview data from students and teachers in order to
enhance credibility.
2.2 Participants
This study focused on Science, Technology, Engineering, and Mathematics
(STEM) students in grades 11 and 12 who were taking General Biology, which is
a 40-hour specialized subject in the K-12 curriculum offered to Senior High School
(SHS) students. It consists of sequential courses: General Biology 1, which focuses
on life processes at the cellular and molecular levels including energy
transformations, and General Biology 2, which deals with the topics of heredity,
variation, diversity, structure, function, and evolution of organisms.
A total of 221 SHS STEM students consented to participate in the study, with 53%
coming from grade 11, 85% from General Biology 1, and 77% using digitized
modules. We used intensity sampling for online personal semi-structured
interviews of students (n = 10) and their teachers (n = 11), both of whom used the
BSLM during the conduct of the study. This sampling method allows researchers
to choose participants that represent rich cases, manifesting the phenomenon of
interest for in-depth analysis (Patton, 1990). The interviews sought clarifications
and elaborations from students on their responses to the survey. Teachers were
interviewed about the BSLM design and their interactions with students as they
used BLSM, including any interventions provided.
2.3 Instrument
We adapted the 3-point Likert scale questionnaire developed by Çimer and
Coşkun (2018), which has a Cronbach’s alpha reliability coefficient of .94. This
instrument was designed following a constructivist learning framework and was
utilized to evaluate printed Biology textbooks. In this study, we modified the
original questionnaire to evaluate a similar curriculum resource, the BSLM, both
in printed and digitized forms. The modified instrument consists of four parts: (1)
content, 26 items; (2) learning-teaching activities, 30 items; (3) assessment and
evaluation activities, 15 items; and (4) structure (originally “physical design”), 14
items. The items in the first three parts of the instrument were retained with minor
changes in terminology; “textbook” was changed to “modules”. The items in the
last part were rephrased to appropriately refer to the two available BSLM formats.
For instance, we replaced the word “letterpress” with “text” since the former is
applicable only to the printed version. Similarly, references to size, weight,
durability, and paper quality were altered to refer to visual features instead.
The adapted questionnaire retained the 3-point scale for the same reason stated
by Çimer and Coşkun (2018); it offers a clearer interpretation of the findings.
According to Lehmann and Hulbert (1972), a 3-point Likert scale is good enough
for a study whose main interest is to obtain averages across a group of people and
not generate a behavior profile of individual persons. The adapted questionnaire
was evaluated and approved by eleven education experts composed of master
teachers and experienced Biology teachers.
2.4 Data Collection

We sought permission from various DepEd division offices across the country.
Only four division offices officially endorsed and disseminated the link to the
254
online survey through their respective science education program supervisors.

Observing ethical conduct of research, only students whose parent or guardian
signed the printed consent forms coursed through their teachers were included in
the study. The link to the online survey in Google Forms was sent to student-
respondents. Only students who have used either General Biology 1 or General
Biology 2 module for at least two months were included in the data gathering and
analysis. Interview data were collected after the preliminary analysis of survey
data. This allowed the crafting of specific and targeted questions to be used in the
online interview sessions.
2.5 Data Analysis

We used a 3-point Likert scale questionnaire (1-Disagree, 2-Undecided, and 3-
Agree) to obtain students’ evaluation of the modules according to a constructivist
learning theory. The ordinal data obtained from the questionnaire were subjected
to descriptive statistical analysis and presented accordingly: students’ views on
the content (A1-A26), learning-teaching activities (B1-B30), assessment and
evaluation activities (C1-C15), and structure (D1-D14).
3. Findings
In this section, we present the findings of the survey with respect to the four
aspects of the BLSM evaluated by students. We highlight the items with the high
and low percentage agreement in order to identify features of the curriculum that
could be retained and improved upon, respectively, in future iterations. Also, we
provide complementary interview data in order to clarify particular results and
provide a fuller understanding of contextual influences on students’ interaction
with BSLM.
3.1 Students’ Views on the Content of Biology Modules

The majority of the students perceived the content of the modules used during the
COVID-19 pandemic as fostering understanding of biology concepts, as shown in
Table 1. Many agreed that the visuals used (e.g., diagrams, pictures, tables,
graphs, figures, etc.) coordinated well with the topics and helped them interpret
information (A24, A2, A19). They also found the topic presentation to be well-
designed—being grade-appropriate and logically arranged (A22). The links
between topics (A4) were also noted by many, with almost two-thirds of students
agreeing that the module content exhibits interdisciplinarity (Jacobs, 1989) in its
appreciable association with other subjects like Physics and Chemistry (A1). In
addition, almost half of them agreed with the real-world applicability of the
module content (A6, A26, A12). Timeliness, suitability, and integrability of
discipline-based knowledge are given importance in constructivist approaches.
These features allow learners to cross boundaries between disciplines and
effectively apply their learning to solve authentic and real-life problems (Bada &
Olusegun, 2015; Doolittle, 1999; Gross & Gross, 2016).
255
Table 1: Students’ views on the content of Biology modules

Disagree Undecided Agree
Items*
(%) (%) (%)
A24 Pictures, graphs, figures, and tables in the 1 12 87
module are appropriate to the topics.
A2 Tables, graphs, or figures provided in the 4 13 83
module help us acquire and interpret
information.
A22 The topics in the module are appropriate to our 1 16 83
learning level.
A4 The topics are connected to each other. 3 15 82
A19 Descriptions are provided under pictures, 5 18 77
graphics, figures, and tables.
A25 Topics are arranged in a logical sequence. 4 26 71
A21 At the end of a topic, there are additional 7 24 69
resources that allow us to obtain more detailed
information.
A1 When necessary, the topics relate to those in 5 28 67
other subjects (Physics, Chemistry, etc.).
A5 The topics in the module are taught using brief 9 26 65
and concise information.
A6 When presenting the topics, current events are 9 26 65
used as examples.
A10 Essential knowledge related to the topics in the 5 30 65
module is distinctly emphasized.
A18 Summaries are provided at the end of sections. 11 24 65
A3 Topics are discussed in a detailed manner. 10 27 63
A8 The module is sufficient for us to learn basic 11 26 63
biological concepts.
A26 At the end of each topic, reading texts are 7 30 63
provided in the module that make it easy to
establish a connection with current events.
A15 The historical development of biology is also 10 29 62
discussed in the module.
A11 I must use other resources because the topics in 9 30 61
the module are superficially presented.
A16 The module includes numerous definitions. 15 24 61
A23 The information provided in the module is up 5 36 60
to date.
A12 I can use the knowledge I have acquired from 8 33 59
the module to solve biology-related problems
that I encounter in my daily life.
A9 Detailed definitions of biological concepts or 10 33 57
terms are provided in the module.
A14 The topics are enriched with examples from 10 39 51
daily life.
A17 The number of examples for each topic is 13 39 48
adequate.
A20 The module includes excessive and unnecessary 46 37 17
knowledge that I will not use in my daily life.
A7 The topics in the module are not presented 41 46 13
according to the class level.
A13 I often encounter inaccurate information in the 35 47 18
module.
Note: *Adapted from Çimer and Coşkun (2018).
256
In addition, a large percentage of students agreed that the module content is brief,
concise (A5), adequate (A21, A8), and presented in a detailed (A3) and logically
arranged manner (A25). Section summaries and links to additional resources
helped students deepen their understanding of topics. However, most students
were undecided about information accuracy (A13 = 47%), even as some assessed
the content to contain no misinformation at all (37%). Perhaps the former felt they
could not possibly make a competent judgment on this aspect. Although most
students found the content sufficient, still others viewed its presentation as
superficially because they would still need to use supplemental resources. Based
on interviews with teacher, the information could appear superficial because they
only included the MELCs. As the name suggests, MELCs are trimmed down
content representing the most basic topics that the DepEd determined students
needed to know during this time of pandemic learning.
Constructivist approaches do not require outrightly providing detailed

expositions of concepts to be learned. Instead, they guide students to integrate
prior and newly presented knowledge to construct meanings through searching,
analyzing, and interpreting (Çimer & Coskun, 2018). Thus, students are expected
to use various resources while teachers guide them in the discovery, construction,
and application of knowledge. This was done in BSLM through scaffolded lessons
with links to other resources provided to supplement learning. Specifically, BSLM
developers, as explained by one teacher interviewee, followed the 5E inquiry
teaching model (Bybee et al., 2006). It features an instructional sequence (Engage,
Explore, Explain, Evaluate, and Elaborate) that is used to practically implement
an implicit constructivist teaching and learning approach (Boddy et al, 2003;
Bybee, 1997; Bybee & Powell, 1993). Scaffolding features employed in BSLM align
with a constructivist approach and guide the development of students’ conceptual
understanding.
3.2 Students’ Views on the Learning-Teaching Activities of Biology Modules

A majority of the students viewed the BSLM as helping to stimulate their thinking
(B2, B21), enable self-expression (B18), encourage the use of other resources (B25),
and construct their own understanding (B5), as shown in Table 2. These results
correspond to what emerged in the survey as the most salient learning activity
features for students. Constructivism enjoins teachers: to encourage students to
be independent and accountable for their own learning; provide them multiple
perspectives and representations; and allow them to express learning in various
forms (Bada & Olusegun, 2015; Doolittle, 1999).
Table 2: Students’ views on the learning-teaching activities of Biology modules

Items*
(%) (%) (%)
B2 Activities and questions in the module stimulate 1 10 88
us to think.
B18 The activities in the module enable us to express 2 15 82
our ideas and opinions related to the lesson.
B21 Questions and activities in the module 3 15 82
encourage us to think about the topics and
interpret them.
257
B25 The activities in the module encourage us to 3 15 81

learn the subjects by researching other sources.
B5 Activities in the module encourage us to take 3 17 80
responsibility for our learning.
B20 The module serves as a guide that shows us 2 20 78
how to use the knowledge, skills, and
experiences that we have acquired from the
lessons.
B30 Application instructions for the activities are 2 20 78
provided.
B11 The module includes preparatory questions and 3 20 77
activities that attempt to gauge our existing
knowledge before the beginning of each unit or
topic.
B16 The activities in the module reinforce our 1 22 77
understanding and learning of the topics.
B19 The module includes activities that allow us to 6 17 77
look at daily life problems from different
perspectives while trying to solve them.
B26 The module contains various activities that 3 20 77
enable us to present our knowledge and skills in
different ways.
B4 The module includes activities that allow us to 5 25 70
participate in the lesson actively.
B15 The activities in the module allow us to learn 9 21 70
other students’ opinions and ideas on the topics.
B6 Activities in the module offer us different 8 24 68
learning environments.
B8 Activities in the module are suitable for our 4 29 67
ability level.
B1 The module offers problems to solve that are 8 27 65
related to daily life.
B24 The teacher must describe the activities in the 13 23 64
module.
B13 The provided experiments facilitate our 6 40 54
understanding of the topics.
B22 The activities in the module allow us to discuss 12 38 50
the subjects in class.
B29 The topics in the module are arranged in an 11 39 50
increasing order of difficulty.
B14 The module contains activities or questions 9 45 47
requiring us to design our own experiments.
B27 The methods to conduct experiments are 10 45 45
described in detail in the module.
B28 The module includes such activities as 13 42 45
structured grids or componential analysis
tables.
B12 The activities in the module enable us to acquire 21 36 43
knowledge without the teacher’s help.
B17 The module includes such activities as field 31 33 37
visits, observation, and investigations.
B23 The activities in the module are focused on 15 48 37
solving daily life problems or situations.
B3 Teachers’ role in the activities is minimal. 19 49 33
B10 Activities in the module require us to work in 26 45 29
groups.
258
B9 The time allocated for conducting activities is 18 41 41

not enough.
B7 We can easily carry out the laboratory activities 35 48 17
given in the module.
Based on the interviews, the teachers, using the 5E model, presented stimulating
problems at the beginning of each module to encourage students to recall past
lessons in order to activate prior knowledge. For example, some of the recall
questions in the Engage phase for the lesson Prokaryotic vs Eukaryotic Cells include:
“Have you experienced having an infected wound? How about a ringworm?
What do you think causes these infections/skin diseases? What is usually
prescribed when you have an infected wound? A ringworm? Are these two
caused by the same organism?” To enable expression of learning, teachers
provided activities in the Extend phase that solicited students’ insights by
completing prompts such as “I have learned that…” and “I wish to ask my teacher
about…” Furthermore, to enrich learning through varied representations,
students were engaged in different performance activities, such as singing a rap
song related to the cell theory, making a 3-D cell model, investigating enzyme
activities using chicken liver, and watching videos through weblinks provided in
the modules.
Remarkably, a little more than one-third of students unequivocally disagreed that

they can easily carry out laboratory activities (B7) whereas almost half of them
were ambivalent about this statement. In addition, half of the students agreed that
the time allocated for conducting activities is not enough (B9). These results point
to the changes in mode of learning due to the pandemic. The difficulty felt by
students in performing activities may be attributed to the unavailability of the
needed materials and equipment at home (Tria, 2020; Williamson et al., 2020). A
challenge to module designers is to create activities that use materials that are
easily sourced. However, according to teachers interviewed, they in fact designed
laboratory activities that could be performed using materials found at home (Tria,
2020; Williamson et al., 2020). They instructed students to ask their parents’
assistance to ensure safety when performing the experiments. Anticipating that
students would have questions and experience difficulties, teachers opened
communication channels through online group chats, text messaging, or phone
calls. They even gave deadline extensions and provided work plans with agreed
timelines to relieve some pressure due to time constraints.
Students were divided on the question of module activities requiring

collaboration with others (B10). According to constructivist approaches, learning
involves social negotiation and mediation (Bada & Olusegun, 2015; Doolittle,
1999). The social interaction is strongly related to students’ learning enjoyment
and may affect their attitude towards education (Marcial et al., 2015). During the
interviews, teachers said that they avoided group activities as much as possible
during the pandemic to prevent students from COVID-19 exposure and infection.
They only allowed group tasks for students who can perform online group
activities. However, in order for online group learning to be successful, there is a
need to minimize the cost of, and improve the quality of access to, the Internet
(Marcial et al., 2015; Tria, 2020; Williamson et al., 2020). This technological
259
infrastructure challenge is a systemic problem in a third world country like the

Philippines (Albert et al., 2016; Akamai, 2017) and is more acute in rural areas than
in urban ones. Given this situation, it seems that group work would have to be
paused until the pandemic ends. The problems students encountered in
performing tasks online, such as the unavailability of materials and constraints on
time and conducting group work, to some extent parallel the challenges
experienced by Pakistani students (Adnan & Anwar, 2020). The difficulties
perceived by students suggest the need to aim for a socially just digital
inclusiveness in designing modules (Williamson et al., 2020).
3.3 Students’ Views on the Assessment and Evaluation of Biology Modules

Most of the students had a positive appraisal of the assessment and evaluation
activities of BSLM, as shown in Table 3. They regarded the module questions as
letting them identify the level of their current knowledge (C2), prompting
thinking (C4, C1), and evaluating their own learning (C10, C11). These results
reveal that the most noticeable constructivist features for students are eliciting
prior knowledge, inducing thinking, and evaluating one’s own learning. Each
module was designed with pre- and post-tests, with an answer key provided at
the end of the module to allow students to evaluate their own learning. In
addition, students were also provided reflection questions—what they have
learned and what they still wanted to learn—to assess their current understanding
of the lesson.
Table 3: Students’ views on the assessment and evaluation of Biology modules

Items*
(%) (%) (%)
C4 The module contains questions that prompt 2 14 84
us to rethink and interpret the relevant
information.
C10 The module contains activities enabling us to 1 15 84
assess our own learning.
C2 The module contains questions or activities 3 14 83
that aim to clarify the level of our existing
knowledge at the beginning of a new topic.
C11 The module contains different types of 2 17 81
questions seeking to evaluate if we have
understood the subjects.
C1 Preparatory activities related to the topics 2 18 80
stimulate us to think and make inquiries.
C5 The module contains evaluative questions at 3 19 78
the end of each topic or section.
C7 The module contains questions that encourage 3 20 77
us to express our own ideas and experiences.
C14 The module contains questions requiring us to 6 25 68
interpret the tables, graph, or pictures
included within a topic.
C12 The module contains performance tasks that 5 30 65
we can mentioned in our portfolios.
C13 The module contains various questions, tasks, 8 29 63
or assignments that encourage us to conduct
research and investigation.
260
C15 We generally face difficulties in answering the 7 31 62

questions in the module.
C9 Preparatory questions or activities at the 13 36 51
beginning of the topics attract our attention.
C8 Performance tasks and assignments attract 18 33 50
our attention.
C6 The questions provided at the end of each 37 40 23
topic or section are not adequate to evaluate if
we have understood the topic.
C3 Questions in the module are not sufficient to 33 46 21
reinforce our learning and understanding of
the topics.
More than half of the students reported that they faced difficulties in answering
modules questions (C15). In an interview, a student who used the digitized
version of BSLM shared that the slow internet connection, limited mobility due to
community restrictions during the pandemic, and lack of tools, such as mobile
phones, computers, and laboratory instruments, led to his difficulties in
completing learning tasks, including accessing weblinks to supplemental
materials. These impediments to accessing knowledge resources in the
community and the Internet constrained students’ engagement with the learning
materials (Pear & Crone-Todd, 2002; Tria, 2020; Williamson et al., 2020). In
response, teachers sought ways to support and provide assistance to students
experiencing difficulties. For instance, they enlisted the help of persons (e.g., a
student’s classmates or an official in the community) to relay necessary
information. In some cases, teachers visited students to personally attend to their
educational needs following COVID-19 protocols.
3.4 Students’ Views on the Structure (Printed and Digitized) of Biology

Modules
Students’ positive appraisal of the structure of BSLM in both printed and digitized
format (Table 4) referred to its clear, intelligible, and comprehensible use of
language (D8, D5), use of visuals that appropriately facilitate better
understanding (D13, D3, D7), and proper technical aspects of writing (D9).
Table 4: Students’ views on the structure of Biology modules

Printed Digitized
Items (n = 67) (n = 154)
1 2 3 1 2 3
(%) (%) (%) (%) (%) (%)
D8 The module uses clear and intelligible 6 19 75 3 20 77
language
D5 The module uses language that has a 3 18 79 3 21 76
natural flow and is comprehensible.
D13 Pictures, graphs, figures, and tables in the 2 27 72 4 21 75
module are appropriate to our learning
level.
D3 Visuals and graphic organizers are 6 22 72 5 22 73
available to facilitate better understanding
of the lesson.
261
D7 Visual materials such as pictures, graphs, 6 22 72 5 22 73

figures, and tables are placed appropriately
in the module.
D9 Punctuation and spelling rules are strictly 2 21 78 5 26 70
followed in the text.
D2 The lessons are arranged from simple to 8 28 64 8 29 62
complex.
D6 The module is visually engaging and 6 27 67 8 35 57
interesting.
D11 The colors used of visual materials such as 9 33 58 8 36 57
photos, pictures, and tables in the module
are attractive and interesting.
D10 There are too many typographical errors in 46 43 10 54 32 14
the module.
D4 Remarkable, motivating, and vibrant colors 19 31 49 9 42 49
are used sufficiently in the module.
D1 The texts in the module are clear and easy 19 36 45 12 40 49
to understand.
D14 Long sentences are used for presenting 9 31 60 13 42 44
information in the module.
D12 There are too many unfamiliar words in the 24 37 39 35 43 21
module that we cannot understand.
Notes: 1 = Disagree, 2 = Undecided, and 3 = Agree. *Adapted from Çimer and Coşkun (2018).
Many of the students reported that there are too many unfamiliar words (D12),
more from students who used printed modules (39%) compared to those who
used digitized modules (21%). Students who opted for print modules generally
belong to the low socio-economic group who have problems in internet
connectivity and/or lack electronic gadgets (Tria, 2020). As such, they have no
access to supporting online resources when faced with learning difficulties, such
as when confronted with unfamiliar words. In designing BSLM, teachers
provisioned a glossary section that students could refer to. Some of them also
offered helplines through various communication channels. Teachers recognized
that it is important that the BSLM are complemented by external modes of
support.
4. Conclusion
This study set out to understand students’ experiences with the Biology self-
learning modules, which was rapidly developed and deployed as part of
emergency distance education. We investigated students’ interactions with the
new curriculum material using an online survey questionnaire informed by
constructivist learning theory and conducted online interviews with students and
teachers to better understand the contextual influences on their participation in
science learning and teaching.
Students' evaluation of the BSLM is largely positive. In general, they considered

the content and activities as timely, suitable, sufficient, applicable to real-life
situations, and relatable to other disciplines. The activities, assessment, and
evaluation in the modules encourage independence and responsibility towards
one own learning, provide various opportunities to express learning, and guide
to develop new understandings. The use of scaffolding and visual elements, like
262
diagrams and pictures, helped them to better understand concepts. Students

viewed the structure of the module positively, noticing its clarity,
comprehensibility, and correctness of language, grammar, and spelling. These
BSLM design features that were salient to a majority of the students suggest that
constructivist learning could somehow be achieved even in an emergency
distance education context.
However, students’ engagement with the BSLM was not without any challenges.
There were students who perceived the presentation of content as superficial and
some activities unimplementable (e.g., laboratory activities) at home because of
the unavailability of necessary materials. Some recognized that group tasks may
be difficult to accomplish when they are homebound and lack access to
technologies for remote collaboration. Still others found some module questions
difficult to answer because of, again, impediments in accessing knowledge
resources in the community and the Internet. Digital inequalities remained acute
and contributed to students’ differential participation in learning activities.
Teachers found ways to try to mitigate the consequences of digital inequalities.

They offered alternative activities, requested parents’ assistance, adjusted
deadlines, suggested doable workplans, and opened various channels of
communication and support. Teachers’ accommodations and extensions of help
became necessary elements of a pandemic pedagogy.
Learning about the students’ experiences with new curriculum materials is

productive in informing future design efforts. The results of this study may be
used by teachers and other instructional designers to improve materials for
emergency distance learning. It does not only provide insight into design features
that could support constructivist learning but also suggest complementary
support structures needed to address digital inequalities. With the COVID-19
pandemic suspending conventional education, putting at greater risk the
intellectual and social development of children and adolescents, it is important to
appraise proposed remedies in order to fine tune our emergency response.
5. References
Adnan, M., & Anwar, K. (2020). Online learning amid the COVID-19 pandemic: Students’
perspectives. Journal of Pedagogical Sociology and Psychology, 2(1), 45-51.
https://doi.org/10.33902/JPSP.%202020261309
Albert, J. R. G., Serafica, R. B., & Lumbera, B. T. (2016). Examining trends in ICT statistics:
How does the Philippines fare in ICT? (No. 2016-16). PIDS Discussion Paper Series.
Akamai (2017). Akamai’s state of the internet. Akamai, 10(1).
https://www.akamai.com/us/en/multimedia/documents/state-of-the-
internet/q1-2017-state-of-the-internet-connectivity-report.pdf
Ayaz, M. F., & Sekerci, H. (2015). The effects of the constructivist learning approach on
student's academic achievement: A meta-analysis study. Turkish Online Journal of
Educational Technology-TOJET, 14(4), 143-156.
Bada, S. O., & Olusegun, S. (2015). Constructivism learning theory: A paradigm for
teaching and learning. Journal of Research & Method in Education, 5(6), 66-70.
Boddy, N., Watson, K., & Aubusson, P. (2003). A trial of the five Es: A referent model for
constructivist teaching and learning. Research in Science Education, 33(1), 27-42.
https://doi.org/10.1023/A:1023606425452
263
Bybee, R. W. (1997). Achieving scientific literacy: From purposes to practices. Portsmouth, UK:
Heinemann.
Bybee, R. W., & Powell, J. C. (1993). Investigating diversity and limits, middle school science
and technology. Dubuque, IA: Kendall Publishing.
Bybee, R. W., Taylor, J. A., Gardner, A., Van Scotter, P., Powell, J. C., Westbrook, A., &
Landes, N. (2006). The BSCS 5E instructional model: Origins and effectiveness.
Biological Sciences Curriculum Study.
Caggiano, G., Castelnuovo, E., & Kima, R. (2020). The global effects of COVID-19-induced
uncertainty. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.3598710
Cakir, N. K. (2017). Effect of 5E learning model on academic achievement, attitude and
science process skills: Meta-analysis study. Journal of Education and Training
Studies, 5(11), 157-170. https://doi.org/10.11114/jets.v5i11.2649
Çimer, A. (2012). What makes biology learning difficult and effective: Students’ views.
Educational Research and Reviews, 7(3), 61-71. https://doi.org/10.5897/ERR11.205
Çimer, A., & Coskun, S. (2018). Students' opinions about their ninth grade biology
textbook: From the perspective of constructivist learning approach. Journal of
Education and Learning, 7(4), 201-214. https://doi.org/10.5539/jel.v7n4p201
Creswell, J. W. & Creswell, J. D. (2018). Research design: qualitative, quantitative, and mixed
methods approaches (5th ed.). Los Angeles: SAGE publications.
https://doi.org/10.1002/nha3.20258
Department of Education. (2020a). The basic education learning continuity plan. (DepEd
Order No. 12, s. 2020).
Department of Education. (2020b). Policy guidelines for the provision of learning resources in
the implementation of the basic education learning continuity plan (DepEd Order No.
018, s. 2020). Department of Education.
Department of Education. (2019a). Policy guidelines on the k to 12 basic education program
(DepEd Order No. 021, s. 2019). Department of Education.
Department of Education. (2019b). Alternative delivery mode learning resource standards
(ADMLR STANDARDS) [version 12]. Department of Education.
Dochy, F., Segers, M., Van den Bossche, P., & Gijbels, D. (2003). Effects of problem-based
learning: A meta-analysis. Learning and Instruction, 13(5), 533-568.
https://doi.org/10.3102%2F00346543075001027
Doolittle, P. E. (1999). Constructivism and online education. Virginia Polytechnic Institute
and State University.
Funa, A. & Prudente, M. (in press). Effectiveness of problem-based learning on secondary
students’ achievement in science: A meta-analysis. International Journal of
Instruction.
Gross, K., & Gross, S. (2016). Transformation: Constructivism, design thinking, and
elementary STEAM. Art Education, 69(6), 36-34.
https://doi.org/10.1080/00043125.2016.1224869
Gutierrez, A. F. (2014). Development and effectiveness of an educational card game as
supplementary material in understanding selected topics in biology. CBE life
sciences education, 13(1), 76–82. https://doi.org/10.1187/cbe.13-05-0093
Jacobs, H. H. (1989). Interdisciplinary curriculum: Design and implementation. Association for
Supervision, and Curriculum Development, 1250 N. Pitt Street, Alexandria, VA
22314.
Lehmann, D., & Hulbert, J. (1972). Are three-point scales always good enough? Journal of
Marketing Research, 9(4), 444-446. https://doi.org/10.2307/3149313
Marcial, D. E., Caballero, R. D. B., Rendal, J. B., & Patrimonio, G. A. (2015). “I am offline”:
Measuring barriers to open online learning in the Philippines. Information
264
Technologies and Learning Tools, 45(1), 28-41.

https://doi.org/10.33407/itlt.v45i1.1170
Patton, M. (1990). Qualitative evaluation and research methods (pp.169-186). Beverly Hills,
CA: Sage. https://doi.org/10.1002/nur.4770140111
Pear, J. J., & Crone-Todd, D. E. (2002). A social constructivist approach to computer-
mediated instruction. Computers & Education. 38(1-3), 221-231.
https://doi.org/10.1023/A:1024558414766
Sañosa, M. (2013). Implementation of K to 12 curriculum program among grade 7 science
teachers in Eastern Visayas. Journal of Society and Technology, 3, 37-44
Talidong, K. J. B., & Toquero, C. M. D. (2020). Philippine teachers’ practices to deal with
anxiety amid COVID-19. Journal of Loss and Trauma, 25(6-7), 573-579.
https://doi.org/10.1080/15325024.2020.1759225
Toraman, C., & Demir, E. (2016). The effect of constructivism on attitudes towards lessons:
A meta-analysis study. Eurasian Journal of Educational Research, 62, 115-142.
https://doi.org/10.14689/ejer.2016.62.8
Tria, J. Z. (2020). The COVID-19 pandemic through the lens of education in the
Philippines: The new normal. International Journal of Pedagogical Development and
Lifelong Learning, 1(1), 2-4.
Van Bergen, P., & Parsell, M. (2019). Comparing radical, social and psychological
constructivism in Australian higher education: A psycho-philosophical
perspective. The Australian Educational Researcher, 46(1), 41-58.
https://doi.org/10.1007/s13384-018-0285-8
Williamson, B., Eynon, R., & Potter, J. (2020). Pandemic politics, pedagogies and practices:
Digital technologies and distance education during the coronavirus
emergency. Learning, Media and Technology, 45(2), 107-114.
https://10.10bslm80/17439884.2020.1761641
265

Vol. 20, No. 3, pp. 265-281, March 2021
The COVID-19 Pandemic and the Challenges of

E-Assessment of Calculus Courses in
Higher Education: A Case Study in Saudi Arabia
Fatima M. Azmi* and Heba Bakr Khoshaim

Department of General Sciences, Deanship of Educational Services,
Prince Sultan University, Saudi Arabia
https://orcid.org/0000-0001-9275-0965
https://orcid.org/0000-0002-5898-0991
Abstract. The COVID-19 pandemic has affected many aspects of our

lives, including education. Due to this unexpected catastrophe,
education has shifted to virtual-learning and auto-grading models in
most parts of the world. This study explores the validity and
appropriateness of auto-grading-assessment for online exams by
comparing students’ online exam scores where they are first auto-
graded and then manually graded. Furthermore, it investigates whether
the mean differences in their scores are statistically significant. The
study included two calculus courses taught by the authors, during the
spring semester 2019-2020 at a private university in Saudi Arabia. The
online exam was performed on the WebAssign platform, which has
built-in calculus questions. The sample consisted of fifty-five students
who were registered on those calculus courses. The quantitative data
was analysed using the SPSS statistical tool. A paired t-test at an alpha
level of 0.05 was performed on differences in mean exam scores between
auto-graded and manually-graded scores. The statistical analysis results
revealed a statistically significant difference in students' mean scores.
Our findings illustrate the importance of human intelligence, its role in
assessing students' achievements and understanding of mathematical
concepts, and the extent to which instructors can currently rely on auto-
grading. A careful manual investigation of auto-graded exams revealed
different types of mistakes committed by students. Those mistakes were
characterized into two categories: non-mathematical mistakes (related to
Platform Design) and minor mathematical mistakes, which might
deserve partial credit. The study indicated a need to reform the auto-
grading system and provided some suggestions to overcome its
setbacks.
Keywords: COVID-19 pandemic; E-assessment; validity of auto-

grading; higher education; mathematics
*Corresponding author: Fatima M. Azmi; Email: fazmi@psu.edu.sa
©Authors
266
1. Introduction
Pandemics threaten people's existence and health status. The associated damage
also affects our economic, social, and educational systems. The most recent
pandemic is coronavirus—known as COVID-19. An estimated 300 million
students are at home from school due to this unforeseen pandemic (McCarthy,
2020). Despite conflicting arguments about the effectiveness of keeping children
at home (Viner et al., 2020), decisions to keep all schools closed were consistent
worldwide. Both primary/secondary and university students' educational
journey has been unexpectedly and severely disturbed in order to contain the
virus. In the Kingdom of Saudi Arabia (KSA), to keep students on track, several
electronic platforms have been adopted to switch classes to virtual teaching.
Online education has traditionally been regarded as an alternative system of
teaching, but due to the COVID19 pandemic, educators and students of all levels
primary/secondary and university quickly adopted virtual teaching and
learning (Hodges, 2020). Thus, the year 2020 has seen increased adoption of
electronic resources which can be integrated into virtual learning, for example
Zoom and Google Meet (Fulton, 2020).
Technology undoubtedly brings innovative tools and opportunities to the

education field (Broughton et al., 2013; Parshall & Guille, 2015). It can support
educators to teach and help students to learn (Raja & Nagasubramani, 2018).
Particularly in the situation of COVID-19, experiential learning has led to
innovations such as virtual labs and field trips (Pennisi, 2020). However, any
opportunity is expected to present some challenges (Azevedo, 2015), especially
when assessing mathematical knowledge. For example, students must possess
other knowledge and master new skills (when using a computer to submit
answers) and a lack thereof will affect their success (Alqahtani & Rajkhan, 2020;
Parshall & Guille, 2015). Complexity ensues in terms of additional requirements
that will add an extra load onto students: familiarity with the online assessment
platform, how to write the mathematical formulae, and how to enter the
answers in a proper format.
E-assessment or online assessment (auto-grading) refers to the use of

information technology to assess students’ performance in online exams or
homework. One of the challenges of using E-assessment is the restriction in the
format of the items: MCQs (multiple choice questions), true/false, matching, or
short answer submission type items. Although such a format is convenient for
large-stake exams as it provides easy and consistent grading (Stankous, 2018), it
might indicate that only the final answer is what matters. In mathematics
education, we do not only consider the final answer; we evaluate the logical
thinking and decisions made by the student at each step. Thus, the final answer
carries little weight when it comes to mathematical problem assessments. In
that, partial scoring is important.
Moreover, MCQ or true/false formats might not be appropriate to assess

students' analysis and process skills, which are crucial for mathematical
competencies. However, there is an ongoing debate concerning the best item-
format that can be used for mathematical assessment (Stankous, 2018). Hence,
267
there is a need to explore the validity of online auto-grading compared to

manual-grading. Previous research has shown inconsistencies between manual-
grading and auto-grading (Bejar et al., 2017; Stankous, 2018). Stankous argued
that true assessment of mathematical knowledge can only be examined through
constructive response items.
1.1 Significance of the Study

The use of E-assessment is inevitably growing every day. Due to the COVID-19
pandemic, education shifted to E-learning and E-assessment modes overnight
for all schools and universities in Saudi Arabia. Educators did not get a chance
to think, evaluate, or even design their E-assessment approaches. Instead, they
were limited to choosing from several available platforms, adapt to online
teaching, and immediately apply E-assessment procedures. It is now essential to
address this approach, evaluate it, and suggest solutions for its limitations.
Especially as the COVID-19 pandemic cases began to rise again worldwide,
indicating more reliance on virtual teaching and online exams in the future.
This study involves two calculus courses (Calculus I and Business Calculus),
which are taught at a private university in Riyadh, KSA. The instructors of the
courses (who are the authors of this paper) utilized the WebAssign platform by
Cengage for the E-assessment processes for both courses during the spring
semester of the academic year 2019-2020. The sample consisted of a total of fifty-
five students who were registered on those two courses. This study addresses
the following three questions:
Q1: Was there a significant difference in students’ online midterm and final
exam scores between auto-graded and then manually graded scores for the
Calculus 1 course?
Business Calculus (B.C.) course?
Q3: Were there some common mistakes committed by students during the
online exams due to their unfamiliarity with the WebAssign platform, and did
this result in zero credit when auto-graded?
The article findings illustrate the importance of human intelligence, its role in
assessing students' achievements and understanding of mathematical concepts.
The results provide some suggestions and guidelines for E-assessment methods
to assess undergraduate mathematical skills and the need to make auto-grading
more intelligent.
2.1. E-Learning & E-Assessment in Mathematics
The use of technology is increasingly common in our lives, and the education
system is no exception. E-learning in mathematics classrooms refers to the use of
technology in the learning process, which can be through multimedia (e.g.,
videos) to present and articulate a task, software applications to facilitate
understanding of mathematical concepts, or online platforms to practice and
visualize the problems. Such software has been shown to support students'
268
conceptual understanding and the depth of their investigation (Heid, 1988), shift
their attention from computation and memorizing formulas to help them focus
instead on the mathematics itself (Camacho Machín et al., 2010), and improve
their visualization skills (Baki et al., 2011). Therefore, E-learning has become a
prevalent method of effective learning (Kerzic et al., 2018). In fact, E-learning has
shifted from being luxurious to being essential during the COVID-19 pandemic
(Alqahtani & Rajkhan, 2020). Most educational institutions transformed to e-
learning mode to control the spread of the virus. However, although educators
see it as a promising future for teaching and learning (Alanazi & Alshaalan,
2020) and students consider it a time saving method (Khalil et al., 2020), E-
learning is not always the preferred approach by students, especially in applied
sciences (Abbasi et al., 2020).
Assessment is defined as "the process of collecting evidence regarding students'

learning, interpreting the evidence and defining an action" (Shahbari & Abu-
Alhija, 2018. p. 1316). The process itself is valuable and has critical consequences
for students' futures. Its validity and appropriateness have been investigated in
the literature. Until now, assigning numerical or alphabetical grades has been
the most common approach in education to represent students' achievements or
knowledge and is the ultimate goal of most educational systems (Rešić et al.,
2017). However, the assigned grades are not always satisfiable or agreed upon
by students or their families (Rešić & Halilčević, 2014). E-assessment, on the
other hand, refers to any assessment done electronically, in fact it is any type of
computer-based assessment using a given platform. In recent years, the
approach has grown in popularity and is expected to be adopted by higher
education institutions in the future.
2.2. Benefits of E-assessment

E-assessment is a crucial aspect of online education. Some of its benefits include
its fast scoring and cheap administration (Broughton et al., 2013; Rupp &
Leighton, 2016; Smith, 2019). There is a bank of questions that instructors can
utilize to compile their assessment, therefore saving them time. Moreover, items
designed to be auto-graded usually do not require a high level of reading
comprehension (Kan et al., 2019). High-stakes assessments, such as SATs or
GCSEs, use E-assessments exclusively. In many cases, students can get
immediate feedback. Several features make E-assessment of exams even more
pleasing; for example, each student will receive a unique set of numerical values,
the order of items will be shifted among students, time limits will be applied per
exam or item, submissions can be restricted, and items are displayed one at a
time.
2.3. Challenges of E-assessments

Regardless of the benefits of E-assessments, there are some limitations. First, the
lack of technical skills, limitations of access, and poor infrastructure might affect
the assessment process (Alruwais et al., 2018). In addition, the items' format
has a substantial influence (Kan et al., 2019). The use of MCQ is widespread in
the E-assessment process in mathematics. Recent research claimed MCQ
assessment had been proven effective (Azevedo, 2015) and that students'
performance does not differ between an MCQ or long-response format
269
(Lawrence & Singhania, 2004). However, as Kan et al. (2019) suggested, different
item formats require distinct cognitive demands and skills to solve the
mathematical problem. It is thus essential to match the item's format to the
assessment goal. However, the format of the items in E-assessment is restricted
to MCQ, true/false, matching, or short answer submission, so some aspects of
students' mathematical understanding—such as their logical and analytical
skills—might not be addressed (Wang, 2011). Moreover, such formats might not
reflect an accurate evaluation of students' understanding, as some students
might simply guess the correct answer. In fact, Stankous (2018) argued that only
constructive response items are effective in assessing mathematical
competencies.
Mathematics is a subject that requires the knowledge and skills to analyse a

problem accurately before solving it. The process (i.e., the sequential steps
necessary to solve the mathematical problem) plays a significant role in
assessing whether students have achieved the aim of the subject (Rešić &
Halilčević, 2014). Evaluating problem-solving problems is thus essential but
cannot be accomplished with MCQ or short answer responses (Wang, 2011).
3. Research Methodology
3.1. The WebAssign Platform
WebAssign is an online education platform designed by educators to enrich the
teaching and learning experience (https://webassign.com/). The students of the
Calculus 1 and Business Calculus courses were using the WebAssign platform
for all their homework for many years. The platform provides valuable
electronic recourses that foster the learning process. For the purpose of
assessments, the platform was used only for online homework assignments.
However, due to COVID-19 and the lockdown order, the Saudi higher education
system converted the educational process into a virtual one. Therefore, the
instructors of the two courses chose to conduct each assessment (midterm and
final exam) for these two courses on the WebAssign platform. We should
mention that the settings for the homework assignments differs from the setting
for the exam. For example, when students are doing homework, they are then
given up to five attempts to submit answer, in case their previous attempts were
wrong, they also have hints and solutions to similar type problem options, as the
main purpose of the assignments is to help students learn and master
mathematical problem-solving techniques. On the other hand, the setting for the
exam was completely different, only one attempt per question was permitted
and no help or hint was allowed. The duration of the exam was limited to 60
minutes for midterm and two hours for the final exam.
3.2. Design and Data Collection

The total number of students who started the Calculus I and Business Calculus
courses was 62, and the number of students who actually completed both
courses was 55. Thus, our sample consisted of a total of 55 freshman and
sophomore students registered on the Calculus I and Business Calculus courses
(46 students in Calculus I, and 9 students in Business Calculus). Both courses
were taught by the authors of this paper during spring 2019/2020. The study
270
investigated midterm and final exam items and students' responses in these two
courses. It is important to mention here that all assessment items were selected
from the large questions bank provided by WebAssign. The teachers were not
able to develop new items or modify the existing ones. All assessment items
were either MCQs or short answer submissions. For the Calculus I course, the
midterm had a total of 29 items, the final exam had a total of 36 items, while
Business Calculus had a total of 18 items in the midterm exam, and the final
exam had a total of 23 items. This study addressed the following three questions:
Calculus 1 course?
3.3. Limitation of the Study

This study considers only two courses, Calculus I and Business Calculus courses
taught by both authors at a private university, with a sample size of 55 students.
3.4. Data Analysis

After each online exam, either midterm or final, the exam was immediately auto-
graded and scores were recorded, then, manual grading began. Thus, instructors
reviewed each student's answers and their auto-grading score, then by using
human intelligence it was manually-graded. Thus, depending on the student’s
mistakes, either full, partial, or no credit was awarded. The manual grading
scores were recorded separately for later analysis. The study addressed three
questions, and the data analysis is as follows:
(Q1) and (Q2), the quantitative data were analysed using SPSS statistical tools.
Thus, using a paired t-test, we explore whether the differences in the auto-
graded and manually graded mean scores is statistically significant or not at
alpha = 0.05. The paired t-test is robust to non-normality (Schmider et al., 2010).
Regarding the qualitative data in (Q3), while instructors were manually grading
the exam they recorded the types of common mistakes committed by students.
Those mistakes were characterized into two categories: non-mathematical
mistakes (related to WebAssign Platform Design) and minor mathematical
mistakes, which might deserve partial credit. All those common mistakes were
recorded and samples were collected for the most common types of mistakes.
One of the main reasons for this categorization of students’ common mistakes is
that we can educate our students and try to familiarize them more about the
WebAssign platform and the right method to submit their answers so that they
take extra care when submitting answers in the future.
271
4. Results
4.1. Answer to Q1
Calculus 1 course?
Students of the Calculus I course were given midterm and final exams on the
WebAssign website, using the platform’s questions bank. The midterm exam
had 29 items (5 MCQs and 24 short answer submissions), while the final exam
had a total of 36 items (9 MCQs and 27 short answer submissions).
Each exam, whether a midterm or final was immediately auto-graded, then a

manual-grading was carried out. Each item's answer was examined. In
particular, when a student obtained zero credit, their answer was analysed. The
grades were assigned according to the type of mistakes committed by the
student (more detail is provided on this when answering Q3). The differences in
the grades between auto-grading and manual-grading were recorded.
The paired t-test revealed a statistically significant difference in students'

midterm mean scores at p = 0.000 < 0.05. (Table 1). A similar investigation for
final exam grades revealed a statistically significant difference in students' mean
scores at p = 0.000 < 0.05. (Table 2).
Table 1. Significant difference in auto and manual grading for midterm exam of
Calculus I.
Calculus 1
Midterm Exam Std. Std. Error Sig.
Mean Deviation Mean t df (2-tailed)
Auto -Graded -2.79948 2.08683 .30121 -9.294 45 .000
Manual-Graded
Table 2. Significant difference between auto and manual grading for final exam of
Calculus I.
Calculus 1
Final exam Std. Std. Error Sig.
Mean Deviation Mean t df (2-tailed)
Auto - Graded -1.649565 1.443055 .212767 -7.753 45 .000
Manual -Graded
The auto-graded midterm scores for Calculus I had a mean score of 38.04 with
7.351 standard deviation, when it was manually-graded the midterm scores had
a mean of 40.84 with 6.45 standard deviation. As for the final exam, the auto-
graded students had a mean score of 37.82 with 8.36 standard deviation, and
when manually-graded, it had a mean score of 39.47 with 7.74 standard
deviation. (Figure 1)
272
Figure 1. The difference in the mean scores between auto-graded and manual-graded
exams for the Calculus I course.
4.2. Answer to Q2
Students of the Business Calculus course were given midterm and final exams
on the WebAssign website, using the platform’s questions bank. They had a total
of 18 items in the midterm (3 MCQs and 15 short answer submissions) and the
final exam had a total of 23 items (no MCQs, all short answer submission).
Each exam was immediately auto-graded, then a manual-grading was carried

out. Each item's answer was examined. In particular, when a student obtained
zero credit, their answer was analysed. The grades were assigned according to
the type of mistakes committed by the student (more detail is provided on this
when answering Q3). The differences in the grades between auto-grading and
manual-grading were recorded.
The paired t-test revealed a statistically significant difference in students'

midterm mean scores at p = 0.002 < 0.05. (Table 3). The final exam results were
little different. Observation showed that students committed fewer mistakes on
the online final exam than in the midterms. The paired t-test gave a p-value of p
= 0.101 > 0.05, indicating that the mean differences were not statistically
significant (Table 4).
Table 3. Significant difference between auto and manual grading for midterm exam of B.C.
BC Midterm Std. Error Sig.
Exam Mean Std. Deviation Mean t df (2-paired)
Auto Graded -9.72222 6.66667 2.22222 -4.375 8 .002
Manual Graded
273
Table 4. The difference between auto and manual grading for the final exam of B.C.
BC Final Exam Std. Error Sig.
Mean Std. Deviation Mean t df (2-paired)
Auto -Graded -2.44667 3.95505 1.31835 -1.856 8 .101
Manual Graded
The auto-graded midterm grades for the Business Calculus course had a mean
score of 20.14 with 12.52 standard deviation, while the manual-graded had a
mean score of 29.86 with 11.24 standard deviation. As for the final exam, the
auto-graded students had a mean score of 37.302 with 12.4 standard deviation,
while the manual-graded students had a mean score of 39.749 with 11.816
standard deviation (Figure 2).
Figure 2. The difference in the mean scores between auto-graded and manual-graded
exams for the Business Calculus course.
4.3. Answer to Q3
To investigate whether students' unfamiliarity with the platform affected their
overall assessment, the authors analysed each student's response to each item on
the midterm and final exam. Common mistakes were characterized into two
categories:
1- Platform Design: In some cases, students did not follow the appropriate
website format and hence lost the credit for that item. These are not
mathematical mistakes, nor do they represent a lack of comprehension – they
only indicate students' unfamiliarity with the website requirements.
2- Partial Credits: In other cases, students committed minor mathematical
mistakes; they deserved partial credit but received no credit.
274
4.3.1. Platform Design

Some students did not adhere to the typical writing design required by the
software platform of the WebAssign. They used different symbols,
capitalization, or representations.
The following samples represent innocent "mistakes" committed by students
while submitting their answer on the platform, it does not indicate a lack of
students' mathematical skills or knowledge:
1- When solving an indefinite integral task, a student performed the integration
accurately but wrote the integration constant as a small letter 'c' instead of a
capital letter 'C', and the constant of integration is just a symbol, thus writing
it as a small or large letter does not alter the answer, but this mistake
resulted in zero credit (Figure 3).
2- Instead of writing the variable "x," some students used the Greek letter chi
“ꭓ”, which looks similar to the "x," and all computations were correct. This is
not a mathematical mistake, but it resulted in zero credit (Figure 4).
3- A student computed the derivative of the function correctly. However, when
submitting the answer, the student wrote the symbol of the derivative and
then the equals sign as usually one does when they write the answer on
paper. But on the WebAssign platform, they are supposed to only enter the
answer without . This made WebAssign not recognize the correct answer
and resulted in zero credit (Figure 5).
4- After a long process of taking the derivative and finding the inflection point,
the student submitted a correct answer as an ordered pair, and while
submitting she inserted brackets as is usually done when we write on paper,
she did not notice that the brackets were already included. This made
WebAssign not recognize her correct answer due to extra brackets and she
received zero credit. (Figure 6).
5- The definite integral computation was all correct. The students submitted the
answer in a decimal format instead of a fraction. This resulted in zero credit
(Figure 7).
6- This is a Business Calculus problem; it is a long problem related to finding
the producer’s surplus. When submitting the final answer, the student
rounded it to one decimal place instead of two decimal places (89.3 instead
of 89.33). All computations were accurate, and this resulted in zero credit
(Figure 8).
275
Samples of students who submitted a correct answer and received zero credit
when auto-graded.
Figure 3. Using the lowercase letter “c” as the integration constant
Figure 4. Using the Greek letter chi “ꭓ” instead of “x”
Figure 5. Repeated article
Figure 6. Extra brackets
Figure 7. Using decimal format
276
Figure 8. Rounding to one decimal place instead of two decimal places
4.3.2. Partial Credits

Sometimes students commit minor mathematical mistakes, which result in
partial credit when the problem is manually graded. Unfortunately, the current
auto-grading system does not offer partial credit. Here is a list of examples of
minor mathematical mistakes which deserve partial credit.
1- The student computed the derivative but while entering the answer the
student missed the negative (-) sign in one place. All other computations
were accurate. When we manually grade such a problem the student
deserves partial credit, but auto-grading resulted in zero credit. (Figure 9).
2- The student found the derivative correctly but when submitting her answer
on the platform, in one place she missed a single variable ``x``, which she
forgot to write. However, the WebAssign platform cannot distinguish this
small error, so it resulted in zero credit, however when such a problem is
graded manually then she absolutely deserves partial credit (Figure 10).
on the platform she misplaced one single bracket in a multi-bracket item, this
resulted in zero credit when auto-graded (Figure 11).
on the platform, in one place instead of writing variable ``t”, by mistake she
wrote variable "x". This is just an answer submission mistake, the auto-
grading system does not distinguish this small mistake and results in zero
credit (Figure 12).
Figure 9. Missing a negative sign (-)
277
Figure 10. Missing a variable “x” in a single place
Figure 11. Misplacing a bracket
Figure 12. Using the variable “x” instead of “t” in single place by mistake
5. Discussion
This study aimed to evaluate the E-assessment process used in two calculus
courses at one higher education institution in the KSA. Students were used to
submitting online homework on the WebAssign platform eight weeks before the
lockdown. However, during exams, it is understandable that students might
commit the aforementioned non-mathematical mistakes while submitting their
answers due to exam pressure, the limited time duration, and the restriction of a
single submission attempt. It must be noted that mistakes provoked by the
platform’s design are only relevant during an E-assessment process.
Unfortunately, WebAssign's auto-grading system will either assign a full credit
(100%) or no credit (0%) to an item. No partial credit is considered. This issue is
consistent with the limitations reported by Alruwais et al. (2018), Kan et al.
(2019) and Smith (2019).
The statistical analysis results demonstrated a significant difference between

auto-graded and manual-graded exams scores for both midterm and final exams
for the Calculus I course (Tables 1 and 2). Whereas the bar chart (Figure 1)
displayed the differences in the mean scores between auto-graded and manual-
278
graded midterm and final exams for the Calculus I course. The literature
indicated a discrepancy between auto-grading and manual-grading (Bejar et al.,
2017), which is consistent with our research results.
As for Business Calculus, the result was a little different. Table 3 displayed
statistically significant differences in the midterm mean scores between the auto-
graded and manually-graded exams. However, for the final exam, the difference
was not statistically significant (Table 4). One factor that must be considered is
that the Business Calculus course is mandatory for the Finance programme only;
hence it has a lower number of students than Calculus I (which is offered for
Computer Information, Engineering, and Communication & Network
programmes). In the online midterm exam of Business Calculus, most students'
mistakes were related to platform design rather than mathematical mistakes.
Thus, they became meticulous about submitting their answers on the
WebAssign platform by growing their cognizance of the WebAssign platform's
tools and continually asking the instructor for every detail to minimize their
mistakes related to the platform design. The final exam result demonstrated
fewer mistakes related to platform design. This revealed the importance of
educating our students in detail about online platform design. The bar chart
(Figure 2) displayed the differences in the mean scores between auto-graded and
manual-graded midterm and final exams for the Business Calculus course.
To investigate whether students' unfamiliarity with the platform affected their

overall assessment, the authors analysed each student's response to each item on
the midterm and final exam. The common mistakes were characterized into two
categories: Platform Design and Partial Credits. By examining the samples of
answers provided by students, it becomes clear that students deserved either
partial or full credit when auto-graded awarded them zero credit.
The assessment items' type influences the assessment's validity. The results
reflected a surprising number of inconsistencies between the auto-grading and
manual-grading. Figures 3-8 illustrated the most common mistakes committed
by students that resulted in zero credit when auto-graded.
In many cases, the mistake was only due to a lack of familiarity with the
formatting, not a lack of mathematical understanding (Figures 3-8). For example,
submitting answers with additional brackets (Figure 6) resulted in zero credit for
the answer, this mistake was apparent when students wrote an ordered-pair
number (an inflection point), intervals of an increasing and decreasing function,
or a function's concavity status. This is consistent with what Smith (2019) stated
– that formatting and technical issues could result in losing credits.
In some other cases, if the student made a minor mathematical mistake, then
she/he deserves a partial credit, which is a standard agreement in mathematics
assessments. For example, after carrying out a complicated derivative task, a
student incorrectly missed a "minus sign" (Figure 9). In the auto-grading
process, the student received zero credit for the item, whereas if human
intelligence were applied then partial credit would have been awarded.
279
As our analysis demonstrated, E-assessment has some limitations. It has been

argued that students might misinterpret an item or some minor terminology and
hence fail to choose the correct answer (Wang, 1998). Moreover, sometimes
students commit simple mistakes if they misread an item in E-assessment, which
can otherwise be addressed with short feedback from the instructor. However,
this is not possible with auto-grading (Smith, 2019), which is consistent with our
research results.
Moreover, considering partial grading is essential. Even if a student must only

submit the final answer, this final answer can be evaluated from several angles.
In some cases, students do not submit a 100% accurate answer, but a partially
valid answer deserves partial credit. Unfortunately, this is not always possible
with auto-grading (Smith, 2019), as our research indicated.
6. Conclusion
The demand for E-learning and E-assessments are increasing, especially during
the recent COVID-19 pandemic as traditional teaching and assessments methods
are no longer favourable. In this study, we have investigated the validity and
appropriateness of E-assessments in Calculus courses, by comparing the
difference in the means scores of auto-graded and manually graded exam scores.
The differences were statistically significant, thus illustrating the importance of
manual grading and its role in assessing students' achievements and
mathematics understanding. From the results, we can conclude that we cannot
solely rely on auto-grading, as it will not be a valid indicator of a student's
mathematical proficiency. Also, a partial score is essential; however, this aspect
is not considered in auto-grading. Hence, auto-grading is a useful approach, but
it has some limitations and needs to be more intelligent. E-learning and E-
assessments put an extra non-Mathematical burden on the students—the need to
learn about each online platform design, which they will use. Therefore,
students need to master new skills (when using a computer to submit answers),
and a lack thereof will affect their success (Parshall & Guille, 2015). Our study
has characterized the most common type of mistakes committed by students
during online exams. We hope our research will guide the software designer to
target those common mistakes and upgrade their auto-grading software to
account for partial credit and to correct platform design errors.
Acknowledgment
The authors would like to thank the Deanship of Educational Services at Prince
Sultan University for their support. Also, the authors would like to thank the
reviewers for their helpful comments, which helped the paper takes its current
shape.
Declaration
The authors indicate no conflict of interest.
9. References
Abbasi, S., Ayoob, T., Malik, A., & Memon, S. I. (2020). Perceptions of students regarding
E-learning during COVID-19 at a private medical college. Pakistan Journal of
Medical Sciences, 36, 57–61. https://doi.org/10.12669/pjms.36.COVID19-S4.2766
280
Alanazi, A. A, & Alshaalan, Z. M. (2020). Views of faculty members on the use of e-

learning in Saudi medical and health colleges during COVID-19 pandemic.
Journal of Nature and Science of Medicine, 3(4), 308–317.
https://doi.org/10.4103/JNSM.JNSM_82_20
Alqahtani, A. Y., & Rajkhan, A. A. (2020). E-Learning critical success factors during the
COVID-19 pandemic: A comprehensive analysis of E-Learning managerial
perspectives. Education Sciences, 10(9), 1–16.
https://doi.org/10.3390/educsci10090216
Alruwais, N., Wills, G., & Wald, M. (2018). Advantages and challenges of using e-
assessment. International Journal of Information and Education Technology, 8(1), 34–
37. https://doi.org/10.18178/ijiet.2018.8.1.1008
Azevedo, J. M. (2015). E-Assessment in mathematics courses with multiple-choice
questions tests. In Proceedings of the 7th International Conference on Computer
Supported Education, CSEDU-2015 (pp. 260–266).
https://doi.org/10.5220/0005452702600266
Baki, A., Kosa, T., & Guven, B. (2011). A comparative study of the effects of using
dynamic geometry software and physical manipulative on the spatial
visualization skills of pre-service mathematics teachers. British Journal of
Educational Research, 42, 291–319. https://doi.org/10.1111/j.1467-
8535.2009.01012.x
Bejar, I. I., Mislevy, R. I., & Zhang, M. (2017). Automated scoring with validity in mind.
In A. A. Rubb, & J. P. Leighton, The handbook of cognition and assessment:
Frameworks, methodologies and applications (pp. 226–247). Wiley-Blackwell.
Broughton, S. J., Robinson, C. L., & Hernandez-Martinez, P. (2013). Lecturers'
perspectives on the use of a mathematics-based computer-aided assessment
system. Teaching Mathematics and its Application, 32, 88–94.
https://doi.org/10.1093/teamat/hrt008
Camacho Machín, M., Depool Rivero, R., & Santos-Trigo, M. (2010). Students' use of
Derive software in comprehending and making sense of definite integral and
area concepts. CBMS Issues in Mathematics Education, 16, 29–61.
Fulton, C. (2020). Collaborating in online teaching: inviting e-guests to facilitate learning
in the digital environment. Information Learning Sciences, 121(7/8), 579–585.
Heid, M. K. (1988). Resequencing skills and concepts in applied calculus using the
computer as a tool. Journal for Research in Mathematics Education, 19, 3–25.
Hodges, C., Moore, S., Lockee, B., Trust, T. & Bond, M. A. (2020, March 27). The
difference between emergency remote teaching and online learning. Educause
Review. https://er.educause.edu/articles/2020/3/the-difference-between-
emergency-remote-teaching-and-online-learning#fn12
Kan, A., Bulut, O., & Cormier, D. C. (2019). The impact of item stem format on the
dimensional structure of mathematics assessments. Educational Assessment, 24(1),
13–32. https://doi.org/10.1080/10627197.2018.1545569
Kerzic, D., Umek, I., Tomazevic, N., & Aristovnik, A. (2018). Evaluating the impact of e-
learning on students' perception of acquired competencies in a university
blended learning environment. Journal of e-learning and Knowledge Society. 14(3),
65–76.
Khalil, R., Mansour, A. E., & Fadda, W. A. et al. (2020). The sudden transition to
synchronized online learning during the COVID-19 pandemic in Saudi Arabia: a
qualitative study exploring medical students’ perspectives. BMC Medical
Education, 20(285), 1–10. https://doi.org/10.1186/s12909-020-02208-z
Lawrence, J. A., & Singhania, R. P. (2004). A study of teaching and testing strategies for a
required statistics course for undergraduate business students. Journal of
281
Education for Business, 79(6), 333–383. https://doi.org/10.3200/JOEB.79.6.333-

338
McCarthy, K. (2020). "The Global Impact of Coronavirus on Education." [Video]. ABC
News Network. https://abcnews.go.com/International/global-impact-
coronavirus-education/story?id=69411738
Parshall, C. G., & Guille, R. A. (2015). Managing ongoing changes to the test: Agile
strategies for continuous innovations. In F. Drasgow (Ed.), Technology and
Testing: Improving Educational and Psychological Measurement. National Council of
Teachers of Mathematics.
Pennisi, E. (2020). Courses bring field sites and labs to the small screen. Science,
369(6501), 239–240. https://doi.org/10.1126/science.369.6501.239
Raja, R, & Nagasubramani, P. C. (2018). Impact of modern technology in education.
Journal of Applied and Advanced Research, 3(1), 33−35.
https://doi.org/10.21839/jaar.2018.v3S1.165
Rešić, S. & Halilčević, S. (2014). Tracking, evaluation, and grading students'
accomplishments in mathematics classes. Human: Journal for Interdisciplinary
Studies, 4(3), 41–50.
Rešić, S., Bajramović, A., & Hrnjičić, A. (2017). Grade as the motivational factor in
learning mathematics. Human: Journal for Interdisciplinary Studies, 7(2), 10–21.
https://doi.org/10.21554/hrr.091702
Rupp, A. A., & Leighton, J. P. (2017). Introduction to handbook. In A. A. Rubb, & J. P.
Leighton (Eds.), The handbook of cognition and assessment: Frameworks,
methodologies and applications (pp. 1–11). Wiley-Blackwell.
Schmider, E., Ziegler, M., Danay, E., Beyer, L., & B hner, M. (2010). Is it really robust?
Reinvestigating the robustness of ANOVA against violations of the normal
distribution assumption. Methodology: European Journal of Research Methods for the
Behavioral and Social Sciences, 6, 147–151.
Shahbari, J. A., & Abu-Alhija, F. N. (2018). Does training in alternative assessment
matter? The case of prospective and practicing mathematics teachers' attitudes
toward alternative assessment and their beliefs about the nature of mathematics.
International Journal of Science and Mathematics Education, 16, 1315–1335.
https://doi.org/10.1007/s10763-017-9830-6
Smith, J. S. (2019, June), Determining optimal deployment strategies of MATLAB autograder to
maximize student learning and engagement. Paper presented at 2019 ASEE Annual
Conference & Exposition, Tampa, Florida. https://peer.asee.org/32627
Stankous, N. V. (2018). Constructive response vs. Multiple-Choice tests in math:
American experience and discussion (Review). European Scientific Journal (special
ed.), 308–316. https://core.ac.uk/download/pdf/328025438.pdf
Viner, R. M., Russell, S. J., Croker, H., Packer, J., Ward, J., Stansfield, C., Mytton, O.,
Bonell, C., & Booy, R. (2020). School closure and management practices during
coronavirus outbreaks including COVID-19: A rapid systematic review. Lancet
Child & Adolescent Health, 4, 397–404. https://doi.org/10.1016/S2352-
4642(20)30095-X
Wang, J. (1998). A content examination of the TIMSS results. Phi Delta Kappan, 80(1), 36–
38.
Wang, J. (2011). Re-examining test item issues in the TIMSS Mathematics and Science
Assessments. School Science and Mathematics, 111(7), 33–34.
Vol. 20, No. 3, pp. 282-303, March 2021
The Effects of Physical, Emotional, and Cognitive

Demands on Academic Leaders’ Performance in
Malaysian Research Universities
Mayadah Graizi
University of Malaya, Kuala Lumpur, Malaysia
https://orcid.org/0000-0002-8298-5637
Kenny S. L. Cheah
https://orcid.org/0000-0001-9881-582X
Kazi Enamul Hoque

https://orcid.org/0000-0002-8152-9250
Abstract. This study attempted to investigate the possible impact of

physical, emotional, and cognitive job demands on burnout among
Malaysian academic leaders at Research Universities (RU). Another
objective of the study was to study the direct and mediating role of
burnout on the job performance of the target population. Through a
quantitative study and by using a five-Likert point, 250 academic
leaders in Malaysian Research Universities (MRUs) were surveyed. The
obtained primary data were subject to quantitative analysis through
outer loading of the items using Smart PLS software. The exploratory
and confirmatory tests applied to the primary data earlier to the
inferential tests started with testing the direct hypotheses structured
followed by the indirect effect. Findings indicated that based on the
model extracted and the loaded factors, it was found that cognitive,
emotional, and physical demands have a significant impact on burnout.
Burnout also showed a significant effect on in-role and extra-role
performance of the academic leaders and an indirect effect of burnout
between job demands and job performance was observed. As an
implication, this study can have pedagogical implications for
educational policymakers, education syllabus designers, and academic
leaders. Assessing the interaction role of gender type suggests further
research, which benefit the policymakers in diversifying the job demand
for each type.
Keywords: job demands; job performance; burnout; academic

leadership; research university
©Authors
283
1. Introduction
An emerging concept among academicians is academic leadership (Lorello et al.,
2020). Academic leadership is an interest among scholars at Research
Universities (RU) who seek excellence in research and education (Burkinshaw &
White, 2019). Rathmell et al. (2019) explain that academic leadership is the ability
to use various skills to overcome the challenges encountered by academicians.
They also state that academic leadership in one generation affects the leadership
among the next generation of academicians.
Burkinshaw and White (2019) take a gender-based view and explain that males
and females have not been granted equal opportunities to take an academic lead.
Qamar et al. (2019) believe that non-academic aspects of work affect academic
leadership. They refer to these variables as external variables. However, the
focus of most scholars has been on the effects of academic leadership on
academic performance (e.g., Qamar et al., 2019; Sinniah et al., 2018; Sung et al.,
2017).
In line with many western societies, academic leadership has also been
investigated in Malaysia (Arbae et al., 2019; Ismail & Noor, 2016; Omar, 2018;
Rahman et al., 2019). Although these studies mostly look into the professional
role of academic leaders, they mostly neglect the possible factors that can affect
academic leadership (Zarb, 2016). The most common variables investigated in
these studies are unproportioned job demands (Bakker & Demerouti, 2007),
academic leaders' wellbeing (Bakker & Demerouti, 2014), and performance
(Saleem et al., 2017).
It seems that unproportioned job demand is one of the most common themes
studied in Malaysia and in the global context. Bakker and Demerouti (2018)
assert that every job demand has a cost for the employees. In case there is an
imbalance between the cost and energy for the employees, they undergo
job-related stress. Bakker and Demeroutil (2007) identify the main aspects of job
demand as physical, emotional, and cognitive aspects. Also, and according to
Ilies et al. (2015) and Nahrgang et al. (2011), while emotional and physical
demands cause burnout, cognitive demands result in mental tiredness. Burnout
(also known as strain) can eventually affect job efficacy among academic leaders
(Friedman, 2000). This indicates that a serious look at the factors that result in
burnout among academicians is required, as excessive physical and emotional
demands decrease work efficacy among academicians (Bowen et al., 2016).
However, research conducted in Malaysia has mostly focused on pedagogy, and
less attention has been given to job demands (e.g., Rahman et al., 2019).
In terms of academic leaders' performance, most research has accorded focus to

professional role and relationships rather than job demands and burnout (Arbae
et al., 2019; Omar, 2018; Ismail & Noor, 2016). The question left open is how
burnout and job demands are associated among Malaysian academic leaders.
While burnout can possibly affect performance (Al-Dubai et al., 2013), and
decrease organizational commitment (Bte Marmaya et al., 2011), little attention
has been given to it in the context of Malaysia. Needless to say that the job
284
performance in educational contexts is as important as any other context.

Williams and Anderson (1991), who provide a lateral taxonomy of job
performance (in-role vs. extra-role), assert that in-role job performance (task
performance) and extra-role performance (organizational citizenship behavior
(OCB) or contextual performance) should be equally attended.
The educational sector in Malaysia is experiencing very swift changes, as the

number of universities is increasing and the need for qualified staff is obvious
(Yousefi & Abdullah, 2019). Mustapha (2013) stated that little research has dealt
with the academic leaders in RUs at the centre of such changes. In line with
these swift changes, the knowledge-based economy within Malaysian higher
education requires adroit academic leaders (Grapragasem et al., 2014). This
indicates that maintaining an academic position is harder than before, and the
educators are under stress to produce knowledge and show high levels of in-role
and extra-role performance. Such difficulties are the result of unproportioned
job demands which have not been solved yet.
There are scholastic pieces of evidence that indicate academicians' burnout

should be subject to more research in the context of Malaysia. Not only is
burnout one of the main influential factors on work performance in the
educational sector (Al-Dubai et al., 2013), but also it can decrease organizational
support (Bte Marmaya et al., 2011). On the other hand, burnout is the result of
the stressors the Malaysian academicians encounter in higher education (Fullan
& Scott, 2009). Yet, whether these stressors are emotional, cognitive, or physical
is not fully known.
Khairuddin and Makhbul (2011) who point to the academic performance of the
academic leaders assert that at Research Universities (RUs), academic
performance is not studied satisfactorily. The consensus in the literature is that
not only should the academic leadership problems in Malaysia be investigated,
but also the factors affecting the academic leaders' performance should be
investigated. These two main problems form the main intention to conduct this
study.
According to the problems stated above, this study has two main objectives,
namely, 1) To find out how various forms of job demand affect burnout; 2) To
determine how burnout affects job performance. To have an accurate look at this
issue, both direct and indirect impacts of burnout on job demand are studied.
Also, job performance is viewed as in-role and extra-role performance of
Malaysian academicians.
Various theories that discuss job demand consider three main aspects, i.e.,
physical, psychological, and social (organizational) for job demand (DeFrank &
Ivancevich, 1998; Schaufeli & Bakker, 2004). Job demand, if assigned
proportionately, can have positive effects; however, unproportionate job
demands can have negative effects on the performance of the employees and the
outcome of an organization. According to Bakker and Demerouti (2018), some
285
variables also play a role in the relationship between job demand and job
performance. Burnout and motivation are the most important variables. In
simple terms, long working shifts and excessive work pressure can result in
burnout or even depression and eventually affect the performance of the
employees. These variables have been subject to scientific studies and a number
of theoretical bases have emerged as a result of these studies.
3. Theoretical Framework
The main theory used in this study is that of Job Demand-Resource (JDR) theory.
However, to justify the use of JDR theory, some of the relevant previous theories
should be explained. One of the most significant theories is the Path-Goal theory.
Yukl and Lepsinger (2006) explain that the Path-Goal theory is a leadership
theory that sees the leaders' behaviour as the result of their satisfaction,
motivation, and eventually, their performance. This theory associates the
leaders' behaviour to their subordinate's abilities that compensate for
deficiencies (Bakker & Demerouti, 2014). Thus, the researchers should attempt to
see how the contingency perspective to leadership is associated with conditions
or situational variables that affect the relationship between leaders' behavior and
effectiveness (Madonko & April, 2020). The types of leaders' behaviour in
Path-Goal theory, as stated by Muchinsky (2006), are directive, supportive,
participative, and achievement-oriented. While directive behaviors tend to
remove obstacles that cause frustrations, supportive behaviors lead to
understanding subordinates’ needs and increasing their wellbeing. Participative
and achievement-oriented behaviors lead to investments in the ideas of
subordinates, and encouraging a high performance, respectively (Muchinsky,
2006).
Raziq et al. (2018) divide the behavior in Goal-Path theory into three main
categories. These categories centre on satisfaction and refer to immediate and
future satisfaction, along with tools to satisfy subordinates' needs contingent
with effective performance (House & Dessler, 1974). The main tools are support
and guidance in the work environment.
3.1 Stress Theories

The relationship between stress and job environment is a conditional-stimuli
relationship (Jex et al., 1992). Jex et al. (1992) also assert that the stimulus for
stress in the job environment can be any job-related demand that causes burnout.
Thus, according to Kinman and Jones (2005), there should be a balance between
job demands and personal abilities. This relationship is known as job-personal
resources. Job-personal resources are the basis of many job stress theories, which
have mature in different ways. One of the mostly used theories in this regard is
that of the Job Demand-Resource model.
3.2 The Job Demand-Resource Model (JDR)

The job demand-resource (JDR) model is the theoretical basis of this study. This
theory does not only centre on specific types of job conditions. In this theory, the
job characteristics are broad (Bakker & Demerouti, 2017). The theory is a
globally-proven theory and has been used in many studies. According to Bakker
286
and Demerouti (2014), it has been mainly used to describe employees' wellbeing
(like burnout/strain, motivation, health, engagement, satisfaction, commitment)
and performance (in-role and extra-role). Although the theoretical building
block of JDR theory is based on certain assumptions, the theory asserts that the
working conditions can introduce new variables in the analysis (Bakker &
Demerouti, 2007).
3.3 Conceptual Framework

The model below (Figure 1) shows the JDR theory based on the variables under
investigation in the current study. As can be seen in the model, and based on the
JDR model, three main categories of job demand, i.e., physical, emotional, and
cognitive, are investigated as possible influencers of burnout. This possible
impact may affect in-role and extra-role performance of the academic leaders
which is also investigated based on this model. Finally, the mediating role of
burnout is explored.
Individual demands consist of: Academic performance roles consist of:
Physical
Demands In-Role
Performance
Emotional Burnout
Demands
Extra-Role
Hypothesized Mediator Performance
Cognitive
Demands
Figure 1: Academic leaders’ performance in Malaysia research universities
4. Previous Empirical Studies

Some studies have been conducted on the relationship between job demands
and job performance in Malaysia. Reviewing these studies brought the
researchers to the conclusion that there is a certain need to study the impact of
job demands on burnout and the direct and indirect impact of burnout on job
performance, and these variables are not deeply investigated in the literature yet
(Zysberg et al., 2017).
The correlations between occupational stress and job performance with a focus
on emotional intelligence were investigated by Ismail et al. (2009). They studied
104 academicians at private universities in Malaysia and realized that
occupational stress and job performance correlate. They also found out that
emotional intelligence is a significant role player and mediates the relationship
between these two variables. Watts and Robertson (2011) also worked on
burnout and stress. By reviewing previously published documents from 6 data
resources, they could conclude that burnout and teachers' stress in higher
287
education are associated. Their content analysis also showed that gender has a
predictive role and female staff are more subject to burnout.
Studies conducted in other parts of the world are mostly congruent with these
results. For example, in the context of Austrialia, Winefield et al. (2003) saw a
significant relationship between burnout and lack of job satisfaction. By
surveying 9000 academicians in seventeen universities, they realized that the
new generation of academicians is more subject to burnout compared to the
older staff. They also found that self-report measures of psychological wellbeing
were highly had associations with objective measures of university wellbeing. In
another study in Spain, burnout was observed among school teachers in Spain.
Prieto et al. (2008) realized that work overload could predict exhaustion and
prediction. They found the results by studying 274 teachers at 23 different
secondary schools.
In Pakistan, Saleem et al. (2017) realized that the type of leadership affects
burnout. They reported that the principals’ leadership styles and teachers’
organizational commitment are highly correlated. Through looking at the
indirect impact of emotional intelligence (EI) and perceived organizational
support (POS), they reported that the principals prefer a democratic leadership
style (EI and POS). Another significant finding in their study was that the
leadership styles and organizational commitment are highly correlated.
The previous studies discussed within Table 1 lack the challenging cognitive
demands at work (Meyer & Hünefeld, 2018). Hence, the need to highlight the
cognitive demand in academic work is rising. To fill this gap, this study identified
cognitive demand as a latent factor as a determinant of burnout at academic work.
The researcher reviewed more studies. The synopsis of these studies is presented
in Table 1.
Table 1: Summary of the previous studies

N Authors and Design Instrument Participants and Result
Date Type Context
1 Kasinathan, Quantitative Survey Malaysian There are
and Questionnaire Universities interventions at
Arokiasamy organizational
(2019) level to promote
well-being of
academicians.
2 Zysberg et al. Quantitative Likert scale 1230 daycare Burnout has
(2017) questionnaire educators and associations with
2209 school personality and
teachers emotional
intelligence. In this
way, stress affects
burnout
3 Mérida-López, Review Researcher 13 academic There is a negative
and Extremera Study (Content articles in 3 association
(2017) Analysis) scientific between emotional
databases intelligence and
burnout.
288
4 Saleem et al. Quantitative Survey 50 principals, 300 Democratic

(2017) Questionnaire teachers-Pakistan leadership style is
preferred by the
principals. They
also observed a
significant
relationship
between
leadership styles
(EI and POS) and
organizational
commitment.
5 Makhbul and Review Content Malaysian Both

Khairuddin study analysis academicians environmental
(2013) factors and
personal factors
affect the
academicians'
wellbeing.
6 Bakker et al. Quantitative Survey 3753 Australian Neuroticism
(2010) Questionnaire Academicians correlates with the
health impairment
process, and that
extroversion
correlates with
motivational
process.
7 Watts and Qualitative Researcher 6 valid databases. Burnout is the
Robertson content No human result of having
(2011) analysis participation-Glob large classes with
al many students.
Gender and age
also played a
predictive role as
female staff and
younger staff were
more subject to
burnout. They
correlated this to
emotional
exhaustion of the
female staff.
8 Prieto et al. Quantitative Survey 274 teachers-Spain Quantitative
(2008) survey Questionnaire overload has a
predictive impact
on exhaustion and
dedication. It was
also realized that
role conflict
impacts cynicism
and role ambiguity
impacts
dedication.
9 Winefield et al. Quantitative Survey 9000 academic The academic staff
(2003) survey questionnaire staff -Australia were worse off
than general staff.
The new staff were
worse off than the
289
older staff in terms

of strain and job
satisfaction.
Psychological
wellbeing was
highly correlated
with objective
measures of
university
wellbeing.
10 Ismail et al. Quantitative Survey 104 Occupational
(2009) Questionnaire Academicians- stress and job
Private performance
Universities correlate, and
Kuching emotional
intelligence play a
significant role
and mediates the
relationship.
The review of previous studies also highlights that the quantitative design is the
mostly used research design. The main form of instrument used in these studies
is that of a Likert-scale questionnaire.
5. Methods
This section clarifies the methodology progress adopted by this study, which
explains the research design, population and sampling, and the instrumentation.
5.1 Research Design

This quantitative study has a cross-sectional design, as data were collected
through distributing Google forms. Spector (2019) argues that cross-sectional
design provides much valuable information that explains the relationship
among the model variables, specifically the lack of temporal components. As a
quantitative study, the study has its roots in the positivist perspective that
considers reality as observed rather than perceived (Crossan, 2003).
5.2 Population and Sampling

To conduct the study, the researchers made use of five research universities in
the context of Malaysia, i.e., University of Malaya (UM), University of Putra
Malaysia (UPM), University of Teknologi Malaysia (UTM), University of
Kebangsaan Malaysia (UKM), and University Sains Malaysia (USM). The total of
academic staff determined at 2,000 as reported by the Malaysian Ministry of
Education. These universities were entitled Malaysian Research Universities
(MRUs) (MOHE, 2007). The researchers targeted the academic leaders at various
positions and sent the questionnaire to about 2000 academic staff, which is
processed via the human resource department of each university, the generated
link from the Google survey platform posted to the academic leaders’ social
network community of these universities. As a result, out of which, 250
respondents replied. The probability sampling technique was used to lower the
sampling bias, and to ensure sampling diversification as suggested by Acharya
et al. (2013), which provides better understanding of the population perspective.
290
5.3 Instruments
The Likert scale questionnaire was used in this study. The questionnaire had 81
items. It was a combination of 7 different Likert scale questionnaires. However,
as this study is part of a larger study, only the questions relevant to this study
are discussed. The items relevant to job demand were chosen from "Copenhagen
Psychosocial Questionnaire; COPSOQ II" (Pejtersen et al., 2010). The
questionnaire entails questions relaxant to cognitive, physical, and emotional job
demands. Items relevant to burnout were selected from "Copenhagen Burnout
Inventory; CBI", to assess academics burnout (Kristensen et al., 2005). Finally,
the items related to job performance (in-role and extra-role) were selected from
Williams and Anderson (1991).
6. Results
The objectives behind this study were threefold. First, it aimed at exploring the
effects of physical, emotional, and cognitive demands on the burnout of
academicians. Second, it investigated the effect of burnout on in-role and
extra-role of academicians; and finally, it studied to what extent burnout
mediated the effect of physical, emotional, and cognitive demands on in-role
and extra-role.
The data collected in this study were analyzed using SmartPLS V. 3.2.8. All
results were bootstrapped 1,000 times to compute confidence intervals for the
computed indices. The conceptual model being explored is displayed in Figure
4.1. As displayed in this model, the physical, cognitive, and emotional demands
directly affect burnout. They also have indirect effects on in-role and extra-role
through the mediation of burnout. Some of the indicators (items) related to
latent variables were dropped out, which will be discussed below.
Figure 2: Conceptual PLS model
291
6.1 Cronbach’s Alpha and Composite Reliability Indices

Cronbach’s alpha and composite reliability (CR) indices were computed for the
components of the present model. Both Cronbach’s alpha and CR, which are
complementary indices, were reported because, as noted by (Hair et al., 2017, p.
112), Cronbach’s alpha is a conservative measure of reliability (i.e., it results in
relatively low-reliability values). In contrast, composite reliability tends to
overestimate the internal consistency reliability, thereby resulting in
comparatively higher reliability estimates. Therefore, it is reasonable to consider
and report both criteria. When analyzing and assessing the measures’ internal
consistency reliability, the true reliability usually lies between Cronbach’s alpha
(representing the lower bound) and the composite reliability (representing the
upper bound)”.
Cronbach’s alpha reliability indices should be equal to or higher than .70, as

suggested by Tseng et al. (2006) and Dörnyei and Taguchi (2009). Except for
OCBO, all other constructs enjoyed Cronbach’s alpha indices higher than .70;
moreover, all probabilities were lower than .05, and none of the confidence
intervals were negative or zero. Although OCBO’s reliability index of .589 was
lower than .70, it enjoyed a statistical significance (p = .000), and its confidence
intervals of .412 and .694 were neither negative nor zero. Based on these results,
it can be concluded that the present constructs enjoyed statistically significant
Cronbach’s alpha reliability indices.
Table 2: Cronbach’s Alpha reliability indices

Confidence Intervals
Alpha Mean SD t-value p-value
2.5 % 97.5 %
Burnout 0.889 0.888 0.015 57.753 0.000 0.854 0.915
Cognitive 0.766 0.760 0.040 19.071 0.000 0.673 0.829
Emotional 0.812 0.811 0.027 29.724 0.000 0.752 0.856
In-Role 0.755 0.746 0.059 12.891 0.000 0.609 0.828
OCBI 0.844 0.842 0.025 33.629 0.000 0.784 0.885
OCBO 0.589 0.578 0.075 7.821 0.000 0.412 0.694
Physical 0.724 0.722 0.037 19.393 0.000 0.640 0.787
Table 3 displays the CR indices for the present data. Composite reliability
indices should be equal to or higher than .60; however, they should not be
higher than 0.95 (Hair et al. 2016, p. 112). As displayed in Table 3, all CR indices
were within the ranges of .60 to .95, they all enjoyed statistical significance, and
all confidence intervals were positive and non-zero. Thus, it can be concluded
that the present constructs enjoyed significant CR indices.
Table 3: Composite reliability indices

Confidence
Composite Intervals
Mean SD t-value p-value
Reliability 2.5 % 97.5 %
Burnout 0.916 0.915 0.011 84.694 0.000 0.891 0.933
Cognitive 0.851 0.848 0.022 38.997 0.000 0.802 0.887
292
Emotional 0.877 0.877 0.015 56.857 0.000 0.844 0.903

In-Role 0.721 0.578 0.272 2.653 0.008 0.005 0.863
OCBI 0.885 0.884 0.016 54.970 0.000 0.848 0.913
OCBO 0.735 0.731 0.038 19.593 0.000 0.649 0.790
Physical 0.827 0.824 0.022 37.714 0.000 0.778 0.863
6.2 Average Variance Extracted

The average variance extracted (AVE) for the components of the model were
computed. As noted by Garson (2016), AVE indices should be at least equal to
.50. The average variance extracted indicates the probability that the construct is
measuring what it is supposed to measure. For example, the AVE for burnout
was .644. That is to say, there was a 64.4 percent chance that burnout measured
what it was supposed to measure. The results indicated that the AVE for all
constructs was higher than .50, except for in-role and OCBO. Despite these low
AVE indices, all results were statistically significant; i.e. p = .000, and none of the
confidence intervals were negative or zero. Thus, it can be concluded that the
constructs of this model enjoyed significant AVE indices.
Table 4: Average variance extracted

Confidence Intervals
AVE Mean SD t-value p-value
2.5 % 97.5 %
Burnout 0.644 0.643 0.031 20.522 0.000 0.577 0.701
Cognitive 0.588 0.585 0.040 14.522 0.000 0.507 0.662
Emotional 0.643 0.643 0.032 19.971 0.000 0.578 0.701
In-Role 0.332 0.355 0.074 4.475 0.000 0.261 0.515
OCBI 0.564 0.563 0.038 14.851 0.000 0.485 0.636
OCBO 0.322 0.322 0.037 8.637 0.000 0.247 0.391
Physical 0.565 0.566 0.029 19.287 0.000 0.503 0.623
6.3 Exploring Outer Loadings

In a PLS model, outer loadings refer to the contribution of indicators (items) to
the latent variables. Based on the results displayed in Table 6, it can be
concluded that all indicators had significant (p < .05) contributions to their latent
variables, except for the four of the items related to in-role, i.e., items 40 to 43.
Table 5: Outer loadings

Outer t-valu p-val 27.50
M SD
Loadings e ue 2.50% %
Q10 <- Burnout 0.816 0.815 0.030 27.111 0.000 0.751 0.869
Q11 <- Burnout 0.810 0.807 0.031 26.313 0.000 0.738 0.859
Q12 <- Burnout 0.831 0.830 0.024 34.833 0.000 0.781 0.868
Q13 <- Burnout 0.817 0.814 0.031 26.486 0.000 0.742 0.862
Q14 <- Burnout 0.766 0.767 0.030 25.150 0.000 0.701 0.821
Q15 <- Burnout 0.771 0.773 0.030 25.442 0.000 0.709 0.824
Q16 <- Physical 0.832 0.833 0.024 35.300 0.000 0.784 0.874
Q17 <- Physical 0.889 0.889 0.016 54.123 0.000 0.853 0.918
293
Q18 <- Physical 0.808 0.806 0.034 23.652 0.000 0.734 0.864
Q19 <- Physical 0.354 0.338 0.113 3.125 0.002 0.091 0.539
Q20 <- Cognitive 0.714 0.710 0.051 14.115 0.000 0.599 0.794
Q21 <- Cognitive 0.805 0.805 0.027 29.466 0.000 0.746 0.852
Q22 <- Cognitive 0.771 0.762 0.046 16.796 0.000 0.656 0.840
Q23 <- Cognitive 0.775 0.774 0.030 25.911 0.000 0.706 0.824
Q24 <- Emotional 0.761 0.760 0.036 21.323 0.000 0.682 0.822
Q25 <- Emotional 0.769 0.767 0.037 20.573 0.000 0.683 0.832
Q26 <- Emotional 0.899 0.900 0.013 70.789 0.000 0.873 0.922
Q27 <- Emotional 0.769 0.770 0.032 24.176 0.000 0.701 0.827
Q40 <- In-role 0.393 0.287 0.363 1.083 0.279 -0.482 0.759
Q41 <- In-role 0.361 0.246 0.397 0.909 0.364 -0.574 0.775
Q42 <- In-role 0.374 0.259 0.388 0.962 0.336 -0.540 0.778
Q43 <- In-role 0.434 0.317 0.380 1.142 0.254 -0.490 0.814
Q44 <- In-role 0.830 0.756 0.233 3.563 0.000 0.418 0.937
Q45 <- In-role 0.830 0.750 0.216 3.848 0.000 0.454 0.920
Q46 <- OCBO 0.437 0.433 0.082 5.333 0.000 0.249 0.563
Q47 <- OCBO 0.624 0.622 0.052 11.975 0.000 0.507 0.706
Q48 <- OCBO 0.625 0.617 0.072 8.727 0.000 0.450 0.733
Q49 <- OCBO 0.473 0.472 0.069 6.823 0.000 0.319 0.586
Q50 <- OCBO 0.679 0.677 0.042 16.009 0.000 0.583 0.750
Q51 <- OCBO 0.524 0.522 0.060 8.739 0.000 0.397 0.625
Q52 <- OCBI 0.769 0.769 0.028 27.462 0.000 0.711 0.819
Q53 <- OCBI 0.711 0.711 0.041 17.392 0.000 0.628 0.782
Q54 <- OCBI 0.752 0.750 0.038 19.804 0.000 0.664 0.815
Q55 <- OCBI 0.743 0.741 0.036 20.795 0.000 0.664 0.808
Q56 <- OCBI 0.827 0.825 0.027 30.427 0.000 0.764 0.869
Q57 <- OCBI 0.695 0.692 0.049 14.277 0.000 0.588 0.777
6.4 Exploring Direct Effects (First Four Research Questions)

The first four research questions explored the direct effects of physical, cognitive
and emotional demands on burnout; and direct effects of burnout on in-role and
extra-role. Table 6 and Figure 3 display the standardized path coefficients. Based
on these results, it can be concluded that;
A: Physical demand had a significant direct effect on burnout (b = .161, t = 8.75,
p = .000, 97.5 % CI [.120, .194]). Thus, it can be concluded that the first
null-hypothesis was rejected.
B: Emotional demand had a significant direct effect on burnout (b = .258, t =
10.04, p = .000, 97.5 % CI [.194, .300]). Thus, it can be concluded that the second
C: Cognitive demand had significant direct effect on burnout (b = .249, t = 7.91, p
= .000, 97.5 % CI [.179, .303]). Thus, it can be concluded that the third
294
Table 6: Direct effects (research questions one to four)

Direct P
M SD t-Value 2.5% 97.5%
Effect Values
Burnout -> Extra-Role 0.213 0.209 0.085 2.491 0.013 0.040 0.385
Burnout -> In-Role 0.314 0.320 0.100 3.133 0.002 0.185 0.442
Cognitive -> Burnout 0.249 0.247 0.031 7.918 0.000 0.179 0.303
Emotional -> Burnout 0.258 0.255 0.026 10.046 0.000 0.194 0.300
Physical -> Burnout 0.161 0.161 0.018 8.750 0.000 0.120 0.194
D: Burnout had a significant direct effect on in-role (b = .314, t = 3.13, p = .002,

97.5 % CI [.185, .442]); and extra-role (b = .213, t = 2.49, p = .013, 97.5 % CI [.040,
.385]). Based on these results, it can be concluded that the fourth null-hypothesis
was rejected, although the results should be interpreted cautiously due to the
near zero lower bound confidence interval of .040 for the direct effect of burnout
on extra-role.
Figure 3 displays the direct and indirect effects of the PLS model. The values on
the arrows are standardized path coefficients.
Figure 3: Final PLS model (standardized path coefficients)
6.5 Exploring Indirect Effects (Fifth Research Question)

The last research question explored the indirect effects of demands on in-role
and extra-role through the mediation of burnout. Based on the results displayed
in Table 8, it can be concluded that;
Demands had significant indirect effect on in-role after being mediated with
burnout (b = .169, t = 2.807, p = .005, 97.5 % CI [.086, .277]). It also had a
significant indirect effect on extra-role after being mediated with burnout (b =
295
.114, t = 2.05, p = .040, 97.5 % CI [.018, .242]). Thus, it can be concluded that the
fifth null-hypothesis was rejected, although the results should be interpreted
cautiously because the lower bound confidence intervals for these indirect
effects were almost zero; i.e. .086 and .018.
Table 7: Indirect effects (fifth research question)

Indirect t-Valu P 2.5% 97.5%
M SD
Effect e Values
Demands -> Burnout .086 .277
0.169 0.173 0.060 2.807 0.005
-> In-Role
Demands -> Burnout .018 .242
0.114 0.115 0.056 2.056 0.040
-> Extra-Role
Figure 4 displays the final model. The relationships between variables are
displayed as t-values.
Figure 4: Final PLS model (t-values)
7. Discussions
The findings of this study are generally in line with the previously published
documents. Numerous resources have been reported on the effects of job
demands (i.e., cognitive, emotional, and physical) on burnout. In most cases,
these studies have observed significant positive effects (e.g., Bakker et al., 2010;
Ismail et al., 2009; Kasinathan and Arokiasamy (2019); Makhbul & Khairuddin,
2013; Mérida-López & Extremera, 2017; Prieto et al., 2008; Saleem et al., 2017;
Watts & Robertson, 2011; Winefield et al., 2003; Zysberg et al., 2017). For
example, Kasinathan and Arokiasamy (2019), who studied Malaysian
academicians' well-being, realized that emotional and psychological wellbeing
affects the profitability of Malaysian Universities. Therefore, they suggested
enough attention to the emotional and psychological aspects of the
academicians' job demands. Elsewhere, Zysberg et al. (2017) associated burnout
with emotions and personality. This study was also conducted among academic
leaders at RUs. Congruent with these two studies, it was realized that the
296
emotional aspect of academic staffs' job demand in research universities could

affect their burnout at work.
Various aspects of job demands at Malaysian universities were studied by

Makhbul and Khairuddin (2013) who realized that excessive job demands
(psychical, emotional, and psychological) are the main resources of job-related
stress which eventually reduces job performance. They also mention that
burnout is the result of stress. Similar results were found in the current study
where emotional, physical, and cognitive demands showed significant effects on
burnout among Malaysian academicians at research universities. Also, it was
observed that burnout could affect the in-role and extra-role performance of the
Malaysian academicians at RUs. The findings are in line with the majority of
previous studies, as they have observed a direct effect of burnout on
performance. For example, Brown and Roloff (2013); Petitta and Vecchione (2011)
observed a direct effect of burnout on extra-role performance. These two studies
were conducted in the educational and non-educational sectors respectively;
however, they both showed a decrease in commitment due to burnout. Thus, the
findings are in line with the current study. Watts and Robertson (2011) posit that
burnout might be found in different occupations; however, only excessive
burnout can affect performance, as it causes emotional stress. This indicates that
the working situation in Malaysian research universities should be restudied to
make sure it does not cause emotional stress.
A direct impact on in-role performance was also observed as a result of burnout.

In line with this finding, the impact of burnout on in-role performance was
already observed among bank employees (Yavas et al., 2013), flight attendants
(Chen & Kao, 2012), and teachers (Cohen & Abedallah, 2015). Therefore, it can
be concluded that the findings of the current study are congruent with the main
trends in the literature on burnout and performance.
Burnout showed a mediating role between job demands and performance. The
majority of previous studies see burnout as the main result of job-related stress
(e.g., Cohen & Abedallah, 2015; Makhbul & Khairuddin, 2013; Watts &
Robertson, 2011). For example, Watts and Robertson (2011) assert that burnout
causes emotional stress and emotional stress affects performance. Cohen and
Abedallah (2015) also mentioned that burnout mediates the relationship of
between EI and self-efficacy among teachers. Congruent with such views
concerning burnout, in this study, the researchers observed that burnout could
mediate the impact of job demands on job performance, although this study was
not an attempt to realize what causes such a mediation. Some scholars such as
Celik (2013) believe that where job performance is negatively affected, there is a
sign of burnout. He made this conclusion by looking into role ambiguity among
teachers, which causes burnout and affects performance. In his study, burnout
was observed as a mediating role.
The findings of this study are also in line with the results gained in some other
fields. The mediating role of burnout on job performance has also been
observed in other work environments. For example, Leiter and Maslach (2009)
297
supported the mediation model of burnout, in which areas of work-life

predicted burnout.
8. Conclusion
In this study, the effects of various job demands on burnout among Malaysian
academic leaders at RUs were observed. The job demand included emotional,
cognitive, and physical demands which prove to have significant impacts on
burnout among the target population. These demands were dealt with at
individual level. In addition, the impact of burnout on in-role and extra-role
performance of Malaysian academicians was observed. This impact was
significant for both in-role and extra-role performance. In addition, the
mediating (indirect) effect of burnout between job demands and performance
was significant. This indicates that the effect of burnout on performance has a
direct and an indirect effect.
This study was a response to a need for a research on Malaysian RUs in line with
the high demands on the academicians in these institutes of higher education.
The findings can be practical for policymakers who are held accountable for
making leadership decisions in these universities. In addition, the educational
syllabus designers who set the educational syllabus for each educational year
should bear in mind that high physical, emotional, and even cognitive demand
posed through long working hours can result in burnout among the
academicians. This study is limited to the MRUs. This is due to the high
concerns of Malaysian higher education about this category of universities,
besides the high need for cognitive demand to be tested in the context of
research universities category. It is highly recommended that the Malaysian
higher education pays much attention to boosting the emotional demand within
the academicians, as it has the highest reduction impact on burnout. It is highly
suggested that the prospective researchers should focus on strategies to reduce
burnout among academic leaders. These strategies should be well studied so
that they can be used in educational contexts.
9. References
Al-Dubai, S. A. R., Ganasegeran, K., Perianayagam, W., & Rampal, K. G. (2013).
Emotional burnout, perceived sources of job stress, professional fulfillment, and
engagement among medical residents in Malaysia. The Scientific World Journal,
11(2), 86. https://doi.org/10.1155/2013/137620
Acharya, A. S., Prakash, A., Saxena, P., & Nigam, A. (2013). Sampling: Why and how of
it. Indian journal of medical specialities, 4(2), 330-333.
https://doi.org/10.7713/ijms.2013.0032
Arbae, M. N. A., Othman, N. A. M., Mahamad, N. F., Bakri, S. F. Z., & Hariri, A. (2019).
Study of perceived stress among academicians in higher institutional Malaysia.
Journal of Safety, Health & Ergonomics, 1(1).
http://fazpublishing.com/jshe/index.php/jshe/article/view/4
Bakker, A. B., Boyd, C. M., Dollard, M., Gillespie, N., Winefield, A. H., & Stough, C.
(2010). The role of personality in the job demands‐resources model. Career
Development International. https:// doi.org/10.1108/13620431011094050
Bakker, A. B., & Demerouti, E. (2007). The job demands‐resources model: State of the
art. Journal of Managerial Psychology.
https://doi.org/10.1108/02683940710733115
298
Bakker, A. B., & Demerouti, E. (2014). Job demands–resources theory. Wellbeing: A

complete reference guide, 1-28. https://doi.org/10.1002/9781118539415.wbwell019
Bakker, A. B., & Demerouti, E. (2017). Job demands–resources theory: taking stock and
looking forward. Journal of occupational health psychology, 22(3), 273.
http://dx.doi.org/10.1037/ocp0000056
Bakker, A. B., & Demerouti, E. (2018). Multiple levels in job demands-resources theory:
Implications for employee well-being and performance. In Handbook of
well-being. DEF Publishers. https://doi.org.nobascholar.com
Brown, L. A., & Roloff, M. E. (2015). Organizational citizenship behavior, organizational
communication, and burnout: The buffering role of perceived organizational
support and psychological contracts. Communication Quarterly, 63(4), 384-404.
https://doi.org/10.1080/01463373.2015.1058287
Bowen, P., Rose, R., & Pilkington, A. (2016). Perceived stress amongst university
academics. American International Journal of Contemporary Research, 6(1), 22–28.
http://www.aijcrnet.com/journals/Vol_6_No_1_February_2016/3.pdf
Bte Marmaya, N. H., Hitam, M., Zawawi, N., & Jody, J. M. (2011). Organizational
commitment and job burnout among employees in Malaysia. Proc Intl Conf on
Bus and Econs Res, 185-187. http://www.ipedr.com/vol1/40-B10039.pdf
Burkinshaw, P., & White, K. (2019). Networking and gender equality in academic
leadership. In Women, Business and Leadership. Edward Elgar Publishing.
https://doi.org/10.4337/9781786432711.00017
Celik, K. (2013). The effect of role ambiguity and role conflict on performance of vice
principals: The mediating role of burnout. Eurasian Journal of Educational Research,
51, 195–213. https://files.eric.ed.gov/fulltext/EJ1059818.pdf
Chen, C. F., & Kao, Y. L. (2012). Moderating effects of work engagement and job tenure
on burnout–performance among flight attendants. Journal of Air Transport
Management, 25, 61–63. https://doi.org/10.1016/j.jairtraman.2012.08.009
Cohen, A., & Abedallah, M. (2015). The mediating role of burnout on the relationship of
emotional intelligence and self-efficacy with OCB and performance. Management
Research Review. https://doi.org/10.1108/MRR-10-2013-0238
Crossan, F. (2003). Research philosophy: Towards an understanding. Nurse Researcher,
11(1), 46.
DeFrank, R. S., & Ivancevich, J. M. (1998). Stress on the job: An executive update.
Academy of Management Executives, 12, 55–66.
https://doi.org/10.5465/ame.1998.1109050
Dörnyei, Z., & Taguchi, T. (2009). Questionnaires in second language research:
Construction, administration, and processing. Routledge.
http://golshanlc.com/wp-content/uploads/2019/09/Questionnaires_in_Secon
d_Language_Research__Construction__Administration__and_Processing__Seco
nd_Language_Acquisition_Research_Series_.pdf
Friedman, I. A. (2000). Burnout in teachers: Shattered dreams of impeccable professional
performance. Journal of Clinical Psychology, 56(5), 595–606.
Fullan, M., & Scott, G. (2009). Turnaround leadership for higher education. John Wiley &
Sons. https://doi.org/10.2202/1940-1639.1687
Garson, G. D. (2016). Partial least squares: Regression and structural equation models.
Statistical Associates Publishers.
http://www.statisticalassociates.com/pls-sem.htm
Grapragasem, S., Krishnan, A., & Mansor, A. N. (2014). Current trends in Malaysian
higher education and the effect on education policy and practice: An overview.
International Journal of Higher Education, 3(1), 85–93.
http://dx.doi.org/10.5430/ijhe.v3n1p85
299
Hair, J. J. F., Sarstedt, M., Matthews Lucy, M., & Ringle Christian, M. (2016). Identifying
and treating unobserved heterogeneity with FIMIX-PLS: part I – method.
European Business Review, 28(1), 63-76. http://doi.org/10.1108/EBR-09-2015-0094
Hair, J. F., Jr., Hult, G. T. M., Ringle, C., & Sarstedt, M. (2017). A primer on partial least
squares structural equation modeling (PLS-SEM). Sage publications.
https://us.sagepub.com/en-us/nam/a-primer-on-partial-least-squares-structur
al-equation-modeling-pls-sem/book244583
House, R. J. (1971). A path goal theory of leader effectiveness. Administrative Science
Quarterly, 321–339.
House, R. J. (1996). Path-goal theory of leadership: Lessons, legacy, and a reformulated
theory. The Leadership Quarterly, 7(3), 323–352.
https://doi.org/10.1016/S1048-9843(96)90024-7
House, R. J., & Dessler, J. (1974). The pathgoal theory of leadership: Some post hoc and a
priori tests. In J. G. Hunt & L. L. Larson (Eds.), Contingency approaches to
leadership, 29-55. Southern Illinois University Press.
https://doi.org/10.2307/255527
Ilies, R., Huth, M., Ryan, A. M., & Dimotakis, N. (2015). Explaining the links between
workload, distress, and work–family conflict among school employees: Physical,
cognitive, and emotional fatigue. Journal of Educational Psychology, 107(4), 1136.
http://dx.doi.org/10.1037/edu0000029
Ismail, N. H., & Noor, A. (2016). Occupational stress and its associated factors among
academician in a research university, Malaysia. Malaysian Journal of Public Health
Medicine, 16(1), 81–91.
Ismail, A., Suh-Suh, Y., Ajis, M. N. E., & Dollah, N. F. (2009). Relationship between
occupational stress, emotional intelligence and job performance: An empirical
study in Malaysia. Theoretical & Applied Economics, 16(10), 3–16.
Jex, S. M., Beehr, T. A., & Roberts, C. K. (1992). The meaning of occupational stress items
to survey respondents. Journal of Applied Psychology, 77(5), 623.
https://doi.org/10.1037/0021-9010.77.5.623
Kasinathan, J. P., & Arokiasamy, L. (2019). A Study on Academicians' Well-Being in
Malaysian Universities: A Conceptual Paper. Global Business and Management
Research, 11(1), 446-454.
Khairuddin, S. M. H. S., & Makhbul, Z. K. M. (2011). Stress at the workplace and
productivity: A pilot study on faculty administrators in a Malaysian research
university. In proceedings of the 2nd International Conference on Business and
Economic Research (2nd ICBER), Malaysia.
Kinman, G., & Jones, F. (2005). Lay representations of workplace stress: What do people
really mean when they say they are stressed? Work & Stress, 19(2), 101–120.
https://doi.org/10.1080/02678370500144831
Kristensen, T. S., Hannerz, H., Høgh, A., & Borg, V. (2005). The Copenhagen
Psychosocial Questionnaire—A tool for the assessment and improvement of the
psychosocial work environment. Scandinavian Journal of Work, Environment &
Health, 438–449. http://www.jstor.org/stable/40967527
Leiter, M. P., & Maslach, C. (2009). Nurse turnover: The mediating role of burnout.
Journal of Nursing Management, 17(3), 331–339.
https://doi.org/10.1111/j.1365-2834.2009.01004.x
Lorello, G. R., Cil, T., & Flexman, A. M. (2020). Women anesthesiologists’ journeys to
academic leadership: A constructivist grounded theory-inspired study. Canadian
Journal of Anaesthesia, 67(9), 1130–1139.
https://doi.org/10.1007/s12630-020-01735-z
Madonko, N., & April, K. A. (2020). The Effectiveness of Leaders in Communicating
Organizational Values. Effective Executive, 23(4), 22-38.
300
Makhbul, Z. M., & Khairuddin, S. M. H. S. (2013). Stress among Malaysian academics: A

conceptual study. International Journal of Academic Research in Business and Social
Sciences, 3(1), 196.
Mérida-López, S., & Extremera, N. (2017). Emotional intelligence and teacher burnout: A
systematic review. International Journal of Educational Research, 85, 121–130.
https://doi.org/10.1016/j.ijer.2017.07.006
Meyer, S.-C., & Hünefeld, L. (2018). Challenging cognitive demands at work, related
working conditions, and employee well-being. International journal of
environmental research public health, 15(12), 2911.
https://doi.org/10.3390/ijerph15122911
Ministry of Higher Education (2009). The Malaysian Higher Education System - An
Overview.
https://www.studymalaysia.com/international/the-national-education-system
/the-malaysian-higher-education-system-an-overview
Muchinsky, P. M. (2006). Psychology applied to work: An introduction to industrial and
organizational psychology. Cengage Learning.
https://psycnet.apa.org/record/2001-06886-000
Mustapha, N. (2013). Measuring job satisfaction from the perspective of interpersonal
relationship and faculty workload among academic staff at public universities in
Kelantan, Malaysia. International Journal of Business and Social Science, 4(15),
120–124.
Nahrgang, J. D., Morgeson, F. P., & Hofmann, D. A. (2011). Safety at work: A
meta-analytic investigation of the link between job demands, job resources,
burnout, engagement, and safety outcomes. Journal of Applied Psychology, 96(1),
71. https://doi.org/10.1037/a0021484
Omar, M. (2018). Occupational stress and its associated factors among academician in
northern Malaysian universities. http://repo.uum.edu.my/24709/
Pejtersen, J. H., Kristensen, T. S., Borg, V., & Bjorner, J. B. (2010). The second version of
the Copenhagen Psychosocial Questionnaire. Scandinavian Journal of Public Health,
38(3_suppl), 8–24. https://doi.org/10.1177/1403494809349858
Petitta, L., & Vecchione, M. (2011). Job burnout, absenteeism, and extra role behaviors.
Journal of Workplace Behavioral Health, 26(2), 97–121.
https://doi.org/10.1080/15555240.2011.573752
Prieto, L. L., Soria, M. S., Martínez, I. M., & Schaufeli, W. (2008). Una ampliación del
modelo demandas-recursos laborales en la predicción del burnout y del
engagement en profesores. [Extension of the job demands-resources model in
the prediction of burnout and engagement among teachers over time.].
Psicothema, 20(3), 354-360. https://psycnet.apa.org/record/2008-10293-003
Qamar, K., Khan, M. G. S. A., Khan, S. A., & Tasawar, A. (2019). Educational and
academic leadership. The Professional Medical Journal, 26(05), 769–775.
https://doi.org/10.29309/TPMJ/2019.26.05.3475
Rahman, R. A., Mustaffa, W. S. W., Wahid, H. A., & Yunus, N. K. Y. (2019). Exploring the
factors affecting quality of work life among millennial academic staff in
Malaysian public universities. International Journal of Asian Social Science, 9(6),
379–389. http://www.aessweb.com/html/4606
Rathmell, W. K., Brown, N. J., & Kilburg, R. R. (2019). Transformation to academic
leadership: The role of mentorship and executive coaching. Consulting Psychology
Journal: Practice and Research, 71(3), 141. http://dx.doi.org/10.1037/cpb0000124
Raziq, M. M., Borini, F. M., Malik, O. F., Ahmad, M., & Shabaz, M. (2018). Leadership
styles, goal clarity, and project success. Leadership & Organization Development
Journal, 39(2), 309-323. https://doi.org/10.1108/LODJ-07-2017-0212
301
Saleem, Z., Batool, S., & Khattak, S. R. (2017). Relationship between leadership styles and
organizational commitment: Moderating role of emotional intelligence and
organizational support. Journal of Managerial Sciences, 11(1).
https://www.qurtuba.edu.pk/jms/default_files/JMS/11_1/JMS_January_June
2017_71-84.pdf
Sinniah, S., Makhbul, M., Khan, Z., Perumal, G., Mohamed, H., & Moona, R. K. (2018).
Career motivation among lecturers’ working at private universities in Malaysia.
International Journal of Business & Management Science, 8(2).
Schaufeli, W. B., & Bakker, A. B. (2004). Job demands, job resources, and their
relationship with burnout and engagement: A multi-sample study. Journal of
Organizational Behaviour, 25, 293-315. https://doi.org/10.1002/job.248
Spector, P. E. (2019). Do Not Cross Me: Optimizing the Use of Cross-Sectional Designs.
Journal of Business and Psychology, 34(2), 125-137.
https://doi.org/10.1007/s10869-018-09613-8
Sung, T. P., Yee, G. C. S., Bahron, A., & Rahim, I. H. A. (2017). The influence of training,
employee engagement and performance appraisal on turnover intention among
lecturers in Sabah private higher education institutions. Journal of Global Business
and Social Entrepreneurship, 1(3), 89–98.
http://gbse.com.my/v1no3jan17/Paper-48-.pdf
Tseng, W. T., Dörnyei, Z., & Schmitt, N. (2006). A new approach to assessing strategic
learning: The case of self-regulation in vocabulary acquisition. Applied linguistics,
27(1), 78-102. https://doi.org/10.1093/applin/ami046
Watts, J., & Robertson, N. (2011). Burnout in university teaching staff: A systematic
literature review. Educational Research, 53(1), 33–50.
https://doi.org/10.1080/00131881.2011.552235
Williams, L. J., & Anderson, S. E. (1991). Job satisfaction and organizational commitment
as predictors of organizational citizenship and in-role behaviors. Journal of
Management, 17(3), 601–617. https://doi.org/10.1177/014920639101700305
Winefield, A. H., Gillespie, N., Stough, C., Dua, J., Hapuarachchi, J., & Boyd, C. (2003).
Occupational stress in Australian university staff: Results from a national
survey. International Journal of Stress Management, 10(1), 51.
https://psycnet.apa.org/buy/2003-06067-006
Yavas, U., Babakus, E., & Karatepe, O. M. (2013). Does hope moderate the impact of job
burnout on frontline bank employees’ in-role and extra-role performances?
International Journal of Bank Marketing.
https://doi.org/10.1108/02652321311292056
Yousefi, M., & Abdullah, A. G. K. (2019). The impact of organizational stressors on job
performance among academic staff. International Journal of Instruction, 12(3),
561–576. https://eric.ed.gov/?id=EJ1220197
Yukl, G., & Lepsinger, R. (2006). Leading change: Adapting and innovating in an
uncertain world. Leadership in Action: A Publication of the Center for Creative
Leadership and Jossey‐Bass, 26(2), 3–7. https://doi.org/10.1002/lia.1154
Zarb, N. (2016). Stories from the middle: Perceptions of the roles of curriculum
development managers in a Maltese further education organization. Journal of
Advanced Research in Social Sciences and Humanities, 1(1), 18–29.
https://dx.doi.org/10.26500/JARSSH-01-2016-0103
Zysberg, L., Orenshtein, C., Gimmon, E., & Robinson, R. (2017). Emotional intelligence,
personality, stress, and burnout among educators. International Journal of Stress
Management, 24(S1), 122. https://doi.org/10.1037/str0000028
302
Appendix
Variables Measurements
Keywords:
BO = Burnout
IDW= Physical demands
IDC= Cognitive demands
IDE= Emotional demands
IN-P= In-role performance
OCB-O= Extra-role performance (organizational citizenship behavior toward organization)
OCB-I= Extra-role performance (organizational citizenship behavior toward individual)
Code NO Items Never Seldom Sometimes Often Always
BO1 10 How often have you felt
1 2 3 4 5
worn out?
BO2 11 How often have you been
1 2 3 4 5
physically exhausted?
BO3 12 How often have you been
1 2 3 4 5
emotionally exhausted?
BO4 13 How often have you felt
1 2 3 4 5
tired?
BO5 14 How often do you think: ’’I
1 2 3 4 5
can’t take it anymore’’?
BO6 15 How often do you
1 2 3 4 5
susceptible to illness?
IDW1 16 My workload is unevenly
1 2 3 4 5
distributed so it piles up.
IDW2 17 I don’t have time to
complete all my work 1 2 3 4 5
tasks.
IDW3 18 I do get behind with my
1 2 3 4 5
work schedule.
IDW4 19 I have enough time for my
1 2 3 4 5
work tasks (rs).
IDC1 20 I have to keep my eyes on
lots of things during my 1 2 3 4 5
work.
IDC2 21 My tasks need to remember
1 2 3 4 5
a lot of things.
IDC3 22 My work demands that I
am improving at coming 1 2 3 4 5
up with new ideas.
IDC4 23 My work requires me to
1 2 3 4 5
make difficult decisions.
IDE1 24 My work puts me in
emotionally disturbing 1 2 3 4 5
situations.
IDE2 25 I have to relate to other
people’s problems as part 1 2 3 4 5
of my work.
IDE3 26 My task is emotionally
1 2 3 4 5
demanding.
IDE4 27 I get emotionally involved
1 2 3 4 5
in my work.
303
Strongly Disagree Neutral Agree Strongly

Code NO Items
disagree agree
IN-P1 40 I fulfill all the responsibilities
specified in my job 1 2 3 4 5
description.
IN-P2 41 I meet the formal performance
1 2 3 4 5
requirements of my job.
IN-P3 42 I conscientiously perform
1 2 3 4 5
tasks that are expected of me.
IN-P4 43 I adequately complete all of
1 2 3 4 5
my assigned duties.
IN-P5 44 I sometimes fail to perform
1 2 3 4 5
essential duties of my job (rs).
IN-P6 45 I sometimes neglect aspects of
the job that I am obligated to 1 2 3 4 5
perform (rs).
OCB-O1 46 I sometimes take undeserved
1 2 3 4 5
work breaks (rs).
OCB-O2 47 I adhere to informal
organizational rules devised 1 2 3 4 5
to maintain order.
OCB-O3 48 I always give advance notice
when I am unable to come to 1 2 3 4 5
work.
OCB-O4 49 I sometimes spend a lot of

time on personal phone 1 2 3 4 5
conversations (rs).
OCB-O5 50 My attendance at work is
1 2 3 4 5
above the norm.
OCB-O6 51 I sometimes complain about
minor things at work (rs). 1 2 3 4 5
OCB-I1 52 I generally help others who

have been absent. 1 2 3 4 5
OCB-I2 53 I take a personal interest in the

well-being of other 1 2 3 4 5
employees.
OCB-I3 54 I generally help others who

1 2 3 4 5
have heavy workloads.
OCB-I4 55 I go out of the way to help
1 2 3 4 5
new employees.
OCB-I5 56 I generally take time to listen
to co-workers’ problems. 1 2 3 4 5
OCB-I6 57 I pass along work-related

1 2 3 4 5
information to co-workers.
PUBLISHER International Journal of Learning, Teaching and
Society for Research and Knowledge Management Educational Research
Port Louis
Republic of Mauritius
www.ijlter.org The International Journal of Learning, Teaching
and Educational Research is an open-access
Chief Editor journal which has been established for the dis-
Dr. Antonio Silva Sprock, Universidad Central de semination of state-of-the-art knowledge in the
Venezuela, Venezuela, Bolivarian Republic of field of education, learning and teaching. IJLTER
welcomes research articles from academics, ed-
Editorial Board
ucators, teachers, trainers and other practition-
Dr. Fatima Zohra Belkhir-Benmostefa
ers on all aspects of education to publish high
Dr. Giorgio Poletti
quality peer-reviewed papers. Papers for publi-
Dr. Chi Man Tsui
cation in the International Journal of Learning,
Dr. Fitri Suraya Mohamad
Teaching and Educational Research are selected
Dr. Hernando Lintag Berna
through precise peer-review to ensure quality,
Dr. Charanjit Kaur Swaran Singh
originality, appropriateness, significance and
Dr. Abu Bakar
readability. Authors are solicited to contribute
Dr. Eglantina Hysa
to this journal by submitting articles that illus-
Dr. Mo'en Salman Alnasraween
trate research results, projects, original surveys
Dr. Hermayawati Hermayawati
and case studies that describe significant ad-
Dr. Selma Kara
vances in the fields of education, training, e-
Dr. Michael B. Cahapay
learning, etc. Authors are invited to submit pa-
Dr. Bunmi Isaiah Omodan
pers to this journal through the ONLINE submis-
Dr. Vassiliki Pliogou
sion system. Submissions must be original and
Dr. Meera Subramanian
should not have been published previously or
Dr. Muhammad Kristiawan
be under consideration for publication while
Dr. Wahyu Widada
being evaluated by IJLTER.
Dr. Som Pal Baliyan
Dr. Reem Khalid Abu-Shawish
Dr. Froilan Delute Mobo
Dr. Mohamed Ali Elkot
Dr. Anabelie Villa Valdez
Mr. Teody Lester Verdeflor Panela

Vol 20 No 3 March 2021

Uploaded by

Copyright:

Available Formats

Vol 20 No 3 March 2021

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Vol 20 No 3 March 2021

Uploaded by

Copyright:

Available Formats

IJLTER.

Society for Research and Knowledge Management

The International Journal of Learning, Teaching and Educational

Aims and Objectives

The main objective of this journal is to provide a platform for educators,

Indexing and Abstracting

The International Journal of Learning, Teaching and Educational

We are very happy to publish this issue of the International Journal of

The International Journal of Learning, Teaching and Educational

We are grateful to the editor-in-chief, members of the Editorial Board

The Editorial Board is composed of renowned people from across the

We will endeavour to ensure the reputation and quality of this journal

Editors of the March 2021 Issue

Innovative Teaching: A Qualitative Review of Flipped Classrooms ............................................................................. 18

International Journal of Learning, Teaching and Educational Research

Effective Social Studies Pedagogy:

Sunday Obro, Clifford Edore Ogheneakoke and Williams P. Akpochafo

Abstract. This study explored the effects of game simulation and

Keywords: brainstorming; pedagogy; Social Studies; simulation game;

Simulation games challenge students’ understanding to comprehend difficult

Study outcomes of other research have shown that brainstorming and

will not be found among brainstorming, simulation-games, and lecture

The hypotheses listed below were formulated and tested:

1.1 Practical gaps

Simulation games give students the understanding to comprehend difficult

According to Ezeudu and Ezinwanne (2013), a simulation game is often referred

However, despite the several positive educational benefits or advantages, it still

2.2 Brainstorming as a Pedagogical Strategy

Jack and Kyado (2017) opined that a brainstorming strategy is an instructional

However, in spite of all the usefulness and qualities of a brainstorming strategy,

2.3 Studies Related to Simulation Games and Students’ Learning Outcome

2.4 Studies Related to Brainstorming Strategy and Students’ Learning

et al. (2016) explored the brainstorming strategy efficacy on students’ previous

The theory provided the theoretical foundation for organising simulation-game

Figure 1: The conceptual framework

Table 1: Research blueprint

The codes are explained as follows:

As detailed previously, an intervention was introduced to the two experimental

Table 2: Test blueprint for Social Studies learning outcomes (SSLOT)

Instrument reliability was confirmed by using 30 students in a test-retest

4.4 Research Procedure

After six weeks of experimentation or treatment, a posttest was administered to

Table 3: Students’ pretest and posttest learning outcome scores of simulation-game

Table 3 shows that students taught using a simulation-game pedagogical

Table 4: Results of ANCOVA of learning outcome test according to simulation game

Table 4 shows that students involved in a simulation-game pedagogical strategy

Table 5: Students’ retest and posttest learning outcome scores of brainstorming

Table 6: Results of ANCOVA of learning outcomes test according to brainstorming

As shown in Table 6, students involved in the brainstorming conditions

Table 7: Pretest/Posttest of students’ learning outcomes by strategies

Table 8: ANCOVA summary of the posttest according to strategies

Table 8 displays a statistically significant difference among simulation-game,

Table 9: Scheffe’s posthoc analysis by strategies

A simulation game pedagogical strategy is significantly superior to

simulation-game pedagogical strategy in improving students’ learning outcomes

9. Implication of the study

EFL learners’ background knowledge in essay writing classes. Indian Journal

International Journal of Learning, Teaching and Educational Research

Innovative Teaching: A Qualitative Review of