Vol 22 No 7 July 2023

IJLTER.
ORG
International Journal
of
Learning, Teaching
And
Educational Research
p-ISSN: 1694-2493
e-ISSN: 1694-2116
Vol.22 No.7
International Journal of Learning, Teaching and Educational Research
(IJLTER)
Vol. 22, No. 7 (July 2023)
Print version: 1694-2493
Online version: 1694-2116
IJLTER
International Journal of Learning, Teaching and Educational Research (IJLTER)
Vol. 22, No. 7
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part
of the material is concerned, specifically those of translation, reprinting, re-use of illustrations,
broadcasting, reproduction by photocopying machines or similar means, and storage in data banks.
Society for Research and Knowledge Management

The International Journal of Learning, Teaching and Educational

Research is a peer-reviewed open-access journal which has been
established for the dissemination of state-of-the-art knowledge in the
fields of learning, teaching and educational research.
Aims and Objectives
The main objective of this journal is to provide a platform for educators,

teachers, trainers, academicians, scientists and researchers from over the
world to present the results of their research activities in the following
fields: innovative methodologies in learning, teaching and assessment;
multimedia in digital learning; e-learning; m-learning; e-education;
knowledge management; infrastructure support for online learning;
virtual learning environments; open education; ICT and education;
digital classrooms; blended learning; social networks and education; e-
tutoring: learning management systems; educational portals, classroom
management issues, educational case studies, etc.
Indexing and Abstracting

Research is indexed in Scopus since 2018. The Journal is also indexed in
Google Scholar and CNKI. All articles published in IJLTER are assigned
a unique DOI number.
Foreword
We are very happy to publish this issue of the International Journal of

Learning, Teaching and Educational Research.

Research is a peer-reviewed open-access journal committed to
publishing high-quality articles in the field of education. Submissions
may include full-length articles, case studies and innovative solutions to
problems faced by students, educators and directors of educational
organisations. To learn more about this journal, please visit the website
http://www.ijlter.org.
We are grateful to the editor-in-chief, members of the Editorial Board

and the reviewers for accepting only high quality articles in this issue.
We seize this opportunity to thank them for their great collaboration.
The Editorial Board is composed of renowned people from across the

world. Each paper is reviewed by at least two blind reviewers.
We will endeavour to ensure the reputation and quality of this journal

with this issue.
Editors of the July 2023 Issue

VOLUME 22 NUMBER 7 July 2023
Table of Contents
Assessment Literacy, Current Assessment Practices and Future Training: Reflections of Teachers in Higher
Education ................................................................................................................................................................................. 1
Sophia Gaikwad, Ashwini Wadegaonkar, Gargee Mitra, Devjani Chakravarty
Principals’ Leadership Orientation and Students’ Academic Performance in Secondary Schools of Gedeo Zone,
Ethiopia .................................................................................................................................................................................. 30
RJ (Nico) Botha, Seyoum Gari Aleme
Chinese University English-Major Students’ Attitude Toward Literature Circles and Literature Reading ............. 52
Lei Ma, Lilliati Ismail, Norzihani Saharuddin
Performance Analysis Towards Excellent Schools ........................................................................................................... 70

Abednego Abednego, Patrisius Rahabav, John Rafafy Batlolona
Experiences of Student-Teachers: Implications for Refined Student-Support .............................................................. 87

Adesegun Olayide Odutayo, Sarita Ramsaroop
Stakeholders’ Perspectives of Early Childhood Education Language and Literacy Laboratories in the United Arab
Emirates................................................................................................................................................................................ 104
Aysha AlShamsi, Phil Quirke
Challenges and Opportunities of AI-Assisted Learning: A Systematic Literature Review on the Impact of
ChatGPT Usage in Higher Education .............................................................................................................................. 122
Alfonso Renato Vargas-Murillo, Ilda Nadia Monica de la Asuncion Pari-Bedoya, Francisco de Jesús Guevara-Soto
Project-based Learning to Promote Learner Autonomy in Training Hospitality Education at a Technical and
Vocational Education and Training College ................................................................................................................... 136
Shawn Lourens Green, Elizabeth Catharina (Elize) du Plessis
Implementation of Strategy Instruction in Teaching English as a Foreign Language: A Systematic Review ........ 156
Li Su, Nooreen Noordin, Joanna Joseph Jeyaraj
Stakeholders’ Perceptions of Teaching and Technological Skills in EFL Vocabulary Instruction: Implications for
Remote Learning ................................................................................................................................................................. 173
Paul Gonzalez-Torres, Paola Cabrera-Solano, Luz Castillo-Cuesta
Self-Instructional Teaching Internship Module: An Evaluation ................................................................................... 193

Imelda C. Montalbo, Maria Nancy Quinco-Cadosales, Angeline M. Pogoy, Jo Ann M. Petancio
The Role of Multi-dimensional Curriculum Design in Improving Higher-Order Thinking Skills .......................... 219
Anita Yosepha, Mohammad Ali, Dinn Wahyudin, Rusman Rusman
GO-DEEP: A Potential Reflection Model for Experiential Learning ............................................................................ 240

Ngoc Thi-Nhu Bui, Pratchayapong Yasri
An Investigation into the Benefits and Challenges of International Student Exchange Programs: Perspectives
from Student Teachers ....................................................................................................................................................... 258
Huynh Thi Thuy Diem, Mai Phuc Thinh, Tran Thi Mung
Application of the Chatbot in University Education: A Bibliometric Analysis of Indexed Scientific Production in
SCOPUS, 2013-2023 ............................................................................................................................................................ 281
Omar Chamorro-Atalaya, Soledad Olivares-Zegarra, Lisle Sobrino-Chunga, Rosemary Guerrero-Carranza, Ademar Vargas-
Diaz, Madison Huarcaya-Godoy, José Rasilla-Rovegno, Raul Suarez-Bazalar, Jorge Poma-Garcia, Yreneo Cruz-Telada
Applying E-Writing Therapy to Improve Mental Wellbeing among Malaysian University Students Following the
COVID-19 Pandemic .......................................................................................................................................................... 305
Ying Qin Tee, Kee Pau, Mahmoud Danaee
Effectiveness of a Mixed Methods-Based Literacy Program in Improving Reading Comprehension, Vocabulary

Mastery, and Reading Fluency Skills of Early Grade Students .................................................................................... 324
Indah Nurmahanani
A Conceptual Analysis of What it Means to Decolonize the Curriculum ................................................................... 344

Bunmi Isaiah Omodan, Pretty Thandiswa Mpiti, Nomxolisi Mtsi
The Impact of Project-Based and Experiential Learning Integration on Pre-Service Teacher Achievement in
Evaluation and Assessment ............................................................................................................................................... 356
Apantee Poonputta
Teachers’ Professional Development and Pedagogical Shift towards Dialogic Teaching in Malaysian Lower
Secondary ESL Classrooms ............................................................................................................................................... 371
Sheila Adelina Ramasamy, Azlin Zaiti Zainal
Critical Factors for Enhancing Students’ Collaborative Learning Experiences in a Project-based Connectivism
Learning Environment ....................................................................................................................................................... 388
Fui Theng Leow, Mai Neo
Innovation in Early Reading Instruction: The Development of e-Learning Materials in Mother Tongue.............. 411
J-Roel B. Semilla, Venus R. Parmisana, Loreta L. Fajardo, Ruben L. Abucayon, Angeline P. Dinoro, Josefina M. Tabudlong
Empowering Children through Sex Education: A Study on Kindergarten Policies in Indonesia ............................ 434
Munir Yusuf, Firman Firman, Hasriadi Hasriadi, Mirnawati Mirnawati
Contributions and Partnership Strategies of External Stakeholders in the Implementation of the Alternative
Learning System in Bontoc District: Insights from Teachers ........................................................................................ 454
Johnny P. Cayabas, Christie Lynne C. Codod, Delia A. Sumeg-ang, Elizabeth Lacaben
Faculty Members’ Awareness of Artificial Intelligence and Its Relationship to Technology Acceptance and Digital
Competencies at King Faisal University .......................................................................................................................... 473
Sherif Adel Gaber, Hussein Ahmed Shahat, Ibrahim Abdullah Alkhateeb, Sumaia Attia Al Hasan, Mohammed Ahmed
Alqatam, Sultan Mubarak Almughyirah, Mohammed keshar Kamel
Enhancing Reading-Comprehension Abilities and Attitudes of EFL Students through utilising Content-Creation
Tools in Classroom Presentations ..................................................................................................................................... 497
Pattama Panyasai
Engaging English as a Foreign Language (EFL) Students through the Game-Based Learning Approach in Higher
Education ............................................................................................................................................................................. 517
Cesar Ochoa-Cueva, Luz Castillo-Cuesta, Paola Cabrera-Solano
A Survey of Mathematics Pre-Service Teachers' End-of-Teaching Practice Reflections of Educational Contexts . 535
Benjamin Tatira
Flipped Direct Instruction (FDI): A New Practicum Learning Model in Vocational Education .............................. 547
Akrimullah Mubai, Ambiyar Ambiyar, Dedy Irfan, Mohamad Sattar Rasul
Mentors’ and Mentees’ Experiences in a Sino-Foreign Cooperative Education Programme: A Phenomenological

Study in a Public University in China .............................................................................................................................. 566
Ying Tang, Lin Siew Eng
Factors that Parents in South Africa Consider in Support of their Children’s Continuous Use of Online Learning
............................................................................................................................................................................................... 586
David Mutambara
1

Vol. 22, No. 7, pp. 1-29, July 2023
https://doi.org/10.26803/ijlter.22.7.1
Received Apr 16, 2023; Revised July 7, 2023; Accepted July 18, 2023
Assessment Literacy, Current Assessment

Practices and Future Training:
Reflections of Teachers in Higher Education
Sophia Gaikwad
Symbiosis Teaching Learning Resource Centre (STLRC),
Symbiosis International (Deemed University), India
Ashwini Wadegaonkar
Symbiosis Teaching Learning Resource Centre (STLRC),
Symbiosis International (Deemed University), India
Gargee Mitra
Symbiosis Schools Central Directorate, India
Devjani Chakravarty
Savitribai Phule Pune University, India
Abstract. The aim of this study is to explore assessment literacy, to

analyse the current assessment practices of teachers in Higher Education
(HE) and to understand the need for alternative assessment practices in
light of teachers’ reflections. The research employed incidental sampling
to select (n=58) HE teachers from a single multidisciplinary university in
Maharashtra State, India. A comprehensive literature review on
assessment literacy provided the theoretical foundation, and a specialized
assessment tool was developed to assess the teachers' assessment literacy
level, perspectives, and practices in assessment. Data were collected
through voluntary participation and analysed using quantitative and
qualitative techniques. The findings revealed that the majority of HE
teachers (79%) demonstrated a moderate level of assessment literacy,
while 16% exhibited low levels and only 5% showed high levels.
Additionally, a discrepancy was observed between the preferred
assessment types of teachers and learners, with learners emphasizing
active participation and real-world applications, while teachers leaned
toward traditional evaluation methods. Participants were found to be
literate only in specific aspects such as meaning, basic forms of
assessments, and the purpose of educational assessments. Furthermore,
teachers minimally acknowledge and have a lesser preference for
alternative assessment methods, such as article analysis, team projects,
case studies, and discussions. Overall findings indicate teachers' lack of
©Authors
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0
International License (CC BY-NC-ND 4.0).
2
knowledge about assessment practices that involve evaluating student

performance, fostering collaboration, assessing higher-order thinking
skills, and utilizing assessment as a tool for learning and improvement.
This research confirms the need for training to enhance teachers'
assessment literacy, promoting the adoption of alternative assessment
practices in higher education to optimize student learning experiences.
Keywords: Assessment literacy; Current assessment practices;

Alternative assessments; Higher education teachers
1. Introduction
Assessment is crucial in our rank-dominated system but also leads to high
dropout rates in higher education (HE). Effective assessment practices can help to
identify and address the factors contributing to dropout rates such as learning
difficulties, lower engagement, or inadequate support. By understanding
students' learning needs and progress through assessment, educational
institutions can implement targeted and timely interventions and personalized
support to improve retention rates, thereby ensuring the success of all learners.
India's Higher Education (HE) sector is critical in meeting the needs of its vast
youth population, which exceeds 38.5 million. The National Education Policy,
2020 (NEP 2020), set a target to increase the HE Gross Enrollment Ratio (GER)
from 26.3% in 2018 to 50% by 2035. This expansion is expected to be facilitated by
the adoption of Open and Distance Learning methods. Furthermore, the NEP 2020
is committed to offering high-quality higher education, with equity and inclusion.
However, reports reveal enormous student dropout rates between the secondary
stage and higher education (Ministry of Human Resource Development, 2016).
The constructivist approach to assessments emphasizes that ‘learners are the

constructors of their knowledge.’ It promotes student initiative, self-discipline,
and choice. Furthermore, it encourages learner engagement and provides ample
opportunities for students to express their learning through preferred assessment
tasks. Alternative assessments require responsive instruction, allowing teachers
to make use of learners' abilities to improve the quality of learning.
Reports have shown that there is an enormous student dropout rate from
secondary stage onwards, extending to higher education (MHRD report,
2016). When it comes to educational assessment, the stakes are very high, since
it determines students’ individual learning and sways development at the macro
level. Therefore, assessment standards must be carefully determined in terms of
quality enhancement, rather than relying upon the instinctive judgment of
teachers. To achieve this, assessment literacy must be given due importance in
teachers’ professional development. However, the question remains whether the
current assessment practices effectively indicate learners’ skill levels or whether
they are merely a tool for testing the learners’ memories.
Underachievement in assessments could be a significant factor in inducing

dropout (Can et al., 2017; Paura & Arhipova, 2014). In this vein, the NEP 2020
states that Higher Education Institutions shall move to a criterion-based grading system
that assesses student achievement based on learning goals for each programme, making
http://ijlter.org/index.php/ijlter
3
the system fairer and outcomes more comparable. HEIs shall move away from high-stakes
examinations towards more continuous and comprehensive evaluation –teachers will
also have more autonomy in terms of the selection of assessment method. The
questions remains, however, whether teachers possess the necessary skills to
make informed decisions regarding the most appropriate assessment format,
which is a crucial determinant of students' future outcomes.
Assessment in education has a decisive role to play. It measures the learning and
competence of students in terms of their scores and grades while also assisting
teachers with reflective and remedial teaching. In a formal system of learner-
centric education, assessment has a greater impact on learners than the teacher or
the institution. Several factors can potentially affect the output of assessments.
From the learners' perspectives, these could be individual learning styles,
memory, interest, and other psychological factors. From an institutional
standpoint, factors can include the teaching style, teachers’ assessment literacy,
classroom environment, instruments used for assessment, subject, and type of
assessment (DeLuca et al., 2019). Nevertheless, teachers’ knowledge of assessment
is an undisputed factor in determining the validity of assessment.
Assessment literacy can be understood as teachers’ abilities to comprehend the

meaning, forms, purposes, strategies, and techniques of assessment, and apply
them appropriately. According to the Michigan Assessment Consortium (2015),
assessment literacy encompasses a range of beliefs, knowledge, and practices that
enable teachers and other stakeholders to utilize assessment effectively for
enhancing student learning and achievement. It involves having the necessary
knowledge, skills, and processes to design, select, implement, and score. It utilizes
high-quality assessments that contribute to improved student learning outcomes.
In essence, assessment literacy empowers educators to make informed decisions
about assessment methods and utilize assessment data to support and enhance
student learning.
The essential knowledge and performance components are presented in Figure 1

below.
Figure 1: Essentials of assessment literacy

(Adapted from Michigan Assessment Consortium, 2020, pg. 5)
4
Traditionally, the paradigm of assessment was utilized as a mere tool for learners’
evaluation. Over time, educational researchers have broadened the dimensions to
encompass ‘Assessment as Learning’, ‘Assessment for Learning’ and ‘Assessment
of Learning’. ‘Assessment for learning’ provides information about learners
during their learning and assists teachers in obtaining information about students'
learning and teaching practices that can be modified to improve learning. On the
other hand, 'Summative and Formative assessments:’ refer to the traditional
summative evaluation process, which provides evidence of students' learning at
the end of the learning period, while ‘Assessment as Learning’ is a self-monitoring
tool used by learners to evaluate their learning. In a nutshell, assessment literacy
is demonstrated when teachers have sufficient knowledge of assessment as well
as the capacity to comprehend its shifting paradigm from teachers to learners and
traditional to alternative assessments. These assessments are to be practiced in
classrooms to cater to learners' autonomy, transparency, and evaluation
preferences.
Studies have found that teachers’ assessment literacy has a statistically significant
impact on learners’ achievement, thus illustrating the need for an effective
teaching environment and motivated assessment design (Mellati & Khademi,
2018). Teachers’ assessment styles, preferences, and actions significantly influence
students’ learning experiences and achievement (Hattie & Timperley, 2008;
DeLuca et al., 2018). In a closed academic environment, it is difficult for teachers
to develop assessment literacy by themselves without intervention. However,
researchers also report that teachers’ knowledge of assessment is insufficient and
argue that even when teachers have a considerable understanding of assessment,
they do not put their knowledge into practice (Nurdiana, 2020).
The emphasis of the educational evaluation system relies on the cognitive

capacities of learners; that is, memory and verbal skills, rather than 21st-century
skills such as collaboration, critical thinking, and problem-solving. Studies have
identified a disparity between teachers' practice and students’ preference for
assessment. The contemporary approach to assessment focuses on utilizing
alternative methods such as projects, portfolios, and active performance tasks,
which are learner-centric education approaches. Alternative assessments, when
used with the right strategies, can be used effectively to assess learning outcomes
(Adama et al., 2023).
Alternative assessment is characterized as an alternative to standardized, norm-

referenced, multiple-choice testing. It includes student involvement in setting
learning goals, assessment criteria, and even task alignment. In line with the skill-
oriented context of learning, alternative assessment involves the application of
higher order thinking, problem solving skills, meta-cognition, collaboration, and
intrapersonal skills. Furthermore, it follows a constructive approach to instruction,
wherein contextualization in real-world applications, the use of specified criteria
and defined standards of performance (Madellan, 2004) are considered.
Additionally, the contemporary assessment approach also advocates equity in
assessment, students’ preferences for assessment, and transparency of assessment,
all of which significantly influence the achievement and learning environment.
This is based on learner-centric education; learner-centric teaching and learning
5
assessment focuses on helping learners to think critically, solve real-life problems,

evaluate evidence, analyse situations and progress towards higher-order thinking
skills.
The traditional practices, based on teachers’ instinctive judgment or preferences,

affect the learning process. Assessment practices must come from teachers’
judgement based on assessment theories, conceptual clarity, and learners’
preferences. In all forms of education, assessment drives learning and, therefore,
it is necessary to study students’ attitudes toward different assessment formats
before implementing a new curriculum (Holzinger et al., 2020). Nevertheless, the
level of teachers' understanding and proficiency in assessment strategies
significantly influences their assessment practices at all educational levels
(Deneen & Brown, 2016). As emphasized in a previous study by Popham (2009),
assessment literacy remains a crucial domain that demands continuous attention
and investment in faculty development, both in the present and future contexts.
Thus, the discussion highlights certain gaps that include the need for further
exploration of learners' preferences and needs in assessment (Holzinger et al.,
2020; Thomas & Jessop, 2018), enhanced assessment literacy among teachers in
higher education (Mellati & Khademi, 2018; Zulia, 2020), and adapting assessment
practices for online learning environments (Dutta, 2020; Joshi et al., 2021). One
specific problem that has been identified is the disparity between learner-
preferred and teacher-preferred assessment tools in higher education (HE). There
is a need to ensure that assessment methods align with students' learning needs
and preferences (Holzinger et al., 2020). Addressing these gaps in assessment
literacy and exploring alternative assessment methods, such as projects,
portfolios, and active performance tasks, can lead to a more equitable and learner-
centred approach to assessment in higher education (Adama et al., 2023).
In the context of a multidisciplinary university within the higher education

ecosystem, the present study aims to explore the assessment literacy of higher
education (HE) teachers. It examines HE teachers’ preferences and reflections on
assessment practices and highlights the disparities between learner-preferred and
teacher-preferred assessment tools. Additionally, the study discusses the
identified gaps, orientation needs, and the importance of training in alternative
assessment methods.
Operationally, assessment literacy in the present research was based on the

knowledge component (Fig. 1), with the perspective of understanding teachers’
conceptual comprehension of the concept – ‘Assessment in education’. Though
this may not be the comprehensive meaning of the term, it includes the essence of
it. Thus, the aim was to assess the conceptual assessment literacy of practicing
teachers to understand their preconceived understanding of assessment with
respect to its meaning, forms and purpose, as well as the strategies and techniques
of assessment in education. Teachers’ current assessment practices were also
considered.
Thus, the present study aims:
(A) to gauge assessment literacy among teachers of higher education;
6
(B) to identify the current assessment practices in higher education; and
(C) to determine the need for alternative assessment practices in light of teachers’
reflections.
2. Literature Review
The traditional approach to assessment in higher education is facing significant
issues in terms of its alignment with learning outcomes and objectives. These
issues include compromised reliability, limitations in assessing higher-order
learning, and a lack of transparency in the evaluation and scoring process.
Conventional assessment relies on ‘pen and paper’ types of assessments with
simple measurement instruments such as quizzes, true/false questions, and
matching types, while this remains at odds with the learning objectives and aims
of higher education. Thus, learners’ ability to perform independently or in novel
contexts is not represented through scores and the learners are not informed of
how learning has occurred (Pereira et al., 2015; Knight, 2002). Research suggests
that the most used traditional assessment tools are multiple-choice tests,
true/false tests, short answers, and essays (Dikli, S., 2003). When the traditional
assessment methods are used for summative assessment, the same limitations are
carried forward into formative assessment in the absence of teachers’ assessment
literacy. In response to these challenges, exploring alternative assessment
practices becomes imperative. These can be better connected to real-life
applications, providing opportunities for learners to utilize their analytical,
critical thinking, reasoning skills. Methods include self-assessment and peer
assessment in a problem-based learning environment, stimulating deep learning
and critical thinking (Segers & Dochy, 2001 assessment). These practices offer
promising alternatives to traditional assessment methods, providing
opportunities for learners to engage actively in their own learning process and to
develop higher-order cognitive skills. The assessment must not completely
depend on situational learning or problem-based learning. Considering that the
students’ approach to learning may depend on the type of assessment used,
teachers should employ a diverse range of assessment methods to actively engage
students in the learning process. However, researchers also argue that the use of
multiple methods of assessment may be confusing for learners to internalize the
goals of assessment (Thomas & Jessop, 2018). Different assessment practices can
influence students’ achievement and student-oriented factors such as
commitment, difficulty level, study skills, parental support, and the institution’s
student support system also play a major role (Mekonnen & Besha, 2019). It has
been found that teacher’s assessment literacy also has a statistically significant
impact on learners’ writing achievements, reinforcing the need for effective and
motivated assessment designs (Mellati, M., & Khademi, M., 2018).
Research has also highlighted other factors influencing achievement that can be
explained at the micro-level, meso-level, and macro-level. Micro-levels include
factors such as teachers’ personal beliefs, knowledge, experience, and
conceptions. Meso-levels deal with institution-level culture and practices, and
macro-levels deal with system assessment policies, values, and protocols (DeLuca
et al., 2019). Therefore, the selection and integration of multiple assessment
methods should be undertaken thoughtfully to ensure clarity and coherence in
7
communicating the learning objectives to learners. Researchers have also found

volume and variety in assessment methods to be an independent factor affecting
achievements. The modern assessment practice emphasizes learners' preferences
in selecting assessment tools. Holzinger et al. (2020) studied 439 medical students
and found that learners prefer objective question formats (aka Multiple-Choice
Questions) above other types of assessment. Thomas and Jessop (2018) studied
programme assessment data from 73 courses across 14 universities in the UK to
determine the difference in assessment loads across the courses. The volume of
summative and formative assessment, examinations proportions and varieties of
assessment methods were put into use. The study found that research-intensive
courses have higher summative assessment loads and a greater proportion of
examinations compared to teaching-intensive courses, which have greater
varieties of assessment.
Mellati and Khademi (2018) studied the impact of teachers’ assessment literacy on
current assessment practices and writing outcomes using teachers’ assessment
literacy inventory, semi-structured interviews, non-participatory observation,
and Writing Competence Rating Scale (WCRS). The findings highlighted that
teachers’ assessment literacy has a statistically significant impact on learners’
achievement. Furthermore, teachers’ assessment awareness induces an effective
teaching environment and motivated assessment design. Zulia (2020) explored
teachers’ perceptions of classroom-based assessment and the extent to which they
are reflected in their practice. The study involved a survey of 22 participants,
interviews with five participants and documentary evidence of assessments. The
research concluded that the teachers had a good level of assessment literacy and
were aware of the principles of classroom-based assessment, although the quality
of the assessment methods used was questioned. DeLuca et al. (2019), in their
descriptive study, provided empirical evidence for assessment literacy as a
differential and situated professional competency. Contemporary views were
gathered on five common classroom scenarios. The study involved 453
participants to explore the assessment literacy of teachers and examine their
approaches towards assessment in different classroom scenarios. The study
provided empirical evidence supporting the notion that assessment literacy is a
differentiated and context-dependent professional competency. The findings
revealed significant differences in teachers' assessment approaches across
teaching divisions and career stages. The complexity of factors such as teaching
grade, subject, and individual characteristics also influenced teachers' assessment
practices within specific contexts. These findings highlight the need for teachers
to develop assessment literacy skills to enhance their understanding and
implementation of effective assessment practices. In a review study, Nurdiana
(2020) found that some teachers’ assessment literacy is insufficient, while other
teachers have a high degree of assessment literacy but do not put their knowledge
into practice. Khadijeh and Rezaei (2015) stated that assessment literacy is
important as it enables teachers to perceive, analyse, and use data on student
performance to thereby improve their teaching. The absence of assessment
literacy can therefore be seen as a form of ‘professional suicide’ (Popham, 2011, p.
269).
8
In the Indian context, researchers have mainly focused their attention on

assessment literacy among schoolteachers, with the exception of a few studies that
have addressed the need for faculty development in Higher Education.
Govindarajan and Srivastava (2020) focus on the potential impact of remote
teaching in higher education, and indirectly touch upon the importance of
assessment literacy in this context. Their article explores the transformative
potential of remote teaching in higher education. The authors compare the current
shift to virtual learning. As the education landscape transitions to online
platforms, it becomes crucial for educators to possess a strong understanding of
assessment practices and techniques that can effectively measure student learning
in virtual environments. Therefore, as the future of higher education evolves
towards remote teaching, assessment literacy remains a critical component for
educators to navigate the challenges and optimize student learning outcomes.
Dutta (2020) explored the impact of digital social media on Indian higher
education during the COVID-19 pandemic crisis. The author examined the use of
social media platforms for disseminating learning resources to students and
analysed the effectiveness of online classes and e-learning pedagogy through
qualitative analysis. The findings reveal that the lockdown and shift to online
learning have had a significant impact on students, causing stress, anxiety, and a
sense of helplessness. On the other hand, the provision of online classes has
proven beneficial, not only in terms of educational advancement but also for
students' mental well-being. The study indicated the importance of assessment
literacy in online education and suggested the need to be proficient in designing
and conducting assessments online. In a similar study, Joshi et al. (2021) explored
the impact of the COVID-19 pandemic and focussed on teachers' perspectives on
online teaching and assessments. Using interpretative phenomenological
analysis, the study identified four main categories of barriers faced by teachers:
challenges in home environment settings; institutional support barriers; technical
difficulties; and personal problems. The findings underscore the importance of
assessment literacy among teachers to overcome these barriers and ensure
effective online teaching and assessment practices. Thus, while assessment
literacy has not received significant attention directly, researchers have
highlighted the need for increased focus on faculty development in higher
education, confirming the necessity for further research on assessment literacy.
3. Methodology
This study employed a descriptive survey design. 58 teachers from 16 disciplines
within a single multidisciplinary university participated in the study. Incidental
sampling method was employed, with selection being based on voluntary
participation, accessibility, and availability. Individuals had the autonomy to
decide whether or not to participate in the study. This ensured that participants
had a genuine interest in the research topic and were more likely to provide
valuable insights (Creswell, 2014; Cohen, Manion, & Morrison, 2018). The
participation in the survey is presented in Table 1.
9
Table 1: Survey sample details
Discipline n Discipline n
Health Science 2 English Language 2
Medicine 2 Liberal Arts 1
Nursing 5 Economics 1
Information Technology 3 Banking and Finance 2
Computer Studies 4 Management Studies 13
Media and
Technology 13 Communication 2
Design 4 Telecommunication 1
Architecture 1 Biological Science 2
Total Sample Size (N)= 58
Where 'n' stands for number of samples
There were 58 participants from 16 disciplines, as presented in Figure 2.
Figure 2: No. of participants according to participants’ disciplinary affiliations
3.1 Tool for data collection

A researcher-made questionnaire (Appendix I) was used for assessing
‘Assessment Literacy’ including both open-ended and close-ended questions. A
systematic process was followed to design and develop the questionnaire.
Tool development: The tool development process included:-
1. Group discussion with a panel of five experts, consisting of faculty members

from teacher education, experienced researchers, and practitioners in the field
of assessment, with the purpose of outlining the scope, identifying key
constructs and dimensions of tools.
2. Designing close-ended questions (MCQs) and framing open-ended questions.
3. Content validation from researcher and language experts.
4. Pilot testing the questionnaire.
10
5. Expert validation after refinement.

6. Finalization of the questionnaire.
Nature of the questionnaire: The questionnaire included both open- and close-
ended questions to capture two aspects: 1. Assessment literacy; and 2. Assessment
practices. The assessment literacy aspects encompassed dimensions such as the
concept of assessment, forms and functions of assessment, purpose of assessment
literacy, and strategies and techniques of assessment. Assessment practices
aspects focused on the purpose of assessment, ongoing strategies, learner-
preferred assessment strategies, and teacher-preferred assessment strategies.
The questionnaire was designed for measuring HE teachers' assessment literacy.

Close-ended MCQs were added to retain objectivity and simplicity in assessment
(Ben-Simon et al., 1997), while open-ended questions were included in order to
avoid bias through suggested responses (Reja, U., et al. 2003). Also, they were
used to gain HE teachers’ insights into their current assessment practices.
Thus, the questionnaire (Appendix I) included a set of 15 questions - both closed

and open-ended. The main aims were to gauge the teachers’ assessment literacy
and to identify the current assessment strategies used by the teachers, as shown
in Table 2.
Table 2: Nature of questionnaire

No. of
S.N. Purpose questions
Details
I Assessment literacy aspects
Multiple choice
questions (MCQs) -
1 Meaning of assessment 3
• Incomplete
statement format
2 Forms and functions of assessment 3 • Scenario-based
3 Purpose of assessment literacy 2 • Image-based
4 Strategies and techniques of assessment 2 • Single response
II Assessment practices
5 Purpose of assessment 1
6 Ongoing strategies for assessment 1
• Open-ended
7 Learner-preferred assessment strategies 1
reflections
8 Teacher-preferred assessment strategies 1
9 Successful assessment practices 1
3.2 Research Process

Initially, a literature review was performed to establish the theoretical
foundations of assessment literacy and related concepts. This involved examining
national and international documents, research papers and articles in the field,
highlighting studies and findings to establish a strong theoretical framework. This
helped outline the essential components for assessing the assessment literacy of
the HE teachers. Subsequently, three rounds of discussion with a panel of six
11
experts were held to develop the structure of the 15-question tool for the
Assessment of Higher Education Teachers’ Assessment Literacy and to obtain content
validity for it. The questionnaire data were collected using a Google form after
obtaining permission. The data were gathered using both quantitative and
qualitative processes. This helped to investigate the assessment literacy and gain
insights into the assessment practices employed by teachers. The collected data
were then analysed using basic statistical methods. The findings were presented
using percentages, tables, and graphs to represent the results and help uncover
the patterns in the data. These patterns ultimately gave insights for drawing the
conclusions of the research. The research process is summarized in Figure 3.
Figure 3: The research process
4. Data Analysis and Results

Descriptive statistics were used for the close-ended questions, mostly using
percentage and graphical representation. The qualitative data gathered through
open-ended questions were subjected to thematic analysis. Furthermore, the
quantification of qualitative data was achieved using frequency of occurrence to
understand the trends and patterns for objective 2. The data analysis is
represented under the three sections presented below.
Section A – (For objective 1: Assessment literacy among teachers of higher

education)
1) Participants’ assessment literacy (Tables 3 and 4)
2) Teachers’ responses on the purpose of assessment (Table 5)
Section B – (For objective 2: To identify the current assessment practices in higher

education)
3) Online and offline assessment strategies used by teachers (Figure 6)
4) Teachers’ responses on learner-preferred current assessment practices (Figure
7)
5) Teacher-preferred current assessment practices (Figure 8)
12
Section C- (For objective 3: To determine the need for alternative assessment

practices in light of teachers’ reflections)
6) Rationalizations for preferred assessment strategies (Table 8)
7) Teachers’ reflections on successful and unsuccessful assessment strategies
(Table 9)
The major findings are:

1. The majority of teachers (79%) have moderate levels of assessment
literacy, compared to 16% teachers with a low level and only 5% showing
a high level of assessment literacy.
2. Teachers lack clarity in aspects of assessment literacy such as tests,
‘Assessment for learning’ and ‘Assessment of learning’, various types of
assessments, purpose of evaluation, strategies and techniques of
alternative assessments.
3. Currently, teachers vastly prefer offline practices and so training in
alternative online strategies is needed.
4. Greater emphasis was laid on individual performance and knowledge
recall in teacher-preferred assessments.
5. The top three assessments preferred by learners were Case

analysis/Discussion/Presentation (20%), Quiz/MCQs (16%), and Project
(group/Field/live) (14%).
6. Though teachers have been using varied assessment strategies, there is a

gap in teacher-preferred and student-preferred strategies.
7. Teachers are aware of the specific purpose/reasoning for the assessment

strategies they have been practicing.
8. Certain students’ preferences for strategies such as debates, research-
based assignments, simulation, seminar and reflections are not taken into
consideration for regular assessment.
9. Descriptive questions, online written exams, group presentations, open
book online exams and use of whiteboard are examples of strategies that
need training to convert them into successful strategies.
10. There is a need to improve assessment literacy among teachers.
A. Assessment Literacy
After recording the individual scores of the respondents, these scores were
grouped into three assessment literacy levels; low, medium and high. This scale
is shown in Table 3.
Table 3: Assessment literacy level

Assessment Literacy (AsL)
AsL Level Scores n
Low Under 33% 9
Moderate Up to 66% 46
High Above 66% 3
N 58
13
The distribution of the assessment literacy levels (Figure 4) indicates moderate

assessment literacy in the majority of respondents (79%), compared to 16% of
teachers who showed only a low level and just 5% showing a high level.
Figure 4: Assessment literacy levels
Data gathered using the MCQs were analysed, as shown in Table 4.
Table 4: Assessment literacy

Assessment Components of AL Component Percentage of HE
Literacy code teacher with
(AL)
aspects Correct Incorrect
response response
Concept of Meaning of a test M 44.83% 55.17%
assessment Individual assessment needs –
altering assessment IAN 58.62% 41.38%
‘Assessment for learning’ and

‘Assessment of learning’ AoL/AfL 12.07% 87.93%
Forms and Difference between formative and

functions summative assessment FA/SA 81.03% 18.97%
of
assessment Function of formative assessment
Fn. 72.41% 27.59%
Various types of assessment T 27.59% 72.41%

Purpose of Purpose of assessment PA 68.97% 31.03%
assessment Purpose of evaluation PE 32.76% 67.24%
Strategies Appropriateness of assessment
and strategies A 55.17% 44.83%
techniques
of Strategies and techniques for
assessment alternative assessment S/T 17.24% 82.76%
Where total number of sample (N)=58
14
The assessment literacy aspects have been depicted in Figure 4, shown below:
Figure 5: Teachers’ assessment literacy
The data revealed that more than half of the teachers were not clear about concepts
such as testing, measurement, assessment and evaluation in education. Many of
the participants lacked knowledge regarding alternative approaches to
assessment while the majority of them (88%) were unsure about the essential
concepts related to ‘Assessment for learning’. Thus, teachers lack clarity in their
understanding of the term assessment. Although most of the teachers (81%) were
aware of the meaning of – and difference between – formative and summative
assessment, they had lesser understanding of other forms of assessment such as
diagnostic, norm-referenced, and criterion-referenced assessments. The purpose
of assessment and evaluation was not clear to many of the teachers (68% and 67%,
respectively). Very few teachers (17%) demonstrated an understanding of the
strategies and techniques of assessment, which confirms the need for conceptual
clarity. Furthermore, a lack of clarity on the appropriateness of the assessment
strategies was found in almost half of the respondents (45%).
The results of the open responses received from teachers regarding the purpose
of assessment are listed below in Table 5, indicating the 21 key themes that
emerged from the thematic analysis.
Open-ended item 1: In your view, what is the purpose of assessment?

Presented in Table 5, below, are the key themes that emerged from teachers'
responses on the purpose of assessment, reflecting a diverse range of
perspectives.
15
Table 5: Teachers’ responses on the purpose of assessment

Teachers'
Teachers' responses Teachers' responses
responses on the
SN on the 'Purpose of SN on the 'Purpose of SN
'Purpose of
Assessment' Assessment'
Assessment'
Evaluate students’ Identify learners’ Ensure student
1 8 15
understanding interest engagement
Assess learners’
Understand best Judge the
2 9 16 acquired skills and
teaching practices knowledge
abilities
Collect relevant
information on Understand learning Decide on
3 10 17
students’ requirements promotion
performance
Differentiate
Understand difficult Make decisions
4 between excellent 11 18
topic/content about progression
and poor students
Identify difficult
Provide help to
topics and modify Provide feedback
5 students having 12 19
teaching methods based on learning
learning difficulties
accordingly
Determine how well
Reflect on the
learning matches
6 effectiveness of the 13 20 For exams
with the outcomes/
adopted pedagogy
expectations
Gauge the transfer
Identify learning Understand the
7 14 and assimilation of 21
styles learning process
knowledge
The responses indicate that teachers believe assessment serves multiple purposes,
including supporting holistic development, promoting reflective teaching and
learning, driving student engagement and progress, enabling personalized
instruction, and encompassing a range of objectives to enhance the educational
experience. The responses can broadly be classified under five main themes
indicating the purposes of assessment: 1) Evaluation and Differentiation; 2)
Understanding Teaching and Learning; 3) Judging Learners' Interest and
Knowledge; 4) Understanding Difficult Topics and Learning Outcomes; 5)
Student Engagement, Progression, and Feedback. Even though some responses
lacked clarity and specificity, making it difficult to understand the intended
actions or strategies associated with those purposes, it can be inferred that the
majority of the responses focus on quantitative assessment methods, potentially
overlooking the value of qualitative approaches that provide deeper insights into
students’ learning. Also, there are implicit assumptions about the purpose of
assessment, such as ranking students or making progression decisions, which are
teacher-centric. These should be critically examined to ensure a more
comprehensive and student-centred approach. Furthermore, student
involvement in the assessment process is not prominently mentioned, despite its
potential to foster student ownership of learning.
16
B. Current Assessment Practices

To understand the current assessment practices implemented by teachers, data
related to learners’ preferred strategies and teachers’ preferred strategies were
gathered and the results are presented as follows.
Open-ended item 2: What are the strategies you are using to assess your
students?
Data revealed that the teachers generally use offline assessment strategies as
opposed to online strategies, as shown below in Figure 6.
Figure 6: Use of the offline and online assessment strategies mainly used by teachers
Practical, demonstration, open book test, article writing/evaluation, debates,

tutorials, written assignments, seminars, mind maps, open-ended questions, pair-
share, portfolios, panel discussion, journal writing, term-end papers, research-
based assignments, lab tests, analysing company white papers, report writing,
movie reviews, video analysis and role plays are the offline strategies used by 2-
3% teachers.
When asked for the online assessment strategies they employ, respondents listed
such strategies as personal interaction in online mode, experiential learning,
dialogue, online lectures, class participation, punctuality in joining classes,
involvement and participation, attentiveness, timely submission of assignments,
self-assessments, Google forms, Edmodo, Google classroom, peer review of class
work, exercises that have a reflective component, online extempore sessions, oral
practical activities, problem solving, break-out rooms, interactive online tasks and
rubrics for evaluation. Many of these are not even assessment strategies,
indicating the teachers’ lack of conceptual clarity.
Open-ended item 3: What are the assessment strategies most preferred by

your students?
The main themes that emerged through thematic analysis for learners’ preferred
assessment strategies are shown in Figure 7, along with their frequency of
occurrence.
17
Table 6: Current Assessment Practices: Learner Preferred

Current Assessment Practices: Learner Preferred
Assessment types Percent f
Case analysis/Discussion/Presentation 20% 12
Quiz/MCQs 16% 9
Project (group/field/live) 14% 8
Discussion 12% 7
Online assessment (Mentimeter, Spinwheel) 11% 6
Assignment 10% 6
Written exam/ paper-pencil tests 8% 5
Viva / Oral presentation 6% 3
Seminars/ Lab tests 3% 2
Total 58
Figure 7: Learner-preferred current assessment practices
Apart from these commonly used strategies, other strategies such as home
assignments, open-ended questions, practical tasks, demonstrations, debates,
journals, research-based assignments, role plays, tutorials, written tests, games,
group discussions, think-pair-share, mini cases, simulation, choral reading,
Socratic seminars, interactive videos, reflections and mock interviews were
additional strategies sometimes used by some of the teachers.
Open-ended item 4: Which assessment strategies do you prefer, and why?

The responses were categorized based on the assessment types preferred by the
teachers and represented in the table below (Table 7).
Table 7: Current Assessment Practices: Teacher Preferred

Current Assessment Practices: Teacher Preferred
Assessment types Percent f
Quiz/MCQs 27% 16
Viva 15% 9
Online assessment (e.g. Mentimeter, Spinwheel) 15% 9
Case analysis/Discussion/Presentation 12% 7
18
Presentation 7% 4
Assignment (group/individual) 7% 4
Project (group/field/live) 5% 3
Written exams 5% 3
Discussions 5% 3
Practical tasks 2% 1
Total 58
Figure 8: Teacher-preferred current assessment practices
In terms of the assessment types preferred by learners, the top three choices were
case analysis/discussion/presentation (20%), quiz/MCQs (16%), and project
(group/field/live) (14%). These assessment methods tend to promote active
engagement, collaborative learning, and practical application of knowledge. On
the other hand, the most preferred assessment types of teachers were quiz/MCQs
(27%), viva (15%), and online assessments (e.g. Mentimeter, Spinwheel) (15%).
This indicates a greater emphasis on individual performance and knowledge
recall in teacher-preferred assessments.
One notable difference was the higher preference for discussion as an assessment
type among learners (12%) compared to teachers (5%). This suggests that learners
value opportunities for dialogue and exchanging ideas during assessments, which
aligns with their desire for active participation and engagement in the learning
process.
Assignment was chosen by 10% of learners compared to 7% of teachers, indicating

that learners appreciate assignments as a means to demonstrate their
understanding and apply their knowledge. Conversely, teachers placed a higher
emphasis on viva (15%) and presentation (7%) as preferred strategies for
assessment, suggesting a focus on evaluating oral communication skills and the
ability to deliver information effectively.
Despite these differences, both learners and teachers showed a shared preference
for online assessments, with 11% and 15%, respectively. This suggests a
recognition of the benefits and convenience offered by technology-mediated
assessments in facilitating learning and providing timely feedback.
19
Overall, the data highlights a difference in preferences between learners and

teachers regarding assessment types (shown in Figures 7 and 8). Learners tend to
gravitate towards assessments that involve active participation, collaborative
activities, and real-world applications, while teachers lean towards assessments
that focus on individual performance and traditional evaluation methods.
Understanding these differences can help in designing assessment strategies that
align with the needs and preferences of both learners and teachers, promoting
effective and engaging assessment practices.
The main reasons given for the selection of assessment strategies are presented in
section C.
C. Teachers’ Reflections
Reflections by the teachers on their preferred assessment strategies and the
reasons for their selections are presented below in Table 8.
Table 8: Rationales for preferred assessment strategies

Teachers’ preferred Teachers’ reasons for their selection of assessment
assessment strategies strategies
Quiz • Objective
• Useful for formative assessment
• Fast, less time-consuming
• Accurate analysis of the learning
Viva • Clear picture about each student
• Overall understanding of students
• No room for copying, easy to gauge students
• Opportunity for individual assessment
• Real-time analysis
• Immediate overview of students’ understanding
without any bias
Case studies/ • Evokes real-time thinking
analysis/discussion/ • Students study a lot
presentation • Allow students to write without fear of judgment
MCQs • Preferred by students
• Easy to conduct and evaluate
• Less time-consuming
• Reduce chances of ambiguity
• Easy to check the basics
• Convenient
Presentation • Opportunity for personal interaction
• Student involvement
• Real-life applications
• Presentation skills, develops confidence
• Quick feedback can be given
Projects • Summarization
(group/ field/live) • Knowledge application
• Peer learning
• Students choose what they like to work on
20
• Encourages team spirit and student interaction

• Scope for creativity
Written exams • Test the learning of students
• Check writing ability and knowledge of students
Discussions • Room for group work
Open-ended • Opportunities for students to express themselves

questions
Article analysis • Analytical abilities
• Cover contemporary issues not found in textbooks
Game-based learning • Assists self-evaluation

• Enhances practical skills
Open book test • Challenging
Interactive video case • Apply theoretical learning to a practical scenario
The findings indicated that teacher-centric strategies focus on assessing learning

outcomes and gaining insights into students' progress, while student-centric
strategies emphasize assessment for learning and creating an interactive learning
environment. The reasons given for the selection of these strategies include
objectivity, time efficiency, real-time analysis, personal interaction opportunities,
and skill development. Thus, more attention is currently paid to ‘Assessment of
Learning’ rather than ‘Assessment as learning’ or ‘Assessment for learning’.
The HE teachers' preferences for assessment tools and their rationales indicate
that there is a lower priority on promoting deeper understanding, critical
thinking, problem-solving skills, creativity, collaboration, and self-evaluation. On
the other hand, alternative assessment strategies prioritize engagement and the
active involvement of students in the assessment process. The participants’
preferred assessment strategies are primarily based on traditional methods. These
strategies are chosen for their objectivity, efficiency in administration, ability to
provide a clear understanding of students' performance, opportunities for
individual assessment, and real-time analysis. Consequently, the preferences
expressed in the table reflect a predominant reliance on teacher-centric assessment
approaches aimed at evaluating students' knowledge and comprehension of the
subject matter.
Furthermore, the table displayed the HE teachers’ rationales for selecting specific
assessment tools, which were based on factors such as objectivity, formative
assessment opportunities, real-time analysis, student engagement, ease of
administration and evaluation, and the ability to measure overall learning. These
considerations reflect the goals and priorities of the teachers in assessing student
progress and promoting effective learning outcomes.
The table also highlights the limited evidence of teachers' knowledge and
utilization of alternative assessment methods. Alternative strategies such as case
21
studies, discussions, projects, game-based learning, open-book tests, article

analysis, and interactive video cases offer opportunities for students to
demonstrate practical skills, engage in collaborative learning, apply theoretical
knowledge to real-world scenarios, and express their thoughts and ideas. These
methods promote critical thinking, problem-solving, creativity, and self-
evaluation.
The findings indicate that there is a lack of clarity among teachers regarding
assessment-related terms such as strategies, tools, and approaches. Certain
student-preferred strategies, including debates, research-based assignments,
simulations, seminars, and reflections, were not taken into consideration.
However, the reasons for their selection of specific strategies indicated clarity
among teachers in using them for assessments.
Thus, the limited mention of alternative assessment tools (as shown in the table)
suggests that teachers may have limited awareness or understanding of the
potential benefits and applications of these methods. It is essential for educators
to enhance their assessment literacy and explore diverse assessment approaches
that cater to the varying needs and preferences of students. By incorporating
alternative assessment methods into practice, teachers can create a more inclusive
and engaging learning environment that fosters deeper understanding, critical
thinking, and greater skill development.
Open-ended item 5: Reflect on your ‘successful’ and ‘unsuccessful’ assessment

practices during the past two years.
Reflections on item 5 were collected from the respondents. The results are
indicated in Table 9, as follows.
Table 9: Reflections on successful and unsuccessful assessment strategies
Successfully implemented assessment Unsuccessful assessment

strategies strategies
• MCQs and short questions • Descriptive questions

• Viva (online/offline) • Online written exams
• Continuous quizzes • Group presentations
• Verbal assessment • Open book online exams
• Quizzes • Term-end conventional
• Tests tests
• Projects • Use of whiteboard
• Home assignments
• Case analysis
• Article writing
• Project-based learning
Reflections on the successes and failures of various assessment strategies in online

and offline modes were mixed. MCQs, short questions, viva (online/offline),
continuous quizzes, verbal assessment, quizzes, tests, projects, home assignments,
case analyses, article writing, and project-based learning were successful
22
strategies, while descriptive questions, online written exams, group presentations,

open book online exams, term-end conventional tests, and use of whiteboard were
unsuccessful.
Analysis revealed that there is an emerging need for training on innovative,

alternative assessments. Comparison between the learner-preferred strategies and
the teacher-preferred strategies revealed a mismatch. Apart from commonly used
assessment strategies, several other forms – such as home assignment, open-
ended questions, practical tasks, demonstrations, debates, journaling, research-
based assignments, role plays, tutorials, games, group discussions, think-pair-
share, mini-cases, simulations, choral readings, Socratic seminars, interactive
videos, reflections, and mock interviews – were preferred by students but were
infrequently practiced by teachers. This reinforces the need for the adoption of
alternative assessment strategies by teachers. Furthermore, convenience of
implementation, less consumption of time, and ease of evaluation were found to
be among the prominent reasons given for the selection of assessment strategies.
Comparison between Table 8 and Table 9 highlights both the existing knowledge
and potential gaps in assessment practices among HE teachers. Table 1 reveals
that teachers have a concrete understanding of traditional assessment strategies,
as indicated by their preferences. Their reasons for selecting these strategies, such
as objectivity, time efficiency, and real-time analysis, demonstrate their awareness
of the benefits of these methods in evaluating student performance. However, the
limited mention or absence of alternative assessment tools, as shown in Table 8,
suggests a need for further training and exploration. Strategies such as debates,
research-based assignments, simulations, and seminars, which promote critical
thinking and creativity, were not considered by teachers for assessment purposes.
This highlights a potential gap in their knowledge of alternative assessment
methods. Table 9 provides additional insight by showcasing the assessment
strategies that have been successfully implemented and those that have not
yielded the desired outcomes. While traditional strategies such as MCQs and
written exams were deemed successful, some alternative methods, including
group presentations and open book online exams, were not perceived to be as
effective. These findings underscore the importance of addressing the training
needs of HE teachers in terms of assessment literacy and alternative assessment
tools. Comprehensive training programs can equip teachers with the necessary
knowledge, skills, and resources to implement a wider range of assessment
strategies that cater to diverse student needs and promote deeper learning
outcomes.
5. Discussion
The present study, based on current assessment practices, showed that assessment
literacy among teachers in higher education needs to be improved. Overall, the
results suggest that higher education teachers are inclined towards traditional
assessment methods. Similar findings were observed in previous studies,
indicating persistently low levels of assessment literacy among teachers and a lack
of theory-driven instruments in formative assessments (Yan & Pastore, 2022).
23
Traditional assessment methods are frequently preferred by teachers, compared

to other assessment strategies. The findings also underscore persistent gaps
between learner-preferred and teacher-preferred assessment practices. Though
many teachers use online assessment strategies, few use student-preferred
strategies such as debates, research-based assignments, simulations, seminars,
and reflections. There was a lack of clarity among teachers on assessment
strategies, tools and approaches, thereby indicating inadequate awareness and
preparation. The present study thus found differences in the preferred assessment
strategies among teachers and learners. The study conducted by Pereira and
Flores (2016) provides evidence of such a contradiction between teachers’
conceptions of assessment and the practices on similar lines.
Developing ‘assessment literacy’ or ‘learning to assess’ is a complex process that

demands continuous negotiation with shifting paradigms of assessment,
alongside other evolving educational and pedagogical theories related
to individualization, self-assessment, constructive feedback, peer-assessment,
spaced learning, and differentiated assessment. A lack of substantial knowledge
on assessment practices significantly influences learning outcomes (Oo et al., 2023;
DeLuca et al., 2019; Bennett, 2011, as cited in Mellati & Khademi, 2018). Research
emphasizes that improved assessment literacy among teachers can develop clarity
among learners on the overall process and evaluation criteria. Assessment literacy
enhances student engagement and motivation in learning (Hannigan et al.,
2022). Teachers’ assessment literacy has a statistically significant impact on
learners’ achievement (Mellati & Khademi, 2018). It is apparent from the findings
that teachers' reasoning and judgement on assessment methods is more in favour
of conventional assessment practices that assess lower-order thinking and recall
skills, focusing on evaluating oral communication skills, the ability to deliver
information effectively and so on. This translates into a lower inclination towards
active participation, collaborative activities, real-world applications, and
performance-based approaches that support learner-centred practices.
Contrary to the findings of some similar studies (Sun & Zhang, 2022), the teachers’
responses were not completely unsatisfactory. Many teachers practiced strategies
such as MCQs, short questions, viva (online/offline), continuous quizzes, verbal
assessment, quizzes, tests, projects, home assignments, case analyses, article
writing, and project-based learning, which are found to be more successful than
strategies such as descriptive questions, online written exams, group
presentations, open book online exams, term-end conventional tests, and
assessments using interactive whiteboards. Deep-level strategies involving higher
order thinking are the preferred assessment tasks and are considered to be
successful strategies by higher education teacher participants. The teacher-
preferred strategies that elicit responses restricted to lower-level thinking, while
involving learners only superficially, have been considered less successful
strategies. In their study, Kim and Lee (2021) observed that various factors are
responsible for low scores among teachers in assessment literacy, including
personal factors, professional factors, institutional culture and factors related to
state policy. By promoting a deeper understanding of assessment principles and
strategies, educators can design assessments that align with learners' needs and
foster higher-order thinking.
24
The present study reveals that teachers prefer assessments that are convenient to
implement, less time-consuming and easy to evaluate. These preferences indicate
an inclination towards the surface approach of completing assessment tasks
without demonstrating in-depth learning. Performance-based assessments serve
as an alternative to traditional methods and promote deep-level learning. The
present findings contradict those of previous research which indicate that learners
prefer objective styles of question format (MCQ) above other assessment types
(Holzinger, 2020; Dang & Tsang, 2022). However, the findings of the present
study indicate learners’ preference for performance-based assessments. Tomas
and Jessop (2018) speculate that learners focus on achieving grades and thus
mainly concentrate on work that counts towards these. Also, attention needs to be
paid to the minimal use of alternative performance-based strategies and the lack
of consideration being given to learners’ preferences in assessments. The present
research findings align with those of Areekkuzhiyil (2019), who states that
deliberate efforts are required from academicians and authorities to make
assessments more dynamic and fruitful.
Overall, the present study addresses the low levels of assessment literacy, lack of
accommodation of learners’ preferred strategies and minimal use of performance-
based and alternative assessment strategies. Similar findings were reported in the
literature in China, which reveals a lack of assessment literacy among university
English teachers (Xu & Brown, 2017).
Furthermore, the present work also highlights the need for improving awareness
among teachers about various assessment strategies. Previous studies have
confirmed that learner-centric assessment practices enhance the active
involvement of the students, produce feedback, enable collaboration between
students and faculty and allow teachers to realise how learning occurs (Webber,
2012 cited in Pereira et al., 2016). The conventional approach to assessment must
be used judiciously, in combination with performance-based assessments,
considering the learning outcomes. Holzinger et al. (2020) suggest that MCQs
must be well constructed, allowing for the evaluation of taxonomically higher-
order skills rather than simply recall or recognition-type questions.
Unlike previous research carried out in the area of assessment literacy, this
research work presented an apparent depiction of assessment literacy and
practices. The study utilized the teacher-preferred assessment strategy (i.e.
MCQs) for participants to self-assess their own assessment literacy.
Although the perceptions on assessment-related concepts cannot be generalized,

they are noteworthy at both local and institutional levels.
6. Recommendations
The results indicate that strategies supporting skill enhancement, collaboration,
creativity, and performance are less practiced. Teachers need formal training to
implement these alternative strategies alongside their regularly practiced
assessment strategies. These reflections require further exploration and
subsequent data gathering in order to understand the nature of training
programmes that should be offered to faculty. Similar surveys can be conducted
in future to verify the differences between learner-preferred and teacher-preferred
25
assessment practices at higher education level. Learners can also be involved to

understand the current assessment practices offered by their teachers in offline as
well as online modes of teaching and learning. Teachers' readiness to adopt new
alternative means of assessment needs to be addressed as early as possible. In
view of the observations and findings, the researchers recommend that further
exploration is needed in the area of assessment literacy among higher education
teachers and further analysis is needed in terms of alternative assessments. This
study can be taken ahead by obtaining data from all the stakeholders and then
triangulating it, which will give insight into the concrete needs for alternative
assessment training.
7. Conclusion
In conclusion, the National Education Policy - 2020 emphasizes the need for a shift
towards scientific and formative assessments that focus on the application of
knowledge. It highlights the significance of assessment literacy among teachers to
ensure the validity and fairness of assessments, as well as the importance of
faculty autonomy in fostering innovative teaching and assessment practices (NEP,
2020). However, the findings of the present study address specific research gaps
in the field of assessment in higher education. In light of policy expectations and
the increasing demands of the modern education system, this study has
significant implications for faculty development programs and educational
policymakers. The study reveals a need to improve HE teachers' assessment
literacy, as evidenced by their lack of clarity regarding tests, the discrepancy
between teachers’ and students’ preferences for assessment methods and the
limited reference to performance-based, formative and alternative assessment
strategies. While some educators exhibit proficiency in specific aspects of
assessment, there remains a clear need for comprehensive training on the
fundamental principles of educational assessment and evaluation, as well as the
implementation of student-centred and performance-based assessment practices
(Govindarajan & Srivastava, 2020; Dutta, 2020; Joshi et al., 2021). To address these
gaps, targeted training on assessment, reforms in teacher training and
development programs are necessary to enhance assessment literacy and promote
effective assessment practices that are aligned with learning outcomes. Further
research on a larger scale is recommended to better understand the status of
assessment literacy as well as the assessment practices in Indian universities and
to explore the impact of enhanced assessment literacy on student learning
outcomes and the overall learning environment.
8. References
Adama, E. A., Graf, A., Adusei-Asante, K., & Afrifa-Yamoah, E. (2023). COVID-19 and
alternative assessments in higher education: Implications for academic integrity
among nursing and social science students. International Journal for Educational
Integrity, 19(1). https://doi.org/10.1007/s40979-023-00129-0
Areekkuzhiyil, S. (2019). Assessment Practices in Higher Education: Myths and Realities.
Online Submission.
Bayat, K., & Rezaei, A. (2015). Importance of Teachers’ Assessment Literacy. International
Journal of English Language Education, 3, 139-148.
https://doi.org/10.5296/ijele.v3i1.6887
26
Al-Thani, S. J., Abdelmoneim, A., Daoud, K., Cherif, A., & Moukarzel, D. (2021).
Assessment of student learning outcomes for assurance of learning at Qatar
University. Journal of Assessment and Institutional Effectiveness, 4(2), 116-135.
https://doi.org/10.5325/jasseinsteffe.4.2.0116
Atjonen, P., Pöntinen, S., Kontkanen, S., & Ruotsalainen, P. (2022). Enhancing preservice
teachers' assessment literacy: Focus on knowledge base, conceptions of
assessment, and teacher learning. Frontiers in Education, 7, 891391.
https://doi.org/10.3389/feduc.2022.891391
Bennett, R. E. (2011). Formative assessment: a critical review. Assessment in Education:
Principles, Policy & Practice, 18(1), 5–25.
https://doi.org/10.1080/0969594x.2010.513678
Can, E., Aktas, F., & Arpacioglu, I. (2017). The reasons of school dropouts in higher
education: Babaeski Vocational College case. Universal Journal of Educational
Research, 5(12A), 84-88.
Chatterjee, S., & Bhattacharjee, K. K. (2020). Adoption of artificial intelligence in higher
education: a quantitative analysis using structural equation modelling. Education
and Information Technologies, 25(5), 3443–3463.
https://doi.org/10.1007/s10639-020-10159-7
Cohen, L., Manion, L., & Morrison, K. (2018). Research methods in education (8th ed.).
Routledge.
Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed methods
approaches (4th ed.). Sage Publications.
Dang, B. Y., Ho, E., & Tsang, A. (2022). Learner’s assessment preferences in higher
education: A comparison study of high-achievers and low-achievers. Asia-Pacific
Education Researcher. https://doi.org/10.1007/s40299-022-00679-w
DeLuca, C., Coombs, A., MacGregor, S., & Rasooli, A. (2019). Toward a differential and
situated view of assessment literacy: Studying teachers' responses to classroom
assessment scenarios. Frontiers in Education, 4.
Deneen, C. C., & Brown, G. T. L. (2016). The impact of conceptions of assessment on
assessment literacy in a teacher education program. Cogent Education, 3(1).
DePascale, C., Sharp A., Ryan K., Damian Betebenner, D. (2018). Building a Conceptual
Framework for Assessment Literacy. Assessment Literacy Framework.
Dikli, S. (2003). Assessment at a distance: Traditional vs. alternative assessments. Turkish
Online Journal of Educational Technology-TOJET, 2(3), 13-19.
Dutta, A. (2020). Impact of Digital Social Media on Indian Higher Education: Alternative
Approaches of Online Learning during COVID-19 Pandemic Crisis. International
Journal of Scientific and Research Publications, 10(5), 604.
http://dx.doi.org/10.29322/IJSRP.10.05.2020.p10169
Gallego, M. G., Perez de los Cobos, A. P., & Gallego, J. C. G. (2021). Identifying students
at risk to academic dropout in higher education. Education Sciences, 11, 427.
https://doi.org/10.3390/educsci11090427
Government of India. (2020). National Education Policy 2020. Ministry of Education.
Government of India, Ministry of Education. (2020). All India survey on higher education.
Govindarajan, V., & Srivastava, A. (2020). What the Shift to Virtual Learning Could Mean
for the Future of Higher Ed. Harvard Business Review. Retrieved from
https://hbr.org/2020/03/what-the-shift-to-virtual-learning-could-mean-for-
the-future-of-higher-ed
Hannigan, C., Alonzo, D., & Oo, C. Z. (2022). Student assessment literacy: Indicators and
domains from the literature. Assessment in Education: Principles, Policy &
Practice, 29(4), 482-504. https://doi.org/10.1080/0969594X.2022.2121911
27
Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research,
77(1), 81-112. https://doi.org/10.3102/003465430298487
Hidri, S. (2020). Language assessment literacy: Where to go? In S. Hidri (Ed.), Perspectives
on Language Assessment Literacy: Challenges for Improved Student Learning
(pp. 10). Routledge. https://doi.org/10.4324/9781003016083
Holzinger, A., Lettner, S., Steiner-Hofbauer, V., & Capan-Melser, M. (2020). How to
assess? Perceptions and preferences of undergraduate medical students
concerning traditional assessment methods. BMC Medical Education, 20(1), 1-9.
https://doi.org/10.1186/s12909-020-02239-6
Janisch, C., Liu, X., & Akrofi, A. (2007). Implementing alternative assessment:
opportunities and obstacles. The Educational Forum, 71(3), 221-230.
https://doi.org/10.1080/00131720709336947
Joshi, A., Vinay, M., & Bhaskar, P. (2021). Impact of coronavirus pandemic on the Indian
education sector: Perspectives of teachers on online teaching and assessments.
Interactive Technology and Smart Education, 18(2), 205-226.
https://doi.org/10.1108/ITSE-06-2020-0087
Knight, P. T. (2002). Summative assessment in higher education: Practices in disarray.
Studies in Higher Education, 27(3), 275-286.
https://doi.org/10.1080/03075070220000721
Kremmel, B., & Harding, L. (2019). Towards a comprehensive, empirical model of
language assessment literacy across stakeholder groups: Developing the language
assessment literacy survey. Language Assessment Quarterly, 16(3), 315-335.
https://doi.org/10.1080/15434303.2019.1612823
Latif, M. W., & Wasim, A. (2022). Teacher beliefs, personal theories and conceptions of
assessment literacy - A tertiary EFL perspective. Language Testing in Asia, 12(1),
1-22. https://doi.org/10.1186/s40468-022-00158-5
Maclellan, E. (2004). How convincing is alternative assessment for use in higher
education? Assessment & Evaluation in Higher Education, 29(3), 311-321.
https://doi.org/10.1080/0260293042000200497
Mellati, M., & Khademi, M. (2018). Exploring teachers’ assessment literacy: Impact on
learners’ writing achievements and implications for teacher development.
Australian Journal of Teacher Education, 43(6), 1-18.
https://doi.org/10.14221/ajte.2018v43n6.2
Ministry of Human Resource Development. (2016). All India survey on higher education
2015-16. Department of Higher Education, Government of India, New Delhi.
https://www.education.gov.in/sites/upload_files/mhrd/files/statistics-
new/AISHE2015-16.pdf
Michigan Assessment Consortium. (2015). Assessment Literacy Standards: A National
Imperative, Version 5.0 (2015). www.michiganassessmentconsortium.org
Michigan Assessment Consortium. (2020). Assessment Literacy Standards: A National
Imperative. Assessment Literacy Standards // VERSION 6.0.
https://www.michiganassessmentconsortium.org/assessment-literacy-
standards/
NAAC. (2017). Assessment and accreditation standards for quality assurance in higher
education institutions. National Assessment and Accreditation Council, India.
National Statistical Office, Ministry of Statistics and Programme Implementation,
Government of India. (2022). Youth of India 2022
Newfields, T. (2006). Teacher development and assessment literacy. In Authentic
Communication: Proceedings of the 5th Annual JALT Pan-SIG Conference.
28
Nieminen, J. H., Bearman, M., & Tai, J. (2023). How is theory used in assessment and
feedback research? A critical review. Assessment & Evaluation in Higher
Education, 48(1), 77–94.
Nurdiana. (2020). Language Teacher Assessment Literacy: A Current Review. Journal of
English Language and Culture, 11(1).
Oo, C. Z., Alonzo, D., & Davison, C. (2023). Using a Needs-Based Professional
Development Program to Enhance Pre-Service Teacher Assessment for Learning
Literacy. International Journal of Instruction, 16(1), 781-800–800.
https://doi.org/10.29333/iji.2023.16144a
Paura, L., & Arhipova, I. (2014). Cause Analysis of Students’ Dropout Rate in Higher
Education Study Program. Procedia - Social and Behavioral Sciences, 109, 1282–
1286.
Pereira, D. R., & Flores, M. A. (2016). Conceptions and practices of assessment in higher
education: A study of Portuguese university teachers. Revista Iberoamericana de
Evaluación Educativa, 9(1), 9-29.
Pereira, D., Flores, M. A., & Niklasson, L. (2015). Assessment revisited: A review of
research in Assessment and Evaluation in Higher Education. Assessment &
Evaluation in Higher Education, 41(7), 1008–1032.
Popham, W. J. (2009). Assessment Literacy for Teachers: Faddish or Fundamental? Theory
Into Practice, 48(1), 4-11. https://doi.org/10.1080/00405840802577536
Popham, J. (2011). Assessment literacy overlooked: A teacher educator’s confession. The
Teacher Educator, 46, 265-273.
Segers, M., & Dochy, F. (2001). New Assessment Forms in Problem-based Learning: The
value-added of the students’ perspective. Studies in Higher Education, 26(3), 327–
343. https://doi.org/10.1080/03075070120076291
Sun, H., & Zhang, J. (2022). Assessment literacy of college EFL teachers in China: Status
quo and mediating factors. Studies in Educational Evaluation, 74, 101157.
https://doi.org/10.1016/j.stueduc.2022.101157
Tomas, C., & Jessop, T. (2018). Struggling and juggling: A comparison of student
assessment loads across research and teaching-intensive universities. Assessment
& Evaluation in Higher Education, 1–10.
UGC. (2019). Assessment and accreditation norms in higher educational institutions.
University Grants Commission, India.
Wadhwa, S. (2021). Ensuring integrity and security in digital assessments: Challenges and
solutions for higher education in India. The International Review of Research in
Open and Distributed Learning, 22(2), 80-98.
https://doi.org/10.19173/irrodl.v22i2.5629
Wang, H., Sun, W., Zhou, Y., Li, T., & Zhou, P. (2022). Teachers’ assessment literacy
improves teaching efficacy: A view from conservation of resources theory.
Frontiers in Psychology, 13. https://doi.org/10.3389/fpsyg.2022.1007830
Webber, C. (2012). Standardized examination and higher education in India: Evaluating
the influence of the AIEEE on engineering education. Asia Pacific Education
Review, 13(2), 205-215. https://doi.org/10.1007/s12564-011-9182-0
Webber, K. (2012). The use of learner-centered assessment in US colleges and universities.
Research in Higher Education, 53(2), 201-228.
Xu, Y., & Brown, G. T. L. (2017). University English teacher assessment literacy: A survey-
test report from China. Papers in Language Testing and Assessment, 6(1).
Yan, Z., & Pastore, S. (2022). Are teachers literate in formative assessment? The
development and validation of the Teacher Formative Assessment Literacy Scale.
Studies in Educational Evaluation, 74.
29
Youngjoon, K., & Okseon, L. (2021). Autoethnography of a Novice Teacher’s Assessment

Literacy in Elementary Physical Education. International Journal of Qualitative
Studies on Health and Well-being, 16(1).
https://doi.org/10.1080/17482631.2021.1882066
Zulaiha, S., Mulyono, H., & Ambarsari, L. (2020). An Investigation into EFL Teachers’
Assessment Literacy: Indonesian Teachers’ Perceptions and Classroom Practice.
European Journal of Contemporary Education, 9(1), 189-199.
https://doi.org/10.13187/ejced.2020.1.189
Zulaiha, S., Suryanti, E., & Pramudya, I. (2020). Issues and challenges in assessment of
diverse students in higher education: A study in Indonesian context. Journal of
Educational Science and Technology, 6(1), 91-101.
https://doi.org/10.26858/est.v6i1.12696
30

Vol. 22, No. 7, pp. 30-51, July 2023
Received Apr 8, 2023; Revised Jun 15, 2023; Accepted July 10, 2023
Principals’ Leadership Orientation and Students’

Academic Performance in Secondary Schools of
Gedeo Zone, Ethiopia
RJ (Nico) Botha*
University of South Africa, Pretoria, South Africa
Seyoum Gari Aleme

Institute of Education and Behavioural Science
Dilla University, Dilla, Ethiopia
Abstract. As accountability in educational leadership has increased,

interest in finding the most rewarding type of principal leadership
orientation that helps to improve student performance is enhanced. The
lack of agreement on the most rewarding type of leadership orientation
among task-focused, relationship-focused or/and change-focused
behaviours, as well as the incidence of poor performance by students has
resulted in the Gedeo Zone of Ethiopia commissioning us to conduct this
study. The objective of the study was to identify the most profitable type
of principal leadership orientation that enhances the success of students
in the secondary schools of the zone. Three ineffective (least achievers)
and three effective (best achiever) schools of the zone were chosen
purposively, as sample for the study by using the maximum variation
strategy. The total of the sample was 339, of which 321 (n=321)
participated in the quantitative part of the study, while 18 were involved
in the qualitative part. A questionnaire, semi-structured interviews and
document analysis were used as instruments to collect the data. The
quantitative data were analysed by using means, standard deviations,
correlations, regressions, and line graphs, while the qualitative data were
analysed via a content-analytical approach. The findings of this study
revealed that high task and change-oriented behaviours among school
leadership enhanced the students’ success, while high relationship-
oriented behaviours intended to get a mere affiliation affected the
learners’ success negatively. High task-oriented behaviours enable
principals to initiate work, directing members towards goals, and
monitoring members’ performance. The active engagement of principals
in teaching and learning activities enhances students’ academic
‘achievements more than mere passive involvement to deal with the
challenges they may encounter. Thus, principals are advised to exhibit
high task and high change-oriented behaviours, as their engagement
*
Corresponding author: RJ (Nico) Botha, botharj@unisa.ac.za
©Authors
31
affects other stakeholders to play their role in improving students’

learning.
Keywords: leadership; task-oriented; relationship-oriented; change-

oriented; Ethiopian secondary schools; students’ performance; Gedeo
zone
1. Introduction
The role of school principals becomes more demanding and complex, as school
activities expand in size and complexity, because of the high expectations of the
learners and the parents. Principals, as school leaders, must be capable of
inspiring the school community with what they do and how they do it, so that all
stakeholders in the school context are motivated to realise the success of all
students (Hallinger & Heck, 2010; Onorato, 2013; Tesfaye & Ayalew, 2020). Thus,
the complex and continuously changing school environment requires capable
school principals, who exert a positive influence by exhibiting the appropriate
leadership behaviours that would motivate the school community to work
enthusiastically, in order to realise the success of all the students.
Studies of effective schools, where nearly all the students are assumed to have
reached their performance targets, have demonstrated the importance of studying
principals’ leadership behaviours as a major factor determining school success (cf.
Louis, et al., 2010; Budohi, 2014; Pinto, 2014; Day et al., 2016; Chia & Lia 2017). In
relation to this, the leadership styles of principals are an instrument that helps to
influence and shape the process and behaviour of the school community towards
realising better achievements for all the students (Hallinger, 2011).
More importantly, most researchers have considered only task and relationship-
oriented behaviours in their analysis of school effectiveness, while school leaders
themselves have in fact exhibited changed behaviours in accomplishing their
leadership goals. Consequently, there is a shortage of studies that have considered
the effect of all three meta-categories (task, relationship, and change-oriented
behaviours) in the field of school leadership and management. Such observed
discrepancy in the field makes this study important; as it may contribute to
resolving the existing lack of conceptual clarity in the category of leadership
orientation and contradictory findings on the most effective type of leadership
orientation that enhances better student performance.
2. Objectives and a hypothesis for the study

Taking this context into consideration, the objective of the study was to identify
the most effective type of principal’s leadership behavioural orientation that has
a positive effect on students’ academic performance in secondary schools in the
Gedeo zone of Ethiopia. To achieve the above objective, the research question was
posed as follows: Which type(s) of principals’ leadership behavioural
orientation has/have a positive effect on students’ academic performance in the
secondary schools of Gedeo zone, Ethiopia?
According to Cohen et al., (2007), “an alternative way of operationalising research

questions takes the form of hypothesis raising and hypothesis testing” (p. 82). In
32
line with this, we have set a hypothesis as an additional tool to guide our enquiry
with the intention of detecting the type and direction of relationship behavior that
exists between each type of principal’s leadership orientation and students’
academic performance. The following hypothesis with respect to the effect of a
principal’s leadership orientation on the academic performance of students (to be
rejected or confirmed by analysing the data collected for this purpose) was
phrased:
H1: All three types of principals’ leadership orientation (task, relationship, and
change) yield a statistically significant academic achievement of students in the
secondary schools of Gedeo zone in Ethiopia.
3. The Literature review

Leadership behaviours, in the broadest sense, refer to the style leaders exhibit in
work that exclusively scrutinises what they do, and how they act in the process of
directing people, implementing plans, or motivating followers in pursuit of the
goals commonly agreed on (cf. Mullins, 2005; Yukl, 2010; Northouse, 2013). In a
school context, the study of leadership behaviours is concerned with what the
principals do regarding their respective activities, roles, and responsibilities; and,
in addition, how they act instead of searching for the trait or personality
characteristics endowed by nature (Yukl, 2010, Northouse, 2016).
Leadership behaviours exhibited by a leader at work are important for ensuring

a smooth and effective functioning of an organisation and attaining shared goals.
Many educational researchers offered evidence of a positive correlation between
student academic success and effective leadership behaviours of principals (cf.
Brady, 2012; Day et al., 2016; Cruickshank, 2017; Chia & Lia, 2017). Such research
findings made the study of leadership behaviour a major focus point for
researchers in this field; since it is believed to be significant for increasing
personal, as well as organisational satisfaction and performance of employees
(Yukl, 2010; Northouse, 2016). Since the leadership style affects aspects, such as
the acceptance of decisions, the commitment of stake holders, the satisfaction and
productivity of the school, principals must select the appropriate behaviour for
the existing context (Botha, 2012). In this sense, the leadership behaviours
exhibited by school principals should be appropriate, in order to exert a positive
influence on the behaviours of teachers, students, parents and the other
stakeholders.
The main period of behavioural approaches to leadership occurred between 1945

with the Ohio State and Michigan studies and the mid-1960s, with the
development of the Managerial Grid (cf. Mullins, 2005; Yukl, 2010; Northouse,
2016). During this period, most scholars specified two broad dimensions of
leadership behaviours that incorporate task accomplishment and satisfy the
personal and organisational needs of followers. Task-oriented and relationship-
oriented leadership behaviours are therefore a foundation for various types of
leadership styles. This dichotomy is multi-faceted, referring to the way in which
power is distributed and decisions are made on what needs should be met.
33
Regarding this point, Bass (1990) relates the features of autocratic and democratic
leadership styles to task-oriented and relationship-oriented behaviours.
Furthermore, Bass (1990) has shown that the autocratic cluster of behaviours
refers to the performance dimension, while the democratic cluster denotes the
maintenance dimension.
Even though the two-factor dichotomy discussed above encompasses many

leadership styles, it fails to consider change-oriented behaviours that are
concerned with encouraging and facilitating change, innovation and emotional
commitment to the mission of the unit (Yukl, 2010). Furthermore, Yukl (2010)
elaborated that by the 1980s, change-oriented behaviour was implicit in some
theories of charismatic and transformational leadership. Change-oriented
leadership behaviours, as a third meta-category, was mentioned independently in
the 1990s by researchers in Sweden (Ekvall & Arvonen, 1991) and in the United
States (Yukl, 1999a). Each of the three meta-categories of leadership has a different
primary purpose. As stated by Yukl (2010), task-oriented leadership behaviour is
primarily concerned with accomplishing the task in an efficient and reliable way,
while relations-oriented behaviour is concerned with increasing mutual trust,
cooperation, job satisfaction and identification with the organisation. Likewise,
the third meta–category of change-oriented behaviour, is primarily concerned
with understanding the environment, finding innovative ways to adapt to it and
implementing the major changes in strategies, products, or processes.
In accordance with this development, the early fixation on considering and

initiating a leadership structure appears to have come to an end, as many
researchers now examine a broader range of behaviours and types of behaviours
that are more specific (Yukl, 2010). Indeed, the classification of the leadership
behaviours of task-oriented, relationship-oriented, and change-oriented is a
generalised taxonomy that is functional in all types of organisations in a similar
way.
Most researchers (cf. Armstrong, 2009; Yukl, 2010; Northouse, 2016) who have
conducted studies on the effect of leadership orientation on employees’
performance have merely deliberated task and relationship behaviours,
overlooking change-oriented behaviours, which school principals are practicing
continually in their leadership roles. Evidently, Northouse (2016) stated that
“whenever leadership occurs, leaders are acting out both task and relationship behaviours,
although in some situation they need to focus on task, whereas in others condition, they
may give more emphasis for relationship” (p.83). Based on an overall pattern of
research findings, Yukl (2010) asserts that ‘high-task’ and `high-relationship’
oriented leadership behaviours tend to be more effective, even though
concentrating on one type of behaviour and less on the others, could also make
the organisation effective in specific situations. According to Yukl (2010),
depending on circumstances, both styles could lead to an increase in the
performance and productivity.
From the discussion above it is evident that change-oriented leadership

behaviours, recently mentioned and discussed by various scholars as a third meta-
category, is indeed exhibited as a leadership behaviour in practice. There are,
however, limited research findings that have considered the impact of all three
34
meta-categories of leadership behaviour (task, relationship and change-oriented)

on the academic achievement of students.
In spite of various attempts made by researchers, no consensus has been reached

regarding what principals need to do to be effective in realising the improved
academic performance of students. Garland (2018) supports the above comment,
when stating that there is consensus about the determination of leaders, but not
yet on the type of meta-category of principal leadership behaviour that leads to
better student achievement. With the expansion of better understanding of the
determinateness of leadership in enhancing the academic performance of
students, different issues related to principals’ leadership style in becoming an
important research topic.
4. Methods
The mixed research method designs were used for this study; because it offers
better opportunity to collect, analyse, and interpret quantitative and qualitative
data interactively (Creswell & Plano Clark, 2011). The quantitative approach
enabled us to collect quantifiable data and to analyse the data statistically,
whereas the qualitative approach helped them to get different perspectives on the
respondents of the issue under study. Specifically, an explanatory sequential
mixed-method design was employed in this study. In line with the notion of this
design, the data were analysed in two separate phases, while the results were
triangulated to verify whether the findings of the two phases agreed.
The population of this study was 23 government general secondary schools in the
Gedeo zone, comprising 23 school principals, 46 vice-principals, 536 teaching
staff, 161 parents-teachers-students-association (PTSA) members, 22, 672 students
and 12 district-head supervisors and zone- educational experts (GZED, 2019). Out
of the total of 23 secondary schools in the zone, six sampled schools were
deliberately selected. The three least-achiever schools and three best-achiever
schools of the zone were selected by means of a maximum variation strategy,
based on the results of their tenth-grade students in the national examination for
the three consecutive years of 2016/17/ 2018/19.
We assumed that the source of variation in percentages of students’ pass rate to

preparatory education could be determined by the leadership competence
(exhibiting appropriate leadership behaviours/styles) of their school principals.
The justification given to this assumption is that the government secondary
schools of the zone have basic similarities almost in all school-related aspects, such
as school infrastructure, teachers’ qualifications, socio-economic status of parents,
except for the leadership competence of the school principals. Furthermore, we
took the necessary measures to minimise the interference of any other extraneous
variables. For instance, we excluded private schools from the study, as their
context differs from that of the government secondary schools.
A total of 141 teachers and 180 students (n=221) from the sampled schools
completed the questionnaires, as part of the quantitative phase of the study data.
These respondents were chosen from the six sampled schools by using the
proportional stratified sampling technique. This technique enabled us to find a
proportional number of respondents that represented the population (Johnson &
35
Christensen, 2014). In the qualitative phase of the study, the data were collected
from six principals and six PTSA chairmen of the sampled schools, and from four
district supervisors and two zone-education experts. These participants were
deliberately selected; as they could have some knowledge regarding the
leadership behaviours of the principals of the sampled schools; as they interact
consistently, due to their interdependent work relationships.
Questionnaires, interviews, and document analysis were used to collect relevant

data from the participants. The purpose for which we used questionnaires as a
data-gathering tool was to collect quantitative data from the large number of
respondents. Two sets of questionnaires were prepared separately for teachers
and students. Interviews and document reviews were employed to collect the
qualitative data to help shed more light on the quantitative results and to cross-
check the findings. The interviews were conducted by following the one-on-one
approach, as this approach assisted us to ask elaborative questions to justify the
respondents’ assertions. We recorded the interviews with the consent of the
participants, as a back-up to minimise the possible errors during transcription.
This occurred anonymously as no individuals’ or the school’s name was
mentioned.
Before starting the actual data-collection process by means of the designed

questionnaires, various attempts were made to refine this research instrument.
Amendments to the draft questionnaire were made according to the comments of
some colleagues, who were capable and willing to assist us technically. Besides
this, a pilot study was also conducted at two non-participant schools and an
improvement was enacted on the results. Based on the results obtained from the
pilot-study, the contents of four items were refined and the language clarity of
seven items was improved. The participants were treated in an ethical manner in
that we respected their decisions and protected them from any forms of
psychological, social, economic and/or physiological harm that could intimidate
their well-being. We applied the principles of informed consent, anonymity and
confidentiality of the participants to safeguard their security.
5. The Results
The objective of this study was to examine the effect of principal leadership
behavioural orientation on the academic achievements of the students and to
identify the effective types of leadership behaviours that could improve the
performance of the students. In examining the effect of the principal’s leadership
behavioural orientation on the students’ academic achievements, the recent
development in the categorisation of leadership orientations, which broadly
divides this into three meta-categories of behaviours, that were actualised (Yukl,
2010).
Accordingly, we included all three meta-categories, namely task-focused,

relationship-focused, and change-focused leadership behaviours in the analysis.
To examine the effect of the three types of leadership orientations of principals,
we used 12 items as indicators. One type of leadership orientation is represented
by four selected items in the questionnaire. By keeping this in mind, the
36
leadership orientations of principals of the sampled schools were analysed with

descriptive and inferential statistics in the following sections indicate.
5.1 Descriptive analysis of principals’ leadership behavioural orientation

To understand the leadership behavioural orientation of principals in the two
groups of schools, the participants were asked to rate the extent to which their
school principals frequently exhibited the specified leadership behaviour. Thus,
the score obtained by each principal on each item was used as an indication to
show the extent to which the school principals valued specific behaviour in their
daily leadership practices. Task-oriented leadership behaviours will be discussed,
followed by relationship-orientated behaviours and change-oriented leadership
behaviours of the two groups analysed comparatively.
As indicated above, the three least-achiever and the three best-achiever schools of
the zone were selected deliberately, based on their tenth-grade students’ results
in national examination in the three consecutive years of 2016/17 to 2018/19. To
make the analysis easy and to ensure the anonymity of the participants, a code
name was given to each sampled school, as well as to the two groups of schools.
The code name Group 1 was given to the category of the three least achievers,
whereas the code name Group 2 was given to the category of the three best-
achiever schools of the zone. Likewise, the code name of RLAS-23, RLAS-22 and
RLAS-21 have been assigned to the three least achievers; while the code names of
RBAS-3, RBAS-2 and RBAS-1 bwasgiven to the three best achiever schools of the
zone respectively.
5.1.1 Analysis on the task-oriented leadership behaviours of the principals

As described above, the descriptive analysis of the task-oriented leadership
behaviours of the principals was represented by comparing the two categories of
the sampled schools, in order to assess whether they showed any difference. To
this effect, the score that the sampled schools’ principals obtained in the form of
mean values and standard deviations were compared, as indicated in the table
below.
Table 1: Task-oriented leadership behaviours of principals’ leadership orientation

Leadership N LEADERSHIP BEHAVIOURAL ORIENTAION OF SCHOOL PRINCIPALS
orientation 0 Items Group 1 Group 2
M SD M SD
Task- 1 Set standards of performance & follow up its 2.46 1.14 3.67 1.01
oriented accomplishment
leadership 2 Define role & responsibilities of each member 2.56 .739 3.81 .936
behaviours 3 Provide plan that specify what task & how it to be done 2.40 1.11 3.79 .896
4 Aware members about what is expected of them 2.51 .806 3.76 .918
Mean score task-oriented behaviours 2.48 2.05 3.76 1.88
Note: The code name Group 1 represents the the three least achievers while the code name
Group 2 represents the the three best achiever schools of the zone
In item 1 of table 1, the respondents of both groups were asked to identify the
extent to which their school principal had achieved the ‘set standards of
performance for members and monitors of its accomplishment. In their reply, the
37
respondents in the Group 1 schools revealed that principals rarely did (M=2.46;
SD= 1.14), while the respondents in Group 2 replied that the school principals
usually did (M=3.67; SD=1.01). From the analysis, it is possible to infer that the
principals of the Group 2 schools placed a high emphasis on the specified task,
and, as a result they, were better achievers than those from Group 1. This might
have been emanated from their ambition to be successful and to control the trends
in their school. Some earlier studies, which have been carried out in this field
stated that well-organised monitoring practices have a positive effect on
enhancing managerial effectiveness (cf. Kim & Yukl, 1995; Amabile et al., 2004).
Engagement of principals in the task of ‘defining roles and responsibilities for

each member and committee was assessed in item 2 of Table 1. The respondents
replied that the principals of the Group 1 schools sometimes engaged in task-
defining roles (M=2.56; SD=.739), while the respondents in Group 2 revealed that
the principals usually performed the task themselves (M=3.81; SD=.936). This
implies that the principals in Group 2 were effective in engaging the entire school
community and various committees by prompting them frequently to accomplish
their roles and responsibilities adequately. Thus, they were better achievers than
the principals in Group 1.
With reference to item 3 in Table 1, the respondents in Group 1 replied that the
principals rarely provided a work plan with a clear description of the tasks to be
carried out together with details on how the tasks should be carried out (M=2.4;
SD=1.1), while the respondents in Group 2 revealed that the principal of their
school usually exhibited this specific behaviour (M=3.79; SD=.896). With reference
to this point, Yukl (2010) illustrated that even a subordinate, who is highly
competent and motivated may fail to achieve a high level of performance if he or
she is confused about his/her responsibilities and priorities. This implies that the
principals in Group 2 performed better than the principals in Group 1 in keeping
the school community on the right track by using a work plan that encompasses
a clear description of the tasks to be carried out together with details on how this
needs to be done.
With reference to item 4 of Table 1, the respondents replied that the principals in
the Group 1 schools reminded the school members about what was expected of
them sometimes (M=2.51; SD= .806), while the respondents in Group 2 replied
that their school principals often exhibited the specified behaviour (M=3.76; SD=
.918). Regarding the importance of clarifying the expectations for managerial
effectiveness, various earlier studies have found a positive relationship between
clarifying expectation and managerial effectiveness (cf. Bauer & Green, 1998; Kim
& Yukl, 1995). Depending on the results of the analysis, it is possible to infer that
the principals of Group 2 were effective in making the school community
contribute to the schools; as these principals frequently reminded them of what
was expected of them.
In complementing these quantitative data, qualitative data were gathered from

the principals, district supervisors, chairmen of PTSA and zone education experts.
The supplementary qualitative data were collected and analysed with the aim of
38
counterchecking whether the result obtained from the qualitative approach match
the quantitative findings of the study. Thus, the qualitative data collected through
interviews and some facts collected from documents were analysed to determine
whether the participants’ opinions, beliefs and perceptions supplement the results
obtained in the quantitative phase of the data analysis.
In order to identify the useful and justifiable types of principal leadership,

behavioural orientations that help to enhance better students’ achievements; the
next interview question was raised: “In your opinion, which meta-category of
leadership behaviours do school principals need to enhance better academic
achievement of their students? And why?” Depending on the response given by
the interviewees, it could be possible to organise their answer under three
different sets of views.
The largest proportion of participants (55.5 %) stressed the importance of putting

high emphasis on the task, above average emphasis for the relationship and
average emphasis for change-oriented behaviours in descending order from high
to low. For instance, the response given by P-6 was presented here, as it could
signify those participants who argued for the importance of emphasising task-
oriented behaviours. P-6 suggested, “To be effective, principals should spend more
time on task-oriented behaviours like setting plan, organising the work and the people,
directing the people and controling the teaching-learning process”. A proponent of this
view argued that when principals frequently exhibit task-oriented behaviours, the
school community becomes alert of what is expected of them in improving
students’ academic achievement. Generally, the following major concepts were
synthesised from the interviewee’s responses:
• Principals have to prioritise the accomplishment of the teaching-learning task,
as it is the major objective for which the school has been established.
• The task first, relationship second and change third view could not work as a
rigid rule, but it can be reversed in accordance with the need.
5.1.2 Analysis on the relationship-oriented leadership behaviours of principals

The relationship-oriented leadership behaviours of school principals were
analysed by comparison to identify whether the two groups of school principals
emphasised the specified behaviours at a similar or different rate. To this effect,
the score given for items number 5 to 8 by the respondents regarding the
principals of the sampled schools were compared, using mean scores and
standard deviations.
39
Table 2: Analysis of relationship-oriented leadership behaviours of principals

Leadership Relationship-oriented leadership behaviours of principals
orientation N items Group-1 Group2
o M SD M SD
Relationship- 5 Concern for the well-being of members 3.25 .705 3.32 1.08
Oriented 6 Communicate actively with members 2.76 1.09 3.36 .969
Leadership 7 Help members to get along each other 3.13 .667 3.36 .951
behaviours 8 Respond happily to comments of the stakeholders 3.13 .629 3.33 .951
Mean score on relationship-oriented behaviours 3.07 1.76 3.34 1.98
The results of the analysis showed that the principals of all the sampled schools
of the best and least achiever schools sometimes exhibited relationship-oriented
behaviours (moderate emphasis given to relationship) in a similar way. This can
be seen from the overall mean score on the relationship-oriented behaviours of
Group 1 (M=3.07; SD=1.76) and Group 2 (M=3.34; SD=1.98). For instance, with
reference to the well-being of the school community (item 5), the respondents of
both groups replied that the principals of their schools `sometimes’ emphasised
their need. This is evident from the mean scores (M=3.25; SD=0.84) of Group 1
and that of Group 2 (M=3.32; SD=1.08). It implies that both groups of respondents
were partially satisfied with the emphasis placed on their well-being.
Similarly, with reference to item 6, the respondents in both Group 1 (M=2.76;

SD=1.04) and Group 2 (M=3.36; SD=.969) revealed that the principals `sometimes’
communicate with the school community. Likewise, engagement of principals in
helping members to get along with each other (item 7) was rated ‘sometimes’ by
both Group 1 (obtaining a mean score of M=3.13; SD=817) and Group 2 (obtaining
a mean score of M=3.36; SD=.951). With reference to item 8, the respondents of
both groups verified that the principals ‘sometimes’ respond favourably to the
suggestions of the school members, with Group 1 obtaining a mean score of
(M=3.13; SD=.793) and Group 2 obtaining a mean score of (M=3.33; SD=.951).
The second group that comprises a moderate proportion of the participants

(33.33%) advised secondary school principals to give the highest emphasis for
relationships than for tasks or change behaviours. The participants in this group
proposed high relationship and average change and task-oriented behaviours
with the assumption that satisfied staff would show high commitment to realising
better achievement of students. They further argued that exhibiting high task
behaviours may not be helpful, as high school teachers are educated and they are
mature enough to guide themselves.
5.1.3 Analysis of change-oriented leadership behaviours of principals

The change-oriented behaviours of principals in the two categories of schools
were analysed and compared to identify whether it differs in the two sets of
groups. Based on the perception of the respondents, the scores given to the
principals of the sample schools for items number 9 to 12 were compared in terms
of mean values and standard deviations.
40
Table 3: Change-oriented leadership behaviours of principal leadership orientation

Leadership N Leadership behavioural orientation of school principals
orientation o Items Group-1 Group-2
M SD M SD
Change- 9 Envision exciting new possibilities for the school 2.72 .975 3.61 .935
oriented 10 Encourage a staff to innovate a new work strategy 2.38 1.06 3.6 .929
leadership 11 Interpret events to explain a need for urgent change 2.76 1.01 3.45 .979
behaviours 12 Encourage the staff to brought major change 2.78 1.04 3.54 1.02
Mean score of change- oriented behaviours 2.66 2.02 3.55 1.93
As evident from item 9 in table 3, the respondents were asked to verify the extent
to which the principals of their schools envisage exciting new possibilities for the
school. The respondents in Group 1 responded that the principals sometimes
exhibited the specified behaviour (M=2.72; DS=.987), whereas the respondents in
Group 2 replied that the principals of their schools frequently exhibited this
behaviour (M=3.61; DS=.935). The result implies that the principals of the best
achiever schools were actively searching for new possibilities and initiating the
school community to use these with the emerged opportunity, in order to bring
about change, and to realise the success of the school.
The respondents of Group 1 replied that the principals of their schools rarely
encouraged the school community to innovate a new work strategy (M=2.38;
SD=1.06), while the respondents in Group 2 replied that the principals of their
schools frequently engaged in this specified leadership behaviour (M=3.60;
DS=.929). This implies that the teachers and other members in Group 2 have the
freedom of proposing and exercising new ways of thinking; and they were
effective in motivating staff for the innovation of new strategies. Regarding this
point, Leithwood, Harris and Hopkins (2008) stated that to be effective, principals
need to serve as catalysts to unleash potential capacities, which already exist in
the staff and other members. With reference to item 11, the respondents replied
that the principals of both Group 1 and Group 2 sometimes interpreted new
events to explain a need for urgent change with mean scores of (M=2.76; DS=1.01)
and (M=3.45; DS=.979) respectively.
This result implies that the emphasis on the specific behaviours of principals in
both groups differed. With reference to item 12, the respondents replied that the
principals of Group 1 `sometimes’ (M=2.78; DS=1.04) encouraged the staff to bring
about major change, while the respondents asserted that the principals of Group
2 schools `frequently’ engaged in the specified leadership behaviours (M=3.54;
DS=1.02). Therefore, it is fair to say that the teachers in the Group 2 schools are
encouraged more to employ innovative ideas than the teachers in the Group 1
schools.
A moderate proportion of informants have asserted that principals must strive to

bring about change in all the aspects of the school life, but it should not be done
merely for the sake of change; it should also be applied when necessary. Thus,
41
when there is clear purpose and adequate reasons, principals could emphasise
changed behaviours.
There was a small proportion of participants (11.11 %) who argued for equality of
emphasis to all the three meta-categories of leadership behavioural orientation.
They believed that the principals should exhibit all the three meta-categories of
behaviours in equal magnitude in their daily work. They advised the principals
to put equal emphasis on all three meta-categories of leadership behaviours in all
the contexts.
The data extracted from document analysis revealed the presence of visible
differences in the leadership orientation of Group 1 and Group 2 school principals.
For instance, slogans, pictures as well as motivational quotations mounted in the
whole of Group 1 schools are urging stakeholders to hold up positive
interpersonal relationships, while in the case of Group 2 schools such document
insist that the school community should bring about change and ensure the
culture of hard work. Furthermore, the document reviewed of the participant
schools revealed that principals of Group 2 schools to perform better in preparing
and implementing strategic plans than the principals of Group 1 schools
implemented.
5.2 Analysis of principal-leadership behavioural orientation (PLBO) with

inferential statistics and a hypothesis test
In this section, the analysis of principal-leadership orientation has been treated by
associating each style with the effect it has on improving students’ academic
achievement. The analysis was carried out with the help of inferential statistics.
Firstly, the mean scores of the principals of the three least and the three best
achiever schools of the zone were calculated, based on the degree of emphasis
they have given for task-, relationship- and change-oriented leadership
behaviours. Simultaneously, the percentages of students who secure minimum
pass marks in the tenth-grade national exams in three consecutive years were
computed for these schools; and they were used as dependent variables of the
study. Secondly, the associations between dependent and independent variables
were examined with multiple line-graphs, by using the same data.
Finally, the research question that was phrased earlier was examined by means of
inferential statistics, followed by the analysis of the related hypothesis for the
study. The results are presented in Table 4 below. The table illustrated the extent
to which the principal of each sampled school focused on task-, relationship-
and/or change-oriented behaviours in juxtaposition with the percentage of
students who had succeeded in joining the preparatory schools in the last three
years.
Table 4: Summary of Leadership Behavioural Orientation of School Principals

No Summary of Leadership Behavioural Orientation of School Principals
Leadership Group-1 score per school Group-1 Group-2 score per school Group-2
score
Orientatio
RLAS-23 RLAS-22 RLAS-21 score RBAS-3 RLAS-2 RLAS-1 score
n
Task-oriented M 2.44 2.48 2.53 2.48 3.67 3.76 3.84 3.76
1 Behaviours SD 2.10 1.94 2.1 2.05 1.94 1.91 1.78 1.88
42
2 Relationship- M 3.05 3.03 3.11 3.07 3.27 3.32 3.47 3.34

oriented SD 1.81 1.79 1.71 1.76 1.9 2.07 1.91 1.98
behaviours
3 Change-oriented M 2.64 2.57 2.74 2.66 3.43 3.53 3.73 3.55
Behaviours SD 2.14 1.74 2.11 2.02 1.86 1.04 1.85 1.93
Percentage of students’ able to
29.56 30.62 35.95 - 58.16 61.13 66.68 -
pass & join preparatory level
Rank out of 23 Secondary
3rd 1st
rd nd nd
23 22 21st - 2 -
School
In item 1 above, the engagement of principals in task-oriented behaviours in the

two categories of schools were assessed. The data revealed an unequal level of
emphasis on task-oriented leadership behaviours of principals of the least and the
best achiever schools. The engagement of principals in Group 1 schools on task-
oriented leadership behaviours was rated as rare. This can be revealed from the
mean score of Group 1 (M=2.48, SD=2.05). However, the Group 2 schools
respondents rated the frequent engagement of principals in task-oriented
behaviours. This can be ascertained from the mean score of Group2 (M=3.76,
SD=1.8).
This result indicated that there was more emphasis on the task-oriented role of
the principals of Group 2 schools in realising better student achievement. This
implies that high emphasis on task-oriented behaviours could enable principals
to realise better student achievements.
The summarised data on the engagement of principals in the two categories of

schools in terms of relationship-oriented behaviours revealed moderate emphasis
on relationship-oriented behaviours by principals in both categories. This can be
identified from the mean score of Group 1 (M=3.07, SD=1.76) and the mean score
of Group 2 (M=3.34, SD=1.98). This ascertained occasional engagement by all the
sampled school principals. This implies that it is difficult to infer whether frequent
engagement-relationship behaviours result in better student achievement.
Item 3 of Table 4 illustrates the summarised result of change-focused behaviours

of principals in the two categories of schools. The rating given for engagement of
all the principals of Group 1 in change-oriented behaviours was occasional, while
the rating score of two principals out of the three revealed ‘frequent’ engagement
in the Group 2 best achievers category. As the result of this, the mean score of
principals for change-oriented behaviours revealed occasional engagement (M=
2.66, SD=2.2) for Group 1, whereas the score of Group 2 principals (M=3.55,
SD=1.93) revealed frequent involvement of leaders in change-oriented activities.
From this result, it can be assumed that there is a positive relationship between
increasing change-oriented behaviours of principals’ and the improvement of
students’ achievements.
To supplement the above analysis that was carried out by using inferential
statistics regarding the relationship between principals’ leadership orientation
43
and students’ performance in the two groups of schools of the zone, the next
figure illustrates the associational trend observed between the two variables
considered in this study. According to Cohen et al., (2007), “multiple line graphs
are useful for showing trends in continuous data on several variables in the same graph”
(p. 507). With this understanding, the trends of the three meta-categories of
behavioural orientation of principals are visualized graphically by representing
them with three different lines in the graph. In the graph, the names of the
sampled schools are indicated on the ‘x’ axis, while the score obtained for each
type of behavioural orientation is pointed on the ‘y’ axis. The trends of the three
behavioural orientations of the principals of the least and best achiever schools
are graphically illustrated in Figure 1 below.
Figure 1: Line graph shown the behavioural orientation of the principals of the sampled
schools
The above line graph visualises the trends in the three types of leadership
orientation in the least achiever schools of RLAS-23, RLAS-22, and RLAS-21 and
that of Group 2 schools indicated as RBAS-3, RBAS-2, and RBAS-1. As can be
observed from the above graph, the overall engagement of the best achiever
schools in all the three mega-categories of leadership behaviours (task,
relationship, and change) exceed those principals of the least-achiever schools of
the zone. This means that the principals of the best-achiever schools were active
in exerting influence on their followers. The overall leadership orientation of
principals in the best- and the least-achiever schools of the zone, illustrated by the
above line graph, which exhibits the degree of emphasis they gave to the task,
relationship and change behaviours described here comparatively.
• The principals of the least achiever schools put the highest emphasis on
relationship behaviours, whereas the principals of the best-achiever
category put the lowest emphases on the relationship behaviours. The line
44
graph representing relationship-oriented behaviours is almost as it appears

horizontally, which implies the degree of emphasis on the specified
behaviours is similarly moderate in both groups of schools.
• However, with reference to task-oriented leadership behaviours, the
principals of the least- achiever schools engaged minimally, whereas the
principals in the best-achiever category exhibiedt task-oriented behaviours
at the highest level.
• With respect to the change-oriented leadership behaviours in both groups
of schools, it becomes clear that in best-achiever schools, top priority is given
to the tasks, while in the least-achiever schools, where the top priority is
given to relationships. The group of least- achiever schools showed a
declining, or slowly increasing trend, while the best-achiever category
showa fast-increment.
• By placing higher emphasis on task- and change-related behaviours,
principals of the best achiever schools succeed in realising better academic
performance of the students. The graph visualises that the principals of the
best-achiever schools are more influential than those of the least-achiever
schools in all types of leadership orientation. This implies that principals in
Group 2 are more decisive than those in Group 1, who illustrate passiveness
in their actions.
5.3 Analysis on principals’ leadership orientation and hypothesis testing

In addition to the above, line graph, computing the correlation between the
dependent and independent variables and testing a related hypothesis with
appropriate inferential statistics helped us to draw a conclusion. (See the
regression result in the table below).
Table 5: Regression Result on Leadership Behavioural Orientation
Orientation Coefficient Std. Error t-Statistic Prob. R2 Adjusted R2

C -0.201640 0.017235 -11.69958 0.0000 0.86 0.85
Relationship -0.003043 0.002506 -1.214153 0.2256
Task 0.015571 0.002169 7.178986 0.0000**
Change 0.007654 0.000710 10.77711 0.0000**
Table 5 above clearly shows that more engagement in relationship behaviour

negatively affects students’ academic achievement by almost 0.3%, provided that
other factors are equal. However, the probability value (p=0.2256) indicates that
there is no statistical evidence on whether the effect of such behaviour is
significant. In conclusion, relationship-oriented leadership behaviour had a
negative impact on students’ academic achievement, although the effect was
insignificant.
However, task and change-oriented leadership behaviours could more likely

affect the academic achievement of students positively. In the case of task-oriented
leadership behaviour, the coefficient 0.015571 showed that an increase in task-
leadership behaviour by 1% could add 1.5% increase in students’ achievement.
Similarly, change-oriented leadership behaviour affects students’ achievement by
45
0.7%. The probability values (p=0.0000) for both orientations confirmed that there
is a strong statistical evidence that task- and change-oriented leadership
behaviours have a significant effect on students’ academic achievement. It can be
inferred from the coefficients 0.015571 and 0.007654 of the task and change
behaviours respectively; and this implies that an increase in these two types of
leadership could yield statistically significant improvement in students’
achievements.
Hence, with statistical evidence, it is possible to infer that task- and change-
oriented leadership behaviours have a positive effect on students’ achievement,
although no evidence was found on whether the relationship behaviour of
leadership affect students’ achievement. In general, the positive hypothesis that
was stated as ‘all types of principals’ leadership orientation (task, relationship and
change) yield a statistically significant academic achievement of students in the
secondary schools of Gedeo zone’ is rejected, as there was no statistically
significant positive correlation found between the relationship-oriented
leadership behaviors of principals and the academic achievement of students (r
= -0.003043, ρ = 0.2256).
6. Discussion of the findings

In this sub-section, the findings obtained from the descriptive and inferential
analysis of empirical data, content review of qualitative data and literatures
reviewed with respect to the impact of principals’ leadership orientation (task,
relationship, and change behaviours) on students’ achievement are presented.
Principals who put more emphasis on task-behaviours are generally more likely
to exert a positive effect on the behaviours, thoughts, and actions of the school
community, which, in turn, contributes to realising a better academic achievement
of students. Comparatively, the principals of Group 2 schools put high emphasis
on task-behaviours, which enabled them to achieve better results in students’
learning, as focusing on the core activities of their teaching and learning roles
make the school community to perform more effectively, Such task-oriented
behaviours of Group 2 principals may emanate from their strong sprit of success,
and the ambition they have to control the trends in their respective schools (cf.
par. 5.5.3.1).
The findings obtained from both quantitative and qualitative data analysis
revealed the presence of a strong positive bond between exhibiting task-oriented
behaviours and the improvement of students’ achievement. The regression result
of (r=0.015571; p=0.0000) obtained from the quantitative data analysis showed
that when school principals increase task-oriented behaviours by 1%, it could
yield 1.5% improvement in student achievement (cf. par. 5.5.3.2).
In the same way, the findings obtained from qualitative data analysis also
indicated that exhibiting task-oriented behaviours help to exert a positive
influence on the behaviours and actions of the school community, so that they
work together to enhance better student achievement. Many interviewees
suggested that principals spend more time on task-oriented behaviours, like
setting up a plan, organising work and people, directing people, and controlling
46
the teaching-learning process and encouraging the school community to realise

better student achievements. Most interviewees argued that when the principal
frequently exhibits task-oriented behaviours, the school community would
become more attentive to what is expected of them and will therefore contribute
to the improvement of the academic achievement of students (cf. par. 5.6.4).
Furthermore, by supplementing the results mentioned above, the findings

obtained from document review of the sample schools indicated that principals of
the Group 2 (best achiever) schools exhibited more task-oriented leadership
behaviours than the principals of Group 1 schools. For instance, a review of the
school minutes of Group 2 schools indicated lists of different committees with
specific duties and responsibilities which is indicative of provision of in-school
training, as well as a checklist to control accomplishment and discussions on the
performance reports of many sections revealed that principals in the best achiever
category exhibited task-oriented behaviours (cf. par. 5.6.4).
The results obtained from descriptive analysis indicated that task-oriented

behaviours exhibited by Group 1 and Group 2 principals were entirely different.
The results generally indicated low engagement of Group 1 principals on task-
behaviours, while Group 2 schools principals engaged at a higher level.
Specifically, in terms of ‘providing a plan specifying the task and how it is done’,
‘define role and responsibilities of each member’ and ‘setting standards of
performance and monitoring members’ accomplishment, Group 2 principals have
exhibited high task-oriented leadership behaviours. High emphasis on task
behaviours offers the opportunity to initiate work, direct members towards the
designed goals and the monitoring of members’ performance, which, in turn,
enhances2 better student achievement (cf. par. 5.5.3.1).
The relationship-oriented behaviour has a negative effect on students’ academic

achievement, although this effect is insignificant. High emphasis on relationship
behaviours could not result in the improvement of academic achievement of
students. In this study, the results obtained from the computed regression
coefficient (r=-0.0030043, p=0.2256) revealed that an increased engagement in
relationship behaviour by 1%, negatively affected students’ academic
achievement by almost 0.3%. Furthermore, the probability value (p=0.2256)
indicated that there was no statistical evidence of whether the effect of such
behaviour is significant (cf. par. 5.5.3.2).
Exhibiting relationship-oriented behaviours may not have a direct impact on

students’ achievement. Slightly different findings were found from the qualitative
data analysis of relationship-oriented behaviours. Most interviewees argued that
apart from task-behaviours, principals put the second-most emphasis on
relationship-behaviours to be effective, although the results obtained from the
quantitative data analysis revealed principals’ high or low engagement in
relationship behaviours did not make any difference to students’ achievement.
Evidently, the emphasis on relationship behaviours by Group 1 and Group 2
school principals was moderate; however, by nullifying the influence of
relationship behaviours, the student’ achievement scores were significantly
47
different in the two groups of schools. Thus, the incompatibility of the findings
obtained from the two types of data analysis made us cautious about the
importance of keeping relationship behaviours only at a reasonable level of
emphasis.
Increased engagement in change behaviour could result in increased student

achievement. The results obtained from the computed regression coefficient of
(r=0.007654; p=0.0000) indicated that when a change-oriented leadership
behaviour increases by 1%, students’ achievements’ increase by 0.7%. The result
of the analysis indicated that change-focused behaviours of Group 1 principals
rated as ‘moderate’, while that of Group 2 principals rated `higher’. This indicates
that there is a direct relation between change-oriented behaviours of principals
and students’ achievement (cf. par. 5.5.3.2).
The results summarised from the qualitative data analysis indicated that
principals must strive towards bringing about change in all aspects of the school
life, but it should not merely be done for the sake of change only; it should only
be applied when necessary. Thus, when there are clear purpose and adequate
reasons, principals could put high emphasis on changed behaviours. Specifically,
regarding ‘envisaged exciting new possibilities for the school’, Group 1 principals
exhibited average emphasis, while Group 2 principals’ engaged highly in
specified leadership roles. This implies that the principals of Group 2 schools were
actively searching for new possibilities and initiating members to engage in the
emerged opportunities. In terms of ‘encouraging members to innovate new
strategies, the emphasis given by Group 1 principals was low, while Group 2
principals’ engagement rated higher. This implies that members in best achiever
schools are encouraged to think out of the box and to exercise innovated strategies
(cf. par. 5.6.4).
Furthermore, the relationship between principals’ leadership orientation and

students’ achievement in the least and best achiever categories of schools that
were examined with the help of three separate line graphs indicated in figure 1
and its summarised findings are discussed below. From the results it is visualised
by the three separate lines, the following conclusions can be drawn.
Principals of Group 2 schools did better than the principals of Group 1 schools in
their engagement in all three types of leadership categories, although the
difference observed in terms of relationship behaviour appears to be insignificant.
The line graphs indicated that the principals of Group 2 schools were more active
than the principals of Group 1 schools. This implies that the principals of best-
achiever schools are more influential than the principals of the least-achiever
schools. The findings obtained from the analysis of the data revealed the necessity
of offering a moderate and above-average emphasis on all types of leadership
orientations that include task-, change- and relationship-behaviours to be effective
in realising better achievements of the students. Particularly, high emphasis on
task- and change-oriented leadership behaviours could yield higherz return in
terms of students’ academic achievements (cf. par. 5.5.3.2).
48
The graph also illustrated that the emphasis by principals of Group 2 on task and
changed behaviours were high. This implies that high emphasis on task- and
change-behaviours may help to realise better academic achievements. The line
graph that represented relationship behaviours ascertained a moderate level of
engagement of both groups. This indicated that there is no direct relation between
relationship behaviours and students’ achievements, as the same degree of
emphasis resulted in a different level of student achievement.
From the findings, we can assume that Group 1 principals engaged in task-
accomplishment and initiating change at lower levels to minimise challenges that
may encounter them if they exhibit high task and change behaviours. Thus, it is
being reasonable if someone correlates high engagement of principals in
accomplishment of tasks and endorsement of change with better student
achievements.
7. Recommendations
Based on the findings obtained from the analysis of both quantitative and
qualitative data of this study and an understanding gained from the review of
previous related studies, recommendations were made on how principals’
leadership orientation could yield higher students’ academic performance.
• Principals have to prioritise (emphasis) the accomplishment of the teaching-
learning process, as it is the major objective; the school is established for high
emphasis on task behaviours, which helps school principals to succeed, as it
affords them the opportunity to initiate the work of direct members towards
the specified goals and enable them to monitor community members’
performance. When school, principals exhibit task-oriented behaviours, the
school community becomes vigilant for what is expected of them; and
consequently by doing as expected, students’ academic achievement is
realised in the process (cf. par. 5.6.4).
• Principals are required to spend more time on task-oriented behaviours, such
as setting a vision, planning, organising work and people, directing people,
and controlling the teaching-learning process (cf. par. 5.5.3.1).
• Principals need to be aware that high relationship-oriented leadership
behaviours may obstruct students’ achievements, unless there is a concrete
reason to do so. Thus, they should not unnecessarily exhibit high relationship
behaviours simply for the purpose of being liked and to avert challenges that
they may encounter (cf. par. 5.5.3.2).
• Even though, principals’ engagement in relationship behaviours, such as ‘
emphasising the well-being of the school community’, ‘helping school
members to get along with one another’, ‘communicating actively with
members’ and ‘responding favourably to the comments of stakeholders’ may
not directly make a difference to students’ achievements; since it is a necessary
condition for improving emotional integrity in the school Thus, school
principals are required to put reasonable emphasis on relationships and
behaviours (cf. par. 5.6.4).
• Although exhibiting high relationship behaviours, merely for the purpose of
getting affiliation from the school community, may not bring about
performance improvement, satisfying members’ needs without
49
compromising the work to be done, in order to enable the school principals to

secure co-operation and emotional integrity.
• Principals must strive towards bringing about change in all aspects of the
school life; but it should not be done merely for the sake of change itself; it
should be done only when necessary. Thus, when there is clear purpose and
adequate reasons, principals need to put high emphasis on change
behaviours.
• When principals actively search for new possibilities and initiate the school
community needs to use these, should the need arise; they may support their
school by increasing the productivity of members that might improve
students’ achievements. Similarly, putting high emphasis on change
behaviour or ‘envisioning exciting new possibilities for the school’ and
‘encouraging members to innovate new work strategies, in order to inspire
members to think out of the box, and to apply new strategies, which would
pave the ways for increasing student achievements (cf. par. 5.5.3 & 5.6.4).
8. Conclusion
School principals exhibiting task- and change-focused leadership behaviours may
help to improve student achievements, whereas high emphasis on relationship-
behaviours may not result in high student performance. High emphasis on task-
behaviours enables school principals to succeed, as it gives the opportunity to
initiate work, direct members towards specified goals, and help them to monitor
staff members’ performance. The result of the quantitative data analysis indicated
that relationship-oriented behaviour has a negative impact on students’ academic
achievements, although the effect is insignificant. Thus, putting a high priority on
relationship-behaviours may be inversely related with student achievement.
Exhibiting high relationship- behaviours merely for the purpose of getting
affiliation from the school community may not bring about any improvement in
performance; however, satisfying members’ needs without compromising the
work to be done might enable the school principals to secure co-operation and
emotional integrity.
Furthermore, to be effective in realising the better academic performance of

students, school principals need to engage actively in all the affairs of the school,
although the degree of emphasis on the three meta-categories of leadership
behaviours may differ. Furthermore, the results of the study indicated that the
increasing change-oriented behaviours of school principals could enable them to
improve students’ achievements, unless the situation merely demands
maintenance and not endorsing change. Further research is recommended in
future to assess whether the results obtained in this study are found reliable, or
not, by repeating the study on a larger scale and in a diversified school context.
50
9. References
Amabile, T. M., Schatzel, E. A., Moneta, G. B., & Kramer, S. J. (2004). Leader behaviours
and the work environment for creativity: Perceived leader support. Leadership
Quarterly, 15 (1), 5–32.
Armstrong, M. (2009). Handbook of Management and Leadership: A guide to managing for
results. 2nd ed. London and Philadelphia, KOGAN PAGE. ISBN 978-0-7494-5417-
3
Bass, B. M. (1990). Handbook of leadership: A survey of theory and research. New York: Free
Press.
Bauer, T. N., & Green, S. G. (1998). Testing the combined effects of newcomer information
seeking and manager behavior on socialization. Journal of Applied Psychology, 83,
72–83.
Brady, L. (2012). Principal Evaluation: A Description of current practice. San Diego state
University.
Budohi, L. A. (2014). The Effect of Principals’ Leadership Styles on the Academic Achievement of
Students in Public Secondary Schools in Lurambi Division, Kakamega County, Kenya. .
KCA University. DOI: 10.13140/RG.2.1.1427.8807
Botha, R. J. (2012). “Exploring Practices Determining School Effectiveness in South
African Schools.” International Journal of Educational Studies 4 (2): 137– 145
Choi A., & Gil M. (2017). Does school leadership affect student academic achievement? What
Works in Education? Using evidence to improve education. Institut Catala d’Avaluacio
de Politiques Publiques
Cohen, L., Manion, L. & Morrison, K. (2007). Research methods in education. (6th ed.)
London and New York: Routhledge. Taylor & Francis e-Library.
Creswell, J. W. (2012). Educational research: Planning, conducting, and evaluating quantitative
and qualitative research (4th ed.). Upper Saddle River, NJ: Pearson Education.
Creswell, J. W., & Plano Clark, V. L. (2011). Designing and conducting mixed methods
research (2nd ed.). Thousand Oaks, CA: Sage.
Cruickshank, V. (2017). The Influence of School Leadership on Student Outcomes. Open
Journal of Social Sciences, 5, 115-123.
Day, C., Gu, Q. & Sammons, P. (2016) The Impact of Leadership on Student Outcomes:
How Successful School Leaders Use Transformational and Instructional Strategies
to Make a Difference. Educational Administration Quarterly, 52, 221-258.
Ekvall, G., & Arvonen, J. (1991). Change-centred leadership: An extension of the two-
dimensional model. Scandinavian Journal of Management, 7, 17–26.
Garland, J. E. (2018). "The Relationship between Principal Leadership Behaviour and
Student Achievement in Low Performing Schools". Seton Hall University
Dissertations and Thesis. (ETDs). 2511.
Hallinger, P. (2011). Leadership for learning: Lessons from 40 years of empirical research.
Journal of Educational Administration, 49(2), 125–142.
Hallinger, P., & Heck, R. H. (2010). Collaborative leadership and school improvement:
Understanding the impact on school capacity and student learning. School
Leadership and Management, 30(2), 95–110.
Johnson, R. B, & Christensen, L. (2014). Educational Research: Quantitative, qualitative, and
mixed approaches. (5TH ed). SAGE Publications Ltd.
Kim, H., & Yukl, G. (1995). Relationships of self-reported and subordinate-reported
leadership behaviours to managerial effectiveness and advancement. Leadership
Quarterly, 6, 361–377.
Leithwood, K., Harris, A., & Hopkins, D. (2008). Seven strong claims about successful
school leadership. School Leadership and Management, 28(1), 27-42.
51
Louis, K. S., Leithwood, K., Wahlstrom, K. L., &Anderson, S. E. (2010). Investigating the
links to improved student learning: Final report of research findings. St. Paul:
University of Minnesota.
Mullins, L, J. (2005). Management and Organisational Behaviour. (7th ed.). Prentice Hall.
http://www.booksites.net/mullins
Northouse, P. G. (2013). Leadership. Theory and Practice. Western Michigan University:
SAGE Publications Asia-Pacific Pty.
Northouse, P.G (2016). Leadership. Theory and Practice. Thousand Oaks, CA: SAGE
Publishers.
Onorato, M. (2013). Transformational leadership style in the educational sector: an
empirical study of corporate managers and educational leaders. Academy of
Educational Leadership Journal, Volume 17, Number 1, 2013 organizing schools (pp.
157‐179). Greenwich, CT: Information Age Publishing.
Pinto, D. A. (2014). Effective principal leadership behaviour: the student perspective.
electronic theses, projects, and dissertations. paper 11
Sithole, S. L., Higson-Smith, C., & Bless, C. (2013). Fundamentals of social research methods:
An African perspective. Lusaka: Juta and Company Ltd.
Tesfaye D., & Ayalew T. (2020). The Effects of Leadership Style on Students Performance.
Journal of Education and Practice www.iiste.org ISSN 2222-1735 Vol.11, No.34
Yukl, G. (2010). Leadership in organizations (7th ed.). University at Albany State. University
of New York. Upper Saddle River, NJ: Prentice Hall
52

Vol. 22, No. 7, pp. 52-69, July 2023
Received May 9, 2023; Revised Jul 6, 2023; Accepted Jul 7, 2023
Chinese University English-Major Students’

Attitude Toward Literature Circles
and Literature Reading
Lei Ma , Lilliati Ismail* and Norzihani Saharuddin

Faculty of Educational Studies, Universiti Putra Malaysia
Serdang, Selangor, Malaysia
Abstract. Of the four basic skills, namely, reading, is a fundamental tool

that supports the development of other skills. In Chinese tertiary
education, reading is a compulsory core subject for students. Although
the benefits of teaching English reading through literature circles have
been recognized in other countries for decades, there is still an insufficient
practice of this activity among English-major students in the Chinese
university context. This study aims to investigate the attitudes of Chinese
English-major students toward literature circles as a method of teaching
reading, and to determine the benefits of reading literature in language
learning. The study employs a one-shot case study involving only one
group exposed to the treatment, followed by a measure. Quantitative and
qualitative data were collected to determine the participants' attitudes
toward literature circles and their perception of the benefits of reading
literature. An intact class of 41 students participated in this
extracurricular reading activity through the convenience sampling
method. The result obtained from the eight closed-ended questions
indicated that the participants generally held favorable views toward
literature circles. The results from the focus group interviews confirmed
the benefits of reading literature in terms of cultural, linguistic, and
personal enrichment. The findings suggest that more longitudinal studies
involving comparison groups or qualitative studies are necessary to
better understand the benefits of literature circles.
Keywords: English-major students; literature circles; reading skills;

reading motivation; English learning
*
Corresponding author: Lilliati Ismail, lilliati@upm.edu.my
©Authors
53
1. Introduction
Over the past few decades, China has significantly strengthened its international
standing and expanded its global influence. In this era of challenges and
opportunities, proficiency in English is regarded as a crucial skill in China, both
at the individual and societal levels (You & Dörnyei, 2016). At the societal level,
English remains the preferred lingua franca for international communication,
with economic and political implications (Bachman, 2010). On an individual level,
English proficiency provides access to numerous economic, social, and
educational opportunities (Liu et al., 2023). Currently, the largest English-learning
population is in China, with an estimated number of over 300 million people
learning English as a foreign language. The State-enforced curriculum requires all
citizens to start learning English in third grade and English has become the most
popular major offered in over 1300 universities in China, reflecting its prominence
in foreign language education.
The Teaching Syllabus for English Majors (NACFLT, 2000) in China emphasizes
the significance of independent and creative thinking, as well as the development
of students' ideological, moral, cultural, and psychological qualities, alongside the
mastery of English language skills such as listening, speaking, reading, and
writing. Among these skills, reading is considered the fundamental tool for
learning, and the ability to read is viewed as the most stable and enduring
language skill that supports the development of other competencies, including
grammar, vocabulary, and writing (Koch, 1974). According to Grabe (2009), being
a proficient reader is essential for success in the modern world. It enables
individuals to pursue advanced studies, secure better job opportunities, access
information, and effectively communicate with others. English language
educators in China are committed to enhancing learners' reading experiences and
improving their English reading literacy through continuous reading practices
(Mei, 2018).
In Chinese tertiary education, reading is a compulsory subject for English-major

students during the first half of their four-year program. However, the current
situation is far from satisfactory. In the reading classroom, teachers primarily
focus on teaching language and cultural knowledge, while students are often
engaged in notetaking (Yi & Bai, 2022). The first prevailing problem faced by
Chinese English-major students is poor reading comprehension skills, which is
related to various aspects of reading, such as limited reading materials, restricted
reading time, lack of interest in reading, and insufficient autonomy in learning
(Li, 2022; Zhong & Suwanthep, 2022; Chen, 2022; Gu et al., 2017; ). Students'
motivation for reading is closely related to the issue of reading comprehension
ability. According to the investigation of reading among English majors by Quan
(2022), students were generally not interested in reading, which was consistent
with the questionnaire-based research by Li and Zhou (2017) on the demotivation
of Chinese university ELF learners, showing that most students had little or
moderate interest for learning English respectively.
According to Nuttall (1996), the lack of reading skills and ability would lower
students' willingness to read, which, in the long run, reduces their learning
54
interest. Meanwhile, motivation plays an important part in students’ reading

process, reading engagement, and reading achievement (Wolters et al., 2014). This
phenomenon of low motivation affects the teaching effects of language and the
English-major students’ interest in reading. Therefore, cultivating the reading
interest of English-major students and improving their reading skills are crucial
in foreign language teaching.
Driven by the need to produce competent language professionals capable of

meeting international challenges, there has been a re-evaluation of the close
relationship between reading literary texts and language learning by language
experts in China (Zhao, 2021; Jiang, 2019; Gu, 2016). Literature, with its rich
lexical, syntactic, and aesthetic discourse, provides an ideal source for classroom
activities that motivate learners to grasp a foreign language more effectively
(Bland, 2019). The benefits of literature in language learning have been
demonstrated by researchers (Hall, 2005; Krashen, 2004; Brumfit, 1981). This has
led to an increasing number of studies advocating for the integration of language
and literature in foreign language classes (e.g., Tucker, 2022; Compton et al., 2014).
Various reading activities have been proposed and implemented by researchers

to promote foreign language reading, such as sustained silent reading
(McCrachen, 1971), reading labs (Stoller, 1994), free reading time (Kirmizi, 2010),
and literature circles (Daniels, 2002). Manning and Manning (1984) confirmed
previous assumptions that peer interaction is the most successful recreational
reading model regarding reading attitudes and achievement. Elliot (1990) also
suggested that significant learning can occur effortlessly when learners
participate in literature reading groups facilitated by teachers.
Daniels (2002) suggested that literature circles are an excellent approach to

integrating the curriculum through authentic book reading and rich student
discussions. Ragland and Palace (2017) further confirmed that using literature
circles offers a better way for English teachers to create a student-centered class.
As an effective learning method for developing proficient readers, literature
circles can help students enhance their reading skills and foster their interest in
reading through collaborative interactions in a relaxed reading environment
(Bedel, 2016; Daniels, 2006; DaLie, 2001). Therefore, incorporating literature
circles into the curriculum of EFL/ESL learners is a better choice for language
teachers.
Although literature circles were initially designed for young readers in K-12
education, they have expanded beyond young readers in America to learners of
all age groups in different countries with different language backgrounds.
However, in the Chinese context, limited research has focused on either high
school students (e.g., Qin, 2022) or non-English-major students (e.g., Li et al., 2021;
Xu, 2021; Liang, 2014), and the benefits of literature circles for college English-
major students have not been adequately examined. Thus, the current study
hopes to encourage students to have a new experience of reading by participating
in collaborative activity, to provide inspirations for language teachers to design
more motivating reading activities and contribute to the knowledge concerning
55
literature circles. Considering the research findings on the benefits of literature

circles in fostering learner autonomy, increasing reading motivation, improving
reading comprehension, and providing opportunities for extensive reading and
collaborative discussions, the current study aimed to address the following
research questions:
Research Question 1: What is the attitude of Chinese English-major students

towards literature circles as a reading activity?
Research Question 2: How do English-major students perceive literature reading
in language learning?
2.1 Literature Circles
Literature circles, initiated by Harvey Daniels in 1994, have gained popularity in
America for their versatility in accommodating different ages, circumstances, and
needs (DaLie, 2001). This activity involves small student reading groups
following a specific framework that allows EFL or ESL students to engage in
authentic and meaningful discussions about literature in English (Furr, 2004).
Shelton-Strong (2012) specifically defined literature circles as small peer-led
discussion groups that read the same literary work and meet regularly to share
their understanding.
According to Daniels (2002), the success of literature circles relies on assigning

different roles or tasks to each participant in the reading group. The four basic
roles designed by Daniels are the connector, questioner, literary
luminary/passage master, and illustrator. However, depending on the type of
literature, additional optional roles such as summarizer, researcher, vocabulary
enricher/word wizard, and travel tracer/scene setter can be included. For higher-
level reading groups, Furr (2004) suggested six roles, adding a group discussion
leader and culture collector. These roles can vary and rotate for each new reading
cycle to ensure that learners focus on different aspects of reading (Shelton-Strong,
2012). By performing various tasks and fulfilling their assigned roles, learners
develop cooperation and participation skills within a collaborative learning
environment. DaLie (2001) emphasizes that the authenticity, empowerment, and
collaboration embedded in the activity promote open discussions and challenge-
taking among students.
2.2 The Effects of Literature Circles

Researchers have explored the advantages of using literature circles in EFL/ESL
learning from different perspectives. Among these benefits, the impact on reading
comprehension performance has been the most prominent and extensively
studied. Karatay (2017) conducted action research to examine the effectiveness of
literature circles in improving students' ability to analyze texts and their reading
motivation. The study concluded that literature circles contribute to students'
reading comprehension skills in areas such as text analysis, social learning
atmosphere, critical reading ability, independent reading motivation, speaking
confidence, speaking ability, and tolerance towards cultural and socio-economic
differences. Xu (2021) confirmed Karatay's findings in an academic classroom
56
setting, where 85% of Business English majoring university students perceived

literature circles as effective in improving their Business English reading abilities.
Similarly, Chou (2022) and Huljanah et al. (2022) investigated the effects of
literature circles on reading comprehension through experimental and control
groups, and both studies demonstrated the effectiveness of literature circles in
promoting reading comprehension, reading motivation, and reading strategy
practice.
The effect of literature circles on social skills has also been a focus of research.
Abdelrasoul (2014) implemented a proposed program based on literature circles
to develop critical reading skills and social skills among Egyptian EFL second-
year preparatory school students and found that literature circles increased
students' critical reading skills and social skills effectively. Venegas (2019)
highlighted the benefits of literature circles through a case study of Grace, a 10-
year-old African American girl in fifth grade. The study showed that Grace's
participation in literature circles enhanced both her self-management skills
(intrapersonal) and social awareness (interpersonal). Dogan and Kaya-Tosun
(2020) conducted an experimental study on 74 fourth-grade students in a public
elementary school, concluding that the literature circles method positively
influenced the social skill development of students in the experimental group.
Literature circles have also been found to improve students' intercultural
communicative competence and promote open-mindedness and tolerance
towards cultural differences (Imamyartha et al., 2020).
In addition to the instrumental benefits, research has shown that students develop
a high level of motivation toward literary reading and become more interested
and engaged in literature when participating in literature circles (Bouazid, 2022;
Mabe, 2022; Hassan, 2018 ). Reading motivation, according to Stutz et al. (2016)
and Schiefele et al. (2012), is positively associated with reading volume and
reading comprehension.
Recognizing the various benefits of literature circles, some researchers (e.g.,

Nurhadi & Anggrarini, 2022; Imamyartha et al., 2020) have incorporated literature
circles as instructional scaffolding in extensive reading programs, demonstrating
that shared reading experiences can provide students with new reading
experiences and enrich their intercultural knowledge and understanding.
3. Method
The current study is designed as a one-shot case study involving only one group
through the convenience sampling method, exposed to the treatment followed by
measures. Both quantitative and qualitative data were collected and analyzed to
identify the attitude of Chinese English major students toward literature circles
and determine the benefits of reading literature in EFL/ESL learning. The
research was conducted over four weeks, and feedback from the participants was
collected through questionnaires and focus group interviews for analysis.
57
3.1 Participants
The study was conducted in a private university located in the Midwest of China.
According to the National Syllabus of English Major Education in Universities
(2000), English-major students in China are required to read 20 books of liberal
arts written in English during their first two years to acquire 2 credits. Typically,
the reading task in this university is conducted independently by the students,
but each class is assigned a tutor to provide guidance. With permission from the
Dean of the English Department, the researcher contacted the tutor of a class that
agreed to participate in the research. The selected class consisted of 41 second-
year undergraduate students majoring in English, including 5 males and 36
females. All participants were preparing to take the national proficiency test for
English majors (TEM-4) the following April, which is a criterion-referenced
English language test specifically designed for English major sophomores. The
TEM-4 assesses listening, reading, and writing skills. Therefore, the participants
had a strong motivation to improve their reading ability to pass the exam and earn
the 2 credits from after-class reading. The tutor was responsible for evaluating the
participants' reading performance and teaching one core subject of the class.
Before the study, meetings were held with the tutor to discuss the intervention
procedures, and the participants were informed about the purpose, procedures,
reporting, dissemination, benefits, and risks of participating in the reading
activity. Confidentiality and anonymity were ensured to make the participants
feel secure and respected.
3.2 Procedures
3.2.1 Grouping
Before the reading program, preliminary instructions were provided to the
participants regarding the research purpose, functions of different roles, and
procedures. To implement the literature circles, the class was divided into six
reading groups. The students were encouraged to form small temporary groups
based on their own choice, ensuring that an outgoing and active student served
as the group leader. Since there were 41 students, six groups were formed, with
five groups consisting of seven members and one group consisting of six
members. In the groups with seven members, two participants assumed the role
of the passage person. Each group was provided with a role sheet (Table 1), which
described the various roles and helped participants understand their respective
responsibilities.
Table 1: Role sheet (Furr, 2004)
Roles Job description
Group
Facilitating the whole discussion
discussion
Opening the discussion with open-ended questions
leader
Providing a complete summary of the plot at the
Summarizer
beginning or end of the discussion
Finding connections between the text and the real

Connector
world, i.e., between the thoughts, feelings, or actions of
58
characters in the story and stories of family members,

friends, or classmates
Focusing on at least five single words or short phrases

Word master
to share, explain and elaborate
Passage Making a careful reading of the text and looking for

person well-written passages to share the understanding
Looking for the cultural and historical background of
Culture
the story and noting differences and similarities
collector
between the culture in the text and their own culture
3.2.2 Selecting Reading Materials

Each group was given the freedom to select their reading materials, with the
restriction that the texts had to be English literature or English translations of
literature. During the first week, the students finalized their chosen texts. The
selected materials primarily consisted of short stories for the first two weeks a
lengthy fiction for the last two weeks. Table 2 shows the reading materials selected
by students:
Table 2: Reading materials

Group Week 1 Week 2 Week 3 & Week 4
1 Araby Ambush The Little Prince
The Story of an The Call of the
2 A Rose for Emily
Hour Wild
The Story of an
3 Araby The Great Gatsby
Hour
A Woman on a The Colour
4 The Little Prince
Roof Purple
The Rocking- The Secret
5 The Open Boat
Horse Winner Garden
A clean, well- The Old Man and
6 Recitatif
lighted place the Sea
3.2.3 Reading Tasks and Regular Discussion

After selecting their reading materials, the participants were assigned reading
tasks and asked to prepare slide presentations based on their respective roles
within one week. Every Tuesday, following the regular class hour, a 90-minute
reading-sharing session was held. Each group took turns sharing their readings,
with the group discussion leader facilitating the discussion. Once the first round
was completed, the second round began immediately, and participants were
required to assume different roles from their previous round. According to Furr
(2004), literature circles promote the integration of reading, speaking, listening,
and writing skills through regular discussions. The intervention spanned one
month, allowing each participant to share four times in various roles. Table 3
provides the guideline for administering literature circles sharing sessions.
59
Table 3. Guideline for Sharing Session
Time: 2 pm – 3.30 pm Tuesday
Location: Multimedia classroom
Duration: 4 weeks
Preparing for sharing session
 Students read the text before the sharing session.

 Students take notes while reading.
 Group members meet and discuss according to their
specific roles before the sharing session.
 Groups prepare slides for the sharing session.
Conducting the sharing session
 The group leader opens the discussion by introducing and

posing related open-ended questions.
 The summarizer gives a summary of the story.
 The connecter makes a connection between the literary
work and the real world.
 The word master teaches words, phrases, and their usage.
 The passage person analyses the features of one or two
well-written paragraphs.
 The culture collector presents the cultural background of
the work.
Reflection after the sharing session
 The tutor comments on the presentation of each group or
specific group member based on the following:
✓ Did the group members actively participate in the
discussion and sharing session?
✓ Did the group demonstrate a deep understanding of
the text?
✓ Did the group leader ask thoughtful open-ended
questions?
✓ Did each group member effectively fulfill their
assigned roles?
✓ Did each group member use appropriate language
and communication skills?
✓ Did each group member demonstrate a collaborative
and supportive manner within their groups?
✓ Did each group member demonstrate a love for
reading each time?
4. Data Collection and Analysis

Once the intervention concluded, data was collected in two forms by the tutor. A
questionnaire (see Appendix 1) consisting of 8 close-ended questions adapted
from Smith and Feng's (2018) study, was distributed in online form to investigate
the participants' attitudes toward literature circles. The questionnaire was
submitted by 40 out of the 41 students, for 1 student was absent for the last reading
session. Descriptive statistics were used to analyze the results, providing insights
60
to answer research question one.
In addition to the quantitative data, qualitative data were collected to gain deeper
insights into the benefits students derived from the activity. The day after the
survey, a focus group interview facilitated by the tutor, was conducted with 5
active participants in the class through purposeful sampling, with the assumption
that they can provide rich information concerning the benefits of literature
reading. Focus group interviews are particularly useful in case studies involving
a few selected individuals, as the data generated is socially constructed within the
group interaction (Merriam, 2014). Given that the participants read different
books and may interpret them differently, the focus group interview provided an
opportunity for them to share their perspectives. To ensure that all relevant issues
were covered, an interview schedule was prepared by the researcher as a guide
(see Appendix 2). The interview took place in a relaxed café environment to
encourage open conversation. It centered around several general questions, such
as "Do you like reading literature? Why or why not?" "What abilities have you
improved through reading literature, such as language competency, cultural
awareness, personal development, and others?" "Do you find literature circles
helpful in learning English compared to traditional independent reading?" and
"Among the roles you have performed in literature circles, which one is your
favorite and why?" All five participants were cooperative and provided
informative responses for the study.
The interview was recorded to ensure accuracy during transcription. An

interview protocol was also used to capture keywords, non-verbal expressions,
and reflections of the interviewer. Following the interview, an edited transcription
was created as a written record of the data. Content analysis was then employed
to analyze the students' interview responses.
5. Results and Discussion

Regarding the attitudes of students toward adopting literature circles in English
learning, the quantitative results from the survey were analyzed using descriptive
statistics in SPSS 28.0 and then interpreted according to the criteria suggested by
Ketkham (2004). Table 4 and Table 5 present the criteria and the results of the
Likert scale survey, respectively.
Table 4: Criteria of Interpreting the Questionnaire (Ketkham, 2004, p. 74)

Mean score Degree of Agreement
1.00 – 1.80 Strongly Disagree
1.81 – 2.60 Disagree
2.61 – 3.40 Undecided
3.41 – 4.20 Agree
4.21 - 5.00 Strongly agree
61
Table 5: Results of the Attitude Toward Literature Circles

Interpretati
Items Statements Mean
on
Reading is fun when I take Strongly
1 4.30
part in literature circles. agree
I prefer literature circles to
Strongly
2 traditional reading 4.22
agree
instruction.
I would like to continue
3 participating in literature 4.20 Agree
circles.
Literature circles help me
4 4.18 Agree
enjoy reading literature more.
Participating in literature
Strongly
5 circles inspires me to read 4.25
agree
more books.
I like to try different roles to
6 4.13 Agree
improve my different skills.
Literature circle meetings help
Strongly
7 me understand the books 4.35
agree
better.
Participating in literature
Strongly
8 circles motivates me to dig 4.30
agree
deeper into the text.
Strongly
Total 4.24
agree
In response to the statements of the questionnaire concerning the attitude toward

adopting literature circles, the summary of the statistics showed that students
generally hold a strong and positive attitude toward this activity (M = 4.24). Of
the 8 statements, item 7 scored the highest (M = 4.35), followed by item 1 and item
8 (M = 4.30), indicating that the respondents highly agree that literature circles are
useful, fun, and motivating in helping them understand the books better. There
were no negative opinions toward any of the statements. The tables showed that
the respondents agree or strongly agree with the opinions that reading is
interesting and enjoyable (items 1, 4), inspiring (item 5), helpful (items 6, 7), and
motivating (item 8). Most of them prefer literature circles to traditional instruction
(item 2) and would like to continue participating in literature circles (item 3).
According to Su and Wu (2016), literature circles can provide students with a safe
and low-risk environment to experience dynamic interactions among readers and
texts and can nurture the pleasure of reading literature and literacy skills
simultaneously. The research results are consistent with the conclusion by Smith
and Feng (2018) that literature circles could encourage students to read more and
actively participate in group discussions with their peers. The results are also
aligned with the findings by Carrison and Ernest-Slavit (2005) that students can
be actively engaged in using authentic language as literature circle members for
real purposes. The survey results have further strengthened the findings of
Hassan (2018) from teachers' perspective that students can acquire vocabulary,
62
develop key reading skills, and literature circles are considered fun and make
students interested in literature. The results further support the findings of
Kaowiwattanakul (2020) in the context of Thailand that students hold highly
positive attitudes towards the use of literature circles in improving their speaking
skills.
The responses from the participants after the interview provided insights into the
benefits of reading literature on language competence, cultural awareness,
personal development, as well as other abilities. The interview responses were
analyzed using qualitative content analysis techniques. A total of 13 pieces of
content were collected from the interviewees and subjected to content analysis.
The content was segmented into 34 text units and entered into Excel for line-by-
line coding. They were then grouped into code categories, as is common in content
analysis, to classify the words of texts (Weber, 1990). The thematic categories were
based on pre-established models by Carter and Long (1996) for literature reading,
which include the cultural model, the linguistic model, and the personal model.
Any references related to interpreting meanings were categorized as cultural, and
texts involving decoding linguistic aspects were marked as language-based
themes. References concerning feelings and reading interest were marked as
personal responses.
Table 6. Categorization of Codes concerning the benefits of literature reading
Categories Cultural Linguistic Personal

Expressing (2) Appreciate (2)
Background (7) Share ideas (2) Interesting (1)
Customs (2) Discussing (1) Leadership (1)
Codes
Horizons (2) Analyzing (2) Empathy (3)
Author (2) Writing skills (3) Experience (1)
Vocabulary (1) World (2)
Number 13 (38%) 11 (32%) 10 (30%)
According to Fraenkel et al. (2012), a common way to interpret content analysis

data is to use frequencies and percentages to determine the occurrence of specific
categories relative to the total occurrences. As shown in Table 6, 13 (38%) codes
are related to cultural knowledge, 11 (32%) to linguistic skills, and 10 (30%) to
personal development, respectively. Among the three models of reading
literature, the cultural model is valued the most (38%), indicating that students
read literature mainly to expand their knowledge of cultures outside of their own.
This is followed by a focus on improving language skills in reading, writing, and
speaking. Some participants also mentioned personal development aspects such
as perception, empathy, aesthetics, as well as cognitive and evaluative abilities.
The results from the interview discussions confirmed previous findings by

researchers (e.g., Ashrafuzzaman et al., 2021; Baytar & Timuçin, 2021; Hasan &
Hasan, 2019; Aghagolzadeh & Tajabadi, 2012; Yeasmin et al., 2011) that literature
helps learners develop a wide range of skills, including language competency
through exposure to authentic language, familiarity with universal themes related
63
to human psychology, increased cultural understanding, enhanced reading

motivation, analytical thinking skills, and enriched reading experiences.
According to Ashrafuzzaman et al. (2021), literature helps build interpretive
abilities by exposing students to a wide range of vocabulary, grammar, and
pronunciation, thus being a valuable tool for language learning. Baytar and
Timucin’s (2021) research with university students also confirmed the benefits of
literature in improving students' linguistic, cultural, and personal abilities. The
current research and previous studies also reinforced the benefits of integrating
literature into language learning proposed by Collie and Slater (1987) that
literature can be used in a language class as valuable authentic material, cultural
enrichment, language enrichment, and personal involvement.
As a highly developed form of collaborative learning guided by reader response

principles (Daniels, 2002), this shared reading by literature circles can provide
students with a strong reader-text relationship, and an improved classroom
learning environment more conducive to language learning (Lin, 2004). It can also
provide students with a multidimensional learning platform to naturally develop
learners’ language skills, contextualized discussions and interpretations (Shelton-
Strong, 2012). Therefore, this reading activity fits Chinese English-major students'
learning to inspire their interest in L2 learning and engage them more in linguistic,
cultural, and personal enrichment.
6. Conclusions and Implications

The study aimed to investigate the attitudes of English-major students toward
literature circles and the benefits of reading literature in a Chinese non-key
university. While literature circles had not been practiced in this university before,
the findings from the research were quite promising.
Firstly, most participants in the survey expressed high interest in participating in

literature circles and agreed that they had benefited from the activity.
Additionally, most participants indicated their willingness to continue reading
literary texts in literature circles in the future and expressed motivation to read
more extensively and deeply.
Secondly, the study confirmed the positive impact of incorporating literature in a

Chinese university context. Discussions from the interviews revealed that
participants recognized the vital role of literature in language learning,
acknowledging its ability to improve their language competence, enhance cultural
awareness, and develop other abilities such as perception, leadership, cognition,
and critical thinking.
However, it is important to note that not all students in the class enjoyed or were
highly motivated to actively engage in the literature circles, as indicated by the
survey results. From these findings, several implications can be drawn. Firstly, it
is crucial to motivate all participants to transition from negative readers to
positive readers to ensure smooth and successful group work. Secondly, as
literature reading and the subsequent literature circle discussions can be time-
consuming, students may need to allocate sufficient time after regular classes to
64
fulfill their responsibilities. Teachers or researchers should consider the length

and difficulty level of the reading materials and allocate suitable periods for the
sharing sessions. Lastly, in this study, the reading groups were formed based on
students' friendships, which may have resulted in homogeneous groups (Abrami
et al., 1995). According to Webb (1989), cooperative learning groups perform best
when they contain members of different abilities, therefore, the grouping
procedure was also worth consideration in future studies.
Meanwhile, as a small-scale intervention study conducted to identify the attitude

of Chinese English-major students toward literature circles and literature reading,
this study can prompt the recommendations for ESL/EFL teachers to integrate
this collaborative reading activity into language teaching, thus enhancing the
reading motivations and learning efficiency. Several limitations in this study are
worthy to point out. First, the participants in this study are all English-major
students with intermediate English levels. A more comprehensive study that
involves non-English-major students at the tertiary level should be replicated to
provide a better insight into the effectiveness of literature circles in improving
tertiary-level learners’ reading ability. Secondly, since the present study was
conducted for only 4 weeks, it is recommended that further longitudinal studies
that may last for one academic year or longer be undertaken to better understand
the benefits of literature circles. Thirdly, for this study, only literary texts were
adopted as reading materials in literature circles, and the other genres were not
used. It is recommended that different types of materials be integrated into
literature circles in future studies to confirm the effectiveness of literature circles.
Lastly, with the wide use of modern technology, literature circles can be
implemented in virtual forms conveniently. Therefore, more future studies can be
conducted by integrating the convenience of modern technology to provide a
different and easy avenue for language learning.
Acknowledgments
We would like to express our gratitude to everyone who helped during this
review and improved the quality of our final manuscript.
7. References
Abdelrasoul, M. M. I. (2014). Using reading circles strategy for developing preparatory students'
critical reading skills and social skills. Online Submission.
https://doi.org/10.21608/mjat.2014.103616
Abrami, P. C., Chambers, B., Poulsen, C., de Simone, C., D’Apollonia, S., & Howden, J.
(1995). Classroom Connections: Understanding and Using Cooperative Learning.
Harcourt Brace.
Aghagolzadeh, F., & Tajabadi, F. (2012). A debate on literature as a teaching material in
FLT. Journal of Language Teaching and Research, 3(1), 205–210.
https://doi.org/10.4304/jltr.3.1.205-210
Ashrafuzzaman, M., Ahmed, I., & Begum, M. (2021). Learning English language through
literature: insights from a survey at university level in Bangladesh. Journal of
Language and Linguistic Studies, 17(2). https://doi.org/10.17263/jlls.904150
Bachman, L. F. (2010). Foreword. In English Language Assessment and the Chinese Learner
(p. x). Routledge.
65
Baytar, İ., & Timuçin, M. (2021). Harnessing literature in EFL teaching: A case in point.
Journal of Language and Linguistic Studies, 17, 327–345.
https://doi.org/10.17263/jlls.903426
Bedel, O. (2016). Collaborative learning through literature circles in EFL. European Journal
of Language and Literature, 6(1), 96–99. https://doi.org/10.26417/ejls.v6i1.p96-99
Bland, J. (2019). Learning through literature. In S. Garton & F. Copland (Eds.), The
Routledge Handbook of Teaching English to Young Learners (pp. 269–287). Routledge.
Bouazid, T. (2022). Fostering motivation and adopting learner-centered teaching strategy
through EFL literature circles. FKIP Universitas Lambung Mangkurat Banjarmasin,
5(1), 11–28. https://doi.org/10.20527/jetall.v5i1.10530
Brumfit, C. (1981). Reading skills and the study of literature in a foreign language. System,
9(1), 243–248. https://doi.org/10.1016/0346-251x(81)90047-6
Carrison, C., & Ernst-Slavit, G. (2005). From silence to a whisper to active participation:
Using literature circles with ELL students. Reading Horizons, 46(2), 93–113.
Carter, R., & Long, M. N. (1996). Teaching Literature. Longman.
Chen, T. (2022). An investigation and analysis of college English majors’ autonomous
learning ability in ubiquitous learning environment. Journal of Environmental and
Public Health, 2022, 1–10. https://doi.org/10.1155/2022/9103148
Chou, M. H. (2022). Using literature circles to teach graded readers in English: an
investigation into reading performance and strategy use. Innovation in Language
Learning and Teaching, 16(2), 144–163.
https://doi.org/10.1080/17501229.2021.1885412
Collie, Joanne., & Slater, S. (1987). Literature in the language classroom: a resource book of ideas
and activities. Cambridge University Press.
Compton, D. L., Miller, A. C., Elleman, A. M., & Steacy, L. M. (2014). Have we forsaken
reading theory in the name of “quick fix” interventions for children with reading
disability? Scientific Studies of Reading, 18(1), 55–73.
https://doi.org/10.1080/10888438.2013.836200
DaLie, S. O. (2001). Students becoming real readers: literature circles in high school
English classes. Teaching Reading in High School English Classes, 84–100.
Daniels, H. (2002). Literature circles: voice and choice in book clubs and reading groups (2nd
ed.). Stenhouse Publishers.
Daniels, H. (2006). What’s the next big thing with literature circles? Voices from the Middle,
13(4), 10–15.
Dogan, B., & Kaya-Tosun, D. (2020). An effective method in improving social skills:
literature circles. International Journal of Educational Methodology, 6(1), 199–206.
https://doi.org/10.12973/ijem.6.1.199
Elliott, R. (1990). Encouraging reader-response to literature in ESL situations. ELT Journal,
44(3), 191–198. https://doi.org/10.1093/elt/44.3.191
Fraenkel, J., Wallen, N., & Hyun, H. (2012). How to Design and Evaluate Research in
Education (8th ed.). McGraw-Hill.
Furr, M. (2004). Why and how to use EFL literature circles.
Http://Www.Eflliteraturecircles.Com/Howandwhylit.Pdf.
Grabe, W. (2009). Reading in a Second Language Moving from Theory to Practice. Cambridge
University Press.
Gu, X., Hong, Y., & Yang, R. (2017). “大量阅读输入”教学模式下英语专业学生的阅读态度
——基于学生学习日志的历时质性研究 [English majors’ reading attitudes in the
extensive reading input teaching model practice: a longitudinal qualitative study
based on students’ learning journals]. Foreign Languages and Their Teaching, 3(294),
68–79. https://doi.10.13458/j.cnki.flatt.004361
Gu, Y. (2016). 回归经典阅读：英语专业的人文性与基于阅读经验的文学教育 [Back to the
classics: the English major and experience-based teaching of literature]. FLLTP, 2,
42–47.
66
Hall, G. (2005). Literature in Language Education. Palgrave Macmillan.

Hasan, A. M., & Hasan, Z. F. (2019). Students’ perception towards literature integration
in the English language departments at Duhok and Zakho Universities. Advances
in Language and Literary Studies, 10(4), 130–152.
https://doi.org/10.7575/aiac.alls.v.10n.4p.130
Hassan, H. (2018). Instructors’ attitude on the use of “literature circles” in teaching
literature. Polytechnic Journal, 8(2), 142–158.
https://doi.org/10.25156/ptj.2018.8.2.140
Huljanah, M., Akbarjono，Ali, & Martina, F. (2022). The effect of literature circle strategy
on students’ English reading comprehension. Jurnal Pendidikan Tematik, 1(3), 77–
86.
Imamyartha, D., Andayani, R., A’Yunin, A., Puspa, A., Hudori, R. F. A., Fardhani, A. E.,
& Dafik. (2020). Engaging EFL readers in literature circles to escalate intercultural
communicative competence. IOP Conference Series: Earth and Environmental
Science, 485(1), 0–11. https://doi.org/10.1088/1755-1315/485/1/012090
Jiang, Hongxin. (2019). 推动构建中国特色英语类本科专业人才培养体系 [Promoting the
construction of the cultivation system for undergraduate English majors with
Chinese Characteristics]. Foreign Language World, 5(194), 1–6.
Kaowiwattanakul, S. (2020). Using literature circles to promote the English-speaking
skills of engineering students in English for specific purposes classrooms. LEARN
Journal: Language Education and Acquisition Research Network, 13(2), 414-425.
Karatay, H. (2017). The effect of literature circles on text analysis and reading desire.
International Journal of Higher Education, 6(5), 65–75.
https://doi.org/10.5430/ijhe.v6n5p65
Ketkham, K. (2004). Students’ attitudes toward the computer-based reading program in
the developmental reading course: The impact on their reading performances.
Thai Journal of Development Administration, 44(2), 67–94.
Kirmizi, F. S. (2010). Relationship between reading comprehension strategy use and daily
free reading time. Procedia - Social and Behavioral Sciences, 2(2), 4752–4756.
https://doi.org/10.1016/j.sbspro.2010.03.763
Koch, R. E. (1974). Relationships Between Reading Interests and Reading Comprehension Among
Fourth Grade and Sixth-Grade Students. University of Illinois at Urbana-
Champaign.
Krashen, S. D. (2004). The Power of Reading: Insights from The Research (2nd ed.). Libraries.
Laličić, A., & Dubravac, V. (2021). The role of reading in English language classrooms.
MAP Social Sciences, 1(1), 27–36. https://doi.org/10.53880/2744-2454.2021.1.1.23
Li, R. (2022). Foreign language reading anxiety and its correlates: a meta-analysis. Reading
and Writing, 35(4), 995–1018. https://doi.org/10.1007/s11145-021-10213-x
Li, Y., Chen, K., Su, Y., & Yue, X. (2021). Do social regulation strategies predict learning
engagement and learning outcomes? A study of English language learners in
wiki-supported literature circles activities. Educational Technology Research and
Development, 69(2), 917–943. https://doi.org/10.1007/s11423-020-09934-7
Li, C., & Zhou, T. (2017). A Questionnaire-based Study on Chinese University Students’
Demotivation to Learn English. English Language Teaching, 10(3), 128–135.
https://doi.org/10.5539/elt.v10n3p128
Liang, Y. (2014). Application of “reading circles” in college English teaching. Conference
on Information in Education, Management and Business, 409–411.
https://doi.org/10.2991/iemb-14.2014.122
Lin, C.-H. (2004). Literature circles. Teacher Librarian, 31(3), 23–25.
Liu, Y., Nam, B. H., & Yang, Y. (2023). Revisiting symbolic power and elite language
education in China: a critical narrative ethnography of the English education
major at a top language university in Shanghai. Educational Review, 1-27.
https://doi.org/10.1080/00131911.2023.2184774
67
Mabe, K. (2022). Students’ perceptions of reading circles in the EFL classroom. Journal of
Multilingual Pedagogy and Practice, 2, 129–140.
Manning, G. L., & Manning, M. (1984). What models of recreational reading make a
difference? Reading World, 23(4), 375–380.
https://doi.org/10.1080/19388078409557787
McCracken, R. A. (1971). Initiating Sustained Silent Reading. Journal of Reading, 14(8), 521–
583. http://www.jstor.org/stable/40009700
Mei, D. (2018). 2018 TESOL Shanghai Proclamation. 2018 TESOL China.
https://www.tesolsh.com/news_content.asp?id=11229
Merriam, S. B. (2014). Qualitative Research: A Guide to Design and Implementation. Wiley.
NACFLT. (2000). The National College English Teaching Syllabus for English Majors.
Shanghai Foreign Language Education Press.
Nuttall, C. (1996). Teaching Reading Skills in a Foreign Language. Macmillan Heinemann.
Nurhadi, K., & Anggrarini, N. (2022). Adopting literature circle to guide work
undergraduate students in extensive reading program: lessons learned from
initial teacher education. JELLT, VI (1), 40–50.
https://doi.org/10.36597/jellt.v6i1.12036
Qin, Z. (2022). The effects of engaging students in literature circles on reading
comprehension: an English reading program in a senior high school. ICELA, 637,
604–607. https://doi.org/10.2991/assehr.k.220131.109
Quan, L. (2022). An Analysis of the Influence of Different Ways of Context Construction
on the Teaching Effect of Intensive Reading and Extensive Reading for English
Majors. Advances in Multimedia, 2022, 1–12.
https://doi.org/10.1155/2022/6833203
Ragland, J., & Palace, C. (2017). Literature circles for adolescent developmental readers.
The English Journal, 106(6), 35–40.
Schiefele, U., Schaffner, E., Möller, J., Wigfield, A., Nolen, S., & Baker, L. (2012).
Dimensions of reading motivation and their relation to reading behaviour and
competence. Reading Research Quarterly, 47(4), 427–463.
https://doi.org/10.1002/RRQ.030
Shelton-Strong, S. J. (2012). Literature circles in ELT. ELT Journal, 66(2), 214–223.
https://doi.org/10.1093/elt/ccr049
Smith, A., & Feng, J. (2018). Literature circle and gifted students: boosting reading
motivation and performance. Online Submission, 1–36.
Stoller, F. L. (1994). Developing a focused reading lab for L2 students. Reading in a Foreign
Language, 10(2), 33–53.
Stutz, F., Schaffner, E., & Schiefele, U. (2016). Relations among reading motivation,
reading amount, and reading comprehension in the early elementary grades.
Learning and Individual Differences, 45, 101–113.
https://doi.org/10.1016/j.lindif.2015.11.022
Su, Y.-C., & Wu, K. H. (2016). How Literature Circles Support EFL College Students’
Literary and Literacy Learning in a Children’s and Adolescent Literature Course.
International Journal for Cross-Disciplinary Subjects in Education, 7(3), 2874-2879.
https://doi.org/10.20533/licej.2040.2589.2016.0311
Tucker, A. (2022). Reading texts, reading people: cognitive literary science and pedagogy.
Arts and Humanities in Higher Education, 21(1), 94–110.
https://doi.org/10.1177/14740222211013757
Venegas, E. M. (2019). “We listened to each other”: socioemotional growth in literature
circles. Reading Teacher, 73(2), 149–159. https://doi.org/10.1002/trtr.1822
Webb, N. M. (1989). Peer interaction and learning in small groups. International Journal of
Educational Research, 13, 21–39. https://doi.org/10.1016/0883-0355(89)90014-1
Weber, R. P. (1990). Basic Content Analysis (2nd ed.). Sage.
68
Wolters, C. A., Denton, C. A., York, M. J., & Francis, D. J. (2014). Adolescents’ motivation
for reading: Group differences and relation to standardized achievement.
Reading and Writing, 27(3), 503–533. https://doi.org/10.1007/s11145-013-9454-
3
Xu, Q. (2021). Incorporating reading circles into a task-based EAP reading scheme. ELT
Journal, 75(3), 341–350. https://doi.org/10.1093/elt/ccab012
Yeasmin, N., Azad, Md. A. K., & Ferdoush, J. (2011). Teaching language through
literature: designing appropriate classroom activities. ASA University Review,
5(2), 283–297.
Yi, X., & Bai, M. (2022). A study on teaching college English reading from the perspective
of invitational rhetoric in China. International Journal of English Linguistics, 12(2),
70. https://doi.org/10.5539/ijel.v12n2p70
You, C., & Dörnyei, Z. (2016). Language learning motivation in China: results of a large-
scale stratified survey. Applied Linguistics, 37(4), 495–519.
https://doi.org/10.1093/applin/amu046
Zhao, W. (2021). 阅读经典：何谓？何以？如何？[Reading classics: what， why， how].
Zhejiang Social Sciences, 12, 152–155. https://doi:10.14167/j.zjss.2021.12.018
Zhong, Y., & Suwanthep, J. (2022). Effects of a Lexical Approach to Chinese English Major
Students’ Reading Comprehension Ability. Journal: Language Education and
Acquisition Research Network, 15(2), 468–497.
Appendix 1: Attitude for Literature Circles Questionnaire

1. Strongly disagree 2. Disagree 3. Neither agree nor disagree
4. Agree 5. Strongly agree
Statements 1 2 3 4 5
Reading is fun when I take part in
literature circles.
I prefer literature circles to traditional
reading instruction.
I would like to continue participating in
literature circles.
Literature circles help me enjoy reading
literature more.
Participating in literature circles
inspires me to read more books.
I like to try different roles to improve
my different skills.
Literature circle meetings help me
understand the books better.
Participating in literature circles
motivates me to try to dig more deeply
into the text.
69
Appendix 2: Interview Protocol
Topic：How is your attitude toward literature reading？

Part 1: Basic Information
Time of Interview:
Date:
Place:
Interviewer:
Interviewees:
Part 2: Introduction
Firstly, I would like to express my sincere thanks for meeting me today.The main purpose of this
interview is to get an understanding of your attitude toward literature reading. The interview would
take you around one hour. Thank you for your time again. Also, if you don’t mind, I would like to
tape record our conversation so as not to miss any important information. Although I would take
notes during our conversation, sometimes I may not be fast enough to catch up with you. So, will you
agree to be that? If so, be sure to speak up anything you can share with me. Thank you very much for
the conversation. For your information, all the data will be kept confidential, secure and stored safely,
and I will make sure not to disclose your name and information at any time. All the Meanwhile,
helping with this research interview is completely voluntary, so you have the right to end the talk at
any time and you can always have the right to decide what to share or not.
Part 3: Opening remarks
In the past weeks, we have conducted literature circles and each group has performed quite well.
Reading literature in literature circles is a new experience for you and hope you have benefited from
this activity.
Part 4: Content Questions

Q1: Do you like reading literature? Why or why not?
Notes:
Probes: Please elaborate on the reasons you mentioned.
Q2: What abilities have you improved through reading literature, such as language competency,
cultural awareness, personal development, and others?
Notes:
Probes: Please tell me more about it.
Q3: Do you find literature circles helpful in learning English compared to traditional independent
reading?
Notes:
Probes: What’s next?
Q4: Among the roles you have performed in literature circles, which one is your favorite and why?
Notes:
Probes: Please talk more about this?
Part 5：Closing Instruction

Thank you so much. Is there anything you still want to add? Have a good day.
70

Vol. 22, No. 7, pp. 70-86, July 2023
Performance Analysis Towards Excellent Schools
Abednego Abednego*
Nonformal Education Study Program, Faculty of Teacher
Training and Education, Pattimura University, Indonesia
Patrisius Rahabav
Educational Administration, Faculty of Teacher
Training and Education, Pattimura University, Indonesia
John Rafafy Batlolona

Department of Physics Education, Faculty of Teacher
Training and Education, Pattimura University, Ambon, Indonesia
Abstract. The elementary level is the foundation for children to continue

their education to a higher level. Therefore, governance is essential. This
study measures school performance and factors inhibiting elementary
school performance in Ambon City. The results showed that 1) the
dimension of performance action; sub-dimensions that have been
achieved by the school, namely the school's physical appearance, staff
cooperation, and excellent service. The sub-dimensions that have not
been achieved maximally, namely the championship rankings that have
not been achieved much, especially in accredited schools B and C. 2) The
dimension of performance achievement; sub-dimensions that the school
has achieved, namely: all schools have been accredited, the average
score on the education report is 7; the average student attendance is
good; the Drop-Out presentation is minimal; the average repeating
classes and repeating subjects is less, and students show civilized
behavior or good character. The sub-dimension that has yet to be
achieved is the ranking of championships or non-academic activities. 3)
The dimension of performance outcomes: The sub-dimension achieved,
namely the attendance of graduates who continue their studies to junior
high schools, is 100%. The sub-dimensions that have yet to be achieved,
namely the level of customer satisfaction and community support for
school programs, still need improvement. Therefore, it is concluded that
several dimensions still need to be achieved in heading toward a
superior school. It is a challenge for the school to improve and
repackage the program so that it runs well.
*
Corresponding author: Abednego Abednego, abednegodr01@gmail.com
©Authors
71
Keywords: school performance; factors inhibiting performance;

governance; excellent schools; primary schools
1. Introduction
Good school governance is the dream of all parties. Good school governance has
started since the enactment of the decentralization of education. At the school
level, the decentralization of education came to be known as School-Based
Management (SBM). SBM is the delegation of some of the administered
education authority to schools in making decisions, implementing, and
evaluating school programs in partnership with stakeholders to support
improving school performance (Guha, 2021). In that spirit, each school competes
creatively in managing the school to achieve good performance towards a
superior school (Qian & Kong, 2018; Parker et al., 2022).
The elementary school level is the foundation for children to continue their
education to a higher level, namely junior high school. Thus, elementary school
governance is essential. This statistic is important because good school
governance will increase good school performance (Leasa et al., 2023). Various
empirical studies have found that the performance of schools in Indonesia could
be better. One of the fundamental problems is that the competence of school
principals is inadequate due to: 1) recruitment to schools is not based on a merit
system approach but a spoil system; 2) school principals are not well prepared
with the prerequisite training as school principals; 3) training in the position of
the school principal is not carried out. The implication could be better
governance in various policies and implementation of school programs
(Rahabav & Souisa, 2021).
School performance can be seen from the average student learning outcomes in
each subject and accumulates in the graduation percentage each year. School
performance is more comprehensively seen from the achievements of the 8
national education standards. If the 8 national education standards are used as a
reference, then in the aggregate, the performance of elementary schools in
Ambon City is categorized as good. It is evident from the results of elementary
school accreditation. Based on data from The National Accreditation Agency
(BAN) of Maluku Province Middle Schools, there are 207 elementary schools in
Ambon City, with details of 24 public and 25 private schools (BPS, 2020).
Based on the results of brief observations at several accredited primary schools,

the general symptoms were found that the performance of the primary schools
needed to follow the accreditation rating obtained. This condition is
understandable because the accreditation of schools, including elementary
schools, still emphasizes the complaint aspect in the achievement of documents
according to the 8 standard assessment rubrics and has yet to emphasize actual
performance. In that context, schools generally only prepare documents
according to accreditation requirements. The assessor's assessment targets the
adequacy of standard 8 documents and is not based on performance.
Unsurprisingly, schools are satisfied when they receive accreditation certificates
72
and tend to refrain from following up on the results of BAN SM's

recommendations for improving school performance.
The practice above has become a habit that holds elementary schools in Ambon
City and Indonesia. In this regard, this study is relevant to conduct so that it can
be dissected in detail on the performance of the elementary school. The aims of
this research were: 1) Measuring the performance of the elementary school in
Ambon City; 2) Identifying the problem of measuring the elementary school's
performance in Ambon City.
The results assessed here should be related to the time input and the cost input
incurred. Therefore, in performance measurement, effectiveness indicators are
not only used but also efficiency. Performance is the process or activity of an
organization solving problems. Thus, organizational success should be assessed
from adaptability and flexibility rather than the benefits achieved and
productivity (Notanubun et al., 2019; Ahmed et al., 2022).
In its development, the concept of performance is examined from three

perspectives, namely (1) individual performance, in the form of employee
contributions according to status and role in the organization (2) teamwork, in
the form of contributions made by employees as a whole; and (3) organizational
performance is the actual contribution of individual and team performance as a
whole (Burnier, 2018).
This study analyzed school performance; Thus, we used organizational

performance as the theoretical basis. Organizational performance is defined as
the ability of an organization to achieve its goals through efficient and effective
utilization of resources (Nosratabadi et al., 2020). Organizational performance is
measured by how far an organization achieves results when compared with
previous performance with other organizations (benchmarking) and to what
extent it achieves the goals and targets set (Medne & Lapina, 2019); Martin et al.,
2020; Farida & Setiawan, 2022). In this regard, school performance is the extent
to which the school achieves predetermined goals, targets, and standards by
efficiently and effectively utilizing the potential of the school's human, physical,
and social resources.
The results of studies in Catalonia-Spain, Chile, and Virginia-USA show that the
performance of elementary school students is influenced by socio-economic
factors, school characteristics, and teacher quality, disrupting students' academic
performance, both cognitive and non-cognitive achievement (Clayton, 2011;
Candia et al., 2022). In addition, a study in the UK showed that older students
placed in classes with older students had higher academic achievement scores
than younger students. (Gutiérrez-domènech, 2012). Sixth graders in Latin
America perform well in math and reading literacy. It is because one-third of
students have a computer at home, and schools provide computers to help with
learning. (Román Carrasco & Murillo Torrecilla, 2012). In addition, the results of
the Belgian study showed that in improving the performance of elementary
school students for the first language (L1), the teacher showed a video. The
73
results showed that the students' oral communication skills improved when the
teacher showed the video. (Colognesi et al., 2023). The information above is a
strength in improving school performance, but the school performance related to
good governance towards superior schools has not been considered. Therefore,
this study highlights that.
In managing school performance, there needs to be a method agreed upon by

experts. In general, the measurement of school performance is carried out by
comparing the performance of one school to another, namely through the cross-
sectional comparative method (Broadbent, 2017; Almoslamani, 2022). School
performance can be measured based on parameters (1) educational outcomes
both in the field academic as well as development in the personal and social
dimensions of students; (2) the quality of the educational process in the school
concerned, such as culture or ethos, and the level of satisfaction felt by both staff
at the school (service providers) and students (service recipients) towards
schools; (3) the capacity or ability of schools to provide the inputs needed for a
quality educational process to take place, such as the ability to recruit qualified
prospective students, the availability of funds and adequate educational
facilities and infrastructure and the ownership of professional teacher staff.
This study measured school performance parameters based on three

dimensions: performance action, performance achievement, and performance
outcomes. Details can be visualized in Figure 1.
School Performance
Performance Action Performance Achievement Performance Outcomes
1. Accreditation Rating
1. Efficiency 2. Graduation 1. The percentage of
2. Effectiveness presentation students received in
3. Physical appearance 3. Average IP/Raport higher schools
4. Open management 4. Presence average 2. The percentage of
5. Teacher competence 5. Percentage of DO students entering the
6. Staff cooperation 6. The percentage of school
7. Quality commitment grade repetition 3. Customer
8. Achievement motive 7. Civilized behavior satisfaction
9. 9.Excellent service 8. Championship 4. Support received
ranking
9. Benchmarking
Figure 1: School performance (Rahabav, 2014)
74
2. Methodology
2.1. Research Design
This research approach was descriptive-qualitative, which is carried out to
analyze non-numeric information such as words, expressions, actions, and
behavior (Ahmad et al., 2019). A study with qualitative characteristics is a
research based on descriptive records such as document reviews and interviews
to produce descriptive information.
This research design is classified as a case study. A case study is a detailed study
of a person or a social unit over a certain period. Case studies can combine data
collection methods such as archives, interviews, questionnaires, and
observation. Data/evidence can be qualitative data (words) or quantitative
(numbers). Case studies have the advantages of 1) Examination of data, 2)
variation in intrinsic, instrumental, and collective approaches, and 3) detailed
qualitative reports. In addition, the limitations of case studies are 1) Feels rigid,
which allows vague evidence or biased views to influence the direction of
findings and conclusions 2) Provide very little basis for scientific generalization
because they use a small number of subjects; some are done with only one
subject. 3) Case studies are often labeled too long, difficult to conduct, and
produce a large amount of documentation (Zainal, 2007; Quintão et al., 2020).
This qualitative research generally uses observation and interview approaches,

where respondents are not randomly but selected according to specific criteria
(Daniel, 2016). This study used a purposive sampling method to take
respondents. The purpose of the purposive sampling method is to investigate
and obtain rich information from a case so that it answers the questions posed in
a study (Busetto et al., 2020).
2.2 Participant
By using a purposive sampling technique, the informants for this study with
details of 3 school principals, 3 deputy heads of curriculum, and 3 treasurers
from 3 elementary schools, namely 1 Xaverius Elementary School C (accredited
A), Public Elementary School 11 (accredited B) and Inpres Elementary School 1
Hative Kecil (accredited C). Informant research was based on the criteria of
mastering the substance of the problem under study.
2.3 Data Collection Technique

The collection techniques used were assessment rubrics, structured interviews,
participatory observation, and documentation studies. Documentation study
data taken were accreditation certificates, quality report cards, input data,
student output, lesson plans, RKJM (Medium Term Work Plan) documents, RKT
(the annual work plan), RKS (school activity plan and budget), bookkeeping,
annual performance reports, attendance data, partnership documents with
stakeholders, and academic and non-academic achievement data.
2.4 Data Analysis

The data analysis technique used was qualitative-descriptive, through three
stages of analysis, namely data condensation, data display (presentation of data),
75
and conclusion drawing/verification (drawing conclusions or verification)

(Palinkas et al., 2015).
1) Data condensation is a form of analysis to sharpen, sort, focus, discard, and
organize data in such a way as to produce a "final" and verifiable conclusion.
The data condensation process is a process to verify the validity of the data.
The condensation process is done by selecting, focusing, simplifying,
abstracting, and even changing the data obtained through field notes,
interview transcripts, documents, and other empirical data.
2) Data display is the process of data analysis and data display. Qualitative data
can be presented as narrative text in field notes, matrices, graphs, networks,
and charts. The presentation of data will combine information arranged in a
coherent and easy-to-reach form, making it easier to re-analyze what is going
on and whether the conclusions are correct or vice versa.
In order to test the validity of the data, the authors used the triangulation
method. Triangulation was used to test the credibility of the data by checking
the data obtained from several sources. The triangulation techniques used were
technical triangulation, source triangulation, and time triangulation. We used
source triangulation to test the credibility of the data. It was done by checking
the data obtained at the sample schools through several data sources (principal,
vice principal of curriculum, and treasurer) and then asking for agreement
(member check) with the three data sources. Triangulation techniques are used
to test the credibility of the data. Technical triangulation was done by checking
data from the same source using different techniques. The data obtained from
the informants with the assessment rubric was checked using interviews,
participatory observation, and documentation studies. Suppose the techniques
for testing the credibility of the data are different. In that case, we discussed
further with the informants to ascertain which data was considered the most
appropriate (correct).
Time triangulation was carried out, considering that it often affects the
credibility of the data. For example, the validity of the data collected from the
informant during the day is very doubtful because the informant may be tired.
Therefore, we test the data's credibility by checking the data's accuracy using
assessment rubrics, interviews, participatory observation, and documentation
studies in different times or situations. If the results differ, they must be done
repeatedly to obtain valid data.
3. Result and Discussion

3.1 Dimensions of Action Performance (Action Achievements)
The results of the analysis of the sub-dimensions, efficiency revealed that: there
is a balance between student inputs and outputs, with graduation rates reaching
90-100% each year. On the other hand, there are still inefficiencies in time, effort,
cost, and facilities used. It can be seen from implementing programs that were
not realized on time due to limited funds. Funds were not realized over time,
misuse of the budget plan, and reduced number of staff participating in several
programs due to limited/lack of funds.
76
The results of the analysis of the sub-dimensions, Effectiveness: The results of

the data analysis show that the school program could have been more effective.
The school program has yet to be realized according to the time target;
performance indicators; are not yet relevant to the RKJM (Medium Term Work
Plan), Operational Plan (RENOP), and school vision and mission. The results of
studies on documents, RKAS (school activity plans and budgets), accountability
reports, and other supporting data show that program implementation could
have been more effective.
It is evidenced by the results of an interview with “L” as follows:

“School programs so far have yet to be achieved effectively because (1) funding is always
late; generally, stage 1 is only realized in July. They must complete a report as soon as
possible for the search for stage 2 in September and stage 3 in December. The fund
disbursement mechanism made it difficult for us to reschedule programs/activities and
implement them; (2) there are many activities of the Ambon City Education and Sports
Office that are not listed in the RKAS, which require school support or participation; (3)
School needs are urgent and time is minimal, so they are forced to adopt policies that are
no longer following the RKAS documents so that accountability reports are not late; (4)
Our school has no other funding sources than the primary sources of BOS (School
operational assistance), BOP (Implementation Operational Assistance) and committee
fees which is very limited so that many of the programs listed in the RKAS cannot be
realized.”
Sub-dimensional analysis results showed that the physical appearance of the

school generally meets all the indicators surveyed, namely the physical
condition of the building is generally well-organized, clean, and beautiful school
environment; the number of classrooms is proportional to the number of
students; adequate laboratory facilities and learning support facilities are
available; available sports fields; sufficient toilet facilities are available to serve
the needs of school personnel and clean.
The open management sub-dimensional analysis results reveal that the

indicators were met, like joint planning involving the committee, participation of
all teachers and management in the program, and joint meetings and
evaluations. However, the results of a documentation study conducted by the
author on the RKAS (School Budgeting Planning) documents, books, and
accountability reports on transparency and accountability in the management of
activities and funds are not good; broad access to information disclosure for all
parties regarding school programs and funds as well as routine internal audits
has not been carried out optimally. In this regard, school management needs to
be better managed.
The results of the analysis of the Teacher Competency Sub-Dimension, the

indicators that have been fulfilled are that teachers have academic qualifications
equivalent to S1/D4, teach following the field of science; teachers sufficiently
master ICT, able to relate to all parties and have empathy for students (social
competence). The teacher has a good personality, so it becomes a model for
students to emulate (personality competence). Meanwhile, the indicators that
have yet to be maximized are the teacher's lack of mastery of the scientific field
77
(professional competence); and lack of pedagogic competence. It was

acknowledged by “A” as follows:
“As a teacher, professional competence is an unsolved problem. I continuously deepen
professional competence through discussions, reading, and participating in hose
training. However, science and technology developments happen so fast that what I learn
sometimes needs to be updated. In addition, even though our school has implemented
K13 with an emphasis on the 5M approach, most teachers still use lectures. Technically,
innovative learning models are de facto written in lesson plans but have not been
implemented in learning practice.”
The interview results with “Al” were confirmed by a document study conducted
by the author on the teacher's lesson plans. Lesson plans are mostly copy-paste.
Lesson plans complement the learning administration and are not used as a
reference in classroom learning practices. From the authors' observations, it was
found that there was an inconsistency between the teacher's lesson plans and
actions in class. Based on the description above, teachers in Ambon City have
mastered two competency clusters, personal competence, and social
competence. In contrast, teachers have not fully mastered professional and
pedagogical competence. For this reason, the main thing that teachers need is to
master scientific and pedagogic content (Viseu et al., 2016; León et al., 2021).
All indicators have been achieved to analyze the sub-dimensions of staff

cooperation: participation in various activities; lack of conflict; mutual
reinforcement; mutual trust; and a sense of l'spirit de corps among school
personnel. The achievement of this dimension is authentic when conducting
research. The author found that the sample schools could build human relations
among principals, teachers, education staff, and even with students and parents.
Staff cooperation is needed so that all policies and programs launched by the
school can be appropriately implemented.
All indicators have yet to be achieved to analyze the sub-dimensions of

commitment to quality. It can be seen from the absence of a quality committee;
the absence of quality standard documents; no SOP (Standard Operating
Procedures) on quality; teachers' lack of commitment to teaching professionally;
no commitment to make all non-academic activities end with quality; and
evaluation and quality improvement have not been carried out continuously.
The results of the documentation study conducted by the author at the sample
schools strengthen the informants' answers. It is evident from the absence of an
internal quality team decision letter and quality standard documents such as
school academic regulations, quality guidelines, quality instruments, and SOP
regarding quality. There needs to be a teacher's commitment to teaching
professionally. For example, it was found that there are still teachers who taught
not according to the Lesson Implementation Plan (LPP), taught without teaching
aids, used fewer computers, and did not commit to making all non-academic
activities that lead to quality, evaluation, and quality improvement had not been
carried out continuously. No supporting documents were found, such as
assessment results and remedial data.
78
In an analysis of the sub-dimensions motive for achievement from the

respondents' answers, all indicators are in the less category, namely: the teacher
is very enthusiastic if he excels; teachers have not set realistic goals and taken
calculated risks; teachers still believe in good fortune; teachers do not want to
take responsibility for their work results; teachers have not acted as
entrepreneurs, most teachers have not chosen challenging assignments and
demonstrated more initiative behavior; teachers do not yet want quick concrete
feedback on their performance; teachers work primarily for money or power.
The results of the interview with “P” are as follows:

“As a school principal, I have to admit honestly that the teachers at this school do not
have good achievement motives. It is evident from the fact that the majority of teachers
lack initiative in their work. They are still waiting for orders. The entrepreneurial spirit
that should appear in behaviors such as hard work, the ability to manage time, the ability
to withstand pressure, innovation, and creation is lacking in teachers; Most teachers are
still motivated to work when given adequate incentives.”
Studies on teacher motivation show that teachers who receive incentives,

especially financial incentives, become more effective at work. From the
perspective of economic concepts, the primary motivation for humans to work is
to earn money. Thus, when rewarded, humans are motivated to do work
(Forson et al., 2021). In this regard, without financial motivation, teacher
performance will decrease. This constatation is only sometimes valid; because
humans are not always motivated to work because of money. Some people are
motivated because they want to pursue psychological satisfaction. Teachers with
such motivation will be committed to transforming students' academic
knowledge and skills (Mintrop & Ordenes, 2017; Bektaş et al., 2020). In that
context, schools must create effective management because it supports teacher
motivation. If the school system and structure are not supportive, teachers do
not play a good role and tend to lose their commitment and professional
responsibility (Bashir et al., 2020).
In the analysis of the sub-dimensions excellent service, all surveyed indicators

have been achieved, namely: the teacher shows a friendly attitude; the services
provided by the teacher are fast; the work procedure is not complicated; the
services provided by the teacher are fair to all customers (non-discriminatory);
customers feel comfortable when dealing with teachers, and there is satisfaction
in the customers. Excellent service is customer-centered (the customer is king),
and the teacher positions himself as a servant. Excellent service will be realized
in efficient, effective, without discrimination, not bureaucratic, and humane,
responsive, and highly responsive services.
3.2 Dimension of Performance Achievement (Achievement Results)

Based on the results of the analysis of the accreditation ranking sub-dimensions,
the sample schools met the indicators surveyed, namely: schools have
accreditation certificates; both schools with accreditation rating A (Xaverius
Elementary School C), schools with accreditation B (Public Elementary School
11) and schools with accreditation C (Elementary School Inpres 1 Hative Kecil)
Ambon City. According to the analysis of the National Examination Percentage
79
Passing Sub-Dimensions, all the surveyed indicators have been achieved,

namely: comparison of the inputs and outputs number of students is achieved;
the average score compared to the school program target (KKM) is achieved; the
average score compared to other schools at the district, the provincial level was
achieved.
For all indicators surveyed in the analysis of sub-dimension 2, the average of

education reports was achieved, namely: the average of education reports;
compared to other schools (benchmarking) achieved. The results of this study
indicate that students with high achievement motivation also tend to show high
achievement. This study's results align with previous studies that there is a
significant relationship between achievement motivation and academic
achievement. It shows that motivational factors play an essential role in
academic achievement. Individuals with high academic achievement tend to
have good academic achievement (Shin & Hur, 2020). One factor contributing to
motivation is students' attitudes toward the field of study (De Sousa Sabbagha et
al., 2018).
Besides that, achievement in the academic field is influenced by economic factors

(Koob et al., 2021). Parents' income or socio-economic status influences students'
academic performance (Gobena, 2018; Fenanlampir et al., 2021). Another
dominant factor, namely teacher competency, significantly impacts student
academic achievement. A teacher's performance greatly influences student
performance (Getie, 2020; Batlolona & Diantoro, 2023).
In dimensional analysis, the average attendance of all surveyed indicators has

been achieved: the number of attendance compared to the effective learning
calendar and the number of attendance compared to other schools
(benchmarking) is achieved. A documentation study by the author reviewing
teacher attendance lists and teaching manuals found that the frequency of
teacher attendance in teaching and curriculum targets in the sample schools was
generally achieved, except when Covid-19 was generally not reached.
All indicators surveyed in the analysis of sub-dimensions and the percentage of

drop-outs were achieved: the number of drop-outs compared to the total
population of students, development trend in recent years, and the number of
drop-outs compared to other schools (benchmarking). Based on the
documentation study, there were no student drop-outs either in the sample
schools or in comparison with other schools. There was a balance between the
number of student inputs and the number of outputs (no students who dropped
out were found).
All the indicators surveyed in the analysis of the sub-dimensions, the percentage
of grade repetition, was the number of students who repeat compared to the
total student population were not balanced; the development trend in recent
years continues to push up; there was no significant difference in the number of
students repeating compared to other schools (benchmarking). The repetition
rate was only found in 1 sample school, which was 0.10%, while other schools
80
were not found. This figure occurred in grade 1 due to the impact of covid - 19.
The trend of repeating development from year to year does not show an
increasing trend (0%).
All indicators in the analysis of the civilized behavior sub-dimensions have been
achieved: students were polite in speaking, respectful to teachers and others,
and not involved in a brawl or other criminal acts. It is a very encouraging
development that the sample schools successfully fostered and developed their
students' character.
All indicators in the analysis of the sub-dimensions of championship ranking

have yet to be achieved in schools with B and C accreditation, while schools with
A accreditation have achieved all indicators, namely: cleanliness champion,
sports and other arts champions, and writing contest winners. Based on the
results of a documentation study, schools with B and C accreditation generally
emphasize academic achievement and pay less attention to non-academic
achievements. It is evident from the few winners in the sports and/or arts that
were won.
3.3 Dimension of Performance Outcomes (Final Achievement)

Indicators measured in sub-dimensions percentage of junior high school
enrolment was the amount received compared to outputs; entry route through
selection; accreditation rating is the preferred choice. The percentage of entering
junior high school in the three sample schools was 100%. It means that all
graduates can be absorbed at junior high school. Based on free interviews with
informants, it is known that students with accreditation rating A were accepted
without testing. Meanwhile, several junior high schools, such as Public Junior
High School 1 and 2, Lantera Junior High School, and Xaverius Junior High
School, apply an entrance test policy for accreditation B and C. The percentage
of students who pass the entrance test reaches 70%. It shows that the students in
the sample schools have adequate achievements and can compete with other
elementary school graduates to be accepted into junior high school.
Analysis of sub-dimensions indicators measure the percentage of students who

are accepted by schools each year: (1) the number received is compared to the
number of the previous year; (2) the trend of students' interest each year (up or
down) and (3) the average education report. Based on the analysis results of the
informants' answers, it was found that the percentage of students accepted at
accredited A schools (Elementary School Xaverius C) showed an increasing
trend from year to year. It is because the location is very strategic, the
competence of teachers is above average, and vocational skills such as
computers, English, and Mandarin skills were trained. In addition, the non-
academic achievements that have always been achieved by Elementary School
Xaverius C were the main attraction. Meanwhile, the other two schools, Public
Elementary School 11 and Elementary School Inpres 1 Hative Kecil with
Accreditation A at Ambon City show an increasing trend, but lower than
Elementary School Xaverius C. The average report on education in the three
sample schools is 7.
81
Indicators measured in the analysis of sub-dimensions of customer satisfaction

were (1) a good perception of school products or services; (2) a high interest in
using school services. Based on the analysis results, information was obtained
that customers are generally satisfied with the services provided by schools with
accreditation A. It has triggered high interest in parents using teacher services,
for example, requests for additional tutoring. This condition is different from
schools with B and C accreditation. Based on a documentation study, the two
schools still need to measure customer satisfaction with school products.
Therefore, parents' interest in using teacher services to do additional tutoring,
for example, needed to be improved. Even for Accreditation C, it did not even
happen.
Measuring service quality is fundamental to building and maintaining a moving

image and customer relationship. Many organizations try to find customer
reactions to service quality because it triggers increased competitiveness,
especially in attracting new customers and increasing good customer
relationships (Simões et al., 2022).
The quality of service perceived by customers is the forerunner to their level of

satisfaction. It is a determinant of satisfaction for an organization, including
schools, in improving its quality and competitiveness. Perceptions of service
quality can include input, process, and output from the perspective of
stakeholders/customers (Alam & Noor, 2020). Therefore, a progressive and
influential institution must adapt all its activities to customer expectations.
Customers' quality preferences and perceptions about quality must be
maintained by management (Mosadeghrad, 2014); (Gemnafle et al., 2018).
For analysis of sub-dimensions of support received, it is known that the sample

schools lack support from parents. The support received was still limited to
financial support (committees). Meanwhile, other supports such as cooperative
partnerships, staff support, and supporting facilities needed to be improved in
schools.
3.4 Identification of Factors Inhibiting School Performance

Based on the results of the analysis of data collected by the authors from various
sources, several factors were identified that hindered school performance,
including the following: First, low-quality commitment: Low-quality
commitment in the sample schools can be seen from the absence of quality
standard documents, the absence of academic regulations, the absence of SOPs
regarding quality; there is no teacher commitment to teaching professionally; the
school's commitment has not yet been created to make all non-academic
activities end with quality; evaluation and quality improvement that has not
been carried out continuously.
Second, mismanagement: The symptoms of mismanagement in the sample

schools are seen from the discrepancy between the RKAS documents and the
realization of activities and funds. Bookkeeping activities were not carried out
properly; schools needed to apply the principles of good governance in
82
managing activities and funds; there were still many policies that deviated from
the RKAS; the principles of transparency and accountability needed to be
applied consistently.
Third, the competency of the school principal: the principal had yet to show
himself as a visionary leader. It was evident from the school's vision, mission,
and policies that they needed to be placed on an open and transparent platform.
A grand design of the School Work Plan has yet to be found standardized in the
systemized documents. Thus, schools still need creative, clear, measurable
development directions. It is a logical consequence of recruiting school
principals not based on a competency approach but instead on a spoil system
approach (nepotism). Therefore, school principals are affiliated with politicians,
bureaucrats, and regional elites. The school principals were immediately
appointed to positions without being prepared through preparatory training as
school principals. Even in the office, they do not receive capacity-building
training as school principals.
Inadequate teacher competence: Two less main competencies in the teachers

surveyed needed to be stronger professional and pedagogical competence. It
was due to the need for more frequency and intensity of education and training
conducted by the Youth and Sports Education Office and those carried out in the
Teacher Working Group (TWG) or school initiation programs. This condition is
still exacerbated by the teacher's commitment to growing in the office through
independent learning, which is still felt to be very minimal.
Fourth, the teacher's lack of achievement motives: Teachers lack achievement

motives. It is evident from creation, minimal initiation; teachers were less
adaptive to changes in science and technology; teachers were still waiting for the
principal's orders to carry out work, lack of entrepreneurial spirit, teachers were
less able to manage time, were less able to withstand pressure and most teachers
were still motivated to work if given money (economic motives).
Fifth, lack of funds: Fund is one of the triggers for school performance that could
be more optimal. It is because schools entirely rely on financing activities from
BOS (School Operational Assistance), BOP (Educational Operational Assistance),
and Committee contributions. Schools have yet to be able to create products that
are of public interest. It will encourage people to buy school products which
have implications for contributing to school income.
Sixth, lack of community support: Community support is one of the great hopes
of schools to support various programs and activities that have been designed.
Support can be from financial contributions, thoughts, personnel, or facilities.
From the results of the data analysis, informants admit that it still needs to be
improved. Community support has been very limited due to 1) misconceptions
about the importance of education which has implications for lack of public
awareness of children's education; 2) school financial management that is less
transparent and accountable; 3) the influence of free education campaigns
carried out by candidates for Mayor and Regional People's Representative
83
Council (DPRD) in each election campaign for the Mayor or Regional People's
Representative Council (Rahabav, 2020).
4. Conclusion
Based on the analysis and discussion above, several conclusions are drawn: 1)
Dimensions of performance action: dimensions that the school has achieved,
namely Sub dimensions: Physical appearance of the school, staff cooperation,
and excellent service. Sub-Dimensions that have not been achieved to the
maximum are sub-dimensions: championship rankings that have not been
achieved, mainly in schools accredited B and C. 2) Performance achievement
dimensions; sub-dimensions that have been achieved by schools, namely: all
schools have been accredited, the average score on education reports is 7; the
average attendance of students is good; minimal Drop-Out percentage; The
average of repeating classes and repeating subjects is less, and students show
civilized behavior or good character. Sub-Dimensions that have yet to be
achieved, namely championship rankings or non-academic activities, generally
schools with B and C accreditation, still need to be improved. 3) Dimensions of
performance outcomes (final achievement): The sub-dimension that has been
achieved, namely the attendance of graduates going on to junior high school
100% and student presentations accepted at all schools, show a non-decreasing
trend even Elementary School Xaverius, showing an upward trend. The sub-
Dimensions that have yet to be achieved, namely the level of customer
satisfaction and community support to support school programs, still need to be
improved.
Factors inhibiting school performance include a low-quality commitment:

mismanagement; insufficient competence of the principal: insufficient
competence of teachers; lack of teacher achievement motivation; lack of funds,
and lack of community support. Therefore, several dimensions have been
achieved and have not been achieved in leading to excellent schools. The
limitation of this research is that it only analyzes the performance of excellent
schools at the elementary level, and future suggestions can be carried out at a
high level, namely junior and senior high school. The school should note the
existing dimensions to be able to improve and repackage the program so that it
runs well.
5. References
Ahmad, S., Wasim, S., Irfan, S., Gogoi, S., Srivastava, A., & Farheen, Z. (2019).
Qualitative v/s. quantitative research- a summarized review. Journal of Evidence
Based Medicine and Healthcare, 6(43), 2828–2832.
https://doi.org/10.18410/jebmh/2019/587
Ahmed, W., Hizam, S. M., & Sentosa, I. (2022). Digital dexterity: employee as consumer
approach towards organizational success. Human Resource Development
International, 25(5), 631–641. https://doi.org/10.1080/13678868.2020.1835107
Alam, M. M. D., & Noor, N. A. M. (2020). The relationship between service quality,
corporate image, and customer loyalty of generation y: an application of s-o-r
paradigm in the context of superstores in Bangladesh. SAGE Open, 10(2).
https://doi.org/10.1177/2158244020924405
Almoslamani, Y. (2022). The impact of learning strategies on the academic achievement
of university students in Saudi Arabia. Learning and Teaching in Higher Education:
84
Gulf Perspectives, 18(1), 4–18. https://doi.org/10.1108/LTHE-08-2020-0025

Bashir, A., Amir, A., Jawaad, M., & Hasan, T. (2020). Work conditions and job
performance: An indirect conditional effect of motivation. Cogent Business and
Management, 7(1), 1–16. https://doi.org/10.1080/23311975.2020.1801961
Batlolona, J. R., & Diantoro, M. (2023). Mental models and creative thinking skills in
students ’ physics learning. Creativity Studies, 16(2), 433–447.
Bektaş, F., Kılınç, A. Ç., & Gümüş, S. (2020). The effects of distributed leadership on
teacher professional learning: mediating roles of teacher trust in principal and
teacher motivation. Educational Studies, 00(00), 1–23.
https://doi.org/10.1080/03055698.2020.1793301
BPS. (2020). Jumlah Sekolah, Guru, dan Murid Sekolah Menengah Pertama (SMP) di Bawah
Kementerian Pendidikan dan Kebudayaan Menurut Kecamatan di Kota Ambon,
2019/2020. 18–19.
Broadbent, J. (2017). Comparing online and blended learner’s self-regulated learning
strategies and academic performance. Internet and Higher Education, 33, 24–32.
https://doi.org/10.1016/j.iheduc.2017.01.004
Burnier, D. L. (2018). Reimagining performance in public administration theory and
practice: creating a democratic performativity of care and hope. Administrative
Theory and Praxis, 40(1), 62–78. https://doi.org/10.1080/10841806.2017.1420746
Busetto, L., Wick, W., & Gumbinger, C. (2020). How to use and assess qualitative
research methods. Neurological Research and Practice, 2(1), 1–10.
https://doi.org/10.1186/s42466-020-00059-z
Candia, C., Oyarzún, M., Landaeta, V., Yaikin, T., Monge, C., Hidalgo, C., & Rodriguez-
Sickert, C. (2022). Reciprocity heightens academic performance in elementary
school students. Heliyon, 8(12), 1–8. https://doi.org/10.1016/j.heliyon.2022.e11916
Clayton, J. K. (2011). Changing diversity in US schools: The impact on elementary
student performance and achievement. Education and Urban Society, 43(6), 671–695.
https://doi.org/10.1177/0013124510380909
Colognesi, S., Coppe, T., & Lucchini, S. (2023). Improving the oral language skills of
elementary school students through video-recorded performances. Teaching and
Teacher Education, 128, 1–11. https://doi.org/10.1016/j.tate.2023.104141
Daniel, E. (2016). The Usefulness of qualitative and quantitative approaches and
methods in researching problem-solving ability in science education curriculum.
Journal of Education and Practice, 7(15), 91–100. https://doi.org/2222-288X
De Sousa Sabbagha, M., Ledimo, O., & Martins, N. (2018). Predicting staff retention from
employee motivation and job satisfaction. Journal of Psychology in Africa, 28(2), 136–
140. https://doi.org/10.1080/14330237.2018.1454578
Farida, I., & Setiawan, D. (2022). Business strategies and competitive advantage: the role
of performance and innovation. Journal of Open Innovation: Technology, Market, and
Complexity, 8(3), 1–16. https://doi.org/10.3390/joitmc8030163
Fenanlampir, A., Leasa, M., & Batlolona, J. R. (2021). The development of homogeneity
psycho cognition learning strategy in physical education learning. International
Journal of Evaluation and Research in Education, 10(3), 1047–1059.
https://doi.org/10.11591/IJERE.V10I3.21713
Forson, J. A., Ofosu-Dwamena, E., Opoku, R. A., & Adjavon, S. E. (2021). Employee
motivation and job performance: a study of basic school teachers in Ghana. Future
Business Journal, 7(1), 1–12. https://doi.org/10.1186/s43093-021-00077-6
Gemnafle, M., Waimuri, S. P., & Batlolona, J. R. (2018). Organizational climate of the
school and teacher performance improvement in the 21 st century. International
Journal of Science and Research (IJSR), 7(2), 119–126.
https://doi.org/10.21275/ART20179865
Getie, A. S. (2020). Factors affecting the attitudes of students towards learning English as
a foreign language. Cogent Education, 7(1), 0–37.
85
https://doi.org/10.1080/2331186X.2020.1738184
Gobena, G. A. (2018). Family Socio-economic status effect on students’ academic
achievement at college of education and behavioral sciences, Haramaya University,
Eastern Ethiopia. Journal of Teacher Education and Educators, 7(3), 207–222.
Guha, P. (2021). Are disadvantaged schools slow to adopt school-based management
reforms? Evidence from India. Policy Design and Practice, 4(4), 517–533.
https://doi.org/10.1080/25741292.2021.1950349
Gutiérrez-domènech, M. (2012). Student performance in elementary schools. Revista de
Economía Aplicada, 59(20), 135-164
Koob, C., Schröpfer, K., Coenen, M., Kus, S., & Schmidt, N. (2021). Factors influencing
study engagement during the COVID-19 pandemic: A cross-sectional study among
health and social professions students. PLoS ONE, 16(7), 1–19.
https://doi.org/10.1371/journal.pone.0255191
Leasa, M., Abednego, A., & Batlolona, J. R. (2023). Problem-based learning (PBL) with
reading questioning and answering (RQA) of preservice elementary school
teachers. International Journal of Learning, Teaching and Educational Research, 22(6),
245–261. https://doi.org/10.26803/ijlter.22.6.14
León, H. C. R., Navarro, E. R., Meléndez, L. V., del Rosario Merino Salazar, T., Yuncor,
N. R. C., & María, E. M. M. (2021). Job satisfaction factors in secondary school
teachers, public and private institutions in a Peruvian Region. Cypriot Journal of
Educational Sciences, 16(6), 3317–3328. https://doi.org/10.18844/cjes.v16i6.6566
Martin, J., Elg, M., & Gremyr, I. (2020). The many meanings of quality: towards a
definition in support of sustainable operations. Total Quality Management and
Business Excellence, 0(0), 1–14. https://doi.org/10.1080/14783363.2020.1844564
Medne, A., & Lapina, I. (2019). Sustainability and continuous improvement of
organization: Review of process-oriented performance indicators. Journal of Open
Innovation: Technology, Market, and Complexity, 5(3), 49.
https://doi.org/10.3390/joitmc5030049
Mintrop, R., & Ordenes, M. (2017). Motivação dos professores para trabalhar na era dos
incentivos extrínsecos extrínsecos: Metas de desempenho e compromissos pró-
sociais ao serviço da equidade. Education Policy Analysis Archives, 25(44), 1–43.
https://doi.org/10.14507/epaa.25.2482
Mosadeghrad, A. M. (2014). Factors influencing healthcare service quality. International
Journal of Health Policy and Management, 3(2), 77–89.
https://doi.org/10.15171/ijhpm.2014.65
Nosratabadi, S., Bahrami, P., Palouzian, K., & Mosavi, A. (2020). Leader cultural
intelligence and organizational performance. Cogent Business and Management, 7(1),
1–17. https://doi.org/10.1080/23311975.2020.1809310
Notanubun, Z., Ririhena, R. L., & Batlolona, J. R. (2019). The effect of organization
restructuring on organization performance viewed from employee performance
and leadership effectiveness at Maluku Provincial education office. Journal of
Education and Learning (EduLearn), 13(1), 118–124.
https://doi.org/10.11591/edulearn.v13i1.11222
Palinkas, L. A., Horwitz, S. M., Green, C. A., Wisdom, J. P., Duan, N., Hoagwood, K.,
Angeles, L., & Northwest, K. P. (2015). Purposeful sampling for qualitative data
collection and analysis in mixed method implementation research. Administration
and Policy in Mental Health and Mental Health Services Research, 42(2), 533–544.
https://doi.org/10.1007/s10488-013-0528-y.Purposeful
Parker, R., Thomsen, B. S., & Berry, A. (2022). Learning through play at school – a
framework for policy and practice. Frontiers in Education, 7, 1–12.
Qian, J., & Kong, L. (2018). When secular universalism meets pluralism: religious schools
and the politics of school-based management in Hong Kong. Annals of the American
86
Association of Geographers, 108(3), 794–810.

https://doi.org/10.1080/24694452.2017.1372175
Quintão, C., Andrade, P., & Almeida, F. (2020). How to improve the validity and
reliability of a case study approach? journal of Interdisciplinary Studies in Education,
9(2), 273–284. https://doi.org/10.32674/jise.v9i2.2026
Rahabav, P. (2014). Kepemimpinan Kepala Sekolah di Era Otonomi Pendidikan [Principal
Leadership in the Era of Educational Autonomy]. Yogyakarta: Kanisius
Rahabav, P. (2020). Strategi Peningkatan Kompetensi Kepala Sekolah [Strategies to
Improve Principal Competence]. Presented at the DIKLAT for School Principals
organized by the Maluku Provincial Office of Education and Culture. Di Hotel
Golden Ambon 12 -14 Oktober 2020.
Rahabav, P., & Souisa, T. R. (2021). Evaluation of non-formal education management in
Maluku Province , Indonesia. International Journal of Evaluation and Research in
Education (IJERE), 10(4), 1395–1408. https://doi.org/10.11591/ijere.v10i4.21116
Román Carrasco, M., & Murillo Torrecilla, F. J. (2012). Learning environments with
technological resources: A look at their contribution to student performance in
Latin American elementary schools. Educational Technology Research and
Development, 60(6), 1107–1128. https://doi.org/10.1007/s11423-012-9262-5
Shin, Y., & Hur, W. M. (2020). Supervisor incivility and employee job performance: the
mediating roles of job insecurity and amotivation. Journal of Psychology:
Interdisciplinary and Applied, 154(1), 38–59.
https://doi.org/10.1080/00223980.2019.1645634
Simões, S., Oliveira, T., & Nunes, C. (2022). Influence of computers in students’ academic
achievement. Heliyon, 8(3), 1–13. https://doi.org/10.1016/j.heliyon.2022.e09004
Viseu, J., de Jesus, S. N., Rus, C., & Canavarro, J. M. (2016). Motivación docente,
satisfacción en el trabajo y capital psicológico positivo: Una revisión. Electronic
Journal of Research in Educational Psychology, 14(2), 439–461.
https://doi.org/10.14204/ejrep.39.15102
Zainal, Z. (2007). Case study as a research method. Jurnal Kemanusiaan, 9, 1–6.
87

Vol. 22, No. 7, pp. 87-103, July 2023
Received Jun 2, 2023; Revised Jul 15, 2023; Accepted Jul 28, 2023
Experiences of Student-Teachers: Implications

for Refined Student-Support
Adesegun Olayide Odutayo*
Department of Childhood Education,
University of Johannesburg 2006, South Africa
Sarita Ramsaroop
Department of Childhood Education,
University of Johannesburg 2006, South Africa
Abstract. Every institution has support services available for students,

especially first-year students, that would help deal with issues they
might need assistance with during their early years. Student support
services offered at universities to assist students’ academic, social, or
psycho-emotional needs help to enhance their overall welfare and
academic achievement. This study investigated university student-
teachers experiences and their implications for refined student support.
A case study qualitative research design was employed for this study
with 12 student-teachers (five male and seven female) as the sample. A
focus group interview method was employed to generate responses, and
the information was evaluated with thematic content analysis. This
study reported that first-year students encountered various challenges
classified into academic (inconsistency in keeping the timetable of
lectures and lecturers, misplacement, and missing academic results),
social (inadequacies in the transportation system and accommodation
issues), and psycho-emotional (low self-esteem, external academic
pressure, and suicidal ideation). It was also revealed that the
respondents benefitted from extra classes by lecturers, tutorials from
student associations and peers, assistance from the student affairs unit
and students’ associations, and aid from the counselling unit, staff
advisers, and clerics support services in the institution. Also, there is a
need for improvements in the various support services available in the
institution. Therefore, it was recommended that institutions only offer
admission to students applying for education-related courses.
Keywords: Experiences; Student-teachers; Academic; Social; Psycho-

emotional; Support Services
*
Corresponding author: Adesegun Olayide Odutayo; aodutayo@uj.ac.za
©Authors
88
1. Introduction
Higher education institutions (HEIs) aspire to provide students with positive
academic and social campus-based experiences (Conley, 2017; Njageand &
Athiemoolam, 2021). However, many challenges hinder student success at
university. Homesickness, the pressure of studies (Maymon et al., 2019), making
friends, accommodation, budgeting/finance, and transitioning to university life
(Nelson-Royes, 2013) impacted dropout rates of first-year students negatively,
both internationally and in Nigeria (Ogbeide & Ugwu, 2016). Students who do
not have an academic, emotional, or social connection to the university are more
likely to drop out of the university without effective student support services
(Chandra, 2020). Developing and implementing comprehensive student support
is crucial, especially in the first year at university (Sosu & Pheunpha, 2019).
These services, which include academic, social, and psycho-emotional needs, are
aimed at improving student well-being and student retention (Edwards et al.,
2015). Every institution has its respective support services available for students,
especially first-year students, that would help deal with issues they might need
assistance with during their early years (Sosu & Pheunpha, 2019). However,
students do not optimally utilize these support services (Scanlon et al., 2020).
Over the last few years, support from the Nigerian government to HEIs has
declined, compromising the quality of services provided by these institutions,
especially regarding support services (Borishade et al., 2021).
There are two categories of student support services, academic and non-
academic, which include peer support sessions, tutoring, academic mentoring,
and counselling sessions (Conley, 2017). Non-academic support services refer to
centralised services offered by universities, such as social, psychological (Green,
2021), and emotional support (Chou & Chang, 2019) that are not directly related
to academic issues (Amirkhan & Kofman, 2018). However, they are crucial for
providing students with suitable study conditions to excel outside of the
classroom system. On the other hand, academic support is provided by
academic staff. Academic support services are programs and strategies
employed to improve the learning outcome of students, particularly for those
who are showing signs of weak academic performance (Ugwu, 2021). Tertiary
institutions are responsible for assisting students to enhance their intellectual
capacity (cognitive, affective, and psychomotor domains of learning), which
would help them to proffer solutions to societal challenges (Ene, 2016). To attain
their aims, these institutions engage in teaching, research, knowledge creation,
and dissemination through a range of programs, including certificate, diploma,
undergraduate, and postgraduate courses (National Universities Commission
(NUC), 2014).
Teacher education is one of the programmes offered at universities in Nigeria,

aimed at the professional development of teachers to help them develop the
attitudes, abilities, and knowledge deemed necessary to function effectively and
efficiently in the teaching profession (Oyekan, 2016). Teacher education is
categorised into pre-service and in-service programmes (Ogunyinka et al., 2015).
A student-teacher is an individual pursuing a career in education at teacher
training institutions and placed in a classroom under the guidance of an
experienced teacher. In specific instances, these individuals are posted to schools
89
to observe and participate in classroom activities, assist with lesson planning,

and gradually take on teaching responsibilities, gaining hands-on experience
while learning from their mentor (Odutayo, 2021). This research focuses
specifically on students studying early childhood education, also referred to as
pre-primary education. Graduating quality teachers for the early years cannot be
underestimated, as they play a crucial role in supporting young children’s
learning and development and laying the foundations for learning in later years
(Federal Republic of Nigeria, 2014; Alabi, 2018).
In Nigeria, most pre-service teachers are unwilling or reluctant to enter the

teaching profession (Yusuf, 2010) but do so because they do not meet the
minimum requirements to study their preferred career pathway (Odutayo,
2021). Most students are admitted into faculties of education with lower scores
than their counterparts in other faculties. This implies that more scaffolding and
support are required for academic and psychological well-being (Ogunojemite,
2020). As a result, most of these students don’t take teacher education seriously,
some dropout, do poorly in the university, and in other circumstances, they
graduate but aren’t passionate about being teachers (Yusuf, 2014). From the first
author’s personal experiences as an undergraduate in the university, the benefits
and limitations of adequate support services were observed and experienced.
In one instance, the first author was conflicted about rejecting the teacher
education admission offered to him. The assistance provided by the academic
adviser facilitated the researcher to accept the admission and subsequently
pursue a career in the teaching profession. The research can provide a
comprehensive understanding of student teachers’ experiences, challenges, and
opportunities. It can shed light on their perspectives, emotions, and perceptions
throughout the teacher preparation process, offering valuable insights into their
professional development journey. In this research, we set out to examine
student-teachers’ first-year experiences and their implications for refined
student support. The following research questions would guide this study.
a. What are the challenges encountered by first-year students at the
university?
b. What are student-teachers’ experiences on the support services
(academic, social and psycho-emotional) provided at the university?
c. How adequate are these support services (academic, social and psycho-
emotional) at the university as articulated by student-teachers?
Student success at higher education institutions depends on several factors,
including infrastructure supporting teaching and learning and the available
academic, social, and psycho-emotional support. The following empirical
studies relating to infrastructure support are identified and discussed. Subair et
al. (2012) investigated the place of infrastructure in maintaining quality in
Nigerian universities. It was gathered that the availability and adequacy of
physical infrastructure, well-equipped libraries, ICT, and science laboratories
with modern equipment are necessary, provided it meets the minimum
standard and are an essential part of a conducive learning environment.
Omogbadegun et al. (2014) researched Covenant University in Nigeria as a case
90
study of the infrastructure needed to improve tertiary institutions’ academic

performance. According to their research, the uninterrupted academic
programme, faculty support programme, cutting-edge teaching and learning
tools, internet services, students’ assessment system, and 24-hour electricity and
water supply have all contributed to Covenant University students’ academic
improvement. In another study, Nugroho and Wibowo (2019) studied the
influence of school infrastructure on student learning activeness. Their study
explained that having adequate and appropriate rooms and learning spaces in
good conditions is decisive for students to achieve the expected academic
outcome and encourages learning activeness.
Researchers such as Murillo et al. (2011); Amsterdam (2013); Alkadri et al. (2017);
and Marmoah et al. (2019) have all carried out their separate studies and found
that infrastructure such as structures, classroom space, libraries, computer and
science laboratories, with renewed public spaces can contribute to improved
students’ academic learning outcome and quality of education. However,
students’ academic improvement might be threatened due to inadequate
structures and facilities, housing, and overcrowded classrooms (Zhirnova &
Absalyamova, 2013; Conolly & Lampe, 2016; Hyun et al., 2017). It is also
necessary to measure the success of student support services. Here, Busaba et al.
(2019) evaluated the use of a remedial management system in student support
services in Thailand. The Student Support Service Workflow Information System
(SSS WIS) comprises a student information system to evaluate the issues that
may arise in finance, health, social, and psychological issues. They submitted
that SSS WIS assists students in graduating from the university on time.
Regarding academic support services, Paideya and Bengesai (2017) investigated

the University of KwaZulu-Natal’s academic support. Their findings show that
academic support services, such as curriculum-based interventions, alternate
access interventions, peer support program evaluations, and innovative teaching
programmes, can help achieve institutional and student objectives. Ramsaroop
and Petersen (2020) carried out a study on portraits of primary school pre-
service teachers at a South African university: Implications for nuanced student
support. The profiles aided with understanding the intricacy of the themes and
how identifying changeable and non-malleable variables impacting students’
lives may influence and shape behaviors for a smooth transition to university. A
multifaceted academic support approach was proposed for student assistance in
the form of individualised and small group tutorials for students who are not
coping, along with strategies for building conceptual knowledge, student
tracking and follow-up, and improved pedagogical approaches. Similarly,
research conducted by Cage et al. (2021) opined that effective tutoring
programmes provide personalized attention, direct teaching, and address errors
and misconceptions, keeping students motivated and improving academic
success. The goal is to increase achievement, regardless of the additional
assistance’s shape (Venezia et al., 2017). Peterson et al. (2014); Omogbadegun et
al. (2014); Higgins-Opitz and Tufts (2014); and Mngomezulu and Ramrathan
(2015) in separate studies submitted that tutoring programs (individualised or
class-wide), study skills coaching, adoption of remote learning, supplemental
91
instruction, academic mentoring, and academic counselling could promote

positive learning outcomes.
Concerning social support services, Tamulien (2014) carried out a study on

designing college students’ social and psychological services to address student
needs in Lithuania. The research was carried out to determine the connection
between the academic, collegiate, vocational, and non-conformist students
enrolled in Lithuanian institutions and the need for student emotional and
psychological support services. It was revealed that first-year students’
orientation courses, language training, and community feeling development
activities are collegiate and non-conformist students’ most critical SS services. It
was noticed that the collegiate type of students also needs psychological
counselling and accommodation services. Meanwhile, academic and vocational
students require career counselling services. Awang et al. (2014) looked at first-
year college students’ perceptions of academic, SS, and well-being in their study.
It has been noted that incoming students’ capacity to receive academic assistance
from friends (supporting friendship) and families affects their ability to
transition academically, socially, and emotionally. They argued that the idea of
social support should be expanded to encompass environments other than
universities. The results of this study point to the value of older students, family
networks, and the student body in helping first-year students adjust to
university environments.
Ogbeide and Ugwu (2016) investigated the link between academic achievement
and social support among undergraduate students at Madonna University in
Nigeria. These results corroborate the linear beneficial association between SS
and academic achievement. In other words, the higher the social support
available and accessed, the better the students’ academic performance (Ratelle et
al., 2013; Poots & Cassidy, 2020). Adequate social support like excursions, a
curriculum that emphasises social learning, informal interactions between
students and staff, members of clubs and societies, sporting activities from
educational institutions to students irrespective of age, ethnicity, gender,
religion, socio-economic status can contribute to improved learning outcome as
reported in studies such as Ermer and Proulx (2020); Ramsaroop and Petersen
(2020); Aloba et al. (2021); and Eze et al. (2021). McLean et al. (2021) evaluated
perceived SS and stress in a survey of first-year students in Ireland. According to
their research, both male and female students reported moderate levels of
perceived stress, but those with more social support also reported less stress.
The universities attempting to improve SS and deal with stress may need a
gender-specific focus because female students require greater social support and
stress than male students (Ugwu, 2021). Social support statistically substantially
influenced the undergraduates’ social well-being. SS is critical in determining an
undergraduate’s social well-being (Wu et al., 2020).
Lastly, Arifin (2018) investigated how psychological support services might

improve students’ persistence in the setting of open universities: a lesson from
Indonesia. They submitted that psychological support was a significant factor in
students’ perseverance with open university systems. Students who experience
academic stress but have solid emotional support can attain greater academic
92
heights. In related studies, Chinelo (2017) and Peretomode and Ugbomeh (2018)
submitted that psychological support services are paramount in helping
undergraduates navigate their early years in the university. Mental health
professionals, counsellors, and academic advisers should be available and
adequate to provide psychological services to students. The relevance of the
quality and source of the assistance received for a student’s well-being was
examined by Maymon et al. (2019) in their study on helping first-year students
throughout their transition to higher education. Overall, the results reveal the
value of assessing first-year students’ psychological support level throughout
the transition to higher education and demonstrate that faculty/staff support is
a significant factor in student well-being.
Perspectives on emotional and psychological learning in tertiary education were

examined by Millett (2020). It was reported that emotional and psychological
abilities, reading, and numeracy are increasingly acknowledged as crucial for
success in school and beyond. Higher education institutions compete with one
another for students, but they also work together by sharing information about
what makes students successful there. Njageand and Athiemoolam (2021)
investigated the efficiency of secondary school counseling programs in assisting
students in making a smooth transition at Moi University, Kenya. Their study’s
conclusions showed that high school counseling programs do not sufficiently
assist students in preparing to transition to higher education. For a smooth
transition, the report advised the introduction of increased collaboration
between universities and schools. In a sample of Afghan university students
exposed to COVID-19, Green et al. (2021) investigated how psychological
support reduces academic stress and promotes mental health. They concluded
that receiving moderate to high levels of emotional support may mitigate the
damaging effects of greater academic focus on mental health. Ogunode (2020)
submitted that peer emotional support and regular counselling sessions with
first-year students are psychological services essential for students to attain their
educational objectives.
3. Methodology
This study adopted a qualitative research approach. Qualitative research
involves collecting and analsying non-numerical data to understand the
respondent’s experiences. This approach is appropriate because it allows the
researcher to gather in-depth insights into the students’ experiences within the
university at the point of entry. The research design appropriate for this study
was a case study design. A case study research design generates a multifaceted
understanding of a complex issue in its real-life context (McCombes, 2023). The
population for this study was all second-year student-teachers in the
Department of Adult & Primary Education, University of Ilorin. In contrast, the
target population comprised 175 second-year Early Childhood student-teachers
in the department. The respondents in this study are second-year students who
have spent one academic session (2020/2021), which should allow them to
appraise and access the available support services within the university. The
respondents were randomly selected, after which they were assigned randomly
to groups. A focus group interview method was employed to generate responses
from the participants. The respondents were classified into four groups, with
93
three participants in each group. The sample for this study was 12 randomly
selected student-teachers. Interviews using open-ended questions were used for
data collection from the respondents. The researcher made an audio recording of
the interview sessions with explicit verbal approval from each of the
participants. Reliability and validity in data analysis were observed as the first
author worked with the data, which the second author then verified. Secondly,
triangulation took place by comparing the responses from different participants
using the comparative method. All ethical considerations/parameters were
adhered to. Data generated were analysed using thematic content analysis. The
researchers opted to adopt Braun and Clarke’s thematic analysis method, a
circular procedure involving the following steps. Becoming familiar with the
information, creating cyphers, creating, assessing, describing themes, and
locating patterns.
4. Findings
In this section, the researchers present the interview results with the
respondents. The respondents were five male and seven female students in the
department who provided insight into the three themes that guide this study.
Theme one: Negative Academic and Social Experiences Influences Students’

Well-being
This theme provides a searchlight into the academic, social, and psycho-
emotional challenges experienced by the respondents in their first years in the
institution.
In discussing the academic challenges, inconsistency in keeping the timetable of

lectures and lecturers was identified. As stipulated by a female respondent from
group 1 “part of the challenges that I faced academically was the constant sudden
change of lecture period after so many struggles to get to lecture areas and front seats
due to overcrowded lecturer room...and after all that ‘wahala’ (struggles), the lecturer
would end up not showing up”. Strike actions by lecturers was another academic
difficulty as stated by a female respondent from group 3 “the strike action
embarked upon by lecturers created a serious issue for her academically. As a result of
the strike action proposed by federal lecturers, the university shut lectures. Immediately
school opened like this, examination timetable released. I didn’t know what hit me, like
play like play, examination starts, and my performance was not encouraging”. The
respondents also identified misplacement and missing academic results. A male
respondent from group 4 said “the most trying academically difficulty encountered as
a first-year student was missing examination result, which subsequently meant that I
failed the course. It was a stressful time for me, and I even thought I was cursed after
four years of being at home. And now, battling with missing scores (sighs). Finally, after
stressing the lecturer in charge for months, the script was found, and my grade was
released. However, the experience had already taken a toll on my academic activities”.
Other issues that affected the respondents in their first year were delays in the
release of results, clash of timetables, poor teaching methodologies, and
compulsory imposition of textbook handouts by lecturers.
Regarding social challenges, the respondents encountered unfriendly lecturer-

student interaction, peer pressure, financial challenges, sexual harassment,
94
cultural diversities, insecurity, and homesickness. Other issues were

inadequacies in the transportation system. A female respondent from group 2
said, “transportation was my social achilles heel… the excitement of being an
undergraduate quickly disappeared on the first day I attended classes. Two hundred
buses could not take the number of people I saw at the bus stop to school. After so much
shunting, struggling, and begging, I was able to get to campus around 11am since
7:45am that I got there. This was a regular occurrence for the better part of the first
semester. I was already exhausted most times on my arrival in class… and this has even
discouraged me so many times from going to school, and as a result, I used to miss
numerous classes”. Also identified by A male respondent from group 3 was
accommodation issues “securing appropriate accommodation was not an easy task.
The hostel operated by the university could not cater to a large chunk of the student
population, while the privately controlled hostels were too expensive. Before getting to
my apartment, I had to squat with a coursemate for a few weeks. The room was
inadequate for two people to live in, not to talk of four male students. I secured a room
off-campus, but there was no power supply, water and difficulty getting to school due to
insufficient transportation were matters arising”. In the same vein, difficulty in
making friends, as stated by a female respondent from group 4 “growing up, it
was difficult to interact with people who she was not related to, and it takes a lot of time
to make friends…So, being a first-year student at the university was difficult, and I had
to make friends to avoid being isolated. I was found at club, bars, and any other event
that had music and drinks…. It became so bad that I got mixed up with the wrong
crowd. It almost got me thrown out of school… However, that’s all behind me now”.
Lastly, the researchers gathered that psychological intimidation, procrastination,

peer pressure, and students’ ambition were some of the psycho-emotional
challenges encountered by the respondents. Other challenges include lack of
family support and depression “admitted to study primary education in the
university was a waste of time according to relations and family members. When you
have been denied admission for five years, securing an admission offer into any
programme is a welcome development. The lack of support from people closest to me got
me into a state of depression and used different substances as a relief... I was able to get
assistance from one of my lecturers” as submitted by a male respondent from group
2. Other challenges mentioned by the respondents were low self-esteem, external
academic pressure, and suicidal ideation. A group 1 male respondent added, “I
come from a more competitive family than cooperative…, where every sibling
achievement is designed to challenge the rest of us”. The pressure is unbearable. After
the first semester examinations and results were released, my performance was not too
bad, but it was also not to the standard set for me. I had self-esteem issues and later
suicidal ideation. I believed it was better for me to be ‘dead’ rather than face the
disappointment from my family… Obviously, I didn’t go through (smiles) with it, but I
was very close”. A female respondent from group 3 also reported that academic
stress and examination sickness, “the rate at which they bombard us with lectures
and tests before exam commences usually increases the tension…I remember this
pressure made me feel sick during the exam, to the extent that I missed two exams while
on admission at the school clinic during second-semester examinations” were psycho-
emotional challenges encountered as a first-year student.
95
Theme Two: Hoping for the experience to be better than anticipated

Based on the challenges highlighted in theme 1, the respondents reported on the
support services (academic, social, and psycho-emotional) available and utilised
in the institution. These services made their early years issues less dreadful.
Relating to academic support services, some respondents opined those extra

classes by lecturers and tutorials from student associations, and coursemates
assisted them in surviving their first years academically. As stated by a female
respondent from group 1 “If not for the extra classes and tutorial organised by some
teaching assistants and higher-level students, I wouldn’t have understood many of my
courses offered and even passed some of my first-semester examinations...Kudos to the
student association”. Access to library facility (both conventional and e-library
facilities) “the university has a massive building dedicated for library purposes. There is
also the e-library, and construction on another building (library annex) has been
completed. Library and e-library are accessible to all students with their university
identity cards” were also available, as reported by a female respondent from
group 3. Likewise, the respondents also submitted that internet facilities and
access to online learning materials were provided, and this made learning
interesting. To state it clearly, a male respondent from group 2 shared that
“wireless internet facilities are available at the lecture area, around the hostels, and office
buildings. it is accessible to all students at the university…I mostly make use of it to do
my assignments”.
On social support services provided by the institution, the respondents

commented that the departmental student association helped with issues
relating to course registration, timetable issues, and locating lecturers’ offices
and lecture halls “upon my arrival on campus, I did not know what was expected of me
or where to go, until I was directed to the department student association. I was able to
meet my level adviser, my coursemates and other necessary information I required to get
my footing” as reported by a female respondent from group 2. Another support
provided by the institution was the regulation of on and off campus
accommodation by the student affairs unit. A male respondent from group 1
said “in my first few weeks as an undergraduate in the university, there was an issue of
theft in my off-campus hostel. The case was reported at the student affairs unit, and it
got resolved…Onwards, I understood the role of the unit”. Other social support
services coordinated and regulated by the unit are school-owned and private
transportation, extra-curricular activities, organizing of orientation programmes,
registering of students’ associations (university, faculty, and department), social,
religious and political societies. While other respondents submitted that the
school’s security unit provided protection from harassment, theft and general
safety on-campus. In the words of another female respondent from group 4
added “during examinations the presence of the university’s security details helps to
secure my possessions… On a rare occasion, an individual tried taking someone else’s
possession, he was caught, and appropriate punishment was meted out”.
With regard to the psycho-emotional support services provided by the

institution, the researchers enquired more from the respondents. This was based
on the emotional/psychological challenges (examination anxiety, depression,
and suicidal ideation) reported by the respondents in theme 1. On the issue of
96
examination anxiety, the female respondent from group 3 reported that due to
my concern, my level adviser directed me to the faculty counsellor... He recommended
techniques in helping me manage my anxiety”. While the male respondent from
group 2 submitted on the issue of depression, “regular sessions were organised with
my cleric to measure my progress”. In conclusion, the male respondent from group
3 who spoke about suicidal ideation said, “I did not go through with the suicidal
ideation due to the help I am receiving from a lecturer in counselling education”.
Theme Three: More help required, but it’s a start

This theme discusses the adequacy of the respondents’ academic, social, and
psycho-emotional support services as stipulated in theme 2.
Many respondents conclude that academic support services available and

utilised in the institution require serious improvement. “The university library is
equipped with mostly outdated materials… while the e-library is also not always useful
due to insufficient computers. Sometimes, students must wait for other students to finish
with their tasks…And the library is closed by a certain time which hinders student’s
academic productivity”. On the other hand, “due to congestion and overcrowding, the
internet facilities within the institution are inadequate and epileptic. I had to either
subscribe personally with my mobile carrier to the internet, stay late in school or go to
school on weekends depending on the urgency in submitting the assignments”. It was
also gathered that the tutorials were not an official activity by the university. It
was not readily available and is primarily dependent on the tutors, which
eliminates individualised instruction to assist struggling students.
Their assessment was unanimous on the adequacy of the social support services
the respondents benefitted from in their first year. They rated the services as
mildly adequate. The submission of some respondents as it relates to state of the
campus hostel “the experience in the school-run hostel was not encouraging… the
rooms did not have reading area, functional fans, and proper lighting. The bathrooms
and toilets did not have water supply and were not properly cleaned. Students were
treated at the school clinic for different forms of infection. Some students moved out of
the hostel”. While others reported that “staying off-campus is demanding due to
incessant power outages and theft. The buses available for transportation were limited,
and this created untold hardships. We had to leave home hours before classes to get
public transport. Otherwise, we had to trek or ask for lift from private motorist (PS or
Tanke Sir). It was the same situation after classes. We are fatigued before classes and too
tired to do assignments in the evening…Things improved in the second semester”.
Whereas the psycho-emotional support services provided for students is less

than adequate. Factors such as insufficient manpower, inadequate space for
sessions, inexperienced counsellors, and unawareness of the counselling unit
due to inadequate information are some of the reasons for this submission by the
respondents. As responded by the students, “except one is directed to the
counselling unit, it is difficult to be aware of its existence due to its location…On the
other hand, the faculty counsellor’s office does not assure a safe space”.
97
5. Discussion
It is the opinion of the first researcher that some of the academic and social
challenges these first-year students face could be attributed to the system of
secondary education and the upbringing they experienced. These students are
trained in a rigid and controlled system where such schools’ authorities
determine learning decisions. Due to the flexibility in the system of the
university, they struggle. Parents and guardians also contribute to their wards’
struggles as they do not allow them to make decisions independently while in a
lower educational level. It is only logical that these students struggle in the
university since they do not have the luxury of their secondary school teachers,
and parents to make decisions on their behalf. Transiting from one educational
level to another is bound to create challenges, the important thing is either to
find solutions or adapt quickly. This study’s findings correlate with the
assertions of Wangeri et al. (2012), Millet (2015), and Brooker et al. (2017), who
submitted in different studies that students face separate struggles during their
early years in any institution. This shows that difficulties in adjusting are
expected in the early years of university students, and institutions should put
necessary mechanisms in place for solutions. For students to attain their
educational objectives, they must quickly adapt to the environment of the
institution in all ramifications. The challenges undergraduates face in their first
year can substantively impact their academic productivity if adequate steps are
not undertaken.
This study also found that students’ learning might be threatened due to
inadequate facilities and spaces, which are crucial components in attaining
academic excellence. What then is the essence of attending a university where
appropriate facilities to promote learning and ease students’ academic stress are
not readily available? It means that in addition to doing well academically, they
must also be burdened with self-providing materials and resources for learning.
State-of-the-art teaching and learning tools, internet services, an efficient system
for monitoring students’ progress, 24-hour power supply, and adequate internet
services are facilities the university is expected to provide, creating an
environment where learning is continuous, comprehensive, and robust.
Researchers such as Murillo et al. (2011); Amsterdam (2013); Alkadri et al. (2017);
and Marmoah et al. (2019) have all carried out their separate studies and found
that provision of necessary academic services should increase educational
productivity which the findings of this study support.
First-year undergraduates’ transition from home and secondary education to

tertiary education is vital to attaining their academic objectives. Institutions are
encouraged to provide an environment that promotes empathy, trust, care,
hope, and love among and between undergraduates to enhance social and
academic life quality. By so doing, an atmosphere that promotes immersion in
sporting activities, socio-cultural participation, religious tolerance, and other co-
curricular activities to reduce academic pressure and stress. University
education should not be totally about academic endeavours alone, but it is an
environment that should promote the social experiences of students. When a
student focuses solely on academics, they lose out on exploring the social culture
within the institution. Social interactions help individuals to understand the
98
social psychology of people. The findings of Ermer and Proulx- (2020);

Ramsaroop and Petersen (2020); Aloba et al. (2021), and Eze et al. (2021) have
been supported by this present study. The provision of appropriate and
adequate social support services can contribute to improved learning outcomes.
The emotional needs of undergraduates should be efficiently met, which results

in the elimination of serious roadblocks to educational goals. Educational
institutions should assist students in developing the emotional fortitude
required to succeed inside and outside the classroom. Equipping students in the
process of navigating sticky social challenges, either self-inflicted or due to
external factors, should be areas of emotional support institutions should
provide. There are incidences where undergraduates have committed suicide
because they failed a course, did not meet up with expectations, or did not
graduate with their coursemates. If the appropriate counselling services were
provided from their first year, there is a possibility of reduced incidences of
suicide cases. The responsibility does not lie solely with the university; parents
and guardians have also left the training and upbringing of their children in the
hands of teachers, house helpers, and caregivers.
The findings of this study show that the absence of parental support in
providing appropriate guidance for their children is a serious emotional red flag.
As a black person, we are trained to believe that asking for assistance or talking
about one’s challenges is a sign of weakness. This view is problematic, resulting
in increasing rates of suicide among youths. Parents should also provide
emotional support to complement the psychological support services available
at the university. It is pertinent that administrators should have individuals who
have the required expertise to guide students through challenging situations.
Students who experience academic stress but have strong emotional support can
accomplish better academic heights (Chinelo, 2017; Peretomode & Ugbomeh,
2018; Njageand & Athiemoolam, 2021).
6. Conclusion
From the interview, the researchers gathered that the first challenge some
respondents had before stepping on campus was the conflict of accepting or
rejecting the admission offer. In some ways, this conflict contributed to the
difficulties (academic, social, and psycho-emotional) they encountered in their
first year at the university. Many respondents opted for their present course of
study due to the rejection of their preferred programme. This shows that from
the onset, they were not open-minded about their programme, because they
believed only students who were not good enough studied education-related
courses. On the other hand, a few respondents submitted that they applied for
admission into their desired courses at separate institutions and faculties while
holding on to their present admission offer. This situation had varying impacts
on their academic, social, and psychological well-being. They added that since
they were not going to be completing the degree in education, there was no need
to take the course seriously or create relationships with other students. Some
added that they were not in the right state of mind to explore the opportunities
and possibilities available within the institution. The researchers also gathered
that due to the unwillingness of most students in education-related
99
programmes, some engaged in dangerous and reckless activities, and it

impacted their learning outcomes. As a result of the disappointment of not
performing as expected (either due to internal or external motivation), they
failed to get the required assistance (reluctance to ask for help or
unprofessionalism by the support services staff. The first author insisted that the
institution is handicapped and doing its best with the limited resources
provided by supervisory agencies. Furthermore, the institution is underfunded
and understaffed, which overstretches the available staff.
7. Limitations and Recommendations

The study provides valuable insights into student-teachers’ challenges in their
first year and highlights the need for improved support systems. However, it is
essential to acknowledge certain limitations that may impact the generalizability
and applicability of the findings. Firstly, the study focused on a specific cohort of
student-teachers from a particular educational institution, which limits its ability
to capture diverse experiences across different contexts and settings.
Additionally, the study’s sample size was relatively small, which may restrict
the representativeness of the findings. Furthermore, the study relied heavily on
self-reported data, introducing potential biases and subjectivity. Lastly, resource
constraints and time limitations are also presented as limitations in this research.
Therefore, it was recommended that tertiary institutions should make adequate

provision for mental health practitioners to guide undergraduates, especially
first-year students, in navigating their academic journey. Supervisory agencies
should provide necessary assistance to these institutions, which allows them to
provide required support services to their students. The institution should
formally provide adequate academic assistance to assist weak learners. The
institution should strongly encourage and reward participation in sporting and
social activities. Institutions should only offer admission to students who apply
for education-related courses.
8. References
Alabi, A. T. (2018). Evaluation of the impact of Universal Basic Education process on
primary school enrolment in Kwara state. Nigerian Journal of Educational Research
and Evaluation, 4(1), 7-12.
Alkadri, H., Ningrum, T. A., Santoso, Y., & Afriansyah, H. (2017). Essentiality of
management of facilities and infrastructure toward a number of students of
early years institution. Paper presented at the International Conference of Early
Childhood Education (ICECE 2017). https://doi.org/10.2991/icece-17.2018.18
Aloba, O., Opakunle, T., & Ogunrinu, O. (2019). Psychometric characteristics and
measurement invariance across genders of the multidimensional scale of
perceived social support (MSPSS) among Nigerian adolescents. Health Psychology
Report, 7(1), 76-83. https://doi.org/10.5114/hpr.2019.82629
Amirkhan, J. H. & Kofman, Y. B. (2018). Stress overload as a red flag for freshman failure
and attrition. Contemporary Educational Psychology, 54, 297-308.
https://doi.org/10.1016/j.cedpsych.2018.07.004
Amsterdam, C. (2013). School infrastructure in South Africa: Views and experiences of
educators and learners. Paper presented at the Conference Paper: International
Conference on Education, June 4-8.
100
Arifin, M. H. (2018). The role of student support services in enhancing student

persistence in the open university context: lesson from Indonesia open
university. Turkish Online Journal of Distance Education-TOJDE, 19(3), 156-168
Awang, M. M., Kutty, F. M. & Ahmad, A. (2014). Perceived social support and well-
being: First-Year student experience in university. International Education Studies,
7(13), 261-270. https://doi.org/10.5539/ies.v7n13p261
Borishade, T. T., Ogunnaike, O. O., Salau, O., Motilewa, B. D., & Dirisu, J. I. (2021).
Assessing the relationship among service quality, student satisfaction and
loyalty: the NIGERIAN higher education experience. Heliyon, 7(7), 1-8.
https://doi.org/10.1016/j.heliyon.2021.e07590
Braun, V. & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research
in Psychology, 3(2), 77-101. https://doi.org/10.1191/1478088706qp063oa
Brooker, A., Brooker, S. & Lawrence, J. (2017). First year students’ perceptions of their
difficulties. Student Success, 8(1), 49-62. https://doi.org/10.5204/ssj.v8i1.352
Busaba, B., Tanawastien, S. & Tantaswadi, P. (2019). Student support services with
remedial management system. TEM Journal, 8(2), 610-616.
Cage, E., Jones, E., Ryan, G., Hughes, G. & Spanner, L. (2021). Student mental health and
transitions into, through and out of university: Student and staff perspectives.
Journal of Further and Higher Education, 1, 1–14.
https://doi.org/10.1080/0309877x.2021.1875203
Chandra, Y. (2020). Online education during COVID-19: Perception of academic stress
and emotional intelligence coping strategies among college students. Asian
Education and Development Studies, 10, 229–238.
Chinelo, O. D (2017). An analysis of psychological problems encountered by
postgraduate students in Nigerian universities. Journal of Social Science, 22(2), 75-
83.
Chou, S. C. & Chang, C. C. (2019). The relationship between academic stress and health
status – The moderating role of social support. In L. Rønningsbakk, T. T. Wu, F.
Sandnes & Y. M. Huang (Eds.), Innovative technologies and learning. ICITL 2019.
Lecture notes in computer science (Vol. 11937). Springer.
Conley, D. T. (2017). The new complexity of readiness for college and careers. In K. L.
McClarty, K. D. Mattern & M. N. Gaertner (Eds.), Preparing Students for College
and Careers (pp. 9–22). Routledge.
Conolly, A. & Lampe, M. (2016). How an active learning classroom transformed IT
executive management. International Systems Education Journal (ISEDJ), 14(1), 25-
32.
Edwards, D., Hawker, C., Carrier, J. & Rees, C. (2015). A systematic review of the
effectiveness of strategies and interventions to improve the transition from
student to newly qualified nurses. International Journal of Nursing Studies, 52(7),
1254–1268. https://doi.org/10.1016/j.ijnurstu.2015.03.007
Ene, I. E. (2016). Quality assurance in Nigerian tertiary institutions: A significant factor
for sustainable democracy. International Journal of Education and Research, 4(7), 98-
116.
Ermer, A. E., & Proulx, C. M. (2020). Social support and well-being among older adult
married couples: A dyadic perspective. Journal of Social and Personal Relationships,
37(4), 8-15.
Eze, J. E., Chukwuorji, J. B. C., Ettu, P. C., Zacchaeus, E. A., Iorfa, S. K., & Nwonyi, S. K.
(2021). Bullying and suicide ideation: Testing the buffering hypothesis of social
support in a Sub-Saharan African Sample. Journal of Child and Adolescent Trauma,
14(1), 75-82. https://doi.org/10.1007/s40653-019-00294-w
Federal Republic of Nigeria (2014). National Policy on Education. Lagos: NERDC Press
101
Green, Z. A. (2021). Strengthening career adaptation among school teachers in Pakistan:

Test of strengths-based career intervention imparted through emotionalized
learning experiences. International Journal for Educational and Vocational Guidance,
6, 5-17.
Green, Z. A., Faizi, F. Jalal, R. & Zadran, Z. (2021). Emotional support received
moderates academic stress and mental well-being in a sample of Afghan
university students amid COVID-19. International Journal of Social Psychiatry, 1, 1-
8. https://doi.org/10.1177/00207640211057729
Higgins-Opitz, S.B. & Tufts, M. (2014). Performance of first-year health sciences students
in a large, diverse, multidisciplinary, first-semester, physiology service module.
Advances in Physiology Education, 38, 161–169.
https://doi.org/10.1152/advan.00067.2013
Hyun, J., Ediger, R., & Lee, D. (2017). Students’ satisfaction with their learning process in
active learning and traditional classrooms. International Journal of Teaching and
Learning in Higher Education, 29(1), 108-118.
Marmoah, S., Adela, D., & Fauziah, m. (2019). Implementation of facilities and
infrastructure management in public elementary schools. Al-Tanzim: Jurnal
Manajemen Pendidikan Islam, 3(1), 102-134. https://doi.org/10.33650/al-
tanzim.v3i1.507
Maymon, R., Hall, N. C. & Harley, J. M. (2019). Supporting first-year students during the
transition to higher education: The importance of quality and source of received
support for student well-being. Student Succes Journal, 10(3), 64-79.
https://doi.org/10.5204/ssj.v10i3.1407
McCombes, S. (2023). What is a case study? Definition, examples & methods. Retrieved
from https://www.scribbr.com/methodology/case-study/
McLean, L., Gaul, D. & Penco, R. (2021). Perceived social support and stress: A study of
1st year students in Ireland. International Journal of Mental Health and Addiction,
1(1), 1-21. https://doi.org/10.1007/s11469-021-00710-z
Millett, C. M. (2020). Perspectives on social and emotional learning in tertiary education.
Policy Information Report and ETS Research Report Series. Retrieved from
www.ets.org/research/perc
Mngomezulu, S. & Ramrathan, L. (2015). Academic Intervention Experiences of ‘at-risk’
students in a South African university. Interdisciplinary Journal for the Study of the
Arts and Humanities in Southern Africa, 17, 116–141.
Murillo, F. J., Román, M. J. S. E., & Imapa, s. (2011). School infrastructure and resources
do matter: analysis of the incidence of school resources on the performance of
Latin American students. Social Sciences Journal, 22(1), 29-50.
https://doi.org/10.1080/09243453.2010.543538
Naidoo, A. & Lemmens, J. (2015). Faculty intervention as support for first-year students.
Journal of Student Affairs in Africa, 3(2), 17-32.
Nelson-Royes, A. M. (2013). Tutors can improve students’ reading skills. Reading
Improvement, 50(2), 48-53. https://doi.org/10.14426/jsaa.v3i2.133
Njageand, R. & Athiemoolam, L. (2021). Effectiveness of secondary school guidance
programs in preparing learners for successful transition to university: A case of
Moi University. European Journal of Education and Pedagogy, 2(3), 88-95.
https://doi.org/10.24018/ejedu.2021.2.3.84
Nugroho, A. A. & Wibowo, U. B. (2019). The influence of school infrastructure on
student learning activeness: A research study. Advances in Social Science,
Education and Humanities Research, 397, 607-612.
https://doi.org/10.2991/assehr.k.200129.076
102
Odutayo, A. O. (2021). Assessment of the content knowledge and pedagogical skills of

university economics education students in South-west, Nigeria. [Unpublished PhD
Thesis]. Department of Social Sciences Education, University of Ilorin.
Ogbeide, E. D. O. & Ugwu, C. C. (2016). Relationship between social support and
academic performance among undergraduate students of Madonna University,
Nigeria. Global Journal of Applied, Management and Social Sciences (GOJAMSS), 13,
70-77.
Ogunode, N. J. (2020). An investigation into the causes of unstable academic calendar in
higher institutions: A case study of Federal University Wukari, Taraba, Nigeria.
Social Science Researcher, 6(2), 78-91.
Ogunojemite, G. B. (2020). Importance of teacher education in the teaching profession in
Nigeria. In Oyinloye, G. O. & Adeosun, O. V. (eds). Fundamentals of Curriculum
Studies (1-5). Universal Publishers.
Ogunyinka, E. K., Okeke, T. I., & Adedoyin, R. C. (2015). Teacher education and
development in Nigeria: An analysis of reforms, challenges, and prospects.
Education Journal, 4(3), 111-122.
Omogbadegun, Z., Oyedepo, T., Fasina, F. & Omotosho, E. (2014). Student support
services infrastructural framework for excellent academic performance in
tertiary institutions: A case study of Covenant University, Nigeria. Proceedings of
EDULEARN14 Conference 7th-9th July 2014, Barcelona, Spain
Oyekan, S.O. (2006). Foundations of teacher education. Ben Quality Prints.
Paideya, V. & Bengesai, A. (2017). Academic support at the University of KwaZulu-
Natal: A systematic review of peer-reviewed journal articles, 2010–2015. Journal
of Student Affairs in Africa, 5(2), 55–74.
Peretomode, V. F & Ugbomeh, A. N. (2018). Psychological problems faced by
undergraduate and diploma students and their survival strategies in Nigeria: An
Institutional Analysis. International Journal of Educational Planning &
Administration, 3(1), 25- 33.
Peterson, R. L., O’Connor, A. & Strawhun, J. (2014). Academic supports and tutoring.
Retrieved from http://k12engagement.unl.edu.
Poots, A. & Cassidy, T. (2020). Academic expectation, self-compassion, psychological
capital, social support, and student well-being. International Journal of Educational
Research, 99, 25-34. https://doi.org/10.1016/j.ijer.2019.101506
Ramsaroop, S. & Petersen, N. (2020). Portraits of primary school pre-service teachers at a
South African university: Implications for nuanced student support. Journal of
Education, 81, 82-98. https://doi.org/10.17159/2520-9868/i81a05
Ratelle, C. F., Simard, K., & Guay, F. (2013). University students’ academic well-being:
The role of autonomy support from parents, friends, and the romantic partner.
Journal of Happiness Studies, 14(3), 2-9. https://doi.org/10.1007/s10902-012-9360-
4
Scanlon, M., Leahy, P., Jenkinson, H. & Powell, F. (2020). My biggest fear was whether or
not I would make friends’: Working-class students’ reflections on their transition
to university in Ireland. Journal of Further and Higher Education, 44(6), 753–765.
https://doi.org/10.1080/0309877x.2019.1597030
Sosu, E. M. & Pheunpha, P. (2019). Trajectory of university dropout: Investigating the
cumulative effect of academic vulnerability and proximity to family support.
Frontiers in Education, 4, 6-17. https://doi.org/10.3389/feduc.2019.00006
Subair, S. T., Okotoni, C. A. & Adebakin, A. B. (2012). Perceived quality of infrastructure
in selected Nigerian universities. Makerere Journal of Higher Education, 4(1), 111-
124. https://doi.org/10.4314/majohe.v4i1.9
103
Tamuliene, R. (2014). Adjusting college students’ support services to students’ type:

Lithuania’s case. Procedia - Social and Behavioral Sciences 141, 438-446.
Ugwu, K. T. (2021). Social support as a predictor of social well-being of undergraduates
in Enugu State. International Journal of the Arts and Sciences, (IJOTAS), 4(3), 69-85.
Venezia, A., Jaeger, L., Venezia, A. & Jaeger, L. (2017). Transitions from high school to
college. The Future of Children, 23(1), 117–136.
https://doi.org/10.1353/foc.2013.0004
Wangeri, T., Kimani, E. & Mutweleli, S. M. (2012). Transitional challenges facing
university first year students in Kenyan public universities: A case of Kenyatta
University. Interdisciplinary Review of Economics and Management, 2(1), 41-50
Wu, T. J., Wang, L. Y., Gao, J. Y., & Wei, A. P. (2020). Social support and well-being of
Chinese special education teachers—an emotional labor perspective.
International Journal of Environmental Research and Public Health, 17(18), 34-41.
Yusuf, H. T. (2010). An evaluation of concurrent and consecutive university-based teacher-
education curricula in Kwara, Oyo and Osun states, Nigeria. [Unpublished PhD
Thesis]. Faculty of education, University of Ilorin, Nigeria.
Yusuf, M. A. (2014). Professionalisation of teaching through functional teacher education
in Nigeria. European Scientific Journal, 10(4), 28-34.
Zhirnova, G. I., & Absalyamova, S. G. (2013). Global innovation gap and quality of education.
Paper presented at the 2013 International Conference on Interactive
Collaborative Learning (ICL). https://doi.org/10.1109/icl.2013.6644558
104

Vol. 22, No. 7, pp. 104-121, July 2023
Received Mar 16, 2023; Revised Jul 7, 2023; Accepted Jul 14, 2023
Stakeholders’ Perspectives of Early Childhood

Education Language and Literacy Laboratories
in the United Arab Emirates
Aysha AlShamsi*
Higher Colleges of Technology, Al Ain
Phil Quirke
Higher Colleges of Technology, Abu Dhabi
Abstract. Language and literacy laboratories (LLLs) have been

instrumental in the preparation of early childhood education (ECE), as
reading is considered pivotal for novice learners. This study focused on
the perspectives of pre- and in-service teachers and their instructors on
the level of readiness to teach literacy through LLLs using the
professional development school model of teacher education. The study
employed a pragmatism-driven sequential mixed-methods research
design consisting of a self-administered survey of 720 respondents who
were randomly selected during the 2021–2022 academic year. In-depth
interviews were subsequently conducted with 42 pre- and in-service
teachers and their instructors. Following the descriptive analysis of the
quantitative data, the qualitative data were thematically analyzed. The
three emergent themes were participants’ experiences in the ECE
program, their views on the implementation of LLLs, and the
effectiveness of literacy strategies. The paper concludes with
recommendations for the strategic implementation of LLLs and adoption
of multiple language strategies for language teaching and assessment.
Keywords: early childhood education; language and literacy laboratories;

professional development school model; United Arab Emirates
1. Introduction
In the United Arab Emirates (UAE), early childhood education (ECE) contributes
to supporting the National Agenda for Education as well as the success of
initiatives such as the Program for International Student Assessment (PISA) and
the Trends in International Mathematics and Science Study (TIMSS) (United
Nations, 2017). It is important that learning environments offer language-learning
opportunities; thus, language and literacy laboratories (LLLs) are crucial in ECE
*
Corresponding author: Aysha AlShamsi, aalshamsi4@hct.ac.ae
©Authors
105
teacher preparation and support during field-based experiences. According to

Maxwell et al. (2018), LLLs allow universities and elementary schools to
collaborate and to enhance teaching and collaborative skills. Moreover, teachers
can apply what they have learnt in these laboratories.
A central pillar of ECE programs involves investment in preparing high-quality,

motivated teachers who possess competencies and up-to-date knowledge of the
curriculum to effectively develop learners’ language and literacy skills (Pianta et
al., 2016). Literacy programs support children with different abilities and
additional needs. Investment in innovative teacher training and capacity-building
initiatives is therefore essential, because teaching is a multifaceted activity that
requires a variety of skills and knowledge (Bransford et al., 2005; Stürmer et al.,
2016). It is also important to implement customized and contextualized
instructional strategies that match individual student needs (AlShamsi et al.,
2022). Because of the demands inherent in these roles, pre-service teachers (PSTs)
often feel unprepared or experience high levels of anxiety while undergoing
classroom practice (Ismail & Jarrah, 2019).
Some of this anxiety may be attributed to the educational preparation of bilingual

Emirati children (AlShamsi & Alsheikh, 2022). ECE programs in the UAE target
4- to 9-year-old children and aim at offering high-quality instruction (von
Suchodoletz et al., 2020). Over the years, ECE in the UAE has increasingly
recruited teachers from the West to fill the gap in the shortage of teachers (von
Suchodoletz et al., 2020). Naturally, these expatriates also bring their cultural
experiences, which influence the teaching phases of the education system.
Consequently, more strategic efforts are needed in the preparation of teachers in
ECE, which rationalizes the importance of LLLs to enhance teachers’ preparation
for the classroom.
Professional development models of teacher education are intended to facilitate

the creation of communities of practice to assist trainee teachers. School–college
partnerships are used worldwide to develop confidence and field skills among
these trainees. One key aspect of the model is that it enables higher education
institutions (HEIs) to provide trainee teachers with placement opportunities in
schools, which exposes them to authentic teaching and learning (Dietrichson et
al., 2021; Teitel, 2003). These placements are often supported by the colleges’
literacy laboratories, especially when placement becomes a mandatory
requirement of their degree (Maxwell et al., 2018). Given the success of such
laboratories in various countries (Walker et al., 2020), the establishment of LLLs
in the UAE, in collaboration with the educational regulatory bodies, seems
appropriate as an essential part of a teacher’s college education.
Therefore, this pragmatic sequential mixed-methods study aimed to identify the

factors that contribute to the scope of LLLs in the UAE from the perspective of key
stakeholders, comprising in-service teachers (ISTs) and PSTs and their instructors.
The study further aimed to explore these results in depth through qualitative
analysis. To achieve these objectives, the following research questions were
formulated:
106
1. What factors do ECE stakeholders consider pivotal in implementing

language and literacy teaching and assessment strategies?
2. How instrumental are LLLs in international ECE programs?
3. In what ways can educational partnerships enhance LLLs and the teaching
and assessment practices of education stakeholders?
1.1 The Present Study

The study context is an HEI in the UAE that offers an applied ECE teaching
program. This program focuses on the skills and competency development of ECE
pre-service homeroom teachers of English language, mathematics, and science.
PSTs begin their teaching practicum (TP) as early as Year One. It is in these cohort-
specific practicum courses that PSTs begin learning how to link theory to the
practicum experience. To facilitate this process, they are assigned specific tasks
and supervised throughout their TP experience. During TP, PSTs are expected to
plan and teach lessons based on their semester level and to provide reflections
after each lesson taught. The TP coordinator serves as the liaison between the HEI
and the UAE’s education regulatory bodies. The TP coordinator also assigns each
PST a college mentor teacher (MCT) and organizes school placements according
to the regulatory bodies, who then assign each student to a specific grade-level
school mentor teacher (MST). This authentic experience provides each PST with a
rich opportunity to link theory to practice. It also serves as a supportive practice
alongside the PST’s experience with LLLs to enable teaching efficacy and
knowledge acquisition and better define factors that predict success through
professional development.
1.2 Professional Development School Model

The professional development school (PDS) model of teacher education is
associated with the Holmes Group (1986, 1990). The group’s central intent was the
development of the PDS model, which recognizes the associations between
teaching and teaching education and the need to create partnerships to improve
teaching and learning for prospective teacher education candidates and students.
Researchers such as Ball and Cohen (1999) supported the PDS model, as they felt
the need for a more pragmatic model of teacher professional development (TPD).
Most TPD models differed from the theoretical and pedagogical contexts and
were not helpful to teachers during their TP (Ball & Cohen, 1999). Teitel (2003)
opined that a PDS model should involve groups of PSTs working jointly as a
cohort who are placed in different schools with different teachers over the long
term and for interactive internships. To test the effectiveness of the PDS model,
Sandoval-Lucero et al. (2011) examined the experiences of three groups of PSTs,
who showed sufficient theoretical and practical experience during their school
placements. The early learning environment (Walker et al., 2020) and teacher
efficacy (Haverback & Parault, 2011) are pivotal for language intervention and
communication. Therefore, it is reasonable to expect that teachers who are
responsible for teaching literacy are well prepared to identify difficulties and
select appropriate intervention strategies.
The effectiveness of the PDS model in supporting literacy for PSTs is a topical
subject in education literature. Lefever-Davis and Heller (2003) argued that PSTs
should learn with children and practice the art of teaching, reading, and writing.
107
While examining a Teach for America program, Gabriel (2011) applied the PDS
model and concluded that it was pragmatic, accommodative, and enabled
teachers to address the needs of diverse learners within their environments. More
recently, Herro et al. (2019) described a faculty-in-residence program that applied
a practice-based PDS model to a sample of three STEAM (science, technology,
engineering, art, and mathematics) teachers with different needs. The authors
concluded that this PDS model was more responsive to the needs of learners, as it
helped teachers finetune their instructional practices. This study modelled these
approaches while being guided by the PDS model in supporting PSTs and their
use of LLLs established on campuses and in schools in the UAE. This research is
expected to benefit the ongoing mutual collaboration among MSTs, college
instructors, ECE PSTs, and their MCTs. Examination of these aspects of the PDS
model within the teaching education of the UAE may help better define the factors
that predict success for new teachers.
2.1 Language and Literacy Teaching and Assessment Strategies
Several strategies are adopted to enhance language and literacy teaching among
preschoolers, including code-focused strategies, language enhancement
interventions, and reading techniques. Reading is proposed as one strategy that
builds children’s vocabulary; therefore, teachers’ comments during reading
sessions can further augment their language and reading skills (Barnes &
Dickinson, 2017). Regarding language assessment, Lam (2015) observed that
language teachers from Hong Kong were hindered in their assessment of
language and literacy because of under-preparedness due to inadequate training,
which is exacerbated by the use of traditional standardized tests. Notably, Xu
et al. (2014) found that both summative and formative assessment methods were
crucial in teachers’ planning of language lessons. Assessment becomes more
critical at the kindergarten level, as literacy serves as the foundation of future
reading skills.
When executed effectively, language and literacy teaching and assessment

strategies are effective in enhancing students’ acquisition of valuable skills. These
skills relate to alphabet knowledge, phonological awareness, rapid automatic
naming, writing, and phonological memory, among other skills. In their study of
220 Estonian-speaking primary school learners, Kasper et al. (2018) established
that prioritizing strategies that develop reading interests is useful for reading and
vocabulary acquisition. Moreover, in a preschool setting in the United States,
Davison and Qi (2017) observed that shared book reading and intentional
exposure to vocabulary in context were useful for preschoolers, especially those
keen on second language acquisition. Focusing on teaching strategies that might
enhance the acquisition of communication skills among preschoolers is important.
Therefore, any additional assistance, such as the use of visual pictures to enable
learners to form mental images of words or physical models paired with verbal
commands to reinforce what is being taught, is crucial (Creger, 2019).
With the advancement in technology and contemporary ways of teaching and

learning, international ECE programs are now utilizing modern language,
108
literacy, and assessment practices to meet the teaching and learning goals of PSTs.
In a Hungarian experiment to enhance children’s language acquisition, a PST
engaged learners in play with sounds and invited them to guess the source of the
sounds and to sing songs in the target language (Noel et al., 2019). These strategies
mirrored arguments advanced by Creger (2019) on the use of play to enhance
skills development among preschoolers. Translanguaging and bilingual aspects,
for example, as well as the bilingual labeling of classroom displays, also work
towards enhancing literacy acquisition (AlShamsi & Alsheikh, 2020; Bronteng,
2018).
2.2 Language and Literacy Laboratories

Establishing LLLs in schools that collaborate with university literacy and
language programs has been one of the most creative and effective methods of
preparing teacher candidates for the classroom (Maxwell et al., 2018). Notably,
many PSTs struggle to formulate well-articulated learning goals and face
difficulty in lesson planning and making assessment schemes for their students
(Cavanagh et al., 2019). This supports the need for high-quality learning for PSTs,
especially in literacy and special needs (Al Otaiba et al., 2012). Effective
partnerships between schools and colleges can create a productive teaching and
learning environment for PSTs and enhance current IST practices. These
partnerships are intended to create opportunity for quality TP experiences.
Haverback and Parault (2011) maintained that field and laboratory settings
impacted PSTs’ beliefs and perceptions of students with different needs, learning
styles, reading skills, and strategies. Moreover, the laboratory experience is
strongly linked with PSTs’ fieldwork experience while in TP. PSTs and MCTs can
experiment and test hypotheses in literacy laboratories. More recently, researchers
have started investigating the effectiveness of teacher development programs and
strategies to produce high-quality teachers who can better align learning goals
with quality standards (Elmahdi & Fawzi, 2019). Similarly, research has indicated
that PSTs’ performance and readiness levels could be enhanced by exposing them
to real-life contextualized training with supervised clinical experiences (Marttinen
et al., 2020). Therefore, to have trained and ready PSTs, an authentic learning
context with clinical or laboratory experience is critical for teacher education
programs (Sandoval-Lucero et al., 2011). Furthermore, PSTs must have additional
experiences to develop expertise (El-Abd & Chaaban, 2020).
Wilson and Thayalan (2007) observed that language laboratories provide a

platform to assess students’ speech. These laboratories allow students to listen to
the model pronunciation, repeat and record various sounds, listen to their
performance, compare it with the available models, and conduct self-assessments.
The authors emphasized that providing PSTs with this work experience allows
them to gain a good command of the language for communication with clarity
and accuracy. Maxwell et al. (2018) suggested that providing student teachers
with laboratory field experience will assist them to practice evaluating students,
analyzing the results, and developing assessment-based reading lessons based on
individual needs. The PDS model is proposed to guide laboratory experiences to
109
provide an authentic learning experience that can benefit all education

stakeholders in the UAE.
3. Methodology
3.1 Research Design
This study employed a pragmatism-driven sequential mixed-methods research
design. The study was conducted during two semesters in the 2021–2022
academic year and consisted of two research approaches conducted in two
phases. The findings of the first, quantitative phase were used to inform the
second, qualitative phase (Creswell & Clark, 2011; Saunders et al., 2016), as it was
perceived that one technique alone could not adequately respond to the current
inquiry. In the quantitative phase, two self-administered survey questionnaires
were distributed to the target population of PSTs and ISTs, and the collected data
were descriptively analyzed. For the second, qualitative phase, focus-group
interviews were conducted with PSTs and ISTs who were recruited from the first
phase, MCTs, and other ECE instructors. The qualitative data were thematically
analyzed. The criteria for inclusion in the quantitative phase were: a) current
enrollment in an ECE program and b) willingness to volunteer. The criteria for
exclusion were: a) absent persons (on study leave, maternity leave, etc.) and
b) unwillingness to volunteer. Of the 1000 survey questionnaires distributed in
Phase 1, a total of 720 PSTs and ISTs (n = 352 and 370, respectively) responded,
representing a collective response rate of 72%. PST respondents were between the
ages of 18 and 23 years, whereas IST respondents were between the ages of 26 and
55 years.
3.2 Participants
The final list of respondents in Phase 1 consisted of registered PSTs and ISTs. In
this group, all respondents were female, because the ECE field in the UAE is
dominated by female instructors. Participants for the qualitative stage (Phase 2)
were recruited from Phase 1, with the sample consisting of 42 participants (female
PSTs, n = 17; female ISTs, n = 15; MCTs, n = 6 females and n = 4 males). The
inclusion of participants from Phase 1 in Phase 2 satisfied the integrative nature
of the mixed-methods approach.
3.3 Sampling
In the quantitative stage, all respondents were recruited via simple random
sampling (Bryman, 2012) using a master list solicited from the federal HEIs and
the Ministry of Education (MOE). This sampling procedure was used to ensure
that each member of the targeted population had an equal and independent
chance for selection.
For Phase 2, a purposive sampling technique was used to recruit participants from
the quantitative phase. According to Creswell and Clark (2011), “[p]urposeful
sampling in qualitative research means that researchers intentionally select or recruit
participants who have experienced the central phenomenon or the key concept being
explored in the study” (p. 174).
110
3.4 Ethical Considerations

The study followed the principles of voluntariness and involved minimal risk.
The MOE and HEIs approved the study. The survey questionnaire was
anonymized to protect participant privacy and distributed to the target groups
after receiving their informed consent. The focus-group interviews were
conducted via Zoom over three weeks, and permission was sought and received
from participants to record the sessions.
3.5 Data Collection and Analysis

3.5.1 Phase 1: Quantitative phase
We developed questionnaires from a pool of items compiled from previous
literature on ECE language and literacy-related studies and laboratory experience
in higher education. The questionnaires were compiled in English, and data were
gathered during the 2021–2022 school year. The survey responses comprised the
main data source; however, focus-group interviews expanded on the closed-
ended questionnaire, thus enabling triangulation of the data.
3.5.2 Pilot testing

Before dissemination, the survey was sent to six educators from HEIs, faculty, and
heads of faculties in schools, and amendments were made based on their
feedback. The instrument was then pilot tested with 10 students in the first
researcher’s class and five ISTs. The students and ISTs indicated that some of the
questions relating to the teaching and assessment strategies were repetitive, and
these were subsequently revised.
3.5.3 Perspectives of ECE stakeholders of LLLs

The 133 items of the survey were based on a 5-point Likert scale ranging from
1 (strongly disagree) to 5 (strongly agree). The survey was composed of six sections:
demographics (e.g., sex, length of teaching tenure, and school classification);
language and literacy teaching and assessment strategies (e.g., acquisition of teaching
skills and program support); teaching strategies (e.g., planning skills and content
knowledge); assessment strategies (e.g., using assessment tools such as the
phonological awareness inventory); effectiveness of language and literacy strategies
(e.g., for vocabulary building and slow learners); and new global language, literacy,
and assessment practices (e.g., practical exercises in the teaching of reading, writing,
and alphabet knowledge). The survey required approximately 20 minutes to
complete. Descriptive analyses were conducted to obtain frequencies, means, and
standard deviations. There were no incomplete questionnaires.
3.5.4 Phase 2: Qualitative phase

We developed the interview protocol and formulated 12 questions. Part A
included three general questions about the experience in the ECE field. Part B
included nine questions about the importance of LLLs in teacher preparation
programs, their components, the manner of implementation, their importance in
preparing PSTs for teaching literacy, their importance for the community, their
goals, and their proposal for development. The protocol was sent to three faculty
members and four students from the first author’s class, and some questions were
revised based on their feedback. All interviews were conducted via Zoom in
111
English over three weeks and were recorded and transcribed by the research team.
By the end of the study period, participants were repeating the same information,
thus confirming that data saturation had been reached (Creswell & Clark, 2011).
We supplemented the electronic data with written field notes during the TP visits.
Participants were issued pseudonyms due to ethical considerations. The data
were thematically analyzed using NVivo 12 software following Braun and
Clarke’s (2006) six-step approach. We then sent our findings to members of the
research group and ISTs to allow them an opportunity to validate our
transcription. Engaging our participants in member-checking also served as an
important aspect of triangulation (Candela, 2019). Table 1 presents the phases of
the thematic analysis of perspectives of ECE stakeholders of LLLs.
Table 1: Phases of the thematic analysis of the perspectives of ECE stakeholders of

LLLs
Phase Description of the process
1. Becoming familiar with We transcribed, read, and re-read the data and
the data noted initial ideas
2. Generating initial codes We systematically coded interesting features of
the data and collated data relevant to each
code, for example, experience, resource type, and
camaraderie
3. Searching for themes We collated data into potential themes and
gathered all data relevant to each potential
theme
4. Reviewing themes We checked if themes worked with the coded
extracts (Level 1) and the entire data set
(Level 2). We generated a thematic map of our
analysis.
5. Defining and naming We performed an ongoing analysis to refine
themes the specifics of each theme. An overall story
began to emerge, after which we generated
clear definitions and names for each theme.
6. Analyzing the results We selected vivid, compelling extracts as
examples. We created a final analysis of the
selected extracts and related this analysis to the
research questions and secondary literature.
4. Results
4.1 Descriptive Data from Survey Results
Of the 720 respondents who responded to the survey, all were female (100%).
Approximately 70% taught in public schools and 30% taught in private schools.
Furthermore, 60% had taught ECE for more than five years. For 80% of the
respondents, the highest level of qualification was a BA in Education/ECE, and
20% had an M.Ed qualification.
4.1.1 Stakeholder perceptions of training

Most respondents (53%) agreed with the extent to which training on language and
literacy teaching and assessment strategies was provided (M = 4.11, SD = 0.78).
112
4.1.2 Stakeholder perceptions of language and literacy teaching strategies currently

implemented
Most respondents (55%) appeared confident about their knowledge of children,
their knowledge of and ability to teach the content, and their ability to modify the
content to suit the situation even if it involved information and communication
technology (M = 4.16, SD = 0.75).
4.1.3 Stakeholder perceptions of language and literacy assessment strategies

Most respondents (58%) felt confident using and applying assessment strategies
to provide feedback to their students and intuitively assess their needs (M = 4.17,
SD = 0.72).
4.1.4 Stakeholder perceptions of new global language, literacy, and assessment practices
Most respondents (57%; M = 4.15) agreed that students’ literacy performance
reflected the effectiveness of the strategies implemented in early childhood
educators’ training, teaching, and assessment. Specific categories under this
heading (and their respective scores) are literacy (57%; M = 4.18), print (50%;
M = 4.18), comprehension/vocabulary (54%; M = 4.26), alphabet knowledge and
letter-pronunciation practices (50%; M = 4.30), phonological awareness activities
(53%; M = 4.24), reading-aloud practices (52%; M = 4.26), and writing activities
(53%; M = 4.28).
Respondents agreed that the training included in the ECE teacher preparation
program is crucial in implementing effective language, literacy, and assessment
teaching strategies. In addition, they agreed that the current implemented
strategies and new global language, literacy, and assessment practices are
essential and should be emphasized to enhance current practices.
4.2 Qualitative Analysis

Three general themes were identified as they pertain to the research objectives of
this study. In the focus-group sessions, participants reflected on the processes that
occurred during TP and aligned this with what they learned in the classroom. One
key theme was how this experience strengthened their confidence in their
competence and willingness to continue their ECE career. The results are
presented using the pseudonyms assigned to participants.
4.2.1 Experiences in the ECE program

Participants identified learning new concepts in the ECE program, which
consisted of awareness of language, morphology, syntax, and phonology. One IST
reported that:
“The ECE program helped us understand and practice authentic teaching
through theoretical and practical courses that raised our awareness of
various language aspects and the teaching and assessment strategies to
improve them. I remember Miss Susan taught us about linguistic
awareness and the strategies that should be employed in the classroom.
For example, I learned about the literacy inventory and how to apply it to
measure my children’s literacy readiness.” (Alya)
113
MCTs observed that they enjoyed working with PSTs. Such comments
highlighted the collaborative nature of the engagement and objective of the PDS
model.
“I enjoyed teaching. I enjoyed watching my pre-service teachers teaching
young children and applying what we teach them in the classroom. I feel
proud of them when I see the latest literature-supported best practices
applied in ECE classrooms. Also, we teach them how to contextualize
practices that are unique to this country.” (Kyle)
The role of the PDS model in the bilingual context was also highlighted.
“I like how ECE programs are structured. We teach our pre-service
teachers to integrate subjects through play and improve literacy and
biliteracy through strategies such as translanguaging and code-
switching. We teach them to apply strategies that support reading skills
such as the big books, storytelling, and many other strategies.” (May)
PSTs added that overseas teachers were beneficial as they helped them fulfill their
duties as ECE interns. One PST stated that:
“… for me, it was a huge benefit. My mentor teacher was absolutely
phenomenal, and we worked very well together and we’re still in contact.
Her guidance was beneficial and helped to develop my confidence to teach
and lead a classroom. It wasn’t easy for her to teach children whose
English is not their first language, but with the support of her co-teacher,
she was able to implement her best practice and help me apply learned
strategies.” (Salma)
4.2.2 Implementation of LLLs

MCTs shared how LLLs with the appropriate resources could be useful for
bilingual students.
“I would envision [LLLs] as having a space where you have all the
resources that you need to teach all subjects with a linguistic focus. You
have the children there that you can teach, with pre-service teachers
learning how to do actual research [and] they’re learning how to collect
data and use that data to push children forward. I can see a language lab
as a place where math and science activities are integrated to support
concept development with an authentic link to enable literacy
development among children, and reflection.” (Talia)
ISTs also added that LLLs led to improvement in practice and school teaching,
specifically language practice through activities that included the use of
technological aids.
“My main goal as a KG (kindergarten) teacher is to apply a strategy or a
teaching methodology that enhances children’s literacy in English and
Arabic within an integrative teaching and learning environment. I
imagine having a pre-service teacher who gathers and analyzes data, so
we can think about how to improve our children’s linguistic abilities. I
respect the reflective practices many of my pre-service teachers apply.”
(Talia)
114
An analysis of the HEI instructors’ views on how LLLs support students showed
results with more accurate assessments. LLLs also led to confidence in literacy
assessment, inquiry-based learning, and cognitive skills. One MST observed that:
“I think you can’t start anywhere without getting a background of where
the students are at, I mean hands-on games, hands-on materials inside of
literacy, that’s very important, but before we get there, we need to know
where the students are at, so accurate assessment would be one of my top
priorities in a literacy lab.” (Salwa)
One PST mentioned inadequate resources for implementation, which included

programs and specific equipment.
“The resources—what’s lacking is the implementation, yeah, the practical
part. We need a space where we can bring children on campus, apply a
uniquely designed literacy program for the community children, assess
the progress, and finalize it with research. Having PSTs actually
implement what they’re seeing and understanding how to implement
what they’re doing is challenging sometimes. We want to apply all
listening, speaking, reading, and writing activities with young children
at our campus, where we can assess them and provide strategic guidance
and support.” (Sara)
One MCT also noted that:

“You need a good comprehensive phonic program because it’s a good
starting point. In the school, teachers don’t need to have the phonics
program; they’re moving children through the system.” (Gillian)
Participants shared that LLLs aim for unprompted engagement with resources,
which could be a motivational factor. One MCT noted that:
“The LLLs provide unexpected engagement with the different resources.
It really motivates students right when they are unprompted.” (Jane)
4.2.3 Effectiveness of literacy strategies

One of the PSTs indicated that they are prepared as future teachers through
advanced teaching strategies using technology and new hybrid spaces via LLLs.
“These labs provide children with a rich interactive learning
environment. Children are supported by technology and physical
guidance, where teachers can scaffold and guide their progress. They can
support children to work independently and evaluate their progress.”
(May)
Participants also indicated that LLLs are crucial in providing inquiry-based

learning, practical learning experiences, rich learning strategies, a semiotic
system, and teaching through drama.
“Everything we do with children starts with a question and then they
start thinking about how or where they want to go with the idea.”
(Gillian)
“LLLs are important hubs for children to practice language through

activities, audio, video, visuals, etc.” (Aysha)
115
“LLLs provide rich teaching, learning, and assessment strategies,

especially when working on drama lessons.” (Alyazia)
“… it is basically teaching children how to deal with language better to

make their ways through the language. You have some kind of semiotic
system, official language to transfer meaning to students and students
construct meaning.” (Tony)
Participating PSTs, ISTs, and MCTs mentioned that their teaching methods
changed and had a positive effect on their students’ literacy achievements. In
addition, the confidence of students improved. LLLs prepared students as future
teachers when they used learning tools to teach. One PST observed that:
“We have moved toward the integrative method in teaching and learning.
And as you said before, literacy is everywhere—in math, you’re writing;
in science, we learn how to link English and Arabic subjects to facilitate
science concepts and learning. We teach children how to speak and write
in all classes. Labs would help us measure the effectiveness of any kind of
implementation strategically and systematically.” (Najla)
ISTs mentioned that LLLs provided practical orientation for students. One noted
that:
“Some of the schools [I taught at] don’t care about or know how to
implement shared reading. They are not doing it. Others don’t even have
literacy or reading and writing programs at their schools. It depends on
the school’s leadership. Having a consistent practice of LLLs that are based
on campus and support schools’ practices is highly needed.” (Alia)
HEI instructors highlighted the importance of setting clear goals and visions.
“We want a transformational curriculum implementation where an
integrative approach is implemented through technology and data-based
practices. Many leading education systems applied this transformational
aspect of the curriculum to ensure that data leads to practice. We need
this critical stance and the power to create this collaboration between
HEIs’ labs and schools to ensure guided best-practice implementation.”
(Ama)
“LLLs are important aspects of the community. LLLs can be initiated as

independent centers or in partnerships with colleges as evening programs.
This will help children not only to learn but to have fun. It is a good
opportunity to support parents. It is crucial to have community-based
partnerships.” (Nikishia)
5. Discussion
This study used a mixed-methods approach to investigate the perspectives of key
stakeholders in ECE on LLLs in the UAE. The ensuing discussion contextualizes
the findings of the results and triangulates them with the secondary literature.
116
5.1 Diverse Strategies in Language Learning Acquisition and Implementation

Most participating PSTs and ISTs expressed comfort in employing diverse
strategies in ECE learning, such as literacy, print, vocabulary, and phonological
awareness, and understood their importance. Our results support those of Kasper
et al. (2018), who argued that language and literacy strategies enhanced the
acquisition of learners’ language skills. Barnes and Dickinson (2017) and Davison
and Qi (2017) also concurred that knowledge of these activities strengthened
students’ communication skills, so PSTs and ISTs need to know how to implement
them.
Participants expressed how their experiences in the ECE program integrated the
theoretical aspects of their learning activities with practical strategies. This
acknowledgment helped increase their competence and self-efficacy, and most
expressed confidence in their ability to develop children’s linguistic abilities, such
as phonology, morphology, and syntax. The ability to master these activities and
to perceive how they would be helpful in their teaching careers was evident in the
responses. For example, most participants acknowledged the benefits of the
pronunciation aspects of languages with the use of LLLs (Wilson & Thayalan,
2007). As highlighted by the Western ECE teachers, the provision of bilingual co-
teachers for different activities in teaching and assessment in the early years could
benefit young learners. Indeed, collaboration with foreign teachers was helpful
and enriched the program, as it enabled the implementation of bilingual teaching
strategies such as translanguaging and code-switching. Maxwell et al. (2018)
observed that collaboration among the relevant stakeholders, such as universities
and schools, or even among the teachers, was one of the highlights of LLLs and
the PDS programs.
Factors related to well-structured integrative lessons were also highlighted.

Demonstrating the associations during the lessons helped learners avoid
confusion about the different learning concepts. Consequently, they were
motivated to learn and gain more knowledge and practice, especially when
concepts and literacy were emphasized in all subjects (Davison & Qi, 2017).
Moreover, while focusing on teaching strategies that might enhance the
acquisition of communication skills and development among preschoolers,
Creger (2019) suggested the use of visual cues to reinforce what was being taught,
among other strategies.
5.2 Language Learning, Literacy, and PDS Programs

Participating PSTs and ISTs found that their TP extended beyond the theoretical
components, as they had the chance to apply them in schools and campuses
through physical and digital spaces (Quirke & AlShamsi, 2023). However, there
was a need for a consistent and sustainable practice that links HEI programs and
schools through LLLs to apply the different strategies taught. Language and
literacy programs engender a practical perspective on learning where learners are
offered the opportunity to demonstrate what they are learning rather than just
gaining theoretical knowledge (Maxwell et al., 2018; Pianta et al., 2016).
Furthermore, the PDS model, which anchored this study, focuses on infusing
learners with practical skills, thereby enabling them to be knowledgeable and
117
practical (Sandoval-Lucero et al., 2011). Generally, the findings show that

participants had an enriching experience at the HEI, with language and literacy
classes that were supported by the campus–school LLLs. Most participants
considered it an opportunity to create partnerships with the community to
enhance learners’ language and literacy skills.
5.3 Implementation of LLLs

LLLs were considered useful for assessments in ways that could enhance the
diversity of learning among ECE learners through inquiry-based learning
(AlShamsi, 2022, 2023). Language assessments are challenging areas for teachers
as they often rely on traditional approaches that may not be pragmatic. In this
regard, Lam (2015) noted that teachers of language are often challenged in
performing assessments due to the use of traditional approaches and a lack of
understanding of their implementation. This suggests that using LLLs is a
practical approach to language assessments. The results show how LLLs were also
found to be instrumental in enhancing reading habits. One of the challenges as far
as language and literacy are concerned is gaining reading skills and related
concepts, such as vocabulary acquisition. However, when these laboratories are
utilized and backed by appropriate strategies, learners make immense gains
(Barnes & Dickinson, 2017).
LLLs would be useful in solving inquiry- and problem-based aspects of learning

(AlShamsi, 2022, 2023) and for the development of cognitive and metacognitive
abilities in young children (AlShamsi, 2021). For example, when the LLLs are
embedded within the lessons, practical learning is enabled. This observation is
supported by researchers such as Maxwell et al. (2018), who noted the practical
components of LLLs. This aspect supports the focus of PDS models on supporting
this level of learning (Ball & Cohen, 1999). Therefore, using LLLs encourages
practical learning and fits well within the PDS model (Sandoval-Lucero et al.,
2011).
6. Conclusions
Education stakeholders’ perspectives on LLLs in ECE programs are important for
ensuring competency and efficacy in language teaching and learning in countries
such as the UAE. The PDS model is unique in that it allows a form of collaborative
immersion between institutions, known for theory, and teacher educators, known
for practicum, without barriers or other impediments. This meaningful
association between theory and practice is perceived to benefit early learners of
language and literacy. This study showed strong support for the PDS model as
proposed by the Holmes Group. PSTs and ISTs welcomed the opportunity to
engage with and learn from the MCTs and other instructors in the UAE education
system, who in turn admitted that they also enjoyed imparting knowledge and
engaging with their mentees. The strength of the field experiences in predicting
the teacher trainees’ readiness was apparent. Their competence and efficacy will
remain buoyant if they are placed in under-resourced schools or schools with
lackluster leadership, which will be important when assessing the sustainability
of activities and the training provided in PDS programs.
118
The outcome of the study highlights the important role played by mentor teachers
in the preparation process and suggests the need for more long-term support even
when PSTs and ISTs are placed in their respective schools. To this end, careful
selection and training of new entrants will be important as the profession and the
current state of education require creativity and foresight to address the
limitations that may arise after these teachers become full-fledged instructors. All
teachers will need to apply unconventional methods to accommodate
socioeconomic, psychological, and cultural diversity when faced with real-world
challenges. LLLs are best practiced in partnership with schools, HEIs, and the
community.
7. Recommendations
Several recommendations emanate from this research. First, teachers can develop
learning hubs for the learners in their respective schools in partnerships with HEIs
to enable them to learn through activity-based curricular episodes. Simulations
could be of immense value in this regard, as the dramatic presentations of
language teaching and learning involve all human senses and can improve
children’s higher order thinking skills. Participants noted that the learners’ oral
skills were employed more frequently, so there was a greater need to develop
their reading habits. Incorporating technology into the teaching and learning of
language would be a valuable addition to enable learning in hybrid spaces.
Second, participants expressed a desire for subject-specific assessment techniques

or strategies for language teaching assessment. Therefore, on-task or in-the-
laboratory assessment through projects and assignments at the individual and
group levels might be useful. This can take the form of in-class and out-of-class
activities. This aligns with the PDS model, which advocates for a practical
approach to language and literacy teaching in schools.
Third, students have different personalities, so it is recommended that teachers

adopt different approaches to accommodate these differences and cater to
individual needs. Instructors need to devise unconventional strategies to enhance
learners’ literacy skills and ensure a congenial learning environment.
8. References
Al Otaiba, S., Lake, V. E., Greulich, L., Folsom, J. S., & Guidry, L. (2012). Preparing
beginning reading teachers: An experimental comparison of initial early literacy
field experiences. Reading and Writing, 25(1), 109–129.
https://doi.org/10.1007/s11145-010-9250-2
AlShamsi, A. S., Alshamsi, A. K., & AlKetbi, A. N. (2022) Training teachers using action
research for innovation in early childhood education literacy. International Journal
of Learning, Teaching and Educational Research, 21(11), 54–72.
AlShamsi, A. S., & Alsheikh, N. O. (2020). Sociocultural integration of bilingualism and
biliteracy of Emiratis: Exploratory study in a group of fifth-grade students. 3L:
Language, Linguistics, Literature: The Southeast Asian Journal of English Language
Studies, 26(3), 66–80. https://doi.org/10.17576/3L-2020-2603-06
AlShamsi, A. (2022). Equity in online assessment: A qualitative case study.
NeuroQuantology, 20(10), 6478–6491.
119
AlShamsi, A., & Alsheikh, N. (2022). Reconnoitering the influence of linguistic aspects on
Emirati fifth grade students’ bilingualism practices: A mixed method study.
Journal of Positive Psychology and Wellbeing, 6(2), 1214–1231.
AlShamsi, A. (2023). Perspective chapter: Performance-based assessment through inquiry-
based learning. Education and Human Development.
https://doi.org/10.5772/intechopen.109217
AlShamsi, A. (2021). Cognitive and metacognitive skills on elementary school students:
Mixed methods study. TESOL International Journal, 61(6.1), 7–36.
Ball, D., & Cohen, D. (1999). Developing practice, developing practitioners. In L. Darling-
Hammond, & G. Sykes (Eds.), Teaching as the learning profession: Handbook for policy
and practice (pp. 3–32). Josey Bass.
Barnes, E. M., & Dickinson, D. K. (2017). The impact of teachers’ commenting strategies
on children’s vocabulary growth. Exceptionality, 25(3), 186–206.
https://doi.org/10.1080/09362835.2016.1196447
Bransford, J., Darling-Hammond, L., & LePage, P. (2005). Introduction. In L. Darling-
Hammond, & J. Bransford (Eds.), Preparing teachers for a changing world: What
teachers should learn and be able to do (pp. 1–39). Jossey-Bass.
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research
in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa
Bronteng, J. E. (2018). A study of Ghanaian kindergarten teachers’ use of bilingual and
translanguaging practices [Doctoral dissertation]. University of South Florida,
Florida.
https://scholarcommons.usf.edu/cgi/viewcontent.cgi?article=8865&context=et
d
Bryman, A. (2012). Social research methods (4th ed.). Oxford University Press.
Candela, A. G. (2019). Exploring the function of member checking. Qualitative Report, 24(3),
619–628. https://doi.org/10.46743/2160-3715/2019.3726
Cavanagh, M., Barr, J., Moloney, R., Lane, R., Hay, I., & Chu, H. E. (2019). Pre-service
teachers’ impact on student learning: Planning, teaching, and assessing during
professional practice. Australian Journal of Teacher Education, 44(2), 66–81.
Creger, E. (2019). Supporting early childhood students’ communication skills within the
classroom. Hamline University.
Creswell, J. W., & Clark, V. L. (2011). Educational research: Planning, conducting, and
evaluating quantitative and qualitative research. Pearson.
Davison, M. D., & Qi, C. H. (2017). Language teaching strategies for preschool English
learners. Perspectives of the ASHA Special Interest Groups, 2(1), 170–178.
https://doi.org/10.1044/persp2.SIG1.170
Dietrichson, J., Ringsmose, C., Christoffersen, M., & Svinth, L. (2021). Participatory
professional development and preschool quality: Evidence from a mixed-
methods study. Nordic Studies in Education, 41(2), 80–110.
https://doi.org/10.23865/nse.v41.2342
El-Abd, M., & Chaaban, Y. (2020). The role of vicarious experiences in the development
of pre-service teachers’ classroom management self-efficacy beliefs. International
Journal of Early Years Education, 29(3), 282–297.
https://doi.org/10.1080/09669760.2020.1779669
Elmahdi, I., & Fawzi, H. (2019). Preservice teachers’ perception of readiness to teach in
light of teachers’ standards. American Journal of Educational Research, 7(4), 304–308.
https://doi.org/10.12691/education-7-4-1
120
Gabriel, R. (2011). A practice-based theory of professional education: Teach for America’s

professional development model. Urban Education, 46(5), 975–986.
https://doi.org/10.1177/0042085911400324
Haverback, H. R., & Parault, S. J. (2011). High efficacy and the preservice reading teacher:
A comparative study. Teaching and Teacher Education, 27(4), 703–711.
https://doi.org/10.1016/j.tate.2010.12.001
Herro, D., Hirsch, S. E., & Quigley, C. (2019). A faculty-in-residence programme: Enacting
practice-based professional development in a STEAM-focused middle school.
Professional Development in Education, 48(4), 559–575.
https://doi.org/10.1080/19415257.2019.1702579
Holmes Group. (1986). Tomorrow’s teachers: A report of the Holmes Group. Author.
Holmes Group. (1990). Tomorrow’s schools. Author.
Ismail, S. A. A., & Jarrah, A. M. (2019). Exploring pre-service teachers’ perceptions of their
pedagogical preferences, teaching competence and motivation. International
Journal of Instruction, 12(1), 493–510. https://doi.org/10.29333/iji.2019.12132a
Kasper, M., Uibu, K., & Mikk, J. (2018). Language teaching strategies’ impact on third-
grade students’ reading outcomes and reading interest. International Electronic
Journal of Elementary Education, 10(5), 601–610.
https://www.iejee.com/index.php/IEJEE/article/view/509
Lam, R. (2015). Language assessment training in Hong Kong: Implications for language
assessment literacy. Language Testing, 32(2), 169–197.
https://doi.org/10.1177/0265532214554321
Lefever-Davis, S., & Heller, M. (2003). Trends in teacher certification and literacy: Literacy
partnerships in education. Reading Teacher, 56(6), 566–568.
https://www.jstor.org/stable/20205247
Marttinen, R., Daum, D. N., Banville, D., & Fredrick, R. N. (2020). Pre-service teachers
learning through service-learning in a low SES school. Physical Education and Sport
Pedagogy, 25(1), 1–15. https://doi.org/10.1080/17408989.2019.1670153
Maxwell, N., Hilaski, D., & Whelan-Kim, K. (2018). Enhancing teacher education through
field-based literacy laboratories. The Reading Professor, 41(1), 22–29.
https://scholar.stjohns.edu/thereadingprofessor/vol41/iss1/19
Noel, A. M., Lord, K. M., & Varga, L. (2019). Supporting second language vocabulary: A
scenario from Hungarian preschools. Kappa Delta Pi Record, 55(1), 24–29.
https://doi.org/10.1080/00228958.2019.1549437
Pianta, R., Downer, J., & Hamre, B. (2016). Quality in early education classrooms:
Definitions, gaps, and systems. The Future of Children, 26(2), 119–137.
https://www.jstor.org/stable/43940584 .
Quirke, P., & AlShamsi, A. S. (2023). Perspective chapter: Peer observation of teaching in
phygital communities of inquiry. Education and Human Development.
https://doi.org/10.5772/intechopen.109380
Sandoval-Lucero, E., Shanklin, N. L., Sobel, D. M., Townsend, S. S., Davis, A., &
Kalisher, S. (2011). Voices of beginning teachers: Do paths to preparation make a
difference? Education, 132(2), 336–350.
Saunders, M., Lewis, P., & Thornhill, A. (2016). Research methods for business students
(7th ed.). Pearson Education.
Stürmer, K., Seidel, T., & Holzberger, D. (2016). Intra-individual differences in developing
professional vision: Preservice teachers’ changes in the course of an innovative
teacher education program. Instructional Science, 44(3), 293–309.
https://doi.org/10.1007/s11251-016-9373-1
Teitel, L. (2003). The professional development schools handbook: Starting, sustaining, and
assessing partnerships that improve student learning. Crowin Press.
121
United Nations. (2017). UAE and the 2030 Agenda for Sustainable Development:
Excellence in implementation. National Committee on Sustainable Development
Goals.
https://sustainabledevelopment.un.org/content/documents/20161UAE_SDGs
_Report_Full_English.pdf
von Suchodoletz, A., Barza, L., & Larsen, R. A. A. (2020). Examination of teacher–child
interactions in early childhood education programmes in the United Arab
Emirates. International Journal of Early Years Education, 28(1), 6–21.
https://doi.org/10.1080/09669760.2019.1594720
Walker, D., Sepulveda, S. J., Hoff, E., Rowe, M. L., Schwartz, I. S., Dale, P. S.,
Peterson, C. A., Diamond, K., Goldin-Meadow, S., Levine, S. C., Wasik, B. H.,
Horm, D. M., & Bigelow, K. M. (2020). Language intervention research in early
childhood care and education: A systematic survey of the literature. Early
Childhood Research Quarterly, 50, 68–85.
https://doi.org/10.1016/j.ecresq.2019.02.010
Wilson, D. D., & Thayalan, V. (2007). The significance of the language laboratory in
communication. Karen’s linguistics issues.
http://www3.telus.net/linguisticissues/lab
Xu, Y., Chin, C., Reed, E., & Hutchinson, C. (2014). The effects of a comprehensive early
literacy project on preschoolers’ language and literacy skills. Early Childhood
Education Journal, 42(5), 295–304. https://doi.org/10.1007/s10643-013-0613-6
122

Vol. 22, No. 7, pp. 122-135, July 2023
Received Mar 16, 2023; Revised Jul 7, 2023; Accepted Jul 14, 2023
Challenges and Opportunities of AI-Assisted Learning:

A Systematic Literature Review on the Impact of
ChatGPT Usage in Higher Education
Alfonso Renato Vargas-Murillo

Facultad de Derecho
Universidad Privada del Norte, Lima, Perú
Ilda Nadia Monica de la Asuncion Pari-Bedoya

Facultad de Derecho
Universidad Tecnológica del Perú, Lima, Perú
Francisco de Jesús Guevara-Soto

Grupo de Investigación Interdisciplinaria en Gestión de la Educación de Calidad
Universidad Privada de Tacna, Tacna, Perú
Abstract. In recent years, ChatGPT has become a noteworthy subject in

the educational field due to the popularity it gained among students all
over the world. Applications such as text creation, data processing, and
curriculum development, among other applications, prove that ChatGPT
is one of the most advanced chatbots available, albeit it posing a series of
challenges that prevents students from developing their critical thinking
skills in some cases. In this sense, this study’s objective is to analyze the
current literature regarding the use of ChatGPT in higher education. The
study questions its usage, the overall impact it has on education, and the
challenges and opportunities of artificial intelligence- (AI) assisted
learning, all within the research area of digital education. For this, a
systematic literature review (SLR) was conducted across multiple journal
databases, analyzing the articles found and filtering them with the use of
specific criteria. The results of this analysis afforded the conclusion that
that ChatGPT can potentially enhance both academic- and librarian-
related processes, although it is important to reconsider the ethics of using
technology such as this. This is because this kind of AI technology also
has the potential to modify the way people take on different jobs, tasks,
and processes at an academic level, for which the correct and responsible
use of ChatGPT needs to be assessed in the field of higher education.
Keywords: artificial intelligence; ChatGPT; higher education; learning

methodologies; smart learning
©Authors
123
1. Introduction
In recent years, ChatGPT has remained an interesting subject that attracts the
attention of students and teachers alike. Its somewhat controversial introduction
to the educational field has spurred the debate of whether its usage for academic
work is beneficial or not, considering the potential capabilities of this tool for
completing tasks such as research and data comparison, to name a few. In this
sense, this form of artificial intelligence- (AI) assisted learning represents a
remarkable technological advancement in smart learning. It paves the way for
future applications within the field of education, in a world more driven towards
the development and optimization of digital tools, with the help of machine
learning. Nevertheless, this type of technology should be questioned when its
application permeates deeply in the performance and development of students
and their learning process, especially when taking into consideration the level of
accessibility that ChatGPT has worldwide. Students should have an ethical
standpoint on whether they want to use ChatGPT to complement their learning
process and how much input this technology will have in their academic work.
They can then learn to use it more effectively and avoid the abuse of ChatGPT
usage, to seize the benefits that this AI technology may have in higher education.
According to Kung et al. (2023), the past decade has brought many advances
regarding deep learning and AI in the way that professionals approach their tasks
across different industries. This is due to the ability of AI-assisted tools to build
classifications of different objects and subjects regardless of the input, such as
images and text, among other media. This enables the development of systems
with many applications, such as automated tagging of photographs, translation,
scanning, and text generation.
AI technologies have gradually become more visible in various aspects of society,

and ChatGPT is part of said technologies. Presented as one of the most advanced
AI applications to date, ChatGPT has drawn public attention from every corner of
the world, seeing that its applications are developed in different fields, including
education (Tlili et al., 2023).
It has always been remarkable how AI is able to reproduce human-like behavior

and writing styles that are so convincing that it becomes difficult to tell the
difference between machine- and human-produced content. According to
Abdullah et al. (2022), machine learning and data processing have come a long
way, so much so that the continuous development and evolution of these
technologies constantly set new standards for what is perceived of AI. It is not so
difficult nowadays to find literature written entirely by AI, and ChatGPT is a clear
example of that. This natural language processing model developed by OpenAi
in 2022 was created for open-ended conversations, being able to power purely
conversational AI applications such as chatbots and virtual assistants (OpenAI,
2022). Nevertheless, its application in the educational field has begun to flourish,
as students across different levels of education all over the world use this
technology to assist with their academic homework. This transforms ChatGPT
into some sort of auxiliary tool that aids them with the completion of certain tasks
124
that would normally take more time to complete, such as research and data
comparison, to name a few examples.
AI is definitely at the helm of digital transformation and machine-powered media

content, and the way ChatGPT enables students to complement their learning
process is something to take into consideration. ChatGPT allows users to enter
text prompts to effectively generate text responses that are drawn from
knowledge gathered via machine learning, through the use of the internet (Pavlik,
2023). Its uses and applications are certainly not unlimited, but students’ ability
to exploit this technology’s capability is somewhat of a problematic subject.
Regarding the impact that ChatGPT could have on the educational field, it is
important to assess the ethics behind the correct usage of this tool so that it does
not hinder students’ ability to write or to conduct academic work without the need
of an AI tool. Abusing these technologies can make a student dependent on them.
In addition, even though there may be a level of enthusiasm regarding how
ChatGPT could change the educational setting, there are some people who advise
approaching their use with caution (Tlili et al., 2023). To see the bigger picture
regarding the educational transformation, these technologies would need to be
implemented to determine the usefulness of its application and the user
experiences in the learning process, as scenarios such as cheating and
misinformation could create a negative impact in education. In this regard, it is
only rational that academics and educational institutions all over the world
should consider safe measures to adopt technologies such as ChatGPT in
education, or to develop strategies that prevent students from abusing its power.
1.1 Research Questions

To optimize and enhance the process of this research, it was important to apply
research questions that would help redirect the findings towards a specific goal
and grasp a better focus on this study. These questions were used to filter the
results of the research and to make relevant conclusions from the specific
literature selected from this systematic literature review (SLR) (Guyatt et al.,
2008). This was done by proposing a general research question and dividing it
into four specific questions (Q1, Q2, Q3, and Q4) that were used to guide the
different aspects of this research, to keep the focus of the investigation. Table 1
shows the research questions proposed to conduct this SLR and each question’s
purpose.
Table 1: Research questions and their purpose

General research
Specific question Purpose of the question
question
In order to understand
Q1: How is ChatGPT used
ChatGPT’s impact in education,
and/or applied in
it is important to know how it
Does ChatGPT impact education?
can be used in this field
on higher education?
Q2: What are the risks and It is necessary to know the risks
challenges of ChatGPT and challenges presented by
usage in education? using this tool
125
Q3: What are the It is important to know what

opportunities created by kind of opportunities are
using ChatGPT in education? created by using this tool
Q4: What is the overall It is necessary to assess the
impact of ChatGPT in overall impact ChatGPT has on
education? the field of education
2. Methodology
2.1 Research Approach
This study focused entirely on analyzing the impact of ChatGPT usage in
education, while filling the gap left by previous studies regarding the use of
ChatGPT in education. The aim was to gain a comprehensive understanding of its
potential benefits and challenges within the field. As the integration of AI and
machine learning technologies continues to shape the educational landscape, it is
crucial to explore the implications of ChatGPT in depth. Addressing this subject
will help students and educators harness the full potential of ChatGPT to
revolutionize education, while safeguarding its integrity and effectiveness.
Taking into consideration the ethics revolving such matter, an SLR was conducted
to look for specific knowledge of the subject at hand, and to have a better
understanding of the use of this technology in the educational field through the
course of this research (Nightingale, 2009). Additionally, analyzing the ethical
considerations surrounding ChatGPT’s use can help develop recommendations
that ensure responsible implementation of this technology in educational settings.
An SLR, as stated by Moreno et al. (2018), permits investigators to recollect

existing studies and articles regarding the subject at hand and enables them to
build a curated collection of literature that uses various approaches to assess the
same subject, which was the main focus of this research. With all the knowledge
gathered from the different literature sources, it is necessary to start filtering every
single finding, including or excluding distinct literature through the use of
specific criteria and research questions that will enable aiding in the recollection
of the exact knowledge needed for this study.
For this research, the SLR was conducted by searching for literature sources
through the journal databases of sites such as Scopus, ScienceDirect, ProQuest,
IEEE Xplore, and ACM Digital Library. This SLR has opened up the possibility to
have a better understanding regarding the way different authors approach
ChatGPT application in education and its impact on said field. In addition, with
the help of keywords, search filters, and criteria, the SLR was able to focus even
more on specific findings, hence the analysis made could assess the subject at
hand in the best way possible (Nightingale, 2009).
2.2 Keywords and Criteria Used for Research

With the research question already established, we needed to look through the
journal databases selected for this study with the help of a series of keywords that
would help gather enough information while keeping the research within the
same field (in this case, the educational field). We also had to keep in mind that
the information had to be of relevance for later analysis. For this research, the
126
series of keywords used to filter the search results in each of the different journal
databases selected were the following:
((“ChatGPT) AND (“education” OR “learning” OR “e-learning” OR
“teaching”) AND (“AI” OR “artificial intelligence”))
These keywords helped to navigate through the results obtained from the
databases consulted to categorize the literature found and review the articles that
assess the study subject. Nevertheless, these keywords would not have sufficed if
the SLR’s objective were to obtain more specific and in-depth knowledge
throughout this research. With this in mind, therefore, some criteria were applied
to exclude articles that did not fall in line with the purpose of this study. Table 2
shows the criteria applied to filter out any unwanted literature that did not
contribute any sort of insight for this research.
Table 2: Article inclusion and exclusion criteria

Criterion Criterion description Inclusion and exclusion status
Language Articles written in English Article is included
Year of publication Articles published in the last year Article is included
Articles that are related to the
Article is included
educational field
Articles that are related to the learning
Field process, teaching methodologies, or Article is included
different types of learning
The article is not related to the
Article is not included
educational field in any way
Article does not mention ChatGPT Article is not included
The article does not mention ChatGPT
or artificial intelligence in any way
Article content
Articles that have high similarities
across databases
Article data are inconclusive Article is not included
The article talks about different
applications of ChatGPT or AI Article is included
technology in education
Relevance of content
The article talks about applications of
ChatGPT or AI in other fields besides Article is not included
education
After applying these criteria, a series of articles were collected. The number of
articles found in each database searched are shown in Table 3.
Table 3: Number of articles found in each journal database searched
Journal database Number of articles

Scopus 5
ScienceDirect 4
ProQuest 4
IEEE Xplore 2
ACM Digital Library 1
Total 16
127
It is safe to say that the criteria applied helped to narrow down the search results
and filter out the best results for this study. Table 4 shows the full list of articles
selected from this search, after applying the criteria and categorizing each article
by their respective journal database.
Table 4: List of articles selected

Database Author(s) and year of
Article title
journal publication
The role of ChatGPT, generative language
models, and artificial intelligence in medical
Scopus Eysenbach (2023)
education: A conversation with ChatGPT and
a call for papers
Game of algorithms: ChatGPT implications
Ivanov and Soliman
Scopus for the future of tourism education and
(2023)
research
The future of medical education and
Scopus research: Is ChatGPT a blessing or blight in Arif et al. (2023)
disguise?
ChatGPT in education: Strategies for
Scopus Halaweh (2023)
responsible implementation
Commentary: ChatGPT use in higher
Scopus education assessment: Prospects and Benuyenah (2023)
epistemic threats
Science ChatGPT in nursing education: Is there a role
Thakur et al. (2023)
Direct for curriculum development?
Chatting or cheating? The impacts of
Science
ChatGPT and other artificial intelligence Choi et al. (2023)
Direct
language models on nurse education
The promise and peril of ChatGPT in
Science
geriatric nursing education: What we know Qi et al. (2023)
Direct
and do not know
ChatGPT for good? On opportunities and
Science
challenges of large language models for Kasneci et al. (2023)
Direct
education
Impact of artificial intelligence on dental
ProQuest education: A review and guide for Thurzo et al. (2023)
curriculum update
Can an artificial intelligence chatbot be the
ProQuest Lee (2023)
author of a scholarly article?
Chatbots in education and research: A critical
ProQuest examination of ethical implications and Kooli (2023)
solutions
ChatGPT utility in healthcare education,
research, and practice: Systematic review on
ProQuest Sallam (2023)
the promising perspectives and valid
concerns
IEEE Xplore Use of ChatGPT in ESP teaching process Kovačević (2023)
Exploring students’ perceptions of ChatGPT:
IEEE Xplore Shoufan (2023)
Thematic analysis and follow-up survey
ACM Digital
ChatGPT in education: Partner or pariah? Joyner (2023)
Library
128

Collecting the necessary data for this SLR made it possible to gain a clearer
understanding of the specific applications of ChatGPT in education and its impact
on the educational field. This research aimed to identify the various ways in which
ChatGPT is utilized in educational settings, as well as its effects on teaching and
learning outcomes. Through a comprehensive analysis of existing studies, this
SLR uncovered valuable insights into the benefits, challenges, and implications of
incorporating ChatGPT into educational practices.
To answer each of the research questions more directly while keeping the main
focus of the SLR, the findings regarding the general view of ChatGPT usage,
alongside its risks, challenges, opportunities, and overall impact, were divided
and organized into different categories according to the subject and themes they
addressed. This categorization of research results is shown in Table 5.
Table 5: Categorization of selected articles

Research
Article title Author(s) Category
question
The role of ChatGPT, generative language
models, and artificial intelligence in
Eysenbach (2023)
medical education: A conversation with
ChatGPT and a call for papers
Game of algorithms: ChatGPT implications
Ivanov and
for the future of tourism education and
Q1: How is Soliman (2023)
research
ChatGPT used ChatGPT uses and
ChatGPT in nursing education: Is there a Thakur et al.
and/or applications in
role for curriculum development? (2023)
applied in education
Impact of artificial intelligence on dental
education? Thurzo et al.
education: A review and guide for
(2023)
curriculum update
Use of ChatGPT in ESP teaching process Kovačević (2023)
Exploring students’ perceptions of
ChatGPT: Thematic analysis and follow-up Shoufan (2023)
survey
The future of medical education and
research: Is ChatGPT a blessing or blight in Arif et al. (2023)
disguise?
Q2: What are
Chatting or cheating? The impacts of
the risks and
ChatGPT and other artificial intelligence Choi et al. (2023) Risks and challenges
challenges of
language models on nurse education of ChatGPT in
ChatGPT
The promise and peril of ChatGPT in education
usage in
geriatric nursing education: What we know Qi et al. (2023)
education?
and do not know
Can an artificial intelligence chatbot be the
Lee (2023)
author of a scholarly article?
Q3: What are ChatGPT in education: Strategies for
Halaweh (2023)
the responsible implementation Opportunities of
opportunities ChatGPT for good? On opportunities and ChatGPT in
Kasneci et al.
created by challenges of large language models for education
(2023)
using education
129
Research
Article title Author(s) Category
question
ChatGPT in
education?
Commentary: ChatGPT use in higher
education assessment: Prospects and Benuyenah (2023)
epistemic threats
Q4: What is Chatbots in education and research: A
the overall critical examination of ethical implications Kooli (2023) Overall impact of
impact of and solutions ChatGPT in
ChatGPT in ChatGPT utility in healthcare education, education
education? research, and practice: Systematic review
Sallam (2023)
on the promising perspectives and valid
concerns
ChatGPT in education: Partner or pariah? Joyner (2023)
Table 5 shows the impacts of ChatGPT usage in education. As shown in the table,
four categories were generated. The analysis of each article’s content helped to
answer the research questions of this study. Said analysis and answers to these
questions are presented in the following sections of this paper.
3.1 ChatGPT Uses and Applications in Education

Kovačević (2023) stated that ChatGPT emerged recently, bringing with it a large
number of opportunities for applications of AI. It deals with text creation and data
processing using materials gathered from machine learning, proving effective as
a time-saving tool when it comes to the preparation and implementation of
teaching units to evaluate students’ assignments. Moreover, ChatGPT has also
shed some light regarding ideas on how to apply chatbots in education, showing
its capabilities to generate virtual simulation and quizzes for students (Eysenbach,
2023). With this said, it is clear that this AI approach to education can help
educators evaluate their students, but only if they do not abuse the usage of this
tool, as the information supplied by ChatGPT is not always 100% correct.
This AI model has revolutionized education regarding what students and

researchers can do to create and process texts and assignments. In this context,
universities need to reconsider the way their teaching methods and learning
strategies incorporate the use of these kinds of tools in their teaching and become
more receptive towards written text that is generated partially by AI.
Nevertheless, they need to always remain vigilant so that students do not rely
completely on AI (Ivanov & Soliman, 2023).
AI is advancing fast, and students today have become accustomed to using

grammar correction tools, style guides, and different kinds of word processing
programs. They are not relying on some other person but solely on their own
ability to use AI tools such as ChatGPT to complement their student work. In this
case, the introduction of ChatGPT is not something new for students that are
already accustomed to the use of AI tools. However, the effective usage of said
tools is what causes concern among educators, especially when it comes to aspects
such as plagiarism (Thurzo et al, 2023). Nevertheless, in the context of AI, such
scenario is hard to prove, because there is no original author in an AI-generated
130
text. In this sense, ChatGPT proves to be useful and without risk when used
effectively if a student wishes to use this tool to verify syntax and complement an
academic work with additional information. However, it is also crucial to note
that students need to be aware of the potential legal implications when using
AI-generated text, taking note of their universities’ own rules regarding the usage
of said tools.
ChatGPT was received with both excitement and skepticism in the educational
field. Analyzing its impact on teaching and learning processes is as important as
understanding the perception students have of this AI model, as well as the
potential challenges they face when using this tool. According to Shoufan (2023),
students find ChatGPT’s capabilities interesting and helpful for their academic
work. They also find this AI tool easy to use and user-friendly, as the human-like
interaction when giving the AI-generated answers becomes part of the whole
experience. Nevertheless, as Shoufan (2023) stated, many students also feel that
these AI-generated answers are not completely accurate and they require
additional work to have a good theoretical background. The students therefore
need to have a level of understanding of the subject they want to study before
using ChatGPT in order to filter out any wrong or inaccurate information that this
AI tool may supply them. In this sense, ChatGPT can prove useful in education,
but students need to be aware of its limitations.
Thakur et al. (2023) indicated that, as an innovative AI model, ChatGPT can

facilitate different areas in the field of education, such as curriculum development,
undergraduate education, professional development, admissions process, etc.
However, to do this, it needs to be given the correct prompts to effectively aid the
processes in which this tool is used, hereby acknowledging ChatGPT’s limitations
and opportunities.
3.2 Risks and Challenges of ChatGPT in Education

The appearance of ChatGPT caused great reactions in the educational field, as this
chatbot proved itself to be very capable regarding the usage it could provide.
However, it also proved capable of deception, making its way into academic
papers, putting at risk the integrity of many researchers’ credibility when using
this tool for academic work (Lee, 2023). With the rapid evolution of technology,
authors have become more reliant on AI as an easy-to-use tool that complements
their work. However, a raising concern is the ethical aspects of these AI tools,
where misuse can lead to the lack of creative thinking and research integrity, not
to mention the inaccuracies this technology yields when not given exact prompts.
Moreover, students can also lose their ability to produce original ideas and to
present correct arguments to prove their research (Arif et al., 2023). There has yet
to be implemented a correct way to manage AI-assisted learning and to control AI
tool usage in educational institutions, but the risks that are present in its
implementation are visible and worthy of attention and analysis.
For instance, while ChatGPT is a powerful tool that can provide assistance and
information, its misuse undermines the principles of academic integrity and
critical thinking. Students who rely heavily on ChatGPT for completing their
131
homework assignments may succumb to a dependency that inhibits their own

intellectual growth. It affects their ability to engage with the material, explore
different perspectives, and develop their own ideas. In this sense, it is important
for educators and institutions to emphasize the responsible and ethical use of
ChatGPT, encouraging students to utilize it as a resource for understanding and
inspiration, rather than a shortcut to completing assignments.
ChatGPT is one of the most advanced chatbots available, but it poses a series of
challenges that prevents students from developing their critical thinking skills.
Students start to rely even more on this AI tool to answer questions during their
learning process, instead of engaging actively with the materials given to them by
their educators (Choi et al., 2023). Students must hone their independent thought
to learn to identify, analyze, and synthesize information on their own. Another
concern regarding the use of ChatGPT is the academic dishonesty, as students rely
on this tool to generate their written assignments, submitting them as their own
work. In this sense, educators need to play an important role in teaching their
students about academic honesty and the importance of critical thinking, while
also teaching them the consequences of the misuse of AI-generated texts.
Nevertheless, ChatGPT has proven advantageous to facilitate continuing

education, so its benefits should not be dismissed. Especially in an era when it is
unrealistic to ban the use of new technologies, teachers need to enhance their
efforts to teach their students regarding the correct use of chatbots and other AI
tools to complement their learning process and its potential uses in education. In
addition, as AI technology continues to grow and students become more
comfortable with its use, it is important to note that AI-assisted learning, even
though it can enhance teaching and learning processes, cannot replace human
interaction in the educational field (Choi et al., 2023).
AI application in education has become even more relevant with the constant
technological advancements. ChatGPT helps to surpass challenges due to its
potential applications to augment students’ learning experience, but it is
important to emphasize that relying completely on this technology can affect the
learning process (Qi et al, 2023), as AI tools are susceptible to misuse and prone to
be exploited by students to cheat in their exams. ChatGPT, like any AI model, is
susceptible to errors and is not 100% infallible, as many of the answers generated
by it need to be corrected. The phenomenon called “hallucination effect”, which
causes an AI technology to invent terms that it is familiar with, is present in many
AI models and is a risk that needs to be assessed when implementing these tools
in an educational context.
With this said, although ChatGPT has proven to effectively assist students with
writing text from online research, it is necessary to acknowledge that this chatbot
cannot conduct comprehensive and thorough literature analyses or critical
discussions of the articles it uses to generate their answers. Many scientific
communities thus reject the use of ChatGPT due to its irrational behavior and lack
of critical thinking (Qi et al., 2023).
132
3.3 Opportunities of ChatGPT in Education

Ever since ChatGPT was launched, teachers have expressed their concern
regarding its applications in the educational field. As critical thinking and
originality are necessary skills that need to be harnessed by the students, and as it
was discussed before, said skills can be put at risk when using ChatGPT to
generate academic work. Plagiarism is another concern, although students and
academic writers alike can still plagiarize without using ChatGPT, which is why
it may be better to assess the conduct behind these actions before banning an AI
tool without considering its benefits. For instance, Halaweh (2023) explained that
ChatGPT could be used to recollect potential ideas and issues of one subject to be
studied furthermore in the future; this would just be an aid for students to
kickstart their research without being dependent on these technologies. Even so,
teachers could also benefit from using different prompts to generate new ideas
and arguments to assess and discuss with their students in class, without putting
creativity at risk. ChatGPT’s quickness regarding these scenarios could prove
beneficial if used correctly, while understanding the potential misuse it could
have but emphasizing its positive uses.
New technologies are key to progress innovation, and despite the critical views
that many educators and institutions may have, the overall impact of AI-assisted
learning can benefit the educational field, only if students and teachers alike
deeply understand the possibilities that this technology can bring in the future.
Perhaps the development of competencies directly linked to the use of chatbots
and large language models for education could help to better understand this
technology and exploit its advantages and new dynamics that can be brought to
the learning process (Kasneci et al., 2023). This should be coupled with the
continuous oversight of responsible tutors and educators that watch over the
correct use of AI applications in education, remaining vigilant so that these
models are used in the most ethical and responsible manner by students, teachers,
and academics alike.
3.4 Overall Impact of ChatGPT in Education

New technologies diminish the importance of previous methodologies, and
teachers who are against change force themselves and their students to be against
new skills that can be taught to make effective use of said technologies. This is a
common phenomenon, in which educators will need to create new skills to adapt
to the constant technological changes (Joyner, 2023). For example, the computer
lowered the importance of the traditional pen and paper, but it allowed for the
non-linear construction of written essays at a much faster rate. Even though it had
its detractors, with time, the computer was welcomed by everyone in the
educational field. Alarmists often threat the educational systems with risks that
can be surpassed by learning, understanding, and adapting to the new
technologies. In that regard, learning ChatGPT’s limits and potential advantages
could only help educators and students optimize its use and lower the risks of its
usage.
Furthermore, as a new era of education is just around the corner, with AI-based
learning experiences growing in every corner, teachers and students need to learn
to adapt to these changes quickly. However, as Kooli (2023) indicated, the main
133
issue regarding the limits and risks of ChatGPT and other AI-based technologies
when applied to education will mainly be related to ethics. The ethical challenges
of these systems thus need to be surpassed with specific strategies that help the
educational community adapt to new technology, mitigating the potential risk for
misuse and exploitation. What is important here are the constant use of ChatGPT
in education, the potential misuse of this tool, and the regulations applied for
those who need to learn how to use this technology correctly and ethically. This
is because, when properly implemented, ChatGPT has the potential to promote
innovation in the educational field (Sallam, 2023).
While living in an era of AI-assisted learning, digital assessments will only begin
to be more innovative and engaging. Adaptability is therefore extremely
important for the educational field to adopt these technologies correctly and to let
AI systems such as ChatGPT create opportunities for development, raising
awareness of its misuse and improving the learning process. Benuyenah (2023)
indicated how, just about 23 years ago, the impact of computer usage on teachers
was met with similar disbelief at first, with the same concerns that are presented
against AI tools such as ChatGPT. Yet, teachers slowly adapted to this
technological change and continue using computers to this day, because new tools
will always emerge. If educators want to prevent potential risks, they need to
implement strategies that control AI-assisted learning and the usage of ChatGPT
to improve the learning process for their students to be prepared for new
challenges within the context of constant technological change.
4. Conclusion and Recommendations

ChatGPT is still an emerging innovation, and it will stay as an application in many
fields, education included, for a long time, as it has the potential power to enhance
both academic- and librarian-related processes. However, this potential needs to
be considered along the risks and challenges of AI-assisted learning, ensuring the
ethical and responsible use of this technology. In this sense, AI technology has the
potential to modify the way people assess different jobs, tasks, and processes, both
at a professional and academic level. ChatGPT has the special advantage that its
public availability makes it one of the most popular tools among professionals,
scholars, and students alike.
Educators should train themselves and their students to understand and properly
use the functions of this technology, depending on the scenario in which it may
be used. To stop students from misusing ChatGPT, educators could also inform
them of the difference between text generation (such as writing, editing, and
paraphrasing) and idea generation (related entirely to creativity). The further
development of tools such as this will continue, and teachers and students alike
need to be constantly educating themselves in the use of these technologies,
without putting their learning process at risk. Due to the fast transformation and
emergence of new technologies, advancements in AI technology are going to keep
improving upon what is already established, and students and educators need to
be prepared for that scenario.
134
With time, AI-assisted learning will not necessarily be a subject of debate, but
maybe just part of the new normality. Nevertheless, it is still necessary to
acknowledge that AI implementation in education needs to be overseen by
educators that understand the risks and potential benefits of these technologies to
adapt to this constantly changing world in the face of emerging new technologies
that can change the course of the educational field.
5. References
Abdullah, M., Madain, A., & Jararweh, Y. (2022). ChatGPT: Fundamentals, applications and
social impacts [Conference paper]. 2022 Ninth International Conference on Social
Networks Analysis, Management and Security (SNAMS), 1–8.
https://doi.org/10.1109/SNAMS58071.2022.10062688
Arif, T. B., Munaf, U., & Ul-Haque, I. (2023). The future of medical education and research:
Is ChatGPT a blessing or blight in disguise? Medical Education Online, 28(1),
2181052. https://doi.org/10.1080/10872981.2023.2181052
Benuyenah, V. (2023). Commentary: ChatGPT use in higher education assessment:
Prospects and epistemic threats. Journal of Research in Innovative Teaching and
Learning, 16(1), 134–135. https://doi.org/10.1108/JRIT-03-2023-097
Choi, E., Lee, J. J., Ho, M.-H., Kwok, J., & Lok, K. (2023). Chatting or cheating? The impacts
of ChatGPT and other artificial intelligence language models on nurse education.
Nurse Education Today, 125. https://doi.org/10.1016/j.nedt.2023.105796
Eysenbach, G. (2023). The role of ChatGPT, generative language models, and artificial
intelligence in medical education: A conversation with ChatGPT and a call for
papers. JMIR Medical Education, 9. https://doi.org/10.2196/46885
Guyatt, G., Meade, M. O., Richardson, S., & Jaeschke, R. (2008). What is the question? In
G. Guyatt, D. Renni, M. O. Meade, & D. J. Cook (Eds.), Users’ guides to the medical
literature: A manual for evidence-based clinical practice (2nd ed.; pp. 17–28).
McGraw-Hill.
Halaweh, M. (2023). ChatGPT in education: Strategies for responsible implementation.
Contemporary Educational Technology, 15(2), Article ep421.
https://doi.org/10.30935/cedtech/13036
Ivanov, S., & Soliman, M. (2023). Game of algorithms: ChatGPT implications for the future
of tourism education and research. Journal of Tourism Futures, 9(2), 214–221.
https://doi.org/10.1108/JTF-02-2023-0038
Joyner, D. (2023). ChatGPT in education: Partner or pariah? XRDS: The ACM Magazine for
Students, 29(3), 48–51. https://doi.org/10.1145/3589651
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F.,
Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., Krusche, S. Kutyniok, G.,
Michaeli, T., Nerdel, C., Pfeffer, J., Poquet, O., Sailer, M., Schmidt, A., Seidel, T.,
… Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large
language models for education. Learning and Individual Differences, 103, 102274.
https://doi.org/10.1016/j.lindif.2023.102274
Kooli, C. (2023). Chatbots in education and research: A critical examination of ethical
implications and solutions. Sustainability, 15(7), 5614.
https://doi.org/10.3390/su15075614
Kovačević, D. (2023). Use of ChatGPT in ESP teaching process [Symposium paper]. 22nd
International Symposium INFOTEH-JAHORINA (INFOTEH), 1–5.
https://doi,org/10.1109/INFOTEH57020.2023.10094133
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., de Leon, L., Elepaño, C.,
Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., & Tseng, V. (2023).
Performance of ChatGPT on USMLE: Potential for AI-assisted medical education
135
using large language models. PLoS Digital Health, 2(2), e0000198.

https://doi.org/10.1371/journal.pdig.0000198
Lee, J. Y. (2023). Can an artificial intelligence chatbot be the author of a scholarly article?
Journal of Educational Evaluation for Health Professions, 20(6).
https://doi.org/10.3352/jeehp.2022.20.6
Lund, B. D., & Wang, T. (2023). Chatting about ChatGPT: How may AI and GPT impact
academia and libraries? Library Hi Tech News, 40(3).
https://doi.org/10.1108/LHTN-01-2023-0009
Moreno, B., Muñoz, M., Cuellar, J., Domancic, S., & Villanueva, J. (2018). Revisiones
sistemáticas: Definición y nociones básicas [Systematic reviews: Definition and
basic notions]. Revista Clínica de Periodoncia, Implantología y Rehabilitación Oral,
11(3), 184–186. https://dx.doi.org/10.4067/S0719-01072018000300184
Nightingale, A. (2009). A guide to systematic literature reviews. Surgery, 27(9), 381–384.
https://doi.org/10.1016/j.mpsur.2009.07.005
OpenAi. (2022, November 30). Introducing ChatGPT. https://openai.com/blog/chatgpt
Pavlik, J. V. (2023). Collaborating with ChatGPT: Considering the implications of
generative artificial intelligence for journalism and media education. Journalism
and Mass Communication Educator, 78(1), 84–93.
https://doi.org/10.1177/10776958221149577
Qi, X., Zhu, Z., & Wu, B. (2023). The promise and peril of ChatGPT in geriatric nursing
education: What we know and do not know. Aging and Health Research, 3(2),
100136. https://doi.org/10.1016/j.ahr.2023.100136
Sallam, M. (2023). ChatGPT utility in healthcare education, research, and practice:
Systematic review on the promising perspectives and valid concerns. Healthcare,
11(6), 887. https://doi.org/10.3390/healthcare11060887
Shoufan, A. (2023). Exploring students’ perceptions of ChatGPT: Thematic analysis and
follow-up survey. IEEE Access, 11, 38805–38818.
https://doi.org/10.1109/ACCESS.2023.3268224
Thakur, A., Devanshi, P., & Thakur, A. (2023). ChatGPT in nursing education: Is there a
role for curriculum development? Teaching and Learning in Nursing, 18(3), 450−451.
https://doi.org/10.1016/j.teln.2023.03.011
Thurzo, A., Strunga, M., Urban, R., Surovková, J., & Afrashtehfar, K. (2023). Impact of
artificial intelligence on dental education: A review and guide for curriculum
update. Education Sciences, 13(2), 150. https://doi.org/10.3390/educsci13020150
Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., &
Agyemang, B. (2023). What if the devil is my guardian angel: ChatGPT as a case
study of using chatbots in education. Smart Learning Environments, 10(1), 15.
https://doi.org/10.1186/s40561-023-00237-x
136

Vol. 22, No. 7, pp. 136-155, July 2023
Received Mar 9, 2023; Revised Jun 30, 2023; Accepted Jul 7, 2023
Project-based Learning to Promote Learner

Autonomy in Training Hospitality Education at a
Technical and Vocational Education and
Training College
Shawn Lourens Green
Tshwane South TVET College, Pretoria, South Africa
Elizabeth Catharina (Elize) du Plessis

University of South Africa, Pretoria, South Africa
Abstract. Project-based learning (PjBL) is a teaching and learning

approach that involves students actively working on real-world projects,
to develop knowledge and skills. This method has been shown to be
effective in promoting learner autonomy (LA) and skills development in
language learning studies. There is, however, a dearth of research on the
use of PjBL in hospitality education, especially in a technical and
vocational education and training (TVET) environment. Using a case
study research strategy, this study aimed to investigate whether PjBL
could be used to promote autonomy and develop employable skills in
students in hospitality education. A convergent mixed methods approach
was used to develop a framework for PjBL in promoting LA in a TVET
environment. The study included a closed-structured questionnaire with
responses from 144 (n=144) students from two hospitality programmes,
as well as a semi-structured interview with 18 student participants. The
findings indicate that PjBL is an effective teaching and learning approach
in hospitality education that can promote LA, lead to a deeper
understanding of the subject matter, while facilitating the development
of a variety of important skills and competencies. Students can apply their
knowledge and skills in meaningful ways through hands-on, real-world
activities, which can foster a greater sense of autonomy and responsibility
for their own learning. Moreover, this study emphasises the significance
of incorporating PjBL into hospitality education programmes, to prepare
students for success in a rapidly changing world.
Keywords: hospitality education; further education; learner autonomy;

project-based learning
©Authors
137
1. Introduction and background of the study

Autonomy is considered both a precondition and a vital learning outcome in the
academic success of students (Ding & Yu, 2021) and their ability to independently
gain knowledge and skills (Bei et al., 2019). Little et al. (2017) posit that learner
autonomy (LA) is not an option for educational institutions, but rather a
pedagogical imperative. Current trends show an expanding recognition of the
significance of LA in academic achievement (Ginting et al., 2020; Melvina & Julia,
2021) and the role of students in directing their own learning process (Alonazi,
2017; Saeed, 2021; Tomasouw & Marantika, 2020; Yu, 2020). The successful
transition of students to the technical and vocational education and training
(TVET) environment is not only about academic competence (i.e., the skills,
attitudes and behaviours that contribute to a student's academic success), but also
about adjusting to a learning environment that requires greater autonomy and
individual responsibility. One of the approaches used in promoting LA involves
integrating project-based learning (PjBL) into the curriculum.
Scholars believe that PjBL, as an experiential learning teaching and learning

pedagogy, is a crucial instructional approach that enables students to develop
content knowledge and academic skills (Danko, 2019; Indrawan et al., 2020;
Pradanti & Muqtada, 2023), develop and improve skills for future success (ESEI,
2021; Stehling & Munzert, 2018), develop LA (Boggu & Sundarsingh, 2019; Yuliani
& Lengkanawati, 2017; Zaidi et al., 2020), and build the necessary personal agency
to meet the challenges of life and the wider world (ESEI, 2021; High-Quality
Project Based Learning, 2018; Stehling & Munzert, 2018). However, although
implementing PjBL in curricula may prove to be advantageous, educators may
experience challenges in its implementation and application (Aldabbus, 2018;
Juliet, 2020; Vasiliene-Vasiliauskiene et al., 2020), especially in a TVET context (H.
Liu, 2019; Mustapha et al., 2020).
Research shows that the use of PjBL in a TVET context does not always have the
desired teaching and learning effects. This may be because vocational educators
are not well versed in PjBL, are unable to adequately supervise students during
this process, and encounter difficulties when executing PjBL in practice (H. Liu,
2019). Van de Pol et al. (2019) argue that, in practice, it is not always clear how to
support students, or how the educator can ensure that students are ready and
capable to assume responsibility for their studies. Furthermore, research shows
that traditional, educator-centred styles remain dominant in the practice of PjBL
in TVET, and that educators are overly involved in the implementation of projects,
depriving students of control and ownership (H. Liu, 2019). Moreover, students
lack the autonomy necessary to plan for, prepare, engage in and/or manage their
own learning (Borg & Alshumaimeri, 2019; H. Liu, 2019). When students are
involved in a project, they are not always sufficiently active, leading to a lack of
interest, motivation and creativity on their part, to achieve good learning
outcomes (H. Liu, 2019). Studies also show that students are not aware of the
concept of autonomy, that educators and students do not favour autonomous
learning (Boggu & Sundarsingh, 2019), and that some students need guidance and
support from their educators to become autonomous (Reswari & Kalimanzila,
138
2021). Lastly, Jansen et al. (2020) concede that many students struggle to
successfully regulate their learning process.
2. Aim of the study

In order for students to develop into autonomous learners, they require ongoing
support, guidance and commitment from their educators. In doing so, there is a
need for the design and implementation of PjBL to promote LA, and to create a
learning environment in which students can acquire the necessary skills to
succeed both academically and within the hospitality industry.
This study aims to contribute insight into how PjBL, as a teaching and learning
pedagogy, promotes hospitality students’ autonomous learning in gaining
work-related skills and competencies based on their beliefs around and
perceptions of LA in a TVET context. The study addresses a gap in the research,
which is currently dominated by literature on students’ beliefs and perceptions in
promoting LA in the context of the English language and English as a foreign
language (EFL) (Barin & Eyerci, 2021; Bhattarai, 2021; Iamudom &
Tangkiengsirisin, 2020; Jose et al., 2020; Kim & Yoon, 2021; Pham, 2021; Reswari
& Kalimanzila, 2021; Yaprak, 2021; Yu, 2020; Yuliani & Lengkanawati, 2017;
Zourez, 2019), or has been conducted at either the primary and secondary
schooling level (Wirapatni et al., 2021; Yuliani & Lengkanawati, 2017; Zaidi et al.,
2020) or the university level (Padmadewi et al., 2020; Tran, 2020; Yasmin et al.,
2020). There is a dearth of research into how PjBL can promote LA, especially
within TVET hospitality education, to enable students to acquire employable
skills and competencies.
Furthermore, this is to provide a framework for PjBL in promoting LA in the TVET

milieu. The outcomes of this study can contribute to educators designing,
implementing and facilitating better learning experiences for students, using PjBL
to promote autonomous learning and help students achieve academic success. To
that end, the study addresses the following research question: How can PjBL
promote LA in hospitality students studying hospitality education at a TVET
college?
3. Review of the literature

3.1. Conceptualising learner autonomy
Autonomy is not a universal concept that is understood similarly around the
world. As Teng (2019) notes, theoretical discussions around LA are far from
coherent, consistent or systematic. Many scholars use LA synonymously with
autonomous learning (Kyu, 2018; Thanh, 2019; Yu, 2020; Zaidi et al., 2020), learner
independence (Kim & Yoon, 2021; Zaidi et al., 2020), independent learning,
self-directed learning or self-direction (Al-Khawlani, 2018; Gulyamova &
Kadirova, 2021; Hawkins, 2018), learner-centredness (Kyu, 2018; Yu, 2020),
learner self-regulation (Hawkins, 2018; Oxana et al., 2020) and self-managed
learning (Marsevani, 2021). It is a difficult and complex concept to define precisely
(Gulyamova & Kadirova, 2021; Kim & Yoon, 2021; Sereti & Giossos, 2018; Teng,
2019), because of its broad and abstract nature (Oxana et al., 2020).
139
Holec (1981, as cited in Tran, 2020, p. 134) views autonomy as “the ability to take
charge of one’s own learning”, in an “individual capacity”. Benson (as cited in
Tomasouw & Marantika, 2020, p. 505) describes autonomy “as the capacity to take
control of one’s learning”. Although the terms ability and capacity are used in these
definitions, they are often widely used interchangeably. Saglam (2018) defines
autonomy as “the ability of the learner to take responsibility for his/her own
learning and monitor own learning process”. Scharle and Szabó (2000, as cited in
Iamudom & Tangkiengsirisin 2020, p. 201) define autonomy as “the freedom and
ability to manage one’s own affairs, which entails the right to make decisions as
well”. Note the use of “take charge of”, “take control of”, “take responsibility for”
and “manage” (Blidi, 2017, pp. xxiv; 82) in these four definitions.
In addition to emphasising ability/capacity and taking

charge/control/responsibility, scholars mention two affective factors that affect
students’ autonomy, namely motivation and willingness (Bhattarai, 2021; Ceylan,
2021; Iamudom & Tangkiengsirisin, 2020; Little et al., 2017; Orakcı, 2021; Yu,
2020). Nguyen (2014, as cited in Alrabai, 2017, p. 212) defines LA as a “learner’s
willingness and ability to take responsibility to plan, implement, monitor and
evaluate his/her learning in tasks that are constructed in negotiation with and
support from the teacher”. The emphasis is on students being positive and active
in their learning.
3.2. Project-based learning as a teaching and learning pedagogy

PjBL involves an inquiry-based instructional method (Albar & Southcott, 2021;
Guo et al., 2020) that involves students in the construction of knowledge by
having them accomplish meaningful projects and develop real-world products
(Guo et al., 2020; Pradanti & Muqtada, 2023), presentations, or performances over
a given period (Albar & Southcott, 2021). These projects usually involve elements
of researching a complex problem, question or challenge, as an extension of what
has already been learned in class, before presenting it as a project (Güven & Valais,
2014). A collaborative learning environment is created in which students work in
teams or pairs, supervised by an educator, a facilitator or a mentor (Albar &
Southcott, 2021; Budhai & Skipwith, 2022; Roland, 2017). Krajcik and Shin (2014)
report that the PjBL environment comprises six key elements: (i) a driving
question, (ii) a focus on learning goals, (iii) participation in educational activities,
(iv) engaging through collaboration, (v) scaffolding with the use of learning
technologies, and (vi) creating a tangible product.
In engaging with the project, students may encounter problems that need to be
addressed, requiring them to construct and present an end product in response to
a driving question (Albar & Southcott, 2021; Pradanti & Muqtada, 2023). The
educator interacts with the students to guide them to frame meaningful questions,
facilitate student dialogue in knowledge development and organise tasks, and he
or she provides ongoing feedback to students on what they have learned from
their experiences (Budhai & Skipwith, 2022; Güven & Valais, 2014). Educators and
students thus develop an “inclusive relationship learning partnership” (Güven &
Valais, 2014, p. 184).
140
Moreover, in PjBL the role of the students is to investigate significant questions

that require them to gather information and think critically (H. Liu, 2019). This
allows them to learn by stimulating their interest and motivation, and capacitates
them to apply new knowledge learned in a problem-solving context (Budhai &
Skipwith, 2022). Fini et al. (2018) assert that PjBL allows different groups of
students to work together to solve practical problems, before presenting and
defending their approaches and solutions. This entails promoting their
intellectual and social development, requiring them to actively participate in the
process of acquiring knowledge, while improving their communication and
interpersonal skills, and enhancing their leadership skills and creativity (Fini et
al., 2018).
In PjBL, the “voice and choice” of students are fostered through carefully
managed and planned instructional benchmarks (Güven & Valais, 2014, p. 184),
with regular formative assessments of these benchmarks serving to guide them,
even as their progress with the project encourages them to dig deeper into the
concepts learned.
3.3. Project-based learning in promoting learner autonomy in hospitality

education
Zaidi et al. (2020) conclude that the use of PjBL enhances LA and has been proven
to be effective in improving cognitive ability, improving students’ use of critical
thinking skills, enhancing students’ ability to obtain, retain and retrieve the
knowledge to increase the achievement of students. This is furthermore
supported by studies by Tran and Tran (2020), Ayu Sukerti and Yuliantini (2018),
Van Loi (2017), and Yuliani and Lengkanawati (2017), which emphasise the
promotion of LA through PjBL. By promoting LA, students are more likely to
actively engage, explore their interests and develop the necessary knowledge and
skills to excel in their chosen hospitality profession, thus promoting lifelong
learning (X. Liu et al., 2020), critical thinking and adaptability, which are all
essential attributes in the dynamic ever-evolving field of hospitality.
4. Theoretical framework
Kolb’s experiential learning theory (ELT) is an important and influential approach
to education that focuses on how students learn best by experiencing the material
they are studying. ELT is derived from the work of 20th-century foundational
scholars such as John Dewey, William James, Kurt Lewin,
Lev Vygotsky, Jean Piaget, Paulo Freire, Carl Jung, Carl Rogers and Mary Follett,
who placed experience at the centre of the learning process, thereby envisaging a
learner-centred educational system (Kolb & Kolb, 2017; Passarelli & Kolb, 2020).
Dewey (as cited in Vasiliene-Vasiliauskiene et al., 2020) postulates that the nature
of the experience is continuous, and the experiential learning process is
fundamentally important in the shaping of students’ learning. Dewey (as cited in
Passarelli & Kolb, 2020) considers experience, inquiry and reflection to be key
components of experiential learning. In addition, practical and varied experiences
are deemed to improve students’ preparedness for life holistically, with varied
activities being more beneficial than traditional curricula (Dewey, 1986 as cited in
Bradbury, Schwarz & Lenton, 2021).
141
Currently, ELT (Passarelli & Kolb, 2020) is known as a dynamic, holistic model
that defines learning as the major process of human adaptation, involving the
person in his or her entirety (Bell & Bell, 2020; Kolb & Kolb, 2017; Passarelli &
Kolb, 2020). Kolb’s ELT proposes that individuals learn through a four-stage
process that includes concrete experiences, reflective observation, abstract
conceptualisation, and active experimentation (Kolb & Kolb, 2018; Passarelli &
Kolb, 2020). According to Kolb (cited in Kolb & Kolb, 2017, 2018; see also Passarelli
& Kolb, 2020), these stages form a continuous cycle in which individuals engage
in experiences, reflect on those experiences, form new concepts and theories, and
test those concepts and theories in new experiences.
Kolb’s theory has been widely applied in various fields, including education,
information science, business and psychology (Passarelli & Kolb, 2020). In
education, Kolb’s theory has been used to design instructional strategies to
promote active and reflective learning (Askren & James, 2021; Patil et al., 2020).
Empirical studies have provided support for Kolb’s theory. The findings reported
by Calderón Carvajal et al. (2021) support the hypothesis of four learning modes.
As Pamungkas et al. (2019) found, implementing Kolb's ELT could enhance
students’ conceptual understanding, and allow them to directly develop their
knowledge and abilities.
Some are, admittedly, critical of Kolb’s theory, based on systematic reviews

(Calderón Carvajal et al., 2021; Johnson et al., 2020; Kumar & Bhandarker, 2020;
Matsuo & Nagata, 2020; Morris, 2020; Ndlovu & Nyane, 2018). Many believe ELT
lacks sound theoretical and empirical foundations (Bell & Bell, 2020; Burch et al.,
2019; Morris, 2020), and they question the premise on which it is based, the design
and acceptance of its constructs, and its generalisability and effectiveness (Kumar
& Bhandarker, 2020). Calderón Carvajal et al. (2021, p. 605) found that Kolb’s
model reflects the presence of the four learning modes, but not the “orthogonal
bipolar structure”. These results therefore confirm the existence of learning
modes, but not of learning styles, which are deemed non-viable due to their failure
to comply with the orthogonal bipolar structure.
Despite these criticisms, Kolb’s ELT has been influential in the field of education
and continues to be widely used in various disciplines (Kolb & Kolb, 2018; Morris,
2020), especially hospitality education (Askren & James, 2021; Dillette & Sipe,
2018; Zisan, Albattat, & Bvvasar, 2021). It provides a valuable framework for
understanding strategies that promote active and reflective learning.
5. Research methodological approach

5.1. Research approach and the data collection instrument
This study employed the convergent mixed-methods research approach to gain a
thorough understanding of the topic under study. Creswell and Guetterman
(2021) define convergent mixed methods as a research approach in which
quantitative and qualitative (QUAN + QUAL) data are collected concurrently,
weighed equally, analysed independently, and then interpreted jointly. The
142
researchers triangulated the methods for corroboration and validation by directly

comparing the QUAN results with the QUAL findings.
Two scales were used to collect QUAN data on students' perceptions of LA and
experiential learning. The researchers used Bei et al. (2019) five-point Likert-type
scale, developed to measure LA from the dimensions of personal and educational
autonomy. The second scale, a seven-point Likert-type experiential learning scale,
developed by Clem et al. (2014), was used to collect data from respondents' PjBL
experiences and assess their perceptions of experience-based instruction.
Furthermore, the QUAL data was collected through semi-structured interviews
whereby only two of the 20 questions posed to answer the research question, are
addressed in this article. They are: (i) What skills do you have that allow you to
be an autonomous student?, and (ii) Is the self-reflection report important or not
important in your development as a student? Please explain your answer.
5.2. Data collection and the analysis process

Before conducting research, ethical clearance was obtained by the University of
South Africa’s College of Education Ethics Review Committee and the TVET
college where the study was conducted. Students enrolled in the selected TVET
college during the 2021 and 2022 academic years, who completed a project in
either Catering Theory and Practical N6 (on the National Accredited Technical
Education Diploma [NATED] level) or Hospitality Services level 3 (on the
National Certificate Vocational [NC(V)] level), were invited to volunteer for the
study. Those who completed the paper-based survey were asked to also
participate in the semi-structured interviews.
In total, n=144 responses were received from a population of N=181, selected

using simple random sampling. A 99.0% confidence level with an E=4.87% was
achieved. Noori (2021) and Cohen et al. (2018) postulate that, in educational
research, confidence levels of 95 to 99 per cent are commonly used. Cronbach’s
alpha was used to test the reliability of both the LA and experiential learning
scales. In the end, 18 students participated in the semi-structured interviews.
Once the paper-based surveys were completed, the researcher captured the data
in Google Forms and then the data was uploaded into IBM SPSS (version 6) for
analysis. The QUAN data were run on all the survey items to understand the
pattern of responses within the sample and to describe the sample in terms of the
constructs under investigation. In order to answer the research question, both
simple linear and multiple regression analyses were performed to determine
whether a statistically significant relationship exist (Pallant, 2020). The QUAL
data obtained through semi-structured interviews were recorded and then
transcribed in Microsoft Word® and uploaded to Atlas.ti ™ 22.2 to store and
organise the data, compare codes and produce visual representations, that is, a
word cloud of the QUAL data of the most commonly used words among
responses.
143
5.3. Reliability of the questionnaire

To determine whether the QUAN instrument was successful in gathering accurate
data on students’ perceived LA and PjBL, the Cronbach α was used to evaluate
internal consistency and reliability (see Table 1).
Table 1: Table of Cronbach Alpha co-efficient reliability estimates for the personal
autonomy scale (n=144)
Scales Items Items Cronbach Reliability Skewness
left α co- interpretation
out efficient
Personal 1, 2, 3, 4, 5, 6, 7 None 0.600 Moderate -0.507
autonomy
Educational 8, 9, 10, 11, 12, 13, None 0.507 Moderate -0.215
autonomy 14, 15, 16
Experiential 1, 2, 3, 4, 5, 6, 7, 8, None 0.802 High -0.897
learning 9, 10, 11, 12, 13, 14,
15, 16, 17, 18, 19,
20, 21, 22, 23, 24,
25, 26, 27, 28
Sub-scales of experiential learning scale
Sub-scales Items Items Cronbach Reliability Skewness
left α co- interpretation
out efficient
Authenticity 1, 2, 4, 5 3 0.626 Moderate -1.210
Active 6, 7, 8, 10, 11, 12 9 0.578 Moderate -0.622
learning
Relevance 13, 14, 15, 16, 17, None 0.721 High -0.804
18, 19, 20, 21
Utility 22, 23, 24, 25, 26, None 0.606 Moderate -0.897
27, 28
As the study sought to investigate what is involved in project-based teaching and

learning which seeks to promote LA in hospitality students at a TVET college, the
researcher sought to obtain reliability on the global scales of personal autonomy,
educational autonomy and experiential learning, for both the descriptive statistics
and the simple linear regression analysis. As seen in Table 1, all three scales
attained adequate reliability: The personal autonomy scale had a Cronbach α of
0.600 (moderate reliability), the educational autonomy scale 0.507 (moderate
reliability), and the experiential learning scale 0.802 (high reliability).
Next, the sub-scales of the experiential learning scale were examined for
reliability, to complete a multiple regression analysis. For the two sub-scales
– authenticity and active learning – to be deemed reliable, one item of each
subscale was omitted, having achieved a negative corrected item correlation.
Item 3 of the authenticity subscale “The environment I learn in does not enhance
the learning experience” had an item correlation of -0.260, while item 9 of the
active learning subscale “I find this experience boring” had an item correlation of
-0.073. Both items received a negative correlation, as they were negatively stated
in the questionnaire which contained both positive (regular) and negative
(reversed) statements. Scholars who have researched similar questionnaires with
144
both types of structured questions and/or statements combined in a single test,

argue that the reliability of such questionnaires may be significantly negatively
affected (Chyung et al., 2018; Suárez-Alvarez et al., 2018; Zeng et al., 2020).
After removing the aforementioned two items, the Cronbach α for authenticity
was 0.626 (moderate reliability), and for active learning 0.578 (moderate
reliability), relevance 0.721 (high reliability), and utility 0.606 (moderate
reliability). A Cronbach α of 0.5 and above is acceptable, according to Hinton et
al. (2014). As a result, it was deemed that the three scales and four sub-scales, with
Cronbach's α values ranging from 0.507 to 0.802, were adequate and could be used
in the study (Hinton et al., 2014; Suntharalingam et al., 2021).
6. Results and findings of the study

6.1. The results on the contribution of project-based learning in promoting
learner autonomy using simple and multiple regression analysis
To find out if the data was multicollinear, correlation analysis was utilised. The
first regression analysis performed was simple linear regression analysis to
estimate the relationship between two variables. Two simple linear regressions
were run to predict personal and educational autonomy from experiential
learning. Table 2 shows the results of these regressions.
Table 2: Simple linear regression results showing the contribution of experiential

learning, to promoting personal and educational autonomy
ANOVA Coefficients
Adjusted R2
F Sig B t Sig
Personal autonomy 0.076 12.710 <0,001 0.071 3.565 <0,001
Educational autonomy 0.101 17.147 <0,001 0.091 4.141 <0,001
Experiential learning statistically significantly predicted personal autonomy and

explained 7.6% of its variance (F=12.170, p< 0.01, adj. R2 = 0.076). There was a
significant positive relationship between these two variables, with a one unit
increase in experiential learning associated with a 0.071 unit increase in the total
personal autonomy score. Similarly, experiential learning statistically
significantly predicted educational autonomy and explained 10.1% of its variance
(F=17.147, p < 0.05, adj, R2 = 0.0101). There was also a significant positive
relationship between these variables, with a one unit increase in experiential
learning associated with a 0.091 unit increase in the educational autonomy score.
In addition, multiple regressions were run to determine the contribution of

experiential learning sub-scales to promoting personal and educational
autonomy. The results are shown in Table 3.
145
Table 3: Multiple regression results showing the contribution of experiential learning

sub-scales to promoting personal and educational autonomy
ANOVA Coefficients
Adjusted R2
F Sig B t Sig
Personal autonomy
Experiential learning
0.091 4.600 0.002
combined model
Authenticity -0.040 -0.475 0.636
Active learning 0.231 2.573 0.011
Relevance 0.019 0.297 0.767
Utility 0.110 1.560 0.121
Educational autonomy
Experiential learning
0.105 5.193 0.001
combined model
Authenticity 0.124 1.317 0.190
Active learning -0.040 -0.408 0.684
Relevance 0.240 3.334 0.001
Utility -0.046 -0.585 0.559
Active learning made a statistically significant unique contribution to the

prediction of personal autonomy after all other sub-scales were controlled for
(t=2.573, p< 0.05), with a one unit increase in active learning associated with a
0.231 unit increase in personal autonomy. As for educational autonomy, it was
found that relevance scores made a statistically significant unique contribution to
its prediction when other sub-scales were controlled for (t=3.334, p<0.05). A one
unit increase in the relevance subscale score was associated with a 0.240 unit
increase in educational autonomy.
6.2. The findings of the qualitative data

Two questions were used from the QUAL data findings to support the
development of the framework for this study. Participants were asked what skills
they possess, to allow them to be autonomous. Figure 1 illustrates the participants’
word frequency, using Atlas.ti 22, in describing their autonomous skills.
Figure 1: Participants’ opinions on the skills that allow them to be autonomous
146
The most widely used term participants used to express their skills was motivation,
followed by teamwork. Words that received the same count were creative/creativity,
leadership, responsibility, planning, organising and goals. Other common words were
communication, confidence, decision making, independence, positivity, problem-solving,
social interaction, critical thinking and time management. This indicates that
participants viewed themselves as applying the aforementioned skills in their
studies, and while completing the project. All the skills shown in Figure 1 are
necessary for promoting LA.
Next, participants were asked whether or not self-reflection is important in their

development as students. The majority stated that self-reflection is an important
part of PjBL, as it helps them reflect on how they performed during the project.
Participant 9 stated:
Yes, it is important, because you have to give feedback on the reason for
the rating, things you find challenging, and what you experienced, what
could you have done to sell more products. I gave myself a rating of how
I performed at the service. So it is important for me to reflect on what I
have done there. I learned something from this. I learned about the
challenges I got from the services and the feedback of the customers
regarding the service.
Participants indicated that, by reflecting, they could identify areas for growth, or
where they need to acquire new skills.
6.3. Proposed framework based on the findings of the study

The aim of this study was to investigate what is involved in project-based teaching
and learning, to promote LA in hospitality students at a TVET college and develop
a framework for promoting such autonomy. A proposed framework, based on the
synthesis of the QUAN results and the QUAL findings, was developed by the
authors for PjBL in promoting LA in a TVET environment (see Figure 2).
147
Figure 2: Framework for project-based learning in promoting learner autonomy in

technical and vocational education and training contexts
The two main participants in the framework – the educator and the student – form
an “inclusive relationship learning partnership” (Güven & Valais, 2014, p. 184).
Here, the educator’s role is to facilitate the learning process for students, rather
than simply delivering content (Budhai & Skipwith, 2022). The educator should
assist students in defining and clarifying project goals, ensure that resources are
available to students for PjBL, provide guidance and support as needed, and
ensure that students stay on track and make progress towards completing the
project (Budhai & Skipwith, 2022; Güven & Valais, 2014). Moreover, the educator
serves as a mentor (Roland, 2017), creating an environment in which students can
develop the necessary knowledge, skills, attitudes and competencies required for
workplace and academic success, as well as LA (Bei et al., 2019; Boggu &
Sundarsingh, 2019; Danko, 2019; Indrawan et al., 2020; Pradanti & Muqtada, 2023;
Yuliani & Lengkanawati, 2017; Zaidi et al., 2020). Furthermore, the educator
should provide students with continuous feedback on the progress of their work,
assist them in refining their ideas and improving the quality of their projects
(Budhai & Skipwith, 2022), and encourage them to delve deeper into concepts
learned (Güven & Valais, 2014). Lastly, the educator must evaluate the
competency of the students’ final product.
The students’ role is to own their learning and actively participate in the learning
process (Alonazi, 2017; Saeed, 2021; Tomasouw & Marantika, 2020; Yu, 2020) to
gain the necessary experience by reflecting on their experiences and linking that
to future action (Kolb & Kolb, 2017, 2018; Passarelli & Kolb, 2020). They are also
responsible for identifying and researching a topic of interest, developing a plan
148
to learn more about it, and presenting and defending their findings in a final
project (Fini et al., 2018; Güven & Valais, 2014). Students should play an important
role in the collaborative aspect of PjBL by brainstorming ideas, sharing resources,
reflecting, providing feedback, and supporting one another throughout the
project. This will assist them in developing the necessary knowledge, skills,
attitudes and competencies, and allow them to become autonomous (Fini et al.,
2018).
A positive and significant relationship was found between PjBL and personal and
educational autonomy. As a result, the more effective PjBL is in its design to
achieve the desired outcomes, the greater the degree of LA in the form of personal
and educational autonomy (Boggu & Sundarsingh, 2019; Yuliani &
Lengkanawati, 2017). Thus, educators and PjBL designers must ensure that the
project encourages students to participate actively in real-world problems or
challenges that are personally meaningful to them (Guo et al., 2020; Pradanti &
Muqtada, 2023). A positive and significant relationship was found between active
learning and personal autonomy. As the results indicate, students’ level of
engagement in the project increased their level of personal autonomy. They were
therefore more likely to be able to explore their own interests and feel in control
of their own learning, when they were mentally and/or physically engaged in the
development of an authentic product for the project (Guo et al., 2020; Pradanti &
Muqtada, 2023).
Relevance and educational autonomy were found to have a positive and

significant relationship. Learning becomes more meaningful and effective when
linked to students’ own experiences and interests, allowing them to see the
relevance of the material they are learning to their own lives and futures (Budhai
& Skipwith, 2022; Güven & Valais, 2014). Students are therefore more likely to
invest time and effort in learning when they believe the material is important or
useful to them. Moreover, PjBL is seen as an effective way of promoting relevance,
as it allows students to apply the knowledge and skills they learn, to real-world
problems or challenges (Guo et al., 2020). Therefore, incorporating PjBL into the
hospitality education curriculum in TVET is viewed as a teaching and learning
pedagogy in which students learn by actively engaging in real-world experiences,
reflecting on those experiences, and participating in personally meaningful
projects that promote their autonomy.
7. Conclusion
The purpose of this study was to investigate what is involved in project-based
teaching and learning aimed at promoting LA in hospitality students at a TVET
college. The findings were used to develop a framework for PjBL, to promote
autonomy in TVET hospitality students. In this study, PjBL was shown to be an
effective teaching and learning approach in hospitality education that can
promote LA, lead to a deeper understanding of the subject matter, and facilitate
the development of a variety of important skills and competencies needed for the
world of work. Students can apply their knowledge and skills in meaningful ways
by actively participating in hands-on, real-world activities, which can foster in
them a greater sense of ownership and responsibility for their own learning. This
149
approach allows students to take the lead in their own learning, and make
decisions about the direction of their projects, thereby promoting independence
and developing self-direction. Moreover, this study emphasises the significance
of incorporating PjBL into hospitality education programmes to promote LA
through active learning and student-centered projects. PjBL is thus recognised as
a valuable experiential learning approach, with the potential to promote deeper
learning outcomes. The implication for practice is that the findings of the study
and the suggested framework could be used by educators in designing,
implementing and facilitating a better learning experience for their students
through the use of PjBL to support student independence and help them achieve
academic success.
8. Recommendations and future research

This study recommends the following improvements or suggestions for
educators, to ensure that PjBL is effectively applied to the TVET hospitality
education curriculum:
1. The educator should provide guidance, support and inspiration to students
throughout PjBL. Many students will be experiencing PjBL for the first time,
so educators must strike a balance in assisting them during the process, so as
not to take control away from them, but rather to guide, support and mentor
them to achieve the project outcomes.
2. The educator should engage with students in selecting the project topic,
problem, challenge and/or content, based on the project structure provided
by the assessment guidelines. Allowing students to select their own project
topic, problem, challenge and/or content ensures that the project is relevant
to their needs. It can also be an effective way of engaging students in the
learning process, as they are more likely to be invested in the project and
motivated to complete it – all of which encourages autonomous learning.
3. Following the agreement by both students and educators on the project topics
and outcomes, educators should assist students in developing realistic and
measurable goals to achieve through PjBL. Setting goals allows students to
focus their efforts, giving them a clear picture of what they are working
towards. Furthermore, involving students in the process of setting goals and
identifying the next steps during the project, will allow them to take
ownership of their learning and development.
4. Educators should provide students with continuous feedback (not only at the
end of the project), to help them understand how their work is progressing,
and identify areas where they need to focus additional effort. It may also help
them understand what is expected of them, and how they can improve.
5. Self-reflection is an important aspect of PjBL. Reflection provides students
with an opportunity to analyse information, solve problems and make
decisions. Educators should ensure that students possess the required skills to
self-reflect, in order for them to learn and develop new concepts or skills and
produce a final product for the project.
The focus of this study was on two specific areas of PjBL in promoting LA, namely
hospitality education and the TVET environment. Future research could look into
other fields/programmes within this environment, as well as hospitality
150
education at other types of institutions, to determine whether the results and/or

findings will yield a similar framework. Furthermore, research could be
conducted to determine whether other types of classroom-based, online or virtual
experiential learning teaching and learning approaches can be used to promote
LA amongst TVET hospitality students.
9. References
Al-Khawlani, A. (2018). The influence of the learning environment on learner autonomy:
A comparative study of Polish and Yemeni EFL undergraduate learners. Sino-US
English Teaching, 15(3), 109–124. https://doi.org/10.17265/1539-
8072/2018.03.001
Albar, S. B., & Southcott, J. E. (2021). Problem and project-based learning through an
investigation lesson: Significant gains in creative thinking behaviour within the
Australian foundation (preparatory) classroom. Thinking Skills and Creativity,
41(May), 1–19. https://doi.org/10.1016/j.tsc.2021.100853
Aldabbus, S. (2018). Project-based learning: Implementation & challenges. International
Journal of Education, Learning and Development, 6(3), 71–79.
https://www.eajournals.org/journals/international-journal-of-education-
learning-and-development-ijeld/vol-6-issue-3-march-2018/project-based-
learning-implementation-challenges/
Alonazi, S. M. (2017). The role of teachers in promoting learner autonomy in secondary
schools in Saudi Arabia. English Language Teaching, 10(7), 183.
Alrabai, F. (2017). Saudi EFL teachers’ perspectives on learner autonomy. International
Journal of Linguistics, 9(5), 211–231. https://doi.org/10.5296/ijl.v9i5.11918
Askren, J., & James, W. (2021). Experiential learning methods in culinary course can
bridge the gap: Student perceptions on how hands-on curriculum prepares them
for industry. Journal of Hospitality and Tourism Education, 33(2), 111–125.
https://doi.org/10.1080/10963758.2020.1791134
Barin, M., & Eyerci, A. (2021). A survey on the relationship between willingness to
communicate and learner autonomy of Turkish EFL students. Shanlax
International Journal of Education, 9(S1), 104–116. https://doi.org/10.34293/
education.v9iS1-May.4006
Bei, E., Mavroidis, I., & Giossos, Y. (2019). Development of a scale for measuring the
learner autonomy of distance education students. European Journal of Open,
Distance and E-Learning, 22(2), 133–144. https://doi.org/10.2478/eurodl-2019-
0015
Bell, R., & Bell, H. (2020). Applying educational theory to develop a framework to
support the delivery of experiential entrepreneurship education. Journal of Small
Business and Enterprise Development, 27(6), 987–1004.
https://doi.org/10.1108/JSBED-01-2020-0012
Bhattarai, P. (2021). English language teachers’ perceptions and practices on learner
autonomy in Nepalese context. Journal of Research and Innovation in Language,
3(1), 17–25. https://doi.org/10.31849/reila.6064
Blidi, S. (2017). Collaborative learner autonomy: A mode of learner autonomy development.
Springer. https://doi.org/10.1007/978-981-10-2048-3
Boggu, A. T., & Sundarsingh, J. (2019). An experiential learning approach to fostering
learner autonomy among Omani students. Journal of Language Teaching and
Research, 10(1), 204–214. https://doi.org/10.17507/jltr.1001.23
Borg, S., & Alshumaimeri, Y. (2019). Language learner autonomy in a tertiary context:
Teachers’ beliefs and practices. Language Teaching Research, 23(1), 9–38.
https://doi.org/10.1177/1362168817725759
151
Bradbury, T., Schwarz, E. C., & Lenton, A. (2021). Experiential learning from an
international perspective: An empirical study of the United States, Canada,
Australia and New Zealand. International Journal of Sport Management Volume, 22,
55–93. https://vuir.vu.edu.au/42188/8/5DecemberECS-
FINAL_EDITS_Experiential_learning_from_an_international_perspective.pdf
Budhai, S. S., & Skipwith, K. B. (2022). Best practices in engaging online learners through
active and experiential learning strategies (2nd ed.). Routledge.
https://doi.org/10.7771/1541-5015.1764
Burch, G. F., Giambatista, R., Batchelor, J. H., Burch, J. J., Hoover, J. D., & Heller, N. A.
(2019). A meta-analysis of the relationship between experiential learning and
learning outcomes. Decision Sciences Journal of Innovative Education, 17(3), 239–
273. https://doi.org/10.1111/dsji.12188
Calderón Carvajal, C., Ximénez Gómez, C., Lay-Lisboa, S., & Briceño, M. (2021).
Reviewing the structure of Kolb’s learning style inventory from factor analysis
and Thurstonian Item Response Theory (IRT) model approaches. Journal of
Psychoeducational Assessment, 39(5), 593–609.
https://doi.org/10.1177/07342829211003739
Ceylan, N. O. (2021). The relationship between learner autonomy and motivation.
TOJET: The Turkish Online Journal of Educational Technology, 20(4), 150–158.
https://files.eric.ed.gov/fulltext/EJ1313495.pdf
Chyung, S. Y., Barkin, J. R., & Shamsy, J. A. (2018). Evidence-based survey design: The
use of negatively worded items in surveys. Performance Improvement, 57(3), 16–
25. https://doi.org/10.1002/pfi.21749
Clem, J. M., Mennicke, A. M., & Beasley, C. (2014). Development and validation of the
Experiential Learning Survey. Journal of Social Work Education, 50(3), 490–506.
https://doi.org/10.1080/10437797.2014.917900 To
Cohen, L., Manion, L., & Morrison, K. (2018). Research methods in education (8th ed.).
Routledge. https://doi.org/10.4324/9781315456539
Danko, T. (2019). Student perceptions in homeland security and emergency management
education: Experiential learning survey. Journal of Experiential Education, 42(4),
417–427. https://doi.org/10.1177/1053825919873678
Dillette, A., & Sipe, L. (2018). A systematic framework of experiential learning:
Challenging educators to make college more than an aggregation of credits.
Creative Education, 09, 1426–1443. https://doi.org/10.4236/ce.2018.99106
Ding, F., & Yu, B. (2021). First year university students’ perception of autonomy: an
individualistic approach. Journal of Further and Higher Education.
https://doi.org/10.1080/0309877X.2021.1905154
ESEI, I. B. S. (2021). Project-based learning at its best: Our students presenting at the Majestic
Hotel. https://www.eseibusinessschool.com/the-project-based-learning/
Fini, E. H., Awadallah, F., Parast, M. M., & Abu-Lebdeh, T. (2018). The impact of project-
based learning on improving student learning outcomes of sustainability
concepts in transportation engineering courses. European Journal of Engineering
Education, 43(3), 473–488. https://doi.org/10.1080/03043797.2017.1393045
Ginting, D., Djiwandono, P. I., Woods, R., & Lee, D. (2020). Is autonomous learning
possible for Asian students? The story of a MOOC from Indonesia. Teaching
English with Technology, 20(1), 60–79.
Gulyamova, M. K., & Kadirova, Z. M. (2021). The main pecularities of autonomous
learning in foreign language acquisition. Innovations in Pedagogy and Psychology,
4(4), 85–89. https://doi.org/10.26739/2181-9513-2021-4-16
Guo, P., Saab, N., Post, L. S., & Admiraal, W. (2020). A review of project-based learning
in higher education: Student outcomes and measures. International Journal of
Educational Research, 102(2020), 1–13. https://doi.org/10.1016/j.ijer.2020.101586
152
Güven, Z., & Valais, T. (2014). Project based learning: A constructive way toward learner
autonomy. International Journal of Languages’ Education and Teaching, 2(3), 182–
193.
https://www.academia.edu/22066911/PROJECT_BASED_LEARNING_A_CO
NSTRUCTIVE_WAY_TOWARD_LEARNER_AUTONOMY
Hawkins, M. W. (2018). Self-directed learning as related to learning strategies, self-
regulation, and autonomy in an English language program: A local application
with global implications. Studies in Second Language Learning and Teaching, 8(2),
445–469. https://doi.org/10.14746/ssllt.2018.8.2.12
High Quality Project Based Learning. (2018). A framework for high quality project based
learning. https://hqpbl.org/
Hinton, P. R., McMurray, I., & Brownlow, C. (2014). SPSS explained. In SPSS Explained
(2nd ed.). Routledge. https://doi.org/10.4324/9781315797298
Iamudom, T., & Tangkiengsirisin, S. (2020). A comparison study of learner autonomy
and language learning strategies among Thai EFL learners. International Journal
of Instruction, 13(2), 199–212. https://doi.org/10.29333/iji.2020.13214a
Indrawan, E., Jalinus, N., & Syahril. (2020). Project-based learning in vocational
technology education: Study of literature. International Journal of Scientific and
Technology Research, 9(2), 2821–2825. http://www.ijstr.org/final-
print/feb2020/Project-based-Learning-In-Vocational-Technology-Education-
Study-Of-Literature.pdf
Jansen, R. S., van Leeuwen, A., Janssen, J., Conijn, R., & Kester, L. (2020). Supporting
learners’ self-regulated learning in Massive Open Online Courses. Computers and
Education, 146, 1–17. https://doi.org/10.1016/j.compedu.2019.103771
Johnson, J. L., Khan, A. A., & Saeed, K. (2020). Learning styles inventory and experiential
learning theories an integrative review of literature. Pakistan Journal of Society,
Education and Language, 6(1), 1–12.
https://www.pjsel.jehanf.com/index.php/journal/article/download/58/43
Jose, A. E. S., Cartajena, V. T., Decena, N. P., & Geromo, J. C. (2020). Taking control:
Fostering learners’ autonomy in learning English language. Journal of Humanities
and Social Sciences Invention, 2(2), 13–17. https://doi.org/10.35210/jhssi.2.2.01
Juliet, M. P. (2020). Students’ reflections on collaborated project-based learning in the
department of ecotourism. African Journal of Hospitality, Tourism and Leisure, 9(1),
1–14.
https://www.ajhtl.com/uploads/7/1/6/3/7163688/article_41_vol_9_1__2020_
dut.pdf
Kim, N., & Yoon, Y. (2021). A comparative study on blended learning and flipped
learning: EFL students’ learner autonomy, independence, and attitudes. Korean
Journal of English Language and Linguistics, 21(March), 171–188.
https://doi.org/10.15738/kjell.21..202103.171
Kolb, A. Y., & Kolb, D. A. (2017). Experiential learning theory as a guide for experiential
educators in higher education. ELTHE: A Journal for Engaged Educators, 1(1), 7–44.
https://apps.weber.edu/wsuimages/tlf/Experiential Learning Theory as a
guide for experiential educators in higher education.pdf
Kolb, A. Y., & Kolb, D. A. (2018). Eight important things to know about the experiential
learning cycle. Australian Educational Leader, 40(3), 8–14.
https://learningfromexperience.com/downloads/research-library/eight-
important-things-to-know-about-the-experiential-learning-cycle.pdf
Krajcik, J. S., & Shin, N. (2014). Project-based learning. In R. K. Sawyer (Ed.), The
Cambridge handbook of the learning sciences (2nd ed.). Cambridge University Press.
http://daleydoseoflearning.weebly.com/uploads/1/8/7/7/18774020/chapter_
19_pbl_kraichik.pdf
Kumar, S., & Bhandarker, A. (2020). Experiential learning and its efficacy in management
153
education. Purushartha, XIII(1), 35–55. https://doi.org/10.21844/16202013103

Kyu, T. T. (2018). Effective strategies to motivate students to become active participants. 1st
International Conference on Engineering Education and Innovation.
https://meral.edu.mm/record/7562/files/Thin Thin Kyu, EE2,Dec. 13-
14,2018.Dept,Eng.TU(H).pdf.
Little, D., Dam, L., & Legenhausen, L. (2017). Language learner autonomy: Theory,
practice and research. In D. Sungleton & S. E. Pfenninger (Eds.), Multilingual
Matters (Second Lan).
https://www.researchgate.net/publication/320022166_Language_Learner_Aut
onomy_Theory_Practice_and_Research
Liu, H. (2019). Barriers of project-based learning in teaching and learning of Chinese
Technical and Vocational Education and Training (TVET): A review.
TVET@Asia, 12, 1–14. https://tvet-online.asia/issue/12/liu/
Liu, X., Liu, Y., & Tu, J. F. (2020). Multimedia technology and learner autonomy: An
experimental study for asymmetric effects. Symmetry, 12(3), 1–11.
https://doi.org/org/10.3390/sym12030462
Marsevani, M. (2021). Learners’ perception and practices on autonomous language
learning in EFL settings. International Journal of Language and Literature, 5(1), 54–
65.
https://pdfs.semanticscholar.org/4110/d6673d42c7108e4f6e6fc956f5a09dab2f9d
.pdf
Matsuo, M., & Nagata, M. (2020). A revised model of experiential learning with a
debriefing checklist. International Journal of Training and Development, 24(2), 144–
153. https://doi.org/10.1111/ijtd.12177
Melvina, M., & Julia, J. (2021). Learner autonomy and English proficiency of Indonesian
undergraduate students. Cypriot Journal of Educational Sciences, 16(2), 803–818.
https://doi.org/10.18844/cjes.v16i2.5677
Morris, T. H. (2020). Experiential learning–a systematic review and revision of Kolb’s
model. Interactive Learning Environments, 28(8), 1064–1077.
https://doi.org/10.1080/10494820
Mustapha, R., Sadrina, Nashir, I. M., Azman, M. N. A., & Hasnan, K. A. (2020).
Assessing the implementation of the Project-based Learning (PJBL) in the
department of mechanical engineering at a Malaysian polytechnic. Journal of
Technical Education and Training, 12(1 Special Issue), 100–118.
https://doi.org/10.30880/jtet.2020.12.01.011
Ndlovu, T. P., & Nyane, E. M. M. (2018). The value of workplace learning in attaining
discipline-specific skills for ecotourism employment. African Journal of
Hospitality, Tourism and Leisure, 7(4), 1–13. http//:www.ajhtl.com
Orakcı, Ş. (2021). Exploring the relationships between cognitive flexibility, learner
autonomy, and reflective thinking. Thinking Skills and Creativity, 41, 1–13.
https://doi.org/10.1016/j.tsc.2021.100838
Oxana, C., Aleksandr, L., Irina, T., & Galina, G. (2020). Challenges in building up learner
autonomy in Russian universities. International E-Journal of Advances in Education,
6(16), 7–13. https://doi.org/10.18768/ijaedu.616001
Padmadewi, N. N., Artini, L. P., & Agustini, D. A. E. (2020). Promoting autonomous
learning and 21st century skills of English education students through
empowering their pedagogical skills. Advances in Social Science, Education and
Humanities Research, 438, 106–111. https://doi.org/10.2991/assehr.k.200513.024
Pallant, J. (2020). SPSS survival manual: A step by step guide to data analysis using IBM SPSS
(7th ed.). McGraw-Hill Education.
Pamungkas, S. F., Widiastuti, I., & Suharno, S. (2019). Kolb’s experiential learning as an
effective learning model in creative product and entrepreneurship subjects.
Journal of Mechanical Engineering and Vocational Education, 2(1), 27–32.
154
https://doi.org/10.20961/jomeve.v2i1.28352
Passarelli, A. M., & Kolb, D. A. (2020). The learning way: Learning from experience as
the path to lifelong learning and development. In M. London (Ed.), Handbook of
lifelong learning (2nd ed.). University Press.
Patil, T., Hunt, M., Cooper, K., & Townsend, R. (2020). Developing a case-based
experiential learning model at a program level in a regional university:
Reflections on the developmental process. Australian Journal of Adult Learning,
60(2), 225–244. https://files.eric.ed.gov/fulltext/EJ1267932.pdf
Pham, T. (2021). Employing project work to enhance aspects of learner autonomy for
tertiary EFL learners. International Journal of Advance Research, Ideas and
Innovations in Technology, 7(3), 386–395.
https://www.researchgate.net/publication/351774465_Employing_project_wor
k_to_enhance_aspects_of_learner_autonomy_for_tertiary_EFL_learners
Pradanti, P., & Muqtada, M. R. (2023). Students’ perceptions on learning, motivation,
and performance through project-based learning: Undergraduate students’ case.
PYTHAGORAS: Jurnal Program Studi Pendidikan Matematika, 12(1), 16–26.
https://doi.org/10.33373/pythagoras.v12i1.5011
Reswari, G. P. A., & Kalimanzila, J. (2021). Re-promoting autonomous learning for
university students: A lesson from pandemic Covid-19. Jurnal Ilmiah Profesi
Pendidikan, 6(1), 38–47. https://doi.org/10.29303/jipp.v6i1.144
Roland, K. (2017). Experiential learning: Learning through reflective practice.
International Journal for Cross-Disciplinary Subjects in Education, 8(1), 2982–2989.
https://doi.org/10.20533/ijcdse.2042.6364.2017.0405
Saeed, M. A. (2021). Learner autonomy: Learners’ perceptions on strategies to achieve
autonomy in an EFL classroom. International Journal of Linguistics, Literature and
Translation, 4(3), 150–158. https://doi.org/10.32996/ijllt
Saglam, A. L. (2018). The integration of educational technology for classroom-based
formative assessment to empower teaching and learning. In A. Khan & S. Umair
(Eds.), Handbook of Research on Mobile Devices and Smart Gadgets in K-12 Education.
IGI Global. https://doi.org/10.4018/978-1-5225-2706-0.ch020
Sereti, S., & Giossos, Y. (2018). Development of a Learner Autonomy Scale. The Journal for
Open and Distance Education and Educational Technology, 14(1), 94–103.
https://doi.org/10.12681/jode.18212
Stehling, C., & Munzert, U. (2018). Project-based learning. In J. Drummer, G. Hakimov,
M. Joldoshov, T. Köhler, & S. Udartseva (Eds.), Technical and Vocational Education
and Training: Issues, Concerns and Prospects (Vol. 28, pp. 111–115). Springer Open.
https://doi.org/10.1007/978-3-319-73093-6
Suárez-Alvarez, J., Pedrosa, I., Lozano, L. M., García-Cueto, E., Cuesta, M., & Muñiz, J.
(2018). Using reversed items in likert scales: A questionable practice. Psicothema,
30(2), 149–158. https://doi.org/10.7334/psicothema2018.33
Suntharalingam, C., Rathakrishnan, T., & Safari, S. (2021). Determinants of fruit
purchasing decision among Singaporean consumers: An empirical study. Journal
of International Food and Agribusiness Marketing, 1–31.
https://doi.org/10.1080/08974438.2021.2006106
Teng, M. F. (2019). Learner autonomy: An educational goal of teaching English as a
foreign language. In Autonomy, agency, and identity in teaching and learning English
as a foreign language. Springer. https://doi.org/10.1007/978-981-13-0728-7_1
Thanh, N. T. (2019). Promoting learner autonomy through self-assessment and reflection.
VNU Journal of Foreign Studies, 35(6), 146–153.
https://scholar.google.co.za/scholar_url?url=https://rikkyo.repo.nii.ac.jp/%3F
action%3Drepository_action_common_download%26item_id%3D16046%26item
_no%3D1%26attribute_id%3D18%26file_no%3D1&hl=en&sa=X&ei=6qsJZMWSJ
4KLmwGN2aygDQ&scisig=AAGBfm1wCHaf1sXFiDx43Amn
155
Tomasouw, J., & Marantika, J. E. R. (2020). Learner autonomy as strategy to enhance the
quality of learner. 3rd International Conference on Learning Innovation and Quality
Education (ICLIQE 2019) Learner, 397, 504–510.
Tran, K. M. A. (2020). Promoting learners’ autonomy: To what extent do students
practice autonomy activities for their learning? Journal of Science and Technology,
43(B), 133–144. https://doi.org/10.46242/jst-iuh.v43i01.615
Van de Pol, J., Mercer, N., & Volman, M. (2019). Scaffolding student understanding in
small-group work: Students’ uptake of teacher support in subsequent small-
group interaction. Journal of the Learning Sciences, 28(2), 206–239.
https://doi.org/10.1080/10508406.2018.1522258
Vasiliene-Vasiliauskiene, V., Vasiliauskas, A. V., Meidute-Kavaliauskiene, I., & Sabaityte,
J. (2020). Peculiarities of educational challenges implementing project-based
learning. World Journal on Educational Technology: Current Issues, 12(2), 136–149.
https://doi.org/10.18844/wjet.v12i2.4816
Wirapatni, A. A. I. S., Nitiasih, P. K., & Artini, L. P. (2021). Exploring senior high school
EFL teachers’ beliefs regarding learner autonomy. Journal of Educational Research
and Evaluation, 5(1), 118–124. https://doi.org/10.23887/jere.v5i1.29945
Yaprak, Z. (2021). Accountable learner autonomy model: Developing digital materials in
second language education. The Reading Matrix: An International Online Journal,
21(1), 51–71. https://readingmatrix.com/files/24-b9lg083e.pdf
Yasmin, M., Naseem, F., & Abas, N. (2020). Constraints to developing learner autonomy
in Pakistan: University lecturers’ perspectives. Educational Research for Policy and
Practice, 19, 125–142. https://doi.org/10.1007/s10671-019-09252-7
Yu, R. (2020). On fostering learner autonomy in learning English. Theory and Practice in
Language Studies, 10(11), 1414–1419. https://doi.org/10.17507/tpls.1011.09 On
Yuliani, Y., & Lengkanawati, N. S. (2017). Project-based Learning in promoting learner
autonomy in an EFL classroom. Indonesian Journal of Applied Linguistics, 7(2), 285–
293. https://doi.org/10.17509/ijal.v7i2.8131
Zaidi, N. H., Khan, N., & Oad, L. (2020). Promoting learners’ autonomy at elementary
level through experiential learning: A quantitative study. Journal of Elementary
Education, 29(2), 0–17.
https://www.researchgate.net/publication/341254513_Promoting_Learners’_A
utonomy_at_Elementary_Level_through_Experiential_Learning_A_Quantitative
_Study
Zeng, B., Wen, H., & Zhang, J. (2020). How does the valence of wording affect features of
a scale? The method effects in the undergraduate learning burnout scale.
Frontiers in Psychology, 11, 1–12. https://doi.org/10.3389/fpsyg.2020.585179
Zisan, K. Al, Albattat, A., & Basar, S. (2021). Impact of experiential learning on student
performance in hospitality and tourism education. Journal of Management and
Science, 19(1), 1–9.
https://www.researchgate.net/publication/352876627_Impact_of_Experiential_
Learning_on_Student_performance_in_Hospitality_and_Tourism_Education
Zourez, L. (2019). Teachers’ awareness of the role of EFL learner autonomy in promoting
learning outcomes. Milev Journal of Research and Studies, 5(2), 450–463.
https://www.asjp.cerist.dz/en/article/106100
156

Vol. 22, No. 7, pp. 156-172, July 2023
Implementation of Strategy Instruction in

Teaching English as a Foreign Language: A
Systematic Review
Li Su
Faculty of Educational Studies, Universiti Putra Malaysia, 43400, UPM
Nooreen Noordin*
Joanna Joseph Jeyaraj

Abstract. Strategy instruction (SI) has played an essential role in English

as a foreign language (EFL). Its usage looks to provide potential benefits
for improving learners’ strategic competence for lifelong education.
However, the current state of research on SI in EFL is fragmented and
does not sufficiently elaborate upon the optimal conditions for effectively
implementing SI. The present study aims to address this gap by
synthesizing existing studies on SI in EFL and employing a systematic
literature review using Petticrew and Roberts’ (2008) framework. The
related 25 articles were selected from databases of Educational Resources
Information Centre (ERIC), Scopus, and Google Scholar from 2010 to
2022. The findings indicated that Strategy-based Learning (SBL) is one of
the most effective approaches for implementing SI. The factors impacting
the effectiveness are also revealed. Finally, this study recommends that
future studies should integrate self-regulation into SI research and that
the collaborative approach should be conducted by scholars and
educators to enhance the efficiency of IS.
Keywords: strategy instruction (SI); language learning strategy (LLS);

strategic competence; self-regulated learning (SRL); learner autonomy
(LA)
*
Corresponding author: Nooreen Noordin, nooreen@upm.edu.my

157
1. Introduction
Proficiency in English has become a crucial factor for success in various domains
in the contemporary globalized world. As the most widely spoken language
worldwide, English has emerged as the primary language for international
business, politics, and academia (Phillipson, 2017). In numerous countries,
English is included as a mandatory subject in school curricula, and a competent
level of English proficiency is often a prerequisite for gaining admission to
universities and securing employment opportunities. However, the acquisition of
a new language can be an arduous and challenging task, particularly for adult
learners such as college students who have already established their primary
language skills.
To address the challenge of enhancing English as a Foreign Language (EFL),

several scholars (e.g., Amiryousefi, 2019; Zhang et al., 2020) have sought to
investigate influential factors such as learners’ language proficiency, learning
motivation, learner autonomy (LA), and foreign language anxiety. Among these
approaches, the utilization of learning strategies (LS) has garnered significant
attention (Habók et al., 2021). LS is commonly defined as the deliberate actions
taken by learners to enhance the efficiency of their language learning process
(Sardegna et al., 2018). Although there is variation in the taxonomies of LS, most
researchers in this field distinguish among three categories of strategies in
language learning: cognitive, metacognitive, and socio-affective. Cognitive
strategies are concerned with behaviors and mental processes that facilitate
learning (e.g., keyword, rehearsal, note-taking). Metacognitive strategies pertain
to the awareness of the learning process itself (e.g., focusing attention, and
planning for learning). Finally, socio-affective strategies are related to interactions
with others and personality traits that affect language learning (e.g., seeking out
conversation partners and managing anxiety).
Despite some existing debates surrounding the clarity of the definition and
categorization of language learning strategies (LLS), scholars generally agree on
the facilitative role that learning strategies play in foreign language acquisition
(Lou & Noels, 2019). Empirical investigations on the impact of LLS on foreign
language acquisition (FLA) have been abundant, with a multitude of studies
demonstrating their positive influence on learners’ strategy utilization,
motivation, and language proficiency (e.g., Cheng et al., 2019; Zhang et al., 2019).
Mazzetti et al. (2020) argued that the extent of achievement in language learning
is correlated with the number and frequency of strategies employed by learners.
They further suggested that the effectiveness of particular strategies in language
learning is dependent on learners’ goals, indicating that the efficacy of certain
strategies may differ based on the intended outcome. Additionally, the authors
advocated for learners to experiment, explore, and evaluate different strategies to
identify the most effective ones, with metacognitive strategies being particularly
advantageous for all learners and learning styles.
In the realm of foreign language education, a significant breakthrough occurred

in the 1970s when scholars and practitioners redirected their focus from the
teacher-centric teaching outcome to the learner-centric learning procedure
158
(Dwivedi et al., 2019). As a result of this paradigm shift, a multitude of claims

have been made regarding the importance of strategy instruction (SI), which aims
to empower students and meet their learning needs by increasing their self-
awareness and expertise in effective strategy utilization. Empirical studies on
foreign language strategy instruction have been identified as a promising avenue
for both theoretical and practical advancements in the field (Thomas et al., 2021).
SI refers to “any intervention which focuses on the strategies to be regularly
adopted and used by language learners to develop their proficiency, to improve
particular task performance, or both” (Hassan et al., 2005, p. 1). As a result of the
growing interest in LLS, a significant number of researchers have developed
models for SI. These models have been employed in experimental intervention
studies, and they highlight the importance of cultivating students’ metacognitive
awareness of the value of learning strategies. Overall, there is a consensus among
scholars that SI can have a positive impact on both learners and the language
learning process (Lin et al., 2017). However, the current state of research on SI in
EFL is still fragmented and does not provide a comprehensive explanation of the
optimal conditions for effectively implementing SI. For example, it does not detail
the factors that may impact the effectiveness of SI and the operative approaches
necessary to employ SI in EFL learning. This systematic literature review aims to
address this gap by synthesizing existing studies on SI in EFL and identifying
trends and gaps in the research.
Strategic learning is a dynamic and iterative process aimed at efficiently solving
learning problems and improving the learning process (Häkkinen et al., 2017).
This process involves identifying difficult or novel tasks and analyzing task
demands, one’s own resources, the available instruments, strengths and
weaknesses, and contextual factors that may affect task completion. The initial
analysis constitutes a foundation for making strategic decisions and generating
appropriate plans. These plans are subsequently monitored and adjusted when
executed. Once the task is finished, strategic learners engage in reflection and
assess the effectiveness of the strategies they employed during the process, which
can be adjusted and transferred to similar tasks in the future. The acquisition of
strategic learning in approaching unfamiliar tasks may initially involve
intentional and gradual steps. However, through consistent practice, strategic
learners develop an extensive range of strategies that can be readily applied to
analogous tasks. Experienced strategic learners have the ability to consciously
employ strategies, while the implementation, monitoring, and adjustment can
take place automatically, even without the learner’s conscious awareness (Bae &
Kwon, 2021). Many examples of strategic processing can be found in various
domains (Wolf & Floyd, 2017), such as for master sports players (Patton et al.,
2020), chess players (De Jorge-Moreno, 2020), and successful language learners
(Teng & Zhang, 2020).
The dynamic and iterative nature of strategic learning highlights the importance
of metacognition and self-regulated learning (SRL). The ability to monitor one’s
own learning process and make necessary adjustments is crucial for successful
strategic learning (Lapitan et al., 2021). The accumulation of a repertoire of
159
strategies requires not only the knowledge of different strategies but also the
ability to select and adapt them to different tasks and contexts. The expertise of
strategic learners is not only characterized by their ability to deploy effective
strategies but also by their ability to reflect on their learning and adjust their
strategies accordingly. Furthermore, strategic learning can be considered a subset
of SRL (Valtonen et al., 2017), which involves a variety of cognitive, metacognitive,
and motivational processes. The emphasis on SI in language learning reflects the
recognition of the importance of learner autonomy (LA) and the need to shift the
focus from the teacher to the learner. Effective SI should aim to develop learners’
metacognitive understanding of the value of learning strategies and assist in the
development of these for rapid and effective deployment in various tasks and
contexts.
Strategic competence comprises declarative, procedural, and conditional

knowledge of the strategies (Teng et al., 2021). Declarative knowledge
encompasses learners’ awareness and understanding of specific learning
strategies and their functions. It includes knowledge of different categories of
strategies, such as cognitive, metacognitive, and socio-affective strategies, and the
ability to differentiate among them. Procedural knowledge, on the other hand,
involves the actual implementation of strategies, including knowing how to
initiate, execute, and monitor their use. This type of knowledge is developed
through practice and feedback. Conditional knowledge refers to the ability to
determine when and why to use specific strategies based on the task at hand, the
learner’s goals, and the learning context. This knowledge is more complex and
situational as it requires learners to consider a variety of factors when selecting
strategies, such as the difficulty of the task, the time available, the learner’s prior
knowledge and experience, and the learning resources and support available (Sun
& Li, 2019). The effective utilization of learning strategies is not entirely reliant on
the available declarative, procedural, and conditional knowledge. Although these
modes of knowledge form the necessary foundation for strategic learning,
effective utilization is also contingent on the learner’s consistent motivation,
volition, and beliefs (Teng & Zhang, 2018). The active involvement and decision-
making abilities of the learner play a crucial role in determining the utilization of
strategies. This learner agency enables the intentional exploration of novel
strategies, the selection from existing ones, and the ability to coordinate and
maintain strategic learning behavior. Moreover, strategic learning encompasses
the metacognitive management of the entire learning process, rather than solely
focusing on individual learning tasks.
The acquisition of strategic competence can be facilitated through SI, although

individual differences (ID) may arise (Elleman & Oslund, 2019). A fundamental
aspect of effective differentiated instruction is the teacher’s awareness of each
learner’s strategic learning needs’ strengths and weaknesses. Some learners may
have a limited repertoire of learning strategies while others may claim to possess
a diverse range of approaches but lack the ability to implement them effectively.
Additionally, some learners may demonstrate proficiency in applying strategies
to specific tasks but struggle to transfer these approaches to new contexts.
Furthermore, learners may possess a range of learning strategies but may be
160
predisposed to utilizing certain strategies due to personal factors such as past

experiences, cultural beliefs, or time constraints (Day-Vines et al., 2021).
Moreover, there are instances where learners can competently use strategies to
complete a task but are unable to apply these when managing their own long-
term learning and development goals. These diverse scenarios add to the intricacy
of differentiated instruction, necessitating individualized approaches that account
for a thorough assessment of each learner or student group and their strategic
learning requirements.
3. Methodology
3.1 Procedure
For this study, the researchers employed the methodology proposed by Petticrew
and Roberts (2008) to conduct a systematic review, which is widely utilized in
social science research. Systematic reviews are designed to minimize systematic
errors or biases by comprehensively identifying, evaluating, and synthesizing all
relevant research about a specific research problem or set of problems. Following
the framework outlined by Petticrew and Roberts (2008), the present study
consisted of seven stages: formulating research questions or hypotheses,
identifying the types of studies to include, conducting a thorough literature
search, screening the search results, appraising the studies that met the inclusion
criteria, synthesizing the findings, and assessing the heterogeneity among the
included studies. For the current study, the research questions were confirmed
based on the systematic literature review of SI in the field of English as a Foreign
Language (EFL) learning.
RQ1: What approaches have been employed in the EFL context for the
implementation of SI?
RQ2: What factors would influence the effectiveness of SI in the EFL context?
3.2 Search Criteria

For the second stage, the related criteria were determined to clarify the types of
studies for the further search process. The study exclusively considered research
papers written in the English language and published in peer-reviewed journals
from 2010 to 2022. An emphasis was placed on ensuring the quality, applicability,
and ease of access to the selected studies. Subsequently, certain inclusive and
exclusive criteria were defined in Table 1 as follows.
Table 1 Literature review selection criteria

Inclusion criteria Exclusion criteria
Published in English and from 2010 to Published in other languages and outside
2022 of 2010 to 2022
Book reviews, conference papers,
Published in peer-reviewed journals
dissertation
Studies exploring the implementation of
Studies outside of the EFL context
SI in the EFL context
Inadequate information on research design
The full texts are available
and data analysis
161
3.3 Search Process

The next stage involved determining the related databases and search terms to
carry out the comprehensive search process. Several academic journal databases
were utilized, including Educational Resources Information Centre (ERIC),
Scopus, and Google Scholar, to ensure a complete set of results was collected.
Pertaining to search terms, the key concepts related to the research aims were
identified as strategy instruction, implementation approaches, and teaching
English as a foreign language. To facilitate the search process, the study identified
relevant search terms, including synonyms and alternative spellings, which had
been used in previous studies. By considering various terms, the aim was to
ensure comprehensive coverage and retrieval of relevant literature.
3.4 Screening Process

The screening process comprised stages 4 to 7, which were presented as follows.
The search query resulted in 360 total related articles. From these, 96 were in ERIC,
89 in Scopus, and 175 in Google Scholar. During the fourth stage, a screening
process was conducted using the inclusion and exclusion criteria outlined in Table
1. As a result, a total of 57 articles met the criteria and were selected for further
analysis.
In the fifth stage, the researchers conducted an appraisal of the articles based on
their titles, abstracts, keywords, and adherence to the inclusion criteria. As a result,
a total of 45 articles were identified for further analysis. Subsequently, in the sixth
stage, we synthesized the findings from these articles, and in the seventh stage,
the researchers assessed the heterogeneity among them. Through this process, a
total of 30 articles remained for further examination. To expand the search, the
researchers employed the snowball method by scanning the reference lists of the
30 selected articles using the Google Scholar databases. This additional step
resulted in the identification of three duplicate articles, which were subsequently
removed. Ultimately, 25 articles were determined to provide support for self-
regulated learning in the context of English as a Foreign Language (EFL) learning.
The overall process of searching and screening was depicted in Figure 1.
162
Figure 1: Searching and screening procedures
4. Result and Discussions

In accordance with the research questions confirmed in stage one, the content of
the 25 articles was analyzed. The result and discussions are presented in the
following sections.
4.1 RQ1: What approaches have been employed in the EFL context for the
implementation of SI?
SI for foreign language acquisition is a crucial aspect of language education that
aims to equip learners with the necessary cognitive tools and techniques to
become strategic learners. The goal of SI is to help learners develop the
competence to use a wide range of effective learning strategies that may facilitate
the acquisition and use of a foreign language.
Various approaches have been developed over the past four decades to practically
and beneficially adapt language learning strategy research for language learners.
One such approach is Strategy-Based Instruction (SBI), which involves integrating
language learning strategy instruction (LLSI) into foreign language classrooms.
The Cognitive Academic Language Learning Approach (CALLA) is an example
of an SBI program that emphasizes the use of metacognitive strategies and
provides a framework for teaching these strategies to language learners
(Albashtawi, 2019). Other general approaches include 1) the use of stand-alone
“learning to learn” courses, which are designed to teach students how to learn
effectively and efficiently; 2) the use of learner guidebooks that provide learners
with a set of strategies and techniques that they can employ to support their
163
language learning outside the classrooms. Some language textbooks also include
built-in strategy training, providing students with opportunities to practice and
develop their language learning skills; 3) Keys to Learning, which offers learners
a systematic and comprehensive approach to developing effective learning
strategies; 4) learner guidance websites and self-access materials, such as learning
tips, which are widely available at many self-access centers around the world and
provide learners with additional resources to support their language learning
strategies.
From the above approaches, SBI has emerged as a promising approach in

language teaching that focuses on cultivating learners' specific competencies to
help them study a language more effectively. Specifically, explicit and integrated
SI offers an additional advantage, since empirical evidence suggests that engaging
in strategy practice within authentic language tasks could promote effective
strategy transfer, foster learners’ task awareness, improve comprehension and
retention, and serve as a means of sustaining or enhancing learners’ motivation
(Chinpakdee & Gu, 2021; Machili et al., 2020). In the realm of foreign language
education, the empirical evidence from the 25 included papers has confirmed the
effectiveness of implementing explicit and integrated strategy training. Such
studies have consistently demonstrated favorable outcomes, including
heightened self-reported use of learning strategies (Martínez-Adrián et al., 2019),
enhanced language proficiency (Namaziandost et al., 2019), increased
metacognitive awareness (Maftoon & Alamdari, 2020), and improved motivation
and autonomy among learners (Duke et al., 2021).
Since its conception, the Cognitive Academic Language Learning Approach

(CALLA) model developed by Chamot and O’Malley (1987) has emerged as the
prevalent approach for Language Learning Strategy Instruction (LLSI) in the
foreign language education field. The CALLA model is primarily intended to
enhance the academic performance of learners who are studying in a second
language. It integrates content, language, and learning strategies, and the
instructional approach typically follows a five-stage sequence that gradually
transitions the responsibility for learning from the teacher to the students
themselves, empowering them to become more autonomous. It should be noted
that the CALLA model is not exclusively designed for second language
acquisition, and LLSI is just one component of the overall instruction. Numerous
published studies have examined the effectiveness of the CALLA model and have
consistently reported positive outcomes, with some studies even reporting
dramatic improvements in language learning outcomes as a result of
implementing LLSI.
4.2 RQ2: What factors would influence the effectiveness of SI in the EFL
context?
The use of SI has been found to boost language learners’ motivation, increase their
use of language learning strategies, and enhance their language learning
outcomes. While SI has been shown to be efficient in the EFL field, the
effectiveness of SI may be moderated by a multitude of factors concerning learners’
individual traits, context and treatment based on the results of the target papers.
164
4.2.1. Learners’ individual traits

In light of the literature, learners’ characters, comprising learning stage, age,
culture, and learning style, are assumed to moderate the effects of SI in foreign
language acquisition (Iqbal et al., 2022). Specifically, learners may use different
strategies at different stages in the learning process, and adults and children may
choose different strategies based on cognitive ability and activity level. Cultural
background can also influence strategy selection, as culture results in both
opportunities and limitations on behavior. Additionally, learning style preference
can vary greatly between individuals and impact strategy selection.
Furthermore, the literature results suggest that numerous other elements,

involving personality, gender, autonomy, beliefs, affect, aptitude, motivation and
volition, can influence the effectiveness of SI. These elements conduce to the
learner’s identity, which has become an essential domain of research orientation
in recent years. Learners’ sense of identity can impact their motivation (Perez et
al., 2014), autonomy (Ushioda, 2011), willingness to invest time and effort in
learning (Kahu, 2013), and affective reactions (Barcelos, 2015). As a result, teachers
need to consider learners’ identities seriously, as resistance may develop if this is
not the case, which can be counterproductive to effective learning. Ultimately,
learners’ sense of identity plays a critical role in whether they could become
successful in target language acquisition and impacts the strategies they choose to
implement to achieve their goals.
4.2.2. Context
Scholars have long acknowledged that the effectiveness of an intervention can
vary depending on contextual factors (e.g., Alibali et al., 2019; Fayyaz & Omar,
2014). In the case of SI interventions, these factors may include the setting in which
the second language (L2) or foreign language (FL) is studied, the age of the learner,
their educational level, proficiency, and the typology of the new language. This
review has explored the potential influence of these variables on the effectiveness
of SI interventions and relevant implications.
L2 and FL. Previous studies have confirmed the effectiveness of SI interventions

in both FL and L2 (Chen, 2022; Plonsky, 2011). However, it can be challenging for
studies to account for the differences between these settings. In the meta-analysis
conducted by Plonsky (2011), it was found that the effectiveness of SI
interventions was nearly two times larger in L2 contexts than in FL contexts.
However, the more specific meta-analysis conducted by Taylor (2014) revealed an
opposite result, with effect sizes favoring FL contexts. Ardasheva et al. (2017)
contended that the differences among effect sizes are not statistically significant,
which suggested that SI interventions may be equally effective in enhancing
language achievements across L2 and FL settings.
Proficiency. Many studies have shown that learners who receive SI tend to
perform better in terms of L2 proficiency compared to those who do not receive
it. Moreover, the positive effect of SI on L2 proficiency seems to increase as
learners’ proficiency level increases. For example, Rao (2016) claimed that the
employment of learning strategies is considerably influenced by the English
proficiency level of the students, wherein students with a higher level of
165
proficiency tend to utilize a more diverse range of strategies and with greater
frequency compared to those with a lower level of proficiency.
In addition, studies have found that the positive effect of SI on FL proficiency is

cumulative, meaning that learners who receive strategy training over an extended
period tend to show greater gains in language proficiency than those receiving
training over a shorter period. For instance, Ardasheva et al. (2017) indicated that
learners who received intervention over a period of 20 weeks showed greater
gains in L2 proficiency than those who received strategy instruction over a period
of 10 weeks. These findings suggest that intervention can be particularly effective
in improving L2 proficiency among advanced proficiency learners and over an
extended period.
Language typology. Language typology refers to the structural and typological

characteristics of a language, such as its phonology, syntax, morphology, and
lexicon. These features can vary across different languages and language families.
One potential influence of language typology is the degree of similarity between
a learner’s first language (L1) and the target foreign language (L2). For example,
shared structural and typological features between a learner’s L1 and L2 may
make the transfer of strategies easier from the L1 to L2 (Guo & Huang, 2020).
Conversely, learners whose L1 is vastly different from the L2 may struggle to
apply strategies learned in one language to the other. Another potential influence
is the complexity and transparency of the L2’s grammar and orthography
(Goodwin, 2016). Languages with complex or opaque grammar and orthography
may present greater challenges for strategy use and acquisition. For example, in
languages with highly inflected grammar, learners may need to use different
strategies to process and learn grammar rules compared to languages with
simpler grammar structures. Overall, research on the relationship between
language typology and SI is still under-researched, and more studies are needed
to better understand these potential influences. Some studies suggest that SI may
be effective across a range of language typologies and that learners may benefit
from strategies that are tailored to the specific features and demands of their target
language.
Educational level and age. Age and educational level are two variables directly
related to SI in EFL research. The theory that learners of different education levels
prefer strategies at different complexity levels is consistent with previous research
in foreign language acquisition. For example, Pfenninger and Singleton (2017)
found that primary students (elementary school level) preferred social strategies,
such as asking classmates or teachers for help, while secondary and higher-level
students preferred more advanced metacognitive strategies, such as setting goals,
monitoring their own learning, and evaluating their progress.
The results also indicated that younger learners tend to benefit more from SI in
terms of language outcomes (Stefánsson, 2013). Certain evidence indicated that
older students might face a language disadvantage in self-regulated learning (SI)
due to potentially lower language learning abilities compared to younger learners.
As learners age, their cognitive abilities and working memory capacity may
166
decline, which can make it more difficult for them to process and retain new
information. Additionally, older learners may have less exposure to the language
they are learning and may have more difficulty acquiring new vocabulary and
grammar structures.
4.4.3. Treatment
The present review identified several factors that can impact the effectiveness of
SI, which include the type of strategies taught (such as metacognitive, cognitive,
or socio-affective strategies), the scope of strategies taught (whether a single
strategy is taught or multiple strategies are packaged together), the duration of
the treatment (ranging from up to 2 weeks to up to a school year), the instructional
approach used (such as awareness-raising or behavior-modeling) and SI scope
and delivery mode. These features should be taken into account when designing
and implementing SI programs, as they could impact the effectiveness of the
instruction on language learning outcomes.
Concerning the number of strategies chosen by the learners, the researcher

provides strong support for a “less-is-more” approach. Rather than
overwhelming learners with a large number of strategies, researchers suggest that
focusing on a smaller set of high-impact strategies can be more effective for
improving language learning outcomes (Yeh, 2021). This approach allows
learners to develop a deeper understanding and better master each strategy,
leading to the effective use and application of strategies in their language learning
tasks. Furthermore, focusing on a smaller set of strategies can help to simplify
instruction and reduce cognitive overload for learners, which can enhance
motivation and engagement in the learning process.
With regard to the instructional approach utilized, according to Ranalli (2013), the
effects of SI utilizing the awareness-raising approach were found to be more
pronounced compared to the behavior-modeling approach. The awareness-
raising approach involves the explicit instruction and explanation of strategies to
learners, while the behavior-modeling approach involves demonstrating the use
of strategies through modeling and observation. The greater effectiveness of the
awareness-raising approach may be due to the fact that it provides learners with
a more explicit and structured understanding of how to use strategies, which can
enhance their metacognitive awareness and control over the learning process.
The available evidence suggests that both short-term and longer-term

interventions may be equally efficient for learners and that the decision regarding
the optimal intervention length may be best determined by the instructors
according to the specific needs and characteristics of the learners. While further
longitudinal research with delayed post-tests is needed to better gauge the long-
term impacts of SI, these findings provide some assurance and allay earlier
concerns that SI may take valuable time away from direct language instruction.
Instead, they lend support to current pedagogical recommendations that
instruction should aim to balance language and (meta)cognitive skills
development.
167
Two noteworthy findings regarding the delivery methods and agents of SI have
emerged from the literature. Firstly, the statistically similar effect sizes (ESs) for
technology-delivered and teacher-delivered SI indicate that the former is not
inferior to the latter in terms of its effectiveness (Fogarty et al., 2017). This finding
has important implications for pedagogical practice, as it suggests that
technology-based SI can be a viable and effective alternative to conventional
teacher-led instruction. Secondly, the finding indicated that researcher-led and
teacher-led SI are equally effective in improving language outcomes (Lyster, 2019).
This might provide some support for the ecological validity of SI research. This
finding suggests that the effects of SI observed in research settings are
generalizable to real-world classroom settings and that the instructional strategies
and techniques used by researchers can be effectively implemented by classroom
teachers.
5. Recommendations for Future Study

While we have made significant progress in understanding effective LLS, there is
still much to do in terms of developing and implementing effective SI. It is critical
to move beyond exploratory and descriptive research by developing and
researching a range of practical and feasible approaches to implementing SI and
bridging the theories to efficient practice. There are still some issues concerning
SI in the field of EFL that should be taken into consideration.
5.1 The Evaluation Issue of SI

The results from the 25 articles indicate that intervention studies typically focus
on assessing whether students’ performance on a singular task has been enhanced
following SI. Such tasks may involve the retention of word lists, comprehension
of texts, or the logical organization of written work. Moreover, these studies have
consistently found that the instruction provided leads to significant
improvements in learners’ performance in the specific task at hand.
It is imperative to acknowledge that the success of instructional interventions

ought not to be confined solely to ameliorating performance on a singular task.
As alluded to earlier, the fundamental objective of instruction is to cultivate
learners’ strategic competence and promote their autonomy. This necessitates
instruction that surpasses the confines of isolated tasks and strives to empower
learners with self-directed study and independence so that acquired knowledge
and skills are easily transferable to novel tasks and contexts. It is paramount that
SI extends beyond the scope of language learning tasks and the confines of foreign
language classrooms. The overarching objectives of such instruction ought to
encompass the development of enhanced learner efficacy, refined communicative
proficiency, and the cultivation of responsible citizenship for the future. These
aims require learners to possess self-regulated and self-directed behaviors,
counterbalanced by a heightened awareness of their social responsibilities. Thus,
the researcher of the current study highly recommends integrating the construct
of ‘self-regulation’ into the current SI intervention to foster EFL learners’ learning
autonomy for lifelong learning.
168
5.2 The Linkage between SI Research and Pedagogical Practice

Despite the innovative nature of the specific aspect of LLS under investigation or
the data collection methods employed, the primary criterion for evaluating the
empirical evidence will be through the success of its application in practical
foreign language instruction. Although certain components of SI have been
incorporated into language learning coursebooks and the significance of strategic
interventions has been acknowledged in national curricula, the emphasis on LLS
remains relatively limited, with educational materials frequently neglecting to
provide structured strategy-based instruction in a principled manner.
Additionally, teachers may be hesitant to allocate time towards strategic
instruction due to competing educational priorities. Despite widespread
discourse regarding the integration of LLS in the classroom, specialists have not
adequately illustrated the contributions of LLS to language proficiency, nor have
they succeeded in effectively conveying research findings to educators in a
comprehensible manner, which should be highlighted in future research.
To date, SI has largely been conducted using a fundamental research paradigm,

with a focus on outcomes rather than the practicalities of implementing SI in
diverse classroom contexts with learners at varying developmental stages. SI
research has primarily been conducted by scholars and has infrequently been
initiated by educators themselves. To maximize the potential effectiveness of SI,
rather than relying solely on conventional one-way workshops for teachers, it is
recommended that educators should be offered ongoing opportunities to
establish their objectives, access resources, contextualize research-based
knowledge within their own work contexts, monitor and reflect on outcomes, and
receive tailored support. This approach emphasizes the importance of providing
teachers with sustained and personalized support in order to promote the
effective implementation of SI. For this purpose, it would be advantageous to
design a viable research-to-practice pathway. To address this issue, the
collaborative approach should be integrated when designing and implementing
SI. During the process, the researcher serves as a facilitator and collaborator and
works alongside the teacher and students in the planning, implementation, and
assessment of SI. This approach involves multiple rounds of feedback, adjustment,
and fine-tuning, with the ultimate goal of enhancing student learning outcomes.
For example, the researcher may work with the teacher and students to identify
appropriate learning strategies, develop lesson plans, and implement
instructional activities. Throughout the process, the researcher may provide
guidance and support, while also gathering data on the effectiveness of the
instruction. Based on this feedback, the team may then adjust and fine-tune the
instruction to better meet the personalized needs of the students.
6. Conclusion
Possessing a high level of strategic competence is deemed essential for academic
success. Numerous scholars attempt to explore the approaches of SI
implementation and factors influencing its effectiveness. In light of the literature
review, research on the two issues in the EFL context is scarce.
169
Based on the analyses of the 25 related articles, Strategy-Based Instruction (SBI) is

considered highly efficient in the EFL realm, particularly in explicit and integrated
modes of training, which can yield positive outcomes. Such outcomes include an
increase in the self-reported utilization of learning strategies, improvement in
language proficiency, enhanced awareness of metacognitive processes, and a
boost in motivation and autonomy among learners. Meanwhile, the systematic
literature review also confirmed the factors that could influence the effect of SI,
involving learners’ individual traits, context and treatment. The exploration of the
two issues in the current systematic review could shed light on the essence of the
effective SI module and provide practical implications for policy-makers and
practitioners. Subsequently, this literature review also provides some suggestions
for future research. On the one hand, the success of instructional interventions
ought not to be confined solely to ameliorating performance on a singular task.
The integration of ‘self-regulation’ into the current SI intervention to foster EFL
learners’ learning autonomy for lifelong learning should also be highlighted. On
the other hand, researchers and educators should work together by utilizing a
collaborative approach to design and implement SI in the EFL context.
7. References
Albashtawi, A. H. (2019). Improvement of EFL Students’ Academic Reading
Achievement Through the Cognitive Academic Language Learning Approach
(CALLA). Reading Psychology. https://doi.org/10.1080/02702711.2019.1658669
Alibali, M. W., Brown, S., & Menendez, D. (2019). Understanding Strategy Change:
Contextual, Individual, and Metacognitive Factors. Advances in Child
Development and Behavior, 227–256. https://doi.org/10.1016/bs.acdb.2018.11.004
Amiryousefi, M. (2019). The incorporation of flipped learning into conventional classes
to enhance EFL learners’ L2 speaking, L2 listening, and engagement. Innovation
in Language Learning and Teaching, 13(2), 147–161.
https://doi.org/10.1080/17501229.2017.1394307
Ardasheva, Y., Wang, Z., Adesope, O., & Valentine, J. C. (2017). Exploring Effectiveness
and Moderators of Language Learning Strategy Instruction on Second Language
and Self-Regulated Learning Outcomes. Review of Educational Research, 87(3),
544–582. https://doi.org/10.3102/0034654316689135
Bae, H., & Kwon, K. (2021). Developing metacognitive skills through class activities:
what makes students use metacognitive skills? Educational Studies, 47(4), 456–
471. https://doi.org/10.1080/03055698.2019.1707068
Barcelos, A. R. G. (2015). Unveiling the relationship between language learning beliefs,
emotions, and identities. Studies in Second Language Learning and Teaching, 5(2),
301–325. https://doi.org/10.14746/ssllt.2015.5.2.6
Chamot, A. U., & O’Malley, J. M. (1987). The Cognitive Academic Language Learning
Approach: A Bridge to the Mainstream. TESOL Quarterly, 21(2), 227.
https://doi.org/10.2307/3586733
Chen, A. (2022). The Effects of Writing Strategy Instruction on EFL Learners’ Writing
Development. English Language Teaching, 15(3), 29.
Cheng, L., Ritzhaupt, A. D., & Antonenko, P. D. (2019). Effects of the flipped classroom
instructional strategy on students’ learning outcomes: a meta-analysis.
Educational Technology Research and Development, 67(4), 793–824.
https://doi.org/10.1007/s11423-018-9633-7
170
Chinpakdee, M., & Gu, P. Y. (2021). The impact of explicit strategy instruction on EFL
secondary school learners’ reading. Language Teaching Research,
136216882199415. https://doi.org/10.1177/1362168821994157
Day-Vines, N. L., Cluxton-Keller, F., Agorsor, C., & Gubara, S. M. (2021). Strategies for
Broaching the Subjects of Race, Ethnicity, and Culture. Journal of Counseling and
Development, 99(3), 348–357. https://doi.org/10.1002/jcad.12380
De Jorge-Moreno, J. (2020). Relationship between University Education in Strategic
Management and Chess in A Both Learning Process: An Approach Across
Learning Paths. Business, Management and Education, 18(2), 226–246.
https://doi.org/10.3846/bme.2020.12369
Duke, N. C., Ward, A. E., & Pearson, P. D. (2021). The Science of Reading
Comprehension Instruction. The Reading Teacher, 74(6), 663–672.
https://doi.org/10.1002/trtr.1993
Dwivedi, A., Dwivedi, P., Bobek, S., & Zabukovšek, S. S. (2019). Factors affecting
students’ engagement with online content in blended learning. Kybernetes, 48(7),
1500–1515. https://doi.org/10.1108/k-10-2018-0559
Elleman, A. M., & Oslund, E. L. (2019). Reading Comprehension Research: Implications
for Practice and Policy. Policy Insights From the Behavioral and Brain Sciences, 6(1),
3–11. https://doi.org/10.1177/2372732218816339
Fayyaz, S., & Omar, H. (2014). A Study of Contextual Situatedness of English Language
Teachers’ Beliefs and Practices about the Form-focused Instruction: A Case
Study in Sandakan District, Sabah. Procedia - Social and Behavioral Sciences, 134,
201–212. https://doi.org/10.1016/j.sbspro.2014.04.240
Fogarty, M., Clemens, N. H., Simmons, D. C., Anderson, L., Davis, J. M., Smith, A. W.,
Wang, H., Kwok, O., Simmons, L. E., & Oslund, E. L. (2017). Impact of a
Technology-Mediated Reading Intervention on Adolescents’ Reading
Comprehension. Journal of Research on Educational Effectiveness, 10(2), 326–353.
https://doi.org/10.1080/19345747.2016.1227412
Goodwin, A. P. (2016). Effectiveness of word solving: Integrating morphological
problem-solving within comprehension instruction for middle school students.
Reading and Writing, 29(1), 91–116. https://doi.org/10.1007/s11145-015-9581-0
Guo, X., & Huang, L. (2020). Are L1 and L2 strategies transferable? An exploration of the
L1 and L2 writing strategies of Chinese graduate students. Language Learning
Journal. https://doi.org/10.1080/09571736.2018.1435710
Habók, A., Kong, Y., Ragchaa, J., & Magyar, A. (2021). Cross-cultural differences in
foreign language learning strategy preferences among Hungarian, Chinese and
Mongolian University students. Heliyon, 7(3), e06505.
Hassan, X., Macaro, E., Mason, D., Nye, G., Smith, P., & Vanderplank, R. (2005). Strategy
training in language learning: A systematic review of available research. London,
England: University of London, EPPI-Centre.
Häkkinen, P., Järvelä, S., Mäkitalo-Siegl, K., Ahonen, A., & Valtonen, T. (2017). Preparing
teacher-students for twenty-first-century learning practices (PREP 21): a
framework for enhancing collaborative problem-solving and strategic learning
skills. Teachers and Teaching, 23(1), 25–41.
https://doi.org/10.1080/13540602.2016.1203772
Iqbal, M., Shet, J. P., Alsaraireh, M. Y., Rad, D., Ignat, S., Hernández, R. M., Sameem, M.
a. M., & Beltran, J. A. (2022). Interplaying Factors of Students Personal
Characteristics in Online Learning Modality: Evidence in Asian Context. World
Journal of English Language, 12(2), 392. https://doi.org/10.5430/wjel.v12n2p392
Kahu, E. R. (2013). Framing student engagement in higher education. Studies in Higher
171
Lapitan, L. D., Tiangco, C., Sumalinog, D. a. G., Sabarillo, N. S., & Diaz, J. M. S. (2021).
An effective blended online teaching and learning strategy during the COVID-19
pandemic. Education for Chemical Engineers, 35, 116–131.
https://doi.org/10.1016/j.ece.2021.01.012
Lin, C. E., Yining, Z., & Zheng, B. (2017). The roles of learning strategies and motivation
in online language learning: A structural equation modeling analysis. Computers
& Education, 113, 75–85. https://doi.org/10.1016/j.compedu.2017.05.014
Lou, N. M., & Noels, K. A. (2019). Promoting growth in foreign and second language
education: A research agenda for mindsets in language learning and teaching.
System, 86, 102126. https://doi.org/10.1016/j.system.2019.102126
Lyster, R. (2019). Making research on instructed SLA relevant for teachers through
professional development. Language Teaching Research, 23(4), 494–513.
https://doi.org/10.1177/1362168818776667
Machili, I., Papadopoulou, I., & Kantaridou, Z. (2020). Effect of strategy instruction on
EFL students’ video-mediated integrated writing performance. Journal of Second
Language Writing, 48, 100708. https://doi.org/10.1016/j.jslw.2019.100708
Maftoon, P., & Alamdari, E. F. (2020). Exploring the Effect of Metacognitive Strategy
Instruction on Metacognitive Awareness and Listening Performance Through a
Process-Based Approach. International Journal of Listening, 34(1), 1–20.
https://doi.org/10.1080/10904018.2016.1250632
Martínez-Adrián, M., Del Puerto, F. G., & Basterrechea, M. (2019). On self-reported use
of communication strategies by CLIL learners in primary education. Language
Teaching Research, 23(1), 39–57. https://doi.org/10.1177/1362168817722054
Mazzetti, G., Paolucci, A., Guglielmi, D., & Vannini, I. (2020). The Impact of Learning
Strategies and Future Orientation on Academic Success: The Moderating Role of
Academic Self-Efficacy among Italian Undergraduate Students. Education
Sciences, 10(5), 134. https://doi.org/10.3390/educsci10050134
Namaziandost, E., Neisi, L., K., & Nasri, M. (2019). Enhancing oral proficiency through
cooperative learning among intermediate EFL learners: English learning
motivation in focus. Cogent Education, 6(1), 1683933.
https://doi.org/10.1080/2331186x.2019.1683933
Nguyen, L. V., & Gu, Y. (2013). Strategy-based instruction: A learner-focused approach
to developing learner autonomy. Language Teaching Research, 17(1), 9–30.
https://doi.org/10.1177/1362168812457528
Patton, D., Huber, C. M., McDonald, C. C., Margulies, S. S., Master, C. L., & Arbogast, K.
B. (2020). Video Confirmation of Head Impact Sensor Data From High School
Soccer Players. American Journal of Sports Medicine, 48(5), 1246–1253.
https://doi.org/10.1177/0363546520906406
Perez, T., Cromley, J. G., & Kaplan, A. (2014). The role of identity development, values,
and costs in college STEM retention. Journal of Educational Psychology, 106(1),
315–329. https://doi.org/10.1037/a0034027
Pfenninger, S. E., & Singleton, D. (2017). Beyond age effects in instructional L2 learning:
Revisiting the age factor (Vol. 113). Multilingual Matters.
Petticrew, M., & Roberts, H. (2008). Systematic Reviews in the Social Sciences: A Practical
Guide. John Wiley & Sons.
Phillipson, R. (2017). Myths and realities of ‘global’ English. Language Policy, 16(3), 313–
331. https://doi.org/10.1007/s10993-016-9409-z
Plonsky, L. (2011). The Effectiveness of Second Language Strategy Instruction: A Meta-
analysis. Language Learning, 61(4), 993–1038. https://doi.org/10.1111/j.1467-
9922.2011.00663.x
Ranalli, J. (2013). Online Strategy Instruction for Integrating Dictionary Skills and
Language Awareness. Language Learning & Technology, 17(2), 75–99.
http://llt.msu.edu/issues/june2013/ranalli.pdf
172
Rao, Z. (2016). Language learning strategies and English proficiency: interpretations

from information-processing theory. Language Learning Journal, 44(1), 90–106.
https://doi.org/10.1080/09571736.2012.733886
Sardegna, V. G., Lee, J., & Kusey, C. (2018). Self-Efficacy, Attitudes, and Choice of
Strategies for English Pronunciation Learning. Language Learning, 68(1), 83–114.
https://doi.org/10.1111/lang.12263
Stefánsson, E. G. (2013). Second language acquisition: The effect of age and
motivation (Doctoral dissertation).
Sun, D., & Li, Y. (2019). Effectiveness of Digital Note-Taking on Students’ Performance in
Declarative, Procedural and Conditional Knowledge Learning. International
Journal of Emerging Technologies in Learning (Ijet), 14(18), 108.
https://doi.org/10.3991/ijet.v14i18.10825
Taylor, A. M. (2014). L1 glossing and strategy training for improving L2 reading
comprehension: a meta-analysis. International Journal of Quantitative Research in
Education. https://doi.org/10.1504/ijqre.2014.060973
Teng, L., & Zhang, L. J. (2018). Effects of motivational regulation strategies on writing
performance: a mediation model of self-regulated learning of writing in English
as a second/foreign language. Metacognition and Learning, 13(2), 213–240.
https://doi.org/10.1007/s11409-017-9171-4
Teng, L., & Zhang, L. J. (2020). Empowering learners in the second/foreign language
classroom: Can self-regulated learning strategies-based writing instruction make
a difference? Journal of Second Language Writing, 48, 100701.
https://doi.org/10.1016/j.jslw.2019.100701
Teng, M. F., Qin, C., & Wang, C. (2021). Validation of metacognitive academic writing
strategies and the predictive effects on academic writing performance in a
foreign language context. Metacognition and Learning, 17(1), 167–190.
https://doi.org/10.1007/s11409-021-09278-4
Thomas, N., Bowen, N. E. J. A., Reynolds, B. L., Osment, C., Pun, J., & Mikolajewska, A.
(2021). A Systematic Review of the Core Components of Language Learning
Strategy Research in Taiwan. English Teaching & Learning, 45(3), 355–374.
https://doi.org/10.1007/s42321-021-00095-1
Ushioda, E. (2011). Why autonomy? Insights from motivation theory and research.
Innovation in Language Learning and Teaching, 5(2), 221–232.
https://doi.org/10.1080/17501229.2011.577536
Valtonen, T., Sointu, E., Kukkonen, J., Häkkinen, P., Järvelä, S., Ahonen, A., Pöysä-
Tarhonen, J., & Mäkitalo-Siegl, K. (2017). Insights into Finnish first-year pre-
service teachers’ twenty-first century skills. Education and Information
Technologies, 22(5), 2055–2069. https://doi.org/10.1007/s10639-016-9529-2
Wolf, C., & Floyd, S. W. (2017). Strategic Planning Research: Toward a Theory-Driven
Agenda. Journal of Management, 43(6), 1754–1788.
https://doi.org/10.1177/0149206313478185
Yeh, H. W. M. (2021). A Study of the Relationship Between the Use of Grammar
Learning Strategies and Student Achievement. International Journal of Adult
Education and Technology, 12(3), 34–46. https://doi.org/10.4018/ijaet.2021070103
Zhang, H., Dai, Y., & Wang, Y. (2020). Motivation and Second Foreign Language
Proficiency: The Mediating Role of Foreign Language Enjoyment. Sustainability,
12(4), 1302. https://doi.org/10.3390/su12041302
Zhang, L. J., Thomas, N., & Qin, T. L. (2019). Language learning strategy research in
System: Looking back and looking forward. System, 84, 87–92.
https://doi.org/10.1016/j.system.2019.06.002
173

Vol. 22, No. 7, pp. 173-192, July 2023
Stakeholders’ Perceptions of Teaching and

Technological Skills in EFL Vocabulary
Instruction: Implications for Remote Learning
Paul Gonzalez-Torres
Universidad Técnica Particular de Loja
Loja, Ecuador
Paola Cabrera-Solano
Loja, Ecuador
Luz Castillo-Cuesta
Loja, Ecuador
Abstract. This study examines stakeholders' perceptions of teaching and

technological skills in English as a foreign language (EFL) vocabulary
instruction and how this relates to remote learning. This research
included 280 participants from 30 public and private institutions of
secondary and higher education in southern Ecuador. All participants
volunteered and consented to contributing data for the study. The data
collection process consisted of administering questionnaires based on a
Likert scale and conducting structured interviews with teachers. The aim
of these instruments was to collect participants' views on teaching and
technological skills in EFL vocabulary instruction. The researchers
adopted a mixed-method approach, with quantitative analysis of the
questionnaire responses using descriptive statistics and qualitative
analysis of interview data. The findings indicated that participants had
positive perceptions of EFL vocabulary instruction in terms of teachers'
classroom management, technological skills, technological resources, and
teaching methods. However, some aspects, such as learning
environments, teachers’ training, online classes, and the use of
information and communication technologies (ICT), were perceived more
positively in private institutions. The findings have implications for EFL
vocabulary instruction, particularly in remote learning contexts, and can
inform the development of appropriate classroom management,
technological resources, and teacher training programs to enhance this
aspect of the EFL teaching and learning process.
©Authors
174
Keywords: feedback; technological skills; ICT; remote learning; EFL

instruction; EFL vocabulary
1. Introduction
The EFL teaching and learning process plays a vital role in education and
employment in today's globalized world. The English language has become
essential for social and economic mobility and intercultural communication,
emphasizing the significance of EFL in promoting multiculturalism and diversity
by enabling individuals to communicate with people from different cultural
backgrounds (Zülküf, 2017).
The COVID-19 pandemic has had a profound impact on our lives, especially in
developing countries (Tadesse & Muluye, 2020). One aspect that was deeply
affected, specifically in Latin America, was education, including EFL teaching and
learning. Regarding the teaching difficulties in EFL instruction during this
pandemic, learners faced challenges such as Internet connectivity, accessibility
and interaction in virtual lessons, utilization of technological tools, and
downloading didactic resources (Mahyoob, 2020). Under these circumstances,
vocabulary, which is a crucial element of language proficiency, requires special
attention in the EFL classroom. Therefore, it is important for instructors to have a
better understanding of the EFL vocabulary teaching and learning process
(Bergström et al., 2022).
The integration of technology has transformed the teaching of EFL vocabulary,

providing numerous opportunities for interactive, cooperative, communicative,
and informative learning (Kassem, 2018). For instance, multimedia resources
such as educational videos, multimedia presentations, teleconferences, chats, and
webinars are popular audiovisual online tools utilized by EFL teachers (Agaltsova
et al., 2020). Additionally, technology related to virtual reality and gaming can
complement more traditional teaching tools such as infographics, digital boards,
and collaborative tools (Bikowski, 2018). These multimedia materials have gained
widespread popularity in EFL teaching and learning, their implementation being
facilitated by modern media resources.
Several studies have examined perceptions of the use of ICT in EFL instruction
(e.g. Pardede, 2020; Rahim & Chandran, 2021; Li, 2022). Nevertheless, despite
recognizing the potential benefits of technological resources in EFL teaching, there
is still a need for a comprehensive understanding of perceptions related to
teaching and technological skills specifically in EFL vocabulary instruction,
especially in the case of remote learning in Latin America. This study aims to
address this gap, considering the situation of remote learning during the
lockdown period of the COVID-19 pandemic. Furthermore, identifying
perceptions related to EFL vocabulary instruction in a remote learning context is
crucial for a more thorough comprehension of the experiences encountered
during the teaching and learning process, and for the effective addressing of any
challenges.
175
In view of the challenges faced when teaching and learning EFL vocabulary,
especially in the use of technology in remote learning during the COVID-19
pandemic, the following research questions will be addressed in this study:
1. What are the participants’ perceptions regarding classroom management
in EFL vocabulary instruction?
2. What are the participants’ perceptions regarding the teachers’
technological skills used for teaching vocabulary in EFL remote
education?
3. What are the participants’ perceptions regarding the technological
resources and methods used when teaching vocabulary in EFL remote
education?
2.1 Technology in EFL teaching
In language teaching, technology has been a subject of great interest in research.
The result is that ICT has had a fundamental role in the field of EFL teaching. In
this respect, EFL teachers are expected to integrate ICT in the EFL classroom;
however, this application of technology in learning requires not only language
skills and strategies but also technological skills (Cakici, 2016). The use of ICT is
necessary in the context of EFL, and it can be applied as an effective resource that
can help students in the learning process (Al-Munawwarah, 2015). With careful
planning and clear objectives, the benefits of using ICT in the classroom include
boosting autonomous learning, as well as motivating and engaging students in
language learning (Azmi, 2017).
Using ICT has acquired great importance in remote education, especially during
the time of the COVID-19 pandemic, when teachers had to find new ways of
implementing their lessons by using digital tools such as Google Classroom,
Google Meet, Zoom, and Google Forms (Blanco González & Mañoso-Pacheco,
2021). In this sense, it is important to note that educational digital resources can
be used to teach EFL listening, speaking, reading, writing, grammar, and
vocabulary. According to Kurniawati and Sofiyah (2021), there are two types of
ICT tools in the EFL classroom: non-web-based (e.g., interactive multimedia,
computer, PowerPoint, virtual books, interactive whiteboard) and web-based
learning tools (e.g., YouTube, WhatsApp, Facebook, Padlet). Other examples of
these digital tools also include Kahoot, Genially, Canva, Quizlet, Class Dojo, Voki,
Symbaloo, and Snappet.
Mobile devices such as smartphones and tablets are other tools that have gained
popularity in recent years. These devices offer the advantage of providing learners
with access to learning materials anytime and anywhere (Kukulska-Hulme, 2019).
Additionally, mobile devices can be used to deliver multimedia content such as
videos, podcasts, and images, which can help to contextualize the target language
skill and make it more memorable.
Furthermore, social media platforms such as Facebook, Twitter, and Instagram

can also be used in EFL classrooms. The effectiveness of social media platforms in
enhancing learner engagement, and motivation has been acknowledged,
176
indicating that these platforms can serve as useful tools and resources to promote
learning (Noori et al., 2022). Social media can also be used to promote
collaborative learning, as students can share their knowledge and receive
feedback from their peers.
Given the significance of technology and the focus of our research, this study aims
to address the utilization of technology in vocabulary instruction—an essential
component of EFL teaching.
2.2 Technology for teaching EFL vocabulary

Vocabulary is considered an essential part of EFL instruction (Schmitt & Carter,
2000; Khan et al., 2018). Without sufficient knowledge of vocabulary, it becomes
very challenging for students to communicate effectively. As a result, it can be
argued that effective communication depends on acquiring appropriate
vocabulary (Cook, 2013). Given the significance of vocabulary in EFL teaching, it
is crucial to acknowledge that learners must have the necessary skills in this area.
Schmitt (2008) has noted that vocabulary learning is a key indicator of language
proficiency since inadequate knowledge of this aspect can significantly impact
students' communication abilities (Adam, 2016).
Regarding the teaching of vocabulary in EFL, there are some strategies that can be
employed to enhance students' success. For example, implicit and explicit
vocabulary teaching strategies have been extensively used (Asyiah, 2017).
Additionally, Ghalebi et al. (2020) suggest that different vocabulary learning
strategies may reveal individual differences among students; therefore they
encourage EFL instructors and course developers to design materials and
activities that can help learners improve their vocabulary knowledge.
In addition to the diverse teaching strategies, the incorporation of technological

resources has taken the teaching of EFL vocabulary to a new level, engaging
learners by offering opportunities to enrich their vocabulary knowledge (Cabrera-
Solano et al., 2019). In this respect, Agaltsova et al. (2020) affirm that among the
various resources that EFL teachers can incorporate in vocabulary instruction, the
use of multimedia provides several opportunities and allows students to interact,
cooperate, communicate, and exchange useful information. The most common
audiovisual online resources include educational films, videos, multimedia
presentations, teleconferences, chats, and webinars. In addition, Bikowski (2018)
indicates that another option in terms of teaching resources is the use of virtual
reality, avatars, and gaming, which can be combined with more traditional tools
such as infographics, digital boards, and collaborative tools. These resources have
acquired high levels of popularity in the learning process and are easy to
implement using modern media resources.
2.3 Classroom management in EFL instruction

Teacher experience grows with time; in fact, experience not only provides
language instruction but also contributes to learners’ personal growth (Ladd &
Sorensen, 2017). A crucial component of teacher experience is classroom
management, which refers to how teachers regulate and control learners'
movements, behaviour, and interactions during a lesson (Habibi et al., 2018).
177
Likewise, Chandra (2015) asserts that classroom management is a process to

achieve students’ self-control which implies promoting favourable student
achievement and behaviour leading to class control and a suitable teaching-
learning environment.
Effective online classroom management requires the development of several

competencies for teachers; thus, communication skills, technological competence,
provision of informative feedback, administrative skills, responsiveness,
monitoring learning, and providing student support are necessary (Bigatel et al.,
2012). However, achieving effective teacher-student communication in an online
class can be challenging (Song et al., 2016). Similarly, teachers need to have
adequate technological skills to solve issues that may arise during virtual sessions,
as students might encounter difficulties accessing materials. Finally, building a
positive rapport and monitoring students' progress are important in establishing
an effective and supportive online learning environment (Kaufmann & Vallade,
2022).
2.4 Previous work

Silviyanti and Yusuf (2015) conducted a study to identify EFL teachers’
perceptions of the use of ICT in their instruction. The participants were 42 EFL
teachers enrolled in two state universities in Indonesia. To collect data, a closed-
ended questionnaire based on the technology acceptance model (TAM) was
administered, and interviews were conducted. The results showed that 31
teachers perceived themselves to have a high level of motivation to use ICT
because it promotes an interesting learning environment; however, 11 instructors
believed that even though the use of technology is a fundamental aspect in the
EFL classroom, their motivation remained low. Moreover, the non-user teachers
believed that the implementation of ICT requires much effort and training
because, although ICTs offer positive outcomes in the teaching and learning
process, some barriers such as cost and insufficient knowledge prevent instructors
from integrating them into their lessons.
Irzawati and Hasibuan (2020) determined students’ perceptions of using ICT for
English learning. The study involved 134 university students who completed
questionnaires to share their perceptions. A survey design was applied to obtain
information related to the students’ perceptions of the use of ICT. The findings
revealed that using ICT was helpful in promoting learning activities, maintaining
motivation, and improving achievement. However, there were also negative
consequences such as hindering students' focus and exposing them to data
misuse. To optimize the benefits and minimize the negative aspects, it is
important to take measures such as setting rules, educating students on proper
data usage, and monitoring their learning activities.
Pardede (2020) analyzed the teachers’ perceptions of ICT use in EFL learning
activities. The participants included 32 EFL instructors from several educational
institutions in Greater Jakarta. Surveys and interviews were employed to gather
quantitative and qualitative data. The findings revealed that most of the teachers
had integrated technological resources in their EFL classrooms. Their perceptions
regarding the use of ICT in teaching activities were positive; however, they
178
acknowledged that they had to face several challenges with respect to facilities
and technical expertise. They also agreed on the necessity of continuous training
for improving their use of technological skills in the classroom.
Rahim and Chandran (2021) examined the perceptions of EFL students in

Afghanistan towards the implementation of e-learning at the university level
during the COVID-19 pandemic. To identify the obstacles and opportunities
associated with e-learning in EFL education, the study utilized a qualitative
method employing semi-structured interviews to elicit the students’ perceptions.
The study selected 21 male and female students from universities in Afghanistan
as respondents through purposeful sampling, while the data from the interviews
were thematically analyzed. The findings indicated that EFL learners view e-
learning as a preferable alternative to traditional face-to-face classrooms.
However, the lack of regular electricity, limited Internet bandwidth, high costs of
ICT tools, inadequate infrastructure, and lack of ICT knowledge among teachers
and students were viewed as obstacles to implementing e-learning in higher
education in Afghanistan.
Li (2022) carried out a study to investigate teachers’ acceptance of ICT and their
ICT literacy during the COVID-19 pandemic in China. A total of 186 high school
EFL teachers participated in this research. The quantitative and qualitative data
were collected through an online survey which included four sections and 29
items. The results evidenced that the participants´ perceptions regarding their
acceptance and knowledge of ICT were overall positive, indicating their readiness
for applying technological tools in COVID-19 emergency remote teaching.
Nevertheless, there were critical difficulties when integrating technology with
pedagogy during an unprecedented crisis.
The aforementioned studies have addressed perceptions of ICT and EFL

instruction; however, they do not specifically focus on EFL vocabulary instruction
in remote learning. Therefore, this study aims to address this gap and provide a
comprehensive understanding of perceptions related to teaching and
technological skills in EFL vocabulary, particularly in the context of remote
learning in Latin America.
3. Methodology
3.1 Setting and participants
A total of 280 participants voluntarily participated in this study, including 90 EFL
teachers, 30 educational authorities, 110 students, and 50 parents from 30 public
and private secondary and tertiary institutions in southern Ecuador (see Figure
1). In other words, purposeful sampling was employed across the stakeholder
groups. Students belonged to different proficiency levels that were aligned with
the standards of the Common European Framework of Reference (Council of
Europe, 2020). These students were studying English as a foreign language based
on the curriculum established by the Ecuadorian Ministry of Education. Owing to
the COVID-19 pandemic, the students were involved in a remote learning
environment.
179
70%
60% 60%
55%
45%
40% 40%
30%
Teachers Students Authorities Parents
Figure 1. Sample distribution
Note: The percentages in red represent stakeholders from private institutions,

while the percentages in blue represent stakeholders from public institutions.
3.2 Data collection instruments

Various data collection instruments were developed by the researchers to carry
out this study. Firstly, a 15-item questionnaire was administered to students to
assess their perceptions of their EFL teachers' classroom management in EFL
vocabulary instruction, as well as their own learning experience in this aspect.
Secondly, a 27-item questionnaire was administered to EFL teachers to determine
their perceptions regarding their classroom management experience,
technological skills, technological resources, and methods for vocabulary
instruction. Thirdly, a 10-item questionnaire was administered to parents of
secondary and higher education students to evaluate their perceptions of the
English vocabulary that their children learned via remote learning. In addition, a
10-item questionnaire was administered to education authorities to explore their
opinions on EFL vocabulary instruction. Finally, a structured interview was
conducted with 10 teachers from the sample to obtain further information about
their experience in EFL vocabulary instruction, their technological skills for
teaching vocabulary, as well as their technological resources and teaching
methods.
All the questionnaires were designed using Google Forms and were based on a
Likert scale with five options: Strongly disagree, disagree, neutral, agree, and
strongly agree. These instruments were pilot-tested prior to their final
administration, while their internal consistency was assessed using Cronbach's
alpha coefficient. The results showed an acceptable level of internal consistency
for the questionnaires with an r coefficient of 0.7.
3.3 Procedure
A mixed-method approach was utilized in this study, combining quantitative
analysis of the questionnaire responses and qualitative analysis of the interview
data. Participants were selected using purposive sampling, and they gave their
180
consent before completing the questionnaires. Permission was also obtained from
private and public institutions. All participants voluntarily completed the
questionnaires and participated in the interviews. Education authorities were
contacted via email and assisted in distributing the questionnaires to EFL
teachers, parents, and students.
Confidentiality and anonymity were ensured during the data collection process,
which lasted for four months. The questionnaires were administered online
through Google Forms, while the interviews were conducted via Zoom sessions.
The data obtained were organized into frequency tables using the SPSS software,
while the questionnaire results were triangulated with the interview data during
the analysis and discussion of the results to enhance the validity and reliability of
the findings.
4. Results
The results presented in Table 1 indicate teachers’ perceptions regarding their
experience in EFL vocabulary instruction. Most EFL teachers agreed that they had
effective presentation and communication skills for teaching vocabulary in both
public (98.1%) and private (100%) institutions. They also believed that they could
provide clear explanations of complex vocabulary-related issues. Regarding
didactic experience, 85.4% of teachers from private institutions reported being
able to create a comfortable classroom environment when teaching vocabulary,
68.8% could provide personalized interactions, and 81.3% could increase students'
interest in learning vocabulary.
In terms of adapting materials and activities for teaching vocabulary, 87.2% of

private institution teachers and 61.1% of public institution teachers could adjust
didactic resources for remote education. They also have experience in selecting
appropriate activities for teaching vocabulary and transitioning between them.
When asked about their knowledge of English vocabulary, 78.7% of private
institution teachers and 51.9% of public institution teachers strongly agreed that
they had sound lexical knowledge. Furthermore, authorities, students, and
parents also expressed their positive perceptions regarding the aforementioned
aspects, especially those from private institutions. The findings of the teachers’
interviews also revealed that teachers from private education have more teaching
experience with respect to vocabulary instruction.
Table 1. Teachers’ perceptions regarding their experience in EFL vocabulary

instruction
Items Strongly Disagree Neutral Agree Strongly
disagree agree
1 2 1 2 1 2 1 2 1 2
% % % % % % % % % %
I have effective 0 0 0 0 1.9 0 44.4 39.6 53.7 60.4

presentation and
communication skills for
teaching vocabulary.
181
I provide clear 0 0 0 0 1.9 2.1 43.4 37.5 54.7 60.4

explanations of complex
issues related to
vocabulary.
I have experience in 0 0 0 0 9.3 2.1 50 29.2 40.7 68.8

personalized interactions
for teaching vocabulary.
0 0 0 0 0 0 29.6 14.6 70.4 85.4

I try to create a
comfortable environment
in the classroom when
teaching vocabulary.
I have the creativity to 0 0 0 0 3.8 0 41.5 18.8 54.7 81.3

motivate students to
increase their interest in
learning vocabulary.
I am able to conduct a 0 0 0 0 5.7 0 50.9 31.3 43.4 68.8

smooth transition from
one activity to the next.
I can adapt materials and 0 0 0 0 1.9 2.1 37.0 10.6 61.1 87.2
resources to use them in
online classes.
I have experience choosing 0 0 1.9 0 7.4 2.1 38.9 31.9 51.9 66

activities relevant to
vocabulary teaching.
I have a vast knowledge of 0 0 0 0 1.9 0 46.3 21.3 51.9 78.7

EFL vocabulary.
* 1 = public; 2 = private
The results in Table 2 show teachers' perceptions regarding technological skills

for teaching vocabulary in EFL remote education. Most instructors from private
institutions (87.2%) strongly agreed that they could use technology to promote
collaborative work in their instruction. Additionally, 73.6% of teachers from
public institutions considered themselves to be able to promote collaboration
when teaching vocabulary through ICT tools.
With respect to teachers' skills in using ICT tools in EFL vocabulary teaching, the
majority of instructors from private institutions (93.6%) agreed that they
conformed to this requirement. Similarly, most teachers from public education
(85.2%) also had a positive perception of their ability to use technology effectively.
Likewise, 72.3% of teachers from private institutions agreed that they had the
skills to implement a variety of technological resources in EFL vocabulary
instruction. On the other hand, 44.4% of instructors from public education
182
affirmed that they could make use of a variety of ICT tools in vocabulary
teaching.
Regarding teachers' perceptions of their technological skills to find and evaluate

authentic web-based content for teaching vocabulary, most teachers agreed with
this statement, both in public (62.2%) and private institutions (91.5%).
Furthermore, the majority of teachers in private (93.6%) and public (88.9%)
education affirmed that they could create interactive digital resources for their
virtual vocabulary lessons. In terms of assessment and feedback, most teachers in
public (50%) and private (68.1%) institutions considered that they could assess
their students' vocabulary by using technological tools. It is important to note that
a significant percentage of teachers (40.7% in public and 27.7% in private
institutions) responded "neutral" to this aspect. Additionally, teachers' opinions
(64.8% in public and 87.6% in private institutions) show that they can use
technological tools to provide feedback on vocabulary.
Table 2. Teachers’ perceptions regarding their technological skills for teaching EFL
vocabulary

disagree agree
1 2 1 2 1 2 1 2 1 2
% % % % % % % % % %
I can use technological 0 0 1.9 0 3.8 0 20.8 12.8 73.6 87.2

tools to promote
collaborative activities
for vocabulary learning.
I know how to use 0 0 5.6 0 9.3 6.4 33.3 23.4 51.9 70.2
technological tools for
teaching EFL
vocabulary.
I can use a variety of 7.4 4.3 16.7 6.4 31.5 17 18.5 38.3 25.9 34
technological tools for
teaching EFL
vocabulary.
I can assess my students’ 0 0 9.3 4.3 40.7 27.7 33.3 42.6 16.7 25.5
vocabulary knowledge
by using technological
tools.
I can find and evaluate 0 0 17 2.1 20.8 6.4 26.4 36.2 35.8 55.3
authentic web-based
content for teaching
vocabulary.
183
I can create interactive 0 0 1.9 2.1 9.3 4.3 46.3 21.3 42.6 72.3
digital resources to be
used in my virtual
vocabulary lessons.
I provide feedback on 3.7 0 5.6 6.3 25.9 6.3 40.7 43.8 24.1 43.8
my students’ vocabulary
performance by using
technological tools.
The participants’ perceptions regarding technological resources and methods

used when teaching vocabulary in EFL remote education are presented in Table
3. With respect to the use of virtual platforms in remote teaching, the majority of
participants in public (88.9%) and private (97.9%) institutions affirmed that they
used tools such as Zoom, Microsoft Teams, Google Classroom, CANVAS, Cisco
Webex, and Moodle in their online classes. These platforms enabled teachers to
implement collaborative work using different technological resources. Thus, the
majority of teachers in both public (62.6%) and private (75.9%) institutions used
technological resources to promote collaborative work.
As for the use of social networks for teaching EFL vocabulary in remote education,
the majority of teachers from private institutions (77.1%) and 38.9% of instructors
from the public education system strongly agreed with this statement. In relation
to the use of audiovisual material to teach EFL vocabulary, 57.4% of teachers from
private education agreed with this aspect, while 31.2% of instructors from public
institutions also affirmed that they use these types of resources in their lessons.
Regarding the use of text files to teach vocabulary, the majority of teachers (75%)
from private institutions and 55.5% of instructors from the public education
system used these resources.
Teachers’ views concerning methodologies showed that they used a variety of

innovative methodologies, strategies, and activities to teach vocabulary in both
public (72.2%) and private institutions (91.7%). It is also necessary to mention that,
when teaching EFL vocabulary, there was a prevalence of student-centered
methods in public (70.4%) and private (83.4%) institutions compared to teacher-
centered methods, which were less frequent in both types of institutions.
Furthermore, in terms of assessment, the majority of teachers responded that they

used online tools (95.9% in private and 85.1% in public) and self-assessment
techniques (79.2% in private and 64.8% in public) to evaluate students’ vocabulary
learning. Moreover, private institutions use online tools for assessment and self-
assessment more frequently than public institutions do. The interview data also
showed that private education institutions had the appropriate infrastructure to
use ICT when teaching EFL vocabulary.
184
Table 3. Technological resources and teaching methods used for teaching EFL
vocabulary in EFL remote education

disagree agree
1 2 1 2 1 2 1 2 1 2
% % % % % % % % % %
I use social networks to 0 0 3.7 0 20.4 6.3 37 16.7 38.9 77.1

teach EFL vocabulary
1 I use virtual platforms 0 0 1.9 0 9,3 2,1 51.9 25 37 72.9

(e.g., Zoom, Microsoft
Teams, CANVAS,
Google Classroom,
Cisco Webex, Moodle)
I use audiovisual 0 10.4 14.8 18.8 27.8 10.4 13 20.8 44.4 39.6
material to teach EFL
vocabulary.
I use text files to teach 3.7 2.1 13 4.2 27.8 18.8 40.7 41.7 14.8 33.3
EFL vocabulary.
I use online tools (e.g., 1.9 0 0 2.1 13 2.1 40.7 31.3 44.4 64.6
quizzes, blogs,
chatrooms) to evaluate
vocabulary.
I use a variety of 1.9 0 7.3 0 18.5 8.3 46.3 29.2 25.9 62.5
innovative
methodologies,
strategies, and activities
to teach vocabulary.
I use technological 0 18.8 11.1 2.1 13 16.7 18.8 31.5 43.8 44.4
resources that promote
collaborative work.
I use student-centered 0 2,1 7,4 2,1 22,2 12,5 42,6 41,7 27,8 41,7
methods and strategies
when teaching
vocabulary.
I use teacher-centered 5,6 16,7 24,1 8,3 27,8 14,6 33,3 37,5 9,3 22,9
methods and strategies
when teaching
vocabulary.
185
I use the 0 2.1 0 2.1 17 12.5 56.6 37.5 26.4 45.8

communicative
approach and the
content and language
integrated learning
approach to teach EFL
vocabulary
I use self-assessment 0 2.1 7.4 4.2 27.8 14.6 35.2 39.6 29.6 39.6
techniques for
evaluating vocabulary.
5. Discussion
5.1 Perceptions regarding classroom management in EFL vocabulary instruction
Most EFL teachers in both public and private institutions believed that they
possess effective presentation and communication skills for teaching vocabulary,
enabling them to provide clear explanations of complex vocabulary-related
issues. This aspect is crucial for effective language learning, as argued by
Bergström et al. (2022). Therefore, EFL instructors should employ different
approaches to help learners enhance their vocabulary knowledge, which implies
that they need to be equipped with effective tools and authentic materials.
The majority of instructors from private institutions reported being able to create
a comfortable classroom environment when teaching vocabulary, providing
personalized interactions, and increasing students' interest in learning
vocabulary. This suggests that teachers from private institutions are better
prepared in these aspects. In this respect, Almusharraf (2021) claims that creating
a comfortable environment can improve rapport with EFL students and help them
focus on learning new vocabulary.
Both private and public institution teachers had experience in adjusting didactic
resources for remote education and selecting appropriate activities for teaching
vocabulary. According to Fatmawaty et al. (2021), adapting teaching materials is
an effective way to address students' needs when teachers do not have sufficient
time to design their own resources. Likewise, private institution teachers also
appeared to be more knowledgeable about vocabulary. In this sense, Rogers
(2018) emphasizes the importance of effective vocabulary input for students;
therefore, EFL teachers should be well-equipped to guide their students in the
vocabulary acquisition process both in and beyond the classroom.
Authorities, students, and parents expressed mostly positive perceptions about

the aforementioned aspects, especially those from private institutions. It was also
revealed that teachers from private institutions were much more skilled in
classroom management regarding EFL vocabulary instruction. This may be
attributed to more training opportunities, available funding, and efficient
administration in private institutions, which, according to Quiñonez et al. (2020),
are the result of the socio-economic inequality in Ecuador.
186
5.2 Perceptions regarding the teachers’ technological skills used for teaching
vocabulary in EFL remote education
Teachers in this study considered themselves capable of promoting collaborative
work using ICT tools. Collaborative work through technology can provide an
effective environment for activities that promote group work, which is crucial in
the learning process (Graham & Misanchuk, 2004). Similarly, these teachers had a
positive perception of the use of ICT in EFL vocabulary instruction. Owing to the
importance of ICT in education, it is essential that teachers have appropriate
technological skills to perform tasks related to the use of audio and video files,
PDFs, and PPTs, among others (Ja'ashan, 2020).
There was also greater variety in terms of implementing technological resources

in EFL vocabulary instruction among teachers from private institutions. It is
worth mentioning that ICT is regarded as a resource that can have a positive effect
on learning (Nikolopoulou, 2018), including the development of vocabulary
(Maureen et al., 2018). On the other hand, the majority of teachers in private and
public institutions affirmed that they could create interactive digital resources for
their virtual vocabulary lessons. In terms of online teaching and learning, teachers
must have appropriate preparation and training to improve their teaching
practice, including aspects such as the evaluation and creation of digital material
(Atmojo & Nugroho, 2020).
With respect to assessment and feedback, most teachers in public and private
institutions indicated that they could assess their students' vocabulary using
technological tools. In fact, a wide range of technological tools can be effectively
used for implementing assessment and providing feedback, which are crucial
elements in language learning (González-Lloret, 2020). Nevertheless, a significant
number of teachers may not use technological tools for this purpose.
The perceptions of authorities, students, and parents regarding technological

skills were more positive in private institutions, specifically in the aspects of
vocabulary teaching, interactive resource design, feedback, and assessment.
Moreover, teachers from public institutions expressed the urgent need for more
training and infrastructure related to technological resources that can be used in
the EFL classroom. All in all, training and access to technology are aspects that
must be prioritized today in the field of ICT for EFL learning (Ja'ashan, 2020).
5.3 Perceptions regarding the technological resources and methods used when
teaching vocabulary in EFL remote education
Teachers, in general, affirmed that they used tools such as Zoom, Microsoft
Teams, and Google Classroom in their online classes to implement collaborative
work using different technological resources. These virtual workspaces are useful
and stimulating learning environments that enable students to increase their
interest in learning and work in collaborative activities, in which they can
spontaneously show originality when delivering their projects in class (Castillo et
al., 2022).
With respect to social networks, their use was more frequent in private
institutions, likely owing to their better technological infrastructure, which favors
187
the application of different types of ICT tools. As Siddig (2020) acknowledges, the
use of social media platforms allows English language learners to develop and
enhance their listening, speaking, writing, and reading skills because they can
immerse themselves in the culture associated with the target language, increase
their interaction with native speakers, and improve their learning experience.
Regarding the use of audiovisual material to teach EFL vocabulary, teachers from
private institutions were more capable of using different types of technological
resources in remote education. Moreover, audiovisual materials help teachers
explain meaning more clearly by representing connections in a more effective way
than when using words (Chiekezie & Inyang, 2021).
Teachers used a variety of innovative methodologies, strategies, and activities to

teach vocabulary in both public and private institutions. Student-centered
methods, such as the communicative approach and the content and language
integrated learning approach (CLIL), prevail. Implementing new methodologies
helps students improve their target language skills and achieve their academic
goals (Ortega-Auquilla et al., 2020).
As for assessment, the majority of teachers claimed that they use online tools and
self-assessment techniques in the evaluation of vocabulary learning. Online
resources such as self-test quiz tools, discussion forums, and e-portfolios foster
formative assessment, which is beneficial in terms of students’ engagement in the
learning process (Gikandi et al., 2011).
Perceptions regarding technological resources and methods for teaching EFL

vocabulary indicated that, in private institutions, teachers use a wider variety of
technological resources to promote collaborative work. Both types of institutions
favor student-centered methods, such as the communicative approach and CLIL.
Moreover, private institutions used online tools for assessment and self-
assessment more frequently than public institutions do. It is also important to
mention that private education institutions had better opportunities and
infrastructure to use ICT when teaching EFL vocabulary, particularly in terms of
collaborative work, teaching methodologies, and assessment.
6. Conclusions
The stakeholders' perceptions regarding classroom management in vocabulary
instruction are highly positive. EFL instructors from both private and public
institutions have effective presentation and communication skills which play a
significant role in language instruction. Teachers can also manage complex issues
when teaching new lexicon in the target language.
EFL instructors from private institutions are perceived to create more comfortable
environments when teaching vocabulary, provide more personalized
interactions, and increase their students' interest by adapting activities and
materials to be used in remote education. The participants also perceived that
instructors from private education have a sound knowledge of lexicon and more
effective classroom management than teachers from the public system.
188
Regarding the perceptions of technological skills, it can be concluded that teachers

from private schools have better preparation and access than teachers in public
institutions in terms of teaching EFL vocabulary using ICTs. This situation is
evidenced in the participants' views regarding their technological skills, as well
as their pertinent training and technological infrastructure.
Teachers from private institutions in this study also claim to have more efficient
management of online classes than teachers from public institutions. This aspect
has been noticed in their perceptions related to the promotion of collaborative
work, students' assessment, feedback, variety in the use of technological tools in
the EFL classroom, as well as the evaluation and creation of digital didactic
material.
The participants' perceptions regarding technological resources used when

teaching vocabulary revealed that teachers in private institutions utilized virtual
platforms, social networks, audiovisual materials, online assessment resources,
and other technological tools more frequently during EFL remote education than
those in public institutions. Also, private institutions had better technological
infrastructure than the public ones concerning access to virtual platforms and
other ICTs for working collaboratively, as well as for presenting and assessing
EFL vocabulary.
Regarding the methods used for teaching EFL vocabulary in remote education,
instructors in both public and private institutions used student-centered
approaches. Thus, the communicative approach and CLIL were the
methodologies that they preferred when teaching vocabulary. These approaches
involve the implementation of innovative strategies and activities that were used
by teachers in both types of educational institutions.
One limitation of the study is that it was conducted in a specific geographic region
(southern Ecuador) and comprised a small sample of participants from 30 public
and private institutions of secondary and higher education. As a result, the
findings may not be generalizable to other contexts or populations. Another
limitation is that the study relied on self-reported perceptions of participants
based on online questionnaires, which may be subject to bias.
Despite these limitations, these findings have implications for EFL vocabulary
instruction, particularly in remote learning contexts, and can inform the
development of appropriate classroom management, teaching methods,
technological resources, and teacher training programs to enhance EFL
vocabulary learning outcomes. The study also highlights the importance of
classroom management, technological skills, and the use of a variety of
technological resources in promoting appropriate teaching practices of EFL
vocabulary. Additionally, the finding that private institutions were perceived
more positively in some aspects of EFL vocabulary instruction also raises
important questions about equity and access in remote learning.
189
Overall, the study provides valuable insights into stakeholders' perceptions of

teaching and technological skills in EFL vocabulary instruction and their
implications for remote learning. However, further research is needed to address
the study's limitations and to explore these issues in other contexts and
populations. One of these important issues would be comparing the results of this
research with those of traditional training approaches in non-pandemic
conditions.
7. Acknowledgments
We wish to extend our sincere appreciation to the Universidad Tecnica Particular
de Loja for their generous support and funding which made this research possible.
We would also like to express our gratitude to the EFL Learning, Teaching, and
Technology research group for their invaluable assistance in promoting and
facilitating this study.
8. References
Adam, M. (2016). Role of vocabulary learning strategies in promoting EFL learners' performance
[Doctoral dissertation, Sudan University of Science and Technology].
https://repository.sustech.edu/handle/123456789/15469
Agaltsova, D., Rodinova, N., Sabekia, R., Kagosyan, A., & Ostroukhov, V. (2020). Webinar
as an innovative technology of online education with the use of modern media
resources. Revista Inclusiones, 7(S3-5), 119-132.
https://revistainclusiones.org/index.php/inclu/article/view/1149.
Al-Munawwarah, S. (2015). Teachers’ perceptions on the use of ICT in Indonesian EFL
learning context. English Review: Journal of English Education, 3(1), 70-80.
https://journal.uniku.ac.id/index.php/ERJEE/article/view/116/78
Almusharraf, A. (2021). Bridging the gap: Saudi Arabian faculty and learners' attitudes
towards first language use in EFL classes. Issues in Educational Research, 31(3), 679-
698. https://www.iier.org.au/iier31/almusharraf.pdf
Asyiah, D. (2017). The vocabulary teaching and vocabulary learning: Perception,
strategies, and influences on students’ vocabulary mastery. Jurnal Bahasa Lingua
Scientia, 9(2), 293-318. http://dx.doi.org/10.21274/ls.2017.9.2.293-318
Atmojo, A., & Nugroho, A. (2020). EFL classes must go online! Teaching activities and
challenges during COVID-19 pandemic in Indonesia. Register Journal, 13(1), 49-76.
http://dx.doi.org/10.18326/rgt.v13i1.49-76
Azmi, N. (2017). The benefits of using ICT in the EFL classroom: From perceived utility to
potential challenges. Journal of Educational and Social Research, 7(1), 111.
https://doi.org/10.5901/jesr.2017.v7n1p111
Bergström, D., Norberg, C., & Nordlund, M. (2022). “Words are picked up along the way”
– Swedish EFL teachers’ conceptualizations of vocabulary knowledge and
learning. Language Awareness, 31(4), 393-409.
http://dx.doi.org/10.1080/09658416.2021.1893326
Bigatel, P., Ragan, L., Kennan, S., May, J., & Redmond, B. (2012). The identification of
competencies for online teaching success. Journal of Asynchronous Learning
Networks, 16(1), 59-77. http://dx.doi.org/10.24059/olj.v16i1.215
Bikowski, D. (2018). Technology for teaching grammar. The TESOL Encyclopedia of English
Language Teaching, 1-7. http://dx.doi.org/10.1002/9781118784235.eelt0441
Blanco González, N. & Mañoso-Pacheco, L. (2021). Spanish EFL teachers and ICT tools. A
survey-based analysis. Revista Inclusiones, 8(1), 174-186.
https://revistainclusiones.org/index.php/inclu/article/view/192
190
Cabrera-Solano, P., Gonzalez-Torres, P., Solano, L., & Castillo-Cuesta, L. (2019). Using
visual vocabulary app as a metacognitive strategy in EFL learning: A case of pre-
service teachers. International Journal of Learning, Teaching and Educational Research,
18(12), 302-315. https://doi.org/10.26803/ijlter.18.12.17
Cakici, D. (2016). The use of ICT in teaching English as a foreign language. Participatory
Educational Research, 4(2), 73-77.
http://www.perjournal.com/archieve/spi_16_4/per_16_spi_4_10.pdf
Castillo-Cuesta, L., Ochoa-Cueva, C., & Cabrera-Solano, P. (2022). Virtual workspaces for
enhancing collaborative work in EFL learning: A case study in higher education.
International Journal of Emerging Technologies in Learning (iJET), 17(2), 4-18.
http://dx.doi.org/10.3991/IJET.V17I02.25937
Chandra, R. (2015). Classroom management for effective teaching. International Journal of
Education and Psychological Research, 4(4), 13-15.
https://www.researchgate.net/publication/313889949_Classroom_Managemen
t_for_Effective_Teaching#fullTextFileContent
Chiekezie, P., & Inyang, M. (2021). The significance of audio-visual aids in teaching of
English vocabulary. GNOSI: An Interdisciplinary Journal of Human Theory and
Praxis, 4(2), 54-70. http://gnosijournal.com/index.php/gnosi/article/view/84
Cook, V. (2013). Second language learning and language teaching. Routledge.
Council of Europe (2020). Common European framework of reference for languages: Learning,
teaching, assessment – Companion volume. Council of Europe Publishing.
Fatmawaty, R., Setyaningrum, R., & Faridah, F. (2021). Vocabulary materials based on
picture media for autistic students. Linguistic, English Education and Art (LEEA)
Journal, 4(2), 425-435. https://doi.org/10.31539/leea.v4i2.1777
Ghalebi, R., Sadighi, F., & Bagheri, M. (2020). Vocabulary learning strategies: A
comparative study of EFL learners. Cogent Psychology, 7(1),
1824306. https://doi.org/10.1080/23311908.2020.1824306
Gikandi, J., Morrow, D., & Davis, N. (2011). Online formative assessment in higher
education: A review of the literature. Computers & Education, 57(4), 2333-2351.
https://doi.org/10.1016/j.compedu.2011.06.004
González‐Lloret, M. (2020). Collaborative tasks for online language teaching. Foreign
Language Annals, 53(2), 260-269. http://dx.doi.org/10.1111/flan.12466
Graham, C., & Misanchuk, M. (2004). Computer‐mediated learning groups: Benefits and
challenges to using groupwork in online learning environments. In T. S. Roberts
(Ed.), Online collaborative learning: Theory and practice (pp. 181–202). Information
Science Publishers. http://dx.doi.org/10.4018/978-1-59140-174-2.ch008
Habibi, A., Mukminin, A., Najwan, J., Sofwan, M., Haswindy, S., Marzulina, L., Sirozi, M.,
& Harto, K. (2018). Investigating EFL classroom management in pesantren: A case
study. The Qualitative Report. https://core.ac.uk/download/pdf/291991664.pdf
Irzawati, I., & Hasibuan, A. (2020). Students’ perceptions of the utilization of ICT in
English learning: Way or barrier? In 3rd International Conference on Innovative
Research Across Disciplines (ICIRAD 2019) (pp. 68-73). Atlantis Press.
Ja'ashan, M. (2020). The challenges and prospects of using e-learning among EFL students
in Bisha University. Arab World English Journal, 11(1), 124-137.
https://dx.doi.org/10.24093/awej/vol11no1.11
Kassem, M. (2018). The effect of a suggested in-service teacher training program based on
MALL applications on developing EFL students' vocabulary acquisition. Journal
of Language Teaching and Research, 9(2), 250-260.
http://dx.doi.org/10.17507/jltr.0902.05
191
Kaufmann, R., & Vallade, J. I. (2022). Exploring connections in the online learning
environment: Student perceptions of rapport, climate, and loneliness. Interactive
Learning Environments, 30(10), 1794-1808.
https://doi.org/10.1080/10494820.2020.1749670
Khan, R., Radzuan, N., Shahbaz, M., Ibrahim, A., & Mustafa, G. (2018). The role of
vocabulary knowledge in speaking development of Saudi EFL learners. Arab
World English Journal (AWEJ), 9(1), 406-418.
http://dx.doi.org/10.2139/ssrn.3151128
Kukulska-Hulme, A. (2019). Mobile language learning innovation inspired by migrants.
Journal of Learning for Development, 6(2), 116-
129. https://doi.org/10.56059/jl4d.v6i2.349
Kurniawati, D., & Sofiyah, S. (2021). The implementation of using ICT in teaching English
at EFL class of Ronggolawe University Tuban. EJI (English Journal of Indragiri):
Studies in Education, Literature, and Linguistics, 5(1), 1-14.
https://ejournal.unisi.ac.id/index.php/eji/article/download/1277/893
Ladd, H., & Sorensen, L. (2017). Returns to teacher experience: Student achievement and
motivation in middle school. Education Finance and Policy, 12(2), 241-
279.http://dx.doi.org/10.1162/EDFP_a_00194
Li, B. (2022). Ready for online? Exploring EFL teachers’ ICT acceptance and ICT literacy
during COVID-19 in mainland China. Journal of Educational Computing Research,
60(1), 196-219. https://doi.org/10.1177/07356331211028934
Mahyoob, M. (2020). Challenges of e-learning during the COVID-19 pandemic
experienced by EFL learners. Arab World English Journal (AWEJ), 11(4).
http://dx.doi.org/10.24093/awej/vol11no4.23
Maureen, I., Van der Meij, H., & De Jong, T. (2018). Supporting literacy and digital literacy
development in early childhood education using storytelling activities.
International Journal of Early Childhood, 50(3), 371–389.
http://dx.doi.org/10.1007/s13158-018-0230-z
Nikolopoulou, K. (2018). Mobile technologies and early childhood education. In M.
Tsitouridou, J.A. Diniz, T.A. Mikropoulos, & S. Hadjileontiadou (Eds.),
Proceedings of the 1st International Conference on Technology and Innovation in
Learning, Teaching and Education—TECH-EDU 2018, Aristotle University of
Thessaloniki, Greece. Springer. http://dx.doi.org/10.1007/978-3-030-20954-4_33
Noori, A., Orfan, S., Akramy, S., & Hashemi, A. (2022). The use of social media in EFL
learning and teaching in higher education of Afghanistan. Cogent Social Sciences,
8(1), 1-12. https://doi.org/10.1080/23311886.2022.2027613
Ortega-Auquilla, D., Sigüenza-Garzón, P., Chumbay, J., & Heras, E. (2022). Motivation in
English learning at university: A mixed-methods study investigating the
perceptions of different stakeholders. International Journal of Learning, Teaching and
Educational Research, 21(12), 175-196. https://doi.org/10.26803/ijlter.21.12.10
Pardede, P. (2020). Secondary school EFL teachers’ perception of ICT use in learning and
teaching: A case study in Greater Jakarta. Journal of English Teaching, 6(2), 144-157.
https://doi.org/10.33541/jet.v6i2.1734
Quinonez-Beltran, A., Cabrera-Solano, P., Gonzalez-Torres, P., Castillo-Cuesta, L., &
Ochoa-Cueva, C. (2020). A comparative study of EFL listening difficulties in
public and private Ecuadorian high schools. Pertanika Journal of Social Sciences and
Humanities, 28 (3), 2081-2100.
http://www.pertanika.upm.edu.my/resources/files/Pertanika%20PAPERS/JS
SH%20Vol.%2028%20(3)%20Sep.%202020/27%20JSSH-4909-2019.pdf
192
Rahim, M., & Chandran, S. (2021). Investigating EFL students’ perceptions on e-learning
paradigm-shift during COVID-19 pandemic. Elsya: Journal of English Language
Studies, 3(1), 56-66. https://doi.org/10.31849/elsya.v3i1.5949
Rogers, J. (2018). Teaching/Developing vocabulary through metacognition. The TESOL
encyclopedia of English language teaching. https://doi.org/10.1002/
9781118784235.eelt0737
Schmitt, N. (2008). Review article: Instructed second language vocabulary learning.
Language Teaching Research, 12(3), 329–363.
https://doi.org/10.1177/1362168808089921
Schmitt, N., & Carter, R. (2000). The lexical advantages of narrow reading for second
language learners. TESOL Journal, 9(1), 4-
9. https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=720c5ccf
c4b155f99007fe652638862284763314
Siddig, B. (2020). Social media in teaching of languages. International Journal of Emerging
Technologies in Learning (iJET), 15(12), 72-80.
https://doi.org/10.3991/ijet.v15i12.12645
Silviyanti, T., & Yusuf, Y. (2015). EFL teachers' perceptions on using ICT in their teaching:
To use or to reject? Teaching English with Technology, 15(4), 29-43.
Song, H., Kim, J., & Luo, W. (2016). Teacher–student relationship in online classes: A role
of teacher self-disclosure. Computers in Human Behavior, 54, 436-443.
https://doi.org/10.1016/j.chb.2015.07.037
Tadesse, S., & Muluye, W. (2020). The impact of COVID-19 pandemic on education system
in developing countries: A review. Open Journal of Social Sciences, 8(10), 159-170.
https://doi.org/10.4236/jss.2020.810011
Zülküf, A. (2017). Globalization, English language teaching and Turkey. International
Journal of Languages’ Education and Teaching, 5(4), 764–776.
https://doi.org/10.18298/ijlet.2238
193

Vol. 22, No. 7, pp. 193-218, July 2023
Received Apr 14, 2023; Revised Jul 9, 2023; Accepted Jul 15, 2023
Self-Instructional Teaching Internship Module:

An Evaluation
Imelda C. Montalbo , Maria Nancy Quinco-Cadosales ,
Angeline M. Pogoy and Jo Ann M. Petancio
Cebu Normal University, Cebu City, Philippines
Abstract. Teaching internship is the apex in the academic life of

preservice teachers to ensure their readiness to teach in the actual field.
To help teaching interns (TIs) thrive in their practicum, the self-
instructional teaching internship (STI) module is developed. This study
evaluated the module using the Plan-Do-Study-Act action research and
mixed-method research design to provide evidence to aid in its second
revision. The participants were 13 state university TIs. The data were
gathered by using a validated questionnaire, performance appraisal, and
interview guide. While descriptive statistics, percentages, and weighted
means were computed for the data. The module structure and format
were present and compliant with the design framework, according to
experts. The TIs unanimously agreed that the module adhered to
Outcome-Based Education (OBE), Technological Pedagogical Content
Knowledge (TPACK), Universal Design for Learning (UDL), and
comprised a useful learning resource that allowed them to recognise their
shortcomings, develop relationships, and become competent as
evidenced by their progressive performance. The use of the STI module
in Philippine Teacher Education Institutions is encouraged.
Keywords: quality education; teaching internship; teaching interns; self-

instructional teaching module; action research; mixed-method
Philippines
1. Introduction
The primary objective of education is to enable students to acquire the essential
knowledge, skills, dispositions, and values (Bersoto et al., 2014) while educational
institutions have the responsibility to provide support services to achieve the
intended learning outcome (Javier, 2012). However, during the COVID-19 crisis,
educational systems worldwide experienced massive interruptions of classes,
thereby identifying "hi-tech, low-tech, and no-tech solutions" to assure the
continuity of learning was deemed imperative by UNESCO Director-General
Audrey Azoulay as cited by Huang et al. (2020).
©Authors
194
In developing education and training programs, instructional design is seen as a

systematic, dependable, and reliable approach (Reiser & Dempsey, 2007, as cited
in Wagner, 2018). Also, instructional design entails the design, development, and
delivery of educational products and teaching-learning experiences (What is
Instructional Design?, n.d.). This includes the crafting of instructional modules to
enhance learning as students acquire knowledge and skills, which encourage
them to be more engaged, as they experience a deeper understanding of the lesson
(Merrill et al., 1996). With recent technological advances, instructional design has
evolved, and its future lies in flexible learning methods (Greene-Harper, 2023).
Flexible learning provides students with a diverse array of options for accessing
learning materials, accommodating their preferences and circumstances (Collis et
al., 1997; Lundin, 1999). Students have the opportunity to engage with the course
through various means, including in-person classes, online learning, or a
combination of both, facilitated by technologies like Augmented Reality (AR).
However, when the COVID-19 pandemic hit, colleges and tertiary institutions
were quick to switch from face-to-face lectures to virtual instruction. To meet
current needs, institutions are re-inventing their learning environments with the
expansion of digital technologies, while supplementing engagements between
learner and teacher (Schleicher, 2020) for learning to continue during the
pandemic. As a result, online or virtual, or e-teaching internships came to be
devised.
With online teaching internships, learner autonomy is developed among the

interns (Ugalingan et al., 2021). This could be attributed to the provision of diverse
ways in which students can engage with the educational content and materials in
flexible learning (Goode et al., 2007). And as students embrace the multi-
dimensional perspective, they learn through a constructivist approach by taking
responsibility for how they learn (Lewis & Spencer, 1986; Goode et al., 2007). They
become more self-regulated and skilled, while the teachers prepared them to be
more engaging (Collis, 1998).
Before the pandemic, most teachers resorted to using traditional face-to-face

instruction (Dayagbil et al., 2018). The flexible learning approaches in the era of
"Internet+" are varied, including individual or collaborative learning; learning
resources-based or thorough electronic devices or gadgets; and self-regulated or
inter-disciplinary learning methods (Huang et al., 2020). Despite the numerous
approaches and benefits reported matters; some educators and education
students in the Philippines lacked awareness and training in the use of flipped
classrooms, blended learning, and distance learning (Plaisent et al., 2016).
Consequently, challenges identified during the online teaching internship are
along the lines of technology. These include readiness for virtual instruction;
technical and technological adeptness; internet connectivity and accessibility
(Guiamalon, 2023; Lampadan et al., 2022; Ugalingan et al., 2021).
In response to these technological hurdles, the development of self-instructional

modules that students can access, either online or offline, is both promising and
practical. Self-instructional modules are designed as autonomous units of
195
instruction that aim to achieve specific learning objectives (Macarandang, 2009).

These modules possess two notable characteristics that facilitate self-paced
learning and accessibility. Firstly, they enable students to progress at their own
speed, independently, regardless of the group's pace. Secondly, the availability of
these self-instructional modules empowers students to learn at their convenience,
without being constrained by external timetables (Torrefranca, 2017).
The development of flexible instructional materials for teacher-educators is

anchored on constructivism and supported by blended learning with the TPACK
framework. Constructivism is a fusion of the tenets of behaviourism and
cognitivism. Dewey advocated active learning through experiences (Kliebard,
1992); Piaget emphasized the construction of understanding through reflecting on
experiences (Smith & Ragan, 1999); and Vygotsky introduced the setting up of
scaffolds and the zone of proximal development. On the other hand, blended
learning is the amalgamation of in-person and virtual instructional modalities
(Graham, 2013), referred to as the "new normal" in education with its extensive
adoption in higher education institutions (Norberg et al., 2011). Blended learning
is the mixture of the finest features of e-learning, structured in-person activities,
and real-life practice (Semler, 2005)
In this study, the Flexible Instructional Materials Development for Teacher

Educators are modules developed and anchored on the TPACK framework that
educators need to focus on in the design and implementation of the selected
professional education curricula. To provide quality education and life-long
learning, the TPACK directs students' thoughts and introduces technological
strategies (Koehler et al., 2013; Cetin-Berber & Erdem, 2015). The TPACK
framework was first created by Schulman (1986) and developed by Koehler et al.
(2013).
As Teacher Education Institutions (TEIs) prepare education students by

combining their pedagogical understanding with their technological and content
expertise, the flexible instructional materials developed specifically the module
for Teaching Interns expose them to the teaching and learning system. Moreover,
modules that connected classroom knowledge with workplace realities, provided
teaching interns with experimental experience and enabled them to pursue a place
in the dynamic teaching job in the modern world of work. This provides quality
education as stated in Article I Section 6.2 of CMO No. 74 and No.75 s. 2017 on
The Policies, Standards and Guidelines for Bachelor of Elementary Education
(BEEd) and Bachelor of Secondary Education (BSEd) and CMO No.74, which is
per the CHED mandate as explicitly stated in Article I Section one of CMO no. 46
s.2012 or the Policy Standard on Outcome-Based and Typology-Based Quality
Assurance. The implementation of the Teaching Internship module is anchored
on OBE, TPACK, UDL, and Diversity and Inclusion.
To ensure academic continuity and resilience amid the pandemic, or i any crisis,
the research-based flexible instructional module crafted for the teaching interns
serves as a resource guide for teachers in ensuring optimum student engagement
and learning. The Self-instructional Teaching Internship (STI) module responds
196
to the call for continuity, access, and flexibility in times of crisis and post-crises.
The STI module best engages students in remote and open- learning classes where
students become self-determined and independent learners, while teachers
facilitate using varied and flexible learning activities. It provides work-related
learning and practical preparation that is critical for the personal and professional
development of teaching interns ,as pre-service teachers and future in-service
teachers in real-world settings. This STI module was evaluated by the experts in
terms of content, structure, and form,at, and then further assessed and revised
after pilot-testing and implementation. Hence, the second revision of the STI
module evaluation is evidence-based.
2. Objectives of the Study

This study dealt with the validation, implementation, and evaluation of the self-
instructional teaching internship (STI) module. Specifically, it aimed to address
these questions:
a. How did the validators rate the STI module in terms of the structure and
format, and extent of compliance with the design framework?
b. What were the teaching interns' perceptions of the STI module?
c. What were the views and experiences of the teaching interns in the use of
the STI module?
d. How did the teaching interns perform during the actual teaching
demonstration?
3. Methodology
3.1 Research Design
This action-research adopted the Plan-Do-Study-Act (PDSA) model (Deming,
1993) and Creswell's (2014) mixed-methods research design in evaluating the
teaching interns' practice of using the module in the Teaching Internship course.
The PDSA is a methodical procedure in the acquisition of significant knowledge
and relevant insights geared towards the enhancement of a product, process, or
service (The Deming Institute, 2022).
3.2 Research Participants

The 13 participants were BEd teaching interns enrolled in the Academic Year 2021
- 2022 at a State University in Central Visayas, Philippines. In this study,
purposive sampling was used (Campbell et al., 2020; Ilker Etikan et al., 2016).
Those TIs who utilized the STI module in their virtual teaching internship and
completely taught for 4 consecutive weeks on-campus were included in the study.
These TIs were also under the guidance of the same teaching internship-mentor,
who will be rating their performance while on campus.
3.3 Research Instruments

Teaching Internship is the last course to be enrolled by the pre-service teachers to
complete the degree program of Bachelor of Elementary Education. To help the
preservice teachers, the module on teaching internship is developed and
validated. This module has 4 units with sub-topics to package the important
lessons in teaching internship. Various research instruments were used in for
evaluating the modules crafted. The modules were validated by 2 experts using
197
the validated checklist for compliance to design the framework. The evaluation
focused on the content, structure, and format in terms of how the modules
manifest adherence to the 4 frameworks of OBE, TPACK, UDL, and Diversity and
Inclusion.
A validated questionnaire consisting of 24 items on Students' Perceptions of the

module was formulated. It has 5 sub-scales, namely OBE, TPACK, UDL, Diversity
and Inclusion, and Learners' Satisfaction with statements for each sub-scale for
the interns to specify their agreement on a 4-point Likert scale, in which 4 =
strongly agree, 3 = agree, 2 = disagree, and 1 = strongly disagreed. The online
questionnaire was administered by using Google Forms. A Cronbach's alpha =
0.98 was computed after the questionnaire was pilot-tested.
A performance appraisal sheet for teaching interns was also used in rating the
teaching demonstration of the interns in the aspects of lesson design, teaching
strategies, class-room management, and communication skills. The scores of 9.30
- 10.00 = outstanding, 8.00 -9.29 = very satisfactory, 4.7 - 7.99 = satisfactory, 3.0 -
4.69 = fair, and 2.99 and below = unsatisfactory.
A focus-group discussion was carried out with the teaching interns, in order to
obtain feedback on their experiences in using the self-instructional module. The
probing questions focused on how they were assisted in the conduct of the phases
in mentoring, their overall experience in teaching on-campus and off-campus, and
the recommendations they can give to improve the self-instructional module.
3.4 Data-Gathering Procedure

The experts used a standardised tool to evaluate the self-instructional modules.
The comments and suggestions of the experts were considered in the first revision
of the modules. Then, the revised modules and the questionnaire on Students'
Perceptions were sent to pre-service teachers during the pilot test. After the
interns had used the modules during a period of 4 weeks, the revised modules
and the questionnaire on Students' Perceptions were distributed to the teaching
interns at the end of the 4 weeks.
To describe the teaching performance of the teaching interns, a performance

appraisal sheet was used. A focus-group was also conducted to describe the
interns' views and experiences in answering the self-learning teaching module.
3.5 The Data Analysis

Quantitative data collection was done through the experts' extensive presence and
compliance with the design framework checklists, and the Students' Perceptions
of the questionnaire. Descriptive statistics, such as percentage and weighted-
means were computed.
The qualitative data analysis was through Braun and Clarke's (2020) 6-step
thematic analysis that includes the raw data familiarisation, as wel as the initial
code generation, as well as the production thereof.
198

The study observed some protocols to ensure the safety of the participants.
Approval from the immediate supervisor was secured before conducting the
study. The participants signified their willingness to be included in the study via
the consent form. Then, a meeting was conducted to orient the participants on the
Students' Perception questionnaire and on how to evaluate the self-instructional
modules via Google form. Furthermore, a scheduled date and time for the focus
group was finalised, which was conducted via Google Meet. The identity of the
participants was protected at all times. The data gathered from the participants
were kept confidential at all times.

This section summarises the results through the different tables and figures, based
on the data gathered by using the different research instruments after performing
the necessary and appropriate data analyses; a discussion of the results is also
provided accordingly.
4.1 Evaluation of Validators to the STI Module

The STI module consists of 4 chapters, namely: 1) Introduction to Teaching
Internship students; 2) The Teaching-Learning Process (On & Off Campus); 3)
Action Research, and 4) Anciliary Activities in Teaching Internship. Chapter 1 has
two lessons; while Chapters 2 to 4 have three lessons each, making the module a
total of 11 lessons.
In the evaluation of the validators for the Self-instructional Teaching Internship

(STI) Module, there were two parts: 1) Content, Structure, and Format, and 2)
Compliance with the Design Framework. The validation results by the two
experts in the field served as the basis for the first revision of the STI Module
before pilot testing of the module and implementation during the second semester
of the school year.
The STI Module structure and format were first revised, based on the ratings of
the experts in Table 1. The module overview was partially present and the module
writers needed to revise the module, based on the remarks and recommendations
of the experts. All of the experts considered the course content and learning
outcomes as fully present; however, there was a need to organise some of the
contents. So, in the first revision, the chapters were re-organised from the
Introduction to the Teaching Internship, the Teaching-Learning Process (On & Off
Campus), Action Research, and Ancillary Activities in Teaching Internship. The
learning outcomes were retained, such as explaining the policies and standards in
teaching anchored on the code of ethics for teachers, performing the duties
necessary in the efficient and effective delivery of the processes in actual teaching
by being mindful of the recommendations given during conferences with the co-
operating teachers; filling out correctly the forms required by the Department of
Education and other relevant documents related to teaching; performing the
duties necessary in the efficient and effective delivery of the processes in actual
teaching by being mindful of the recommendations given during conferences with
the co-operating teachers; sharing the results of one's action research; and
199
demonstrating one’s skills in performing the ancillary activities to enhance

students' learning.
Table 1: STI Module Structure and Format as Rated by the Experts

Extent of those Present
Structure and
Partially Fully Description/Remarks
Format Absent
present present
Module
 Needs revision
Overview
Module/Course Needs to be

Contents organized
Course
Learning  Maintain as is
Outcomes
Learning
 Add varied activities
Experiences
Assessment of Needs to be

Learning organized
More research-based
activities and
provision for
Enhancement 
synchronous and
asynchronous
activities
References
Books
Articles
Pictures
Videos  Use APA format
Research
articles
Accessibility
and copyright
The learning activities for both online and physical, or blended, synchronous, or
asynchronous were added for the teaching interns; and vided varied experiences.
The assessment activities were organised with the experts' comments and
suggestions as the basis. More research-based activities for synchronous and
asynchronous activities for enhancement were included. The references and
materials used in the modules were also retained when using the APA format.
The STI Module compliance to design the framework, as rated by the experts, is
presented in Table 2. The results given by the validators were the basis for revising
the STI module before the pilot testing. For the OBE framework, it was rated
average to highly compliant. There was a revision of the learning objectives of the
module that were reflected as the desired outcomes. Specific lesson objectives
aligned with the course outcomes and assessment strategies were made to
measure the desired learning outcomes, which was to hone their teaching skills
during their internship. This is considered since teaching internship experiences
influences how they also perceive their ability to teach (Clark et al., 2015).
200
Table 2: STI Module Compliance to Design Framework as Rated by the Experts
Extent of Compliance
Framework Design Remarks
Low Average High
OBE
Learning objectives clearly reflected
2
the desired outcomes.
Revise the
Specific lesson objectives map to
2 learning
the course outcomes.
objectives
Teaching and learning activities are
based on
designed to accomplish the 2
course
intended learning outcomes.
learning
Activities encourage students to
outcomes of
engage deeply in what they are
2 student
learning and provide them with
teaching
expanded opportunities.
internship
Assessment strategies measure the
2
desired learning outcomes.
60% 40%
TPACK
Appropriate technology tools were
2
used throughout the lessons.
Concise, accurate, and relevant
concepts of the lessons were 2
Revise using
presented.
appropriate
Activities in the module reflected
technology
essential strategies and techniques 2
tools
that promote student learning.
throughout the
The use of technology enabled
lesson that
students’ engagement with their 2
supports the
lessons.
learning
The integration of technology
2 content
promoted independent learning.
Provisions for the use of technology
supported the content and the 2
learning of the concepts.
83% 17%
UDL
Options were provided for the
engagement of the learning
2
material for online or offline
learning.
Choices of examples and learning Provide
activities were relevant to the needs 2 relevant and
and interests of the learners. varied online
The design promoted learning and offline
continuity for both online and 2 activities for
offline modalities. diverse
The diverse capabilities of learners students
were considered in the timing and 2
pacing of the activities.
Choice of activities encouraged
2
varied ways of action and
201
expression to include individual

differences and multiple
intelligences.
40% 40% 20%
GAD
Topics and inputs were gender
2
sensitive.
Consider
Learning activities gave equal
gender
opportunity and respond to diverse 2
sensitivity in
students to learn.
terms of
Learning materials (e.g.,
lessons, and
illustrations, pictures, and 2
materials to be
diagrams) were gender sensitive.
inclusive
Lesson examples were carefully
2
chosen and inclusive.
75% 25%
In the TPACK design framework, it was found that validators rated the
compliance as average to highly compliant. This means that there was a need to
revise the module by using appropriate technological tools throughout the lesson
that would engage students actively, as it promotes independent learning. These
technological tools support the contents of the module.
In the UDL design framework, the STI module was revised for there was low to
average compliance. In the first revision, based on the ratings and comments
given by the experts, the provision of options for relevant engagement in both
online and offline activities was ensured, while making continuity of the activities
of the lessons was also considered for the diverse students. The use of the UDL
Framework in this module was to improve and maximise instruction with more
inclusive and transformative teaching and learning experiences (CAST Inc., 2022).
In the gender and developmental framework, the module was revised to be more
gender-sensitive, as stipulated by the ratings of the experts. The learning activities
and learning materials were revised to respond to the students' diverse needs.
This is also required of higher education institutions, as stipulated in CHED
Memorandum Order (CMO) No. 01, series of 2015 in the establishment of Gender
and Development to help create a gender-sensitive learning environment.
Hence, the first revision of the STI module was based on the evaluation of the
experts or consultants, as to the content, format, structure, and design framework
compliance. The revision was done before pilot testing and later at the
implementation of the module.
4.2 Teaching Interns’ Perceptions of the STI Module

The tables that follow reveal the perceptions of the teaching interns on the first
revision of the self-instructional module in terms of five facets, namely OBE,
TPACK, UDL, Diversity and Inclusion, and Learners’ Satisfaction.
202
Table 3: Teaching Interns’ Perceptions: OBE (N=13)
Statement SD D A SA Mean Sd
1) The learning objectives
clearly reflected the desired
outcomes. 0% 0% 0% 100% 4 0
2) The specific lesson objectives
are aligned with the course
learning outcomes. 0% 0% 0% 100% 4 0
3) The activities are designed to
accomplish the intended
learning outcomes. 0% 0% 0% 100% 4 0
4) The activities encouraged
students to engage deeply in
what they are learning. 0% 0% 7.69% 92.31% 3.92 0.28
5) The activities provided the
students with opportunities for
further learning. 0% 0% 0% 100% 4 0
6) The assessment strategies
measured the desired learning
outcomes. 0% 0% 0% 100% 4 0
Overall Perception 3.99
Table 3 shows that the teaching interns strongly agreed with the statements that
the STI module adheres to the OBE Framework, as reflected in the overall
perception of 3.99. They deemed that there is alignment of the lesson objectives,
activities, and assessment strategies with the intended learning outcomes. This
group of teaching interns has been exposed to OBE ever since their first year in
college. The extent to which a program is designed is based on the OBE principles
is the basis for that program to be legitimately called outcome-based (Aldridge et
al., 2006). The same could be said for instructional materials, such as the self-
instructional module. In OBE, what students can do and demonstrate, when
instruction commences, is the focus and basis for all processes in the educational
system (Spady, 1994).
The results in the table show how the teaching interns perceived the module to
embody the OBE principles. This perception arises from their first-hand
experiences of such principles while utilizing the STI module. In return, the
experiences interns have of various teaching strategies and methods during their
internship, which can influence how they also perceive their ability to teach (Clark
et al., 2015). Their exposure to the outcome-based STI Module is a form of support
(Lamp Adan et al., 2023) and an avenue to model teaching strategies (G et al.,
2021) to the teaching interns as they cope with various challenges in their virtual
teaching internship.
203
Table 4: Teaching Interns’ Perceptions: TPACK (N = 13)
1) Appropriate technology
tools were used
throughout the lessons. 0% 0% 0% 100% 4 0
2) Concise, accurate, and
relevant concepts of the
lessons were presented. 0% 0% 30.77% 69.23% 3.69 0.48
3) Activities in the module
reflected essential
strategies and techniques
that promote student
learning 0% 0% 7.69% 92.31% 3.92 0.28
4) The use of technology
enabled students’
engagement with their
lessons. 0% 0% 7.69% 92.31% 3.92 0.28
5) The integration of
technology promoted
independent learning. 0% 0% 0% 100% 4 0
6) Provisions for the use of
technology supported the
content and the learning of
the concepts. 0% 0% 0% 100% 4 0
Based on Table 4, the overall perception of 3.92 means that the interns strongly
agreed that the STI Module adheres to the TPACK Framework. The TPACK is
both a knowledge type and technological integration framework (Koehler et al.,
2013) that requires activities and procedures that would aid in developing it
(Archambault & Barnett, 2010). The teaching interns believe that the module
contains the necessary tasks and techniques to cater to this special kind of
knowledge.
In Oner (2020), there was an increase in the complexity of pre-service teachers'

TPACK representations, as a manifestation of the effectiveness of virtual
internships. Also, pre-service teachers with TPACK modules in their courses
acquired more TPACK than those without them (Lachner et al., 2021). Meanwhile,
in the present study, the mode for internship of the participants was purely virtual
or online with synchronous and asynchronous classes for the learners in the
laboratory school. This was supplemented by the STI module that the interns
deemed to be TPACK aligned. If the virtual internships and TPACK modules
were found to promote the acquisition of TPACK, then the actual internship
through virtual or online modality plus the STI module can also potentially do the
same or possibly even more for the participants in this study. Furthermore,
Ismaeel and Al Mulheim (2022) advocated for the integration of conventional and
e-teaching internship strategies to develop TPACK among prospective educators.
204
Table 5: Teaching Interns’ Perceptions: UDL (N = 13)
1)Options were provided for
the student’s engagement
with the learning materials
during online and offline
sessions. 0% 0% 23.08% 76.92% 3.77 0.44
2) Examples and learning
activities were relevant to the
needs and interests of the
learners. 0% 0% 0% 100% 4 0
3) The design of the module
promoted learning continuity
for both online and offline
sessions. 0% 0% 23.08% 76.92% 3.77 0.44
4) The diverse capabilities of
learners were considered in
the timing and pacing of the
activities. 0% 0% 7.69% 92.31% 3.92 0.28
5) The learning activities
encouraged varied ways of
action and expression that
reflect individual differences
and multiple intelligences
among the students. 0% 0% 7.69% 92.31% 3.92 0.28
Table 5 reflects the overall perception of the teaching interns towards the STI
module in terms of the Universal Design for Learning Framework, which is 3.88.
This means that the interns strongly agree with all the statements that the module
is grounded in the said framework. Based on a scientific understanding of how
individuals learn, the UDL Framework was created for the improvement and
optimisation of teaching and learning for everyone, in order to make learning
more inclusive and transformative (CAST Inc., 2022).
In a study by Lowrey et al. (2019), teaching pre-service teachers the fundamentals

of UDL did not result in a significant shift in the way that planning and instruction
were carried out, with the teacher-candidates adhering first to traditional lesson
planning, and UDL lesson planning next. On the other hand, based on various
study findings, teacher candidates who received relatively brief instruction in the
three UDL principles of engagement, representation, action, and expression were
more successful in creating lessons that enabled students with and without
disabilities to master the same material in inclusive classrooms (Lee & Griffin,
2021). Although the interns in this present study believed that the STI module
exemplifies the principles of UDL, it is yet to be known to what extent their ability
to design lessons has improved. With interns identifying challenges in their online
internships, such as their ability to facilitate online classroom interaction (Lamp
Adan et al., 2023; Galligan et al., 2021) and their seemingly inadequate actual
teaching experiences (Bulawat, 2021), providing them with the STI module can be
205
viewed as an innovation-related experience that is associated with learning

progress (Guindon, 2022).
Table 6: Teaching Interns’ Perceptions: Diversity and Inclusion (N= 13)
1)The topics and inputs in the
module were gender sensitive. 0% 0% 0% 100% 4 0
2) The learning activities provided
equal opportunities for students
with diverse ways of learning. 0% 0% 7.69% 92.31% 3.92 0.28
3) The learning materials (e.g.,
illustrations, pictures, diagrams,
etc.) used in the lessons were
gender sensitive. 0% 0% 15.38% 84.62% 3.85 0.38
The results presented in Table 6 show that the teaching interns strongly agree that
the STI module has provisions for diversity and inclusion. An overall perception
level of 3.92 manifests this. They view the topics, inputs, learning activities, and
learning materials in the STI module to be gender-sensitive.
Teacher preparation programs that include coursework on inclusion and special

education allowed pre-service teachers to engage with learners from diverse
backgrounds (Harvey et al., 2010). Also, participants, who had taken an inclusive
education course were more supportive of inclusion in later years of study
(Kraska & Boyle, 2014). In a virtual internship, Bulawat (2021) found out that one
of its limitations is the non-reinforcement of the interns' knowledge of the
differences among their students. But since the teaching interns in this study
perceived the STI module to be gender-sensitive and inclusive, it is potentially
impactive on their attitudes towards inclusion and diversity of their students in
the online classroom set-up.
Table 7 shows how the teaching interns perceive the STI module to be satisfying
to them as learners. They strongly agree with all the statements, as shown in the
table. Furthermore, the overall perception rating of 3.86 reflects their satisfaction
as learners who have utilised the STI module.
Table 7: Teaching Interns’ Perceptions: Learners’ Satisfaction (N = 13)
1)The time allotted for the
activities was manageable. 0% 0% 23.08% 76.92% 3.77 0.44
2) The activities/tasks
enabled students’ learning
to be independent and self –
regulated. 0% 0% 30.77% 69.23% 3.69 0.48
3) The module provided
opportunities for the 0% 0% 0% 100% 4 0
206
students to be actively
engaged in learning.
4) The module enabled the
students to attain the
maximum learning
competencies of the course. 0% 0% 7.69% 92.31% 3.92 0.28
In the study by Ahmed Abdullah and Sultana Mirza (2020), students expressed
satisfaction with the teaching practice module, and such feedback was taken into
consideration for continuous improvement. Also, Mandera (2023) reported how
the practice teachers gained in the acquisition of the needed knowledge and skills
through both online and virtual internships. Likewise, the interns in the present
study were highly satisfied with the STI module, as a complementary resource to
their ongoing virtual internship. Obtaining feedback and suggestions from
teaching interns is valuable for every teacher in an educational institution.
In totality, the perceptions of the teaching interns ranged from 3.86 to 3.99, as
shown in Tables 3-7, which means that they are in strong agreement that the
module adheres to the tenets of OBE, TPACK, UDL, and Diversity and Inclusion.
Furthermore, they were very satisfied with the STI module, which they used
during their practicum. Their perceptions could be attributed to the revisions
made by the authors in response to the expert consultants' evaluation and
recommendations before the pilot implementation. Consequently, the course
contents, outcomes, online and offline activities, and assessment tasks in the
module can be sustained.
4.3 Views and Experiences of the Teaching Interns

Employing the thematic analysis of Braun and Clarke (2020), the following views
and perceptions of the teaching interns were revealed.
4.3.1 Getting learning support from STI Module

Learning resources can be changed from physical forms, like books and
magazines, to non-physical forms, like cyberspace, software programs and
applications, and, most notably, the social media (Kilpatrick, et al., 2019). Also, as
revealed in the study of Petancio and Bonotan (2018), student teachers identified
reading books and other references as a way to cope throughout their teaching
internship. When the COVID-19 pandemic struck, the teaching interns performed
their virtual teaching internship. To assist them in their daily tasks, a self-
instructional teaching internship module (STI) was crafted. The STI module served
as a learning support as revealed by teaching interns (TI 2,5,9). The teaching interns
acknowledged the learning support they got as they utilised the module.
4.3.1.1 Finding the relevance of the STI

The topics, activities, and assessment of the STI module was carefully validated
and evaluated by experts. Thus, the teaching interns found the STI module
relevant; as it provided simple explanations on the different dimensions from pre-
teaching and observation conferences, full teaching immersion to post-conference.
207
It served as their guide, as they performed the virtual teaching demonstrations.

The online videos provided in the module served as a guide in the conduct of
authentic observations before actually teaching. They found it comprehensive to
manage a class and boost their level of confidence in classroom teaching.
4.3.1.2 Providing a digital aid

The STI module provided links to online resources that the teaching interns could
use as they prepare for their virtual teaching demonstrations. They believed ICT
is simple to use, even when it has academic benefits (Sayaf et al., 2022). Students
who have access to ICT are highly probable to consider the value of technology in
teaching. Similarly, the teaching interns acknowledge the great contribution of the
STI module in integrating technology into virtual teaching.
4.3.2 Acknowledging one’s insufficiencies

The online teaching internship has brought about some degree of discomfort
among teaching interns. As teaching interns narrated that “Student teaching
internship was like a rollercoaster ride and a tough one (TI10).”
4.3.2.1 Exploring one’s self

Using the STI module allows the teaching interns to evaluate themselves as they
reported that the activities helped them to explore their strengths and weaknesses.
Their experiences in virtual teaching internship support the theory regarding the
phases of melting among teaching interns when using the online instructional
delivery mode (Quinco-Cadosales, 2022). According to this theory, the teaching
interns engage in introspection by being aware of their feelings, expectations, and
anxieties.
4.3.2.2 Becoming a reflective thinker

To better prepare teaching interns for the classroom, teacher education programs
should place a premium on the integration of practice-based activities into the
curriculum. During these activities, the interns must be able to demonstrate their
abilities to reflect and think critically. Explicit teaching and scaffolding of these
essential abilities, while teaching interns, are immersed in actual and practical
experiences that can enhance their self-reflection and critical thinking skills (Harn
& Meline, 2021).
4.3.3 Developing relationships during mentoring

One important aspect of teaching internships is building relationships in
mentoring. This refers to the collaborative effort of the teaching intern and the
teaching internship mentor in terms of co-planning in lesson designing and
several concerns that need to be addressed before the actual teaching
demonstration (Posner, 2005). During the mentoring stage, the teaching interns
developed relationships with their mentors and co-interns. Iradel et al. (2021)
identified receiving support as one of the emergent themes in their
phenomenological study involving teaching interns during the time of the
pandemic. The teaching-intern participants revealed that they had received
support from their parents, friends, mentors, and co-teaching interns. Cadosales
et al. (2021) emphasised the role of a mentor in coaching and feedbacking for the
success of teaching interns in their teaching internship mentoring framework.
208
4.3.3.1 On-campus and off-campus teaching

The teaching interns placed a premium on their on-campus teaching experiences.
Their actual participation in teaching demonstrations, as an application of what
they had learned from the STI module honed their teaching skills. A high-quality
pre-conference lays the groundwork for trust-building between the teaching
interns and the teaching internship-mentors during on-campus teaching. Pre-
conferences boost motivation and confidence, which eventually leads to higher
accomplishments on the part of the teaching interns and a sense of purpose on the
part of the teaching-internship mentors (Kuijpers et al, 2010).
Moreover, the desirable relationship between teaching interns and co-operating

teachers was sustained during off-campus teaching. The co-operating teachers
take on roles, such as modelling, guiding, leading, planning, and motivating that
provided opportunities for teaching interns to develop competencies during off-
campus experiences (Abas, 2016). Both the on- and off-campus teaching
opportunities strengthened the teaching interns' teaching competencies.
4.3.4 Becoming a competent teaching intern

Teaching internship experiences is a process of developing understanding,
capabilities, and perspectives that equip teaching interns how to become teaching
professionals. Teaching interns themselves recognise that they are life-long
learners, who never cease to learn, even when they become teachers themselves
in the future (Petancio, 2019). Pepito (2022) concluded that their teaching
internship experiences helped them to develop unanticipated knowledge,
abilities, and values. Additionally, Montalbo et al. (2021) suggested that higher
education institutions give pre-service and professional teachers thorough
training for multi-grade teaching and different student groups, in order to further
develop their teaching competence.
4.3.4.1 Gaining knowledge on teaching internships

Experiences in the field allow practising teachers to establish a strong link
between what they know in theory and what is being practised in schools
(Hudson et al., 2008). The assimilation of theory and practice is aided further, as
they critically observe and analyse the lessons proved by their more experienced
co-operating teachers and mentors (Genc & Buyukkarci, 2013). The STI module
helped the teaching interns to gain knowledge about the entirety of teaching
internship. It provided them with an avenue to value the various tasks of a
teacher.
4.3.4.2 Acquiring teaching skills

In the study of Dsilva et al. (2022) on the implementation of an intervention that
included a self-instructional module, the average knowledge and practice scores
at the post-test of the 51 nursing students that were considerably higher than those
at the pre-test (P < 0.05). Knowledge had increased by 36%, and competence had
improved by 41% since the intervention. In the same vein, this study also revealed
that the teaching interns had enhanced their teaching skills through the use of the
self-instructional teaching module. The teaching intern (TI1) shared that “The
teaching demonstrations enhanced my skills in choosing appropriate teaching strategies,
209
managing the virtual classroom, improving my communication skills, choosing the

appropriate assessment of learning, asking the right questions, and more importantly
using the learning management system.”
4.3.4.3 Developing positive attitudes towards teaching

Time management and an optimistic attitude were also factors that allowed the
students to effectively finish their modules, according to the findings of Villonez
et al. (2022). The perseverance and time management skills of the students
allowed them to complete the module on time. As a teaching intern (TI5) narrated
that “I am excited during my teaching demonstrations. I enjoy the moment by being really
involved in the teaching-learning process. I realised that the teaching demonstration is fun
as well as our interaction with the students.”
Figure 1: Teaching Interns’ Views and their Experiences with the STI Module
At a glance, Figure 1 displays the teaching interns' views and experiences with the
use of the self-instructional teaching internship (STI) module. It can be gleaned in
the figure the emergence of 4 themes and their corresponding sub-themes. The
cyclical relationship of the themes is founded on the realisation of the TIs that the
self-instructional teaching internship (STI) module is a relevant learning resource;
and it serves as a digital aid. Through the STI module, they can acknowledge their
insufficiencies by exploring oneself and becoming a reflective thinker. Their
teaching internship experience allows them to develop relationships during the
mentoring sessions with their respective mentors, both in on-campus and off-
campus teaching. At the end of the cyclical process, they become competent; as
they had gained knowledge on teaching internship; acquired the necessary
teaching skills; and developed positive attitudes towards teaching.
4.4 Teaching-Interns’ Performance

The teaching performance of the interns during the actual demonstration was
progressive in the aspects of lesson design, strategies in teaching, classroom
management, and communication skills. The STI module was used to improve
the performance of the teaching interns. Moreover, interns who participated in
online and modular internships gained knowledge and skills that they could
apply in their field of internship (Mancera, 2023).
210
Figure 2 shows the performance of teaching interns in lesson design. In week 1,

77% of the student interns got a rating of 6 as being satisfactory, which can be
attributed to late submission of the lesson designs for checking and approval; and
some parts of the lesson design were not included or correctly presented. Lesson
planning is a crucial skill that beginning teachers must master; and it typically
takes some time because there is confusion over the exact designs and format, and
difficulty in identifying instructional objectives and learning experiences that are
appropriate to the learners (Gafoor & Farooque, 2010). Likewise, they require a
thorough awareness of the critical link that exists between planning and teaching.
The clarity and conciseness of the module's guidelines and activities in terms of
lesson planning aided the interns in developing a good lesson (Mancera, 2023).
Figure 2: TIs Performance in Lesson Designing
In week 2, there was an increase in the rating of the performance of teaching

interns inwhich 38% of the teaching interns got a rating of 8, which can be
associated with the use of proper formatting and editing. In week 3, 53% of the
teaching interns got a rating of 8, due to the correct presentation and accuracy of
the lesson design. In week 4, 54% of the teaching interns had submitted the lesson
design five days before the demonstration. Likewise, all parts of the lesson design
were completed by using the correct format with fluency and accuracy in the use
of the language. As mentioned by Prastonwo and Listyani (2020), teaching interns
can enhance their skills in writing lesson designs through practice in authentic
classroom settings. They believed that detailed lesson plans are relevant tools
during their teaching internship journey (Gafoor & Farooque, 2010).
211
Figure 3: TIs Performance in Strategies in Teaching
Figure 3 shows the teaching interns' performance in the use of strategies in

teaching during the actual demonstration. In week 1, more than half (62%) of the
teaching interns had satisfactory performances in the teaching demonstration.
This means that their methods were not varied and comprehensive. Likewise,
they used common teacher-directed strategies. According to Le Donne et al.
(2016), direct instruction enables learners to accomplish easier tasks; but it does
not do much in preparing them to perform more difficult tasks.
In week 2, almost 40% of them got a rating of 8, and 25% of them got a rating of
10. This means that their methods of teaching were very satisfactorily facilitated
and enhanced the development of the lesson; and lesson activities provided active
participation among students. In week 3, 61% of the teaching interns had very
satisfactorily exhibited their mastery of the subject matter. However, in week 4,
more than half (54%) of the teaching interns got a rating of 10 in their actual
demonstration. This means that they utilised a more active learning strategy and
cognitive activation strategy; as they engaged students in group work and let
them explain their thinking on complex problems, which has the strongest
association with raising students' achievements (Le Donne et al., 2016). Setting
specific activities and outcomes in the STI module for internship and training
content ensures that interns absorbed the key lessons, content, and procedures
(Fullington & Harrick, 2017).
Figure 4 shows the teaching interns' performance in classroom management. In

week 1, almost half (46%) of the teaching interns got a rating of 8. This means that
they have very satisfactorily prepared the needed instructional materials before
teaching time. Likewise, they have similar performance in terms of managing the
class in weeks 1 and 2 with only 62% of them being able to minimise discipline
problems.
This implies that more emphasis should be placed on classroom discipline in the
teaching practice of teaching interns. Teaching practice, as described by Prastomo
and Listyani (2020), certainly causes a mixture of excitement and concern among
teaching interns when they begin their teaching practice . Inadequate classroom
management is not just among the most pressing issues for teaching neophytes; it
212
is also one of the leading causes of stress, frustration, and teacher turnover
(Harmsen et al., 2018).
Figure 4: TIs Performance in Classroom Management
However, at the end of week 4, 69% of the teaching interns got a rating of 10. This
suggests that there was evidence of order and structure in how the class activities
were carried out, as well as sensitivity to student reactions throughout recitation.
Similarly, they were able to maintain the students' attention and start and end the
class on time. Classroom management consists of strategies and procedures that
teachers employ to achieve and sustain a learning environment conducive to
instruction, employing tools and tactics to achieve behavioural change (Lew &
Nelson, 2016). The ability to manage a class is essential in practice teaching
(Shafqat, 2016). The STI module included information on how to create
developmentally appropriate behaviour rules in classrooms, allowing interns to
connect their activities to the students' behavioural expectations.
Figure 5: TIs Performance in Communication Skills
Figure 5 shows the performance of teaching interns in communication skills. It

can be noted that starting in week 1, 15% of them demonstrated satisfactory
performance in speaking English or Filipino when talking to the students during
the actual demonstration. Teachers that lack communication skills often have
difficulty in creating engaging lessons, and they struggle to connect with their
students (Jakhanwal, 2021).
213
However, their communication abilities improved dramatically in weeks 3 and 4,

with 54% and 92% of the interns receiving a rating of 10. This signifies that their
voices were modulated properly during the continuous actual demonstrations in
the classroom, and their communication is clear and distinct. They also
demonstrated effective abilities to communicate with fluency and accuracy. Good
communication skills enhance relationships and increase understanding between
students and teachers (Khan et al., 2017). The communication skill links provided
in the STI module attempted to address many of the core issues related to
communication skills; and improved these skills among the teaching interns.
In general, the teaching interns' teaching performance was progressive because

they transmitted content, engaged in meaningful experiences, innovated means
of teaching, and critically reflected on their performance. The STI module served
as a learning tool for teaching interns to be aware of their responsibilities and
enhanced their performance as they eventually learned the science and art of
teaching.
5. Conclusion
The self-instructional teaching internship module was created to serve as a
resource guide for teaching interns during their in-person and/or virtual teaching
internships. The expert evaluation paved the way for the initial revision of the
module and the subsequent pilot implementation to teaching interns during their
online practicum. The perceptions, perspectives, and performance of the teaching
interns were used to further evaluate the revised module.
The STI module was a valuable learning resource for the teaching interns. The use
of the module aided them in improving their multi-dimensional teaching
performance, as well as developing their teaching competence, skills, and
character. The first revision of the module will be maintained, along with its
components - overview, contents, outcomes, learning experiences, assessment,
enhancement, and references. Hence, the module is recommended for use by
teaching interns in Philippine Teacher Education Institutions.
A limitation of this study is its pilot implementation to interns exposed to online

teaching internship only. Another evaluation study for module utilisation among
teaching interns in full face-to-face or blended mode during their practice teaching
is recommended.
6. Acknowledgements
The authors are grateful to the Commission on Higher Education Revised and
Expanded Continuing Professional Education (RECPE) Grants - Flexible
Instructional Materials Development for Teacher Educators (FIMDTED) and the
Cebu Normal University.
7. References
Abas, M. (2016). Pre-Service teachers’ experiences during off-campus observation: Basis
for improving the roles of teacher education institutions and cooperating schools.
Journal of Education and Learning, 10(2), 187-202.
https://doi.org/10.11591/edulearn.v10i2.3449
214
Ahmed Abdullah, N. & Sultana Mirza, M. (2020). Evaluating pre-service teaching practice
for online and distance education students in Pakistan. The International Review of
Research in Open and Distributed Learning, 21(2), 81–97.
https://doi.org/10.19173/irrodl.v21i2.4606
Aldridge, J.M., Laugksch, R.C. & Barry J. Fraser, B.J. (2006). School-level environment and
outcomes-based education in South Africa, 9(2), 123–147.
https://doi.org/10.1007/s10984-006-9009-5
Archambault, L.M. & Barnett, J.H. (2010). Revisiting technological pedagogical content
knowledge: Exploring the TPACK framework. Computers and Education, 55(4),
1656–1662. https://doi.org/10.1016/j.compedu.2010.07.009
Bersoto, L.D., Lescano, J.D., Maquimot, N.I., Santorce, M.J.N., Simbulan, A.F. &
Pagcaliwagan, A.M. (2014). Status of implementation and usefulness of outcomes-
based education in the engineering department of an Asian university.
International Journal of Multidisciplinary Academic Research, 2(4), 14-25.
https://research.lpubatangas.edu.ph/wp-content/uploads/2014/12/IJMDAR-
STATUS-OF-IMPLEMENTATION-AND-USEFULNESS-OF-OUTCOMES-
BASED-EDUCATION-IN-THE-ENGINEERING-DEPARTMENT-OF-AN-
ASIAN-UNIVERSITY.pdf
Braun, V. and Clarke, V. (2020). Can I use TA? Should I use TA? Should I not use TA?
Comparing reflexive thematic analysis and other pattern-based qualitative
analytic approaches. Wiley Online Library. https://doi.org/10.1002/capr.12360
Bulawat, A.C. (2021). Practice teaching amidst the pandemic: Challenges and
opportunities. International Journal of Scientific Research in Multidisciplinary Studies,
7(5), 19-26.
Cadosales, M. N. Q., Cabanilla, A. B., Elcullada, R. O., Lacea, R. & Beltran, N. (2021). A
meta-synthesis on mentoring framework on teaching internship. Turkish Online
Journal of Qualitative Inquiry (TOJQI), 12(6), 8955 - 8964.
https://tojqi.net/index.php/journal/article/view/3399
Campbell, S., Greenwood, M., Prior, S., Shearer, T., Walkem, K., Young, S., Bywaters, D.
& Walker, K. (2020). Purposive sampling: complex or simple? Research case
examples. Journal of Research in Nursing, 0(0), 1-10.
https://doi.org/10.1177/1744987120927206
CAST Inc. (2022). About Universal Design for Learning.
https://www.cast.org/impact/universal-design-for-learning-udl
Cetin-Berber, D. & Erdem, A. (2015). An investigation of Turkish pre-service teachers’
technological, pedagogical and content knowledge. Computers, 4(3), 234–250.
https://doi.org/10.3390/computers4030234
Clark, S.K., Byrnes, D., & Sudweeks, R. (2015). A comparative examination of preservice
and in-service teacher perceptions based on length and type of student teaching
assignment. Journal of Teacher Education, 66(2), 170–183.
https://doi.org/10.1177/0022487114561659.
Collis, B. (1998). New didactics for university instruction: Why and how? Computers and
Education, 3(3), 373-393. https://doi.org/10.1016/S0360-1315(98)00040-2
Collis, B., Vingerhoets, J. & Moonen, J. (1997). Flexibility as a key construct in European
training: Experiences from the TeleScopia Project, British Journal of Educational
Technology, 28(3), 199-218. https://doi.org/10.1111/1467-8535.00026
Creswell, J. W. (2014). Research Design: Qualitative, Quantitative and Mixed Methods
Approaches (4th ed.). Sage.
Dayagbil, F.T., Pogoy, A.M., Suson, E.I.B. & Derasin, C.V. (2018). Flipped classroom:
Maximizing face time in teaching and learning. CNU Journal of Higher Education,
12, 30-41. https://jhe.cnu.edu.ph/index.php/ojs3/article/view/21/22
Dsilva, F., Mathew, S. & Joseph, G. (2022). Effectiveness of a self-instructional module on
knowledge and observed practices of nurses with regard to prevention of central
215
line–associated blood stream infection: A before–after intervention study. Journal

of Health Management, 24(2), 233-239.
https://doi.org/10.1177/09720634221087809
Fullington, L. A. & Harrick, M. (2017). Applying Instructional Design Principles to an
Internship. City University of New York Academic Works Publications and
Research. https://academicworks.cuny.edu/bc_pubs/127
Gafoor, A. K. & Farooque, U.T.K (2010). Student teachers’ perspective of the difficulties in
lesson planning and their suggestions for remediation. Journal of Studies in Teacher
Education. 4(1), 19-28. https://www.researchgate.net/profile/Kunnathodi-
Gafoor/publication/262924040_Student_Teachers%27_Perspective_of_the_Diffi
culties_in_Lesson_Planning_and_Their_Suggestions_for_Remediation/links/54f
208b50cf2f9e34eff401b/Student-Teachers-Perspective-of-the-Difficulties-in-
Lesson-Planning-and-Their-Suggestions-for-Remediation.pdf
Genc,B., & Buyukkarci, K. (2013). An Assessment of pre-service language teachers’
practicum observation forms: descriptive observation vs. critical observation.
Educational Research, 2(2), 83-91.
https://doi.org/10.5838/erej.2013.22.01
Goode, S., Willis, R.A., Wolf, J.R. & Harris, A.L. (2007). Enhancing IS education with
flexible teaching and learning. Journal of Information Systems Education, 18(3), 297-
302. https://jise.org/Volume18/n3/JISEv18n3p297.pdf
Graham, C. R. (2013). Emerging practice and research in blended learning. In M. G. Moore
(Ed.), Handbook of distance education (3rd ed., pp. 333–350). Routledge.
Greene-Harper, R. T. (2023). The Future of Instructional Design: Engaging Students
Through Gamified, Personalized, And Flexible Learning with AI And
Partnerships. eLearning Industry. https://elearningindustry.com/future-of-
instructional-design-engaging-students-through-gamified-personalized-flexible-
learning-with-ai-and-partnerships
Guiamalon, T.S. (2022). Internship in times of pandemic: A qualitative phenomenological
study. Res Militaris, 12(6), 1039-1051. https://resmilitaris.net/menu-
script/index.php/resmilitaris/article/view/2604/2171
Harmsen R., Helms-Lorenz, M., Maulana, R., & Veen, K. (2018). The relationship between
beginning teachers’ stress causes, stress responses, teaching behaviour and
attrition. Teachers and Teaching Theory and Practice. 4(6), 626-643.
https://doi.org/10.1080/13540602.2018.1465404
Harn, B. & Meline, M. (2021). Developing critical thinking and reflection in teachers within
teacher preparation. https://doi.org/10.4018/978-1-7998-3022-1.ch014
Harvey, M. W.; Yssel, N.; Bauserman, A. D.; Merbler, J. B. (2010). Preservice teacher
preparation for inclusion: An exploration of higher education Teacher-Training
Institutions. Remedial and Special Education, 31(1), 24–33.
https://doi.org/10.1177/0741932508324397
Huang, R.H., Liu, D.J., Tlili, A., Yang, J.F., Wang, H.H., Jemni, M., & Burgos, D. (2020).
Handbook on Facilitating Flexible Learning During Educational Disruption: The Chinese
Experience in Maintaining Undisrupted Learning in COVID-19 Outbreak. Beijing:
Smart Learning Institute of Beijing Normal University
Hudson, P.B., Nguyen, T.M.H. & Hudson, S. (2008). Challenges for preservice EFL
teachers entering practicum. In Proceedings 2008 Asia TEFL International
Conference: Globalizing Asia: The Role of ELT, Bali , Indonesia.
https://eprints.qut.edu.au/14517/1/14517.pdf
Ilker Etikan, Sulaiman Abubakar Musa, Rukayya Sunusi Alkassim. (2016). Comparison of
convenience sampling and purposive sampling. American Journal of Theoretical and
Applied Statistics, 5(1), 1-4. https://doi.org/ 10.11648/j.ajtas.20160501.11
Iradel, C. M., Cadosales, M. N. Q., & Perez, N. (2021). Lived experience of teaching interns
during the COVID-19 Pandemic. Journal of Research, Policy & Practice of Teachers
216
and Teacher Education, 11(2), 74-87.

https://doi.org/10.37134/jrpptte.vol11.2.6.2021
Ismaeel, D. A., & Al Mulhim, E. N. (2022). E-teaching internships and TPACK during the
covid-19 Crisis: The case of Saudi pre-service teachers. International Journal of
Instruction, 15(4), 147-166. https://doi.org/10.29333/iji.2022.1549a
Jakhanwal, M. S. (2021). Professional and Communication Skills for Teachers, ZBW - Leibniz
Information Centre for Economics, Kiel, Hamburg.
https://www.econstor.eu/bitstream/10419/228530/1/MEETA-JAKHANWAL-
PAPER.pdf
Javier, F.V. (2012). Assessing an Asian university’s organizational effectiveness using the
Malcolm Baldridge model. Asian Journal of Business and Governance, 2, 37-55.
http://dx.doi.org/10.7828/ajobg.v2i1.110
Khan, A., Khan, S., Islam, S., Khan, M. (2017). Communication skills of a teacher and its
role in the development of the students’ academic success. Journal of Education and
Practice, 8(1), 18-21. https://files.eric.ed.gov/fulltext/EJ1131770.pdf
Kilpatrick, A. L., Sengchanh, S., Namvongsa, V., & Gray, A. Z. (2019). Medical education
in Lao People’s Democratic Republic: The challenges students face in accessing
learning resources. The AsiaPacific Scholar, 4(2), 39-47.
https://doi.org/10.29060/TAPS.2019-4-2/OA2034
Kliebard, H.M. (1992). Constructing a History of the American Curriculum. In P.W.
Jackson (Ed.), Handbook of Research on Curriculum. MacMillan.
Koehler, M.J., Mishra, P. & Cian, W. (2013). What is Technological Pedagogical Content
Knowledge (TPACK)? Journal of Education, 193(3), 13-19.
https://doi.org/10.1177/002205741319300303
Kraska, J. & Boyle, C. (2014) Attitudes of pre-school and primary school pre-service
teachers towards inclusive education. Asia-Pacific Journal of Teacher Education,
42(3), 228-246. https://doi.org/10.1080/1359866X.2014.926307
Kuijpers, J., Houtveen, A. and Wubbels, Th. (2010). An integrated professional
development model for effective teaching. Teaching and Teacher Education, 26,
1687-1694. https://doi.org/10.1016/j.tate.2010.06.021
Lachner, A., Fabian, A., Franke, U., Preiß, J., Jacob, L., Führer, C., … Thomas, P. (2021).
Fostering pre-service teachers’ technological pedagogical content knowledge
(TPACK): A quasi-experimental field study. Computers & Education, 174, 104304.
https://doi.org/10.1016/j.compedu.2021.104304
Lampadan, N., Naltan, C.U. & Maidom, R. (2023). Challenges with online teaching
internship and coping strategies: A phenomenological study of lived experience
of student-teachers during the Covid-19 pandemic. In Proceedings 9th
International Scholars’ Conference: Impactful Investigations: Advancing
Research for Social Change, Saraburi, Thailand.
https://repository.unai.edu/id/eprint/344/1/Full%20Paper%20Proceeding%2
09ISC%202022.pdf#page=558
Le Donné, N., Fraser, P., & Bousquet, G. (2016). Teaching Strategies for Instructional
Quality. OECD Education Working Papers.
https://doi.org/10.1787/5jln1hlsr0lr-en
Lee, A. & Griffin, C.C. (2021) Exploring online learning modules for teaching universal
design for learning (UDL): Preservice teachers’ lesson plan development and
implementation. Journal of Education for Teaching, 47(3), 411-425.
https://doi.org/10.1080/02607476.2021.1884494
Lew, M. & Nelson, R. (2016). New teachers’ challenges how culturally responsive
teaching, classroom management, & assessment literacy are intertwined.
Multicultural Education, 23(3-4), 7-13.
217
Lewis, R. & Spencer, D. (1986). What is open learning? Open learning guide 4, London
Council for Education Technology, 9-10.
https://doi.org/10.1080/0268051860010202
Lowrey, K.A., Classen, A. & Sylvest, A. (2019). Exploring ways to support preservice
teachers’ use of UDL in planning and instruction. Journal of Educational Research
and Practice, 9(1), 261-281. https://doi.org/10.5590/JERAP.2019.09.1.19
Lundin, R. (1999). Flexible teaching and learning: Perspectives and practices, UniServe
Science News, 13. https://core.ac.uk/download/pdf/229417091.pdf
Macarandang, M. A. (2009). Evaluation of a proposed set of modules in principles and
method of teaching. E-International Scientific Research Journal, 1, 1-24.
https://doi.org/ 10.12691/education-2-11-20
Mancera, R.V. (2023). Internship program in virtual and modular instruction as perceived
by students. International Journal of Current Science Research and Review, 6(6), 3134-
3139. https://doi.org/10.47191/ijcsrr/V6-i6-06
Merrill, M.D., Drake, L., Lacy, M.J., Pratt, J. & the ID2 Research Group. (1996). Reclaiming
instructional design. Educational Technology, 36(5), 5-7.
https://mdavidmerrill.files.wordpress.com/2019/04/reclaiming.pdf
Montalbo, I., Pogoy, A., Pepito, G. & Cabanilla, A. (2021). Teachers’ experiences in multi-
grade instruction: A meta-synthesis. Multicultural Education, 7(10), 387 - 397.
http://doi.org.10.5281/zenodo.5567812
Norberg, A., Dziuban, C.D., & Moskal, P.D. (2011). A time-based blended learning model.
On the Horizon, 19(3), 207-216. https://doi.org/10.1108/10748121111163913
Oner, D. (2020). A virtual internship for developing technological pedagogical content
knowledge. Australasian Journal of Educational Technology, 36(2), 27–42.
https://doi.org/10.14742/ajet.5192
Pepito, G. M. (2022). Student teaching internship amidst the pandemic: The lived
experiences of early childhood education student teachers. Multicultural
Education, 8(3), 255 - 262. https://doi.org.10.5281/zenodo.6373538
Petancio, J.A.M. & Boonotan, A.M. (2018). The realities of teaching elementary
mathematics: A phenomenological probe. Asia Pacific Journal of Multidisciplinary
Research, 6(1), 59-66. http://www.apjmr.com/wp-
content/uploads/2018/02/APJMR-2017.6.1.07.pdf
Petancio, J.A.M. (2019). Becoming the teacher one needs: An emergent theory on
pedagogic empathy. American Journal of Humanities and Social Sciences Research,
3(6), 122-126. https://www.ajhssr.com/wp-
content/uploads/2019/06/P1936122126.pdf
Plaisent, M., Dayagbil, F., Poroy, A. and Prosper B. (2016). Is Flipped Classroom a Tendency
or a Fad? The Point of View of Future Teachers in the Philippines. Hershey, USA: IGI
Global, 2016 – a chapter of Mobile and Blended Learning innovations for
improved Learning Outcomes. https://doi.org/10.4018/978-1-5225-0783-3.ch104
Posner, G. (2003). Analyzing the Curriculum. McGraw-Hill
Prastomo, Y. and Listyani. (2020). Problems Encountered by Student-Teachers in Two
Junior High Schools in Central Java. Humanizing Language Teaching. Issue 4.
https://www.hltmag.co.uk/aug20/problems-encountered
Quinco-Cadosales, M. N. (2022). Exploring the student teaching interns’ journey in an
online mode of teaching delivery. International Journal of Scientific Research and
Management, 10(5), 2236-2256. https://doi.org.10.18535/ijsrm/v10i5.el01
Sayaf, A.M., Alamri, M.M., Alqahtani, M.A., & Alrahmi, W.M. (2022). Factors influencing
university students’ adoption of digital learning technology in teaching and
learning. Sustainability, 14, 493. https://doi.org/10.3390/su14010493
Schleicher, A. (2020). The impact of COVID – 19 on education: Insights from education at
a glance 2020. OECD 2020. https://www.oecd.org/education/the-impact-
of-covid-19-on-education-insights-education-at-a-glance-2020.pdf
218
Semler, S. (2005). Use Blended Learning to Increase Learner Engagement and Reduce Training
Cost. http://www.learningsim.com/content/lsnews/ blended_learning1.html
Shafqat, A., Khalid, M., Naqvi, S. & Aamir, S. (2016). Teaching practice: Prevalent and
expected performance level of prospective teachers. The Dialogue, 11(1).
https://www.thefreelibrary.com/Teaching+Practice%3a+Prevalent+and+Expec
ted+Performance+Level+of...-a0448247713
Smith, P. & Ragan, T.J. (1999). Instructional Design. John Wiley & Sons.
Spady, W.G. (1994). Outcomes Based Education: Critical Issues and Answers. American
Association of School Administration: Arlington, Virginia.
https://files.eric.ed.gov/fulltext/ED380910.pdf
The Deming Institute (2022). PDSA Cycle. https://deming.org/explore/pdsa/
Torrefranca, E.C. (2017). Development and validation of instructional modules on rational
expressions and variations. The Normal Lights, 11(1), 43-73.
https://po.pnuresearchportal.org/ejournal/index.php/normallights/article/vi
ew/375/235
Ugalingan, G., Edjan, D. & Valdez, P.N. (2021). Online Internship Experiences Among Pre-
service ESL Teachers in the Philippines: Challenges and Opportunities. The
Electronic Journal for English as a Second Language, 25(3), 1-13.
https://doi.org/10.59588/2961-3094.1020
Villonez, G.L., Quia-eo, J.E.G. & Jalotjot, E.H. (2022). Phenomenological study on
completing self-instructional modules in new normal education. Journal of
Education Sakon Nakhon Rajabhat University. https://edu.snru.ac.th/wp-
content/uploads/2022/01/35.1.pdf
Wagner, E. (2018). What is This Thing Called Instructional Design? Foundations of Learning
and Instructional Design Technology .
https://lidtfoundations.pressbooks.com/chapter/what-is-instructional-design/
What is Instructional Design? (n.d.). Home.
https://www.instructionaldesigncentral.com/whatisinstructionaldesign
219

Vol. 22, No. 7, pp. 219-239, July 2023
The Role of Multi-dimensional Curriculum

Design in Improving Higher-Order Thinking
Skills
Anita Yosepha* , Mohammad Ali ,
Dinn Wahyudin and Rusman
Department of curriculum development,
Universitas Pendidikan Indonesia, Bandung, Indonesia
Abstract. The purpose of this study was to investigate the effectiveness of

a multidimensional curriculum in improving the students' high-level
thinking skills in middle and elementary schools. The method used was
the experimental method with a quasi-experimental design by testing a
multidimensional curriculum model integrated with scientific, creative,
and future thinking competencies in the experimental group and
comparing it to the control group. The participants involved in this study
were 500 students divided into two groups, namely the experimental and
control. The experimental group consisted of 250 students with a
composition of elementary school (grades 4-6) and junior high school
students (grades 9-12). The instrument used to measure the students'
thinking skills was a questionnaire that contained three dimensions,
namely scientific thinking, creative thinking, and future thinking. Data
analysis was then carried out. First, factor analysis was used for the item
scale analysis for each competency, the Pearson correlation to investigate
the relationships between the competencies, paired sample tests to
investigate the pre-test and post-test differences, and repeated tests to
determine the results based on all variables. The results show that a
multidimensional curriculum is proven to be effective at increasing the
high-level thinking skills of elementary and middle school students. The
improvement in higher-order thinking skills at the junior high school
level is more significant than that of the elementary school students
because the development of high-level thinking skills are by then
sufficiently trained to think scientifically, creatively, and for the future
compared to elementary school students. The dimensions of ability that
have the most significant improvement are the ability to think about the
future and the dimensions of the ability to think creatively. The strongest
correlation between higher-order thinking skills and curriculum
dimension elements was found in the correlation between creative
thinking (identifying and solving problems). This research has the
implication that the multidimensional curriculum in its implementation
*
Corresponding author: Anita Yosepha; anitahandjaya19@upi.edu
©Authors
220
must be accompanied by the use of learning methods that are able to

encourage students to think scientifically, creatively, and be able to
predict the future.
Keywords: higher order thinking skills; multidimensional curriculum;

thinking skills dimensions; innovative strategies and methods
1. Introduction
Today's teachers need to be better able to support students to improve their skills
both in general and in particular to meet the demands of the world (Hadianto et
al., 2021a; Saido et al., 2018). Schools must prepare their students to answer future
challenges. What kind of education is appropriate and the best to answer the
challenges of the future? This question is of concern to stakeholders both in
government and the school itself. The answer lies in a curriculum design that must
be able to equip students to face challenges in the future. There are several abilities
that must be possessed by students in the 21st century, namely the ability to think
critically and solve problems, the ability to communicate, collaborate, computing
and information technology skills, as well as career planning, cross-cultural,
creative and innovative (Bovill & Woolmer, 2019; Green, 2018). The integration of
critical thinking skills and other abilities is still not enough to create a curriculum
that is able to provide excellent abilities. Education must be appropriate to the
context both locally and globally. In addition, education must be able to meet
specific characteristics according to its culture, demands, and history as well as
the demands of the future, such as digital capabilities (Davis et al., 2023; Fensham,
2022).
Improving the students' thinking skills can be done through various methods and
strategies. However, it is still rare for curriculum models to be designed that are
intended to improve higher-order thinking skills that integrate future thinking
skills, individual views, and concepts, as well as global understanding and the
ability to predict problems in the future (Manassero-Mas & Vázquez-Alonso,
2022; Whalen & Paez, 2021). This ability is needed by students today. The ability
to plan for the future must be given to students. This ability usually depends on
the age level of the student. The ability to plan for the future using multiple
perspectives requires the ability to analyse problems and solve them. The
multidimensional curriculum is used by researchers to better determine its role in
improving higher-order thinking skills by focusing on three aspects, namely
scientific thinking (scientific questions), creative thinking (creative problem
solving), and future thinking (individual and time perspectives) (Hadianto et al.,
2022; Sung et al., 2019). The difference between this study and the previous
research is intervention design in the form of a multidimensional curriculum and
the competencies that it develops. Most of the previous research used
interventions that were teaching methods and focused only on higher-order
thinking skills (Lu et al., 2021; Miedijensky et al., 2021). This is in contrast to this
study where the intervention used was a multidimensional curriculum design
and the competencies developed not only focused on higher-order thinking skills
but all abilities that belong to higher-order thinking, namely scientific thinking,
creative thinking, and predicting the future.
221
Through this research, the researchers tried to investigate the contribution of a

multidimensional curriculum design to the students' higher-order thinking skills,
the dimensions of said higher-order thinking skills, and also to see whether there
is a connection between the aspects of school level and gender. This research
provides knowledge on how to improve the level of thinking skills and future
thinking skills through curriculum design. The researchers investigated the
results of the intervention in an experimental group that learned certain material
using three types of thought processes (scientific, creative, and future thinking) in
a multidimensional curriculum. Next, they compared it with the learning
outcomes of students who studied using conventional methods who were in the
control group. In addition, this study investigated the results of the intervention
of learning programs with a multidimensional curriculum design at different
school levels and their relationship with the student’s gender. The formulation of
the research problem is as follows: 1) What is the effect of the multidimensional
curriculum program intervention on the students' higher-order thinking skills?
and 2) Does the school level and gender of the students have a relationship with
higher-order thinking skills when receiving interventions from a
multidimensional curriculum?
2.1 Curriculum development
The basis for developing this multidimensional curriculum is a constructivist
approach which is believed to be able to improve higher-order thinking skills and
future thinking skills. The multidimensional curriculum framework is built on a
curriculum model made for gifted children. The multidimensional curriculum
model is based on an integrated curriculum and parallel curricula and programs
to provide future problem-solving skills (Guo et al., 2022; Saido et al., 2018). The
integrated curriculum focuses on three aspects, namely content dimensions, issue
dimensions, and process and product dimensions. The parallel curriculum
focuses on the interdisciplinary curriculum, the involvement of personality
aspects, and expert practicum (Cross, 2021; Oberauer et al., 2022). The problem-
solving program contains creative problem-solving competencies and abilities
that are needed in the future so then students can adapt to the demands of the
world. It is not enough for students at school to just learn about the past but they
must also be equipped with the ability to understand and predict possible future
choices about a problem (Guo et al., 2022; McConnery et al., 2021). Students must
be able to actively imagine surviving in an era that is experiencing very rapid
changes. If students are equipped with the ability to predict the future, they will
use their imagination to see problems within a modern paradigm, as well as to
find, analyse, explore and produce new views of a problem that are appropriate
to the present (Fensham, 2022). In order for students to have the ability to think in
the future, they must be trained in historical views, and equipped with short-term
and long-term planning skills. Students should be encouraged to develop their
competencies in three aspects, namely product development, concepts, and views.
When designing a multidimensional curriculum, there are three additional

dimensions suggested by researchers, namely personal, global, and the time
perspective (Carroll & Harris, 2020; Heron & Palfreyman, 2021). Personal
perspective reflects personal identity. This dimension encourages personal
222
student involvement, increases self-awareness, fosters interest, and encourages

intrinsic motivation. Some questions that reveal the personal dimension include
1) How do you feel about the problem? 2) How did you get involved in the
process? and 3) How do you improve your competence so then you can contribute
to society. The personal perspective focuses on developing the students'
awareness in order for them to be involved in the learning process, to participate
in social life, and to be active when predicting future problems (Green, 2022;
Hadianto et al., 2021b). The personal perspective aspect in the curriculum
dimension is useful for training students to use procedures, and to compare and
contrast, when building a global perspective and perspective based on future
time. The personal perspective is used as a basis for improving future thinking
skills. Next, there is the global perspective dimension. The global perspective
views the world as integrated. Events that occur in one country will affect other
countries in various aspects. Students need to be given the ability to identify
problems or concepts from micro and macro views, to analyse differences and
similarities as well as cultural and geographical issues, and other global trends
that will definitely affect the conditions of countries worldwide (Barfod &
Bentsen, 2018; Carroll & Harris, 2020).
Some of the questions used to reveal the students' global perspectives include 1)
What issues are currently emerging in developed countries? and 2) What are the
issues that are currently developing globally in each country? The students'
awareness of this global perspective will be useful as part of creating a global
perspective when the students grow up. The dimension of the time perspective
contains the competence to understand developments and changes over time. The
time perspective provides competency opportunities to understand problems, to
optimise processes and products, and to understand the aspects of the past,
present, and future. Questions that reveal the perspective of time include where
did it come from, what was it like at first, and what are the current conditions like?
What's the future direction? These questions can improve the students' ability to
predict and respond to problems or demands in the future using their past
knowledge. Through a multi-dimensional curriculum, students are given the
tools to analyse and predict various possibilities for the future.
2.2 Development of higher-order thinking skills

The learning methods and strategies used in the implementation of the
multidimensional curriculum promote higher-order thinking skills. The levels of
cognitive ability drawn from Bloom's taxonomy have been revised into
identification, memory, understanding, implementation, analysis, evaluation,
and creation. The ability to think at a higher level cognitively is involved in the
process of constructing new knowledge (Bovill & Woolmer, 2019; Gore et al.,
2018). New knowledge is constructed through processes that promote critical
thinking, creative, and problem-solving skills. There are several types of thinking
framework that can be used when creating new knowledge including pedagogy
or instructional design, productive or philosophical approaches, cognitive
structures and development, and comprehensive frameworks (Lu et al., 2021;
Miedijensky et al., 2021).
223
One approach that can be used to improve thinking skills is the infusion approach.
This approach uses pedagogy and encourages the ability to think directly so then
the students are able to identify patterns, find similarities and differences, guess,
give rational reasons, provide different views, solve problems, make decisions,
and evaluate the results of their work (Lin & Chuang, 2018). Classical scientific
inquiry abilities such as formulating problems, making hypotheses, making
experiments, and making conclusions are included in higher-order thinking skills.
Cognitive activities that can train higher-order thinking skills include giving
arguments, comparing, solving problems, responding to different views, making
decisions, and finding implicit assumptions. From previous studies, a relationship
was found between knowledge and certain thoughts. This finding confirms that
the development of thinking in a particular domain is largely determined by the
level of knowledge of said domain. In addition, these findings also confirm that
individual creativity is highly dependent on the domain (Elfeky, 2019; Zhang &
Chan, 2020). Individual metacognitive abilities use the ability to think critically
and creatively simultaneously. Teaching that encourages the students' critical
thinking is teaching that contains the teacher’s instructions or open-ended teacher
questions, as well as providing space for the students to think critically and
creatively so then this higher-order thinking activity becomes a routine.
2.3 Increasing higher-order thinking skills through a multidimensional

curriculum
Teaching thinking skills is a complex activity because thinking processes can
involve inquiry, critical thinking, and creative thinking. Teaching can be done by
giving examples of thinking or practicing thinking skills including making
concepts, understanding concepts, drawing conclusions, and solving problems
(Baghaei et al., 2020; Carroll & Harris, 2020). Teaching thinking processes focuses
on three dimensions, namely training scientific thinking skills (inquiry),
practicing creative thinking skills (identification and problem solving), and
training the ability to think about the future based on personal views (looking at
a problem from the perspectives of the past, present, and future). The ability to
think about the future is one of the new competencies that must be taught in order
to equip students to be ready to face the future and prepare their competencies in
order to face future competition (Green, 2022; Nurwanto & Cusack, 2018). The
ability to think scientifically is the ability to study scientific concepts, and to
explore and test them through scientific experiments. Some of the abilities
involved in the scientific thinking process are identifying problems, making
hypotheses, making experiments, analysing results, and making conclusions.
Creative thinking teaches students to identify and solve problems. The ability to
think creatively is a type of critical thinking skill used in investigating problems.
Critical thinking skills are carried out based on objectivity criteria, problem-
solving strategies, reflection, and practicing decision-making. Parallel, lateral,
divergent, and convergent thinking are the cornerstones of the process of creative
thinking and problem-solving (Manassero-Mas & Vázquez-Alonso, 2022; Saido et
al., 2018). The ability to solve problems includes the ability to identify problems,
determine problems, formulate solutions, evaluate criteria, choose the criteria for
solutions, and plan actions. Creativity has various meanings, so the ability to think
creatively is represented by the ability to solve problems. The ability to think
224
about the future is a thinking ability that was often used by ancient humans to
predict future conditions. With this future thinking ability, individuals can make
judgments for the future, assess the significance of an intervention, provide
predictions based on existing information, and evaluate their abilities (Hwang et
al., 2019; Sung et al., 2019). The ability to predict oneself in the future according to
one's abilities is a very important aspect of future thinking skills. The ability to
think about the future and make plans appears across all ages of student. This
ability gradually develops until the students are 25 years old. By the time the
students are grown up, their ability to plan for the future reaches an average of
three years into the future (Baghaei et al., 2020; Carroll & Harris, 2020). The
utilisation of the recorded past and a global perspective can increase the chances
of providing more accurate predictions. Awareness of the ability to predict the
future can be increased using several strategies, namely predictions from time to
time, designing scenarios for selected events, imagining the future through a
global perspective, and providing several solutions to events that occur (Hadianto
et al., 2021a; Whalen & Paez, 2021). The four ranges of awareness that are often
used to increase the awareness of future predictive abilities are event continuity,
correlated events, event duration, and the acceleration or deceleration of events.
The five time spans of awareness include the immediate range of 5 years, the
short-term range of 6-10 years, the median range of 11-31 years, the long-term
range of 31-51 years, and the very long range of 52-100 years.
2.4 Multidimensional curriculum design

The multidimensional curriculum design must consider various aspects including
content, thinking strategies, evaluation tools, products, and reflection
(McConnery et al., 2021). These aspects focus on three types of thinking skills,
namely the ability to think scientifically, to think creatively, and to think for the
future. Aspects that need to be considered in a multidimensional curriculum
design are how the content of the curriculum must be interdisciplinary and that
the concepts must be understood comprehensively. In addition, the thinking
process in the curriculum must focus on three types, namely the ability to think
scientifically through investigation, the ability to think creatively through
identifying and solving problems, and the ability to think about the future which
can be used to build new concepts or knowledge (Cross, 2021).
The scientific thinking strategy includes several stages, namely formulating the
problem, obtaining information, presenting the results, and drawing conclusions.
Creative thinking includes determining the problem, providing solutions, making
criteria for selecting solutions, and planning actions from various perspectives.
Thinking about the future includes several stages, namely identifying the
components, analysing and classifying, comparing, identifying relationships and
processes, organising, and making predictions (Green, 2022). The instrument used
for improving scientific thinking skills was the TASC (Thinking Actively in a
Social Context) Thinking Wheel. The way that this instrument works is to organise
individual thoughts by forming a wheel and using inquiry to investigate the
problems or concepts. Creative thinking skills are developed by using complex
approaches to several types of problem (types 4, 5 and 6), practicing the stages of
solving problems, and exploring the perspectives using some of the chosen topics.
Future thinking skills can use problem illustrations with mind maps to write
225
future plans. Tools for evaluating the products produced by the students must be
discussed with the students and mutually agreed upon. This assessment must
include formative and summative evaluations and the assessment must also be
carried out using various methods such as self-assessment, and the views of
friends and other teachers. The final product of the scientific thinking skills
training process should be a small research project.
The final product in the process of improving creative thinking skills can be the
formulation of alternative solutions and future action plans. The development of
future thinking skills can be oriented towards several products including concept
maps, model development, timescales, future plans, and future model designs.
The products selected for training future thinking skills must vary including
written, spoken, and creative products to acquire new knowledge and new
perspectives (Sung et al., 2019; Whalen & Paez, 2021). In addition, there must be
time or the opportunity for the students to reflect on the processes they have gone
through in relation to metacognition and personal reflection. This can be done by
asking specific questions or questions about the different strategies. The design of
the multidimensional curriculum components is presented in Table 1.
Table 1. Components of the multidimensional curriculum

Content Appropriate interdisciplinary content or product orientation
Thought Think scientific Creative thinking Future thinking
process (inquiry) (identify and solve (building and analysing
problems) concepts or materials)
Strategy Formulation of the Define the problem Defining and identifying
thinking problem Provide solutions components
hypothesis Create criteria Classify and analyse
Gather information Identify perspective Compare
Organising Choose the best Identify relationships
information solution Identify processes
Presenting graphs Respond to different Organising sections
perspectives Prediction development
Tool Thinking wheel model problem type Mind map
Inquiry stages Problem solving Future scenarios
stages
Thinking topics
Assessment Create criteria for product assessment with the students
instrument Use formative and summative assessments
Self-assessment, peers, and teachers
Product Mini research Problem solution Concept maps
Action plan Model development
Multicategory Deadline
Written, spoken, and creative Future models
Future scenarios
Reflection Metacognition/personal reflection on the thought processes
General questions on the learning process
Question-based thinking strategies
The teaching strategies that can be used in a multidimensional curriculum design

include project-based learning, problem-based learning, and blended learning
that combines technology, language skills, and relevant issues (Cheng et al., 2020;
226
Diamond et al., 2020). The formulation of the subject units in a multidimensional

curriculum uses several stages, namely a) introducing the content to students
where the mental representation of students must be formed by the teacher
through involving the content, process and product components, integrating
them with personal, global and time perspectives; b) the teacher must provide
alternative choices of content, taking into account suggestions from the students;
and c) the learning procedures must be introduced to students and must involve
the students actively in deciding something in the learning process (Oberauer et
al., 2022; van Leent & Spina, 2022). Teachers in this curriculum are required to
have flexible, open, and to use democratic criteria during the learning process.
Planning by actively involving the students can increase student motivation and
interest. The design of the learning process tools in the multidimensional
curriculum uses a constructivist approach and integrates perspectives with
innovative learning instruments and strategies. This design process can improve
the ability to think scientifically, creatively, and for the future.
3. Methodology
3.1 Participants
This study used an experimental method with a quasi-experimental design to
investigate the students' thinking skills as a result of the intervention of a
multidimensional curriculum design. There were 500 participants in this study
divided into two groups, namely the experimental and control where the groups
had the same number of participants who had the same socioeconomic status. The
experimental group consisted of 250 students with a composition consisting of
elementary (grades 4-6) and middle school (grades 9-12). The control group
consisted of 250 students with an equal number of elementary and junior high
school students (grades 9-12). The experimental group received the learning
program intervention using a multidimensional curriculum, while the control
group studied using conventional methods. An explanation of the demographics
of the research participants is presented in Table 2.
Table 2. Demographics of the research participants

Experimental Control Total
Grade Level Elementary 125 125 250
Secondary 130 120 250
Total 255 245 500
Gender Male 120 120 240
Female 135 125 260
Total 255 245 500
The selection of the student sample was carried out proportionally according to
school level, class and gender. This research was conducted in 8 schools and 15
classes from elementary school to junior high school in West Java, Indonesia. The
number of students in one class averaged 30-40. The proportion of gender was 260
female students and 240 male students. The sample in this study represents the
culture in the West Java region, namely the Sundanese culture. In one school,
about 70% of the students were born in the West Java region, where the Sundanese
culture is predominant. In general, the participants involved in this research had
227
academic abilities that were evenly distributed, which means that it can be
interpreted that their thinking abilities are not much different.
3.2 Procedure
This research went through several procedures, including the development of an
intervention program designed with a multidimensional curriculum in mind. The
fields of study that are the focus of the intervention are communication skills,
economic capacity and globalisation, culture, understanding of the internal
organs, and mathematics. All of these areas are studied at school. Each unit
contains 10-15 lessons conducted at secondary school level. All fields of study
contain scientific, creative, and future thinking processes. Future thinking
competence focuses on the personal and temporal views of a concept. First, all
schools integrate these three types of thinking process in their implementation
process, namely scientific thinking, creative thinking, and future thinking,
specifically thinking scientifically using inquiry, thinking creatively using
problem-solving steps and thinking themes, and thinking in the future using
mind maps and scenarios or future planning. The control class was taught using
the direct or inquiry methods but not too often.
Next, the second stage was piloted. The assessment instruments and indicators
were made based on the scale and questionnaire items. The indicators were made
based on agreement between the assessor and the teacher who checked the
student's answers to the open questionnaire. The scale used was a scale of 1-5 in
each category. The agreement used was 90%, so all teachers assessed the results
of the questionnaire using these indicators. The third stage was testing the higher-
order thinking skills before and after receiving the intervention consisting of a
learning program for a multidimensional curriculum. In the pretest session, the
students' thinking skills were measured by them answering an open
questionnaire individually which took about 30 minutes for the pretest and 45
minutes for the posttest. At the end of the study, the assessment carried out the
same procedure again to see the consistency of the higher-order thinking skills.

The instrument used to assess higher-order thinking skills was a higher-order
thinking questionnaire. The questionnaire consisted of several parts. The first part
contained general topics that assessed the students' scientific thinking abilities.
The second part assessed the ability to think creatively which provided
information on the possibility of life in outer space and its relationship to earth.
The third section assessed future thinking skills covering general topics and
encouraged the students to provide their personal and timely perspectives. The
questionnaire was used in the experimental and control groups before and after
the learning program intervention. Instrument validation was carried out by two
experts according to their fields using content validity, factor analysis, and
varimax rotation. The reliability of the instrument was assessed using the
student's responses to the questionnaire with a value of 0.96. The factor analysis
assessment obtained three scales according to the focus of thinking competence
(scientific, creative, and future). The following are the results of the assessment
for each competency. Scientific thinking competence (investigation) consisting of
7 items had a Cronbach’s alpha value = 0.90, and a total score of 35. Several of the
228
questionnaire questions assessed scientific thinking skills, namely ‘What are the
main findings presented in the figure?’ This question tests classification and
analytical skills, for example.
‘What can you conclude from this explanation?’ This question tests the ability to
draw conclusions. Creative thinking skills test the ability to identify and solve
problems, which contained 7 items, with a Cronbach’s alpha value = 0.93 and a
total score of 35. Some of the questions used in this section of the questionnaire
were "Identify the problem that you found in the data? (problem identification
ability) and ‘Create alternative solutions of at least five alternative solutions to
solve the problem you specify!’ (ability to provide solutions). The ability to think
about the future was divided into two parts, namely personal perspective and
time perspective. Personal perspective consisted of 5 items, with a Cronbach’s
alpha value of 0.86 and a total score of 25. An example of a question in this section
is ‘Create a paragraph that contains your personal views on the subject matter
presented’. This task can be positioned in the first person. The time perspective
consisted of 5 items with a Cronbach's alpha value = 0.83 and a total score of 25.
An example of a question in this section is ‘In your opinion, how is the
development of the issue, problem/subject?’ (ability to describe processes). The
maximum total score possible in the higher-level thinking skills questionnaire is
100. The results of the factor analysis on the questionnaire are presented in Table
3.
Table 3. Results of the factor analysis.

Scientific Future
Creative thinking
thinking Thinking
Dimensions Inquiry Problem- Personal Time
solving perspective Perspective
Alpha (total 0.95) .90 .93 .86 + .83 = .92

Formulate .80
hypotheses
Information .79
organisation and
graphical
representation
Integration .71
Classification and .69
analysis
Define conclusions .67
Make a problem .66
statement
Define the problem .76
Create criteria .72
Provide solutions .71
Identify the problem .70
Provide the best .69
solution
Create an action plan .52
Make a different .78
view
229
Related to problems .70

Have leadership .60
Write in first person .58
Understand the .75
process
Understand the .72
relationship between
processes
Describes three .50
different times
Appropriate content .43
Variance (in 25.35 19.30 13.80 9.43
Percentage)
Average Score 17.82/41 21.90/32 13.10/23 12.50/23
(SD) (7.89) (7.10) (5.50) (5.31)
3.4 Data analysis

From the results of the trial analysis, the factor analysis of the 500 student
responses through the questionnaire resulted in one item being deleted, one
repeated, and two integrated. Regarding the questionnaire instrument, it was
found that the reliability value of Cronbach's alpha was 0.95 which explained the
three dimensions of higher-order thinking skills. The following is an explanation
of the results of the analysis of the three dimensions of higher-order thinking
competence: a) scientific thinking contained 7 items with a Cronbach's alpha =
0.86, b) creative thinking competence contained 7 items with a Cronbach's alpha
= 0.92, and c) future thinking competence contained 8 items with a Cronbach's
alpha value = 0.90. The students' thinking ability was calculated using a scale and
the items from the factor analysis with a score range of 0 - 5 points. The maximum
score of all questionnaire items was 100. The indicators were developed based on
the readiness of the assessors and they were processed using the percentage of
approved statements multiplied by 100. The reliability of the random sampling
had a reliability value of 90%. Pearson's correlation analysis was used to
investigate the relationship between the dimensions of thinking process
competence. The higher-order thinking ability scores were calculated separately
from the general score and the score for each dimension. The difference between
the two groups at the pretest and posttest was calculated using the paired sample
test. The variable differences by group, school level, gender, pretest-posttest, and
the separate dimension scores were calculated using repeated measures analysis.
3.5 Research ethics

All participants involved filled out the consent form in order to take part in the
research, thus all participants in this study were involved voluntarily without any
coercion. In addition, all participants were anonymous. The research data on the
students' higher-order thinking skills in this study was not used for academic
purposes in the respective schools but it was used for study purposes.
230
4. Results
4.1 Higher-order thinking skills in general
To answer the first formulation of the problem, the researcher presents the data
on the effect of a learning program intervention using a multidimensional
curriculum to increase the use of higher-order thinking skills. Based on the results
of the analysis, the intervention program was found to have a significant impact
on the high-level thinking skills of the experimental group students. The
differences between the two groups were apparent in the pretest and posttest
phases. In the pretest phase, in general, the two groups (experimental and control)
showed relatively the same thinking skills and did not differ much. Based on the
results of the analysis, the mean and standard deviation values were obtained.
The experimental group was 22.20 [16.70] and 20.15 [14.45]). Looking at the pretest
and posttest phases, significant differences in scores were found in each group.
The intervention group obtained the pretest and posttest phase values of (22.20
[16.70] and 63.10 [20.40], MD = -40.80, t = -30.90, p <.001), while the control group
obtained mean and standard deviation values in the pretest and posttest of (19.15
[14.45] and 30.80 [16.30], MD = -4.20, t = -6.60, p < .001). The main effect of the
multidimensional curriculum intervention was found in the experimental group
as having a value of (F[1,195] = 640.50, p <.001, Effect Size = .856). This value
indicates that the experimental group experienced a better improvement than the
control group in the posttest phases. Based on the results of the multivariate
analysis, a significant difference was found between the two groups with scores
of (F[1,462] = 857.70, p < .001, ES = .956). This value indicates that the difference
in the increase in higher-order thinking skills in the experimental group and the
control group is around 45%. The data on the comparison of the scores for the
higher-order thinking skills of the experimental and control groups in each phase
is presented in Table 4. For clarity, multivariate test analysis was carried out based
on gender and school level.
Table 4. Higher-order thinking skills of the two groups in the pretest and posttest
phases.
Pre-test Post-test MD T F(df = Size
mean mean 1,462) effect
(SD) (SD)
Intervention 22.20 63.10 -40.80 -30.90 857.70 0.956

group (16.70) (20.40)
(n = 199)
Control 19.15 30.80 -4.20 -6.60
Group (n = (14.45) (16.30)
195)
p < .001.
To answer the second problem formulation, multivariate test analysis was carried
out based on the time and sex variables to determine the differences between the
two groups. Based on the results of the analysis, the main effect value was
obtained based on the gender variable, the F value and the effect size of male
students, namely F[1,190] = 310.85, p <.001, ES = .840). This value indicates that
the male students in the experimental group had a different average score in each
231
phase. Based on the analysis results, the mean score for the male sex in the posttest
phase is greater than in the pretest phase (22.46 [18.89] < 60.90 [21.35]) and the
average score of the control group in the posttest phase is greater than the pretest
but not too significant for the control group (20.56 [13.50] < 24.10 [15.95]). The
main effect value was found for female students (F[1,198] = 489.89, p < .001, ES =
.890). In addition, it was found that the intervention group of female students was
superior to the control group sequentially in each phase with a score (22.10 [15.78]
< 65.30 [16.50) and the control group (17.60 [13.53] < 21.52 [16.60). Another finding
was that there was a significant difference in the experimental group between the
thinking abilities of the male and female students in the posttest phase with a
score of (60.90 [21.35] < 65.30 [15.78], MD = -5.40, F [1.198] = 6.15, p < 0.05, ES =
0.040). From the results of the analysis, it can be concluded that female students
have more high-level thinking skills in the experimental group. This is different
from the control group which shows the opposite. The results of the data analysis
for the experimental and control groups based on phase and sex are presented in
Table 5.
Table 5. Higher-order thinking skills of the two groups based on student phase and
gender.
Intervention group Control group
Mean (SD) Mean (SD)
Pre-test Post- MD Pre-test Post- MD F (df = Size
test test 3) effect
Boys (n = 22.46 60.90 -37.67** 20.56 24.10 -5.40** 310.85** 0.840
250) (18.89) (21.35) (13.50) (15.95)
Girls (n = 22.10 65.30 42.22** 17.60 21.52 -4.70** 489.89** 0.890
250) (15.78) (15.78) (13.50) (16.60)
__p < .001.
Still answering the second problem formulation, an analysis of higher thinking

skills based on school level was carried out. Based on the results of the analysis, a
significant difference was found. The data on the average score (standard
deviation), F value, and effect size are presented as follows. Differences were
found in the experimental and control groups based on elementary and high
school levels. From the results of the analysis, it was found that the curriculum
had a significant main effect at the elementary school level with a score (F[3,227]
= 478.10, p <.001, SE = 0.856). The acquisition of the average score and mean
difference in the experimental group's average value in the posttest phase
experienced a significant increase with a value of (20.31 [17.90] < 61.70 [20.45], MD
= -42.40, p < 0.001), different from the control group. The scores in the control
group also showed an increase in the posttest phase but not significantly (16.41
[12.50] < 20.42 [15.85], MD = -4.87, p < 0.001). At the secondary school level, the
multidimensional curriculum provided a significant effect size with a score
(F[3,230] = 315.25, p <.001, SE = 0.820). In addition, significant mean scores and
mean differences were also found in the experimental group in the pretest and
posttest phases (32.90 [6.87] < 64.71 [6.20], MD = -35.81, p <.001). The control
group also experienced an increase but not as significantly as the experimental
group (34.21 [6.4] < 40.21 [6.20], MD = -5.12, p <.001). From the results of the
analysis, the higher-order thinking skills of the high school students experienced
232
a more significant increase than that of elementary school students in both the
experimental and control groups. In addition, it was also found that the main
effect was seen in the elementary school level with grades (MD = -42.40, p <.001)
and secondary school grades with grades (MD = -4.12, p <.001), as well as a
difference of 4 points between the average scores posttest and pretest (61.70
[20.41] < 65.72 [6.20]). The experimental group of elementary school students in
the pretest phase had lower initial abilities than those from the middle school but
the increase in the posttest phase was greater. In the posttest phase, the
elementary school students had almost the same thinking skills as the high school
students. The results of the analysis of the higher-order thinking skills based on
school level are presented in Table 6.
Table 6. Results of the analysis of the higher-order thinking skills based on school
level.
Pre- Post- MD Pre- Post- MD F (df = Size
test test test test 3) effect
Elementary 20.31 61.70 - 16.41 20.42 - 478.10 0.856
**
(n = 231) (17.90) (20.45) 42.40** (12.50) (15.85) 4.87**

Secondary 32.90 64.71 -35.80 34.21 40.21 - 315.25** 0.820
(n = 163) (6.87) (6.20) ** (6.4) (6.20) 5.12**
**p < .001.
Table 7. The relationship between the dimensions of higher-order thinking skills in

the dimension curriculum.
Scientific Creative thinking Future thinking
thinking
Scientific thinking _______ .680** .546**
Creative thinking _______ _______ .782**
**p < .001.
Based on the results of the correlation test in Table 7, a significant correlation was
found between the three dimensions of thinking competence found in the
multidimensional curriculum design. The strongest correlation was found in the
correlation between creative thinking (identifying and solving problems) and
future thinking (personal and time perspectives) with a value of 0.782. This was
followed by the second correlation, namely the relationship between scientific
thinking (inquiry) and creative thinking (identifying and solving problems) with
a value of 0.680, and finally the relationship between scientific thinking (inquiry)
and future thinking (personal and time perspective) with a value of 0.546. It can
be concluded that the Pearson correlation between the dimensions of thinking
competence in the multidimensional curriculum had a significant main effect.
From this data, it was interpreted that the ability to think creatively has a stronger
relationship with future thinking than scientific thinking.
233
4.2 Analysis of higher-order thinking skills in each dimension

The researcher conducted a more detailed analysis of each dimension of the
multidimensional curriculum to find out more about the role of each dimension
in relation to the students' higher-order thinking skills. Based on the results of the
analysis, it was found that the main effect of each dimension was as follows. The
main effect on scientific thinking ability was (F[3,412] = 212.785, p < .001, SE =
0.903), creative thinking ability (F[3,412] = 290.135, p < .001, SE = 0.893), and future
thinking ability in general (F[3.412] = 189.412, p < .001, SE = 0.852), in addition to
when it was based on personal perspective (F[3.412] = 120.754, p <.001, SE = 0.742),
and time perspective (F[ 3.412] = 189.425, p<.001, SE = 0.825). Each of the
dimensions was analysed based on the group variables, school level, and gender,
which will be presented in Table 8.
Table 8. Higher-level thinking skills for each dimension of thinking

competence
Pre- Post- MD Pre- Post- MD F (df = Size
test test test test 3) effect
Scientific Inquiry 9.41 17.82 -9.25** 8.65 10.40 - 180.34** 0.581
thinking (30p.) (6.70) (7.89) (5.80) (6.50) 1.70**
Creative Problem- 7.84 26.89 - 6.70 5.89 -0.26 470.20** 0.784
thinking Solving (6.21) (8.03) 17.12** (5.20) (5.24)
(30p.)
Future Personal 4.92 13.10 -8.25** 3.80 5.32 - 180.80** 0.668

thinking P. (20p.) (5.78) (5.52) (4.90) (5.50) 1.65**
Time P. 4.25 13.60 - 2.89 3.56 -0.81 270.70** 0.758
(20p.) (5.24) (6.32) 10.42** (3.80) (5.10) **
Personal 7.89 25.50 - 4.57 6.65 - 314.61** 0.782

& Time (9.90) (8.24) 17.23** (6.18) (7.45) 2.10**
P. (40p.)
Based on the results of the analysis in Table 8, a significant difference was found
in each dimension of the ability to think highly in the experimental group. The
increase in higher-order thinking skills in the pretest and posttest experimental
groups experienced an increase of 9-15%. Higher-order thinking skills were very
low but experienced a significant increase of around 65% in the posttest phase in
all dimensions. The dimensions of thinking competence that experienced an
increase in the experimental group were mass thinking competence with a value
of (7.89 [9.90] < 25.50 [8.24], MD = -17.23, p <.001) and creative thinking
competence (6.91 [5.24] < 25.90 [7.05], MD = -16.04, p <.001). The lowest thinking
ability score was for the future thinking ability score in the time perspective in the
pretest with a score of (3.20 [4.25]) but this increased in the posttest phase by about
9 points (12.56 [5.21]). The value of future thinking skills from the time perspective
in the posttest phase was in the middle category. Furthermore, the ability to think
scientifically was found to have the highest value in the pretest phase and showed
a significant increase. The increase occurred more in the experimental group than
in the control group with the following values: (9.41[6.70] < 17.82 [7.89], MD = -
234
9.25, p <.001 and 8.65 [5.80] < 10.40 [6.50], MD = -2.70, p < 001). The increase in
higher order thinking skills in the control group was very small. There was also a
significant difference in higher order thinking skills based on school level, namely
creative thinking skills at the elementary school level had a value (F[3,189] = 11.89,
p <.001, SE = 0.063), whereas at the secondary school level, it was (6.07 [3.54] <
21.70 [3.70] and 9.41 [4.70] < 21.30 [4.89] However, no significant differences were
found in each dimension based on the gender variable.
5. Discussion
This study investigated the role of a multidimensional curriculum design in
increasing higher-order thinking skills among both elementary and secondary
school students. To investigate the role of the proposed multidimensional
curriculum design, a learning intervention program was created for both groups
of experimental students, while the learning intervention for the control group
used conventional methods. In the integrated experimental group intervention,
three dimensions of thinking competence were integrated with the
multidimensional curriculum, namely scientific, creative, and future thinking
competencies. The research findings show that this multidimensional curriculum
design can improve the high-level thinking skills of elementary and secondary
school students. This finding is consistent with the theory that thinking
competence will be more effective if it is taught in an integrated manner in the
curriculum design as well as the teaching process (Hwang et al., 2019; McConnery
et al., 2021). Students will indirectly be trained in thinking competence through
the processes and products that the curriculum demands. So, the use of an
integrated approach, explicit instruments, and the implicit teaching of the
thinking competence in the curriculum have been proven to be effective at
increasing the students' higher-order thinking skills. The multidimensional
curriculum was designed by introducing thinking competency evaluation
instruments, selecting appropriate content, promoting collaboration, integrating
it with technology, and providing opportunities for the students to provide
different perspectives. The design of the multidimensional curriculum component
was able to improve the competence of the students' higher-order thinking skills.
This research is in accordance with the previous research which has tested a
thinking ability improvement program that was proven to be effective at
improving the academic abilities of students in their schools. This was in contrast
to the students who did not receive the teaching of thinking competency
improvement programs with a relatively poor academic achievement (Barfod &
Bentsen, 2018; Saido et al., 2018).
Teaching thinking skills is not enough if you only use teaching instruments as it
needs to be accompanied by other components. This was done in this study. A
multidimensional curriculum designed to improve higher-order thinking skills
using various components that are integrated into the curriculum can include
thinking topics, thought wheels, problem solving stages, instructions for writing
future plans, and other relevant units (Elfeky, 2019; Lin & Chuang, 2018). A
multidimensional curriculum that integrates the three dimensions of thinking
(scientific, creative, and future thinking) was able to effectively improve higher-
order thinking competencies in the experimental group at both the elementary
and secondary school levels. The main effect size given by the multidimensional
235
curriculum as a whole is 0.90. From these findings, it can be concluded that

students at the primary and secondary school levels have potential in their
thinking abilities if they are optimised using appropriate intervention programs
(Fang et al., 2021; Lu et al., 2021).
Students at the elementary school level who have low higher-order thinking skills
can also improve their thinking skills using a multidimensional curriculum
design, meaning that the high-order thinking skills of the elementary school
students are almost equal to the students' high-order thinking skills at the middle
school level. This improvement can be seen in both the thinking competencies in
general, as well as in every dimension of the scientific, creative, and future
thinking competencies more generally. Teaching thinking competencies such as
scientific thinking competencies (inquiry), creative thinking (problem solving),
and future thinking (personal and time perspectives) carried out among early age
students (elementary school) will greatly assist the students in getting used to
continuing to use high-level thinking skills high in each learning process
(Miedijensky et al., 2021; Wijnen et al., 2021).
From the results of the analysis, the increase in the dimensions of the ability to
think scientifically increased at least from pretest to posttest among the other
thinking competencies. This is inseparable from the complexity of the scientific
thinking competencies which require quite long stages of involvement. Scientific
thinking competence is an inquiry process that contains quite complex stages
including defining problems, formulating hypotheses, planning experiments,
analysing results, and drawing conclusions (Bovill & Woolmer, 2019; Gore et al.,
2018). The difference in improvement in each significant dimension of thinking in
the experimental group is caused by the other additional aspects used by the
teachers to increase student motivation, for example, the use of innovative
learning strategies, interesting topics, involving students in formulating the
assessment criteria and materials, as well as the use of a challenging learning
process and encouraging the students' higher-order thinking skills. Furthermore,
an increase in the dimensions of creative thinking can be seen in the students'
ability to identify and solve problems (Whalen & Paez, 2021; Zhang & Chan, 2020).
This increase also occurred for the students who had never been involved in the
problem-solving process before in class, such as the elementary school students
who rarely used this method.
All students at the primary and secondary school levels are able to demonstrate
their ability to identify problems and solve them in various fields of study.
Students who have very low high-order thinking skills are able to improve their
thinking skills to almost match the thinking abilities of the middle-level students.
This finding was quite a surprise to the researchers, as well as providing an
illustration that younger students are able to optimise their thinking skills very
quickly if given the right intervention (Lin & Chuang, 2018; Lu et al., 2021).
Furthermore, future thinking competence also increased but the increase was not
too significant in the experimental group when viewed according to each
dimension of personal perspective and time perspective. This happens because
the competence to think about the future for elementary and middle school
students is something new and rarely used either at school or in everyday life.
236
This is consistent with the theory that future thinking contains higher thinking
competencies that require inter-process understanding, meaning that it takes a
longer time to master them (Manassero-Mas & Vázquez-Alonso, 2022; Saido et al.,
2018). In general, more than 55% of elementary and middle school students were
able to use the future thinking ability in the posttest phase. This indicates that
students can be taught the ability to think well in the future even though it is
difficult unless the right intervention program is used.
6. Conclusion and Implication

The multidimensional curriculum has been proven to be effective at increasing
the higher order thinking skills of primary and secondary school students. The
improvement is seen in the three abilities that are at the core of higher-order
thinking, namely the ability to think scientifically, to think creatively, and to think
about the future. From the two school levels, the increase in higher-order thinking
skills at the secondary school level was more significant than that of the
elementary school students because the development of high-level thinking skills
was more sufficiently trained. Furthermore, the effect of the multidimensional
curriculum on the higher thinking skills of female students was more significant
than that of male students. The strongest correlation between higher-order
thinking skills and curriculum dimension elements was found in the correlation
between creative thinking (identifying and solving problems) and future thinking
(personal and time perspectives), followed by the second correlation, namely the
relationship between scientific thinking (inquiry). by thinking creatively
(identifying and solving problems), and lastly the relationship between scientific
thinking (inquiry) and thinking about the future (personal and time perspectives).
A teaching process that uses a multidimensional curriculum can facilitate the

students in acquiring scientific, creative thinking, and future thinking skills in a
relatively short time. The multidimensional curriculum in its implementation
must be accompanied by the use of learning methods that are able to encourage
students to think scientifically, creatively, and be able to predict the future. This
research facilitates several implications, namely that a) curriculum designers
should pay attention to several aspects, including the design of the inputs,
processes, and outputs as part of the curriculum implementation, the teaching of
integrated thinking competencies, the training of teachers, and the development
of future thinking competencies with a multidimensional curriculum design; b)
students who receive the learning program interventions have better
opportunities in the future to get a decent life; c) thinking competence must be
experienced directly according to the context so then the students can optimally
develop their ideas from various perspectives; d) teachers at all school levels must
have the capacity to provide a learning process that improves the students' higher-
order thinking skills; e) the teaching process must promote future thinking skills
so then the students are able to identify the processes involved and relate them,
allowing them to provide predictions of a problem in the future.
7. Limitations and Recommendations

This study has several limitations, including the limited sample of students both
in terms of number and school level, research data that only relies on quantitative
data, no investigation of previously acquired learning experiences, and focusing
237
only on three dimensions of thinking, namely scientific, creative, and future.

Based on these limitations, the recommendations for further research are that the
samples that must be expanded in terms of both number and school level, for
example, up to senior high school and tertiary education levels. The research data
can also be better equipped with qualitative data which can be obtained through
interviews. Experiential investigations into the previous learning need to be done
so then any deficiencies in the teaching process can be corrected, and thinking
dimensions can be expanded or added to the thinking dimensions needed at this
time. The combination of scientific, creative, and future thinking competencies
provides opportunities for researchers to carry out further research on future
thinking literacy.
8. References
Baghaei, S., Bagheri, M. S., & Yamini, M. (2020). Analysis of IELTS and TOEFL reading
and listening tests in terms of Revised Bloom’s Taxonomy. Cogent Education, 7(1).
https://doi.org/10.1080/2331186X.2020.1720939
Barfod, K., & Bentsen, P. (2018). Don’t ask how outdoor education can be integrated into
the school curriculum; ask how the school curriculum can be taught outside the
classroom. Curriculum Perspectives, 38(2), 151–156. https://doi.org/10.1007/s41297-
018-0055-9
Bovill, C., & Woolmer, C. (2019). How conceptualisations of curriculum in higher
education influence student-staff co-creation in and of the curriculum. Higher
Education, 78(3), 407–422. https://doi.org/10.1007/s10734-018-0349-8
Carroll, K. A., & Harris, C. M. (2020). Using a Repetitive Instructional Intervention to
Improve Students’ Higher-Order Thinking Skills. College Teaching, 69(2), 82–90.
https://doi.org/10.1080/87567555.2020.1823310
Cheng, S. C., Hwang, G. J., & Lai, C. L. (2020). Effects of the group leadership promotion
approach on students’ higher order thinking awareness and online interactive
behavioral patterns in a blended learning environment. Interactive Learning
Environments, 28(2), 246–263. https://doi.org/10.1080/10494820.2019.1636075
Cross, R. (2021). Correction to: Creative in finding creativity in the curriculum: the CLIL
second language classroom (The Australian Educational Researcher, (2012), 39, 4,
(431-445), 10.1007/s13384-012-0074-8). Australian Educational Researcher, 48(1), 209.
https://doi.org/10.1007/s13384-020-00427-3
Davis, K. A., Grote, D., Mahmoudi, H., Perry, L., Ghaffarzadegan, N., Grohs, J.,
Hosseinichimeh, N., Knight, D. B., & Triantis, K. (2023). Comparing Self-Report
Assessments and Scenario-Based Assessments of Systems Thinking Competence.
Journal of Science Education and Technology, 0123456789.
https://doi.org/10.1007/s10956-023-10027-2
Diamond, K., Kandola, S., & Weimerskirch, M. (2020). Developing Problem-Solving Skills
in Active Learning Pre-Calculus Courses. Primus, 0(0), 1–16.
https://doi.org/10.1080/10511970.2020.1772917
Elfeky, A. I. M. (2019). The effect of personal learning environments on participants’
higher order thinking skills and satisfaction. Innovations in Education and Teaching
International, 56(4), 505–516. https://doi.org/10.1080/14703297.2018.1534601
Fang, J. W., Chang, S. C., Hwang, G. J., & Yang, G. (2021). An online collaborative peer-
assessment approach to strengthening pre-service teachers’ digital content
development competence and higher-order thinking tendency. Educational
Technology Research and Development, 69(2), 1155–1181.
https://doi.org/10.1007/s11423-021-09990-7
238
Fensham, P. J. (2022). Correction to: The Future Curriculum for School Science: What Can
Be Learnt from the Past? (Research in Science Education, (2016), 46, 2, (165-185),
10.1007/s11165-015-9511-9). Research in Science Education, 0123456789, 11165.
https://doi.org/10.1007/s11165-022-10076-4
Gore, J., Banks, A. P., & McDowall, A. (2018). Developing cognitive task analysis and the
importance of socio-cognitive competence/insight for professional practice.
Cognition, Technology and Work, 20(4), 555–563. https://doi.org/10.1007/s10111-
018-0502-2
Green, B. (2018). Understanding curriculum? Notes towards a conceptual basis for
curriculum inquiry. Curriculum Perspectives, 38(1), 81–84.
https://doi.org/10.1007/s41297-017-0038-2
Green, B. (2022). Understanding curriculum as practice, or on the practice turn(s) in
curriculum inquiry. Curriculum Perspectives, 42(1), 77–83.
https://doi.org/10.1007/s41297-022-00160-0
Guo, X., Hao, X., Deng, W., Ji, X., Xiang, S., & Hu, W. (2022). The relationship between
epistemological beliefs, reflective thinking, and science identity: a structural
equation modeling analysis. International Journal of STEM Education, 9(1).
https://doi.org/10.1186/s40594-022-00355-x
Hadianto, D., Damaianti, V. S., Mulyati, Y., & Sastromiharjo, A. (2021a). Enhancing
scientific argumentation skill through partnership comprehensive literacy. Journal
of Physics: Conference Series, 2098(1). https://doi.org/10.1088/1742-
6596/2098/1/012015
Hadianto, D., Damaianti, V. S., Mulyati, Y., & Sastromiharjo, A. (2021b). The role of
multimodal text to develop literacy and change social behaviour foreign learner.
International Journal of Instruction, 14(4), 85–102.
Hadianto, D., S. Damaianti, V., Mulyati, Y., & Sastromiharjo, A. (2022). Effectiveness of
Literacy Teaching Design Integrating Local Culture Discourse and Activities to
Enhance Reading Skills. Cogent Education, 9(1), 0–13.
https://doi.org/10.1080/2331186X.2021.2016040
Heron, M., & Palfreyman, D. M. (2021). Exploring Higher-Order Thinking in Higher
Education Seminar Talk. College Teaching, 0(0), 1–8.
https://doi.org/10.1080/87567555.2021.2018397
Hwang, G. J., Yin, C., & Chu, H. C. (2019). The era of flipped learning: promoting active
learning and higher order thinking with innovative flipped learning strategies and
supporting systems. Interactive Learning Environments, 27(8), 991–994.
https://doi.org/10.1080/10494820.2019.1667150
Lin, C. T., & Chuang, S. S. (2018). The role of empathy between functional competence
diversity and competence acquisition: a case study of interdisciplinary teams.
Quality and Quantity, 52(6), 2535–2556. https://doi.org/10.1007/s11135-018-0794-6
Lu, K., Pang, F., & Shadiev, R. (2021). Understanding the mediating effect of learning
approach between learning factors and higher order thinking skills in collaborative
inquiry-based learning. Educational Technology Research and Development, 69(5),
2475–2492. https://doi.org/10.1007/s11423-021-10025-4
Manassero-Mas, M. A., & Vázquez-Alonso, Á. (2022). An empirical analysis of the
relationship between nature of science and critical thinking through science
definitions and thinking skills. SN Social Sciences, 2(12), 1–27.
https://doi.org/10.1007/s43545-022-00546-x
McConnery, J. R., Bassilious, E., & Ngo, Q. N. (2021). Engagement and learning in an
electronic spaced repetition curriculum companion for a paediatrics academic half-
day curriculum. Perspectives on Medical Education, 10(6), 369–372.
239
https://doi.org/10.1007/s40037-021-00680-x
Miedijensky, S., Sasson, I., & Yehuda, I. (2021). Teachers’ Learning Communities for
Developing High Order Thinking Skills—A Case Study of a School Pedagogical
Change. Interchange, 52(4), 577–598. https://doi.org/10.1007/s10780-021-09423-7
Nurwanto, N., & Cusack, C. M. (2018). Correction to: Addressing multicultural societies:
lessons from religious education curriculum policy in Indonesia and England.
Journal of Religious Education, 66(3), 237–237. https://doi.org/10.1007/s40839-018-
0059-7
Oberauer, K., Schickl, M., Zint, M., Liebhaber, N., Deisenrieder, V., Kubisch, S., Parth, S.,
Frick, M., Stötter, H., & Keller, L. (2022). The impact of teenagers’ emotions on their
complexity thinking competence related to climate change and its consequences on
their future: looking at complex interconnections and implications in climate
change education. Sustainability Science, 18(2), 907–931.
https://doi.org/10.1007/s11625-022-01222-y
Saido, G. A. M., Siraj, S., DeWitt, D., & Al-Amedy, O. S. (2018). Development of an
instructional model for higher order thinking in science among secondary school
students: a fuzzy Delphi approach. International Journal of Science Education, 40(8),
847–866. https://doi.org/10.1080/09500693.2018.1452307
Sung, H. Y., Hwang, G. J., & Chen, S. F. (2019). Effects of embedding a problem-posing-
based learning guiding strategy into interactive e-books on students’ learning
performance and higher order thinking tendency. Interactive Learning Environments,
27(3), 389–401. https://doi.org/10.1080/10494820.2018.1474235
van Leent, L., & Spina, N. (2022). Teachers’ representations of genders and sexualities in
primary school: the power of curriculum and an institutional ideological code.
Australian Educational Researcher, 0123456789. https://doi.org/10.1007/s13384-022-
00515-6
Whalen, K., & Paez, A. (2021). Student perceptions of reflection and the acquisition of
higher-order thinking skills in a university sustainability course. Journal of
Geography in Higher Education, 45(1), 108–127.
https://doi.org/10.1080/03098265.2020.1804843
Wijnen, F., Walma van der Molen, J., & Voogt, J. (2021). Primary school teachers’ attitudes
toward technology use and stimulating higher-order thinking in students: a review
of the literature. Journal of Research on Technology in Education, 0(0), 1–23.
https://doi.org/10.1080/15391523.2021.1991864
Zhang, Y., & Chan, K. K. (2020). Infusing visual analytics technology with business
education: An exploratory investigation in fostering higher-order thinking in
China. Innovations in Education and Teaching International, 00(00), 1–10.
https://doi.org/10.1080/14703297.2020.1793799
240

Vol. 22, No. 7, pp. 240-257, July 2023
GO-DEEP: A Potential Reflection Model for

Experiential Learning
Ngoc Thi-Nhu Bui
Institute for Innovative Learning, Mahidol University, Thailand
Pratchayapong Yarsi*
Faculty of Science, King Mongkut’s University of Technology Thonburi,
Thailand
Abstract. Experiential learning is a critical approach in higher education

that emphasizes learning by doing, where students engage in activities
that promote hands-on learning experiences. Reflection is a crucial
component of experiential learning, enabling learners to extract valuable
insights and knowledge from their experiences. This study presents our
synthesized reflection model, the GO-DEEP reflection model, which
includes six main stages to reflect before, during, and after experiences.
This study implemented a pre-experimental design to investigate the
model’s usefulness and effectiveness in a work-integrated learning
course. The course comprised of eight learning sessions with 39 life
science students. During each session, the students actively participated
in the process of reflection. The students were asked to complete the
GO-DEEP level of use questionnaires after sessions four and eight and
they submitted their reflection reports after each session through Google
Forms. In addition, they responded to the open-ended questions about
their perception of the GO-DEEP reflection model and reflection
practices. The collected data were analyzed using descriptive, thematic,
and t-test analyses. The findings reveal that the university students who
used the GO-DEEP reflection model were highly interested in its use
and found it beneficial in extracting valuable insights about their
learning experiences by constructing knowledge, promoting, and
facilitating learning. These outcomes highlight the efficacy and potential
of GO-DEEP as a helpful tool in fostering meaningful and impactful
educational experiences. Moreover, based on student experience, GO-
DEEP has the potential to be applied not only in academic aspects but
also in non-academic activities and life experiences.
Keywords: experiential learning; higher education; work-integrated

learning; reflective practice; GO-DEEP reflection model
*
Corresponding author: Pratchayapong Yarsi, pratchayapong.yasr@kmutt.ac.th
241
1. Introduction
Higher education plays a vital role in equipping students with the necessary
competencies for lifelong learning, employment, and personal growth. However,
achieving these goals requires active engagement from students, as learning
achievement is closely linked to their level of engagement. In this context,
learning engagement can be defined as students’ interest in learning, which
includes affective, cognitive, and behavioral domains (Kahu, 2013). Various
approaches are applied to promote learning engagement, such as teaching
methods, classroom discussion, engaging learning activities, project-based
learning, peer grading (Collaço, 2017).
Besides focusing on academic courses and teaching methods, students must

have a chance to participate in hands-on activities to enhance employability in
the competitive labor market, such as experiential activities, work experiences,
entrepreneurship modules, and career orientation (Knight & Yorke, 2010).
Experiential programs and courses aim to provide hands-on practices and work-
integrated learning to increase learning engagement. Occasionally, they can
involve the industry stakeholders in the academic curriculum and training
(Ferns et al., 2014; Jackson, 2018; Rowe & Zegwaard, 2017).
Reflection is a necessary stage of any experience, which can promote intention

and purpose to develop a deliberate, professional after experience. It could also
enhance learning engagement, satisfaction, and self-efficacy (Phan, 2014; van
Seggelen-Damen & van Dam, 2016); develop lifelong learning (McKauge et al.,
2011); and enable individuals’ unlearning, to stop using outdated knowledge
and affect work engagement (Matsuo, 2019).
Although various reflection models are applied in different learning disciplines

and forms, there is still a lack of a systematic model for experiential approaches
or work-integrated learning. This study, therefore, aims to synthesize a reflection
model from existing frameworks for experiential learning, demonstrate how to
use it in a virtual work-integrated learning environment, and determine the
usefulness level from both quantitative and qualitative perspectives.
Experiential learning is an essential part of education that promotes bridging the
gap between theory and practice. Kolb has defined experiential learning since
1984 — also known as Kolb’s learning cycle and Kolb’s experiential learning
theory. The theory focuses on four stages of the learning process, learning styles,
and learning spaces.
In the initial cycle, the idea mentions four stages of learning: concrete experience,
reflective observations, abstract conceptualization, and active experimentation.
In the concrete experience stage, students encounter a new experience or
learning situation. In the reflective observation stage, students review their
learning carefully and recognize their experience and understanding. The
abstract conceptualization stage means students focus on constructing
knowledge, theories, or concepts from experiences. Finally, students practice
242
what they have learned by implementing it in different situations for active

experimentation (Kolb, 1984).
In addition, the experiential learning theory proposes learning as the process by

which learners can create, recreate, and construct their knowledge through
concrete experience (Kolb & Kolb, 2005).
Morris (2019) provided a revision of the Kolb learning cycle, emphasizing

contextually rich concrete experience, which means that the present moment is a
hands-on, real-world experience, rather than in all situations and arenas of life.
The author also stressed critical reflection, which is necessary for the learning
process and acts as a meditator of meaning-making. In addition, contextual-
specific, abstract conceptualization and practical active experimentation were
the optimized stages in the cycle. In higher education, experiential learning
could be work-integrated learning, work-based practice, cooperative learning,
collaborative learning, service learning, career development, internships, and
practicums (Harvey et al., 2016).
Reflection can be viewed as a mediating stage between experience and learning,

fostering the transition to learned experience (Kuk & Holst, 2018). Student
learning can be promoted through reflection in constructing knowledge,
identifying the missing knowledge, and promoting personal development (Ash
& Clayton, 2009; Chang, 2019). Moreover, the reflection process allows students
to achieve internship learning goals, including professional development,
personal growth, and civic learning (Ash & Clayton, 2009). Reflection is also a
powerful approach to improve leadership practice and become an effective
leader by helping address personal values, establishing expectations, thinking
critically, and matching the team with the organization’s direction (Moon, 2013;
Thaanraj, 2016).
Learners can reflect on their learning through various tools to maximize the
effectiveness of reflection on learning, such as journaling (Dyment & O’Connell,
2010) and portfolios (Beka & Kulinxha, 2021). Over than that, scholars have
suggested the criteria of high-quality reflection include describing experience
context, linking experiences to learning (integrating the current knowledge to
understand and connect recent experiences with previous experiences to
construct their learning strategy); integrating learning goals and learning
materials; providing deep reflection by helping clarify values and assessing in
terms of critical thinking; and occurring regular scaffolding processes and
specific guided questions (Moon, 2013; Roberts, 2016).
In addition, scholars developed well-structured reflection models to promote

deep reflection for reflection advantages, such as
• GIBBS (description, feelings, evaluation, analysis, conclusion, action
plan) (Gibbs, 1998).
• SHARP (set learning objectives, how did it go? address concerns, review
learning points, plan) (Ahmed et al., 2013).
• The questioning “what? so what? now what?” (Skinner & Mitchell, 2016).
243
• Schön’s reflection (reflect in action; reflect on action) (Schön, 1983, cited

in Edwards, 2017).
3. Synthesizing Process of the GO-DEEP Model

The process of synthesizing the model included analyzing the characteristics of
experiential learning; reviewing the critical criteria of high-quality reflection for
experiential learning; mapping and comparing the differences and similarities of
some current reflection models; identifying the missing criteria of the existing
models for experiential learning; proposing the prototype model; and revising
the reflection model through discussion to reach the agreement between authors.
In this study, we proposed nine criteria of high-quality reflection for experiential
learning, by following other scholars’ suggestions and Kolb’s learning cycle
(1984).
They included:
1. Describing previous capacities.
2. Setting learning goals.
3. Describing the experience.
4. Providing deep reflection.
5. Linking experiences to learning (Moon, 2013).
6. Leading a future action plan (Ahmed et al., 2013).
7. Being suitable for experiential learning.
8. Occurring regularly.
9. Scaffolding process and specific guided questions (Roberts, 2016).
The nine criteria were used as criteria for the benchmark’s current reflection
models. The selected reflection models were mapped with criteria of high-
quality reflection in Table 1.
Table 1: Benchmarking the current reflection models to criteria

Criteria of high-quality reflection
Reflection approach Author (year)
1 2 3 4 5 6 7 8 9
Description, Feelings,
Evaluation, Analysis, Gibbs, 1998 ✓ ✓ ✓ ✓ ✓ ✓
Conclusion, Action Plan
SHARP (Set learning
objectives; How did it go?
Ahmed et al.,
Address concerns; Review ✓ ✓ ✓ ✓ ✓ ✓ ✓
2013
learning points, Plan
ahead)
What? So what? Now Skinner &
✓ ✓ ✓
what? Mitchell, 2016
Schön (1983,
Reflect in action; Reflect
cited in ✓ ✓ ✓
on action
Edwards, 2017)
(1) Describing previous capacities, (2) Setting learning goals, (3) Describing the experience, (4)
Providing deep reflection, (5) Linking experiences to learning, (6) Leading a future action plan,
(7) Being suitable for experiential learning, (8) Occurring regularly, (9) Scaffolding process
and specific guided questions
244
As evident in Table 1, all selected reflection models were well-mapped with

criteria 3, 6, and 7, namely, describing learning experiences, leading plans, and
being suitable for experiential learning, respectively. However, all approaches
do not show the link with criterion 1 about the previous capacities relevant to
the new experience. They were unsure how often reflection occurs, which may
lead to missing information and may not improve their reflection ability during
the experience (criterion 8). Gibbs and SHARP reflection approaches provide a
scaffolding reflection process and specific guided questions (criterion 9) and link
experience to learning strategically (criterion 5). SHARP reflection is the only
model which requires the learner to set learning goals (criterion 2). Thus, there is
a need to synthesize the existing approaches to generate a unified reflection
framework for experiential learning that includes all these nine criteria.
In the preliminary investigation stage of the GO-DEEP model, an invitation was

sent via email to students who had finished experiential courses (i.e., internship)
to respond to their reflections’ perceptions about the need for reflection during
their internship. The 20 students responded to the online open-ended questions
in Vietnamese; participant identification was anonymous and kept confidential.
The majority of 15 students (75%) confirmed that it is necessary to reflect during
an internship because of several reasons, such as firstly recognizing strengths
and weaknesses in the working periods so that students can improve
immediately; and second recording experiences in detail. However, no effective
reflection model suitable for an internship has been offered. For this reason, the
authors synthesized the reflection approach for experiential learning, including a
reflection process and guided questions by following nine high-quality reflection
criteria, Kolb’s learning cycle, Gibbs, and SHARP models.
Following the theoretical and practical approaches, the synthesized reflection

model guides users to reflect before, during, and after experiences, and allows
them to link previous experiences to future experiences. Before participating in a
new experience, the students must list the gained knowledge, skills, or
competencies which are relevant and help to succeed in the unique experience.
In addition, learning objectives which include professional and personal
purposes should be described carefully.
After careful consideration of the preliminary findings, and based on the

existing reflection models, it was decided to develop the GO-DEEP model which
stands for Gained - Objective - Describe - Evaluate - Establish - Plan (Figure 1).
After finalizing the GO-DEEP model, it was subjected to further validation. The
educational experts reviewed the model and provided critical feedback to
improve the model and guided questions.
245
Figure 1: A synthesized framework for reflective practice: The GO-DEEP reflection

model
Figure 1 shows the different stages of the GO-DEEP model. The gained stage
should be implemented before any experience and during the experience. It
requires the learners to look back on previous knowledge, skills, and
competencies that are relevant and help them succeed in future experiences
(criterion 1). In addition, the learners can update the new information cultivated
from the current experience. The suggested questions could be “What
knowledge/skills have you gained which relate to and help to succeed in the
future experience?”.
The objective stage is setting learning objectives, in which the learners could
have the proper motivation and direction during the experience. The learning
objectives are the measurable statements in different learning aspects, such as
cognitive, psychomotor, and affective domains. Other than those, learners could
think about any personal purposes that can promote learning (criterion 2). The
prompt could be “What do you want to gain after learning?” or “What are the
desired results/abilities/skills?”.
The describe stage focuses on learning description (criterion 3). The guided
questions could be “What have you learned?”, “What did you do?”, “How did
you feel?” and “When did you do it?”. During the describe stage, students
articulate their learning content, activities, feelings, and perceptions (resembling
the concrete experience phase).
The evaluate stage is an important stage to self-discover and self-evaluate

through experience. Students reflect deeply on the development of ideas, skill
sets, knowledge, and performances and uncover areas of strength and
discomfort. In addition, the reflection allows students to determine the value of
experiences/learning (criterion 4). Some questions to be asked are “In what
ways did you succeed or do well in this situation? How can experience promote
achieving your learning objectives? What have you gained? Which are areas of
strength and discomfort?”. The evaluate stage serves as a crucial stage for self-
discovery and self-evaluation, similar to reflection in Kolb’s cycle.
246
The establish stage allows students to conceptualize from experience, construct

their knowledge, skills and competencies from their expertise, and promote
professional learning. If they can describe every detail in the previous stages,
they will build knowledge and information in this stage. It helps a student to
self-direct their learning. In addition, students can look back at the information
in the gained stage to add new concepts or modify the outdated
concept/information they have accumulated from the presented experience
(criterion 5). Some questions are “How can experience promote to achieve your
learning objectives?” and “Which knowledge/principles/skills have you gained
from experience?”. In this stage, students conceptualize knowledge, skills, and
competencies acquired from their experiences, paralleling abstract
conceptualization.
The plan stage proposes action plans for future implications which are based on
new knowledge. Students can determine critical actions, resources and timelines
for future implications or engage in an iterative cycle of drafts; receive and
reflect on feedback; and pursue the following steps until the task is completed
(criterion 6). The guided questions could be “How would you achieve learning
objectives?” or “How could you apply the knowledge in the future?”.
Additionally, the plan stage empowers students to determine critical actions,
resources, and timelines based on newfound knowledge, mirroring active
experimentation. This mapping highlights the suitability of Kolb’s learning
model for experiential learning (criterion 7).
Furthermore, the authors suggest the implementation and application of GO-

DEEP for educators to facilitate the learning process in an online approach on a
regular basis, ranging from before, during and after experiencing certain
learning and teaching activities. The suggestion is presented in Table 2 (criterion
8).
Table 2: Instructional strategy with GO–DEEP
Learning and teaching activities

Stage
Educators Students
Introduce about GO-DEEP model Raise questions and concerns
Introduce learning objectives and
Discuss and ask questions
content
Before Guide students to complete Review competencies and set
experience “gained and objective” stages learning objectives
Review students’ response and
Analyze learning materials
deliver learning suggestion to reach
Build learning strategy
learning objectives
Write reflection during an
experience (describe, evaluate,
Provide proper learning activities
During establish, propose action stages)
for specific content
experience Repeat the reflection process
Feedback and discussion
during the experience
Look back gained and objective often
After Record the information and
Deep discussion and feedback
experience respond
247
4. Methodology
4.1 Context and Learning Environment
The context was a virtual work-integrated learning (WIL) program for a
bachelor’s degree in Ho Chi Minh City in the summer of 2022. The program
consists of eight consecutive sessions to introduce fundamental mindset,
knowledge, and skills about the labor market opportunities and requirements,
marketing strategies, technical sales principles, and practices in life science,
health care, and medicine trading for bio-biotechnology students, as well as
prepare students for future jobs. Students were invited to participate in different
learning environments in the eight-sessions WIL program, including lecturing,
role-playing, and lecturing with group work. The sequence of learning
approaches is described in Table 3.
Table 3. Learning environments sequence in each learning session

Sessions
1 2 3 4 5 6 7 8
Lecture Lecture Lecture Lecture

Final
Learning and and and and
Lecture Lecture Lecture presentation
environments role group group group
(role play)
play work work work
4.2 Participants
The first author emailed students an invitation and introduced the study and
GO-DEEP model to them. There were 39 students (33 female students, six male
students), and all were undergraduate students in life science (12 juniors and 27
seniors). All participants did not have any experience in writing reflections.
4.3 Research Instrument

4.3.1 Level of Use Behavior
Level of use is a framework to describe and evaluate the changes or innovation
that helps define an end user’s experience level. In general, there are eight levels
of use, including non-use (level 0), preparation (level I), orientation (level II),
mechanical use (level III), routine (level IV), refinement (level V), integration
(Level VI), and renewal (level VII). Levels 0 to II could be grouped as non-use,
from levels III to VII show the use of intervention (Hall et al., 2006; Roach et al.,
2009). This study adapted the questionnaire initially published by
Threekunprapa & Yasri (2021).
4.3.2 Open-ended Questions - Student’s Perception on Reflection Practice by Using

GO-DEEP
Open-ended questions were used to collect students’ comments and suggestions
about the GO-DEEP model and reflection approach. The questions include:
What do you think about the reflection using the GO-DEEP model? How did
GO-DEEP impact your learning? and please provide other comments or
suggestions to improve the GO-DEEP model.
4.4 Data Collection Procedure

This study is a pre-experimental design so that students participated in the same
group and responded to the requirements by following the data collection
248
procedure which was announced. The first author introduced the reflection and
GO-DEEP model in the introduction session. After each session, all participants
were asked to write reflections following the GO-DEEP instruction and submit
them via Google form voluntarily. Participants submitted 289 reflection reports
in Vietnamese in total; this number of written reflections allowed us to analyze
the textual information more profoundly and quantify the reflection outcomes in
different categories. The number of reflection reports for each session is
presented in Table 4.
Table 4: The number of submitted reflections reports after each session

Sessions
1 2 3 4 5 6 7 8
Number of reports 39 37 39 32 33 34 36 39
Number of missing 0 2 0 7 6 5 3 0
Students were asked to complete the level of use questionnaire sent in sessions
four and eight. Students responded to the questionnaire virtually via Google
Forms.
After experiencing the GO-DEEP reflection model, participants were asked to

provide comments and suggestions about GO-DEEP and the reflection approach
by responding to open-ended questions via Google form in session eight.

This study was a part of PhD thesis research which was ethical approval by IRB
of Mahidol University (COA. No. 2022/02-017). All participants were informed
about the purposes of the study, and they could withdraw without conditions.
The reflection results do not affect the course result. They were asked to submit
consent forms virtually, and the first author also explained the study and asked
them for oral consent in the course introduction session. The participant
identification was anonymous and kept confidential.
4.6 Data Analysis

The analysis strategy for qualitative and quantitative data includes:
Reflection reports. The authors discussed the criteria for evaluating reflection
reports. Four categories scheme is used to determine levels of reflection,
including non-reflection - 1, understanding - 2, reflection - 3, and critical
reflection – 4 (Kember et al., 2008). The rubric and reflection samples are shown
in the appendix. All reflection reports were analyzed and provided scores by
three evaluators (the first author and two invited lecturers). The mean results of
three evaluators were used for further analysis. After that, data analysis was
analyzed by performing descriptive statistics and comparisons of means through
t-tests.
Qualitative data from the open-ended questions. Thematic analysis was conducted to
analyze students’ responses. The first author read students’ responses several
times to become familiar with the language and ideas. Then, the themes were
identified, the students’ opinions about the reflection practice and GO-DEEP
249
model were categorized into themes, and the representative statements were
quoted. The first author selected the proper quotations, translated them into
English, and discussed them with the second author until a consensus was
reached.
The level of use. The results from level-of-use questionnaires were calculated as
the descriptive analysis and the frequency percentage.
5. Results
5.1 GO-DEEP Level of Use
Table 5 shows the GO-DEEP levels of use by the students. In the virtual course,
in session 4, 54.1 % of students chose either non-use, orientation, or preparation
level (non-use group), and only 45.9 % chose either mechanical routine,
refinement, or renewal level (use group). The percentage is changed in session 8
for the non-use and use levels, 29.7 % and 70.3 %, respectively. The result shows
an increasing change in GO-DEEP levels of use towards a more positive position
of perceived usefulness.
Table 5: The GO-DEEP levels of use

Session 4 Session 8
Level of use Statement
No % No %
I think GO-DEEP is NOT useful to me at
0. Non–use 1 2.7 2 5.4
all.
I am INTERESTED in using GO-DEEP in
1. Orientation 7 18.9 2 5.4
my own study.
2. Preparation I MAY use GO-DEEP in my own study. 12 32.4 7 18.9
3. Mechanical I WILL use GO-DEEP in my own study. 8 21.6 9 24.3
I WILL REGULARLY use GO-DEEP in my
4. Routine 5 13.5 9 24.3
own study.
I am CONFIDENT I can use GO-DEEP to
5. Refinement 3 8.1 6 16.2
improve my study.
I am confident I can TEACH others how to
6. Integration 0 0.0 2 5.4
use GO-DEEP.
I am confident I can DEVELOP a reflection
7. Renewal 1 2.7 0 0.0
model like this for my own study.
5.2 Students’ Perceptions of GO-DEEP

Based on a thematic analysis, the results show three benefits that the students
could get from using GO-DEEP, namely 1) construct knowledge; 2) promote
learning proactively and facilitate learning; and 3) expand application in
different subjects and promote personal development.
Theme 1: Construct knowledge

Students were interested in reflecting on learning by following GO-DEEP
instruction because it helps them construct knowledge systematically from
previous to current learning and future applications.
Student 19 wrote:
“GO-DEEP model is advantageous in systematizing knowledge.”
250
The students confirmed that reflection by GO-DEEP is the proper approach to

synthesizing what they have learned or experienced, preparing for the future,
and applying knowledge. In addition, by following GO-DEEP instructions,
students can look back and evaluate their personal learning goals and think
about the application in other contexts.
In this regard, student 36 mentioned:
“…an effective method to review and update the knowledge…new
knowledge which is combined with personal opinion/experience, lead the
knowledge have learned from instructors, alumni become of mine.”
In addition, the model positively affects students by asking students to set

learning goals for upcoming lessons, which can lead to the best performance.
Student 30 wrote:
“Model positively affected me because I know what to do in the next
lesson to achieve the highest results.”
Theme 2: Promote learning proactively and facilitate learning

Besides constructing knowledge, GO-DEEP could promote learning proactively
and facilitate learning. GO-DEEP is not only a proper model to support students
learning in aspects of knowledge but also encourages students to consider their
professional skills and well-prepared attitude about learning.
Student 31 explained:
“By using GO-DEEP, I become more concentrated during the lesson,
try to take note of the information…prepare for the next lesson. It helps
me understand my learning approach’s strengths and improvement
areas.”
In addition, GO-DEEP could be a potential learning facilitation approach

that encourages learning proactively and effectively through a well-
structured process.
Student 7 mentioned:
“Help me to overview all learning and activities and recognize what I
have done, what I need to develop and improve.”
Theme 3: Expand application in different subjects and promote

personal development
The results showed that the students widened the application of the GO-DEEP
model in different subjects and promoted personal development.
Student 28 explained:
“This is an interesting model I applied when learning new things in life
and work. It helps me to recognize my potential and limitation during
learning; also foster me to accumulate skills more quickly and
effectively.”
251
In specific, a student suggested that they can apply GO-DEEP to learn English.
Student 27 wrote:
“I can apply GO-DEEP in school and learn English. I find my learning
becomes systematic and reach my learning expectations.”
While students expressed their interest in GO-DEEP, it also needs improvement

in some aspects, such as modifying the guided questions, extending the duration
of reflection, and considering the frequency of reflection during the learning.
5.3 Reflective Practice Change in the Different Learning Environments of the

Work-Integrated Learning Program
In this study, three evaluators assessed reflection reports independently by
following the rubric in the appendix. Figure 2 shows the mean score of reflection
reports performed by the students in response to the activity set for each session.
Sessions 1, 4, and 8 had the highest mean score of 2.5. During these sessions,
students could reflect on their learning; they began to apply it in practical
situations with personal insight. Session 6 had the lowest score, which was 1.6.
Sessions 2, 3, 5, and 7 were around 2.0, meaning that the students could
understand the learning concepts and topics but still lack personal experiences,
real-life applications, or practical situations. In sessions 4 and 8, students
participated in hands-on activities/role-playing to become a salesperson, such as
contacting strangers, selling some products in session 4, and implementing the
mini workshop in session 8.
Figure 2: Reflection levels at each learning session
The results indicated that GO-DEEP can guide students on how to reflect on
their learning experiences. To identify the reflection changes in different learning
environments. We calculated the reflection level into three main learning
environment groups: the traditional lecture for sessions 1, 2, and 3; the
traditional lecture combined with group work for sessions 5, 6, and 7; and the
role-playing in sessions 4 and 8. The mean reflection score of each learning
environment is shown in Table 6.
252
Table 6: The reflection levels in learning environments

A. Descriptive analysis
Number of
Learning environment Mean (M) SD
reflection reports
Lecture 115 2.17 0.591
Lecture and group work 103 1.96 0.494
Role-playing 71 2.48 0.679
B. T-test comparison
Two-tailed (α = 0.05)
Paired t
p-value
Role-playing and lecture 2.86 0.007
Role-playing and group work 5.62 0.000
Lecture and group work 2.74 0.010
The t-test analysis of the findings revealed that the students exhibited
significantly higher levels of reflection during their participation in role-playing
activities compared to the other settings, with a significance level of 95%. In
contrast, no statistically significant difference was found in the level of reflection
displayed by the students during lecture sessions, whether conducted
individually or coupled with group work (Table 6.B T-test comparison). In other
words, a role-playing learning environment, a form of experiential learning,
improved student reflective thinking using the GO-DEEP model, compared to
different learning approaches, such as lecturing and group work.
6. Discussion
This study focuses on the synthesis of the new reflection model for experiential
learning and investigates its effectiveness and usefulness. We systematically
reviewed and synthesized the potential reflection model for experiential
learning, namely GO-DEEP. The six stages, with guided questions, can
encourage students to reflect on the action regularly, set learning goals, review
past capabilities, link experiences, construct new knowledge, and propose future
action. In terms of usefulness, the results demonstrate that over 70% of users are
at use behavior of the GO-DEEP at the end of the program.
The results confirmed that students could accept the GO-DEEP intervention to
reflect on their learning. In addition, students increasingly changed their
intention about using GO-DEEP in the user group from 45.9% in session 4 to
70.3% in session 8. Most students responded optimistically to using the GO-
DEEP reflection model at levels 3 (mechanical use) and 4 (routine use) by
replying to the levels of use questionnaire.
The spectrum of product usage levels spans from non-users to future developers
of the product (i.e., renewal). In this particular context, students are classified as
users, so it was not anticipated for them to reach the developer end of the
spectrum. It is argued that enthusiastic users typically remain at the mechanical
or routine use stage. Various factors may affect the higher level of use, such as
learning context, individual capacities, attitudes, learning support, and learner
efforts (Roach et al., 2009; Tung & Chang, 2008).
253
Furthermore, students also shared the application of GO-DEEP outside class in

different experiences, such as learning English and developing soft skills. This
means that GO-DEEP is a potential reflection model for experiential learning,
which allows recognition of the interconnections of knowledge and transforms
student learning (Chang, 2019).
Regarding the effectiveness of reflection practice, students participated in the

virtual course with different learning activities in each session. The means score
of the reflection level was assessed and calculated. We recognized interesting
findings, including a high level of reflection at the beginning and an actual
fluctuation in reflecting practice between sessions. The students performed a
high level of reflective practice by using GO-DEEP in the first session, followed
by a drop in the subsequent two lecture sessions. Although the reflection quality
went up again in sessions 4 and 8, it declined in sessions 5, 6, and 7.
The novelty effect could explain this phenomenon: students began to decrease
engagement motivation for using the new intervention (Hur & Oh, 2012). In
addition, emotional valence may decline if the learning activities are repeated
(Tatiana et al., 2021). Thus, educators should consider reflection frequency
because the students suggested reducing the frequency of reflection. If students
have to reflect overwhelmingly during the course, it could reduce their interest
in reflecting. For future implications, we encourage educators interested in
experiential learning to seriously consider the usefulness and effectiveness of the
GO-DEEP reflection model and its guided questions presented in this paper.
Initially, not all students reflected; perhaps there should be a training session,
allowing students to become familiar with reflection practices and the GO-DEEP
reflection model. It could take time and challenge to learn reflection practices at
the beginning. Thus, students can reflect by simply responding to the first
stages, such as listing the previous capacities, setting up learning objectives, and
describing tasks and experiences. Moreover, students may face boredom and
lose the motivation to reflect if they must reflect too often. Educators can suggest
minimal reflection practices should be conducted during the experience.
Although Google Forms was used to collect reflection reports in this study,
educators can facilitate students to use other tools (e.g., journaling, diary
notebook, recording, etc.) in which they are interested, with the GO-DEEP
reflection model serving as a guideline to organize the reflection content.
The GO-DEEP reflection model and reflective practices should be used as

learning movements and support learning rather than assessments to reduce
stress. Educators and institutions can encourage reflection by implementing a
debriefing session and providing bonus scores or rewards for proper situations.
Last, educators must create a trustworthy learning environment where students
can share the reflection report and ensure student privacy.
Future research should explore the reflection frequency in experiential learning

and how to enhance individual interest in reflection. Balancing reflection and
254
validity helps students develop self-assessments properly (Melrose, 2017). In

addition, we realized that the learning environments may affect reflection
practice. The result confirmed that the role-playing learning environment is
linked with the highest mean score of reflection level compared to lecturing
learning environment. In general, the findings informed the link between the
learning environment and the quality of reflection, and reflective practice could
be trained and changeable (Bruno & Dell’Aversana, 2018).
7. Conclusion
According to the results, this study demonstrated the effectiveness and
usefulness of the GO-DEEP reflection model in experiential learning with
specific role-playing learning environment. Generally, the findings
demonstrated a notable shift in GO-DEEP usage levels towards a more positive
perception of usefulness.
Through thematic analysis, three key benefits of employing GO-DEEP were

identified for students: the ability to construct knowledge, proactive learning
promotion, and the facilitation of learning across various subjects, thereby
fostering personal development. Notably, in a comparative analysis with other
learning approaches, such as lecturing and group work, a role-playing learning
environment, which aligns with experiential learning, proved to enhance
students’ reflective thinking when utilizing the GO-DEEP model.
8. References
Ahmed, M., Arora, S., Russ, S., Darzi, A., Vincent, C., & Sevdalis, N. (2013). Operation
debrief: A SHARP improvement in performance feedback in the operating room.
Annals of Surgery, 258(6), 958–963.
https://doi.org/10.1097/SLA.0b013e31828c88fc
Ash, S. L., & Clayton, P. H. (2009). Generating, Deepening, and Documenting Learning:
the Power of Critical Reflection in Applied Learning. Journal of Applied Learning
in Higher Education, 1, 25–48. https://doi.org/10.57186/jalhe_2009_v1a2p25-48
Beka, A., & Kulinxha, G. (2021). Portfolio as a Tool for Self- Reflection and Professional
Development for Pre-Service Teachers. International Journal of Learning, Teaching
and Educational Research, 20(2), 22–35. https://doi.org/10.26803/ijlter.20.2.2
Bruno, A., & Dell’Aversana, G. (2018). Reflective practicum in higher education: The
influence of the learning environment on the quality of learning. Assessment and
https://doi.org/10.1080/02602938.2017.1344823
Chang, B. (2019). Reflection in learning. Online Learning Journal, 23(1), 95–110.
https://doi.org/10.24059/olj.v23i1.1447
Collaço, C. M. (2017). Increasing Student Engagement in Higher Education. Journal of
Higher Education Theory and Practice, 17(4), 40–47.
https://articlegateway.com/index.php/JHETP/article/view/1545
Dyment, J. E., & O’Connell, T. S. (2010). The quality of reflection in student journals: A
review of limiting and enabling factors. Innovative Higher Education, 35(4), 233–
244. https://doi.org/10.1007/s10755-010-9143-y
Edwards, S. (2017). Reflecting differently. New dimensions: reflection-before-action and
reflection-beyond-action. International Practice Development Journal, 7(1), 1–14.
https://doi.org/10.19043/ipdj.71.002
Ferns, S., Campbell, M., & Zegwaard, K. E. (2014). Work-integrated learning. In S. Ferns
255
(Ed.), Work integrated learning in the curriculum (pp. 1–6). Higher Education
Research and Development Society of Australia (HERDSA Inc).
https://researchcommons.waikato.ac.nz/handle/10289/9196
Gibbs, G. (1998). Learning by Doing, A Guide to Teaching and Learning Methods. Oxford
Brooks University.
Hall, G. E., Dirksen, D. J., & George, A. A. (2006). Measuring implementation in schools:
Level of Use. The Southwest Educational Development Laboratory.
https://sedl.org/cbam/lou_manual_201410.pdf
Harvey, M., Coulson, D., & McMaugh, A. (2016). Towards a theory of the Ecology of
Reflection: Reflective practice for experiential learning in higher education.
Journal of University Teaching & Learning Practice, 13(2), 2.
https://doi.org/10.53761/1.13.2.2
Hur, J. W., & Oh, J. (2012). Learning, Engagement, and Technology: Middle School
Students’ Three-Year Experience in Pervasive Technology Environments in
South Korea. Journal of Educational Computing Research, 46(3), 295–312.
https://doi.org/10.2190/ec.46.3.e
Jackson, D. (2018). Developing graduate career readiness in Australia: Shifting from
extra-curricular internships to work-integrated learning. International Journal of
Work-Integrated Learning, 19(1), 23–35.
Kahu, E. R. (2013). Framing student engagement in higher education. Studies in Higher
Kember, D., Mckay, J., Sinclair, K., & Kam Yuet Wong, F. (2008). A four‐category scheme
for coding and assessing the level of reflection in written work. Assessment &
https://doi.org/10.1080/02602930701293355
Knight, P. T., & Yorke, M. (2010). Employability and Good Learning in Higher
Education. Teaching in Higher Education, 8(1), 3–16.
https://doi.org/10.1080/1356251032000052294
Kolb, A. Y., & Kolb, D. A. (2005). Learning Styles and Learning Spaces: Enhancing
Experiential Learning in Higher Education. Academy of Management Learning &
Education, 4(2), 193–212. https://doi.org/10.5465/amle.2005.17268566
Kolb, D. A. (1984). Experiential Learning: Experience as the Source of Learning and
Development (Vol. 1). Prentice-Hall.
https://www.researchgate.net/publication/235701029_Experiential_Learning_
Experience_As_The_Source_Of_Learning_And_Development
Kuk, H. S., & Holst, J. D. (2018). A Dissection of Experiential Learning Theory:
Alternative Approaches to Reflection. Adult Learning, 29(4), 150–157.
https://doi.org/10.1177/1045159518779138
Matsuo, M. (2019). Critical reflection, unlearning, and engagement. Management Learning,
50(4), 465–481. https://doi.org/10.1177/1350507619859681
McKauge, L., Stupans, I., Owen, S. M., Ryan, G., & Woulfe, J. (2011). Building Critical
Reflection Skills for Lifelong Learning in the Emergent Landscape of a National
Registration and Accreditation Scheme: Journal of Pharmacy Practice, 24(2), 235–
240. https://doi.org/10.1177/0897190010397373
Melrose, S. (2017). Balancing Reflection and Validity in Health Profession Students’ Self-
Assessment. International Journal of Learning, Teaching and Educational Research,
16(8), 65–76. https://www.ijlter.org/index.php/ijlter/article/view/979
Moon, J. A. (2013). A handbook of reflective and experiential learning: Theory and
practice. In A Handbook of Reflective and Experiential Learning: Theory and Practice.
Routledge. https://doi.org/10.4324/9780203416150
256
Morris, T. H. (2019). Experiential learning – a systematic review and revision of Kolb’s

model. Interactive Learning Environments, 28(8), 1064–1077.
https://doi.org/10.1080/10494820.2019.1570279
Phan, H. P. (2014). Self-efficacy, reflection, and achievement: A short-term longitudinal
examination. Journal of Educational Research, 107(2), 90–102.
https://doi.org/10.1080/00220671.2012.753860
Roach, A. T., Kratochwill, T. R., & Frank, J. L. (2009). School-Based Consultants as
Change Facilitators: Adaptation of the Concerns-Based Adoption Model
(CBAM) to Support the Implementation of Research-Based Practices. Journal of
Educational and Psychological Consultation, 19(4), 300–320.
https://doi.org/10.1080/10474410802463304
Roberts, P. (2016). Reflection: A renewed and practical focus for an existing problem in
teacher education. Australian Journal of Teacher Education, 41(7), 19–35.
Rowe, A. D., & Zegwaard, K. E. (2017). Developing graduate employability skills and
attributes: Curriculum enhancement through work-integrated learning. Asia-
Pacific Journal of Cooperative Education, Special Issue, 18(2), 87–99.
https://www.ijwil.org/files/apjce_18_2_87_99.pdf
van Seggelen-Damen, I., & van Dam, K. (2016). Self-reflection as a mediator between self-
efficacy and well-being. Journal of Managerial Psychology, 31(1), 18–33.
https://doi.org/10.1108/jmp-01-2013-0022
Skinner, M., & Mitchell, D. (2016). “What? So What? Now What?” Applying Borton and
Rolfe’s Models of Reflexive Practice in Healthcare Contexts. Health and Social
Care Chaplaincy, 4(1), 10–19. https://doi.org/10.1558/hscc.v4i1.28972
Tatiana, S., Hanna, J., Kristiina, M., Jouni, P., & Sanna, J. (2021). The Changes in Lower
Secondary School Students’ Interest During Collaborative Learning. Scandinavian
Journal of Educational Research, 66(7), 1127–1140.
https://doi.org/10.1080/00313831.2021.1958377
Thaanraj, A. (2016). Improving Leadership Practice through the Power of Reflection: An
Epistemological Study. International Journal of Learning, Teaching and Educational
Research, 15(9), 28–43.
https://www.ijlter.org/index.php/ijlter/article/view/757
Threekunprapa, A., & Yasri, P. (2021). The role of augmented reality-based unplugged
computer programming approach in the effectiveness of computational
thinking. International Journal of Mobile Learning and Organisation, 15(3), 233–250.
https://doi.org/10.1504/ijmlo.2021.116506
Tung, F. C., & Chang, S. C. (2008). Nursing students’ behavioral intention to use online
courses: A questionnaire survey. International Journal of Nursing Studies, 45(9),
1299–1309. https://doi.org/10.1016/j.ijnurstu.2007.09.011
Appendix
Level of
Indicators Sample
reflection
● The answer shows no evidence
of the student attempting to
understand the concept or theory
“Company structure and speaker’s experiences,
underpinning the topic.
improve job application, learn marketing course,
1 Non-reflection ● Material has been placed into an
prepare LinkedIn profile, and improve teamwork”
essay without the student
— Student 11 (week 2)
thinking seriously about it, trying
to interpret the material, or
forming a view.
257
● Largely reproduction, with or

without adaptation, of the work of
others.
“I have learned communication, negotiation,

● Evidence of an understanding of pursue the customers. I practiced buying goods,
defining prices, finding new customers, pursuing,
a concept or topic.
etc. It must show clear original, suitable price, nice
● Material is confined to theory.
attitude to pursue and sales products. In general, I
● Reliance upon what was in the
2 Understanding have achieved a good result. The limitations include
textbook or the lecture notes.
not finding a cheaper product, improving
● Theory is unrelated to personal
communication with strangers, and negotiating
experiences, real-life applications, capabilities. I am to complete the current course to
or practical situations. accumulate the knowledge and experience.” —
Student 14 (week 4)
● Theory is applied to practical “I have learned […]. In addition, I discussed this
situations. with my classmate […]. At this time, I can identify
● Situations encountered in the mistakes in my personal CV. However, I am
practice will be considered and still not proactive during the discussion. […] I also
3 Reflection
successfully discussed according lack the skills for sales relevant position at the
to what has been taught. There company. […] In the future, I will try to improve
will be personal insights that go my English capacity, communication skills, and
beyond book theory. sales […] — Student 15 (week 1)
During the program, I learned proactively (asking
questions, responding to the questions, taking
notes, completing all assignments, being open-
minded with friends, presenting group projects,
etc.) […] After the program, my strengths are
confidence to discuss with strangers, brave to try
the new things. […] My weakness is postponing the
● Evidence of a change in deadline. I have accumulated fundamental
perspective over a fundamental knowledge about commercial and selling
Critical belief of understanding a key biotechnology products and professional skills to
4
reflection concept or phenomenon. become a salesperson in life science. I also recognize
● Critical reflection is unlikely to that becoming a technical sale is not irrelevant to
occur frequently. the major. Before the program, I only knew about
being a lab technician or working in a lab position
for biotechnology employment. […] I have reached
all the learning objectives which I aimed to. In some
ways, the program is beyond my
estimation/expectation. I will apply the knowledge
and skills which I have learned in my future
career…” — Student 38 (week 8)
258

Vol. 22, No. 7, pp. 258-280, July 2023
An Investigation into the Benefits and

Challenges of International Student Exchange
Programs: Perspectives from Student Teachers
Huynh Thi Thuy Diem* , Mai Phuc Thinh
and Tran Thi Mung
Can Tho University, Can Tho City, Vietnam
Abstract. This study explores the perspectives of student teachers from

Can Tho University, Vietnam, on the benefits and challenges of engaging
in student exchange programs. Utilizing a quantitative research
approach, data was collected through a survey instrument consisting of
multiple-choice and Likert-scale questions. The survey examined student
teachers' perceptions of various advantages, including access to new
knowledge, enhanced teaching skills, and valuable support from foreign
partners and mentors. Additionally, it investigated challenges related to
language differences, cultural norms, and time management. The
findings shed light on the experiences of student teachers in exchange
programs and reveal that one of the most significant reasons for their
participation was the desire to expand their international networks and
foster meaningful connections with students from other countries. These
insights provide valuable guidance for educational institutions in
developing effective strategies and support systems to optimize the
participation and satisfaction of student teachers in these programs.
Keywords: international student exchange; exchange programs;

internationalization
1. Introduction
In today's globalized labor markets, having a university degree is no longer
enough to guarantee employment. The rapid growth in the number of university
graduates has led to a decline in the perceived quality and value of a degree, as
educational credentials have become more common and accessible (Tomlinson,
2008). As a result, employers are increasingly seeking graduates with additional
employability skills, such as communication, problem-solving, and teamwork, to
enhance their chances of securing meaningful employment (Gale & Parker, 2017;
Nilsson & Ripmeester, 2016).
*Corrresponding author: Huynh Thi Thuy Diem, httdiem@ctu.edu.vn
©Authors
259
Vietnam, a country in Southeast Asia, recognizes the urgent need for higher
education internationalization and cooperation to meet the demands of a rapidly
evolving job market. With the increasing interconnectedness of the world,
Vietnamese higher education institutions are actively engaged in efforts to
enhance the quality of global learning experiences and improve job opportunities
for their students (Phan et al., 2019). To address these demands, international
student exchange programs have emerged as a powerful tool for providing
students with firsthand exposure to global markets and facilitating interactions in
diverse cultural contexts (Ritz, 2011). International student exchange programs
refer to structured educational initiatives that facilitate the exchange of students
between institutions in different countries, allowing them to study, work, or
engage in cultural immersion experiences abroad (Knight, 2012). In the context of
this study, international exchange programs for student teachers at Can Tho
University are short-term teaching exchanges facilitated by partner international
universities.
International exchange programs aim to promote cross-cultural understanding,
enhance participants' academic and professional skills, and foster global
citizenship. The benefits of participating in international student exchange
programs are manifold. Students who engage in these programs gain access to
study and work opportunities that enhance their professional competencies and
provide exposure to international environments (Wright & Cluke, 2010). They
acquire real-world career experience and secure positions related to international
affairs, effectively equipping themselves for the global job market (Di Pietro,
2015). Moreover, immersion in native cultures and local life during exchange
programs allows students to reflect on their experiences, gain new insights, and
undergo personal growth (Curtis & Ledgerwood, 2018).
However, while international student exchange programs present valuable
opportunities, they also pose challenges for Vietnamese students. Language
barriers, unfamiliar learning environments, and cultural differences can pose
significant obstacles to their adjustment and academic success (Nguyen, 2011).
Therefore, it is essential to examine and address the specific challenges and
opportunities associated with these exchange programs in the context of Vietnam.
The objective of this journal article is to examine the experiences of student
teachers who participate in international student exchange programs at Can Tho
University, a prominent higher education institution in Vietnam. Our focus is on
capturing the subjective experiences, perceived benefits, and challenges faced by
student teachers during their participation in these programs. By delving into the
specific context of Can Tho University, we seek to provide valuable insights that
contribute to the existing knowledge base and enhance our understanding of the
effects of international student exchange programs on student teachers.
This study is important and relevant for several reasons. Firstly, it contributes to
the field of international education by exploring the impact of exchange programs
on a specific group of students, namely student teachers. While previous research
has investigated the effects of internationalization on students in various
disciplines, there is a paucity of literature specifically examining the experiences
and outcomes of student teachers in the context of international exchange
programs. Therefore, this study fills a gap in the existing research and provides a
260
deeper understanding of the unique challenges and opportunities faced by

student teachers in international exchange programs.
Secondly, this study is relevant to the field of teacher training. As teacher
education becomes increasingly globalized, it is crucial to examine how
international experiences shape the professional development of student teachers.
Research has shown that participation in international exchange programs can
have a positive impact on student teachers by expanding their cultural awareness,
developing intercultural competencies, and enhancing their ability to effectively
teach in diverse and multicultural classrooms (Goodwin, 2020). Understanding
the specific impact of international exchange programs on student teachers can
inform the development of teacher training programs that better prepare future
educators. By integrating international experiences and incorporating strategies
to address the challenges faced during these programs, teacher training programs
can effectively equip student teachers with the necessary skills and knowledge to
promote inclusive and culturally responsive teaching practices (Schleicher, 2012).
This study aims to explore the benefits and challenges associated with
international student exchange programs for student teachers at Can Tho
University, in order to contribute to the enhancement of teacher training strategies
and curricula. To address these objectives, the article aims to answer the following
research questions:
- What are the perceived benefits of international student exchange programs for
student teachers at Can Tho University?
- What challenges do student teachers face during their participation in
international exchange programs?
By addressing these research questions, the article aims to provide valuable
insights into the experiences of international student exchange programs on
student teachers at Can Tho University, thereby contributing to the fields of
international education and teacher training.
The Role of International Student Exchange Programs in Education
In recent years, the globalization of education has become a prominent
phenomenon, driven by advancements in technology, increased mobility, and the
growing interconnectivity of nations. It has brought about significant changes in
the way education is perceived and delivered, transcending national borders and
fostering a globalized learning environment (Berg & Schwander, 2019).
International student exchange programs have emerged as a crucial component
of the global education landscape. These programs facilitate the movement of
students across countries, allowing them to immerse themselves in different
cultural, academic, and social contexts (Goldstein, 2022). By participating in such
exchange programs, students gain unique opportunities for personal growth,
academic enrichment, and the development of global competencies (Iskhakova et
al., 2022; Atalar, 2020).
International student exchange programs play a vital role in promoting cross-
cultural understanding and educational collaboration. They provide students
with a platform to engage with diverse perspectives, traditions, and values,
261
fostering intercultural dialogue and mutual respect (Whatley et al, 2021). Through
these exchanges, students are exposed to different educational systems, teaching
methods, and research environments, broadening their horizons and expanding
their academic repertoire.
Furthermore, international student exchange programs serve as catalysts for
international collaboration and cooperation among educational institutions. They
facilitate the exchange of knowledge, ideas, and best practices, encouraging
partnerships between universities and promoting research collaborations on a
global scale (De Wit & Altbach, 2021). Such collaborations contribute to the
advancement of scientific research, innovation, and the dissemination of
knowledge across borders.
Benefits of Participating in International Student Exchange Programs

Participating in international student exchange programs offers numerous
benefits to students, enhancing their skills and employability in an increasingly
globalized and interconnected world. These programs provide students with
firsthand exposure to diverse cultures, traditions, and perspectives, fostering
cultural competence and global awareness (Nilsson & Ripmeester, 2016; Teichler
& Janson, 2007). Through immersion in a foreign language environment, students
have the opportunity to improve their language skills, enhancing their fluency,
vocabulary, and overall language proficiency (Neumann et al., 2019). The
exposure to different educational systems, teaching methods, and research
environments broadens students' academic horizons, fosters critical thinking
skills, and encourages intellectual flexibility (Loes & An, 2023).
In addition to academic growth, international student exchange programs
promote personal development and independence. Students are required to step
out of their comfort zones and adapt to new cultural, social, and academic
environments, fostering valuable life skills such as problem-solving, adaptability,
and intercultural communication (Berg & Schwander, 2019; Mikulec, 2019). The
programs also provide opportunities for students to build an extensive network
of international contacts, fostering the development of lifelong friendships,
international connections, and professional networks (McManus, 2019).
Participating in international student exchange programs significantly enhances
students' skills and employability in a global job market (Fakunle, 2021). The
programs develop strong intercultural communication skills and the ability to
collaborate effectively with individuals from different cultural backgrounds
which are highly valued in today's globalized workplaces (Roy et al., 2019).
Students also gain adaptability, resilience, and a global perspective, enabling
them to navigate diverse and unpredictable settings (Hua et al., 2019). Language
proficiency gained through immersion in a foreign language environment
enhances students' competitiveness and opens up a wider range of career
opportunities in international organizations and multinational companies
(Fakunle, 2021). The experience of studying abroad through international student
exchange programs demonstrates students' global mindset, adaptability, and
openness to diverse perspectives, making them attractive candidates for
employers seeking globally minded individuals (Gil & Reyes, 2020).
262
In conclusion, international student exchange programs provide students with a

myriad of benefits, including the development of cultural competence, language
proficiency, academic growth, personal development, and the enhancement of
skills and employability. These programs play a vital role in preparing students
for a globalized world by equipping them with the necessary skills and
experiences to thrive in diverse and interconnected environments.
International Student Exchange Programs in the Vietnamese Context

In the context of Vietnam, the urgent need for international cooperation and
higher education internationalization is driven by the demand for high-quality
labor in today's fast-paced world (Nguyen et al, 2016). Recognizing the
importance of internationalization, Vietnamese higher education institutions have
actively engaged in efforts to enhance the quality of global learning experiences,
integrate into the global arena, and improve job opportunities for students after
graduation (Le, 2016).
International student exchange programs have proven to be beneficial for
Vietnamese students, offering them opportunities to develop professional skills,
improve English proficiency and access quality education (Huong et al, 2017). In
other words, through international exchanges, Vietnamese students can gain
exposure to different educational systems, teaching methods, and research
environments, enabling them to broaden their perspectives and acquire
knowledge and skills that are applicable in their home country (Brisset et al, 2010).
However, international student exchange programs also present challenges for
Vietnamese students, such as language barriers, unfamiliar learning
environments, and cultural differences (Pham & Saltmarsh, 2013). Language
proficiency, particularly in English, is often a significant hurdle for Vietnamese
students, as it can impact their academic performance and social integration in the
host country (Huong et al., 2017). Moreover, adjusting to a new educational
system, different teaching methods, and cultural norms requires adaptability and
resilience on the part of Vietnamese students (Huong et al., 2017).
Given the unique challenges and opportunities associated with international
exchange programs in Vietnam, it is crucial to examine and address these factors
to ensure the effectiveness and success of such programs. By understanding the
specific needs and concerns of Vietnamese students participating in international
student exchange programs, educational institutions and policymakers can
develop targeted support mechanisms, language support programs, and cultural
orientation initiatives to maximize the benefits and minimize the challenges faced
by Vietnamese students.
3. Methods
Participants
The participants in this study were 56 student teachers who had participated in at
least one international student exchange program at Can Tho University. The
sampling technique employed in this study was convenience sampling.
Convenience sampling is a non-probability sampling method where participants
are selected based on their availability and willingness to participate. In this case,
the participants were selected from the pool of student teachers who had
263
previously participated in international student exchange programs at Can Tho

University. Due to the relatively new implementation of international student
exchange programs at Can Tho University, the number of participants available
for the study was limited. Additionally, some participants had already graduated,
and their contact information was no longer available. To maximize the
participation rate and ensure a diverse representation, the research team
collaborated with the Office of International Affairs and relevant faculties and
departments to obtain a list of student teachers who had participated in the
programs. Using the provided list, the research team contacted the participants
via email. The email invitations explained the purpose of the research,
emphasized the significance of their participation, and provided a link to the
online survey questionnaire. By using convenience sampling, the research team
aimed to include a range of participants from different cohorts who had varying
experiences and perspectives regarding international student exchange programs.
Data Collection
Data collection for this study was conducted through the administration of a
survey questionnaire distributed to the participants via email. The questionnaire
was delivered electronically to the participants between July and August 2022,
allowing them sufficient time to provide thoughtful responses. To ensure a
smooth and efficient delivery process, the contact information of the participants
was obtained from the Office of International Affairs at Can Tho University. This
office maintains a comprehensive list of students who have participated in
international student exchange programs. Utilizing this list, the questionnaire was
sent directly to the participants' email addresses.
To enhance the questionnaire's reliability, several steps were taken. First, the
questionnaire was developed based on an extensive review of the existing
literature on international student exchange programs and their impact on
student teachers. This helped ensure that the survey items aligned with
established constructs and concepts in the field. Second, the questionnaire
included a mix of multiple-choice and Likert scale questions. The multiple-choice
questions provided participants with predefined response options, allowing for
quantitative analysis of specific aspects. The Likert scale questions, on the other
hand, employed a 5-point scale, ranging from 1 (Strongly Disagree) to 5 (Strongly
Agree), enabling participants to express their level of agreement or disagreement
with various statements. This allowed for a more nuanced understanding of the
participants' perceptions and experiences. Moreover, the questionnaire
underwent a rigorous pilot testing phase involving a small group of student
teachers and lecturers who provided feedback on the clarity, relevance, and
comprehensiveness of the questions. This iterative process helped refine the
questionnaire and ensure its reliability.
Data Analysis
The collected data were analyzed using the SPSS for Window 16.0 software. The
analysis involved both descriptive and inferential statistical techniques. For
descriptive analysis, the SPSS for Windows 16.0 software was utilized. Descriptive
statistics, including measures such as the mean (average score) and standard
deviation, were calculated to summarize the data collected from the survey
264
questionnaire. The mean score provided an indication of the participants' overall

level of agreement or disagreement with the statements presented in the Likert
scale. The standard deviation helped assess the degree of variability or dispersion
within the responses. Specifically, to calculate the mean score, the Likert scale
responses were assigned numerical values ranging from 1 (Strongly Disagree) to
5 (Strongly Agree). The average score was then determined by summing up all
the responses and dividing the total by the number of participants. The range of
the scale (0.8) was computed using the formula (Maximum - Minimum)/n, where
the maximum value was 5 and the minimum value was 1, and n represented the
number of response options.
4. Results
Demographic Description
The demographic description of the participants in the study revealed interesting
insights. The study included a total of male and female participants, with males
comprising 57.1% and females accounting for 42.9% of the total sample. The
participants were enrolled in various international exchange programs, each
varying in duration. The program duration distribution among the participants
was as follows: 32.1% of participants had a program duration of 1 week or less,
42.8% had a duration of over 1 week to 2 weeks, 16.1% had a duration of over 2
weeks to 4 weeks, and 9% had a program duration of over 1 month.
Table 1. Demographic Descriptions of Participants
Number of
Percentage
respondents
Characteristics
n %
Gender
Male 32 57.1
Female 24 42.9
Program duration
1 week or less 18 32.1
Over 1 week to 2 weeks 24 42.8
2 weeks to 4 weeks 9 16.1
Over 1 month 5 9.0
Exchange destinations
Thailand 44 78.6
Taiwan 5 8.9
Korea 3 5.3
Japan 3 5.3
Czech 1 1.9
265
In terms of exchange destinations, the majority of participants (78.6%) traveled to

Thailand for their international exchange program. Other destinations included
Taiwan (8.9%), Korea (5.3%), Japan (5.3%), and Czech (1.9%). Notably, the focus
of the exchange programs primarily revolved around Asian countries, with
Thailand being the most popular destination.
It is important to highlight that all of the participants in the study had only
engaged in an international exchange program once. This indicates that the
participants had limited prior experience with international exchange programs,
making their perspectives and experiences particularly valuable for the study.
Students’ Motivations to Participate in International Exchange Programs

Table 2 provides valuable insights into the reasons behind students' participation
in international exchange programs. The data revealed several key motivations
that influenced the students' decision to take part in these programs.
Table 2. Reasons for students’ participation in international exchange programs
Reasons Percentage (%)
1. Recommendation from the faculty advisor 50.0
2. Expansion of relationships with international students 89.3
3. Desire to improve language skills 85.7
4. Promotion of Vietnamese culture 60.7
5. Sharing and discussion of topics related to their field of study 67.9
One prominent factor that influenced students' participation was the

recommendation by their faculty advisor. A significant percentage of students
(50.0%) indicated that they were encouraged to participate in the exchange
programs based on the advice and guidance of their academic mentors. This
highlighted the influential role that faculty members played in shaping students'
decisions and promoting the value of international experiences.
Another strong motivation for students to participate in international exchange
programs was the desire to expand relationships with international students. An
overwhelming majority (89.3%) expressed their eagerness to connect with
students from different countries, reflecting a genuine interest in fostering cross-
cultural understanding and establishing global networks. The opportunity to
interact with peers from diverse backgrounds offered students the chance to
broaden their perspectives, develop intercultural communication skills, and build
lifelong friendships.
Improving language skills was identified as a significant motivation for
participation, with 85.7% of students indicating their desire to enhance their
language proficiency through immersion in a foreign language environment.
Recognizing the importance of language competency in today's globalized world,
students acknowledged the international exchange program as a valuable
opportunity to sharpen their language abilities, particularly in international
266
languages such as English. The program provided an immersive setting where

students could practice their language skills and gain confidence in
communicating effectively with people from different linguistic backgrounds.
Promoting Vietnamese culture emerged as another motivation for students to
participate in international exchange programs, with 60.7% expressing a desire to
showcase their country's traditions, customs, and values to international peers.
This reflects a sense of cultural pride and a commitment to representing Vietnam
on a global stage. By sharing their unique cultural heritage, students contributed
to the promotion of cross-cultural understanding and appreciation among their
international counterparts.
Furthermore, a considerable proportion of students (67.9%) expressed their
interest in engaging in academic discussions and knowledge sharing related to
their field of study during the international exchange program. This demonstrates
their eagerness to expand their knowledge, gain new insights, and explore
different perspectives within their chosen academic discipline. The opportunity
to engage with experts, attend specialized courses, and collaborate with
international peers could greatly enrich students' academic experience and
contribute to their personal and professional growth.
In conclusion, the analysis of Table 2 highlights a range of motivations that drive
students to participate in international exchange programs. These motivations
include the recommendation of faculty advisors, the desire to establish
international relationships, the aspiration to improve language skills, the interest
in promoting Vietnamese culture, and the eagerness to engage in academic
discussions. Understanding these motivations is crucial for program organizers
and educators to design meaningful and impactful international exchange
experiences that meet the needs and expectations of participating students.
Perceived Benefits of International Student Exchange Programs

According to Table 3, international student exchange programs offer a range of
benefits to participants, contributing to their personal growth, skills development,
and global perspective. The following groups summarize the similar benefits
reported by the participants, along with their mean ratings and standard
deviations.
267
Table 3. The benefits of international student exchange programs from students’

perspectives
Category Mean SD Ranking
Acquisition of independent living skills 4.21 1.258 1
Global perspective development 4.21 1.287 1
Improved critical thinking abilities 4.14 1.239 2
English language skills progression 4.07 1.215 3
Self-evaluation and self-improvement 4.04 1.232 4
Integration into a multicultural environment 4.04 1.232 4
Cultural understanding 4.00 1.186 5
International network building 4.00 1.217 5
Teamwork skills enhancement 4.00 1.217 5
Acquisition of valuable future work skills 3.93 1.215 6
Growth in maturity and confidence 3.89 1.227 7
Preparedness to become a global citizen 3.89 1.227 7
Easy adaptation to new environments 3.79 1.258 8
Personal Growth and Skills Development

International student exchange programs offered several benefits that contributed
to personal growth and skills development. These benefits were determined
based on mean ratings and standard deviations provided by the participants,
reflecting their experiences and perceptions.
Acquisition of independent living skills emerged as a significant benefit, with a
high mean rating of 4.21. This aspect encompassed the development of essential
skills required for living independently in a foreign country. Participants had the
opportunity to enhance their abilities in managing daily tasks, adapting to new
environments, and navigating diverse cultural norms. The relatively low standard
deviation of 1.258 indicated that a general agreement among participants
regarding the importance of acquiring these skills.
Similarly, the development of a global perspective received a mean rating of 4.21,
indicating its significance in international student exchange programs. This
benefit encompassed the ability to adopt a broader worldview, comprehend
global issues, and approach work from a global standpoint. The relatively low
standard deviation of 1.287 showed a general consensus among participants
regarding the value of cultivating a global perspective.
Improvement in critical thinking abilities was also identified as a notable benefit,
with a mean rating of 4.14. This aspect highlighted the enhancement of critical
thinking skills, enabling participants to analyze and evaluate information from
diverse cultural contexts. While there may be some variation in individual
experiences, the overall acknowledgment of the importance of this skill
development is evident.
Language skills acquisition and progression, with a mean rating of 4.07, emerged
as another valuable benefit. Participants had the opportunity to learn and improve
their vocabulary and English language proficiency, which were essential for
268
effective communication in an international setting. The standard deviation of

1.215 indicated some variation in individual experiences, but overall, participants
recognized the significance of language skills in their exchange program journey.
In addition, opportunities for self-evaluation and self-improvement were highly
valued, with a mean rating of 4.04. This benefit provided participants with the
chance to reflect on their strengths and areas for growth, fostering personal
development and self-awareness. While there might be some variation in
individual experiences, the recognition of these opportunities as valuable was
evident.
The personal growth and skills development offered by international student
exchange programs empowered participants to acquire essential life skills,
develop a global perspective, enhance critical thinking abilities, and improve
language proficiency. These experiences facilitated personal growth, cultural
understanding, and readiness to navigate an interconnected world. The relatively
low standard deviations indicated a general consensus among participants,
emphasizing the significance of these benefits within the context of international
student exchange programs.
Cultural Understanding and International Connections

Cultural understanding, and international connections were two closely
intertwined dimensions that held great importance for students participating in
international student exchange programs. These dimensions emphasize the value
of engaging with different cultures, developing cultural competence, and building
meaningful friendships with individuals from diverse backgrounds.
Cultural understanding received a mean rating of 4.00, indicating its significant
impact on participants. This dimension encompassed the opportunity for students
to immerse themselves in a different culture, gaining a deeper understanding of
its customs, traditions, and perspectives. By actively engaging in the local
community and experiencing daily life in a foreign country, students broadened
their horizons and developed a more global perspective. The relatively low
standard deviation of 1.186 indicated a general consensus among the participants
regarding the value of cultural immersion and understanding. This dimension
ranked 5th among the listed benefits.
International network building, with a mean rating of 4.00, highlights the
importance of forming friendships with students from different countries. By
building meaningful relationships, students create a network of international
contacts and expand their intercultural communication skills. These connections
foster a sense of global citizenship, promote cross-cultural understanding, and
provide opportunities for future collaborations. The standard deviation of 1.217
suggested some variations in individual experiences, but overall, the participants
recognize the significance of international friendship building. This dimension
also ranked 5th among the listed benefits.
Together, these two dimensions underlined the transformative power of
international student exchange programs. Through cultural immersion and
understanding, students developed a broader perspective, enhanced cultural
competence, and a deeper appreciation for diversity. Simultaneously,
269
international connections and friendship building contributed to students'

personal growth, global citizenship, and the formation of lifelong relationships.
These dimensions reflected the richness and transformative potential of
international student exchange programs in fostering intercultural understanding
and promoting global engagement.
Adaptability and Multicultural Integration

Adaptability and multicultural integration were key aspects of international
student exchange programs that contributed to a transformative experience for
participants. These benefits, as indicated by mean ratings and standard
deviations, reflected the importance of navigating new environments and
integrating into diverse cultural contexts.
The ability to easily adapt to new environments received a mean rating of 3.79
with a standard deviation of 1.258, indicating its relevance in international student
exchange programs. This theme emphasized the importance of being flexible and
open-minded when confronted with unfamiliar cultural and social environments.
Participants who could quickly adapt to new situations were better equipped to
navigate the challenges and opportunities that arose during their exchange
program. While there might be some variation in individual experiences, the
overall recognition of the value of adaptability was evident.
Integration into a multicultural environment received a mean rating of 4.04 with
a standard deviation of 1.232, highlighting its significant role in international
student exchange programs. This benefit encompassed the ability to engage and
connect effectively within a diverse cultural setting, fostering cross-cultural
communication and understanding. Participants who actively sought
opportunities to engage with individuals from different backgrounds could
broaden their perspectives and gained a deeper appreciation for cultural
diversity. Integration into a multicultural environment enabled participants to
develop intercultural competence and navigate the complexities of a globalized
world. This benefit ranked 4th among the listed benefits, indicating its importance
to the overall exchange program experience.
The relatively low standard deviations associated with both benefits indicated a
general consensus among participants regarding their importance, with limited
variation in individual ratings. This consensus underscored the significance of
adaptability and multicultural integration in creating a supportive and inclusive
environment for participants, where they could thrive and engage with the host
culture and community effectively.
Future career preparation and confidence

In the context of international student exchange programs, two significant aspects
emerged, future career preparation and personal growth and confidence. These
aspects represented the multifaceted benefits that students derived from
participating in such programs.
Future career preparation entailed the acquisition of valuable future work skills.
The mean rating for the acquisition of valuable future work skills was 3.93,
indicating its significance in preparing students for their future careers. This
component encompassed the development of skills highly sought after by
270
employers, such as teamwork, problem-solving, and intercultural

communication. The relatively low standard deviation of 1.215 suggested a
consensus among the participants regarding the importance of future career
preparation. This benefit ranked 6th among the listed benefits.
Parallel to future career preparation, personal growth and confidence were
integral to international student exchange programs. Students experienced
growth in maturity and confidence, as reflected by a mean rating of 3.89. This
aspect highlighted the transformative journey students embarked on during their
exchange programs, leading to increased self-assurance and a broader perspective
on their capabilities. The standard deviation of 1.227 showed some variation in
individual experiences within this aspect. Preparedness to become a global citizen
also received a mean rating of 3.89. This component signified students' readiness
to engage with global issues, contribute to international communities, and
embrace a global perspective. Both personal growth and confidence elements
ranked 7th among the listed benefits.
In summary, the analysis of the data revealed the significant impact of
international student exchange programs on future career preparation and
personal growth and confidence. These programs provided students with the
opportunity to acquire valuable skills for their future careers and undergo a
transformative personal journey. By participating in these programs, students
developed the necessary skills and mindset to succeed in the globalized world
while gaining newfound self-assurance and a broader perspective.
Challenges Faced by Students in International Student Exchange Programs

International student exchange programs offered invaluable opportunities for
students to immerse themselves in new cultures, gain academic and professional
experiences abroad, and broaden their horizons. However, such programs were
not without their challenges. Table 4 indicates the various challenges that students
often encounter during their international student exchange journeys. These
challenges can be grouped into several topics, each representing a distinct aspect
of the exchange experience.
Table 4. Challenges of Participating in International Student Exchange Programs
Challenges Percentage (%)
1. Language barrier 67.9

2. Weather conditions 32.1
3. Cultural customs and traditions 17.9
4. Working methods 21.4
5. Working hours 25.0
6. Food 25.0
7. Accommodation 10.7
8. Workload 14.3
271
9. Lack of expertise in teaching or working 75.0

10. Lack of new knowledge abroad 21.4
11. Insufficient teaching skills 25.0
12. Inadequate curriculum content 67.9
13. Lack of interaction between teachers and students in class 14.3
14. Lack of skills for laboratory work 28.6
15. Inadequate lesson planning 75.0
16. Limited interaction between teachers and students in class 89.3
Note: N = 56
Cultural and Practical Adjustments

One of the primary challenges faced by students in international student exchange
programs is adapting to new cultural customs, traditions, and practical aspects of
daily life. Students may initially experience a language barrier, making it
challenging to communicate and fully engage with the local community and
academic environment. This challenge (67.9%) highlighted the importance of
language preparation and ongoing language support throughout the exchange
program. Additionally, students might encounter differences in weather
conditions (32.1%), which could impact their daily routines and require
adjustment. Understanding and adapting to new cultural norms, social etiquettes,
and local customs were also essential for students to feel integrated and
comfortable in their host country.
Academic and Professional Challenges

Another set of challenges revolved around the academic and professional aspects
of the exchange program. Students might encounter difficulties with working
methods (21.4%), such as different approaches to research, assignments, or
classroom participation. Adjusting to new working hours (25.0%) and managing
a potentially higher workload (14.3%) could also pose challenges. Furthermore,
students might face the challenge of lacking expertise in teaching or working
(75.0%), as they navigate unfamiliar educational systems and professional
environments. Adapting to different teaching styles, curriculum content, and
assessment methods could be demanding. Moreover, limited interaction between
teachers and students in class (89.3%) could hinder effective learning and
engagement. These challenges highlighted the need for students to be proactive
in seeking academic support, building relationships with professors, and
embracing a growth mindset to navigate these academic and professional
obstacles successfully.
Accommodation and Practicalities

Students might also face challenges related to accommodation (10.7%) and
practicalities associated with their stay abroad. Finding suitable housing, dealing
with contracts, and managing expenses in a foreign currency could be daunting.
Navigating local transportation systems and understanding the logistics of daily
life, such as grocery shopping or accessing healthcare services, may also present
272
challenges. Overcoming these challenges requires students to be resourceful, seek

guidance from program coordinators or local resources, and develop a sense of
independence and adaptability.
Laboratory and Technical Skills

For students engaged in scientific or technical fields, challenges might arise in
laboratory work. Around 28.6% of students reported facing challenges related to
the lack of skills for laboratory work. This highlighted the need for adequate
training and support in acquiring the necessary technical competencies for
conducting experiments and research. Institutions should provide comprehensive
orientation and training programs to ensure students were equipped with the
required laboratory skills and safety knowledge.
Participating in international student exchange programs offers numerous
benefits, but it also presents various challenges for students. These challenges
encompass cultural adjustments, academic and professional obstacles,
accommodation and practicalities, and specific technical skill requirements.
Understanding and anticipating these challenges could help students better
prepare and equip themselves for a successful and rewarding exchange
experience. Moreover, institutions and program coordinators should provide
adequate support, resources, and guidance to help students navigate these
challenges and make the most of their international student exchange journey. By
addressing these challenges proactively, students can overcome barriers, enhance
their cross-cultural competencies, and foster personal and academic growth
during their time abroad.
5. Discussion
The findings of this study shed light on the motivations, benefits, and challenges
associated with international student exchange programs, particularly in the
context of student teachers and their teaching experience. These findings provide
valuable insights that can be compared and contrasted with the existing body of
literature on the subject.
The motivations identified in this study align with previous research, which
emphasizes the value of international exchanges in promoting cross-cultural
understanding and academic enrichment (Goldstein, 2022; Iskhakova et al., 2022).
Participants expressed a desire to expand relationships with international
students, improve language skills, promote their own culture, and engage in
academic discussions related to their field of study. These motivations reflect the
students' eagerness to embrace diverse perspectives, foster intercultural dialogue,
and develop global competencies, which is consistent with the literature.
Similarly, the benefits reported by participants in this study align with the
documented advantages of international student exchange programs. The
acquisition of independent living skills, development of a global perspective,
improvement in critical thinking abilities, and progression of language skills were
among the benefits identified (Nilsson & Ripmeester, 2016; Teichler & Janson,
2007). These findings support the notion that international exchanges contribute
to personal growth, cultural understanding, and the acquisition of valuable skills
that enhance employability in a globalized job market.
273
However, the study also uncovered new insights specific to the context of student
teachers and their teaching experience during international exchanges. The
challenges identified, such as the lack of expertise in teaching or working,
insufficient teaching skills, and limited interaction between teachers and students
in class, highlight the unique difficulties faced by student teachers in navigating
foreign educational systems. These challenges highlight the need for tailored
support mechanisms, language support programs, and cultural orientation
initiatives that address the specific needs of student teachers. The challenges
identified in this study are consistent with the existing literature on the obstacles
faced by students participating in international exchange programs (Pham &
Saltmarsh, 2013; Huong et al., 2017). Language barriers, cultural customs and
traditions, and academic adjustments have been previously highlighted as
common challenges. The current study reinforces these findings and underscores
their relevance in the context of student teachers and their teaching experience
abroad.
By integrating the study’s findings with the existing literature, we can gain a more
comprehensive understanding of the complexities and outcomes of international
student exchange programs, particularly within the context of student teachers at
Can Tho University. Can Tho University holds a prominent position as a leading
institution in training human resources for the Mekong Delta region, an area that
faces numerous disadvantages and challenges. Therefore, examining the
experiences and perspectives of student teachers in international exchange
programs becomes crucial in addressing the specific needs of this region. The
findings highlight the transformative nature of international exchanges for
student teachers, aligning with previous research that emphasizes the personal
and professional development outcomes of participating in such programs. The
acquisition of independent living skills, the development of a global perspective,
improvement in critical thinking abilities, and enhancement of language
proficiency are all vital for student teachers who will play a crucial role in shaping
the education system of the Mekong Delta region. These outcomes contribute to
their preparedness to become global citizens and navigate diverse and
interconnected educational environments.
Recommendations for Universities and Policymakers

Based on the findings and the specific context of Can Tho University and the
Mekong Delta region, the following recommendations and strategies can be
integrated to ensure the effectiveness and success of international exchange
programs for student teachers:
• Clear policies and guidelines: Establish comprehensive policies and
guidelines that outline the objectives, procedures, and expectations of
international exchange programs. Include provisions for integrating
international exchange experiences into the curriculum.
• Expansion of partner institutions and collaboration: Actively collaborate
with a diverse range of partner institutions worldwide to offer a broader
range of academic programs and destinations. Develop joint academic
programs, research opportunities, and cultural initiatives.
• Financial support and scholarships: Allocate sufficient funding and
resources to support student teachers financially during their exchange
274
period. Establish scholarships, grants, and other financial aid options

specifically for student teachers participating in international exchanges.
• Pre-departure Orientation and Ongoing Support: Provide comprehensive
pre-departure orientation programs that prepare student teachers for the
challenges they may encounter during their exchange program. Establish
robust support systems with dedicated staff members to assist student
teachers throughout their international exchange journey.
• Promotion and Awareness: Actively promote international exchange
programs to student teachers through various channels. Emphasize the
integration of international exchange programs into the curriculum.
Limitations of the Study

While this study provides valuable insights into the motivations, perceived
benefits, and challenges of international student exchange programs for student
teachers at Can Tho University, there are several limitations that should be
acknowledged. First, the sample size and representation may limit the
generalizability of the findings. The study focused on a specific group of student
teachers from a single university, which may not fully capture the experiences and
perspectives of student teachers from other institutions or regions.
Second, the reliance on self-reporting measures introduces the possibility of
response bias. Participants may have provided socially desirable responses or
may not have accurately recalled their experiences. Future studies could employ
mixed-method approaches, incorporating qualitative methods such as interviews
or focus groups, to gain a deeper understanding of the nuances and complexities
of students' experiences in international exchange programs.
Furthermore, the study's findings are specific to the context of Can Tho University
and the Mekong Delta region. The cultural and institutional factors unique to this
context may influence students' motivations, experiences, and challenges in
international exchange programs. Therefore, caution should be exercised when
generalizing the findings to other contexts.
6. Conclusion
In conclusion, this study sheds light on the motivations, perceived benefits, and
challenges faced by student teachers participating in international student
exchange programs at Can Tho University. The findings highlight the importance
of these programs in promoting cultural understanding, personal growth, and
academic development among student teachers.
Despite the limitations, this research emphasizes the transformative potential of
international exchange programs for student teachers. It underscores the need for
universities and policymakers to prioritize and support these programs to
enhance students' experiences and outcomes. By establishing clear policies and
guidelines, expanding partner institutions, providing financial support, and
offering comprehensive pre-departure orientation and ongoing support,
universities can create an environment conducive to successful international
exchange experiences for student teachers.
Moving forward, future research should aim to address the limitations of this
study by employing larger and more diverse samples, incorporating qualitative
275
methods, and considering the specific cultural and institutional factors at play.
This will contribute to a more comprehensive understanding of students’
experiences in international exchange programs and inform the development of
targeted interventions and support mechanisms.
As Nelson Mandela once said, “Education is the most powerful weapon which
you can use to change the world”. International student exchange programs have
the potential to equip student teachers with the necessary tools and perspectives
to make a positive impact in their future classrooms and beyond. By fostering
intercultural understanding, personal growth, and academic advancement, these
programs contribute to the development of student teachers as global citizens
who can actively engage in a diverse and interconnected world. It is imperative
that universities and policymakers continue to prioritize and support
international exchange programs to cultivate a generation of educators who are
prepared to navigate the complexities of our global society.
Acknowledgements
The authors would like to express their sincere gratitude to the lecturers and
students who participated in the pilot questionnaire and provided valuable
feedback, which greatly contributed to the refinement of the research instrument.
The support received from the Office of International Affairs and the School of
Education at Can Tho University is also acknowledged with appreciation.
Special thanks are extended to all the former exchange student teachers who took
the time to respond to the questionnaire. Their valuable insights, experiences, and
perspectives were invaluable in shaping the findings and conclusions of this
research. Their willingness to share their experiences and provide feedback is
greatly appreciated.
Furthermore, the authors acknowledge the partial funding provided by Can Tho
University through the Research Project with code T2023-68. This financial
support was instrumental in covering the research expenses and ensuring the
smooth progress of the study.
The authors would also like to extend their gratitude to all individuals who have
contributed to the development and completion of this research in various ways.
Their support, guidance, and encouragement throughout the research process
were indispensable.
Finally, the authors express their deep appreciation to the anonymous reviewers
for their valuable feedback and suggestions, which significantly enhanced the
quality of this article.
276
7. References
Atalar, A. (2020). Student exchange: The first step toward international collaboration.
Successful global collaborations in higher education institutions, 63-71.
https://link.springer.com/chapter/10.1007/978-3-030-25525-1_7
Berg, T. M. V., & Schwander, L. (2019). The Long-Term Impact of a Short-Term Study
Abroad Program: Perspectives on Global Citizenship. Journal of Education and
Learning, 8(4), 18-31. https://eric.ed.gov/?id=EJ1219299
Brisset, C., Safdar, S., Lewis, J. R., & Sabatier, C. (2010). Psychological and sociocultural
adaptation of university students in France: The case of Vietnamese international
students. International journal of intercultural relations, 34(4), 413-426.
https://www.academia.edu/download/47238329/Psychological_and_sociocult
ural_adaptati20160714-11464-ttwbuy.pdf
De Wit, H., & Altbach, P. G. (2021). Internationalization in higher education: global trends
and recommendations for its future. Policy Reviews in Higher Education, 5(1), 28-46.
https://doi.org/10.1080/23322969.2020.1820898
Di Pietro, G. (2015). Do study abroad programs enhance the employability of graduates?.
Education Finance and policy, 10(2), 223-243.
https://doi.org/10.1162/EDFP_a_00159
Curtis, T., & Ledgerwood, J. R. (2018). Students’ motivations, perceived benefits and
constraints towards study abroad and other international education
opportunities. Journal of International Education in Business, 11(1), 63-78.
https://doi.org/10.1108/JIEB-01-2017-0002
Fakunle, O. (2021). International students' perspective on developing employability
during study abroad. Higher Education Quarterly, 75(4), 575-590.
https://doi.org/10.1111/hequ.12308
Gale, T., & Parker, S. (2017). Retaining students in Australian higher education: Cultural
capital, field distinction. European Educational Research Journal, 16(1), 80-96.
https://doi.org/10.1177/1474904116678004
Goldstein, S. B. (2022). A systematic review of short-term study abroad research
methodology and intercultural competence outcomes. International Journal of
Intercultural Relations, 87, 26-36. https://doi.org/10.1016/j.ijintrel.2022.01.001
Goodwin, A. L. (2020). Globalization, global mindsets and teacher education. Action in
Teacher Education, 42(1), 6-18. https://doi.org/10.1080/01626620.2019.1700848
Gil, M., & Reyes, M. (2020). International Short-Term Trips and the Development of a
Global Mindset in Business Students. Journal of Teaching in International Business,
31(4), 358-379. https://doi.org/10.1080/08975930.2020.1851623
Hua, J., Fan, J., Walker, A., Hou, N., Zheng, L., & Debode, J. (2019). Examinations of the
role of individual adaptability in cross-cultural adjustment. Journal of Career
Assessment, 27(3), 490-509. https://doi.org/10.1177/1069072718765193
Huong, L., Koo, F. K., Arambewela, R., & Zutshi, A. (2017). Voices of dissent: Unpacking
Vietnamese international student experience. International journal of educational
management. https://doi.org/10.1108/IJEM-07-2015-0099
Iskhakova, M., Bradly, A., Whiting, B., & Lu, V. N. (2022). Cultural intelligence
development during short-term study abroad programs: the role of cultural
distance and prior international experience. Studies in Higher Education, 47(8),
1694-1711. https://doi.org/10.1080/03075079.2021.1957811
Jackson, J. (2015). Preparing students for the global workplace: The impact of a semester
abroad. Language and Intercultural Communication, 15(1), 76-91.
https://doi.org/10.1080/14708477.2014.985307
277
Knight, J. (2012). Concepts, rationales, and interpretive frameworks in the

internationalization of higher education. The SAGE handbook of international higher
education, 27-42. https://doi.org/10.4135/9781452218397.n2
Le, Q. A. (2016). The impact of globalisation on the reform of higher education in Vietnam.
International Journal of Business and Economic Affairs, 1(1), 29-35.
https://doi.org/10.24088/IJBEA-2016-11005
Loes, C. N., & An, B. P. (2023). Study Abroad Enhances Cognitive Development. Journal of
student affairs research and practice, 1-14.
https://doi.org/10.1080/19496591.2023.2176770
Mathews-Aydinli, J. (Ed.). (2017). International education exchanges and intercultural
understanding: Promoting peace and global relations. Springer International
Publishing.
McManus, K. (2019). Relationships between social networks and language development
during study abroad. Language, Culture and Curriculum, 32(3), 270-284.
https://doi.org/10.1080/07908318.2019.1661683
Mikulec, E. (2019). Short-Term Study Abroad for Pre-Service Teachers: Personal and
Professional Growth in Brighton, England. International Journal for the Scholarship
of Teaching and Learning, 13(1), 11. https://doi.org/10.20429/ijsotl.2019.130111
Neumann, H., Padden, N., & McDonough, K. (2019). Beyond English language proficiency
scores: Understanding the academic performance of international undergraduate
students during the first year of study. Higher Education Research & Development,
38(2), 324-338. https://doi.org/10.1080/07294360.2018.1522621
Nguyen, D. P., Vickers, M., Ly, T. M. C., & Tran, M. D. (2016). Internationalizing higher
education (HE) in Vietnam: Insights from higher education leaders–an
exploratory study. Education+ Training. https://doi.org/10.1108/ET-08-2015-
0072
Nguyen, C. T. (2011). Challenges of Learning English in Australia towards Students
Coming from Selected Southeast Asian Countries: Vietnam, Thailand and
Indonesia. International Education Studies, 4(1), 13-20.
https://eric.ed.gov/?id=EJ1066393
Nilsson, P. A., & Ripmeester, N. (2016). International student expectations: Career
opportunities and employability. Journal of International Students, 6(2), 614-631.
https://doi.org/10.32674/jis.v6i2.373
Pham, L., & Saltmarsh, D. (2013). International students’ identities in a globalized world:
Narratives from Vietnam. Journal of Research in International Education, 12(2), 129-
141. https://doi.org/10.1177/147524091348117
Phan, H. L. T., Tran, L. T., & Blackmore, J. (2019). Internationalization, student
engagement, and global graduates: A comparative study of Vietnamese and
Australian students’ experience. Journal of Studies in International Education, 23(1),
171-189. https://doi.org/10.1177/102831531880371
Ritz, A. A. (2011). The educational value of short-term study abroad programs as course
components. Journal of Teaching in Travel & Tourism, 11(2), 164-178.
https://doi.org/10.1080/15313220.2010.525968
Roy, A., Newman, A., Ellenberger, T., & Pyman, A. (2019). Outcomes of international
student mobility programs: A systematic review and agenda for future research.
Studies in Higher Education, 44(9), 1630-1644.
https://doi.org/10.1080/03075079.2018.1458222
Schleicher, A. (2012). Preparing teachers and developing school leaders for the 21st century:
Lessons from around the world. OECD Publishing. 2, rue Andre Pascal, F-75775 Paris
Cedex 16, France. https://doi.org/10.1787/9789264174559-en
278
Teichler, U., & Janson, K. (2007). The professional value of temporary study in another
European country: Employment and work of former ERASMUS students. Journal
of studies in International Education, 11(3-4), 486-495.
https://doi.org/10.1177/102831530730323
Tomlinson, M. (2008). ‘The degree is not enough’: students’ perceptions of the role of
higher education credentials for graduate work and employability. British journal
of sociology of education, 29(1), 49-61. https://doi.org/10.1080/01425690701737457
Whatley, M., Landon, A. C., Tarrant, M. A., & Rubin, D. (2021). Program design and the
development of students’ global perspectives in faculty-led short-term study
abroad. Journal of Studies in International Education, 25(3), 301-318.
https://doi.org/10.1177/102831532090615
Wright, N. D., & Clarke, I. (2010). Preparing marketing students for a global and
multicultural work environment: The value of a semester-long study abroad
program. Marketing Education Review, 20(2), 149-162.
https://doi.org/10.2753/MER1052-8008200206
279
Appendix. Questionnaire
Thank you for participating in this survey about international student exchange programs.
Your input is valuable in understanding the experiences and perspectives of students.
Please answer the following questions:
Background Information
1. Gender
❑ Male
❑ Female
2. Program duration
❑ 1 week or less
❑ Over 1 week to 2 weeks
❑ 2 weeks to 4 weeks
❑ Over 1 month
3. Exchange destinations (short answers): _________________________________________
Reasons for Participating in International Exchange Programs

Please indicate the reasons influenced your decision to participate in an international
exchange program:
❑ Recommendation from the faculty advisor
❑ Expansion of relationships with international students
❑ Desire to improve language skills
❑ Promotion of Vietnamese culture
❑ Sharing and discussion of topics related to their field of study
Others (Please specify):
______________________________________________________________________________
Benefits of International Student Exchange Programs

Please rate the following benefits of international student exchange programs based on
your personal experiences:
(Use a scale of 1 to 5, where 1 represents “Strongly disagree” and 5 represents “Strongly
agree”)
• Cultural understanding 
• International network building 
• Acquisition of independent living skills 
• Global perspective development 
• Improved critical thinking abilities 
• Easy adaptation to new environments 
• Acquisition of valuable future work skills 
• English language skills progression 
• Teamwork skills enhancement 
• Self-evaluation and self-improvement 
• Growth in maturity and confidence 
• Integration into a multicultural environment 
• Preparedness to become a global citizen 
280
Challenges of Participating in International Student Exchange Programs:

Please indicate the challenges you faced during your international student exchange
program:
❑ Language barrier
❑ Weather conditions
❑ Cultural customs and traditions
❑ Working methods
❑ Working hours
❑ Food
❑ Accommodation
❑ Workload
❑ Lack of expertise in teaching or working
❑ Lack of new knowledge abroad
❑ Insufficient teaching skills
❑ Inadequate curriculum content
❑ Lack of interaction between teachers and students in class
❑ Lack of skills for laboratory work
❑ Inadequate lesson planning
❑ Limited interaction between teachers and students in class
Others (Please specify):__________________________________________________________
Thank you for taking the time to complete this survey. Your feedback is greatly
appreciated in enhancing the understanding and improvement of international student
exchange programs.
281

Vol. 22, No. 7, pp. 281-304, July 2023
Application of the Chatbot in University Education:

A Bibliometric Analysis of Indexed Scientific
Production in SCOPUS, 2013-2023
Omar Chamorro-Atalaya , Soledad Olivares-Zegarra
Facultad de Ingeniería y Gestión,
Universidad Nacional Tecnológica de Lima Sur, Perú*
Lisle Sobrino-Chunga , Rosemary Guerrero-Carranza

Facultad de Psicología,
Universidad Femenina del Sagrado Corazón, Perú
Ademar Vargas-Diaz
Facultad de Psicología,
Universidad César Vallejo, Perú
Madison Huarcaya-Godoy , José Rasilla-Rovegno , Raul Suarez-Bazalar

Facultad de Ciencias Administrativas,
Universidad Nacional del Callao, Perú
Jorge Poma-Garcia
Facultad de Arquitectura,
Universidad Nacional del Centro del Perú, Perú
Yreneo Cruz-Telada
Facultad Ciencias de la Salud,
Universidad Autónoma de Ica, Perú
Abstract. Universities today are employing tools based on artificial

intelligence to improve the educational service in general. The chatbot
represents an online communicative interaction tool which contributes
to solving daily queries required by students, teachers or graduates.
However, there are few bibliometric review studies on the chatbot that
show in which areas there is a greater concentration or lack of scientific
production at the university level. The objective of this article is to
explore and describe the research trends regarding the application of the
chatbot in university education through the bibliometric analysis of
*
Corresponding author: Omar Chamorro-Atalaya, ochamorro@untels.edu.pe
©Authors
282
publications indexed to Scopus. The research is of an exploratory-

descriptive level, developed under a quantitative approach. The study
covered the scientific production between the years 2013 and 2023,
identifying 210 manuscripts. It was identified that there is a growing
trend in scientific production, particularly in scientific articles and
conference papers. The most cited article was published in 2018 and has
162 citations. Therefore, it is concluded that there is a greater
concentration in the scientific production of manuscripts focused on
improving the university educational service, applying significantly to
the improvement of academic performance, administrative management
and university wellbeing. However, there is a gap that needs to be
reduced in terms of the lack of scientific studies in which the chatbot is
used as a tool to identify the satisfaction of university students. In
addition, there is a lack of research on the use of a regulatory framework
that regulates the application of the chatbot at the university level.
Keywords: chatbot; education; university; bibliometric analysis;

scientific production; Scopus
1. Introduction
The development of science and technology is booming, and education a cannot
be exempt from these advances; therefore, a broader view is required to
recontextualise the function of the academy in the labour and social sphere
(Villarroel, 2021). In recent years, globalisation has had a great impact on world
society and, as a consequence, has generated a higher level of technological
penetration that aims to streamline and facilitate many of the everyday
processes, including education (Artavia-Díaz & Castro-Granados, 2021).
However, today it is becoming increasingly evident that artificial intelligence, a
field of computer science that attempts to understand and simulate
characteristics of human intelligence (Jiménez et al., 2023), has acquired a solid
scientific foundation and has produced many successful applications, including
in academia, and that it has significant implications for the teaching and learning
process (Vera, 2023). In the educational field, artificial intelligence, hand-in-hand
with various knowledge regarding education, has as its main objective to
generate programs that allow the development of adaptive and personalised
learning environments with a high capacity for interaction between students
(Ocaña-Fernández et al., 2019; Padilla, 2019). This implementation of new
paradigms in the educational field requires knowledge, resources and planning,
which, at this juncture, are framed in virtual systems based on artificial
intelligence (Martín-Ramallal et al., 2022). Under the aforementioned, artificial
intelligence is capable of altering various forms of social interaction, has
affording it the potential to revolutionise and transform educational institutions
(Flores-Vivar & García-Peñalvo, 2023). Thus, an area of vital importance and
topicality for artificial intelligence is the conversational agents called chatbots
(Cotrina-Aliaga et al., 2021; Vázquez et al., 2018). Chatbots allow the creation of
a communication channel that is capable of simulating a communication
interface that serves as an authentic virtual tutor in learning (Manzano et al.,
2020); or even becoming a teaching assistant, learning companion or personal
283
tutor of the student (Deng & Yu, 2023).
Chatbots represent virtual human communication interaction tools (Alagarsamy

& Mehrolia, 2023; Wang et al., 2023); through the exchange of audio or text (Hsu
& Lin, 2023; Smutny & Schreiberova, 2020; Xing et al., 2022). Chatbots make use
of natural language processing, and today are part of the most advanced
technological tools for automatic and personalised interaction (Baabdullah et al.,
2022; Brustenga et al., 2018; Drouin et al., 2022; Zhou et al., 2023). Human-
chatbot interactions represent a form of social interaction carried out online
(Dippold, 2023]. They provide the user with the ability to understand their
emotions and feelings through affective computing tools (Benke et al., 2022).
Compared to traditional IT tools that are used to interrelate the service offered
with the customer, chatbots are made up of functionalities that allow them to
provide social and emotional understanding (Song et al., 2022). They show a
high capacity to process requirements from various users and then suggest a
collaboration process that responds to their needs (Cheng et al., 2022). In
general, they join other user-oriented applications and fulfil the purpose of being
easy and consistent means between organisations and clients (Nguyen et al.,
2022), helping them to find information, provide feedback and even file
complaints (Chen et al., 2022, Fan et al., 2022; Liu et al., 2023). The rapid increase
in the production of chatbot applications has attracted the attention of various
fields, such as industry and academia (Balakrishnan et al., 2022). To examine the
performance of the interaction in the use of the chatbot as a conversational
agent, the time spent in communication and the satisfaction of the event
experienced by the user are taken into account (Rhim et al., 2022).
In the field of universities, the use of chatbots is yet to be explored, but it

certainly offers many possibilities to improve the educational service in general
(Suárez et al., 2022), generating a new learning space (Llugcha, 2023). Chatbots
contribute to expedite learning as well as the resolution of doubts in the student
(Bueno, 2022), while., in the case of the teacher, chatbots help them to maintain
permanent contact with students and free them from repetitive tasks (Miguel et
al., 2022). In general, chatbots present characteristics such as interaction, self-
management, and accessibility, that is, resources that contribute to the
construction of educational technological processes (Castillo, 2020; Yang &
Chen, 2023). They facilitate the issuance of quick responses with high service
availability regardless of the place and time in which the student is (Dokukinaa
& Gumanova, 2020; Lee & Yeo, 2022; León-Granizo & León-Granizo, 2020). A
chatbot can not only provide the student with benefits for interdisciplinary
learning, but also promote the ability to classify information and knowledge
formation (Iku-Silan et al., 2023; Sáiz-Manzanares et al., 2023). However, it is
important to highlight that these assistants do not replace people, since their
function is complementary; therefore, the work of teachers and administrative
staff of an institution continues to be vital (Torres et al. 2022). On the other hand,
it is necessary to train the teacher and students with a certain level of knowledge
to be able to manipulate or develop a chatbot, particularly in subjects that are
284
not related to computing (Rodríguez et al., 2021). In addition, great care must be
taken that students are fully aware that the exchange of information is being
carried out with a conversational agent and not with a human being, since
various studies indicate the presence of effects on the level of satisfaction at
student expectations (Go & Sundar, 2019; Huang & Lee, 2022).
Based on what has been stated, this article aims to explore and describe research
trends regarding the application of the chatbot in university education through
bibliometric analysis of scientific production indexed in the Scopus database.
The analysis will be carried out for the period from 2013 to 2023. This study
seeks to contribute to the generation of prior knowledge for the purpose of
preparing systematic review studies or meta-analysis in the field of chatbots and
its contribution to the improvement of the processes imbedded in university
education. Thus, this manuscript has been structured based on the following
points: introduction, methodology, results, discussion, conclusions, limitations,
and future studies. Likewise, the research questions (RQ) defined for the
development of this bibliometric review article are detailed below:
• RQ1: What is the scientific production of manuscripts by year of
publication regarding the chatbot and its application in university
education?
• RQ2: What are the types of manuscripts published regarding the chatbot
and its application in university education?
• RQ3: What is the scientific production of open and restricted access
manuscripts regarding the chatbot and its application in university
education?
• RQ4: What are the most cited manuscripts regarding the chatbot and its
application in university education?
• RQ5: Which are the magazines with the largest number of publications
regarding the chatbot and its application in university education?
• RQ6: What are the words with the highest rate of occurrence in the titles
of the manuscripts regarding the chatbot and its application in university
education?
• RQ7: What are the words with the highest rate of occurrence in the
abstracts of the manuscripts regarding the chatbot and its application in
university education?
• RQ8: What are the thematic areas with the highest incidence that have
been published regarding the chatbot and its application in university
education?
2. Methodology
2.1 Research level and focus
This study is exploratory-descriptive level. It is of an exploratory level since it
seeks, in principle, to investigate the scientific production of chatbots applied to
university education through the identification of bibliometric indicators, from
the Scopus database, such as the number of manuscripts published per year,
types of manuscripts, number of manuscripts published in open access and
restricted access sources, most cited manuscripts and journals with the largest
number of manuscripts in this field of study. Exploratory studies investigate
285
patterns that are initially established as little known or that there is little
information in this regard, which is why the researcher is interested in
examining their characteristics (Galarza, 2020). Thus, it is also descriptive since it
seeks to determine the research trends regarding the chatbot in university
education, through the content analysis of the manuscripts included for the
study, also relying on the analysis of the words with the highest rate of
occurrence in the titles and abstracts identified in the manuscripts under
analysis. Descriptive research is carried out when seeking to describe, based on
its main components, a specific reality or context (Alban et al., 2020). In this way,
the quantitative approach will also be used for the analysis of the collected data,
Given that it is intended to develop a bibliometric analysis regarding the
scientific production of the chatbot and its application in university education, it
will seek to quantify the scientific activity through the application of quantitative
treatments. Studies with a quantitative approach seek accurate and objective
knowledge of reality, knowledge that is observable, measurable and
quantifiable, for which the aid of mathematics and statistics is required (Rojas et
al., 2022).
2.2 Database and manuscript search equation

In order to define and establish the manuscripts that will be part of the
bibliometric analysis on the scientific production of the chatbot and its
application in university education, the database from which the data will be
extracted was established as a relevant aspect in this process of documents and
bibliometric information; therefore, for this study, it was decided to use the
Scopus database. This is due to its recognition in the academic world of the rigor
that it submits to different scientific journals for the indexing of manuscripts to
this database. Likewise, Scopus stores scientific documents related to the subject
under study and provides bibliometric information for processing and analysis.
Scopus represents one of the main scientific information databases, allowing
researchers to access current information, of greater specialty and
preponderance by technological topics (Garcés-Giraldo et al., 2022). On the other
hand, Scopus is the largest database of citations and abstracts of peer-reviewed
literature and has a greater reach than WoS (Web of Science) both
geographically and thematically, so it is considered a highly suitable database
for conducting research bibliographic reviews (Pedraza-Navarro et al., 2022).
Once the database to be used was defined, the search equation was established
in order to identify the manuscripts with a higher level of relationship and link
with the topic under study. It should be noted that the search equation in the
case of the Scopus database responds to a certain syntax, which makes it
particular or different compared to other databases. In this way, in accordance
with the topic under study, the search equation was expressed as follows:
(TITLE-ABS-KEY (chatbot)) AND (TITLE-ABS-KEY (university AND students)).
This search equation will optimise the manuscript selection process (Sastoque et
al., 2020), through which the analysis of the bibliometric indicators that will
contribute to answering each of the research questions will be carried out
(Quezada et al., 2020).
286
2.3 Manuscript extraction method

The method used for the extraction of manuscripts is shown in Figure 1, the
same as validated by Izhar et al. (2023), in which three phases are defined that
will lead to the determination of the manuscripts to be included in the analysis
of the bibliometric indicators. This method comprised firstly establishing the
subject under study, scope or specific criteria of the manuscripts (year of
publication and type of manuscript) and eligibility related to the result obtained
from the search equation. A total of 212 manuscripts was identified for this
study at this stage. The second phase consisted of filtering the manuscripts
identified in the previous phase applying the criterion that defines the period of
years of publication of the manuscripts (2013-2023). In this study, it was
identified that only two manuscripts were not considered within the study time
frame; therefore, when developing this second phase, the number of
manuscripts was reduced to 210. The third phase consisted of defining the
manuscripts included for their analysis and processing of their bibliometric
indicators; this was achieved from the exhaustive review of the title, abstract and
full content of each manuscript, ultimately identifying 114 manuscripts.
Figure 1. Method used to extract manuscripts
3. Results
3.1 Scientific production of manuscripts by year of publication regarding the
chatbot and its application in university education
Of the 114 articles included for the phase of analysis and processing of
bibliometric indicators, it was identified that the years in which the least
scientific production was carried out on the application of the chatbot in
university education, were 2013 and 2017, both with a single publication.
Likewise, from 2018 to 2021, it has been identified that scientific production
experienced a sustained growth, reaching a total of 70 manuscripts published in
those four years. In addition, in 2021 it was identified that the scientific
production reached its maximum value, with 33 published manuscripts.
287
Another aspect to highlight is that, considering that in these ten years the
average scientific production is 11 manuscripts, from the year 2022 to date the
scientific production has remained above average; even in 2023 it has, to date,
already exceeded the average in eight manuscripts. Figure 2 shows the scientific
production of manuscripts by year of publication.
Figure 2. Production of manuscripts by year of publication
3.2 Types of published manuscripts regarding the chatbot and its application
in university education
In relation to the types of manuscripts that have been developed regarding the
application of the chatbot in university education, of the 114 manuscripts
extracted from the Scopus database, five types of manuscripts were identified,
these being: "Scientific articles", " Chapter of the book”, “Conference paper”,
“Letter”, “Review articles”. Of the 114 manuscripts reviewed, 55 are "Scientific
articles" representing 48.246%, 52 are "Conference paper" representing 45.614%,
five are "Chapter of the book" representing 4.386%, and finally "Letter" and
"Review articles" with only one manuscript each, representing 0.877% of the
total number of manuscripts. Figure 3 shows the percentage distribution of types
of manuscripts identified in the Scopus database regarding the application of the
chatbot in university education.
Figure 3. Percentage distribution of the types of manuscripts
288
3.3 Scientific production of open and restricted access manuscripts regarding the
chatbot and its application in university education
Regarding the scientific production of open and restricted access manuscripts, it
is necessary to refer to Tosar (2022) who established that open access
manuscripts are those scientific documents in which researchers or readers can
access the entire content for free. Casate-Fernández and Senso-Ruiz (2017), on
the other hand, state that restricted access manuscripts are those scientific
documents in which researchers who are not registered with the journal that
published the manuscript have restricted access to the entire document. In
many cases, only the summary of the published research is accessed. Based on
what was indicated from the 114 manuscripts obtained from the Scopus
database, 79 manuscripts are restricted access representing 69.30%, and 35
manuscripts are open access representing 30.70%. Figure 4 shows the percentage
of open access and restricted access manuscripts.
Figure 4: Percentage distribution of open and restricted access manuscripts with

respect to the chatbot application
When carrying out a more exhaustive analysis regarding the types of open
access manuscripts, it was identified that these, in turn, can be categorised as
golden, green, hybrid and bronze. In this regard, Alhuay-Quispe and Bautista-
Ynofuent (2021) point out that gold category manuscripts are published in an
open access journal indexed by the DOAJ (Directory of Open Access Journals);
so also, the green category comprises paid access manuscripts on the publisher's
page, but with a free copy in a repository, while hybrid category manuscripts are
free manuscripts under an open license in a paid access journal. Finally, the
bronze category manuscripts are free access manuscripts on the publisher's
page, but without a clearly identifiable license. Table 1 shows the categorisation
of the manuscripts identified as open access in which 71.43% are gold access,
14.29% are green access, 5.71% are hybrid access and 8.57% are bronze access.
Table 1: Categorisation of open access manuscripts
Open Access Manuscripts Number of

Percentage distribution
Category manuscripts
Golden 25 71.43%
Green 5 14.29%
289
Hybrid 2 5.71%
Bronze 3 8.57%
3.4 Most cited manuscripts regarding the chatbot and its application in
university education
In relation to the most cited manuscripts, the findings of Ronda-Pupo (2021)
were taken into account, which established that the number of citations of a
manuscript represents the impact and significance that its results had with
respect to other studies: it represents the degree of significant contribution to
other manuscripts in the same field of studies. Based on Table 2, the list of the
twenty manuscripts with the highest number of citations in the Scopus database,
linked to the chatbot and its application in university education, is shown. In
other words, of the 114 manuscripts chosen for the bibliometric review study,
the manuscript with the highest number of citations is "Using psychological
artificial intelligence (Tess) to relieve symptoms of depression and anxiety:
Randomised controlled trial" with 162 citations, which represents 19.19 % of the
total citations of all the manuscripts under analysis. The second most cited
manuscript is "Chatbot for university related FAQs", with 155 citations, which
represents 18.36% of the total citations.
Table 2: The twenty manuscripts with the highest number of citations

Number of
Reference Manuscript Title Percentage
citations
Fulmer et al. Using psychological artificial
(2018) intelligence (Tess) to relieve
symptoms of depression and 162 19.19%
anxiety: Randomized controlled
trial
Ranoliya et al.
Chatbot for university related FAQs 155 18.36%
(2017)
Ghose and Barua Toward the implementation of a
(2013) topic specific dialogue based
72 8.53%
natural language chatbot as an
undergraduate advisor
Colace et al. Chatbot for e-learning: A case of
52 6.16%
(2018) study
Dibitonto et al. Chatbot in a campus environment:
(2018) Design of Lisa, a virtual assistant to 44 5.21%
help students in their university life
Villegas-Ch and Proposal of an Architecture for the
Palacios (2020) Integration of a Chatbot with
Artificial Intelligence in a Smart 40 4.74%
Campus for the Improvement of
Learning
Santoso et al. Dinus Intelligent Assistance (DINA)
(2018) Chatbot for University Admission 36 4.27%
Services
Dekker et al. Optimizing Students' Mental Health
(2020) and Academic Performance: AI-
30 3.55%
Enhanced Life Crafting
290
Patel et al. (2019) AI and Web-Based Human-Like

Interactive University Chatbot 29 3.44%
(UNIBOT)
Almahri et al. Understanding Student Acceptance
(2020) and Use of Chatbots in the United
27 3.20%
Kingdom Universities: A Structural
Equation Modeling Approach
Vázquez-Cano Chatbot to improve learning
and López (2021) punctuation in Spanish and to
25 2.96%
enhance open and flexible learning
environments
Al-Ghadhban
and Al-Twairesh Nabiha: An Arabic dialect chatbot 23 2.73%
(2020)
Huang et al. Designing and evaluating three
(2019) chatbot-enhanced activities for a 23 2.73%
flipped graduate course
Chang et al. Promoting students' learning
(2022) achievement and self-efficacy: A
21 2.49%
mobile chatbot approach for
nursing training
Ralston et al. A voice interactive multilingual
(2019) student support system using IBM 21 2.49%
Watson
Mckie and Enhancing the Academic Library
Narayan (2019) Experience with Chatbots: An
21 2.49%
Exploration of Research and
Implications for Practice
Gabrielli et al. Engagement and effectiveness of a
(2021) healthy-coping intervention via
chatbot for university students
19 2.25%
during the COVID-19 pandemic:
Mixed methods proof-of-concept
study
Lee et al. (2020) Using a Multiplatform Chatbot as
an Online Tutor in a University 15 1.78%
Course
Singh et al. (2019) Rule-based Chabot for student
15 1.78%
inquiries
Liu et al. (2022) Using AI chatbots to provide self-
help depression interventions for
14 1.66%
university students: A randomized
trial of effectiveness
Total 844 100%
3.5 Journals with the largest number of publications regarding the chatbot
and its application in university education
Regarding the journals with the highest number of manuscripts published and
indexed to the Scopus database on the subject of chatbots and their application
in university education, it was identified that, of the 92 journals that published
the 114 manuscripts under analysis in this bibliometric review study, 13 present
at least two publications, while 72 present only one publication. Table 3 shows
291
the scientific journals with the highest number of publications, the journal with
the largest number of publications being Lecture Notes in Computer Science
(Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in
Bioinformatics) with nine publications, representing 7.89% of the total articles
under analysis. This journal presents an H index equal to 446 according to the
Scimago SJR 2022 ranking; it is located in the Q3 quartile. However, there are
four journals located in the best quartile, that is, quartile Q1, namely: "Education
and Information Technologies", "Educational Technology and Society",
"Interactive Technology and Smart Education" and "Sustainability", whose
number of manuscripts published regarding the topic under study are two.
Table 3: Scientific journals with the highest number of publications

Scimago
SJR Number of
Name of journals H-index Percentage
Quartile publications
2022
Lecture Notes in Computer Science
(Including Subseries Lecture Notes in
446 Q3 9 7.89%
Artificial Intelligence and Lecture
Notes in Bioinformatics)
ACM International Conference not
137 4 3.51%
Proceeding Series quartile
Lecture Notes in Networks and
27 Q4 4 3.51%
Systems
not
CEUR Workshop Proceedings 62 2 1.75%
quartile
Education and Information
61 Q1 2 1.75%
Technologies
Educational Technology and Society 103 Q1 2 1.75%
IAENG International Journal of
26 Q3 2 1.75%
Computer Science
Intelligent Systems Reference Library 35 Q4 2 1.75%
Interactive Technology and Smart
27 Q1 2 1.75%
Education
International Journal of Mechanical
15 Q3 2 1.75%
Engineering and Robotics Research
Lecture Notes in Electrical
40 Q4 2 1.75%
Engineering
Sustainability (Switzerland) 136 Q1 2 1.75%
3.6 Words with the highest rate of occurrence in the titles of the manuscripts
regarding the chatbot and its application in university education
Using the VOSviewer software, we proceeded to analyse the words with the
highest occurrence rate in the titles of the 114 manuscripts chosen from the
Scopus database. For this, the VOSviewer software was configured to show the
results of those words that have at least three occurrences, obtaining 17 words as
a result. Table 4 shows the words of the titles of the manuscripts, the number of
occurrences, the percentage distribution of occurrence and the link strength of
each word. The words of the titles with the highest occurrence are "Chatbot"
with 34.93%, followed by "Students" with 14.38%, while the words with the
292
lowest occurrence are "Anxiety" with 2.05% and "Academic performance" with
the same percentage.
Table 4. Words with the highest rate of occurrence in the titles of the manuscripts.
Title words Occurrence number Percentage Link strength

Chatbot 51 34.93% 39
Student 21 14.38% 21
Chatbots 10 6.85% 9
University Student 8 5.48% 16
University 7 4.79% 10
Higher Education 6 4.11% 5
Artificial intelligence 5 3.42% 8
Ai Chatbot 4 2.74% 5
Covid 4 2.74% 8
Implementation 4 2.74% 3
Knowledge 4 2.74% 8
Motivation 4 2.74% 6
Pandemic 4 2.74% 6
Wear 4 2.74% 7
Virtual Assistant 4 2.74% 6
Academic performance 3 2.05% 5
Anxiety 3 2.05% 5
Total 146 100% 167
In addition, through the VOSviewer software, the co-occurrence network can be

generated, which shows the relationships or links between the words used most
frequently in the titles of the analysed manuscripts. At this level the relationship
or link of each word is known as link strength. From Figure 5 it can be seen that,
of the seventeen words with the highest occurrence, the word "chatbot" is the
one with the greatest link strength or is the word that has a higher level of
relationship or link with the other sixteen words. Graphically, it can be seen that
the largest circle belongs to the word "chatbot", referring to the fact that it
presents the greatest link strength among all the words with the highest
occurrence in the 114 titles of the manuscripts.
Figure 5. Network of co-occurrence between the words most frequently used in the
titles of the manuscripts
293
However, in order to establish how the words with the highest level of
occurrence are grouped or associated among all of them, the cluster density
network was generated, in which five clusters were identified, differentiated by
colours, as shown in Figure 6. These are composed as follows:
• The first cluster is made up of the words: chatbot, knowledge, student,
university and virtual assistant.
• The second cluster is made up of the words: academic performance, AI
chatbot, motivation, university students and use.
• The third cluster is made up of the words: chatbots, higher education and
implementation.
• The fourth cluster is made up of the words: anxiety and artificial
intelligence.
• The fifth cluster is made up of the words: Covid and pandemic.
Figure 6. Cluster density network of words with the highest level of occurrence in
manuscript titles
3.7 Words with the highest rate of occurrence in the abstracts of the manuscripts
Using VOSviewer, the Scopus database was analysed to obtain the abstract
words with the highest occurrence by author. For this, the VOSviewer software
was configured to show the results of those words that have at least seventeen
occurrences, obtaining as a result the 20 words of the abstracts with the highest
occurrence. This result is displayed in Table 5, which details the number of
occurrences of the words, their percentage and the value of the link strength. It
shows that the most relevant keyword is "chatbot" (7.96%), followed by
"students" (7.44%) and "university" (5.63%).
294
Table 5. Words of the abstracts with the highest occurrence

Abstract words Occurrence
Percentage Link strength
number
Chatbot 105 13.19% 651
Student 98 12.31% 605
University 75 9.42% 468
Studies 59 7.41% 379
Paper 40 5.03% 249
Technology 35 4.40% 248
Wear 34 4.27% 228
User 34 4.27% 220
Research 30 3.77% 210
College student 30 3.77% 186
Education 29 3.64% 219
Question 29 3.64% 199
Time 29 3.64% 197
Course 28 3.52% 187
Info 28 3.52% 188
Application 26 3.27% 201
Interaction 25 3.14% 185
Artificial intelligence 21 2.64% 157
Data 21 2.64% 143
Participant 20 2.51% 147
Total 796 100% 5267
Thus, it was also possible to generate the co-occurrence network, which shows
the relationships or links between the words used most frequently in the
abstracts of the manuscripts under analysis. From Figure 6, it can be seen that, of
the 20 words with the highest occurrence, the word "chatbot" is the one with the
greatest link strength, with a value of 651. Graphically, it can be seen that the
largest circle belongs to the word "chatbot", referring to the fact that it presents
the greatest link strength among all the words with the highest occurrence in the
114 titles of the manuscripts.
Figure 6. Network of co-occurrence between the words used most frequently in the
abstracts of the manuscripts
295
3.8 Thematic areas with the highest incidence that have been published
In relation to the thematic areas with the highest incidence that have been
published regarding the chatbot and its application in university education, a
content analysis of the 114 manuscripts was carried out, in order to categorise it
according to the thematic area developed, with which five thematic areas could
be identified. Table 6 shows the categories of the five categories, in which it is
observed that the category with the greatest number of manuscripts developed
is the one related to the "Improvement of academic performance using chatbot in
the teaching and learning process" with 49 manuscripts. Next, there are the
manuscripts related to "Administrative and resource management in the
university environment through the chatbot" and "Monitoring of the wellbeing
of the university student through the chatbot" with 29 and 23 manuscripts,
respectively. The thematic areas with the least scientific production are those
related to "Student learning a second language through chatbot" and "Academic
tutoring with chatbot", with eight and five manuscripts, respectively.
Table 6. Categorisation of the manuscripts analysed by thematic area of study

Number of
Thematic areas Percentage
manuscripts
Improvement of academic performance using chatbot in
49 42.982%
the teaching and learning process
Administrative and resource management in the
29 25.439%
university environment through chatbot
Monitoring the wellbeing of university students
23 20.175%
through the chatbot
Learning a second language with chatbot 8 7.018%
Academic tutoring with chatbot 5 4.386%
4. Discussion
From the results obtained regarding the scientific production of manuscripts by
year of publication on the application of the chatbot in university education, it
was identified that, between the years 2018 and 2021, a sustained growth in the
scientific production of manuscripts indexed to the Scopus database has been
identified, reaching a total of 70 manuscripts published in those four years. Prior
to these years, that is, from 2013 to 2017, only two indexed manuscripts were
identified in the Scopus database. In addition, in the year 2021 it was identified
that the scientific production reached its maximum value, with 33 published
manuscripts. In this regard, in their study on the history of the chatbot and its
applications, Adamopoulou and Moussiades (2020) show that there is a
significant increase in publications indexed to Scopus in recent years. Although
this study does not strictly focus on education, it does show a growth in
scientific production on chatbots in general. Likewise, also supporting what was
identified in this bibliometric review study regarding the trend in the growth of
scientific production on the chatbot application specifically in the educational
field. In their systematic review study on chatbot and its applications in
education, Okonkwo and Ade-Ibijola (2021) point out that, between 2015 and
296
2021, 624 manuscripts on the mentioned subject were identified in the Scopus
database, of which only 73 manuscripts were freely accessible to their content.
This number reflects the growing trend in scientific production on chatbot
applications in education. Likewise, regarding the availability of the complete
content of the manuscript, it was identified that, of the 114 manuscripts obtained
in the Scopus database, 79 manuscripts were restricted access and 35
manuscripts were open access. In this regard, it has become evident in recent
years that manuscripts on chatbots as applied to university education are being
published to a greater extent in restricted access journals, making it difficult to
access the full content of the manuscripts.
In addition, by continuing to review the work developed by Okonkwo and Ade-

Ibijola (2021), it was identified that they define as inclusion criteria for the
extraction of documents for their systematic review study, that these must be
scientific articles and conference papers; therefore, it is understood that the 73
manuscripts to which they refer are composed of only these two types of
documents, while in this bibliometric review it was identified that, of the 114
manuscripts reviewed, 55 were "Scientific articles" and 52 were "Conference
paper", numbers well above other types of documents such as "book chapters",
"letters to the editor" or "review articles". Although there is evidence of a
coincidence between the "types of documents" in which research on chatbot in
education is reflected to a greater extent, the numbers differ due to the fact that
the cited study only focused between the years 2015 and 2021, while this
bibliometric review covers until April 2023; being the scientific production from
the year 2022 to this date and consisting of 44 manuscripts.
In relation to the most cited manuscripts regarding the chatbot and its
application in university education, it was identified that the manuscript with
the highest number of citations is that developed by Fulmer et al. (2018), entitled
"Using psychological artificial intelligence (Tess) to relieve symptoms of
depression and anxiety: Randomized controlled trial", with 162 citations. This
article shows the positive impact of using a chatbot as a therapeutic agent or tool
for university students who suffer from some type of mental problem. The
second most cited manuscript is that by Ranoliya et al. (2017), entitled "Chatbot
for university related FAQs", with 155 citations, in which a chatbot application
for a university is designed, which is used for students to consult common
questions regarding the services offered by universities. In both cases, it is
shown that the topic of the chatbot turns out to be relevant in these times, since
the most cited manuscript, in less than five years and has achieved an average of
32 citations per year, a relatively significant number for a scientific publication.
Regarding the journals with the highest number of manuscripts published and
indexed to the Scopus database on chatbot and its application in university
education, it was identified that, of the 92 journals that published the 114
manuscripts under analysis, 13 present at least two publications, while 72
present only one publication. The journal with the largest number of
publications being Lecture Notes in Computer Science (Including Subseries Lecture
Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) with an H index
297
equal to 446 and, according to the Scimago SJR 2022 ranking, located in quartile
Q3. However, when searching for a relationship between this journal and the
most cited manuscripts, such as those developed by Ranoliya et al. (2017) and
Fulmer et al. (2018), it is identified that these works were not published in this
journal, with a high H index, and, on the contrary, these manuscripts were
published in journals that only contain a publication on chatbot in university
education. Therefore, it opens the possibility for researchers who develop
studies in this field to have a greater number of journals to publish, with the
possibility of making their research visible regardless of whether the journal has
published a greater number of manuscripts in previous years, or if it belongs to a
particular quartile or whether the impact number of the journal is high or low.
Finally, in relation to the words with the highest rate of occurrence in the titles
and abstracts of the manuscripts regarding the chatbot and its application in
university education, it was identified through the VOSviewer software that
these are "chatbot" and "students". While when analysing the co-occurrence or
link between words, five clusters were identified, with the first and second
clusters containing the largest number of associated words. The words that
make up these clusters are “chatbot, knowledge, student, university and virtual
assistant” and “academic performance, AI chatbot, motivation, university
students and use”. In this regard, Auqui (2021), in his systematic review study
on chatbot in the university student's learning process, based on research
published from 2015 to 2020, identified that the most used words in 24 selected
articles are chatbot, learning and artificial. Although this result supports what
was found in this bibliometric review study, the studies were carried out at
different time intervals, so it could be established that the trend of manuscripts
on chatbot in university education is increasing in recent years. In addition, in
accordance with what has been indicated, it was also identified that the thematic
areas with the highest incidence that have been published regarding the chatbot
and its application in university education are: Improvement of academic
performance using chatbot in the teaching and learning process (42.982%);
Administrative and resource management in the university environment
through the chatbot (25.439%); Student wellbeing monitoring through the
chatbot (20.175%); Learning a second language with chatbot (7.018%); and
Academic tutoring with chatbot (4.386%). In their systematic review on the use
of chatbots in education. Okonkwo and Ade-Ibijola (2021) indicated that the
topics addressed in publications on chatbot in education are teaching and
learning (66%), administration (5%), evaluation (6%), consulting (4%) and
research and development (19%). From the qualitative point of view, the results
of this bibliometric review and the results obtained in the cited research, there is
agreement in the categorisation carried out in the manuscripts analysed. From
the quantitative point of view, the differences that exist are due to the fact that
the cited study took Scopus, ScienceDirect, Springer, IEEE Xplore, ERIC and
Taylor & Francis as its databases, while in this study only the Scopus database
was used.
298
5. Conclusion
From the bibliometric review study on the application of the chatbot in
university education, it was identified that there is a growing trend in scientific
production, particularly scientific articles and conference papers regarding this
field of study. In addition, it was identified that, to a greater extent, these
manuscripts have been published in journals with restricted access to the full
content of the manuscript. Thus, it was also identified that the journal with the
largest number of publications is Lecture Notes in Computer Science (Including
Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics).
Finally, it was identified that the thematic areas with the highest incidence
regarding the chatbot and its application in university education are:
Improvement of academic performance using chatbot in the teaching and
learning process; Administrative and resource management in the university
environment; Monitoring of the wellbeing of the student; Learning a second
language; and Academic tutoring with chatbot. Based on what has been
indicated, it is concluded that there is a greater concentration in the scientific
production of manuscripts focused on improving the university educational
service, being applied significantly in the improvement of academic
performance, administrative management and university wellbeing. However,
there is a marked gap regarding the scientific production in which the chatbot is
used as a tool to identify university student satisfaction. This also identifies the
lack of published studies on the use of a regulatory framework that regulates the
application of the chatbot at the university level. Therefore, it is recommended
that future studies cover these fields of knowledge based on systematic reviews
of the literature and meta-analysis.
6. References
Adamopoulou, E., & Moussiades, L. (2020). Chatbots: History, technology, and
applications. Machine Learning with Applications, 2, 1–18.
https://doi.org/10.1016/j.mlwa.2020.100006
Al-Ghadhban, D., & Al-Twairesh, N. (2020). Nabiha: An Arabic dialect chatbot.
International Journal of Advanced Computer Science and Applications, 11(3), 1-8.
https://pdfs.semanticscholar.org/85a7/efbe21c45fdd756f9ae704bbc0c263152ad9
.pdf
Alagarsamy, S., & Mehrolia, S. (2023). Exploring chatbot trust: Antecedents and
behavioural outcomes. Heliyon, 9(5), e16074.
Alban, G. P. G., Arguello, A. E. V., & Molina, N. E. C. (2020). Educational research
methodologies (descriptive, experimental, participatory, and action research).
Recimundo, 4(3), 163–173. https://doi.org/10.6018/eglobal.441711
Alhuay-Quispe, J., & Bautista-Ynofuente, L. (2021). Altimetric impact and type of access
in Peruvian social science articles. Library Research, 35(89), 91–110.
https://doi.org/10.22201/iibi.24488321xe.2021.89.58369
Almahri, F. A. J., Bell, D., & Merhi, M. (2020, March). Understanding student acceptance
and use of chatbots in the United Kingdom universities: a structural equation
modelling approach. In 2020 6th International Conference on Information
Management (ICIM), pp.27-29 (March), UK.
https://doi.org/10.1109/ICIM49319.2020.244712
Artavia-Díaz, K. Y., & Castro-Granados, A. (2021). Artificial intelligence: digital
transformation and innovation in distance education. Analysis of the UNED,
Costa Rica. Electronic Journal Training and Educational Quality (REFCalE), 9(3), 1–
299
15.
https://observatorioturisticobahia.uleam.edu.ec/index.php/refcale/article/vie
w/3483
Auqui, J. A. O. (2021). Chatbot of the university learning process: A systematic review.
Scientific and Technological Research Journal, 2(2), 1–16.
https://doi.org/10.47422/ac.v2i2.22
Baabdullah, A. M., Alalwan, A. A., Algharabat, R. S., Metri, B., & Rana, N. P. (2022).
Virtual agents and flow experience: An empirical examination of AI-powered
chatbots. Technological Forecasting and Social Change, 181, 1–15.
https://doi.org/10.1016/j.techfore.2022.121772
Balakrishnan, J., Abed, S. S., & Jones, P. (2022). The role of meta-UTAUT factors,
perceived anthropomorphism, perceived intelligence, and social self-efficacy in
chatbot-based services? Technological Forecasting and Social Change, 180, 1–12.
https://doi.org/10.1016/j.techfore.2022.121692
Benke, I., Gnewuch, U., & Maedche, A. (2022). Understanding the impact of control
levels over emotion-aware chatbots. Computers in Human Behavior, 129, 1–14.
https://doi.org/10.1016/j.chb.2021.107122
Brustenga, G. G., Alpiste, M. F., & Castells, N. M. (2018). Briefing paper: Chatbots in
education. Open University of Catalonia (UOC), 1(1), 1–33.
https://openaccess.uoc.edu/bitstream/10609/80185/6/BRIEFING_PAPER_CH
ATBOTS_EN.pdf
Bueno, D. V. (2022). Creation of a chatbot in the educational domain. BS. Thesis, Department
of Computer Languages and Systems, University of Alicante, Spain.
http://hdl.handle.net/10045/124543
Casate-Fernández, R., & Senso-Ruiz, J. A. (2017). Cuban open access scientific production
in Scopus in the period 2010-2014. Cuban Journal of Information on Health Sciences,
28(1), 2–25.
Castillo, C. C. A. (2020). Artificial intelligence in education: use of the chatbot in an
undergraduate course on Academic Research at a private university in Lima. MSc.
Thesis, Graduate School, Pontifical Catholic University of Perú, Perú.
http://hdl.handle.net/20.500.12404/20996
Chang, C. Y., Hwang, G. J., & Gau, M. L. (2022). Promoting students' learning
achievement and self‐efficacy: A mobile chatbot approach for nursing training.
British Journal of Educational Technology, 53(1), 171-188.
https://doi.org/10.1111/bjet.13158
Chen, Q., Gong, Y., Lu, Y., & Tang, J. (2022). Classifying and measuring the service
quality of AI chatbot in frontline service. Journal of Business Research, 145, 1–17.
https://doi.org/10.1016/j.jbusres.2022.02.088
Cheng, X., Zhang, X., Yang, B., & Fu, Y. (2022). An investigation on trust in AI-enabled
collaboration: Application of AI-Driven chatbot in accommodation-based
sharing economy. Electronic Commerce Research and Applications, 54, 1–12.
https://doi.org/10.1016/j.elerap.2022.101164
Colace, F., De Santo, M., Lombardi, M., Pascale, F., Pietrosanto, A., & Lemma, S. (2018).
Chatbot for e-learning: A case of study. International Journal of Mechanical
Engineering and Robotics Research, 7(5), 528-533.
https://doi.org/10.18178/ijmerr.7.5.528-533
Cotrina-Aliaga, J. C., Vera-Flores, M. Á., Ortiz-Cotrina, W. C., & Sosa-Celi, P. (2021). Use
of Artificial Intelligence (AI) as a strategy in higher education. Ibero-American
Journal of Education, Especial 1, 1–11. https://doi.org/10.31876/ie.vi.81
Dekker, I., De Jong, E. M., Schippers, M. C., De Bruijn-Smolders, M., Alexiou, A., &
Giesbers, B. (2020). Optimizing students’ mental health and academic
performance: AI-enhanced life crafting. Frontiers in Psychology, 11, 1-15.
https://doi.org/10.3389/fpsyg.2020.01063
300
Deng, X., & Yu, Z. (2023). A Meta-Analysis and Systematic Review of the Effect of
Chatbot Technology Use in Sustainable Education. Sustainability (Switzerland),
15(4), 1–19. https://doi.org/10.3390/su15042940
Dibitonto, M., Leszczynska, K., Tazzi, F., & Medaglia, C. M. (2018). Chatbot in a Campus
Environment: Design of LiSA, a Virtual Assistant to Help Students in Their University
Life.. In M. Kurosu (Ed.), Human-Computer Interaction Technologies, pp. 103–116.
Springer.
Dippold, D. (2023). “Can I have the scan on Tuesday?” User repair in interaction with a
task-oriented chatbot and the question of communication skills for AI. Journal of
Pragmatics, 204, 21–32. https://doi.org/10.1016/j.pragma.2022.12.004
Dokukina, I., & Gumanova, J. (2020). The rise of chatbots-new personal assistants in
foreign language learning. Procedia Computer Science, 169, 542–546.
https://doi.org/10.1016/j.procs.2020.02.212
Drouin, M., Sprecher, S., Nicola, R., & Perkins, T. (2022). Is chatting with a sophisticated
chatbot as good as chatting online or FTF with a stranger? Computers in Human
Behavior, 128, 1–10. https://doi.org/10.1016/j.chb.2021.107100
Fan, H., Han, B., & Gao, W. (2022). (Im)Balanced customer-oriented behaviors and AI
chatbots’ Efficiency–Flexibility performance: The moderating role of customers’
rational choices. Journal of Retailing and Consumer Services, 66, 1–11.
https://doi.org/10.1016/j.jretconser.2022.102937
Flores-Vivar, J. M., & García-Peñalvo, F. J. (2023). The algorithmic life of education:
artificial intelligence tools and systems for online learning. In G. Bonales Daimiel
& J. Sierra Sánchez (Eds.). Challenges and Challenges of Social Networks in the
Communication Ecosystem, 1, pp. 109–120. McGraw Hill.
http://repositorio.grial.eu/handle/grial/2871
Fulmer, R., Joerin, A., Gentile, B., Lakerink, L., & Rauws, M. (2018). Using psychological
artificial intelligence (tess) to relieve symptoms of depression and anxiety:
Randomized controlled trial. JMIR Mental Health, 5(4), 1–15.
https://doi.org/10.2196/mental.9782
Gabrielli, S., Rizzi, S., Bassi, G., Carbone, S., Maimone, R., Marchesoni, M., & Forti, S.
(2021). Engagement and effectiveness of a healthy-coping intervention via
chatbot for university students during the COVID-19 pandemic: mixed methods
proof-of-concept study. JMIR mHealth and uHealth, 9(5), e27965.
https://doi.org/10.2196/27965
Galarza, C. A. R. (2020). The scope of an investigation. ScienceAmerica, 9(3), 1–6.
http://dx.doi.org/10.33210/ca.v9i3.336
Garcés-Giraldo, L. F., Benjumea-Arias, M., Cardona-Acevedo, S., Bermeo-Giraldo, C.,
Valencia-Arias, A., Patiño-Vanegas, C., Moreno-López, G., & Bao, R. G. (2022).
Use of artificial intelligence in information management: a bibliometric review.
Iberian Journal of Information Systems and Technologies, E54, 506–518.
https://www.proquest.com/openview/cd91b567d2231af1ab3b1838a62b3948/1
?pq-origsite=gscholar&cbl=1006393
Ghose, S., & Barua, J. J. (2013, May). Toward the implementation of a topic specific
dialogue based natural language chatbot as an undergraduate advisor. In 2013
International Conference on Informatics, Electronics and Vision (ICIEV), Dhaka,
Bangladesh, 2013, pp. 1-5. https://doi.org/10.1109/ICIEV.2013.6572650
Go, E., & Sundar, S. S. (2019). Humanizing chatbots: The effects of visual, identity and
conversational cues on humanness perceptions. Computers in Human Behavior, 97,
304–316. https://doi.org/10.1016/j.chb.2019.01.020
Hsu, C. L., & Lin, J. C. C. (2023). Understanding the user satisfaction and loyalty of
customer service chatbots. Journal of Retailing and Consumer Services, 71, 1–10.
https://doi.org/10.1016/j.jretconser.2022.103211
301
Huang, S. Y. B., & Lee, C. J. (2022). Predicting continuance intention to fintech chatbot.
Computers in Human Behavior, 129, 1–8.
Huang, W., Hew, K. F., & Gonda, D. E. (2019). Designing and evaluating three chatbot-
enhanced activities for a flipped graduate course. International Journal of
Mechanical Engineering and Robotics Research, 8, 813-818.
http://dx.doi.org/10.18178/ijmerr.8.5.813-818
Iku-Silan, A., Hwang, G. J., & Chen, C. H. (2023). Decision-guided chatbots and cognitive
styles in interdisciplinary learning. Computers and Education, 201, 1–22.
Izhar, N. A., Ishak, N. A., & Baharudín, S. M. (2023). Bibliometric Analysis of 21st
Century Learning Using Scopus Database. International Journal of Learning,
Teaching and Educational Research, 22(3), 225–240.
Jiménez, J. V., Gomez, E. E. B., & Álvarez, P. J. R. (2023). ChatGPT and artificial
intelligence: Obstacle or advantage to higher medical education?. Higher Medical
Education, 37(2), 1–5. https://ems.sld.cu/index.php/ems/article/view/3851
Lee, D., & Yeo, S. (2022). Developing an AI-based chatbot for practicing responsive
teaching in mathematics. Computers and Education, 191, 1–17.
Lee, L. K., Fung, Y. C., Pun, Y. W., Wong, K. K., Yu, M. T. Y., & Wu, N. I. (2020, August).
Using a multiplatform chatbot as an online tutor in a university course. In 2020
international symposium on educational technology (ISET), 24-27(August), Thailand.
https://doi.org/10.1109/ISET49818.2020.00021
León-Granizo, O., & León-Granizo, M. (2020). Development of a virtual assistant
(chatbot) to improve access to recurring information by students of Higher
Education Institutions. Ecuadorian Science Journal, 4(2), 111–116.
https://doi.org/10.46480/esj.4.2.49
Liu, H., Peng, H., Song, X., Xu, C., & Zhang, M. (2022). Using AI chatbots to provide self-
help depression interventions for university students: A randomized trial of
effectiveness. Internet Interventions, 27, 1–8.
https://doi.org/10.1016/j.invent.2022.100495
Liu, Y., Hu, B., Yan, W., & Lin, Z. (2023). Can chatbots satisfy me? A mixed-method
comparative study of satisfaction with task-oriented chatbots in mainland China
and Hong Kong. Computers in Human Behavior, 143, 1–14.
Llugcha, N. F. J. (2023). The chatbot in the academic tutoring of mathematics in the students of
higher basic general education of the private educational unit “RICARDO DESCALZI”
of the Ambato Canton. B.S., Faculty of Human Sciences and Education, Technical
University of Ambato, Ecuador.
https://repositorio.uta.edu.ec/jspui/handle/123456789/12640
Manzano, J. L. Z., Rodríguez, S. A. B., González, T. O., & Paciente, M. M. (2020). Artificial
intelligence applied to the process of teaching learning in law: Teachbot and
adaptive learning. Teachbot and Adaptive Learning, 1(1), 360–362.
https://accedacris.ulpgc.es/bitstream/10553/121245/1/Inteligencia_artificial_a
plicada.pdf
Martín-Ramallal, P., Merchán-Murillo, A., & Ruiz-Mondaza, M. (2022). Virtual trainers
with artificial intelligence: levels of acceptance among university students.
Educar, 58(2), 427–442. https://doi.org/10.5565/rev/educar.1482
Mckie, I. A. S., & Narayan, B. (2019). Enhancing the academic library experience with
chatbots: An exploration of research and implications for practice. Journal of the
Australian Library and Information Association, 68(3), 268-277.
https://doi.org/10.1080/24750158.2019.1611694
302
Miguel, Y. C. B., Porras, J. G. V., & Toledo, H. L. H. (2022). Virtual education and the
program I learn at home, in unequal conditions, use of the chatbot for
WhatsApp. Ciencia Latina Multidisciplinary Scientific Journal, 6(1), 4774–4795.
https://doi.org/10.37811/cl_rcm.v6i1.1831
Nguyen, Q. N., Sidorova, A., & Torres, R. (2022). User interactions with chatbot
interfaces vs. Menu-based interfaces: An empirical study. Computers in Human
Ocaña-Fernández, Y., Valenzuela-Fernández, L. A., & Garro-Aburto, L. L. (2019).
Artificial intelligence and its implications in higher education. Purposes and
Representations, 7(2), 536–552. https://doi.org/10.20511/pyr2019.v7n2.274
Okonkwo, C. W., & Ade-Ibijola, A. (2021). Chatbots applications in education: A
systematic review. Computers and Education: Artificial Intelligence, 2, 1–10.
https://doi.org/10.1016/j.caeai.2021.100033
Padilla, R. D. M. (2019). The arrival of artificial intelligence to education. Information
Technology Research Journal, 7(14), 260–270.
https://doi.org/10.36825/riti.07.14.022
Patel, N. P., Parikh, D. R., Patel, D. A., & Patel, R. R. (2019, June). AI and web-based
human-like interactive university chatbot (UNIBOT). In 2019 3rd International
conference on Electronics, Communication and Aerospace Technology (ICECA),
Coimbatore, India, 2019, pp. 148-150
https://doi.org/10.1109/ICECA.2019.8822176
Pedraza-Navarro, I., & Sánchez-Serrano, S. (2022). Analysis of publications in WoS and
Scopus Search possibilities to avoid fugitive literature in systematic revisions.
Interuniversity Journal of Research in Educational Technology, 13, 41–61.
https://doi.org/10.6018/riite.548361
Quezada, M. E., Valero, C. Z. V., & Quiroz, C. N. (2020). Bibliometric and content
analysis of the scientific production of psychology related to the study of
entrepreneurship: towards a new conceptual proposal. Journal of Psychology and
Behavioral Sciences of the Legal and Social Sciences Academic Unit, 11(2), 169–195.
https://doi.org/10.29059/rpcc.20201215-123
Ralston, K., Chen, Y., Isah, H., & Zulkernine, F. (2019, December). A voice interactive
multilingual student support system using IBM Watson. In 2019 18th IEEE
International Conference On Machine Learning And Applications (ICMLA), Boca
Raton, FL, USA, 2019, pp. 1924-1929.
https://doi.org/10.1109/ICMLA.2019.00309
Ranoliya, B. R., Raghuwanshi, N., & Singh, S. (2017). Chatbot for university related
FAQs. 2017 International Conference on Advances in Computing, Communications
and Informatics, ICACCI 2017, 04(January), pp. 1525–1530.
https://doi.org/10.1109/ICACCI.2017.8126057
Rhim, J., Kwak, M., Gong, Y., & Gweon, G. (2022). Application of humanization to
survey chatbots: Change in chatbot perception, interaction experience, and
survey data quality. Computers in Human Behavior, 126, 1-12.
Rodríguez, J. A., Santana, M. G., Perera, V. A., & Pulido, J. R. (2021). Embodied
conversational agents: artificial intelligence for autonomous learning. Pixel-Bit,
62, 107–144. https://doi.org/10.12795/pixelbit.86171
Rojas, J. A. H., Noa, L. L. T., & Flores, W. A. M. (2022). Epistemology of quantitative and
qualitative research. Science Horizon, 12(33), 27–47.
https://doi.org/10.26490/uncp.horizonteciencia.2022.23.1462
Ronda-Pupo, G. A. (2021). Scientific production and impact of the Latin American and
Caribbean science system in journals of the region. Library Research, 35(88), 45–
62. https://doi.org/10.22201/iibi.24488321xe.2021.88.58358
303
Sáiz-Manzanares, M. C., Marticorena-Sánchez, R., Martín-Antón, L. J., González Díez, I.,

& Almeida, L. (2023). Perceived satisfaction of university students with the use
of chatbots as a tool for self-regulated learning. Heliyon, 9(1).
Santoso, H. A., Winarsih, N. A. S., Mulyanto, E., Sukmana, S. E., Rustad, S., Rohman, M.
S., ... & Firdausillah, F. (2018, September). Dinus Intelligent Assistance (DINA)
chatbot for university admission services. In 2018 International Seminar on
Application for Technology of Information and Communication, 21-22(September),
Indonesia. https://doi.org/10.1109/ISEMANTIC.2018.8549797
Sastoque, J. A., García-Arango, D. A., Restrepo-Garizabal, S., Vasco-Ospina, A. M., &
Arboleda-Posada, G. (2022). Emotions in virtual learning: Systematic review
from prism guidelines. Iberian Journal of Information Systems and Technologies, E54,
451–467.
https://www.proquest.com/openview/44087bc30783c670c2c389fb2c90d5dd/1?
pq-origsite=gscholar&cbl=1006393
Singh, J., Joseph, M. H., & Jabbar, K. B. A. (2019). Rule-based chabot for student
enquiries. Journal of Physics: Conference Series, 1228(1), 1–5.
https://doi.org/10.1088/1742-6596/1228/1/012060
Smutny, P., & Schreiberova, P. (2020). Chatbots for learning: A review of educational
chatbots for the Facebook Messenger. Computers and Education, 151, 1–11.
Song, M., Du, J., Xing, X., & Mou, J. (2022). Should the chatbot “save itself” or “be helped
by others”?. The influence of service recovery types on consumer perceptions of
recovery satisfaction. Electronic Commerce Research and Applications, 55, 1–17.
https://doi.org/10.1016/j.elerap.2022.101199
Suárez, E. M. del V., Martínez-Pinilla, E., Álvarez, E. G.-, Díaz, M. G., & Incio, A. M. N.
(2022). Creation of a chatbot to answer frequently asked questions about the
running of a subject. Advances and Challenges for Educational Transformation, 1(1),
226–231. https://dialnet.unirioja.es/servlet/articulo?codigo=8582769
Torres, M. T. M., Álvarez, L. M. S., & Osorio, A. S. O. (2022). Initial study of a chatbot for
virtual modality students from the inter-American school of librarianship.
Library Research, 36(90), 13–30.
https://doi.org/10.22201/iibi.24488321xe.2022.90.58452
Tosar, J. P. (2022). The cost of article processing charges (APC) for Uruguay: the price of
open access is high. Informatio, 27(1), 221–252.
https://doi.org/10.35643/info.27.1.1
Leyva V. M., Escobar J. R., Espín R. C., & Pérez T. K. (2018). Facebook as a tool for
collaborative learning of artificial intelligence. Didasc@ Lia: Didactics and
Education, 9(1), 27–36.
https://revistas.ult.edu.cu/index.php/didascalia/article/view/728
Vázquez-Cano, E., Mengual-Andrés, S., & López-Meneses, E. (2021). Chatbot to improve
learning punctuation in Spanish and to enhance open and flexible learning
environments. International Journal of Educational Technology in Higher Education,
18(1), 1-20. https://doi.org/10.1186/s41239-021-00269-8
Vera, F. (2023). Integration of artificial intelligence in higher education: Challenges and
opportunities. Transform, 4(1), 17–34.
https://www.revistatransformar.cl/index.php/transformar/article/view/84
Villarroel, J. J. G. (2021). Implication of artificial intelligence in virtual classrooms for
higher education. Orbis Tertius-UPAL, 5(10), 31–52.
https://www.biblioteca.upal.edu.bo/htdocs/ojs/index.php/orbis/article/view
/98
Villegas-Ch, W., Arias-Navarrete, A., & Palacios-Pacheco, X. (2020). Proposal of an
Architecture for the Integration of a Chatbot with Artificial Intelligence in a
304
Smart Campus for the Improvement of Learning. Sustainability (Switzerland),

12(4), 1-20. https://doi.org/10.3390/su12041500
Wang, C., Li, Y., Fu, W., & Jin, J. (2023). Whether to trust chatbots: Applying the event-
related approach to understand consumers’ emotional experiences in
interactions with chatbots in e-commerce. Journal of Retailing and Consumer
Services, 73, 1–11. https://doi.org/10.1016/j.jretconser.2023.103325
Xing, X., Song, M., Duan, Y., & Mou, J. (2022). Effects of different service failure types
and recovery strategies on the consumer response mechanism of chatbots.
Technology in Society, 70, 1–12. https://doi.org/10.1016/j.techsoc.2022.102049
Yang, T. C., & Chen, J. H. (2023). Pre-service teachers’ perceptions and intentions
regarding the use of chatbots through statistical and lag sequential analysis.
Computers and Education: Artificial Intelligence, 4, 1–10.
Zhou, Q., Li, B., Han, L., & Jou, M. (2023). Talking to a bot or a wall?. How chatbots vs.
human agents affect anticipated communication quality. Computers in Human
305

Vol. 22, No. 7, pp. 305-323, July 2023
Applying E-Writing Therapy to Improve Mental

Wellbeing among Malaysian University Students
Following the COVID-19 Pandemic
Ying Qin Tee*
Sultan Idris Education University, Perak, Malaysia
Kee Pau
Sultan Idris Education University, Perak, Malaysia
Mahmoud Danaee
University of Malaya, Kuala Lumpur, Malaysia
Abstract. The COVID-19 pandemic is a global crisis that has significantly

impacted people’s lives worldwide, particularly in terms of mental
health and wellbeing. The implementation of frequent and restrictive
movement control orders has resulted in prolonged online learning for
university students. However, limited research has been conducted to
test practical interventions aimed at alleviating the negative impact on
the mental wellbeing of university students during the pandemic. This
mixed-method research study aimed to investigate the usability of e-
writing therapy as an intervention tool to improve the mental wellbeing
of 40 university students in a public Malaysian higher education
institution. A one-group pretest-posttest design was employed, and a
mental wellbeing questionnaire based on the Patient Health
Questionnaire (PHQ-9), Generalised Anxiety Disorder (GAD-7), and the
Warwick-Edinburgh Mental Wellbeing Scale (WEMWBS) was
administered to the participants before and after the intervention. The
results of paired-samples t-tests for each instrument indicated that there
was no significant improvement in the students’ mental wellbeing after
the four-week intervention period. However, qualitative data obtained
from the students’ experiences with e-writing therapy revealed that it
provided a flexible and personal safe space for emotional expression. It
empowered them to address their mental health needs during the post-
pandemic era, while minimising concerns about stigma, privacy and
confidentiality, despite having certain challenges and concerns.
Therefore, there is potential for e-writing therapy to be used as a
*
Corresponding author: Ying Qin Tee, teeyingqin@fpm.upsi.edu.my
©Authors
306
complementary tool alongside conventional psychological interventions

to address mental wellbeing issues among university students.
Keywords: mental wellbeing; e-writing therapy; pandemic; university

students
1. Introduction
Mental health and wellbeing are crucial for the optimal functioning of all
individuals, as a mentally fit state reflects physical health. However, the COVID-
19 pandemic has presented a global crisis that has impacted people’s lives
worldwide. The prolonged struggle to overcome this highly contagious viral
outbreak, along with movement control orders, has had a significant
psychological and mental health impact on the public. Cullen et al. (2020)
emphasised the importance of addressing “psychological and psychiatric needs”
(p. 1) during a pandemic outbreak. Unfortunately, the focus on managing
patients, vaccination programmes, and government policies often led to the
neglect of mental health in practice and research (Bäuerle et al., 2020; Cullen et
al., 2020).
Numerous studies have examined mental wellbeing following the onset of the
pandemic, with healthcare workers being the most investigated group.
University students, another vulnerable group, have also been significantly
affected by the COVID-19 pandemic. Researchers have established that the
mental health of university students has been impaired by the pandemic (Faisal
et al., 2021; Jiang, 2020; Khan et al., 2020; Parfa, 2020; Savage et al., 2020; Son et
al., 2020; Wei et al., 2020; Wang et al., 2020). These international findings align
with research evidence in Malaysia, where studies by Sundarasen et al. (2020)
and Wong et al. (2021) indicate that Malaysian students aged 19 to 25 display
higher anxiety levels and a greater vulnerability to mental health issues than
their counterparts.
The research data strongly emphasise the urgency of appropriate interventions

for university students and the role of university management in supporting and
developing students’ mental resilience during the pandemic (Jiang, 2020; Son et
al., 2020; Wong et al., 2021). However, limited studies have examined practical
interventions to mitigate the negative impact of mental wellbeing among
Malaysian university students in the post-pandemic era.
In addition to the increased mental health needs, the COVID-19 pandemic has
revealed unforeseen needs in the field of counselling. Conventional consultation
methods need to be adjusted to ensure the fulfilment of clients’ and health
professionals’ needs through safe, flexible and effective approaches, given the
risk of infection through direct contact. Taking advantage of the technological
literacy of digital natives, e-counselling services for university students can
provide accessible support anytime, anywhere, with or without the presence of a
professional.
307
Considering the positive effects of writing therapy on physical and psychological

health (e.g., Mugerwa & Holden, 2012), e-writing therapy is believed to be a
suitable tool during the pandemic or in post-pandemic circumstances, reducing
the reliance on face-to-face counselling sessions. Malaysia, like many other Asian
countries, is often characterised as having a more conservative cultural and
societal context. Individuals may feel constrained by social norms, cultural
expectations or fear of judgement, which can limit their ability to freely express
themselves. This indicates that there is a significant need for platforms that allow
for personal expression in Malaysia where certain topics or emotions may not be
easily shared with others.
However, even though writing therapy holds various advantages, its specific
efficacy in promoting mental well-being among university students during the
pandemic has not been empirically studied within the Malaysian context.
Therefore, this article aims to evaluate the effectiveness of e-writing therapy as
an intervention tool to improve the mental wellbeing of Malaysian university
students following the COVID-19 pandemic. This proposed intervention serves
as an online, asynchronous tool that empowers undergraduate students to
address their mental health without concerns of stigma, privacy or
confidentiality. Once tested, this tool could complement conventional
counselling strategies to alleviate mental wellbeing issues in this high-risk group,
particularly in the post-pandemic era. Furthermore, it would contribute to the
development of a mentally strong and resilient generation of future leaders in
Malaysia.
Impact of the Pandemic Towards University Students in Malaysia
As Malaysia has experienced significant changes from the beginning of the
pandemic to the current post-COVID-19 circumstances, Shanmugam et al. (2020)
emphasised that an increased possibility of mental health issues is expected to
persist among the Malaysian population. These issues are multifaceted,
stemming from financial difficulties, fears and uncertainties regarding public
health, inconveniences and restrictions imposed by movement control orders,
prolonged isolation, as well as stress and anxieties from various sources. A more
recent study by Tan et al. (2023) reported a statistically significant impact of the
COVID-19 pandemic on the mental health of the general Malaysian population.
Similarly, the COVID-19 pandemic has brought about unexpected and inevitable
changes for university students, particularly undergraduate students who are
primarily full-time students. In a study conducted by Son et al. (2020) on
American college students, 71% reported higher levels of stress and anxiety
caused by the pandemic. These adverse effects on students include concerns
about their own health and the health of family members, difficulties with
concentration and sleep, disruptions to social interactions and worries about
their academic progress.
In the Malaysian context, the Ministry of Higher Education announced in May

2020 that online lectures would continue in all universities until the end of the
year. As of May 2021, after a one-year period, with the enforcement of MCO3.0,
308
most students were still studying online with minimal to no face-to-face contact
with peers, instructors and the university community. Students, whether new or
continuing, must adapt to the “new norms” in education, which involve learning
virtually through online platforms (Al-Kumaim et al., 2021; Selvanathan et al.,
2020).
A plethora of research has been conducted on the impact of mental health and
wellbeing among Malaysian university students during the COVID-19 outbreak
(e.g., Tze Wei et al., 2020; Wan Mohd Yunus et al., 2021; Wong et al., 2021). Wong
et al. (2021) reported that over 55% of students in their study exhibited
symptoms of depression and anxiety. The study further identified young people,
particularly students, as a high-risk group prone to experiencing mental health
symptoms. In contrast, Kamaludin et al. (2020) found that only 30% of university
students demonstrated some level of anxiety due to the pandemic, which aligns
with the findings of Sundarasen et al. (2020) where 30% of university students
were found to have varying levels of anxiety.
The results in Malaysia present a contrast to international research evidence,

which often indicates higher percentages (e.g., Son et al., 2020). This discrepancy
may be attributed to differences in the timing of data collection, as different
points in time may reflect varying severity levels of the COVID-19 outbreak in
respective countries.
The researchers’ own experiences support the notion that a significant number of
university students have faced psychological and mental challenges during the
pandemic, impairing their learning. This is consistent with the study by Yassin et
al. (2021), which highlighted that the pandemic has caused anxieties and
psychological stress, making it difficult for students to focus on their studies.
Additionally, the prolonged period of remote online learning, whether
synchronous or asynchronous, has further burdened the psychological and
mental health of university students. This aligns with the evidence from
Sundarasen et al. (2020), which identified “remote online learning” as a key
stressor contributing to anxiety. As emphasised by Kamaluddin et al. (2020),
many students perceive themselves as lonely and in need of support, which they
often do not receive. This prolonged negative self-perception can easily lead to
mental illness if not addressed promptly (Kamaludin et al., 2020).
The review of available literature underscores the need for prompt action to
support and empower students in the face of an uncertain crisis such as the
pandemic. The effects of the pandemic have a lasting impact that extends into
the post-pandemic period. A study conducted by Duan et al. (2022) reported that
63% of 1 445 college students in Wuhan, China experienced depression during
the post-pandemic era. As recommended by Wong et al. (2021) and Kamaluddin
et al. (2020), practical intervention strategies must be devised to address the
psychological and mental needs of students during and after the COVID-19
period. This aligns with Jamshaid et al. (2023), who also emphasised the long-
term psychological impact of the COVID-19 pandemic on international
university students in the post-pandemic era.
309
Writing Therapy
The COVID-19 pandemic has led to an increase in mental health issues beyond
the capacity of conventional support services, and movement restrictions have
limited access to face-to-face counselling services. The use of online platforms,
including psychotherapy and counselling, has become crucial. Therefore, there is
an urgent need for innovative methods and measures to address these challenges
(Marković et al., 2020).
The pandemic has highlighted unforeseen needs in the field of counselling.

Conventional consultation methods need to be adjusted to align with these needs
and ensure the safety, flexibility and effectiveness of both clients and health
professionals. In the post-pandemic era, there remains a need for flexible online
counselling tools for university students, who are often technologically savvy
and prefer accessible options (Marković et al., 2020). These tools should be easily
accessible anytime, anywhere, with or without the presence of a professional.
Writing therapy, also known as expressive writing (EW), has been introduced as
a tool to improve mental and physical health. Despite decades of research on its
benefits, there are still gaps and discrepancies that need further exploration to
understand its applicability in different counselling and psychotherapy contexts
(Marković et al., 2020).
Writing therapy has shown various benefits for different client groups. Recent
research has demonstrated improvements in depression levels for socially
inhibited individuals, enhanced psychological health for patients with
posttraumatic disorders and depression, reflective benefits for adolescents with
depression, and reduced test anxiety among students. However, there are also
studies where writing therapy did not significantly improve psychological
wellbeing (Marković et al., 2020; Niles et al., 2014; Parfa, 2020). Interestingly,
studies applying writing therapy as an intervention during the COVID-19
pandemic did not find statistically significant positive effects on alleviating
psychological difficulties, although qualitative data reported positive benefits
(e.g. Marković et al., 2020).
Nevertheless, writing therapy is considered an inexpensive and accessible tool

that requires minimal input from experts. It can be easily applied online, which
was particularly relevant during the pandemic, as emphasised by Bechard et al.
(2021). Previous studies have explored the use of writing therapy through online
platforms such as e-diaries, Facebook and other online portals. Considering its
characteristics and flexibility, writing therapy is selected as the proposed e-
intervention tool to improve university students’ post-pandemic mental
wellbeing.
In conclusion, this research study aims to address research gaps by

implementing e-writing therapy as an intervention tool to improve the mental
wellbeing of university students following the COVID-19 pandemic. It also
explores the effectiveness and usability of e-writing therapy in the post-
310
pandemic context among undergraduates in Malaysia, considering the

discrepancies found in previous studies.
3. Methodology
This article represents a portion of the entire developmental process including
needs analysis, product design, development, implementation, and evaluation
(Richey & Klein, 2005). The following section focuses on and explains the
implementation and evaluation of e-writing therapy among the selected research
subjects.
Research Procedures
Upon ascertaining the need for an e-writing therapy intervention through an
extensive literature review, undergraduate university students from a Malaysian
public university were recruited online through email and texting applications as
potential research subjects based on voluntary participation.
Through a specially designed website, participants were informed of the details

of the research process, including how to carry out the entire e-writing therapy
exercise. In addition, subjects were informed that the e-writing therapy is not a
standalone intervention that functions as a mental health treatment for
individuals with severe mental health issues. Rather, it is expected to act as a tool
of prevention that will empower students as a high-risk group in managing
mental health issues. It allows them to act on and be mindful of their own
current state of mental wellbeing in a consistent, regular way. In the long run,
this will in turn minimise their need for prescribed mental health treatment
before developing more severe mental or psychological issues which will take a
long time and complex therapies or treatment to overcome.
Upon ensuring that all participants had adequate understanding about the
intervention tool through the research process, as well as their involvement and
participation rights, informed consent was obtained from participants who
agreed to sign up for the research.
Prior to the intervention, an online mental wellbeing questionnaire with 30

questions was administered as a pretest to the research subjects to measure their
mental wellbeing. All participants engaged in 20 minutes of continuous writing
for three consecutive days each week, which was conducted over a four-week
period, leading to 12 sessions. The exercise was done asynchronously, in their
own preferred time, place and format. In the context of this study, participants
were given the freedom to select any online writing platform on their electronic
devices. Platforms used by research subjects for the intervention include
Microsoft Word, Google Docs or writing applications on their devices such as
Notepad, Notes and Penzu.
This enabled convenient, asynchronous, cost-effective access to the writing

exercise by empowering students to take charge in regulating their mental health
conditions under varying stress factors impacting them.
311
Research subjects were required to submit their weekly entries through Google
Forms to track their participation, which were kept confidential and used for the
purpose of this research only. Weekly reminders through text messages were
sent to the participants.
The e-writing therapy exercise was conducted based on the following

instructions:
During the next 20 minutes, write about any experiences and thoughts
on your life during / post-pandemic on your chosen online platform.
Write everything that comes to your mind, try to follow your thoughts
as they come to you. Describe your thoughts and feelings that you may
not have said or told anyone. As you write, examine your moods and
deeper thoughts about the experience. Make sure you let yourself go
and come into contact with your emotions and deeper thoughts. Feel
free to write everything that comes to your mind. Don’t read back,
delete or change your text; just write your thoughts, and don’t stop for
20 minutes.
The instructions for the intervention were developed by adapting existing

writing therapy models, specifically drawing from Procaccia et al. (2021) and
Marković et al. (2020). These studies focused on evaluating the effectiveness of
Expressive Writing (EW) during the COVID-19 pandemic, one involving Italian
medical healthcare workers, and the other targeting the general public, both
conducted in response to the pandemic's impact.
Upon completion of the four-week intervention, participants’ mental wellbeing

was measured again through the same pretest questionnaire. In addition, an
open-ended qualitative questionnaire was also administered online to research
subjects. Collecting both quantitative and qualitative data ensured
comprehensive data analysis about the practicality of this intervention as a tool
to improve students’ mental wellbeing during the post-pandemic era, both
statistically as well as gaining detailed perspectives based on user experience.
Furthermore, evaluating the usability of the e-writing therapy intervention solely
based on quantitative measures reflected through mental wellbeing scores is
inadequate as the scores could be affected by many other external factors.
Research Subjects
Out of 62 voluntary participants who signed up as research subjects, 40
successfully completed the four-week e-writing therapy intervention. Criteria for
participation included: a) an active undergraduate student, b) comfortable in
self-expression through writing, and c) owns an electronic device with basic
internet access.
The demographic profile of the research subjects is shown in Table 1.
312
Table 1: Demographic profile of participants

Demographic Data Categories Frequency (n=40)
Age 18 4
19 8
20 5
21 8
22 7
23 5
24 3
Gender Male 5
Female 35
Level of Study Diploma 10
Bachelor’s Degree 30
Instrument
Quantitative data of the research were gathered through an online questionnaire
to measure the research subjects’ mental wellbeing before and after the e-writing
therapy intervention. Questions were adapted and organised into three parts
based on three mental wellbeing questionnaires as shown in Table 2, namely the
Patient Health Questionnaire (PHQ-9), Generalised Anxiety Disorder (GAD-7),
and the WEMWBS (University of Warwick, 2006).
Table 2: Questionnaire employed in the study
Questionnaire References Number Description

of Items
Part A Patient Health 9 Measures the frequency of depressive
Questionnaire symptoms over the past two weeks
(PHQ-9) using the scale of “0” (not at all) to “3”
(nearly every day).
Part B Generalised 7 Assesses symptoms of generalised
Anxiety anxiety disorder over the past two
Disorder (GAD- weeks. Respondents rate each item on a
7) scale ranging from “0” (not at all) to “3”
(nearly every day).
Part C WEMWBS 14 Assesses an individual’s subjective
wellbeing, psychological functioning
and overall positive affect over the past
two weeks. Each item on the WEMWBS
is rated on a 5-point Likert scale,
ranging from “1” (none of the time) to
“5” (all of the time).
All three questionnaires are self-report questionnaires that have been widely
used worldwide. The PHQ-9 and the GAD-7 were reported to demonstrate
adequate formal psychometric properties (American Psychological Association,
2020; Pranckeviciene et al., 2022; Richardson et al., 2017). Both the PHQ-9 (Sun et
al., 2020) and the GAD-7 (Dhira et al., 2021) has been found to exhibit strong
internal consistency and demonstrate good convergent validity.
313
On the other hand, an open-ended questionnaire was employed online to collect

detailed qualitative perspectives about user experience of the e-writing therapy
intervention upon participants’ completion of the 4-week sessions. An open-
ended questionnaire was selected instead of interviews to ensure the anonymity
and privacy of the participants at the highest level. Validated by three content
experts, questions include detailed description of experience, benefits and
challenges faced by participants while conducting the e-writing therapy
intervention.
Data Analysis
A paired-samples t-test was conducted on the quantitative data gathered to
determine the effectiveness of the e-writing therapy intervention in improving
students’ mental wellbeing using SPSS version 26.
Furthermore, the qualitative data collected from participants through the open-
ended questionnaire underwent thematic analysis to identify emerging patterns.
Text segments with similar patterns were coded and categorised, and these codes
were then organised, modified and grouped into relevant categories. Finally, the
categories were rephrased into themes, which are discussed in the following
sections. To facilitate the analysis process, matrix tables were created to
systematically organise the data according to the assigned codes, allowing for the
identification of similarities, differences and other significant patterns.
4. Findings
Quantitative Findings
Three different sets of mental wellbeing questionnaires were administered to the
research subjects before and after the intervention, including 1) Patient Health
Questionnaire (PHQ-9), 2) Generalised Anxiety Disorder (GAD-7), and 3)
WEMWBS. As tabulated in Table 3, results from pretest and posttest for each set
of instruments indicated no significant improvement in students’ mental
wellbeing scores after the four-week intervention period: t(39) = 1.12, p =.27; t(39) =
1.30, p =.20; and t(39) = -1.45, p =.15 respectively.
Table 3: Pre- and post-intervention mental wellbeing scores via paired-samples t-test
Instrument Test n Mean SD T value df P value
PHQ-9 Pretest 40 9.73 6.05 1.12 39 .27
Posttest 8.80 5.47
GAD-7 Pretest 7.15 5.05 1.30 39 .20
Posttest 6.38 4.59
WEMWBS Pretest 46.00 10.49 -1.45 39 .15
Posttest 47.90 11.13
Qualitative Findings
In order to gain deeper insights into participants’ experiences during the
intervention, qualitative data on the usability of e-writing therapy in improving
university students’ mental wellbeing were collected through an open-ended
questionnaire. Upon analysis, three different themes were formulated, namely 1)
personal safe space; 2) emotional outlet; and 3) flexibility within structure. For
314
each aspect, both positive and negative perspectives are outlined based on
benefits and challenges encountered by participants during the intervention
period.
Personal safe space

Feedback from e-writing therapy users revealed that the tool provided them
with a personal private space to express themselves without the fear of
judgement or opinion from others. This is supported by multiple excerpts below:
Writing therapy feels like I’m writing a letter to someone unknown who always
listens to me and never judges. (Participant 12, Question 2, Line 1)
I feel very calm as I write each session. It was as if I was talking to someone who
really understood me. I have written many stories about my background that
many people don’t know. My background is a secret that I don’t want my friends
to know but I hope I can tell them. With this [tool], I can at least express my
feelings and share secrets... (P17, Q1, L1-5)
As I write, I feel a feeling of satisfaction... even though I told many people about
my feelings, the response given did not satisfy me. With that I prefer to express my
feelings…while monologuing with myself. (P19, Q1, L3-5)
I could write down everything without feeling anxious, scared or fearful that other
people might invalidate my feelings. Since it is only between me and myself so I
really felt at ease. (P32, Q2, L1-2)
I felt I had an outlet to let my thoughts free without worrying that I would be
judged or [worry if] people not get my peculiar way of viewing things. (P09, Q1,
L2-4)
It feels like you’re in [a] comfort zone where you’re letting the most honest feelings
when you’re writing it. It’s no doubt that you won’t be uncomfortable to talk about
your problems on the paper since it’s between you and the keyboard. (P59, Q2, L1-
2)
I was able to freely express myself without having to face anyone since I don’t
really like making eye contact when talking about my problems... It also gave me
the opportunity to feel safe and comfortable in my own safe space. (P60, Q2, L1-3)
I felt like I was communicating with myself when I write my thoughts, ideas and
emotions. I have become the active listener for myself. (P57, Q1, L1-2)
This writing therapy is a “me time” for me with myself. During the 4 weeks, I felt
like I gave myself more space and time than ever before. (P30, Q1, L3-4)
Despite the clear positive benefits, some concerns about uncertainty, anonymity,
and privacy on the use of e-writing therapy emerged when participants were
asked about potential application of the intervention in future:
315
I’m not keen on saving documents on my gadgets that have my feelings being
expressed. Maybe I will use anonymous accounts to write about my feelings
without leaving any information that will date back to me. (P04, Q5, L1-2)
I feel paranoid where I think that those who read my writing may judge me and
think that I’m pathetic and make my problems a big deal. (P12, Q3, L1-2)
Emotional outlet
Participants also pointed out that the e-writing therapy acted as an appropriate
emotional outlet for self-reflection, improving their mindfulness as they process
and organise their thoughts and emotions in the process. This is evident in
participants’ responses below:
Due to this writing therapy, I was able to allocate some time every week and self-
reflect emotionally. It also enables me to build up the habit of tracking my daily
emotions... (P10, Q1, L1-3)
Reflective, calmer, mindful. (P24, Q1, L1)
I feel like I can express my feelings calmly without having to hurt myself. (P52,
Q1, L1-2)
It helps me to keep my emotions in check. (P08, Q2, L3-6)
First, helped me understand why I’m feeling a certain emotion. Second, made me
understand that it is important to validate my emotions. Third, after I finish
writing about myself, my head would be in a calm state. Finally, it helped me to
sleep in peace. (P49, Q2, L1-3)
I was avoided from bottling up my emotions. (P57, Q2, L3)
This experience had helped me in recognising some hidden emotions which I have
been suppressing all this while. It alarms me to look deeper into some matter as
well as not bottle up emotions, even expressing them through words helps a lot.
(P10, Q4, L1-3)
It helps clearing out the burden in my heart. (P36, Q4, L1)
The writing therapy helped me validate my emotions... So, when I am facing

problems and just have a bad day, I would just shut down and take care of myself
first by writing. (P49, Q4, L1-3)
This experience has help me reflect a lot and gave me the time to truly let go of my
anger or hatred. It has helped me to slowly learn to make peace with myself, and let
go of the past that had hurt me for a long time. It became a reminder to me that it’s
okay to feel what I felt, but I shouldn’t hold on to it for a long time because it will
only cause pain. This experience has really help me feel better. (P60, Q4, L1-4)
316
One respondent beautifully described the role of e-writing therapy in emotional

regulation:
By writing, it gives me relief. It’s like breathing fresh air again, it’s like a therapy
to heal the wounds in the soul. (P50, Q4, L1-3)
Undeniably, the e-writing therapy intervention played the role of helping

participants to express, regulate and validate their emotions in the process.
However, it also saw several participants struggling to deal with some “ugly
past” or undesired emotions that came along the process:
Throughout this writing therapy journey, there are times where I’ll feel extreme
sadness due to reflection on some past life events. It evokes a depressive mood,
causing me to feel down and moody and thus finding myself reluctant to keep on
writing. (P10, Q3, L6-8)
The challenge while writing is when I was being influenced by emotions. It

happened during 3… stressful weeks. When I write I tend to exaggerate to the
point of relating bad assumptions that have not and will not happen. (P18, Q3,
L1-3)
Nevertheless, these participants’ responses clearly indicated how emotions and

thoughts were progressively and successfully processed from the beginning of
the intervention till the end:
Sometimes it does make you cry a bit but after crying you feel a lot better after
expressing your problems. (P38, Q4, L1-2)
At first, I was not sure whether to continue writing because of many past
experiences that haunted me. But after entering the second week... I began to see
how many things I had been through and here I am standing still. Entering the
third week I started to be confident, I wrote how I went through the “dark life” I
was going through, how miserable it feels but I keep all of that to focus on what lies
ahead. In the final week I began to describe how I knew what I was about to go
through was not all sweet and all bitter. (P61, Q1, L1-5)
At first, I felt a little awkward to write about what I feel. I used to write about my
days in a diary when I was younger, but as I grow up, I barely even write
anything anymore. Growing older makes me tend to keep a lot of things to myself.
I don’t share my feelings and thoughts even on papers. However through this 4-
week journey, I was able to give a try at writing out my feelings again. (P08, Q1,
L1-4)
To be honest, this is a new way for me to express my feelings. At first, I had a lot of
doubts to participate in this therapy since I’m not used to write down my feelings
and I couldn’t believe that the issues I’m dealing would get any better just by
writing it down but suprisingly it did. It really works for me since...I often avoid
to express my thoughts or feelings to other people [due to] fear... I think that this is
one of the best way for people to express their feelings especially for an introvert
person like me. (P32, Q1, L1-6)
317
Several participants also described their challenges in their expression of

emotions during the intervention:
The challenging part is I need to find the perfect words to represent my feelings
and thoughts. I need to have the correct terms for every emotion that I used in the
writing. (P03, Q3, L1-4)
I explain it very well in my head but when I need to actually let it out, I found it
really hard to do so. (P05, Q3, L1)
Sometimes I don’t know how to describe and put into words to write down what I
feel. (P09, Q3, L1)
I found it challenging to organise the paragraph and how to talk about the issues.
Also, it is a bit difficult to express since I may not know the proper word or term to
use. (P60, Q3, L1-2)
Flexibility within Structure

Findings implied that e-writing therapy provided them with sufficient amount of
flexibility in expressing themselves within a preset structure of reference that
guided them on the duration and management of their writing routines. This
allowed users of the intervention to develop the consistency of employing the e-
writing therapy while being able to conduct the exercise anywhere and anytime,
with no restriction on the content they wrote, as highlighted in the following
excerpts:
It is not a burden for me to write just three times continuously in a week. Just
nice. The suggested time 20 minutes is also perfect for me to find ideas about what
should I write… (P50, Q1, L1-3)
There was a lot of flexibility - I didn’t have to necessarily write things that made
sense. I just had to write whatever was on my mind (P04, Q2, L1-2)
I like the fact that I can choose to write in English if I want to. I am more
comfortable in using English because my thoughts are mostly in English, rather
than in Malay language. I also prefer the fact that I can do this online instead
writing in front of someone. (P08, Q2, L3-5)
I like the consistency [to be able to write] for 4 weeks. (P23, Q2, L1)
Writing therapy can be done everywhere, whenever you feel stressed out, you can
do the therapy on the spot. (P38, Q2, L1)
Limitless writing while spilling everything regarding my feelings and interest.

(P58, Q2, L1)
I am free to write what I think and what I want to say. (P44, Q2, L1)
Although given some degree of freedom, some participants reported their

struggle in keeping their consistency while conducting the e-writing therapy in
318
accordance with the suggested duration of intervention given, as shown in the

excerpts below:
I like that I am not tied to any style [of writing]. But I don’t like it because I have
to do it 3 days in a row. (P53, Q2, L1-2)
Sometimes I do feel forced to write because it has to be written in few days straight.
(P01, Q1, L2)
Determination to write [for] three days continuously [is a challenge]. (P25, Q3,
L1)
I find the duration a little bit challenging to be done continuously for 3 days. (P57,
Q3, L2-3)
Additionally, there were also participants who pointed out the challenge of not
knowing what to write within the specific time limit during the process of e-
writing therapy:
I don’t know what to write... I need to take more than 20 minutes to think about
what to write. (P31, Q3, L1-2)
My life is quite mundane and repetitive, so I honestly don’t have much things to
write about. Most of the times I also feel calm than sad, so again, I don’t know
what to really write rather than just repeating the same thoughts. However, after
getting myself familiar with this writing therapy, I feel much more relaxed to write
about my days even if it’s repetitive and boring. (P08, Q3, L1-4)
The first thing I find challenging is the excess of items to be recorded in my

writing therapy, [or] the lack thereof. So I believe it really depends on one’s life
events; whether you’re having a cool, decent, calm day or vice versa. Sometimes,
there are days where I find it hard to think of any suitable ideas/items/explanations
to be included in the written record. However, there are other times where there is
just too much to be written down in which I’ll end up using more than the
predetermined time limit. (P10, Q3, L1-5)
Sometimes I feel demotivated to write, especially on those days where I don’t feel
any emotions at all. I would just feel like, this is just another daily routine of mine.
I don’t feel any feelings, hence the difficulty in writing. (P36, Q3, L1-3)
However, as users grew in their familiarity with the structure of e-writing

therapy, findings indicated gradual positive progress in employing the therapy
within the specified time frame:
I felt that writing within 20 minutes was quite challenging at first, because it was
difficult for me to express my feelings and emotions and to use up 20 minutes to
write it. The first week was very difficult. But [for] the second week, I made
progress because I wrote faster and I felt relieved. During the third and fourth
week, everything was easier - I was able to maximise those 20 minutes even though
there were days when I still have difficulty writing. (P22, Q3, L3-7)
319
In the first week, it was very difficult for me to concentrate to write within 20
minutes. But after 4 weeks of writing, slowly I was able to adapt. I started to be
able to focus and follow the flow with what I wanted to write. (P61, Q3, L1-3)
5. Discussion
Based on the quantitative results obtained from the users of e-writing therapy, no
statistically significant positive effects on participants’ mental wellbeing were
observed before and after implementing the intervention. This aligns with the
findings of previous research studies by Marković et al. (2020) and Parfa (2020),
which also explored the use of online writing therapy as an intervention tool
during the COVID-19 pandemic. Notably, research findings of this study are also
aligned to the study of Marković et al. (2020) where its qualitative data also
reported positive benefits of writing therapy for the users involved. The
qualitative findings gathered in this study resonate with the research conducted
by Mikocka-Walus et al. (2020), reaffirming the numerous advantages of the
writing therapy intervention. These benefits include its cost-effectiveness, ease of
online application, capacity to facilitate social seclusion, and the potential for
wide distribution both locally and internationally, making it a valuable resource
in handling future calamities.
In contrast, a study by Robertson et al. (2023) found that COVID-focused online

EW reduced symptoms of anxiety and distress, but not depression. Additionally,
online EW was found to be effective in alleviating psychological distress among
asymptomatic COVID-19 patients in the study by Zheng et al. (2023). Moreover,
concurring with the findings in the study of Tay et al. (2022), qualitative findings
for this research also asserted the role of the writing therapy intervention as a
platform for self-reflection even during the post-pandemic era.
The discrepancies in these findings regarding the evaluation of online writing

therapy intervention tools may be attributed to the different samples used and
the varying circumstances in which the studies were conducted. However, these
discrepancies do not diminish the potential of e-writing therapy as a platform to
help improve the overall psychological wellbeing among Malaysian
undergraduate university students.
6. Conclusion
This research was conducted in response to the need for empirically tested
alternatives to conventional psychotherapy and counselling sessions in light of
the aftermath of the COVID-19 pandemic through application of an e-
intervention tool among Malaysian undergraduate university students.
While quantitative findings from the study did not show statistically significant
positive effects of e-writing therapy in improving the mental wellbeing of the 40
research subjects, qualitative findings strongly emphasised the potential of e-
writing therapy as a powerful intervention technique in the post-COVID-19
pandemic era. Participants’ feedback revealed that it provided a safe, flexible
and consistent means of expressing and regulating emotions for university
students despite several challenges.
320
Despite inconclusive statistical evidence, the potential of e-writing therapy as an

intervention for university students in the post-COVID-19 era should not be
underestimated. Moreover, it is crucial to acknowledge that mental wellbeing is
a complex construct that should not be measured or reflected solely through
psychological scales and statistics alone.
Several limitations in this study include quantitative assessment of participants’

mental wellbeing in this study that was based on self-report questionnaires. It is
also important to note that the assessment was administered during a turbulent
and uncertain period, where university students were still undergoing online
studies in isolation from their usual academic communities. Furthermore, future
studies can provide more robust evidence by increasing the number of
participants, extending the intervention period and exploring the development
of a standardised online platform specifically designed for e-writing therapy.
Relying solely on conventional face-to-face counselling sessions in universities

may not be sufficient, and there is a need to complement these services with
online psychological interventions. By actively supporting and promoting the
use of e-writing therapy, it is crucial for higher education institutions in Malaysia
and around the world to empower students to take proactive steps towards
improving their psychological and mental wellbeing.
7. Acknowledgment
This research study was funded by the Sultan Idris Education University
through the Research Grant 2021-0059-107-01 (GGPU).
8. References
Al-Kumaim, N. H., Mohammed, F., Gazem, N. A., Fazea, Y., Alhazmi, A. K., & Dakkak,
O. (2021). Exploring the impact of transformation to fully online learning during
COVID-19 on Malaysian university students’ academic life and performance.
International Journal of Interactive Mobile Technologies; 15(5).
https://doi.org/10.4018/IJICTE.2021010102
Allen, S. F., Wetherell, M. A., & Smith, M. A. (2020). Online writing about positive life
experiences reduces depression and perceived stress reactivity in socially
inhibited individuals. Psychiatry Research, 284. https://doi.org/
10.1016/j.psychres.2019.112697
Bäuerle, A., Graf, J., Jansen, C., Dörrie, N., Junne, F., Teufel, M., & Skoda, E. M. (2020). An
e-mental health intervention to support burdened people in times of the COVID-
19 pandemic: CoPE It. Journal of Public Health, 42(3), 647–648.
https://doi.org/10.1093/pubmed/fdaa058
Bechard, E., Evans, J., Cho, E., Lin, Y., Kozhumam, A., Jones, J., Grob, S., & Glass, O.
(2021). Feasibility, acceptability, and potential effectiveness of an online
expressive writing intervention for COVID-19 resilience. Complementary Therapies
in Clinical Practice, 45, 1-8. https://doi.org/10.1016/j.ctcp.2021.101460
Cullen, W., Gulati, G., & Kelly, B. D. (2020). Mental health in the COVID-19 pandemic.
QJM: An International Journal of Medicine, 113(5), 311–312.
https://doi.org/10.1093/qjmed/hcaa110
Dhira, T. A., Rahman, M. A., Sarker, A. R., & Mehareen, J. (2021). Validity and reliability
of the Generalized Anxiety Disorder-7 (GAD-7) among university students of
321
Bangladesh. PloS One, 16 (12), 1-16.

Dinastuti, D. (2018). Expressive writing in Indonesian context. Jurnal Ilmiah Psikologi
MANASA, 7(1), 1–15.
https://ejournal.atmajaya.ac.id/index.php/Manasa/article/view/144
Duan, H., Gong, M., Zhang, Q., Huang, X., & Wan, B. (2022). Research on sleep status,
body mass index, anxiety and depression of college students during the post-
pandemic era in Wuhan, China. Journal of Affective Disorders, 301, 189-192.
https://doi.org/10.1016/j.jad.2022.01.015.
Faisal, R. A., Jobe, M. C., Ahmed, O., & Sharker, T. (2021). Mental health status, anxiety,
and depression levels of Bangladeshi university students during the COVID-19
pandemic. International Journal of Mental Health and Addiction, 20(3), 1500–1515.
https://doi.org/10.1007/s11469-020-00458-y
Hirai, M., Dolma, S., Vernon, L. L., & Clum, G. A. (2020). A longitudinal investigation of
the efficacy of online expressive writing interventions for Hispanic students
exposed to traumatic events: competing theories of action. Psychology and Health,
35(12), 1459–1476. https://doi.org/10.1080/08870446.2020.1758324
Jiang, R. (2020). Knowledge, attitudes and mental health of university students during
the COVID-19 pandemic in China. Children and Youth Services Review, 119, 1-4.
https://doi.org/10.1016/j.childyouth.2020.105494
Jamshaid, S., Bahadar, N., Jamshed, K., Rashid, M., Afzal, M. I., Tian, L., Umar, M., Feng,
X., Khan, I., & Zong, M. (2023). Pre- and post-pandemic (COVID-19) mental
health of international students: Data from a longitudinal study. Psychology
Research and Behavior Management, 16, 431-446.
https://doi.org/10.2147/PRBM.S395035
Kamaludin, K., Chinna, K., Sundarasen, S., Khoshaim, H. B., Nurunnabi, M., Baloch, G.
M., Sukayt, A., & Hossain, S. F. A. (2020). Coping with COVID-19 and movement
control order (MCO): Experiences of university students in Malaysia. Heliyon,
6(11), 1-7. https://doi.org/10.1016/j.heliyon.2020.e05339
Khan, A. H., Sultana, M. S., Hossain, S., Hasan, M. T., Ahmed, H. U., & Sikder, M. T.
(2020). The impact of COVID-19 pandemic on mental health & wellbeing among
home-quarantined Bangladeshi students: A cross-sectional pilot study. Journal of
Affective Disorders, 277, 121–128. https://doi.org/10.1016/j.jad.2020.07.135
Lee, S. W., Kim, I., Yoo, J., Park, S., Jeong, B., & Cha, M. (2016). Insights from an
expressive writing intervention on Facebook to help alleviate depressive
symptoms. Computers in Human Behavior, 62, 613–619.
Marković, V. M., Bjekić, J., & Priebe, S. (2020). Effectiveness of expressive writing in the
reduction of psychological distress during the COVID-19 pandemic: A
randomized controlled trial. Frontiers in Psychology, 11.
Metsäranta, K., Kurki, M., Valimaki, M., & Anttila, M. (2019). How do adolescents use
electronic diaries? A mixed-methods study among adolescents with depressive
symptoms. Journal of Medical Internet Research, 21(2).
https://doi.org/10.2196/11711
Mikocka-Walus, A., Olive, L., Skvarc, D., Beswick, L., Massuger, W., Raven, L., Emerson,
C., & Evans, S. (2020). Expressive writing to combat distress associated with the
COVID-19 pandemic in people with inflammatory bowel disease (WriteForIBD):
A trial protocol. Journal of Psychosomatic Research, 139.
https://doi.org/10.1016/j.jpsychores.2020.110286
Mugerwa, S., & Holden, J. D. (2012). Writing therapy: A new tool for general practice?
British Journal of General Practice, 62(605), 661–663.
https://doi.org/10.3399/bjgp12X659457
322
Niles, A. N., Haltom, K. E. B., Mulvenna, C. M., Lieberman, M. D., & Stanton, A. L.
(2014). Randomized controlled trial of expressive writing for psychological and
physical health: the moderating role of emotional expressivity. Anxiety, Stress,
and Coping, 27(1), 1–17. https://doi.org/10.1080/10615806.2013.802308
Parfa, N., & Olle, O. (2020). A challenge or a threat : Exploring expressive writing as a
reframing tool to cope with the transition to online learning [Master’s thesis, UMEA
University]. UME DiVA Portal. https://www.diva-
portal.org/smash/get/diva2:1525682/FULLTEXT01.pdf
Pascoe, P. E. (2016). Using patient writings in psychotherapy: Review of evidence for
expressive writing and cognitive-behavioral writing therapy. American Journal of
Psychiatry Residents’ Journal, 11(3), 3–6. https://doi.org/10.1176/appi.ajp-
rj.2016.110302
Pranckeviciene, A., Saudargiene, A., Gecaite-Stonciene, J., Liaugaudaite, V., Griskova-
Bulanova, I., Simkute, D., Naginiene, R., Dainauskas, L. L., Ceidaite, G., &
Burkauskas, J. (2022). Validation of the Patient Health Questionnaire-9 and the
Generalized Anxiety Disorder-7 in Lithuanian student sample. PloS One, 17(1), 1-
18. https://doi.org/10.1371/journal.pone.0263027
Procaccia, R., Segre, G., Tamanza, G., & Manzoni, G. M. (2021). Benefits of expressive
writing on healthcare workers’ psychological adjustment during the COVID-19
pandemic. Frontiers in Psychology, 12, 1-10.
Richardson, T., Wrightman, M., Yeebo, M., & Lisicka, A. (2017). Reliability and score
ranges of the PHQ-9 and GAD-7 in a primary and secondary care mental health
service. Journal of Psychosocial Rehabilitation and Mental Health, 4, 237–240.
https://doi.org/10.1007/s40737-017-0090-0
Savage, M. J., James, R., Magistro, D., Donaldson, J., Healy, L. C., Nevill, M., & Hennis, P.
J. (2020). Mental health and movement behaviour during the COVID-19
pandemic in UK university students: Prospective cohort study. Mental Health and
Physical Activity, 19, 1-6. https://doi.org/10.1016/j.mhpa.2020.100357
Selvanathan, M., Hussin, N. A. M., & Azazi, N. A. N. (2020). Students learning
experiences during COVID-19: Work from home period in Malaysian higher
learning institutions. Teaching Public Administration, 41(1), 13-22.
https://doi.org/10.1177/0144739420977900
Shanmugam, H., Juhari, J. A., Nair, P., Chow, S. K., & Ng, C. G. (2020). Impacts of
COVID-19 pandemic on mental health in Malaysia: A single thread of hope.
Malaysian Journal of Psychiatry, 29(1), 78–84.
https://journals.lww.com/mjp/Abstract/2020/29010/Impacts_of_COVID_19_
Pandemic_on_Mental_Health_in.10.aspx
Shen, L., Yang, L., Zhang, J., & Zhang, M. (2018). Benefits of expressive writing in
reducing test anxiety: A randomized controlled trial in Chinese samples. PLOS
ONE, 13(2). https://doi.org/10.1371/journal.pone.0191779
Son, C., Hegde, S., Smith, A., Wang, X., & Sasangohar, F. (2020). Effects of COVID-19 on
college students’ mental health in the United States: Interview survey study.
Journal of Medical Internet Research, 22(9), 1-14. https://doi.org/10.2196/21279
Sun, Y., Fu, Z., Bo, Q., Mao, Z., Ma, X., & Wang, C. (2020). The reliability and validity of
PHQ-9 in patients with major depressive disorder in psychiatric hospital. BMC
Psychiatry, 20, 1-7. https://doi.org/10.1186/s12888-020-02885-6
Sundarasen, S., Chinna, K., Kamaludin, K., Nurunnabi, M., Baloch, G. M., Khoshaim, H.
B., Hossain, S. F., & Sukayt, A. (2020). Psychological impact of COVID-19 and
lockdown among university students in Malaysia: Implications and policy
recommendations. International Journal of Environmental Research and Public
Health, 7(17), 1-13. https://doi.org/10.3390/ijerph17176206
323
Tan, E. S. S., Chin, S. A. F. X., Sathapan, M. S. P., Dewi, A. D., Amini, F., Bustami, N. A.,
Tan, P. Y., Ho, Y. B., & Tan, C. K. (2023). Mental health and the COVID-19
pandemic: Observational evidence from Malaysia. International Journal of
Environmental Research and Public Health, 20(5), 1-14.
https://doi.org/10.3390/ijerph20054046
Tay, H. Y., Yu, C., Wong, C. S., & Prihadi, K. D. (2022). Expressive writing during the
COVID-19 pandemic: Themes of mixed expressive writing. International Journal of
Public Health Science, 11(1), 195-203. https://doi.org/10.11591/ijphs.v11i1.21101
Tee, Y. Q., Mohamed, A., & Alias, N. (2014). Application of Fuzzy Delphi Method in
exploring Facebook as a tool for writing therapy among young adults. Malaysian
Online Journal of Educational Technology, 2(3), 1–10.
https://eric.ed.gov/?id=EJ1086401
University of Warwick. (2006). Warwick Edinburgh Mental Wellbeing Scale (WEMWBS).
https://warwick.ac.uk/wemwbs/using/register/resources
Wei, T. T., Yee, Y. X., Baskaran, K., Subramaiam, T. D. N., & Partheeban, G. (2020). Self-
reported compliance and mental health concerns towards COVID-19 pandemic:
Malaysian undergraduate student’s perspective. International Journal of Biomedical
and Clinical Sciences, 5(3), 204–216.
http://www.aiscience.org/journal/paperInfo/ijbcs?paperId=4995
Wan Mohd Yunus, W. M. A., Badri, S. K. Z., Panatik, S. A., & Mukhtar, F. (2021). The
unprecedented movement control order (lockdown) and factors associated with
the negative emotional symptoms, happiness, and work-life balance of
Malaysian university students during the Coronavirus disease (COVID-19)
pandemic. Frontiers in Psychiatry, 11, 1-15.
https://doi.org/10.3389/fpsyt.2020.566221
Wang, X., Hegde, S., Son, C., Keller, B., Smith, A., & Sasangohar, F. (2020). Investigating
mental health of US college students during the COVID-19 pandemic: Cross-
sectional survey study. Journal of Medical Internet Research, 22(9), 1-11.
https://doi.org/10.2196/22817
Wong, L. P., Alias, H., Md Fuzi, A. A., Omar, I. S., Mohamad Nor, A., Tan, M. P.,
Baranovich, D. L., Saari, C. Z., Hamzah, S. H., Cheong, K. W., Poon, C. H.,
Ramoo, V., Che, C. C., Myint, K., Zainuddin, S., & Chung, I. (2021). Escalating
progression of mental health disorders during the COVID-19 pandemic:
Evidence from a nationwide survey. PLOS ONE, 16(3), 1-14.
Yassin, A. A., Razak, N. A., Saeed, M. A., Al-Maliki, M. A. A., & Al-Habies, F. A. (2021).
Psychological impact of the COVID-19 pandemic on local and international
students in Malaysian universities. Asian Education and Development Studies, 10(4),
574-586. https://doi.org/10.1108/AEDS-05-2020-0098
Zheng, X., Qu, J., Xie., J., Yue., W., Liang, X., Shi, Z., Bai, J., Sun, Z., Cheng, F., Li, X., &
Liu, C. (2023). Effectiveness of online expressive writing in reducing
psychological distress among the asymptomatic COVID-19 patients in Fangcang
Hospitals: A quasi-experiment study. Frontiers in Psychology, 13, 1-10.
324

Vol. 22, No. 7, pp. 324-343, July 2023
Effectiveness of a Mixed Methods-Based Literacy

Program in Improving Reading Comprehension,
Vocabulary Mastery, and Reading Fluency Skills
of Early Grade Students
Indah Nurmahanani*
Department of Elementary School Teacher Education, Universitas Pendidikan
Indonesia, Indonesia
nurmahanani@upi.edu
Abstract. Many young children still experience reading difficulties when

they enter school age. This study aimed to examine the effectiveness of a
mixed methods-based reading literacy program (computer-assisted
reading, independent tiered reading, and group-based reading teaching)
to improve reading comprehension skills, vocabulary mastery, and
reading fluency among early grade students. The study used a
quantitative approach with experimental research methods. A quasi-
experimental design was used to achieve the research objectives by
involving 350 elementary school students who were in the fourth to sixth
grade range of elementary school. The sample selection in this study
involved the purposive sampling technique using several criteria. The
teachers involved in this study totaled 150 teachers from 8 schools who
acted as program implementers at school. The reading literacy program
was implemented for two semesters for five days each week. The
research findings show that the mixed methods-based reading literacy
program had a significant effect on all competencies (word reading
efficiency, reading comprehension ability, and reading fluency) in
Grade 3 students, but this increase in ability was not seen in grades 5 and
6 students. More specifically, the effect size of the intervention on oral
reading fluency in Grade 3 students was greater than the effect size in
Grade 5 students. Furthermore, the effect size in Grade 4 was greater than
the effect size in grades 5 and 6. This is because the treatment of reading
difficulties in younger students is easier and more effective than in older
students. In addition, based on the results of the correlation analysis, the
number of words read during reading activities using a computer had a
positive relationship with word reading scores and visual word reading
efficiency in the post-test phase. Therefore, mixed methods-based
literacy programs have a significant impact on reading fluency, and on
vocabulary mastery and reading comprehension competence and
reading fluency. This research has implications in that improving
*
Corresponding author: Indah Nurmahanani; nurmahanani@upi.edu
©Authors
325
reading ability in the early grades is better done by using a combination

of methods, such as computer-assisted reading, independent tiered
reading, and group-based reading teaching. In this way, the
improvement of students’ reading skills is more optimal.
Keywords: early grade students; literacy program; mixed methods;

reading comprehension; reading fluency; vocabulary mastery
1. Introduction
Currently, researchers are focusing on various methods or programs to prevent
early grade students from experiencing reading difficulties when they enter
school age. This prevention is not only done by researchers but also supported by
the government, who facilitates various literacy programs for early grades. Early
grade literacy programs are not only held in Indonesia but also in many other
countries (e.g., the United States, England, Japan, South Korea, and Finland).
These programs take the form of partnership literacy programs, interactive
reading literacy programs, head start literacy programs, early reading literacy
programs, and many others (Higgins et al., 2015; Sampa et al., 2018). Prevention
of reading difficulties is better than overcoming reading difficulties when
students enter a higher school level (Al Jefri & Areepattamannil, 2019; Anderson
et al., 2019). Even though prevention of reading difficulties is better, in reality, the
literacy programs implemented are mostly used to overcome reading difficulties
experienced by students in lower grades, especially in schools that have not paid
optimal attention to their students’ reading abilities. One of the factors that greatly
affects reading ability is the pattern of reading habits that is carried out at home.
Children who regularly, that is at least five days a week, read story books with
their parents at home in their early grades (grades 1–3 of elementary school) tend
to have better reading skills when they enter Grade 4 of elementary school.
However, on the other hand, children who do not regularly read with their
parents at home will have difficulty in reading while at school. Naturally, the low
levels of reading literacy in the early grades will have a significant impact on
students’ academic abilities when they grow up or enter the middle class.
The Indonesian Government has held a literacy program for early childhood
students to overcome reading difficulties at school. Intervention programs
designed by teachers must accommodate the heterogeneity of students’ reading
difficulties, for students in early grades 1 to 3 and also early grades 5 to 6
(Ruotsalainen et al., 2022; Yang et al., 2018). Components that need to be
considered in improving reading skills in the early grades are phonemic
awareness, phonic awareness, vocabulary mastery, reading fluency, and reading
comprehension (Steiner et al., 2022). All the required competencies must be
accommodated in reading instructions, which will require the integration of
various methods or mixed methods. Reading difficulties are not caused by one
aspect only but several, hence the need for a literacy program that accommodates
these aspects. One example is the youth reading model, which accommodates
several aspects of reading, such as phonological awareness, word reading,
decoding, reading fluency, and language skills (vocabulary, structure, and syntax)
(Juanjuan & Mohd Yusoff, 2022; Thomas et al., 2020). Based on the method that
326
accommodates these various abilities, the researcher adapted a literacy program

with mixed methods intervention. This program is part of a literacy program
designed by the government, but the technique has been modified by the
researcher to obtain more optimal results because the literacy program of the
government is general in nature. The modification of the technique of this
program lies in the intervention of reading skills used in implementation during
the program, such as computer-assisted reading, independent tiered reading, and
group-based reading teaching.
Students who have difficulty in reading at the elementary school level will usually
show delays in participating in the learning process and low academic
achievement will start to be seen among students in grades 4 to 6 of elementary
school. In addition, students will often avoid reading learning activities and have
no motivation to learn to read (Hadianto et al., 2022; Steiner et al., 2022).
Differences in the intensity of reading activities that students receive at home have
an impact on their reading ability. Given this, teachers often find it difficult to
minimize the difference in the intensity of reading activities between students
who have good reading skills and those who have poor reading skills (Counihan
et al., 2022; Te Maro et al., 2019). Teachers must at least provide students with low
abilities exposure to intensive reading activities. This will be difficult if intensive
reading is done in the classroom, because the teacher must be fair and pay equal
attention to all students, so a program is needed that embraces both abilities
(individual and community attention). Based on these problems, the researcher
modified the reading literacy program by combining various reading-method
interventions (computer-assisted reading, independent tiered reading, and
group-based reading teaching) for early grades. The aim was to improve reading
comprehension skills, vocabulary mastery, and reading fluency in class, for both
students who are still having difficulties and those who have excelled. Differences
in reading ability on the intensity of reading literacy activities can show the
residual variance of reading ability scores in the post-test phase. This research can
serve to fill the gap in the deficiencies that were not investigated in previous
studies. Therefore, this study attempted to, first, investigate the effectiveness of a
mixed methods-based reading literacy program in improving reading
comprehension skills, vocabulary mastery, and reading fluency in early grade
students, especially students who experience difficulty in reading in early grades.
Second, the study attempted to investigate correlations between literacy program
interventions and variables of ethnicity, level of reading ability, and grade level.
2. Literature Review: Mixed Methods-Based Literacy Program

A mixed methods-based reading literacy program is a program that is designed
to overcome reading difficulties in early grade students, especially in reading
comprehension competence, vocabulary mastery, and reading fluency. This
program is designed by integrating various methods to improve early grade
reading skills (Gutierrez et al., 2021; Thomas et al., 2020). This program is
implemented by combining several methods, namely the use of computers, tiered
books, and small-group-based teaching. The integrated treatment used in the
literacy program is computer-assisted reading, independent tiered reading, and
group-based reading teaching. This program is implemented five times a week
327
for one year. Some of the books used are non-fiction books, including science,
social science, language, and mathematics, with the aim of improving students’
reading skills as well as knowledge at the elementary school level (Ioannidou
et al., 2019; Jung, 2019). In addition, teachers also provide direct instruction to
facilitate students in mastering vocabulary in these various fields of study. For
example, in the field of social sciences, teachers generate schemata of students by
asking questions about the contents of the text, carrying out reading activities
together, and introducing new words on the topic (Heyne et al., 2023; Morris et al.,
2019). In teaching vocabulary, the teacher begins by reading the word being
studied, providing an explanation of its meaning, using the word in a sentence,
asking questions whose answers must include the word, and providing additional
explanations about the word (Byrne et al., 2023; Higgins et al., 2015). Furthermore,
group teaching is done by grouping students into several small groups, where
each group consists of four or five students. In these groups, students take turns
carrying out several activities, namely doing computer-assisted reading
instructions, reading tiered books independently, and using reading models (Kim
& Riley, 2021).
This mixed methods-based literacy program is widely used in several countries,

such as the United States, England, Japan, South Korea, and Finland, where the
method is modified and adapted to the level of difficulty and reading problems
of students. The modifications made are interventions that are used according to
the level of students’ initial reading ability. There are several previous studies that
encourage researchers to use this reading literacy program in Indonesia with
method modifications (Borman et al., 2021; Hadianto et al., 2022; Karatza, 2020).
Ten studies conducted in several countries on the effect of the mixed methods-
based reading literacy program had one central finding, namely an average ability
increase of 50% in the ability to read comprehension from the initial ability
(Mawyer & Johnson, 2019). Furthermore, several previous studies have revealed
a lack of ability to read efficiently and of reading fluency among early grade
students (grades 1–6 in elementary school), which are very necessary and are a
top priority in literacy programs for early grade students. In addition, the
evaluation of a mixed methods-based reading literacy program has not optimally
separated the pure impacts of program instruction from interventions to increase
students’ learning abilities, so the pure impacts of the literacy program have not
been optimally evaluated (Morris et al., 2018, 2023). In this evaluation, the
experimental group received more intensive intervention than the control group,
so that the abilities of the students in the experimental group were superior.
Furthermore, the mixed methods employed for the literacy activity program were
mixed instruction designed to increase students’ intensity in receiving reading
literacy activities.
3. Methodology
3.1 Participants and Design
This study used the experimental method with a quasi-experimental research
design to examine the effectiveness of a mixed methods-based reading literacy
program in improving reading comprehension skills, vocabulary mastery, and
reading fluency. This study also aimed to overcome reading difficulties in early
328
grade students who are in the grade 4 to 5 range at the elementary school level.
The study participants totaled 350 elementary school students selected from 8
elementary schools in the Bandung area, Indonesia. The sample selection in this
study involved the purposive sampling technique using several criteria, namely
the school cluster, grade, gender, and ethnicity of the students. The participants
involved were identified first to determine what aspects of reading they found
difficult. Students were divided into two groups. The experimental group
received intervention through a mixed methods-based reading literacy program;
and the control group received a reading literacy program for 60 minutes after
school hours. The elementary school students involved in this study were those
whose language proficiency scores were below the minimum standard criteria.
The value of this language ability is closely related to the ability to read, so that a
comprehensive improvement is needed in reading competence. In addition, to
determine reading ability, a power analysis was carried out to investigate the
necessity of reading competence. The results of the effect size analysis yielded a
standard deviation of 0.30 on the reading ability standard. Based on the standard
correlation of 0.65–0.80 in the evaluation of the pre-test and post-test phases, it
was found that the strength of 300 to 350 participants showed an effect size of 0.30
standard deviation tested using value a = 0.05. This value is in line with the
average effect size, with a value of 0.28 in other evaluations. Based on the initial
assessment, the number of students who met the selection criteria and had
expressed their willingness to participate in the study totaled 350. Table 1 shows
the demographic characteristics of the student participants.
Table 1: Demographic aspects of student participants

Variable Percentage (%)
Grade
4 35.40
5 38.10
6 30.60
Gender
Female 50.30
Male 49.70
Ethnicity
Sundanese 85.08
Non-Sundanese 15.12
3.2 Procedure
The mixed methods-based reading literacy program adapted for this study was
carried out over two semesters or one year in the 2021–2022 academic year. The
teachers involved in this study totaled 150 teachers from 8 schools, who acted as
program implementers at school. Participating schools were scattered in several
areas in Bandung, especially schools in the disadvantaged category (low school
accreditation, state school status, and free school fees) to include many students
who still have difficulty reading. Data on students with reading difficulties were
collected from the schools’ academic records of the previous year. The mixed
methods-based reading literacy program is designed to address readers’ reading
difficulties in the early grades, which is why it has several different interventions
in comparison to the 60-minute reading literacy program. The difference lies in
329
the treatment, media, intensity of reading, and implementation of the process. The
differences between the two literacy programs for reading can be seen in Table 2.
The researcher adapted this mixed methods-based reading literacy program
based on the principles of the literacy program for early grades so that the
implementation of the literacy program was appropriate and optimal to meet the
needs of early grade students.
Table 2: Components and learning activities in the mixed methods-based reading

literacy program and 60-minute reading literacy program
Mixed methods-based reading 60-minute after-school
Program activity
literacy program literacy program
Reading using the Reading activities using a No
computer computer and assisted by
individually videos, tiered texts, and word
studies with topics from
various fields
Reading tiered books Books according to student No
independently level:
a) 25 titles (250–350 Lexiles)
b) 25 titles (400–600 Lexiles)
c) 25 titles (650–850 Lexiles)
d) 12 titles with audio
Group-based reading The teacher gives reading Teaching reading in
teaching instructions in groups. groups is carried out with
15 activities:
Students carrying out a) reading activities on
instructions in their groups social science topics;
carry out activities on reading, b) math practice through
vocabulary, reading fluency, cooperative games;
and reading comprehension. and
c) activities to develop
vocabulary and social
awareness through
book exchange and
discussion.
The mixed methods-based reading literacy program differs from the 60-minute
reading literacy program in its implementation. During the implementation of the
intervention program, students received individual computer-based reading
training for 30 minutes a day. Then, students received training in reading using
scaffolded videos and tiered text. This activity was conducted to measure
students’ reading comprehension ability. Finally, reading activities were carried
out to practice vocabulary mastery and reading fluency. Computer reading
activities use a variety of interesting topics according to student subjects, such as
society and culture, science, mathematics, and other social sciences. In the process
of reading with a computer, students follow several stages. That is, students are
given the opportunity to watch the video first to form a mental model and support
the ability to understand the text that will appear after watching the video.
Furthermore, after they have finished watching the video, the students perform
the listening activity of the text being spoken and end with an evaluation by
330
answering 15 questions to assess their level of understanding. This evaluation is

useful to determine the extent of the impact of the activities that were carried out
before.
After reading using a computer, the next activity students need to complete is to
read tiered books. These are books according to the students’ lexical level and
students are given the freedom to record reading activities as self-evaluation
material. During independent reading activities, students gain access to various
types of books facilitated by the teacher according to the students’ Lexile level. In
the third reading activity, students are placed in small groups. Students then need
to read the designated word. This intervention was carried out on students,
especially those who found it difficult to read phonetically. Students also receive
training in fluent reading by giving demonstrations; providing reading
comprehension strategies, for example by using words; knowing the context; and
analyzing key words in the reading. For more details, the difference between a
mixed methods-based reading literacy program and a 60-minute reading literacy
program can be seen in Table 2 above. Literacy program activities in the post-test
phase are assessed on all competencies to determine the results of the literacy
program intervention.
3.3 Evaluation
Several assessment activities in the pre-test and post-test phases were conducted
to determine the overall impact of the mixed methods-based reading literacy
program. These activities include assessments on word reading skills and their
efficiency, reading comprehension skills and vocabulary mastery, and reading
fluency. The researcher also recorded student attendance, conducted a survey in
the post-test phase regarding after-school activities, and measured the number of
words read during the intervention using a computer.
3.4 Research Ethics

All participants involved completed consent forms to take part in the research and
were thus involved voluntarily without any coercion. Additionally, all
participants in this study participated anonymously. Research data on students’
reading abilities in this study were used for academic purposes in their schools to
improve students’ reading literacy skills.
3.5 Assessment of Word Reading Efficiency

In evaluating word reading efficiency, the researcher adapted Torgesen’s (1999)
assessment of word reading efficiency. Assessment was carried out on a measure
of accuracy and fluency in reading words with national standards. This
assessment was carried out individually to determine the development of skills
that are very necessary in reading activities, namely the ability to recognize
vocabulary as a unit or in context (maximum score of 120) and ability to
pronounce pseudo and complex vocabulary (maximum score of 70). From the
analysis results, the reliability coefficient of the questionnaire was obtained, with
a value of more than 0.91, and the reliability coefficient of repeated assessment
was in the range of 0.85. In addition, group reading assessment was also carried
out using group assessment, with several subtests, namely vocabulary mastery,
understanding sentences, and understanding parts of sentences. Each student was
331
given a set of assessments according to their level, namely grades 4 to 6. From the
results of the reliability test, the alternative form obtained a reliability score of 0.90
for grades 4, 5, and 6. Furthermore, the reliability test obtained reliable values for
grades 4, 5, and 6, at 0.97, 0.80, 0.95, respectively, at each level.
3.6 Assessment of Reading Fluency with DIBELS Oral Reading Fluency

(DORF)
Assessment of students’ reading fluency was carried out using DIBELS (Dynamic
Indicators of Basic Early Literacy Skills). This test measures students’ ability to
read by measuring their accuracy and speed. Students are instructed to read a part
of a text made up of several words (10 words). This reading activity is carried out
by speaking for one minute at a time by completing the missing parts (jumbled
sentence), with about three seconds to complete the missing parts. Words that are
corrected accurately in three seconds indicate good reading ability. The
measurement was carried out by counting the number of words that were
successfully read correctly by students every minute. The count shows the level
of fluency in students’ oral reading. The reliability value of the DIBELS
assessment obtained was 0.93 to 0.98. Students were divided into three class
sections according to pre-test and post-test. In the post-test phase of the 60-minute
reading literacy program, students filled out a survey with 35 items to gather
information about their motivation to read and the experience they had gained
after participating in the 60-minute literacy program after school. Items 1–20
gather data on students’ intrinsic motivation in reading. Furthermore, items 21−35
gather data on students’ experiences in participating in the literacy program. In
addition, these items also reveal students’ attitudes and involvement during the
60-minute after-school literacy program.
In addition, four items collect feedback on activities that were carried out by
students in the 60-minute after-school literacy program. These are: 1) I got a
dancing learning experience in the literacy program; 2) I got a lot of knowledge and
competence in the literacy program; 3) I am enthusiastic to take part in the literacy
program; and 4) The literacy program has many rules that must be implemented. The
options provided for these items were 1 = strongly agree, 2 = somewhat agree, 3 = do
not really agree, and 4 = strongly disagree. In addition, three items attempt to identify
students’ activities at school and their opinions on the 60-minute literacy program.
These are: 1) What do you think about the activities in the 60-minute literacy program
(answer options: 1 = I don't like it, 2 = good enough, 3 = I like the program activities);
2) How many times do you read books in a week? (answer options: 1 = none, 2 = once a
week, 3 = 2–3 times a week, 4 = every day); 3) How long does it take for your family to
help you with your chores at home? (answer options: 1 = never helped, 2 = less than 20
minutes, 3 = 20−35 minutes, 4 = more than 35 minutes).
Assessment of the results of the implementation of the mixed methods-based

reading literacy program was carried out in several ways. First, the assessment
was carried out through self-reporting regarding the frequency of students’
reading activities (M = 2.80, SD = 1.20, min = 1 day, max = 5 days) to identify the
number of days in a week students read books while participating in the literacy
program. The second stage assessed the number of words read by students using
computer media. This stage assisted the researcher in predicting students’ literacy
332
levels while participating in the mixed methods-based reading literacy program.

At the end of the session, students were given 10 questions to measure
understanding and vocabulary in reading activities using a computer. From the
results of this computer reading activity, data were obtained on the number of
words read by students during the program intervention (M = 16.258, SD = 8.782).
4. Results
Before carrying out the intervention, the researcher tested the initial abilities of
the students in the experimental group (intervention mixed methods-based
reading literacy program) and the abilities of students in the control group
(60-minute literacy program). The results of the initial-ability analysis indicate no
significant differences in the three abilities of the two groups (i.e., efficiency in
reading vocabulary, reading comprehension and vocabulary mastery, and
reading fluency). There was no drastic reduction in the number of students as the
phases progressed (pre-test, intervention, and post-test) because there was control
through the assessment of student attendance. The reduction in participation was
less than 5%, which has no impact on the condition X2 (390, 1) = .025. From the
results of this analysis, the students’ reading ability and slight reduction had no
impact on the internal validity of the program intervention. Furthermore, it was
found that the total scores on word reading ability and comprehension were not
significantly different in each phase. Students who took part in all phases, from
the pre-test to the post-test, had a similar score (M = 92.47, SD = 12.40) to the five
students who took the pre-test only (M = 89.76, SD = 13.02), with t(350) = -.75 and
p > .05. Some of these findings reinforce the notion that external validity does not
interfere with the difference between students with low and high abilities.
To answer the formulation of the problem regarding the effectiveness of a mixed

methods-based reading literacy program, all the average values and standard
deviations of the abilities of word reading ability, reading comprehension and
vocabulary mastery, and reading fluency are presented in Table 3. Data analysis
shows that all abilities experienced a significant increase in the post-test phase,
especially in the efficiency of reading words and reading comprehension. In
addition, the increase in ability is also strengthened by the results of the paired t-
test, in which an increase was seen in the total score of efficiency in word reading
and phonetic decoding, as well as in the total score of vocabulary reading and
To further analyze the impact of the mixed methods-based reading literacy

program, an analysis of covariance (ANCOVA) test was performed on the post-
test scores using the scores in the appropriate pre-test phase and used as
covariates. There are several findings from the results of the ANCOVA regarding
the impact of the mixed methods-based reading literacy program and the
60-minute after-school literacy program. First, the abilities of students who
participated in the mixed methods-based literacy program intervention in the
aspect of evaluating word reading efficiency did not differ significantly from the
abilities of students who participated in the 60-minute after-school literacy
program intervention, with a value of F(1, 272) = 0.10 as well as in phonetic
decoding (F(1, 272) = 0.50) and read words (F(1, 265) = 1.89), with p = .18. In
333
addition, the value was similar to the results of the intervention of the two
programs, with the total score for reading comprehension and comprehension
having a value of F(1, 275) = .35, comprehension subtest, F(1, 275) = .42, and the
vocabulary subtest, F(1, 276) = 0.09. However, in the best proportion on the aspect
of reading fluency, it shows that the reading fluency of the experimental group
(reading literacy program based on mixed methods) was significantly superior to
the reading fluency of students in the control group (60-minute after-school
literacy program) (F(1, 275) = 4.53, p = 0.040).
Table 3: Comparison of students’ abilities in the two literacy programs

Mixed methods-based 60-minute after-school Effect
Variable
reading literacy program literacy program size
M SD Min Max M SD Min Max
Pre-test
Word reading
efficiency
Total 90.25 13.50 57 125 91.89 14.45 53 121
Reading sight
92.10 11.13 55 118 91.70 11.44 53 115
words
Phonetic
92.21 13.31 62 127 93.30 14.71 60 124
parsing
Reading
vocabulary and
comprehension
Total 91.88 12.40 60 120 91.40 12.52 55 118
Understanding 91.45 12.42 61 126 90.20 12.89 53 122
Vocabulary 93.62 13.61 57 124 94.12 12.90 55 120
Oral reading
fluency
DORF 88.54 35.30 6 180 89.20 35.30 55 170
Post-test
Word reading
efficiency
Total 96.89 14.67 62 142 93.32 12.23 70 135 0.05
Reading sight
98.21 11.56 70 125 94.61 12.30 60 125 0.15
words
Phonetic
97.10 15.10 72 142 98.64 13.57 64 132 0.05
parsing
Reading
vocabulary and
comprehension
Total 94.61 14.32 68 132 93.60 13.10 62 128 0.15
Understanding 95.80 14.72 65 124 91.50 11.30 70 120 0.10
Vocabulary 96.12 14.25 60 130 94.12 14.42 60 129 0.05
Oral reading
fluency
DORF 112.13 41.64 10 242 110.42 37.52 8 189 0.25
In addition, there is no difference in the students’ initial language skills obtained

from the results of the national standard assessment of the two programs.
Students who are involved in these two programs have language skills that do not
334
meet the minimum criteria. Furthermore, based on the results of the ANCOVA, it
was found that the total scores in vocabulary reading and reading comprehension
were higher for students who were in the experimental group than the control
group, with a value of F(1, 350) = 1.20, with p = .35. Finally, the attendance rate of
students in the experimental group (reading literacy program based on mixed
methods) was superior (M = 71.12, SD = 20.35) compared to the attendance of
students in the control group (60-minute after-school literacy program)
(M = 62.35, SD = 24.20), with t(275) = 2.642 and p = .0132. To see the effect of each
program, Table 3 presents the effect size in the post-test phase by adjusting the
covariates divided by the number of standard deviations. From the results of the
analysis, a positive and significant effect was found on oral reading fluency
(ES = 0.25) and the attendance aspect (ES = 0.35). Furthermore, the effect size on
reading comprehension competence (0.10) and vocabulary mastery competence
(0.05) were smaller than for oral reading competence and word reading efficiency.
To answer the problem formulation, the researcher presented the impact of a

mixed methods-based reading literacy program based on several variables,
namely ethnicity, gender, reading level, and clarity level. The current study also
investigated the interaction between the intervention and student characteristics
based on these variables because the mixed methods-based reading literacy
program is designed to facilitate students in groups. From the test results, no
significant differences in impact were found based on ethnicity and gender
variables. This means that ethnicity and gender do not have a strong relationship
with students’ abilities. This difference can be seen in the attendance scores of
Grade 4 students who took part in the mixed methods-based reading literacy
program, who had a better score (M = 70.45, SD = 20.50) than the students who
took part in the 60-minute after-school literacy program (M = 58.60, SD = 23.80),
with t(93) = 3.562 and p = .009. However, this contrasted to students in grades 5
and 6, who did not show a significant difference in attendance rates between the
experimental and control groups. Furthermore, based on the results of the
ANCOVA, the oral reading fluency scores of Grade 4 students in the experimental
group were superior to those of students in the control group (F(1, 93) = 9.89,
p = 0.004). Another finding is that the difference in reading fluency in grades 5
and 6 was not significant, so the effect size on reading fluency in Grade 4 was
greater than the effect size on reading fluency in Grade 5 (ES = .01) and Grade 6
(ES = .06) students in the mixed methods-based reading literacy program. From
these findings, it can be concluded that the effect that appears to be the most
significant is on the competency of oral reading fluency and the enthusiastic
attendance aspect of students participating in mixed methods-based reading
literacy programs.
Furthermore, to see students’ attitudes towards all the instructions in the two
programs, the researcher analyzed the survey data to reveal their experiences
while participating in these two literacy programs. The results of the analysis of
student responses to the questionnaire items show that students who were in the
experimental group (reading literacy program based on mixed methods) gave
more affirmative responses to the positive items given compared to students who
were in the control group (60-minute after-school literacy program). The response
335
scores on several items were: a) I learned new and interesting things in the program
(t(283) = 2.20, p > .05) and b) I learned a lot (t(283) = 2.80, p > .01). In addition,
students in the experimental group were also found to be more intensive in
reading books compared to students in the control group (M = 2.20, SD = 1.09),
with t(280) = 7.78 and p >.001. The students in the experimental group received
classes for five days a week, while those in the control group only one to two days
a week. In addition, there were no significant differences in the survey results
regarding the experiences of the two programs. The results of the descriptive
analysis of student literacy resulting from the intervention of the mixed methods-
based reading literacy program and the results of the intercorrelation analysis of
reading skills in both phases and the impact of the mixed methods-based reading
literacy program are presented in tables 4 and 5, respectively.
Table 4: Descriptive analysis of student literacy in the mixed methods-based reading

literacy program
Measures Range Percentile
M SD Min Max. 10 25 50 75 90
Total words
15.423 10.758 525 70.658 5.582 8.742 13.125 19.224 28.230
(raw score)
Total words
14.52 1.08 10.10 17.10 13.32 13.89 14.60 15.18 15.70
(Log 2)
Attendance 73.46 16.92 15 90 56 70 78 84 88
Table 5: Intercorrelation analysis of reading skills in both phases and the impact of
the mixed methods-based reading literacy program
1 2 3 4 5 6 7 8 9 10 11 12 13
1. Post-test
–
Comprehension
2. Post-test
.72 –
Vocabulary
3. DIBELS post-
test
.54 .55 –
Oral reading
fluency
4. Post-test
Phonemic .50 .55 .60 –
decoding
5. Post-test
Sight word .50 .55 .68 .72 –
reading
6. Pre-test
.70 .68 .45 .45 .55 –
Comprehension
7. Pre-test
.63 .71 .53 .57 .60 .67 –
Vocabulary
8. DIBELS pre-
test
.53 .50 .90 .57 .60 .50 .56 –
Oral reading
fluency
9. Pre-test
Phonemic .47 .56 .60 .88 .73 .49 .60 .60 –
decoding
10. Pre-test .48 .52 .63 .63 .82 .55 .53 .62 .74 –
336
Sight word
reading
11. Mean days
per week of .07 -.06 -.11 .14 .02 -.08 -.03 -.16 -.12 .12 –
book reading
12. Words read
.35 .42 .54 .38 .45 .32 .30 .50 .40 -.37 .04
(raw score)
13. Words read
.40 .50 .52 .30 .48 .40 .30 .50 .36 .45 -.06 .87 –
(log base 2)
Next, multiple regression analysis was performed on reading comprehension and

vocabulary scores, with the results presented in Table 6. The results of the
multiple regression analysis on reading fluency scores, word reading, and
phonemic decoding are presented in Table 7. These two tables depict the impact
of the two literacy programs on these competencies. Model 1 includes the reading
pre-test scores and variables that represent impact measures of the literacy
program in Model 2. The effects of scores from the pre-test phase of reading
comprehension and vocabulary competence, reading fluency and phonemic
decoding, and word reading are omitted from the regression analysis in Model 2.
Next, it is determined whether competence in reading books and words in reading
literacy activities based on mixed methods with the help of a computer can
explain the variance in the post-test phase. The results of the analysis are
presented in full, including the non-standard regression coefficient (B), the
standard error B (SE B), the standard regression coefficient (b), and the
correlation (R2).
Table 6: Results of the multiple regression analysis of reading comprehension skills

and vocabulary in the mixed methods-based reading literacy program
Dependent variable B SE B Final b R2
Reading comprehension
Model 1
1. Understanding .55 .11 .48**
2. Vocabulary mastery .28 .11 .25*
3. Reading fluency .09 .04 .20*
4. Phonemic breakdown -.04 .12 -.04
5. Reading sight words .05 .14 .05 .60
Model 2
2. Vocabulary mastery .30 .11 25*
3. Reading fluency .08 .05 .20*
4. Phonemic breakdown .04 .12 .03
5. Reading sight words .03 .14 .03
6. Average number of days per week for
.90 .89 .07
reading
7. Number of words read on the computer .90 .94 .08 .60
Vocabulary
Model 1
1. Understanding .40 .11 .33***
2. Vocabulary mastery .43 .10 .40***
3. Reading fluency .03 .05 .06
337

Model 2
2. Vocabulary mastery .45 .11 .43***
6. Average number of days per week for
.84 .85 .08
reading
7. Number of words read on the computer 3.81 .90 .25** .65
Note. * p < .05, ** p < .01, *** p < .001
Table 7: Results of multiple regression analysis of reading fluency competence, word

reading, and phonemic decoding of students in the mixed methods-based reading
literacy program
Dependent variable B SE B Final b R2
DIBELS oral reading fluency (DORF)
Model 1
1. Understanding .04 .20 .02
2. Vocabulary mastery .11 .20 .05
3. Reading fluency .83 .07 .78***
Model 2
3. Reading fluency .80 .08 .78***
6. Average days per week for reading books 1.40 1.72 .05
7. Number of words read on the computer 3.90 1.81 .10 .80
Sight word reading
Model 1
4. Phonemic breakdown .23 .08 .25**
5. Reading sight words .60 .09 .58*** .75
Model 2
4. Phonemic breakdown .25 .08 .27**
5. Reading sight words .60 .10 .58***
6. Average days per week for reading books 1.90 .58 .12*
7. Number of words read on the computer 1.41 .60 .15* .82
Phonemic decoding
Model 1
338

4. Phonemic breakdown .96 .10 .83***
5. Reading sight words -.07 .12 -.05 .78
Model 2
4. Phonemic breakdown .95 .10 .82***
6. Average days per week for reading books .62 .76 .05
7. Number of words read on the computer .53 .79 .06 .78
Note. * p < .10, * p < .05, ** p < .01, *** p < .001
From the results of the two regression analyses, a significant correlation was
found between the competence of the number of words read in the mixed
methods-based reading literacy program in the pre-test and post-test phases.
Based on the results in Table 6, the number of words read has a positive
relationship with vocabulary scores and reading comprehension in the post-test
phase. From the results of the analysis, an additional 5% variance was found in
the post-test phase vocabulary score. Furthermore, based on the results in Table 7,
the number of words read has a positive relationship with the score of the ability
to read sight words in the post-test phase. The number of words read in this
computer-assisted reading activity provides an additional 4% variance in the
post-test phase for word reading competence. From these findings, it can be
concluded that computer-assisted word reading has a positive relationship with
post-test vocabulary scores and sight word reading. The multilevel model was
applied to the experimental group students. In addition, the researcher also
replicated the regression results presented in Tables 6 and 7.
5. Discussion
Three instructions were used in this mixed methods-based reading literacy
program designed to improve competence in word reading efficiency, reading
comprehension and vocabulary mastery, and reading fluency. Activities in the
modified mixed methods-based reading literacy program are computer-assisted
reading, independent tiered reading, and group-based reading teaching. Based on
the results of the research, this mixed methods-based reading literacy program
provides instruction that encourages students to master vocabulary with a high
frequency of occurrence in all fields (Farkas & Jang, 2019; Koutsouris et al., 2021).
This direct instruction on learning vocabulary can help students who have
difficulty understanding words to gain useful knowledge directly towards
improving reading comprehension skills when reading texts. The main cause of
difficulties in students’ reading ability is vocabulary, which is limited when
students are in early grades 1 to 3. This difficulty thus continues and is very visible
when students are required to have good reading comprehension competence in
grades 4 to 6 of elementary school and middle and high school. This finding is in
accordance with the theory that the main strength of reading comprehension skills
is students’ ability to understand each word that forms the text (Bayless et al.,
2018; Lo & Leung, 2022).
339
From the results of the study, although there were several competencies that
showed insignificant improvement, this mixed methods-based reading literacy
program as a whole was able to improve reading comprehension skills because it
presented various activities that accommodated student characteristics, such as
learning through computers, reading tiered books, using models, and teaching
group-based reading (Metsala & Kalindi, 2022; Steiner et al., 2022). With these
various methods, a significant increase was found in aspects of students’
vocabulary mastery and reading fluency, compared to other competencies. The
curriculum used by schools is also one of the factors that can increase students’
ability to read words efficiently and their ability in reading comprehension. Based
on the results of an analysis of the curriculum material, the teachers involved in
the 60-minute literacy program were given the opportunity to freely choose
various reading activities to improve students’ reading skills, so that the teachers
carried out a variety of reading activities (Dellisse et al., 2021; Yan & Cai, 2022).
This is why the 60-minute literacy program has a different and less than optimal
effect on students’ abilities. In contrast to the mixed methods-based reading
literacy programs, this literacy program was initially designed to improve early
grade students’ reading skills using three uniform methods so that teachers have
guidance in its implementation. The two programs share a similar method,
namely group-based reading teaching.
The combination of reading methods using computer media, reading of tiered

books, and group-based teaching can improve students’ reading skills more
optimally. This is in accordance with the theory that a combination of digital and
traditional-based methods will be more optimal in increasing students’ academic
achievement abilities (Hadianto et al., 2021a; Snow et al., 2020). This also applies
to reading competence. Furthermore, correlation studies and experiments
conducted have shown that it is more difficult to improve the reading skills of
students who are late in developing their reading skills when they enter grades 4
to 6 compared to improving students’ skills during grades 1 to 3 of elementary
school (Counihan et al., 2022; Metsala & Kalindi, 2022). Several other longitudinal
studies have also made the same finding, namely that there is an ability gap
between students who have good and poor reading skills when they enter a
higher grade (Jung, 2019; Karatza, 2020). Students who master the alphabet when
they are in the early grades will have more opportunities to become proficient
readers than students who are late in developing reading skills, including the
competencies of word reading efficiency, reading comprehension and vocabulary,
and reading fluency.
The results of this study reinforce this theory in several ways. Notably, the trials
of the mixed methods-based reading literacy program had a significant effect on
all competencies (word reading efficiency, reading comprehension ability, and
reading fluency) in Grade 3 students, but this ability increase was not seen in
students in grades 5 and 6. More specifically, the effect size of the intervention on
oral reading fluency in Grade 3 students was greater than the effect size in Grade
5 students. Furthermore, the effect size in Grade 4 was greater than the effect size
in grades 5 and 6. These findings are in accordance with the theory that treatment
of reading difficulties in younger students will be easier and more effective than
340
treatment of reading difficulties in older students (Hadianto et al., 2021b; Thomas

et al., 2020). These findings are consistent with findings of remedial reading
interventions with a multi-grade sample, which show that improving reading
difficulties in older students is more difficult than in younger students.
Furthermore, based on the results of the regression analysis on students’ ability in
mixed methods-based reading literacy programs, this reading literacy program
can improve overall reading ability in several competencies. Improvements were
seen in the competencies of word reading efficiency, reading comprehension and
vocabulary, and reading fluency. In addition, based on the results of the
correlation analysis, the number of words read during reading activities using a
computer has a positive relationship with word reading scores and visual word
reading efficiency in the post-test phase (Thomas et al., 2020; Waldron, 2018).
6. Conclusion, Limitations and Recommendations

The reading literacy program based on mixed methods was more effective in
treating reading difficulties in grades 4 to 6 students than the 60-minute after-
school literacy program. Improvements in student competence were seen in the
aspects of word reading efficiency, reading comprehension, sight word reading,
and reading fluency, with the most significant increase seen in students’ reading
fluency. Students in Grade 4 experienced the most significant improvement in
competencies compared to those in grades 5 and 6. This is because it is easier to
correct reading difficulties for younger students than for older students. In
addition, of the several reading methods used in literacy programs, computer-
assisted reading activities have proven to be the most effective in increasing
competence in word reading efficiency and reading fluency. The implication of
this research is the need for integration of various literacy activities in reading
literacy programs to improve students’ reading skills. The various reading
methods in the literacy program means various student characteristics can be
accommodated.
The study had several limitations, including the small sample and number of
schools, and the schools not representing developed and developing areas.
Furthermore, the timespan of the study was still relatively short and the study
compare only two different programs. Future research may compare several
programs at the same time. In addition, the study did not consider the aspect of
gender and the need for feedback regarding program deficiencies from
stakeholders.
Based on some of these deficiencies, several recommendations are made for future
research. First, the sample in the study must be enlarged by involving both
students who experience difficulty and those who are proficient in reading.
Second, schools must be included that represent developed and developing areas.
Third, various times for each treatment in the literacy program must be allowed.
Fourth, teaching reading to early grade students should be done using various
methods. It is better to focus early reading skills on basic abilities such as
vocabulary mastery and reading fluency, only after which more complex abilities,
namely reading comprehension and effective reading, can be targeted. In
addition, teaching reading in the early grades should not only be carried out in
341
the classroom but must be accompanied by literacy programs outside the

classroom. Research is needed to obtain feedback from users and stakeholders
involved in the literacy programs.
7. References
Al Jefri, H. M., & Areepattamannil, S. (2019). Examining the relations of early literacy
activities and skills to reading dispositions, engagement, and achievement among
fourth-grade students in the United Arab Emirates. Social Psychology of Education,
22(4), 901–920. https://doi.org/10.1007/s11218-019-09504-7
Anderson, K. L., Atkinson, T. S., Swaggerty, E. A., & O’Brien, K. (2019). Examining
relationships between home-based shared book reading practices and children’s
language/literacy skills at kindergarten entry. Early Child Development and Care,
189(13), 2167–2182. https://doi.org/10.1080/03004430.2018.1443921
Bayless, S. D., Jenson, J. M., Richmond, M. K., Pampel, F. C., Cook, M., & Calhoun, M.
(2018). Effects of an afterschool early literacy intervention on the reading skills of
children in public housing communities. Child and Youth Care Forum, 47(4),
537−561. https://doi.org/10.1007/s10566-018-9442-5
Borman, G. D., Yang, H., & Xie, X. (2021). A quasi-experimental study of the impacts of
the Kids Read Now Summer Reading Program. Journal of Education for Students
Placed at Risk, 26(4), 316–336. https://doi.org/10.1080/10824669.2020.1855985
Byrne, E., Tulloch, C., Sohail, N., & Diazgranados Ferráns, S. (2023). Harnessing cost data
to improve early grade reading: Cost evidence from a large-scale literacy initiative
in Pakistan. Journal of Development Effectiveness, 15(1), 43–76.
https://doi.org/10.1080/19439342.2022.2034915
Counihan, C., Humble, S., Gittins, L., & Dixon, P. (2022). The effect of different teacher
literacy training programmes on student’s word reading abilities in government
primary schools in Northern Nigeria. School Effectiveness and School Improvement,
33(2), 198–217. https://doi.org/10.1080/09243453.2021.1991960
Dellisse, S., Dumay, X., Galand, B., Dupriez, V., Dufays, J. L., Coertjens, L., De Croix, S., &
Penneman, J. (2021). Does the treatment integrity of a literacy instruction program
foster student reading comprehension? A field experiment. European Journal of
Psychology of Education, 36(3), 573–597. https://doi.org/10.1007/s10212-020-
00499-w
Farkas, W. A., & Jang, B. G. (2019). Designing, implementing, and evaluating a school-
based literacy program for adolescent learners with reading difficulties: A mixed-
methods study. Reading and Writing Quarterly, 35(4), 305–321.
https://doi.org/10.1080/10573569.2018.1541770
Gutierrez, A., Lowe, K., & Guenther, J. (2021). Indigenous student literacy outcomes in
Australia: A systematic review of literacy programmes. Asia-Pacific Journal of
Teacher Education, 49(1), 37–60. https://doi.org/10.1080/1359866X.2019.1700214
Hadianto, D., Damaianti, V. S., Mulyati, Y., & Sastromiharjo, A. (2021a). Does reading
comprehension competence determine level of solving mathematical word
problems competence? Journal of Physics: Conference Series, 1806(1), 012049.
https://doi.org/10.1088/1742-6596/1806/1/012049
Hadianto, D., Damaianti, V. S., Mulyati, Y., & Sastromiharjo, A. (2021b). The role of
multimodal text to develop literacy and change social behaviour foreign learner.
Hadianto, D., Damaianti, V. S., Mulyati, Y., & Sastromiharjo, A. (2022). Effectiveness of
literacy teaching design integrating local culture discourse and activities to
enhance reading skills. Cogent Education, 9(1), 2016040.
https://doi.org/10.1080/2331186X.2021.2016040
Heyne, N., Gnambs, T., Lockl, K., & Neuenhaus, N. (2023). Predictors of adolescents’
342
change in reading literacy: The role of reading strategies, reading motivation, and
declarative metacognition. Current Psychology, 0123456789.
https://doi.org/10.1007/s12144-022-04184-7
Higgins, E., Fitzgerald, J., & Howard, S. (2015). “Literacy Lift-Off”: An experimental
evaluation of a reading recovery programme on literacy skills and reading
self-concept. Educational Psychology in Practice, 31(3), 247–264.
https://doi.org/10.1080/02667363.2015.1030592
Ioannidou, E., Kiourti, E., & Christofidou, C. (2019). Literacy education in prison:
Developing a social literacy programme in the prison school of Cyprus. Research
Papers in Education, 34(5), 597–620.
https://doi.org/10.1080/02671522.2018.1524925
Juanjuan, G., & Mohd Yusoff, N. (2022). The shared features of effective improvement
programmes for teachers’ assessment literacy. Pedagogies: An International Journal,
1–18. https://doi.org/10.1080/1554480X.2022.2084398
Jung, Y. (2019). Associations between in-the-moment behavior of children, family literacy
program use, and Latina mother–child book reading. Early Child Development and
Care, 189(13), 2071–2084. https://doi.org/10.1080/03004430.2018.1438423
Karatza, S. (2020). Multimodal literacy and language testing: Visual and intersemiotic
literacy indicators of reading comprehension texts. Journal of Visual Literacy,
39(3−4), 220–255. https://doi.org/10.1080/1051144X.2020.1826222
Kim, Y. & Riley, D. (2021). Accelerating early language and literacy skills through a
preschool–home partnership using dialogic reading: A randomized trial. Child and
Youth Care Forum, 50(5), 901–924. https://doi.org/10.1007/s10566-021-09598-1
Koutsouris, G., Norwich, B., & Bessudnov, A. (2021). Interpreting RCT, process evaluation
and case study evidence in evaluating the Integrated Group Reading (IGR)
programme: A teacher-led, classroom-based intervention for Year 2 and 3 pupils
struggling to read. Educational Review, 73(3), 346–373.
https://doi.org/10.1080/00131911.2018.1557597
Lo, Y. Y., & Leung, C. (2022). Conceptualising assessment literacy of teachers in content
and language integrated learning programmes. International Journal of Bilingual
Education and Bilingualism, 25(10), 3816–3834.
https://doi.org/10.1080/13670050.2022.2085028
Mawyer, K. K. N., & Johnson, H. J. (2019). Eliciting preservice teachers’ reading strategies
through structured literacy activities. Journal of Science Teacher Education, 30(6),
583–600. https://doi.org/10.1080/1046560X.2019.1589848
Metsala, J. L., & Kalindi, S. C. (2022). The effects of a computer-based early reading
program on the literacy skills of kindergarten students. Computers in the Schools,
39(4), 373–393. https://doi.org/10.1080/07380569.2022.2127344
Morris, R., See, B. H., Gorard, S., & Siddiqui, N. (2023). Literacy for life: Evaluating the
National Literacy Trust’s bespoke programme for schools. Educational Studies,
49(2), 369–387. https://doi.org/10.1080/03055698.2020.1867077
Morris, S. P., Wishart, R., Husain, F., Marshall, L., & Vojtkova, M. (2019). Evaluating the
effectiveness of a family literacy programme on the attainment of children with
English as an additional language: A cluster randomised controlled trial.
Educational Research, 61(4), 408–424.
https://doi.org/10.1080/00131881.2019.1657779
Ruotsalainen, J., Soodla, P., Räikkönen, E., Poikkeus, A. M., Kikas, E., & Lerkkanen, M. K.
(2022). Literacy instruction activities and their associations with first graders’
reading performance in two transparent orthographies. Compare, 52(1), 92–109.
https://doi.org/10.1080/03057925.2020.1742093
Sampa, F. K., Ojanen, E., Westerholm, J., Ketonen, R., & Lyytinen, H. (2018). Literacy
programs efficacy for developing children’s early reading skills in familiar
language in Zambia. Journal of Psychology in Africa, 28(2), 128–135.
343
https://doi.org/10.1080/14330237.2018.1435050
Snow, P. C., Graham, L. J., Mclean, E. J., & Serry, T. A. (2020). The oral language and
reading comprehension skills of adolescents in flexible learning programmes.
International Journal of Speech–Language Pathology, 22(4), 425–434.
https://doi.org/10.1080/17549507.2019.1652343
Steiner, L. M., Hindin, A., & Rizzuto, K. C. (2022). Developing children’s literacy learning
through skillful parent–child shared book readings. Early Childhood Education
Journal, 50(4), 539–553. https://doi.org/10.1007/s10643-021-01170-9
Te Maro, P., Lane, C., Bidois, V., & Earle, D. (2019). Literacy, achievement and success in
a Māori tourism certificate programme: Reading the world in order to read the
word. International Journal of Lifelong Education, 38(4), 449–464.
https://doi.org/10.1080/02601370.2019.1648576
Thomas, N., Colin, C., & Leybaert, J. (2020). Interactive reading to improve language and
emergent literacy skills of preschool children from low socioeconomic and
language-minority backgrounds. Early Childhood Education Journal, 48(5), 549–560.
https://doi.org/10.1007/s10643-020-01022-y
Torgesen, J. K., Wagner, R. K., & Rashotte, C. A. (1999). Test of word reading efficiency.
PRO-ED.
Waldron, C. H. (2018). “Dream more, learn more, care more, and be more”: The
imagination library influencing storybook reading and early literacy. Reading
Psychology, 39(7), 711–728. https://doi.org/10.1080/02702711.2018.1536094
Yan, J., & Cai, Y. (2022). Teachers’ instruction of reading strategies and primary school
students’ reading literacy: An approach of multilevel structural equation
modelling. Reading and Writing Quarterly, 38(2), 139–155.
https://doi.org/10.1080/10573569.2021.1923100
Yang, G., Badri, M., Al Rashedi, A., & Almazroui, K. (2018). The role of reading
motivation, self-efficacy, and home influence in students’ literacy achievement: A
preliminary examination of fourth graders in Abu Dhabi. Large-Scale Assessments
in Education, 6(1), 10. https://doi.org/10.1186/s40536-018-0063-0
344

Vol. 22, No. 7, pp. 344-355, July 2023
Received Mar 4, 2023; Revised May 9, 2023; Accepted Jul 22, 2023
A Conceptual Analysis of What it Means to

Decolonize the Curriculum
Bunmi Isaiah Omodan , Pretty Thandiswa Mpiti and Nomxolisi Mtsi
Walter Sisulu University, South Africa
Abstract. The concept of decoloniality has been subject to wide-ranging

debates among academics and students alike. These discussions have
often looked at the issue from a subjective stance, narrowing its meaning
down to interpretation based on individuals’ backgrounds and contexts.
Consequently, the understanding of decoloniality as it relates to
university curricula has remained fragmented, leading to inconsistencies
in how it is put into practice. This study, therefore, seeks to address this
gap by unpacking the conceptual ambiguity surrounding what it means
to decolonize the curricula in university classrooms. Hence, this study
intends to deconstruct the decoloniality of the curriculum as it relates to
the pedagogical disposition of the classroom in universities using
decolonial theory as the basis of argument. The study answers questions
about the assumptions of decoloniality suitable to understand the
decoloniality of the curriculum. This conceptual analysis is located within
a transformative worldview as a lens and employs conceptual analysis as
a tool to make sense of the argument deductively from the decolonial
premises. The study argues that decolonizing the curriculum is beyond
any personal, contextual, historical, and environmental subjectification,
and should instead be viewed as advocacy to challenge existing power
dynamics towards incorporating traditionally overlooked or excluded
ways of doing. Therefore, it is essential to understand “decolonizing the
curriculum” from the process of knowing, empowering the
disempowered, self-determination, and an anti-oppressive perspective.
Keywords: conceptual analysis; curriculum; decoloniality; decolonizing

the curriculum; transformative worldview
1. Introduction
Decoloniality has become an increasingly prominent concept for academics and
students, who have examined it in depth regarding its implications for diverse
sociocultural landscapes. This has stimulated vibrant dialogue about the impact
of unequal power dynamics by exploring mechanisms such as those associated
with colonization and imperialism that constrain the ability of an array of people
to shape their own future. Discourses around decoloniality demand, perhaps
more than anything else, a systematic examination of how colonial structures can
©Authors
345
be made visible so they can be disrupted appropriately (Sauerbronn et al., 2021;

Sium et al., 2012). However, critics have criticized decolonial writers as failing to
fully understand their circumstances and as engaging in simplistic writing
without sufficient understanding of context or nuance (Dunlap, 2022; Martínez-
Novo, 2018; McDowall & Ramos, 2018). Nonetheless, it remains important to
understand that decoloniality is not merely a topic for theoretical discourse but
has important implications for real-world social movements seeking to create
lasting change (Deem, 2019). Therefore, these debates not only uncover existing
systems that act as impediments to liberation but also give voice to alternative
forms of knowledge production grounded in indigenous thinking and local
practices (Martineau & Ritskes, 2014).
While this decolonial worldview offers an undeniably useful source of discursive

material and ideas, it is essential to recognize the subjective nature of its
application (de França Sá & Marsico, 2022). This is because the accomplishments
brought about by the said discussions or the scholars of it are not always
applicable in each context or based on individuals’ backgrounds. As such,
interpretations of the same subject matter may vary profoundly despite
originating from the same perspective. Therefore, a better understanding of its
reach and implementation can be achieved by paying attention to how subjective
viewpoints can affect meaning-making.
The lack of consensus among theorists and academics regarding the concept of
decoloniality has propagated a deep-rooted divide in how it is implemented in
university curricula. This has led to distorted interpretations of decoloniality,
obscuring its true purpose and creating conflicting ideas of its role within the
academic context. As a result, ambiguity persists, while universities struggle to
define and implement meaningful strategies for deconstructing power relations
between different knowledge systems and pedagogies. We noticed a considerable
disinterest or lack of knowledge among university students, lecturers, and other
scholars when it comes to an understanding of a decolonized curriculum,
decolonized university classrooms, and the implications of decoloniality in higher
education. As such, this study is motivated to develop a comprehensive
understanding of these concepts and facilitate better critical inquiry and thought
about the ways in which power structures within universities can become more
equitable for all. It is hoped that through careful consideration of these ideas, light
may be shed on the far-reaching effects of colonial legacies (Maseland, 2018)
found in modern education systems.
Therefore, in order to rectify these discrepancies, there is an urgent need to

conceptually deconstruct the foundations of decoloniality from the ground up. By
doing so, it would be possible for the targeted audience, such as students,
lecturers, and academics, to understand the values at the core of decoloniality on
a greater level and bring about more substantial change towards achieving a truly
clear knowledge curriculum and classroom practices within the principles of
decoloniality. Hence, this study intends to deconstruct the decoloniality of the
curriculum (decolonizing the curriculum) as it relates to the classroom
pedagogical disposition using decolonial theory as the basis of argument.
346
2. Research Questions
Based on the above problem, the following research questions were raised to
guide the study:
• What is decoloniality?
• What are the assumptions of decoloniality capable of defining “decolonizing
the curriculum”?
3. Methodology
This conceptual paper is located within the transformative paradigm. Given the
current ambiguity in the decolonization of the curriculum as a concept, it is
necessary to deconstruct the concept. Therefore, this conceptual paper is lensed
within the transformative worldview, allowing us to unpack and examine the
potential of decoloniality vis-à-vis its correlational tendencies with university
classrooms. This paradigmatic approach is particularly appropriate for evaluating
this phenomenon given its capabilities of providing a critical analysis of current
conditions and suggesting innovative solutions that are inspired by an alternative
standpoint from which to pursue meaningful reform (Carangio, 2021; Mertens,
2007, 2010). Since higher education institutions are one of the entities that is
uniquely positioned to initiate transformative empowerment that empowers
individuals, including students, academics, and lecturers, this paper therefore
contributes to transformative discourse in academic literature. By changing the
existing situation in ways that broaden perspectives and encourage growth in
knowledge, attitudes, and skills across multiple aspects of life, academic
communities achieve deeper levels of understanding which support an ever-
evolving capacity to live more sustainably. Such transformations can lead to
meaningful academic research, civic engagement, and an increase in
opportunities for the academic community.
Conceptual analysis was employed as a tool to make sense of the argument

deductively from the decolonial premises. Conceptual analysis is a well-suited
tool for deductive argumentation due to its ability to precisely identify important
concepts specific to the required evidence (Furner, 2004; Valchev, 2018). When
dealing with an argument, conceptual analysis can be used to parse out key terms
(von Colln-Appling & Giuliano, 2017), as well as to critically examine the
underlying assumptions of each claim that is made. This offers the resource of
evidence from which meaningful conclusions can be drawn. Additionally,
conceptual analysis helps to depict a more precise meaning behind a statement by
making subtle distinctions that may have been overlooked (Valchev, 2018; von
Colln-Appling & Giuliano, 2017). Ultimately, though, this precise identification
makes it particularly suitable for studying the strength of logical arguments
through rigorous scrutiny and comparison. In this study, decoloniality concept is
presented, within which its assumptions are argued alongside its relationships
with the decoloniality of the curriculum and university classroom practices.
To start with, the concept of decoloniality is unpacked, and its assumptions vis-à-
vis their relationship with the curriculum towards understanding the term
decolonized curriculum or decoloniality of the curriculum are presented.
347
4. Presentation and Analysis

This section presents the concept of decoloniality, its assumptions, and how they
can be explored to clarify what it means to “decolonize the curriculum”.
4.1 Decoloniality as a Concept

Decoloniality is an academic, political, and social movement that emerged to
provide an alternative to both colonial structures of power and post-colonialism,
mostly linked to Latin America (Lentin, 2021) but with growing acceptance from
around the world. It focuses on challenging the dominant Western European
colonizer-imperialist narrative which has dominated for centuries and aims to
empower oppressed people worldwide by legitimizing their knowledge,
languages, and cultures (Kerr, 2014; Manning, 2018). The aim of decoloniality is
to help create a more equitable relationship between colonizers and those
previously colonized, working towards a more just society in terms of politics,
socioeconomics, and culture (Shizha, 2010). Crucially, it seeks justice through not
only undoing but also decolonizing cultural categories such as race, class, gender,
and sexuality – which are often deeply embedded in power structures (Ndlovu‐
Gatsheni, 2015a, 2015b; Schiwy, 2007). Consequently, one can argue that
decoloniality works beyond understanding history mainly as a struggle between
conquerors and conquered; using a critical lens, this framework shifts to how we
should respond to colonialism while looking at past circumstances within today’s
context so as to work towards meaningful transformation.
From the perspectives of Moyo and Mutsvairo (2018) and Ndlovu-Gatsheni

(2019), decoloniality brings forth a counter-hegemonic framework of
categorization and understanding the complexities of modern society. The
framework questions how knowledge is constructed not only through systems of
colonialism but also within different communities and social groups which have
suffered from centuries of underrepresentation. That is, by emphasizing the
importance of self-determination for individuals, decoloniality puts the agency in
the hands of people who have been traditionally excluded from different societies,
changing oppressive power dynamics in education and collective understanding.
Although decoloniality is often understood as being related to decolonization, the

two terms are not synonymous. While decoloniality shares some common traits
with decolonization, namely rejecting hierarchies of power structures based on
imperialism or colonization, it is not to be confused as synonymous. That is,
decolonization pertains to past processes or struggle for liberation from the
colonizer’s political grip (Omanga, 2020), whereas decoloniality opens up further
avenues for post-colonial thought and action through an analytical approach that
centers anti-oppressive narrative practices (Trisos et al., 2021). Central to the
concept of decoloniality is the emphasis on understanding how systems
embedded in colonial knowledge produce inequality. However, decoloniality
recognizes that current power dynamics enabled by imperial and colonial forces
whose distortions remain visible today in global culture, economics, politics, and
education signify a proactive stance towards reorienting established knowledge
production methods and disrupting colonial forms of oppression at all levels.
348
Based on the above, one can argue that decoloniality challenges Euro- or
Americentric epistemology and seeks a decolonized knowledge in discourse. This
knowledge can be created through reclaiming or revitalizing indigenous
knowledge and by considering how Western education may be oppressive or
Eurocentric (Bergström, 2021), for example the concept of settler colonialism. In
practical terms, decoloniality may guide, for instance, the selection of literature on
school reading lists; it might also inform more principal aspects of curriculum
design, such as student-centered learning. Ultimately, decoloniality strives to
re-orient spaces of education around polyvocal perspectives, which include
narratives from various sources representing different identities. This is an
indication that decoloniality is a movement that cuts across all sectors, including
education, which justifies the need to decolonize the curriculum in university
spaces. Such endeavor involves a critical analysis of courses, instructional
materials, methodological approaches, and assessment tools that reflect the values
of all learning communities, free from oppressive narratives, intolerant beliefs,
and politics of hate.
4.2 Assumptions of Decoloniality Concept

From the above presentation of the decolonial concept, the following assumptions
were deduced as some of the features of decoloniality that will be considered in
this study. That is, decoloniality advocates challenging the process of knowing,
empowering the disempowered, and preaching for self-determination, and is
anti-oppressive in nature. These four assumptions are discussed below.
4.2.1 Challenging the process of knowing

From the above concept of decoloniality, it was deduced that decoloniality is a
form of resistance which seeks to confront and disrupt the domination of
knowledge systems based on Euro- or Americentric power. It works to challenge
epistemology, the processes and theories through which people gain knowledge
about the world (Keet, 2014; Mitova, 2020). Controlling the kind of knowledge
imposed by Westerners for centuries creates new spaces for exploration and
recognition of inherent diversity and plurality in how knowledge is produced,
accessed, and acted upon (Mignolo, 2012). Decoloniality not only questions
existing epistemologies but also offers innovative roads that create more equitable
knowledge production, proving ground-breaking value to our understanding of
issues of power, knowledge, and liberation in a global context (Kubota, 2022).
Challenging the colonial way of knowing provides an inclusive opportunity for

people to become part of the knowledge production process and opens up
possibilities to build dynamic spaces for intellectual inquiry never before grasped
(Fellner, 2018). Therefore, through decolonial frameworks, we can engage in a
discourse that interrogates thoughts, enhances understanding of how power
systems create knowledge in an oppressive manner, and reveals how these
systems can be challenged or re-ordered. In this way, decoloniality is important
not only in its ability to challenge epistemology and the process of knowing but
also to construct an environment where ordinary people are not limited by those
who dictate.
349
4.2.2 Empowering the disempowered

Decoloniality is a movement that aims to disrupt the systems of colonialism in
place by empowering and liberating those who have been subjugated by it
throughout history (Mays, 2019). This is achieved by recognizing and challenging
the oppressive, colonial power structures that exist, both on the macro-level, such
as dominant civilizations and nations, and on the micro-level, such as within
educational curricula or other aspects of everyday life. In bringing attention to
these structures, decoloniality seeks to create an environment wherein efforts can
be made to restore agency and dignity to these communities which have been
deprived for so long. Ultimately, this serves to promote equity and justice among
all peoples regardless of prior uneven distributions of power due to the influence
of colonial structures (Garcia, 2018).
Based on this argument, one can view decoloniality from a theoretical level, where
it examines the power structures of colonialism, which shaped the current socio-
political landscape and advocates for resisting and overturning these embedded
unequal systems. In practice, on the other hand, decoloniality foregrounds the
voices of people who have traditionally been excluded from the “mainstream”
narrative – striving towards an equitable society where all individuals are
empowered regardless of their background. This is achieved through initiatives
such as educationally re-centering indigenous knowledge, creating opportunities
through, for example, job training programs and mentoring schemes, and
expanding access to resources. Ultimately, decoloniality works to build a more
equitable future by righting both past and present injustices, through which those
who have been disempowered can gain liberation from unjust structural forces in
society.
4.2.3 Respecting self-determination

From the above presentation of decoloniality concept, one can deduce that
decoloniality is a burgeoning movement of self-determination that foregrounds
the need for the liberation of marginalized communities across the world. This
emancipatory philosophy is grounded in the principle of self-reflection and
historical reparation, divorced from preconceived notions about progress or
advancement imposed by colonial powers. Its purpose is to assert political
agency, overcome disenfranchisement, abolish unequal power structures, create
autonomous forms of knowledge production, and engender conversations about
relationality to build a more equitable society (Harms-Smith & Rasool, 2020). This
also justifies that decoloniality holds tremendous possibilities not only for
emerging nations and their citizens but also for those living within established
nation-states. Recognizing the systemic dynamics that keep certain populations
subjugated is the first step needed to contest this state of affairs and actively work
towards a form of collective liberation.
In clear terms, decoloniality is a critical tool that champions autonomy, self-

governance, and self-determination for individuals and communities. Through
this assumption, people tend to confront the legacies of systemic exploitation in
order to enact positive change. By identifying and critically analyzing the
oppressive structures that uphold unequal power dynamics, decolonial
movements are able to devise creative strategies for establishing equitable
350
societies (Agboka, 2014). However, it is an approach rooted in solidarity and

collective action: working together for the greater good.
4.2.4 Anti-oppressive in nature

Based on the above conceptual presentation, one can deduce that decoloniality is
a form of resistance against oppression and domination in the current world. It
seeks to undo the structure of power that often exists within different forms of
hegemony (Naidu, 2021), such as classism, racism, and sexism. By rejecting
imperial domination and its legacies, decolonizing practices can be seen to
undermine systems of power in favor of creating more equitable institutions and
societies (Auerbach, 2021; Saurombe, 2018). Therefore, decoloniality values
plurality, diversity, and autonomy in order to de-emphasize oppressive forms of
knowledge while uplifting subjugated identities. Thus, it can be argued that
decoloniality works towards creating contexts where oppressive conditions are
confronted and equality is established in conceptualization, theories, practice, and
praxis.
In the practical sense, this means that this results in the acceptance or recognition
of voices typically excluded from mainstream discourse, as well as in an
appreciation for cultures previously dismissed by the oppressive system within
which they exist. In our argument, this is an attempt to provide power to those
affected by the oppression produced by colonialism, ultimately resulting in the
disruption of oppressive and hierarchical systems which prioritize certain people
while denigrating others. Structural racism, capitalism, patriarchy, and other
forms of systemic inequality are rendered amenable to change when
anti-oppressive principles of decoloniality are embraced.
4.3 Nexus Between the Assumptions and “Decolonizing the Curriculum”

This section presents the relationship between the assumptions of decoloniality
concept and the understanding of the concept of decolonizing the curriculum or
decoloniality of the curriculum. This is done with cognizance of the assumptions
as it correlates with the understanding of decolonizing the curriculum. They are:
challenging the process of knowing and the understanding of decolonizing the
curriculum, empowering the disempowered and the understanding of
decolonizing the curriculum, self-determination and the understanding of
decolonizing the curriculum, and anti-oppression and the understanding of
decolonizing the curriculum.
4.3.1 Challenging the process of knowing and decolonizing the curriculum

In order to understand the concept of decolonizing the curriculum, the
assumption of decoloniality that talks about challenging the process of knowing
needs to be envisioned because it explains how the curriculum should be
challenged. This means that decolonizing the curriculum involves interrogating
how knowledge has been constructed within an academic context, with a
particular focus on the persistent colonial power structures that continue to shape
education spaces. This interrogation demands the definers to consider the
assumption of decoloniality – that is, critically reflecting on how we come to know
what we do. By being attuned to these dynamics, students may be provided with
351
alternatives critiquing oppressive curricula and replacing them with pedagogies

more appropriate for enabling decolonial thinking and action. This calls for noting
the multi-layered and complex nature of decolonization and its implications for
teaching, learning, and assessment practices in education today.
At its heart, the concept of decoloniality outlines a process of revisioning: How

can we reimagine the curriculum to reflect a more holistic, equitable perspective?
To understand this process of revisioning, it is necessary to unpack concepts such
as post-colonialism; the importance of power structures within knowledge
production; decolonial literacy; and self-reflexivity. From here, educators can
begin to move away from existing colonial frameworks in order to produce
knowledge that better empowers communities. This, then, helps to create a
vibrant explanation of what it means to decolonize the curriculum and knowledge
production perspective.
4.3.2 Empowering the disempowered and decolonizing the curriculum

In order to understand the concept of decolonizing the curriculum, the
assumption of decoloniality that talks about the fact that the curriculum should
be tailored towards empowering the students and creating equal opportunities
for students is fundamental. That is, decolonization of the curriculum is a kind of
social and knowledge empowerment for disempowered students at any
educational level. This involves tailoring curricula to empower students, create
equal opportunities, and foster inclusion within learning environments. This
approach seeks to break through longstanding power systems that have rendered
certain demographics disempowered and unequal in terms of their access to
knowledge and educational resources. By actively decolonizing a curriculum,
institutions tend to create meaningful learning experiences while mitigating
longstanding inequities that have plagued learning environments. As such, it is a
concept that warrants further research and consideration as it has significant
implications for student engagement and educational outcomes.
Therefore, a decolonized curriculum is one that can promote social justice and
combat exclusion by providing genuine support from various aspects to replace
systems of inequity. Hence, decolonizing the curriculum enables equal
opportunities for every student regardless of race, class, and gender by giving
them more control over their learning experiences. The argument of Trout et al.
(2018) also supports our argument that decoloniality fundamentally addresses
power differentials within knowledge production. This contributes towards an
ultimately equitable environment – where each student is allowed to gain
valuable skills and knowledge without any kind of academic negation or
disadvantage.
4.3.3 Self-determination and decolonizing the curriculum

To understand the concept of decolonizing the curriculum, the assumption of
decoloniality that talks about self-determination upholds that students should be
provided an opportunity to participate in the process of curriculum development
for the purpose of inclusivity and informed knowledge production in university
classrooms. That is, self-determination emerges as a central concept in this
process, stressing the importance of allowing students to actively participate in
352
developing curricula that reflect their perspectives. In this way, university

classrooms become more conscious of systemic power dynamics within higher
education institutions, offering a meaningful opportunity for informed
knowledge production. Ultimately, decolonizing the curriculum involves
dismantling colonial ideas and privileging indigenous ways of knowing through
self-determination (Harvey & Russell-Mundine, 2019). It is up to the curriculum
stakeholders to develop this capacity among teachers/lecturers and students alike
to ensure the success of decolonial ideologies within academic settings. Hence,
such an approach actively seeks to increase inclusivity and encourages
consideration of different cultural views. Recognizing these collective experiences
allows for more meaningful connections between communities and universities,
thereby producing an enhanced learning experience for all stakeholders. By doing
this, one must have the principle of self-determination in mind when
conceptualizing or ruminating on what it means to decolonize the curriculum.
4.3.4 Anti-oppression and decolonizing the curriculum

The assumption of decoloniality that talks about anti-oppression is fundamental
to the understanding of decolonizing the curriculum. That is, a decolonized
curriculum is devoid of any form of oppression, be it systemic or power politics.
This entails eradicating any form of hierarchy, power dynamics, and cultural
hegemony from curricula to ensure the perspectives of those who have been
historically oppressed are given priority in education. A decolonized curriculum
must be built on a foundation entailing principles such as inclusion and equality
for all people regardless of racial makeup, gender identity, and socioeconomic
status. If educators move away from upholding oppressive attitudes and instead
embrace everyone in the learning environment from diverse backgrounds with an
open heart, then collectively decolonizing curricula can effectively reshape
society’s educational structures.
Therefore, a decolonized curriculum is one that prioritizes the elimination of all

oppressive structures in classrooms. To achieve this goal, a central concept is that
of decoloniality, which challenges oppressive power dynamics (Onafuwa, 2018).
This indicates that decolonizing the curriculum is beyond simply reducing the
influence of dominant ideologies and instead seeking to genuinely rework
educational systems to proactively recognize diverse narratives and experiences
of oppression. In doing this, universities create an equitable learning environment
where everyone is afforded their basic human rights and dignity.
5. Conclusion and Recommendations

The study provided an answer to the two research questions on decoloniality of
the curriculum and how its assumptions can contribute to a clear understanding
of what it means to decolonize the curriculum. By answering these questions,
arguments exist that assumptions such as challenging the process of knowing,
empowering the disempowered, self-determination, and anti-oppressive
tendencies of decoloniality are the dimensions that can bring more clarity to the
concept of decolonizing the curriculum. In other words, to gain deeper insights
into this issue, one must examine the arguments around it from a variety of angles.
These angles were broken down above into multiple facets to reach
353
decolonization – challenging existing ways of knowing, emphasizing self-

determination and anti-oppression initiatives, while also providing an avenue for
the disempowered to regain their agency over their own affairs. Thus, these
arguments exist to provide more concrete guidelines within broader standards
that objectives must meet in order to claim their status as “decolonial” curricula.
Through this lens, we can gain further clarity on what it means to leave behind
the legacy of colonialist systems and instead embrace a new era built with
empowered and knowledge-rich communities at its center.
Based on this, the following recommendation was made. When attempting to

conceptualize decolonizing the curriculum, students, lecturers, and academics
should be cognizant of several key values: challenging the process of knowing
and questioning what is being taught, empowering the disempowered by
representing diverse cultures in the syllabus, recognizing and respecting
self-determination, and enacting an anti-oppressive approach. These values
transform the longstanding ambiguity of decoloniality of the curriculum.
6. References
Agboka, G. Y. (2014). Decolonial methodologies: Social justice perspectives in intercultural
technical communication research. Journal of Technical Writing and Communication,
44(3), 297–327. http://dx.doi.org/10.2190/TW.44.3.e
Auerbach, J. (2021). Decolonial science: Towards more equitable knowledge practices.
Advocate: Journal of the National Tertiary Education Union, 28(2), 34–36.
Bergström, J. (2021). Whose knowledge counts? The struggle to revitalise indigenous
knowledges in Guatemala. Sustainability, 13(21), 11589.
https://doi.org/10.3390/su132111589
Carangio, V. (2021). Responsible research practice: Revisiting transformative paradigm in
social research. Ethnic and Racial Studies, 44(8), 1406–1408.
https://doi.org/10.1080/01419870.2020.1831569
de França Sá, A. L., & Marsico, G. (2022). Decoloniality and disruption of the scientific
status quo: Dissemination of universal theoretical assumptions in international
research. Review of General Psychology, 26(4), 416–425.
https://doi.org/10.1177/10892680211065169
Deem, A. (2019). Mediated intersections of environmental and decolonial politics in the
No Dakota Access Pipeline Movement. Theory, Culture & Society, 36(5), 113–131.
https://doi.org/10.1177/0263276418807002
Dunlap, A. (2022). ‘I don’t want your progress! It tries to kill … me!’ Decolonial encounters
and the anarchist critique of civilisation. Globalisations, 1–27.
https://doi.org/10.1080/14747731.2022.2073657
Fellner, K. D. (2018). Embodying decoloniality: Indigenising curriculum and pedagogy.
American Journal of Community Psychology, 62(3-4), 283–293.
https://doi.org/10.1002/ajcp.12286
Furner, J. (2004). Conceptual analysis: A method for understanding information as
evidence, and evidence as information. Archival Science, 4, 233–265.
http://dx.doi.org/10.1007/s10502-005-2594-8
Garcia, G. A. (2018). Decolonizing leadership practices: Towards equity and justice at
Hispanic-serving institutions (HSIs) and emerging HSIs (eHSIs. Journal of
Transformative Leadership & Policy Studies, 7(2), 25–39.
https://doi.org/10.36851/jtlps.v7i2.505
Harms-Smith, L., & Rasool, S. (2020). Deep transformation toward decoloniality in social
work: Themes for change in a social work higher education program. Journal of
354
Progressive Human Services, 31(2), 144–164.

https://doi.org/10.1080/10428232.2020.1762295
Harvey, A., & Russell-Mundine, G. (2019). Decolonising the curriculum: Using graduate
qualities to embed indigenous knowledges at the academic cultural
interface. Teaching in Higher Education, 24(6), 789–808.
https://doi.org/10.1080/13562517.2018.1508131
Keet, A. (2014). Epistemic ‘othering’ and the decolonisation of knowledge. Africa Insight,
44(1), 23–37. https://hdl.handle.net/10520/EJC161966
Kerr, J. (2014). Western epistemic dominance and colonial structures: Considerations for
thought and practice in programs of teacher education. Decolonisation: Indigeneity,
Education & Society, 3(2), 83–104.
https://jps.library.utoronto.ca/index.php/des/article/view/21148
Kubota, R. (2022). Decolonising second language writing: Possibilities and challenges.
Journal of Second Language Writing, 58, 100946.
https://doi.org/10.1016/j.jslw.2022.100946
Lentin, A. (2021). Decolonial thought: A lecture for the Institute for Culture and Society
Key Thinkers & Concepts series. Alana Lentin.net.
https://www.alanalentin.net/2021/07/05/decolonial-thought-a-lecture-for-the-
institute-for-culture-and-society-key-thinkers-concepts-series/
Manning, J. (2018). Becoming a decolonial feminist ethnographer: Addressing the
complexities of positionality and representation. Management Learning, 49(3),
311−326. https://doi.org/10.1177/1350507617745275
Martineau, J., & Ritskes, E. (2014). Fugitive indigeneity: Reclaiming the terrain of
decolonial struggle through indigenous art. Decolonisation: Indigeneity, Education
& Society, 3(1), I-XII.
Martínez Novo, C. (2018). Ventriloquism, racism and the politics of decoloniality in
Ecuador. Cultural Studies, 32(3), 389–413.
https://doi.org/10.1080/09502386.2017.1420091
Maseland, R. (2018). Is colonialism history? The declining impact of colonial legacies on
African institutional and economic development. Journal of Institutional Economics,
14(2), 259–287. https://doi.org/10.1017/S1744137417000315
Mays, K. T. (2019). Decolonial hip hop: Indigenous hip hop and the disruption of settler
colonialism. Cultural Studies, 33(3), 460–479.
https://doi.org/10.1080/09502386.2019.1584908
McDowall, A., & Ramos, F. (2018). Doing decoloniality in the writing borderlands of the
PhD. The Australian Journal of Indigenous Education, 47(1), 54–63.
https://doi.org/10.1017/jie.2017.23
Mertens, D. M. (2007). Transformative paradigm: Mixed methods and social
justice. Journal of Mixed Methods Research, 1(3), 212–225.
https://doi.org/10.1177/1558689807302811
Mertens, D. M. (2010). Transformative mixed methods research. Qualitative Inquiry, 16(6),
469–474. https://doi.org/10.1177/1077800410364612
Mignolo, W. (2012). Decolonising Western epistemology/building decolonial
epistemologies. Decolonising Epistemologies: Latina/o Theology and Philosophy, 19–43.
https://doi.org/10.5422/fordham/9780823241354.003.0002
Mitova, V. (2020). Decolonising knowledge here and now. Philosophical Papers, 49(2),
191−212. https://doi.org/10.1080/05568641.2020.1779606
Moyo, L., & Mutsvairo, B. (2018). Can the subaltern think? The decolonial turn in
communication research in Africa. In B. Mutsvairo (Ed.), The Palgrave handbook of
media and communication research in Africa (pp. 19–40). Palgrave Macmillan.
Naidu, T. (2021). Modern medicine is a colonial artifact: Introducing decoloniality to
medical education research. Academic Medicine, 96(11S), S9−S12.
https://doi.org/10.1097/ACM.0000000000004339
355
Ndlovu‐Gatsheni, S. J. (2015a). Decoloniality as the future of Africa. History Compass,

13(10), 485–496. https://doi.org/10.1111/hic3.12264
Ndlovu-Gatsheni, S. J. (2015b). Decoloniality in Africa: A continuing search for a new
world order. The Australasian Review of African Studies, 36(2), 22–50.
Ndlovu-Gatsheni, S. J. (2019). The struggle for epistemic freedom and decolonisation of
knowledge in Africa [Webinar lecture]. Convival Thinking Collective in
collaboration with European Association of Development Research and Training
Institute (EADI).
Omanga, D. (2020, January 14). Decolonisation, decoloniality, and the future of African
studies: A conversation with Dr. Sabelo Ndlovu-Gatsheni. Items: Insights from the
Social Sciences. https://items.ssrc.org/from-our-programs/decolonization-
decoloniality-and-the-future-of-african-studies-a-conversation-with-dr-sabelo-
ndlovu-gatsheni/
Onafuwa, D. (2018). Allies and decoloniality: A review of the intersectional perspectives
on design, politics, and power symposium. Design and Culture, 10(1), 7–15.
https://doi.org/10.1080/17547075.2018.1430995
Sauerbronn, F. F., Ayres, R. M., da Silva, C. M., & Lourenço, R. L. (2021). Decolonial studies
in accounting? Emerging contributions from Latin America. Critical Perspectives on
Accounting, 102281. https://doi.org/10.1016/j.cpa.2020.102281
Saurombe, N. (2018). Decolonising higher education curricula in South Africa: Factoring
in archives through public programming initiatives. Archival Science, 18, 119–141.
https://doi.org/10.1007/s10502-018-9289-4
Schiwy, F. (2007). Decolonisation and the question of subjectivity: Gender, race, and
binary thinking. Cultural Studies, 21(2–3), 271–294.
https://doi.org/10.1080/09502380601162555
Shizha, E. (2010). Rethinking and reconstituting indigenous knowledge and voices in the
academy in Zimbabwe: A decolonisation process. In D. Kapoor, & E. Shizha
(Eds.), Indigenous knowledge and learning in Asia/Pacific and Africa: Perspectives on
development, education, and culture (pp. 115–129). Palgrave Macmillan.
Sium, A., Desai, C., & Ritskes, E. (2012). Towards the ‘tangible unknown’: Decolonization
and the indigenous future. Decolonisation: Indigeneity, Education & Society, 1(1),
i−xiii.
Trisos, C. H., Auerbach, J., & Katti, M. (2021). Decoloniality and anti-oppressive practices
for a more ethical ecology. Nature Ecology & Evolution, 5(9), 1205−1212.
https://doi.org/10.1038/s41559-021-01460-w
Trout, L., McEachern, D., Mullany, A., White, L., & Wexler, L. (2018). Decoloniality as a
framework for indigenous youth suicide prevention pedagogy: Promoting
community conversations about research to end suicide. American Journal of
Community Psychology, 62(3–4), 396–405. https://doi.org/10.1002/ajcp.12293
Valchev, H. I. (2018). What is conceptual analysis? Balkan Journal of Philosophy, 10(2),
131−142. https://doi.org/10.5840/bjp201810215
von Colln-Appling, C., & Giuliano, D. (2017). A concept analysis of critical thinking: A
guide for nurse educators. Nurse Education Today, 49, 106–109.
https://doi.org/10.1016/j.nedt.2016.11.007
356

Vol. 22, No. 7, pp. 356-370, July 2023
The Impact of Project-Based and Experiential

Learning Integration on Pre-Service Teacher
Achievement in Evaluation and Assessment
Apantee Poonputta
Mahasarakham University, Mahasarakham, Thailand
Abstract. The present study aimed to explore the effectiveness of

integrating project-based learning (PBL) and experiential learning in
fostering the evaluation and assessment skills of pre-service teachers. To
assess the impact of this integration, a self-efficacy assessment form and
evaluation of teacher performance, learning achievement and mentality
aspects were employed. The findings of the study indicated that the
integration of PBL and experiential learning yielded positive outcomes,
enhancing the evaluation and assessment skills of pre-service teachers.
This research contributes to the existing body of literature by providing
empirical evidence supporting the potential of integrating PBL and
experiential learning as an effective approach to augment pre-service
teachers’ evaluation and assessment abilities. The study also underscores
the significance of creating hands-on and collaborative learning
environments within teacher education programmes. These results have
significant implications for the design of future teaching strategies aimed
at improving the evaluation and assessment skills of pre-service teachers.
By incorporating PBL and experiential learning, teacher education
programmes can create dynamic learning experiences that engage
students in practical, real-world contexts. This approach encourages
active participation, critical thinking and problem-solving skills, which
are essential for effective evaluation and assessment practices. The study
emphasises the importance of providing pre-service teachers with
opportunities to apply their knowledge in authentic settings, fostering a
deeper understanding of evaluation and assessment principles. By
equipping pre-service teachers with these skills, they will be better
prepared to meet the demands of the classroom and contribute to
improved student learning outcomes.
Keywords: project-based learning; experiential learning; teacher

education, evaluation and assessment; teaching skill development
1. Introduction
Project-based learning (PBL) has gained increasing attention in the field of teacher
education as a beneficial tool for offering pre-service teachers active learning
experiences. In the twenty-first century, with the rapid advancement of
©Authors
357
technology and the changing demands of the workforce, there is a growing need
for educational approaches that prepare pre-service teachers for real-world
challenges and equip them with essential skills for lifelong learning and future
career development. According to Thomas (2000), PBL is a form of learning that
centres around students and is structured by projects that are oriented toward
solving real-world problems. Throughout the processes, collaborative projects
allow students to learn new content and develop new sets of skills via active
learning, critical thinking, and collaboration among students (Boss & Larmer,
2018). By incorporating PBL into the curriculum, educators can create a more
engaging and effective learning environment that prepares pre-service teachers
for success in the twenty-first century.
According to Krajcik and Shin (2014), PBL is based on four core ideas: active
construction of understanding by the learners, collaboration among learners in
authentic learning contexts, effective scaffolding provided by the teacher or other
cognitive tools, and engagement in meaningful, real-world problems or
challenges. This means that learners are encouraged to engage with the content
by asking questions, seeking out information, making connections to their prior
knowledge, and applying the concepts they are learning in real-world contexts.
By actively constructing their understanding, learners develop a deeper and more
meaningful understanding of the content (Chrestensen, 2007; Bruno et al., 2019).
Moreover, learners work together in groups to solve real-world problems or
complete authentic tasks that are relevant to their lives and interests. By working
collaboratively, learners can share ideas, build on each other’s strengths, and
develop a more sophisticated understanding of the content (Xiaodan et al., 2019;
Jayashree et al., 2021). Regarding teacher facilitation, learners are provided with
the necessary guidance, support and resources to help them complete the task or
solve the problem. Scaffolding may include explicit instruction, modelling,
feedback, or other forms of support that help learners move towards greater
independence and mastery of the content (Taber, 2018). Lastly, learners are
presented with tasks or problems that are relevant to their lives and interests, and
that require them to apply the concepts and skills they are learning in authentic
and meaningful ways. By engaging in these types of tasks, learners are more likely
to be motivated, engaged and invested in their learning, and are more likely to
develop the skills and knowledge necessary for success in the real world (Alshare
& Nitham, 2004; Mahmoud & Idris, 2021).
It can be noted that PBL emphasises the importance of real-world practice and
authentic, meaningful tasks that allow students to gain hands-on experience and
develop essential skills. This makes it consistent with the tenets of experiential
learning. Experiential learning is a pedagogical approach that emphasises the
importance of learning through first-hand experience and reflection on those
experiences (Kolb, 1984). It involves a cycle of concrete experience, reflective
observation, abstract conceptualisation and active experimentation, which allows
learners to engage with the material in a more active and meaningful way. The
first step is concrete experience, which entails engaging in a hands-on experience,
such as participating in an activity or interacting with an object or environment.
The second step is reflective observation, which involves examining the
experience, highlighting noteworthy observations or discoveries, and assessing
its relevance to broader concepts or theories. The abstract conceptualisation step
358
focuses on creating new concepts or theories, connecting the experience to broader

ideas or principles, or identifying patterns or links within the experience. Lastly,
the active experimentation step entails applying new theories, methods or
assumptions based on earlier steps. This cycle provides learners with an active,
engaging, and reflective approach to learning, enabling them to gain a deeper
understanding of the material through direct experience and reflection.
Assessment and evaluation are crucial components of teaching and learning, and
pre-service teachers must develop the necessary skills to be successful in their
future careers (Wright, 2008). Effective assessment and evaluation practices allow
teachers to monitor student progress, identify areas of strength and weakness, and
make informed decisions about instructional strategies and interventions (Amua-
Sekyi, 2016). In addition, teachers with strong assessment and evaluation skills
are better equipped to meet the diverse needs of their students, create a positive
learning environment, and support student success. Furthermore, the ability to
design and implement assessments and evaluations is essential for meeting the
requirements of educational policies and standards, ensuring accountability, and
maintaining quality in education (Drovnikov et al., 2018). Therefore, it is vital that
pre-service teachers receive adequate training in assessment and evaluation
practices and can develop and apply these skills in real-world settings through
project-based and experiential learning opportunities.
Calkins et al. (2018) presented five elements for assessment design including
providing accessible and actionable information, being understood and valued by
students, aligning with curriculum and instruction, building strong identities and
promoting equity. In detail, effective assessment should provide students with
information that is easily accessible and actionable. This means that the
information should be clear, concise and relevant and should provide students
with specific feedback that they can use to improve their learning. Assessment
should also be designed to be inclusive so that all students have access to the same
information and opportunities to demonstrate their knowledge and skills (Banta
et al., 2009; Anette, 2020). Moreover, students need to understand and value the
assessment as an authentic and worthwhile learning experience. This means that
assessment should be relevant to students’ lives and interests and should be
designed in a way that is engaging and meaningful to them (McTighe, 2021). In
addition, assessment should align with the curriculum and instruction to support
knowledge transfer as it should be designed to assess the specific learning
objectives and outcomes that have been taught in class and integrated into the
instructional process as a natural part of learning (Brennan, 2010). Furthermore,
assessment should be designed to promote a growth mindset and to encourage
students to see themselves as capable and successful learners who can achieve
their goals. Lastly, assessment should be designed to promote equity by
addressing the diverse needs and experiences of all students (Taras & Wong,
2022). Differences in learning styles, abilities and backgrounds should be taken
into consideration to create opportunities for all students. Assessment should also
be designed to promote social justice and challenge stereotypes and biases that
can limit student achievement.
However, developing assessment and evaluation skills is not an easy task. It
requires a significant amount of time, effort and practice to master the various
359
techniques and strategies involved. Pre-service teachers must be willing to engage

in ongoing professional development and seek out opportunities to refine their
skills (Lumadi, 2013). Moreover, assessment and evaluation practices are
constantly evolving, and teachers must stay up to date with the latest research and
best practices in the field. Despite these challenges, the development of
assessment and evaluation skills is critical for pre-service teachers to succeed in
their future careers and make a positive impact on their student’s learning and
development.
At a contextual level, teacher education in the Thai context also prioritises
evaluation and assessment skills. The knowledge of assessment types and
implementation, the use of multiple measures, the use of technology in
assessment, data analysis and feedback methods are included in the curriculum.
However, the literature showed that pre-service teachers face challenges in
developing such skills (Buathong, 2018; Sirisiriwat et al., 2016;
Sripranomthanakorn & Thanwadi, 2019). This includes a lack of practical
experience, limited exposure to diverse assessment practices, and limited
opportunities for feedback and collaboration with experienced teachers. In
addition, there may be cultural barriers that impact the implementation of
effective evaluation and assessment practices, such as a focus on rote
memorisation and a reluctance to challenge traditional teaching methods
(Buathong, 2018). To address these challenges, teacher education programmes in
Thailand should prioritise hands-on, experiential learning opportunities that
allow pre-service teachers to develop and apply effective evaluation and
assessment skills in real-world settings. Furthermore, it is important to provide
ongoing support and professional development opportunities to ensure that
teachers have the skills and knowledge they need to promote student learning
and success.
Project-based learning and experiential learning are both educational approaches
that prioritise active, student-centred learning experiences that emphasise the
practical application of knowledge and skills in real-world contexts. By
integrating experiential learning principles into PBL, students are provided with
opportunities to engage in meaningful and authentic tasks that encourage active
learning, critical thinking and collaboration – the skills needed to develop
teachers’ evaluation and assessment abilities. Previous studies also indicated the
benefits of both instructional approaches in teacher education (Alrajeh, 2020;
Biasutti & EL-Deghaidy, 2015; Eckardt et al., 2020; Ernst, 2013; Gao, 2015;
Goldstein, 2016; Howard, 2002; Legge & Smith, 2014.; Miller et al., 2021; Roessingh
& Chambers, 2011; Williams & Sembiante, 2022). The preceding studies provide
evidence that PBL is an instructional approach that can be seamlessly integrated
with technology, as demonstrated by the works of Alrajeh (2020), Biasutti and EL-
Deghaidy (2015), and Howard (2002). Furthermore, earlier research advocates for
the integration of PBL with other instructional principles to further enhance its
effectiveness. Additionally, integrating experiential learning into teacher
education programmes can help address the challenges faced by pre-service
teachers in developing assessment and evaluation skills in the Thai context. Thus,
this study seeks to apply these two principles, project-based and experiential
learning, to the development of pre-service teachers’ evaluation and assessment
abilities. Two objectives are proposed: 1) to investigate the effects of project-based
360
and experiential learning integration on pre-service teacher achievement of

evaluation and assessment and 2) to investigate pre-service teacher self-efficacy
of evaluation and assessment after learning with project-based and experiential
learning integration.
2. Methods
The study used a one-group pre-post-test design. The detail of research
methodology can be seen below.
2.1 Participants
The study involved 32 pre-service teachers who were enrolled in a university in
Thailand, a context that has faced challenges in the quality of its teachers,
particularly regarding evaluation and assessment skills (Buathong, 2018;
Sirisiriwat et al., 2016; Sripranomthanakorn & Thanwadi, 2019). The researchers
used a cluster random sampling method to ensure a representative sample. In
detail, one out of two groups of social studies major students enrolling in the
Measurement and Evaluation Design in Social Studies course was selected. The
study was conducted following human research ethics, and ethical considerations
were taken into account to ensure the safety and well-being of the participants.
Prior to the study, participants were provided with a clear and detailed
explanation of the research project, its objectives and the expected outcomes.
Informed consent was obtained from each participant, ensuring that they fully
understood the purpose of the study and agreed to participate voluntarily. The
participants were informed that their involvement in the study was entirely
voluntary and that they had the right to withdraw from the study at any time
without penalty or consequence. Confidentiality was also maintained throughout
the study, with all data being stored securely and anonymously to ensure the
privacy and anonymity of the participants. These measures were put in place to
ensure that the rights and welfare of the participants were protected, and to
maintain the integrity and credibility of the research findings.
2.2 Instrumentation/Data Collection
2.2.1 The Integrated PBL and Experiential Learning Circle
The current study used a set of learning activities that were carefully designed to
incorporate the principles of project-based learning and experiential learning.
Project-based learning emphasises a problem-based and student-centred
approach to learning, while experiential learning emphasises the importance of
learning through direct experience and reflection on that experience. By
integrating these two principles, the following activities were used in the class.
Firstly, students participated in brainstorming sessions to identify and select a
real-world problem or challenge regarding evaluation and assessment. Once a
problem had been selected, students engaged in research and gathered relevant
information related to the problem or challenge. They were assigned to search
various sources of information, including online resources, surveys and
interviews, to gain a deeper understanding of the issue.
After gathering information, students collaborated with their peers to develop a
project plan and timeline. They were separated into groups of four and instructed
to develop a plan that indicates the tasks and responsibilities of members as well
361
as a working schedule to ensure that the project is completed within a specific

timeframe. Throughout the project, students designed and implemented their
ideas using a variety of resources and tools, such as technology, art materials or
physical resources.
As the project progressed, students were encouraged to engage in the first-hand
learning experience, for example, conducting interviews or surveys with
professional teachers, administrators, students and parents to gather information
related to evaluation and assessment, visiting schools and communities and
engaging in other hands-on activities related to the project. Students were
instructed to use various reflection methods, such as journals, group discussions
or individual assessments, to deepen their understanding of the project and their
learning experience.
Finally, students presented their projects to their peers. An exhibition was held to
give students an opportunity for students to showcase their learning and
communicate the importance of their project. In terms of assessment, students
were assessed on a variety of criteria regarding their ability to identify and solve
real-world problems, collaborate with peers, use technology and resources
effectively, and communicate their ideas clearly and effectively through
presentations.
Overall, the learning circle emphasises concrete experience, reflective observation,
abstract conceptualisation and active experimentation to let learners develop their
assessment skills through experiential learning experiences gained while doing
hands-on activities. The learning circle was used after the pre-test session, and it
was evaluated to be appropriate (x̄ = 4.80, SD = 0.44) before the implementation
by five experts including scholars in evaluation and assessment, learning
management and teacher education.
2.2.2 Evaluation and Assessment evaluation form (Rubric Scoring)
The study used a form to assess the pre-service teachers’ evaluation and
assessment skills, comprising two main parts: the ability to develop an assessment
and to evaluate students’ attributes. The assessment form consisted of 11 issues
related to the ability to develop an assessment and seven issues related to the
ability to evaluate students’ attributes. The form was administered only after the
treatment and was used by the researcher to evaluate the participants’ skills after
learning with the PBL and experiential learning integration. The evaluation
criteria were based on a 4-point rating scale. The scale ranged from 0 (no ability)
to 3 (advanced ability) for each evaluating issue, allowing for a comprehensive
evaluation of the participants’ assessment and evaluation skills development. The
content validity of each evaluation item was at 0.67-1.00. The rubric scale was
tested using a Rater Agreement Index and was found to be 0.99.
2.2.3 Self-efficacy evaluation form (Rating Scale)
The study used a self-efficacy form to assess the participant’s level of confidence
in their own evaluation and assessment skills. The form contained content that
was similar to the previously used skill assessment form; however, it allowed
students to rate their skills on a scale from 1 to 5. This form aimed to provide
insight into the participants’ perceived abilities concerning evaluation and
assessment and to track any changes or improvements in self-efficacy throughout
362
the learning programme. The self-efficacy form was considered an important

measure in evaluating the effectiveness of the learning programme, as it allowed
for a deeper understanding of the participants’ confidence in their skills, and how
this may impact their future teaching practices. The form was employed before
and after the treatment to investigate the extent to which the participants believed
they could develop their evaluation and assessment skills. The content validity of
each evaluation item was at 0.67-1.00. The discrimination of each evaluation item
was 0.55-0.87, and the reliability was 0.96 as tested by Cronbach’s alpha
coefficient.
2.2.4 Learning achievement test
To assess the participants’ knowledge of evaluation and assessment issues, a
learning achievement test was employed. The test was designed to evaluate the
participants’ ability to develop learning achievement assessments, evaluate
students’ attributes, develop psychomotor assessments, manage tests, ensure the
quality of written and multiple-choice tests, and assess the quality of students’
attributes and psychomotor assessments. The test was employed after the
treatment. The content validity of each item was found to be at an acceptable level
(IOC = 0.67-1.00), ensuring that the test accurately measured the participants’
knowledge of evaluation and assessment issues in the course description. The test
consisted of 50 items, each with four choices, with appropriate levels of difficulty
(P = 0.38-0.75) and discrimination (D = 0.25-0.88). The reliability was 0.89 as tested
by Kuder - Richardson Method (KR-20).
2.3 Data Analysis
In summary, a one-group pre-post-test design was used. Prior to the intervention,
a self-efficacy assessment was administered. Participants then engaged in
learning activities that were designed using the principles of project-based
learning and experiential learning, with a duration of one semester. After the
intervention, participants’ evaluation and assessment skills were assessed, and
they were asked to self-evaluate their skills once more. In addition, a learning
achievement test was administered. The collected data were analysed using
various statistical methods such as percentage, mean score, standard deviation,
paired sample test and one-sample t-test.
3. Results
3.1 Test Normality
The results show that all the data sets gathered in the current study were in a
normal distribution. Both Kolmogorov-Smirnova and Shapiro-Wilk indicated no
significant abnormality in data distribution in Table 1. Therefore, parametric
statistics were used to identify the results of the study.
363
Table 1: Normality of data

Data Kolmogorov-Smirnova Shapiro-Wilk
Statistic df p Statistic df p
Participants’ learning
0.14 31 0.13 0.94 31 0.10
achievement
Ability to develop an assessment
0.12 31 0.20 0.95 31 0.16
(Post-test)
Ability to evaluate students’
0.10 31 0.20 0.94 31 0.07
attributes (Post-test)
Overall participants’ Self-efficacy
in evaluation and assessment 0.13 31 1.23 0.13 31 0.11
(Pre-test)
Participants’ Self-efficacy in the
ability to develop an assessment 0.11 31 1.35 0.11 31 0.09
(pre-test)
ability to evaluate students’ 0.14 31 0.98 0.14 31 0.08
attributes (pre-test)
Overall participants’ Self-efficacy
in evaluation and assessment 0.09 31 0.78 0.09 31 0.67
(Post-test)
ability to develop an assessment 0.12 31 0.87 0.12 31 1.05
(post-test)
ability to evaluate students’ 0.11 31 0.69 0.10 31 0.58
attributes (post-test)
3.2 Evaluation and Assessment Skills Assessed by Teachers

The findings suggest that pre-service teachers who participated in the learning
activities designed by integrating PBL and experiential learning exhibited a high
level of evaluation and assessment skills (x̄ = 43.16). When the evaluation and
assessment skills were examined in detail, it was revealed that the participants
possessed a very high level of ability to develop an assessment (x̄ = 27.32), while
their ability to evaluate students’ attributes was rated at a high level (x̄ = 15.84).
The results indicated that the participants excelled in classifying learning
objectives according to Bloom’s taxonomy, developing multiple-choice
assessments, selecting supervisors for test quality, analysing the quality of
multiple-choice assessments, analysing the quality of open-ended assessments,
writing reports on assessment quality, selecting supervisors for attribute
assessment quality, publishing attribute assessments and writing reports on
students’ attribute assessments. They were able to identify the test structure,
develop an assessment according to the planned structure and learning objectives,
develop an open-ended assessment and provide clear instructions. They also
demonstrated a satisfactory level of ability to scope the structure of an attribute
assessment, set questions following operational definitions, and select supervisors
for attribute assessment quality. However, they faced difficulties in identifying
the operational definitions of subject matters (Table 2).
364
Table 2: Participants’ evaluation and assessment skills after the treatment

Evaluation issues Min Max x̄ SD % Interpretation
1. The ability to develop an

22 33 27.32 4.13 82.80 Very High
assessment
1) Be able to classify learning
objectives following Bloom’s 2 3 2.84 0.37 94.62 Very High
taxonomy
2) Be able to scope test

2 3 2.29 0.46 76.34 High
structure
3) Be able to develop an
assessment following the
2 3 2.29 0.46 76.34 High
planned structure and
learning objectives
4) Be able to develop a
2 3 2.84 0.37 94.62 Very High
multiple-choice assessment
5) Be able to develop an
open-ended assessment and 2 3 2.29 0.46 76.34 High
provide a clear instruction
6) Be able to select
2 3 2.65 0.49 88.17 Very High
supervisors for test quality
7) Be able to publish the
2 3 2.29 0.46 76.34 High
assessment
8) Be able to manage an
2 3 2.29 0.46 76.34 High
assessment
9) Be able to analyse the
quality of multiple-choice 2 3 2.45 0.51 81.72 Very High
assessment
quality of open-ended 2 3 2.65 0.49 88.17 Very High
assessment
11) Be able to write a report
2 3 2.45 0.51 81.72 Very High
on assessment quality
2. The ability to evaluate
11 21 15.84 3.80 75.42 High
students’ attributes
1) Be able to identify the
operational definitions of 1 3 1.74 0.89 58.06 Limited
subject matters
2) Be able to scope the
structure of an attribute 1 3 2.13 0.67 70.97 Average
assessment
3) Be able to set questions
following the operational 1 3 2.13 0.67 70.97 Average
definitions
365
Evaluation issues Min Max x̄ SD % Interpretation
4) Be able to select
supervisors for the attribute 2 3 2.65 0.49 88.17 Very High
assessment quality
quality of an attribute 2 3 2.29 0.46 76.34 High
assessment
6) Be able to publish an
2 3 2.45 0.51 81.72 Very High
attribute assessment
7) Be able to write a report on
students’ attribute 2 3 2.45 0.51 81.72 Very High
assessment
Average 33 54 43.16 7.91 79.93 High
3.3 Participants’ Evaluation and Assessment Skills

The present study used a one-sample t-test, a parametric statistical analysis, as all
collected data sets exhibited normal distribution. A determining score of 70
percent of the full marks was set, which indicated a significant difference between
the participants’ overall evaluation and assessment skills (x̄ = 43.16) and the
determining score (t = 3.77, p = 0.00). Additionally, there was a significant
difference between the participant’s ability to develop an assessment (x̄ = 27.32)
and the determining score (t = 5.70, p = 0.00), as well as a significant difference
between the participants’ ability to evaluate students’ attributes (x̄ = 15.84) and
the determining score (t = 5.70, p = 0.00). It can be inferred that the integration of
PBL and experiential learning was effective in developing pre-service teachers’
evaluation and assessment skills, as all mean scores were higher than the
determining scores. Therefore, it can be concluded that the learning programme
was successful in developing pre-service teachers’ evaluation and assessment
skills to a desirable level (Table 3).
Table 3: One sample t-test for the participants’ evaluation and assessment skills
Evaluation Full Determini x̄ SD Mean t p
issues ng mark
mark Differences
The ability to
develop an 33 23.10 27.32 4.13 4.22 5.70 0.00*
assessment
The ability to
evaluate
21 14.70 15.84 3.80 1.67 1.14 0.06
students’
attributes
Overall 54 37.80 43.16 7.91 5.36 3.77 0.00*
*p<0.05
3.4 Participants’ Learning Achievement after the Treatment
The study found that the integration of PBL and experiential learning was
effective in enhancing pre-service teachers’ learning achievement of evaluation
366
and assessment. The results of a one-sample t-test revealed a statistically

significant difference between the participants’ average score on the learning
achievement test (x̄ = 38.06) and the determining criteria (t= 3.93, p = 0.00). The
finding suggests that the designed learning activities using PBL and experiential
learning were successful in improving the participants’ learning achievement in
evaluation and assessment to an expected extent.
Table 4: Participants’ learning achievement of evaluation and assessment
Full Determining Min Max  SD t p
mark score
Learning
achievem 50 35 27 45 38.06 4.34 3.93 0.00*
ent
*p<0.05
3.5 The Participants’ Self-Efficacy in Evaluation and Assessment Before
and After the Treatment
The present study reports that pre-service teachers’ self-efficacy in evaluation and
assessment skills was at an average level before the intervention (x̄ = 3.26), but
significantly increased to a very high level after the treatment (x̄ = 4.27), t = 6.12,
p = 0.00. Notably, both the ability to develop assessments (x̄ pre-test = 3.30; x̄
post-test = 4.27), t = 5.88, p = 0.00 and the ability to evaluate students’ attributes
(x̄ pre-test = 3.20; x̄ post-test = 4.26), t = 6.10, p = 0.00 showed the same positive
effect. These results suggest that the integration of project-based learning and
experiential learning positively impacted pre-service teachers’ self-efficacy in
evaluation and assessment skills. It can be inferred that the intervention was
effective in enhancing their perceived ability to evaluate and assess students, as
reflected in their self-reported improvement.
Table 5: The participants’ self-efficacy in evaluation and assessment
before and after the treatment
Self-efficacy x̄ SD Paired Differences t-test p
x̄ SD
The ability to develop an Post 4.27 0.53 0.97 0.92 5.88 .000*
assessment
Pre 3.30 0.93
The ability to evaluate Post 4.26 0.55 1.06 0.97 6.10 .000*
students’ attributes
Pre 3.20 1.03
Overall Post 4.27 0.53 1.01 0.91 6.12 .000*
Pre 3.26 0.95
*p<0.5
4. Discussion
This study aimed to integrate PBL and experiential learning to develop pre-
service teachers’ evaluation and assessment skills. Additionally, we sought to
address a gap in the literature by incorporating learners’ aspects into the
principles, which had been neglected in previous studies (e.g., Alrajeh, 2020;
Eckardt et al., 2020; Miller et al., 2021; Williams & Sembiante, 2022). To investigate
367
the effectiveness of this approach, we assessed pre-service teachers’ self-efficacy

in evaluation and assessment skills before and after the intervention. Specifically,
we employed a self-efficacy assessment form and found that the data gained from
it contributed to the study by demonstrating that the principles of PBL and
experiential learning are effective in promoting students’ confidence in evaluation
and assessment work.
The study’s key findings reveal that the integration of PBL and experiential
learning was beneficial in developing pre-service teachers’ evaluation and
assessment skills. This is demonstrated through the positive results in the teacher
assessment aspect, as the researchers rated the participants’ performance above
the expected level. Additionally, the participants’ knowledge aspect showed that
their learning achievement score was above the expected criteria, and the
mentality aspect indicated that the treatment made participants more confident in
their evaluation and assessment skills.
These findings support previous studies (e.g., Alrajeh, 2020; Eckardt et al., 2020;
Miller et al., 2021; Williams & Sembiante, 2022) have also identified PBL and
experiential learning as beneficial principles in teacher education. These
principles provide a hands-on and collaborative learning environment for
students, allowing them to actively engage in the learning process by solving real-
world problems, working in groups and applying their knowledge and skills to
create a product or solution. Experiential learning emphasises the importance of
learning through direct experiences, reflection and application. By integrating
these principles, pre-service teachers can practise and develop their evaluation
and assessment skills in a real-world context, enhancing their confidence and self-
efficacy in these skills.
Furthermore, the study’s use of self-efficacy assessments provides further
evidence of the positive impact of PBL and experiential learning on pre-service
teachers’ efficacy in evaluation and assessment skills. Self-efficacy assessments
allow participants to assess their confidence in their abilities, providing insight
into the effectiveness of the intervention. Overall, the study’s findings highlight
the importance of PBL and experiential learning in developing pre-service
teachers’ evaluation and assessment skills, providing valuable insights for teacher
education programmes seeking to enhance their students’ learning outcomes.
Indeed, while prior research has demonstrated the efficacy of both PBL and
experiential learning in enhancing pre-service teachers’ evaluation and
assessment skills, the current study is unique in that it highlights the benefits of
integrating these two principles. This finding underscores the potential
synergistic effects of combining different pedagogical approaches to create more
effective learning environments. Furthermore, the use of a self-efficacy
assessment form in the study allowed for a more nuanced understanding of the
impact of PBL and experiential learning on pre-service teachers’ confidence in
their evaluation and assessment skills. Self-efficacy refers to an individual’s belief
in their ability to perform a specific task or achieve a particular outcome. In the
context of education, self-efficacy is an important predictor of academic
performance or performing actions at designated levels and achievement
(Bandura, 1997; Dele & Maria, 2015). By measuring self-efficacy concerning
evaluation and assessment skills, the current study provides evidence of the
368
positive impact of PBL and experiential learning on pre-service teachers’

perceived competence in these areas.
5. Conclusion
This study aimed to integrate PBL and experiential learning to enhance pre-
service teachers’ evaluation and assessment skills. It also aimed to address a
research gap by incorporating learners’ perspectives, which had been overlooked
in previous studies to include additional elements that can enhance PBL
processes.
Given the promising results of this study, there is a need for further investigation
into the effectiveness of integrating PBL and experiential learning in other aspects
of teacher education, such as curriculum development or classroom management.
Future research could explore the extent to which these principles can be applied
across different settings and contexts, and how they might be modified to suit the
needs of different learner populations. Additionally, as self-efficacy assessments
are effective in measuring the impact of these principles, future studies could
further refine and validate such assessments to ensure their reliability and validity
in different educational settings. Moreover, studies that explore the long-term
impact of integrating PBL and experiential learning on pre-service teachers’
evaluation and assessment skills could provide insights into the sustainability of
these approaches. Finally, it would be useful to investigate the impact of these
approaches on pre-service teachers’ attitudes toward evaluation and assessment,
as well as their perceived value and relevance in their future teaching practice.
One limitation of this study is its heavy reliance on quantitative measures, which
limits the depth of insights into pre-service teachers’ experiences and perceptions.
By not incorporating qualitative data, important nuances and contextual
understanding may be missed, hindering a comprehensive understanding of the
topic. Future research could benefit from incorporating qualitative methods to
gain richer insights into the subjective experiences and perspectives of pre-service
teachers involved in PBL and experiential learning.
Acknowledgement
This research project was financially supported by Faculty of
Education Mahasarakham University.
6. References
Alrajeh, T. S. (2020). The value and use of project-based learning in teacher preparation
programs. Cypriot Journal of Educational Sciences, 15(5), 989–1010.
Alshare, K., & Hindi, N. (2004). The importance of presentation skills in the classroom:
Students and instructors perspectives. Journal of Computing Sciences in Colleges,
19, 6–15.
Anette, B. (2020). Opportunities to display knowledge during national assessment in
mathematics: a matter of access and participation. European Journal of Special
Needs Education, 1-14. https://doi.org/10.1080/08856257.2020.1853970
Amua-Sekyi, E. T. (2016). Assessment, student learning and classroom practice: A
review. Journal of Education and Practice, 7(21), 1–6.
369
Banta, T. W., Black, K. E., & Jones, E. A. (2009). Designing effective assessment: Principles
and profiles of good practice. Jossey-Bass.
Biasutti, M., & EL-Deghaidy, H. (2015). Interdisciplinary project-based learning: An
online wiki experience in teacher education. Technology, Pedagogy and Education,
24(3), 339–355. https://doi.org/10.1080/1475939X.2014.899510
Boss, S., & Larmer, J. (2018). Project based teaching: How to create rigorous and engaging
learning experiences. ASCD.
Brennan, B. (2010). Designing effective assessment: Principles and profiles of good
practice. Studies in Continuing Education, 32(3), 271–272.
https://doi.org/10.1080/0158037X.2010.519566
Bruno, R. B., Simone, C., Dos, S., & Breno, J. D. (2019). Toward effectiveness and
authenticity in PBL: A proposal based on a virtual learning environment in
computing education. Computer Applications in Engineering Education, 27(2), 452-
471. https://doi.org/10.1002/CAE.22088
Buathong, S. (2018). Measurement and assessment of learning skills in the 21st Century.
Veridian E-Journal, Silpakorn University, 10(2), 1856–1867.
Calkins, A., Conley, D., Heritage, M., Merino, N., Pecheone, R., Pittenger, L., Udall, D., &
Wells, J. (2018). Five elements for assessment design and use to support student
autonomy. Jobs for the future. Retrieved from:
https://files.eric.ed.gov/fulltext/ED587401.pdf
Chrestensen, A. (2007). Real-world context, interest, understanding, and retention (Master’s
thesis, Michigan Technological University).
https://digitalcommons.mtu.edu/cgi/viewcontent.cgi?article=1535&context=et
ds
Dale, H. S. & Maria, K. D. (2015). Self-efficacy: Education aspects. (pp. 515-521).
https://doi.org/10.1016/B978-0-08-097086-8.92019-1
Drovnikov, A., Nikolaev, E., Afanasev, A., Ivanov, V., Petrova, T., Tenyukova, G.,
Maksimova, N., & Povshednaya, F. (2016). Teachers’ professional competence
assessment technology in qualification improvement process. International
Review of Management and Marketing, 6, 111–115.
Eckardt, P. N., Craig, M., & Kraemer, L. (2020). The impact of project-based learning on
student content knowledge in an undergraduate, teacher preparation,
foundations of education course. Journal for Leadership and Instruction, 19(1), 38–
42.
Ernst, J. V. (2013). Impact of experiential learning on cognitive outcome in technology
and engineering teacher preparation. Journal of Technology Education, 24(2), 31–
40.
Gao, X. (2015). Promoting experiential learning in pre-service teacher education. Journal
of Education for Teaching, 41(4), 435–438.
https://doi.org/10.1080/02607476.2015.1080424
Goldstein, O. (2016). A project-based learning approach to teaching physics for pre-
service elementary school teacher education students. Cogent Education, 3(1),
1200833. https://doi.org/10.1080/2331186X.2016.1200833
Howard, J. (2002). Technology-enhanced project-based learning in teacher education:
Addressing the goals of transfer. Journal of Technology and Teacher Education,
10(3), 343–364.
Jayashree, S., Awati., U., A., Kamerikar., R., T., Patil., & A., A., Prabhavalikar. (2021).
Implementation of project based learning for best practices activities: A case study.
2341(1), 020045. https://doi.org/10.1063/5.0049975
Kolb, D. (1984). Experiential learning: Experience as the source of learning and development. In
Journal of Business Ethics (Vol. 1). Prentice-Hall.
370
Krajcik, J. S., & Shin, N. (2014). Project-based learning. In R. K. Sawyer (Ed.), The
Cambridge Handbook of the Learning Sciences (2nd ed., pp. 275–297). Cambridge
University Press. https://doi.org/10.1017/CBO9781139519526.018
Legge, M., & Smith, W. (2014). Teacher education and experiential learning: A visual
ethnography. Australian Journal of Teacher Education (Online), 39(12), 95–110.
https://doi.org/10.3316/informit.852580049030403
Lumadi, M. (2013). Challenges besetting teachers in classroom assessment: An
exploratory perspective. Journal of Social Sciences, 34, 211–221.
https://doi.org/10.1080/09718923.2013.11893132
Mahmoud, H., & Idris, A. (2021). Problem-based learning (PBL): A Deep approach to learning
in the 21st century. (pp. 38-56). https://doi.org/10.4018/978-1-7998-4534-
8.CH003
McTighe, J., Ferrara, S., & Brookhart, S. (2021). Assessing student learning by design:
Principles and practices for teachers and school leaders. Teachers College Press.
Miller, E. C., Severance, S., & Krajcik, J. (2021). Motivating teaching, sustaining change in
practice: Design principles for teacher learning in project-based learning
contexts. Journal of Science Teacher Education, 32(7), 757–779.
https://doi.org/10.1080/1046560X.2020.1864099
Roessingh, H., & Chambers, W. (2011). Project-based learning and pedagogy in teacher
preparation: Staking out the theoretical mid-ground. International Journal of
Teaching and Learning in Higher Education, 23(1), 60–71.
Sirisiriwat, K., Srisaard, O., & Wawangboon, T. (2016). The development of learning
measurement and evaluation system in primary Auban Sri Prachanukul School:
Empowerment evaluation. Journal of Educational Measurement Mahasarakham
University, 22, 16–26.
Sripranomthanakorn, S., & Donviset, T. (2019). A study of learning assessment of basic
education schools in Chanthaburi Province. Rajanhat Ranhai Barni Research
Journal, 12(12), 167–176.
Taber, K. (2018). Scaffolding learning: Principles for effective teaching and the design of
classroom resources. In Effective teaching and learning: perspectives, strategies and
implementation (pp. 1–43). Nova Science.
Taras, M., & Wong, H. M. (2022). Student self-assessment: An essential guide for teaching,
learning and reflection at school and university. Routledge.
https://doi.org/10.4324/9781003140634
Thomas, W. (2000). A review of research on project-based learning. Autodesk Foundation.
Williams, L., & Sembiante, S. F. (2022). Experiential learning in U.S. undergraduate
teacher preparation programs: A review of the literature. Teaching and Teacher
Education, 112, 103630. https://doi.org/10.1016/j.tate.2022.103630
Wright, R. (2008). Educational assessment: Tests and measurements in the age of accountability.
SAGE. https://doi.org/10.4135/9781483329673
Xiaodan Zhou, Ling-Hsiu Chen, & Chin-Ling Chen. (2019). Collaborative learning by
teaching: A pedagogy between learner-centered and learner-driven. Sustainability,
11(4). https://doi.org/10.3390/su11041174
371

Vol. 22, No. 7, pp. 371-387, July 2023
Teachers’ Professional Development and

Pedagogical Shift towards Dialogic Teaching in
Malaysian Lower Secondary ESL Classrooms
Sheila Adelina Ramasamy
English Language Teaching Centre
Bandar Enstek, Negeri Sembilan, Malaysia
Azlin Zaiti Zainal*

Faculty of Languages and Linguistics, Universiti Malaya
Kuala Lumpur, Malaysia
Abstract. This study reports on teachers’ pedagogical changes upon

attending a professional development programme. In supporting
learners of a second language (L2), dialogic teaching was emphasised as
an intervention programme in Malaysian lower secondary ESL
classrooms. Dialogic teaching is a pedagogical approach that focuses on
the importance of classroom talk in constructing knowledge collectively
(Alexander, 2018). This study explored the perceptions of four rural L2
teachers who were participants in a professional development on dialogic
teaching and investigated their classroom practices using the approach.
The study was qualitative in nature which comprised four individual
semi-structured interviews and nine classroom observations for the data
collection. The case study method was employed to boost the reliability
of the interview findings in which three of the teachers’ classes were
observed, video-taped, transcribed, and analysed to identify the dialogic
features prevalent in the teachers’ discourse pattern. The findings of this
study indicated that the teachers perceived dialogic teaching as a
meaningful interactive discourse structure that facilitated L2 learning
and, to an extent, demonstrated a pedagogical shift that took advantage
of the dialogic teaching principles and talk repertoire introduced during
the professional development. Dialogic teaching strategies such as whole
class and small group discussions increased the participation and
engagement of students which led to better student outcomes. The
teachers had acquired the skills of questioning to facilitate talk amongst
the students. Nevertheless, the teachers were challenged with issues of
students’ proficiency that obstructed their effective intervention process.
The findings have key pedagogical implications in terms of teacher
discourse patterns and for further investigation on the enactment of
dialogic teaching in facilitating second language.
*
Corresponding author: Azlin Zaiti Zainal; azlinzainal@um.edu.my
©Authors
372
Keywords: classroom discourse; dialogic teaching; pedagogical

approach; second language learning; teacher professional development
1. Introduction
Classroom talk refers to the use of oral language to engage students in thinking
and learning (Chang & Chang 2017; Díez-Palomar et al., 2021). Jocuns (2021) states
that classroom talk provides a wider language practice which increases student
proficiency. Hence, teachers play a pivotal role in shaping students’ talk and
promoting L2 learning (Alexander, 2018; Hardman, 2019; Loewen & Sato, 2018).
Understanding the manner by which teachers support and mediate second
language learning is crucial to L2 communities. One-way teachers afford spoken
interaction opportunities to students is through their choice of discourse (Doley,
2019; Reznitskaya & Gregory, 2013). This form of classroom discourse is known
as dialogic teaching (Alexander, 2018).
Dialogic teaching has begun to receive attention in the educational arena

(Alexander, 2018; Cui & Teo, 2021; Kim & Wilkinson, 2019). As a pedagogical
approach, it harnesses the power of talk between teacher and students and among
students (Alexander, 2018). Through this approach, teachers facilitate talk among
students in the classroom, developing students’ thinking, understanding and
learning. In adopting this approach, teachers are expected to strategically employ
different kind of discourse practices to address specific instructional goals for
their students (Wilkinson et al., 2017).
Grounded in social constructivism, dialogic teaching is a collective effort that

promotes a genuine sequence of utterances encouraging student talk. The teacher
is instrumental in managing the interaction and is responsible for the organisation
of the discursive flow of interactions in the L2 classroom (Sedova, 2017) so as to
mediate learning and enhance student talk (Boyd, 2016). These discursive patterns
develop students’ communicative and academic competencies (Boyd, 2016).
Teachers facilitate the discursiveness through elicitation techniques. This form of
classroom discourse is said to assist students to participate in elaborate talks
(Barekat & Mohammadi, 2014; Boyd, 2016) such that it encourages them to expand
their ability to contribute further to the classroom discussion. This practice is
expected to enhance language learning. The respective students’ output and
contribution of ideas would then be used to further develop the talk. Hence, the
talk becomes the focus of the learning process. The approach encourages students
to become engaged to attain higher reasoning, and construct knowledge
collectively as well as better spoken proficiency (Alexander, 2018; Sedlacek &
Sedova, 2017).
From a sociocultural perspective, spoken interaction manifested through

classroom discourses and in particular collaborative dialogues are crucial in
facilitating second language learning. It is seen as a vital tool for reasoning,
collectively constructing knowledge and learning (Hennessy et al., 2016; Swain &
Watanabe, 2012). For learners who have limited opportunities for L2 learning, the
373
classroom functions as an important source and plays a key role in providing the
language experience (Barekat & Mohammadi, 2014).
Research on dialogic teaching in content-based classrooms demonstrated its

successful application (Lee, 2016; Mercer et al., 2009; Teo, 2013). Dialogic teaching
has also been shown to facilitate learners’ cognitive and linguistic development
(Alexander, 2018). Nevertheless, the enactment of dialogic teaching is not without
problems and has raised a challenge for teachers (Asterhan et al., 2020; Sedova,
2017; Sedova et al., 2014; van de Pol et al., 2017). There are tensions noted within
the literature on concepts of dialogic teaching that can cause confusion and vague
understanding, which affects the enactment of it (Kim & Wilkinson, 2019). This
implies that there is a need for more research to understand how dialogic teaching
is perceived by teachers and how it is implemented in teaching and learning of
different subjects. Given the scarcity of research on dialogic teaching in L2
classrooms, this study attempts to fill the gap by exploring how teachers develop
professionally by attending a teacher professional development on dialogic
teaching.
Study Objectives
This study explores the perceptions and practices of four L2 teachers on dialogic
teaching in facilitating L2 learning. The study attempts to address a gap in the
literature by exploring dialogic teaching in second language learning as most of
the past studies focused on content subjects such as Mathematics and Science. In
this context, language is not only the medium for talk but the outcome of talk. The
introduction of dialogic teaching as a pedagogic discourse to Malaysian L2
teachers was through a professional development programme known as Oral
Proficiency in English (OPS-English). The programme was aimed at supporting a
group of Malaysian L2 lower secondary teachers to adopt dialogic teaching in an
effort to help students to attain better outcomes in the English language and
specifically to enhance students’ oral communication skills.
1.1 Teachers’ Professional Development in Dialogic Teaching

Research shows that professional development (PD) approaches support teachers
in adopting dialogic teaching (Ruthven et al., 2017; Sedova, 2017; Sedova et al.,
2017; Böheim et al., 2021; Hennessy et al., 2021). According to Osborne et al.
(2019), changing teachers’ discourse practice towards a dialogic approach is an
arduous task that involves a carefully considered teacher professional
development programme. Acknowledging this, a professional development was
designed that focused on dialogic teaching as a classroom discourse in lower
secondary English language classrooms (English Language Teaching Centre,
2013). Alexander’s (2018) dialogic teaching model was selected and adapted for
professional development as it was considered a holistic model which embodied
the linguistic and paralinguistic features of classroom talk that integrated the
pedagogical, curricular and cultural dimensions which shaped policies and
classroom practices (Alexander, 2018; Cui & Teo, 2021). Nystrand’s (1997)
dialogically organised instruction model was also utilised as the guiding
framework with the emphasis on teacher questioning,. Nystrand’s dialogically
organised instruction model was specifically selected based on its close
374
resemblance to Alexander’s principles; indeed Alexander’s model was largely

conceptualised based on Nystrand’s. Both models focus on the reciprocal element
of teacher and students. The view of dialogic teaching as a pedagogical discourse
pattern was developed from the combination of both models above. Nystrand’s
(1997) authentic questions, uptake, and high-level evaluation questions influence
and shape students ‘thinking’ (Kim & Wilkinson, 2019). Although Alexander’s
framework was comprehensive in suggesting the principles, repertoires and
indicators for talk, the focus on authentic questions and uptake was emphasised
by Nystrand. Thus, the PD adopted both frameworks to upskill the teachers in
enacting dialogic teaching in English lessons to provide students with
opportunities for second language learning.
The training for the teachers involved both theory and practice. The training was
provided by the in-service teacher training institute in Malaysia known as English
language Teaching Centre. The training was structured in a manner that teachers
were given exposure to the fundamentals of dialogic teaching. The teachers were
then guided in the application of the pedagogical discourse in the teaching of
spoken English by focusing on the principles, the talk repertoires and indicators
as proposed by Alexander (2018). A specific module was developed for the
teachers which was aligned with the curriculum and mapped to the syllabus of
Form One and Two English.
The teachers’ module known as the ‘Teacher Companion’ (MoE, 2012) focused on
the application of the five dialogic principles by Alexander (2018) and the talk
repertoire into the English lessons to facilitate student talk. The teachers were
introduced to the concept of talk manifested through interactive discourse
patterns involving discussions and dialogues and the application of the talk types
in the English language lessons as proposed by Alexander (2018).
Alexander’s (2018) five dialogic principles and talk repertoires were depicted
through the discussions held in the classroom. The dialogic teaching principle of
collectiveness was reflected as the teacher initiated the whole class discussion by
posing an open-ended question where students addressed the learning task
together. Teachers were introduced to open-ended questions and the importance
of them during the training. Teachers were guided to pose open-ended questions
spontaneously during the training. They were introduced to Repertoire 4-
teaching talk and were instructed to focus on discussions and dialogues as talk
strategy for students to be engaged in talk. Simultaneously, Repertoire 3- learning
talk, was also introduced to expose teachers to the talk types expected of students.
Students have to respond by narrating, explaining, exploring and justifying
(Alexander, 2018). The principle of collective was further emphasised during the
small group discussions. The students once again applied talk types from
Repertoire 3 such as narrating, explain and justifying in addressing the task
together.
Second, the principle of reciprocity is seen where both teacher and students
through the whole class and group discussions, would listen attentively to one
another, share their views, and consider alternative viewpoints. Repertoire 3
375
(learning talk) and Repertoire 4 (teaching talk) based on the dialogic teaching
model by Alexander (2018), were emphasised and demonstrated to teachers.
In the application of principle two – reciprocal, particular learners’ talk type such
as analysing, evaluating, questioning, arguing and justifying were modelled. The
framing and facilitating of talk through open-ended questions posed by teacher
and students would enable the second principle – reciprocal to take place.
Third, the principle of supportiveness, where students articulate ideas freely and
support one another in attaining a common understanding is seen during the
whole class and group discussions. Fourth, is the principle of cumulativeness,
where students build on answers, and chain them into coherent lines of thinking.
Finally, the principle of purposefulness meant the lesson was planned and
structured according to the demands of the curriculum. The teacher’s role is to
model the language and to engage students in talk through dialogues and open-
ended questions.
The teacher’s module also consists of suggested strategies and activities to be

carried out in the language classroom to ensure that opportunities are created for
L2 learning by using English as the medium. Procedural guidelines and suggested
questions that teachers can use to initiate talk on the topic of the lesson were also
provided. A module known as the ‘Student’s Handbook’ was also developed for
students. The handbook consists of pictures that serve as a guide for students to
talk.
Nystrand’s importance of authentic questions such as open-ended questions was

discussed and modelled during the training. There were also hands-on sessions
with the teachers to demonstrate the use of open-ended questions. This was aimed
at equipping the teachers with practical applications for their classrooms. The
teachers were also exposed to the Socratic style of questioning so that this could
be used to help their students become more engaged in the learning process. The
aim was to encourage the students to express their views and to justify them. The
questioning approach was purposefully emphasised during the professional
development course.
Hence, this programme equipped the L2 teachers with the ability to use dialogic
features and assist them to operationalise the five principles of dialogic teaching
through oral discourses made up of discussions and dialogues to enhance student
engagement and learning. Upon receiving their professional development in
dialogic teaching, the teachers then implemented the pedagogical discourse in
their classrooms, based on their understanding developed through the training.
From the onset, their practices were designed to facilitate second language use to
enhance the oral skills of the students. It is on this basis that the current study was
conceptualised, which is to identify and examine the teachers’ perspectives, and
their manner of implementing the discourse structures.
Study Questions:
This study aims to answer the following research questions:
376
1. How do Malaysian L2 lower secondary teachers perceive dialogic teaching

as a classroom discourse in facilitating L2 learning?
2. How is dialogic teaching conceptualised in real classroom practices of
Malaysian L2 lower secondary teachers following the professional development?
2. Methodology
A comprehensive two-week training on dialogic teaching was given to the
teachers to enable them to support and enhance students’ L2 learning. A
qualitative approach was employed to explore the teachers’ perceptions of
dialogic teaching in facilitating second language learning and to investigate
teachers’ practices of the approach in the ESL classrooms upon receiving the
professional development. The teachers’ use of dialogic teaching principles and
talk repertoires in facilitating L2 learning were examined.
Study Sample
The participants for the study were purposefully sampled. The teachers involved
are four Lower Secondary English language teachers from rural schools in the
state of Perlis, Kedah, Penang and Johor which had low-achieving English
language results. They were specifically selected as they were the participants of
the professional development and based on their willingness to participate in the
study. They attended the Professional Development Programme on dialogic
teaching with the aim of enhancing students’ second language learning. The
teachers taught English to form 1 and 2 (Lower Secondary level in Malaysia)
students and conducted the intervention for a year. The four female teachers had
a range of teaching experience, from 13 to 30 years. The teachers held a C1
proficiency level based on the Common European Framework of Reference for
Languages (CEFR). To ensure their anonymity, the teachers were given identifiers
such as T (teacher) followed by numbers.
Study Instrument
The instruments for the study involved interviews and classroom observations. A
semi-structured interview was selected as it serves as an effective tool for
interpretive research to gain in depth data on participants experiences (Balushi,
2016, Creswell, 2013). Thus, the semi-structured interviews were used to gain rich,
intensive, and holistic input of dialogic teaching based on the teachers’
experiences and challenges with the approach. The questions were developed and
adapted based on the Interview Protocol Refinement Framework (IPR) by
Castillo-Montoya, (2016). The IPR framework was specifically selected so as to
strengthen the reliability of the questions. The interviews were held individually
with all four teachers to investigate their perceptions of dialogic teaching as a
pedagogical approach and classroom discourse in facilitating second language
learning. Each interview lasted for about 90 minutes. Their perceived
understanding of dialogic teaching, experiences and challenges of employing
dialogic teaching in facilitating L2 learning were derived from the interview data.
Being a semi-structured interview, the researchers were afforded the flexibility to
modify the questions when necessary. The interviews were then recorded,
transcribed, and subsequently categorised according to key themes.
377
In the next stage of the study, classroom observations were conducted on the
teachers’ discourse pattern and their classroom practices to gain an in-depth
analysis of the approach. The classroom observations would provide a real-life
scenario of the teaching process (Flick, 2013). Both the interview and classroom
observations data strengthen the findings of the study that the teachers did adopt
dialogic teaching in their lessons which facilitated second language learning. The
observations took place in four English language lessons (two from each level)
and were 80 minutes in length per lesson for each teacher. The video recordings
of real classroom practices allowed the researchers to analyse teachers’ enactment
of dialogic teaching. To ensure the objectivity of the review and analysis, an
observation checklist adapted from Alexander‘s Dialogic Teaching Principles
(2010) and Nystrand’s Dialogically Organised Instruction Model (1997) was
developed and utilised to analyse teachers’ usage of the dialogic features in the
lessons to unearth the role of dialogic teaching in facilitating L2 learning.
Data Analysis
The data were analysed thematically (Braun & Clarke, 2006) supported by the
qualitative software ATLAS ti8. The analysis of data was driven by the research
questions which were informed largely by Alexander’s Dialogic Teaching
Framework (2018) and Nystrand’s Dialogically Organised Instruction Framework
(1997). A coding framework was also established based on Braun and Clark (2006)
where Alexander’s dialogic teaching principles, talk repertoire and indicators as
well as Nystrand’s authentic questions, uptake and high-level thinking questions
informed the coding process. This enabled the researchers to focus on specific
characteristics of the data, identify important sections from the transcripts and
attach labels to index them as they related to a theme. A set of codes was derived
from the data as below:
C1: The teacher poses open-ended questions to initiate and extend the
talk.
C2: Teacher facilitates talk through scaffolding of open-ended questions.
C3: The teacher creates a non-threatening environment for talk-through
discussions.
C4: Teacher frames and facilitates talk-through discussions.
C5: Discussions generate talk and facilitate the construction of
knowledge collectively.
C6: The emergence of new topics and knowledge through discussions
C7: Teacher uses different talk types to facilitate talk
C8: Students’ responses contribute to the construction of knowledge
collectively.
Among the eight codes that guided the analysis, code 1 had the greatest number
of counts followed by codes 2, 4, 5 and code 7. To ensure trustworthiness, the
coding framework was peer reviewed and reflexive writing was conducted
throughout the process.
378
3. Findings
The analysis revealed three broad themes, which answered the two research
questions 1) How do Malaysian L2 lower secondary teachers perceive dialogic
teaching as a classroom discourse in facilitating L2 learning and 2) how is dialogic
teaching conceptualised in real classroom practices of Malaysian L2 teachers
following the professional development?
3.1 A Pedagogical Shift towards Dialogic Teaching

The analysis attained from the interview and classroom observation data
indicated that the teachers (T1, T2, T3, and T4) perceived dialogic teaching as a
new pedagogical approach which focused on the kind of teacher talk that
promotes student talk in the learning process and proposed this approach to be
adopted. Both T1 and T2 affirmed the need for a shift towards this pedagogical
approach as stated below:
Excerpt 1
T1:
“ Um…before we were introduced to the OPS-English programme, class
lessons were teacher centered. I started to focus on listening and speaking
when we were introduced. Through OPS-English, all students had the
opportunity to share their ideas. That’s where we found that students
could speak. It’s not only the teacher’s talk. So this method…ah…
dialogic teaching had to be emphasised because I felt that more
opportunities, two-way communication and learning took place in a not
stressed environment”.
Excerpt 2
T2:
“For me, the concept of dialogic teaching is more than talk because it
involves teachers and learners building on each other’s ideas, you
know…posing questions, asking questions, you know constructing
interpretations of what is trying to be conveyed. I would say that we
should have this kind of discourse right from the beginning. It’s where I
see that those who are good can add more and those who are weak, through
this discourse, can speak and gain some knowledge even though it’s a
little”.
T3 also stated that dialogic teaching leverages on talk in facilitating learning:
Excerpt 3:
T3:
“We need a positive environment to learn. I want them to feel happy to
learn English. Dialogic teaching gives a chance for the students to talk.
They will share their ideas and they are very relaxed. I can see they learn
from the interactions”.
T3 noticed that the chain of open-ended questions and responses during the
classroom discussions allowed for the co-construction of knowledge. Students
379
learn better through talking with others – “I realise during the discussions they
have a lot of ideas to share”. However, she acknowledged that there were
language difficulties amongst some of the students that hampered their talk. –“It’s
just the language barrier” There seemed to be some interactions where there was
a display of their ability to think and respond critically, “So I feel like…you
know…it creates a platform for them to talk…So I find practically everyone is
engaged and somehow or rather, even their friends will help the other friends.”
T4 summarised her perceptions regarding the usage of dialogic teaching as:
T4:
“Um… for me, we teachers need to throw our egos away if we want to
employ dialogic teaching. We are no longer the traditional teacher like our
own teachers. The teacher needs to change first so that we can give the
opportunity to talk to our students. We have to give students the
opportunity to discuss their ideas in class, to share their opinions and to
be like our friends. If they are comfortable with us and are able to give
their opinions, then that itself can open up to a lot of learning. I think the
most important thing is the teacher”.
The teachers perceived that they changed in their pedagogical approach

upon implementing dialogic teaching during the intervention phase. This
was evident in the classroom observations where teachers applied the
principles, repertoires and indicators of dialogic teaching with the aim of
enhancing student talk to facilitate L2 learning as in the excerpt below:
Excerpt 5:
T3: What do you think about living in a village?
S7: In a village, you have fresh air and a lot of friends to play together.
S8: We can have strong friendship with neighbours
T3: That’s very good… yes, nowadays we hardly know our neighbours. What
else?
S9: Village is better because not so many cars …no pollution.
T3: Okay, good, where would you like to live?
The teacher’s application of the dialogic teaching principles, talk repertoires and
indicators indicated her adoption of the pedagogical approach. She demonstrated
her role as a facilitator in framing and facilitating talk through teacher
questioning. Her talk type was limited to questioning which exhibited
Alexander’s Repertoire 4 – teaching talk.
Based on the excerpts above, it could be gleaned that the teachers viewed dialogic
teaching as a pedagogical approach which afforded students learning
opportunities through talk. Nevertheless, in the context of L2 users, the enactment
of dialogic teaching in facilitating L2 learning was a difficult task because English
was not only the medium of instruction but the objective of learning. The teachers
380
found that the students’ low proficiency of English hindered the discursiveness of
the approach.
The findings indicated that all four teachers (T1, T2, T3, T4) perceived dialogic
teaching as an interactive and meaningful classroom discourse. The teachers
viewed discussions and dialogues as important interactive features of dialogic
teaching which allowed for greater engagement and participation of students in
the lessons. The teachers noticed that the students were engaged in the learning
process through dialogic teaching. There was also an increase in the participation
of students in the discussions held. The whole class and group discussions
encouraged sharing of views, and students were said to be interested in talk. For
instance, T1 states “Most of my lessons, I will have discussions… “70% of my
lesson is discussion. When we have discussions, so they’ll share and sometimes
question each other, “why did you say that?” T1 states that the whole class
discussions created space for students to share their views.
This was echoed by T2 in which she viewed discussions as means for increasing
engagement and participation in class - “So this dialogic teaching is more
interactive, and I feel that the information and knowledge obtained by the
students are more interesting and learning happens at the same time” The
teachers also viewed discussions as their main pedagogical strategy. Students’ L2
learning developed through the whole class and group discussions. Whole class
or group discussions remained as a consistent feature throughout the lessons
observed.
The use of authentic questions such as open-ended questions during the
discussions sustained and extended the interactions and encouraged meaningful
responses. The teachers perceived open-ended questions as another core feature
of dialogic teaching which facilitated student talk and functioned as a discourse
strategy to extend talk amongst the students. The teachers employed open-ended
questions as a dominant communicative strategy to initiate talk and broaden the
scope of talk. The coding on all four semi-structured interview transcripts
indicated that open-ended questions were constantly used as a discourse strategy
to engage students in talk. This was also seen in the classroom observations where
open-ended questions were constantly used throughout the discussions. (T1)
stated that she found that open- ended questions had the capacity to generate talk
when she said” I would just pose an open-ended question just like… “How was
your weekend?” or “how did you spend your holidays?” where they’ll be sharing
ideas and collecting information”. She noticed that the questions posed during the
whole class discussions received meaningful responses, in particular on topics
that were familiar to the students, and created a chain of responses and further
questions. This discursive pattern encouraged talk. Thus, the individual semi
structured interviews with the teachers and the classroom observations provided
insights to the intervention process. This provided an understanding of the
teachers’ ability to enact dialogic teaching in their language classrooms and to
identify if it was a challenging task.
T4 stated that dialogic teaching involved asking open-ended questions as those

questions were authentic and related to students’ prior knowledge about the
381
topic. She asserted that she posed “a lot of questions, WH questions, open-ended
questions to get the students to talk.” For example, “questions as prompters
because that’s how they start their discussion. “So most of the questions are open-
ended questions because we need to share thoughts and through our sharing,
learning happens”. For her, open-ended questions are “the most effective to
encourage students to talk because students have the ideas to share”. This was
also evident in her lessons. The following extract demonstrated how she
facilitated talk through open-ended questions which led to second language
learning:
Excerpt 4:
T4: Look at this sign. What is this sign about? What can you tell me about
this sign?
S1: OKU ( Orang Kurang Upaya)
S2: OKU
S3: Handicapped
S4: Disable
S5: Disable People
T4: Okay good. Where can you find this sign?
S6: At the mall
S7: At the parking lot
T4; Good. Where else?
S8: At the roadside, teacher
The excerpt above is evidence of how dialogic teaching is interactive and

meaningful as a discourse structure. Both the teacher and students kept the
interactions going through the teacher questions and responses which exhibited
Alexander’s dialogic principle of collective where students address the leaning
task together. Students are seen listening and responding to the questions by
sharing their views which demonstrate Alexander’s reciprocal principle. Students
were attempting to respond despite language issues- such as OKU – the Malay
word for a disabled person. This showed students were comfortable and not
embarrassed to respond, which was another principle of dialogic teaching-
supportive. The interactions allowed for vocabulary to be acquired. In other
words, the talk facilitated the acquisition of vocabulary. The teacher did not
evaluate the response as right or wrong but instead attempted to extend the
dialogue by posing further questions in an effort to afford more student talk which
facilitated L2 learning. In this context, the open-ended question functioned as a
scaffolding tool that promoted second language learning.
The excerpt below demonstrated how the interactive discourse afforded second
language learning of vocabulary and grammar. T1 had indirectly introduced
collective nouns to students during the whole class discussion. A student
responded as scissors, and she reiterated by saying “Ah yes… A pair of scissors”.
Similarly, she also facilitated the learning of a new vocabulary when one student
responded as saying cream while another was precise in saying antiseptic cream.
Hence, the word antiseptic was learnt. The particular student might know the
word antiseptic but perhaps not the other student. Once again, the construction
of knowledge collectively on the vocabulary took place which demonstrated
meaningful learning.
382
Excerpt 5:
S3: Scissors.
T1: Ah yes… A pair of scissors. Okay, what else did you put in your
first aid kit?
S4: Cream
S6: Antiseptic cream
T1: Antiseptic cream…good!
3.4 Dialogic Teaching in Practice

The classroom observations demonstrated the application of the dialogic features-
the principles, repertoires, indicators in facilitating L2 learning and acquisition.
They were observed to have framed and facilitated talk through discussions, in
particular whole class and group discussions. Teachers were seen to pose open-
ended questions to initiate discussions to facilitate talk. The four classroom
observations showed teachers initiating talk through whole class discussions by
posing an open-ended question. However, the uptake was largely influenced by
students’ language proficiency. Students did display their engagement, but their
responses were rather limited to words and phrases and simple sentences. It was
also short in length due to their inability to construct complex sentences to clearly
express themselves in L2. Teachers facilitated the construction of knowledge
collectively through scaffolding of questions. They deliberately used open-ended
questions to scaffold the discussion that was taking place in the classroom. The
classroom excerpts above demonstrate that teachers use discussions to create the
talk environment and the open-ended questions as a scaffolding tool to extend
student talk.
4. Discussion
This study demonstrated a group of Malaysian L2 lower secondary teachers’
ability to employ dialogic teaching following a professional development
programme to facilitate second language learning. The implementation of
dialogic teaching based on the professional development programme functioned
as an intervention programme to enhance students’ oral communication skills and
second language learning. This finding supports the findings of (Ruthven et al.,
2017; Sedova, 2017; Sedova et al., 2017; Böheim et al., 2021; Hennessy et al., 2021)
that professional development enabled teachers to adopt a more dialogical
practice in their classrooms which also indicated pedagogical shifts.
In terms of the teachers’ professional development, the findings suggest that the
teachers had put into practice dialogic teaching as introduced. The perceptions of
the teachers on dialogic teaching as a new pedagogical approach focusing on talk
was largely based on the PD and in conducting the intervention. Their experiences
of the teaching process and their reflections of the approach throughout the
intervention informed their perceptions. The classroom observations
demonstrated that Alexander’s dialogic principles (2018) were applied in the
teaching process. The findings showed that the teachers involved utilised the five
dialogic principles of Alexander in their lessons but to a lesser degree. Focusing
on Alexander’s dialogic teaching principle of collectivity, the teachers involved
383
organised the learning tasks in whole class and group discussions to enable the
students to address the task together. The principle of reciprocality (Alexander,
2018) was also reflected in the teachers and students’ interactions. Through the
open-ended questions, students listened to each other attentively and then shared
their ideas by considering alternative viewpoints. This principle appeared to be
well comprehended by the teachers as it was applied adequately. The findings
concur with the study by Sedlacek & Sedova, (2017) where the open-ended
questions demonstrated higher engagement amongst students through
discussions which led to better reasoning. Nevertheless, in the context of this
study, observations, indicated that there were less argumentative responses
stimulated by alternative viewpoints. Instead, the responses comprised responses
that supported previous responses.
Findings also illustrated that the supportive principle (Alexander, 2018) was
applied through the use of authentic questions (Nsytrand, 1997). The teachers
posed open-ended questions to the class to encourage students to articulate their
ideas freely without risk of embarrassment over ‘wrong’ answers and help them
reach a common understanding. However, it was limited as the students involved
were constrained by language proficiency. Consequently, they were less fluent in
articulating their ideas freely. The cumulative principle was also reflected in
which students built on the responses provided by others. These responses were
then clustered and built into coherent lines of thought and understanding. By
integrating open-ended questions in the existing lower secondary Form One
English syllabus, the teachers’ tasks became purposeful. The teachers had
demonstrated Alexander’s principle of purposeful. The professional development
programme provided the teachers with the ability to adopt dialogic teaching in
the L2 classroom.
From a pedagogical perspective, the implementation of dialogic teaching in the

L2 classrooms reflected through the classroom observations indicated that the
teachers had changed their practices. Findings from the observations appear to
support the perceptions of the teachers that they had changed their practice
towards dialogic teaching. The emphasis of dialogic teaching is on the discourse
functions rather than the discourse structure which implies that it serves as a
functional construct instead of a structural construct. In this manner, the
discussions contributed to the learning of L2. The consistent use of discussions
and open-ended questions indicated a shift in the teachers’ pedagogical approach.
The open-ended questions were purposefully used because teachers were
convinced that it would provide every student with the opportunity to practise
and acquire the target language competencies such as grammar, vocabulary and
phonological awareness similar to the findings of (Chow et al., 2021). The teachers’
questions enabled the students to acquire vocabulary, pronunciation, and correct
grammatical structures.
The findings of this study also showed that the teachers’ adoption of dialogic
teaching demonstrated their appreciation of the approach as a discourse pattern
in which students were able to speak in English besides assuming more active
roles as students. The findings concur with the study by Snell and Lefstein (2018)
384
which showed how students who were considered as having lower ability
managed to be engaged in the learning process. The use of discussions and
dialogues created a collaborative culture in the learning process. Nevertheless,
dialogic teaching was far above and beyond just spoken language practices. This
outcome could mean that the teachers’ application of Alexander’s dialogic model
(2018) in this study might vary from the actual dialogic model. The two-week-
long professional development on dialogic pedagogy focused on both the
theoretical as well as the practical aspect which meant that teachers had to link
broad theoretical ideas about dialogic teaching into classroom practices. Thus, it
might be insufficient for the teachers to have digested the underpinning of this
new discourse approach to be translated into their classroom practices. Despite
the positive findings derived from the study, the sampling was small and thus
cannot be generalised to the wider group of Malaysian L2 teachers. This study
was also limited to teacher discourse pattern specifically teacher questioning and
did not investigate their other aspects in the enactment of dialogic teaching.
5. Conclusion
In summary, this study was aimed at exploring and investigating a group of
Malaysian L2 teachers’ perceptions and experiences of dialogic teaching in
facilitating second language learning. It demonstrates the possibility of teachers
to adopt dialogic teaching in the teaching of English (ESL) with the aim of
supporting students to attain better outcomes in the English language and
specifically to enhance students’ oral communication skills. This study contributes
to the body of research on dialogic teaching and specifically on professional
development programmes aimed at enacting dialogic teaching in classroom
settings. This study implies that the specific training on the infusion of dialogic
discourse given to teachers through teacher professional development enabled
the adoption of dialogic teaching in the L2 classrooms. The implementation of
dialogic teaching created a shift in the teachers’ pedagogical practices influenced
by the need to address the issues of student’s oral proficiency. The findings
demonstrated that teachers’ use of dialogic strategies created opportunities for
language use which led to oral proficiency in the English language. The ability of
teachers to adopt dialogic pedagogy demonstrated that their pedagogic intentions
had shifted which influenced the changes in curricula, in the teacher’s role and
the underpinning theories of education. Theoretically, dialogic teaching is
underpinned by the sociocultural theory which demonstrates a move from a
behaviourist theory of second language learning which is teacher centred to a
more learner centred approach. Hence, the study adds to the literature in that
dialogic teaching is a ‘new pedagogical approach’ in the context of Malaysian ESL
classrooms with the aim of changing teachers’ current pedagogical practices.
Future professional development for L2 teachers would require an in-depth
understanding of the dialogic models and skills to develop competence to frame
classroom talk amongst L2 learners. Teachers may require more training on the
application of the dialogical approach in the context of L2 learners. Opportunities
to apply the dialogical approach in the context of L2 learners should be provided
during future professional development to allow teachers some practice and to
obtain feedback from trainers on their teaching. Activities involving teacher
385
reflection on their implementation of dialogical approach should also be

incorporated in future training to support teachers’ professional development.
6. References
Alexander, R. (2018). Developing dialogic teaching: genesis, process, trial. Research Papers
in Education, 33(5), 561–598. https://doi.org/10.1080/02671522.2018.1481140
Al Balushi, K. (2016). The Use of Online Semi-Structured Interviews in Interpretive
Research. International Journal of Science and Research (IJSR) ISSN (Online Index
Copernicus Value Impact Factor, 57(4), 2319–7064.
https://doi.org/10.21275/ART20181393
Asterhan, C. S. C., Howe, C., Lefstein, A., Matusov, E., & Reznitskaya, A. (2020).
Controversies and consensus in research on dialogic teaching and learning.
Dialogic Pedagogy, 8(January), S1–S16. https://doi.org/10.5195/DPJ.2020.312
Barekat, B., & Mohammadi, S. (2014). The Contribution of the Teachers’ Use of Dialogic
Discourse Pattern to the Improvement of the Students’ Speaking Ability. Procedia
- Social and Behavioral Sciences, 98, 353–362.
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative research
in Psychology, 3(2), 77-101.
Böheim, R., Schnitzler, K., Gröschner, A., Weil, M., Knogler, M., Schindler, A. K., Alles,
M., & Seidel, T. (2021). How changes in teachers’ dialogic discourse practice
relate to changes in students’ activation, motivation and cognitive engagement.
Learning, Culture and Social Interaction, 28.
https://doi.org/10.1016/j.lcsi.2020.100450
Boyd, M. P. (2015). Relations Between Teacher Questioning and Student Talk in One
Elementary ELL Classroom. Journal of Literacy Research, 47(3), 370–404.
https://doi.org/10.1177/1086296X16632451
Castillo-Montoya, M. (2016). Preparing for interview research: The interview protocol
refinement framework. Qualitative Report, 21(5), 811–831.
https://doi.org/10.46743/2160-3715/2016.2337
Chang, B. (2017). Dialogues in Promoting Knowledge Construction, 0–5.
Chow, B. W. Y., Hui, A. N. N., Li, Z., & Dong, Y. (2021). Dialogic teaching in English-as-
a-second-language classroom: Its effects on first graders with different levels of
vocabulary knowledge. Language Teaching Research.
https://doi.org/10.1177/1362168820981399
Creswell, J.W. (2013) Research Design: Qualitative, Quantitative, and Mixed Methods
Approaches. 4th Edition, SAGE Publications, Inc., London.
Cui, R., & Teo, P. (2021). Dialogic education for classroom teaching: a critical review.
Language and Education, 35(3), 187–203.
https://doi.org/10.1080/09500782.2020.1837859
Díez-Palomar, J., Chan, M. C. E., Clarke, D., & Padrós, M. (2021). How does dialogical
talk promote student learning during small group work? An exploratory study.
Learning, Culture and Social Interaction, 30 (December 2020), 1–17.
https://doi.org/10.1016/j.lcsi.2021.100540
Doley, S. K. (2019). Discourse patterns in English as an L2 classroom in India.
English Language Teaching Centre (2013). Oral Proficiency In English For Secondary
Schools (OPS-ENGLISH) Trainer’s Guide. Tinta Press Sdn. Bhd
Flick, U. (Ed.). (2013). The SAGE handbook of qualitative data analysis. Sage.
386
Hardman, J. (2019). Developing and supporting the implementation of a dialogic

pedagogy in primary schools in England. Teaching and Teacher Education, 86.
https://doi.org/10.1016/j.tate.2019.102908
Hennessy, S., Kershner, R., Calcagni, E., & Ahmed, F. (2021). Supporting practitioner-led
inquiry into classroom dialogue with a research-informed professional learning
resource: A design-based approach. Review of Education, 9(3).
https://doi.org/10.1002/rev3.3269
Hennessy, S., Rojas-Drummond, S., Higham, R., Márquez, A. M., Maine, F., Ríos, R. M.,
& Barrera, M. J. (2016). Developing a coding scheme for analysing classroom
dialogue across educational contexts. Learning, Culture and Social Interaction, 9,
16–44. http://doi.org/10.1016/j.lcsi.2015.12.001
Kim, M. Y., & Wilkinson, I. A. G. (2019). What is dialogic teaching? Constructing,
deconstructing, and reconstructing a pedagogy of classroom talk. Learning,
Culture and Social Interaction, 21(February), 70–86.
https://doi.org/10.1016/j.lcsi.2019.02.003
Lee, R. (2016). Implementing Dialogic Teaching in a Singapore English Language
Classroom. RELC Journal, 47(3), 279–293.
https://doi.org/10.1177/0033688216631171
Mercer, N., Dawes, L., & Staarman, J. K. (2009). Dialogic teaching in the primary science
classroom. Language and Education, 23(4), 353-369.
Nystrand, M., Gamoran, A., Kachur, R., & Prendergast, C. (1997). Opening Dialogue.
Teachers College Press.
Osborne, J. F., Borko, H., Fishman, E., Gomez Zaccarelli, F., Berson, E., Busch, K. C.,
Reigh, E., & Tseng, A. (2019). Impacts of a practice-based professional
development program on elementary teachers’ facilitation of and student
engagement with scientific argumentation. American Educational Research
Journal, 56(4), 1067-1112.
Reznitskaya A., & Gregory, M. (2013). Student Thought and Classroom Language:
Examining the Mechanisms of Change in Dialogic Teaching. Educational
Psychologist, 48(2), 114–133. https://doi.org/10.1080/00461520.2013.775898
Ruthven, K., Mercer, N., Taber, K. S., Guardia, P., Hofmann, R., Ilie, S, Luthman S., Riga,
F. (2017). A research-informed dialogic-teaching approach to early secondary
school mathematics and science: the pedagogical design and field trial of the
epiSTEMe intervention. Research Papers in Education, 32(1), 18–40.
https://doi.org/10.1080/02671522.2015.1129642
Sedlacek, M., & Sedova, K. (2017). How many are talking? The role of collectivity in
dialogic teaching. International Journal of Educational Research, 85, 99–108.
https://doi.org/10.1016/j.ijer.2017.07.001
Sedova, K. (2017). A case study of a transition to dialogic teaching as a process of
gradual change. Teaching and Teacher Education, 67, 278–290.
https://doi.org/10.1016/j.tate.2017.06.018
Sedova, K., Salamounova, Z., & Svaricek, R. (2014). Troubles with dialogic teaching.
Learning, Culture and Social Interaction, 3(4), 274–285.
Snell, J., & Lefstein, A. (2018). “Low Ability,” Participation, and Identity in Dialogic
Pedagogy. American Educational Research Journal, 55(1), 40–78.
https://doi.org/10.3102/0002831217730010
Swain, M., & Watanabe, Y. (2012). Languaging: Collaborative Dialogue as a Source of
Second Language Learning. The Encyclopedia of Applied Linguistics, January.
https://doi.org/10.1002/9781405198431.wbeal0664
387
Teo, P. (2013). “Stretch your answers”: Opening the dialogic space in teaching and
learning. Learning, Culture and Social Interaction, 2(2), 91–101.
van de Pol, J., Brindley, S., & Higham, R. J. E. (2017). Two secondary teachers’
understanding and classroom practice of dialogic teaching: a case study.
Educational Studies, 43(5), 497–515.
https://doi.org/10.1080/03055698.2017.1293508
Wilkinson, I. A. G., Reznitskaya, A., Bourdage, K., Oyler, J., Glina, M., Drewry, R., Kim,
M-Y., &Nelson, K. (2017). Toward a more dialogic pedagogy: changing teachers’
beliefs and practices through professional development in language arts
classrooms. Language and Education, 31(1), 65–82.
https://doi.org/10.1080/09500782.2016.1230129
388

Vol. 22, No. 7, pp. 388-410, July 2023
Critical Factors for Enhancing Students’

Collaborative Learning Experiences in a Project-
based Connectivism Learning Environment
Fui Theng Leow
Nottingham University Ningbo
Ningbo, China
Mai Neo
Multimedia University
Cyberjaya, Malaysia
Abstract. The advent of technology has led to a significant challenge in

redesigning today’s learning environment, as much technology is
under-utilized in many classrooms, and educators still lack the
confidence and proper theoretical frameworks to redesign their classes
efficiently. The objective of this study was to design a project-based
connectivism learning environment that transforms the traditional
teacher-centered class environment into more student-centered learning
approaches to enhance their collaborative learning experiences. Students
were engaged in developing a group project and constructivism
learning activities to complete their projects collaboratively. A mixed-
method research design using four data collection instruments (survey,
open-ended questions, interviews and online posts) was deployed on
300 participating students in a Malaysian university to investigate the
critical factors for improving students’ learning experiences within this
learning environment. The Exploratory Factor Analysis performed
yielded four factors, 1) Group Dynamics and Experiences, 2) Motivation,
3) Use of Technologies, and 4) Real-World Relevance. The findings
demonstrated that students improved their collaborative skills,
technology was better optimized, and Connectivism Learning effectively
enhanced the student learning process. The resulting Multimedia-based
Connectivism Learning Framework (MCLF) was successfully aligned
with four shifts in the Malaysia Blueprint for Higher Education and may
serve as a practical guideline for redesigning classrooms for 21st-century
education.
Keywords: Connectivism; project-based learning; multimedia; Malaysia;

online learning
©Authors
389
1. Introduction
The 21st century has experienced the rise of globalization, the advent of the
digital revolution, the new paradigm of the knowledge economy, and the
emergence of the net generation. The ever-changing information and shifting
realities bring a rapid growth of knowledge and the lifelong learning trend. The
popularity of technologies has also stimulated innovation in learning that
requires higher information literacy and digital fluency. In preparing university
students for future challenges, the traditional learning approach has shifted
towards the trends of learning through technologies and making connections
with communities and resources. As digital technologies advance in their
capacities and innovations, new possibilities in the teaching and learning context
arise (Halili et al., 2021; Pelletier et al., 2022). However, it also brings challenges
in optimizing the use of technology and ensuring the quality of teaching and
learning to enrich students’ learning experiences in university classroom
contexts (Kukulska-Hulme et al., 2020; Moorhouse & Wong, 2022). Therefore,
redesigning today’s learning environment becomes a challenging process of
exploring different ways of learning while conforming to the social requirements
and students’ needs. Contemporary educational approaches are moving towards
blended learning. Blended learning utilizes a range of technological resources in
facilitating traditional face-to-face (F2F) instruction; and flipped classroom (or
inverted classroom), which engages students in learning activities (Al-Samarraie
et al., 2021; Koh, 2019). These educational approaches focus on engaging
students in collaborative activities, developing students’ high-order thinking,
and connecting all resources (Demosthenous et al., 2020; Murillo-Zamorano et
al., 2019). Therefore, in the 21st-century learning context, implementing flipped
classrooms, utilizing digital technologies, and fostering students’ responsibility
and ownership of their learning have become important components in
constructing a conceptual framework (Al-Samarraie et al., 2020; Andres et al.,
2017; Murillo-Zamorano et al., 2019).
Several research gaps motivated this research study. In Malaysia, the 2018
Malaysian Blueprint (Higher Education) tasked Malaysian higher education
institutions with redesigning their learning spaces using 21st-century pedagogy
and utilizing the latest teaching and learning technologies (Halili et al., 2021;
Ministry of Higher Education Malaysia, 2015). It prescribed 10 shifts in the
Blueprint, four of which were relevant to this study. These were: 1) Produce
holistic, entrepreneurial and balanced graduates (Shift 1), Become a nation of
lifelong learners (Shift 3), 3) Globalized online learning and transformation of
educational delivery (Shift 9) and 4) Transformed higher education learning
(Shift 10). However, studies have shown that the lecture model remains
dominant in many Malaysian university classrooms (Halili et al., 2021; Müller &
Mildenberger, 2021; Rahayu, 2019), creating a gap between the Malaysian
government’s call for using technology in classrooms with the actual scenarios
in universities. Educators still focus on students’ achievement and consider
activities outside school that require collaboration as secondary to the university
teaching and learning context (Brown et al., 2020; Halili et al., 2021; Ministry of
Higher Education Malaysia, 2015).
390
There is also a research gap in the level of technology use among instructors.
Studies have found that instructors’ competencies and confidence in using
technologies play an essential role in peer interaction and the community of
support (Fuad et al., 2020; Gomez, 2020; Moorhouse & Wong, 2022; Rahayu,
2019). However, many instructors still lack confidence in creating engaging
learning environments other than simply uploading their lecture notes online.
This is due to ineffective pedagogical models that support and enable instructors
to create such learning environments. Studies found that there remains a lack of
frameworks that engage students with the experiences of ‘learning how to learn’,
develop their digital literacy, as well as enhance their capabilities in managing
resources (Fuad et al., 2020; Moorhouse & Wong, 2022; Siemens et al., 2020).
Research has also shown that students remain reluctant to engage in
collaborative activities in the classroom due to poorly designed learning
environments. This problem is more prevalent in Malaysia, where the secondary
education system still operates on an individual basis, making it challenging for
students matriculating to university to be comfortable working and
collaborating in groups without effort from their instructors to create
opportunities for collaborative work. With the increased use of technologies
among this young generation of students, it has become more critical for
instructors to efficiently select and utilize technology and web resources to
effectively improve the student learning process (Alkhawaja et al., 2021; Fuad et
al., 2020; Moorhouse & Wong, 2022; Ministry of Higher Education Malaysia,
2015).
Recent literature highlighted that a well-designed learning space and

appropriate pedagogy complement each other in stimulating students’ creativity
and inspiration (Brown et al., 2020; Bulow, 2022; Fuad et al., 2020). These spaces
are also fertile soil for nurturing students’ 21st-century skills and transforming
them to be self-directed, communicative, and resourceful in the diverse 21st-
century workforce (Pelletier et al., 2021). Hence, this study sought to investigate
the impact of redesigning the learning environment, through a pedagogically
sound learning theory and with the support of multimedia and web
technologies, on the student learning process. In this study, the Connectivism
Learning theory was integrated into redesigning the learning environment
supported by the Internet and digital technologies.
2. Literature review
Recently, an increasing number of universities have started using social media to
engage students in their learning activities and online communication
(Moghavvemi et al., 2018). Studies have found that appropriate uses of digital
technologies in the university learning environment can potentially stimulate
interactivity with other people outside the classroom, engage students in
collaborative learning, and enhance students’ ability in creative content
development (Cheung, 2021; Moorhouse, 2023; Müller & Mildenberger, 2021;
Rahayu, 2019). With social media and Web 2.0 tools, educators can better
transition to more student-centred and technology-supported learning
environments. However, studies have found that today’s university class
environments are still mainly organized by traditional teacher-led approaches,
391
supported by textbooks and lecture notes, controlled by instructors who conduct

the class by delivering the lectures (Cheung, 2021; Fuad et al., 2020). In other
words, there are still challenges for many educators making this transition, as
educators with more technology-supported or e-learning teaching experiences
can better transition to the virtual environment than those with little to no
experience. Research has shown that there is still a lack of confidence in many
educators to use e-learning technology and Web 2.0 tools in their classrooms,
especially when the efficacy of these learning environments depends on the
learning support of the educator.
The Malaysia Education Blueprint 2015-2025 (Higher Education) showed that

Malaysian graduates lack communication, problem-solving skills, critical
thinking skills and creativity skills (Azmi et al., 2018; Bakar, 2023; Fuad et al.,
2020; Ministry of Higher Education Malaysia, 2015). The inadequacy of
employability skills among Malaysian university graduates has resulted in a
national drive towards nurturing students to be more innovative and adaptable
to changes in today’s learning environments (Goh & Abdul-Wahab, 2020;
Ministry of Higher Education, 2015). Studies suggest that engaging students in
collaborative projects with sufficient exposure to industry practices and social
resources will allow them to acquire real-world experiences (Alexander et al.,
2019). This further emphasizes the need to redesign learning environments to
prepare university graduates with employability competencies in collaborative
workplaces (Bakar, 2023; Pelletier et al., 2021).
Project-based learning (PjBL) has been widely applied in today's educational

contexts, especially in higher education institutions, to promote a dynamic and
action-oriented learning strategy. It engages students in group learning and
assuming responsibility for their decisions. It also enables them to become active
participants and to experience the role of the community, as it focuses on
students’ ability to work collaboratively to solve real-life problems (Almazroui,
2023; Mursid et al., 2022; Surur et al., 2023). PjBL requires students to work with
a community and use various digital technologies to solve complex problems
while developing an interdisciplinary project. Studies have found that successful
PjBL is determined by the level of peer interaction and project complexity. By
periodically tracking students’ learning processes, the instructor can provide
necessary resources to support students in managing their time, reflecting on
their ideas, and setting priorities for co-constructing project work (Mursid et al.,
2022; Tsybulsky, 2023). By providing more complex project contexts, more
opportunities arise for students to co-develop the content, discuss desirable
outcomes, and evaluate solutions from different perspectives (Markula &
Aksela, 2022; Tsybulsky, 2023). In recent years, PjBL has been known as an
effective method of developing students’ competencies. Through the learning
opportunities given by the project, students are equipped with not only subject-
matter knowledge but also advancing students’ critical thinking,
communication, problem-solving and group collaboration skills (Markula &
Aksela, 2022). Therefore, in this research, project-based learning (PjBL) was used
as an instructional strategy for enhancing the student learning process with
community building, problem-solving, and content-co-creation experiences.
392
In support of redesigning the learning environment from the conventional

classroom to project-based learning within a technology-supported
environment, the Connectivism Learning theory (Siemens, 2005) was used in
this research. Connectivism is growing dominant as a theoretical framework
that addresses learning in complex, social, networked environments and
knowledge with its multiple perspectives (Downes, 2022; Siemens, 2005;
Siemens et al., 2020). Connectivism learning is the process of connecting with
various information sources and social networking with continuously shifting
elements and diverse opinions (Downes, 2019; Siemens et al., 2020). It promotes
‘learning how to be a learner’ and evaluates learning based on the ability to filter
information valuable to a situation (Chandrappa, 2018; Dziubaniuk et al., 2023).
Connectivism learning emphasizes information flow in networked
environments and emphasizes informal learning where students are encouraged
to use social media, Web 2.0 tools and multimedia to engage with others who
have different learning styles and capabilities. Connectivism learning provides
opportunities for students to gain exposure to media and social information,
adopt new tools used by others, acquire the necessary skills or resources, and
express personal understandings. It also develops students’ interests in
becoming content producers, engaging in collaborative learning, and supporting
others in various situations. Therefore, informal learning helps students gain
more social presence and recognition in the community (Downes, 2019; Downes,
2022; Dziubaniuk et al., 2023; Siemens et al., 2020).
As such, Siemens’s (2005) Connectivism Learning was deemed a suitable

learning theory in this research study to support the transition from the
traditional lecture-based approach of teaching towards a more student-centred,
project-based learning environment that engages students in building a learning
community, supported by digital technologies, social media and web resources.
Figure 1 presents the conceptual framework for this research.
Figure 1: The conceptual framework for the research study
393
Thus, this research sought to investigate the following question: "What are the
factors for improving students’ learning experiences in a project-based
Connectivism learning environment?”. By identifying the key factors
contributing to the efficacy of this learning environment, educators would be
better able to transition from conventional lecture-based classes to more
pedagogically sound project-based and collaborative learning environments
with better student learning experiences.
3. Method: Designing the Connectivism Learning Environment

Siemens’s (2005) Connectivism principles were incorporated into designing a
connectivism learning environment that allows students to network connections
with peers via social networking or online collaboration tools. The mapping of
Siemens’s (2005) Connectivism principles to the connectivism learning activities
in the class is shown in Table 1.
Table 1: Mapping of Siemens’s (2005) Connectivism to the classroom
Siemens’ Connectivism Implementation in the class

Learning Principles
Principle 1: “Learning and Students were tasked to solve problems in a

knowledge rests in multimedia group project with their peers by
diversity of opinions” using digital technologies to produce a shared
solution.
Principle 2: “Learning is a Students were encouraged to consolidate the

process of connecting sources and publish with the references in a
information sources” blog post as the Background Study Journal.
Principle 3: “Learning may Students accessed learning content through e-

reside in non-human library resources, their university’s LMS, e-
appliances” learning platforms, video streaming sites,
online databases, and web-based search tools
to search for media files, books, and websites.
Principle 4: “Capacity to Here, students were engaged in an

know more is more critical “apprenticeship” with senior students,
than what is currently industry experts, online forum users, guest
known” lectures, and campus-wide academic-related
events.
Principle 5: “Maintaining Students published their productions on

connections is needed to YouTube video channels, and competitions,
facilitate continual showcasing their productions in their social
learning” profile.
Principle 6: “Ability to see Here, students shared their project prototypes

connections between ideas in Facebook Groups for communicating their
and concepts is a core skill” ideas and seeking more ideas.
Principle 7: “Currency is the Students were required to record the accessed

intent of all connectivism documents, achieve tasks, and summarize the
394
learning” current results for posting in the Work

Progress Journal.
Principle 8: “Decision- Students’ were responsible for all levels of

making is a learning decision-making during the project duration.
process. Choices affect
outcomes”
In this research, 300 IT undergraduate students at INTI International University,

Malaysia, participated in this study. These students were taking the subject,
'Graphic Design and Animations.’ They signed a consent form and agreed to
participate in the study. With convenience sampling, the research samples were
formed by the participating students who had 1) fulfilled the requirements of the
subject entirely without dropping out; 2) agreed with a signature on the
acknowledgement form to participate voluntarily in this research study; and 3)
provided valid responses and feedback in both written and oral forms upon
completion of the multimedia group project. Thus, the research samples
comprised 300 first-year undergraduate IT students as participating students,
who were, on average, aged 18 to 20 years old. These 300 students were divided
into 83 project groups, each with three to four members within the first two
weeks of the semester.
A multimedia group project was assigned to all groups, and the project required
students to work collaboratively in solving the problem. Students were tasked to
rebrand the existing advertisement content, which had an unattractive and
outdated design, within 14 weeks. Figure 2 shows an example of a student’s
project outcomes, and Figure 3 illustrates their online collaborations.
Figure 2: Student team’s final project outcome
395
Figure 3: Student online collaboration with web tools
Data collection procedure

A mixed-method research design was employed in this research study. In this
mixed-method research design, the triangulation data analysis approach was
used to combine various viewpoints, different opinions, and multiple forms of
data collected from all research instruments. Four research instruments were
utilized for collecting both quantitative and qualitative data from the
perspectives of what students think, say and do. These research instruments
were 5-point Likert scale questionnaires, open-ended questions, face-to-face
interviews, and students’ online posts and activities.
The 5-point Likert scale questionnaire was administered to all participating

students to gauge their perceptions and attitudes about the Connectivism
learning environment they were exposed to. This questionnaire consisted of 40
survey items with 5-point Likert scale, with 1 for strongly disagree (SDA), 2 for
disagree (DA), 3 for undecided (U), 4 for agree (A), 5 for strongly agree (SA), and
administered at the end of the trimester. Additionally, open-ended questions
were administered together with the questionnaire form, allowing students to
respond simultaneously with the questionnaire. These open-ended questions
allowed them to provide more detailed comments on the learning environment.
Face-to-face interviews were conducted on another day but in the same week
after the participating students completed the questionnaire and open-ended
questions. After obtaining consent from the participants, the interviews were
recorded and subsequently transcribed. Finally, students’ online posts and
activities were culled from their online blogs, which included their chat
transcripts, media uploads, updates of the events, written reports, discussion
threads and replies, announcements, and exchange of resources, which were all
made available and updated by the groups throughout the entire project
development process.
396
An Exploratory Factor Analysis and Descriptive Analysis were performed on the

quantitative dataset, while discourse analysis was employed on students’
comments, online comments, and the frequency counts in the learning
environment, communication channels, and social networks. The discourse
analysis used the factors identified from the exploratory factor analysis as the
main themes for coding and constructing various aspects.
4. Analysis and Findings

To determine the critical factors that reflected student perceptions and learning
experience in this learning environment, the students’ survey response on the
40-item questionnaire was used to analyze the statistical results through
exploratory factor analysis (EFA), Cronbach’s Alpha Test, and descriptive
analyses. The IBM Statistical Package for the Social Sciences (SPSS) was used to
analyze the quantitative data and perform the Exploratory Factor Analysis
(EFA). In performing the discourse analysis with the sub-categories, the NVivo
Software, a qualitative and mixed-method data analysis software tool, was used
to study students’ feedback and comments. The process included inserting,
organizing, coding, and sorting the data using query tools to cross-examine the
patterns or areas for further analysis and interpret the factors identified from the
EFA.
Although there are no absolute thresholds for minimum sample size, Williams et
al. (2010) suggested that a larger sample size is more accurate for defining the
number of factors in an EFA. Hence, in this research study, the research sample
size of 300 respondents was considered adequate to perform EFA and achieve
good factor recovery as long as the average communality ranges between 0.5 to
0.6. This study employed Principal Component Analysis (PCA) with the
orthogonal varimax rotation method to make the factors more interpretable.
Based on the anti-image correlation matrix performed, all survey diagonal
variables yielded values above 0.9, and all off-diagonal variables had minimal
correlations. This result showed that the survey items correlated significantly,
which made it conducive to performing an exploratory factor analysis (EFA).
The EFA was performed using the Principal Component Analysis (PCA) factor
extraction method. The Kaiser-Meyer-Olkin (KMO) Measure of Sampling
Adequacy and Bartlett’s Test of Sphericity were used to determine the adequacy
of the data for factor analysis. Guidelines in the literature suggest that a KMO of
more than 0.5 indicates the data is appropriate for applying factor analysis, and
the data is considered good at above 0.7. In this research study, the data yielded
a KMO sampling adequacy of 0.928, indicating that the data sample can be
considered adequate for factor analysis. The correlations between items also
proved sufficiently large for PCA, with the result of Bartlett’s test of Sphericity
showing X2 (300) = 3453.604, p <.001. Table 2 presents the results of the KMO
and Bartlett’s tests performed.
397
Table 2. The KMO measure and Bartlett’s test performed

Kaiser-Meyer-Olkin Measure of Sampling Adequacy. .928
3453.60
Approx. Chi-Square
4
Bartlett's Test of Sphericity
df 300
Sig. .000
In reducing the number of survey items and organizing them into the factors for
further interpretation, Kaiser’s criteria were selected as the method in the
process of factor extraction in this research study, with the average communality
falling in the range between 0.5 to 0.6 (Field, 2013; Williams et al., 2010). In this
analysis, 25 survey items yielded commonalities within that range and were
retained. In addition, based on the results of factor extraction and rotation
performed, a total of four extracted components (factors) with an eigenvalue
greater than one were extracted, with an accumulative percentage of variance of
56.723% of the variability in the total number of variables. Specifically, the first
factor explained 26.831% of the total variance, with the second, third, and fourth
factors explaining 11.967%, 10.332%, and 7.594% of total variance respectively.
The orthogonal varimax rotation was used in this study to produce factor
clusters or structures that can be better interpreted. The factor loading was set at
0.5 as the cut-off point for cleaning the factor structure. In the filtering process,
four rounds of the EFA process were executed to suppress the loadings below .5.
The first round of cleaning removed six items with a factor loading below 0.5,
the second round removed five items, and the third round removed four items.
Hence, the rotated component matrix remained with 25 variables, with 14 items
loading onto factor 1, four items loading onto factor 2, four items loading onto
factor 3, and three items loading onto factor 4. These four factors were then
named based on the context and attributes or the variables identified within the
factors through PCA.
In factor 1, the survey items consisted of the attributes surrounding peer

interaction and personal experience in teamwork, as well as group settings and
activities such as problem-solving, meetings and task allocation. It is consistent
with the literature on group learning with project settings which emphasize
team effort in complementing each other’s skills, distributing responsibility, and
increasing the vibrancy of interaction. Therefore, factor 1 was named Group
Dynamic and Experiences. In factor 2, the survey items comprised the attributes
surrounding the rise of students’ motivation and inspiration, pleasant feelings,
and the sense of satisfaction through developing the project. As presented in
other studies, generating deeper understanding and increasing students’
capabilities required motivating students to take charge of their learning process
and be inspired to develop real-world skills by participating in the project
development. Therefore, factor 2 was named Motivation.
In factor 3, the survey items consisted of the attributes regarding the use of web
resources, the contacts between students and instructor, constant updates on the
work progress, and skills enhancement during the learning process. The
398
literature supports that advanced digital technologies and social media have
enhanced today's 21st-century learning for distributing knowledge across
networks, expanding interactivity beyond classroom settings, and encouraging
self-directed learning to complement classroom-based lecture models. Therefore,
factor 3 was named Use of Technologies. In factor 4, the survey items consisted
of the attributes surrounding managing project tasks, generalizing the learned
skills for future enhancements and new achievements. It is consistent with the
concept of connectivism learning where creating, manipulating, and utilizing
information flow are vital activities in promoting active learning. It also
encourages students to engage in informal learning and social interaction by
joining or building the learning communities and repurposing the resources.
Therefore, factor 4 was named Real-World Relevance.
Factor 1 - Group Dynamic and Experiences

The descriptive statistics for factor 1, Group Dynamic and Experiences is
presented in Table 2, where M is the means, STD for standard deviation, and p
represents the percentage of positive responses on the survey (i.e., students
reporting “Agree” or “Strongly Agree” on the survey).
Table 2. Survey responses for the factor of Group Dynamics and Experiences
Survey Items Mean (M) p
1. “I got to know my group members well.” 4.17 80.8
2. “My group helped me do my best in the project.” 4.07 77.6
3. “My group leader was very effective.” 4.04 75.2
4. “My group was supportive of member's problems
3.98 73.6
and helped resolve them.”
5. “My group communicated well with each other.” 3.97 72.3
6. “My group taught me some things I would not have
3.96 70.3
learnt on my own.”
7. “My group was able to solve our problems and
3.95 76.7
conflicts in a positive manner.”
8. “I learn more from the collaboration than on my
3.92 74.0
own.”
9. “I enjoy working in a team.” 3.90 71.3
10. “Our meetings were well attended.” 3.86 66.7
11. “My group's interactions were smooth.” 3.85 69.0
12. “There was a lot of unity in my group.” 3.79 68.0
13. “We were able to organize our work effectively.” 3.67 61.7
399
Student Comments (verbatim):

1. “I’m very like this type of learning…I can’t do all the job by myself…cooperation can
make more perfect project then one person.”
2. “I learn from each other as each one of us has their own area of expertise…”
3. “We improved our leadership skills and our behaviour in working with groups…”
4. “We have a lot of interaction...have our dinner together and discuss about our
group.”
5. “Tolerate with each other, better relationship…I never regret joining this group…”
6. “…enjoy working with this team…we have a strong connection between each other,
every meeting, we all will be there…”
7. “This project has strengthen the bond between our group members, there were minor
conflicts but nothing we couldn't handle.”
As displayed in Table 2 (survey items were numbered based on the mean scores
in descending order), 80.8% of students reported that they got to know their
group members well (Item 1, M = 4.17), 77.6% reported that their group helped
them to do their best in the project (Item 2, M=4.07), and 75.2% reported that
their group leader was very effective (Item 3, M=4.04). Additionally, 73,6% of
students reported that they found support in their group members (Item 4,
M=3.98), with 72.3% and 70.3% of them reporting that they were able to
communicate well and learnt things from the group they would not have been
able to learn on their own (Item 5, M=3.97, and Item 6, M=3.96, respectively).
Peer interaction was also important, as 76.7% and 74% of students reported that
they were better able to solve problems with the group’s help (Item 7, M=3.95)
and learned more from their collaborations (Item 8, M=3.92), respectively, which
resulted in 71.3% of students reporting that they enjoyed working in a team
(Item 9, M = 3.90). In terms of team dynamics, 66.7% of students reported that
their meetings were well attended (Item 10, M = 3.86), 69% reported that there
was unity in the group (Item 11, M = 3.85), and 61.7% reported that they were
able to organize their work effectively. These results showed that feeling familiar
with their peers, building work relationships, and assuming a functional role
were important in forming the project group. Based on the total word frequency
count of all students’ opinions and feedback, the keywords “members”,
“people'', and ''team'' were the most frequently mentioned in their comments. It
also showed that students realized the benefits and values of collaboration. They
have a stronger sense of personal satisfaction that brought positive feelings and
impressions, consistent with the keywords “problems”, “solving”, “asking'' being
the most frequently mentioned words across all student comments.
Factor 2 - Motivation
The descriptive statistics for factor 2, Motivation, are presented in Table 3.
Table 3. Survey responses for the factor of Motivation.

Survey Items Mean (M) STD p
1. “The project made me want to do my best.” 4.15 0.690 85.6
2. “I am very satisfied with my contribution to the
3.88 0.763 70.0
project.”
3. “I enjoyed working on a project like this.” 3.86 0.865 71.0
400
4. “I was very motivated to do this project.” 3.85 0.808 75.7

1. “Our goal is to complete the project in the best way.”
2. “To create an amazing project as possible as we can.”
3. “…almost nothing better than creating something from scratch and then
getting to see the final outcome.”
4. “…learn more stuff is interesting…we learned that there are so many ways to
create just a simple stickman.”
5. “... felt very happy as I was tasked to do what I am best at the most, designing
and drawing.”
As displayed in Table 3 (survey items were numbered based on the mean scores
in descending order), 85.6% of students reported that they were motivated to do
their best work on the project (Item 1, M=4.15), with 75.7% reporting high
motivation levels (Item 4, M = 3.85) and 71% reporting that they enjoyed
working on a group project like this (Item 3, M = 3.86). Overall, 70% of students
reported that they were very satisfied with their contributions to the project
(Item 2, M = 3.88). These findings showed that the factor of Motivation included
the component of opportunities provided by the multimedia group project that
inspired students to devote their efforts and achieve their aims. The second
component was the interest in the learning process, which attracted students’
attention and increased their motivation levels. These results are also supported
by their comments. Students commented that they “felt very happy….” and that
learning about the project was “...interesting”.
Factor 3 - Use of Technologies

The descriptive statistics for factor 3, Use of Technologies, are presented in Table
4.
Table 4. Survey responses for the factor of Use of Technologies
Mean
Survey Items STD p
(M)
1. “I enjoyed using the web to acquire information for my
4.03 0.758 77.9
project.”
2. “I found using the Web to communicate my progress
3.90 0.825 72.9
very useful in my learning.”
3. “The project allowed me to develop and improve my
3.87 0.823 72.3
presentation skills.”
4. “I was able to maintain contact with my lecturer.” 3.85 0.882 69.0
1. “I had researched a lot of information from the internet, including creative ways to
present…”
2. “…get to make full use of the technology with different methods…the freedom to
freely explore…”
3. “I am currently taking CS50 course from Harvard University as part of their Open
Source Learning on iTunes University.”
4. “…we can search through the open world, I very prefer and appreciate the learning
environment…because that everybody has different design ideas.”
5. “…have the chance to share our design ideas to everyone…and gain more creative
design ideas by looking on others creation.”
401
As shown in Table 4 (survey items were numbered based on the mean scores in
descending order), 77.9% of students reported that they enjoyed using the Web
to search for information (Item 1, M = 4.03), with 72.9% reporting that they used
the Web to display and present their progress (Iten 2, M = 3.90), and 69%
reported that they were able to maintain contact with their lecturer (Item 4, M =
3.85). Overall, using technologies helped 72.3% of students to develop and
improve their presentation skills (Item 3, M = 3.87). These findings showed that
the use of Technologies included extending social support, which potentially
transformed the students to be more active and resourceful in sourcing and
managing their learning materials. The second component was creative thinking
in the learning process which stimulated imagination and discoveries for
generating new ideas and proposing better solutions. These were also supported
by students’ comments where they reported that the Web technology allowed
them to “…search through the open world, I very prefer and appreciate the learning
environment” and that having the “...chance to share our design ideas to everyone…”.
Factor 4 - Real-World Relevance

The descriptive statistics for factor 4, Real-World Relevance, are presented in
Table 5.
Table 5. Survey responses for the factor of Real-World Relevance
Survey Items Mean (M) STD p
1. “The project increased my understanding
on how to manage and develop an 4.14 0.736 83.0
interactive application.”
2. “I am now able to apply my skills in a
4.01 0.766 79.6
more effective manner on future projects.”
3. “We were able to complete all our tasks on
3.60 0.987 57
time.”
Student Comments (Verbatim):
1. “…visit a real animator who create a movie animation name ‘Upin and
Ipin’ guided me in the future I might want to be part of them.”
2. “…our trip to Les Copaque, fulfilled my dream of being in an actual
animation production company…none of their work would have been
achieved if not for team work and dedication…”
3. “I learnt the importance of B-Roll with random objects…I look forward to
apply the knowledge that i used in this project, later on.”
4. “…I have gain much from the development team…in the future I want to
create my own style of game.”
5. “…we can share our creativity, manage our time that will be used in real
life.”
Results in Table 5 (survey items were numbered based on the mean scores in
descending order) showed that 57% of students reported being able to complete
their tasks on time (Item 3, M = 3.60), indicating that time management was a
skill they had challenges with. However, overall results for Real-World
Relevance were very positive, with 83% of students reporting that the project
improved their understanding of developing and managing a project, and, more
402
importantly, 79.6% of students reported that they had acquired skills that they
can now apply to their future projects (Item 2, M = 4.01). This is also supported
by their comments that “…we can share our creativity, manage our time that will be
used in real life” and that “…I look forward to apply the knowledge that I used in this
project, later on…”. These findings showed that the Real-World Relevance factor
included the planning component, leading students towards lifelong learning
and being more passionate about future development. The second component
was the flexible minds that allowed students to be more adaptive to resources,
restrictions, and requirements in completing the tasks and gaining new
knowledge.
5. Discussion
In answering the research question, “What are the factors for improving
students’ learning experiences in a connectivism learning environment?”, items
from the survey were analyzed using the IBM Statistical Package for the Social
Sciences (SPSS), where the Exploratory Factor Analysis (EFA) was performed.
The result from the exploratory factor analysis of survey response identified four
factors for improving students’ learning experiences in a connectivism learning
environment, Group Dynamic and Experiences (Factor 1), Motivation (Factor 2),
Use of Technologies (Factor 3), and Real-World Relevance (Factor 4). In addition,
discourse analysis was performed on the qualitative data collected (students’
feedback and comments) using the NVivo Software, to provide and interpret the
factor items identified from the EFA.
Factor 1 – Group Dynamic and Experiences

Factor 1, Group Dynamic and Experiences, consisted of items that focused on
the attributes of peer interaction, team working experience, group settings and
collaborative activities. Findings showed that team members were inspired to do
their best in a project. Teamwork influenced the members to contribute from
various aspects, made the group more united and the interactions smoother to
produce a better outcome. Students perceived that the group’s ability to solve
problems and conflicts in a positive manner was associated with well-attended
meetings and in line with findings in the literature on collaborative activities
(Almazroui, 2023; Demosthenous et al., 2020; Murillo-Zamorano et al., 2019).
Students’ responses revealed that working in a project group gave them a sense
of satisfaction, which developed a good impression and mutual understanding
of collaborative activities.
Factor 2 – Motivation
Factor 2, Motivation, consisted of items that focused on the attributes of
students’ motivation and inspiration. Students’ responses, feedback, and online
posts showed that the project settings provided them with various opportunities
to contribute their best efforts in areas they were familiar with and where they
found the project fun and interesting. Motivation can be recognised as a factor
that stimulates students to take charge of their learning process and improve
their capabilities by co-creating content. As supported by the literature, being
self-directed and self-independent was a critical step in transforming a passive
learner into an active learner and a crucial educational innovation in the recent
403
education reform in Malaysia (Azmi et al., 2018; Bakar, 2023; Goh & Abdul-
Wahab, 2020).
Factor 3 – Use of Technologies

Factor 3, Use of Technologies, consisted of items focused on using digital and
web technologies to make constant contacts, progress updates, and enhance
skills. Students’ responses, feedback, and online posts showed that almost all
students were active and capable of using web technologies to acquire the
necessary support and information for their learning. Findings showed that the
reason students enjoyed using web resources to acquire information was due to
the usefulness of the Web in communicating and sharing their work progress
with others and maintaining contact, whereas maintaining contact with the
instructor was associated with improving students’ presentation skills during
the project development. The Use of Technologies factor played a role in shifting
away from the traditional teacher-centred class environment into community
building and collaborative project development with digital technologies. This
research study found that the connectivism learning environment efficiently
encouraged students to exchange ideas, construct social networks, and co-create
content to enrich their learning experiences in both classroom and digital
settings, aligned with the literature that integrating digital technologies and Web
services in the learning context are effective in engaging students (Alexander et
al., 2019, Brown et al., 2020; Moorhouse & Wong, 2022).
Factor 4 – Real-World Relevance

Factor 4, Real-World Relevance, consisted of items that focused on attributes
such as managing project tasks and generalizing the learned skills for future
success. Students’ responses, feedback, and online posts showed that the
multimedia group project increased students’ knowledge and experiences in
managing resources and solving complicated problems. Students also
demonstrated their capabilities and skills in acquiring and connecting the
learning resources, working around the limitations and restrictions, and
factoring in the environmental issues and others’ opinions for developing a
more successful project in the future. The factor of Real-World Relevance has
been identified as an essential element in leading students towards life-long
learning as Siemens’s (2005) believed that contemporary learning is a continual
process that links with various activities and learners move into a variety of
fields at different times of life (Siemens et al., 2020). This factor also helps
students to make sense of their learning in advancing their level of expertise and
critical thinking, consequently increasing their value in today’s job market
(Kukulska-Hulme et al., 2020; Mursid et al., 2022; Surur et al., 2023).
From the findings, a Multimedia-based Connectivism Learning Framework

(MCLF) was developed. Table 5 shows the learning framework, which can guide
educators to develop students’ skills and 21st-century competencies.
404
Table 5. The MCLF framework for connectivism learning experiences
Factors for Improving Siemens’s (2005)

Students’ Learning Connectivism Principles Connectivism Learning
Experiences (CP) Experiences (CE)
Factor 1 - Group CP1 - “Learning and (CE1) Motivated by social

Dynamic and knowledge rest in support and new ideas
Experiences diversity of opinions.” that bring new values for
Factor 2 - Motivation improvement.
Factor 2 - Motivation CP2 - “Learning is a (CE2) Increased
process of connecting resourcefulness in
Factor 3 - Use of specialized nodes or addressing own needs
Technologies information sources.” and remixing
information.
Factor 1 - Group CP3 - “Learning may (CE3) Adapted to the
Dynamic and reside in non-human online environments with
Experiences appliances.” frequent discussions and
Factor 3 - Use of deeper participation.
Technologies
Factor 2 - Motivation CP4 - “Capacity to know (CE4) Developed new

more is more critical than problem-solving
Factor 3 - Use of what is currently approaches through
Technologies known.” multiple tools and
resources.
Factor 1 - Group CP5 - “Nurturing and (CE5) Participated by
Dynamic and maintaining connections implementing new ideas
Experiences is needed to facilitate in the communities for
continual learning.” further exploration.
Factor 4 - Real-World
Relevance
Factor 1 - Group CP6 - “Ability to see (CE6) Applied prior

Dynamic and connections between experiences and aligned
Experiences fields, ideas, and concepts with preferences in
Factor 2 - Motivation is a core skill.” handling tasks and
planning for
enhancements.
Factor 3 - Use of CP7 - “Currency (CE7) Adapted to changes
Technologies (accurate and up-to-date for reflecting real-time
Factor 4 - Real-World knowledge) is the intent updates and coordinating
Relevance of all connectivism pace.
learning.”
Factor 4 - Real-World CP8 - “Decision-making (CE8) Developed
Relevance is a learning process. purposeful solutions in a
Choices and alterations context for producing
affect the decision and better outcomes.
outcomes.”
405
As can be seen from the framework, the factor of Group Dynamic and
Experiences (factor 1) was successful in improving students’ learning
experiences of being motivated by social support and new ideas (CE1), having
deeper participation in online environments (CE3), implementing new ideas in
communities (CE5), and applying their own prior experiences to current tasks
(CE6). Hence, the factor of Group Dynamic and Experiences is consistent with
Siemens’s (2005) Connectivism principles (CP1), (CP3), (CP5), and (CP6). The
factor of Motivation (factor 2) can improve students’ learning experiences by
being motivated by social support and new ideas (CE1), remixing and producing
new information (CE2), developing a new approach to problem-solving (CE4),
and applying their own prior experiences to current tasks (CE6). Hence the
factor of Motivation is consistent with Siemens’s Connectivism principles (CP1),
(CP2), (CP4), and (CP6).
Furthermore, the factor of Use of Technologies (factor 3) was demonstrated to

improve students’ learning experiences of remixing and producing new
information (CE2), having deeper participation in online environments (CE3),
developing a new approach in problem-solving (CE4), and adapting to changes
and coordinating pace (CE7). Hence the factor of Use of Technologies is
consistent with Siemens’s (2005) Connectivism principles (CP2), (CP3), (CP4),
and (CP7). Finally, the factor of Real-World Relevance was successful in
improving students’ learning experiences of implementing new ideas in
communities (CE5), adapting to changes and coordinating pace (CE7), and being
capable of developing purposeful solutions (CE8). Hence the factor of Real-
World Relevance is consistent with Siemens’s Connectivism principles (CP5),
(CP7), and (CP8).
Therefore, this study demonstrates that the four factors for improving students’
learning experiences in a connectivism learning environment have been
successfully identified from the data analysis. Group Dynamics and Experiences
factor emphasized the aspects of unity in the group, work relationships and
functional roles, collective capabilities, and peer interaction in the social
network. This factor raised students’ responsibility and confidence in
contributing to new knowledge, managing learning resources, and
complementing each other’s skills in a learning community. The factor of
Motivation emphasized opportunities in the learning process, the options and
choices in making decisions, the inspiration and learning interests. It motivated
students to devote more effort to improve their performance, be more
accountable for decisions, and determine their goals and values. The Use of
Technologies factor emphasized the aspects of digital fluency in using social
media, Web technologies and multimedia in the learning activities, the external
exposures to open students’ minds, and the new possibilities for future gain.
This factor stimulated students’ sense of creativity and skills in assimilating
prior experiences into new knowledge. The factor of Real-World Relevance
emphasized the aspects of repurposing resources, knowledge and skills transfer,
and developing context-specific content in presenting solutions. It extended
students’ capacities to learn and relearn, the quality of the interactivity in the
connectivism learning environment, and their employability value in today’s job
406
market. Overall, the study showed strong evidence that connectivism learning
experiences were well-supported by the four identified factors.
6. Contributions, Limitations and Future Research

In this research study, transforming the traditional teacher-centred class
environment into a connectivism learning environment was an effective way to
develop university graduates’ competencies and employability attributes. This
learning environment can be supported by the fact that net generation
dominates the student population in university classrooms, and the current
educational trends are redefining the roles of university instructors and
students. Hence, in transforming the learning environment, it is important to
redesign students’ learning experiences by recognizing their prior experiences,
optimizing digital technologies and social media, engaging students in learning
communities, and enhancing their level of information literacy. In particular,
there were several implications and contributions from this study. In particular,
the findings showed that the learning environment was successful in optimizing
the use of technologies in the classroom to support contemporary students’
learning needs. As Siemens’s (2005) Constructivism Learning Principles were
used to engage students in technology-enhanced learning and informal learning,
students moved away from passive to active learners as they became more
capable of using resources and digital technologies for their projects.
Students experienced improved collaborative learning skills. The results

demonstrated that by engaging in teamwork and group activities online during
their project development stages, students became better team players, problem-
solvers, were more interactive with each other, and were more motivated to
complete their projects. It also motivated them to learn more beyond physical
classrooms. In particular, this research study found that project development
efficiently improved students' capabilities and learning experiences. Many
students demonstrated their abilities to reason out the motives behind their
actions, manipulate the ideas collected from other resources, improve from the
mistakes made, and comprehend the issues clearly in group communication. It
also encouraged students to formulate their strategies for collaborative problem-
solving and efficient teamwork. This blended learning approach is successfully
aligned with four shifts outlined in the Malaysia Education Blueprint (Higher
Education) which were: 1) holistic, entrepreneurial and balanced graduates, 2)
the development of a nation of life-long learners, 3) globalized online learning,
4) transformed higher education learning, and effectively addresses the issue of
limited innovative and engaging learning opportunities in Malaysia.
Connectivism learning was found to be an effective pedagogical strategy to

underpin the learning environment. The study's findings showed that
integrating Siemen’s (2005) eight Connectivism learning activities is effective
and can be used to engage students in project-based classes and collaborative
learning in higher education. Implementing Siemens’s (2005) connectivism
learning principles successfully engaged students in critical thinking, decision-
making, and increased their sense of ownership. Through forming networked
relationships with their peers, students recognized and developed their
407
strengths, made connections with existing knowledge, and planned for future
projects. The Multimedia-based Connectivism Learning Framework (MCLF)
presented in this study can be a strongly supported guideline for such
transformation. As many university students live digital lifestyles, the MCLF can
guide educators and students to harness the benefits of digital technological
advances, Internet connectivity, and Web resources in teaching and learning
contexts. The MCLF supports the nation’s call for redesigning learning
environments to prepare Malaysian graduates with problem-solving and
collaborative skills and encourage them to play a role in learning communities,
which are the most transferable and highly sought-after skills in the Malaysian
work environments today.
This research study had some limitations that can be considered in future
research. In this research, the study’s focus was limited to the class environment
of a university multimedia subject titled ‘Graphic Design and Animations’. As
such, the research samples were limited to students who studied at INTI
International University and enrolled in this subject during the period. For
future research, this connectivism learning environment can be designed for
cross-disciplinary programmes or industrial project collaborations involving
students from different universities, disciplines and academic levels. In addition,
this research study allowed students to make connections based on their
capabilities, experiences, preferences, or needs. Hence, students initiated the
interaction, defined the values, and moved towards their goals without
significant interventions or guidance from the instructors. Future research could
investigate the role of the instructors in making the connections and enhancing
the quantity and quality of the built connections in such a learning environment,
as well as their feedback and experiences in designing the connectivism learning
activities. Doing so may provide more insights and research-informed
knowledge on the significant impacts of instructors’ support in connectivism
learning environments.
7. Conclusion
In conclusion, this research study sought to transform and redesign the
traditional teacher-centred class environment into a project-based connectivism
learning environment to help develop university graduates’ competencies and
employability attributes in Malaysia. Therefore, in transforming the learning
environment, it was important to redesign students’ learning experiences by
recognizing their prior experiences, optimizing digital technologies and social
media, engaging students in learning communities, and enhancing their level of
information literacy. By combining project-based learning with Siemens’s (2005)
Connectivism Learning Principles in redesigning the classroom, critical factors
for effective students’ collaborative learning experiences were identified based
on the findings from the Exploratory Factor Analysis (EFA) performed. These
factors were Group Dynamic and Experiences, Motivation, Use of Technologies,
and Real-World Relevance. As a result of this analysis, the Multimedia-based
Connectivism Learning Framework (MCLF) was presented, where each of the
eight connectivism learning experiences was co-supported and improved by one
408
or more of the four factors identified and were able to address the research
issues investigated in this study successfully.
As can be seen in the quantitative and qualitative results, students experienced

deeper collaboration through project-based learning approaches, gained more
enhanced skills and capabilities through problem-solving and critical thinking,
and developed better strategies in utilizing digital technologies and web
resources to support their learning. This provided strong and encouraging
support for redesigning conventional classrooms to become more
technologically and pedagogically supported. Project-based and Connectivism
learning has been shown to enable more meaningful and richer learning contexts
for students, with the MCLF as an effective guideline for technology-based
classrooms in 21st-century education.
8. References
Alexander, B., Ashford-Rowe, K., Barajas-Murph, N., Dobbin, G., Knott, J., McCormack,
M., Pomerantz, J. Seilhamer, R. & Weber, N. Important Developments in
Educational Technology for Higher Education. Horizon report 2019 higher
education edition, 20-32. https://library.educause.edu/ resources/2019/4/ 2019-
horizon-report
Alkhawaja, M. I., Halim, M. S. A. & Afthanorhan, A. (2021). Technology Anxiety and Its
Impact on E-Learning System Actual Use in Jordan Public Universities during
the Coronavirus Disease Pandemic. European Journal of Educational Research, Vol.
10(4), 1639-1647. https://doi.org/10.12973/eu-jer.10.4.1639
Almazroui, K. M. (2023). Project-Based Learning for 21st-Century Skills: An Overview
and Case Study of Moral Education in the UAE. The Social Studies, 114(3), 125-
136. https://doi.org/10.1080/00377996.2022.2134281
Al-Samarraie, H., Shamsuddin, A., & Alzahrani, A. I. (2020). A flipped classroom model
in higher education: a review of the evidence across disciplines. Educational
Technology Research and Development, 68(3), 1017-1051.
https://doi.org/10.1007/s11423-019-09718-8
Andres, J.M.L., Baker, R.S., Siemens, G., Gasevic, D., and Spann, C.A. (2017). Replicating
21 Findings on Student Success in Online Learning. Technology, Instruction,
Cognition, & Learning, 10(4), 313-333. https://learninganalytics.upenn.edu/
ryanbaker/TICLReplicationManuscript.pdf
Azmi, I. A. G., Hashim, R. C., & Yusoff, Y. M. (2018). The employability skills of
Malaysian university students. International Journal of Modern Trends in Social
Sciences, Vol. 1(3), 1-14. http://www.ijmtss.com/PDF/IJMTSS-2018-03-09-01.pdf
Bakar, A. Y. A. (2023). Fulfilling the Aspirations of Malaysian Education Blueprint 2013-
2025: Issues and Challenges. Journal for ReAttach Therapy and Developmental
Diversities, 6(6s), 13-17. https://jrtdd.com/index.php/journal/article/view/672
Brown, M., McCormack, M., Reeves, J., Brook, D. C., Grajek, S., Alexander, B., Bali, M.,
Bulger, S., Dark, S., Engelbert, N., Gannon, K., Gautheir, A., Gibson, D., Gibson,
R., Lundin, B., Veletsianos, G. & Weber, N. 2020 Educause Horizon Report Teaching
and Learning Edition, 13-31. https://library.educause.edu/
resources/2020/3/2020-educause-horizon-report-teaching-and-learning-edition
Bülow, M. W. (2022). Designing synchronous hybrid learning spaces: Challenges and
opportunities. Hybrid learning spaces, 135-163. https://doi.org/10.1007/978-3-
030-88520-5_9
Chandrappa, P. K. (2018). Connectivism as a learning theory for the digital age.
Adhyayan: a journal of management sciences, Vol. 8(01), 37-47.
https://smsjournals.com/index.php/Adhyayan/article/view/2092
409
Cheung, A. (2021). Synchronous online teaching, a blessing or a curse? Insights from EFL
primary students’ interaction during online English lessons. System, 100, 102566.
https://doi.org/10.1016/j.system.2021.102566
Demosthenous, G., Panaoura, A., & Eteokleous, N. (2020). The Use of Collaborative
Assignment in Online Learning Environments: The Case of Higher Education.
International Journal of Technology in Education and Science, 4(2), 108-117.
https://doi.org/10.46328/ijtes.v4i2.43
Downes, S. (2019). Recent work in connectivism. European Journal of Open, Distance and E-
Learning (EURODL), Vol. 22(2), 113-132. https://doi.org/10.2478/ eurodl-2019-
0014
Downes, S. (2022). Connectivism. Asian Journal of Distance Education, 17(1).
http://asianjde.com/ojs/index.php/AsianJDE/article/view/623/368
Dziubaniuk, O., Ivanova-Gongne, M., & Nyholm, M. (2023). Learning and teaching
sustainable business in the digital era: a connectivism theory approach.
International Journal of Educational Technology in Higher Education, 20(1), 1-23.
https://doi.org/10.1186/s41239-023-00390-w.
Field, A. (2013). Discovering statistics using IBM SPSS Statistics (4th ed.). London: Sage.
Fuad, D. R. S. M., Musa, K., & Hashim, Z. (2020). A Perspective to innovation leadership
in Malaysia education. Journal of Educational Research & Indigenous Studies, 1(1), 1-
17. https://tinyurl.com/3ecdebc3
Goh, P. S. C., & Abdul-Wahab, N. (2020). Paradigms to drive higher education 4.0.
International Journal of Learning, Teaching and Educational Research, Vol. 19(1), 159-
171. https://doi.org/10.26803/ijlter.19.1.9
Gomez, I. N. B. (2020). Reflections on the role of occupational therapy programmes on
the mental health of stakeholders’ transition to e-learning during the COVID-19
pandemic. World Federation of Occupational Therapists Bulletin, 1-5.
https://doi.org/10.1080/14473828.2020.1836791
Halili, S. H., Sulaiman, S., Sulaiman, H., & Razak, R. (2021). Embracing industrial
revolution 4.0 in universities. In IOP Conference Series: Materials Science and
Engineering, Vol. 1088(1), 012111. https://doi.org/10.1088/1757-899x/1088/
1/012111.
Koh, J. H. L. (2019). Four pedagogical dimensions for understanding flipped classroom
practices in higher education: A systematic review. Educational Sciences: Theory &
Practice, 19(4), 14-33. https://doi.org/10.12738/estp.2019.4.002
Kukulska-Hulme, A., Beirne, E., Conole, G., Costello, E., Coughlan, T., Ferguson, R.,
FitzGerald, E., Gaved, M., Herodotou, C., Holmes, W., Mac Lochlain, C.,
Mhichil, M. N. G., Rienties, B., Sergeant, J., Scanlon, E, Sharples, M. & Whitlock,
D. (2020). Innovating pedagogy 2020: Open University Innovation Report 8. 1-49.
https://www.learntechlib.org/p/213818/
Markula, A., & Aksela, M. (2022). The key characteristics of project-based learning: how
teachers implement projects in K-12 science education. Disciplinary and
Interdisciplinary Science Education Research, 4(1), 1-17.
https://doi.org/10.1186/s43031-021-00042-x.
Ministry of Education Malaysia (2015), Malaysia Education Blueprint 2015-2025 (Higher
Education), Kementerian Pendidikan Malaysia: Putrajaya.
https://jpt.mohe.gov.my/portal/index.php/en/corporate/policy-
documents/16-malaysia-education-development-plan-2015-2025
Moorhouse, B. L. (2023). Teachers’ digital technology use after a period of online
teaching. ELT Journal, ccac050. https://doi.org/10.1093/elt/ccac050.
Moorhouse, B. L., & Wong, K. M. (2022). The COVID-19 Pandemic as a catalyst for
teacher pedagogical and technological innovation and development: Teachers’
perspectives. Asia Pacific Journal of Education, 42(sup1), 105-120.
https://doi.org/10.1080/02188791.2021.1988511
410
Müller, C. &. Mildenberger, T. (2021). Facilitating flexible learning by replacing

classroom time with an online learning environment: A systematic review of
blended learning in higher education. Educational Research Review 34: 100394.
https://doi.org/10.1016/j.edurev.2021.100394
Murillo-Zamorano, L. R., Sánchez, J. Á. L., & Godoy-Caballero, A. L. (2019). How the
flipped classroom affects knowledge, skills, and engagement in higher
education: Effects on students' satisfaction. Computers & Education, 141, 103608.
https://doi.org/10.1016/j.compedu.2019.103608.
Mursid, R., Saragih, A. H., & Hartono, R. (2022). The Effect of the Blended Project-Based
Learning Model and Creative Thinking Ability on Engineering Students'
Learning Outcomes. International Journal of Education in Mathematics, Science and
Technology, 10(1), 218-235. https://doi.org/10.46328/ijemst.2244.
Pelletier, K., Brown, M., Brooks, D.C., McCormack, M., Reeves, J., Arbino, N., Bozkurt,
A., Crawford, S., Czerniewicz, L., Gibson, R., Linder, K., Mason, J., and
Mondelli, V. (2021). 2021 EDUCAUSE Horizon Report, Teaching and Learning
Edition. 1-50. https://library.educause.edu/resources/2021/4/2021-educause-
horizon-report-teaching-and-learning-edition
Pelletier, K., McCormack, M., Reeves, J., Robert, J., Arbino, N., Maha, A.F, Dickson-
Deane, C., Guevara, C., Koster, L., Sánchez-Mendiola, M.,Bessette, L.S. & Stine, J.
2022 EDUCAUSE Horizon Report Teaching and Learning Edition, 1-58.
https://library.educause.edu/resources/2022/4/2022-educause-horizon-
report- teaching-and-learning-edition
Rahayu, M. K. (2019). Barriers to use e-learning platform in Indonesia Higher Education:
Factors related to people and organization. In International Conference on
Organizational Innovation (ICOI 2019) 475-479. https://doi.org/10.2991/icoi-
19.2019.83
Siemens, G., Rudolph, J., & Tan, S. (2020). “As human beings, we cannot not learn”. An
interview with Professor George Siemens on connectivism, MOOCs and
learning analytics. Journal of Applied Learning and Teaching, Vol. 3(1), 108-119.
https://doi.org/10.37074/jalt.2020.3.1.15
Siemens, G. (2005). Connectivism: A learning theory for the digital age. International
Journal of Instructional Technology & Distance Learning.
http://www.itdl.org/Journal/Jan_05/article01.htm.
Surur, M., Nurtjahyani, S. D., Agusti, A., & Yana, I. (2023). The Effect of Project Based
Learning on Digital Literacy Skills and Conceptual Understanding in an Online-
Based Flipped Classroom Environment. Edumaspul: Jurnal Pendidikan, 7(1), 849-
856. https://doi.org/10.36681/tused.2022.124
Tsybulsky, D., & Muchnik-Rozanov, Y. (2023). The contribution of a project-based
learning course, designed as a pedagogy of practice, to the development of
preservice teachers' professional identity. Teaching and Teacher Education, 124,
104020. https://doi.org/10.1016/j.tate.2023.104020.
Williams, B., Onsman, A., & Brown, T. (2010). Exploratory factor analysis: A five-step
guide for novices. Australasian Journal of Paramedicine, 8, 1-13.
https://doi.org/10.33151/ajp.8.3.93
411

Vol. 22, No. 7, pp. 411-433, July 2023
Innovation in Early Reading Instruction: The

Development of e-Learning Materials in Mother
Tongue
J-Roel B. Semilla
Department of Professional Education, College of Education,
Mindanao State University-Iligan Institute of Technology, Philippines
Venus R. Parmisana
Department of English, College of Arts and Social Sciences,
Loreta L. Fajardo
Department of English, College of Arts and Social Sciences,
Ruben L. Abucayon
Department of Professional Education, College of Education,
Angeline P. Dinoro
Department of Professional Education, College of Education, Mindanao State
University-Iligan Institute of Technology, Philippines
Josefina M. Tabudlong
Department of Sociology, College of Arts and Social Sciences,
Abstract. Mother Tongue-Based-Multilingual Education has been shown

to improve the academic performance of early-grade learners. However,
challenges such as inaccessibility and lack of contextualized materials in
the mother tongue (MT) have hindered students' full engagement in
learning. To address this, a quasi-experimental study was conducted
with struggling first-grade learners (n=60) to investigate the effectiveness
of researcher-developed e-learning materials for early reading in MT.
Instruments used in this study included a researcher-created storyboard
for video creation, with its dialogue and storyline; the test
questionnaires, adapted from the MT-based Curriculum Guide of the
Department of Education in the Philippines, for measuring participants’
©Authors
412
reading performance in phonic and word recognition. The pre-test and

post-test data were analyzed using mean, standard deviation, and a t-test
at a 5% significance level. Qualitative data extracted through interviews
were analyzed thematically, while a four-point Likert scale was used to
interpret the overall rating of the e-learning materials. The content and
ICT experts measured the e-learning materials’ validity and
effectiveness. The results illustrated e-learning materials like video
stories and PowerPoint presentations, developed using the ADDIE
(Analysis, Design, Development, Implementation, and Evaluation)
model, improved learners' letter-sound and word-blending abilities.
Experts rated the materials excellent for technical standards, content
quality, and effectiveness. The e-learning materials as a pedagogical
intervention in early reading are innovative, interactive, and engaging
and can potentially enhance early graders' reading performance. Future
research can explore design or layout modification of the materials
creatively to enhance learner engagement in early-grade MT instruction.
Keywords: Mother Tongue-Based-Multilingual Education instruction;

e-learning materials; early grade learners; reading performance
1. Introduction
Mother tongue (MT) education is a well-established practice in Asian countries
and has been found to have a positive impact on the academic performance and
self-esteem of grade school learners. In 2012, the Philippines adopted the Mother
Tongue-Based Multilingual Education (MTB-MLE) program under the K-12 Basic
Education Program. The program is aimed to use the learner's first language as
the primary medium of instruction in the early years of education to improve
learning outcomes and preserve cultural diversity. The Department of Education
(DepEd) has confirmed that language is vital and integral to learning. The MTB-
MLE program is integrated into the primary education curriculum, starting from
kindergarten up to grade three level, to prioritize the acquisition of the native
languages present in different localities (Dagalea, Peralta, & Abocejo, 2022). This
paper examines the impact of the MTB-MLE program in the Philippines at the
grade one level and investigates the effectiveness of e-learning materials for early
reading in MT.
Studies suggest that mother tongue-based (MTB) instruction can lead to higher
academic achievement because children learn a familiar language (Tofaris &
Thornton, 2018; UNESCO Bangkok, 2008;). Although no conclusive evidence links
poor education quality to a lack of MT instruction, research indicates that MT
instruction can improve education quality (UNESCO, 2008) and that MTB
prepares graduates for lifelong learning and career development (Apolonio,
2022). Williams et al. (2014) argued that teaching in children's MT lays the
foundation for developing literacy skills and learning additional languages such
as Filipino and English. Several local studies have examined the impact of MT-
based multilingual education on student performance. Ricablanca (2014) found
positive effects on students’ academic performance, while Namanya (2017) found
that MT instruction could improve English proficiency. These findings suggest
that MT instruction can positively impact academic achievement and language
413
learning outcomes. Children’s potential is often wasted without MT, resulting in

educational failure and a lack of development (Mackenzie & Walker, 2015).
Although mother tongue-based education has positive educational impacts, its

implementation faces challenges regarding the accessibility of instructional
materials and teacher training (Eslit, 2017; Lartec et al., 2014;). Insufficient
localized and context-based materials can hinder students' engagement with their
lessons, despite the benefits of using MT. Funding for MTB-MLE training and
material development is scarce, and teachers may have to bear the costs (Eslit,
2017).
Mackenzie and Walker (2015) agree that the challenges of implementing effective
bilingual education for linguistic minorities stem from a shortage of reading and
instructional materials in their local or first language (L1). The availability of
appropriate instructional materials is often identified as a constraint in
developing a localized curriculum (Lone & Efstratopoulou, 2022). This problem
was also underscored in other countries such as Papua New Guinea, India, and
Africa, resulting in low retention rates, poor reading and writing skills, and a
negative impact on teaching practice and literacy development.
Another alarming concern within the MTB-MLE implementation is informed by

the Philippines National Achievement Test (NAT) result in 2012, showing the
overall national performance of Grade Three learners at 56.98%, with English and
reading comprehension at 54.42%. Poor reading performance, particularly in the
early years, can lead to difficulty acquiring advanced skills, struggles in other
subjects, dropout, and poor test results ( Cimmiyotti, 2013; Mwoma, 2017).
The use of ICT in teaching reading has gained popularity in L2 situations. Studies
have shown that teaching reading using ICT, such as e-learning websites and
WebQuest-based, can activate students' prior knowledge, improve their reading
skills and attitudes, and integrate reading with other language skills (Adi Ana &
Nitiasih, 2013; Tuan, 2011). In addition, using ICT tools such as Internet
applications, video technology, and various computer attachments and software
programs has resulted in many positive changes in the educational landscape
(Reid, 2002). Nair and Hindle (2013) show that an ICT-enabled education plays a
catalyst role in promoting inclusive growth and human development for all.
While ICT is increasingly being incorporated into teaching methods (Reid, 2002),
there are limited studies that explore its impact on reading instruction in MT
education in the Philippines. This study aims to address this gap by investigating
the effectiveness of using ICT or e-learning materials for teaching reading in MTB.
1.1 Study’s Objectives and Significance

This study aimed to determine the influence of the e-learning materials on
teaching reading in the MT among Grade One pupils, and in particular, aimed in
particular, at the following:
(1) Determine the pretest scores of the control and experimental groups before
intervention;
414
(2) Establish the salient features of the e-learning materials according to Analysis,
Design, Development, Implementation, and Evaluation by the teachers and the
learners;
(3) Determine the post-test scores of the control and experimental groups after the
intervention;
(4) Verify significant difference between the pre-test scores and post-test scores
when grouped before intervention and after intervention; and
(5) Establish the impact of using e-learning materials on Grade 1 learners’
minimum learning competency in reading.
The null hypothesis proposes no significant difference in pretest scores between

the control group, taught using traditional methods, and the experimental group,
taught using e-learning materials. The alternative hypothesis, on the other hand,
suggests a significant difference in post-test scores between the two groups after
the intervention. In testing these hypotheses, insights can be gained on the
effectiveness of e-learning materials in teaching reading in MT among grade one
pupils.
This study is significant for various stakeholders, emphasizing the importance of

developing reading skills in the L1 and the benefits of ICT integration through E-
learning materials in MTB-MLE. It aims to address the scarcity of instructional
materials and increase reading performance among first-grade Sinugbuanong
Bisaya [Sebuano] speakers. The study's focus on e-learning materials provides a
new avenue of research for investigating transferable reading skills from MT to
English and other critical issues in the MTB-MLE program.
2.1 Theoretical Framework
This study was anchored in three theories. The first is Bruner's Social Interaction
theory (Bruner, 1982; Smidt, 2013) which highlights the importance of using MT
as a language of instruction, a theory relevant to the current study. Children
acquire their MT through conversation and interaction with adults. This sheds
light on the challenges learners face when the language of instruction in schools,
usually English, differs from their home language.
The second theory describes Edgar Dale’s Cone of Experience, which supports
technology use in classroom instruction through audiovisual materials that
provide vivid and memorable experiences for learners (Schifter, 2016).
Dale's theory has been widely used in education in our understanding of
learning and why student learning occurs as an application of the Cone
of Experience. (Davis & Summers, 2015). It has been used to design
effective learning experiences and serves as a useful framework for
curriculum development, instructional materials design, and selecting teaching
methods that promote active and experiential learning.
Unfortunately, the prevailing dilemma in implementing MTB-MLE is the scarcity

of learning resources, not only books but also ICT-based materials (Cabansag,
2016; Cansino, 2023). This is brought about by the limited funding given to
415
schools. Hence, this concept stresses the need to develop MT-based learning and
reading materials. Jeanne Chall's Theory of Reading Development (1983, as cited
in Resnick & Weaver, 2013), the third theory, presents a reading model that
distinguishes between the quality of reading, the complexity of the books that can
be read, and the applications of reading at various stages of development. Chall
identifies six stages: prereading; initial reading or decoding; confirmation,
fluency, and ungluing from print; reading for learning the new; multiple
viewpoints; and constructing and reconstructing a worldview.
The grade one pupils are at the initial reading, writing, and decoding stage (Stage
1), where children aged 6 to 7 learn the relationship between letters and sounds
and how print corresponds to spoken words. Children at this stage can read
simple texts containing high-frequency and phonically regular words and use
decoding skills to read unfamiliar words. Their writing skills progress from
scribbling to controlled scribbling and non-phonetic letter strings, and they are
encouraged to write about familiar words and use invented spellings to promote
beginning writing. (The Literacy Bug, 2018; Westberg, McShane, & Smith, 2006;).
The fourth is based on Mishra and Koehler's (2006, as cited in Mishra, 2019)
Technological Pedagogical Content Knowledge (TPACK) Model, a framework for
integrating technology in education and structuring classrooms for optimal
learning experiences. The model guided the use of e-learning materials in
teaching MT reading through a storytelling strategy in presenting the lesson about
Kaka, which was aligned with the Pedagogical Content Knowledge. Integrating a
video story to present the topic of Kaka reflects Technological Content Knowledge
while combining the video story as technology and the storytelling as strategy
shows Technological Pedagogical Knowledge.
2.2 Mother Tongue-based – Multilingual Education

As defined by the Department of Education (DepEd) in the Philippines, MTB-
MLE is an educational approach that uses the learner's MT and additional
languages in the classroom, resulting in stronger literacy abilities and knowledge
transfer. Through Department Order 74. S, 2009, Mother Tongue-Based
Multilingual Education (MTB-MLE) was institutionalized and adopted in 2012
(Cansino, 2023) and has been implemented in two modules: subject area and
medium of instruction (MOI). As a subject, it focuses on developing listening,
speaking, reading, and writing from Grades 1 to 3 in MT. As an MOI, MT is used
in all learning areas from kindergarten to Grade 3, except in teaching Filipino and
English subjects. This approach promotes inclusion in education and improves
the quality of education by building on learners' and teachers' knowledge and
experience (DepEd, 2018).
According to UNESCO, MT instruction in the early years is essential to quality

education as it allows young learners to express themselves freely, articulate their
thoughts, and add new concepts to their existing knowledge (MacKenzie, 2009).
Using MT positively affects cognitive development and other academic areas, and
pupils who have learned to read and write in their first language acquire
competencies more quickly (DepEd Order No. 74, s. 2009). Furthermore,
appropriate language in education enables teachers to instruct on the language a
416
child speaks most at home and understands well enough to learn academic
content through MT, thereby promoting effective learning (Ricablanca, 2014) and
while MT instruction promotes inclusivity and improves the quality of education.
2.3 ICT Use: Strategy to Improve Reading Performance

Technology has become integral to teaching and learning, integrating ICT tools
into the curriculum and impacting student learning (Reid, 2002). Bolstad (2004)
identified various ICT resources such as digital cameras, computer hardware, and
the Internet. Using ICT as a dynamic learning method provides more benefits than
traditional blackboard and chalk learning (Kaur, 2023). When teachers are
digitally literate, using ICT tools for communication, creation, dissemination, and
information storage can positively impact student learning. Various educational
tools, including interactive exercises and games, can promote student learning
motivation (Gee, 2003; Mayer, 2011), while animations or cartoons can improve
students’ reading fluency (Anestin, 2015). Storytelling or video story is an
effective strategy for enhancing communication and reading skills (Miller &
Pennycuff, 2008). Children are drawn to cartoons due to their captivating visuals,
characters, and educational content (Baran & Davis, 2009; Habib & Soliman, 2015).
Using technology in reading instruction is crucial for all students, as it can
increase motivation and improve learning outcomes, particularly in the digital
age.
3. Methodology
3.1 Design
The study utilized a quasi-experimental design (QED), incorporating a matching-
only pretest-post-test control group design. Quasi-experimental research designs,
as the name suggests, use nonexperimental (or non-researcher-induced) variation
in the main independent variable of interest, essentially mimicking experimental
conditions in which some subjects are exposed to treatment, and others are not
(Gopalan, Rosinger, & Ahn, 2020). Matching participants in the control and
experimental groups based on their reading performance was to minimize
extraneous variables and ensure internal reliability. The experimental group
received e-learning materials, while the control group received traditional
teaching methods.
3.2 Locale and the Participants

The study was conducted in a public elementary school in Northern Mindanao
that implements the MTB-MLE program, with the school’s computer room and
smartboard. The school was selected based on its reputation as a top-performing
school in the area based on the early grade reading assessment (EGRA) results,
with high-performing grade one learners but also non-readers and frustrated
readers.
Through purposive sampling, the study selected 60 first-grade non-reader or

struggling learners (two sections with 30 each). The teachers observed that the
learners had a limited understanding of English, their L2, compared to students
in other classes who were more advanced in literacy skills which could be due to
417
exposure to a literacy-rich environment at home. Selecting participants was based

on pretest scores to ensure similar characteristics between the control and
experimental groups. The study focused on first graders as this is when formal
reading instruction begins, and the MTB-MLE curriculum introduces MT as a
subject (Kalb & van Ours, 2013; Leahyl & Fitzpatrick, 2017; The Literacy Bug
2018;). Many of the chosen pupils struggled with following instructions and
expressing themselves in English, making the use of MT as a medium of
instruction (MOI) necessary.
3.3. Instruments
This study used several instruments, such as computers, and a smart board, as
ICT tools used by the researcher’s video story, and descriptions of the dialogue
and storyline. Another instrument was the pretest and post-test questionnaires
adapted from Learning Material for Grade 1 MTB-MLE to measure participants’
reading performance in phonic and word recognition and determine the
effectiveness of the e-learning materials. A rubric adapted from Paloma (2012)
was used to measure the validity and effectiveness of the e-learning materials in
terms of content and technical aspects. Interview questions were also used to
assess learners’ perceptions of the e-learning materials.
3.4. Procedure and Data Gathering

The ADDIE Model, a popular instructional design framework used in education
and training (Rizal, Rusdiana, Setiawan, & Siahaan, 2021), provided the
framework for the data gathering. ADDIE stands for Analysis, Design,
Development, Implementation, and Evaluation.
Phase 1: Analysis. This stage involved needs analysis, target audience analysis, and
topic and content analysis. This study was motivated by teachers’ concerns about
the lack of appropriate reading and learning materials in mother tongue-based
education which can contribute to educational failure and high dropout rates,
particularly among early-grade learners (Ricablanca, 2014). Primary-level
teachers selected sections where most learners were non-readers or frustrated
readers, using the Early Grade Reading Assessment (EGRA). A pre-test was
conducted for word recognition skills based on the letter and sound of Kk and the
blending of letters to form syllables and words. The sound /k/ is unique and can
pose challenges for young readers regarding tongue placement (The Center for
Speech and Language Development (2018).
Phase 2: Design. The study’s learning competencies (noting details in grade-level

narrative texts and blending letters to form words) were based on the Grade One
MT Curriculum (2016) and Teacher’s Guide. The topic and content focused on the
letter and sound of /Kk/, identified as difficult to recognize due to its low
frequency of use in MT conversations (Imperial, Ya-On, & Ureta, 2019).
Phase 3: Development. Materials were created using Adobe Photoshop, Flash, and
Sony Vegas for image creation, animation, and video production, with the
storyboard utilizing graphics related to the story of Kaka. Then a PPT presentation
was made using Microsoft programs, and both materials were presented on
consecutive days.
418
Phase 4: Implementation. Offline delivery of the e-learning materials to grade one

learners was facilitated using a laptop, projector, speaker, and white screen. The
first material included a video story, differentiated activities, and a question-and-
answer session. The material incorporated a modelling activity, oral and group
reading, and letter-blending exercises.
Phase 5: Evaluation. Grade One learners evaluated the video story through
interviews conducted by the teacher and underwent post-test.
The pretest was administered individually to selected learners in a computer
laboratory setting, and consisted of two parts with five items each. The first part
had photos of animals and objects beginning with the letter k, and below each
photo was a word with a missing initial letter. The second part had photos with
missing initial syllables. The teacher asked “Unsa ang una nga silaba sa pulong nga
kahoy? [What is the beginning syllable for the word Kahoy (tree)?]”. Pupils were
asked to choose the correct syllable from three options and write it to complete
the photo’s name. The post-test was administered to check learners’ word
recognition skills, such as identifying the letter-sound correspondence of the letter
/Kk/ and the beginning syllable ka.
3.5. Coding & Statistical Treatment

ICT and content experts evaluated the MT e-learning materials based on the
technical aspects and the contents, respectively. Pre-test and post-test data were
analyzed using mean, standard deviation, and t-test at a 5% significance level. A
t-test is a statistical test used to determine if there is a significant difference
between the means of two groups. It is commonly used when comparing the
means of a continuous variable (e.g., test scores, comprehension test). A paired t-
test or dependent t-test was used to determine the pre-and post-test scores of the
two groups (controlled and experimental) (Magwilang, 2016). Qualitative data
were analyzed thematically with confidentiality through interviews and
observations. Moreover, a four-point Likert scale assessed the overall raters'
satisfaction with the e-learning materials. The scale was arranged in sequential
order as follows: 4.00-3.25 (Excellent), 3.24-2.50 (Very Satisfactory), 2.49-1.75
(Satisfactory), and 1.74-1.00 (Needs Improvement), with an interval of 0.74 or 0.75,
indicating a consistent interval size between the response options. The ICT
experts’ ratings were based on four items: content accuracy, information
sequence, spelling and grammar, and effectiveness. Meanwhile, the content
experts assessed the e-learning materials based on three items: objective and
content, design and layout, and strategies.
4. Results and Discussions

4.1 Pretest Scores of the Control and Experimental Groups Before Intervention
The pretest was administered individually to selected non-reader grade one
learners who had to name the picture and write the missing initial letter. In
addition, they chose the correct syllable from three options and wrote it to
complete the name shown in the photo. Scores are revealed below.
419
Table 1. Pretest Mean Scores of the Control and Experimental Groups before
Intervention
Compared Groups N Minimum Maximum SD

pretest control 30 3 10 7.90 1.918
pretest experimental 30 4 10 7.97 1.938
The table indicates similar pretest mean scores for both groups, with most pupils
achieving a passing score. The standard deviation for both groups indicates a very
slight or negligible deviation in scores. As observed during pretest
administration, some grade one pupils struggled with word recognition during
the pre-test, particularly with the grapheme Kk and blending letters to form
syllables and words, often guessing or writing inappropriate letters for target
words with missing beginning letters or sounds. For example, for the target word
kandila [candle], the pupil wrote /b/ for the missing initial letter /k/, making the
word bandila [flag] instead of kandila. In the second part of the pretest, the target
word kahoy [tree] has a missing initial syllable, /ka/, and learners resorted to
guessing from the three choices (la, ka, sa), making the word sahoy instead of
kahoy. According to international researchers, phonics, a widely acknowledged
useful strategy for early literacy, could potentially address these difficulties (Ehri,
2020; Yap & Chin, 2020).
4.2 Salient Features of the e-Learning Materials

4.2.1 Analysis Stage
The e-learning materials focused on the challenging concept of the letter and
sound of /Kk/ for early readers, identified as a difficulty for non-readers and
frustrated readers in the study (Imperial et al., 2019)The sound /k/ is produced
by releasing air from the mouth and can be challenging for early-grade learners
because they may have difficulty placing their tongues correctly at the back of
their mouths. The pretest revealed that despite achieving a passing score overall,
many pupils struggled with the letter-sound association and blending letters to
form syllables and words.
4.2.2. Design Stage

E-learning materials were developed based on K-12 Curriculum and Teacher’s
Guide. Learning objectives were determined based on the Grade One MT
Curriculum Guide (2016).
Table 2. Mapping of Learning Competencies to the Learning Objectives

Domain Learning Competencies
Listening Noting important details in grade-level narrative text,
Comprehension listened to
Phonics and Word Blending specific letters to form syllables and words
Recognition Domain
Learning Objectives 1. Note details in a story listened to
from Teacher’s Guide 2. Confirm predictions after listening to a story
3. Respond to the story through the different engagement
activities
a. Act out a portion of the story;
b. Draw the spider;
420
c. Make a character list; and

d. Tell the ending of the story
4. Identify specific letters in the alphabet, both upper and
lower case of the letter /Kk/
5. Pronounce the words that start with the letter Kk clearly
6. Show the relationship between sounds and written
symbols
Designed to be interactive and engaging, the grade one e-learning materials were
created through video production and storytelling about Kaka [spider]. Learning
competencies included noting story details, confirming predictions, and
responding to the story. The second topic focused on the letter /Kk/, with
learning competencies including identifying upper- and lower-case letters,
pronouncing words starting with /Kk/, and understanding the relationship
between sounds and written symbols. Lessons followed an inductive sequence,
beginning with a video story about Kaka [spider] and oral comprehension
questions that learners answered. The PPT presentation started with the letter
sound, and then pupils blended letters to create syllables and produce each sound,
combining the sounds together to form a word. The delivery strategy involved
projected media such as a laptop, projector, speaker, and white screen. For the
evaluation strategy, the materials used a pencil and paper test.
4.2.3. Development Stage

At the development stage, e-learning materials were created, including a video
story, PPT presentation, and lesson guide for class activity. The learning objectives
of the materials were aligned with the learning competencies taken from the MT
Curriculum Guide for Grade One (2016).
A 2-minute video story about Kaka was created, following researchers’ guides to
video length which considered children having a short attention span (Brame,
2015; Brame & Perez, 2016). Using various software, the researcher created a video
containing localized, colorful images with clear audio and subtitles. An animator
sketched the images using Adobe Photoshop and animated them using Adobe
Flash, with audio recorded by a radio broadcaster. The cartoon-like characters
were designed to positively influence children’s learning and socializing skills
(Habib & Soliman, 2015) as children learn from cartoons and often act out what
they have learned, influencing their socializing skills (Baran & Davis, 2009).
The researchers created a PPT presentation for the letter and sound of Kk, with
word-breaking and word-making activities for early readers, and using big
graphics and a conventional font style. The video story (Figures 1 & 2) and PPT
presentation were learner-centered and incorporated ICT-based features.
421
Figures 1 & 2. Screenshots of the e-material
4.2.4 Implementation
On the first day of the lesson guide’s implementation, the teacher started by
unlocking a difficult word in the video story through picture analysis. The
students were asked to identify the animal in the picture, and they answered with
alilawa and damang (MT words for spider). The teacher then asked if they knew
another animal’s name in Binisaya [Sebuano], but when nobody answered, the
teacher said, “Kini nga mananap atoang ginatawag usab nga Kaka. [This is also called
Kaka!]”. The class repeated the name of the animal twice. The teacher asked them
to describe the animal in the picture and set the intention of why the pupils should
watch the video by posing, “Unsa kaha ang nahitabo sa mga kaka? [What happened
to the spiders?]”. After the unlocking activity, the video story (e-learning material
1) was presented to the class twice. As the video started playing, the class was
422
captivated and exclaimed, “Wow!” The e-learning material caught their interest,
and when asked if they wanted to watch it again, they unanimously answered,
“Yes, Sir!”
On the second day, the class reviewed the previous lesson and was introduced to
a PPT presentation to develop the learners’ word recognition skills on blending
sounds with word-breaking and word-making activities and with large-sized
images and a familiar font style. The next day, the class practiced writing and
tracing the letter /Kk/ and connecting syllables to form words. The final day
involved drill reading, identifying pictures beginning with the letter K, arranging
words to form phrases, and practicing reading whole sentences. The activities
were designed to enhance the learners’ reading skills and improve their ability to
form words and phrases.
The teacher presented the video offline using a laptop, projector, speaker, and
white screen, facilitated comprehension questions, and grouped the students for
differentiated instruction with various tasks such as acting, drawing, making
character lists, and predicting the story’s end. The post-activity involved
discussing the importance of taking care of animals and the environment, and the
students demonstrated their understanding by noting important details and
learning important values such as caring for animals and the environment.
4.2.5 Evaluation
4.2.5.1. ICT and Content Experts’ Ratings
Two teams of experts, ICT and Content, evaluated the materials using a rubric
adapted from Paloma (2012), shown in Table 3.
Table 3. Ratings of the ICT Experts and Content Experts

Mean
ICT Experts Content Experts
Category MT1 MT2 Category MT1 MT2
I. Content-Accuracy 4.00 4.00 I. Objective & 4.00 4.00
Content
II.Information 4.00 4.00 II. Design & 3.60 4.00
Sequence Layout
III. Spelling and 4.00 4.00 III. Strategies 3.58 3.92
Grammar
IV. Effectiveness 4.00 4.00
Overall Mean 4.00 4.00 Overall Mean 3.73 3.97
Description Excellent Excellent Description Excellent Excellent
The ICT experts rated materials 1 and 2 with an average rating of 4.00, indicating
excellent technical quality. The materials were accurate and logically organized,
free of spelling and grammar errors, and effectively incorporated necessary
elements such as text, images, sound, and animation to enable a sound
understanding of the subject matter. The Content experts rated the e-learning
423
materials with high ratings of 3.73 and 3.97, indicating excellent quality and
suitability for grade one learners.
The materials effectively met the learning objectives of the Basic Education
Curriculum, with engaging activities that connect to the real world. The content
was clear, concise, and error-free, while the multimedia presentation was well-
structured and easy to follow, utilizing various elements such as text and images
with sound and animation to enhance learning. The materials are well-designed
and organized, with effective, creative, engaging lesson strategies that capture
student interest and foster a participative learning environment. Overall, the
evaluation of the e-learning materials demonstrated their excellent quality,
highlighting their potential to support student learning and promote the
development of critical thinking skills.
4.2.5.2. Learners’ Perceptions

The grade one pupils had a positive and engaging experience with e-learning
material 1, which included a video story featuring children playing in a barrio or
community. They were attentive and could relate to the situation, learning
valuable lessons about animal and environmental care while enjoying the content
in their MT. The PowerPoint (PPT) presentation with colorful pictures facilitated
their active participation and learning. The e-learning material effectively
engaged and taught the students through relatable content, visual aids, and the
use of their MT.
After the 4-day lesson, a focus group discussion (FGD) was conducted, and the
learners recalled the video story they watched about Kaka and elaborated that the
animal is also known as mga lawa, alilawa, or spider, showing that the materials
increase learners’ understanding of the lesson. One of the learners shared that a
spider: “kay nagpuyo sa mga kahoy [lives in trees].” Another also said that the
animal “kay mudako og makita sa supot [grows big and can be seen from its web].”
Such statements illustrate that children could connect their prior knowledge to
the lesson. Moreover, they also mentioned that they learned many things that start
with the letter Kk and the sound /k/, for example, “Kris, Karla, bakaw, kanding,
kalabasa, kutsilyo” among others. This demonstrates that the e-learning materials
used in the classroom help improve learners’ word recognition skills.
During the focus group discussion (FGD), three main themes emerged. The first
is related to the lesson sequence in the e-learning materials. Participants found the
sequence of events in the video clear and easy to follow. Moreover, they could
accurately recount story details in order, indicating that the materials effectively
presented the sequence. The second theme that emerged is the content of the e-
learning materials. Participants displayed a good recall of the video story’s details
and improved their word recognition skills by identifying target letter words. The
third theme was related to the effectiveness of the e-learning materials. The
participants gave positive feedback, finding the colorful images and multimedia
elements engaging, and describing the materials as nindot [good] or ayos [nice]. It
was found that the materials effectively captured their interest and attention,
leading to a positive learning experience.
424
4.3 Results of the Post-test after Intervention

After using e-learning materials as an intervention tool, the participants
completed a post-test, and the results are shown below.
Table 4. Post-test Mean Scores of the Control and Experimental Groups after
Intervention
Compared Groups N Minimum Maximum SD
Post-test controlled 30 7 10 9.53 1.008
Post-test experimental 30 6 10 9.50 0.974
The table shows a small difference in the mean scores of both groups after the
intervention The study confirmed the null hypothesis showing no significant
difference in the reading proficiency of both control and experimental groups.
However, the experimental group’s posttest mean was higher than the control
group, indicating an increased performance in word recognition (such as
identifying the letter-sound correspondence of the letter /Kk/), listening, and
comprehension skills using mother tongue-based e-learning materials in teaching.
The learners could identify the initial sound /k/ in the word kanding [goat] and
complete the missing beginning letter. Additionally, they confidently selected the
missing beginning syllable “ka” from the given options (la, ka, sa) to complete the
word kahoy [tree]. Most pupils in both groups achieved scores above the passing
rate of 6, demonstrating improved reading skills. This illustrates the potential of
using MTB e-learning materials in enhancing early reading skills of grade one
learners to improve their letter-sound correspondence and syllable and word
blending abilities, although some initial difficulties were observed with word
recognition.
4.4 Test Scores

The table below shows the control and experimental groups' pretest and post-test
mean scores before intervention. Although the experimental group showed a
higher mean than the control group in the pre-test, still the result shows no
significant difference. With a computed p-value of 0.894, which is greater than the
significance level of 0.05, this means that there is not enough evidence to reject the
null hypothesis, indicating that both groups had similar pre-test performances. In
the post-test, however, using e-learning materials improved the reading
performance of the experimental group, with a mean score of 9.50 compared to
the pretest’s mean score of 7.97. Furthermore, based on the p-values (0.134 for the
pre-test and 0.130 for the post-test), the observed differences between the groups
are not statistically significant at the commonly used significance level of 0.05.
Results show that since the p-values are greater than 0.05, it suggests that the
observed differences between the control and experimental groups during both
the pre-test and post-test are likely due to chance.
The study found that e-learning materials were as effective as traditional teaching
methods in teaching beginning reading skills. This aligns with the MT Curriculum
Guide (Medilo, 2016) for Grade 1 and Chall’s (1983) Theory of Reading
Development, emphasizing the importance of phonological awareness, letter-
sound knowledge, and the ability for phoneme and syllable manipulation.
425
Table 5. Differences in Test Scores

Compared Groups Sd t-value p-value Remark
Control 7.90 1.918 Not
Pre-test -0.134 0.894
Experimental 7.97 1.938 Significant
Control 9.53 1.008 Not

Post-test 0.130 0.897
Experimental 9.50 0.974 Significant
*p> .05
Our study examined integrating e-learning materials in teaching early reading

skills in the Mother Tongue. Learners participating in the intervention reported
several benefits, including improved word recognition and listening
comprehension skills. Incorporating colorful pictures, moving images, and
talking comic characters within the e-learning materials created an engaging
learning environment. While our results did not show a statistically significant
difference between the e-learning intervention and the traditional method, it is
important to consider the practical significance and potential advantages of e-
learning.
The use of educational technology in the classroom has been shown to increase
student engagement and enthusiasm, as supported by previous research (Gee,
2003; Mayer, 2011) and aligned with Dale’s Cone of Experience. By leveraging
interactive exercises and games, educational technologies offer opportunities for
enhanced learning experiences and increased motivation among early-grade
learners (Alakrash & Razak, 2020). Learners are highly motivated as easy access
to technologies provides them with more learner-centered activities and authentic
learning materials (Brunnet & Portugal, 2016). The students reported a better
learning environment, peer interaction, and participation in learning compared to
the traditional classroom. In our study, the e-learning intervention provided such
interactive elements, contributing to a more engaging and learner-centered
approach.
It is worth noting that individual learner preferences and needs play a significant
role in educational outcomes. While the overall results did not demonstrate a
significant difference, it is possible that certain learners may have benefited more
from the e-learning approach due to factors such as their learning style or prior
experience with technology. This highlights the importance of considering
individual differences and tailoring instructional methods to meet diverse learner
needs.
The findings of this study address the scarcity of learning materials in MT-based
education by introducing new instructional methods that provide learners with
rich learning experiences. These methods emphasize the potential of audio-visual
materials to offer vivid and memorable experiences and promote motivation for
learning (Gee, 2004; West, 2017). In teaching reading, e-learning materials have
been found to activate prior knowledge, improve comprehension, integrate
426
reading skills with other language skills, and foster positive attitudes toward ICT-
based instruction (Adi Ana & Nitiasih, 2013). Integrating technology as a
fundamental part of reading instruction is crucial, especially for students growing
up in the digital age (Capodieci et al., 2004). These findings are consistent with
previous research by Brown and Augustine (2001), Flecknoe (2010), and Molster
(2016), which highlight the innovative pedagogical methods that technology can
offer in enhancing learners’ reading skills.
While technology can be a valuable tool in education, it should not be viewed as

a substitute for effective lesson design and teaching methods. Ertmer and
Ottenbreit-Leftwich (2010) argue that educators should clearly understand how
technology can support learning in relation to curriculum goals, student learning
outcomes, and teaching philosophies. According to Roblyer and Doering (2007),
technology can enhance communication between teachers and learners but cannot
compensate for inadequate teaching practices. Therefore, technology should
enhance, rather than replace, sound pedagogical approaches.
5. Conclusion
The study’s aim was to determine the influence of mother tongue-based (MTB) e-
learning materials on teaching reading among grade one learners. Although the
study confirmed the null hypothesis, indicating no significant difference in
reading proficiency between the control and experimental groups, several
noteworthy findings emerged. First, the experimental group’s post-test mean was
higher than the control group's, suggesting improved performance in word
recognition, listening, and comprehension skills when utilizing MTB e-learning
materials. This highlights the materials’ potential to enhance early reading skills,
particularly in areas such as letter-sound correspondence and syllable and word
blending abilities. However, some initial difficulties were observed with word
recognition. Second, the effectiveness of the MTB e-learning materials was
supported by feedback from learners, who found them innovative, interactive,
and engaging, conforming to Edgar Dale’s Cone of Experience and principles of
experiential learning. The evaluations conducted by the content and ICT experts
confirmed that the e-learning materials met technical standards and provided
high-quality, grade-appropriate content.
Beyond the specific outcomes of this study, the integration of MTB e-learning
materials addresses the scarcity of learning materials in MTB education in the
Philippines. Therefore, by leveraging the benefits of MTB e-learning materials, we
can provide quality education and improve learners’ reading proficiency, aligning
with the statement by UNESCO (2008) that MT instruction can improve education
quality. Finally, integrating these materials can contribute to the overall learners’
educational development, as these materials enhance reading and literacy skills
and promote a more effective, positive, and engaging learning environment for
early-grade learners. The findings of this study corroborate earlier studies that
MT instruction, laying the foundation for literacy skills, can prepare students for
learning additional languages or English as the second language in the
Philippines (Williams et al., 2014), improve English proficiency (Namanya, 2017)
427
academic performance (Ricablanca, 2014) and even prepares learners for lifelong
learning and career development (Apolonio, 2022).
6. Recommendation
The researchers strongly recommend integrating e-learning materials in the MT
in teaching grade one learners alongside traditional methods. However, there is a
need for further improvement of the video story selection and media elements,
such as ensuring more focus on letter Kk words and improving the
synchronization of the character’s voice and mouth movements, and using bubble
dialogue instead of subtitles. Furthermore, the following can be done.
1) Teachers should be allowed to modify the activities in the e-learning
materials to suit better the needs of their learners and their teaching styles;
2) Teachers may provide additional examples and illustrations to
supplement the lesson ;
3) Additional exercises should be provided to give learners more
opportunities to practice their skills;
4) Teachers may ask stimulating questions to help pupils connect the lesson
to real-life experiences;
5) Teachers may modify the design or layout of the materials creatively to
enhance learner engagement;
6) The lesson guide may be modified to align with the specific patterns or
models used in the school.
7) Future researchers may explore factors such as learner motivation,
engagement, satisfaction, and self-directed learning skills to gain a more
comprehensive understanding of the effectiveness of e-learning
interventions.
7. Ethical Consideration
The student has been approved by the ethics review committee of the College of
Education, Mindanao State University-Iligan Institute of Technology, Philippines.
All applicable ethical standards have been followed during the course of the
study.
8. References:
Adi Ana, K.T. & Nitiasih, P.K. (2013). Teaching reading through e-learning website. In
Proceedings of the 3rd International Conference on Foreign Language Learning and
Teaching, Bangkok, Thailand (pp. 15-16). https://docplayer.net/15480582-
Teaching-reading-through-e-learning-website.html
Alakrash, H. M., & Razak, N. A. (2020). Redesigning the English classroom towards fourth
industrial revolution, are the students motivated. The Asian ESP Journal, 16(4), 6-
21. https://www.researchgate.net/profile/Hind-
Hameed/publication/345598048_Oxymoron_in_Day-to-Day_Speech
Anestin, M. (2013). Reading in the digital era: using video self-modeling to improve
reading fluency in at-risk students.
https://scholarcommons.usf.edu/cgi/viewcontent.cgi?referer=https://www.g
oogle.com/&httpsredir=1&article=6831&context=etd
Apolonio, A. L. (2022). Mother tongue-based multilingual education (MTB-MLE) in the
Philippines: Its implications to language learning. Erudio Journal of Educational
428
Innovation, 9(1), 1-12. Retrieved from

http://www.erudio.ub.ac.id/index.php/erudio/article/download/398/296
Baran, J. S., & Davis, K.D. (2009). Introduction to mass communication: Media literacy and
culture. (ebook) McGraw Hill.
Bolstad, R. (2004). School-based curriculum development: principles, processes, and
practices. J Curr Stud, 31(1), 83-97. https://doi.org/10.18296/cm.0068
Brame, C.J. (2015). Effective educational videos. Retrieved June 10, 2020
from http://cft.vanderbilt.edu/guides-sub-pages/effective-educational-
videos/.
Brame, C.J. & Perez, K. E. (2016). Effective educational videos: principles and guidelines
for maximizing student learning from video content. CBE Life Sci Educ. 15(4): es6.
http://doi.org/10.1187/cbe.16-03-0125
Brown, P. J., & Augustine, A. (2001). Screen reading software as an assessment
accommodation: Implications for instruction and student performance. Paper
presented at the American Educational Research Association Annual Meeting,
Seattle, WA. https://files.eric.ed.gov/fulltext/ED458273.pdf
Bruner, J. S. (1982). Towards theory of instruction. Harvard University Press, Cambridge.
https://doi.org/10.1126/science.152.3719.193
Brunnet, N., & Portugal, C. (2016). Digital Games and Interactive Activities: Design of
Experiences to Enhance Children Teaching-Learning Process. International Journal
of Modern Education & Computer Science, 8(12).
https://doi.org/10.5815/IJMECS.2016.12.01
Cabansag, J.N. (2016). The Implementation of Mother Tongue-Based Multilingual
Education: Seeing It from the Stakeholder’' Perspective. International Journal of
English Linguistics, 6, 43. http://doi.org/10.5539/ijel.v6n5p43
Capodieci, A., Cornoldi, C., Doerr, E., Bertolo, L., & Carretti, B. (2020). The use of new
technologies for improving reading comprehension. Frontiers in Psychology, 11, 75.
Cansino, J.A. (2023).The Philippines MTB-MLE Program: Reconciling Its Utilitarian and
Ideological Rationales. East Asian Journal of Multidisciplinary Research.
https://doi.org/10.55927/eajmr.v2i2.2996
Chall, J. (1983). Stages of Reading Development. McGraw-Hill Book. Retrieved from
https://lilaacdotcom.files.wordpress.com/2013/05/challs-stages-of-reading-
development.pdf
Cimmiyotti, C. B. (2013). Impact of reading ability on academic performance at the
primary level.
https://scholar.dominican.edu/cgi/viewcontent.cgi?article=1126&context=mas
ters-theses
Davis, B., & Summers, M.L. (2015). Applying Dale’s Cone of Experience to increase
learning and retention: A study of student learning in a foundational leadership
course. https://doi.org/10.5339/qproc.2015.elc2014.6
DepEd, (2018). Mother tongue-based learning makes lessons more interactive and easier
for students. http://www.deped.gov.ph/press-releases/mother-tongue-based-
learning-makes-lessons-more-interactive-and-easier-students
DepEd. (2016). K to 12 Curriculum Guide MOTHER TONGUE (Grade 1 to Grade 3).
http://www.deped.gov.ph/sites/default/files/page/2016/Mother%20Tongue
%20CG.pdf
Department of Education. (2012). Policy guidelines on the implementation of the mother
tongue-based multilingual education (MTB-MLE). Retrieved from
https://www.deped.gov.ph/wp-content/uploads/2013/01/Policy-Guidelines-
on-the-Implementation-of-MTB-MLE.pdf
429
Department of Education. 2009. Order No. 74. (2009). Institutionalization of the mother
tongue-based multilingual education as a fundamental educational policy and
program in the Department. Manila, Philippines.
http://www.deped.gov.ph/press-releases/mother-tongue-based-learning-
makes-lessons-more-interactive-and-easier-students
Ehri, L. C. (2020). The science of learning to read words: A case for systematic phonics
instruction. Reading Research Quarterly, 55(S1), 45-60. doi:10.1002/rrq.334
Eslit. (2017). “Binisaya” instruction: facing the MTB-MLE challenges head-on. Language,
Culture, Arts and Humanities. http://doi.org/10.13140/RG.2.2.22204.95363
https://www.academia.edu/11535888/_Binisaya_Instruction_Facing_the_MTB
-MLE_Challenges_Head-on
Ertmer, P. A., & Ottenbreit-Leftwich, A. T. (2010). Teacher technology change: how
knowledge, confidence, beliefs, and culture intersect. Journal of research on
Technology in Education, 42(3), 255-284.
https://doi.org/10.1080/15391523.2010.10782551
Dagalea, A. J. L., Peralta, S. B., & Abocejo, F. T. (2022). Evaluation of the mother tongue-
based multilingual education program in the Philippines. Budapest International
Research and Critics in Linguistics and Education (BirLE) Journal, 5(4), 422-431.
https://doi.org/10.33258/birle.v5i4.7269
Flecknoe, M. (2010). How can ICT help us to improve education? Innovations in education
and teaching international, 39(4), 271-279.
https://doi.org/10.1080/13558000210161061
Gee, J. P. (2003). What video games have to teach us about learning and literacy. Computers
in entertainment (CIE), 1(1), 20-20. https://doi.org/10.1145/950566.950595
Gopalan, M., Rosinger, K., & Ahn, J. B. (2020). Use of quasi-experimental research designs
in education research: Growth, promise, and challenges. Review of Research in
Education, 44(1), 218-243. https://doi.org/10.3102/0091732X20903302
Habib, K. & Solima, T. (2015). Cartoons’ effect in changing children mental response and
behavior. Open Journal of Social Sciences, 3, 248-264.
https://doi.org/10.4236/jss.2015.39033
Imperial, J. M. R., Ya-On, C. G. V., & Ureta, J. C. (2019, November). An experimental
Tagalog Finite State Automata spellchecker with Levenshtein edit-distance
feature. In 2019 International Conference on Asian Language Processing (IALP) (pp.
240-243). IEEE. https://doi.org/10.1109/ialp48816.2019.9037687
Kalb, G., & Van Ours, J. C. (2014). Reading to young children: A head-start in life?.
Economics of Education Review, 40, 1-24. https://doi.org/10.2139/ssrn.2267795
Kaur, K. (2023). Teaching and Learning with ICT Tools: Issues and
Challenges. International Journal on Cybernetics & Informatics.
https://doi.org/10.5121/ijci.2023.120302
Lartec, J. K., Belisario, A. M., Bendanillo, J. P., Binas-o, H. K., Bucang, N. O., & Cammagay,
J. L. W. (2014). Strategies and problems encountered by teachers in implementing
mother tongue-based instruction in a multilingual classroom. IAFOR Journal of
Language Learning, 1(1), n1. https://doi.org/10.22492/ijll.1.1.04
Leahy, M. A., & Fitzpatrick, N. M. (2017). Early readers and academic success. Journal of
Educational and Developmental Psychology, 7(2), 87-95.
https://doi.org/10.5539/jedp.v7n2p87
Lone, R. A., & Efstratopoulou, M. (2022). Linguistically-responsive educational
framework for multilingual contexts: supporting children's academic
achievement. Rethinking Inclusion and Transformation in Special Education (pp. 196-
213). IGI Global. https://doi.org/10.4018/978-1-6684-4680-5.ch012
430
MacKenzie, P. J. (2009). Mother tongue first multilingual education among the tribal
communities in India. International Journal of Bilingual Education and
Bilingualism, 12(4), 369-385. https://doi.org/10.1080/13670050902935797
Mackenzie, P.J. & Walker, J. (2015). Global campaign for education policy brief Mother-
tongue education: policy lessons for quality and inclusion.
http://www.campaignforeducation.org/docs/reports/GCE%20Mother%20Ton
gue_EN.pdf
Magwilang, E. B. (2016). Teaching Chemistry in Context: Its Effects on Students’
Motivation, Attitudes and Achievement in Chemistry. International Journal of
Learning, Teaching and Educational Research, 15(4).
Mayer, R. E. (2011). Towards a science of motivated learning in technology-supported
environments. Educational Technology Research and Development, 59(2), 301-308.
https://doi.org/10.1007/s11423-011-9188-3
Medilo, C. (2017). The experience of mother tongue-based multilingual education
teachers in Southern Leyte, Philippines. International Forum Journal, 19(2), 64–79.
http://ojs.aiias.edu/index.php/ojs/article/view/186
Miller & Pennycuff. (2008). The power of story: using storytelling to improve literacy
learning. Retrieved may 23, 2019 from
https://www.academia.edu/21040513/The_power_of_story_using_storytelling
_to_improve_literacy_learning
Mishra, P. (2019). Considering contextual knowledge: The TPACK diagram gets an
upgrade. Journal of Digital Learning in Teacher Education, 35(2), 76-78.
https://doi.org/10.1080/21532974.2019.1588611
Mishra & M. J. Koehler (2006). Technological pedagogical content knowledge: a
framework for teacher knowledge, Teachers College Record, vol. 108, no. 6. pp.
1017–1054. https://doi.org/10.1111/j.1467-9620.2006.00684.x
Mølster, T. (2016). What about ICT for students with reading and writing difficulties?. In
EDULEARN 16 Proceedings (pp. 1859-1867). IATED.
https://doi.org/10.21125/edulearn.2016.1370
Mwoma, T. (2017). Children's reading ability in early primary schooling: Challenges for a
Kenyan rural community. Issues in Educational Research, 27(2), 347-364.
http://www.iier.org.au/iier27/mwoma.pdf
Namanya, S. J. C. (2017). The effects of mother tongue-based multilingual education on
the English literacy of children in Silang, Philippines. In International Forum
Journal (Vol. 20, No. 2, pp. 160-177).
https://journals.aiias.edu/info/article/view/70
Nair, G. & Hindle, R. (2013). Use of ICT in Education. International Journal of Information
Communication Technologies and Human Development (IJICTHD), 5(4), 10-19.
https://doi.org/10.4018/ijicthd.2013100102
Paloma, E. (2012). Development of an e-teaching kit on quadratic functions. Masteral
Thesis (unpublished thesis). MSU-IIT, Iligan City.
The National Achievement Test in the Philippines. Retrieved June 11, 2018, from
http://www.philippinesbasiceducation.us/2013/07/the-national-achievement-
test-in.html
Reid, S. (2002). The Integration of ICT into Classroom Teaching. Alberta Journal of
Educational Research, 48(1). https://doi.org/10.11575/ajer.v48i1.54909
Resnick, L.B., & Weaver, P.A. (eds.) (2013). Theory and Practice of Early Reading: volume 1.
Routhledge.
Ricablanca. J. D. (2014). Effectiveness of mother tongue-based instruction on pupils‘
achievement in mathematics. Unpublished Manuscript. Central Mindanao
University.
431
https://www.academia.edu/10401349/Effectiveness_Of_Mother_Tongue-
Based_Instructionon_Pupils_Achievement_In_Mathematics
Rizal, R., Rusdiana, D., Setiawan, W., & Siahaan, P. (2021). Development of a problem-
based learning management system-supported smartphone (PBLMS3)
application using the ADDIE model to improve digital literacy. International
Journal of Learning, Teaching and Educational Research, 20 (11), 115-131.
Roblyer, A. & Doering, A.H. (2007). Integrating educational technology into teaching. USA:
Pearson, 2007.
Schifter, C. (2016). Bloom's taxonomy vs Dale's cone of experience for twenty-first century
students. In D. Gibson & B. Dodge (Eds.), Proceedings of SITE 2010--Society for
Information Technology & Teacher Education International Conference (pp. 2371-2374).
Association for the Advancement of Computing in Education (AACE).
https://www.learntechlib.org/primary/p/33720/
Smidt, S. (2013). Introducing Bruner: a guide for practitioners and students in early years
education. Routledge. https://doi.org/10.4324/9781315411293
The Literacy Bug (2018). Stages of literacy development. Retrieved from
https://www.theliteracybug.com/stages-of-literacy/
The Center for Speech and Language Development (2018). Developing the G and K
sounds. Retrieved January 10, 2022, from
https://www.thespeechlanguagecenter.com/developing-the-g-and-k-sounds/"
Tofaris, E., & Thornton, R. (2018). Mother Tongue Education Improves Literacy in
Uganda. https://doi.org/10.35648/20.500.12413/11781/ii319
Tuan, L. T. (2011). Teaching Reading through WebQuest. Journal of Language Teaching &
Research, 2(3). https://doi.org/10.4304/jltr.2.3.664-673
UNESCO. (2008). Mother tongue matters: local language as a key to effective learning.
UNESCO. https://unesdoc.unesco.org/ark:/48223/ pf0000161121
UNESCO Bangkok. (2008). Mother tongue-based bilingual education: implications for
education policy. Retrieved from
https://unesdoc.unesco.org/ark:/48223/pf0000153903
Westburg, L., McShane, S., & Smith, L. (2006). Verizon life span literacy matrix: relevant
outcomes, measures and research-based practices and strategies. Washington
D.C. https://unesdoc.unesco.org/ark:/48223/pf0000204386
Williams, A., Metila, R., Pradilla, L. A., & Digo, M. M. (2014). Understanding best practices
in mother tongue-based multilingual education (MTB-MLE) in the Philippines.
Quezon City, Philippines: Assessment, Curriculum, and Technology Research Centre,
University of the Philippines.
http://carinadizonmaellt.com/LANGRES/pdf/22.pdf
https://doi.org/10.1007/s40299-016-0310-5
Yap, J. R., & Chin, M. L. L. (2020). Using systematic synthetic phonics to accelerate rural
indigenous children's acquisition of early literacy skills. International Journal of
Learning, Teaching and Educational Research, 19(10), 1-18.
432
Appendix A- Ang Kaka Storyboard
433
Appendix B- Semi-Structured Interview Questions:

1. Unsa man ang salida na inyo nakita?
[What is the video that you watched?]
2. Mahitabo ba sa tinuod na kinabuhi ang salida na inyong
nakita?
[Do you think what you saw in the video can happen in the real life?]
3. Unsa man ang mga butang na nagsugod sa letrang Kk?
[What are the things that start with letter K?]
4. Unsa man ang nahitabo sa salida?
[What happened in the video?]
5. Giganahan ba mo sa inyong gilantaw? Kung oo, ngano
man?
[Did you like what you saw? If yes, then why?]
6. Mas naka sabot ba mo sa estorya tungod sa salida?
[Did you understand more about the story because you were
watching a video?]
434

Vol. 22, No. 7, pp. 434-453, July 2023
Empowering Children through Sex Education:

A Study on Kindergarten Policies in Indonesia
Munir Yusuf* , Firman , Hasriadi and Mirnawati
Palopo State Islamic Institute, Palopo, Indonesia
Abstract. This article explores the urgency of sexual education for early
childhood, focusing on the role of teachers in delivering this material.
This study aims to understand how sex education can be delivered
according to the psychological developmental stages of early childhood.
Employing ten participants, this qualitative study included observation
and in-depth interviews with teachers and parents at two major
kindergartens in Palopo City, namely Paramata Bunda Palopo
Kindergarten and Datok Sulaiman Palopo Kindergarten. Research
instruments included interview guidelines and observation sheets. Data
were analysed through thematic analysis, considering the perspectives
of educational psychology, educational sociology and religious
education. The results showed that sex education needs to be delivered
with simple language and content adapted to early childhood
development, involving the active role of teachers, parents, and the
community. Sexual education can also help children understand the
values of gender equality and avoid sexual discrimination. In addition,
religious education plays an important role in understanding moral
values in the context of sex. This study concludes that sexual education
for early childhood is important and needs to be done with an approach
appropriate to the stages of child development and involving various
parties. The contributions of this study include recommendations for
developing sexual education curricula in kindergartens and the
education of teachers and parents in providing sexual education. The
results also show potential for future research in developing more
effective strategies in sexual education for early childhood.
Keywords: sex education, early childhood, kindergarten
1. Introduction
Sexual education has become an important and controversial topic in early
childhood education. However, the literature to date is lacking in describing how
sex education should be provided to young children, especially in Indonesia's
education context. In addition, the role of teachers in sexual education for early
childhood has also not been widely researched.
*
Corresponding author: Munir Yusuf, munir_yusuf@iainpalopo.ac.id
©Authors
435
This gap becomes even more critical given that Indonesia has various cultural and
religious backgrounds, each of which has its views on sexual education (Suri &
Chandra, 2021; Wijaya Mulya & Aditomo, 2019). Therefore, the approach used to
provide sexual education needs to be adapted to Indonesia's socio-cultural and
religious context. This research is designed to fill this gap in knowledge by
focusing on how sex education should be provided in the context of early
childhood education in Indonesia, the role of teachers in sexual education, and
how sex education can teach gender equality values and prevent sexual
discrimination.
Literature on the role of teachers in early childhood sexual education in

kindergartens has shown that teachers have a very important role in delivering
appropriate and effective sex education to young children. Teachers are key
facilitators in delivering accurate and thorough information about the body,
gender and healthy interpersonal relationships. They are the primary source of
information for children and are responsible for conveying this information in a
way that children can easily understand. In addition, teachers also serve as
mediators in conversations between children and their parents, helping to
facilitate healthy and effective dialogue about sexuality. In this capacity, teachers
have great potential to influence children's views and attitudes towards sexuality
and gender relations, making their role in early childhood sexual education
particularly important and one which should be prioritized (Bostic et al., 2023;
Scanlon et al., 2022). Teachers also set a good example in demonstrating positive
attitudes and behaviours related to sex.
In providing sex education, teachers must pay attention to early childhood

physical and psychological development (Galbally et al., 2022). Young children
prefer learning through direct experience and examples provided by their
teachers (Brey & Pauker, 2019; Knox-Lane et al., 2023; Leung et al., 2023).
Therefore, teachers need to use creative and innovative methods in providing sex
education to children, such as stories, pictures and games. In this way, children
will be more interested, and it will be easier to understand the material delivered
by the teacher. In addition, teachers should also pay attention to the cultural
norms and values that exist in the surrounding community. For example, in some
communities, the topic of sex is considered a taboo topic and should not be
discussed in front of children. Therefore, teachers should pay attention to the
cultural values around them and ensure that the material delivered is in
accordance with the values adhered to by the local community.
The role of teachers in early childhood sex education is also very important in
building healthy and positive interpersonal relationships between children (Qu et
al., 2022). Teachers should help children understand the concept of friendship and
maintain a good attitude in interacting with their peers (Audley & Jović, 2020;
Salvas et al., 2022). It is important to prevent unwanted behaviours such as
harassment or inappropriate actions towards peers.
In addition, teachers should also build positive relationships with children's

parents. Teachers should realise that parents have an important role in helping
436
their children understand the topic of child sex (Alfaro et al., 2023; Jahng, 2020).
This way, parents will feel more confident in providing appropriate sex education
that aligns with their cultural values.
However, the role of teachers in early childhood sex education is not limited to
providing information and resources. Teachers must also consider children's
safety and privacy when providing sex education. Teachers should ensure that
children feel comfortable and safe asking questions or expressing their feelings
about sex; they should provide a safe and open environment for children to
discuss these topics without feeling embarrassed or afraid (Ayanwale et al., 2022;
Berchiatti et al., 2020; Roche et al., 2023).
In addition, teachers should also pay attention to the sources of information they
use in providing sex education as much information is inaccurate or inappropriate
for young children (Van Loon & Roebers, 2020). Therefore, teachers should choose
appropriate and comprehensive sources of information, such as books or
materials that the school or sex education experts have approved.
In teaching sex education to young children, the very important role teachers play
in helping children understand this topic appropriately and developmentally
appropriately cannot be underestimated (Bloor et al., 2022; Santelli et al., 2021) .
Therefore, teachers should have adequate knowledge about sex education and be
able to provide information creatively and innovatively. In this way, children will
more easily understand the material presented and develop positive attitudes and
behaviours related to sex.
The introduction of sex education for early childhood is still relatively low in
Indonesia (Ismiulya et al., 2022). In addition, many parents and teachers do not
have sufficient knowledge to provide appropriate sex education for young
children. This can have an impact on children's sexual development and the
occurrence of unhealthy sexual behaviour in the future. As a formal education
institution, kindergartens have an important role in providing sex education in
early childhood. However, the role of teachers in providing sex education is still
debated in society, and scientific research on this topic is limited.
This research explores teachers' role in early childhood sex education in

kindergartens, focusing on case studies in several kindergartens in Palopo City,
South Sulawesi, Indonesia. The research will use a qualitative approach with
observation, interview and documentation study methods. Through this research,
it is expected to find valuable information about the role of teachers in early
childhood sex education, as well as factors that influence the quality of sex
education provided by teachers. The results of this study can contribute to
developing a sex education curriculum in kindergarten and provide
recommendations for the development of training and education for teachers and
parents to provide appropriate sex education in early childhood.
437
As this research focuses on understanding and analysing how sexual education is

delivered to young children in kindergarten, several research questions were
formulated, namely:
1. How are sexual education teaching methods applied to early childhood in
kindergartens?
2. What are the perceptions of parents and the community towards the
implementation of sexual education for early childhood?
3. What challenges are faced in delivering sexual education for early
childhood in Kindergarten?
4. What is the school policy in delivering sexual education materials?
2.1. Teacher skills in providing sex education in early childhood
Teacher skills in providing sex education in early childhood are very important.
As such, the pedagogical approach can be a reference for improving such skills.
The pedagogical approach in early childhood sex education is based on the
principle that young children can understand the concept of sex appropriately
and positively if delivered with methods appropriate to their developmental
stages (Gong et al., 2020; Suntheimer & Wolf, 2020). Therefore, teachers must pay
attention to the delivery and selection of appropriate methods for providing sex
education in early childhood.
The pedagogical approach in sex education for early childhood aims to provide
effectively and targeted sex education so that children can understand correctly
about sex and healthy behaviour related to sex and involves participatory
learning methods, where teachers and children interact and learn together. In this
approach, teachers must pay attention to the stages of early childhood
development and adapt learning methods to these stages. As early childhood has
different characteristics in understanding information (Friesen, 2023), teachers
must, therefore, use learning methods suitable for children's developmental
stages, such as picture stories, songs, or games.
In addition, the pedagogical approach in sex education also encourages teachers

to open good communication with children and parents. Teachers need to build
positive relationships with children and parents so that children feel comfortable
and open in discussing sex (Rey-Guerra et al., 2022). In addition, teachers need to
invite parents to be involved in children's sex education, so that parents at home
can reinforce the information teachers provide.
In implementing the pedagogical approach, teachers must also pay attention to

the values and norms prevailing in society. Teachers should ensure the
information provided is in line with the prevailing values and norms so children
can develop positive attitudes and behaviours related to sex. With a pedagogical
approach, sex education for early childhood can be done effectively and on target
so that children will more easily understand sex and healthy behaviour related to
sex and can develop positive attitudes towards themselves and others.
438
Sigmund Freud's theory of psychosexual development suggests that children

experience stages of psychosexual development that affect their understanding of
sexuality (Freud, 2022). The oral, anal, phallic, latent and genital stages play an
important role in shaping children's understanding of the body, gender identity
and social relationships.
In addition, Jean Piaget's theory of cognitive development emphasises the

importance of cognitive development in children's understanding of sexual
concepts (Pakpahan & Saragih, 2022). Early childhood experiences cognitive
development that influences how they understand information about the body,
gender roles and the differences between boys and girls.
Albert Bandura's social learning theory is also relevant in early childhood sex
education (Rumjaun & Narod, 2020). This theory highlights the importance of
environmental influences and behavioural examples in children's learning.
Children learn about sexuality through observing and imitating behaviour from
adults, including teachers and parents. Therefore, the role of teachers and parents
is crucial in providing positive sex education in early childhood.
Furthermore, Urie Bronfenbrenner's ecological theory of development

emphasises the importance of social environment and context in children's
development (Tudge & Rosa, 2020). In early childhood sex education, this theory
considers family, school and community factors in shaping children's
understanding of sexuality and promoting healthy sex education.
2.2. Parental involvement in early childhood sex education

As partners in providing sex education to children, teachers should build effective
communication with parents and engage them in affording early childhood sex
education. Through interactions with parents, teachers can learn about family
values and better understand children's needs (Andoni et al., 2022). This can
strengthen the teacher's role in early childhood sex education and increase
parental involvement in providing sex education to children.
Parental involvement in early childhood sex education is very important because

parents have a huge role in shaping children's values and attitudes about sex
(Yang et al., 2022). Parents need to open communication with children about sex
and answer children's questions honestly and accurately. They also need to model
good behaviour related to sex and teach values and norms as appropriate to their
culture and religion (Lin et al., 2021). In addition, parents can use resources such
as quality books, videos, or websites to assist in providing sex education to their
children. Finally, parents can participate in activities organised by schools related
to children's sex education. With parental involvement, children can obtain
consistent and correct information about sex and form positive attitudes and
behaviours related to sex.
The following are some things parents can do to help teachers in delivering an
understanding of sex education in early childhood:
1) Provide accurate information about children's sexual development:
439
Parents can provide accurate and developmentally appropriate information to

teachers. This will help teachers plan and deliver sex education materials
appropriate for the child's developmental stage.
2) Sharing experiences and insights:

Parents can share their experiences and insights on how to teach sex education to
children correctly and effectively. By sharing experiences and insights, parents
can help teachers understand the needs and challenges in teaching sex education
to young children.
3) Establish good communication with teachers:

Parents can establish good communication with teachers and be open to
providing input or suggestions for children's sex education. This will help
teachers improve or adjust the sex education materials delivered to children
(Barnett et al., 2020; Correia et al., 2020; Davis et al., 2023; Lipscomb et al., 2021).
2.3. The urgency of children's understanding of sex

As educators, teachers have an important role in helping children understand the
concept of sex in a positive and healthy manner. Through appropriate sex
education, teachers can help children understand values about friendship,
communication, decision-making and other social skills to help children develop
a positive sexual identity (Ademuyiwa et al., 2023). Therefore, the role of teachers
in improving children's understanding of sex is very important in early childhood
sex education.
Understanding sex education for early childhood has a very important urgency
because, at that age, children begin to experience physical and psychological
development that affects how they understand their bodies and relationships with
others (Marshall et al., 2020). Therefore, a correct and appropriate understanding
of sex education early on will help children understand and appreciate their
bodies and develop a positive understanding of relationships with others.
In addition, an understanding of sex education at an early age will also help

children to recognise and avoid unhealthy behaviours and sexual harms such as
sexual harassment and sexual violence. With proper understanding, children can
distinguish healthy and unhealthy behaviours in their relationships with others.
Sex education at an early age also helps form positive and healthy relationships
between children and their parents, teachers, and surroundings (Bloor et al., 2022;
Van Leent et al., 2023). In a supportive and positive environment, children will
feel comfortable and open to ask and talk about their sex issues, which can help
them build self-assurance and foster self-confidence.
In order to create a healthy and independent young generation, understanding

sex education in early childhood is very important and must be done
appropriately and correctly. Providing a proper and correct understanding of sex
education early on, will enable children to develop a positive understanding of
themselves and their relationships with others and avoid potential sexual harms
to themselves and others (Leung & Lin, 2019).
440
3. Method
3.1. Research design
This study used a qualitative approach to explore the influence of sexual
education on early childhood character development in Indonesia (Creswell &
Creswell, 2017). The main objective was to understand kindergarten teachers'
perspectives and experiences in implementing sexual education in their
curriculum. This qualitative approach was chosen for its ability to provide an in-
depth and detailed picture of the perceptions, understandings and experiences of
the research subjects, which in this case are kindergarten teachers. This research
design aims to generate rich and meaningful data to provide new insights into
sexual education in Indonesia's early childhood education context.
3.2. Participants
This study involved participants from two leading kindergartens in Palopo City,
namely Paramata Bunda Palopo Kindergarten and Datok Sulaiman Palopo
Kindergarten. Participants were selected based on their involvement in early
childhood sexual education at the kindergarten. From each kindergarten, three
teachers were involved as research participants, comprising a total of six teachers.
In addition, the study also involved four parents, two from each kindergarten.
The decision to involve this number of participants was based on the principle of
data saturation in qualitative research. Data saturation occurs when new
information or data found in subsequent interviews no longer add to the
researcher's understanding or knowledge of the phenomenon under study. In this
study, the interview process was stopped after the researcher felt that the
information obtained from the third participant in each kindergarten was
sufficient and no longer added new knowledge. Therefore, interviews with the
fourth participant and so on were not conducted. This reflects the efficiency
approach in qualitative research, where data are collected until reaching the
point of saturation.
3.3. Data collection

Data collection in this study involved three main methods: observation,
interviews and documentation studies. The research instruments include
observation sheets, interview guidelines and documentation study guides, all of
which have been validated by two experts in the field of early childhood
education, Dr Fatmaridah Sabani and Dr Nur Rahmah. Dr Fatmaridah Sabani is
an expert in early childhood education, while Dr Nur Rahmah is the Head of the
Early Childhood Islamic Education Study Program at Palopo State Islamic
Institute. This validation process helped improve the quality and trustworthiness
of the data collected and the analyses generated.
3.3.1. Observation
Through the observation method, the researcher will directly observe the
situation and interactions in the kindergarten, which is the research location. In
this case, the researcher will observe how the teacher delivers sexual education
material and how students respond to the explanation. In addition, researchers
will also pay attention to the interaction between teachers and parents in the
context of early childhood sexual education. This observation is important to
441
obtain a direct and in-depth picture of the reality in the field and to understand
the broader context in which sex education is provided.
3.3.2. Interview
The interview method will be used to dig deeper into teachers' and parents'
experiences, perceptions and opinions regarding early childhood sexual
education. These interviews will be conducted individually and designed as semi-
structured interviews, where the researcher will have a planned set of questions
and be flexible in exploring topics or ideas that arise during the interview process.
These interviews will provide valuable information about participants' subjective
perspectives and approaches to early childhood sexual education.
3.3.3. Documentation Study

In the documentation study method, researchers will collect and analyse various
documents and learning materials used in sexual education in kindergartens.
These materials may include curriculum, textbooks, handouts and other digital
materials. Through documentation studies, researchers can understand how
sexual concepts and issues are taught and presented to students and how these
materials reflect and support the school's sexual education approach.
3.4. Data analysis

In this study, the process of analysing interview data was carried out with the
help of Atlas.ti software (Friese, 2019). This software allows researchers to
perform several important stages in qualitative data analysis, including coding,
categorisation, visualisation, and data interpretation. Coding in this research
maps and organises the interview transcript data into manageable information
units. Then, the categorisation stage helps group the codes based on similar
themes or topics. Visualisation serves to assist the researcher in understanding the
relationships between categories and sub-categories, as well as in mapping out
common patterns and themes. Furthermore, interpretation is used to extract
meaning and findings from the categorised and visualised data.
Data from observation and documentation studies are analysed descriptively and
are presented in a detailed and systematic narrative, describing the phenomenon
or event based on the empirical data obtained. This descriptive analysis allows
researchers to understand the context, dynamics and meaning of the phenomenon
under study more deeply and comprehensively.
3.5. Research procedures

This procedure ensured the research could be conducted systematically and
efficiently. The steps taken in this research involve the following:
3.5.1. Identification and Selection of Participants

Firstly, the researcher will identify and select kindergartens in Palopo City
involved in early childhood sexual education. Teachers and parents of students
in those schools will be identified as potential participants and will be selected
based on predetermined criteria, including their involvement in sexual education
and their willingness to participate in the study.
442
3.5.2. Data Collection Through Interviews

Once the participants are identified and selected, the researcher will conduct in-
depth interviews with teachers and parents. These interviews will be designed to
understand their experiences, perceptions and approaches to early childhood
sexual education. Interview questions will focus on their experiences in teaching
or supporting sexual education and their perceptions of the effectiveness and
relevance of such education.
3.5.3. Observation
In addition to the interviews, the researcher will conduct observations at the
school to understand more about how sexual education is provided and what role
parents play in this process. This observation will involve direct classroom
observation and interactions between teachers, students and parents related to
sexual education.
3.5.4. Documentation Study

The researcher will collect and analyse educational materials and documents used
in sexual learning at the kindergarten. This analysis will include reviewing the
content, approaches and methods used in the materials and how they are used in
practice.
3.5.5. Data Analysis

Once the data have been collected, the researcher will use a qualitative analysis
approach. This analysis is about describing the data and trying to understand
their meaning. By doing so, the researcher can formulate important findings,
answer research questions, and conclude the research results.
4. Results
The results of this study are based on interviews, observations and document
studies. Through interviews with teachers in several kindergartens, the study
showed how teachers deliver sexual education to young children. This
information also helps understand what challenges teachers face and how they
try to overcome them. Furthermore, direct observation in the classroom allows
researchers to see how sexual education is provided and received by children.
Through observation, researchers can understand the context and nuances of
sexual education practices in kindergartens. The results of the document study
provide insights into how the current sexual education curriculum and materials
are designed and organised. Through document analysis, researchers can see
what is already in place and what may still need to be improved or changed in
the approach to sexual education for early childhood.
The results of the interviews are presented in the form of diagrams generated from
the analysis with Atlas ti. The results showed that the learning content of sex
education conducted by teachers includes two things: a) explaining with simple
language, b) explaining the differences in sex to children, and c) introducing
sexual terms to children.
443
Figure 1. Introducing genitals to children
Views on sex education for children were categorised into two categories: a) sex
education is unnatural, and b) sex education for children is very important.
Figure 2. Views on sex education for children
Teachers' and parents' responses to children's questions about sexuality involved:

a) explaining to children who ask about menstruation, b) feeling taboo to talk
about sex with children, c) children ask about where babies come from, d) children
have never asked about sex.
Figure 3. Teacher and parent responses to children's questions about sexuality
The difficulties experienced in introducing sex education to children include four

things, namely: 1) lack of time with children, 2) difficulty communicating with
language that is easy for children to understand, 3) children prefer to play and not
focus, 4) children are less close to parents.
444
Figure 4. Difficulties experienced in introducing sex education for children

The interviews provide an in-depth picture of how teachers and parents handle
sexual education for young children.
a. Learning Content
Teachers understand the importance of explaining the concept of sexuality in
simple language that is easy for children to understand. They also focus on
recognising the physical differences between boys and girls and introducing
sexual terms to children in an age-appropriate way.
b. Views on Sex Education

Two views were found. Some people felt that sexual education for early
childhood was not appropriate, perhaps out of concern that this would affect the
child's innocence. However, other views emphasise that sexual education is
essential to protect children from sexual abuse and help them understand their
bodies.
c. Response to Children's Questions

Some teachers and parents feel awkward and taboo to talk about sex with
children. However, they also realise that children may question sexuality, such
as menstruation or where babies come from. A good and age-appropriate
response is very important in these situations.
d. Difficulties in Introducing Sex Education

Some challenges include a lack of time with children, difficulty communicating
in a language that children can understand, preferring to play and not focus, and
a lack of closeness between children and parents.
These interviews show that sexual education for early childhood is a complex
and sensitive topic that requires knowledge, expertise and good communication
between teachers, parents and children.
The observations showed various forms of presentation in learning sexuality in

early childhood. Some of the main findings are presented in the following table:
445
Table 1. Form of Presentation in Learning

No. Materials / Serving Description
Materials Techniques
1. Simple stories Storytelling Teachers use simple stories that children easily
Teacher understand to introduce basic concepts of
sexuality and privacy. Stories can be fictional
stories or examples of everyday situations.
Pictures and illustrations are used to help

2. children understand the concepts visually. The
Clear pictures Expository teacher explains and discusses the pictures
with the children.
Teachers use a variety of educational games

3. specifically designed to facilitate learning
Educational Game Play in about sexuality. These games may involve
games class simulations, role plays or other activities that
allow children to learn in a fun and interactive
setting.
This observation shows that teachers use various methods and techniques to
deliver sexual education to young children. These techniques are designed to
ensure that the material is delivered in a way that children can understand and is
appropriate for their stage of development.
a. Simple stories
Teachers utilise narratives that children can grasp to introduce the ideas of
sexuality and privacy using storytelling approaches. These could be imaginary
stories or ordinary occurrences relevant to youngsters' experiences. This strategy
allows youngsters to think about and comprehend things in a situation they can
relate to.
b. Vivid pictures
This expository technique involves using visuals to help children understand
concepts of sexuality. Teachers can use pictures or illustrations to explain concepts
more concretely, allowing children to visualise and understand concepts better.
Pictures can also be a starting point for discussion and answering children's
questions.
c. Educational games
Teachers use educational games as an interactive technique to deliver sexual
education. These games could involve simulations, role-playing or other activities
that allow children to learn in a relaxed and fun environment. Through games,
children can practise the concepts they learn and see how they apply them in real-
life situations.
This observation shows that creative and interactive approaches can help deliver
sexual education to young children in a way that is effective and sensitive to their
developmental stage.
446
Document studies were conducted on the guidelines and handbooks for learners,
teachers, parents and visitors. The handbook is a guideline and policy issued by
the Republic of Indonesia's Ministry of Education, Culture, Research and
Technology. The handbook contains various school policies, including behaviour
and association in the educational environment. These guidelines and handbooks
aim to provide clear and firm guidance to all parties involved in early childhood
education activities at the institution. Learners, teachers, parents and visitors can
refer to the handbook to understand the policies and behaviours that must be
followed in the educational setting.
The document study on the guidelines and handbooks of early childhood

education institutions in Palopo City revealed various policies and guidelines
relevant to sexual education.
a. Uniform Use Policy

The handbook includes a policy on school uniforms for boys and girls. It reflects
efforts to introduce gender concepts to children from an early age.
b. Use of Specific Attributes

The handbook also includes rules on the use of specific attributes for Muslimah
children, showing respect for different cultures and religions within the school
community.
c. Social Behaviour Rules

The handbook includes rules on interactions between students, teachers, parents
and visitors within the school environment. It includes rules on how to
communicate and interact respectfully and safely.
d. Protection for Children with Special Needs

The handbook also includes a protection policy for children with special needs,
demonstrating the school's commitment to ensuring all children have equal access
to education.
This document study shows that school guidelines and handbooks play an
important role in creating a safe, inclusive and supportive learning environment
for all students. They also serve as useful tools to help teachers, parents, and
students understand and comply with the policies and behaviours set by the
school. In addition, they ensure that the policies and behaviours reflect values that
support quality and relevant learning, including in the context of sex education.
5. Discussion
This research was conducted through interviews, observations and document
studies to understand how sexual education is delivered to young kindergarten
children. The interviews showed that the topic is taught through simple language,
the introduction of physical differences between boys and girls, and the
introduction of sexual terms. While some people feel that sex education is not
natural for early childhood, this research shows that many also see it as an
important part of a child's education. However, challenges in teaching include
447
lack of time, communication difficulties, children's lack of focus, and lack of

closeness with parents.
Observations and document studies showed different ways of delivering sexual

education materials, such as teachers use simple stories, pictures and educational
games to explain the concept of sexuality to children. The document study found
school policies on various matters, including uniforms, rules of socialising and
protection for children with special needs. Thus, this research shows that sex
education for early childhood is a complex and sensitive topic, but, with the right
approach, it can be delivered in a way that is effective and respectful of their
developmental stage.
This research found some important intersections and differences with other
studies that focus on sexual education at the nursery school level. In the Chinese
context, research by Chen Jiayue et al. (2022) noted that policies supporting sexual
education in kindergartens are incomplete, and sexual education in kindergartens
lacks teaching instructions. In addition, they also emphasised that sexual
education in kindergartens lacks important family support. These findings align
with this study, reflecting similar challenges in implementing sexual education in
kindergartens.
However, there are important differences when comparing this study with that
by Santelli et al. (2021b) in the United States. They showed that, while sexual
education has strong support from parents and health professionals, acceptance
of sexual education among US adolescents has declined or stagnated over the past
25 years. It contrasts with our findings which show an increase in interest and
participation in sexual education among kindergarten students in Indonesia.
In a study conducted by Ismiulya et al. (2022) in Indonesia, parents'

understanding of providing sexual education to their children was lacking. These
findings coincide with the results of this study, reflecting the communication
challenges between parents and children in delivering sexual education.
However, this study also highlighted variations in approaches to sexual
education, such as the use of uniforms and rules of social interaction, which were
not highlighted in the previous study.
The introduction of sex differences in children also needs to be adjusted to their

developmental stages (Di Pietro et al., 2023; Ibrahim et al., 2021; Swider-Cios et
al., 2023). Through experience, observation and interaction with people around
them, children begin to become aware the physical differences between men and
women early; therefore, parents and teachers must provide a correct and
appropriate understanding of these physical differences so that children do not
misunderstand or get confused about them. A correct and appropriate
understanding can also help children develop positive attitudes towards their
bodies and avoid unnecessary shame or anxiety related to physical differences.
Reflections from this study indicate that sexual education in early childhood is an
important and integral part of their overall education, although it requires an
448
appropriate approach to their development stage. The findings of this study

indicate that there is a need for capacity building for teachers and support for
parents in providing effective and age-appropriate sex education for young
children. Parallel to the study conducted by Nguyen et al. (2022), the results of
this study reflect the need for clearer and more structured school policies to
support holistic and inclusive sexual education. This includes protections for
children with special needs and policies that address issues such as socialising
rules and uniform use. In a broader context, the findings are an important
indicator that early childhood sexual education can and should be an integral part
of the early childhood education curriculum.
The findings of this study have several important implications. Firstly, there is an
urgent need to train teachers and other educators in the effective and age-
appropriate delivery of sexual education to young children. This training should
include ways to communicate about this sensitive topic in language that children
can easily understand and is appropriate to their developmental stage. Secondly,
the findings also point to the need for greater support for parents in
understanding and engaging in their children's sexual education. This could be
through educational resources, workshops or information sessions. Thirdly, the
results underscore the need for clear and consistent school policies that support
inclusive and holistic sexual education, including protection for children with
special needs. The findings imply that sexual education should be placed in a
broader context within the early childhood education curriculum and requires
collaboration between teachers, parents and policymakers.
Based on the findings of this study, several action steps need to be taken. Firstly,
teachers should be engaged in specialised training to deliver sexual education to
early childhood. Ensuring they have the necessary knowledge and skills to
explain this topic appropriately and sensitively is important. Secondly, schools
and governments must develop and implement clear and consistent policies on
sexual education in kindergartens. This includes rules on how and when these
topics are taught, as well as protections for children who may be vulnerable.
Third, an increase of parental involvement and understanding of the importance
of sexual education for their children. This could be through information sessions,
guidance and discussions. Fourthly, the development and use of teaching
materials and methods appropriate to children's age and developmental stage. By
taking these steps, sexual education can be effectively delivered to young children
while respecting and paying attention to their maturity and readiness.
The perspective of religious education in the context of sex education for early
childhood is that religious education views sex as an integral part of human life
that must be managed properly under applicable rules and moral values (Bhana,
2023; Marshall, 2018). Therefore, sex education for early childhood should be
tailored to the religious teachings of the child's community. In addition, religious
education pays attention to the values of purity, so sex education for early
childhood must provide an understanding of the virtues of maintaining personal
purity and avoiding behaviour that is not under religious teachings. Religious
education also emphasises the importance of a humanist approach in providing
449
sex education for early childhood, which means that it must pay attention to
children's psychological and emotional conditions so that trauma or confusion
does not occur.
In terms of the role of parents, religious education also emphasises the importance
of parents and teachers in providing sex education for early childhood. Parents
and teachers are expected to guide children in developing a positive
understanding and attitude towards sex under religious teachings. Finally,
religious education also sees the importance of preventing actions that are
unhealthy or contrary to religious teachings. Therefore, sex education for early
childhood should provide an understanding and knowledge of the negative
impact of unhealthy behaviour contrary to religious teachings.
The novelty of this study lies in the multidisciplinary approach used to explain
and address sexual education for early childhood. It involves the integration of
various perspectives, including educational psychology, educational sociology,
and religious education, each of which brings a unique and important view on the
issue.
From the educational psychology perspective, this research provides new

nuances by emphasising the importance of delivering sexual education materials
appropriate to the child's stage of psychological development. It includes
attention to content, language and teaching methods adapted to cater to children's
comprehension and information-processing capacities at each stage of their
development.
Meanwhile, through the lens of the sociology of education, this study makes a
novel contribution by looking at sexual education as part of a child's socialisation
process. It involves understanding that family, school, and society play important
roles in shaping children's sexual perceptions and attitudes. Therefore, this study
offers a sexual education framework that involves and considers all these parties,
significantly contributing to research in this area.
This study stands out from a religious education perspective by incorporating

religious values and teachings in sexual education. It opens up new insights that
sexual education is not just about factual knowledge but also about shaping
values, attitudes and behaviours that align with the religious teachings of the
child's society.
Thus, this research introduces a new and comprehensive framework for sexual
education in early childhood, which attempts to accommodate the complexity and
uniqueness of this issue in the context of early childhood education. It represents
a step forward from traditional sexual education approaches that are often one-
dimensional and do not consider other important factors. As such, this study
provides valuable new insights for educational practitioners and sets the
foundation for further research in this area.
450
6. Conclusion
The most important finding of this research is that sexual education has started to
be implemented in kindergartens, although it is still a controversial topic in
society. This research shows that sexual education materials can be delivered to
young children using simple language, explaining physical differences between
men and women, and sexual terms through stories, pictures and educational
games. Many parents and educators recognise the importance of sexual education
as part of a child's education.
This research makes a significant contribution to the development of teaching

methods for sexual education in kindergarten. The findings enrich our
understanding of how best to deliver this sensitive topic to children in the context
of early education. The research also emphasises the importance of cooperation
between schools and parents in delivering sexual education and the challenges
teachers face in the teaching process.
While this study provides an important overview of sexual education in

kindergarten, some limitations should be noted. Firstly, the study focused on one
particular educational context and may not cover the whole picture of sexual
education in Indonesia. Secondly, this study only involved a few participants, so
the results may not be generalisable to a wider population. For future research, it
is recommended to conduct a study with a larger and more diverse sample and
explore children's own experiences and views on the sexual education they
receive.
7. Reference
Ademuyiwa, I. Y., Ayamolowo, S. J., Oshinyemi, T. E., & Oyeku, K. J. (2023). Knowledge
and attitude of sex education among secondary school students in south-
western Nigeria: A cross-sectional study. Dialogues in Health, 2, 100085.
https://doi.org/10.1016/j.dialog.2022.100085
Alfaro, J., Carrillo, G., Aspillaga, C., Villarroel, A., & Varela, J. (2023). Well-being, school
and age, from the understandings of Chilean children. Children and Youth
Services Review, 144, 106739. https://doi.org/10.1016/j.childyouth.2022.106739
Andoni, L., Hamsho, N., Blacher, J., & Eisenhower, A. (2022). Psychometric properties of
a parent- and teacher-report measure for autistic children: Parent-Teacher
Relationship Quality Scale (PTRQS). Journal of School Psychology, 95, 25–42.
https://doi.org/10.1016/j.jsp.2022.08.004
Audley, S., & Jović, S. (2020). Making meaning of children's social interactions: The value
tensions among school, classroom, and peer culture. Learning, Culture and Social
Interaction, 24, 100357. https://doi.org/10.1016/j.lcsi.2019.100357
Ayanwale, M. A., Sanusi, I. T., Adelana, O. P., Aruleba, K. D., & Oyelere, S. S. (2022).
Teachers' readiness and intention to teach artificial intelligence in schools.
Computers and Education: Artificial Intelligence, 3, 100099.
Barnett, M. A., Paschall, K. W., Mastergeorge, A. M., Cutshaw, C. A., & Warren, S. M.
(2020). Influences of Parent Engagement in Early Childhood Education Centers
and the Home on Kindergarten School Readiness. Early Childhood Research
Quarterly, 53, 260–273. https://doi.org/10.1016/j.ecresq.2020.05.005
Berchiatti, M., Badenes-Ribera, L., Ferrer, A., Longobardi, C., & Giovanna Maria
Gastaldi, F. (2020). School adjustment in children who stutter: The quality of the
451
student-teacher relationship, peer relationships, and children's academic and

behavioral competence. Children and Youth Services Review, 116, 105226.
Bhana, D. (2023). Comprehensive sexuality education: Young masculinity, cultural
relevance and context matters. In R. J. Tierney, F. Rizvi, & K. Ercikan (Eds.),
International Encyclopedia of Education (4th edn) (pp. 171–178). Elsevier.
https://doi.org/10.1016/B978-0-12-818630-5.08031-3
Bloor, D., Ballantyne, C., Gillespie-Smith, K., Wilson, C., & Hendry, G. (2022).
Investigating the challenges of teaching sex education to autistic learners: A
qualitative exploration of teachers' experiences. Research in Developmental
Disabilities, 131, 104344. https://doi.org/10.1016/j.ridd.2022.104344
Bostic, B., Schock, N., Jeon, L., & Buettner, C. K. (2023). Early childhood teachers' sense
of community and work engagement: Associations with children's social,
emotional, and behavioral functioning. Journal of School Psychology, 98, 133–147.
Brey, E., & Pauker, K. (2019). Teachers' nonverbal behaviors influence children's
stereotypic beliefs. Journal of Experimental Child Psychology, 188, 104671.
https://doi.org/10.1016/j.jecp.2019.104671
Correia, N., Carvalho, H., Fialho, M., & Aguiar, C. (2020). Teachers' practices mediate the
association between teachers' ideas and children's perceived participation in
early childhood education. Children and Youth Services Review, 108, 104668.
Creswell, J. W., & Creswell, J. D. (2017). Research Design: Qualitative, Quantitative, and
Mixed Methods Approaches. SAGE Publications.
Davis, E. M., Schmidt, E., Rothenberg, W. A., Davidson, B., Garcia, D., Barnett, M. L.,
Fernandez, C., & Jent, J. F. (2023). Universal Teacher-Child Interaction Training
in early childhood special education: A cluster randomized control trial. Journal
of School Psychology, 97, 171–191. https://doi.org/10.1016/j.jsp.2023.02.001
Di Pietro, S. V., Willinger, D., Frei, N., Lutz, C., Coraj, S., Schneider, C., Stämpfli, P., &
Brem, S. (2023). Disentangling influences of dyslexia, development, and reading
experience on effective brain connectivity in children. NeuroImage, 268, 119869.
https://doi.org/10.1016/j.neuroimage.2023.119869
Freud, S. (2022). Leonardo da Vinci: A Psychosexual Study of an Infantile Reminiscence.
DigiCat.
Friese, S. (2019). Qualitative Data Analysis with ATLAS.ti. Qualitative Data Analysis with
ATLAS.Ti, 1–344.
Friesen, N. (2023). Phenomenology and education: Researching pedagogical experience.
In R. J. Tierney, F. Rizvi, & K. Ercikan (Eds.), International Encyclopedia of
Education (4th edn) (pp. 131–140). Elsevier. https://doi.org/10.1016/B978-0-12-
818630-5.11015-2
Galbally, M., Watson, S. J., Lappas, M., de Kloet, E. R., Wyrwoll, C. S., Mark, P. J., &
Lewis, A. J. (2022). Exploring sex differences in fetal programming for childhood
emotional disorders. Psychoneuroendocrinology, 141, 105764.
https://doi.org/10.1016/j.psyneuen.2022.105764
Gong, X., Zhang, X., & Tsang, M. C. (2020). Creativity development in preschoolers: The
effects of children's museum visits and other education environment factors.
Studies in Educational Evaluation, 67, 100932.
Ibrahim, K., Kalvin, C., Li, F., He, G., Pelphrey, K. A., McCarthy, G., & Sukhodolsky, D.
G. (2021). Sex differences in medial prefrontal and parietal cortex structure in
452
children with disruptive behavior. Developmental Cognitive Neuroscience, 47,

100884. https://doi.org/10.1016/j.dcn.2020.100884
Ismiulya, F., Diana, R. R., Na’imah, N., Nurhayati, S., Sari, N., & Nurma, N. (2022).
Analisis Pengenalan Edukasi Seks pada Anak Usia Dini. Jurnal Obsesi : Jurnal
Pendidikan Anak Usia Dini, 6(5), Article 5.
https://doi.org/10.31004/obsesi.v6i5.2582
Jahng, K. E. (2020). South Korean mothers' childhood abuse experience and their abuse
of their children with intellectual and developmental disabilities: Moderating
effect of parenting self-efficacy. Child Abuse & Neglect, 101, 104324.
https://doi.org/10.1016/j.chiabu.2019.104324
Jiayue C., Lingfeng G. U. O., & Wenli L. I. U. (2022). Sexuality education in
kindergartens in China based on the perspective of comprehensive sexuality
education. Chinese Journal of School Health, 43(12), 1906–1910.
https://doi.org/10.16835/j.cnki.1000-9817.2022.12.035
Knox-Lane, T., Bailey, C.-S., Ponnock, A., & Denham, S.-A. (2023). Teacher-child racial
congruence and young children's preschool adjustment. Early Childhood Research
Quarterly, 63, 249–263. https://doi.org/10.1016/j.ecresq.2022.12.013
Leung, H., & Lin, L. (2019). Adolescent Sexual Risk Behavior in Hong Kong: Prevalence,
Protective Factors, and Sex Education Programs. Journal of Adolescent Health,
64(6), S52–S58. https://doi.org/10.1016/j.jadohealth.2018.12.007
Leung, S. K. Y., Wu, J., Lam, Y., & Ho, T.-H. (2023). An explanatory study of
kindergarten teachers' teaching behaviours in their visual arts classrooms.
Teaching and Teacher Education, 124, 104018.
Lin, C.-L., Ye, Y., Lin, P., Lai, X.-L., Jin, Y.-Q., Wang, X., & Su, Y.-S. (2021). Safe Sexual
Behavior Intentions among College Students: The Construction of an Extended
Theory of Planned Behavior. International Journal of Environmental Research and
Public Health, 18(12), Article 12. https://doi.org/10.3390/ijerph18126349
Lipscomb, S. T., Hatfield, B., Lewis, H., Goka-Dubose, E., & Abshire, C. (2021). Adverse
childhood experiences and children's development in early care and education
programs. Journal of Applied Developmental Psychology, 72, 101218.
https://doi.org/10.1016/j.appdev.2020.101218
Marshall, C., Semovski, V., & Stewart, S. L. (2020). Exposure to childhood interpersonal
trauma and mental health service urgency. Child Abuse & Neglect, 106, 104464.
Marshall, K. (2018). Global education challenges: Exploring religious dimensions.
International Journal of Educational Development, 62, 184–191.
https://doi.org/10.1016/j.ijedudev.2018.04.005
Nguyen, N. T. P., Chu, A. T. T., Tran, L. H., Pham, S. X., Nguyen, H. N., & Nguyen, V. T.
(2022). Factors Influencing Elementary Teachers' Readiness in Delivering Sex
Education amidst Covid-19 pandemic. International Journal of Learning, Teaching
and Educational Research, 21(2), Article 2.
http://www.ijlter.org/index.php/ijlter/article/view/4839
Pakpahan, F. H., & Saragih, M. (2022). Theory Of Cognitive Development By Jean Piaget.
Journal of Applied Linguistics, 2(1), Article 1.
https://doi.org/10.52622/joal.v2i2.79
Qu, G., Ma, S., Liu, H., Han, T., Zhang, H., Ding, X., Sun, L., Qin, Q., Chen, M., & Sun, Y.
(2022). Positive childhood experiences can moderate the impact of adverse
childhood experiences on adolescent depression and anxiety: Results from a
cross-sectional survey. Child Abuse & Neglect, 125, 105511.
453
Rey-Guerra, C., Maldonado-Carreño, C., Ponguta, L. A., Nieto, A. M., & Yoshikawa, H.
(2022). Family engagement in early learning opportunities at home and in early
childhood education centers in Colombia. Early Childhood Research Quarterly, 58,
35–46. https://doi.org/10.1016/j.ecresq.2021.08.002
Roche, S., Otarra, C., Fell, I., Belle Torres, C., & Rees, S. (2023). Online sexual exploitation
of children in the Philippines: A scoping review. Children and Youth Services
Review, 148, 106861. https://doi.org/10.1016/j.childyouth.2023.106861
Rumjaun, A., & Narod, F. (2020). Social Learning Theory—Albert Bandura. In B. Akpan
& T. J. Kennedy (Eds.), Science Education in Theory and Practice: An Introductory
Guide to Learning Theory (pp. 85–99). Springer International Publishing.
https://doi.org/10.1007/978-3-030-43620-9_7
Salvas, M.-C., Archambault, I., Olivier, E., Vitaro, F., Cantin, S., Guimond, F.-A., &
Robert-Mazaye, C. (2022). Interplay between peer experiences and classroom
behavioral engagement throughout early childhood: Intraindividual and
interindividual differences. Journal of School Psychology, 93, 138–153.
Santelli, J. S., Bell, D. L., Trent, M., Klein, J. D., Grubb, L., Barondeau, J., Stager, M., &
North, S. (2021). School-based Sex Education in the US at a Crossroads: Taking
the Right Path. The Journal of Adolescent Health: Official Publication of the Society for
Adolescent Medicine, 69(6), 886–890.
https://doi.org/10.1016/j.jadohealth.2021.09.007
Scanlon, G., Radeva, S., Pitsia, V., Maguire, C., & Nikolaeva, S. (2022). Attitudes of
teachers in Bulgarian kindergartens towards inclusive education. Teaching and
Teacher Education, 112, 103650. https://doi.org/10.1016/j.tate.2022.103650
Suntheimer, N. M., & Wolf, S. (2020). Cumulative risk, teacher-child closeness, executive
function and early academic skills in kindergarten children. Journal of School
Psychology, 78, 23–37. https://doi.org/10.1016/j.jsp.2019.11.005
Suri, D., & Chandra, D. (2021). Teacher's Strategy for Implementing Multiculturalism
Education Based on Local Cultural Values and Character Building for Early
Childhood Education. Journal of Ethnic and Cultural Studies, 8(4), 271–285.
Swider-Cios, E., Vermeij, A., & Sitskoorn, M. M. (2023). Young children and screen-
based media: The impact on cognitive and socioemotional development and the
importance of parental mediation. Cognitive Development, 66, 101319.
https://doi.org/10.1016/j.cogdev.2023.101319
Tudge, J., & Rosa, E. M. (2020). Bronfenbrenner's Ecological Theory. In The Encyclopedia
of Child and Adolescent Development (pp. 1–11). John Wiley & Sons, Ltd.
https://doi.org/10.1002/9781119171492.wecad251
Van Leent, L., Walsh, K., Moran, C., Hand, K., & French, S. (2023). Effectiveness of
relationships and sex education: A systematic review of terminology, content,
pedagogy, and outcomes. Educational Research Review, 39, 100527.
Van Loon, M. H., & Roebers, C. M. (2020). Using feedback to improve monitoring
judgment accuracy in kindergarten children. Early Childhood Research Quarterly,
53, 301–313. https://doi.org/10.1016/j.ecresq.2020.05.007
Wijaya Mulya, T., & Aditomo, A. (2019). Researching religious tolerance education using
discourse analysis: A case study from Indonesia. British Journal of Religious
Yang, J., Zhang, Q., Li, J., Guan, S., Wang, K., Xu, H., & Liu, Z. (2022). Effect of parental
absence during infancy and early childhood on cognition and depression in later
life: A national household longitudinal study. Journal of Affective Disorders, 319,
562–569. https://doi.org/10.1016/j.jad.2022.09.113
454

Vol. 22, No. 7, pp. 454-472, July 2023
Contributions and Partnership Strategies of

External Stakeholders in the Implementation of
the Alternative Learning System in Bontoc
District: Insights from Teachers
Johnny P. Cayabas, Jr. , Christie Lynne C. Codod ,
Delia A. Sumeg-ang and Elizabeth P. Lacaben
Mountain Province State Polytechnic College, Philippines
Abstract. The involvement and partnership of stakeholders have been

identified as crucial factors influencing program implementation. This
study focuses on examining the contributions and partnership strategies
of external stakeholders involved in the implementation of the
Alternative Learning System (ALS) program in Bontoc District, Mountain
Province. Using a qualitative research design through interview,
observation and focus group discussion, the study identified and
described the contributions and partnership strategies of external
stakeholders in the ALS program implementation, based on insights
provided by the five ALS teachers in the district. Through thematic
approach, the results show that external stakeholders play a significant
role in the implementation of ALS programs in Bontoc District. They
contribute through financial assistance and provision of learning spaces
by the Municipal and Barangay Local Government Units, monetary and
non-monetary donations from alumni, learning materials from the
Provincial Local Government Unit (LGU), free skills trainings and
certifications from Technical Education and Skills Development
Authority (TESDA), and psychosocial support from parents. These
contributions are the result of effective partnership strategies employed
by the stakeholders, including resource sharing, collaboration with
professional colleagues, engagement with alumni, and partnering with
TESDA. ALS teachers recognize the value of these engagements in
enhancing program implementation. However, they also acknowledge
the need to strengthen partnerships with these stakeholders to fully
leverage their potential for strengthening the ALS program
implementation. Establishing networks and linkages with external
stakeholders such as Non-Governmental Organizations (NGOs) and
Higher Education Institutions (HEIs) is identified as a means to
significantly enhance the curricular and instructional efficiency and
effectiveness of the ALS program.
©Authors
455
Keywords: alternative learning system; partnership; strategies;

contributions; implementation; stakeholders
1. Introduction
The Alternative Learning System (ALS) is a program initiated by the Department
of Education (DepEd) in the Philippines to address the educational needs of
individuals who are unable to access formal education due to various challenges
such as poverty, geographical isolation, conflict, and disabilities. ALS offers basic
literacy education and equivalency programs, including the Accreditation and
Equivalency (A&E) Test, to equip learners with functional literacy and numeracy
skills and provide opportunities for further education and employment. The
successful implementation of ALS relies on the collaboration and partnership of
various stakeholders, including the government, Non-Government Oganizations
(NGOs), community leaders, and volunteers. Partnership strategies are the
approaches that guide how two or more entities work together in a collaborative
relationship. In the context of this research, it refers to how schools and its external
stakeholders share their resources to improve the implementation of ALS.
Subsequently, this partnership results to contributions that come in various forms.
United Nations Educational, Scientific and Cultural Organization or UNESCO

(2017) emphasizes the crucial role of local stakeholders in ensuring the
sustainability of literacy programs. The government provides financial support
and resources, while NGOs, community leaders, and volunteers contribute
additional services and resources that align with the program's goals. This
partnership ensures the efficient and effective implementation of the program.
Previous studies have highlighted the significant influence of external
stakeholders in formal educational institutions. However, there is a dearth of
research examining the involvement of external stakeholders in the
implementation of alternative education programs, such as the Alternative
Learning System (ALS). While there have been several studies on the
implementation of ALS in the Philippines, scant attention has been given to
exploring the contributions and support of stakeholders in facilitating its
successful implementation. Thus, the need for this study.
In the Municipality of Bontoc, where this study was conducted, there appears to
be limited involvement of other stakeholders beyond the monetary assistance
provided annually by the Local Government Units (LGUs). During initial
interviews with some ALS teachers, it was noted that parents are not providing
adequate support to their children enrolled in the ALS program. Additionally, the
involvement of barangays, smallest administrative division and the primary unit
of local government in the Philippines, is restricted to the provision of learning
centers, which are typically repurposed barangay halls, Day Care Centers, or
residential homes that serve as temporary housing for ALS learners during
learning sessions. The Mountain Province State Polytechnic College, as a higher
education institution offering teacher education programs, is well-positioned to
lead this program and provide expert support and other resources to ALS teachers
to improve the delivery of the program's curriculum and instruction. This study
is significant because it may contribute to the development of a framework for
456
effective stakeholder partnership in ALS. The findings of this study may inform
policy and practice in promoting effective partnerships among stakeholders in
ALS progam in the Philippines and other similar contexts. The implementation of
the ALS program is a complex endeavor that requires collaboration among
multiple stakeholders, including government agencies, civil society
organizations, private sector entities, and community members. External
stakeholders, in particular, can play a critical role in contributing to the success of
the program through their expertise, resources, and networks. This paper aimed
to explore the contributions and partnership strategies of external stakeholders in
the implementation of the ALS program in the Bontoc District through the lens of
the ALS teachers’ insights. Specifically, it sought answers to the following
problems:
1. What are the contributions of the external stakeholders in the
implementation of the Alternative Learning System?
2. What are partnership strategies of external stakeholders in the
implementation of the Alternative Learning System?
2. Review of Literature
Collaborative partnerships and engaging stakeholders facilitate the exchange of
crucial information, ideas, and resources, which are essential for effective policy
implementation in the modern era. These cooperative arrangements between
multiple organizations are expected to yield enhanced policy outcomes and
organizational performance that surpass what would be achievable in traditional
hierarchical settings. Nonetheless, our understanding of how collaborative
partnerships contribute to a wide range of potential impacts, including both direct
substantive outcomes and indirect process-oriented enhancements, remains
constrained (Conner, 2015). In the context of education, stakeholders’
involvement and partnership have been identified as key factors that can affect
the implementation of an education program. In the study of Çayak and Karsantık
(2020), stakeholders can exert influence over educational activities and elevate
their quality. The importance of active involvement from these stakeholders is
recognized as a key factor in enhancing educational excellence. Relative to this
finding, Darrel, et al. (2020) found that the involvement of stakeholders enhances
both learner performance and the overall quality of education. Within a school
setting, stakeholders encompass individuals who contribute, either directly or
indirectly, to the school's attainment of its goals and objectives.
Parents as Stakeholders. Parental engagement is widely recognized as a positive

factor in improving children's academic achievements. Previous studies have
acknowledged the influence of the family and community on students' learning
(Erdem & Kaya, 2020; Roy & Giraldo-Garca, 2018). However, there is no
universally agreed-upon definition of parental involvement in education, as it
encompasses a range of practices and behaviors related to students' educational
processes. In this study, parental involvement refers to the active and meaningful
participation of biological parents, both financially and behaviorally, in their
children's learning both at home and at school. To encourage parental
involvement, parents are strongly urged to participate in activities such as
attending parent-teacher conferences, which provide an opportunity to
457
familiarize themselves with school policies and initiatives. Regular

communication with teachers is also recommended to stay informed about their
children's academic performance (Newman et al., 2019). Learning at home is
another important aspect of parental involvement, ensuring that families have
access to information that guides them in supporting their children's education
(Ramirez et al., 2022). This type of involvement has been associated with positive
outcomes, including improved academic performance, perception of parents as
educational partners, and higher levels of self-efficacy among students (Eguico et
al., 2022; Garbacz et al., 2017). Furthermore, parents are encouraged to participate
in decision-making processes by taking on roles as committee leaders and
representatives, often through Parent-Teacher Organizations (PTOs) and Parent-
Teacher Associations (PTAs). Numerous empirical investigations and meta-
analyses consistently demonstrate a significant correlation between parental
participation and academic performance, with a predominant positive influence
(Erdem & Kaya, 2020; Tarraga et al., 2017). This involvement has also been linked
to improved self-esteem in children (Garbacz et al., 2017), increased motivation
and self-reliance (Carpenter & Dunn, 2021), as well as better school retention and
attendance (Ross, 2016), leading to improved academic performance regardless of
economic background (Carpenter & Dunn, 2021; Shao et al., 2022). Research
further supports the effectiveness of programs aimed at promoting parental
involvement in education, benefiting not only children but also families and the
broader school community (Sebastian et al., 2017). On the other hand, inadequate
parental involvement has been found to correlate with lower student performance
and engagement (Tarraga et al., 2017).
Community as Stakeholders. When considered as stakeholders, a community

refers to a collective of individuals or entities who possess a direct or indirect
involvement in the affairs of a school. As stakeholders, community members hold
a vested interest or concern in the implementation of school programs. Active
involvement in school management, with a focus on cultural alignment in
teaching and learning, plays a crucial role in fostering a broader acceptance of the
educational process. It has been well-established that community participation in
school activities enhances accountability for learning outcomes and the equitable
allocation of school resources. Additionally, community involvement encourages
the voluntary contribution of local resources, including human capital, materials,
and financial support, to support educational initiatives (Mwila & Meremo, 2022).
When communities and parents are genuinely engaged as stakeholders, actively
contributing to school infrastructure, teacher recruitment, security, and
partnerships, significant achievements can be witnessed in the education sector
(Razzaq, 2015). Ataine and Nkedishu (2017) further highlight the pivotal role
played by community members in providing land for the construction of various
schools. Similarly, community involvement in students' academic activities
encompasses monitoring homework, ensuring regular school attendance,
recognizing children's efforts, and acknowledging teachers. This approach has
been associated with improved attendance, better grades, and effective college
preparation. Thus, community participation in diverse school activities, including
financing and infrastructure development aligned with specific school needs, is
essential to promote quality education in any country. HakiElimu (2017) stresses
458
that school development is not solely the responsibility of the government but
should involve parents and community members, including local business
owners in the school vicinity. Despite the government's provision of free
education to all children, the community and parents play a significant role in
ensuring the delivery of quality education. However, a study by Hussein et al
(2018) revealed that community members supported students but had limited
involvement in school management. This suggests that the level of community
engagement fell short of satisfactory expectations.
Local Government Units as Stakeholders. The Education for All (EFA) 2015
Report in the Philippines accurately emphasizes the significant role of education
in the country's political, economic, social, and cultural aspects. Education has
long been regarded as a crucial element of national development and a key
pathway for social and economic advancement. Reflecting this perspective,
Article XIV, Section 5(5) of the 1987 Constitution explicitly mandates that
education be given the highest budgetary priority by the State. In line with this
directive, the Department of Education (DepEd) consistently receives the largest
allocation from the annual State budget. According to the law, LGUs, which enjoy
fiscal autonomy, are mandated to provide financial assistance for public
education. Through the creation of a Special Education Fund (SEF) and School
Boards under RA 5447, provinces receive a share of the collections from an
additional 1% real property tax. The School Boards are responsible for deciding
the allocation of the SEF, following criteria established by the Bureau of Public
Schools or the Bureau of Vocational Education and approved by the Secretary of
Education. In 1991, the Local Government Code was enacted, amending RA 5447.
This new law stipulates that the proceeds from the additional levy must be
exclusively allocated to the SEF and automatically released to the Local School
Boards (LSBs). In the case of provinces, the proceeds are evenly divided between
the provincial and municipal school boards. The LSBs are tasked with
determining the budget allocation, with priority given to areas such as buildings
and facilities, repair and maintenance of school buildings, establishment and
maintenance of extension classes, and sports activities. The collection and
utilization of the SEF vary among different LGUs, highlighting the importance of
transparency and accountability of the LSB in administering the funds. Evaluating
the actual expenditure becomes a critical mechanism to assess the level of local
support for public schools. In the 2019 National Literacy Conference of the
Department of Interior Local Government Unit, The role of the Locacl
Government was emphasized. LGUs share in the responsibility of enhancing
quality education by improving their collections to fund local education
expenditures through Special Education Fund (SEF). SEF come from an
additional 1% tax on real property that LGUs are mandated to impose and collect
per Local Government Code (RA 7160 of 1991). While SEF spending does not seem
large when compared to either total general government education spending on
basic education (7.4%) or total DepEd spending (8.1%), it is substantial when
reckoned relative to DepEd spending on non-personal services (69%) or DepEd
maintenance and operating expense or MOOE (110%).
459
2.1. Theoretical Framework

The theoretical framework for this study is based on the concept of partnership as
a collaborative relationship between stakeholders involved in the implementation
of the ALS. The study draws on the social partnership theory, which emphasizes
the importance of cooperation and collaboration among different actors in
achieving common goals (Aubry & Kiggundu, 2012; Broucker & Van den Broeck,
2016). This theory highlights the need for stakeholders to work together to address
complex social problems that require collective action, shared resources, and joint
decision-making. The study also draws on the resource dependence theory, which
suggests that organizations rely on external resources and partnerships to achieve
their objectives (Pfeffer & Salancik, 1978). This theory emphasizes the importance
of partnerships in securing resources such as funding, expertise, and support from
external actors, which are critical for the successful implementation of the ALS
Basic Literacy Program. Finally, the study draws on the stakeholder theory, which
suggests that organizations have a responsibility to consider the interests of all
stakeholders in their decision-making processes (Freeman, et al, 1984). This theory
highlights the importance of stakeholder engagement and participation in
ensuring that the ALS Basic Literacy Program meets the needs of all stakeholders.
Figure 1. Conceptual Framework
3. Method
The qualitative research design specifically the narrative approach was used in
this study as it was the most appropriate to describe the perspectives of the ALS
teachers. Qualitative research is often used to gain an in-depth understanding of
a particular phenomenon or experience from the perspectives of the participants
(Creswell, 2014). In this study, the researchers described the contributions and
partnership strategies of external stakeholders in the implementation of the ALS
program based on the insights of the ALS teachers.
460
3.2. Participants and Locale of the Study

The study was conducted in Bontoc, Mountain Province. The Municipality of
Bontoc has 16 barangays, all of which are clustered in seven ALS Community
Learning Centers (CLCs). These centers are comprised of the following:
CLCs Coverage Number of
(Barangays) ALS Teacher
1. Alab CLC Alab, Balili, Gonogon 1
2. Central Bontoc CLC Bontoc Ili, Samoki, Poblacion, 1
Caluttit
3. Bureau of Jail Persons Deprived of Liberty
Management and (PDL) of BJMP Mountain
Penology (BJMP) Province
Mountain Province CLC
4. Caluttit CLC Caluttit 1
5. Maligcong CLC Maligcong,Talubin, Bayyo 1
6. Mainit CLC Mainit 1
7. Guina-ang CLC Guina-ang
Five ALS teachers are handling learners in these CLCs. Three of them hold a
baccalaureate degree in elementary education and the other two in secondary
education. Bontoc district was selected as the study's focus due to its higher
number of ALS centers compared to other districts in the province. Additionally,
the district has a larger enrollment of ALS learners. The respondents for the study
were chosen through total enumeration, as there are only five ALS teachers in the
district. All five teachers willingly participated in the study.
3.3. Data Gathering Tools

Initial data was gathered using an unstructured interview guide. The same
interview, however modified to make it more comprehensive, was used during
the focus group discussion with the teacher-respondents. Open-ended questions
were asked related to the contributions and partnership strategies of external
stakeholders in the implementation of ALS program in the seven CLCs of ALS in
Bontoc district. Some of the questions asked were: What are the contributions of
the external stakeholders in the implementation of ALS program? How did the
external stakeholders establish partnership with ALS schools? Document analysis
was also used to validate the financial assistance of the Municipal LGU.
3.4. Data Gathering Procedure

After permission to conduct the study was sought from the College Management
and the Schools Division of Mountain Province, being the implementer of the
program, the researchers arranged a schedule with the ALS teachers for the
conduct of a focus group discussion with them. Before proceeding with the study,
the researchers obtained the participants' consent to ensure their willingness to
take part in the research. During the FGD, they were asked to identify the external
stakeholders that are actively involved in the program. They were likewise asked
to identify the contributions of these stakeholders and describe how such efforts
have helped improve the implementation of the program. Their responses were
audio recorded and transcribed. Recurring responses were identified and
461
properly coded. These codes were properly categorized into themes that are used
to provide a comprehensive discussion of the problems raised in this study. Prior
to the one-session focus group discussion (FGD), initial data was collected
through interviews with selected teacher-respondents. These interviews helped
inform and refine the interview guide used during the FGD. A chat group was
also created which was used to ask the respondents some follow-up or
clarificatory questions relative to their prior responses during the FGD.
3.5. Treatment of Data

The data collected through the FGD was analyzed using qualitative data analysis
techniques such as thematic analysis and content analysis. The analysis involved
identifying the common responses of the respondents, categorizing these
responses into themes, and establishing connections between the data.

4.1. Contributions of the External Stakeholders in the Implementation of the
Alternative Learning System
External stakeholders play a vital role as essential partners in the successful
implementation of the ALS program in the Bontoc District. Their contributions
extend to various areas, encompassing financial assistance, provision of learning
spaces, both monetary and non-monetary donations (such as alumni
contributions), reproduction of learning materials by the Provincial Local
Government Unit (PLGU), the provision of free skills training and certifications
by the Technical Education and Skills Development Authority (TESDA), as well
as psychosocial support from parents. These collective efforts and support from
external stakeholders greatly enhance the effectiveness and impact of the ALS
program in the Bontoc District despite its observed deficiencies in facilities and
learning resources.
4.1.1. Financial Assistance

A notable and impactful contribution from external stakeholders, greatly
appreciated by the teacher-respondents, is the financial assistance provided by the
Municipal Local Government Unit (MLGU) of Bontoc. This effort was the
initiative of the former Mayor of the Municipality. Annually, the MLGU allocates
50,000 pesos from their Gender and Development (GAD) fund to cater to the
needs of ALS teachers and learners in the district. As confirmed by the Secretary
of the municipal school board of Bontoc, the fund is disbursed upon request
prepared by ALS teachers and approved by the LGU. The records from the
municipal school board indicate that the requested funds are utilized to cover the
expenses associated with the acquisition of school supplies, such as bond papers,
printer inks, ballpoint pens, and pad papers, which are essential for the ALS
learners' educational learning process.
Teacher A confirmed “We receive annual financial assistance from the Municipal LGU
based on our request. The money is being used to purchase school primarily supplies for
our students’ use.” It is important to note that ALS expenses related to school
supplies are not covered by the Maintenance and Other Operating Expenses
(MOOE), a fund specifically designated for formal schools in basic education. The
financial assistance provided by the MLGU not only helps alleviate the burden of
462
purchasing school supplies but also contributes to covering the monthly rental
fees for ALS learning spaces. Among the five learning centers used as classrooms,
two are rented from a church facility. Out of these two, one is paid for through
MLGU fund assistance, while the other one receives support from the Barangay
LGU fund.
Teacher B said “Our barangay LGU defrays our rental expenses because we are using
the Church facility in our place as a learning center because we do not have our own
learning center.” Additionally, there is another learning space hosted in the house
of an ALS teacher, who generously offers the venue free of charge. Some BLGUs
not only cover the rental costs of learning spaces but also pay the electric bills and
provide monetary assistance specifically for purchasing school supplies for the
teachers and learners. Baccali & Ormilla (2021) believed that support from the
parents and government officials or cooperation from stakeholders plays a big
role in the ALS program for the security of the learners and implementers in the
community.
In a focus group discussion with the teacher-respondents, they recognize the

significant impact of the fund assistance on their work enabling them to acquire
essential school supplies and support the rental expenses for learning spaces.
They believe that institutionalizing this support within the MLGU's policies and
programs would ensure its long-term sustainability. By formalizing the provision
of funds specifically designated for ALS, the MLGU can demonstrate its
commitment to inclusive education and the empowerment of ALS learners in the
district. Additionally, they emphasize the need for an increased allocation to
adequately cater to the growing demands of the ALS program. As the number of
ALS learners and the scope of their educational needs expand, a larger financial
allocation would enable the teachers to provide quality instruction and ensure the
smooth operation of the ALS learning centers. With an increased budget, the
teachers would be better equipped to address the various requirements of the ALS
curriculum and improve the overall learning experience for the learners.
4.1.2. Learning Spaces

The ALS Bontoc District comprises seven Community Learning Centers (CLCs)
that cater to ALS learners from all 16 barangays within the Municipality including
the PDLs in the BJMP of Mountain Province. Two of these CLCs are owned by the
barangays and are generously provided free of charge.
Teacher C said “I use my house as a learning center because we do not have one in our
community. I know it is not a conducive place but I thought it is necessary than having
no place at all for my students to learn”. Delmo & Yuzon (2020) found that ALS
students attend the program in community learning centers, which are typically
located in barangay halls, churches, or other designated spaces within the
community where they can gather. Arzadon and Nato (2015) the ALS learning
centers are sometimes found in unlikely places—village meeting hall, jails,
basketball court, or a women shelter. In the Philippines the ALS learning center is
found inside a market area. This makes it convenient for the market workers to go
to the learning center during the lull hours of their working hours. The
463
requirement to go and find learners in difficult places poses a challenge to the

teachers.
The contributions made by these select barangays are highly significant,
particularly considering that the Department of Education-Mountain Province
Division does not have its own dedicated community learning centers, as
indicated in the interviews conducted with the respondents. As stipulated in
Republic Act No. 11510, the DepEd collaborates with LGUs in delivering ALS
programs to their constituents. LGUs, in partnership with other government
agencies and stakeholders, assist in identifying and mobilizing prospective ALS
learners, providing access to suitable learning environments, allocating available
resources to ALS programs (such as Community ALS Implementors and ALS
CLC sites), promoting post-program activities, and introducing local innovations
as necessary. Several private and LGU partners have already been actively
implementing the program with great commitment and enthusiasm to reach all
learners, regardless of their circumstances.
4.1.3. Monetary and Non-Monetary Donations

While the ALS program is still in its early stages and has yet to produce a large
number of successful graduates in Bontoc district, some concerned alumni who
generously contribute to ALS teachers and learners. According to some teacher-
respondents, there are instances when alumni step forward to cover their training
fees when they attend workshops or seminars. In addition, other alumni donate
school supplies. Although these donations are not received regularly, teachers
have expressed that they greatly benefit their teaching and enhance the learning
experience of their students. Teacher D shared “Sometimes, we receive monetary
donations from our alumni. Some of them also provide office suppliese like bondpapers for
the reproduction of our learning materials.” According to Kwarteng and Obing-Ofori
(2021), alumni play a crucial role in the development of an institution, especially
in terms of providing financial support and other contribtions. Justina, et al. (2022)
found that alumni associations have made substantial contributions to the
progress and advancement of their former educational institutions. Numerous
examples can be found where alumni associations have constructed new
structures and furnished their alma mater with chairs and other essential items.
Alumni need to give back to their alma maters, both financially and in other
meaningful ways.
4.1.4. Learning Materials

The Department of Education provides learning modules across the six learning
strands offered in ALS, namely: Communication Skills in English; Kasanayang
Pangkomunikasyon sa Filipino; Scientific and Critical Thinking Skills;
Mathematical and Problem-Solving Skills; Life and Career Skills; Understanding
Self and Society; and Digital Citizenship. In support of the ALS implementation
in the province, the Provincial Local Government Unity (PLGU) of Mountain
Province, through its Sangguniang Kabataan (SK) Federation, takes on the
responsibility of reproducing the learning modules for the students' use. Before
this arrangement, the teacher-respondents shared that they had to personally
reproduce the learning materials since no budget was allocated for their
reproduction. They expressed their frustration over having to spend their own
464
money to provide adequate learning materials for their students. Furthermore,

they do not have MOOEs to cover these expenses, making it a personal sacrifice
on their part. Abad and Galleto (2020) concluded in their study that although, the
support mechanism to ALS instruction is low along with the financial resources,
the scarcity of which may be due to the acquisition of instructional materials,
facilities, and other school equipment needed in carrying out the ALS program.
It can be deduced further that the budget allocated for the implementation
reflects the high availability of instructional materials, facilities, and school
equipment.
4.1.5. Free Skills Trainings and Certifications
The Technical Education and Skills Development Authority (TESDA) plays a
pivotal role in the ALS program by offering valuable skills training and vocational
education to learners. Collaborating with DepEd, TESDA develops and
implements training programs tailored to meet the specific needs of ALS learners.
Their comprehensive range of courses and certifications across diverse fields
equips learners with the necessary skills for employment and entrepreneurship.
TESDA also provides training for ALS teachers, integrating skills development
into the curriculum. Republic Act No. 11510 states that the TESDA shall, assist the
DepEd in equipping ALS learners with technical-vocational skills and provide
access to national certification, as applicable, to improve their work readiness. It
shall support the DepEd in implementing post-program support activities
including providing access to scholarships. Concerning this study, the teacher-
respondents reported that some of their learners had the opportunity to
participate in various community-based training programs offered by TESDA,
completely free of charge. These training programs encompassed a wide array of
disciplines, including Massage Therapy, Bread and Pastry, Beauty Care, Meat
Processing, Dishwashing Soap Making, Pizza Making, Automotive Servicing,
Shielded Metal Arc Welding (SMAW), Bookkeeping, Cookery, and Food
Preservation. As a result of their participation, these learners successfully passed
the assessments and obtained National Certifications (NCs). These certifications,
in turn, served as valuable credentials that enabled some of them to secure
employment opportunities here and abroad. ALS students of Catanduanes were
provided with skills training that helped them in starting up livelihood from their
acquired skills (TESDA conducts free skills training for ALS learners, town
residents in Catanduanes. PIA., n.d.).
4.1.6. Psychosocial Support

Inadequate parental support hinders students from studying effectively at home,
thereby acting as a barrier to the successful implementation of remote learning.
Students who lack emotional support from their parents experience negative
impacts on their mental health and overall well-being. As a result, they are
burdened with the additional responsibility of assisting their siblings with their
schoolwork. Moreover, students who lack effective communication with their
parents, either due to parental absence or excessive busyness, tend to
underperform academically (Mao, 2022).
Teacher E lamented “Sadly, there are parents who do not seem to care for their children
enrolled in ALS. One time, one parent came to me and was surprised to know her child is
465
enrolled in Als.” As key external stakeholders, parents are entrusted with the
crucial role of providing psychosocial support to ALS learners. However,
regrettably, the teacher-respondents have observed that while some parents
initially show support for their children enrolled in the ALS program, their
commitment tends to wane over time. Specifically, they become inconsistent in
monitoring their children's attendance, ensuring the completion of their children’s
modules, and supplying necessary learning materials. According to the
respondents, a significant number of parents exhibit a lack of concern regarding
their children's module submissions and attendance at learning sessions. This
prevailing indifference is particularly notable among parents whose children
come from dysfunctional family backgrounds. The reasons underlying this lack
of parental engagement may vary, but it is disheartening to witness such
diminished involvement in the educational journey of these learners. Teacher E
further shared “Some parents only show involvement if their children get promoted from
ALS.”
Certain parents, particularly those from lower socioeconomic backgrounds or

minority groups, encounter obstacles when it comes to engaging in their
children's learning. In the study conducted by Hanif and Alwi (2019), it was
discovered that parents' attitudes towards their children's academics vary.
However, it was observed that when parents show responsiveness to the school
and engage interactively with teachers, their children tend to achieve better
academically compared to those whose parents do not make an effort to meet with
teachers. Additionally, parental interest in academics fosters positive behavior
among students, reducing disciplinary issues within educational institutions.
Consequently, the responsibility of educating and guiding these learners falls
heavily upon the teachers' shoulders. In the absence of consistent parental
support, they are left to take charge of these students' educational pursuits.
Furthermore, the teacher-respondents emphasized that they bear the sole
responsibility of not only delivering quality instruction but also addressing the
students' psychosocial needs. This added role is necessary to compensate for the
inadequate support the students receive from their homes. This situation poses
significant challenges for teachers, as they must navigate the intricate dynamics
of dysfunctional family environments while striving to create a conducive and
inclusive learning atmosphere for their learners.
4.2. Partnership Strategies of External Stakeholders in the Implementation of

the Alternative Learning System
The successful implementation of the ALS program can be attributed to the
effective partnership strategies employed by the stakeholders, who serve as
significant and invaluable partners in the program's execution. Their active
involvement and collaborative efforts greatly contribute to the achievements and
positive outcomes of the ALS program.
4.2.1. Resources Sharing

In the context of the ALS program, sharing resources involves the collaborative
efforts of various stakeholders to provide and utilize their available resources to
support the program's implementation. This includes sharing physical resources
such as learning spaces, educational materials, or facilities, as well as sharing
466
financial resources, expertise, or manpower to ensure the successful execution of

the program. Sharing resources fosters synergy and maximizes the collective
impact of stakeholders in achieving the goals of the ALS program. The ALS
program in Bontoc District faces challenges due to insufficient physical facilities
and learning resources. It currently lacks Community Learning Centers (CLCs)
and essential laboratory apparatuses, particularly for subjects like Digital
Citizenship that require computer sets. The only available office equipment
consists of a computer and printer for the teacher, along with printed learning
modules for the learners. To address these resource limitations, the ALS program
relies on collaboration and resource sharing among stakeholders. According to
the teacher-respondents, the stakeholders have been highly responsive to their
needs and requests.
Teacher A appreciatively expressed “We are fortunate that our LGUs are very much
ready to respond to our request. Although, still insufficient, their financial assistance
greatly helped us especially in the purchase of office supplies.” Notably, many of these
shared resources are provided free of charge. The teacher-respondents expressed
optimism that by sustaining and strengthening partnerships with these
stakeholders, it is possible to obtain additional resources. They believe that
continued collaboration can lead to an increase in the availability of learning
resources and the improvement of facilities. By leveraging the support and shared
resources from stakeholders, the ALS program in the Bontoc District can
overcome the deficiencies it currently faces. The willingness of stakeholders to
lend their resources at no cost demonstrates their commitment to the success of
the program. With sustained and strengthened partnerships, the ALS program
can access even more resources, thereby enhancing the learning experience and
opportunities for ALS learners in the district. In her report, Salcedo (2023)
identified a significant challenge faced by ALS teachers in Nueva Ecija, namely
the inadequacy of educational resources that fail to meet the students’ needs. As
a result, teachers are compelled to source appropriate learning materials from
external channels. Moreover, certain areas suffer from unstable internet
connectivity, making it difficult to access online resources. Consequently, teachers
must scout for reliable spots where they can download these materials.
Additionally, ALS teachers frequently encounter the need to secure decent chairs,
tables, and learning materials within CLCs. In some instances, they even resort to
using their funds to reproduce modules, purchase paper, and pens, and
sometimes even provide food for their students (Cortez A., et al., 2018).
4.2.2. Collaboration
In this particular context, collaboration entails the sharing of expertise among ALS
teachers to effectively address the diverse needs of ALS learners in terms of
understanding learning content. In the Bontoc District, there are five ALS teachers
responsible for facilitating the program. However, a significant challenge arises
from the mismatch between their teaching workload and their educational and
training preparation. Teacher E, who is a graduate of Secondary Education major
in English shared “The problem is that we were not trained in our baccalaureate
programs to teach ALS learners. Subsequently, I find difficulty teaching other subjects.”
This finding is substantiated by the study conducted by Cortez et al. (2018), which
uncovered that none of the ALS teachers hired by the four SDOs had graduated
467
from an ALS-specific program. Instead, they were graduates of diverse teacher

education programs. Additionally, it was observed that the ALS teachers
employed for elementary and secondary levels were identical, while webinars
and training sessions related to ALS were not conducted as frequently as those
intended for teachers in basic, secondary, and higher education.
These teachers are expected to cater to learners with varied needs, including
aptitude, emotional, and social requirements. However, they lack specialized
training to effectively address these specific learner needs. Three of the teachers
hold degrees in elementary education, making them generalists, yet they are
tasked with teaching subjects at the junior high school level of the ALS program.
Similarly, two of the teachers have degrees in specific fields from secondary
education, but they are required to teach across all disciplines covered in the ALS
curriculum. They openly expressed their concerns about this mismatch, noting
that it often leads to a focus on teaching the content they are most familiar with,
while downplaying or even disregarding topics they struggle to teach. They
attributed this to the fact that during their pre-service teacher training in college,
they were not adequately prepared in terms of pedagogy and content to teach in
the ALS. As a result, this lack of preparation has made their role in teaching ALS
learners a significant challenge. To address this challenge, they collaborate closely
with their colleagues who have expertise in the subjects they find difficult to
teach. By working together, they ensure that learners receive comprehensive
instruction in all necessary concepts. Although effective, they recognize this is
only a short-term intervention. Research has demonstrated that schools
characterized by a high degree of collaboration exhibit more favorable academic
outcomes for students compared to schools with lower levels of collaboration
(Collaborative Teaching- Sharing Best Practice, 2021).
The respondents recognize the importance of being capacitated in terms of

curriculum content, as it would enable them to more effectively teach all subjects.
They acknowledge that their teaching strategies are generally sound, but they
identify a serious issue with their knowledge of certain subject matters. To address
this gap, they emphasize the need for professional development and training that
specifically targets their knowledge and understanding of these content areas.
Such capacity-building efforts would enhance their ability to provide quality
education across all subjects within the ALS curriculum. Moreover, the teacher-
respondents strongly believe that improving their knowledge of subject matters
is crucial for the holistic development of ALS learners. They acknowledge that
effective teaching strategies are important, but without a solid understanding of
the content, their instructional efforts may fall short. Thus, the need to collaborate
with subject specialists, mentors, and resource persons who can provide guidance
and support in areas where they feel less confident.
4.2.3. Alumni Engagement

The teacher-respondents in this context recognize the value and importance of a
partnership strategy focused on alumni engagement within the ALS (Alternative
Learning System) program. While they acknowledge that some ALS alumni have
found employment and are giving donations to the ALS program, they believe
that there is still untapped potential in strengthening the connection between
468
these successful alumni and current ALS learners. One aspect that the respondents
emphasize is the need to go beyond monetary and non-monetary donations.
Teacher D said “I think there is really a need for us to organize an alumni association so
that we can maximize their involvement in the program. By this, we could invite them as
resource speakers for our students.” While financial support is undoubtedly
beneficial for the ALS program, they believe that a more comprehensive approach
is required. They propose the establishment or institutionalization of an alumni
association specifically dedicated to ALS graduates. By creating an alumni
association, the respondents aim to foster a stronger sense of community and
camaraderie among ALS alumni. This association would serve as a platform for
networking, mentorship, and career guidance for current ALS learners. The
teacher-respondents recognize that the experiences and achievements of ALS
alumni can be a valuable resource for those currently enrolled in the program,
providing them with insights, inspiration, and guidance as they navigate their
own educational and career paths. Moreover, this alumni association would
facilitate career orientations and opportunities for ALS learners. It would allow
them to connect with successful ALS graduates who have firsthand knowledge
and experience in various fields. Through mentorship programs, internships,
workshops, and other collaborative initiatives, ALS learners could receive
practical guidance and support to explore different career options, develop
essential skills, and make informed decisions about their future. The respondents
view this proposed alumni association as a means to strengthen the overall impact
and effectiveness of the ALS program. By tapping into the resources and expertise
of ALS alumni, they believe that current learners will not only benefit from the
financial contributions but also gain access to invaluable career guidance and
mentorship. This holistic approach to alumni engagement has the potential to
empower ALS learners, increase their confidence, and enhance their prospects for
success beyond the ALS program.
Alumni engagement plays a vital role in supporting school turnaround strategies

through various means. This includes actively participating in school mentoring
programs, providing tutoring assistance, and ensuring the availability of life
coaching programs for both parents and students. Alumni also contribute to
establishing and sustaining strong school-business partnerships, while raising
funds to support scholarships and other direct benefits that promote engagement.
Additionally, alumni take the lead in coordinating community-building activities
such as job fairs, health fairs, and similar initiatives (Allen, 2017).
4.2.4. Partnership for Skills Training and Certification

One of the goals of the ALS program is not only to integrate its learners into formal
education but also to enhance their skills. As partners in technical education and
skills development, TESDA extends its free services to ALS learners by providing
community-based trainings. Through these platforms, ALS learners can obtain
National Certificates (NCs) that are valuable for securing jobs related to their
skills. This enables them to become economically productive citizens while they
continue their learning journey in the ALS. The teachers participating in the
program have shared that some of their ALS graduates are already working
abroad. However, they recognize the importance of instilling in the learners the
469
value of pursuing and completing a college education as an alternative pathway

to finding a rewarding career. The teachers have observed that many learners are
primarily interested in obtaining an ALS certificate, rather than pursuing higher
education at the college level. “From 2017-2019, TESDA had been closely working
with us in terms of providing skills trainings and national certifications for our students.
We are happy that some of our students who were given these NCs are now working
abroad”, teacher E related.
5. Conclusion
The involvement of external stakeholders in the implementation of the
Alternative Learning System (ALS) program in the Bontoc District has been
identified in this study. This discovery highlights the recognition of ALS teachers
regarding the potential benefits that these engagements can bring to the program.
The contributions made by these external stakeholders are seen as valuable assets
that can contribute to the overall improvement of the ALS program in the district.
However, while ALS teachers appreciate the significance of these contributions,
they also acknowledge the need to strengthen partnerships with these
stakeholders to fully capitalize on their potential. By establishing networks and
linkages with Non-Governmental Organizations (NGOs) and Higher Education
Institutions (HEIs), the curricular and instructional efficiency and effectiveness of
the ALS program can be greatly enhanced. Collaborating with NGOs can bring
several advantages to the ALS program. These organizations often have extensive
experience and expertise in addressing educational issues and providing support
to marginalized communities. They can offer resources, materials, and training
opportunities that can enrich the ALS curriculum and instructional strategies.
Additionally, NGOs can assist in mobilizing community support, advocating for
policy changes, and creating awareness about the importance of education for
out-of-school youth and adults. Similarly, partnering with Higher Education
Institutions (HEIs) can bring numerous benefits to the ALS program. HEIs possess
specialized knowledge and research capabilities that can contribute to the
development and improvement of the ALS curriculum. They can provide access
to academic resources, engage in capacity-building initiatives for ALS teachers,
and offer opportunities for ALS learners to access further education or vocational
training. This research acknowledges that the data collected relied solely on the
perspectives of the teachers. As a result, there is a recognized necessity to expand
the scope of this study through further research. Finally, results of the research
study will be presented to the Local Government Unit through the Local School
Board and the Department of Education, Mountain Province Division for the
possible budget inclusion of ALS and inclusion in the provision of additional
resources such as classrooms, learning materials, and other needs of the program.
6. References
Abad, G. R., & Galleto, P.G. (2020). Alternative learning system program’s
implementation landscape of a division in the Philippines. Cypriot Journal of
Education Science, 15(5) 1078-1088. https://doi.org/10.18844/cjes.v15i5.5173
Allen, J. B. (2017). Community partnerships build school progress: The importance of
alumni engagement. EdLANTA. https://edlanta.org/2017/05/community-
partnerships-build-school-progress-importance-alumni-engagement/
470
Arzadon, M. M., & Nato, R. J. R. (2015). The philippine alternative learning system:
Expanding the educational future of the deprived, depressed.
https://www.researchgate.net/publication/330881893_the_philippine_alternati
ve_learning_system_expanding_the_educational_future_of_the_deprived_depre
ssed_and_underserved
Ataine, A. J., & Nkedishu, V. C. (2017). Relevance of school community relations in the
development of primary education in Delta state, Nigeria. Journal of Research in
Education and Society, 8(2), 18-27.
Aubry, M., & Kiggundu, M. N. (2012). Social partnerships in Canada: An overview.
Canadian public administration/Administration publique du Canada, 55(1), 53-74.
Baccali, V. S., & Ormilla, R. C. (2021). The implementation of alternative learning system
in public schools in Isabela, Philippines. EDUCATUM Journal of Social Sciences,
7(1), 19–29. https://doi.org/10.37134/ejoss.vol7.1.3.2021
Broucker, B., & Van den Broeck, H. (2016). What is social partnership? Theoretical and
empirical considerations. In Social partnership at work (pp. 3-23). Springer.
Carpenter, D., & Dunn, J. (2021). We're all teachers now: Remote learning during
COVID-19. Journal of School Choice, 14(4), 567–594.
https://doi.org/10.1080/15582159.2020.1822727
Çayak, S., & Karsantık, I. (2020, December 31). A stakeholder approach to the
educational process: Parental involvement.
https://www.researchgate.net/publication/348170091_A_Stakeholder_Approa
ch_to_the_Educational_Process_Parental_Involvement
Collaborative teaching- sharing best practice. (2021, May). Www.aitsl.edu.au.
https://www.aitsl.edu.au/research/collaborate/collaborative-teaching-
sharing-best-practice
Conner, T. W. (2015). Exploring the diverse effects of stakeholder engagement on
organizational performance. The American Review of Public
Administration, 47(6), 634–647. https://doi.org/10.1177/0275074015618785
Cortez, A., Rufino, M., Bernard, M., & Cortez, E. (2023). Alternative learning system
delivery services: Basis for capacity building program in post pandemic period.
http://www.vnseameo.org/InternationalConference2022/wp-
content/uploads/2022/07/SEAMEO-Alternative-Learning-System.pdf
Creswell, J. W. (2014). Research design: Qualitative, quantitative and mixed methods
approaches (4th ed.). Thousand Oaks, CA: Sage
Darrel, A. P., Anietie, R., & George, A. (2020). Stakeholders consultation for secondary
educational goals achievement in public schools in rivers state. Global Scientific
Journal, 8(2), 3408–3429.
https://www.globalscientificjournal.com/researchpaper/Stakeholders_Consult
ation_for_Secondary_Educational
_Goals_Achievement_in_Public_Schools_in_Rivers_State.pdf
Delmo, R. O., & Yazon, A. D. (2020). Status of alternative learning system (ALS) at the
city school division of cabuyao: A basis for an improved program
implementation. International Journal of Advanced Research, 8(7), 313-330
Eguico, I. E., Quinco-Cadosales, M. N., Caseros, W. M., Concha, A. D. C. T., &
Omamalin, V. S. (2022). Parents’ views as co-teachers in homeschooling during
the Covid-19 pandemic. Jilin Daxue Xuebao (Gongxueban)/Journal of Jilin University
(Engineering and Technology Edition), 41(7), 330-342.
https://doi.org/10.17605/OSF.IO/BJ4QK
Erdem, C., & Kaya, M. (2020). A meta-analysis of the effect of parental involvement on
students’ academic achievement. Journal of Learning for Development, 7(3), 367-
383.
471
Freeman, R., Harrison, J., Wicks, A., Parmar, B., & De Colle, S. (1984). Stakeholder
theory: The state of the art. Cambridge: Cambridge University Press.
https://doi.org/10.1017/CBO9780511815768
Garbacz, S. A., Herman, K. C., Thompson, A. M., & Reinke, W. M. (2017). Family
engagement in education and intervention: implementation and evaluation to
maximize family, school, and student outcomes. J. Sch. Psychol. 62, 1–10.
HakiElimu, (2017). The impact of the implementation of fee-free education policy on
basic education on Tanzania. Dar es Salaam: HakiElimu.
Hanif, F., & Alwi, S (2019). Impact of parental involvement on academic performance of
students. 10(12), 106-111.
Hussein, Y. A., Mutiri, W., & Samantar, S. M. (2018). The influence of the community
involvement on academic performance of secondary school: International
Journal of Contemporary Applied Researches. 5(8), 86-108.
Justina E., Anyanwu, E., Umunnakwe, G., Benson, O. (2022). The role of community
stakeholders and ministry of education in the effective implementation of school
library guidelines (SLGs) in nigerian schools.
https://www.researchgate.net/publication/362226494_The_Role_of_Communit
y_Stakeholders_and_Ministry_of_Education_in_the_Effective_Implementation_
of_School_Library_Guidelines_SLGs_In_Nigerian_Schools
Kwarteng, H., & Obing-Ofori, D. (2021). Enhancing the role of alumni in the growth of
higher education institutions. Researchgate. https://doi.org/10.21681/IJMSIR-
1.3.831.049173-20201
Mao, W. (2022). Family socioeconomic status and young children’s learning behaviors:
The mediational role of parental expectation, home environment, and parental
involvement. International Journal of Chinese Education, 11(3), 2212585X2211241.
https://doi.org/10.1177/2212585x221124155
Mwila, M. P., & Meremo, G. J. (2019) Bachelor of education programmes offered in
private universities in tanzania within the framework of the inter-university
council of east africa: Stakeholders’ eye on quality of their products. Journal of
Research Innovation and Implications in Education (JRIIE), 3(3) pp. 77-87. Website:
www. Jriie.com
Newman, N., Northcutt, A., Farmer, A., & Black, B. (2019). Epstein’s model of parental
\involvement: Parent perceptions in urban schools. Language Teaching and
Educational Research (LATER), 2(2), 81-100.
https://doi.org/10.35207/later.559732
Pfeffer, J., & Salancik, G. R. (1978). The external control of organizations: A resource
dependence perspective. Harper & Row.
Ramirez, M., Naparan, J., & Naparan, G. (2022). Guiding children though Self-Learning
Modules (SLM): Exploring parents’ experiences. The Normal Lights, 16(1), 1-30.
Razzaq, J. (2015). The 2015 Echidna global scholars policy brief: Community-supported
models for girls’ education in diverse contexts in pakistan: Key issues for policy
and practice.
https://www.brookings.edu/wpcontent/uploads/2016/07/communit y-
supported-models-f
Ross, T. (2016). The differential effects of parental involvement on high school
completion and post secondary attendance. Educ. Policy Anal. Arch. 24, 1–38.
https://doi.org/10.14507/epaa.v24.2030
Roy, M., & Giraldo-García, R. (2018). The role of parental involvement and
social/emotional skills in academic achievement: Global perspectives. School
Community Journal, 28(2), 29-46.
472
Sebastian, J., Moon, J.-M., & Cunningham, M. (2017). The relationship of school-based
parental involvement with student achievement: A comparison of principal and
parent survey reports from PISA 2012. Educ. Stud. 43, 123–146.
https://doi.org/10.1080/03055698.2016.1248900
Shao, M., He, W., Zhao, L., & Su, Y-S. (2022) The influence of parental involvement on
parent satisfaction: The moderating effect of parental educational level and the
number of children. Front. Psychol. 12:752802.
Tarraga, V., García, B., & Reyes, J. (2017). Home-based family involvement and academic
achievement: a case study in primary education. Educ. Stud. 44, 361–375.
https://doi.org/10.1080/03055698.2017.1373636
UNESCO. (2017). Global alliance for literacy within the framework of lifelong learning
(GAL). http://www.unesco.org/new/en/education/themes/education-
building-blocks/literacy/resources/global-alliance-for-literacy/
Appendix 1
Sample Open-ended Guide Questions for FGD and Interview
1. Can you provide examples of specific contributions made by external

stakeholders to support the implementation of the ALS?
2. In your opinion, what are the key benefits of involving external
stakeholders in the implementation of the ALS?
3. How do external stakeholders engage with ALS learners and educators
to enhance the effectiveness of the program?
4. What strategies have external stakeholders employed to mobilize
resources and support for the ALS implementation?
5. What else do you think need to be improved in these strategies?
6. Can you discuss any successful initiatives undertaken by external
stakeholders in the context of the ALS implementation?
7. How do external stakeholders ensure the sustainability and continuity of
their involvement in the ALS implementation?
8. What do you think are some strategies that ALS implementers can
explore to improve its implementation?
473

Vol. 22, No. 7, pp. 473-496, July 2023
Faculty Members’ Awareness of Artificial

Intelligence and Its Relationship to Technology
Acceptance and Digital Competencies at King
Faisal University
Sherif Adel Gaber*
Associate Professor, Faculty of Education, King Faisal University
Al-Ahsa, Saudi Arabia
Hussein Ahmed Shahat

Professor, Faculty of Education, King Faisal University
Ibrahim Abdullah Alkhateeb

Sumaia Attia Al Hasan

Lecturer, Faculty of Education, King Faisal University
Mohammed Ahmed Alqatam

Assistant Professor, Faculty of Education, King Faisal University
Sultan Mubarak Almughyirah

Assistant Professor, Faculty of Education, King Faisal University
Mohammed keshar Kamel

Faculty of Art Education, Minia University,
Minia, Egypt
*
Corresponding author: Sherif Adel Gaber; sagahmed@kfu.edu.sa
©Authors
474
Abstract. Artificial intelligence (AI) is a field of science seeking to

develop computer systems with a level of efficiency similar to that of an
expert human. By employing the most advanced technologies, such
efficiency can contribute significantly to improving the educational
process. Therefore, this study aimed to identify the level of AI
awareness among faculty members at King Faisal University and
examine the relationship between AI awareness and technology
acceptance (TA) and digital competencies (DCs). The study used the
descriptive-correlational research method, and three analyses were
conducted, focusing mainly on AI, TA, and DCs. The sample consisted
of 101 faculty members from all departments in the College of
Education, representing 43.5% of the college’s faculty. They were
selected using the simple random sampling method. After analyzing the
quantitative data, findings revealed that the faculty members had a
medium level of awareness, with a mean score of 3.05 on a 5-point scale.
The findings also revealed that there was no statistically significant
relationship between AI awareness and TA among faculty members,
with a correlation value of 0.139 and a significance value of 0.165. In
contrast, the study found a direct and statistically significant positive
relationship between AI awareness and DCs among faculty members,
with a correlation value of 0.568 and a significance of < 0.001. Therefore,
it is essential to prepare faculty members to use AI in education and
improve their attitudes towards AI by conducting workshops and
providing them with the necessary skills to employ AI applications in
education.
Keywords: artificial intelligence; technology acceptance; digital

competencies; faculty members; King Faisal University
1. Introduction
Modern technology has affected every aspect of human life, especially
education. The world has now moved towards employing the fifth generation of
the Internet, the so-called Internet of Things, in education, and there has been a
growing interest in integrating artificial intelligence (AI) applications into the
teaching and learning process. Hence, AI has grown to hold great significance in
this field (Al-Darayseh, 2023; Ilkka, 2018).
Many educational institutions have tended to employ AI techniques in their

educational systems and benefit from such techniques by advancing the levels of
workers’ performances in their various departments and branches. In fact, such
integration contributes to investing in this reality, facilitating the management of
the education process, and achieving better educational outcomes. Holmes et al.
(2019) found that the use of AI tools to support and enhance learning has
increased in the past decade, and Zawacki-Richter et al. (2019) confirmed that
this use increased after the closure of schools due to COVID-19. Many
commercial companies that are specialized in AI applications reported
significant increases in registered users in education (Maio et al., 2021).
AI is currently the most prominent technique in the world of technology, and it

is based on enabling computer systems to perform tasks that usually require
human intelligence. The field of AI is concerned with the theory and practice of
475
developing systems that simulate the distinctive characteristics of human

intelligence, and its main goal is to understand the principles of intelligence in
human behavior to implement them in a machine (Badawi, 2022; Luo, 2018;
Nadimpalli, 2017; Tecuci, 2012). As the development of AI began with the
intention of creating intelligence similar to human intelligence and implanting it
in machines, making it capable of learning and analyzing data in different
situations, showing reactions, and making decisions according to the situation,
AI is designed based on many disciplines such as computer science, biology,
psychology, linguistics, mathematics, and engineering in order to develop
computer functions related to human intelligence such as thinking, learning, and
problem solving (Ahmed, 2018).
The development of AI began with the intention of creating artificial intelligence

similar to human intelligence by making technology capable of learning,
analyzing data and different situations, showing reactions, and making
decisions. Given this broad scope, AI is utilized in many disciplines, including
computer science, biology, psychology, linguistics, mathematics, and
engineering, to develop computer functions related to human intelligence, such
as thinking, learning, and problem-solving (Tutorial Point, 2015).
AI is increasingly important in academic institutions as faculty members and

students demand high use of the information. Using AI technologies enables
faculty members and students to perform their job duties and scientific
requirements or develop their capabilities and skills in the field of higher
education (Al-Khathami, 2010). AI is thus likely to change education in the
coming decades, both in the classroom and at the system level (Organization for
Economic Co-operation and Development [OECD], 2020). For example, when it
comes to personal learning, AI can provide learning based on the individual
needs of the learner and identify educational resources and methods within the
framework of the student’s learning pace. Luckin (2017) argued that AI
technologies aim to enable every learner to access high quality, personalized,
comprehensive, ubiquitous, and lifelong education, whether formal or non-
formal. Holmes et al. (2022) state that the use of AI in education consists of four
levels: (a) learning with AI, in which AI provides supportive resources for the
learner, teacher, and educational administration; (b) using AI to learn how to
learn through data analysis, identify students’ academic level and effective
learning methods, and make decisions about the learning process and
educational plans; (c) learning about AI, which concerns learners’ knowledge of
AI techniques; and (d) preparing for AI, in which all citizens are prepared for
the possible effects of AI on their lives and are made aware of some of the issues
associated with it, such as ethics of AI, databases, monitoring, and the potential
impact on jobs in the future. In general, current estimates indicate that AI is
impacting the job market, which is leading to rapid changes in the demand for
certain skills and competencies, and the educational system must adapt to such
changes (Ilkka, 2018).
Marvin Minsky and John McCarthy first used the phrase artificial intelligence
(AI) in 1956 as one of the outcomes of a workshop that brought together several
researchers in various fields to build machines capable of simulating human
476
intelligence (Kaplan & Haenlein, 2019). McCarthy et al. (2006,) defined AI as “a

machine that deals with a certain problem in the manner of human intelligence,”
(p. 11) while Kaplan and Haenlein (2019) defined it as “a system that is capable
of correctly interpreting external data, learning from such data, and using the
data to achieve specific goals and tasks through flexible adaptation” (p. 15).
Leslie et al. (2021) defined AI as “algorithmic models that perform cognitive
functions in areas that were subject to human thinking, judgment, and
inference” (p. 8), and UNICEF (2021) defined it as
machine-based systems that are able, by providing them with a set of
goals, to make predictions, recommendations, and decisions that affect
reality or the virtual environment so that AI systems interact with us
and work in our environment directly or indirectly and in an
independent manner that is able to adapt its behavior by recognizing the
context. (p. 16)
Similarly, UNESCO (2021) defined AI as

technical means that are used to process information that integrate
models and algorithms in an attempt to enhance the ability to learn and
perform cognitive tasks that lead to results such as prediction and
decision-making in real and hypothetical conditions independently. (p.
4)
Recently, AI has been defined as a scientific field (or activity) that manufactures
machines that can work appropriately based on the environment. AI is also a
technology that can be used in various ways in several fields, including for
development in education (Ilkka, 2018). Considering the previous definitions, it
is clear that AI is becoming increasingly complex at the level of construction as
well as the functions it performs, and the field is forming an independent
science. Moreover, the field of AI is receiving increased interest, especially in
terms of its potential to improve the quality of life in general. Zhong (2006)
asserts that AI is a branch of modern science and technology that aims to explore
the secrets of human intelligence and transplant it into machines.
Furthermore, the literature indicates that there are three generations of AI. The
first generation is called artificial narrow intelligence (ANI), and in this
generation, AI is applied to specific tasks. The second generation is called
artificial general intelligence (AGI); in this generation, AI can think, plan, and
solve problems independently. The third generation is artificial
superintelligence (ASI), in which AI is considered a conscious system capable of
social skills and creativity (Kaplan & Haenlein, 2019). Notably, AI applications
have raised the expected level of reliability and effectiveness in terms of
processing data for solving problems and making decisions. AI can advance the
learning and thinking processes, which play a significant role in utilizing and
employing knowledge cumulatively. Thus, such features can improve the
quality of decision-making based on analyzing data quantitatively and
qualitatively, regardless of the complexity of the problem (Chowdhury & Sadek,
2012).
477
Regarding AI’s application in education, Al-Darayseh’s study (2023)

demonstrated the positive impact of applying the technology acceptance model
(TAM) to the use of AI applications and the emergence of many positive factors
that improve self-efficacy. In addition, Al-Darayseh’s study emphasized the
need for reinforcing teachers’ awareness of the basic concepts of AI and
providing them with the necessary tools to apply such concepts in the teaching
and learning processes, The study also suggested conducting experimental
studies to develop teachers’ competence in utilizing AI in the classroom.
Similarly, Zhang and Aslan’s study (2021) indicated that AI contributes to
meeting the emotional needs of students, in turn improving learning. They
additionally emphasized the need for more comprehensive designs and
Artificial Intelligence in Education (AIED) technologies to meet the diverse
needs and preferences of students. The AIED Association (2023) concluded that
there is a need to develop a plan for researchers to conduct empirical studies
regarding how AIED can shape the future of education in the next thirty years.
Recently, AIED has become one of the most important goals of the United
Nations; by using AI in education, it is possible to create a world with effective
education at all levels (United Nations, 2023).
Concerning the realistic application of AI in schools, a study by Nazaretsky et al.

(2021) found that teachers viewed AI technologies and tools as highly valuable;
however, they were not sure that they wanted to adjust their teaching methods
to employ such technologies. The study also showed that teachers resisted
adopting recommendations that contradicted their beliefs about education. In
addition, Al-Subhy’s study (2020) aimed to identify the reality of the use of AI
applications by faculty members at Najran University, as well as the challenges
facing their use and the relationship between some variables such as gender and
academic degree. The study had a sample of 301 faculty members at Najran
University, and the study concluded that the use of AI applications in education
by the respondents was very low. Furthermore, it is concluded that there are
many challenges affecting the use of AI applications. Jabali and Al-Qahtani’s
study (2022) also aimed to identify faculty members’ degree of awareness of AI
skills in education and its relationship to teaching experience and training
programs at King Khalid University in the Kingdom of Saudi Arabia. Their
study was based on a sample of 133 faculty members at the university. The
results showed that the faculty members had a high degree of awareness of AI
and that there were no statistically significant differences in the effect of
experience and training programs on the faculty members’ degree of awareness.
One of the most important challenges facing researchers in the field of

information systems and technology is the extent to which beneficiaries accept
or reject this modern technology. For this reason, researchers are interested in
developing theories and models that explain how beneficiaries accept a
particular technology. Technology Acceptance Model (TAM), presented by
Davis in 1989. The TAM is one of the most commonly used models to identify
the factors that affect beneficiaries’ acceptance of technology (Jeong, 2011)
because it analyzes external environments and behavioral factors. It is also
characterized by flexibility, which enables the consideration of external factors
478
that are relevant to the study population and are expected to impact their
technology acceptance (Al-Alawi et al., 2014).
Perceived
Usefulness
Attitude
s Behavioral Actual
Toward Intention Use
Using to Use
Perceived Ease
of Use
Figure 1: Technology Acceptance Model (Davis, 1989)
The TAM assumes that the acceptance of any particular technology is based on
two main factors: perceived usefulness, which expresses the level to which an
individual believes that using a particular system will help him or her enhancing
functional performance, and perceived ease of use, which refers to the degree to
which a person believes that using the technology will be easy. For a particular
system, it will be with the least possible effort (Davis, 1989). These two belief-
based factors are affected by other external variables and indirectly influence the
behavioral intention to use technology (Davis, 1989). Therefore, the TAM is
useful in describing user technology adoption behavior in various
environmental settings (Fathema & Sutton, 2013). For example, in one study, Teo
(2009) attempted to build a model to predict the extent of TA for pre-service
teachers at the Female Teachers Training Institute in Singapore and examined
the relationships between variables associated with factors affecting TA.
Computer self-efficacy was found to directly impact behavioral acceptance of
technology use, while perceived ease of use, technological complexity, and
facilitating conditions affected the behavioral acceptance of use indirectly.
The introduction of new technology in learning tools such as mobile devices,

tablets, laptops, simulators, and virtual laboratories aims to change education in
schools and institutions. The globalization of education requires the application
of digital technologies (Haleem et al., 2022). Therefore, the teaching profession
faces rapidly changing demands that require a set of new competencies that are
increasingly broader and more developed (European Commission, 2023). The
researchers of the current study believe that digital competencies (DCs) are
essential in the present education system and serve as the gateway to the future
of education. Workers at all levels of education must develop their digital skills
and competencies to keep pace with the current escalation of knowledge and
technology. DCs are one of the eight main competencies for the optimal use of a
range of digital information and communication technologies as well as basic
problem-solving in all aspects of life (Akgün, 2020). DCs include the ability to
use digital technologies safely, critically, and judiciously in work, learning,
social engagement, and human interactions to achieve various goals (Caena &
Redecker, 2019). DCs are also a way of using and understanding technologies
and their impacts on the digital world (Becker et al., 2017). In the field of
479
education, DCs include the technical skills that faculty members use to obtain
information, formulate it in the form of digital images, store it in files, and
publish it on the information network (Amayreh, 2019). With technological
development, education professionals must learn to use various tools, improve
their DCs, and harness all available online resources and e-learning platforms to
ensure that their materials are attractive and up-to-date (Haleem et al., 2022).
Shaheen et al. (2021) classified DCs in the educational process into four
categories: (a) computer competencies, (b) computer leadership competencies,
(c) Internet network leadership competencies, and (d) software design and
educational multimedia competencies. Similarly, Al-Alimat (2012) classified
electronic technical competencies into four types: (a) cognitive competencies, (b)
performance competencies, (c) emotional competencies, and (d) productive
competencies. In the European framework, DCs for teachers in 22 areas are
classified into six categories: (a) competencies related to the professional
environment; (b) competencies related to providing, creating, and sharing digital
materials; (c) competencies related to managing digital tools and regulating their
use; (d) competencies related to digital tools and strategies that enhance
evaluation; (e) competencies related to using digital tools to empower learners;
and (f) competencies related to facilitating DCs for learners. This framework
includes DCs for teachers in all stages of education, as it provides a general
reference framework for developers of DC models in educational organizations
(European Commission, 2006).
DCs are essential in the current day, as they help provide teachers with the
appropriate amount of knowledge and skills to use modern technologies in their
field of work and keep pace with the development taking place in the field of
education. Furthermore, in general, DCs help raise the quality of education and
learning (Al-Ghamdi, 2019). Carretero et al. (2017) also identified areas for
evaluating DCs among faculty members: knowledge of information and data;
communication and collaboration; digital content creation (including
programming); safety (including digital well-being and cybersecurity-related
skills); and problem-solving (critical thinking). Finally, Chiu et al. (2021)
emphasized that DCs in higher education must be further researched.
The Gothenburg Summit in 2017 recommended developing an action plan to

enhance digital skills and competencies to expand and increase the purposeful
use of digital and innovative education practices. The following priorities were
identified: more effectively using digital technology in teaching and learning,
developing relevant DCs and skills for digital transformation, and improving
education (Ilkka, 2018). Regarding the development of DCs, Moawad (2019)
aimed to identify the effectiveness of a pervasive training environment based on
the preferred training pattern for developing DCs and TA among faculty
members. In the development of DCs and TA among faculty members at the
College of Education, the second experimental group with a participatory
training style excelled. Ng et al. (2023) also indicated that there is an increasing
need to prepare teachers with DCs to use and teach AI in their institutions,
especially in the presence of a small number of reference guides that direct them
480
to work in these environments; therefore, they emphasized the need to train

teachers with the necessary DCs.
University faculty members are the pillars of the educational process; they
determine the strength, level, and quality of the university, and it is their
responsibility to achieve the desired goals in educational development,
especially in light of technological acceleration, which helps universities
compete at the global level (Sharaf El-Din, 2023). Technical education is a natural
response to the educational opportunities provided by the information and
communications revolution; it can advance teachers’ ability to improve teaching
methods and develop teaching competencies in the university education system
(Al-Khafaji et al., 2021). Teachers have many modern responsibilities and
contemporary roles, including the masterful use of knowledge sources such as
information networks, computer programs, and applications; influencing direct
attitudes and designing activities and experiences based on technology; and
innovating the use of educational technologies and knowledge sources. In order
for a faculty member to fulfil these roles, they must possess many educational
and technical competencies (Abdul Rasul, 2015).
There are several key factors that directly affect institutions’ success in adapting
to AI, the enormous capabilities it possesses, the recent trends of employing it in
the educational process, and what it takes to develop capabilities in various
educational institutions to address this modern innovation. The most important
factors are the workers’ level of awareness of the technology, their acceptance of
it, their willingness to use it, and their possession of the necessary DCs (Ahmed,
2020).
King Faisal University seeks to employ modern technologies in managing the

educational process, delivering education to beneficiaries in line with the
requirements and standards of the present day, and enhancing its efficiency as
an educational institution. However, researchers who have worked at King
Faisal University for between 8 and 15 years have noticed a discrepancy in the
attitudes of faculty members regarding their TA and DCs. In addition, there is a
lack of forums, seminars, programs, training courses, and workshops for faculty
members to develop the knowledge required for using contemporary
technology and understanding its importance in the future of education.
Based on the above considerations and the suggestions of previous literature

(Ahmed, 2020; Kleef et al., 2010; Krumsvik, 2008), it is essential to prepare
individuals for the new era of technology and discuss the challenges that arise in
educational systems. Ahmed (2020) confirmed that digital education in the Arab
world still faces challenges at various levels, and there is an urgent need to
conduct more studies to address this issue and its various elements. Therefore,
the problem of the current study emerged, and the research attempted to answer
the following research questions:
1. What level of awareness do faculty members at King Faisal University have
regarding AI?
481
2. What is the correlation between the level of awareness of AI and TA among

faculty members at King Faisal University?
3. What is the correlation between the level of awareness of AI and DCs among
faculty members at King Faisal University?
2. Methodology
2.1. Approach
After the researchers identified the problem of the study, reviewing the
literature related to the subject, and considering the nature of the study, its
objectives, questions, and the data to be obtained, they concluded that the
appropriate approach to the current study was the descriptive-correlational
approach. This approach was chosen due to its ability to identify detailed facts
about the reality of the studied phenomenon, which enabled the researchers to
obtain a comprehensive description and accurate diagnosis of the problem. The
study sample included faculty members from the College of Education at King
Faisal University in the Al-Ahsa Governorate. They were selected using the
simple random sampling method. A 5-point Likert scale was prepared for the
study and was used to identify the level of faculty members’ cognitive and
performance awareness of AI. Their responses were then analyzed.
2.2. Study population and sample

The study population consisted of all faculty members in the College of
Education at King Faisal University in the Al-Ahsa Governorate, which included
232 members in the third semester of 2022–2023. The study instruments were
distributed to all members, and 101 members, from various departments of the
college, responded, representing 43.53% of the faculty of the College of
Education. The researchers obtained research ethics approval from the Deanship
of Scientific Research at King Faisal University, Ref. No. KFU-REC-2023-APR-
ETHICS775.
2.3. Study instruments

2.3.1. Artificial Intelligence Awareness Scale (AIAS)
The researchers developed a 5-point Likert scale to measure the level of AI
awareness among faculty members by looking at the educational literature and
referring to several measures related to AI. The scale was presented to a group
of experts in educational technologies to ensure its validity. The arbitrators were
asked to express their opinions on the scale in terms of the items’
appropriateness, their clarity, the soundness of their linguistic formulation, and
any other observations. The researchers considered the consensus of 85% of the
arbitrators (six out of seven) sufficient to deem the item acceptable. After
consulting the arbitrators, three items were deleted, and two new items were
added. The linguistic formulations of four items were modified, and the scale
ultimately consisted of 12 items measuring two dimensions: cognitive awareness
of AI and performance awareness of AI.
The faculty members responded to the items using the 5-point Likert scale
method (strongly agree = 5, agree = 4, neutral = 3, disagree = 2, strongly disagree
= 1). Each faculty member then received a score between 12 and 60. A high score
482
indicated a high level of awareness of AI, while a low score indicated a low level
of awareness. To verify the psychometric efficiency of the scale, the researchers
calculated the stability of the scale using Cronbach’s alpha coefficient. The
Cronbach’s alpha coefficient for cognitive awareness of AI was 0.915, and for the
performance awareness of AI, the Cronbach’s alpha coefficient was 0.940. The
Cronbach’s alpha coefficient for the two dimensions reached a value of 0.953.
The validity of the scale was also verified through the validity of its internal
consistency. The internal homogeneity of the scale’s 12 items was determined by
calculating the Pearson correlation coefficient between the degree of each item
and the dimension to which the item belongs, as well as the correlation
coefficient for the dimensions of the scale and the total score. The results of the
correlation coefficient between each statement and the total score of the
dimension to which it belongs.
The results indicated that each item had a positive correlation coefficient with its
corresponding dimension, which is statistically significant at the significance
level of 0.01 or less. This value demonstrated the reliability and validity of the
scale’s endogenous consistency (see Appendix 1, Table 1).
The results of the correlation coefficient between each statement and the total
score of the dimension to which it belongs showed that all correlation
coefficients were statistically significant at the level of 0.01, which indicated the
internal consistency of the scale and its validity for application (see Appendix 2,
Table 2).
2.3.2. Digital Competence Scale (DCS)

The researchers designed a 5-point scale to measure the level of DCs among
faculty members in the College of Education by consulting the educational
literature and referring to several scales related to DCs. The scale was presented
to a group of experts and specialists in educational technologies, curricula, and
teaching methods. To ensure the scale’s validity, the arbitrators were asked to
express their opinions on the scale in terms of the items’ appropriateness, their
clarity, the soundness of their linguistic formulation, and any other observations.
The researcher considered the consensus of 85% of the arbitrators (seven out of
eight) sufficient to deem the item acceptable. According to the arbitrators’
feedback, five items were deleted, three items were added, and the linguistic
formulations of seven items were modified. After making the necessary
modifications based on the observations of the arbitrators, the scale consisted of
14 items measuring three dimensions: technical knowledge competencies,
technical performance competencies, and technical production competencies.
method. Then, each faculty member received a score between 14 and 70. A high
score indicated a high level of DCs, while a low score indicated a low level of
DCs. To verify the psychometric efficiency of the scale, the researchers
calculated the stability of the scale using Cronbach’s alpha coefficient. The
Cronbach’s alpha coefficients for technical cognitive, technical performance, and
technical productivity competencies were 0.862, 0.888, and 0.860, respectively.
The Cronbach’s alpha coefficient for all dimensions was 0.938. In addition, the
483
validity of the scale was verified through the validity of its internal consistency.
The internal homogeneity of the scale’s 14 items was determined by calculating
the Pearson correlation coefficient between the degree of each item and the
dimension to which the item belongs, as well as the correlation coefficient for the
dimensions of the scale and the total score.
score of the dimension to which it belongs indicated that each item had a
positive correlation coefficient with its corresponding dimension, which was
statistically significant at a level of 0.01 or less. This value demonstrates the
reliability and validity of the scale’s endogenous consistency (see Appendix 3,
Table 3).
The results of the correlation coefficient between the dimensions of the scale and
the total score showed that all correlation coefficients were statistically
significant at the level of 0.01, which indicated the internal consistency of the
scale and its validity for application (see Appendix 4, Table 4).
2.3.3. Technological Acceptance Scale (TAS)

The researchers designed a 5-point Likert scale to measure the level of TA
among faculty members in the College of Education by looking at the
educational literature and referring to several measures related to TA. The scale
was presented to experts in educational technologies in order to ensure its
validity. The arbitrators were asked to express their opinions on the scale in
terms of the items’ appropriateness, their clarity, the soundness of their
linguistic formulation, and any other observations. The researcher considered
the consensus of 85% of the arbitrators (six out of seven) sufficient to deem the
item acceptable. Two phrases were deleted, one phrase was added, and the
wording of five phrases was modified. After making the necessary
modifications, the scale comprised 16 items measuring three dimensions: ease of
use, expected benefit, and satisfaction and attitude toward use.
method. Then, each faculty member received a score between 16 and 80, with a
high score indicating a high level of TA and a low score indicating a low level of
TA. To verify the psychometric efficiency of the scale, the researchers calculated
the stability of the scale according to Cronbach’s alpha coefficient. The
Cronbach's alpha coefficients for ease of use, expected benefit, and satisfaction
and attitude toward use were 0.814, 0.900, and 0.895, respectively. The
Cronbach’s alpha coefficient for all dimensions was 0.931. In addition, the
validity of the scale was verified; the internal homogeneity of the scale’s 16 items
was determined by calculating the Pearson correlation coefficient between the
degree of each item and the dimension to which the item belongs, as well as
between the dimensions of the scale and the total score.
score of the dimension to which it belongs indicated that each item had a
positive correlation coefficient with its corresponding dimension, which is
statistically significant at a significance level of 0.01 or less. This value
484
demonstrated the reliability and validity of the scale's endogenous consistency

(see Appendix 5, Table 5).
The results of the correlation coefficient between the dimension of the scale and
the total score showed that all correlation coefficients were statistically
significant at the level of 0.01, demonstrating the internal consistency of the scale
and its validity for application (see Appendix 6, Table 6).
3. Results
3.1. Results for research question 1
The first research question asked: what level of awareness do faculty members
at King Faisal University have about AI?
The mean and standard deviation of the survey responses were calculated. The
results are shown in Table 7.
Table 7. Descriptive statistics of the respondents’ level of cognitive awareness related
to AI (n = 101)
Std. Deviation
Disagree
Disagree
Strongly
Strongly
Neutral
Agree
Agree
Mean
Rank
No. Item
I have sufficient knowledge Freq. 10 37 30 23 1

1 of AI programs and 3.32 0.97 1
applications. % 9.9 36.6 29.7 22.8 1
I have sufficient knowledge Freq. 11 37 27 23 3

of the roles of the faculty
2 3.3 1.03 2
member in the application of
% 10.9 36.6 26.7 22.8 3
AI in teaching.
I have knowledge of the Freq. 9 34 26 22 10

basics of designing and
3 implementing lessons using 3.01 1.14 4
AI programs and % 8.9 33.7 25.7 21.8 9.9
applications.
I have knowledge of Freq. 11 27 30 27 6

methods of evaluating
4 student assignments 3.1 1.1 3
implemented by AI % 10.9 26.7 29.7 26.7 5.9
applications.
Mean* for total 3.2 0.95
According to Table 7, the general mean for the first dimension was 3.20, with a
standard deviation of 0.95. These values indicated that the faculty members had
a medium level of cognitive awareness of AI. Item 1, “I have sufficient
knowledge of AI programs and applications,” was ranked the highest, with a
mean score of 3.32 and a standard deviation of 0.97. Item 3, “I have knowledge
485
of the basics of designing and implementing lessons using AI programs and

applications,” was ranked the lowest, with a mean of 3.01 and a standard
deviation of 1.14.
Figure 2: The means of cognitive awareness related to AI
Table 8. Descriptive statistics of the respondents’ level of performance awareness

related to AI (n = 101)
Std. Deviation
Disagree
Disagree
Strongly
Strongly
Neutral
Agree
Agree
Mean
Rank
No. Item
I can create various files Freq. 12 32 26 26 5

5 3.2 1.1 1
using AI applications. % 11.9 31.7 25.7 25.7 5
Freq. 9 31 21 33 7
I can design course content
6 3.02 1.13 4
using AI applications.
% 8.9 30.7 20.8 32.7 6.9
Freq. 9 29 28 30 5
I can summarize long texts
7 3.07 1.07 2
using AI applications.
% 8.9 28.7 27.7 29.7 5
I can respond to students’ Freq. 12 29 19 31 10

8 inquiries through the use of 3.02 1.22 5
chatbots. % 11.9 28.7 18.8 30.7 9.9
I can convert written texts in Freq. 8 25 24 34 10
the course into audio files
9 2.87 1.14 7
using AI (sound making) % 7.9 24.8 23.8 33.7 9.9
applications.
I can use AI applications Freq. 6 31 25 26 13
10 based on suspense, 2.91 1.15 6
challenge, and scientific % 5.9 30.7 24.8 25.7 12.9
486
competition in the
educational process.
I can convert printed images Freq. 7 32 25 31 6

11 or text into editable text files 3.03 1.072 3
using AI applications. % 6.9 31.7 24.8 30.7 5.9
I can turn written texts into Freq. 4 21 23 42 11
12 educational films using AI 2.65 1.05 8
applications. % 4 20.8 22.8 41.6 10.9
Mean* for total 2.97 0.94
Based on Table 8, the general mean for the performance awareness dimension
was 2.97, with a standard deviation of 0.94. These values indicated that the
faculty members perceived themselves as having a medium level of
performance awareness of AI. Item 5, “I can create various files using AI
applications,” was ranked the highest, with a mean score of 3.20 and a standard
deviation of 1.10. Item 7, “I can summarize long texts using AI applications,”
ranked second, with a mean score of 3.07 and a standard deviation of 1.07.
Item 9, “I can convert written texts in the course into audio files using AI (sound
making) applications,” ranked seventh, with a mean score of 2.87 and a standard
deviation of 1.14. Finally, item 12, “I can turn written texts into educational films
using AI applications,” was ranked the lowest, with a mean score of 2.65 and a
standard deviation of 1.05.
Figure 3: The means of performance awareness related to AI
Table 9. The overall results of the study sample’s level of awareness related to AI (n = 101)
Dimensions Mean Std. Deviation Rank
Level of cognitive
3.20 0.95 1
awareness related to AI
Level of performance
2.97 0.94 2
awareness related to AI
Overall score 3.05 0.89
487
According to Table 9, the general mean of the scale was 3.05, with a standard
deviation of 0.89. These values indicated a medium degree of awareness of AI
among the participants. The first dimension, cognitive awareness of AI, was
ranked higher, with a mean score of 3.20 and a standard deviation of 0.95. The
second dimension, performance awareness of AI, was ranked lower, with a
mean score of 2.97 and a standard deviation of 0.94.
Figure 4: The means of awareness related to AI

The second research question asked: what is the correlation between the level
of awareness of AI and TA among faculty members at King Faisal University?
The Pearson correlation coefficient was calculated between the respondents’

scores on the AIAS and their scores on the TAS. The results are shown in Table
10.
Table 10. Pearson correlation coefficient between the respondents’ scores on the two scales
TA
AI Pearson correlation coefficient 0.139
Sig. 0.165
No. 101
According to Table 10, the value of the correlation coefficient between the
faculty members’ scores on the AIAS and the TAS was 0.139. The significance
value was 0.165, which is greater than 0.05, indicating that there is no statistical
significance between AI awareness and TA among the study sample.
488
Figure 5: The relationship between AI and TA

The third question asked: what is the correlation between the level of
awareness of AI and DCs among faculty members at King Faisal University?
The results are shown in Table 11.
Table 11. Pearson correlation coefficient between the respondents’ scores on the two scales
DCs
AI Pearson correlation coefficient 0.568**
Sig. <0.001
No. 101
**p<0.01
Based on Table 11, the correlation coefficient between the faculty members’
scores on the AIAS and the DCS was 0.568, which is a positive value and
indicates a positive direct correlation. The correlation is statistically significant,
as the significance value was less than 0.001. Therefore, AI awareness is
significantly and directly positively correlated with DCs among the study
sample.
489
Figure 6: The relationship between AI and DCs

4. Discussion
The findings of the study showed that the study sample had a medium level of
awareness with regard to AI. This is perhaps due to the belief that the
educational process depends on the main components of people and value and
that technology is not an end in itself but rather a means. The results of this
study align with the results obtained by Al-Subhy (2020), who found that there
are several challenges to the use of AI in teaching, so the use of AI by faculty
members occurs at a very low rate. It also aligns with the studies conducted by
Badawi (2022) and Jantakun et al. (2021), where the two studies concluded that
there are a set of challenges facing the application of AI on university campuses,
including the need to train faculty and develop their skills to be compatible with
AI. The researchers attribute the average level of AI awareness among the
current study sample to a lack of self-efficacy, the presence of some challenges
preventing the use of AI, and the sample’s lack of experience in integrating AI
techniques in teaching. These findings contrast with the study conducted by Al-
Darayseh (2023), in which the study sample had a high level of acceptance
concerning the use of AI as well as information skills about the basics of
integrating AI technologies in teaching. The findings of this study also
contradict Jabali and Al-Qahtani’s (2022) findings, which indicated a high level
of awareness of AI skills among faculty members. They also found that teaching
experiences and training courses had no statistically significant effect on AI
awareness.
The researchers suggest that the sample might have an average level of AI
awareness because of a scarcity of forums and seminars introducing what AI is
and how to use it in the teaching and learning processes, as well as a lack of
programs, courses, and workshops for training faculty members on AI
applications. The lack of awareness might also be due to the high cost of most AI
490
applications related to the field of teaching. Furthermore, there are few Arab
educational websites specializing in AI applications in the field of education.
The results also showed that there is no statistically significant correlation

between AI awareness and TA. TA is a general concept that may be associated
with technologies other than AI, such as the use of social media, office programs,
graphic design programs, chat, Zoom, Blackboard, and other university systems
that do not include AI. The novelty of the term AI and the lack of awareness of
its concept and applications among university faculty, especially members in the
humanities, may additionally contribute to the medium degree of AI awareness.
This finding aligns with Teo’s (2009) finding that technological complexity
affects behavioral acceptance indirectly. Finally, the results of the study showed
that there is a direct positive correlation between AI awareness and DCs. The
application of AI technology poses some concern to faculty members; however,
Wisskir et al. (2017) noted that young people in developing countries are
optimistic about their future careers and have confidence in their capabilities for
professional development related to AI and their technological skills. Therefore,
faculty members must understand that developing DCs enables them to utilize
the full value of AI in education.
5. Recommendations
Considering the results of this study, the researchers recommend holding
seminars to prepare faculty members to use AI in education and improve their
attitudes toward AI technologies. Training courses and workshops should also
be held for faculty members to learn about what is new in the field of AI
applications and provide them with skills to employ in the educational
environment. Incentives can additionally be offered for faculty members who
use AI in the teaching and learning environment. Universities should be
equipped with the necessary devices to employ AI in education, and they should
conduct more scientific studies aimed at raising the awareness of faculty
members concerning the importance of applying AI and DCs in the teaching
process.
6. Conclusions
Based on the results of the current study, which confirm that the participants
possess a medium degree of awareness of AI, this is due to the fact that
applications of AI in higher education need digital infrastructure and encourage
faculty members to enthusiastically integrate into the AI revolution. Also, the
use of AI by faculty members came at a very low level as a result of fear of AI
and the consequent negative behaviors and practices that are related to the
ethics of scientific research, in addition to the material cost, ignorance, and lack
of knowledge of using AI in teaching and scientific research. Also, there is a gap
between AI and humans, who seem to lack self-awareness. The results also
showed that there is a direct positive correlation between AI and DCs among the
study sample. The researchers of the current study attribute this to the
participants’ understanding of the importance of AI for development, vocational
education, and the positive impact of applying AI in education.
491
Acknowledgments
The authors gratefully acknowledge the financial assistance from King Faisal
University in Saudi Arabia (Grant Number GRANT 3833). The authors are
thankful to all participants who agreed to participate in this study.
7. References
Abdul Rasul, F. (2015). Modern Trends in University Education. Joanna House.
Ahmed, H. (2018). Cloud computing advantages and disadvantages. International Journal
of Scientific Engineering and Applied Science, 4(4), 31-38.
https://ijseas.com/volume4/v4i4/ijseas20180405.pdf
Ahmed, I. (2020). Challenges of digital education in the Arab world (Original Vision). The
Arab Journal of Specific Education. 4(12), 91-108.
http://doi.org/10.33850/ejev.2020.73456
AIED. (2023). AI in Education for Sustainable Society. https://cutt.us/feArZ
Akgün, Ö. (2020). Digital competence: the vital 21st-century skill for teachers and students.
https://cutt.us/rX7FL
Al-Alawi, Y., Al-Saqri, M., & Al-Harassi, N. (2014, March 25-27). Electronic information,
SLA-AGC 20th Annual Conference Doha, Qatar.
http://doi.org/10.5339/qproc.2014.gsla.10
Al-Alimat, p. (2012). The degree of Arabic language teachers' possession of the higher
basic stage in Qasaba Al-Salt educational technology competencies from their
point of view. Studies Journal University of Jordan, 40 (2), 1511-1522.
Al-Darayseh, A. (2023). Acceptance of artificial intelligence in teaching science: Science
teachers' perspective. Computers and Education: Artificial Intelligence, 4,
http://doi.org/10.1016/j.caeai.2023.100132
Al-Ghamdi, A. (2019). Technical competencies of teachers of Islamic education in schools
of the future from the viewpoint of specialists. Pedagogical Pedagogy, 8(4). 113 - 136.
Al-Khafaji, R., Assi, A., & Mohammed, S. (2021). Modern technology, therapeutic approaches,
and educational communication strategies. Nour Al-Hassan Library.
Al-Khathami, M. (2010). A case study for faculty members at the college of computer and
electronic information at Muhammad bin Saud Islamic university in Riyadh.
Journal of the King Fahd National Library, 16(1): 113-130. https://cutt.us/xYMCu
Al-Mahdi, p. (2021). Education and future challenges in the light of the philosophy of artificial
intelligence. https://cutt.us/3i3XP
Al-Subhy, A. (2020). The reality of artificial intelligence applications use in education by
faculty members at Najran University. Journal of the Faculty of Education, Ain Shams
University, 4 (44), 319-368.
Amayreh, M. (2019). The degree of availability of technological competencies for faculty members
in Jordanian universities and their obstacles [Unpublished Master Thesis]. Middle East
University.
Badawi, M. (2022). Applications of artificial intelligence in education: Challenges and
future prospects. Journal of the Egyptian Society for Educational Computers, 10 (2), 91-
108. 10.21608/eaec.2022.155591.1101
Becker, S., Cummins, M., Davis, A., Freeman, A., Hall, C., & Ananthanarayanan, V.
(2017). NMC Horizon Report: 2017 Higher Education Edition. The New Media
Consortium. https://www.learntechlib.org/p/174879/
Caena, F., & Redecker, C. (2019). Aligning teacher competence frameworks to 21st
century challenges: the case for the European digital competence framework for
educators (Digcompedu). European Journal of Education, 54(3), 356-369.
http://doi.org/10.1111/ejed.12345
492
Carretero, S., Vuorikari, R., & Punie, Y. (2017). DigComp 2.1: The digital competence
framework for citizens. Publications Office of the European Union.
http://doi.org/10.2760/38842
Chiu, T., Chu, D., Huang, S., Chang, M., Liu, Y., & Lee, J. (2021). Facing the coronavirus
pandemic: An integrated continuing education program in Taiwan. International
Journal of Environmental Research and Public Health, 18(5), 2417.
http://doi.org/10.3390/ijerph18052417
Chowdhury, M., & Sadek, A. (2012). Advantages and limitations of artificial intelligence.
Artificial intelligence applications to critical transportation issues, 6(3), 360-375.
Davis, F., Bagozzi, R., & Warshaw, P. (1989). User acceptance of computer technology: A
comparison of two theoretical models. Management Science, 35(8), 982–1003.
http://doi.org/10.1287/mnsc.35.8.982
European Commission. (2006). Recommendation of the European parliament and of the
council of 18 December 2006 on key competences for lifelong learning. Official
Journal of the European Union, 394. https://eur-lex.europa.eu/eli/reco/2006/962/oj
European commission.(2023). Digital competence framework for educators (DigCompEdu).
https://cutt.us/3aQ9Q
Fathema, N., & Sutton, K. (2013). Factors influencing faculty members’ Learning
Management Systems adoption behavior: An analysis using the Technology
Acceptance Model. International Journal of Trends in Economics Management &
Technology, 2(6), 20-28.
Haenlein, M., & Kaplan, A. (2019). A brief history of artificial intelligence: On the past,
present, and future of artificial intelligence. California management review, 61(4), 5-
14.
Haleem, A., Javaid, M., Qadri, M., & Suman, R. (2022). Understanding the role of digital
technologies in education: A review. Sustainable Operations and Computers.
http://doi.org/10.1016/j.susoc.2022.05.004
Holmes, W., Bialik, M. & Fadel, C. (2019). Artificial intelligence in education: Promises
and implications for teaching and learning. Center for Curriculum Redesign
Holmes, W., Persson, J., Chounta, I., Wasson, B., & Dimitrova, V. (2022). Artificial
intelligence and education: A critical view through the lens of human rights, democracy
and the rule of law. Council of Europe.
Ilkka, T. (2018). The impact of artificial intelligence on learning, teaching, and education.
European Union.
Jabali, N., & Al-Qahtani, S. (2022). The degree of awareness among faculty members of
artificial intelligence skills in education and its relationship to experience and
training programs at King Khalid University. Journal of the Association of Arab
Universities for Education and Psychology, 19 (3), 92-131
Jantakun, T., Jantakun, K., & Jantakoon, T. (2021). A common framework for artificial
intelligence in higher education (AAI-HE Mode). International Education Studies,
14(11), 94-103. http://doi.org/10.5539/ies.v14n11p94
Jeong, H. (2011). An investigation of user perceptions and behavioral intentions towards
the e-library. Library Collections, Acquisitions, and Technical Services, 35(2-3), 45-60.
http://doi.org/10.1016/j.lcats.2011.03.018
Kaplan, A., & Haenlein, M. (2019). Siri, Siri, in my hand: Who’s the fairest in the land? On
the interpretations, illustrations, and implications of artificial intelligence. Business
horizons, 62(1), 15-25. http://doi.org/10.1016/j.bushor.2018.08.00
Kleef, N., Noltes, J & Spoel, S. (2010). Success factors for Augmented Reality Business Models.
Study Tour Pixel 2010 - University of Twente.
493
Krumsvik, R. (2008). The emerging digital literacy among teachers in Norway (The story
of one digital literate teacher). In R. Kobayashi (Ed.), new educational technology (pp.
105–155). Nova Science Publishers.
Leslie, D., Burr, C., Aitken, M., Cowls, J., Katell, M., & Briggs, M. (2021). Artificial
intelligence, human rights, democracy, and the rule of law: a primer. arXiv preprint
arXiv: 2104.04147. http://doi.org/10.2139/ssrn.3817999
Luckin, R. (2017). Towards artificial intelligence-based assessment systems. Nature
Human Behaviour, 1(3), 0028. https://doi.org/10.1038/s41562-016-0028
Luo, D. (2018). Guide teaching system based on artificial intelligence. Journal of Emerging
Technologies in Learning. 13(8), 90-102. https://doi.org/10.3991/ijet.v13i08.9058
McCarthy, J., Minsky, M. L., Rochester, N., & Shannon, C. (2006). A proposal for the
Dartmouth summer research project on artificial intelligence, august 31, 1955. AI
magazine, 27(4), 12-14. https://doi.org/10.1609/aimag.v27i4.1904
Miao, F., Holmes, W., Huang, R., & Zhang, H. (2021). AI and education: A guidance for
policymakers. UNESCO Publishing.
Moawad, G. (2019). The effectiveness of a diffuse training environment based on the
preferred training style for developing digital competencies and technological
acceptance among faculty members at the University of Education, Al-Azhar
University, 3 (184), 1086-1147. https://doi.org/ 10.21608/jsrep.2019.95899
Nadimpalli, M. (2017). Artificial intelligence risks and benefits. International Journal of
Innovative Research in Science, Engineering and Technology, 6(6).
https://www.researchgate.net/profile/MeenakshiNadimpalli/publication/31932
1806_Artificial_Intelligence_Risks_and_Benefits/links/59a47dd20f7e9b4f7df37ab9
/Artificial-Intelligence-Risks-and-Benefits.pdf
Nazaretsky, T., Cukurova, M., Ariely, M., & Alexandron, G. (2021, September).
Confirmation bias and trust: human factors that influence teachers' attitudes
towards AI-based educational technology. In CEUR Workshop Proceedings (Vol.
3042).
Ng, D., Leung, J., Su, J., Ng, R., & Chu, S. (2023). Teachers’ AI digital competencies and
twenty-first century skills in the post-pandemic world. Educational technology
research and development, 71, 137-161. https://doi.org/ 10.1007/s11423-023-10203-6
Organization for Economic Co-operation and Development. (2020). Trustworthy AI in
education: promises and challenges, background paper for the g20 AI dialogue, digital
economy task force meeting Riyadh, 1-2 February 2020.
https://www.oecd.org/education/trustworthy-artificial-intelligence-in-
education.pdf.
Shaheen, Sh. Al-Awadi, H. Al-Shammari, T. (2021). Competences of e-learning among
teachers of the Arabic language for the fifth grade of primary school, towards their
students towards the subject. Journal of Human Sciences, 28 (3), 1-15.
Sharaf El-Din, M. (2023). The technical competencies of a faculty member at the Faculty
of Physical Education in the light of the highway towards creativity. Beni Suef
Journal of Education and Mathematical Sciences, 6(11), 161-
https://doi.org/17910.21608/obsa.2023.169626.1379
Tecuci. G. (2012). Artificial intelligence. WIREs Computational Statistics, 4(2), 168-180.
Teo, T. (2009). Modelling technology acceptance in education: A study of pre-service
teachers. Computers & Education, 52(2), 302-312, https://doi.org/
10.1016/j.compedu.2008.08.006
The European Commission. (2023). Defines digital competence. https://joint-research-
centre.ec.europa.eu/digcomp/digcomp-framework_en
494
Tutorial point. (2015). Artificial intelligence.

https://www.dcpehvpm.org/EContent/BCA/BCAIII/artificial_intelligence_tutor
ial.pdf.
UNESCO. (2021). Recommendation on the ethics of artificial intelligence.
https://unesdoc.unesco.org/ark:/48223/pf0000380455
UNICEF. (2021), Policy guidance on AI for children. https://cutt.us/mIvdv
United Nations. (2023). Do you know all 17 SDGs? https://sdgs.un.org/goals
Wisskirchen, G., Biacabe, B., Bormann, U., & Intelligence, A. (2017). Robotics and Their
Impact on the Workplace, IBA Global Employment Institute .https://cutt.us/FNpsh
Zawacki-Richter, O., Marín, V., & Bond, M. et al. (2019). Systematic review of research on
artificial intelligence applications in higher education – where are the
educators?. International Journal of Educational Technology in Higher Education, 16, 39.
https://doi.org/10.1186/s41239-019-0171-0
Zhang, K., & Aslan, A. (2021). AI technologies for education: Recent research & future
directions. Computers and Education: Artificial Intelligence, 2,
Zhong, Y. (2006). A cognitive approach to artificial intelligence research. In 2006 5th IEEE
International Conference on Cognitive Informatics (Vol. 1, pp. 90-100).
https://doi.org/10.1109/COGINF.2006.365682
Appendix 1
Table 1. The results of the correlation coefficient between each statement and the total
score of the dimension to which it belongs
Correlation
Dimension No. Sig.
coefficient
1 0.861** < 0.001
Cognitive 2 0.886** < 0.001

awareness of AI 3 0.933** < 0.001
4 0.894** < 0.001
5 0.818** < 0.001
6 0.907** < 0.001
Performance
7 0.826** < 0.001
awareness of AI
8 0.783** < 0.001
9 0.872** < 0.001
10 0.886** < 0.001
11 0.787** < 0.001
12 0.837** < 0.001
**p<0.01
495
Appendix 2
Dimension Correlation coefficient Sig.

Cognitive awareness of AI 0.895** < 0.001
Performance awareness of AI 0.974** < 0.001
**p<0.01
Appendix 3
Correlation
Dimension No. Sig.
coefficient
1 0.815** < 0.001
Technical knowledge 2 0.783** < 0.001

competencies 3 0.878** < 0.001
4 0.904** < 0.001
5 0.769** < 0.001
6 0.855** < 0.001
Technical performance 7 0.819** < 0.001

competencies 8 0.870** < 0.001
9 0.750** < 0.001
10 0.803** < 0.001
11 0.775** < 0.001
Technical productivity 12 0.877** < 0.001

competencies 13 0.890** < 0.001
14 0.824** < 0.001
**p<0.01
Appendix 4
Table 4. The results of the correlation coefficient between the dimensions of the scale
and the total score

Technical knowledge competencies 0.896** < 0.001
Technical performance competencies 0.924** < 0.001
Technical productivity competencies 0.892** < 0.001
**p<0.01
496
Appendix 5
Correlation
Dimension No. Sig.
coefficient
1 0.744** < 0.001
2 0.788** < 0.001
Ease of use
3 0.882** < 0.001
4 0.813** < 0.001
5 0.769** < 0.001
6 0.769** < 0.001
7 0.778** < 0.001
Expected 8 0.823** < 0.001

benefit 9 0.760** < 0.001
10 0.812** < 0.001
11 0.782** < 0.001
12 0.721** < 0.001
13 0.848** < 0.001
Satisfaction 14 0.911** < 0.001
and attitude
toward use 15 0.929** < 0.001
16 0.805** < 0.001
**p<0.01
Appendix 6
Table 6. The results of the correlation coefficient between the dimension of the scale
and the total score

Ease of use 0.836** < 0.001
Expected benefit 0.937** < 0.001
Satisfaction and attitude towards use 0.798** < 0.001
**p<0.01
497

Vol. 22, No. 7, pp. 497-516, July 2023
Enhancing Reading-Comprehension Abilities

and Attitudes of EFL Students through utilising
Content-Creation Tools in Classroom
Presentations
Pattama Panyasai*
Loei Rajabhat University
Loei, Thailand
Abstract. Utilisation of content-creating tools in an environment that

enhances students' reading comprehension also promotes the integration
of language skills and technology for educational purposes. This study
aimed to enhance the reading comprehension skills of Business English
students in Thailand by incorporating content-creation tools to support
their English-language learning. Additionally, it sought to investigate
the students' attitudes towards developing their reading comprehension
through the utilisation of such tools. This study employed a quantitative
research design; and it involved 27 third-year students in a Business-
English program. Various instruments, including English reading
proficiency and comprehension tests for each unit, as well as content-
creation tools, assessment forms for students’ presentations and
questionnaires to gauge students’ attitudes were utilised. Quantitative
data were analysed by using the mean percentage, and the paired t-test.
The results demonstrated a statistically significant improvement in
students’ reading-comprehension abilities after using content-creation
tools for their presentations, with a significance level of two-tails less
than 0.05. This finding highlights the significant effectiveness of
developing students’ reading comprehension skills through
presentations by using content-creation tools for reading
comprehension, which met the pre-determined criterion of 70%.
Moreover, the students displayed a high level of positive attitudes
towards learning English and utilising content-creation tools to develop
their reading comprehension skills through presentations. These results
suggest that the integration of language skills and technology through
content-creation tools creates a stimulating learning environment,
fostering positive outcomes in reading comprehension.
Keywords: reading comprehension; content creation tool; business

English; presentation
*
Corresponding author: Pattama Panyasai, pattama.pan@lru.ac.th
©Authors
498
1. Introduction
More than a decade ago 21st century skills trends emerged in Thailand’s
educational system. Essential skills, such as information and technological work
skills are still necessary to develop students’ learning skills; and the use of
innovation to support their education and to better prepare them for future
careers (Lavi et al., 2021). In addition to the current situation, the social, economic,
and cultural aspects of the world have been affected by globalisation, including
the emergence of an era in which people can easily and quickly access
information. Furthermore, the impact of globalisation on social issues has also
made information more easily accessible in social, economic, and cultural areas.
Students require good reading skills to navigate and understand the large
amount of information available due to this accessibility (Singh et al., 2021).
In English-language teaching, 21st century skills are relevant skills. in which
English teachers should have a well-planned class, in order to provide courses
focusing on practical learning and promoting students’ language abilities
(Ahmadi & Reza, 2018; Newton et al., 2018). In addition, the instructors may need
to accept more responsibilities as mentors and to design more constructive
activities that use technology for enhancing the students’ learning, in order to
solve problems and to produce creative works (Boettcher & Conrad, 2021). The
students require more practices to perform their work or projects appropriated.
Furthermore, the students may be able to create content in which they are
interested; and they can then present it in more more creative ways. Thus, the
students’ projects may not only be shown in the classroom; but they could be
launched through the social-media sites. Performing the students’ content to
others with similar interests, then it would be beneficial for the students to have
a new environment of interactions and discussions. Moreover, it could be an
opportunity for the students to improve their skills accidentally; since it allows
the students to accept their abilities and to learn to improve their work quality
willingly.
Digital-Content Creation (DCC) tools offer a wide variety of features to help the
content creator to plan and manage activities, based on their interests and
preferences. It should be used in the classroom appropriately under teachers’
instructions. According to Puggioni, et. al. (2020), technology has been shown to
be an effective tool in education; because, for students and teachers, it allows for
the simplicity of teaching methods and the improvement of performance by
introducing affordable ways to distribute digital content. With the increase of
technology, students are more comfortable to use digital tools to support their
communicative competence and to learn to present themselves in difference
ways. Following Skrbiš & Laughland‐Booÿ (2019), they point out that people
begin to find ways to express themselves differently, resulting in emerging new
careers, such as Youtubers, online sellers, etc. People rely on online channels to
drive their activities, thereby generating a massive income without entering the
workplace, or having a store to display their products. Thus, content that
students would learn in the classroom should be met with the learner’s needs
and practical application. Panyasai and Paladkong (2019) state that it is relevant
to enable learners to concern regarding the importance of English because they
can apply their language skills learned to apply in their future career. Therefore,
English instructors should focus on the contents. that are in accordance with the
499
students’ needs; and they should attempt to use integrated teaching methods to
provide an effective learning environment in the course alongside the needs of
the learners through the activities, teaching materials, or other facilities.
Drawing on to Business English Reading teaching, this course focuses on
studying the structural patterns of business writing by using essential reading
skills, such as setting the reading objectives, questioning to motivate to read,
capturing business issues, which are consistent with business context, and
enhancing students’ critical thinking skills. The students would learn to present
their knowledge and to practise their communication skills through their
assignments (e.g., answering questions, discussions, and presentations). In
learning and the innovation skills of learning in the 21st century, communication
skills are considered to be an essential skill for learning in the 21st century,
particularly speaking skills, in which students would be judged most in real-life
situations (Parveen, 2016; Brammametta, 2019). In addition, the students should
not learn only how to use language, but also to learn how to apply technology in
their information presentation. Integrating technology in the classroom could
also facilitate the students in learning skills, such as searching for information,
and they ould also learn how to select appropriate information by themselves
(Cucckle et al., 2000). The students, therefore, have received better opportunities
to practice their learning skills and perform a variety of interesting tasks in many
related areas of their learning contents (National Research Council, 2012).
While the use of technology in teaching and learning is still increasing its interest,
creating content to present students’ tasks through online and offline, has become
widespread (Alenezi, 2020; Jayakumar et al., 2022). There are many interesting
programs that could support learners to produce their learning content and share
through presentations, for example https://quizizz.com/, https://socrative.com/,
https://www.canva.com/, and other websites that students can access free of
charge some websites to use and promote the use of knowledge-based content
that they could share online, either in poster form, infographic, or by creating
social-media memes. In addition, using the content-creation tools, it could
provide some advantages in learning languages, for example, by encouraging
learners to use English creatively and to select different ways in practising
English (Chong, 2018).
The emphasis on 21st-century skills in the Thai educational system is expanding,
in response to the changing demands of education and the impact of
globalisation. English language competency has been highlighted as a significant
area of priority in the nation's educational development goals, according to
Thailand's Office of the Education Council (OEC). To increase students'
competitiveness in the international employment market, the OEC has
underlined the necessity to improve English language proficiency (OEC, 2021).
The Thai government has put in place several measures to improve English
language teaching and learning, in order to meet these objectives. For instance,
the English as a Medium of Instruction (EMI) method has been implemented in
a few topics by the Ministry of Education to improve English language
instruction in schools (Ministry of Education, 2021). These initiatives seek to give
students the English language proficiency they need to interact with others to
access the global information, and operate in a global workforce.
500
English language teaching is no exception, with a focus on practical learning and

the incorporation of technology to enhance students' language abilities. To
effectively teach Business English Reading, it is essential to foster essential
reading skills, critical thinking, and communication skills through assignments
and presentations (Erdoğan, 2019). Integrating technology in the classroom
facilitates language learning, information searching, and the selection of
appropriate content. By utilising content-creation tools, students can produce
and share their learning content effectively. This approach aligns with the
demands of the 21st century and could empower students to take an active role
in the learning process. The integration of technology in language classrooms
requires attention from teachers, in order to meet students' needs and to enhance
their motivation and engagement. This research aims to explore how the
integration of technology, specifically through creative content presentation
tools, could enhance reading-comprehension skills and improve students'
presentation abilities in Business English. Specifically, investigating whether the
Content-Creation Tools in presentations could enhance the EFL students' reading
comprehension abilities; and the attitudes of BE students towards presentations
by using content-creative tools in developing their reading comprehension. The
findings of this study could contribute to the field of English language
instruction, could inform teachers' professional development, and could provide
insights for further research in integrating technology in language learning.
2. The Literature Review
2.1 Reading Comprehension Skills
Reading skills include the interplay between the learning skills and cognitive
processes of a reader, as well as the linguistic characteristics of a text. The reader
is required to co-ordinate textual information with existing knowledge, in order
to generate a concept of the text's meaning (Smith et al., 2021). Recent studies
have mentioned two components: word recognition and linguistic
comprehension, which explains a relative percentage of the various aspects in
reading comprehension (Hjetland, et al., 2020; Spiro et al., 2017; Hogan et al.,
2014; Gough & Juel, 2013). In the initial stages of learning to read, the word
recognition is a principal constraint on reading comprehension, but later, when
learners become more confident in word recognition, linguistic comprehension
also becomes a more crucial aspect in reading comprehension (Oakhill, 2019).
The nature of reading comprehension is defined as an effective process of reading
instructions (Duke & Pearson, 2009). Reading comprehension and instruction can
be balanced; and it ought to involve explicit instructions in particular
comprehensive strategies with an approximate time and chance for actual
practising (e.g., reading, writing, and discussion of learning content) (Bogaerds-
Hazenberg et al., 2021 Pearson, 2014; Carlisle et al., 2011). Regarding reading
strategies, the reading process generally includes three reading processes, which
are pre-reading, while-reading, and post-reading (Nazurty et al., 2019;
Banditvilai, 2020). Firstly, getting started with pre-reading strategies, this process
includes preparing a reading plan, eliciting the topic from the beginning of the
paragraph, predicting from the main titles and visualising information,
activating prior knowledge, deciding where to focus, and determining reading
speed. Secondly, while reading, strategies, in this process focus on making
connections between paragraphs, taking notes on the important points, re-
501
reading, high-lighting the main points, using a dictionary to learn vocabularies,

utilising figures, using textual clues, re-analysing when misunderstanding
information is encountered, and building links between previous knowledge and
new issues. Thirdly, post-reading strategies, as the final process: this process has
been pursued a reader to make summarising the text, checking the reading goals,
checking previous predictions, evaluating the main ideas of the text critically,
taking notes for future reference, and expressing the main ideas of the text.
Marzuki et al. (2018) suggested two other ways to assist students to improve their
reading skills: by using the cognitive reading strategies: the first way is to allow
the students to select a topic of reading which is related to their interests. This
way seems to support efficient and pleasant learning for the students, and
importantly, the students can better understand the topic by connecting what
they have recently learnt with their previous knowledge; the second way is
relevant to the instructional procedures of cognitive reading strategies, which are
resourcing, repetition, deduction, skimming and scanning, and so on.
In brief, reading strategies are the essential factors in developing reading
comprehension skills and they also have an impact on the ability of students’
reading comprehension. To develop the students’ reading comprehension, the
students should pay attention to the ways of reading, and to choose them
appropriately, in order to apply them to their reading practices (Manh & Le,
2021). Further, for the classroom instruction, the students should not be merely
the receivers of information, but also active makers of meaning. Reading
processes with different strategies can be used in the classroom to promote the
students’ active learning and to assist them to understand meaning from the text
(Lotfi & Siahpoosh, 2020).
2.2 Content Creation Tools
Content-creation tools can be defined as aids, which support users to render them
able to plan their work by using computer programs or websites to generate their
content in different ways (Amesi et al., 2022). The content-creation tools provide
easy-to-use functions for their users. On the other hand, users can reduce the
designing procedure to produce their content creation faster (Correa & Jeong,
2011; Liapis et al., 2013; Huotari et al., 2015). Due to the increase in the
digitalisation of teaching in recent years, the implication of digital content
creation tools in the classroom has been introduced, in order to enhance the
results of the teaching-learning process. This approach allows for the inclusion
of knowledge and the advancement of students' competencies (Antón-Sancho et
al., 2021; Thyagharajan & Nayak, 2007).
In the English language classroom, utilising different approaches to English
learning and content creation can provide great opportunities for teachers and
students to engage in a wider range of effective language-learning functions.
Also Wayegh at al. (2018) describe that in language classrooms, the challenge for
teachers is to use materials that assist language learning and the material used
can increase the issue of material design. For the content-creation tools, there are
many programs and websites that provide practical functions to utilise in the
creation of contents; and most of them are free. while their purposes are intended
to help the users to develop their contents for the social media channels, Youtube
channels, and so on, the content-creation tools also assist the students with their
502
tasks, such as designing quick graphic and searching for up-to-date topics to
write about. The content-creation tools can be distinguished for different
categories (Collins, 2022): video-content creation tools, content- research tools,
scheduling as well as planning tools, designing and visual-content-creation tools,
and quizzes, and survey tools. In addition, to selecting the content-creation tools
in the classroom, teachers should provide opportunities for students to select the
tools with which they are familiar and can use effectively without complicated
functions (Säljö, 2010; Ordu, 2021). As mentioned above, the utilisation of
content-creation tools in students’ language assignments and the subsequent
improvement in content quality in the language classroom should enable
students to engage in various specific actions. These actions include the selection
of appropriate content-creation tools, the integration of these tools into language
assignments, and the facilitation of a deeper understanding of technology, etc.
Microsoft PowerPoint is commonly used to create engaging presentations and to
support students' assignments (Darmayanti & Lisa, 2019). Online sites like
Canva.com offer user-friendly functions for tasks, such as creating mind-maps,
infographics, and presentation slides. Other recommended sites for classroom
presentations include those of Prezi, Powtoon, and Visme. Technology, including
platforms like TikTok, YouTube, Facebook, and TV broadcasting, provide
opportunities for enhancing English communication competence (Ustun, 2019;
Oktaviani & Mandasari, 2020; Pham & Li, 2022). According to Xiuwen & Razali
(2021) and Fansury et al. (2020), Digital tools, like content-creation tools, have
been investigated for improving reading-comprehension skills. Studies have
shown positive results, such as improved inferential question answering and
increased motivation (Bautista, 2020). However, the use of content-creation tools
for enhancing reading comprehension in EFL contexts remains limited. Most
studies utilise these tools as supportive resources for teachers in developing
teaching materials (Priyanti, 2019; Klimova & Zamborova, 2020; Copoiesis et al.,
2020; Azmuddin et al., 2020).
As given above, selecting appropriate tools also helps students gain more insight
into using technology to develop their language assignments and produce better-
performance content, so that they will become inspired in creating compelling
content in language classrooms. This current study aimed to examine the extent
of the use of content-creation tools that can enhance students’ language
assignments and facilitate the development of better-performing content in the
language classroom. By selecting the appropriate tools, moreover, students were
able to gain a deeper understanding of how technology can be utilised effectively
in their language- earning process.
2.3 Presentation skills
Conducting presentations in front of audiences in different contexts is relevant
to concern many factors, for example speaking fluently with confidence, dealing
with the audiences’ questions, dealing with anxiety, and so on. Al Harun et al.
(2016) argue that if the language used in the presentation is a foreign language,
it can be more challenging for a non-native speaker to have the effective
presentation in English.
In English language learning, conducting a presentation in the classrooms seems
to be a favourable way to encourage students to practise their presentation skills
503
together with learning all language-system areas (vocabulary, grammar, and

phonology) (Agustina, 2019). Panyasai (2022) also says that presentation skills
become more essential in the English class, particularly in English for specific
purposes. Within the courses of English for specific purposes, students are
required to enhance their presentation skills with a variety of learning contents
(i.e., tourism, hospitality, marketing, public relations). In addition, promoting
presentation tasks in the English-language class is necessary to increase the
students’ awareness in using other semiotic resources (i.e., non-verbal, and
visual) to support their speech in delivering presentations. Oktaviani and
Mandasari (2020) clarify that university students need some technological tools
including computer programs or websites to develop their presentations
productively.
Mulyadi et al. (2021) found limited research on how technology-enhanced
learning impacts ESP trainees’ speaking abilities. Their study utilised various
tools, including the TOEFL listening components, online presentations, role-play
activities, and online group discussions. The integration of technology improved
students’ role-playing speech. Meanwhile, Jung (2021) emphasised the
importance of motivating language learners. Interactive technology-enhanced
activities, such as collaborative project creation, can enhance motivation and
engagement. These studies highlight the need for more research in this area and
showcase the potential of technology to motivate and improve language learning
outcomes.
There is currently a lack of research evidence in exploring the impact of speaking
activities on enhancing reading comprehension. However, it can be deduced that
participating in speaking activities, particularly focusing on presentation skills,
holds significant importance in the field of education. Enhancing students'
proficiency in delivering oral presentations through knowledge, preparation,
technical expertise, and extensive practice becomes increasingly vital for their
educational advancement (Rao, 2019). Additionally, acquiring presentation skills
offers benefits, such as fostering self-directed learning and developing life-long
learning capabilities. These skills, in turn, provide students with practical
experiences that help them in their educational endeavours and future
professional development.
3. Methodology
3.1 Design of the study
The research approach chosen is an important decision in the research-design
process; since it determines how relevant information for a study will be
gathered. The goal of qualitative research could be multi-method, with an
interpretive, naturalistic approach to the subject (Williams, 2019). A quantitative
research design was used to investigate this study. The study was designed as
quasi-experimental research, with one-group pretest-posttest-only design. An
independent variable of this study was the teaching procedures used to develop
the students’ reading-comprehension skills through presentation by using
content creation tools. The dependent variables were the students' achievement
in reading comprehension and their attitudes towards teaching procedures.
504
3.2 Participants in the study

This current study was conducted at a government university located in Thailand
by deliberately selecting 27 third-year students who were in a Business English
program and enrolled in the Business English Reading. Perspectives from these
participants provided insights into their attitudes when using content-creation
tools to enhance their presentation in a Business-English course.
3.3 The Research instruments
The instruments used in this study include an English-reading proficiency test, a
reading comprehension test in each unit, content-creation tools (websites or
computer programs, such as, Microsoft PowerPoint, Canva.com, Prezi, Powtoon,
etc.), English students’ presentation assessment form, and the students’ attitude
questionnaires.
Firstly, six lesson plans were designed to be consistent with six different contents
in each unit. Here, the reading comprehension procedures, including pre-
reading, while-reading, and post-reading, were included in the lesson plans
involving the lists of content-creation tools that BE students could select and use
to support their presentations. The content-creation tools could be computer
software or free service charged from different websites that the BE students
were able to use effectively without complicated functions.
Secondly, English-reading proficiency test (30 questions), reading-comprehension
tests in each unit (60 questions, 10 questions for each unit), and English students’
presentation-assessment form were employed to investigate the BE students’
reading comprehension competence. In each English, students’ presentation an
assessment form was generated as the rubric scale, and they were checked for
assessment contents by three experts in the field of ELT. For the English-reading
proficiency test and reading comprehension test in each unit there were created
a test based on the reading comprehension evaluation framework by Bloom
(1956), which were divided by the level of reading comprehension into six levels,
but in the current study were focused on only the four levels of reading
comprehension, which were: 1) knowledge, 2) comprehension, 3) application,
and 4) analysis. The instruments were piloted to examine their reliability and
validity before conducting the pre-test, treatment, and post-test.
Both the English-reading proficiency test and the reading-comprehension test in
each unit were checked for the index of Item-Objective Congruence (IOC) by the
three ELT experts, as well. The difficulty level was tested (with the p value
between 0.20 - 0.80) and the discrimination power (with an r value of 0.20 and
above). The internal consistency of reliability was tested through Kuder-
Richardson Reliability Coefficient. The result shows that the English-reading
proficiency test had a highly reliable coefficient of 0.978. Likewise, the reading-
comprehension test in each unit also shows high reliability with a coefficient of
0.991.
Thirdly, the students’ attitude questionnaire towards developing their reading
comprehension by using content-creation tools were employed to determine the
BE students’ attitudes after using the tools. The questionnaire was in English,
with the Thai version including 15 statements. In order to assess the students’
attitudes, a Likert scale was utilised in the questionnaire. The Likert scale is a
505
commonly used rating scale that allows participants to express their agreement
or disagreement with a series of statements. In this case, the Likert scale ranged
from "totally disagree" to "totally agree" and consisted of five response options (1
= totally disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = totally agree). Regarding
the questionnaire's content and language accuracy, the author sought the
expertise of three experts in the field of English-Language Teaching (ELT) to
review and provide feedback on the questionnaire. In order to check the content
and accuracy of the language used in the questionnaire, the researcher asked for
the three experts who were in the field of ELT, to read through the questionnaire
to check the congruence between the provided statements and the research
objectives by using the Index of Item Objective (IOC) and give some
comments or feedbacks to the questionnaire items. This step ensures that the
content of the questionnaire aligns with the research objectives and that the
language used is appropriate, clear, and accurately reflects the constructs being
assessed. Expert feedback helps to enhance the validity and the reliability of the
questionnaire and to strengthen the overall quality of the study.
3.4 Data collection and analysis
This current study comprised the four phases of collecting the data. Phase one
involved the analysis of the course documents in using a documentary analysis
to gather the details regarding the course’s teaching components, such as the
course description and teaching plan. Phase two comprised the English-reading
proficiency test (pre-test), as it was given to the BE students to examine their
reading comprehension before beginning all the units. Phase three was
conducted to collect the data on BE students’ reading comprehension
competence after they had finished their presentations. Six reading
comprehension tests were assigned to 27 BE students. These tests were based on
the lesson contents; thus, the BE students had to do the tests involving ten
questions for each unit. During this phase, the students’ presentations, which
comprised the assignments requested in six lesson plans from six chapters:
Chapter 1 Ways of Reading, Way to Read, Chapter 2 Business News, Chapter 3
Company Performance, Chapter 4 Rules and Regulations, Chapter 5 Instruction
Manual, and Chapter 6 Social Networking. All the chapters are included in the
following tasks to present a mind-mapping, graphic organiser of the news,
graphs and charts, poster, product instructions and manuals, video presentation,
and all tasks were also evaluated by using English students’ presentation
assessment form, which included he last procedure of data collation was to
assign the students’ attitude questionnaires to the BE students to investigate their
attitudes on developing their reading comprehension through the presentations
by using content-creation tools.
As the quantitative method, this research employed Microsoft Excel software,
which involved the statistical analysis formula developed by Prachanban (2009)
to analyse the research data. A paired t-test was used to examine the significant
difference between the pre-test and the post-test. The tests were administered to
the students to see whether there was any significant difference between their
pre-test and post-test results, based on the four levels of reading comprehension.
According to Mekham (2022), the paired t-test Ho (Null Hypothesis) is accepted
if the significant two-tailed rekult is more than 0.05 (sig>0.05). If the significant
506
two-tailed result is less than 0.05, it means that the Ha (Alternate Hypothesis) is
accepted, and Ho is rejected. The BE students’ assignment scores included the
reading comprehension tests in each unit and the presentation of the assessment;
the data from these instruments were also analysed by using Microsoft Excel
software, and the calculated value of the percentage of the score was not less than
70. Descriptive statistics involving the percentage, mean, and the standard
deviation were also used to analyse the data from the questionnaires.
4. The Results
4.1 The influence of the content-creation tools on the BE students reading-
comprehension skills
After collecting the data, the following table and figures showed the results of
the Business-English students’ reading comprehension competence before and
after exposing them to the content-creation tools. A paired t-test was used to
examine the significant difference between the pre-test and the post-test, as
mentioned earlier.
Table 1: Paired Samples Statistics of BE Students’ Reading Comprehension Skills
Std.
Std. Error
N Mean Deviation Mean Correlation Sig.
Pair 1 Pre-test & 27 12.41 3.129 0.602 0.564 0.002
Post test 23.00 3.138 0.604
The table above presented the mean score of the pre-test as 12.41, while N there
were 27. Meanwhile, the standard deviation for the pretest was 3.129. The mean
standard error for the pretest was 0.602. On the other hand, the mean score of the
post-test was 23.00, with the standard deviation of 3.138. The mean standard
error for post-test was 0.604.
Table 1 shows the statistical results of the BE Students reading comprehension
skills before and after using the content-creation tools. As shown in the table, the
mean scores of the students during the post-test increased by almost half of the
mean score during the pre-test, with a mean difference of 10.59 (23.00-12.41) in
favour of the post-test scores. Using paired sample correlations, this shows that
the correlation value (0.564) between the pre-test and post-test to be highly
significant with a 0.002 level of significance, which is smaller than the critical value
of α= 0.05.
To determine whether the mean difference between pre-test and the post-test
scores obtained by the BE students showed any significant difference, paired t-
test was used, and the results are shown in Table 2.
507
Table 2: Paired Sample Test on BE students’ reading comprehension

Paired Differences
99%
Confidence
Std. Interval of Sig.
Std. Error the Difference (2-
Mean Deviation Mean Lower Upper t df tailed)
Pair 1
Pre-test
10.59 2.925 0.563 9.028 12.157 18.814 26 0.000
and
Post-test
Table 2 showed the result of analysis when using a Paired t-test. The mean
difference between the pretest and the post-test was 10.59, with a standard
deviation of 2.925, and a mean standard error of 0.563. The resultant t test was
18.814, with the degrees of freedom value (df) 25 and a significance of 0.000.
The English-reading proficiency test was conducted by a researcher before and

after the implementation by using content-creation tools to support the BE
students’ presentations. The result showed a significant difference between the
students’ pre-test and their post-test, after developing the students’ reading
comprehension skills through presentations by using content-creation tools;
since the tvalue = 18.814, has a significant value of 0.000, which is lower than the
critical value α= 0.05. It can be concluded that the BE students’ reading
comprehension skills were enhanced after using the content-creation tools in
their presentations.
Table 3: Reading the comprehension test score in each unit
Unit
No. Total score (60) Percentage
1 2 3 4 5 6
1 7 8 9 6 8 5 43 72
2 8 6 8 6 6 8 42 70
3 6 8 8 8 9 5 44 73
4 10 9 6 8 6 6 45 75
5 8 9 9 7 6 5 44 73
6 6 10 8 8 8 4 44 73
7 6 8 6 7 10 8 45 75
8 7 6 6 8 9 9 45 75
9 5 7 7 9 8 8 44 73
10 7 8 6 8 7 8 44 73
11 8 9 9 8 8 7 49 82
12 6 7 9 6 9 10 47 78
13 6 8 8 5 8 9 44 73
14 10 6 8 6 9 6 45 75
15 8 7 9 10 9 7 50 83
16 7 8 8 6 7 8 44 73
17 10 6 6 9 9 9 49 82
18 9 5 8 7 8 6 43 72
19 8 6 9 8 7 7 45 75
20 9 7 8 6 6 9 45 75
21 7 8 9 10 9 8 51 85
508
22 9 7 10 6 9 9 50 83
23 8 10 9 7 8 6 48 80
24 10 9 8 6 6 10 49 82
25 9 10 6 8 8 9 50 83
26 7 8 8 6 6 7 42 70
27 6 7 9 8 7 6 43 72
Mean value 45.70 76.11
Table 3 shows the results of the study on the total score of the task assessment
after completing all the units. Using content-presentation tools to enhance the
reading-comprehension skills in each chapter, it was discovered that 27 students
had a mean value score of 45.70 out of 60 as a total score, representing 76.11% o
thef overall score, indicating that all the students had a passing score of 70%,
which was consistent with the research hypothesis.
4.2. Attitudes of BE students towards presentations when using content-creation tools
in developing their reading comprehension scores
Table 4: Mean score of the students’ attitudes towards developing their reading
comprehension through the presentations by using the content-creation tools
Level of
Statements 𝑥̅ S.D.
Agreement
1) In order to improve my reading
comprehension, I find that the content-
creation tools (Websites that provide useful
4.593 0.681 Strongly agree
functions for developing students’ tasks, for
example Canva.com, Prezi.com, etc.) are
crucial.
2) I can improve my reading comprehension
with the additional assistance of the content 3.889 1.286 Agree
creation tools.
3) I am willing to apply the content creation
tools for enhancing my reading 4.741 0.516 Strongly agree
comprehension.
4) I can select the appropriate content creation
tools to enhance my reading comprehension 4.333 0.903 Agree
skills.
5) I can select the content creation tools based
on my knowledge, experience, and interests 4.519 0.739 Strongly agree
to complete my assignments.
6) I have a positive attitude on using the
content creation tools individually, in pairs, 4.111 1.133 Agree
and in groups.
7) I think that the use of content creation tools
can improve the environment for English
4.370 0.909 Agree
language learning both inside and outside of
the classroom.
8) Using content creation tools can enhance my
creative idea.
9) I am more confident in my ability to use the
content creation tools and improve my 4.667 0.720 Strongly agree
509
10) Using technology, I can adapt to English

learning very well.
11) I think that integrating the content creation
Strongly
tools with teaching and learning can make 4.667 0.667
disagree
my English learning more difficult.
12) I am usually nervous when using the
content creation tools to perform my 4.481 0.787 Disagree
assignments.
13) I think that the content creation tools are not Strongly
4.519 0.833
suitable for educational purposes. disagree
14) I think that using the content creation tools Strongly
4.593 0.782
is boring and incomprehensible. disagree
15) I am not interested in enhancing my reading
comprehension using content presentation 4.481 0.787 Disagree
tools.
Total 4.486 0.854 Agree
From the table above, it can be seen that students’ attitudes towards developing
reading comprehension skills through the use of content creation tools were at a
high level, with an average score of 4.486% (S.D. = 0.854), and that students were
willing to apply the content creation tools for enhancing my reading
comprehension, with an average score of 4.741 (S.D. = 0.516). The statements
regarding the use of content presentation tools in teaching and learning, in which
the content creation tools with teaching and learning can make their English
learning more difficult, had the lowest level of student attitudes towards the
topic, with an average score of 4.667% (SD = 0.667).
5. Discussion
This study focused on the influence of content-creation tools in enhancing the
reading-comprehension skills of Business-English students and their attitudes
towards using these tools.
The purposes of this study were to develop the reading-comprehension skills of
BE students by using content-creation tools to support their English-language
learning, and to investigate the students’ attitudes towards developing their
reading comprehension by using content-creation tools. The quantitative data
showed that the students developed their reading-comprehension skills after
using content-creation tools for their presentations, with a significance level of
two-tailed rsesults of less than 0.05. Furthermore, the assessment of student
presentations and reading comprehension met the predetermined criterion of
70%. These results also resembled the research findings of Pinandito et al. (2020).
Students generally accept the use of technologies to help them learn and
comprehend English-reading comprehension. It can also be said that these
technologies facilitate the creation of tasks from scratch and the reconstruction of
the tasks. Furthermore, students can raise their awareness of how to acquire
reading comprehension in various circumstances, in order to prepare accurate
information based on what they have learnt, in order to deliver their
presentations effectively and confidently through these digital tools. This idea
was in line with the idea of Ahmadi & Reza (2018), who mentioned that many
positive outcomes can occur for students when such technology is used
510
effectively. Students can take advantage of the technological tools because they
answer the common educational challenges and suggest strategies for
implementing knowledge. In addition, technology’s role is crucial in allowing
students to learn a language at their own pace, thereby aiding in self-
understanding, maintaining interaction with the teacher, and generating high
motivation in students to acquire linguistic competence effectively (Lei &
Medwell, 2021). According to Prachansen (2018), Shroff et al. (2011) and Craig
(2013), who also demonstrated that integrating technologies into the
development of student assignments can afford them numerous opportunities to
comprehend, critique, and assess the accuracy and appropriateness of
information. Using language for authentic purposes is a more effective way to
encourage content evaluation and language-skill development among language
students. Students develop their social responsibility and accountability
regarding language use on the basis of the content creation.
Furthermore, regarding the students’ attitudes, the students showed positive
attitudes towards content-creation tools in developing their reading
comprehension through these presentations. The results showed that the
students were willing to use content-creation tools to enhance their reading
comprehension. They were also more confident in using these content-creation
tools and improving their reading comprehension. This result was consistent
with that of Ebadi & Ashrafabadi’s (2022) study, which showed that students
enjoyed using technology and preferred it to traditional reading-comprehension
methods. Furthermore, these technologies increased students’ interest in reading
comprehension tasks and enhanced their willingness to use such technologies as
supporting tools in EFL classes. In addition, the students’ attitude showed that
the content-creation tools provided benefits for enhancing students’ creative
ideas. This result was also found in the studies of Sarangapani et al. (2019) and
McLoughlin & Lee (2010), who reported that students should use technological
tools to improve their command of the language; because these tools play a vital
role in encouraging students’ imaginative growth; while also giving them access
to various engaging and entertaining learning opportunities.
In contrast, the students’ attitudes were lowest towards claims that content-
creation tools used in the classroom could make language-acquisition tasks, like
learning English, more difficult. This result was comparable with the studies of
Chen (2016), Yedla (2013) and Chartrand (2012), who mentioned that there are
several ways to encourage students to listen and create their own materials to
share on the Internet by utilising social net-working websites. Due to costs and
technical limitations, it was formerly challenging to incorporate this type of
activity into lessons; however, these barriers are gradually disappearing, and it
is now possible to use these online tools in the easier way to improve students’
English proficiency. According to the study of Farooq & Javid (2012), technology
in English-language learning (e.g., English as Foreign Language (EFL) in
classrooms) has the potential to encourage student engagement, motivation, and
activity by allowing for the creation of unique, engaging tasks. A motivational,
interactive, and conducive learning environment for English language teachers
and students can be achieved through the judicious use of technology, the
availability and access to online study material, in addition to the integration of
technology with the study material.
511
6. Conclusion and recommendations

The purposes of this study were to develop the reading-comprehension skills of
Business-English students by using content-creation tools to support their
English-language learning, and to study the students’ attitudes towards
developing their reading comprehension by using content-creation tools.
The findings revealed that students’ reading-comprehension skills were
developed. The results demonstrated a statistically significant improvement in
students’ reading and comprehension abilities after using content-creation tools
for their presentations, with a significance level of two-tailed results less than
0.05. Furthermore, the assessment of students’ presentations and reading
comprehension met the predetermined criterion of 70%. While students’
attitudes towards developing reading comprehension skills by using content-
creation tools were at a high level, with an average score of 4.486% (S.D. = 0.854).
Moreover, the students also perceived various advantages in using content-
creation tools, including creative thinking, critical analysis, collaboration,
problem-solving, and digital literacy. However, some students faced challenges
due to limited experience with certain tools, thereby highlighting the need for
additional training and support. Moreover, the results provide better
understanding for the students for using digital-supportive tools in the
classrooms to enhance teachers to pay attention on teaching their students to use
tools to develop their students’ tasks effectively. These skills could be practical
and usable in the students’ future careers.
It is crucial to note that this study specifically examined reading-comprehension
skills and did not assess writing, listening, or speaking abilities, which should be
considered in any future research. The research methodology utilised a single
group, which might limit the generalisability of the results; and more
sophisticated statistical analyses might well have been employed.
Finally, it is recommended to extend this approach to other English, as a Foreign
Language (EFL) and English for Specific Purposes (ESP) courses, as well as,
employing different digital tools to investigate the students’ reading
comprehension in different contexts covering different language skills, and
considering teachers' perspectives. Additionally, future studies could explore the
connections between classes, students' language proficiency levels, age, gender,
and other emerging research topics in English-reading skills.
7. References
Agustina, L. (2019). Stimulating students to speak up through presentation in business
English class. Journal of Applied Studies in Language, 3(1), 21-28.
http://dx.doi.org/10.31940/jasl.v3i1.1148
Ahmadi, R., M. (2018). The use of technology in English language learning: A literature
review. International Journal of Research in English Education, 3(2), 115-125.
http://doi.org/10.29252/ijree.3.2.115
Al Harun, M. O. F., Islam, K. A., & Rahman, M. A. (2016). Challenges in oral presentation
in English for the freshers at tertiary level. Green University Review of Social
Sciences, 3(1), 137-157.
512
Alenezi, A. (2020). The Role of e-Learning Materials in Enhancing Teaching and

Learning Behaviors. International Journal of Information and Education Technology,
10(1), 48–56. https://doi.org/10.18178/ijiet.2020.10.1.1338
Alsowayegh, N., Baradesi, H., & Garba, I. (2018). Creating an effective e-learning
content: implications for a language learning program. In 12TH International
Technology, Education and Development Conference (INTED) (pp. 8603-8611).
http://doi.org/10.21125/inted.2018.2086
Amesi, J., Amaewhule, W. A., & Akoku, A. U. (2022). Digital Skills Required By Business
Education Lecturers For Teaching Accounting Education Courses In Tertiary
Institutions In Rivers State. Nigerian Journal of Business Education (NIGJBED), 9(2),
38-52.
Antón-Sancho, Á., Vergara, D., Lamas-Álvarez, V. E., & Fernández-Arias, P. (2021).
Digital content creation tools: American university teachers’ perception. Applied
Sciences, 11(24), 11649. https://doi.org/10.3390/app112411649
Azmuddin, R. A., Mohd Nor, N. F., & Hamat, A. (2020). Facilitating online reading
comprehension in enhanced learning environment using digital annotation
tools. IAFOR Journal of Education, 8(2), 7–27. https://doi.org/10.22492/ije.8.2.01
Banditvilai, C. (2020). The effectiveness of reading strategies on reading comprehension.
International Journal of Social Science and Humanity, 10(2), 46-50.
http://doi.org/10.18178/ijssh.2020.V10.1012
Boettcher, J. V., & Conrad, R. M. (2021). The online teaching survival guide: Simple and
practical pedagogical tips. 5(1), 294–297. http://doi.org/10.29333/ajqr/10981
Bogaerds-Hazenberg, S. T. M., Evers-Vermeul, J., & Bergh, H. (2020). A meta‐analysis on
the effects of text structure instruction on reading comprehension in the upper
elementary grades. Reading Research Quarterly, 56(3), 435–462.
https://doi.org/10.1002/rrq.311
Capodieci, A., Cornoldi, C., Doerr, E., Bertolo, L., & Carretti, B. (2020). The use of new
technologies for improving reading comprehension. Frontiers in Psychology, 11.
Carlisle, F. J., Kelcey, B., Berebitsky, D., & Phelps, C. G. (2011). Embracing the complexity
of instruction: A study of the effects of teachers’ instruction on students’ reading
comprehension. Scientific Studies of Reading, 15(5), 409-439.
http://doi.org/10.1080/10888438.2010.497521
Chartrand, R. (2012). Social networking for language learners: creating meaningful
output with web 2.0 tools. Knowledge Management & E-Learning, 4(1), 97–101.
https://doi.org/10.34105/j.kmel.2012.04.009
Chen, J. C. (2016). The crossroads of English language learners, task-based instruction,
and 3D multi-user virtual learning in Second Life. Computers & Education, 102,
152-171. https://doi.org/10.1016/j.compedu.2016.08.004
Chong, S. C. (2018). Ten trends and innovations in English language teaching for 2018.
https://www.britishcouncil.org/voices- magazine/ten-trends-innovations-
english-language-teaching-2018
Collins, A. (July 13, 2022). Best Content Creation Tools.
https://www.shopify.com/blog/free-content-marketing-creation-tools
Correa, T., & Jeong, S. H. (2011). Race and online content creation: Why minorities are
actively participating in the Web. Information, communication & society, 14(5), 638-
659. https://doi.org/10.1080/1369118X.2010.514355
Craig, V. D. (2013). Content creators and language learners: Exploring Web 2.0 and
wikis. Journal of Educators Online, 10(2), 1-21. 10.9743/JEO.2013.2.5
Cuckle, P., Clarke, S., & Jenkins, I. (2000). Students’ information and communications
technology skills and their use during teacher training. Journal of Information
Technology for Teacher Education, 9(1), 9-22.
https://doi.org/10.1080/14759390000200072
513
Darmayanti, M. M. L., & Lisa, A. (2019, July). Reflection of disruptive learning And HOTS
to English education: A study in Indonesian school. In Unnes-Teflin National
Seminar, (pp. 60-66).
Duke, N. K., & Pearson, P. D. (2009). Effective practices for developing reading
comprehension. Journal of education, 189(1-2), 107-122.
http://doi.org/10.1598/0872071774.10
Ebadi, S., & Ashrafabadi, F. (2022). An exploration into the impact of augmented reality
on EFL learners’ Reading comprehension. Education and Information Technologies,
27(7), 9745-9765. http://doi.org/10.1007/s10639-022-11021-8
Erdoğan, V. (2019). Integrating 4C skills of 21st century into 4 language skills in EFL
classes. International Journal of Education and Research, 7(11), 113-124.
https://doi.org/10.29228/ijla.25816
Fansury, A. H., Januarty, R., Rahman, A. W., & Syawal. (2020). Digital content for
millennial generations: teaching the English foreign language learner on covid-
19 pandemic. Journal of Southwest Jiaotong University, 55(3). Internet Archive.
https://doi.org/10.35741/issn.0258-2724.55.3.40
Farooq, M. U., & Javid, C. Z. (2012). Attitude of students towards E-learning: A study of
English language learners at Taif University English Language Centre. NUML
Journal of Critical Inquiry, 10(2), 17-28.
Gough, P. B., & Juel, C. (2013). The first stages of word recognition. In Learning to read,
pp. 47-56. Routledge.
Hjetland, H. N., Brinchmann, E. I., Scherer, R., Hulme, C., & Melby-Lervåg, M. (2020).
Preschool pathways to reading comprehension: A systematic meta-analytic
review. Educational Research Review, 30, 100323.
Hogan, T. P., Adlof, S. M., & Alonzo, C. N. (2014). On the importance of listening
comprehension. International journal of speech-language pathology, 16(3), 199-207.
http://doi.org/10.3109/17549507.2014.904441
Huotari, L., Ulkuniemi, P., Saraniemi, S., & Mäläskä, M. (2015). Analysis of content
creation in social media by B2B companies. Journal of Business & Industrial
Marketing, 30(6), 761–770. https://doi.org/10.1108/jbim-05-2013-0118
Jayakumar, P., Suman Rajest, S., & Aravind, B. R. (2022). An Empirical Study on the
Effectiveness of Online Teaching and Learning Outcomes with Regard to LSRW
Skills in COVID-19 Pandemic. Studies in Computational Intelligence, 483–499.
https://doi.org/10.1007/978-3-030-93921-2_27
Jung, C. D. (2021). Perceptions of Collaborative Video Projects in the Language
Classroom: A Qualitative Case Study. International Journal of Instruction, 14(4),
301–320. https://doi.org/10.29333/iji.2021.14418a
Klimova, B., & Zamborova, K. (2020). Use of mobile applications in developing reading
comprehension in second language acquisition—A Review Study. Education
Sciences, 10(12), 391. https://doi.org/10.3390/educsci10120391
Lavi, R., Tal, M., & Dori, Y. J. (2021). Perceptions of STEM alumni and students on
developing 21st century skills through methods of teaching and learning.
Studies in Educational Evaluation, 70, 101002.
Lei, M., & Medwell, J. (2021). Impact of the COVID-19 pandemic on student teachers:
How the shift to online collaborative learning affects student teachers’ learning
and future teaching in a Chinese context. Asia Pacific Education Review, 22, 169-
179. http://doi.org/10.1007/s12564-021-09686-w
Liapis, A., Yannakakis, G., & Togelius, J. (2013). Designer modeling for personalized
game content creation tools. In Proceedings of the AAAI Conference on Artificial
Intelligence and Interactive Digital Entertainment, 9(2) (pp.11-16).
https://doi.org/10.1609/aiide.v9i2.12587
514
Lotfi Sin, S., & Siahpoosh, H. (2020). Looking at the Impact of the Flipped Classroom
Model on Reading Comprehension of Iranian EFL Learners. Arabic Language,
Literature & Culture, 5(2), 14. https://doi.org/10.11648/j.allc.20200502.12
Manh Do, H., & Le Thu Phan, H. (2021). Metacognitive Awareness of Reading Strategies
on Second Language Vietnamese Undergraduates. Arab World English Journal,
12(1), 90–112. https://doi.org/10.24093/awej/vol12no1.7
Marzuki, A. G., Alim, N., & Wekke, I. S. (2018). Improving the reading comprehension
through cognitive reading strategies in language class of coastal area in
Indonesia. IOP Conference Series: Earth and Environmental Science, 156, 012050.
https://doi.org/10.1088/1755-1315/156/1/012050
McLoughlin, E. C., & Lee, M. J. W. (2010). Personalised and self regulated learning in the
Web 2.0 era: International exemplars of innovative pedagogy using social
software. Australasian Journal of Educational Technology, 26(1).
http://doi.org/10.14742/ajet.1100
Mulyadi, D., Wijayatiningsih, T. D., Singh, C. K. S., & Prastikawati, E. F. (2021). Effects of
Technology Enhanced Task-Based Language Teaching on Learners' Listening
Comprehension and Speaking Performance. International Journal of Instruction,
14(3), 717-736.
National Research Council. (2012). Education for life and work: Developing transferable
knowledge and skills in the 21st century. Washington, DC: The National Academies
Press. https://doi.org/10.17226/13398.
Nazurty, R., Priyanto, N., Anggia Pratiwi, S., & Mukminin, A. (2019). Learning strategies
in reading: The case of Indonesian language education student teachers.
Universal Journal of Educational Research, 7(11), 2536-2543.
http://doi.org/10.13189/ujer.2019.071133
Newton, J. M., Ferris, D. R., Goh, C. C., Grabe, W., Stoller, F. L., & Vandergrift, L. (2018).
Teaching English to second language learners in academic contexts: Reading, writing,
listening, and speaking. New York: Routledge.
https://doi.org/10.4324/9781315626949
Oakhill, J., Cain, K., & Elbro, C. (2019). Reading comprehension and reading
comprehension difficulties. Reading development and difficulties: Bridging the gap
between research and practice, 83-115. https://doi.org/10.1007/978-3-030-26550-
2_5
Office of the Education Council. (2021). Policies and strategies for educational development.
Bangkok: Office of the Education Council.
Oktaviani, L., & Mandasari, B. (2020). Powtoon: A digital medium to optimize students’
cultural presentation in ELT classroom. Teknosastik, 18(1), 33-41.
https://doi.org/10.33365/ts.v18i1.526
Ordu, U. B. (2021). The Role of Teaching and Learning Aids / Methods in a Changing World.
Sofia: Bulgarian Comparative Education Society
Panyasai, P. (2022). Students’ perception towards business English presentation:
focusing on needs analysis and preferences. In Proceedings of the 15th National and
the 2nd International Conference on Humanities and Social Sciences, Agility in
Humanities and Social Sciences in a VUCA world, 1012-1025.
Panyasai, P., & Paladkong, A. (2019). Using films to promote motivation in studying
English of Loei Rajabhat university students: a case study in the course of
English for communication and study skills 0001103. In the Proceedings of the 5th
National Conference on Research and Innovation for Sustainable Community
Development, 1061-1071.
Parveen, B. W. (2016). Use of technology in improving speaking skills. Journal of English
language and literature (JOELL), 3(2). 121 – 124.
515
Pearson, P. D. (2014). The roots of reading comprehension instruction. In Handbook of

research on reading comprehension. pp. 27-55. Routledge.
https://doi.org/10.4324/9781315759609-11
Pinandito, A., Wulandari, C. P., Prasetya, D. D., Hirashima, T., Hayashi, Y., & Az-Zahra,
H. M. (2020, November). Students' acceptance towards kit-build concept map
authoring tool in supporting learning of English reading comprehension. In
Proceedings of the 5th International Conference on Sustainable Information Engineering
and Technology, (PP.158-164). https://doi.org/10.1145/3427423.3427464
Phitsanulok: Rattanasuwan Printing House.Prachansen, N. (2018). Effect of graphic
organizer on English reading comprehension of grade 11 students of
Banchamaracharungsarit school [Master’s thesis, Rajabhat Rajanagarindra
University]. http://thesis.rru.ac.th/files/pdf/712_2019_03_21_141143.pdf
Prachanban, P. (2009). Research Methodology in social science.
Priyanti, N. W. I., Santosa, M. H., & Dewi, K. S. (2019). Effect of quizizz towards the
eleventh-grade English students’ reading comprehension in mobile learning
context. Language and Education Journal Undiksha, 2(2).
https://doi.org/10.23887/leju.v2i2.20323
Puggioni, M. P., Frontoni, E., Paolanti, M., Pierdicca, R., Malinverni, E. S., & Sasso, M.
(2020, September). A content creation tool for AR/VR applications in education:
The scholar framework. In International conference on augmented reality, virtual
reality and computer graphics. (pp. 205-219). Springer, Cham.
https://doi.org/10.1007/978-3-030-58468-9_16
Rao, P. S. (2019). The importance of speaking skills in English classrooms. Alford Council
of International English & Literature Journal (ACIELJ), 2(2), 6-18.
Säljö, R. (2010). Digital tools and challenges to institutional traditions of learning:
technologies, social memory and the performative nature of learning. Journal of
computer assisted learning, 26(1), 53-64. https://doi.org/10.1111/j.1365-
2729.2009.00341.x
Sarangapani, V., Kharrufa, A., Leat, D., & Wright, P. (2019). Fostering deep learning in
cross-cultural education through use of content-creation tools. In Proceedings of
the 10th Indian Conference on Human-Computer Interaction. 7, (pp.1–11).
https://doi.org/10.1145/3364183.3364184
Shroff, R. H., Deneen, C. C., & Ng, E. M. (2011). Analysis of the technology acceptance
model in examining students’ behavioural intention to use an e-portfolio system.
Australasian Journal of Educational Technology, 27(4).
https://doi.org/10.14742/ajet.940
Singh, J., Steele, K., & Singh, L. (2021). Combining the Best of Online and Face-to-Face
Learning: Hybrid and Blended Learning Approach for COVID-19, Post Vaccine,
& Post-Pandemic World. Journal of Educational Technology Systems, 50(2),
140–171. https://doi.org/10.1177/00472395211047865
Skrbiš, Z., & Laughland‐Booÿ, J. (2019). Technology, change, and uncertainty:
Maintaining career confidence in the early 21st century. New Technology, Work
and Employment, 34(3), 191-207.
Smith, R., Snow, P., Serry, T., & Hammond, L. (2021). The role of background knowledge
in reading comprehension: A critical review. Reading Psychology, 42(3), 214-240.
https://doi.org/10.1080/02702711.2021.1888348
Spiro, R. J., Bruce, B. C., & Brewer, W. F. (Eds.). (2017). Theoretical issues in reading
comprehension: Perspectives from cognitive psychology, linguistics, artificial intelligence
and education. Vol. 11. Routledge.
Thyagharajan, K. K., & Nayak, R. (2007). Adaptive content creation for personalized e-
learning using web services. Journal of Applied Sciences Research, 3(9), 828-836.
Yedla, S. (2013). MALL (mobile assisted language learning): A paradise for English
language learners. Journal of English Language & Translation Studies, 1(2), 91-99.
516
Ustun, A. B. (2019). Students’ experiences in learning and using Prezi in higher

education. Bartın Üniversitesi Eğitim Fakültesi Dergisi, 8(3), 928–946.
https://doi.org/10.14686/buefad.552138Oktaviani, L., & Mandasari, B. (2020).
Powtoon: A digital medium to optimize students’ cultural presentation in ELT
classroom. TEKNOSASTIK, 18(1), 33. https://doi.org/10.33365/ts.v18i1.526
Pham, Q. N., & Li, M. (2022). Digital multimodal composing using visme: EFL students’
perspectives. The Asia-Pacific Education Researcher.
https://doi.org/10.1007/s40299-022-00687-w
Xiuwen, Z., & Razali, A. B. (2021). An overview of the utilisation of TikTok to improve
oral English communication competence among EFL undergraduate students.
Universal Journal of Educational Research, 9(7), 1439–1451.
https://doi.org/10.13189/ujer.2021.090710
517

Vol. 22, No. 7, pp. 517-534, July 2023
Engaging English as a Foreign Language (EFL)

Students through the Game-Based Learning
Approach in Higher Education
Cesar Ochoa-Cueva , Luz Castillo-Cuesta , Paola Cabrera-Solano
Universidad Técnica Particular de Loja, Loja, Ecuador
Abstract. The present study aimed to implement the Game-Based

Learning (GBL) approach through the use of technology to promote
students’ engagement in the English as a Foreign Language (EFL)
classroom. The participants included 64 undergraduate and postgraduate
students involved in two EFL study programmes. This research followed
a mixed-method approach, which included the administration of an
online diagnostic questionnaire, an observation checklist, an online
perceptions questionnaire and a semi-structured interview. This research
was carried out over 14 weeks in four different courses in which the
game-based approach was implemented through the use of Quizizz. The
results showed that the participants’ perceptions of GBL were positive in
all the stages of EFL lessons since learners were motivated, interested and
had the opportunity to develop their linguistic and pedagogical skills.
Keywords: Game-Based learning approach; EFL students; engagement;

higher education; Quizizz
1. Introduction
English has become increasingly important in the last decade, as it is a useful tool
for effective communication in science, education and business. Being a lingua
franca, mastering English can enhance communication skills with people
worldwide (Jenkins, 2015). According to Alvarez and Guevara (2021), learning
English is essential for obtaining professional scholarships and better job
opportunities in Ecuador. It is now a required academic subject at all levels of
education, including elementary, secondary and university. In this context,
different approaches and methods have been used to teach English, in which
instructors have incorporated the use of technology. To this respect, Brown and
Lee (2014) manifest that as current students have been raised with digital
technologies, they tend to seek innovative methods of learning. Hence, higher
education instructors need to adopt alternative instructional approaches to
facilitate student learning.
©Authors
518
As per Rose’s (2019) statement, it is crucial for EFL teachers in higher education
to ensure that students possess the necessary competencies to acquire the target
language and use English effectively in their future professional endeavours.
Consequently, EFL educators should use different methods and strategies to help
students succeed in the language learning process. In this regard, engaging
learners to succeed is a crucial issue to develop their accomplishment in EFL
language learning (Oga-Baldwin, 2019). One way to foster students’ participation
is engagement, which involves observable and unobservable aspects of learners’
interactions with educational activities (Li & Lerner, 2013). In addition, according
to Fredricks et al. (2019), learners’ engagement is a multidimensional concept
linked to several positive learning and life outcomes. Therefore, engagement is
linked to academic performance and motivation (Barak et al., 2016). As Harmer
(2007) affirms, teachers can promote learners’ engagement by stimulating their
curiosity, involvement, attention and emotion; thus, they will be prepared to start
a new lesson.
In this context, to increase students’ engagement, the implementation of the GBL

approach is crucial to providing students with a sense of control, independent
learning, and academic achievement (Kulpa, 2017; Noroozi et al., 2020). Moreover,
GBL involves the use of game design to improve the educational setting and attain
better learning outcomes (Lim & Lee, 2007; Shaffer et al., 2005). Games with
educational purposes have long been acknowledged as a way to promote
students’ active learning. According to Selvi and Çosan (2018), games can
encourage learners to take an active role in the classroom since they foster their
critical thinking skills. In EFL teaching and learning, the implementation of the
GBL approach creates a setting where the use of games improves the acquisition
of all language skills, and it brings exciting learning experiences (Hitosugi et al.,
2014; Kirriemuir & Farlane, 2004). Moreover, the GBL approach, according to
Tang et al. (2009), uses information and communication technology (ICT) to create
interactive and stimulating virtual learning environments. In this regard, the
integration of GBL through technology offers EFL teachers the opportunity to
design a variety of games by using tools such as Quizizz, which can be used to
create interactive games that students can access on their electronic devices
(Irwansyah & Izzati, 2021). Certainly, the use of games in the EFL classroom
creates a motivating environment in which students can control their own
learning process, which is a crucial aspect of foreign language acquisition
(Cabrera-Solano, 2022).
Several studies have been carried out regarding the use of the GBL approach
through Quizizz in EFL teaching. Munawir and Hasbi (2021) conducted research
on the effectiveness of Quizizz to improve student engagement and learning
outcomes in an EFL university setting. Quizizz had a positive impact on student
engagement, motivation, understanding of the lesson and ease of learning topics.
In addition, Irwansyah and Izzati (2021) analysed the use of Quizizz as a GBL and
assessment tool in an EFL context. The outcomes showed that Quizizz motivated
participants who were actively engaged in the lessons and showed good attitudes
while using this tool. Moreover, Inayati and Waloyo (2022) analysed the level of
learners’ engagement and learning outcomes of Quizizz-online gamification on
519
English learning development in high schools. The findings revealed that

gamification had a positive influence on learners’ engagement, participation and
discipline. Although there have been different studies related to the use of the
game-based approach through Quizizz internationally, in our context, similar
studies are scarce. Therefore, this research aims to address this gap by analysing
the impact of GBL on learners’ engagement during the didactic sequence of the
different stages of a lesson. Therefore, our study would respond to the following
research questions.
RQ1: What are the EFL students’ attitudes towards the use of the Game-Based
Learning approach for classroom engagement?
RQ2: What are the EFL students’ perceptions about the use of the Game-Based
Learning approach in the different stages of a lesson?
2.1 EFL Teaching in higher education
Globalisation and technology have shaped the landscape of higher education.
Institutions are adapting to meet the demands of a globalised world by
incorporating English language proficiency and embracing technology as a means
to enhance teaching and learning (Martín del Pozo et al., 2017). These changes aim
to equip students with the skills and competencies needed to succeed in an
interconnected and technologically driven society (Falloon, 2020). Therefore, few
people can deny that higher education has changed drastically over the course of
the past decades. In this context, globalisation and student mobility have moved
universities to include English in their curriculum as institutions try to meet the
requirements of the global movement of education (Murray, 2016). Undoubtedly,
in the current era of digital progression, the use of technology has become a
popular phenomenon in higher education, which has led to engaging activities
that promote reflection, communication, collaboration, sharing knowledge, and
so forth (Kasuma, 2017). Moreover, according to Brown and Lee (2014), since new-
generation students have grown up with digital technologies, they tend to seek
new learning; therefore, instructors need to use alternative strategies to promote
student learning in higher education institutions. Consequently, English language
higher education teachers need to ensure that students have the requisite skills to
learn the target language and to use English successfully in their future careers
(Rose, 2019). According to Halverson et al. (2018), one alternative to reach such a
goal could be by meeting the needs and expectations of university students who
are greatly interested in online recreational activities such as games, videos and
social networking, among others.
2.2 Students’ Engagement in EFL Teaching

Engagement in EFL education has been explored from different perspectives as
an approach to teaching the language. In this regard, it has been defined as being
involved, committed, or attached to the academic and social activities of an
institution (Li & Lerner, 2013). As Deci and Ryan (2000) affirm, engagement
involves the learners’ traits that can be seen and those that cannot in their
interactions with educational activities. Nevertheless, engagement needs to be
sustained by need fulfilment; therefore, the degree to which students experience
520
need satisfaction affects their interest in different activities. Certainly, Bomia et al.
(1997) claim that engagement is related to learners’ needs and willingness to
involve themselves in the learning process, which can then lead to better
achievement. In foreign language teaching, Oga-Baldwin (2019) acknowledges
that engagement is a highly desirable prerequisite for learning. In this context,
engagement can be considered the energy students spend toward their academic
success; therefore, it constitutes one of the most critical aspects of learners’
accomplishment when studying a foreign language. Likewise, Sadoughi and
Hejazi (2021) state, “EFL teachers expect learners to be as highly engaged as
possible in learning activities relating to the four main language skills (i.e.,
speaking, writing, listening, and reading) as well as sub-skills such as grammar
and vocabulary” (p. 1).
2.3 Game-Based Learning Approach (GBL)

Educational games have been traditionally recognised as an effective approach to
enhancing students’ active learning. Researchers have indicated that using games
can help students participate in their learning process and develop their critical
thinking skills (Huizenga et al., 2009; Selvi & Çosan, 2018). In this context, the GBL
approach involves the effective use of games to promote students’ engagement in
the classroom (Noroozi et al., 2020). In addition, GBL develops students’ problem-
solving skills by using interactive games (Ebner & Holzinger, 2007). In the context
of EFL teaching, GBL constitutes an environment where the use of games
enhances the acquisition of all language skills and fosters students’ abilities to
develop their critical thinking skills (Kirriemuir & Farlane, 2004). Reinhardt and
Sykes (2012) state that the effective use of games has the potential to promote
students’ interactions in the EFL learning process. Furthermore, Hitosugi et al.
(2014) assert that the implementation of games brings exciting learning
experiences to EFL learners who can also develop their cognitive abilities and
reduce their anxiety to learn the target language.
2.4 Implementation of the GBL approach through technology

The rapid development of ICT has revolutionised the way English is taught in the
twenty-first century (Adnan et al., 2020). In this context, the potential of ICT brings
several opportunities for EFL teachers to implement digital game-based
instruction in the classroom. According to Vandercruysse et al. (2012), the use of
games for instructional purposes provides different learning opportunities and
increases students’ motivation. In this regard, the implementation of games
requires teachers to become familiar with technology, which means that they need
to be creative enough to help students become critical thinkers (Qian & Clark,
2016). Considering the aforementioned aspects, EFL teachers can choose from a
variety of ICT tools to integrate games into their lessons. For example, Quizizz,
Genially, Kahoot and Socrative are some of the most recognised tools that allow
the use of games to teach language skills; however, from all these recourses,
Quizizz is considered a fun e-learning tool that has the potential to enhance
learners’ participation and motivation by implementing interactive games in the
teaching and learning process (Dhamayanti, 2021).
521
2.5 Previous Studies

Anak and Hua (2021) carried out a quasi-experimental study to investigate the
effects of Quizizz on Malaysian students’ learning of irregular verbs in English.
This study was conducted in a primary school in Padawan, Sarawak; the
participants were 30 young learners. The researchers used a pre-test/post-test
approach and analysed data through t-tests to explain the participants’
performance. The findings revealed that the implementation of online games
through Quizizz improved the pupils’ learning of irregular past tense verbs.
Therefore, game-based learning helped to enhance the participants’ interest and
learning of the English irregular past verbs.
Another study by Irwansyah and Izzati (2021) analysed the implementation of

Quizizz as a GBL and assessment tool in the English classroom. This study was
carried out in Indonesia during the COVID-19 pandemic. The participants
included students from six classes. They used Quizizz for 20 minutes before
finishing their lessons in which teachers obtained the score reports easily by using
this tool. Students’ perceptions of the use of Quizizz were inquired through a
questionnaire. The results evidenced that the participants felt excited and
motivated when playing Quizizz as a GBL and assessment tool. Also, they
actively participated in the classes and displayed a positive attitude toward the
use of such a tool.
Also, Inayati and Waloyo (2022) studied the level of learners’ engagement and
learning outcomes of Quizizz-online gamification on English learning
development in junior high school. This study was conducted at a private high
school in Batu City, Indonesia. The participants were 62 students in the 9th grade
and an EFL teacher. To gather the information, researchers used observations,
interviews, and students’ test results. The descriptive and thematic analysis
demonstrated that gamification positively influenced students’ engagement,
participation and discipline. However, students’ learning outcomes did not show
equal progress as their engagement level. Therefore, other approaches and
methods that are more cognitively driven should ensure that the balance between
learning outcomes and positive learning experiences is achieved.
Moreover, Munawir and Hasbi (2021) investigated the effectiveness of student

engagement and learning outcomes after using the application Quizizz in an EFL
university context. This research used a design that included a single group of
participants at a single point in time in the explanatory form of quantitative
research. The instruments to collect the data and information were a student
engagement questionnaire, a Quizizz-based test, report documentation, an
observation form and a learning outcomes questionnaire. The results revealed
that Quizizz helped students to be more interested in the lesson and ambitious for
success. Moreover, Quizizz permitted learners to work collaboratively and
increased their motivation. On the other hand, a combination of a gamification
approach with the learning method helped students to understand the lesson
better. Finally, the competition allowed learners to improve their response to time
management and made the learning topics easy to understand.
522
Another study by Pitoyo et al. (2020) aimed to explore the washback effect of the
Quizizz assessment platform on students’ learning. The participants were 18
university students in the third semester who took an English course. The sample
was selected purposively and the instruments included questionnaires,
observation checklists and in-depth interviews. The data obtained were analysed
quantitatively and qualitatively. The results indicated that students were
motivated and wanted to learn more deeply after doing several gamified tests
with Quizizz. Learners were interested in elements of the game such as the
Leaderboard, Meme, Time restriction and Test report. Finally, Web-based
gamification with Quizizz for assessment had a positive impact on students’
learning.
3. Method
3.1. Setting and participants
The participants in this study were 64 male and female EFL learners whose ages
ranged from 20 to 45 years old. The students participated in face-to-face and
distance English language courses and were enrolled in two EFL teaching
undergraduate and postgraduate study programmes offered at a private
university in the southern region of Ecuador. Learners’ proficiency levels
corresponded to B1 and B2 according to the Common European Framework of
References (Council of Europe, 2020). The researchers of this study were the
instructors of the aforementioned courses.

The instruments included the use of an online questionnaire, which was applied
to determine students’ learning expectations about the use of games in EFL
teaching and learning. This instrument comprised a five-point Likert Scale and
open-ended questions. An observation checklist based on five-point Likert Scale
items was used to register students’ involvement in terms of willingness,
participation and motivation in the engagement phase at the beginning, middle
and end of the different stages of EFL lessons. This instrument was designed and
scored by the researchers. An online perceptions questionnaire (close-ended and
open-ended questions) was administered to identify the students’ perceptions
about the use of the GBL approach in the EFL classroom. A semi-structured
interview was also applied to confirm the participants’ views regarding the use of
the GBL approach.
3.3 Procedure
The study followed a mixed-method approach which involved gathering and
analysing qualitative and quantitative data. As Creswell and Creswell (2017) state,
the use of mixed-method approaches provides a better comprehension of a
research problem than using just one research approach. The GBL approach was
implemented to enhance students’ engagement during the didactic sequence of
EFL lessons. The first stage included the administration of the learning
expectations questionnaire. After analysing the results of that questionnaire, an
intervention process was carried out for 14 weeks in four different courses in
which GBL activities (language quizzes) were designed by using Quizizz. The
teaching and learning process was adapted so that games were used to promote
students’ engagement in on-site and online classes. During on-site interactions,
523
students used their mobile phones, tablets and laptops to participate in games
related to linguistic and methodological matters. As for online classes, games
were delivered as supplementary activities that were available on the institutional
Canvas platform. In synchronous meetings, distance students interacted with the
teacher and classmates by means of Zoom to participate in GBL activities. During
the intervention, a total of 20 lessons (five per course) were observed by the
researchers to register the students’ involvement when using games. The aspects
that were seen included learners’ willingness to get involved in the lessons, their
motivation to learn the target language, and their participation level. In the last
stage of the intervention, the online perceptions questionnaire was administered,
and the interview was conducted with 16 EFL students enrolled in the four
courses of this study. It is worth mentioning that the questionnaires were
subjected to a rigorous validation process before being administered. Thus, the
internal consistency of the items was 0.7 (Cronbach’s alpha), which proved that
they were reliable.

In the diagnostic phase, participants (59.3%) indicated that they were skillful at
using technological tools, which had scarcely been used in previous courses. It is
worth mentioning that the participants’ opinions concerning the GBL approach
were diverse because most of them (62.5%) did not know about this method, while
21.87% of the students had been in courses where teachers had eventually used it.
Finally, 15.63% of the participants did not internalise the elements related to this
approach. In addition, students suggested that they wanted their teachers to use
diverse means for engagement such as games (71.8%), PowerPoint presentations
(14.2%), flipcharts (7.8%), questioning (3.1%), and others (3.1%).
Regarding the students’ attitudes towards the use of the GBL approach for
classroom engagement (see Table 1), the results of the observation evidenced that
at the beginning of the lessons, most of the participants (96%) seemed enthusiastic
when using Quizizz. These results are aligned with Zou et al.’s (2019) study, in
which it was found that the implementation of game-based activities enhances
EFL learners’ engagement and enthusiasm.
In addition, the majority of participants (98%) paid attention to the topics that
were going to be discussed in the lesson and responded to questions based on the
activities and resources used by their teachers. In this respect, Wang (2015) asserts
that the use of games has the potential to increase students’ attention and helps
them to keep more focused on the contents to be studied.
Likewise, 98% of the students enjoyed the ICT tools that the teachers used to
present the lesson and all of them liked the resources (PowerPoint presentations,
pictures, realia, games, videos, songs, among others). Certainly, Zulfa and Ratri
(2022) affirm that the advancement of technology facilitates students’ access to
different types of materials and resources, which supports the process of teaching
and learning in the digital age.
524
In the middle of the lessons, 91% of the students seemed to enjoy practising the
contents presented by their teachers through the Quizizz tool. Similarly, 92% of
learners effectively participated in the different classroom discussions promoted
by their instructors on Quizizz. In this regard, by using Quizizz in class, students
can maintain discussions after completing the quiz to review each question and
obtain feedback from the teacher (Zainuddin et al., 2020).
Regarding the use of Quizizz games to develop students’ critical thinking skills,
the majority of participants (98%) carried out different types of tasks that allowed
them to use games as a means to enhance their ability to examine, judge, interpret
and/or evaluate topics related to linguistic and pedagogical issues in the field of
EFL teaching and learning. These results are aligned with what Nashar et al.
(2021) found since the use of the Quizizz application improved the learners’
critical thinking skills.
As for the use of Quizizz games to help learners develop their communication
skills, 81% of them seemed to have a very good performance in most of the
activities that involved oral communication, while 17% of the students performed
at a good level. Another aspect that was observed in the middle of the lessons
dealt with the effect of using Quizizz games to help learners develop their
pedagogical skills. In this respect, the majority of the participants (89%) seemed
to be engaged with this tool, especially when having discussions related to diverse
methodological aspects. In this respect, Anak and Hua (2021) manifest that
Quizizz enhances the teaching and learning of English and also serves as a vital
method for fostering enthusiasm and a strong desire to learn this language.
With respect to the students’ attitudes at the end of the lessons, 92% of the
participants were able to show their knowledge of the contents studied using the
Quizizz tool. Moreover, most of the students (94%) claimed that Quizizz helped
them to reflect on their progress. These results are similar to those found in the
study by Dhamayanti (2021), which demonstrates the benefits of Quizizz as an
effective and motivating tool that supports the learning process in the English
classroom.
On the other hand, the use of games through Quizizz was seen as helpful since
learners (91%) could receive instructors’ immediate feedback based on the tool. In
this same line of thought, participants liked the use of games (89%) because such
activities engaged them to continue participating in future assignments. In this
respect, Maraza et al. (2019) affirm that gamification is an effective learning
strategy to improve feedback processes because it provides self-assessment
opportunities so that language learners can recognise their mistakes without
feeling discouraged.
Finally, most students (76%) reinforced their linguistic and pedagogical skills
through Quizizz; however, 22% of them seemed not to have completely
developed the aforementioned abilities. Regarding the use of Quizizz for
enhancing language skills, Zuhriyah and Pratolo (2020) assert that this tool allows
EFL learners to improve their confidence and language abilities.
525
Table 1. Results of the observation
Statements
Very Very
Not at all Neutral Somewhat
little much
Beginning of the lesson
Students look enthusiastic

when using Quizizz at the 0% 1% 3% 23% 73%
beginning of the lesson.
Students pay attention to

the topics that will be 0% 0% 2% 22% 76%
discussed in the lesson.
The students answer or

respond to questions based
0% 0% 2% 28% 70%
on activities and resources
used by the teacher.
The students like the ICT

tools used by the teacher 0% 0% 2% 17% 81%
when presenting the lesson.
The students like the

resources used by the
teacher when introducing
the lesson: PowerPoint 0% 0% 0% 20% 80%
presentations, pictures,
realia, games, videos,
songs, others.
Middle of the lesson
The students like Quizizz to

practice the contents of the 0% 0% 1% 8% 91%
lesson.
Quizizz allows students to

participate in class 0% 0% 0% 8% 92%
discussions.
The use of Quizizz games

helps learners to develop 0% 0% 2% 11% 87%
their critical thinking skills.

their communication skills.

their pedagogical skills.
End of the lesson
526
Statements
Very Very
Not at all Neutral Somewhat
little much
The use of Quizizz allows

students to show their
0% 0% 2% 6% 92%
knowledge of the contents
learned.
Quizizz helps students to

reflect on their own 0% 0% 1% 5% 94%
progress.
Quizizz games allow

students to receive 0% 0% 1% 8% 91%
immediate feedback.

engages students to 0% 0% 2% 9% 89%
participate in the next class.
The use of Quizizz

reinforces students’
0% 0% 2% 22% 76%
linguistic and pedagogical
abilities.
Table 2. Students’ perceptions
Statements and stages of the Not at Very little Neutral Somewhat Very
lesson all much
I feel motivated when using

Quizizz at the beginning of the 0% 0% 5% 24% 71%
lesson.
I pay attention to the topics

that will be discussed in the 0% 0% 3% 23% 74%
lesson.
I can answer or respond to

questions based on activities
0% 0% 5% 23% 70%
and resources used by the
teacher.
I like the ICT tools used by the

teacher when presenting the 0% 0% 2% 22% 76%
lesson.
I like the resources used by the

teacher when introducing the
lesson: PowerPoint 0% 0% 1% 16% 83%
presentations, pictures, realia,
games, videos, songs, others.
527
Statements and stages of the Not at Very little Neutral Somewhat Very
lesson all much
Middle of the lesson
I like Quizizz to practise the

0% 0% 3% 14% 83%
contents of the lesson.
The use of Quizizz allows me

to participate in class 0% 0% 3% 8% 89%
discussions.

helps me to develop my 0% 0% 2% 14% 84%
critical thinking skills.

communication skills.

pedagogical skills.
End of the lesson
The use of Quizizz allows me

to show my knowledge of the 0% 0% 2% 12% 86%
contents learned.
Quizizz helps me to reflect on

0% 0% 2% 6% 92%
my own progress.
Quizizz games allow me to

0% 0% 3% 13% 84%
receive immediate feedback.

engages me to participate in 0% 0% 2% 12% 86%
the next class.
The use of Quizizz reinforces

my linguistic and pedagogical 0% 0% 3% 25% 72%
abilities.
Table 3. Results of the students’ interview
Stage Students’ perceptions in the interview
Beginning of • Students were motivated when using Quizizz as they were

the lesson engaged with the contents presented in the class.
• The topics were interesting because they were related to the
contents that were going to be discussed in the lesson.
528
• The questioning strategy allowed students to understand the

topics of the class because they had the opportunity to relate
previous knowledge to the new content presented.
• Students liked the resources and ICT tools used by the teachers
when presenting the lesson due to their easy access and appealing
design.
Middle of the • Students enjoyed practicing the contents by using Quizizz because
lesson this tool promoted interaction and collaboration.
• Students developed their critical thinking skills by using Quizizz
games based on the contents studied because they had to analyse,
compare, contrast and evaluate information.
• Quizizz games helped students to practice their communication
and pedagogical skills because they were aligned to the course
syllabus.
End of the • Students could demonstrate their knowledge by using Quizizz
lesson because the games were related to the contents studied in the
lesson.
• Quizizz games helped students to reflect on their own progress
because they received immediate feedback.
• Quizizz games engaged students to participate in classroom
activities, and they developed their linguistic and pedagogical
skills because they could check their own progress.
Regarding the participants’ perceptions of the use of the GBL approach in the EFL
classroom, most of them (95%) affirmed that they felt motivated when using
Quizizz at the beginning of the lesson (see Table 2). These results were confirmed
by the interviews (see Table 3), in which students mentioned that they were
engaged with the contents presented in the class because Quizizz was motivating.
Also, the majority of the participants (97%) mentioned that at the beginning of the
lesson, they paid attention to the topics that were going to be discussed. In fact,
the interview results evidenced that the topics to be analysed in the lesson caught
the students’ interest. As Dhamayanti (2021) asserts, the use of Quizizz in the EFL
classroom can increase learners’ motivation and enthusiasm because the features
of this tool enhance students’ interest and prevent monotony during the lesson.
Furthermore, 93% of the students perceived that the activities and resources used
by their teachers helped them when responding to questions. According to the
participants’ interviews, the questioning strategy allowed them to understand the
topics of the lesson because they had the opportunity to relate previous
knowledge to the new content presented. Certainly, when using Quizizz, students
can evaluate their progress and realise which contents need to be reinforced;
moreover, games increase their eagerness to respond quickly and learn
challenging topics (Munawir & Hasbi, 2021).
As for the use of ICT tools and resources, 98% of the participants affirmed that
they liked the tools used by the teachers at the beginning of the lesson. In this
respect, the results of the interview evidenced that students liked the ICT
resources instructors utilised when presenting the lesson due to their easy access
529
and appealing design. In addition, 99% of the participants liked the resources used
by the teachers to introduce the new topics; these tools included PowerPoint
presentations, pictures, realia, games, videos and songs, among others, which
were utilised at the beginning of the lesson. In this respect, Kupchyk and
Litvinchuk (2021) affirm that the use of technology can enhance teaching practice
through highly engaging activities and foster active learning inside and outside
the classroom.
In the middle of the lesson, 97% of the students affirmed that they liked to use
Quizizz to practice the different concepts taught in the lessons, while only 3% of
them had a neutral position with respect to this aspect. In fact, according to the
interviews, the participants affirmed that they enjoyed practising the contents
through games because Quizizz promoted interaction and collaboration among
them. Certainly, Zulfa and Ratri (2022) affirm that Quizizz offers interesting
features that can be used by teachers and students during e-learning activities,
which means that this tool has many advantages in terms of enhancing learners’
experiences interactively.
Regarding the use of Quizizz to promote students’ participation in class

discussions, 89% of them affirmed that this tool was very helpful for this purpose.
Similarly, most of the students (98%) indicated that Quizizz games helped them
develop their critical thinking skills because, according to the interview, gamified
activities allowed them to analyse, compare, contrast and evaluate information.
In this context, Nashar et al. (2021) assert that the use of Quizizz enhances
learners’ motivation, and it improves their critical thinking abilities, especially
when teachers choose an appropriate model to teach their lessons.
With respect to the effectiveness of Quizizz games to help learners develop their
communication skills, 98% of them perceived that this tool was helpful for this
purpose. These results are aligned with Dhamayanti’s (2021) study, it was found
that the use of Quizizz promotes good communication among students in the EFL
classroom. In relation to the use of Quizizz games to develop students’
pedagogical skills, 95% of the participants perceived that the different games
designed by their teachers were useful. Besides, the interviews revealed that
learners liked using Quizizz games because they were aligned with the course
syllabus.
As for the learners’ views with respect to the use of the GBL approach, at the end
of the lessons, most of them (98%) indicated that Quizizz allowed students to
prove their understanding of the contents studied. The results of the interview
showed that learners perceived that Quizizz games were designed in relation to
the lessons. Furthermore, 92% of the participants claimed Quizizz was helpful, as
this tool enabled them to reflect on what they were learning in the courses. These
results are similar to those found by Mohamad (2020) since the use of the Quizizz
app benefited learners in terms of enhancing the content of the course and
acquiring self-learning skills.
Regarding the use of games through Quizizz to provide feedback, 84% of the
students recognised that the tool allowed them to receive teachers’ prompt
530
feedback; this fact was remarked by learners in the interview since the tool let
them reflect on their own progress. In connection with the point previously
mentioned, Zuhriyah and Pratolo (2020) state that Quizizz’s immediate feedback
feature provides students with the opportunity to review and rectify their
responses, effectively giving them a second chance to address any errors.
Another aspect was the participants’ opinions with respect to the use of Quizizz
games at the end of the lesson. In this context, 86% of the students believed that
the Quizizz games encouraged them to participate in the next class. These findings
are related to Göksün and Gürsoy’s (2019) study which found that the post-class
quizzes strengthened the students’ understanding and readiness for class, as they
were already aware of the pre-class quizzes. This, in turn, encouraged learners to
stay engaged in the course.
Finally, learners (72%) claimed that their linguistic and pedagogical abilities were
very much reinforced thanks to the use of Quizizz, while 25% of the participants
asserted such skills were somewhat developed. According to the interviews,
students affirmed that their linguistic and pedagogical skills were enhanced as
they were able to verify their own progress with the contents. In this respect,
Degirmenci (2021) explains that Quizizz is a beneficial and valuable tool in the
English language education field and has a favourable impact on both the
teaching and learning of this language.
5. Conclusions and Implications

The participants’ perceptions towards the use of the GBL approach at the
beginning of the lessons were mostly positive because they were engaged with
the contents presented. Certainly, the activities and ICT resources used by
teachers to introduce new topics promoted motivation, caught learners’ interest
and allowed them to connect previous knowledge to new content.
The implementation of the GBL approach through Quizizz games favoured

students’ practice and enjoyment of the different contents presented in the middle
of a lesson. Besides, this method promoted interaction and collaboration among
learners, and it helped them develop their critical thinking skills as well as their
communication and pedagogical abilities.
The use of Quizizz was evaluated by learners and found to be beneficial at the end
of the lesson since they claimed that their linguistic and pedagogical skills were
significantly improved. Participants believed that Quizizz allowed them to prove
their understanding of course content and it was helpful in reflecting on what they
were learning. Quizizz permitted learners to receive prompt feedback from
teachers and it encouraged them to participate in future classes, being a valuable
tool for enhancing learning outcomes.
This study suffered from some limitations. First, it was conducted in a single
university; future studies are, therefore, suggested to include more institutions. In
addition, the study was conducted with a limited sample of students (64), which
restricts the generalisability of the findings. Moreover, further research may
utilise a variety of ICT that allow the implementation of the GBL approach.
531
Acknowledgements
The authors are grateful to the Universidad Técnica Particular de Loja for the
support given to our research team. We also express our thankfulness to the EFL
Learning, Teaching and Technology research group for helping to spread the
word about this investigation.
7. References
Adnan, N. H., Norman, H., & Nordin, N. M. (2020). Instructor-generated hand-drawn
2D animations for ESL vocabulary learning in secondary education. The Asian ESP
Journal, 16(1.2), 67–81.
Alvarez, C., & Guevara, P. (2021). EFL teachers’ perceptions on using pedagogical
modules in Ecuador: a focus group interview. Revista Chakiñan de Ciencias Sociales
y Humanidades, (14), 42–56. https://doi.org/10.37135/chk.002.14.03
Anak, C., & Hua, T. (2021). Exploring a gamified learning tool in the ESL classroom: The
case of Quizizz. Journal of Education and e-Learning Research, 8(1), 103–108.
https://doi.org/10.20448/journal.509.2021.81.103.108
Barak, M., Watted, A., & Haick, H. (2016). Motivation to learn in massive open online
courses: Examining aspects of language & social engagement. Computers &
Education, 94, 49–60. https://doi.org/10.1016/j.compedu.2015.11.010
Bomia, L., Beluzo, L., Demeester, D., Elander, K., Johnson, M., & Sheldon, B. (1997). The
impact of teaching strategies on intrinsic motivation. ERIC Clearinghouse on
Elementary and Early Childhood Education (ERIC Document Reproduction
Service No. ED 418925). https://files.eric.ed.gov/fulltext/ED418925.pdf
Brown, H.D., and Lee, H. (2014). Teaching by principles: An interactive approach to language
pedagogy. Pearson.
Cabrera-Solano, P. (2022). Game-based learning in higher education: The pedagogical
effect of genially games in English as a foreign language instruction. International
Journal of Educational Methodology, 8(4), 719–729.
https://doi.org/10.12973/ijem.8.4.719
Council of Europe (2020). Common European Framework of reference for languages: Learning,
teaching, assessment – Companion volume. Council of Europe Publishing.
Creswell, J. W., & Creswell, J. D. (2017). Research design: Qualitative, quantitative, and mixed
methods approaches. SAGE.
Deci, E., & Ryan, R. (2000). The “what” and “why” of goal pursuits: Human needs and
the self-determination of behavior. Psychological Inquiry, 11(4), 227–268.
https://selfdeterminationtheory.org/SDT/documents/2000_DeciRyan_PIWhat
Why.pdf
Degirmenci, R. (2021). The use of Quizizz in language learning and teaching from the
teachers’ and students’ perspectives: A literature review. Language Education and
Technology, 1(1), 1–11.
http://www.langedutech.com/letjournal/index.php/let/article/view/12/1
Dhamayanti, F. I. (2021). EFL students’ perception and motivation toward Quizizz as e-
learning media in English e-classroom. Education of English as Foreign Language,
4(2), 71–78. https://doi.org/10.21776/ub.educafl.2021.004.02.03
Ebner, M., & Holzinger, A. (2007). Successful implementation of user-centered game
based learning in higher education: An example from civil engineering.
Computers & Education, 49(3), 873–890.
532
Falloon, G. (2020). From digital literacy to digital competence: the teacher digital
competency (TDC) framework. Educational Technology Research and Development,
68, 2449–2472. https://doi.org/10.1007/s11423–020–09767–4
Fredricks, J., Reschly, A., & Christenson, S. (2019). Conclusion: Status of student
engagement interventions. In J. Fredricks, A. Reschly & S. Christenson (Eds.),
Handbook of student engagement interventions (pp. 375–389). Academic Press.
https://doi.org/10.1016/B978–0–12–813413–9.00025–5
Göksün, D. O., & Gürsoy, G. (2019). Comparing success and engagement in gamified
learning experiences via Kahoot and Quizizz. Computers & Education, 135, 15–29.
Halverson, R., Kallio, J., Hackett, S., & Halverson, E. (2018). Participatory culture as a
model for how new media technologies can change public schools. The Emerging
Learning Design Journal, 3(1), 1.
https://digitalcommons.montclair.edu/eldj/vol3/iss1/1
Harmer, J. (2007). The changing world of English. In J. Harmer (Ed.), The practice of
English language teaching (pp. 13–22). Pearson Longman.
Hitosugi, C. I., Schmidt, M., & Hayashi, K. (2014). Digital game-based learning (DGBL)
in the L2 classroom: The impact of the UN’s off-the-shelf videogame, Food
Force, on learner affect and vocabulary retention. Calico Journal, 31(1), 19–39.
https://doi.org/10.11139/cj.31.1.19-39
Huizenga, J., Admiraal, W., Akkerman, S., & Dam, G. T. (2009). Mobile game‐based
learning in secondary education: engagement, motivation and learning in a
mobile city game. Journal of Computer Assisted Learning, 25(4), 332–344.
https://doi.org/10.1111/j.1365–2729.2009.00316.x
Inayati, N., & Waloyo, A. A. (2022). The influence of Quizziz-online gamification on
learning engagement and outcomes in online English language teaching. Journal
on English as a Foreign Language, 12(2), 249–271.
https://doi.org/10.23971/jefl.v12i2.3546
Irwansyah, R., & Izzati, M. (2021). Implementing Quizizz as game based learning and
assessment in the English classroom. TEFLA Journal (Teaching English as Foreign
Language and Applied Linguistic Journal), 3(1), 13–18.
https://doi.org/10.35747/tefla.v3i1.756
Jenkins, J. (2015). Global English: A resource book for students. Routledge.
Lim, C. J., & Lee, S. (2007). Pedagogical usability checklist for ESL/EFL e-learning
websites. Journal of Convergence Information Technology, 2(3), 67–76.
Kasuma, S. A. A. (2017). Four characteristics of Facebook activities for English language
learning: A study of Malaysian university students’ needs and preferences.
Advances in Language and Literary Studies, 8(3), 155–171.
http://dx.doi.org/10.7575/aiac.alls.v.8n.3p.155
Kirriemuir, J., & McFarlane, A. (2004). Literature review in games and learning.
https://telearn.archives-
ouvertes.fr/file/index/docid/190453/filename/kirriemuir-j-2004–r8.pdf
Kulpa, A. (2017). Applied gamification: Reframing evaluation in post‐secondary
classrooms. College Teaching, 65(2), 5868.
https://doi.org/10.1080/87567555.2016/1232693
Kupchyk, L., & Litvinchuk, A. (2021). Constructing personal learning environments
through ICT-mediated foreign language instruction. In Journal of physics:
Conference series (Vol. 1840, No. 1, p. 012045). IOP Publishing.
https://doi.org/10.108.8/1742–6596/1840/1/012045
533
Li, Y., & Lerner, R. M. (2013). Interrelations of behavioral, emotional, and cognitive
school engagement in high school students. Journal of Youth and Adolescence, 42,
20–32. http://dx.doi.org/10.1007/s10964-012–9857-5
Maraza, B., Cuadros, L., Fernandez, W., Alay, Y., & Chillitupa, A. (2019). Análisis de las
herramientas de gamificación online Kahoot y Quizizz en el proceso de
retroalimentación de aprendizajes de los estudiantes. Revista Referencia
Pedagógica, 7(2), 339–362.
https://rrp.cujae.edu.cu/index.php/rrp/article/view/193/216
Martín del Pozo, M., Basilotta Gómez-Pablos, V., & García-Valcárcel Muñoz-Repiso, A.
(2017). A quantitative approach to pre-service primary school teachers’ attitudes
towards collaborative learning with video games: previous experience with
video games can make the difference. International Journal of Educational
Technology in Higher Education, 14, 1–18. https://doi.org/10.1186/s41239-017-
0050-5
Mohamad, A. (2020). Student as teacher-alternative revision method via Quizizz app.
MALIM: Jurnal Pengajian Umum Asia Tenggara, 21(1), 150–159.
https://doi.org/10.17576/malim-2020-2101-12
Munawir, A., & Hasbi, N. (2021). The effect of using Quizizz to EFL students’
engagement and learning outcome. English Review: Journal of English Education,
10(1), 297–308 https://doi.org/10.25134/erjee.v10i1.5412v
Murray, N. (2016). Standards of English in higher education: Issues, challenges and strategies.
Cambridge University Press.
Nashar, Nurhasanah, A., & Fauzan, R. (2021). The effectiveness of critical thinking ability
on the basis of Quizizz application viewed from problem based learning model
in history learning of senior high school. In IOP Conference Series: Earth and
Environmental Science (Vol. 747, No. 1, p. 012046). IOP Publishing.
https://doi.org/10.1088/1755-1315/747/1/012046
Noroozi, O., Dehghanzadeh, H., & Talaee, E. (2020). A systematic review on the impacts
of game-based learning on argumentation skills. Entertainment Computing, 35,
100369. https://doi.org/10.1016/j.entcom.2020.100369
Oga-Baldwin, W. Q. (2019). Acting, thinking, feeling, making, collaborating: The
engagement process in foreign language learning. System, 86, 102128.
https://doi.org/10.1016/j.system.2019.102128
Pitoyo, M. D., Sumardi, S., & Asib, A. (2020). Gamification-based assessment: The
washback effect of Quizizz on students’ learning in higher education.
International Journal of Language Education, 4(1), 1–10.
https://dx.doi.org/10.26858/ijole.v4i2.8188
Qian, M., & Clark, K. R. (2016). Game-based learning and 21st century skills: A review of
recent research. Computers in Human Behavior, 63, 50–58.
Reinhardt, J., & Sykes, J. M. (2012). Conceptualizing digital game-mediated L2 learning
and pedagogy: Game-enhanced and game-based research and practice. In H.
Reynders (Ed.), Digital games in language learning and teaching (pp. 32–49).
Palgrave Macmillan. https://doi.org/10.1057/9781137005267_3
Rose, H. (2019). The future of English in global higher education: Shifting trends from
teaching English to teaching through English. CALR Journal, 9.
https://web.aou.edu.lb/research/Documents/Article_1_by_Dr_Heath_Rose.pd
f
Sadoughi, M., & Hejazi, S. Y. (2021). Teacher support and academic engagement among
EFL learners: The role of positive academic emotions. Studies in Educational
Evaluation, 70, 101060. https://doi.org/10.1016/j.stueduc.2021.101060
534
Selvi, M., & Çosan, A. Ö. (2018). The effect of using educational games in teaching
kingdoms of living things. Universal Journal of Educational Research, 6(9), 2019–
2028. https://doi.org/10.13189/ujer.2018.060921
Shaffer, D., Squire, K., Halverson, R., & Gee, J. (2005). Video games and the future of
learning. Phi Delta Kappan, 87(2), 105–111.
https://doi.org/10.1177/003172170508700205
Tang, J., Chen, J., Ku, D., Chao, L., Shih, T., & Weng, M. (2009). Constructing the 2D
adventure game-based assessment system. In M. Spaniol, L, Quing, K. Ralf, & H.
Lau (Eds.), International conference on web-based learning (pp. 376–385). Springer-
Verlag. https://doi.org/10.1007/978-3-642-03426-8_45
Vandercruysse, S., Vandewaetere, M., & Clarebout, G. (2012). Game-based learning: A
review on the effectiveness of educational games. In M. M. Cruz-Cuhuna (Ed.),
Handbook of research on serious games as educational, business and research tools, (pp.
628–647). https://doi.org/10.4018/978-1-4666-0149-9.ch032
Wang, A. I. (2015). The wear out effect of a game-based student response system.
Computers & Education, 82, 217–227.
Zainuddin, Z., Shujahat, M., Haruna, H., & Chu, S. K. W. (2020). The role of gamified e-
quizzes on student learning and engagement: An interactive gamification
solution for a formative assessment system. Computers & Education, 145, 103729.
Zou, D., Huang, Y., & Xie, H. (2019). Digital game-based vocabulary learning: where are
we and where are we going? Computer Assisted Language Learning, 34(6), 1–27.
https://doi.org/10.1080/09588221.2019.1640745
Zuhriyah, S., & Pratolo, B. W. (2020). Exploring students’ views in the use of Quizizz as
an assessment tool in English as a foreign language (EFL) class. Universal Journal
of Educational Research, 8(11), 5312–5317.
https://doi.org/10.13189/ujer.2020.081132
Zulfa, A. I., & Ratri, D. P. (2022). EFL students’ perceptions toward Quizziz as an
assessment tool during online learning. The Journal of English Literacy Education:
The Teaching and Learning of English as a Foreign Language, 9(1), 78–87.
https://dx.doi.org/10.36706/jele.v9i1.17378
535

Vol. 22, No. 7, pp. 535-546, July 2023
A Survey of Mathematics Pre-Service Teachers'

End-of-Teaching Practice Reflections of
Educational Contexts
Benjamin Tatira
Walter Sisulu University, South Africa
Abstract. Pre-service teachers need to identify their understanding and

beliefs about mathematics learning and teaching against the backdrop of
the environment where, they teach during teaching practice. Pre-service
teachers will one day be responsible for full-time teaching and their
current reflections of educational contexts’ knowledge serve as a point of
reference to their acquisition of teacher knowledge. Of the four
knowledge bases for teaching, namely, pedagogy, pedagogical content
knowledge, subject matter and educational contexts, the focus in this
study was on the last. This study purposed to promote and describe
mathematics pre-service teachers’ reflections of educational contexts at
the end of a six-month teaching practice session. The 2022 cohort of 140
Bachelor of Education mathematics students participated in the study.
The judgmental sampling technique was used to select 40 reflection
reports. Moreover, four students were selected for interviews
purposively based on their earlier reflections in the reports. The reflective
reports and the interview transcriptions were analysed thematically
through the lens of the Van Manen framework. The findings revealed that
most of the pre-service teachers operated at the practical reflection level
and very few attained the critical reflection stage. Student teachers took
class preparation seriously, but they were disadvantaged by lack of
teaching resources. They engaged themselves in many school duties, but
this was only helpful if they were guided by experienced teachers. In
response to some school management shortcomings, pre-service teachers
projected wishes that could improve their teaching practices.
Keywords: educational contexts; pre-service teachers; self-reflections;

teacher knowledge; teaching practice
1. Introduction
To try to stem the tide of incessant poor performance in mathematics in South
Africa, the training of new secondary mathematics teachers comes to the fore.
Good grounding of teacher knowledge in pre-service teachers (PSTs) is
fundamental for the future of the teaching profession so much that Shulman’s
(1986) seminal work on teacher knowledge was centred on initial teacher
©Author
536
education. In fact, “researchers have honed Shulman’s ideas to the extent that his
construct now forms the basis for both understanding how teachers work and
how they are educated to become teachers” (Fraser, 2016, p. 142). According to
Shulman, the skills and knowledge that teachers possess contribute immensely to
effective learning and teaching. He identified the following seven types of
knowledge: knowledge of learners; knowledge of educational contexts;
pedagogical knowledge; knowledge of educational ends, values and purposes;
pedagogical content knowledge (PCK); content knowledge and curriculum
knowledge. Shulman later discarded the first four because these were not subject-
matter specific, leading to a model with pedagogical content, content and
curriculum knowledge types. However, the effect of this model was studying
individual teacher’s knowledge in the absence of educational contexts and culture
(Ellis, 2007). In addition, learners’ temperaments in the learning process and other
educational contexts were not adequately explored by Shulman. If school and
learner contextual factors are not addressed, the resultant teacher knowledge
becomes surreal.
In the subsequent derivatives to Shulman’s models of teacher knowledge,

educational contexts’ knowledge was considered as a third type of teacher
knowledge after content and pedagogy (Gess-Newsome, 1999; Grossman, 1990).
The intersection of the three, gives rise to PCK, which is the penultimate. Gess-
Newsome (1999) regarded this model of PCK as transformative, as he believes
that PCK is “new knowledge arising from the act of transforming subject matter,
pedagogical and contextual knowledge for the purposes of instructing students”
(Kind, 2009, p. 180). Grossman (1990) also argues that teachers must possess all
PCK, pedagogy, subject matter and educational contexts. Furthermore, a teacher
needs to know the subject-matter, the instructional strategies and student
knowledge in order to teach effectively. Borich (1995) considers teaching as being
problematic since schools are bedevilled by curriculum, resource, discipline,
family and parental problems. Basically, good teachers must be able to adjust their
teaching according to prevailing contextual factors which ensures their learners
are at ease to learn.
The educational contexts’ knowledge has remarkable influences on teachers’

instructional practices through their experiences in learners’ thinking processes
and school setting. Unlike content knowledge, which is taught in teacher
education at campuses, PSTs accumulate school-contexts’ knowledge in the
teaching field. Thus, teaching practice is a necessary component of the initial
teacher education which is used intentionally to link theory and practice
(Department of Education, 2011a). Teaching practice is an opportunity for PSTs to
experience teaching in the actual environment (Nkhata et al., 2016). To bring
about the necessary links between practice and theory, teacher education uses its
component of school-based teaching practice. According to Leijen, et al., (2014),
teacher reflection is an overarching activity meant to develop PSTs’ practical
knowledge and connecting it with educational theories in teacher education
programmes. Furthermore, reflection is a critical skill and characteristic of an
effective teacher in a teacher education programme.
537
Teachers assert that learning from teaching experience is better than learning from
course work, leading to the age-old saying that “experience is the best teacher”
(Schmidt, 2010, p.3). Some researchers have highlighted the concern that
coursework on its own does not adequately prepare future teachers to link
education theories to the practical situations found in the classroom (Allas et al.,
2016; Altan & Sağlamel, 2015). Knowledge of educational contexts is basically
situated in the intricacies of everyday classroom life, for example, knowledge of
school settings and learners in a particular school. Consequently, PSTs and
beginner-teachers often experience difficulties when dealing with certain
situations encountered in everyday teaching practice and feel inadequately
prepared to deal with the challenges of their profession (Grossman et al., 2009;
Meijer, 2010).
The journey of becoming an expert mathematics teacher is career-long. This

journey involves doing conscious reflections on one’s experiences and making
deliberate attempts to improve on them. Assisting PSTs to develop reflective
practices is one of the goals of teacher preparation programmes (Moore-Russo &
Wilsey, 2014). Reflection can be in-practice (occurs during instruction and is often
spontaneous), on-practice (occurs after instruction and often careful and
deliberate) and for-practice (looking forward to future actions based on current
instruction). This study focused on reflections on-action and for-action, which
necessitated use of end-of-semester reflections and how the reflections create
future-oriented insights (Buchbinder et al., 2021). The purpose of this study was
to instil reflection skills in mathematics PSTs and characterise their reflections on
educational context knowledge at the end of a semester-long teaching practice.
The goal was to explore how PSTs interpret and manipulate the environment
within which they taught secondary school mathematics. The research question
was, “What is PSTs' understanding of reflection on educational contexts and how
does it manifest in practice according to their reflections?” Encouraging PSTs to
reflect on their teaching in a real classroom context provides PSTs with much
needed pedagogical repertoires which have the potential to provide an
understanding of their teaching practices through a commitment for professional
and/or personal growth (Nyaumwe & Mtetwa, 2011).
2. Literature and theoretical framework

The mathematics knowledge for teaching can be summed up as the ability to
know what to teach and how to teach it in each educational context. This is what
Shulman (1986) described as PCK. Shulman (1986, p. 9) in his seminal work on
PCK posits that “there are no single most powerful forms of representation ...” but
teachers should have multiple ways to present given subject matter and make it
meaningful to their learners in diverse educational contexts (Umugiraneza et al.,
2017). The content, pedagogical content and general pedagogy types of
knowledge are taught in teacher education in content, methodology and theory
of education courses respectively, but the actualisation thereof is accomplished
during school teaching practices. Teaching practice is incomplete without PSTs
reflecting on their practice. Reflection acts as evaluation on what worked and did
not work, as well as making decisions for future instructional practices based on
outcomes of the evaluation. To that effect, a study by Almusharraf (2020)
examined the effect of using e-portfolios during teaching practice in developing
538
PSTs’ reflections in an English language course. After considering eight Saudi

PSTs, the findings revealed that while they struggled developing appropriate
teaching philosophies, their levels of reflection improved throughout the eleven-
week period. In another study, Nyaumwe and Mtetwa (2011) sought to give
insight to the on-going debate on ways to improve PST instructional practice
through reflection. By applying the cognitive theory of collaborative reflection to
paired end-of-teaching practice reflection by mathematics PSTs, Nyaumwe and
Mtetwa discovered that reflective dialogues were effective as a means of
providing PSTs with multiple perspectives on implementing and interpreting
teaching strategies in mathematics.
To investigate the assertion that a teacher learns better from teaching experiences
rather than from teacher education, Schmidt (2010) investigated how six pre-
service teachers valued teaching experiences whilst they were enrolled in a music
teacher education programme. Data for that study was collected using PSTs' self-
and instructor assessments over a period of two years. Making use of Dewey’s
(1933) theory of experience framework, findings revealed that participants were
active in their teaching experiences, they each created continuity and derived their
own meanings from them. A balance between action and reflection produced
meaningful learning from all types of teaching experiences, both as individuals
and/or within a community of educators.
The PSTs’ reflections on educational contexts during teaching practice were

viewed and analysed through the lens of Van Manen’s (1977) theoretical
framework. Van Manen developed a three-level hierarchical structure to
understand the development of reflectivity, which consists of technical, practical
and critical. The lowest category is technical reflection which focusses on the
teacher, what happens in the classroom and the post-lesson reflections of events
that occurred during the lesson. For example, technical focusses on whether
learners have done what the teacher asked them to do. At practical reflection level,
PSTs reflect on clarifying assumptions and predispositions while assessing the
educational consequences to which teaching actions lead. In accordance with Van
Manen (1977), the practical reflection facilitates “an interpretative understanding
both of the nature and quality of the educational experience, and of making
practical choices” (p. 226). PSTs are concerned with the learners’ learning
experiences as well as the underlying and predispositions of classroom practice
(Wong et al., 2016). Critical reflection, the highest level, involves deep-thinking
and self-assessment to which a PST conceives broad understanding of the ethics
and morals of an instructional environment. At this level, a PST possesses
interpretive skills of learning outcomes in response to teachers’ actions during
instruction. Critical reflection focusses on the context and the value of knowledge.
Hence, the PSTs’ end-of-teaching reflections fell into the practical and critical
categories.
3. Methodology
This study used a case study design, which is an in-depth empirical inquiry of an
existing phenomenon in its natural settings (Yin, 2014). The author wanted to
determine the mathematics PSTs’ reflections of the educational learning
environment at the end of the school-based teaching practice period. The
539
participants were a cohort of the 2022 final-year Bachelor of Education degree

students majoring in mathematics and a science subject, both at the senior
secondary level. Students in the faculty of education at the university where this
study took place started with school observation in the first and second years of
their study. In the third and fourth years, they spent five weeks and six months
respectively in schools where they practiced teaching under the guidance of a
qualified teacher. Nyaumwe (2001) identified four models of teaching practice,
which are apprenticeship, reflective-practice, practical theorising and theory into
practice. This university adopted the mentorship model of teaching practice,
where a PST is paired with a host teacher at the school. Thus, each PSTs had two
mentors, one for the mathematics class and the other for the science class to which
they were allocated.
The 2022 cohort of 140 Bachelor of Education mathematics students took part in
the study. A judgmental sampling technique was used to select 40 reflection
reports. As a qualitative study, data saturation was arrived at 40. According to
Creswell and Creswell (2018, p. 262), “one stops collecting data when the
categories (or themes) are saturated: when gathering fresh data no longer sparks
new insights or reveals new properties.” In addition, four students were selected
for interviews purposively based on their earlier reflections in the reports. Data
was in the form of end-of-teaching reflections written by all students. Guided
reflection was adopted in this study to elicit certain kind of responses in the
reflection. The main guiding precepts centred on the PSTs’ strengths, weaknesses
and the ways they could do teaching practice differently if given another chance.
Follow-up semi-structured interviews were conducted with individual PSTs to
get a deeper understanding of their reflections. The interviews were recorded and
transcribed by the author. For clarity of data presentation, participants were
labelled P1, P2 and so on. Thematic data analysis was employed where the author
read and re-read all the written reflections and transcriptions to identify patterns
and similarities emerging from the data. The author used the Van Manen levels
to determine the categories of the different levels of PSTS’ reflections in the data.
However, the technical level was not applicable in this study as it focusses on
immediate post-lesson reflections, and it circumvents the contexts of the society
and/or the school (Ballard, 2006). The use of a case study design leads to the
following limitations: the non-generalisability of results from one case study to
another and the possibility of researcher bias, emanating from personal
preferences and opinions. Moreover, the limitation of this study was that it was
conducted with one cohort of PSTs from one teacher education programme.
4. Findings
As mentioned earlier, PSTs were given guidance on what to reflect but in their
responses, some aspects of educational contexts emerged as contributing factors
to the PSTs' training towards becoming mathematics teachers. The results
revealed some insights into PSTs’ appreciation of educational context knowledge
in their quest of becoming effective mathematics teachers. The narrative analysis
of PSTs’ reflections revealed some broad categories of PSTs’ perceptions of
educational contexts as they pertain to teaching mathematics. Studies by Bada &
Jita (2023) and by Boateng and Tatira (2023) also identified broad themes
540
emerging from data as part of data analysis. The categories are stated below, and
evidence is provided under each category.
4.1. Preparing for the class

The participants knew that being PSTs, lesson planning was expected of them.
Moreover, some participants also had further preparations to do, P6 used a
preparation book which contained more information on the problems and their
solutions that were to be used to support learning of a mathematics concept. P9
called it the planning book but it served the same purpose as the preparation book.
P2, admitted he “could not take part in after-school activities due to lesson preparation”.
Some like P30 said they never went to class unprepared, which was supported by
P1 who said, “I do prepare well my lesson because without preparation, no meaning
teaching will be taken place”. Preparation for teaching also meant getting the
learning and teaching support materials (LTSM) ready for instruction. But using
electronic resources meant making necessary preparation. P2 posited one needs
to put LTSMs in place beforehand since digital resources can easily malfunction –
“technology easily fail”. P2 used a laptop and projector in his classes.
4.2. Availability of resources

In the wake of digital technologies, most PSTs attempted to incorporate them in
their lessons. It was smooth sailing for P1 who said, “Use of resources such as ICT
were making things easier for me because I was able to show them the things I was
presenting through video clips by using projectors and laptops”. However, it was not
easy for most PSTs to use digital resources due to their non-existence in rural
schools chosen by many to do their teaching practices. P5 said that his school was
unprepared for e-learning. Not only was his school and others short of electronic
resources for learners-use, internet, printing and copying facilities were non-
existent, according to P5. For P10, his school had “no textbooks so learners had to
depend on what I say”. P4 lamented about the limited resources at his host school.
P6 and P9 encountered a sad reality whereby learners had no scientific calculators
due to poor economic backgrounds. P6 further said that learners could not use the
calculators as a result of the economic backgrounds. As a matter of fact, learners
use calculators “to perform standard numerical computations” (DBE, 2011b, p.
54), and starting in Grade 11, usage of calculators is mandatory, specifically in
Statistics. The Curriculum and Assessment Policy Statement states that “variance
and standard deviation may be calculated using calculators.” (DBE, 2011b, p. 15).
The cumbersome computations of regression analysis and coefficient of
correlation in Grade 12 are entirely calculated using a calculator.
4.3. Extra school duties

Being in the teaching field, PSTs found themselves responsible for extra duties in
addition to instruction of the two major subjects. P3, was “overloaded with full
teacher duties and other non-majored subjects.” The concern by P3 is the overloading
of duties to someone who is still under training. Other duties are normal as they
induct PST into the profession, for example, conducting assembly (mentioned by
P40), setting examination papers, marking the examinations, recording marks and
invigilation. In the interview, P40 posited that “teaching is more than standing in
front of learners”. P30 and P22 “also did administration work and invigilation over and
above teaching duties.” Moreover, any other duties may be done by PSTs as and
541
when necessary, but they should not shoulder the burden alone. For instance, P27
“played multiple roles of counsellor, parent, social worker, psycho-teacher alone since these
teenagers spent more time at school than at home”. It was going to be a noble
experience for P27 had he carried out those duties under the guidance of an
experienced teacher. Nevertheless, sometimes PSTs are given duties beyond the
expected simply because the school or the mentor does not fully know their roles
in coaching PSTs. This happens when universities that have teacher education
programmes do not conduct training or awareness to schools on the expectations
of the roles of schools, mentors and PSTs (Nyaumwe & Mtetwa, 2011). At the same
time some schools were hard hit by teacher shortages in mathematics and sciences
so much that they intentionally recruit PSTs to fill that gap. P28 posited that he
“taught 6 classes including grade 12 with no mentors because of serious teacher shortage.”
In South Africa, Grade 12 is the final grade where learners write the national
examinations in preparation for university. Hence, we have the situation where
PSTs perform all duties of a teacher without guidance.
4.4. Job preparation

The goal of some PSTs to do the final-year second semester teaching practice was
to fulfill the degree programme requirements and qualify to be mathematics
teachers. Many PSTs concurred that this final six-month teaching period was
sufficient to see them as qualified teachers (P6, P7, P38 and P32). In a way, they
regarded this experience as a means to qualify and get a job. “School-based
experiences were enough preparation for next year”, said P6. On the other hand, some
PSTs valued the life-long skills they gained through this exercise (P11, P29, P39
and P33). For example, P11 said that the “experience prepared me for teaching field”,
which was supported by P29 who said, “I will take everything my mentor taught me
throughout my teaching career.” The follow-up interview with P11 revealed that
teaching practice marks the end of the degree programme and he can confidently
walk into any mathematics classroom and deliver.
4.5. Extra classes and envisioned practice

PSTs obviously could not rectify the problems they encountered in schools, for
example, the issue of limited resources. PSTs acknowledged that indeed it was the
responsibility of the school management to do that, but they still voiced their
envisioned practices in their reflections. To increase contact time with their
classes, P26, P9, P11, P25, P5 and P14 wished to conduct extra weekend/afternoon
classes. “I conduct Saturday classes to increase mathematics fluency and familiarity with
calculator usage” cited P26. The desire to have more contact time with classes was
shared by many PSTs. To P10, the desire was rooted in the fact that he was
allocated a weak class so that the extra time is to be used to uplift the condition of
the class-understanding of mathematics. In P15’s case, her learners actually “asked
for extra classes being eager to learn”. Some PSTs admitted they improved their
respective pass rates in the six-month period but they felt they could have done
better had they taught for longer period. P17 explained: “I could have improved or
produced a high passing rate on my class if I found them beginning of the year. Yes, I
improved the results a little bit but I could have done it better than this given enough
time.” Nonetheless, P8 and P18 saw positive results after they successfully
participated in extra lessons. They reported, “my passes rose from 30% to 54% due
to extra lessons” and “improve mathematics results from 55% in term 2 to 87% in term
542
3 – extra lessons played a key role” respectively. In response to some under-

privileged school environments they encountered, P3, P20 and P21 proposed to
build more classrooms, buy own printer and make textbooks available
respectively. Asked how they would do it if school authorities could not provide
such, P3 suggested that it was his desire to work with donors for financing his
projects.
4.6. School management and administration

The realisation of the PSTs’ envisioned practices rested in a good supportive
environment created by the school management team. The school management
was very supportive of P23 who was coached how to handle learner-discipline
and other aspects of the teaching profession whilst P24 was given strict dos and
don’ts on the very first day. P34 was also given rules but they were suppressive
as he was “always reminded that you are a preservice teacher and not allowed to be free
thinkers”. That left P34 no room for initiative in pursuing his aspirations of
becoming a mathematics teacher of his choosing. Two PSTs (P19 and P35)
indicated that they had no choice but to stick to the supplied annual teaching
plans which dictate what was to be taught for that particular period of the year.
P19 said she was “stuck to the annual teaching plan with no flexibility even when faced
with mixed-performing learners.” Furthermore, P16 faced difficult situations as a
result of the school management’s laxity in some instances. She said that
“absenteeism was a challenge: you teach half a class today and another half tomorrow and
the management did not do anything about it.” On the other hand, P6, P12, P13, P37,
P32 and P36 were affectionately treated as professional teachers by the school
management and other teachers, so that they were given absolute control of their
classes. Regrettably, P31 was not only regarded as a colleague by the teachers, but
learners treated her as a peer by virtue of her age. Hence, PSTs still looked up to
the school management team for school-wide contextual factors in order to
maximise their teaching endeavours.
5. Discussion
An important characteristic of effective teaching is knowing what needs to be
done and when to do it. This has pushed to the fore the preparation of reflective
teachers, which has become a central theme in teacher education (Kaasila &
Lauriala, 2010). A teacher should be able to think deeply about the educational
consequences of a teaching action and conceive broad understanding of an
instructional environment. Reflection is a process of self-inquiry and learning
through introspection in order to realise possible visions of practice (Johns, 2010).
This was intended to realise a collaboration of researchers, school mentors and the
PSTs in order to understand the contextual nature of desirable practices and the
possible factors that limit attaining these practices. This coincides with the
practical and critical levels of Van Manen’s (1977) model of PSTs reflection and
fits well with the knowledge of educational contexts (Gess-Newsome, 1999).
Most of the PSTs’ reflections fell in the category of practical reflections, whereby
they narrated the description and interpretation of the school environments they
encountered. This was the case for the categories of lesson preparation, shortage
of teaching resources, extra non-teaching duties, job-training and activities of the
school management teams. For example, P34 and P19 narrated about rigid
543
teaching environments whereby they were always told what to do and instructed
to follow annual teaching plans in curriculum delivery respectively. PSTs’
endeavours to think critically and act on those thoughts were thwarted. However,
the goal of every teacher-education programme is to inculcate critical reflections
by creating opportunities for deep thinking. A few students alluded to the critical
reflection stage when they attempted to give an envisioned practice to the
challenges they identified in the course of instruction in their respective school
environments. These PSTs sought to manipulate the teaching environment to suit
their espoused visions of practice. Reflection involves looking backward and
forward realising “new understandings, greater insights and/or greater
responsibility for future actions” (Anderson, 2019, p. 2). The new insights came
because of some teaching challenges shortage of resources and poor infrastructure
which acted as a hindrance (John, 2019; Bada & Jita, 2023).
Reflection on broader educational contexts remains one of the ways to help PSTs
to better understand what they claim to know and do as they develop their
knowledge of teaching through engaging educational theories, they learn in
teacher education (Loughran, 2002; Leijen et al., 2014). PSTs can perform
complementary duties to instruction but need to be assisted by a qualified teacher
according to the mentor model. However, without any form of training or
induction on mentoring PSTs, some mentors may not know how to offer proper
guidance to PSTs. Reflection of the six-month teaching practice experience builds
confidence in becoming a mathematics teacher as well as raising awareness of
career opportunities that lie ahead. The reflections for the final-year teaching
practice places PSTs at the threshold of mathematics teaching whereby they
would be required to navigate a myriad of educational contextual factors which
both militate and support mathematics instruction. Already, there are concerns
that the initial teacher education programmes do not adequately prepare PSTs to
link theories of learning to the practical classroom and school situations (Allas et
al., 2016). Consequently, beginner-teachers grapple with everyday teaching
practice situations (Meijer, 2010) and feel unprepared to deal with some inherent
challenges of the teaching profession. Even though teacher knowledge is
unconscious and tacit, it is vital to commit PSTs to conscious reflection, which in
turn can be shared and used as a reference point for action (Buchbinder et al.,
2021). The goal for teacher education is to motivate PSTs to take ownership of their
career development and build confidence to make decisions and risks in an ever-
changing education landscape. PSTs’ experiences in the school setting emphasise
self-reflection to be useful in making them good teachers (Boateng & Tatira, 2023).
The PSTs’ practical reflections of educational contexts were detailed and

encompassing, which covered both on-action and for-action. However, they were
presented at pre-critical levels which depicts lack of interpretive skills of
educational consequences where teaching action culminates (Nyaumwe &
Mtetwa, 2011). A PST conceives a broad understanding of an instructional
environment through deep-thinking and self-assessment. Teacher educators
should inculcate critical reflection skills in the methodology courses to make it
easier for PSTs to enact them during teaching practice. A shift away from the
currently used mentor model of teaching practice also facilitates PSTs to reflect
objectively. Currently, PSTs tend to model their mentors by observing, imitating,
544
getting advice on how to teach and overcome other educational issues (Samuel,
2010), instead of becoming themselves. Sometimes mentors dictate what they
deem to be good teaching practices and PSTs are bound to listen. Hence, the
critical reflection aspect is conspicuous as the PSTs fail to construct and
understand their own teaching identities.
6. Conclusion
Conscious reflection of context knowledge may improve secondary school
mathematics attainment through the development of teachers’ PCK. This occurs
when PSTs narrate and critically reflect their experiences in educational contexts
knowledge. Reflection is central to teaching and PSTs’ reflections on educational
contexts contribute to the development of their PCK. PSTs demonstrated the
ability to give practical reflections on the processes of teaching and learning, how
to work in diverse school climates and simultaneously make visionary judgments
leading to modifications of their practices to match the needs of learners.
Although the findings of this study may not apply in all settings, it is important
for teacher educators to recognise the roles played by Van Manen’s levels of
reflections of technical, practical and critical in order to help PSTs balance their in-
depth self-assessment of educational contexts as they consummate teacher
education.
The study has implications for education faculties to assist PSTs develop balanced
critical reflections in teacher education which they would use to navigate the
diversity of educational contexts during teaching practice and beyond. The PSTs
are still expected to be critical even in the South African education system which
unilaterally provides annual teaching plans for all subjects in the curriculum,
which in a way constrains teacher initiatives regarding topic and duration of
teaching units. In addition, the study recommends the national Department of
Education partners with teacher education to conscientise school-based mentors
on the revolving expectations PSTs undergoing teaching practices using the
mentor model. (Boateng & Tatira, 2023). This is critical because all teaching
practices in the South African teacher education system are- based on the
apprentice model. Future research can explore the types of practical and critical
reflections PSTs communicate in content and methodology types of teacher
knowledge in different social and cultural contexts.
7. References
Allas, R., Leijen, A., & Toom, A. (2017). Supporting the Construction of Teacher’s Practical
Knowledge through Different Interactive Formats of Oral Reflection and Written
Reflection. Scandinavian Journal of Educational Research, 61(5), 600-615.
https://doi.org/10.1080/00313831.2016.1172504
Almusharraf, A.M. (2020). Student Teachers’ Development of Reflective Practice
concerning Teaching Philosophy and Peer Observations. Arab World English
Journal, 11(4), 547-564. https://dx.doi.org/10.24093/awej/vol11no4.35
Altan, M.Z., & Sağlamel, H. (2015). Student teaching from the perspectives of cooperating
teachers and pupils. CogentEducation, 2(1), 1-16.
https://doi.org/10.1080/2331186X.2015.1086291
Anderson, J. (2019). In search of reflection-in-action: An exploratory study of the
interactive reflection of four experienced teachers. Teaching and Teacher Education,
86, 1–17.
545
Bada, A.A., & Jita, L.C. (2023). Teaching practice: Experiences of pre-service physics
teachers. EURASIA Journal of Mathematics, Science and Technology Education, 19(6),
em2281. https://doi.org/10.29333/ejmste/13233
Ballard, K.K. (2006). Using Van Manen’s Model to Assess Levels of Reflectivity among Preservice
Physical Education teachers. Unpublished doctoral thesis. Texas A&M University,
College Station, Texas.
Boateng, S., & Tatira, B. (2023). Voices from the Field: Pre-Service Teachers’ First Time
Experiences of Teaching Physical Sciences during School-Based Experience.
International Journal of Learning, Teaching and Educational Research, 22(6), 59-77.
Borich, G. D. (1995). Becoming a teacher and inquiry dialogue for the beginning teacher. The
Flamer Press.
Buchbinder, O., Brisard, S., Butler, R., & Mccrone, S. (2021). Preservice Secondary
Mathematics Teachers’ Reflective Noticing from 360-degree Video Recordings of
Their Own Teaching. Journal of Technology and Teacher Education, 29(3), 279-308.
Creswell, J. W., & Creswell, J. D. (2018). Research Design: Qualitative, Quantitative, and Mixed
Methods Approaches 5th Ed. Sage.
Department of Basic Education. (2011a) Integrated strategic planning framework for teacher
education and development in South Africa, 2011–2025. Pretoria: Departments of
Basic Education and Higher Education and Training. Retrieved from
https://www.education.gov.za/Informationfor/Teachers/ISPFTED2011-
2025.aspx
Department of Basic Education. (2011b). Curriculum and Assessment Policy Statement
(CAPS) FET Band Mathematics Grades 10-12. Department of Basic Education.
Dewey, J. (1933). How we think: A restatement of the relation of reflective thinking to the
educative process. Houghton Mifflin Company.
Ellis, V. (2007). Taking subject knowledge seriously: from professional knowledge recipes
to complex conceptualizations of teacher development. Curriculum Journal, 18(4),
447-462.
Fraser, S. P. (2016). Pedagogical Content Knowledge (PCK): Exploring its Usefulness for
Science Lecturers in Higher Education. Res Sci Educ, 46, 141-161.
https://doi.org/10.1007/s11165-014-9459-1
Gess-Newsome, J. (1999). Pedagogical content knowledge: An introduction and
orientation. In J. Gess-Newsome & N.G. Lederman (Eds.), Pedagogical content
knowledge and science education: The construct and its implications for science education
(pp. 21–50). Kluwer.
Grossman, P.L. (1990). The making of a teacher: Teacher knowledge and teacher education.
Teachers College Press, Columbia University.
Grossman, P., Hammerness, K., & McDonald, M. (2009). Redefining teaching, reimagining
teacher education. Teachers and Teaching: Theory and Practice, 15(2), 273-289.
John, M. (2019). Physical sciences teaching and learning in Eastern Cape rural schools:
Reflections of pre-service teachers. South African Journal of Education, 39(1), S1-S12.
https://doi.org/10.15700/saje.v39ns 1a1660
Johns, C. (2010). Guided Reflection: A Narrative Approach to Advancing Professional Practice.
Wiley Online Library.
Kaasila, R., & Lauriala, A. (2012). How do pre-service teachers’ reflective processes differ
in relation to different contexts? European Journal of Teacher Education, 35(1), 77-89.
https://doi.org/10.1080/02619768.2011.633992
Kind, V. (2009). Pedagogical content knowledge in science education: perspectives and
potential for progress. Studies in Science Education, 45(2), 169-204.
546
Leijen, A, Allas, R., Toom, A., Husu, J., Marcos, J. M., Meijer, P., Knezic, D., Pedaste, M., &
Krull, E. (2014). Guided reflection for supporting the development of student.
Procedia - Social and Behavioral Sciences, 112, 314-322.
Loughran, J. J. (2002). Effective reflective practice: In search of meaning in learning about
teaching. Journal of Teacher Education, 53, 33-43.
Meijer, P. C. (2010). The teacher education knowledge base: Experienced teachers’ craft
knowledge. In E. Baker, P. Peterson & B. McGaw (Eds.), International encyclopedia
of education (3rd ed., pp. 642–649, vol. 7). Elsevier.
Moore-Russo, D. A., & Wilsey, J. N. (2014). Delving into the meaning of productive
reflection: A study of future teachers’ reflections on representations of teaching.
Teaching and Teacher Education, 37, 76-90.
Nkhata, L., Chituta, D., Banda, A., Choobe, B., & Jumbe, J. (2016). Mathematics and Science
Student Teachers’ Perceptions of Their Classroom Management Practices during
Practicum: A Case of Copperbelt University Students. American Journal of
Educational Research, 4(15), 1106-1115. https://doi.org/10.12691/education-4-15-9
Nyaumwe, L. (2001). A Survey of Bindura University of Science Education Student
Teachers' Perceptions of the Mentoring Model of Teaching Practice. Zimbabwe
Journal of Educational Research, 13(3), 230-256.
Nyaumwe, L. J., & Mthethwa, D. K. (2011). Developing a cognitive theory from student
teachers’ post-lesson reflective dialogues on secondary school mathematics. South
African Journal of Education, 31, 145-159.
Schmidt, M. (2010). Learning From Teaching Experience: Dewey’s Theory and Preservice
Teachers’ Learning. Journal of Research in Music Education, 58(2), 131-146.
https://doi.org/10.1177/0022429410368723
Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational
Researcher, 15(2), 4-14.
Umugiraneza, O., Bansilal, S., & North, D. (2017). Exploring teachers’ practices in teaching
Mathematics and Statistics in KwaZulu-Natal schools. South African Journal of
Education, 37(2). https://doi.org/10.15700/saje.v37n2a1306
Van Manen, M. (1977). Linking ways of knowing with ways of being practical. Curriculum
Inquiry, 6, 205-228.
Wong, Y.M., Mansor, R., & Samsudin, S. (2016). Student teachers' level of reflection during
teacher clinical experience: A case study in a Malaysian university. Journal of
Research, Policy & Practice of Teachers & Teacher Education, 6(2), 23-32.
Yin, R. K. (2014). Case study research: design and methods (5th Ed.). SAGE.
547

Vol. 22, No. 7, pp. 547-565, July 2023
Flipped Direct Instruction (FDI): A New

Practicum Learning Model in Vocational
Education
Akrimullah Mubai , Ambiyar* and Dedy Irfan
Universitas Negeri Padang, Padang, Indonesia
Mohamad Sattar Rasul

Universiti Kebangsaan Malaysia, Malaysia
Abstract. The Flipped Direct Instruction (FDI) learning model was

introduced as a new learning model for practicum learning. It was
developed to mitigate the limitations of the Direct Instruction (DI) model
in improving vocational education graduates’ competencies. Therefore,
the purpose of this study is to examine the formation of the FDI learning
model. This research uses an exploratory sequential mixed methods
design with Mile and Huberman methods and Confirmatory Factor
Analysis (CFA). The FDI learning model is formed through conceptual,
theoretical, hypothetical, and final modeling stages. Research data were
obtained through document analysis and Focus Group Discussion (FGD).
FGD was conducted with 7 experts as the research sample and non-test
instruments were used. The result of this study is an FDI learning model
formed from several theories including Joyce (2003), Cooper (2012),
Ktoridou (2018), and Indrajit (2021). The FDI model consists of an
orientation phase, procedure-based simulation phase, case-based
simulation phase, structured practice phase, guided practice phase, and
independent practice phase. Each phase obtained CFA values of 0.464,
0.492, 0.292, 0.009, 0.016, and 0.018. All values are below 2, revealing that
the FDI model has a goodness-of-fit-model criteria. The FDI model is
presented as a contribution to the development of science and as a new
learning model option recommended for vocational education. In the
future, this research will contribute to the evaluation of the FDI learning
model in measure its level of practicality, effectiveness, and impact on
model application.
Keywords: Flipped Direct Instruction; New Learning Model; Vocational

Education
*
Corresponding author: Ambiyar, ambiyar@ft.unp.ac.id
©Authors
548
1. Introduction
The learning model has major implications for the formation of student
competence. In the process of forming student competencies, learning models are
implemented according to the mode of learning (Joyce & Weil, 2003). Popular
learning models currently used in theoretical learning are Problem-Based
Learning (PBL), Discovery Learning, Contextual Learning, Flipped Classroom,
and Project-Based Learning (Joyce & Weil, 2003; Yulianto et al., 2019). Meanwhile,
learning models widely used in practical learning are Self Directed Learning, Role
Playing, Simulation and Direct Instruction (Joyce & Weil, 2003; Yulianto et al.,
2019). Each learning model is organized in structured and systematic phases
(Joyce & Weil, 2003). For example, the Direct Instruction (DI) learning model,
conceived by Jere Brophy and Tom Good in 1986 (Joyce & Weil, 2003, p. 20), is a
learning model that consists of orientation, presentation, structured practice,
guided practice, and independent practice phases. It manages the ability to think
and behave in a practical learning environment (Joyce & Weil, 2003; Patandean &
Indrajit, 2021).
The DI model is one of the earliest learning models found and is widely used
today (Chamidy et al., 2020; Flynn et al., 2012; Glogger-Frey et al., 2015; Gurses et
al., 2015; Warju et al., 2020; Winarno et al., 2018; Winarsih et al., 2019). However,
at present, the implementation of the DI model has several drawbacks, including
the formation of students' initial knowledge. In this model, initial knowledge
cannot be adequately formed because this learning style is very dependent on the
teacher (Warju et al., 2020). In fact, initial knowledge should be obtained before
learning commences, not built during learning in the classroom (Dehham &
Albayati, 2021). This is because knowledge must be assimilated in students’ active
process and through good mental capacities, so that complexity can be developed
through understanding. Following this, understanding is assembled by students
through an equilibration process, where initial knowledge is compared against
acquired knowledge (Gurses et al., 2015; Paulsen et al., 2019).
Interaction in the Direct Instruction learning model is dominated by the teacher.

The domination of the teacher in the classroom suppresses student activity in the
learning process (Aziz et al., 2018). This forms the next problem in the DI learning
model (Gurses et al., 2015; Winarno et al., 2018). A teacher’s suppression of active
learning leads to student passivity in the classroom (Zayyadi et al., 2020).
Students’ passivity in learning reduces their ability to build their own
understanding and knowledge (Winarno et al., 2021). This reduces the interest
and motivation of students to learn. A lack of interest and motivation will make
learning boring, thereby reducing students’ enthusiasm (Warju et al., 2020;
Winarno et al., 2021; Winarsih et al., 2019). This low enthusiasm will impact
students’ ability to understand the learning material provided by the teacher.
High teacher dominance in the learning process reduces the freedom and
independence of students in learning. This in turn will reduce students’ ability to
independently adapt to their learning environment (Winarno et al., 2021; Zayyadi
et al., 2020). A lack of student independence leads to passivity (Budiman et al.,
2020; Warju et al., 2020), and passive learning inhibits students from generating
innovative ideas (Ahmad et al., 2022; Glogger-Frey et al., 2015). Students' ideas
549
impact their problem-solving ability, born from good critical thinking skills. For
this reason, critical ability is one of the weaknesses of the DI learning model
(Budiman et al., 2020; Gurses et al., 2015; Winarsih et al., 2019).
Critical thinking skills can only be built if students are granted sufficient learning
opportunities to be creative and innovative. Creativity and innovation can be
stimulated by problem-based learning or case-based learning (Courtney et al.,
2015; Ktoridou et al., 2018). Students can be given the opportunity to solve
problems or cases independently or collaboratively in teams (Winarno et al.,
2018). Independent learning allows students to build their own knowledge, while
teamwork-based learning promotes students’ creativity, innovation, and
teamwork (Aziziy et al., 2020; Nuris & Istyaningputri, 2021). In problem-solving
or case-based learning, teachers are required to always monitor student work,
provide the necessary facilities, and offer evaluative and constructive directions
and suggestions (Joyce & Weil, 2003).
In the DI learning model, independent learning and teamwork for problem

solving are not well facilitated. This is because this model has the characteristics
of guided and procedural learning (Winarno et al., 2018). Therefore, the DI model
cannot facilitate these capabilities, even though the ability to strengthen initial
knowledge, think critically, be creative, and collaborate are some of the
competencies that students need to have in the 21st century (Huda et al., 2021;
Viinikka et al., 2019). For this reason, several researchers have proposed various
solutions to maximize the use of the DI model.
Gurses et al. (2015) combined the DI model with the Constructivist Learning base,
which was named the Interactive Direct Teaching Based Constructivist Learning
(IDTBCL) model. The IDTBCL model was developed to strengthen learners’
ability to conceptualize ideas and use their mental abilities. Winarsih et al. (2019)
merged the DI model with the Problem Based Learning model in accounting
subjects. The combination was able to improve students' critical thinking skills as
measured by the ANOVA method. Warju et al. (2020) used Real Condition Video
media in the application of the DI model. It was tested in Basic Automotive
Engineering subjects with an increase in student learning outcomes. Winarno et
al. (2021) utilized multimedia in the DI model combined with Problem Based
Learning. The combination of these models is named multimedia Direct Problem
Based Learning (mDPBL). The mDPBL model is able to improve problem solving
skills and mastery of subject matter by students.
Based on the development of the DI model that has been proposed, there is still a
weakness: each proposed model has its own advantages. There is no development
of DI learning models that have comprehensive advantages. On the other hand,
the DI learning model is still widely used, especially in vocational education. This
is based on Stockard's research in 2020 which has reviewed more than 500 articles
from DI learning models which dominate in vocational education (Stockard et al.,
2020).
The implementation of the DI learning model in vocational education has not

produced graduates ready to work in Indonesia. This can be identified from the
550
number of vocational education graduates in Indonesia, who form the largest

contributor to open unemployment in Indonesia with 9.42% as of August 2022
(Central Bureau of Statistics Indonesia (BPS), 2022). Due to the weaknesses of the
DI model in improving initial knowledge, critical thinking skills, independent
learning, cooperation, and creativity of students in improving the competence of
vocational education graduates, the DI learning model requires urgent
improvement. Therefore, this study explores the development of DI learning
models from a different perspective.
The perspective of DI model development in this research occurs by combining

direct and indirect learning models. The reason for this combination is because
direct learning is excellent for teaching facts, rules, and learning sequences that
will improve technical skills, investigation, and discovery, while indirect learning
is excellent for teaching concepts, patterns, and abstractions in improving the
ability to solve problems, think critically, and collaborate (Budiman et al., 2020;
Rüütmann & Kipper, 2011). Therefore, to present a more optimal practicum
learning model, the DI model as a direct learning model is combined with the
Flipped Classroom (FC) learning model. The FC learning model is part of a
blended learning model that can be used indirectly (Staker & Horn, 2012). The
indirect learning model in the FC learning model allows students to prepare their
prior knowledge better (Patandean & Indrajit, 2021).
Stable prior knowledge promotes further creativity, critical thinking, and

collaboration to help achieve practicum learning (Gurses et al., 2015). In practicum
learning, students require cognitive skills, interpersonal skills, and attitudes that
demonstrate competent behavior in a particular field or profession (Holmes et al.,
2021). In achieving all these competencies, it is necessary to use an improved DI
learning model. Therefore, the main objective of this research is to increase the
capability of the DI learning model by combining it with the FC learning model,
where the combination of the two models is named the Flipped Direct Instruction
(FDI) learning model. Specifically, the purpose of this research is to discuss the
process of forming the FDI model from the conceptual, theoretical, hypothesis,
and final model stages as a new alternative in practicum learning.
2. Method
This study used an exploratory sequential mixed methods design research
approach to determine the impacts of the FDI learning model as a new alternative
in practicum learning. The qualitative research approach was conducted using the
Mile and Huberman method. Then, the researcher proceeded to using method
Confirmatory Factor Analysis (CFA) for quantitative research approaches
(Creswell, 2012, p. 543).
2.2. Participants of The Study

The participants in this study were 7 experts in the field of learning models. This
participant selection uses saturated sampling, where the population becomes the
sample. This is because the number of participants in the population is less than
thirty (Riyanti & Parulian, 2023).
551
2.3. Development of The Research Instrument

The instrument used in collecting data is a non-test instrument. This instrument
is based on the factors that make up the FDI model. This instrument uses a Likert
scale. The Likert scale used is one for 'strongly disagree' to five for 'strongly agree'.
The instruments used have passed the validity and reliability tests. Instrument
validity uses Aiken's V formula and reliability uses Cronbach Alpha formula. For
categories in the measurement of validity and reliability can be seen in Table 1.
Table 1. Categories of Validity and Reliability Test Data

Aiken's V value Category Cronbach Alpha r Value Category
0,81 – 1,00 Very High
0,67 – 1,00 Valid
0,61 – 0,80 High
0,41 – 0,60 Medium
≤ 0,66 Invalid 0,21 – 0,40 Low
0,00 – 0,20 Very Low
Source: (Azwar, 2019, p. 113; Tambunan et al., 2021)
The data obtained uses instruments to build the FDI model. The stages of building
the FDI model can be seen in Figure 1.
Figure 1. The Stages of Establishing The FDI Learning Model

Based on Figure 1 above, the Mile and Huberman method was used to construct
a conceptual and theoretical model of the FDI learning model. The CFA method
was used to develop a hypothetical model and the final iteration of the FDI
learning model.
2.4. Data Gathering

Data were obtained through document analysis and through Focus Group
Discussion (FGD) with 7 experts as research participants. Data obtained through
document analysis are used as a conceptual and theoretical FDI model builder
using the Mile and Huberman method. Then, the data obtained through FGD use
the CFA method to hypothetically formulate the FDI model and the final model.
2.5. Data Analysis

Mile and Huberman Method
The Mile and Huberman method is a method for processing and analyzing
qualitative data. The Mile and Huberman method has stages, namely collection,
reduction, display, and verification (Miles et al., 2014, p. 33). Each stage is carried
out to create the conceptual and theoretical nature of the FDI learning model. The
conceptual model has the form of concepts and ideas from the formation of the
FDI model. The concept is built from the analysis of problems and opportunities
obtained from each literature review, while in the theoretical model, the FDI
model has been formed in the learning phase. Here, each phase is built based on
552
the analysis of relevant concepts and theories. For this reason, each stage of the
Mile and Huberman method is used.
The first stage is data collection from document analysis. The document analysis
comes from documents in the form of articles and books themed on learning
models. At this stage, the data obtained from documents is compiled and grouped
for analysis. Data that has been arranged according to groups is processed at the
reduction stage.
At the reduction stage, the necessary data is sorted and analyzed. Analysis is
achieved by comparing, eliminating, and compiling data. Data that has been
organized by group is compared with data from other groups. The same data is
then eliminated, until one item of data is left that is representative of the removed
data. The remaining data is then combined with other data to build a complete
data set. The complete data is then presented. Data presentation is in the form of
descriptive text information which is then verified before a conclusion is made
(Miles et al., 2014). The conclusion is the result of qualitative data analysis.
Confirmatory Factor Analysis (CFA)

Confirmatory Factor Analysis (CFA) is an analytical technique that aims to
determine how many factors exist and whether these factors need to be correlated.
Such factor testing can be used in testing the construct validity of a model (Ashari
et al., 2019; Maksum & Purwanto, 2019, p. 206; Nofriansyah, 2020). The construct
validity test of a model can be used to answer the hypotheses in this study. The
hypotheses in the development of the FDI learning model consist of:
Ha: FDI learning model is valid for use

H0: FDI learning model is not valid to use
The CFA method is used to answer the hypotheses. The CFA method uses
quantitative data obtained from FGD. Obtained data were calculated using the
CFA method with the help of statistical applications, namely Lisrel 8.80. In the
CFA calculation, criteria proposed by Stevens and Mayers is used. The chosen
criteria proposed by Stevens and Mayers are the chi-square (x2) divided by the
degrees of freedom test (df), (Meyers et al., 2013, p. 871). The description of the
technique (x2/df) is presented in Table 2.
Table 2. Categorizing The Fit Index (Test)

Categorizing the fit index (test) Target values
x2/df <2
Source: (Jackson, 2018, p. 5; Meyers et al., 2013, p. 870)
Based on table 1, data (x2/df) can be considered valid if the value is below 2.
Therefore, a phase or learning model is deemed valid or as a goodness-of-fit-
model if the value of (x2/df) is below 2 (Anderson et al., 2001; Jackson, 2018;
Maksum & Purwanto, 2019).
3. Research Result
The results of this research are categorized into qualitative and quantitative
results. Qualitative results are described by themes that constitute conceptual and
553
theoretical models. On the other hand, quantitative results are described

according to the hypothesis model and the final model. In obtaining quantitative
data using non-test instruments, the results of the validity and reliability tests can
be seen in Table 3.
Table 3. Validity and Reliability Test Results of Research Instruments

Test Test Score Category
Validity 0,867 Valid
Reliability 0,751 High
Based on Table 3, the instruments used are valid and reliable. Therefore, this
research instrument is appropriate for use in obtaining research data. The
description of the data analysis of the research results is as follows.
3.1. Conceptual Models

The FDI learning model is established through the development of the DI learning
model combined with the FC learning model. The development of the FDI
learning model is needed to increase the effectiveness of using the DI learning
model as a new practicum model or alternative practicum practice. Therefore,
conceptually, the DI learning model was combined with the Flipped Classroom
(FC) learning model, as reflected in Figure 2.
Figure 2. The Concept of Forming The FDI Learning Model

Conceptually, on Figure 2, the combination of the DI and FC learning models
formed a new learning model called Flipped Direct Instruction (FDI). The FDI
model obtains indirect learning from the FC model and direct learning from the
DI model. Therefore, the concept of this new learning model requires theoretical
554
support, whether from a combination of learning models or by establishing each

phase of the FDI learning model.
3.2. Theoretical Model

The establishment of the FDI learning model is theoretically based on the theory
of Brophy in 1989 as an inventor of the DI learning model and Patandean in 2021
as a developer of the FC learning model (Joyce & Weil, 2003; Patandean & Indrajit,
2021). In particular, the formation of an FDI learning model using these two
theories is possible because the models’ characteristics are complementary. The
FC model was used for asynchronous and indirect learning, while the DI model
was for synchronous and direct learning. The formation of phases in the FDI
model is built based on several theories forming each model phase. The forming
theory of the FDI model phase is presented in Table 4.
Table 4. Theory Formation of the FDI Learning Model Phases

The Source of The Theory of
Phases Factors
Phase Formation Phase Forming
Joyce & Weil (2003, The implementation of
The
pp. 349–350) learning preparation (ORI1)
accommodation
Patandean & The availability of easy and
and phase
Indrajit (2021, p. 28) cheap teaching materials
1 modification in the
Vanek et al. (2020, (ORI2)
Orientat DI model
p. 40) The formation of perception
ion (orientation and
and initial knowledge (ORI3)
(ORI) presentation) and
The implementation of
phases of the FC
asynchronous learning
model (planning,
communication (ORI4)
recording &
Facilitating the independent
sharing)
learning (ORI5)
Cooper et al. (2012) The availability of simulation
Joyce & Weil (2003, tools and materials (SBP1)
pp. 355–357) The reliable simulation tools
Salminen- and materials (SBP2)
Tuomaala (2019) The capability and mobility of
2
Smaldino et al. simulation tools and materials
Procedu
A new phase in the (2014) (SBP3)
re-based
FDI model Warwick et al. The formation of initial
simulati
(2016) knowledge in the Lower Order
on (SBP)
Thinking Skills (LOTS)
category (SBP4)
The communication
implementation for learning
simulation (SBP5)
3 Courtney et al. The cooperation (SBK1)
The new phases of
Case- (2015) Discussion to resolve cases
the FDI model and
based Joyce & Weil (2003, (SBK2)
implementation
Simulati pp. 357–358) The formation of critical
phases of the FC
on Ktoridou et al. thinking skills (SBK3)
model (group)
(SBK) (2018) Growing the creativity (SBK4)
555
The Source of The Theory of

Phases Factors
Phase Formation Phase Forming
Patandean & Growing the Higher Order
Indrajit (2021, p. 28) Thinking Skill (HOTS) (SBK5)
Ridho (2019) Discussion of cases to avoid
Smaldino et al. misperceptions and
(2014) misconceptions (SBK6)
Warwick et al.
(2016)
Joyce & Weil (2003, Procedural-based simulation
pp. 347–350) proofing (PTR1).
Increased the understanding
of form, function, and how to
use practicum tools and
materials (PTR2).
Increased understanding of
The
Standard Operating
4 accommodation of
Procedures and Occupational
The DI model phases
Health & Safety (PTR3).
structur (structured
Psychomotor enhancement
ed practice) and the
(PTR4).
Practice proofing of
Increased motivation (PTR5).
(PTR) procedural basis
Lockstep method in increasing
simulation
understanding and ability
(PTR6).
The meaningful experiences
for the future (PTR7).
Corrective and constructive
feedback to build confidence
(PTR8).
The Joyce & Weil (2003, Case-based simulation
accommodation pp. 347–350) verification (PTB1)
and modification of Patandean & Independence in practice
phases in the DI Indrajit (2021, p. 28) (PTB2)
5
model (guided The ability to solve technical
Guided
practice) and cases (PTB3)
Practice
phases of the FC The ability to cooperate
(PTB)
model (change) as technically (PTB4)
well as proof of Corrective and constructive
case-based feedback in finding solutions
simulations. (PTB5).
Joyce & Weil (2003, Strengthening memory
The
pp. 350–351) (PTM1)
accommodation
Patandean & Strengthening understanding
6 and modification of
Indrajit (2021, p. 28) (PTM2)
Indepen phases in the DI
Strengthening knowledge
dent model (self-
(PTM3)
Practice practice) and
Implementation of
(PTM) phases of the FC
documentation (PTM4)
model
Flexible implementation
(rearranging)
(PTM5)
556
The results of the theoretical analysis of the FDI learning model would strengthen
the foundation for the formation of the learning model. To make the FDI model
more testable, it was necessary to test the validity of the FDI learning model to
ensure that the developed model was ready for use. The FDI model validity test
is presented in the hypothetical model.
3.3. Hypothesis Models

The hypothetical model was formed based on the results of the Focus Group
Discussion (FGD) with 7 experts. The resulting data from the FGD is calculated
and presented in Figure 3.
Figure 3. Validation of Each Phase of The FDI Learning Model

Based on Figure 3, the data obtained from the FGD results were calculated using
the CFA method. Calculations ware continued in CFA to obtain values (x2/df) for
each phase of the FDI model. The first phase was the Orientation phase (ORI) with
five forming factors (ORI1-ORI5) obtaining a Chi-Square (x2) value of 2.32, a df
value of 5, and a value (x2/df) of 0.464. The value of 0.464 < 2 showed that the
Orientation phase (ORI) in the FDI model was valid/fit. The second phase was
the Procedure Based Simulation (SBP) phase with five forming factors (SBP1-
SBP5) obtaining a Chi-Square (x2) value of 2.46. The df value was 5 and the value
(x2/df) was 0.492. The value of 0.492 < 2 indicated that the Procedure Based
Simulation (SBP) phase in the FDI model was valid/fit. The third phase was the
Case Base Simulation (SBK) phase with six forming factors (SBK1- SBK6)
obtaining a Chi-Square (x2) value of 2.63. The df value was 9 and the value (x2/df)
was 0.292. The value of 0.292 < 2 showed that the Case Base Simulation (SBK)
phase in the FDI model was valid/fit.
Furthermore, the fourth phase, the Structured Practice (PTR) phase with eight
forming factors (PTR1-PTR8) obtained a Chi-Square (x2) value 0.18, the df value
20, and the value (x2/df) 0.009. The value of 0.009 < 2 indicated that the Structured
Practice (PTR) phase of the FDI model was valid/fit. The fifth phase, the Guided
Practice (PTB) phase, with five forming factors (PTB1-PTB5) obtained a Chi-
557
Square (x2) value of 0.08, df value of 5, and value (x2/df) of 0.016. The value of
0.016 < 2 indicated that the Guided Practice (PTB) phase of the FDI model was
valid/fit. Finally, the sixth phase was the Independent Practice (PTM) phase with
five forming factors (PTM1-PTM5) obtaining a Chi-Square (x2) value of 0.09, a df
value of 5, and a value (x2/df) of 0.018. The value of 0.018 < 2 indicated that the
Independent Practice (PTM) phase of the FDI model was valid/fit. All phases in
the valid FDI model form the goodness-of-fit-models and answer the research
hypothesis. The hypothesis accepted in this study is the alternative hypothesis
(Ha). The hypothesis (Ha) revealed that the FDI learning model is valid for use.
The results of the accepted hypothesis model could be continued in the formation
of the final model.
3.4. The Final Model

The final model was proposed after going through a conceptual and theoretical
formation which showed that the FDI model was created by combining the DI
model initiated by Brophy in 1986 with the FC model initiated by Patandean in
2021 (Joyce & Weil, 2003; Patandean & Indrajit, 2021). Based on Figure 3, the final
iteration of the FDI model was presented in Figure 4.
Figure 4. Flipped Direct Instruction (FDI) Learning Model

Based on Figure 4, the final iteration of the FDI learning model consisted of
Orientation, Procedure-Based Simulation, Case-Based Simulation, Structured
Practice, Guided Practice, and Independent Practice phases. Based on the model
hypothesis testing, the final FDI model was ready to be used as a new learning
model in contributing as an alternative for practicum implementation.
4. Discussion
The FDI learning model was successfully developed. Conceptually, the FDI
learning model was formed from a combination of the DI learning model and the
FC learning model. The combination of the two models represented a combination
of direct and indirect learning models. The DI learning model is part of the direct
learning model and the FC model is part of the indirect learning model. The
combination of direct and indirect learning models aligns with the theories of
Budiman et al. (2020) and Rüütmann and Kipper (2011), who state that direct and
558
indirect learning models can be combined for enhanced learning. This

combination looks to optimize students’ technical competence, critical thinking,
creativity, and cooperation. The FDI learning model was a learning model
combining direct and indirect learning, as shown in Figure 2.
Theoretically, the FDI learning model is formed by a combination of Brophy’s DI

learning model theory with Patandean’s FC model theory (Joyce & Weil, 2003;
Patandean & Indrajit, 2021). Based on Table 2, the FDI learning model had six
learning phases. The first phase in the FDI learning model was the Orientation
phase, formed from the accommodation and modification of the DI and FC
models and based on the theories of Vanek et al (2020), Joyce & Weil (2003), and
Patandean & Indrajit (2021), according to Table 2. The modification of this phase
was the merging of the Orientation and Presentation phases (DI model section)
with the planning phase, recording phase, and sharing phase (FC model section).
The Planning phase (part of the FC model) was similar in content to the
Orientation phase (section of the DI model). The equation was in the formation of
learning preparation.
The next phase is the Presentation phase (part of the DI model). The Presentation
phase can be combined with the Record phase and the Share phase (part of the FC
model). This can be combined as the recorded presentation can be shared in one
phase. This is a substitute for direct presentations as a mode of distributing
learning information. In this case, direct presentations were in the form of
delivering subject matter directly and synchronously, while presenting,
recording, and sharing are a revolution in conveying learning material indirectly
and asynchronously (Stein & Graham, 2020). Activities in all phases can be
combined into 1 phase, namely the Orientation phase. The Orientation phase of
the FDI model included presentations of learning objectives, explanations of
learning materials, and simulated explanations of learning topics. All presentation
explanations were recorded on video and distributed to students to watch and
learn.
The second phase in the FDI model was the Procedure Based Simulation Phase.
The Procedure-Based Simulation Phase was a novelty in the FDI model and was
formed based on the theory of Cooper (2012), Joyce & Weil (2003), Smaldino
(2014), Salminen-Tuomaala (2019), and Warwick (2016). Warwick (2016) states
that students will be able to understand a lesson better if there are observations in
it. Observation of learning can be carried out in the form of a simulation.
Simulation was initiated by Smith (1966) in (Joyce & Weil, 2003, pp. 355–356).
Simulation involved a comparison of biological control mechanisms (such as

humans) with electromagnetic control mechanisms (such as computers).
Simulation was closely related to simulator media. Therefore, the Procedure
Based Simulation phase used a simulator. The use of simulators can represent
reality with customizable problem complexity (Joyce & Weil, 2003, p. 357). In
addition, the use of simulators can provide students with learning tasks that they
can respond to, but these responses do not have the same consequences as with
real-life situations. Furthermore, simulation can also increase the level of students’
self-confidence when learning in the laboratory (Cooper et al., 2012). This
559
procedure-based simulation phase can be the best way to direct students who are
new to a portion of learning material (Salminen-Tuomaala, 2019).
Simulation is proven to improve skills through complex attributes such as

problem-solving, emotional intelligence, and situational sensitivity through
practice (Forneris et al., 2015). This is also in line with reality: if a person practices
more often, they will become more proficient and agile. Just like an athlete, they
will be able to run fast, weightlifting, play fast, and achieve other things that are
obtained from intense and consistent training. The implementation of a
procedure-based simulation allowed students to carry out exercises and observed
the practical implementation material in the laboratory. This was because, in the
procedure-based simulation phase, students carried out step-by-step practicum
simulations using a simulator according to the procedures that have been given
and carried out outside the laboratory indirectly, asynchronously, and
independently.
This activity allowed students to try out virtual practicum tools and materials in
a structured procedure, so that their Low Order Thinking (LOTS) abilities could
be formed. Procedure-based orientation and simulation phases were solutions to
the weaknesses of the DI model. These weaknesses included strengthening
students' initial knowledge (Warju et al., 2020; Winarno et al., 2021). The problem
of increasing students' initial knowledge in the DI model had actually been
proposed by various previous researchers. Warju et al. proposed providing real
video conditions in the DI model (Warju et al., 2020). While this is helpful, in the
FDI model, learning videos are also provided along with simulation media, which
are distributed to students with the Orientation and Procedure-Based Simulation
phases in the FDI model.
Next, the third phase was the Case Based Simulation Phase. The Case Based
Simulation Phase was the third phase in the FDI model. The Case Base Simulation
phase was also a novelty in the FDI model. The Case-Based Simulation Phase was
formed based on the theories of Courtney (2015), Joyce & Weil (2003), Ktoridou
(2018), Patandean (2021), Ridho (2019), Smaldino (2014), and Warwick (2016).
Case-Based Simulation accommodated the Group phase (part of the FC model),
which aligns with Patandean and Indrajit’s (2021) theory that the group formation
stage aims to separate the topics of the lesson into student observation and student
exploration. Cases are given in this phase, which will be more effective if
completed in practice and in groups by students (Courtney et al., 2015; Hartanto
et al., 2022; Joyce & Weil, 2003, p. 361).
The Case Based Simulation phase was based on the same theory as the Procedure
Based Simulation phase. However, in this phase, there was a case that must be
resolved by students. Cases are suitable for use in simulation because among the
purposes of simulation, problems potentially more complex than real-world
problems can be found or solved (Joyce & Weil, 2003, p. 358). Therefore, a case can
be used to solve more complex problems. The use of cases refers to the theory
from Courtney et al. (2015) which states that the case method provides elements
of effective learning such as discovery, probing, continual practice, contrast and
comparison, as well as involvement and motivation. Based on Corey's theory, the
560
case-solving method is very effective in continuous practical learning, so the main

goal of solving cases is to encourage students' Higher Order Thinking (HOTS)
abilities. The case-based simulation phase was carried out directly in the
laboratory. This is in line with the FC learning model, which suggests that the
formation of students' HOTS abilities needs to be done directly in class (Patandean
& Indrajit, 2021, p. 39).
Providing cases that were relevant to current conditions would be useful in

offering an overview for students to find out the benefits of the practicum being
studied at that time in the industrial field. This can also bring students closer to
the real world that they will encounter (Ridho, 2019). The case-based simulation
phase would increase students' collaborative abilities as the case-solving phase in
the FDI model was carried out using a group technique. Collaboration in groups
would develop collaborative skills as a solid team would solve cases with
theoretical and practical evidence and making a conclusion.
Aspects of students' critical thinking skills are a weakness in the DI learning

model (Gurses et al., 2015; Winarno et al., 2018). This problem was solved by
previous researchers. Winarno et al. (2018) proposed useful problem-based
learning in the DI model. However, in the FDI model, the problems presented and
contained in a case are more complex; in a case, the collection of problems has a
complex pattern (Rohmadi, 2015). For this reason, this case-based simulation was
carried out directly, synchronously, in groups, and with a simulator.
The fourth phase in the FDI model was the Structured Practice Phase. The
Structured Practice Phase was formed from the accommodation of the DI model
which was based on the theory of Joyce and Weil (2003). The Structured Practice
Phase is based on the principle of practice itself, namely to “form” (Joyce & Weil,
2003). This formation is intended so that students can have skills that can be used
independently and with little or no mistakes. In addition, the Structured Practice
phase was intended to meticulously practice the practicum material in a real and
direct way on students who have carried out a procedural basis simulation. In
achieving independent practice with a high degree of accuracy, practicum must
be carried out through structured practice, guided practice, and independent
practice (Joyce & Weil, 2003). This makes it necessary to carry out the structured
practice phase directly and synchronously in the laboratory.
The fifth phase in the FDI model was the Guided Practice Phase. The Guided
Practice phase was formed from accommodation and modified from the DI
model, which is based on the theories of Joyce and Weil (2003) and Patandean and
Indrajit (2021). The modification of this phase was the merging of the Change
Phase (part of the FC model) into the Phase of Guided Practice (part of the DI
model). The Change Phase (part of the FC model) aimed to create demonstrable
changes in student interactions with the learning environment. Therefore, the
Structured Practice phase can facilitate proof of these changes to shape students'
skills (Joyce & Weil, 2003). In the Guided Practice phase, students are given the
opportunity to practice alone or in groups under teacher monitoring in the
laboratory. Students will practice in the laboratory to prove the case-based
simulation that has just been discussed. The previous case-based simulation was
561
used only to strengthen students' understanding and to reduce the level of

accidents that may occur in Guided Practice.
The sixth phase in the FDI model was the Independent Practice Phase. This phase
is formed from the accommodation and modification of the DI model. This phase
is based on the same theory as the fourth and fifth phases. The modification of
this phase involved the merging of the Independent Practice phase (DI model
section) with the Rearrangement phase (FC model section). This can be combined
because the Rearrangement phase (part of the FC model) and Independent
Practice (part of the DI model) aim to strengthen memory and increase students'
knowledge, skills, and work attitudes (Joyce & Weil, 2003; Tasrif et al., 2021). This
phase was a repetition in nature carried out by students outside the laboratory.
Independent practice could be made in the form of assignments. The assignments
can be in the form of simulations repetition or lab reports.
Based on the results of the FDI learning model formation in theory, the FDI
learning model was tested with the Hypothesis model. The hypothetical model of
the FDI learning model was studied in a Focus Group Discussion (FGD). The data
from the study was presented in Figure 3. Based on the data calculation in Figure
4, each sequential phase of the FDI learning model had a value (x2/df), namely
0.464, 0.492, 0.292, 0.009, 0.016, and 0.018. The value (x2/df) obtained for each
phase of the FDI learning model was below 2, which indicated that each phase
was valid/fit (Jackson, 2018; Meyers et al., 2013, p. 870). All valid phases reflecting
the FDI learning model were included in the goodness-of-fit-model category. The
goodness-of-fit-model answered the research hypothesis by accepting the
alternative hypothesis (Ha), revealing that the FDI learning model was valid to
use.
The FDI learning model that had passed the hypothesis test was then incorporated
final model. In accordance with Figure 4, the final FDI learning model had 6
learning phases that were carried out directly and indirectly. The combination of
direct and indirect learning in the FDI learning model aims to optimize technical
competence, critical thinking, creativity, and collaboration by students
(Rüütmann & Kipper, 2011). Good competence would produce efficient
educational graduates in industry, continuing education, or entrepreneurship.
Therefore, the FDI learning model can be presented as a new learning model and
as a good alternative in increasing the effectiveness of implementing practicum
learning.
5. Conclusion
The problem of limitations in the DI learning model has been solved by the
presence of the FDI model. The FDI model had phases of orientation, procedure-
based simulation, case-based simulation, structured practice, guided practice, and
independent practice. Each phase had a valid category with the value (x2/df)
sequentially as 0.464, 0.492, 0.292, 0.009, 0.016, and 0.018. The validity of all phases
reveals that the FDI learning model is in the goodness-of-fit-model category. The
goodness-of-fit-model of the FDI learning model reveals development of this
model is successful and ready to improve students' prior knowledge, technical
skills, critical thinking, creativity, and work ability. Therefore, the FDI learning
562
model was presented as a new learning model to offer new options that help
improve the effectiveness of practicum learning implementation and contribute
to the development of science in the field of learning models. The FDI model is
highly recommended in vocational education, which holds a dominant portion of
practical learning. This research was limited to the stages of forming the FDI
model. In the future, the FDI learning model should continue to be evaluated and
developed to improve the capabilities of this model. The improvement is carried
out to measure the level of practicality, effectiveness, and impact of the
application of the FDI model.
6. References
Ahmad, N. A., Rasul, M. S., Othman, N., & Jalaludin, N. A. (2022). Generating
Entrepreneurial Ideas for Business Development. Sustainability (Switzerland), 14(9),
1–11. https://doi.org/10.3390/su14094905
Anderson, L. W., Krathwohl, D. R., Airasian, P. W., Cruikshank, K. A., Mayer, R. E.,
Pintrich, P. R., Raths, J., & Wittrock, M. C. (2001). A Taxonomy For Learning, Teaching,
and Assessing a Revision of Bloom’s Taxonomy of Education Objectives. Addison Wesley
Longman, Inc.
Ashari, Z. H. M., Azman, N., & Rasul, M. S. (2019). Factors predicting career choice among
Malaysian students in skills-based training institutions. International Journal for
Educational and Vocational Guidance, 19(1), 19–39. https://doi.org/10.1007/s10775-
018-9366-5
Aziz, F., Quraishi, U., & Kazi, A. S. (2018). Factors behind classroom participation of
secondary school students (A gender based analysis). Universal Journal of Educational
Research, 6(2), 211–217. https://doi.org/10.13189/ ujer.2018.060201
Aziziy, Y. N., Sulhadi, Marwoto, P., & Iswari, R. S. (2020). Designing e-learning based
independent learning activity unit through virtual worksheet: Improving student’s
4Cs. Journal of Physics: Conference Series, 1567(4), 1–5. https://doi.org/10.1088/1742-
6596/1567/4/042103
Azwar, S. (2019). Reabilitas dan Validitas Edisi 4 [Reliability and Validity 4th Edition]. Pustaka
Pelajar.
Budiman, A., Samani, M., Rusijono, R., Setyawan, W. H., & Nurdyansyah, N. (2020). The
Development of Direct-Contextual Learning: A New Model on Higher Education.
International Journal of Higher Education, 10(2), 15–26.
https://doi.org/10.5430/ijhe.v10n2p15
Central Bureau of Statistics Indonesia (BPS). (2022). Keadaan Ketenagakerjaan Indonesia
Agustus 2022 [Indonesia’s State of Employment August 2022].
https://www.bps.go.id/publication/2022/12/07/a64afccf38fbf6deb81a5dc0/kead
aan-angkatan-kerja-di-indonesia-agustus-2022.html
Chamidy, T., Degeng, I. N. S., & Ulfa, S. (2020). The effect of problem-based learning and
tacit knowledge on problem-solving skills of students in computer network practice
course. Journal for the Education of Gifted Young Scientists, 8(2), 691–700.
https://doi.org/10.17478/JEGYS.650400
Cooper, S., Cant, R., Porter, J., Bogossian, F., McKenna, L., Brady, S., & Fox-Young, S.
(2012). Simulation based learning in midwifery education: A systematic review.
Women and Birth, 25(2), 64–78. https://doi.org/10.1016/ j.wombi.2011.03.004
Courtney, N., Poulsen, C., & Stylios, C. (2015). Case Based Teaching and Learning for the 21st
Century. Libri Publishing.
Creswell, J. W. (2012). Educational Research : Planning, Conducting, and Evaluating
Quantitative and Qualitative Research. Pearson Education, Inc.
563
Dehham, S. H., & Albayati, W. A. (2021). The Impact of Using Flipped Learning on the
Performance of Iraqi EFL Fifth Preparatory Students in Reading Comprehension.
Turkish Online Journal of Qualitative Inquiry (TOJQI), 12(7), 11783–11795.
Flynn, R. J., Marquis, R. A., Paquet, M. P., Peeke, L. M., & Aubry, T. D. (2012). Effects of
individual direct-instruction tutoring on foster children’s academic skills: A
randomized trial. Children and Youth Services Review, 34(6), 1183–1189.
https://doi.org/10.1016/j.childyouth.2012.01.036
Forneris, S. G., Neal, D. O., Tiffany, J., Kuehn, M. B., Meyer, H. M., Blazovich, L. M.,
Holland, A. E., & Smerillo, M. (2015). Enhancing clinical reasoning through
simulation debriefing: A multisite study. Nursing Education Perspectives, 36(5), 304–
310. https://doi.org/10.5480/15-1672
Glogger-Frey, I., Fleischer, C., Grüny, L., Kappich, J., & Renkl, A. (2015). Inventing a
solution and studying a worked solution prepare differently for learning from direct
instruction. Learning and Instruction, 39, 72–87.
https://doi.org/10.1016/j.learninstruc.2015.05.001
Gurses, A., Dogar, C., & Gunes, K. (2015). A New Approach for Learning: Interactive
Direct Teaching Based Constructivist Learning (IDTBCL). Procedia - Social and
Behavioral Sciences, 197(February), 2384–2389.
Hartanto, S., Huda, A., Wulansari, R. E., Mubai, A., Firdaus, & Shalehoddin. (2022). The
Design of Android-Based Interactive Lean Manufacturing Application to Increase
Students’ Work Skill in Vocational High School: The Development and Validity.
International Journal of Interactive Mobile Technologies (IJIM), 16(13), 130–139.
https://doi.org/10.3991/ ijim.v16i13.30595
Holmes, A. G. D., Tuin, M. P., & Turner, S. L. (2021). Competence and competency in
higher education, simple terms yet with complex meanings: Theoretical and
practical issues for university teachers and assessors implementing Competency-
Based Education (CBE). Educational Process: International Journal, 10(3), 39–52.
https://doi.org/10.22521/EDUPIJ.2021.103.3
Huda, A., Azhar, N., Almasri, A., Wulansari, R. E., Mubai, A., Sakti, R. H., Firdaus, F., &
Hartanto, S. (2021). Augmented Reality Technology as a Complement on Graphic
Design to Face Revolution Industry 4.0 Learning and Competence: The Development
and Validity. International Journal of Interactive Mobile Technologies, 15(5), 116–126.
https://doi.org/10.3991/ ijim.v15i05.20905
Jackson, C. R. (2018). Validating and Adapting the Motivated Strategies for Learning
Questionnaire (MSLQ) for STEM Courses at an HBCU. AERA Open, 4(4), 1–16.
https://doi.org/10.1177/2332858418809346
Joyce, B., & Weil, M. (2003). Models of Teaching Fifth Edition. Asoke K. Ghosh.
Ktoridou, D., Doukanari, E., Epaminonda, E., & Karayiannis, A. (2018). Case-based
learning: Offering a premier targeted learning experience for technology
management students. IEEE Global Engineering Education Conference, Educon, 2018-
April, 1781–1786. https://doi.org/10.1109/ EDUCON.2018.8363450
Maksum, H., & Purwanto, W. (2019). Model Pembelajaran Pendidikan Vokasi Otomotif (PVO)
[Automotive Vocational Education (PVO) Learning Model]. UNP PRESS.
Meyers, L. S., Glenn Gamst, & Guarino, A. J. (2013). Applied Multivariate Research Design
and Interpretation. SAGE Publications, Inc.
Miles, M. B., Huberman, A. M., & Saldaña, J. (2014). Qualitative data analysis: a methods
sourcebook - Third edition. SAGE Publications.
Nofriansyah, D. (2020). DICKY : A New Learning Model Which Is A Combination Of
Artificial Intelligence And ICT In Vocational Education. International Journal of
Evaluation and Research in Education (IJERE), 9 No 3.
564
Nuris, D. M., & Istyaningputri, C. M. (2021). The Factors That Influence the Independent
Learning of Accounting Students at a Vocational High School. KnE Social Sciences,
2021, 49–58. https://doi.org/10.18502/kss.v5i8.9347
Patandean, Y. R., & Indrajit, R. E. (2021). Flipped Classroom Membuat Peserta Didik Berpikir
Kritis, Kreatif, Mandiri dan Mampu Berkaloborasi [Flipped Classroom Makes Learners
Think Critically, Creatively, Independently and Able to Collaborate]. Andi Offset.
Paulsen, K. S., Ungermand, S., Engermann, M. H., Faria, M., Tree, D. J., O’Neill, L., Hladiš,
R., & Gadney, G. (2019). Storytelling Beyond The Screen Creating (K. S. Paulsen (ed.)).
Filmby Aarhus & VIA Film & Transmedia.
Ridho, S. (2019). Pembelajaran Berbasis Kasus [Case-Based Learning]. IPAsiapatakut.
https://ipasiapatakut.blogspot.com/2019/11/pembelajaran-berbasis-kasus-case-
based.html
Riyanti, A., & Parulian, N. A. (2023). The Effect of Communication and Decision-Making
on Teacher Performance in Junior High School. Journal on Education, 05(04), 11498–
11502. https://doi.org/https://doi.org/10.31004/ joe.v5i4.2096
Rohmadi, A. (2015). Case Based Reasoning Untuk Pemilihan Kegiatan Organisasi
Mahasiswa [Case Based Reasoning for Student Organization Activity Selection].
Jurnal Momentum, 17(2), 39–43.
Rüütmann, T., & Kipper, H. (2011). Teaching strategies for direct and indirect instruction
in teaching engineering. 2011 14th International Conference on Interactive Collaborative
Learning, ICL 2011 - 11th International Conference Virtual University, VU’11,
107(September), 107–114. https://doi.org/ 10.1109/ICL.2011.6059556
Salminen-Tuomaala, M. (2019). Development of Multiprofessional Simulation-based
Education in South Ostrobothnia, Finland. Athens Journal of Health & Medical Sciences,
6(3), 143–156. https://doi.org/doi.org/10.30958/ajhms.6-3-2
Smaldino, Lowther, & Russell. (2014). Instructional Technology and Media for Learning
(Tenth). Pearson Education Limited.
Staker, H., & Horn, M. B. (2012). Classifying K-12 Blended Learning. Innosight Institute,
May, 1–22.
Stein, J., & Graham, C. R. (2020). Essentials for blended learning: A standards-based guide.
In Essentials for Blended Learning. Routledge.
https://doi.org/10.4324/9780203075258-7
Stockard, J., Wood, T. W., Coughlin, C., & Rasplica Khoury, C. (2020). All students can
succeed: A half century of research on the effectiveness of Direct Instruction. Lexington
Books.
Tambunan, H., Sinaga, B., & Widada, W. (2021). Analysis of teacher performance to build
student interest and motivation towards mathematics achievement. International
https://doi.org/10.11591/ijere.v10i1.20711
Tasrif, E., Saputra, H. K., Kurniadi, D., Hidayat, H., & Mubai, A. (2021). Designing
Website-Based Scholarship Management Application for Teaching of Analytical
Hierarchy Process (AHP) in Decision Support Systems (DSS) Subjects. International
Journal of Interactive Mobile Technologies, 15(9), 179–191.
https://doi.org/10.3991/ijim.v15i09.23513
Vanek, J., Simpson, D., & Goumas, J. (2020). IDEAL Distance Education and Blended Learning
Handbook, 7th Edition. EdTech Books. https://edtechbooks.org/ ideal_dl_handbook
Viinikka, K., Ubani, M., Lipiäinen, T., & Kallioniemi, A. (2019). 21st Century Skills and
Finnish Student Teachers’ Perceptions about the Ideal RE Teacher Today and in the
Future. International Journal of Learning, Teaching and Educational Research, 18(8), 75–
97. https://doi.org/10.26803/ijlter.18.8.5
Warju, W., Ariyanto, S. R., Soeryanto, S., Hidayatullah, R. S., & Nurtanto, M. (2020).
565
Practical Learning Innovation: Real Condition Video-Based Direct Instruction Model

in Vocational Education. Journal of Educational Science and Technology (EST), 6(1), 79–
91. https://doi.org/10.26858/est.v6i1.12665
Warwick, P., Vrikki, M., Vermunt, J. D., Mercer, N., & van Halem, N. (2016). Connecting
observations of student and teacher learning: an examination of dialogic processes
in Lesson Study discussions in mathematics. ZDM - Mathematics Education, 48(4),
555–569. https://doi.org/10.1007/s11858-015-0750-z
Winarno, S., Andono, P. N., Muthu, K. S., & Ling, L. S. (2021). The effectiveness of mDPBL
as a multimedia-based learning approach. Cypriot Journal of Educational Sciences,
16(6), 2946–2955. https://doi.org/10.18844/ cjes.v16i6.6479
Winarno, S., Muthu, K. S., & Ling, L. S. (2018). Direct Problem-Based Learning (DPBL): A
Framework for Integrating Direct Instruction and Problem-Based Learning
Approach. International Education Studies, 11(1), 119–126.
https://doi.org/10.5539/ies.v11n1p119
Winarsih, S., Sangka, K. B., & Octoria, D. (2019). The effect of direct instruction and
problem based learning on millennial. AIP Conference Proceedings, 2194(December
2019). https://doi.org/10.1063/1.5139873
Yulianto, T., Pramudya, I., & Slamet, I. (2019). Effects of the 21st Century Learning Model
and Problem-Based Models on Higher Order Thinking Skill. International Journal of
Educational Research Review, 4, 749–755. https://doi.org/10.24331/ijere.629084
Zayyadi, M., Nusantara, T., Hidayanto, E., Sulandra, I. M., & Sa’Dijah, C. (2020). Content
and pedagogical knowledge of prospective teachers in mathematics learning:
Commognitive. Journal for the Education of Gifted Young Scientists, 8(1), 515–532.
https://doi.org/10.17478/jegys.642131
566

Vol. 22, No. 7, pp. 566-585, July 2023
Received May 23, 2023; Revised Jul 28, 2023; Accepted Aug 5, 2023
Mentors’ and Mentees’ Experiences in a Sino-

Foreign Cooperative Education Programme: A
Phenomenological Study in a Public University
in China
Ying Tang
UCSI University, Education Department, Kuala Lumpur, Malaysia
Guilin Tourism University, International Hospitality Department, Guangxi,
Guilin, China
Lin Siew Eng*

UCSI University, Education Department, Kuala Lumpur, Malaysia
Abstract. This phenomenological study explores the experiences of

mentors and mentees in a Sino-Foreign Cooperative Education (SFCE)
programme at a public university in China. Since the implementation of
the SFCE programme in 2015, no study has been conducted to investigate
the experience of mentors and mentees. Using purposive sampling, data
were collected through in-depth interviews and classroom observations
with six mentors and six mentees, who were selected from a public
university in China. The data were analysed using a thematic analysis
approach, from which five major themes emerged. The findings highlight
the benefits of the SFCE programme for participants, especially for
mentees. However, mentors and mentees also faced challenges related to
cultural differences and effective mentoring relationships in teaching
assessment, teaching style, and management. The facilities and
educational resources also require further improvement to ensure the
effective implementation of the SFCE programme, and, notably, mentors
do not share enough educational resources with mentees. The
programme schedule was intensive for mentors and mentees and, thus,
needs to be adjusted to improve the SFCE programme. The study
concludes with recommendations for both the foreign and Chinese
universities that participated in the SFCE programme, to set up a
Programme Management Committee to improve communication, to
encourage building up better mentoring relationships, and to create a
supportive training environment.
Keywords: experiences; mentor; mentee; phenomenological study;

thematic analysis
*
Corresponding author: Lin Siew Eng; linse@ucsiuniversity.edu.my
©Authors
567
1. Introduction
Sino-Foreign Cooperative Education (SFCE) refers to foreign legal entities or
international organisations that cooperate with Chinese educational institutions,
or other legal, social organisations, to set up educational institutions or
programmes and carry out educational activities within the territory of China (Li
& Xue, 2022). SFCE allowed China to quickly learn from and absorb valuable
experiences, advanced teaching methods, and curriculum design from developed
countries.
The SFCE programme offers specialised courses from a foreign cooperative

university every semester; the teaching time of the teachers at the cooperative
university is required to be at least one-third of the total class hours (Li, 2021).
Foreign teachers teach in English, and Chinese teachers teach in both Chinese and
English. Foreign teachers and Chinese teachers work together to teach in the SFCE
programme. The foreign teachers are the mentors, and the Chinese teachers are
the mentees.
Chinese teachers can learn advanced teaching methods, improve their teaching
skills, and exchange knowledge about culture with foreign teachers in the
implementation of the programme by working together, which is an important
reason why the government supports SFCE (Song & Wang, 2020).
Mentors and mentees are the core elements for the implementation of SFCE and
an important dimension to test the effectiveness of the introduction and
absorption of quality foreign resources (Xia, 2020). However, since the operational
and management system of the SFCE programme differs from the normal
teaching programme, there are many problems and developmental blocks for the
teaching staff (Lin, 2019). Factors such as cultural and system differences between
China and the foreign countries involved affect the teaching and learning of the
mentors and mentees in the SFCE. The programme has not effectively cultivated
Chinese teachers or promoted professional development in Chinese universities
(Zeng, 2020).
The researchers adopted a phenomenological approach, which provides an

account of the experiences of the mentors and mentees in a SFCE programme at a
public university in China, including their responses to the programme’s teaching
and learning, the educational resources and facilities, the schedule, the level of
motivation, and the assessment methods throughout the SFCE programme. By
determining the experiences of the mentors and mentees, we can highlight how
the SFCE can be improved.
This section presents a literature review on the SFCE programme in China,
teaching in the SFCE and mentoring relationships.
2.1 SFCE in China

There are nearly 2,300 SFCE programmes and institutions at different levels in
China as of June 2020 (Ministry of Education of the People‘s Republic of China
568
[MOE], 2020). More than 600,000 students are enrolled in SFCE programmes at a
bachelor’s level or above (Lin, 2022). In China, the duration of an undergraduate
SFCE programme is 4–6 years. Engineering, finance, automation design,
accounting, architecture, management, and international economics and trade are
some of the majors available in the SFCE programme (Zhu & Huang, 2020).
2.2 Teaching in SFCE

The teaching syllabus of an SFCE programme must meet the requirements of the
teaching policies and content of Chinese and foreign teachers (Zeng, 2020). The
Ministry of Education (MOE) also evaluates whether the training objectives
conform to China’s guiding ideology; whether the programme reflects Chinese
specifics requirements; whether the SFCE educational objective is scientific,
meaningful, and feasible; and whether the objective is consistent with the
commitment made upon obtaining certification (Supervisory Information
Platform for SFCE Ministry of Education, 2021).
The teaching content of the SFCE programme should be in line with the teaching
policy, teaching content, and teaching modes of the Chinese and foreign parties
(Zeng, 2020). Due to the high language requirements of the SFCE programme,
there may be too much English-subject content in the SFCE programme education.
However, students’ weak listening and speaking abilities in English lead to
confusion with the use of professional vocabulary and terms in courses taught in
English (Yan & Chernytskyi, 2022).
The teaching schedule in the SFCE programme is intensive, and there is no

continuous guidance for students and Chinese teachers (Liu, 2018). By looking at
the course schedules, it can be seen that most students prioritise the courses
offered by foreign teachers. Chinese teachers’ time is very limited, and their
courses are even held in the evenings or on weekends (An, 2016). Within the
limited time and conditions, adjusting the schedule of courses can also improve
teaching quality (Li et al., 2020).
In terms of teaching methods, there are differences between Chinese and foreign
teachers’ teaching methods (Wu et al., 2019). In the SFCE programme, teachers
often use case studies, multimedia, game-embedded, mind-brain mapping,
problem-based learning, and online teaching methods to promote students'
learning motivation (Wang, 2021). Amidst the threat of the COVID-19 pandemic,
teachers, students, and schools coped with and adjusted to distance learning
education (Gocotano et al., 2021). In SFCE programmes, final exams are the
primary means of assessment, regardless of individual student differences. The
teachers in the SFCE should strengthen their diversified assessment methods and
place special emphasis on formative assessment in the context of post-COVID-19
(Martínez Huamán et al., 2022; Shi et al., 2023).
A previous study analysed the structure of the SFCE evaluation system and
proposed the selection of school-running conditions, teaching quality, and
cultural differences, such as teaching methods, communication methods, school-
569
running effectiveness and employment competence, as measure indices (Yu &

Dong, 2021).
Other scholars have also attempted to evaluate SFCE programmes from the
perspective of multi-subject participation (Li & Liu, 2019; Pan, 2017). It has been
found that teacher satisfaction is important to the development of the SFCE
programme. The teachers in the SFCE programme feel less satisfied and more
stressed than teachers in normal teaching programmes (Yu, 2023).
Teachers in the SFCE programme have attracted increasing research attention, but
most of the research has focused on the current situation, teaching methods, and
construction of models for bilingual teaching (Feng, 2022; Li, 2021). Teachers’
experiences, feelings, and mentoring relationship during the implementation of
the SFCE programme have not yet been sufficiently studied.
2.3 Mentoring Relationship

Mentoring in education refers to a relationship between two people, one of whom
is experienced and plays a supportive and advisory role for the other who has
little experience (Argente-Linares et al., 2016). The benefits of mentoring in
education have been reported by many scholars (Clutterbuck, 2014), for both
mentors and mentees (Bean et al., 2014).
Mentors are crucial for guiding and nurturing mentees and helping them to build
positive feelings about their careers (Ellis et al., 2020). Mentors can also use their
experience and knowledge in a convenient way to support mentees’ development
(Ellis et al., 2020). Mentors provide mentees with guidance, advice, and feedback,
and they also play many roles, such as that of advisor, sponsor, and advocate.
A mentor can be a professional coach and an emotional support system and play
a sociocultural role in fostering mentoring relationships (Banerjee-Batist et al.,
2019). Mentors can encourage mentees to develop relevant skills and attitudes for
the future and, in particular, help mentees set goals and identify actions.
According to the MOE’s specifications, the proportion of courses in SFCE

programmes taught by foreign teachers should not be less than one-third of the
total number in the four-year curriculum. The Chinese teacher and the foreign
teacher should co-teach together in the programme, forming a mentoring
relationship (Supervisory Information Platform for SFCE Ministry of Education,
2021).
Age, ethnicity and gender affect the mentoring relationships between mentors and
mentees (Daniel et al., 2019). There may be difficulties in mentoring caused by
cultural complexity and sensitivity between cultures that are present across
mentoring relationships. These difficulties may be overcome by a sense of
understanding differences, and a desire for meaningful sharing, trust and respect
(Parker et al., 2022).
570
From the standpoint of the mentoring relationship, there is a lack of research on

mentors and mentees in SFCE programmes; moreover, research on mentoring in
China has concentrated more on vocational education and less on international
education (Ma et al., 2019).
3. Methodology
The phenomenological approach focuses on explaining an experience from the
perspective of the subject by analysing the words that they have said or written
(Sohn et al., 2017). Phenomenological research is conducted on a small number of
participants, which makes the selection of participants essential to obtain
significant and accurate data. Collecting extensive details regarding the
participants and their relationships to the phenomena is essential for qualitative
research (Creswell, 2014). The present researcher used the phenomenological
approach, as the purpose of this research was to investigate the experiences of
mentors and mentees who have participated in the SFCE programme in a public
university in China.
3.1 Setting
The SFCE programme is a collaboration between a public university in China
(pseudonymous University X) and a university in Europe (pseudonymous
University Y). The case was specifically selected (Creswell, 2013) for its unique
SFCE programme in China on Hospitality Management at the undergraduate
level. This study investigated the mentors’ and mentees’ experiences regarding
the implementation of the programme.
3.2 Participants
The purposive sampling method was used, which means that only those who
were willing to participate in and had teaching experience in the SFCE
programme were selected, which ensured the pertinence of the interview
information obtained. Creswell (2013) recommended 8–12 participants as an
appropriate size range for a phenomenological study. There were 12 participants
in the study: six mentors and six mentees. All participants had at least one full
semester of teaching in the SFCE programme and the experience of cooperating
with a mentor (mentee).
Pseudonyms have been used throughout the study to ensure the confidentiality
of the mentors and the mentees in the SFCE programme: for example, M
represents a mentor from University Y, and MT represents a mentee from
University X. Mentors and mentees worked together to teach one course (Table
1).
Table 1: Coding of participants

Course Name Mentor Mentee
Strategic Management (SM) (M1-University Y) (MT1-University X)
Foundations of Hospitality Marketing (M2-University Y) (MT2-University X)
(HM)
Revenue Management (RM) (M3-University Y) (MT3-University X)
Food & Beverage Cost Management (M4-University Y) (MT4-University X)
(FBM)
571
Management Information System (M5-University Y) (MT5-University X)

(MIS)
Finance Management (FM) (M6-University Y) (MT6-University X)
3.3 Data Collection

Phenomenological studies typically collect data from interviews, observations,
audio visual materials, and written documents (Creswell, 2013). Given the
research objectives of the SFCE programme, three main methods of data collection
were applied: document collection, observation, and semi-structured interviews.
Documents related to the SFCE policies, education objectives, teaching standards,
assessment standards, and cooperative agreements were analysed to compare the
gaps within teacher performance and to help researchers understand the
development and history of the programme.
The researcher observed the mentors and mentees at the teaching site using the
observation checklist. Semi-structured interviews were conducted according to
the designed interview protocol. The interview questions were used to address
the key research question: What are the mentors’ and mentees’ reactions towards
the implementation of the SFCE programme at a public university in China with
regard to teaching and learning, educational resources, facilities, schedules,
motivation, and assessment methods? The researcher designed the interview
protocol around the research question separately for the mentors and mentees.
The interviews were audio recorded and transcribed to obtain as much
information as possible from the interview process. The interview questions and
observation checklist were checked by content experts to ensure validity and
reliability.
3.4 Data Analysis

The data from the documents were read carefully. The data from the observations
were recorded as field journals, and the data from the interviews were transcribed
into text documents. Interpretative phenomenological analysis (IPA) was the
main method applied to help the researcher answer the research question. A
number of themes emerged through reading and rereading each transcript and
coding it, looking for common words and themes.
4. Findings
During the data analysis, five themes emerged (Figure 1).
Theme1: Theme 2: Theme 3:
Theme 4: Theme 5:
Suitability of the methods of Suitability and drawbacks of Suitability of course duration and
Motivation and drawbacks Suitability of assessment methods
teaching and learning resources and teaching facilities schedule of SFCE programme
• Motors use student-centred • Mentors and mentees were • Schedule was continuously • Sharing the working experience • Abide by University X’s
teaching methods satisfied with the teaching optimised in the hospitality industry with examination requirements
• Mentors and mentees adjust the facilities and equipment • Schedule was too intensive students • Used a variety of methods to
teaching content and methods • Facilities and equipment needed • Mentors satisfied with the • Mentors want to gain a deeper assess the learning process
to accommodate the students constant maintenance and schedule arrangement understanding of China through • The assessment was necessary
and course itself of this SFCE updating • Mentees complained about the this SFCE programme to stimulate students to study in
programme • Network problems were the scheduling • A good mentoring relationship this SFCE programme
• Epidemic makes teaching in most crucial issue • Some of the arrangements of • Financial rewards • Several factors limited their
SFCE programme difficult • There was also a shortage of the courses were inappropriate • Reputation of University Y assessment practices
software • Contact hours for teaching in • Maintaining English • The assessment should be
• Mentors shared teaching some courses were insufficient communication skills continuously improved
resource with mentees, but • Being encouraged by the
mentees felt the resource was mentors
not necessarily beneficial to
• The preparation of course
them
materials reduced mentors’ and
mentees’ motivation
Figure 1: Five themes
572
4.1 Theme 1: Suitability of the Methods of Teaching and Learning

It is important to adopt appropriate methodologies for delivering a lesson
(Rajagopalan, 2019). All of the mentors stated that, in the SFCE programme, they
commonly use student-centred teaching methods, such as case studies, group
discussions, and presentations, to promote student learning.
“I prefer to use group presentation in the classes because I feel that the
spirit of cooperation is a necessary quality for hotel professionals.” (M1)
“I use questioning in my teaching to encourage students to think.” (M2)
Three mentors and four mentees reported that they adjusted the teaching content
and methods to accommodate the students and the course itself in this SFCE
programme.
“I will adjust the depth of instruction on certain points. Moreover, I
included exercises in class to facilitate student engagement.” (M5)
“The teaching methods I use would depend on the topic and seniority of
the students. I usually use group work in the classes because the number
of students is large and time is limited, not suitable for individual work
with corrections.” (MT6)
“I often assign exercises or quizzes for students to do in the class to test

and reinforce what has been learned. Because it’s a Management
Information System course, students need to know how to apply the
knowledge.” (MT5)
However, the pandemic made teaching SFCE programmes difficult. Due to

COVID-19, foreign teachers were unable to travel to China to teach physical
classes and were forced to rely on distance learning. Teachers’ options for teaching
methods were limited by distance learning because online classes reduced their
interaction with students. Two interviewees described the experiences of student
learning:
“The students responded less in online class; only a few students
answered the teacher’s questions via the Tencent dialogue box.” (MT6)
“In distance learning, interaction in the classroom is significantly
reduced.” (M4)
The researcher’s observations also confirmed the participants’ claims.

Furthermore, the teachers were not adequately equipped with digital media skills.
Mentors and mentees experienced the unavailability of a network, a shortage of
digital devices, a distractive learning environment, and a lack of digital literacy
skills.
4.2 Theme 2: Suitability and Drawbacks of Resources and Teaching Facilities

The SFCE programme should be implemented with good teaching facilities and
equipment. Appropriate facilities and equipment allow the mentors and mentees
involved in the programme to feel the school’s support and attention, which
increases motivation and confidence.
An advantage of the SFCE programme at University Y is that its mentors share

foreign teaching resources. All of the participants indicated that they were
573
satisfied with the teaching facilities and equipment in this SFCE programme. The
mentors and mentees believed that the equipment of this SFCE programme was
better than that of the other teaching programmes at University X. An
independently operated hotel provided a practical teaching environment for
students. The researcher observed that the teaching environment was clean, tidy,
and air-conditioned, and that it had good teaching equipment and facilities.
“In addition to the excellent specialised laboratories, such as wine-tasting,
hotel rooms, and pastry laboratories, the programme also had good
multimedia classrooms, projectors, and computers, which could enhance
teaching and learning.” (MT3)
“I was fascinated by the teaching surroundings and the teaching facilities
were well equipped. “(M2)
In contrast, at University X, since the programme had been in operation for seven
years, the facilities and equipment appeared to be somewhat old and needed
constant maintenance and updating. At least three mentees said that some
equipment could not meet teaching needs.
“The projector in the small lecture hall was not clear.” (MT2)
“The projector is old.” (MT1)
“Some air conditioners do not cool.” (MT5)
Network problems were the most crucial issue. Most mentors and mentees
complained about the network’s system and speed.
“The network was not stable.” (M3)
“The network was bad; I have to restart the system frequently.” (MT4)
“I need to take a long time to connect to the Internet.” (M5)
Three mentees reported that there was also a shortage of the software needed to
teach specific courses or disciplines.
“We do not have the specialised software for finance.” (MT6)
“SPSS and NVivo are necessary for research and data analysis.” (MT1)
“We should have translation software to help students adapt to full
English instruction more effectively.” (MT4)
The sharing of relevant academic resources by the mentors with the mentees was
vital in implementing the programme. The mentors are the sources of technology
and information. All of the mentors mentioned that they were willing to share
their materials, such as PowerPoint displays, reference books, test papers, articles,
cases and even teaching tips and class experiences.
“I gave my mentee the name of the book in the beginning, and then I sent
her all of the course materials.” (M4)
“I like to share my experience, and I have some tips that I like to give to
my mentee.” (M6)
However, four of the six mentees reported that the educational resources that the
mentors shared with them were not necessarily beneficial.
“Sometimes the teaching resources provided by my mentor could not be
downloaded directly.” (MT5)’
574
“The shared resources must be localised because they don’t meet China’s
educational requirements.” (MT1)
4.3 Theme 3: Suitability of Course Duration and Schedule of the SFCE

Programme
In September 2015, the SFCE programme enrolled its first batch of students.
According to the MOE, the duration of an undergraduate programme is 4 years.
Classes are scheduled according to the programme content and instructor
availability, but the total number of student sessions per day should not exceed
eight, and no unscheduled classes may occur. During the implementation of the
SFCE programme, the course schedule was continuously optimised and made
more appropriate than the original arrangement.
“The latest course schedule is more reasonable than the previous one.”
(MT5)
However, most mentees and mentors described the programme’s schedule as too
intensive. Thus, the students felt tired and did not have time to participate in other
activities on campus. The following statement was endorsed by eight
interviewees:
“The freshman courses in the SFCE programme run from 8:30 a.m. to
1:30 p.m. with practical lessons and then run from 2:00 p.m. to 5:00 p.m.
with theory courses, without a real break in between: from morning to
evening, there is no time to reflect and process information.” (MT2)
Mentors were more satisfied with the schedule arrangement. The programme
management team communicated with the mentors prior to scheduling courses
each semester.
“I am consulted for my availabilities at the beginning of the semester. If I
don’t have time, I will not take it.” (M5)
In contrast to the mentors, the mentees complained about the scheduling. The
mentees had to accommodate the mentor’s time. Furthermore, due to the COVID-
19 pandemic, the mentors at University Y could not travel from Europe to China
to teach face-to-face; they had to teach online. Due to the time difference and the
large number of courses, many classes had to be held in the evening and at night,
which made mentees feel tired and ineffective.
During the interview, all of the mentees reported that they had lessons at night.
These quotes reflect the mentees’ feelings about scheduling:

“I’m taught on Friday night from 7:30 p.m. to 10:30 p.m. in China; It’s
almost 12:00 a.m. when I get home from university. That is terrible.”
(MT3)
“I get to the school at 8:20 in the morning and don’t leave until 9:30 at
night.” (MT4)
Another problem with the course schedule was that some of the arrangements of
the courses were inappropriate.
“The Thesis Writing course should be scheduled before the students write
their thesis, but it was scheduled in the last semester of the fourth year, at
575
which point some students had completed or had partially completed their
thesis. “(MT5)
“The spreadsheet course should be scheduled for the semester before the
Revenue Management course.” (MT3)
However, two mentors and two mentees reported that the contact hours for
teaching in some courses were insufficient.
“The Interior Design for Hotels course is very specialised. However, the
students in the programme lacked the relevant professional design
foundation, and it was difficult for them to understand and master the
topic in depth in only 34 hours of study.” (MT4)
“The number of hours in the Revenue Management course was
insufficient, and I would need at least 45 hours to present the same
content in the classes at University Y.” (MT3)
While there were some unsatisfactory aspects of the course schedule, some
participants were optimistic and made concrete suggestions for improvement.
Some mentors suggested that more elective courses could be offered to meet the
needs of students with different interests and schedules, as well as reduce the
course content.
4.4 Theme 4: Motivation and Drawbacks

This theme resonated with three participants who have working experience in the
hospitality industry. They wanted to share what they knew about the hospitality
industry with the students, and sharing was a way to deliver education:
“I have many years of experience in education, and I love education. That
is the most important motivation for my participation in the programme.”
(M2)
“I have 10 years of experience in the hospitality industry, and I would like

to share this with students.” (M4)
“Students are future professionals, and I shared knowledge with students
as a way to contribute to the development of the hospitality industry.”
(M3)
The researcher’s observations also confirmed the enthusiasm of the mentors for
education. Most mentors’ facial expressions were pleasant, they used body
language appropriately, and they spoke at a very proper pace and tone.
Another primary motivation for the mentors to participate in teaching in the SFCE
programme was a desire to gain a deeper understanding of China. Mentors were
very interested in the Chinese market and hoped to learn more about the
developments and changes in the Chinese hospitality industry through teaching
in the SFCE programme. A mentor explained why he agreed to participate in this
programme:
“I hoped to learn more about Chinese culture through co-teaching with
Chinese teachers.” (M4)
576
In addition, having a good mentoring relationship was also a motivation for some
mentors to choose to participate in the SFCE programme. One mentor articulated
her feelings:
“I think the relationship with the SFCE programme members at
University X was a “family” relationship: family members can be honest
and can learn from each other, and I liked the family-type relationship.”
(M6)
Extrinsic motivation, such as rewards, also affected the mentors’ enthusiasm and
motivation to participate in this SFCE programme. A mentor was very candid in
the interview:
“The financial rewards were one reason I taught in the programme.” (M5)
In contrast, all the mentees were motivated to participate in the programme

because University Y is world-renowned in the field of hospitality education, and
the mentees wanted to learn more and improve their professional ability through
the programme.
“The QS ranking of University Y of Hospitality Management is No. 1.”
(MT1)
“The Hospitality Management programme at Y University is well-known

worldwide.” (MT6)
Of the six mentees who participated in the interviews, five had a background of
studying or working abroad and were familiar with the international teaching
model. The English-speaking environment in the SFCE programme helped the
mentees maintain their English communication skills, which is one of the
motivations for wanting to teach in this SFCE programme.
“I graduated in the USA.” (MT1)
“I got my master’s degree in the UK.” (MT3)
Another three mentees stated that part of being motivated was being encouraged
by the mentors. They were inspired by their mentor’s extensive knowledge and
industry experience, excellent teaching skills, and dedication.
“I want to learn effective teaching techniques from mentors.” (MT5)
“I respect the teacher’s ability to thoughtfully prepare materials for each

class, and I should take a cue from him.” (MT2)
However, the preparation of course materials reduced mentors’ and mentees’

motivation to participate in the programme. The biggest problem was the
preparation of teaching materials. A dispiriting point for the mentors and mentees
was the conflict over the teaching requirements, which resulted in mentees
spending a lot of time redoing and collating the teaching materials.
“The biggest problem is the preparation of our teaching materials; the
mentor is not in line with our university’s requirements, which led us to
spend a lot of time redoing and collating.” (MT5)
577
“The specifications for teaching materials differ across University X and

Y. I have to get two distinct sets.” (M6)
4.5 Theme 5: Suitability of Assessment Methods

The assessment of student performance allows programme management staff to
assess how well the programme is being delivered. With regards to the
assessment, all mentors and mentees stated that they must abide by University
X’s examination requirements. Based on compliance with these requirements,
mentors and mentees used a variety of methods to assess the learning process
according to the needs of the course and their preferences including role play,
presentation or “wake-up call” attendance, and completion of class assignments.
The quotes below recounted how mentors designed the assessment:
“I gave students quizzes, three exercises in the class, and some extra
exercises as homework. I also gave students an Excel application
programme or higher programme analysis.” (M5)
“Presentations were my most common assessment method because I

wanted students to talk, apply their learning, and think more deeply. I
wanted students to ask questions when they did a presentation, which was
also the best way to develop a sense of collaborative teamwork.” (M1)
Throughout the interviews, it was evident that assessment was necessary to

stimulate students to study in this SFCE programme. Mentees and mentors
pointed out that assessment methods could motivate students to interact with the
instructor in the classroom.
“Exams make students take the course more seriously and complete all
assigned work.” (M2)
“Students are more active in answering questions in class.” (MT1)
However, mentors and mentees reported that several factors limited their
assessment practices. The first is the number of students. They could not do an
“individual presentation and case study” assessment because of the large number
of students.
“I cannot agree that eighty students do the individual presentation.” (M2)
“The size of the class is quite big, so it is tough to have more discussion.”
(M3)
The second factor is online teaching. Due to the COVID-19 pandemic, as courses
were taught via distance-learning, it was difficult for the teachers to organise
presentations, and students were less motivated to participate compared to
classes taught in person. For these reasons, most mentors and mentees reported
that online teaching also limited assessment.
“If the class is face to face, there are things I would do differently.” (M4)
The third factor is the individual student. Mentors and mentees agreed that the
ability of exams to promote learning depended on the individual student. This
resonated with most mentors and mentees:
578
“Learning is very personal, so students need to have some forms of

assessment; if there is no assessment and measurement, then the teachers
cannot evaluate their effectiveness.” (MT3)
Most mentors and mentees still believed that assessment should be continuously
improved. It would be beneficial to increase the weight of the learning progress
assessment to 50% or 60% of the total score. There should also be more emphasis
on the application of knowledge in assessment; two mentees suggested that
competitions could be incorporated into the course assessment.
“In addition to exams, students should be encouraged to participate in
business projects and discipline-related competitions. Competition results
can be part of the course scores to stimulate their interest in course
learning.” (MT1)
5. Discussion
Due to the demand for SFCE programmes in China over the last decade, it is
critical to understand faculty experiences with the implementation of these
programmes. First, the SFCE programme improves mentees’ teaching skills,
professional knowledge, and international competencies, but cultural sensitivity
and differences are evident in the process of implementing a SFCE programme,
thus reducing the mentors’ and mentees’ experience.
Further discussions are presented as follows:

A difference is in the assessment method. Assessment is an important means to
check the effectiveness of teaching programmes, and the mentees and mentors
agreed that appropriate assessment methods must be used to motivate students’
learning. However, due to the single and mandatory nature of the Chinese
teaching administration, using a final written exam was the primary means and
method of assessment, regardless of individual differences among students. This
finding was consistent with other research (Pan et al., 2020; Sun, 2018). Similar
assessment methods are also used in traditional teaching programmes, which do
not fully reflect the advantages of the SFCE programme.
In terms of assessment, Chinese teachers primarily use examinations and scores

as the final result for assessment, whereas Western teachers use a variety of
teaching evaluation methods and place a greater emphasis on the learning process
(Guo et al., 2022). Therefore, more assessments could be used, such as projects,
performance assignments, concept maps, self-assessments, peer-assessments,
observation, portfolios, drama, diagnostic trees, journals, posters, and instructor
and student interviews (Tosuncuoglu, 2018).
Another difference appears in teaching styles. Foreign teachers (mentors) in the

SFCE programme place a high value on interactive teaching and the development
of students’ abilities, but they frequently overlook the teaching of fundamental
knowledge. Chinese teachers (mentees), in contrast, do not fully use “student-
centred” teaching methods but they allocate more time to teaching in the class.
Teaching model or methods should be matched with course objectives in the SFCE
programme (O. Li, 2021; Liu, 2020; Zhu & Zhao, 2022). Diversified teaching
methods can help achieve the objectives of SFCE programme.
579
The objectives of this SFCE programme only emphasised the description of

specialised competencies but did not adequately integrate moral character and
global employability in terms of students’ worldviews, attitudes, and transferable
skills (Wu et al., 2022).
Management style and culture are the third facets of cultural difference. The
Chinese mentors’ single teaching management style restricts the implementation
of this SFCE programme. For example, Chinese administrators do not understand
their partners’ teaching philosophy. The requisite full compliance with Chinese
regulations in the preparation of teaching materials and the design of criteria in
teaching evaluation does not account for cross-cultural differences.
It is also notable that trust between the mentors and mentees is important in
building effective mentoring relationships. Mutual commitment, respect and trust
are vital elements in forming a good mentorship. The personality and attitude of
the mentee motivate the mentor (Goodsett, 2021).
Teaching resources, teaching environment, and facilities need to be continuously

improved. This study found that, in general, mentors and mentees were satisfied
with the teaching resources, facilities, and equipment of the SFCE programme and
that they were proud to work in such a teaching environment. The faculty felt
distinct from regular teaching programmes (Chang et al., 2020). Studying in a
cofounded programme with advanced facilities and foreign teachers allows
students to obtain international study experience (Knight & Liu, 2016).
The findings from this study also indicated that the Chinese university and the
foreign university need to pay more attention to the renewal and maintenance of
programme facilities and equipment with time. Older facilities and equipment
cannot support the development of the programme in terms of functionality and
aesthetics (Zhang & He, 2019). Sakibu and Kamugisha (2022) identified improving
facilities as the main factor contributing to academic achievement, and the quality,
appropriateness, and adequacy of facilities contribute to quality education. This
finding justifies the notion that ensuring sufficient facilities and space is one of the
ways to upgrade the quality of the SFCE programme. To achieve the integration
of teaching and information technology, more appropriate teaching software
should be added.
The programme’s schedule was intensive and affected the mentees’ teaching
experience. The mentors were consulted before the course was scheduled, but the
mentees had no choice, they had to follow the mentor’s class schedules. These
SFCE programmes tend to prioritise the courses of foreign teachers. The time for
Chinese teachers was very limited. As a result, mentees’ courses were even
scheduled in the evening or on weekends (An, 2016). This is important, as
Marciniak et al. (2022) have demonstrated that course schedules affect mentees’
and students’ academic performance.
In addition to intrinsic motivations, such as enthusiasm, reputation, and self-

improvement, external incentives such as a higher income and good mentorship
580
will affect mentors’ and mentees’ experiences. A similar comparison was with
Nyangau (2018), who found that faculty motivation originated from their desire
to facilitate students’ qualification for the requirements of global citizenship. The
motivations of the mentees and mentors for participating in the SFCE
programmes included self-improvement, self-satisfaction, reputation, making a
contribution, and extrinsic motivations, such as financial rewards. This finding
corroborates the work of Bandura (1989) and Mullins (2005), who noted that
intrinsic motivation is associated with psychological rewards, while extrinsic
motivation is associated with tangible rewards. Thus, it is necessary to improve
the bonuses for teachers participating in the SFCE programmes.
Finally, the global COVID-19 pandemic has had a significant impact on

everyone’s social and professional lives. Without a doubt, higher education faces
significant challenges in the delivery of teaching and research, particularly in
terms of academic mobility. Distance learning limits teachers’ ability to interact
with students in teaching and learning and affects teaching assessment. Therefore,
resuming physical classes as soon as possible was the best way to improve the
teaching experience. Meanwhile, COVID-19 was an opportune time to consider
“internationalisation at home” (Brandenburg et al., 2019) and utilise existing
international quality educational resources, such as international students,
foreign teaching materials, cross-border programmes, diversified curriculum, and
international conferences, to create an international campus with local
characteristics and to provide opportunities for cross-cultural and international
exposure for all students on campus (Wang, 2019).
6. Recommendations
There were three points to consider for recommendation.
6.1 Establish a Programme Management Committee to Run the SFCE
Programme to Improve its Communication Management Mechanism
According to the MOE’s requirements, universities implementing SFCE
programmes should set up a Programme Management Committee to be
responsible for the management, promotion, and strategic planning of the
programme. First, the Programme Management Committee should be responsible
for informing the faculty about programme objectives and strategies. Second, the
Programme Management Committee should respect different pedagogical and
cultural differences and work together to establish applicable standards and
evaluation requirements for the programme. The Programme Management
Committee should create a compromised set of standards for the programme,
taking into account the feasibility and suitability of the programme. University X
and University Y should design distinct management requirements and processes
for the programme. The approach could effectively solve the problem of cultural
differences such as teaching methods, assessment methods, and differences in
management systems. Both partners should keep an open mind and accept any
differences.
6.2 Encourage Adequate Communication Between Mentors and Mentees to

Improve the Mentoring Relationship
The mentoring relationship is significant in implementing the SFCE programme.
Some mentors and mentees mentioned a lack of communication about the
581
programme objectives and the sharing of quality resources, which led to a poor
teaching experience. Therefore, it is necessary to consciously construct a regular
consultation mechanism for running the SFCE programme to facilitate
understanding between the two sides of the school running the programme. For
instance, mentors and mentees should meet regularly at the semester’s beginning,
middle, and end, in a collaborative process to discuss co-teaching styles, content,
assessment, and student feedback to enhance mutual understanding.
Simultaneously, effective feedback is imperative. The mentor must be able to
deliver constructive feedback, and the mentees should be receptive to feedback
and upfront to let the mentor know their goals and needs.
6.3 Establish a Regular Training Mechanism and Set Up a Mentoring Workshop

to Create a Supportive Training Environment
Mentees need more training and mentoring opportunities. The Programme
Management Committee should set up a training workshop and clarify the
relationship between the mentor and mentee from both universities. In the
workshop, mentors should guide mentees’ teaching and research, and assess
them to promote their continuous improvement.
University X should develop a mentoring programme in collaboration with

University Y, and its overseas resources, to create a pathway for sustained faculty
involvement. Also, University X should implement policies that consider the
workload for mentees based on the specific needs of mentoring. There is a need
to improve the training of the personnel involved in the management of the SFCE
programmes, particularly those who have more contact with mentors, so that they
can gain an understanding of how international people from different cultural
backgrounds perceive the world and their values, and then learn to use various
communication skills to cope with and solve problems encountered during
programme implementation.
7. Conclusion
The findings of this study clarified the experiences of the mentors and mentees
who participated in the SFCE programme at a public university in China. By
extracting data from the interviews and observations of mentors and mentees, this
study provides a unique insight into the implementation of the SFCE programme
including: (a) suitability of the methods of teaching and learning; (b) suitability
and drawbacks of resources and teaching facilities; (c) suitability of course
duration and schedule; (d) motivation and drawbacks; and (e) suitability of
assessment methods.
The study’s findings are significant in proposing a new approach to further

develop the SFCE programme in China’s public universities. As part of the call to
improve the quality of the SFCE programme, the Chinese and foreign universities
running the SFCE programme should establish a Programme Management
Committee to improve the communication management mechanism of the
programme, encourage building up better mentoring relationships, and establish
a regular training mechanism to create a supportive training environment. Based
on these findings, future research could be conducted in other SFCE programmes
582
in different majors and at different universities. To achieve exemplary teaching

and learning outcomes, educational programmes require multiple stakeholders to
collaborate, and subsequent research could examine students’ and employers’
experiences regarding SFCE programme implementation.
8. References
An, F. B. (2016). Study on problems and countermeasures of curriculum setting of sino-canadian
cooperation in WeiFang No. 1 Middle School [Master’s Thesis, Shandong Normal
University].
https://d.wanfangdata.com.cn/thesis/ChJUaGVzaXNOZXdTMjAyMzAxMTIS
CUQwMTI2MzUwORoIemE2MnpwdHU%3D
Argente-Linares, E., Pérez-López, M. C., & Ordóñez-Solana, C. (2016). Practical experience
of blended mentoring in higher education. Mentoring & Tutoring: Partnership in
Learning, 24(5), 399–414. https://doi.org/10.1080/13611267.2016.1273449
Bandura, A. (1989). Human agency in social cognitive theory. American Psychologist, 44(9),
1175–1184. https://doi.org/10.1037/0003-066x.44.9.1175
Banerjee-Batist, R., Reio Jr., T. G., & Rocco, T. S. (2019). Mentoring functions and outcomes:
An integrative literature review of sociocultural factors and individual
differences. Human Resource Development Review, 18(1), 114–162.
https://doi.org/10.1177/1534484318810267
Bean, N. M., Lucas, L., & Hyers, L. L. (2014). Mentoring in higher education should be the
norm to assure success: Lessons learned from the faculty mentoring program,
West Chester University, 2008–2011. Mentoring & Tutoring: Partnership in Learning,
22(1), 56–73. https://doi.org/10.1080/13611267.2014.882606
Brandenburg, U., De Wit, H., Jones, E., & Leask, B. (2019). Internationalisation in higher
education for society. University World News, 20.
https://www.researchgate.net/profile/Elspeth-
Jones/publication/340535564_Internationalisation_in_Higher_Education_for_So
ciety/links/5e8f2a97a6fdcca789023e69/Internationalisation-in-Higher-
Education-for-Society.pdf
Chang, Q., Ge, N., & Liu, W. (2020). Research on hardware and infrastructure construction
of Sino-foreign cooperation in running schools. Young Society, (6), 268–269.
https://qikan.cqvip.com/Qikan/Article/Detail?id=7101400297&from=Qikan_S
earch_Index
Clutterbuck, D. (2014). Everyone needs a mentor (5th ed.). Kogan Page.
https://books.google.com.my/books?id=HZkkDwAAQBAJ
Creswell, J. W. (2013). Qualitative inquiry & research design: Choosing among five approaches
(3rd ed.). SAGE. http://www.ceil-conicet.gov.ar/wp-
content/uploads/2018/04/CRESWELLQualitative-Inquary-and-Research-
Design-Creswell.pdf
Creswell, J. W. (2014). Research design: Qualitative, quantitative and mix methods approach.
Sage.
https://www.ucg.ac.me/skladiste/blog_609332/objava_105202/fajlovi/Creswe
ll.pdf
Daniel, A., Franco, S., Schroeder, N. L., & Cenkci, A. T. (2019). Cross-cultural academic
mentoring dyads: a case study. Mentoring & Tutoring: Partnership in Learning,
27(2), 164–189. https://doi.org/10.1080/13611267.2019.1611286
Ellis, N. J., Alonzo, D., & Nguyen, H. T. M. (2020). Elements of a quality pre-service teacher
mentor: A literature review. Teaching and Teacher Education, 92, 103072.
583
Feng, K. (2022). An analysis of Sino-foreign cooperation in Chinese higher education. Atlantis

Press. https://doi.org/10.2991/978-2-494069-45-9_97
Gocotano, T. E., Jerodiaz, M. A. L., Banggay, J. C. P., Nasibog, H. B. R., & Go, M. B. (2021).
Higher education students’ challenges on flexible online learning implementation
in the rural areas: A philippine case. International Journal of Learning, Teaching and
Educational Research, 20(7), 262–290. https://doi.org/10.26803/ijlter.20.7.15
Goodsett, M. (2021). Commitment, respect, and trust: The building blocks of a strong
mentoring relationship. Michael Schwartz Library Publications, 176.
https://engagedscholarship.csuohio.edu/cgi/viewcontent.cgi?article=1191&co
ntext=msl_facpub
Guo, S. H., Chen, P., & Yurava, N. (2022). Cultural conflicts and coping strategies in Sino-
foreign cooperative education. Heilongjiang Education, 6, 19–21.
http://www.cnki.com.cn/Article/CJFDTotal-HLLJ202206007.htm
Knight, J., & Liu, Q. (2016). Crossborder and transnational higher education. Oxford University
Press. https://doi.org/10.1093/obo/9780199756810-0176
Li, J., & Xue, E. (2022). Sino-foreign cooperation of running schools in China. In opening
education to the outside world: rethinking international education in China during post
COVID-19 (pp. 27–42). Springer. https://doi.org/10.1007/978-981-19-4880-0_3
Li, O. (2021). Research on the teaching mode of bilingual computer courses for Chinese-
foreign cooperation in running schools programme. International Journal of Higher
Education Teaching Theory, 2(1), 83–84.
http://www.acadpubl.com/Papers/Vol%202,%20No%201%20(IJHETT%202021
).pdf#page=91
Li, X., Geng, D., Fan, R., & Hou, Z. (2020). Engineering training course arrangement under
the vision of teaching reform. Casting Technology, 41(7), 696–700.
https://doi.org/https://doi.org/10.16410/j.issn1000-8365.2020.07.025
Li, X., & Liu, J. (2019). Research on Chinese-foreign cooperation in running schools in
northwest China in the context of the belt and road initiative—from a
stakeholders’ perspectives. Journal of Research on Education for Ethnic Minorities,
155(6), 79–84.
https://qikan.cqvip.com/Qikan/Article/Detail?id=7100941885&from=Qikan_S
earch_Index
Li, Z. (2021). A brief analysis of the ways to improve the quality of foreign teachers’
specialized courses for junior college students in Sino-foreign cooperation.
International Journal of Higher Education Teaching Theory, 2(4), 111–112.
http://www.acadpubl.com/Papers/Vol%202,%20No%204%20(IJHETT%202021
).pdf#page=119
Lin, F. (2019). Research on the construction of Sino-foreign cooperative school teachers.
Journal of Higher Education, 9, 159–161.
http://www.cnki.com.cn/Article/CJFDTotal-GJXK201909055.htm
Lin, J. (2022, March 31). Promoting high-quality development of Sino-foreign cooperative
education. China Education Daily.
https://baijiahao.baidu.com/s?id=1728800876685562740&wfr=spider&for=pc
Liu, X. (2020). Research on improving the teaching quality of Sino-foreign cooperative
education in colleges and universities. Journal of Higher Education, (28), 15–18.
https://d.wanfangdata.com.cn/periodical/ChlQZXJpb2RpY2FsQ0hJTmV3UzI
wMjMwNDE5Eg1nanhrMjAyMDI4MDA0GghobXJkaHc0bQ%3D%3D
Liu, Y. L. (2018). Policy research on Chinese-foreign cooperation in running schools.
Guangdong Chemical Industry, 45(22), 124–125.
https://doi.org/10.3969/j.issn.1007-1865.2018.22.061
584
Ma, J. M., Cai, L. L., & Yang, J. P. (2019). The application of the modern apprenticeship
system in the higher education of China. Education Teaching Forum, (1), 160–162.
http://61.181.120.82:8081/kcms/detail/detail.aspx?filename=JYJU201901074&d
bcode=CJFD&dbname=CJFD2019
Marciniak, J., Wójtowicz, A., Kolodziejczak, B., Szczepanski, M., & Stachowiak, A. (2022).
Impact of course scheduling on student performance in remote learning. Association for
Computing Machinery. https://doi.org/10.1145/3502718.3524788
Martínez Huamán, E. L., Aguirre Landa, J. P., Lizama Mendoza, V. E., & García Rivas
Plata, C. E., Martínez-Huamán, E. L., Landa, J. P. A., Mendoza, V. E. L., & Plata,
C. E. G. R. (2022). Formative assessment: Component of teaching-learning process
in the university context in post-COVID-19. International Journal of Learning,
Teaching and Educational Research, 21(11), 300–314.
Ministry of Education of the People’s Republic of China (MOE). (2020, December 22).
Ministry of Education: 2, 332 Chinese-foreign cooperative institutions and projects 1,230
undergraduate and above.
http://www.moe.gov.cn/fbh/live/2020/52834/mtbd/202012/t20201222_50695
5.html
Mullins, L. J. (2005). Management and organisational behaviour (7th ed.). FT Prentice Hall.
http://www.mim.ac.mw/books/Management%20&%20Organizational%20Beh
aviour,%207th%20edition.pdf
Nyangau, J. Z. (2018). Motivations of faculty engagement in internationalization: An
agenda for future research. FIRE: Forum for International Research in Education, 4(3),
7–32. https://doi.org/10.32865/fire2018431
Pan, X. (2017). Research on efficiency evaluation of Sino-foreign cooperative education-taking
Wuhan Institute of Technology as an example [Master’s Thesis, Wuhan Institute of
Technology]. https://cdmd.cnki.com.cn/Article/CDMD-10490-1018042001.htm
Pan, X. Q., Li, Y. X., & Hou, C. J. (2020). The problems and changes of university course
assessment are discussed. Journal of Jiangxi University of Technology, 41(6), 83–86.
https://doi.org/10.13265/j.cnki.jxlgdxxb.2020.06.013
Parker, L., Xu, S., & Chi, C. (2022). Chinese preservice teachers’ perspectives of mentoring
relationships in an international learning partnership. Journal of Teacher Education,
73(5), 525–537. https://doi.org/10.1177/00224871221108655
Rajagopalan, I. (2019). Concept of teaching. Shanlax International Journal of Education, 7(2),
5–8. https://doi.org/10.34293/education.v7i2.329
Sakibu, A., & Kamugisha, N. A. (2022). Shortage of school physical facilities as a threat to
provision of quality education in Tanzania: A case morogoro municipal
community secondary schools. East African Journal of Interdisciplinary Studies, 5(1),
231–141. https://doi.org/10.37284/eajis.5.1.918
Shi, M., Chen, X., & Zhou, H. (2023). Exploration of international curriculum in the context
of vocational education. SHS Web of Conferences, 171, 03020.
https://doi.org/10.1051/shsconf/202317103020
Sohn, B. K., Thomas, S. P., Greenberg, K. H., & Pollio, H. R. (2017). Hearing the voices of
students and teachers: A phenomenological approach to educational research.
Qualitative Research in Education, 6(2), 121.
https://doi.org/10.17583/qre.2017.2374
Song, L., & Wang, D. (2020). Research on the reform method of Sino-foreign cooperative education
courses from the perspective of positive psychology. Atlantis Press.
Sun, L. (2018). An analysis of the sustainability of Sino-foreign cooperative education
projects in higher vocational colleges. Theory and Practice of Education, (9), 19–21.
585
https://d.wanfangdata.com.cn/periodical/ChlQZXJpb2RpY2FsQ0hJTmV3UzI
wMjMwNDE5EhdRS0MyMDE4MjAxODA0MjcwMDA1OTM3MxoIcXY2YnQ5b
mY%3D
Supervisory Information Platform for SFCE Ministry of Education. (2021). Contents of
evaluation of Chinese-foreign cooperation in running schools.
https://www.crs.jsj.edu.cn/news/index/24
Tosuncuoglu, I. (2018). Importance of assessment in ELT. Journal of Education and Training
Studies, 6(9), 163. https://doi.org/10.11114/jets.v6i9.3443
Wang, H.-Y. (2019). A preliminary inquiry into the internationalization and
internationalization at home of Taiwan’s higher education. STUST Journal of
Humanities and Social Sciences, (20), 31–68. https://doi.org/10.29841/STUTJHSS
Wang, K. (2021). The education features and teaching methods of economics courses in
Sino-foreign joint education program. Special Economic Zone, (9), 116–123.
http://qikan.cqvip.com/Qikan/Article/Detail?id=7105900089
Wu, S., Zhou, S., Huang, M., & Chen, W. (2022). Employment prospects of graduates from
Sino-foreign cooperative universities in China. Journal of Studies in International
Education, 26(1), 21–43. https://doi.org/10.1177/1028315320957433
Wu, W. Y., Kang, N., Hallett, D., & Li, W. J. (2019). Challenges and countermeasures to
foreign language teaching of Chinese-foreign cooperation in running schools.
Advances in Education, 9(6), 688–694. https://doi.org/10.12677/ae.2019.96112
Xia, B. B. (2020). Short-term overseas training for teachers in Chinese-foreign cooperation
in running schools– A case study based on CIPP evaluation model. Higher
Education Exploration, (8), 111–115. https://doi.org/10.3969/j.issn.1673-
9760.2020.08.019
Yan, T., & Chernytskyi, V. (2022). On improving the teaching quality of Sino-foreign
cooperation education. European Journal of Humanities and Social Sciences, 31–34.
https://doi.org/10.29013/EJHSS‑22-2-31-34
Yu, L. M. (2023). Accreditation and assessment of Sino-foreign cooperative higher education
programme. IATED.
https://library.iated.org/view/YU2023ACC?re=downloadnotallowed
Yu, R., & Dong, C. (2021). Means to elevate employment competence in Sino-foreign cooperative
education. Atlantis Press. https://www.atlantis-press.com/article/125959864.pdf
Zeng, J. (2020). Design and practice on curriculum system of Sino-foreign cooperation
programme in local application-oriented university. Research in Teaching, 43(5),
41–45. https://doi.org/10.3969/j.issn.1005-4634.2020.05.007
Zhang, X. Q., & He, J. X. (2019). Historical evolution and realistic reflection on the policy
of Chinese-foreign cooperative education. Heilongjiang Higher Education Research,
(3), 18–23. https://doi.org/10.3969/j.issn.1003-2614.2019.03.004
Zhu, F., & Huang, Y. J. (2020). Development of Sino-Foreign Cooperatively-Run Schools
and Professional Programs in 42 World-Class University in China. Heilongjiang
Researches on Higher Education, 5, 35–41.
http://www.cqvip.com/qk/80625x/20205/7101997229.html
Zhu, Y. Y., & Zhao, J. Q. (2022). The development path of co-operative education between
China and foreign countries and the development of co-education of curriculum
thinking and politics. Journal of Henan Normal University (Philosophy and Social
Science Edition), 49(5), 144–149. https://doi.org/10.16366/j.cnki.1000-
2359.2022.05.19
586

Vol. 22, No. 7, pp. 586-605, July 2023
Received Mar 30, 2023; Revised Jun 20, 2023; Accepted Aug 7, 2023
Factors that Parents in South Africa Consider in

Support of their Children’s Continuous Use of
Online Learning
David Mutambara*
Department of Mathematics, Science and Technology Education
University of Zululand, KwaZulu-Natal, South Africa
Abstract. One of the lessons learnt during the COVID-19 pandemic is the
need for technology to connect and communicate. Schools learnt to use
technology as tools of teaching and learning, as well as connecting with
parents. Such gains need not be discarded now that the pandemic has
subsided. The purpose of this study was to investigate factors that parents
in South Africa consider significant in allowing their children to continue
using online learning. The extended expectation-confirmation model,
with eight constructs, was used to explain this phenomenon using a
sample of 358 participating parents. The model was analysed using
partial least squares structural equation modeling, while SmartPLS3 was
employed in the analysis of the data. The results showed that the factors
that parents in South Africa consider significant in allowing their children
to continue using online learning were statistically significant. The seven
factors (constructs) identified in the model explained 74.6% of the
variance in support of continuous use of online learning. This was an
overwhelming support for continuance to use online learning by their
children, based on its benefits. This work contributed to the body of
knowledge by developing a model for predicting the continued use of
educational technologies (online learning), especially in developing
countries. One of the shortcomings of the study is that it only included
parents of learners in one district of South Africa. As a result, generalising
the findings to other high schools elsewhere should be done with caution.
Keywords: continuous use; online learning; parents; expectation-

confirmation model; high schools
1. Introduction
Online learning (OL) is rapidly expanding in developed countries, providing new
learning options for learners and parents. According to expert estimates, the
number of high school learners in online courses exceeds one billion (Dong et al.,
* Corresponding author: David Mutambara; mutambarad@unizulu.ac.za
©Authors
587
2020). Parents and learners choose online institutions and online courses for a
variety of reasons stemming from their specific needs. There are a number of
reasons why parents and their children prefer OL (Agaton & Cueto, 2021; Lau &
Lee, 2021). For instance, parents and their children may be looking for subjects
that are not available at their local schools (Watson et al., 2010). Another attraction
is that learning can take place at the learner’s own pace (Agaton et al., 2021).
Learners with certain medical conditions may benefit from OL (Lau et al., 2021).
Learners can gain access, through OL, to specialist teachers who are located far
away, which they would not be able to do in the classroom (Maqableh & Alia,
2021). Parents believe that virtual laboratories (which may not be available in their
locality) can be used to help their children better understand science concepts
(Mutambara & Chibisa, 2022b; Watson et al., 2010). These reasons for choosing
online institutions and OL has led to the proliferation of OL service providers
(Watson et al., 2010).
The demand for OL further increased due to the global spread of COVID-19 which
caused countries to implement national lockdowns (Wang et al., 2021). During
these lockdowns, people completely depended on information and
communication technologies for teaching and learning which, in developing
countries especially, was relatively new (Mutambara & Bayaga, 2021b). This
necessitated the engagement of all educational stakeholders, including the
Department of Basic Education (DBE), learners, teachers, and parents.
The DBE spent a significant amount of money to fund OL (Mulenga & Marbán,
2020). For example, the DBE collaborated with cellular network providers to offer
zero-rated educational learning platforms (Mulenga et al., 2020). Parents assisted
by purchasing mobile gadgets to help their children’s online study (Mutambara
& Bayaga, 2020b; Xu & Jaggars, 2013). Teachers developed a large amount of
subject-specific OL resources (Mutambara & Bayaga, 2020a).
Online learning has been found in studies to boost learners’ interest and
performance (Xu et al., 2013). With all these benefits and the investments made, it
is critical that OL remains a component of teaching and learning post COVID-19.
Online learning will only be a success in developed nations if learners and
teachers continue to use it (Suzianti & Paramadini, 2021; Wang et al., 2021).
Several studies have been undertaken to better understand the factors that
influence users’ acceptance of OL and, as a result, its effective implementation
(Maheshwari, 2021; Mutambara et al., 2021b). However, there has been a limited
amount of research conducted on the post-acceptance of OL, specifically its
continued use (Chandradasa & Galhena, 2021; Huang, 2021; Suzianti et al., 2021;
Wang et al., 2021). Few studies have been undertaken to study the factors that
learners (Chandradasa et al., 2021; Holmes, 2013; Huang, 2021; Wang et al., 2021)
and teachers (Holmes, 2013; Suzianti et al., 2021) consider when deciding whether
or not to continue using OL. However, the question remains as to whether the
research studies included all of the stakeholders in high school education
(Mutambara et al., 2020b).
588
When it comes to the continued use of OL, parents’ perspectives have not received
the consideration they deserve. However, the ability of parents to adapt and
become accustomed to utilising technology is required and necessary for assisting
the learning process of their children with the use of OL (Dong et al., 2020). This
comes in the form of parents assisting with the purchasing of the required gadgets
for their children or even practically helping with the actual homework on these
gadgets, let alone giving the necessary support and supervision required by
learners outside the school environment.
To fill this gap, this study sought to investigate the factors that parents in South
Africa consider to be significant in allowing their children to continue using OL.
These factors will help all stakeholders in education in decision making. Parents
will also be informed of the critical factors on which they must concentrate to
support their children in learning effectively.
2.1 The Impact of COVID-19
During the peak of the COVID-19, all non-essential services were suspended,
including schools. This led the DBE to order schools to switch to OL in order to
save the academic year. The shift to OL was carried out with little to no
forethought (Mulenga et al., 2020), thereby causing a chaotic transition. In the
midst of this chaos, the government suggested that, rather than adding new
technologies (which was costly), teachers should expand educational
opportunities by utilising the existing technologies (Ardington et al., 2021;
Mulenga et al., 2020). The administration also claimed that mobile devices capable
of supporting OL were available in most homes (Landa et al., 2021). In so doing,
OL took the role of traditional education. This meant that parents were required
to monitor their children’s learning at home.
This changeover was fraught with difficulties (Landa et al., 2021; Mulenga et al.,
2020; Mutambara & Bayaga, 2021a; Ramrathan, 2021). The majority of South
African educators were not trained to use OL (Landa et al., 2021). Teachers were
hesitant to teach online because they believed parents were watching them
(Mulenga et al., 2020). Some parents were forced to purchase mobile gadgets that
could facilitate OL (Mutambara et al., 2021a; Ramrathan, 2021). Parents also found
it difficult to monitor and support their children as they were learning online
(Landa et al., 2021). For learners, erratic internet connection and electricity supply
interrupted their OL (Mutambara et al., 2021b). Despite these numerous hurdles,
the DBE, teachers, learners, and parents persisted with OL.
Eventually, the virus infections came to a peak and then subsided and life could
resume as it had been prior COVID-19. Schools had to open, allowing teachers
and learners to return to face-to-face instruction. This risked the loss of all the
gains of OL which did not sit well with parents and other stakeholders. Therefore,
some parents and other stakeholders wished to explore and identify factors that
would allow children to continue using OL.
589
2.2 Online Learning and Parents’ Views

Online learning is defined as learning that occurs over the internet, either
synchronously or asynchronously, in which learners conveniently interact with
content, educators and other learners (Dong et al., 2020; Heba & Sultan, 2020).
Online learning has grown significantly in recent years due to its better flexibility
in terms of time, location, and pace of study (Huang, 2021). Furthermore, OL
offers convenient and highly efficient access to a wider array of knowledge, while
minimising expenses (Huang, 2021; Mutambara et al., 2022b). In developing
countries, OL mitigates shortages of learning materials (Chibisa et al., 2021;
Mutambara et al., 2022b). In some cases, it can be used to enhance learners’
understanding of science concepts using computer simulations (Mutambara et al.,
2021a). However, in emerging nations, accelerated development of OL was
witnessed in institutes of higher learning, rather than schools (Chandradasa et al.,
2021; Mutambara et al., 2021b).
As OL continues to expand rapidly, governments, educational institutions, and

companies worldwide are increasingly advocating for its adoption, resulting in a
shift away from traditional in-person classrooms (Huang, 2021; Mulenga et al.,
2020). The advancement of communication technologies has enabled OL to create
dynamic and authentic learning environments, fostering enhanced collaboration
and interdependence among learners (Chandradasa et al., 2021; Lu et al., 2019).
In the context of OL, authenticity can be achieved through various means. For
instance, Chandradasa et al. (2021) stated that online platforms can offer
simulations, virtual environments, and interactive multimedia resources that
simulate real-world scenarios. The immersive experiences provided enable
students to apply their knowledge and skills within a secure and controlled
environment, replicating real-life situations that they may encounter in their
future professional endeavours (Lu et al., 2019). Lu et al. (2019) added that OL
often involves the use of technology, tools and resources that are commonly
utilised in professional contexts. Students can gain practical experience in using
digital tools, software applications, and online collaboration platforms that are
prevalent in today’s workplace (Xu et al., 2013). By engaging with these
technologies, students develop digital literacy skills and become familiar with the
tools they will likely encounter in their professional lives (Zhu et al., 2020).
Despite the numerous benefits of OL, parents’ views are polarised (Isikoglu
Erdogan et al., 2019; Lu et al., 2019). Some parents have reservations about OL
(Isikoglu Erdogan et al., 2019; Stites et al., 2021). For instance, some parents are
concerned that OL may lead to social isolation and a lack of engagement, thus
impacting their children’s development (Lau et al., 2021). Some parents are also
concerned that their children’s excessive exposure to OL may expose them to
inappropriate information (Stites et al., 2021). They are also concerned about
cyberbullying (Isikoglu Erdogan et al., 2019), while others are anxious about the
time that their children spend online, claiming that it can lead to physical health
concerns (Heba et al., 2020). Some parents are concerned about the necessity to
monitor their children’s usage of these devices by putting parental care tools in
place (Isikoglu Erdogan et al., 2019; Lau et al., 2021).
590
Conversely, certain parents have expressed support for their children’s utilisation
of technology and online learning (OL), recognising the potential for valuable
technological skills’ development and the enhancement of academic growth
(Mascheroni et al., 2016). They believe that equipping children with computer
proficiency will contribute to their future opportunities, including career
prospects (Hatzigianni & Kalaitzidis, 2018; Kumpulainen & Gillen, 2017;
Mascheroni et al., 2016). In the extremely diverse digital environment of the
modern world, some parents believe that a variety of digital and online tools may
provide additional knowledge and learning opportunities for their children
(Hatzigianni et al., 2018). There is a belief among certain parents that digital
technologies and OL can play a crucial role in fostering the development of
children’s learning competencies, language skills, self-expression, and social
acceptance (Nouwen & Zaman, 2018). For most parents, the advantages of OL
outweigh the disadvantages, resulting in its rapid growth.
2.3 Factors that Influence Parents to Allow Children to Continue Using Online
Learning
Numerous studies have investigated parents’ perspectives on OL (Kumar et al.,
2010; Mutambara et al., 2020b; Özdamlı & Yıldız, 2014; Poçan et al., 2021). Özdamlı
et al. (2014) focused on parents’ views towards parent-school collaboration using
mobile devices. Mutambara et al. (2020b) investigated factors that influence
parents’ pre-acceptance views of OL. The results revealed that parents’
behavioural intention to allow their children to use OL was influenced by the
perceived ease of use of OL, the perceived usefulness of OL, perceived attitude
towards OL, and perceived resources needed for OL (Mutambara et al., 2020b).
Kumar et al. (2010) emphasised that parents are concerned with the high cost
involved in OL. Poçan et al. (2021) also noted that parents consider perceived ease
of use, perceived performance impact, and perceived usefulness when allowing
their children to use OL to learn mathematics. Parents consider teachers’ support
and preparedness when they allow their children to use OL (Eutsler, 2018).
Parents’ attitudes towards OL play a very important role in their acceptance of
OL for their children (Mutambara et al., 2021a).
Thus far, the current authors have found no evidence that available research
examined what factors impact parents’ decisions to allow their children to
continue OL. However, there have been few studies that have looked into
learners’ (Zhu et al., 2020) and teachers’ (Holmes, 2013; Holmes & Sime, 2012) )
continuous use of OL. According to the findings, some of the factors that influence
teachers’ pre-acceptance of OL also influence their continued use (Chandradasa
et al., 2021; Zhu et al., 2020). According to the findings of these studies
(Chandradasa et al., 2021; Holmes et al., 2012; Zhu et al., 2020), several of the
factors that influence learners’ pre-acceptance of OL also influence their post-
acceptance. These results can be extrapolated towards the factors that influence
parents’ support of learners’ continued use of OL.
According to some studies, the perceived usefulness of parents influences their

behavioural intention to allow their children to use OL (Mutambara et al., 2021a).
Parents’ behavioural intentions were found to be influenced by cost and perceived
591
resources (Mutambara et al., 2020b). Another study by Agaton et al. (2021) found
that performance impact and educator preparedness influence parents’
willingness to allow their children to use OL. Teachers’ and the DBE’s support has
a significant impact on parents’ behavioural intentions to use OL (Heba et al.,
2020).
3. Framework of the Study

The expectation-confirmation model (ECM) developed by Bhattacherjee (2001)
was used to explain the factors that parents consider important when allowing
their children to use OL. The ECM states that users’ continuous use of an
information system (in this case, OL) is influenced by their satisfaction and
usefulness (Bhattacherjee, 2001). Satisfaction and usefulness are influenced by
confirmation (Bhattacherjee, 2001). The ECM has been widely used in the context
of educational technology to explain users’ continued use of these technologies
(Huang, 2021; Nugroho et al., 2019; Suzianti et al., 2021). Nugroho et al. (2019)
noted that more educational-related variables or constructs should be added to
ECM to improve its explanatory power. This study, like Alarabiat et al. (2021) and
Chandradasa et al. (2021), extended the ECM with educational related constructs
stated as: confirmed performance impact, confirmed costs, confirmed support,
confirmed teachers’ preparedness, attitude towards, and confirmed usability
(CU). These added constructs, together with the ECM constructs, make up the
conceptual framework of this study and are explained as follows.
3.1 Satisfaction (SFN)

Satisfaction refers to the cognitive and emotional state experienced by a user in
response to a particular activity, such as OL, encompassing both physical and
emotional reactions (Alarabiat et al., 2021). Users will be emotionally fulfilled if
the outcomes fit their needs, expectations, task orientation, and goal
determination (Holmes, 2013; Lu et al., 2019). User satisfaction is seen as a
significant predictor of an information system success. The user’s ongoing
intention to use an information system is primarily influenced by their satisfaction
with the system’s usage (Suzianti et al., 2021). Huang (2021) stressed that the key
determinant of continued intention is satisfaction gained after actual use of an
educational technology. When the level of satisfaction is high, it results in a
corresponding high level of continued use (Alarabiat et al., 2021; Xu et al., 2013).
Satisfaction is usually regarded as a mediating factor of post-adoption factors and
continuous use (Nugroho et al., 2019). If parents are satisfied with OL, they would
want their children to continue using it. Therefore, the hypothesis for the construct
satisfaction is:
H1: Parents’ satisfaction influences their children’s continuous use of OL.
3.2 Confirmed Usability (CU)

According to Lewis (2018), usability refers to the degree of effectiveness in which
designated users can successfully accomplish predetermined objectives within a
particular context. In this study, confirmed usability is defined as parents’
confirmed belief that their children can use OL to learn. The amount to which a
task goal is successfully attained (e.g., the proportion of users who are able to
finish a particular task) is referred to as effectiveness (Lewis et al., 2015). This
592
study proposed that if parents have a confirmed belief that their children can learn
using OL, they will be satisfied and have a positive attitude towards it. Therefore,
the hypotheses of the construct confirmed usability are:
H6: Parents’ confirmed usability influences their satisfaction with OL.
H8: Parents’ confirmed usability influences their attitude towards OL.
3.3 Attitude Towards (AT)

In this study, the phrase “attitude towards” (AT) refers to parents’ general
affective reaction to the use of OL (Mutambara et al., 2021b). After permitting their
children to use OL, parents’ views regarding it develop. These attitudes may be
favourable or negative (Alarabiat et al., 2021; Almahamid & Rub, 2011; Nouwen
et al., 2018). If their expectations are met or surpassed, parents will have a positive
attitude towards OL. Attitudes have been shown to influence satisfaction (Huang,
2021). If parents develop a positive attitude with OL, they will be satisfied with it.
Therefore, the hypothesis for the construct attitude towards is.
H5: Parents’ attitude towards influences their satisfaction with OL.
3.4 Confirmed Performance Impact (CPI)

In this study, parents’ confirmed view that OL enhances their children’s
performance is defined as a confirmed performance impact. The goal of
performance impact is to quantify one component of performance impact, which
is parents’ self-report on the perceived influence of OL on their children’s
performance (DeLone & McLean, 2016; El Said, 2015). Despite the fact that
improving users’ efficiency in executing activities involved in information
systems is one of the key aims of an information system, the performance impact
of an information system has seldom been examined, or even considered, in
information system adoption models (Goodhue & Thompson, 1995). This study
postulated that if parents have a confirmed performance impact, they will allow
their children to continue to use OL and be satisfied with it. As a result, the
construct “confirmed performance impact” hypotheses are as follows:
H4: Parents’ confirmed performance impact influences their satisfaction with OL.
H2: Parents’ confirmed performance impact influences their children’s continuous use of
OL.
H9: Parents’ confirmed performance impact influences their attitude towards OL.
3.5 Confirmed Teachers’ Preparedness (CTP)

The concept of teachers’ preparedness encompasses the “state or situation of
being prepared; readiness”, emphasising the attitudinal element of being ready
for something (Gill & Dalgarno, 2008, p. 12). Human factors (such as a lack of
confidence in utilising OL, reluctance to change and unfavourable attitude
towards OL, and lack of perceived benefits, in addition to a lack of time, resources,
and training) were important and constant barriers to teachers’ use of OL (Gill et
al., 2008; Hero, 2020; Nasiri et al., 2014). According to studies, teachers’ readiness
influences their support (Hero, 2020) and attitude towards the OL (Nasiri et al.,
2014). This study postulated that parents’ perceptions of teachers’ readiness to use
OL will influence their confirmed support and attitude towards OL. As a result,
the construct “confirmed teachers’ preparedness” hypotheses are as follows:
H10: Parents’ confirmed teachers’ preparedness influences attitude towards OL.
H11: Parents’ confirmed teachers’ preparedness influences their confirmed support.
593
3.6 Confirmed Support (SUP)

Learner support is yet another indicator of the effectiveness of OL. Academic
assistance, counselling and mentorship, tutoring, and technological assistance are
examples of support services (Watson et al., 2010). Several of these tasks are
handled by educators in some schools; however, other schools dedicate personnel
for part or all of these services. In some schools, learners are directed to a support
website, while in others, support is embedded into the course material and
software (Watson et al., 2010; Yu et al., 2017). Most schools employ a combination
of all of the above (Hossain et al., 2020; Mutambara et al., 2021a).
Learners and parents both require technical assistance with OL devices and, thus,
it is critical that schools provide adequate technical support to alleviate the
irritation caused by technology-based interruptions to learning (Mutambara et al.,
2020b; Stites et al., 2021; Watson et al., 2010). Several studies have investigated
support as part of perceived resources, not as a standalone construct, and they
found that it influences users’ attitude towards the use of an educational
technology (Chibisa et al., 2021; Mutambara et al., 2020a; Mutambara et al., 2022b;
Zhu et al., 2020). This study postulates that if schools can provide academic
assistance, counselling and mentorship, tutoring, and technological assistance to
learners, parents will have a positive attitude towards their children’s OL.
H12: Parents’ confirmed support influences their attitude towards children’s OL.
H7: Parents’ confirmed support influences their satisfaction with OL.
3.7 Confirmed Cost (CC)

There are fees associated with online schooling that parents must bear (Watson et
al., 2010). Parents must purchase data bundles and equipment that can facilitate
OL (Mutambara et al., 2021a; Nasiri et al., 2014). Several studies have found that
OL expenses have a unfavourable effect on user attitudes (Mutambara et al.,
2021b; Poçan et al., 2021). Huang (2021) observed that the cost of OL influences
user satisfaction and continuous use. This study assumes that the expenses of OL
have an impact on parents’ continued use, satisfaction, and attitude towards use.
As a result, the construct “confirmed cost” hypotheses are as follows.
H3: Parents’ confirmed cost influences their children’s continuous use of OL.
H13: Parents’ confirmed cost influences their attitude towards use of OL.
H14: Parents’ confirmed cost influences their satisfaction with OL.
The 14 hypotheses and seven constructs explained above make up the conceptual
model shown in Figure 1. These seven constructs explain continuous use (CONT)
of OL. Hence, CPT predicts SUP and they both predict AT. CC, CU, and CPI are
antecedents of AT and SFN. CC, CPI, and SFN have a direct effect CONT.
594
Figure 1: Conceptual model
4. Methodology
4.1 Research Design
Using a cross-sectional survey design, the research aimed to quantitatively
capture the perspectives of a particular population by analysing a subset of
individuals within that population (Creswell & Creswell, 2017). This design was
utilised to obtain a quantitative description of parents’ perspectives on their
children’s continuous utilisation of OL. The survey design enabled the collection
of a substantial amount of data from parents within a short timeframe and at a
reasonable cost. Additionally, it facilitated the acquisition of opinion-based data
from parents through the use of a questionnaire. The proposed model was
examined using partial least squares structural equation modeling (PLS-SEM)
implemented with the assistance of SmartPLS3.
4.2 Participants
To gather data, the research employed stratified random sampling as described
by (Creswell et al., 2017). In the study, all high schools within South Africa’s King
Cetshwayo District were categorised into quintiles based on their characteristics.
Schools within the same quintile were grouped together to form strata. This
stratification approach ensured that schools with similar characteristics were
grouped together, reducing potential estimation errors (Creswell et al., 2017). A
total of five strata were created as a result. From each stratum, one school was
selected using simple random selection. Additionally, simple random sampling
was employed to choose 550 students from the selected schools. Each of these
students received a questionnaire for each of their parents, resulting in a sample
size of 550 for the parents in the study.
595
Out of the initial 550 questionnaires distributed, a response rate of 71% was
achieved, resulting in the collection of 390 completed questionnaires. However,
only 92% (358 responses) of the collected questionnaires were considered suitable
for inclusion in this study after undergoing data screening, while the remaining
8% were excluded. Regarding the demographics of the respondents, 35% resided
in rural areas, 40% in semi-urban areas, and 25% in urban areas. In terms of gender
distribution, 64% of the respondents identified as female, while 36% identified as
male.
The constructs with the highest number of indicators were “continued use” and
“confirmed support”, each consisting of five indicators. In accordance with the
guideline proposed by Hair et al. (2017), which suggested a minimum sample size
of at least 10 times larger than the number of indicators in the construct with the
most indicators, a sample size of 50 or more was recommended. The sample size
of 358 participants in this study exceeds this recommended minimum by a
significant margin.
4.3 Instrument Formation

The data collection process involved two main parts. Firstly, parents provided
information regarding their demographic background. Secondly, respondents
completed the essential section of the questionnaire, which encompassed scales
designed to measure the constructs of the model. The questionnaire was adapted
from previous research studies conducted by Huang (2021) and Mutambara et al.
(2021a), and customised to suit the specific requirements of the present study. All
items in the survey’s questionnaire were directly framed within the context of OL.
Respondents were asked to indicate their agreement level on a 7-point Likert-type
scale, ranging from strongly disagree to strongly agree.
4.4 Data analysis

The collected data were analysed using partial least squares structural equation
modeling (PLS-SEM), with the assistance of SmartPLS3. The analysis process
involved two main stages. First, the measurement model was evaluated. This step
focused on examining the reliability and validity of the measurement scales used
in the study (Hair et al., 2017). The measurement model assessed the relationships
between the observed variables and their corresponding latent constructs,
ensuring that the indicators accurately measured the underlying constructs (Hair
Jr et al., 2016). Second, the structural model was assessed. This step aimed to
investigate the relationships and effects among the latent constructs in the
proposed theoretical model (Hair Jr et al., 2017). The structural model analysis
examined the direction, significance, and strength of the relationships between
different constructs, providing insights into the hypothesised relationships and
their statistical significance (Hair et al., 2017).
By conducting both the measurement model evaluation and the structural model
assessment, the analysis provided a comprehensive understanding of the
relationships among the constructs and their overall fit to the collected data.
596
5. Results
5.1 Measurement Model
To assess convergent validity, the extracted values of internal consistency, item
reliability, and average variance extracted values (AVE) are used (Hair Jr et al.,
2016; Hair Jr et al., 2017). In this investigation, most of the outer loadings in Figure
2 were greater than the recommended cut-off value of 0.7. The only construct with
outer loading below 0.7 was cu1 (0.649). This construct was not removed from the
model because of its contribution to content reliability. These findings suggested
that item reliability was adequate (Hair Jr, Hult, Ringle, & Sarstedt, 2021). The
composite reliability (CR) threshold value is 0.7 (Hair et al., 2017; Hair et al., 2021).
According to the data in Table 1, all the CR values were greater than the threshold
value of 0.7, suggesting adequate internal consistency. The cut-off value of the
average variance extracted (AVE) is 0.5 (Hair et al., 2017). The results in Table 1
show that all the AVE values were greater than 0.5. Acceptable item reliability,
AVE, and internal consistency all confirmed convergent validity (Hair Jr et al.,
2016).
Table 1: Measurement model

Construct AT CC CONT CPI CPT CU SFN SUP
AVE 0.739 0.733 0.791 0.685 0.704 0.588 0.803 0.781
CR 0.919 0.916 0.949 0.897 0.905 0.850 0.942 0.947
To assess discriminant validity, the Fornell-Larcker criterion was used (Hair Jr et

al., 2016). According to Hair et al. (2017), in order to assess discriminant validity,
the square root of the AVE for each latent variable should be greater than its
highest correlation with any other latent variable. The findings demonstrated that
each construct in the model assesses different factors that parents in South Africa
consider significant in allowing their children to continue using OL.
Table 2: Fornell-Larcker criterion

AT CC CONT CPI CPT CU SFN SUP
AT 0.860
CC 0.830 0.856
CONT 0.651 0.733 0.889
CPI 0.596 0.693 0.775 0.888
CPT 0.652 0.650 0.713 0.825 0.890
CU 0.489 0.542 0.562 0.598 0.504 0.767
SFN 0.599 0.728 0.828 0.779 0.685 0.611 0.896
SUP 0.750 0.637 0.514 0.664 0.735 0.502 0.715 0.884
5.2 Structural Model

Once the validity of the measurement model was established, the structural model
was evaluated following a four-step process proposed by Hair Jr et al. (2016). As
outlined by Hair Jr et al. (2016), this process involves analysing collinearity in the
structural model, assessing the significance and relevance of path coefficients,
evaluating the model’s explanatory power, and finally, examining its predictive
power. This four-step evaluation framework was applied in the current study.
597
To test for collinearity in the measurement model, variance inflation factor (VIF)
values were computed, as suggested by Hair Jr et al. (2017). Table 3 displays the
VIF values; all of them were found to be below four, indicating the absence of
collinearity issues within the model (Hair Jr et al., 2017).
Next, the significance of the path coefficients was evaluated using the
bootstrapping method with 5000 subsamples. Table 3 presents the results,
revealing that three out of the 14 examined hypotheses were rejected, while 11
hypotheses were accepted. Hypotheses were rejected based on p-values greater
than 0.05 and t-values less than 1.96, adhering to the established criteria (Hair Jr
et al., 2017).
Table 3: Structural model

Std
Path t-values p-values Decision f2 VIF
Beta
AT to SFN 0.334 3.002 0.003 Accepted 0.097 3.542
CC to AT 0.652 7.028 0.000 Accepted 0.894 2.226
CC to CONT 0.207 3.093 0.002 Accepted 0.730 2.327
CC to SFN 0.463 5.071 0.000 Accepted 0.203 3.173
CPI to AT -0.269 1.124 0.262 Rejected 0.068 3.966
CPI to CONT 0.264 3.239 0.001 Accepted 0.099 2.781
CPI to SFN 0.314 3.870 0.000 Accepted 0.149 2.617
CPT to AT 0.177 0.660 0.510 Rejected 0.030 3.952
CPT to SUP 0.735 16.347 0.000 Accepted 1.173 1.000
CU to AT 0.021 0.511 0.609 Rejected 0.001 1.683
CU to SFN 0.138 2.202 0.028 Accepted 0.046 1.658
SFN to CONT 0.471 5.125 0.000 Accepted 0.283 3.082
SUP to AT 0.373 3.100 0.002 Accepted 0.264 2.462
SUP to SFN 0.393 4.077 0.000 Accepted 0.214 2.851
The coefficient of determination (R2) and effect size (f2) values are used to evaluate
the model’s explanatory power (Hair Jr et al., 2021). Hair Jr et al. (2021)
demonstrate that R2 values of 0.19, 0.33, and 0.67, respectively, imply a weak,
moderate, and substantial level of accuracy (Hair Jr et al., 2021). According to Hair
Jr et al. (2021), f2 values of 0.02, 0.15, and 0.35 correspond to effect sizes of small,
medium, and large, respectively. The R2 values of SUP (0.540) is considered
moderate, while the R2 values of AT (0.786), SFN (0.747), and CONT (0.746) are
considered a substantial level of accuracy.
The effect sizes were minimal for the routes AT to SFN (0.097), CPI to AT (0.068),
CPT to AT (0.030), CPI to SFN (0.149), CPI to CONT (0.099), CU to SFN (0.046),
and CU to AT (0.001). SFN to CONT (0.283), SUP to AT (0.264), SUP to SFN (0.214),
and CC to SFN (0.203) had medium impact sizes. The routes with the largest effect
sizes were CC to AT (0.894), CC to CONT (0.730), and CPT to SUP (1.173),
indicating that removing the predictor from the model had a significant influence
on the R2 value of its endogenous variable.
598
Figure 2: Structural model

The predictive power of the model was assessed using the Stone-Q-squared
Geisser’s statistic. By running the blindfold procedure, the results show that
endogenous variables AT, CONT, SFN, and SUP have Q2 values of 0.562, 0.580,
0.582, and 0.417, respectively. All Q2 values in the model were found to be greater
than zero, indicating that the model had satisfactory predictive significance (Hair
Jr et al., 2017). This implies that the predictors CPT, SFN, SUP, AT, CPI, CC, and
CU are among the factors that motivate parents of children in South Africa to
continue utilising OL.
6. Discussion
The aim of this study was to investigate the factors that parents in South Africa
consider significant in allowing their children to continue using OL. This was
accomplished by incorporating context-related constructs into the expectation-
confirmation model, as proposed by Foroughi et al. (2019). The model explained
74.6% of the variance in the continuous use of OL by parents in South Africa in
allowing their children to continue using OL. All the Q2 values were greater than
zero. This means that the constructs of confirmed cost, confirmed support,
confirmed teachers’ preparedness, confirmed performance impact, attitude
towards, confirmed usability, and satisfaction are good predictors of parents in
South Africa in allowing their children to continue using OL.
The results revealed that confirmed teachers’ preparedness influence parents’

confirmed support but not their attitude towards the use of OL. These studies
contradict the findings of Nasiri et al. (2014), who reported that teachers’
preparedness influence parents’ attitude towards the use of OL. This result is not
surprising, since most parents in South Africa are not well educated and they do
not know what their children are learning (Nasiri et al. (2014). The results are also
in line with the findings of Hero (2020), who stated that teachers’ confirmed
preparedness influences parents’ confirmed support. The results imply that
599
teachers should be well prepared to provide adequate technical support to both

parents and learners to alleviate the irritation caused by technology-based
interruptions to learning.
Congruent to the findings of Stites et al. (2021) and Watson et al. (2010), parents’
confirmed support influence their confirmed attitude towards use and
satisfaction. These results imply that parents’ overall views on allowing their
children to continue to use OL is influenced by the academic assistance,
counselling, and mentorship, tutoring, and technological assistance that they and
their children receive from the school. It is important for schools to continue to
support parents and learners for continuance of OL.
Parents’ satisfaction was influenced by their attitude towards use. This outcome
is consistent with previous research findings (Alarabiat et al., 2021; Almahamid et
al., 2011; Huang, 2021), which found that after enabling their children to use OL,
parents acquired a good attitude towards it, which influenced their satisfaction.
This positive attitude towards OL is reinforced by the assistance they received
from teachers, the good impact of OL on their children’s performance, and
teachers’ preparedness.
Contrary to the findings of Lewis (2018), the findings of this study demonstrated
that parents’ confirmed usability influences their satisfaction rather than their
attitudes. Given that OL has been in use for some time and that studies
demonstrated that it is effective and increases learners’ performance (Mutambara
& Chibisa, 2022c), one would expect its effectiveness to favourably affect parents’
attitude towards it. The ability of OL to enable learners to learn and improve their
performance influences parents’ satisfaction with it.
Confirmed performance impact had a considerable influence on satisfaction and

continued use but had no effect on attitude towards. This finding is congruent
with the findings of DeLone et al. (2016), who reported that when choosing an OL
platform, parents are mainly concerned with their children’s performance. These
findings show that parents are satisfied with OL and they will continue to allow
their children to use it because of its potential to increase learners’ performance.
The findings also revealed that confirmed costs have a beneficial impact on
attitude, satisfaction, and continued use. These findings are consistent with
previous studies (Huang, 2021; Mutambara & Chibisa, 2022a; Poçan et al., 2021).
One possible explanation for these findings is that learners may already have had
devices that could support OL and, thus, they did not need to invest in new
gadgets. Furthermore, the DBE collaborated with cellular network carriers to offer
zero-rated educational websites. This lowered the overall cost of OL. It is this cost
saving that persuade parents to allow their children to continue using OL.
Congruent to the common belief in the body of knowledge (Alarabiat et al., 2021;
Suzianti et al., 2021; Xu et al., 2013), our results showed that parents’ satisfaction
strongly correlates with their continued use. Information system users’ continued
intention is primarily determined by satisfaction with the usage of that
information system (Suzianti et al., 2021). Huang (2021) stressed that the key
600
determinant of continued intention is satisfaction gained after actual use of

educational technologies. Increased satisfaction levels are associated with higher
levels of continuous use (Alarabiat et al., 2021; Xu et al., 2013). The results also
revealed that satisfaction plays a very important mediating role between
continued use and confirmed support, confirmed teachers’ preparedness, and
confirmed support. It is very important for parents to be satisfied with OL for
them to continue supporting their children in using the technology.
7. Theoretical Implications
The present study adds to the existing literature in four ways. First, the study
presents empirical evidence that, despite being designed two decades ago
(Bhattacherjee, 2001), the expectation-confirmation model may still be utilised to
predict users’ continued use of technology.
Second, the findings validated the recommendation of Nugroho et al. (2019) that
the ECM be expanded by including context-related antecedents to explain the
continued use of technology in a different setting. According to the findings of
this study, confirmed cost influence satisfaction, continued use, and attitude
towards use. Confirmed performance impact influences parents’ satisfaction and
continued use. Confirmed teachers’ preparedness predicts confirmed support,
which influences attitude towards use and satisfaction.
Third, by developing a model for predicting the continued use of educational

technologies, this work contributes to the body of knowledge. This may be
advantageous to developing countries’ continued use of educational technologies.
This is significant because most of the educational technology research has been
conducted in developed countries to date.
Fourth, the study’s findings revealed that satisfaction was the strongest direct
predictor of continued use. This suggests that for users to continue utilising
educational technology, they must be content with them. More factors influencing
user satisfaction with educational technologies should be identified.
8. Practical Contributions
This study and its findings have several practical ramifications. Parents’
confirmed support influences their confirmed attitude towards use and
satisfaction. It is critical for schools to continue to help parents and learners for
OL to continue to be used. When implementing OL, schools in South Africa
should employ OL dedicated technicians to assist parents and learners.
According to the findings of this study, parents’ satisfaction is the best predictor
of their continuous use. The findings also demonstrated that satisfaction plays a
critical role in mediating the relationship between continuous use and confirmed
usability, confirmed teachers’ preparedness, and confirmed support. It is critical
to increase parents’ satisfaction with OL for it to be used on a consistent basis. This
can be accomplished by providing technical and psychological assistance to
parents and learners. This can be achieved by hiring technicians to assist with
technological challenges.
601
Our results revealed that confirmed cost has a positive impact on attitude
towards, satisfaction, and continuous use. The DBE should assist learners by
providing them with devices that are suitable for OL and data bundles. Schools
can also use offline portals, where students can only connect to the school’s Wi-Fi
once a week to get their learning materials. This reduces the amount of data
required to support OL and, as a result, the associated costs.
9. Limitations and Future Studies

The study’s shortcomings include the fact that it only included parents of high
school learners in one district. As a result, generalising the study’s findings to all
high schools in South Africa should be done with caution. Additionally, this study
relied on self-reported data, which introduces the possibility of response bias and
social desirability bias. Participants may not always provide accurate or truthful
information about their attitudes, behaviours, or experiences, leading to potential
errors in the results.
Future studies could consider employing additional measures or alternative data

collection methods to mitigate these potential biases and enhance the robustness
of the findings. Future research may replicate and compare the findings of this
study in other developing countries. More research is needed to identify the
factors that influence the continuous use of educational technologies, particularly
in developing nations. Researchers may also investigate what aspects influence
parents’ satisfaction with educational technologies. This will increase the usage of
educational technology in the long run, as user satisfaction is the best single
predictor of continuous use.
10. Conclusion
The aim of this study was to investigate the factors that parents in South Africa
consider significant when allowing their children to continue using OL.
Confirmed cost, confirmed teachers’ preparedness, attitude towards, confirmed
usability, and confirmed performance impact were used to extend the ECM in this
study. The results revealed that all the added variables are good antecedents of
the ECM. The ECM factors, plus the additional variables, explained 74.6% of the
variance in the continuous use of OL by parents in South Africa in allowing their
children to continue using OL. The results also revealed that confirmed
performance impact, satisfaction, and confirmed support had direct effects on
parents in South Africa in allowing their children to continue using OL.
Attitude towards, confirmed teachers’ preparedness, confirmed usability and

confirmed support are prognosticators of satisfaction. Confirmed costs and
confirmed support influence attitude towards the use of OL. Confirmed teachers’
preparedness influences their confirmed support. The high explanatory power of
the developed model in this study was evidence that the identified factors are
good factors that parents in South Africa consider when allowing their children
to continue using OL.
602
11. References
Agaton, C. B., & Cueto, L. J. (2021). Learning at Home: Parents’ Lived Experiences on
Distance Learning during COVID-19 Pandemic in the Philippines. International
https://doi.org/10.11591/ijere.v10i3.21136
Alarabiat, A., Hujran, O., Soares, D., & Tarhini, A. (2021). Examining students’ continuous
use of online learning in the post-Covid-19 era: an application of the process
virtualization theory. Information Technology & People.
https://doi.org/10.1108/itp-02-2021-0142
Almahamid, S., & Rub, F. A. (2011). Factors that determine continuance intention to use e-
learning system: an empirical investigation. Paper presented at the International
conference on telecommunication technology and applications Proc. of CSIT.
https://doi.org/10.5220/0005748102150222
Ardington, C., Wills, G., & Kotze, J. (2021). COVID-19 learning losses: Early grade reading
in South Africa. International Journal of Educational Development, 86, 102480.
https://doi.org/10.1016/j.ijedudev.2021.102480
Bhattacherjee, A. (2001). Understanding information systems continuance: an expectation-
confirmation model. MIS quarterly, 351–370. https://doi.org/10.2307/3250921
Chandradasa, A., & Galhena, B. (2021). University students’ intention of continuous use of
Zoom for eLearning. https://doi.org/10.4038/jmm.v9i1.31
Chibisa, A., Tshabalala, M. G., & Maphalala, M. C. (2021). Pre-Service Teachers’ Computer
Self-Efficacy and the Use of Computers. International Journal of Learning, Teaching
and Educational Research, 20(11). https://doi.org/10.26803/ijlter.20.11.18
Creswell, J. W., & Creswell, J. D. (2017). Research design: Qualitative, quantitative, and mixed
methods approaches: Sage publications. https://doi.org/10.7748/nr.12.1.82.s2
DeLone, W. H., & McLean, E. R. (2016). Information systems success measurement.
Foundations and Trends® in Information Systems, 2(1), 1–116.
https://doi.org/10.1561/2900000005
Dong, C., Cao, S., & Li, H. (2020). Young children’s online learning during COVID-19
pandemic: Chinese parents’ beliefs and attitudes. Children and youth services review,
118, 105440. https://doi.org/10.1016/j.childyouth.2020.105440
El Said, G. R. (2015). Understanding Knowledge Management System antecedents of
performance impact: Extending the Task-technology Fit Model with intention to
share knowledge construct. Future Business Journal, 1(1-2), 75–87.
https://doi.org/10.1016/j.fbj.2015.11.003
Eutsler, L. (2018). Parents’ mobile technology adoption influences on elementary
children’s use. The International Journal of Information and Learning Technology.
https://doi.org/10.1108/ijilt-05-2017-0035
Foroughi, B., Iranmanesh, M., & Hyun, S. S. (2019). Understanding the determinants of
mobile banking continuance usage intention. Journal of Enterprise Information
Management. https://doi.org/10.1108/jeim-10-2018-0237
Gill, L., & Dalgarno, B. (2008). Influences on pre-service teachers’ preparedness to use ICTs
in the classroom. Hello! Where are you in the landscape of educational technology.
Goodhue, D. L., & Thompson, R. L. (1995). Task-technology fit and individual
performance. MIS quarterly, 213-236. https://doi.org/10.2307/249689
Hair, J., Hollingsworth, C. L., Randolph, A. B., & Chong, A. Y. L. (2017). An updated and
expanded assessment of PLS-SEM in information systems research. Industrial
management & data systems. https://doi.org/10.1108/imds-04-2016-0130
603
Hair Jr, J. F., Hult, G. T. M., Ringle, C. M., & Sarstedt, M. (2016). A primer on partial least
squares structural equation modeling (PLS-SEM): Sage publications.
https://doi.org/10.54055/ejtr.v6i2.134
Hair Jr, J. F., Hult, G. T. M., Ringle, C. M., & Sarstedt, M. (2021). A primer on partial least
squares structural equation modeling (PLS-SEM): Sage publications.
https://doi.org/10.54055/ejtr.v6i2.134
Hair Jr, J. F., Sarstedt, M., Ringle, C. M., & Gudergan, S. P. (2017). Advanced issues in partial
least squares structural equation modeling. Sage publications.
https://doi.org/10.1007/978-3-319-05542-8_15-1
Hatzigianni, M., & Kalaitzidis, I. (2018). Early childhood educators’ attitudes and beliefs
around the use of touchscreen technologies by children under three years of age.
British Journal of Educational Technology, 49(5), 883–895.
https://doi.org/10.1111/bjet.12649
Heba, C., & Sultan, S. (2020). Parents’ Views of their children’s online learning in the UAE
context during the COVID-19 pandemic.
https://doi.org/10.36348/jaep.2020.v04i10.004
Hero, J. L. (2020). Teachers’ Preparedness and acceptance of information and
communications technology (ICT) integration and its effect on their ICT
integration practices. Puissant, 1, 59-76. https://doi.org/10.3402/rlt.v15i1.10912
Holmes, B. (2013). School teachers’ continuous professional development in an online
learning community: Lessons from a case study of an eTwinning learning event .
European Journal of Education, 48(1), 97-112. https://doi.org/10.1111/ejed.12015
Holmes, B., & Sime, J. (2012). Online learning communities for teachers’ continuous professional
development: case study of an eTwinning learning event. Paper presented at the
Proceedings of the 8th International Conference on Networked Learning.
https://doi.org/10.1007/978-3-319-01940-6_10
Hossain, M. N., Talukder, M. S., Khayer, A., & Bao, Y. (2020). Investigating the factors
driving adult learners’ continuous intention to use M-learning application: a
fuzzy-set analysis. Journal of Research in Innovative Teaching & Learning.
https://doi.org/10.1108/jrit-09-2019-0071
Huang, C.-H. (2021). Exploring the continuous usage intention of online learning
platforms from the perspective of social capital. Information, 12(4), 141.
https://doi.org/10.3390/info12040141
Isikoglu Erdogan, N., Johnson, J. E., Dong, P. I., & Qiu, Z. (2019). Do parents prefer digital
play? Examination of parental preferences and beliefs in four nations. Early
Childhood Education Journal, 47(2), 131-142. https://doi.org/10.1007/s10643-018-
0901-2
Kumar, A., Tewari, A., Shroff, G., Chittamuru, D., Kam, M., & Canny, J. (2010). An
exploratory study of unsupervised mobile learning in rural India. Paper presented at
the Proceedings of the SIGCHI Conference on Human Factors in Computing
Systems. https://doi.org/10.1145/1753326.1753435
Kumpulainen, K., & Gillen, J. (2017). Young children’s digital literacy practices in the home: A
review of the literature. Cost Action ISI1410 DigiLitEY.
https://doi.org/10.4324/9780203730638-7
Landa, N., Zhou, S., & Marongwe, N. (2021). Education in emergencies: Lessons from
COVID-19 in South Africa. International Review of Education, 67(1), 167-183.
https://doi.org/10.1007/s11159-021-09903-z
Lau, E. Y. H., & Lee, K. (2021). Parents’ views on young children’s distance learning and
screen time during COVID-19 class suspension in Hong Kong. Early Education and
Development, 32(6), 863-880. https://doi.org/10.1080/10409289.2020.1843925
604
Lewis, J. R. (2018). Measuring perceived usability: The CSUQ, SUS, and UMUX.
International Journal of Human–Computer Interaction, 34(12), 1148-1156.
Lewis, J. R., Utesch, B. S., & Maher, D. E. (2015). Measuring perceived usability: The SUS,
UMUX-LITE, and AltUsability. International Journal of Human-Computer
Interaction, 31(8), 496-505. https://doi.org/10.1080/10447318.2017.1418805
Lu, Y., Wang, B., & Lu, Y. (2019). Understanding key drivers of MOOC satisfaction and
continuance intention to use. Journal of Electronic Commerce Research, 20(2).
https://doi.org/10.4067/s0718-18762019000300103
Maheshwari, G. (2021). Factors affecting students’ intentions to undertake online learning:
An empirical study in Vietnam. Education and information technologies, 1-21.
https://doi.org/10.1007/s10639-021-10465-8
Maqableh, M., & Alia, M. (2021). Evaluation online learning of undergraduate students
under lockdown amidst COVID-19 Pandemic: The online learning experience and
students’ satisfaction. Children and youth services review, 128, 106160.
Mascheroni, G., Livingstone, S., Dreier, M., & Chaudron, S. (2016). Learning versus play
or learning through play? How parents’imaginaries, discourses and practices
around ICTs shape children’s (digital) literacy practices. Media Education, 7(2),
242-261.
Mulenga, E. M., & Marbán, J. M. (2020). Is COVID-19 the Gateway for Digital Learning in
Mathematics Education? Contemporary Educational Technology, 12(2), ep269.
https://doi.org/10.1177/2042753016677858
Mutambara, D., & Bayaga, A. (2020a). Predicting rural stem teachers’ acceptance of mobile
learning in the fourth industrial revolution. Journal of Construction Project
Management and Innovation, 10(2), 14-29.
https://doi.org/10.36615/jcpmi.v10i2.404
Mutambara, D., & Bayaga, A. (2020b). Understanding rural parents’ behavioral intention
to allow their children to use mobile learning. Responsible Design, Implementation
and Use of Information and Communication Technology, 12066, 520.
https://doi.org/10.1007/978-3-030-44999-5_43
Mutambara, D., & Bayaga, A. (2021a). Determinants of mobile learning acceptance for
STEM education in rural areas. Computers & Education, 160, 104010.
Mutambara, D., & Bayaga, A. (2021b). Learners’ and teachers’ acceptance of mobile
learning: An exploratory study in a developing country. International Journal of
Learning Technology, 16(2), 90-108. https://doi.org/10.1504/ijlt.2021.117763
Mutambara, D., & Chibisa, A. (2022a). Analysing the Determinants of learners’ continuous
use of mobile learning during and after the Covid-19 pandemic. Journal of
Educational Studies, 21(2), 126-146. https://doi.org/10.26803/ijlter.21.3.1
Mutambara, D., & Chibisa, A. (2022b). Rural STEM Preservice Teachers’ Acceptance of
Virtual Learning. International Journal of Learning, Teaching and Educational
Research, 21(2). https://doi.org/10.26803/ijlter.21.2.9
Mutambara, D., & Chibisa, A. (2022c). Rural STEM Preservice Teachers’ Acceptance of
Virtual Learning. International Journal of Learning, Teaching and Educational
Research, 21(2), 155-175. https://doi.org/10.26803/ijlter.21.2.9
Nasiri, F., Ghanbari, S., Ardalan, M., & Karimi, I. (2014). Effect of infrastructure and faculty
readiness in effective implementation of e-learning based on technology
acceptance model (TAM). Education Strategies in Medical Sciences, 7(5), 329-328.
https://doi.org/10.1080/10963758.2023.2188217
605
Nouwen, M., & Zaman, B. (2018). Redefining the role of parents in young children’s online
interactions. A value-sensitive design case study. International Journal of Child-
Computer Interaction, 18, 22-26. https://doi.org/10.1016/j.ijcci.2018.06.001
Nugroho, M. A., Setyorini, D., & Novitasari, B. T. (2019). The role of satisfaction on
perceived value and e-learning usage continuity relationship. Procedia Computer
Science, 161, 82–89. https://doi.org/10.1016/j.procs.2019.11.102
Özdamlı, F., & Yıldız, E. P. (2014). Parents’ views towards improve parent-school
collaboration with mobile technologies. Procedia-Social and Behavioral Sciences, 131,
361–366. https://doi.org/10.1016/j.sbspro.2014.04.130
Poçan, S., Altay, B., & Yaşaroğlu, C. (2021). Parents’ Opinions on the Use of Mobile
Technology in Teaching Mathematics. Inonu University Journal of the Faculty of
Education (INUJFE), 22(1). https://doi.org/10.1007/s10639-022-11166-6
Ramrathan, L. (2021). School curriculum in South Africa in the Covid-19 context: An
opportunity for education for relevance. Prospects, 51(1), 383–392.
https://doi.org/10.1007/s11125-020-09490-1
Stites, M. L., Sonneschein, S., & Galczyk, S. H. (2021). Preschool parents’ views of distance
learning during COVID-19. Early Education and Development, 32(7), 923–939.
https://doi.org/10.1080/10409289.2021.1930936
Suzianti, A., & Paramadini, S. A. (2021). Continuance intention of e-learning: The
condition and its connection with open innovation. Journal of Open Innovation:
Technology, Market, and Complexity, 7(1), 97.
https://doi.org/10.3390/joitmc7010097
Wang, T., Lin, C.-L., & Su, Y.-S. (2021). Continuance intention of university students and
online learning during the COVID-19 pandemic: A modified expectation
confirmation model perspective. Sustainability, 13(8), 4586.
https://doi.org/10.3390/su13084586
Watson, J., Gemin, B., & Coffey, M. (2010). A Parent’s Guide to Choosing the Right Online
Program. Promising Practices in Online Learning. International Association for K-
12 Online Learning. https://doi.org/10.4324/9781315721316-10
Xu, D., & Jaggars, S. S. (2013). The impact of online learning on students’ course outcomes:
Evidence from a large community and technical college system. Economics of
Education Review, 37, 46–57. https://doi.org/10.1016/j.econedurev.2013.08.001
Yu, J., Lee, H., Ha, I., & Zo, H. (2017). User acceptance of media tablets: An empirical
examination of perceived value. Telematics and informatics, 34(4), 206–223.
https://doi.org/10.1016/j.tele.2015.11.004
Zhu, Y., Zhang, J. H., Au, W., & Yates, G. (2020). University students’ online learning
attitudes and continuous intention to undertake online courses: A self-regulated
learning perspective. Educational technology research and development, 68(3), 1485–
1519. https://doi.org/10.1007/s11423-020-09753-w
PUBLISHER International Journal of Learning, Teaching and
Society for Research and Knowledge Management Educational Research
Port Louis
Republic of Mauritius
www.ijlter.org The International Journal of Learning, Teaching
and Educational Research is an open-access
Chief Editor journal which has been established for the dis-
Dr. Antonio Silva Sprock, Universidad Central de semination of state-of-the-art knowledge in the
Venezuela, Venezuela, Bolivarian Republic of field of education, learning and teaching. IJLTER
welcomes research articles from academics, ed-
Editorial Board
ucators, teachers, trainers and other practition-
Dr. Fatima Zohra Belkhir-Benmostefa
ers on all aspects of education to publish high
Dr. Giorgio Poletti
quality peer-reviewed papers. Papers for publi-
Dr. Chi Man Tsui
cation in the International Journal of Learning,
Dr. Fitri Suraya Mohamad
Teaching and Educational Research are selected
Dr. Hernando Lintag Berna
through precise peer-review to ensure quality,
Dr. Charanjit Kaur Swaran Singh
originality, appropriateness, significance and
Dr. Abu Bakar
readability. Authors are solicited to contribute
Dr. Eglantina Hysa
to this journal by submitting articles that illus-
Dr. Mo'en Salman Alnasraween
trate research results, projects, original surveys
Dr. Hermayawati Hermayawati
and case studies that describe significant ad-
Dr. Selma Kara
vances in the fields of education, training, e-
Dr. Michael B. Cahapay
learning, etc. Authors are invited to submit pa-
Dr. Bunmi Isaiah Omodan
pers to this journal through the ONLINE submis-
Dr. Vassiliki Pliogou
sion system. Submissions must be original and
Dr. Meera Subramanian
should not have been published previously or
Dr. Muhammad Kristiawan
be under consideration for publication while
Dr. Wahyu Widada
being evaluated by IJLTER.
Dr. Som Pal Baliyan
Dr. Reem Khalid Abu-Shawish
Dr. Froilan Delute Mobo
Dr. Mohamed Ali Elkot
Dr. Anabelie Villa Valdez
Mr. Teody Lester Verdeflor Panela

Vol 22 No 7 July 2023

Uploaded by

Copyright:

Available Formats

Vol 22 No 7 July 2023

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Vol 22 No 7 July 2023

Uploaded by

Copyright:

Available Formats

IJLTER.

Society for Research and Knowledge Management

The International Journal of Learning, Teaching and Educational

Aims and Objectives

The main objective of this journal is to provide a platform for educators,

Indexing and Abstracting

The International Journal of Learning, Teaching and Educational

We are very happy to publish this issue of the International Journal of

The International Journal of Learning, Teaching and Educational

We are grateful to the editor-in-chief, members of the Editorial Board

The Editorial Board is composed of renowned people from across the

We will endeavour to ensure the reputation and quality of this journal

Editors of the July 2023 Issue

Performance Analysis Towards Excellent Schools ........................................................................................................... 70

Experiences of Student-Teachers: Implications for Refined Student-Support .............................................................. 87

Self-Instructional Teaching Internship Module: An Evaluation ................................................................................... 193

GO-DEEP: A Potential Reflection Model for Experiential Learning ............................................................................ 240

Effectiveness of a Mixed Methods-Based Literacy Program in Improving Reading Comprehension, Vocabulary

A Conceptual Analysis of What it Means to Decolonize the Curriculum ................................................................... 344

Mentors’ and Mentees’ Experiences in a Sino-Foreign Cooperative Education Programme: A Phenomenological

International Journal of Learning, Teaching and Educational Research

Assessment Literacy, Current Assessment

Abstract. The aim of this study is to explore assessment literacy, to

knowledge about assessment practices that involve evaluating student

Keywords: Assessment literacy; Current assessment practices;

The constructivist approach to assessments emphasizes that ‘learners are the

Underachievement in assessments could be a significant factor in inducing

Assessment literacy can be understood as teachers’ abilities to comprehend the

The essential knowledge and performance components are presented in Figure 1

Figure 1: Essentials of assessment literacy

The emphasis of the educational evaluation system relies on the cognitive

Alternative assessment is characterized as an alternative to standardized, norm-

assessment focuses on helping learners to think critically, solve real-life problems,

The traditional practices, based on teachers’ instinctive judgment or preferences,

In the context of a multidisciplinary university within the higher education

Operationally, assessment literacy in the present research was based on the

Thus, the present study aims:

(A) to gauge assessment literacy among teachers of higher education;

(B) to identify the current assessment practices in higher education; and

communicating the learning objectives to learners. Researchers have also found

In the Indian context, researchers have mainly focused their attention on

Table 1: Survey sample details

There were 58 participants from 16 disciplines, as presented in Figure 2.

Figure 2: No. of participants according to participants’ disciplinary affiliations

3.1 Tool for data collection

Tool development: The tool development process included:-

1. Group discussion with a panel of five experts, consisting of faculty members

5. Expert validation after refinement.

The questionnaire was designed for measuring HE teachers' assessment literacy.

Thus, the questionnaire (Appendix I) included a set of 15 questions - both closed

Table 2: Nature of questionnaire

3.2 Research Process

Figure 3: The research process

4. Data Analysis and Results

Section A – (For objective 1: Assessment literacy among teachers of higher

Section B – (For objective 2: To identify the current assessment practices in higher

Section C- (For objective 3: To determine the need for alternative assessment

The major findings are:

5. The top three assessments preferred by learners were Case