Computers and Education: Artificial Intelligence 2 (2021) 100041

Contents lists available at ScienceDirect

Computers and Education: Artificial Intelligence

journal homepage: www.sciencedirect.com/journal/computers-and-education-artificial-intelligence

Conceptualizing AI literacy: An exploratory review

Davy Tsz Kit Ng a, *, Jac Ka Lok Leung b, Samuel Kai Wah Chu a, Maggie Shen Qiao a
a
Faculty of Education, The University of Hong Kong, Pokfulam, Hong Kong
b
Center for Education Innovation, Hong Kong University of Science and Technology, Hong Kong

A R T I C L E I N F O A B S T R A C T

Keywords: Artificial Intelligence (AI) has spread across industries (e.g., business, science, art, education) to enhance user
AI literacy experience, improve work efficiency, and create many future job opportunities. However, public understanding
AI learning and teaching of AI technologies and how to define AI literacy is under-explored. This vision poses upcoming challenges for our
AI in education
next generation to learn about AI. On this note, an exploratory review was conducted to conceptualize the newly
AI ethics
AI literacy questionnaire
emerging concept “AI literacy”, in search for a sound theoretical foundation to define, teach and evaluate AI
literacy. Grounded in literature on 30 existing peer-reviewed articles, this review proposed four aspects (i.e.,
know and understand, use and apply, evaluate and create, and ethical issues) for fostering AI literacy based on
the adaptation of classic literacies. This study sheds light on the consolidated definition, teaching, and ethical
concerns on AI literacy, establishing the groundwork for future research such as competency development and
assessment criteria on AI literacy.

1. Introduction cause declines in some occupations (Davenport & Ronanki, 2018;

Manyika et al., 2017).
Artificial intelligence (AI) was first defined as “the science and en­ Second, studies reflect that the rise of AI will create many job op­
gineering of making intelligent machines” in 1956 (McCarthy, 2007, p. portunities in various industries, and AI will probably replace tomor­
2). Throughout several decades of the 20th century, AI has evolved row’s workplace. Even though not all disciplines are not going to be
progressively into intelligent machines and algorithms that can reason replaced by AI, people with AI knowledge will replace those that do not
and adapt based on sets of rules and environment which mimic human in the future of work. In a MicKinsey report, Manyika et al. (2017)
intelligence (McCarthy, 2007). Wang (2019) broadened the definition of estimated that 15% of the global working hours will be automated and
AI which can perform cognitive tasks particularly learning and 47% of American jobs are at high risk of automation by 2030.
problem-solving with the exciting technological innovations such as Furthermore, the situation could be worse among women since over 160
machine learning, natural language processing and neural networks million women worldwide may need to transition between occupations
(Zawacki-Richter, Marín, Bond, & Gouverneur, 2019). often into higher-skilled roles. Among different natures of work, clerical
Artificial intelligence will eventually affect many facets of human life work such as secretaries and bookkeepers will be mostly easily elimi­
rather than merely computer industries and everyone should learn AI. nated by AI, given that 72% of those jobs in advanced economies are
Currently, the use of AI has spread across industries (e.g., business, held by women (Manyika et al., 2017). As such, to gain a competitive
science, art, education) to enhance user experience and improve effi­ advantage at work, similar to classic literacy which includes reading/­
ciency. Applications of AI exist in many parts of our everyday life (e.g., writing and mathematical abilities, AI literacy has emerged as a new
smart home appliances, smartphones, Google, Siri). Vast majority of the skill set that everyone should learn in response to this new era of
public acknowledges the existence of AI services and devices, but seldom intelligence.
do they know about the concepts and technology behind, or aware of Literacy was popularly understood as an ability to read and write
potential ethical issues related to AI (Burgsteiner, Kandlhofer, & Stein­ (McBride, 2015). In today’s digital era, the emergence of the knowledge-
bauer, 2016; Ghallab, 2019). Although AI will generate significant based society implies that every citizen must be ‘digitally literate’ and
benefits for users, businesses and economies, and lift productivity and possess basic competencies in order to be on a better footing in terms of
economic growth, AI is poised to eliminate millions of current jobs and equal opportunities in their workplaces (Bawden, 2008, p. 102). This

* Corresponding author.
E-mail addresses: davyngtk@connect.hku.hk (D.T.K. Ng), jac.leung@ust.hk (J.K.L. Leung), samchu@hku.hk (S.K.W. Chu), sqiao@connect.hku.hk (M.S. Qiao).

https://doi.org/10.1016/j.caeai.2021.100041
Received 12 July 2021; Received in revised form 12 November 2021; Accepted 12 November 2021
Available online 22 November 2021
2666-920X/© 2021 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license
(http://creativecommons.org/licenses/by-nc-nd/4.0/).
D.T.K. Ng et al. Computers and Education: Artificial Intelligence 2 (2021) 100041

term has been extended to new literacies such as media, digital, infor­ began to define AI literacy based on the term ‘literacy’ which has been
mation, computer and AI literacy (Kong et al., 2021). In the twenty-first applied to define skill sets in varied disciplines (Long & Magerko, 2020).
century, students who are equipped with these skills could use related However, few studies have provided comprehensive explanations on
technologies and computers in very advanced ways to learn new how to conceptualize AI literacy. To achieve a better understanding of
knowledge and skills with their counterparts (Bell, 2010; Griffin & Care, the concept of AI literacy, we categorize how researchers define the term
2014; Larson & Miller, 2011). Nowadays, AI technology emerges and in four aspects, inspired from the cognitive domains in Bloom’s taxon­
becomes essential skills to play critical roles across disciplines and in­ omy. Then, we evaluate how educators help learners develop AI literacy
dustries (Ng et al., 2021; Touretzky et al., 2019). Students need to learn skills with emerging technological tools, and evaluate their assessment
how to use AI technologies judiciously, as well as to discriminate be­ accordingly. To fill this gap, this study reviewed the relevant literature,
tween ethical and unethical practices (Robinson, 2020; Rodrí­ and analysed how scholars define “AI literacy”, how it can be learned,
guez-García, Moreno-León, Román-González, & Robles, 2020). AI and what are the ethical concerns. Specifically, the present study poses
potentially becomes one of the important technology skills in the the following four research questions:
twenty-first century. As such, to combine AI and literacy, AI literacy
means having the essential abilities that people need to live, learn and 1. How do researchers define the term “AI literacy”?
work in our digital world through AI-driven technologies, and this 2. How do educators help learners develop AI literacy in terms of
should be taught at the K-12 levels (Steinbauer et al., 2021). learning artefacts, pedagogical approaches and subject matters?
AI learning started in university computer science education which 3. How do researchers evaluate students’ AI literacy skills?
required advanced programming competencies that were not at an 4. What are the ethical concerns in the domain of AI literacy?
appropriate level for K-12 learners. Educators faced challenges in scaf­
folding K-12 children to understand AI concepts through syntax-based 2. Method
programming (e.g., McCarthy, 2007; Wong et al., 2020). The emer­
gence of more age-appropriate hardwares and softwares enabled edu­ 2.1. The search and manuscript selection process
cators to improve the learning process for younger learners in recent
years. The access to a wide range of technologies in day-to-day life, such As AI literacy is an emerging field in the twenty-first century, hence
as chatbots and translation apps, presents opportunities for everyone to available literature is limited. In search for literature on AI literacy, both
understand and use AI in everyday life. This enables educators to peer-reviewed scholarly articles and conference papers from K-12 to
leverage on the availability of AI technologies to inculcate AI literacy for higher education levels published from 2016 to 2021 through the Web of
young learners. For example, prior studies discussed the potential to Science, Scopus, ProQuest Education Collection, IEEE and ACM digital
incorporate AI learning in K-12 STEAM education via playful experience library were included in this review. The first publication year found in
such as gamified and social media tools to prepare children for future the databases was 2016. The aforementioned databases were considered
science, technology, engineering, art and mathematics workforces (e.g., among the world’s most trusted citation indices platforms for evidence-
Ng, 2021; Ng & Chu, 2021; Zou, Wang, & Zhao, 2019). based quality scientific research and hence helped us to ensure the in­
Knowing and using AI for future careers is only one aspect of clusion of quality scientific content (Mongeon & Paul-Hus, 2016). The
teaching AI literacy for educators. Any technology as potent as AI would articles that contained the phrase “AI literacy” OR “Artificial intelli­
also bring new risks due to algorithmic bias and malicious uses of AI gence literacy” in either the title, the abstract, main text or keywords
(Brundage et al., 2018). People often overlook the importance of the were downloaded and reviewed by the researchers. The search resulted
roles of AI ethics, which is considered as extraneous or surplus to in 46 articles.
technical concerns in work settings (Hagendorff, 2020). Software de­ After excluding irrelevant studies, as of Apr 11, 2021, a total of 30
velopers usually feel a lack of accountability and moral significance of articles were identified. The articles were downloaded and reviewed by
their work, especially when economic incentives are easily overriding our researchers during the document review. The selected articles then
commitment to ethical principles and values (Hagendorff, 2020). As were examined by two researchers to determine whether they were
such, educating both citizens and computer scientists AI ethics is suitable for the purpose of this study. During this examination, a set of
essential to strengthen their social responsibility, and consider social inclusion and exclusion criteria were adopted to ensure generalisation of
inclusion and diversity to apply AI for societal good (Dignum, 2019). In the findings and avoid biases in the studies selection (see Table 1). For
this review, we examine the published studies to evaluate the ethical example, Sharma (2019) focused on the impact made by AI in entre­
concerns in the domain of AI literacy. preneurial activities including encouraging social innovation,
According to Google Scholar search, there is a dramatic increase in improving the institutional environment and gaining support from in­
AI literacy publications from 2014 to 2021 (see Fig. 1). As AI becomes ternational organizations, instead of integrating AI in educational
more and more important in work settings and everyday life, researchers settings.

Table 1
Inclusion on and exclusion criteria.
Inclusion criteria Exclusion criteria

(1) the studies had to review articles, (1) Editorials and books are excluded
empirical papers, articles, case studies due to the lack of peer review.
or conference proceedings published (2) Articles that mention the term “AI
in the journals indexed by the literacy” are actually about how
aforementioned databases. AI is applying in particular fields
(2) the studies had to be in the field of and unrelated to education.
education which was related to AI
literacy.
(3) the studies should provide
descriptions of the underlying theory
and methods.
Fig. 1. AI literacy articles from google scholar published by year.

2
D.T.K. Ng et al. Computers and Education: Artificial Intelligence 2 (2021) 100041

2.2. The data coding and analysis processes abovementioned databases were limited, it is observed that the
increasing trend of publications is consistent with Google Scholar’s
This study began with formulating the study objectives, followed by trend. In addition, we listed the nationality information of the first
a review and analysis of AI literacy research trends according to the four author in the AI literacy paper and observed that many countries have
research questions. Then, the full text of the chosen articles was quali­ begun conceptualizing AI literacy. The regions that published two or
tatively classified using the constant comparative method espoused by more AI literacy articles include: the United States (9), China (4), Hong
Glaser (1965), which was used in other recent systematic reviews (e.g., Kong (4), Spain (3) and Austria (3).
Hew & Cheung, 2014; Terras & Warwick, 2013). Through studying the Researchers conducted studies and implemented AI literacy in­
main content in the selected articles, similar meaningful concepts were terventions across various educational levels. Most of the articles
identified and extracted for further thematic analysis. Corresponding focused on primary school (14) and secondary school (14) students that
text segments were coded under the coding schemes in each research covered almost half of the reviewed studies. Only a few studies were
question. To establish coding reliability, six (30%) of the articles were implemented for citizens (4), university students (4) and teachers (2).
randomly picked, blind-coded and analysed by the two researchers. Two Finally, some articles studied AI literacy in less conventional settings in
experienced researchers then read and categorized the papers based on AI to bring up students for their future work, including libraries (1),
the coding scheme. Disagreements were resolved through discussing the medicine (1) and meteorology (1). About one-third of the studies (9)
disputed studies. Cohen’s kappa coefficient (0.9) was found to be were conducted in an informal setting, which included after-school
excellent to show inter-rater reliability between coders (Miles & programs, out-of-school activities and poster presentations. Seven
Huberman, 1994). After validating the coding scheme, the data findings studies were conducted in regular lessons in a formal setting. The
were then descriptively analysed and summarised in terms of frequency, remaining papers did not specify whether the settings are formal or
percentages and identified themes. In the case of discrepancy, the coders informal. One possible reason is that AI literacy is an emerging field, and
resolved this and reached a final decision through discussion. most researchers tend to conduct preliminary studies to explore their
interventions in an informal setting or merely write opinion papers
3. Results and discussion based on their observation.
Overall, there are 1 review paper, 4 conceptual articles and 25
In this section, background information (i.e., publication year, empirical studies. Regarding the research method, most of the empirical
country, levels of education and research method) of the 30 selected studies adopted qualitative (12) methods (see Table 3). Researchers used
studies is first described (see Table 2). Then, we present the results and quantitative methods (5) to assess students’ AI concepts, perceived
discuss these results according to the four research questions. The abilities and other constructs such as confidence in using AI and social
publications of AI literacy papers increase from 2016 (2 articles) to 2019 skills. Seven studies adopted a mixed-method approach (8) to collect
(8 articles). Nineteen published articles were found between 2020 and data via multiple data sources including ability tests, questionnaire
April 2021. Although the number of publications found in the surveys, field notes, interviews and observations. We found one review
article (i.e., Long & Magerko, 2020) in which they searched broader
terms such as “AI education”, “learning about AI” and “AI school” to
Table 2
map the key concepts underpinning AI literacy on their AI4K12 mailing
Frequency (N, %) of the characteristics of the reviewed articles.
list and selected papers. Since AI literacy articles emerged these few
Variables Categories N Percent years, this review discusses how researchers use the specific term “AI
Year 2016 2 6.7% literacy” instead of teaching and learning AI.
2018 1 3.3%
2019 8 26.7%
2020 17 56.7% 3.1. RQ 1. how do researchers define the term “AI literacy”?
2021 2 6.7%
Countries Austria 3 10.0%
Belgium 1 3.3%
Of the 30 articles, 17 articles defined AI literacy based on the ideas of
China 4 13.3% ‘literacy’. Prior to AI literacy, the term “digital literacy” emerged to
Denmark 1 3.3% assess basic computer-related concepts and skills when computer ap­
Hong Kong 4 13.3% plications gained popularity across industries in the 1970s. It was
India 1 3.3%
necessary for users to become competent in using computer systems
Singapore 1 3.3%
Spain 3 10.0% related to their specific task or job. The importance of digital literacy
Sweden 1 3.3% increased as more people depend on the use of computer technologies to
Turkey 1 3.3% develop new social and economic opportunities (Leahy & Dolan, 2010).
USA 9 10.0%
UK 1 3.3%
Publication type Research paper 25 80.0% Table 3
Conceptual paper 4 13.3% Research methods.
Review paper 1 3.3%
Research N Studies
Educational level K-elementary 14 46.7%
methods
Secondary school 14 46.7%
Higher education 4 13.3% Qualitative 12 Burgsteiner et al. (2016); Han et al. (2018); How and Hung
Citizen 4 13.3% (2019); Kaspersen et al. (2021); Leander and Burriss
Teacher 2 6.7% (2020); Long et al. (2019); Rivero (2020); Robinson
Learning artefacts AI-related agents 11 36.7% (2020); Rodríguez-García et al. (2020); Schaper et al.
Hardware-focused artefacts 8 26.7% (2020); Vazhayil, Shetty, Bhavani, & Akshay (2019);
Software-focused artefacts 6 20.0% Watkins (2020).
Unplugged artefacts 7 23.3% Quantitative 5 Chai, Wang, and Xu (2020); Chai, Lin, et al. (2020); Dai
Educational setting Formal 7 23.3% et al. (2020); Gong et al. (2020); Karaca (2020).
Informal 9 30.0% Mixed method 8 Druga et al. (2019); Julie et al. (2020); Kandlhofer et al.
Not specified 14 46.6% (2016); Lin et al. (2021); Wan et al. (2020); Williams, Park,
Research methods Qualitative 12 40.0% & Breazeal (2019); Register & Ko (2020);
Quantitative 5 16.7% Rodríguez-García et al. (2020).
Mixed research 8 26.7% Review articles 5 Long and Magerko (2020); Pegrum et al. (2018); Wong
Review articles/conceptual papers 5 16.7% et al. (2020); Xu (2020); Zou et al. (2019).

3
D.T.K. Ng et al. Computers and Education: Artificial Intelligence 2 (2021) 100041

In succession to digital advancement, AI started to arise and imitate technological knowledge which then was applied in scientific
human intelligence in machines for computers to learn, reason and research-based learning to solve practical problems. Long et al. (2019)
perceive. It was initially used in scientific research and academic envi­ engaged citizens in co-creating AI amenities in public spaces to broaden
ronments but had yet become ubiquitous in our daily lives. In summary, their public AI literacy and experiences. Participants could engage with
four aspects of fostering AI literacy were identified from the review (see public interactive artworks progress sequentially from being initially
Table 4). attracted to an AI-enabled installation to relate their interaction with the
installation and other people.
3.2. Know and understand AI Overall, although these articles showed slight variations on the
definition of AI literacy, they support the notion that everyone, espe­
Twenty-seven articles conceptualize AI literacy as educating learners cially K-12 children, acquire basic AI knowledge and abilities, enhance
about acquiring fundamental concepts, skills, knowledge and attitudes motivation for future career, as well as use AI-enabled technology (Chai,
that require no prior knowledge. On top of being the end users of AI Lin, et al., 2020). In addition to knowing and using AI ethically, AI lit­
applications, learners should understand the technologies behind. eracy serves as a set of competencies that enables individuals to criti­
Burgsteiner et al. (2016) and Kandlhofer et al. (2016) defined AI literacy cally evaluate AI technologies, communicate and collaborate effectively
as the ability to understand the basic techniques and concepts behind AI with AI (Long & Magerko, 2020).
in different products and services. Moreover, some researchers associate
AI literacy with perceived abilities, confidence and readiness in learning 3.5. Bloom’s taxonomy
AI. In K-12 education, Druga et al. (2019) and Lin et al. (2021) designed
learning curriculums and activities that foster AI literacy that focuses on Specifically, a definition for AI literacy learning is presented in the
how learners gain AI concepts. aforementioned three aspects. In fact, the abilities and skills involved in
each aspect could be potentially mapped to the cognitive domains in
3.3. Use and apply AI Bloom’s Taxonomy. Bloom’s Taxonomy is an approach to categorize the
levels of reasoning skills and ordered thinking required across different
All 30 articles emphasized the importance of educating learners to learning contexts. There are six levels in the taxonomy, each requiring a
know how to apply AI concepts in different contexts and applications in higher level of complexity and ordered thinking from the students. The
everyday life. For example, Rodríguez-García et al. (2020) evaluated levels are understood to be successive, so that one level must be
LearningML, a machine learning model builder, to educate citizens to mastered before the next level can be reached (Bloom, 1956; Huitt,
understand AI applications and how it can affect our lives, as well as 2011). The reason why we adopted the Bloom architecture is that AI
knowing the ethical issues regarding AI technologies. In addition, half of literacy is novice to educators and a classification of levels of cognitive
the studies (19) discussed the human-centered and ethical consider­ processes has not yet been developed in the context of AI learning.
ations and focused on using AI concepts and application ethically, which However, this model is a classic pedagogical theory that establishes the
would be further discussed in RQ4. Eight articles borrowed the ideas of core foundation of AI taught to young learners. In our review, it is
computational thinking to interplay AI literacy and AI thinking (see proposed to assign these three aspects (i.e., know and understand, use,
Table 5). AI thinking refers to the construction of logic and algorithms in and evaluate and create AI) into the cognitive levels of Bloom’s Tax­
order to support students’ understanding of how to use knowledge bases onomy. “Know and understand AI” is assigned to the bottom two levels;
for problem-solving, processing semantics and handling unstructured “use and apply AI” in applying concepts and applications is assigned to
data (Vazhayil et al., 2019). For example, How and Hung (2019) the apply level; “evaluate and create AI” are assigned to the top three
leveraged AI thinking through conducting data analytics with levels to analyse, evaluate and create AI (see Fig. 2).
computing, and interpreted new findings from the machine-learned In our review, most studies discussed how to foster learners’ AI lit­
discovery of hidden patterns in data. eracy in knowing and understanding AI (27), as well as how to use AI
applications in everyday life and apply its underlying concepts in
3.4. Evaluate and create AI different contexts (30). Only 19 articles (63.3%) mentioned how to
enhance students to analyse, evaluate and create AI applications
AI augments human intelligence with digital automation and 19 through higher-order thinking activities. A possible reason that existing
articles alluded AI literacy to engage learners in higher-order thinking AI literacy studies focused more on general skills and knowledge about
activities. Other than knowing and using AI with concepts and practices, AI is that AI literacy is a set of fundamental skills and abilities in helping
some studies had extended AI literacy to two other competencies that everyone, including children and citizens, to acquire, construct and
enabled individuals to critically evaluate AI technologies, communicate apply knowledge. They may not necessarily handle how to abstract and
and collaborate effectively with AI (e.g., Long & Magerko, 2020). For decompose AI problems, nor build AI applications; instead, they need to
example, Han et al. (2018) enhanced students’ scientific and know the basic concepts and use AI ethically. As such, most of our

Table 4
Coding framework of AI literacy.
AI literacy Definitions N Sample references Sample studies

Know & Know the basic functions of AI and how 27 Even though transparency in algorithms and AI in general has been Lin et al. (2021)Lin et al. (2021);
understand to use AI applications. acknowledged to be ethically important, the public lacks understanding of Kandlhofer et al., 2016);
AI even the basic functions of AI. Efforts to make AI more comprehensible exist Robinson (2020).
(Robinson, 2020).
Use & Apply AI Applying AI knowledge, concepts and 30 Apply k-means clustering in science contexts.. explore the mapping Druga et al. (2019); Julie et al.
applications in different scenarios. relationship between facial features and data values and apply the concept to (2020); Vazhayil et al. (2019).
brainstorm other objects such as Lego (Wan et al., 2020).
Evaluate & Higher-order thinking skills (e.g., 19 Design & build experiences: Technology exploration and creation activities Druga et al. (2019); Han et al.
create AI evaluate, appraise, predict, design) with supported students in making sense of the underlying AI concepts. (Lee, (2018); How and Hung (2019).
AI applications. 2020).
AI ethics Human-centered considerations (e.g., 19 “AI for social good” measures an individual’s perception of the social Chai et al. (2020); Druga et al.
fairness, accountability, transparency, environment surrounding the behavior, which is related to subjective (2019);
ethics, safety). norms (Chai et al., 2020). Gong et al. (2020).

4
D.T.K. Ng et al. Computers and Education: Artificial Intelligence 2 (2021) 100041

Table 5
Interplay between AI and Brennan-Resnick’s (2012) computational thinking.
Elements Descriptions Examples

AI concepts Technical and conceptual understanding of the basic AI ● Understand the basic AI concepts and their origins such as machine learning, deep learning and
concepts. neural network.
AI practices The techniques and strategies used when applying AI. ● Appreciate the real-world applications of AI concepts such as speech recognition, robotics.
● Training, validation and testing.
● Remixing or reusing code.
AI Attitudes and dispositions adopted while solving ● Collaborating to solve problems, understanding of technology as a problem-solving tool.
perspectives problems. ● Consider the ethical and safety concerns when applying AI technologies in real-world applications.

Fig. 2. Bloom’s Taxonomy and AI literacy.

Fig. 3. AI literacy TPACK framework.

5
D.T.K. Ng et al. Computers and Education: Artificial Intelligence 2 (2021) 100041

selected AI literacy studies put more emphasis on engaging learners in Table 6

lower-level thinking activities. However, when students are promoted to Learning artefacts.
secondary schools and universities, they become knowledgeable to Definition Learning artefacts Sample studies
apply their prior knowledge to create their own artefacts and justify examples
decisions with AI applications and algorithms. Hardware- Use physical Bee-bots, LEGO Kandlhofer et al.
focused artefacts to learn AI Mindstorms NXT, (2016)
3.6. RQ 2. how do educators help learners develop AI literacy in terms of artefacts such as robotics, Cubelets, alpha dog Chai, Wang, and Xu
learning artefacts, pedagogical approaches and subject matters? sensors and robot, Kinect (2020)
Arduino devices. LuminAI, VR Robot Long et al. (2019)
Improv Circus, Druga et al. (2019)
This review aims to fill recognized gaps in knowing the effective Sound Happening, Burgsteiner et al.
means to integrate AI literacy into school curricula and how educators Shape of Story (2016)
help learners develop AI literacy. The elements found in our studies into AI home assistants:
Jibo robot, Anki’s
the Technological, Pedagogical and Content Knowledge (TPACK)
Cozmo robot and
framework are categorized in terms of learning artefacts, pedagogical Amazon’s Alexa
approaches, and subject matters (see Fig. 3). The reason that we adopt Lego Mindstorms
the TPACK model is that it is widely used across studies to identify how NXT
teachers can incorporate technologies into their pedagogical methods Software- Use digital Google maps, Kandlhofer et al.
focused artefacts to learn AI Golog, YAGI, (2016)Wan, Zhou,
and content knowledge, and conceptualizes their capacity and knowl­ artefacts such as block/ ASRAEL Ye, Mortensen, & Bai
edge that is needed to integrate relevant technologies in AI literacy syntax-based SmileyCluster, A* (2020)
education (e.g., Graham, 2011; Koehler et al., 2013). It provides a map programming and algorithm in C#
for understanding how to integrate AI literacy into classrooms effec­ simulation. Burgsteiner et al.
(2016)
tively. For example, Kim et al. (2021, pp. 1–13) based on AI learning
AI-related Use intelligent Scratch, Google’s Lin et al. (2021)
resources to conceptualize TPACK to improve teaching for K-12 AI ed­ agents agents such as Teachable Machine, Vazhayil et al. (2019)
ucation, which offers core foundations of AI taught to young learners. expert systems, Generative How and Hung
Among the three knowledge, technological knowledge involves the machine learning Adversarial (2019)
affordances and use of domain-specific learning tools such as hardware trainers, chatbots Networks (GANS), Druga et al. (2019)
to build their Watson AI services,
and software in AI literacy education, AI-enabled tools (e.g., intelligent custom machine Bayesialab, AI home
agents), and unplugged learning tools (e.g., role-playing). Second, learning models assistants: Jibo
pedagogical knowledge relates to teaching methods and their applica­ without coding. robot, Anki’s Cozmo
tion to promote student AI literacy learning, which entails teaching robot and Amazon’s
Alexa
strategies and scaffolding, feedbacking students’ learning processes
Unplugged Use learning Lectures, career Lin et al. (2021)
(Janssen et al., 2019). Third, content knowledge concerns knowledge activities to learn talk, textbook, case Dai et al. (2020)
about the AI literacy subject matter that specific subjects should be AI without a study, webinar, Schaper et al. (2020)
covered in the curriculum. computer such as role-playing, Rodríguez-García
Learning artefacts: Given the complexity of AI, age-appropriate lecture, case study, storytelling et al. (2020)
role-playing and Julie et al. (2020)
learning artefacts were important to scaffold students’ AI conceptual storytelling.
understandings and stimulate their motivation and interest in learning
AI. In recent years, there has been an increase in hardware and software
that enhance AI concepts accessible to younger learners. Table 6 pro­ example, researchers introduced children to AI concepts in playful and
vides an overview of the types of AI learning artefacts ranging from inquiry approaches via high-order thinking activities such as creating
hardware (8) to software-focused artefacts (6), intelligent agents (11) digital stories (Kandlhofer et al., 2016), performing Turing Test with
and unplugged learning tools (5). The democratization of current AI intelligent agents, creating chatbot and inference algorithms (Wong
technologies encourages students to make intelligent agents and ma­ et al., 2020), and building applications through blockly-based pro­
chine learning models without needing to program such as ML-for-kids gramming (Gong et al., 2020). In addition to understanding the
and Teachable Machine (Kaspersen et al., 2021; Long & Magerko, 2020). connection between those AI techniques and common AI applications,
In this context, we can see an opportunity for educators to democratize secondary school students should have the abilities to apply prior AI
access to AI literacy and reinforce the AI concepts through these knowledge in practical group projects to analyse and solve problems
emerging tools. In addition, AI-driven tools such as chatbot, writing independently (Kandlhofer et al., 2016). Thus, educators could design
assistants and web mapping encourage students to experience the soci­ real-world, collaborative projects based on the principles of con­
etal impact and technological affordances of AI applications. Alterna­ structionism and instructionism (Kandlhofer et al., 2016). Researchers
tively, five studies designed unplugged learning activities to foster suggest various hands-on activities such as robot constructions (Wil­
students’ AI literacy without using a computer through engaging ap­ liams et al., 2019), data and comparative visualization (Wan et al.,
proaches such as case study, role-playing and storytelling (e.g., Julie 2020), as well as training AI models (Vazhayli et al., 2019) as possible
et al., 2020; Rodríguez-García et al., 2020). On the whole, most re­ means to promote AI literacy in secondary school levels.
searchers restricted the development of AI literacy skills within Com­ Adult learners are categorized as university students and the general
puter Science-related learning artefacts, while some researchers public. Since university students have obtained fundamental AI under­
extended AI literacy skills to non-CS elements such as role-playing and standing, they are more ready for further developments in this field.
storytelling. They could conduct projects or research to describe problems formally
and on a higher abstraction level (Kandlhofer et al., 2016). As such, they
3.7. Pedagogical approach could apply AI skills and knowledge to solve real-world problems for
future academic and career challenges (Chat et al., 2020).
The pedagogies including teaching methods and strategies are clas­ To cultivate the general public to understand and use AI applications
sified according to the levels of education. One of the aims of AI literacy ethically, free online resources and courses (Robinson; 2020), public art
education for primary schools is to familiarize children with the basic installations and museum exhibits (Rodríguez-García et al., 2020) are
concepts of AI/computer science and encourage them to discover the viable approaches to establish a collaborative, creative, robust and safe
connection between AI applications and the underlying concepts. For society.

6
D.T.K. Ng et al. Computers and Education: Artificial Intelligence 2 (2021) 100041

3.8. Content knowledge teaching practice and management, and promote personalized learning
to understand students’ learning progress and needs (Xu, 2020). Xu
In K-12 education, studies that involved the design of learning (2020) proposed the importance of learning AI for educators that
curricula and activities focus on how learners gain AI concepts, and how “teachers who know how to use AI may replace the teachers who do not
they apply AI to contexts of their interests (e.g., Druga et al., 2019; Lin know how, because AI can empower teachers and promote their role
et al., 2021). Long and Magerko (2020) and Rodríguez-García et al. transformation which greatly improve the efficiency of management
(2020) mentioned Touretzky et al. (2019)’s five “big ideas” of AI have and the level of decision-making” (p.290). In addition, teachers should
set a sound framework for future research on fostering AI literacy: enable students to use AI-enhanced learning tools such as intelligent
tutors and adaptive learning systems to facilitate personalized learning
● Perceptions: Computers perceive the world using sensors. (e.g., self-diagnosing, providing automatic feedback and promoting
● Representation and reasoning: Agents maintain representation of the online collaboration among learners) (Cavalcanti et al., 2021).
world and use them for reasoning.
● Learning: Computers can learn from data.
● Natural interaction: Intelligent agents require many kinds of 3.10. RQ 3. how do researchers evaluate students’ AI literacy skills?
knowledge to interact naturally with humans.
● Societal impact: AI can impact society in both positive and negative Among 30 studies, researchers adopted quantitative (13) and quali­
ways. tative (18) evaluation methods to examine how to assess students’
mastery and application of AI literacy-related skills (see Table 7), pro­
Inspired by this framework, Wong et al. (2020) further categorized vided that we double-coded “mixed-method research” into quantitative
AI literacy in K-12 into three dimensions: AI concepts, applications and and qualitative evaluation methods. In addition, Robinson (2020) is not
ethics. In another study, Rodríguez-García et al. (2020) evaluated
LearningML, a machine learning model builder, to develop critical Table 7
thinking. This model builder teaches K-12 students on AI fundamentals Assessment constructs and tools to evaluate students’ AI learning.
to understand the applications of AI, how it can affect their lives, and the Research Constructs and tools Some examples Sample studies
ethical issues that arise from AI technologies. methods
In higher education, AI knowledge and skills become more advanced Quantitative Use knowledge tests Could you order the Kandlhofer et al.
to meet the future job demands. Kandlhofer et al. (2016) and Burgsteiner (13) to assess students’ AI major steps for the k- (2016); Wan et al.
et al. (2016) listed a set of AI concepts that have potential to become the cognitive gain and means clustering (2020).
basis for careers in science and engineering: automata, intelligent abilities algorithm? (Wan
et al., 2020)
agents, graphs and data structures, basics of computer science, machine
Use perceived How would you rate Chai, Wang, and
learning, etc., based on the “Artificial Intelligence: A Modern Approach” questionnaire to your knowledge Xu (2020); Chai,
written by Russell Stuart and Norvig (2009). Four studies mentioned the assess the non- about search Lin, et al. (2020);
importance of educating citizens on fundamental AI concepts, and the cognitive aspects, algorithms? ( Dai et al. (2020);
including: perceived Kandlhofer et al., Druga et al.
impacts of AI technologies on their everyday lives. For example, Rob­
ability, confidence in 2016) (2019);
inson (2020) mentioned that the Norwegian policy document, in a using AI, Gong et al.
section titled “AI for everyone: Elements of AI” (p. 44) asserts the gov­ intelligence, (2020); Julie et al.
ernment will make AI learning courses globally accessible in 2020, truthfulness, (2020);
which conceptualizes AI literacy as educating their citizens about the perceived Lin et al. (2021);
understanding, Wan et al. (2020).
elements of AI that require no prior knowledge (Robinson, 2020). In
subjective norms, AI
addition, three studies focused on AI learning in specific disciplines (i.e., anxiety, perceived
meteorology, medicine and library) to describe how AI can be applied in usefulness of AI, AI
vocational training and workplace application (e.g., using healthcare-AI for social good,
attitude toward using
technologies for delivering prevention, diagnosis, treatment and reha­
AI, confidence in
bilitation services) (Karaca et al., 2021; Rivero, 2020; Zou et al., 2019). learning AI, learning
behavioural
3.9. Teacher education intention, AI
optimsm, relevance,
AI awareness, career
From the review, four articles discussed how learning programs adaptability skills.
could strengthen teacher preparation especially for those without prior Qualitative Use videos, Through a follow-up Burgsteiner et al.
knowledge so that they could incorporate AI literacy into school (19) documents, pictures, interview, they (2016)*; Druga
curricula (Vazhayil et al., 2019; Xu, 2020). Vazhayil et al. (2019) presentations, found that children (2019); Julie et al.
students interactions were able to apply (2020);
explored how 34 Indian teachers perceived AI literacy learning after the
with AI agents and their new knowledge Kandlhofer et al.
workshop (e.g., “How did you find the teaching methods used during the projects to examine of ML to their own (2016); Schaper
training?“, “Do you think this workshop will enjoy you the most?“) students’ AI cognitive life and to think up et al. (2020);
(p.74). Teachers need to first update their knowledge of AI concepts that and non-cognitive personally Wan et al. (2020).
potentially be introduced in their schools. Then, they design suitable abilities. meaningful
applications using
teaching methods and strategies (e.g., collaborative problem-solving) ML (Kaspersen
and choose age-appropriate learning materials to stimulate students’ et al., 2021).
interest. They also need to consider various teaching challenges such as The author
insufficient funding, immature AI curricula, tools and evaluation compares how the
three values of trust,
methods (Gong et al., 2020), as well as technical concerns that whether
transparency, and
their schools’ internet infrastructure is ready for students to compile openness are defined
AI-enabled algorithms and applications (Vazhayil et al., 2019). and explored in
Apart from updating teachers’ AI knowledge to solve teaching Nordic AI policy
challenges, educators need to know and use suitable AI-enhanced documents
(Robinson, 2020).
technologies such as adaptive learning systems to facilitate their daily

7
D.T.K. Ng et al. Computers and Education: Artificial Intelligence 2 (2021) 100041

coded in RQ3 since this study aimed to compare how trust, trans­ can be considered as knowledge and skills gained which could be
parency, and openness are defined and explored in AI government policy regarded as quantifiable mastery of knowledge regarding AI compo­
documents in different countries. nents. Since AI will be more widely taught in K-12 and non-computer
Quantitative methods: To evaluate K-12 students’ AI literacy, one science university programs, it is observed that there will be more
important component is to promote their intention to learn and possess reliable and valid knowledge assessment which can be conveniently
basic knowledge about AI. Thirteen studies used quantitative methods to adapted into learning interventions to understand students’ AI knowl­
assess the knowledge acquisition of K-12 and university students via pre- edge for a summative assessment purpose.
and post-knowledge tests (e.g., What are the characteristics of depth- Survey: Surveys are widely used to investigate perceived ability,
first search?), and students’ perceived abilities (e.g., How would you affective and non-cognitive learning outcomes (e.g., motivations, atti­
rate your knowledge about search algorithms?) (Kandlhofer et al., 2016; tudes toward AI learning) in educational research. Eleven studies
Wan et al., 2020). Furthermore, studies discussed other quantitative developed surveys designed quantitative items to understand students’
aspects via surveys to understand students’ perceptions (non-cognitive perceptions of AI, and open-ended questions to collect student self-
aspects) towards AI literacy education such as confidence in using AI, report responses. Although surveys were often used to examine stu­
motivation and AI for societal good. dents’ non-cognitive outcomes, several studies used surveys to elicit
Qualitative methods: Nineteen researchers collected qualitative data students’ perceived AI understandings. For example, Chai, Wang, and
by taking pictures, field notes during teaching, and interviewing stu­ Xu (2020) and Chai, Lin, et al. (2020) designed a 6-item questionnaire to
dents to understand their motivations, expectations and lessons learned. understand students’ confidence, perceived relevance of learning AI and
For example, Druga (2019) recorded students’ interaction with AI readiness towards AI. The survey was then modified by Lin et al. (2021)
agents through field observations and adopted a three-attribute AI who employed structural equation modeling to validate primary stu­
perception questionnaire to evaluate how 102 children (7–12 years old) dents’ motivation for learning AI for the future development of AI
interacted and perceived their AI agents in their lessons. These three curricula and instruction. It is found that AI literacy is significantly
attributes measure whether the agents are smarter, truthful and under­ associated with the aspects including subjective norms, perceived use­
stand them (e.g., “What do you think of Google Voice, an AI-enabled fulness of AI, AI for social good, attitude toward using AI, AI optimism
agent?“). Children replied that the most fun features were playing and confidence in learning AI (Chai, Wang, & Xu, 2020; Lin et al., 2021).
beat-box and music, taking pictures and playing games. Watkins (2020) Another study Register & Ko (2020) applied qualitative thematic anal­
collected exhibition feedback from 367 participants to present the most ysis of students’ open-ended responses about how machine learning
frequently asked questions in a poster session (e.g., “Will librarians be systems work, as well as other aspects including ML model transparency,
able to develop programming with this tool?“) (p.17). critical thinking and learners’ interests and backgrounds. One advantage
of using surveys is that it consists of convenient data collection from a
3.11. Tools for assessing AI literacy large sample size which could produce quantifiable results. However, it
limits students’ rich description from their learning exposure. To fill this
To examine AI literacy assessment, researchers and AI educators now gap, the usage of project portfolio analysis and artefact-based interviews
use quantitative and qualitative tools to examine students’ AI literacy could be incorporated into knowledge tests and surveys to triangulate
development. To better understand the interplay between cognitive and students’ AI learning.
non-cognitive constructs of fostering AI literacy, studies began to Project portfolio analysis and artefact-based interview: Project
explore the changes in attitudes, behaviours and cognitions toward portfolio analysis refers to a purposeful and systematic process of col­
statistics in different AI educational contexts. Table 7 demonstrates the lecting and evaluating various types of students’ learning artefacts such
assessment constructs and tools to evaluate students’ AI cognitive and as products, projects and programs (McMillan, 2013). With students’
non-cognitive development. To further understand how to examine AI project portfolios, researchers and educators can interview students to
literacy through quantitative and qualitative tools, we categorized three examine their AI concepts and practices. Five studies applied project
major assessment types that have been found in the literature, including portfolio analysis with a follow-up interview to examine the attainment
knowledge tests, survey, portfolio assessment and artefact-based in­ of learning targets. For example, Kaspersen et al. (2021) evaluated
terviews. Some studies adopted more than one assessment type to students’ AI models and user interface design through collecting and
triangulate the learning outcomes of students’ AI literacy. labelling data, and building, testing and evaluating models. After ana­
Knowledge test: Six studies developed selected or constructed- lysing the artefacts in students’ projects, researchers found that children
response questions such as multiple choice and structured questions were able to apply their new knowledge of machine learning (ML) to
which are evaluated by correctness and completeness for summative their own life and to think up personally meaningful applications using
purposes. Kandlhofer et al. (2016) used paper-and-pencil exercises to ML. Another study Watkins (2020) asked participants to create 2D
assess students’ existing knowledge of AI concepts such as graphs, trees visualization and related kiosk applications that were demonstrated in
and data structures as evidence of student AI proficiency. Students’ AI the makerspaces and libraries at universities, and further invited visitors
knowledge acquisition and retention of AI skills was assessed in Lin et al. to perceive their AI applications in Cosmology. Kandlhofer et al. (2021)
(2021), Wan et al. (2020) and Rodríguez-García, Moreno-León, studied students’ picture taking, field notes, interaction and project
Román-González, & Robles, 2021 studies via some pre-post knowledge demonstrations during each teaching unit. Then, they performed
tests. Lin et al. (2021) administered AI concepts tests to address common semi-structured interviews and content analysis to examine how stu­
core AI concepts including decision tree, logics system, neural network dents foster their AI understandings. However, it is observed that re­
and machine learning. Wan et al. (2020) conducted pre-post question­ searchers did not generate a grading rubric to indicate the levels of
naires with written answers to questions relating to clustering, similarity achievement for each dimension of AI learning performance and
comparison and k-means clustering process whereas Rodríguez-García, whether a criteria is met. Future research could design rubrics to analyse
Moreno-León, Román-González, & Robles, 2021 selected and modified students’ AI concepts that could be graded by human raters and/or AI
14-item questions from other available tests and online resources, such education systems such as chatbots, expert systems and intelligent tutors
as Machine Learning for Kids website and MOOC platforms on AI. Wil­ (Zhang & Aslan, 2021).
liams et al. (2019) developed three or four multiple-choice questions on Through artefact-based interviews, it is useful to understand which
a tablet or paper to probe what kindergarten children understood about AI components students could understand and use more frequently
AI knowledge such as classification and generative AI to triangulate through communication and students’ projects. Since AI learning is
students’ learning behavior observation in relevant activities. novice to K-12 educators, the application of portfolio assessment and
The usage of this traditional knowledge test suggests that AI literacy follow-up interviews could capture a holistic view of what extent of

8
D.T.K. Ng et al. Computers and Education: Artificial Intelligence 2 (2021) 100041

knowledge and skills students need to obtain, and how educators design 4. Conclusion
and choose their learning materials and tools in their learning design. In
addition, this also encourages educators to use in classrooms and across In this review, a variety of definitions of AI literacy was identified.
platforms to formatively assess students to offer them feedback that is Most defined AI literacy based on different types of ‘literacies’, which
potentially beneficial to their future AI learning. had recently been applied to define skill sets in other disciplines. Most
With the great potential of using artefact-based interviews, re­ researchers advocated that instead of merely knowing how to use AI
searchers usually employed interviews to support and elaborate on applications, learners should learn about the underlying AI concepts for
students’ portfolio assessment by specifying their thinking processes of their future careers and understand the ethical concerns in order to use
using AI skills to solve problems (e.g., how they got started, how the AI responsibly.
project evolved, what was important for them to know to make the Since AI literacy is an emerging field that there is a lack of journals
project, what problems they encountered throughout the process, and published in this field, several limitations were identified. The keyword
how they dealt with those problems). Furthermore, students could search limited the scope of domain specificity within the AI context
reflect on themselves when working on the hands-on projects, such as while other subfields of AI like machine learning, neural network, etc.
what they were most confident of, what they might want to further could potentially be related to this study but were not captured in the
improve, and what engaged them. However, the challenges of using current review. Second, some articles in this review involved in­
interviews include its high cost and long time spent on interviewing and terventions and learning programs that were relevant to AI literacy.
coding the data as well as its small distribution to students, which makes However, the articles did not explicitly define the term AI literacy.
it difficult to be quantified (Tang et al., 2020). Throughout Third, a larger pool of studies discussing AI learning and teaching
artefact-based interviews, researchers were able to have detailed dis­ without mentioning the term “AI literacy” were not included in this
cussions about different AI elements in students’ projects, and to review; however, their interventions could be comparable to AI literacy
develop rich descriptions of their development practices. instructional design. This suggests that future review could broaden the
scope of search to common AI themes and to capture more literature in
3.12. RQ 4. what are the ethical concerns in the domain of AI literacy? AI learning and teaching.
The existing gaps and needs in the AI literacy research were derived
As AI plays an important role in day-to-day decision making, misused to bring forth potential areas for future studies. In this review, the ma­
or poorly designed AI could cause irreparable harm to humans and the jority of the articles (22) are conference papers while the other eight are
society (Fourtané, 2020). AI-concerned scientists and engineers like journal publications. Moreover, 19 of the articles used qualitative
Elon Musk expound on the horrors that future AI technologies may research methods and were exploratory research for preliminary studies.
wreak on humanity in decades to come (Johnson, 2019). In our review, In the near future, it is foreseen that research design will shift to be more
Schaper et al. (2020) reflected that international organizations such as empirical and interventional (e.g., quasi-experiment, design-based
UNICEF and OECD argue for the need of transparency and explainability research) with clearly documented treatment and control groups, as well
in AI to offer meaningful information to understand AI systems, user as varied data analysis procedures (e.g., t-test, one-way analysis of
interactions and societal impacts (Vincent-Lancrin & Van der Vlies, variance, factor analysis, regression, structural equation modeling).
2020; UNICEF, 2019). Nineteen studies had mentioned human-centered Furthermore, there is a need to examine the quality of different AI lit­
considerations, and raised attention to educate citizens to become so­ eracy assessments. Only three studies examine the reliability and val­
cially responsible and ethical users (Ahmad, Teredesai, & Eckert, 2020). idity of scales for AI literacy skills by conducting exploratory and
Gong et al. (2020) found that students pay little attention to ethical confirmatory factor analysis (Chai et al., 2020a, 2020b; Dai et al., 2020).
concerns such as bias in AI and legal responsibility, and intellectual To advance the AI literacy field, priority needs to be placed on proposing
property. In this regard, researchers began to notice the importance of AI definitive frameworks to guide educators to create lesson designs with
human-centered concerns such as inclusiveness, fairness, accountability, appropriate pedagogies, learning artefacts and assessment criteria. We
transparency, and ethics, instead of merely enhancing students’ AI hope this review will inspire scholars, educators, and government offi­
abilities and interests (Hagendorff, 2021; Microsoft, 2021). For example, cers to begin the discussion on how to define, implement and evaluate AI
Lin et al. (2021) designed a middle-school curriculum to develop AI literacy in the future.
literacy through combining AI concepts, ethics, awareness and careers.
Their study envisioned that the foundation of future AI industries would 4.1. Recommendations for future AI literacy education
be built on “principles of inclusivity, provide equitable access, include
consideration of multiple stakeholders and potential users, and mini­ The findings of this review present a preliminary overview of
mize the potential for bias” (p.191). To summarize, conceptualizing AI empirical research literature on AI literacy studies in the education field.
literacy with human-centered considerations is crucial to building a This study contributes to addressing the aforementioned research gaps,
future inclusive society. and provides directions for future research on AI literacy education
To bring up future responsible citizens who are competent in using AI based on the prevalent research questions:
in a reliable, trustworthy and fair manner, broadening participation in
AI for everyone and ensuring inclusive AI learning designs are necessary. ● AI becomes a fundamental skill for everyone, not just for computer
Teachers should address the learning needs of under-represented groups scientists. In addition to reading, writing, arithmetic and digital
including, but not limited to, gender, ethnic minorities, social-economic skills, we should add AI to every learners’ twenty-first century
status and cultural background when teaching AI. For example, Druga technological literacy in work settings and everyday life.
(2019) found that low social-economical children tend to have a harder ● Inspired by Bloom taxonomy, AI literacy possesses basic compe­
time advancing AI concepts because they had less experience with tencies to know and understand, use and apply, as well as evaluate
coding and interacting with these technologies in their everyday life. and create AI. People need to equip themselves cognitively for future
She proposed a set of guidelines to make AI learning inclusive by technological challenges in their workplaces. At the same time, it is
avoiding deceiving technologies, offering ways for children to customize important to foster their social responsibility and ethical awareness
their machines, and encouraging collaboration to share each other’s to use AI for societal good.
work (Druga, 2019). ● Students are not only the end users but potentially be problem-
solvers to use AI technologies in different scenarios, or even create
possible AI-driven hardware and software solutions to make our
society a better place to live in.

9
D.T.K. Ng et al. Computers and Education: Artificial Intelligence 2 (2021) 100041

● AI literacy combines the ideas of data science, computational Fourtané, S. (2020, August). In Ethics of AI: Benefits and risks of artificial intelligence
systems. Interesting engineering. Retrieved from https://interestingengineering.com/et
thinking and multi-disciplinary knowledge to interplay AI literacy
hics-of-ai-benefits-and-risks-of-artificial-intelligence-systems.
and AI thinking. Ghallab, M. (2019). Responsible AI: Requirements and challenges. AI Perspectives, 1(1),
● To facilitate educators’ teaching, the technological, pedagogical and 1–7.
content knowledge framework needs to be considered to provide a Gong, X., Tang, Y., Liu, X., Jing, S., Cui, W., Liang, J., & Wang, F. Y. (2020, October). K-9
artificial intelligence education in qingdao: Issues, challenges and suggestions. In
map for understanding how to integrate AI literacy into classrooms IEEE international conference on networking, sensing and control (ICNSC) (pp. 1–6).
effectively. Age-appropriate learning artefacts and curricula need to IEEE.
be designed to scaffold K-12 students’ AI conceptual understandings Graham, C. R. (2011). Theoretical considerations for understanding technological
pedagogical content knowledge (TPACK). Computers & Education, 57(3), 1953–1960.
and stimulate their motivation and interest in learning AI. Griffin, P., & Care, E. (Eds.). (2014). Assessment and teaching of 21st century skills: Methods
● Educators should update their AI knowledge to solve teaching and approach. Springer.
challenges such as knowing and using suitable AI-enhanced tech­ Hagendorff, T. (2020). The ethics of AI ethics: An evaluation of guidelines. Minds and
Machines, 30(1), 99–120.
nologies such as adaptive learning systems that facilitate their daily Han, X., Hu, F., Xiong, G., Liu, X., Gong, X., Niu, X., & Wang, X. (2018). Design of AI+
teaching practice and management, and promote personalized curriculum for primary and secondary schools in qingdao. In In Chinese automation
learning to understand students’ learning progress and needs. congress (CAC) (pp. 4135–4140). IEEE.
How, M. L., & Hung, W. L. D. (2019). Educing AI-thinking in science, technology,
● Future researcher and educators will develop pedagogical strategies engineering, arts, and mathematics (STEAM) education. Education Sciences, 9(3),
(e.g., collaborative project-based learning, gamification) and theo­ 184.
retical models (e.g., self-determination theory, constructionism) to Huitt, W. (2011). Bloom et al.’s taxonomy of the cognitive domain (Vol. 22). Educational
psychology interactive.
increase students’ motivation and engagement, promote interaction
Johnson, K. (2010, November). AI ethics is all about power. Venturebeat Insider - the
and collaboration, enhance motivation and attitudes, and develop Machine Making Sense of AI. Retrieved from https://venturebeat.com/2019/11/11/
numerous learning skills in the context of AI literacy. ai-ethics-is-all-about-power/.
● Future researchers and educators will develop quantitative and Julie, H., Alyson, H., & Anne-Sophie, C. (2020, October). Designing digital literacy
activities: An interdisciplinary and collaborative approach. In 2020 IEEE frontiers in
qualitative assessments to examine students’ learning performance education conference (FIE) (pp. 1–5). IEEE.
via post-knowledge tests, self-perceived surveys, learners’ artefacts, Kandlhofer, M., Steinbauer, G., Hirschmugl-Gaisch, S., & Huber, P. (2016, October).
projects and conversations. Artificial intelligence and computer science in education: From kindergarten to
university. In IEEE Frontiers in education conference (FIE) (pp. 1–9). IEEE.
● Human-centered considerations are important to raise attention to Karaca, O., Çalışkan, S. A., & Demir, K. (2021). Medical artificial intelligence readiness
educate citizens to become socially responsible and ethical users scale for medical students (MAIRS-MS)–development, validity and reliability study.
such as inclusiveness, fairness, accountability, transparency, and BMC Medical Education, 21(1), 1–9.
Kaspersen, M. H., Bilstrup, K. E. K., & Petersen, M. G. (2021, February). The machine
ethics, instead of merely enhancing students’ AI abilities and learning machine: A tangible user interface for teaching machine learning. In
interests. Proceedings of the fifteenth international conference on tangible, embedded, and embodied
interaction (pp. 1–12).
Kim, S., Jang, Y., Choi, S., Kim, W., Jung, H., Kim, S., & Kim, H. (2021). Analyzing teacher
Declaration of competing interest competency with TPACK for K-12 AI education. KI-Künstliche Intelligenz.
Koehler, M. J., Mishra, P., & Cain, W. (2013). What is technological pedagogical content
The authors declare that they have no known competing financial knowledge (TPACK)? Journal of Education, 193(3), 13–19.
Kong, S. C., Cheung, W. M. Y., & Zhang, G. (2021). Evaluation of an artificial intelligence
interests or personal relationships that could have appeared to influence
literacy course for university students with diverse study backgrounds. In Computers
the work reported in this paper. and education. Artificial Intelligence, Article 100026.
Larson, L. C., & Miller, T. N. (2011). 21st century skills: Prepare students for the future.
References Kappa Delta Pi Record, 47(3), 121–123.
Leahy, D., & Dolan, D. (2010, September). Digital literacy: A vital competence for 2010?.
In IFIP international conference on key competencies in the knowledge society (pp.
Ahmad, M. A., Teredesai, A., & Eckert, C. (2020, January). Fairness, accountability, 210–221). Berlin, Heidelberg: Springer.
transparency in AI at scale: Lessons from national programs. In Proceedings of the Leander, K. M., & Burriss, S. K. (2020). Critical literacy for a posthuman world: When
2020 ACM conference on fairness, accountability, and transparency, 690-690. people read, and become, with machines. British Journal of Educational Technology,
Bawden, D. (2008). Origins and concepts of digital literacy. Digital Literacies: Concepts, 51(4), 1262–1276.
Policies and Practices, 30, 17–32. Lin, P. Y., Chai, C. S., Jong, M. S. Y., Dai, Y., Guo, Y., & Qin, J. (2021). Modeling the
Bell, S. (2010). Project-based learning for the 21st century: Skills for the future. The structural relationship among primary students’ motivation to learn artificial
clearing House, 83(2), 39–43. intelligence. Computers & Education: Artificial Intelligence, 2, Article 100006.
Bloom, B. S. (1956). Taxonomy of educational objectives (Vol. 1, pp. 20–24). New York: Long, D., Jacob, M., & Magerko, B. (2019). Designing co-creative AI for public spaces. In
McKay: Cognitive domain. Proceedings of the 2019 on creativity and cognition (pp. 271–284).
Brundage, M., Avin, S., Clark, J., Toner, H., Eckersley, P., Garfinkel, B., & Amodei, D. Long, D., & Magerko, B. (2020, April). What is AI literacy? Competencies and design
(2018). The malicious use of artificial intelligence: Forecasting, prevention, and considerations. In Proceedings of the 2020 CHI conference on human factors in
mitigation. University of Oxford. arXiv preprint arXiv:1802.07228. computing systems (pp. 1–16).
Burgsteiner, H., Kandlhofer, M., & Steinbauer, G. (2016, March). Irobot: Teaching the Manyika, J., Lund, S., Chui, M., Bughin, J., Woetzel, J., Batra, P., Ko, R., & Sanghvi, S.
basics of artificial intelligence in high schools. In Proceeding of the AAAI conference on (2017, December). Jobs lost, jobs gained: Workforce transitions in a time of
artificial intelligence, 30(1). automation. McKinsey global institute. Retrieved from https://www.mckinsey.
Cavalcanti, A. P., Diego, A., Carvalho, R., Freitas, F., Tsai, Y. S., Gašević, D., & com/~/media/McKinsey/Industries/Public%20and%20Social%20Sector/Our%20I
Mello, R. F. (2021). Automatic feedback in online learning environments: A systematic nsights/What%20the%20future%20of%20work%20will%20mean%20for%20jobs%
literature review. Computers and Education: Artificial Intelligence, Article 100027. 20skills%20and%20wages/MGI-Jobs-Lost-Jobs-Gained-Report-December-6-2017.
Chai, C. S., Wang, X., & Xu, C. (2020a). An extended theory of planned behavior for the pdf.
modelling of Chinese secondary school students’ intention to learn Artificial McCarthy, J. (2007). From here to human-level AI. Artificial Intelligence, 171(18),
Intelligence. Mathematics, 8(11), 2089. 1174–1182.
Chai, C. S., Lin, P. Y., Jong, M. S. Y., Dai, Y., Chiu, T. K., & Huang, B. (2020, August). McMillan, J. H. (2013). Classroom assessment: Principles and practice for effective instruction
Factors influencing students’ behavioral intention to continue artificial intelligence (6th ed.). Boston: Pearson/Allyn and Bacon.
learning. In 2020 international symposium on educational technology (pp. 147–150). Microsoft. (2021). Fate: Fairness, accountability, transparency, and ethics in AI.
IEEE. Retrieved from https://www.microsoft.com/en-us/research/theme/fate/.
Dai, Y., Chai, C. S., Lin, P. Y., Jong, M. S. Y., Guo, Y., & Qin, J. (2020). Promoting Mongeon, P., & Paul-Hus, A. (2016). The journal coverage of web of science and Scopus:
students’ well-being by developing their readiness for the artificial intelligence age. A comparative analysis. Scientometrics, 106(1), 213–228.
Sustainability, 12(16), 6597. Ng, T. K. (2021). New interpretation of extracurricular activities via social networking
Davenport, T. H., & Ronanki, R. (2018). Artificial intelligence for the real world. Harvard sites: A case study of artificial intelligence learning at a secondary school in Hong
Business Review, 96(1), 108–116. Kong. Journal of Education and Training Studies, 9(1), 49–60.
Dignum, V. (2019). Responsible artificial intelligence: How to develop and use AI in a Ng, T. K., & Chu, K. W. (2021). Motivating students to learn AI through social networking
responsible way. Springer Nature. sites: A case study in Hong Kong. Online Learning, 25(1), 195–208.
Druga, S., Vu, S. T., Likhith, E., & Qiu, T. (2019). Inclusive AI literacy for kids around the Ng, T. K., Leung, J. K. L., Chu, K. W. S., & Qiao, M. S. (2021). AI literacy: Definition,
world. In Proceedings of FabLearn 2019 (pp. 104–111). ACM. teaching, evaluation and ethical issues. Proceedings of the Association for Information
Science and Technology, 58(1), 504–509.

10
D.T.K. Ng et al. Computers and Education: Artificial Intelligence 2 (2021) 100041

Pegrum, M., Dudeney, G., & Hockly, N. (2018). Digital literacies revisited. European Touretzky, D., Gardner-McCune, C., Martin, F., & Seehorn, D. (2019, July). Envisioning
Journal of Applied Linguistics and TEFL, 7(2), 3–24. AI for K-12: What should every child know about AI?. In Proceedings of the AAAI
Register, Y., & Ko, A. J. (2020, August). Learning machine learning with personal data conference on artificial intelligence (Vol. 33, pp. 9795–9799). No. 01.
helps stakeholders ground advocacy arguments in model mechanics. In In United Nations Children’s Fund (UNICEF). (2019). Workshop report: AI and child rights
Proceedings of the 2020 ACM conference on international computing education research policy. Retrieved from https://www.unicef.org/globalinsight/media/661/file.
(pp. 67–78). ACM. Vazhayil, A., Shetty, R., Bhavani, R. R., & Akshay, N. (2019, December). Focusing on
Rivero, E. (2020). Library and educational use cases. Library Technology Reports, 56(4), teacher education to introduce ai in schools: Perspectives and illustrative findings. In
14. In 2019 IEEE tenth international conference on technology for education (T4E) (pp.
Robinson, S. C. (2020). Trust, transparency, and openness: How inclusion of cultural 71–77). IEEE.
values shapes Nordic national public policy strategies for artificial intelligence (AI). Vincent-Lancrin, S., & Van der Vlies, R. (2020). Trustworthy artificial intelligence (AI) in
Technology in Society, 63, 101421. education: Promises and challenges. OECD.
Rodríguez-García, J. D., Moreno-León, J., Román-González, M., & Robles, G. (2020, Wan, X., Zhou, X., Ye, Z., Mortensen, C. K., & Bai, Z. (2020, June). SmileyCluster:
October). Introducing artificial intelligence fundamentals with LearningML: supporting accessible machine learning in K-12 scientific discovery. In Proceedings of
Artificial intelligence made easy. In In Eighth international conference on technological the Interaction Design and Children Conference (pp. 23–35).
ecosystems for enhancing multiculturality (pp. 18–20). ACM. Wang, P. (2019). On defining artificial intelligence. Journal of Artificial General
Rodríguez-García, J. D., Moreno-León, J., Román-González, M., & Robles, G. (2021, Intelligence, 10(2), 1–37.
March). Evaluation of an online intervention to teach artificial intelligence with Watkins, T. (2020). Cosmology of artificial intelligence project: Libraries, makerspaces,
LearningML to 10-16-year-old students. In Proceedings of the 52nd ACM technical community and AI literacy. AI Matters, 5(4), 14–17.
symposium on computer science education (pp. 177–183). ACM. Williams, R., Park, H. W., & Breazeal, C. (2019, May). A is for artificial intelligence: The
Russell Stuart, J., & Norvig, P. (2009). Artificial intelligence: A modern approach. Prentice impact of artificial intelligence activities on young children’s perceptions of robots.
Hall. In Proceedings of the 2019 CHI conference on human factors in computing systems (pp.
Schaper, M. M., Malinverni, L., & Valero, C. (2020, October). Robot presidents: Who 1–11).
should rule the world? Teaching critical thinking in AI through reflections upon food Wong, G. K., Ma, X., Dillenbourg, P., & Huan, J. (2020). Broadening artificial intelligence
traditions. In Proceedings of the 11th nordic conference on human-computer interaction: education in K-12: Where to start? ACM Inroads, 11(1), 20–29.
Shaping experiences (pp. 1–4). Shaping Society. Xu, L. (2020, December). The dilemma and countermeasures of AI in educational
Sharma, A. (2019, November). Entrepreneurship and role of AI. In Proceedings of the 2019 application. In In 2020 4th international conference on computer science and artificial
2nd international conference on signal processing and machine learning (pp. 122–126). intelligence (pp. 289–294). ACM.
ACM. Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review
Steinbauer, G., Kandlhofer, M., Chklovski, T., Heintz, F., & Koenig, S. (2021). of research on artificial intelligence applications in higher education – where are the
A differentiated discussion about AI education K-12 (pp. 1–7). KI-Künstliche Intelligenz. educators? International of Journal Education Technology Higher Education, 16, 39.
Tang, X., Yin, Y., Lin, Q., Hadad, R., & Zhai, X. (2020). Assessing computational thinking: https://doi.org/10.1186/s41239-019-0171-0
A systematic review of empirical studies. Computers & Education, 148, Article Zou, L., Wang, Q., & Zhao, P. (2019, October). A preliminary study on the application of
103798. artificial intelligence technology in meteorological education and training. In IEEE/
WIC/ACM international conference on web intelligence-companion volume (pp.
148–153). ACM.

11