Volume 8, Issue 7, July – 2023 International Journal of Innovative Science and Research Technology

ISSN No:-2456-2165

Usability Evaluation of University Putra Malaysia

(UPM) Learning Management System
Nuruddeen Ahmad Sama’ila, 2Mukhtar Abubakar, 3Abdulhafiz Nuhu
1
123
Department of Computer Science and Information Technology, Federal University Dutsinma, Katsina State, Nigeria

Abstract:- The objectives of any academic Learning development taking place in the field of information
Management System is to provide valuable academic dissemination technology shows that, e-learning nowadays
information to its users without any difficulty. metamorphoses and becomes part and parcel of what is
Nowadays, web usability is becoming a crucial issue for presently known as Learning Management System (LMS).
LMS development, most users perceive usability as a In support of this (Pituch & Lee, 2006) argued that e-
key factor in e-learning application planning and use . It learning or virtual learning system (VLS) is gradually
is possible that website visitors will find it very difficult turning into method of teaching and learning through LMS.
in getting their needs due to the issue of usability. In further support, (Wang & Shee, 2007) reaffirmed the
However there is increased concern whether to improve aforementioned argument, thus, e-learning or learning
the UPM learning Management system or not because through internet has become a major phenomenon in recent
various users encountered many problems when using years. Detailed analysis of the developmental trends of LMS
the LMS. These result lead to less use of the system and over period of time concurs with the impressions of many
affect the goals of learning management system. educationalists and researchers world over that, the long
Therefore this study identifies the usability level of UPM awaited revolution in teaching methodology has evolved, i.e.
learning Management System from 376 UPM students the traditional method of acquiring and imparting
perspectives, A proposed UPM-LMS was developed to knowledge is being replaced with e-learning or Learning
overcome the usability problems of UPM-LMS which Management System LMS. In agreement with the assertion,
was achieved through post-test and Expert Evaluation. . people using LMS, particularly end-users, have the notion
The result of the pre survey shows that all the usability that, the method is very influential in the sense that, it
factors were found to be at high level but looking at their encourages good interactions between teachers and students,
point they are not up to the mid-point of high usability by providing various materials one needs within the time
level, their point are closer to low usability level thus, frame. End-users here refers to students, teachers,
more attention needs to be given to the factors for better researchers, administrators, board management, staff etc.
students satisfaction especially efficiency and error people and various materials are refers as education teaching
prevention which their values were found to be at the aids, that are not only cost effective but accessible to
border line of low usability level. However 30 students everybody at any part of the world to carry out their
and 3 usability experts evaluated the proposed UPM activities with up to most ease or with the highest peak level
LMS, and found out that, there is significant of satisfaction. In line with this general concept, Frey (2005)
improvement on all the usability factors. described LMS as a means of assisting learners and
instructors to accomplish their instructional goals through
Keywords: - Usability, Learning, Management, System, the use of problem-solving team, simulation online, and
UPM. questions and answers session, rather than be a tool that just
allows printing lecture notes, evaluating lecturers or seeing
I. INTRODUCTION any updated information made by the instructors. In a
similar concept, Naidu (2006) defined LMS “as a means an
One of the popular phrases used in our present electronic Moodle that include a suite of tools for teaching
education sectors and it operates all over the world is the and learning online”. Based on these definitions, my
electronic Learning, which is synonymously abbreviated as conceptual understanding of the term, LMS is an
e-Learning and is more or less related to other internet electronically designed instructional Moodle which is made
services such as e-research, e-library, e-commerce, e- accessible and cost effective to all and sundry irrespective of
payment and other e-transactions. This method evolves as a distance, time or location in any part of the world.
result of a sort of revolution taking place in the field of
information technology (IT). In a nutshell, this newly II. RELATED WORK
emerged method of learning portrays itself as distance
learning, which is globally made accessible to all and sundry Among the various commendations made on the impart
irrespective of location, distance, cost or time. The method and importance of LMS to the world- wide educational
is electronically designed to be accomplished through sectors, include those from the end users of the system, who
various electronic gadgets, such as internet, intranet, unanimously express their views, that, this globalized
extranet, satellite, audio, video, CD ROM as well as through method is one of the rapidly growing technology in our
others sources audio-video media of information educational institutions, this is as a result of its worldwide
dissemination. Close observation of the exponential advantages such as significant reduction in travelling time,

IJISRT23JUL458 www.ijisrt.com 814

Volume 8, Issue 7, July – 2023 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
cost effective, availability of efficient training materials that many end-users discovered that, not all LMS are efficient;
suite one level of interest and to crown it up, convenience therefore careful assessments are needed in selecting the
as the method is accessible to everybody in the world most suitable LMS. For instance, usability is one of the non-
regardless of distance, time or location. For instance functional requirement for choosing any system especially
(Andersson, & Grönlund, 2009; Linna, 2013) independently LMS, which allows us to know the level of its usability. In
enumerated a number of LMS applications which are collaborative efforts to define the term usability in relation
advantageous to the world educational sectors and they to LMS. Different researchers and standard institute defined
include among others, cost reduction, worldwide usability from different perspective, among them include
accessibility and overall improvement of the general quality [IEEE Std.610.12-1990] as “The ease with which a user can
of education. In line with these enumerated advantages of learn to operate, prepare inputs for, and interpret outputs of a
LMS, institutions in not only technologically advanced system or component”. In view of the fact that, usability is a
countries in the world but also institutions in Sub-Saharan component of Human Computer Interaction (HCI), ISO
countries derived numerous benefits in using LMS, which 9241 part ii, viewed usability as “extent to which a product
range from uplifting their educational institutions to can be used by specified users to achieve specified goals
becoming complementary to various methods used in with effectiveness, efficiency and satisfaction in a specified
teaching and learning. This is buttressed by the adoption of context of use”. Recent reviews of the components of LMS,
various LMS models in many educational institutions in shows that, usability is becoming a strategic factor level that
Sub-Saharan countries. (Munguatosha et al, 2011) expressed needs special attention, particularly in the software
the view that, out of approximately 80.2% of educational development processes. That is why (Juristo et al., 2007)
institutions in Sub-Saharan countries, 78% uses Moodle said; usability evaluation has now become an increasingly
LMS, while the remaining adopts blackboard. Also in Sudan major concern area of human-computer interaction (HCI).
(El-Mubarak et al., 2013) revealed that 25% of tertiary Similarly, (Nielsen, J. 2012) defined usability in relation to
institutions in Sudan use Moodle LMS. In re-affirming the its factors, i.e. efficiency, satisfaction, memorability,
aforementioned, research conducted by (Unwin et al., 2010) learnability and error prevention. He continued to describe
showed tremendous expansion, adoption and utilization of usability as an important factor in designing any web-site,
different LMS, in Sub Saharan countries, these include especially LMS, which is of great concern to many web-site
online system that used to arrange and encourages end-users visitors all over the world. These Learning Management
to learn through the web. In a similar contribution (Pituch & Systems LMS include that of University Putra Malaysia,
Lee, 2006) reported that learning these online systems are UPM, where students from all walks of life avail themselves
ordinarily integrated with learning resource tools for audio- with UPM system. Reactions from end-users of UPM: LMS
video messages and communication (email, discussion especially students revealed that UPM web-site visitors
forum, chat ) and evaluation tools. Moreover (Ssekakubo et encounter with many problems in getting the result expected
al., 2011; Unwin et al., 2010) further confirmed that, the when working with the UPM: LMS due to some usability
most relevant frameworks conveyed in several institutions in hurdles. Based on these encountered problems, the needs to
Sub- Saharan countries are Moodle, blackboard, Atutor, know the usability strength and weaknesses level of UPM:
Sakai, and Kewl as they demonstrate the capacity to LMS becomes absolutely imperative, as the result obtained
alleviate the difficulties confronting education sectors in could be of great helping to the management and website
Sub-Saharan countries. In further commendations made on developers of not only that of UPM but also others
the importance of LMS, coupled with the advanced trends in institution LMSs. In view of the aforementioned, the
teaching and learning, LMS, is as paramount important to researchers of this paper find it of economic importance to
the worldwide educational sectors, as majority of explore the presumed usability factors that affect UPM:
universities, polytechnics and colleges in the most advanced LMS, and at the same time open up related areas for further
countries such as USA, United kingdom, Japan etc. in the research.
world embrace LMS to support instructors showing
activities and students learning process. According to To embark on this research, the entire work is
(Epping, 2010) the most outstanding feature of LMS is to organized as follows; a brief literature review, followed by
provide an enabling environment for learning, irrespective Nielsen and WAMMI usability factors, then, methodological
of distance, time or location. Further observation on the stages of the research, further pilot study, actual study,
expanded technology in internet, LMS has as of now been evaluation metrics, reliability and validity test and sample
connected to different educational institutions round the selection were all described in this paper. The results
world. (Falvo & Johnson, 2007) shared the views that, LMS obtained were analyzed and evaluation of the proposed
helps end-users particularly instructors and lecturers to UPM: LMS was made from both the students and usability
explore various LMS models to discuss course content by experts, and finally, contributions, recommendation and
exchanging information to one another, keep track of conclusion of the research work were summarized.
students learning and control educational activities in an
online environment. On the other hand (Boggs & shore Different researchers from different domains have
2004) emphasized on the way and manner LMS enhances conducted usability studies in Learning Management System
students and instructors learning and teaching activities. LMS) and other applicable field of usability, among such are
the work of (Al-Khalifa, 2010) evaluated the usability of
In view of the numerous factors adduced to explain the JUSUR Learning Management System of the department of
importance of LMS to the world-wide learning sectors, information and technology in King Saudi University. In the

IJISRT23JUL458 www.ijisrt.com 815

Volume 8, Issue 7, July – 2023 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
study, two standard questionnaires were used as an in the study from the faculty of economic, science and
evaluation method, which comprises: a) system usability informatics. Based on the survey, it was observed that the
scale (SUS) developed by John Brooke, 1986, the Moodle LMS was usable in the perspectives of usability
questionnaire consists of ten questions which were used and factors: memorability, effectiveness, efficiency, ease of use
adapted. b) Learning Management System usability and satisfaction.
questionnaire (LMSUQ) , this questionnaire was constructed
and supported by two existing questionnaires which are Web In-order to come up with an appropriate questionnaire
based learning Environment Instrument WLEI (Chang, for UPM Learning Management System, various related
1999) and usability and user satisfaction questionnaire papers based on usability evaluation of LMS were reviewed.
adapted from PSSUQ (Zins et al., 2004). The questionnaire The observation shows evaluation methods of LMS
consists of 31 questions, 18 questions from usability and usability, which includes questionnaire based evaluation,
user satisfaction and 13 from WLEI. A non-probability heuristic evaluation, frameworks, models and checklist
sampling of 155 female students were selected as method, thus, most previous works focus on the Nielsen
respondents for the research work. Furthermore, the concept, as more than half of the researchers adopted
evaluation of JUSUR learning management system (LMS) is questionnaire as an instrument for usability evaluation of
based on 7 factors namely: system usefulness, learnability, LMS upon which 80 percent directly or indirectly adopted
satisfaction, outcome/future use, design/layout and the concepts of Nielsen usability evaluation which
functionality. However some students observed that comprised of 5 usability factors i.e. learnability, error
JUSUR LMS was user-friendly and easy to use but prevention, satisfaction, efficiency and memorability. Some
functionality of the system was complex, as some students researchers used all the Nielsen usability factors, whilst
revealed that the font size is too small and difficult to find its others used two to three of the factors. The remaining
back button. reviewed papers adopted some of the Nielsen usability
factors in their models, framework and heuristic evaluation,
The study of (Gorgi et al, 2008), evaluated the usability but generally most researchers focus on Nielsen concept in
of LMS Moodle from four units, namely learning, evaluating the usability of LMS either directly or indirectly.
synchronous and asynchronous communication, submissions In this particular research which is related to UPM: LMS
of assignment and testing. In the study, four course creators evaluation, three factors from (Nielsen, 2012 ) together with
known as teachers, four university professors and two WAMMI based evaluation questionnaire (Caglar & Mentes,
system administrators were included in the evaluation 2012) similar to Nielsen usability factors were adopted as an
techniques. Also the course creators create courses and add instrument for the study.
blocks and tools into the courses. Three different set of
questionnaires were used with three different types of The figure 1 below shows the relationship between
questions (open, closed ended questions, and scaled WAMMI and Nielsen usability factors both are used for
answers). The questionnaires were given to the students LMS usability evaluation. Efficiency and learnability are the
after one, two and six weeks of using the Moodle LMS common factors between the two.
respectively. A non-probability of 84 students participated

Fig 1 WAMMI- Nielsen Usability Factors

In order to come up with an appropriate usability III. METHODOLOGY

questionnaire we combined the two usability factors of
WAMMI and Nielsen as shown below. The main goal of this paper is to identify the level of
usability strength and weaknesses of UPM Learning
 WAMMI-Nielsen Usability Factors Management System, based on usability factors of WAMMI
and Nielsen (2012) and these has been achieved by adopting
 Attractiveness the methodological phases and activities as described below.
 Controllability
 Efficiency
 Learnability
 Helpfulness
 Memorability
 Satisfaction
 Error prevention

IJISRT23JUL458 www.ijisrt.com 816

Volume 8, Issue 7, July – 2023 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165

Fig 2 Flowchart of the Research Methodology

 Questionnaire age, internet experience, gender, faculty, nationality and

This study adopts WAMMI-Nielsen questionnaire access to UPM LMS. The second section consisted of thirty
based evaluation techniques of (Caglar & Mentes, 2012) two questions, four questions from each of the 8 categorized
together with three Nielsen (2012) usability factors factors namely: attractiveness, satisfaction, memorability,
(memorability, error prevention and satisfaction). The efficiency, learnability, controllability error prevention and
questionnaire comprised of two sections, the first part helpfulness. The questionnaire aims to identify the usability
contained the information about the respondents, such as level (strength and weaknesses) of UPM LMS.

IJISRT23JUL458 www.ijisrt.com 817

Volume 8, Issue 7, July – 2023 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
 Evaluation Metrics Unacceptable”. Cronbach’s alpha, is the most widely used
In-order to evaluate the usability of UPM LMS from objective measure of reliability and it is used to measure the
students’ point of view, responses will be evaluated based reliability of the questionnaire adopted for this study. The
on the adopted merit point of (Islam & Tsuji, 2011). closer the Cronbach’s alpha coefficient is 1.0 the greater the
internal consistency of the items in the scale. Reliability test
Table 1 Usability Merit Point of UPM LMS was conducted on the data that was obtained from the pilot
Option Strongly Disagree Agree Strongly study.
Disagree Agree
Merits 1 2 3 4 Table 3 Reliability Statistic of Pilot Study
Cronbach's Alpha
There are four questions in each usability factors and Based on Standardized
each question represents the participant merit point. Table 1 Cronbach's Alpha Items N of Items
above shows four usability likert scale with their 0.822 0.823 8
corresponding merit points, ranging from 1 to 4. According
to (Abdullah & Wei, 2008), usability point for a category, x, Table 4 Reliability Statistic of Pilot Study
is defined as: X = [Σ (Merit for each question of the Cronbach's Cronbach's Alpha Based
category)] / [number of questions]. Overall mean, minimum Alpha on Standardized Items N of Items
mean, maximum mean, range and standard deviation were 0.896 0.897 8
all calculated to get the usability strength and weaknesses
level of the UPM LM. Table nand 4 above show the Cronbach’s Alpha result
of both pilot and actual study to be 0.823 and 0.897
Table 2 Usability Level and Corresponding Usability respectively. This reveals that the instrument used in the
Points. study is up to expectation as such appropriate for the study.
Usability level Point X
Very low 0 ≤ x ≤1.49  Sample Selection
Low 1.5 ≤ x ≤ 2.49 In order to select the respondents that will participate
High 2.5 ≤ x ≤ 3.49 in the study, a probability sampling called cluster sampling
Very high 3.5 ≤ x ≤ 4.00 method was applied. University Putra Malaysia was
grouped into two clusters, cluster1 consists of institutes and
The usability strength and weaknesses level of UPM cluster2 consists of faculties. Simple random sampling was
LMS was determined by using the corresponding merit randomly applied, in which cluster2 was chosen. All the
values of usability levels. Table 2 above shows the usability faculties were numbered from 1 to 15 and each numbers was
level with their corresponding usability points ranging from written in a small piece of papers, and the papers were
0 to 4. folded and thrown in a bowl. Hence, 4 numbers were
randomly selected from the 15 numbers and the numbers
 Pilot Study that correspond to such faculties are the faculty of economic
In any survey research work, there is need to conduct a and management, science, engineering and computer
pilot study to know whether the instrument used is science.
appropriate for the study or not. Because it is very difficult
for usability evaluation to be free from error. The pilot In-order to know the required number of sample size,
study determines the workability of the actual study. Four slovins formula (Ariola, 2006) was applied:
faculties were randomly selected from the 15 faculties of the
University Putra Malaysia where 30 postgraduate students n = N / (1 + Ne2)
were selected for the study, and in the process of conducting Where n = number of respondents
this research, Comments, suggestion and ambiguities were N= total population
met, and all possible corrections were made, before the e = error tolerance
result was taken into considerations. n= 9585 / (1 + 9585 (0.05)2)
= 383.9 = 384.
 Reliability
Reliability and validity are the main elements used, as The sampling size obtained from slovins formula was
instrument in measurements evaluation, such as compared with that of (krejcie & Morgan, 1970) which
questionnaire. Reliability is the extend in which the sample size of 370. The difference between the number of
questionnaire (instrument) gives the same result samples size obtained from the slovins formula and that of
consistently, whereas validity refers to the degree in which (Krejcie & Morgan 1970) is 384- 370= 14 which is
an instrument such as questionnaire measures what is negligible, therefore sample size of 384 was used for this
intended to measure. The value of alpha measures the study.
internal consistency of the test (Cronbach, 1951) and it is
defined “as number ranging from 0-9” (Cronbach, 1951). In-order to get the exact number of the sample size, 10
(George & Mallery, 2003) (p. 231). Provide the following percent of the sample size was added to the actual number of
rules of thumb: “> .9 – Excellent, > .8 – Good, > .7 – the sample size.To know the number of questionnaire to be
Acceptable, > .6 – Questionable, > .5 – Poor and < .5 –

IJISRT23JUL458 www.ijisrt.com 818

Volume 8, Issue 7, July – 2023 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
distributed, number of sample size was assigned to each  Descriptive Statistics of Respondents
faculty based on the percentage of their population. Descriptive statistics of respondents that participated in
the survey are showed in table 5 below.

Table 5 Descriptive Statistic

Factors Category NQD NQR Percentage
Gender Male 217 57.7
Female 422 159 42.3
Access To UPM LMS Daily 47 12.5
After 2 days 422 31 8.20
Weekly 105 29.7
Two weeks and above 154 41.0
Age 23-33 229 60.9
34-43 422 112 28.9
44-53 33 8.8
54 and above 2 5
Internet Experience < 1 Year 10 2.7
1-5 years 422 62 16.2
6 Years and above 304 80.9
Faculty Science 71.74 69 16.35
52 12.32
Economics and Management 55.95 193 45.73
Engineering 62 14.69
226.74
Computer Science
67.52
Have you ever used Putra Yes 422 337 89.6
LMS? No 39 10.4
Students Local 422 224 59.6
International 156 40.4
NQD = Number of questionnaires distributed
NQR = Number of questionnaires returned

As showed in Table 5 above, more than half of the  Usability Strength and Weaknesses Level of UPM LMS
students were males, whereas 42.3% were females. For In this research work, four usability levels with their
Access to UPM : LMS 12.3%, 8.2%, 29.7% and 41.0% merit points were adopted based on the work of (Abdullah &
students have access to the UPM: LMS daily, after 2 days, Wei, 2008), very low, low, high and very high usability
weekly and more than 2 weeks respectively. More than 60% levels. In-order to determine the usability level of a given
of the students were below 23-33 years. 28.9%, 8.8% are factor, whether it is very low, low, high and very high
students within the age 34-43 and 44-43 years of age usability, we first divide the range of a given factor by the
respectively, whilst 5% of the students were above 54 years number of usability level i.e. RUF/NUL.
of age. This result is obviously true because of the students’
academic nature of age. 51.3% of the students were from Where
faculty of engineering. This result is not surprising because RUF=Range of Usability factor
faculty of Engineering has the highest number of students NUL=Number of usability level
among the other three faculties thus; hence more participants
are expected from the faculty. 13.8% of the students are After the divided range value of a given factor was
from the faculty of economics and 18.4% from the faculty of obtained, then very low usability level is obtained from the
science, whereas 16.5% of the students comes from the minimum mean value of a given factor to the maximum
faculty of science. More than 80% of the students have mean value of that factor i.e. MaxMVVLU = MinMVVLU+
computer experience for more than 6 years whereas 16.2% DRV.
and 2.7% of the students have one to five years’ experience
and less than one years’ experience respectively. 89.6% are Where
found to be the users of UPM LMS, whereas 10.4% were MaxMVVLU = Maximum Mean value of very low usability
not using the UPM LMS. This result is also not surprising if MinMVVLU=Minimum Mean value of very low usability
we consider the students that offer a course while studying, DRV=Divided range value
hence most of the students happen to take one or two
courses before graduating from his or her study, therefore Low usability level was obtained from the maximum
this necessitates the students to make use of the LMS. mean value of very low usability which becomes the
59.6% and 40.4% of the students were local and minimum mean value of low usability i.e. MaxMVVLU =
international students studying in UPM respectively. MinMVLU +0.01.

IJISRT23JUL458 www.ijisrt.com 819

Volume 8, Issue 7, July – 2023 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
Where Very high usability level was determined from the
MinMVLU=Minimum Mean value of low usability maximum mean value of high usability which becomes the
minimum mean value of very high usability i.e. MaxMVHU
And the Maximum Mean value of low usability is also = MinMVVHU +0.01.
obtained from the Minimum Mean value of low usability
plus the divided range value i.e. MinMVLU + DRV. Where
MinMVVHU=Minimum Mean value of very high usability
Similarly high usability level was obtained from the
Maximum value of low usability which becomes the And the Maximum value of very high usability is also
minimum mean value of high usability i.e. MaxMVLU = obtained from the Minimum Mean value of very high
MinMVHU +0.01. usability plus the divided range value i.e. MinMVHU +
DRV.
Where
MinMVHU=Minimum Mean value of high usability IV. RESULT FROM EVALUATION METRICS

And the Maximum Mean value of high usability is also The summary of the usability level of UPM LMS is
obtained from the Minimum Mean value of high usability shown in table 6 below and depicted graphically in figure 3
plus the divided range value i.e. MinMVHU + DRV.

Table 6 Pre Survey Usability Result

Factors Usability level Point
Attractiveness High 2.61
Controllability High 2.75
Helpfulness High 2.67
Efficiency High 2.58
Learnability High 2.78
Memorability High 2.84
Satisfaction High 2.74
Error prevention High 2.66

Fig 3 Students Pre survey Usability Result

Figure 3 above shows the overall responses of 376 level points with 2.54 and 2.58 respectively, and then
students to each factors of WAMMI-Nielsen usability followed by attractiveness and helpfulness with 2.61 and
questionnaire, which varies from 0-4 likert scale. All the 2.67 respectively. Satisfaction, controllability are close to
usability factors were found to be at a “High” level with the each other with the value 2.74 and 2.75, thus this result is
value above 2.50. Even though all the usability factors were not surprising because satisfaction can lead to the
found to be at high level but looking at their points they are acceptance of any factor, but this revealed that, the students
not up to the mid-point of high usability level thus, their are more satisfied towards the controllability. Similarly,
point are closer to low usability level than high usability learnability and memorability points are also close to each
level. Error prevention and efficiency have the low usability other with the point 2.78 and 2.84, hence this result is

IJISRT23JUL458 www.ijisrt.com 820

Volume 8, Issue 7, July – 2023 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
obvious because learnability and memorability go hand in improve the level of the usefulness to reach the peak level of
hand, you cannot remember without learning and vice versa. students’ satisfaction. Therefore the overall mean of
Therefore among all the usability factors students find it usability level of UPM LMS was found to be towards the
easier to remember the features of UPM LMS. Although the high level with the point 2.69 as shown in the figure 4
UPM LMS was found to be usable but still there is need to below.

Fig 4 Overall Usability Levels of UPM LMS

Figure 4 above shows the overall usability level of the features of the Proposed UPM LMS when using the
UPM LMS; however it was found to be towards the “High” system for the first time. However Learnability goes hand
usability level with the point 2.69. in hand with memorability because remembrance is part of
learning thus, if a person cannot remember, is very difficult
V. CONTROL VARIABLES for him to learn. From the authenticated results obtained
from the university more than 10,000 students access the
Due to the differences that occur between the existing existing UPM LMS therefore the efficiency of the system
UPM: LMS and the proposed one, it becomes necessary to depends on the number of students accessing the UPM LMS
control some factors that lead to such differences. There is at a time. Hence the efficiency of the existing UPM LMS
need to controls memorability, efficiency and learnability system cannot be compare with the proposed UPM LMS
factors when evaluating the proposed UPM LMS. which was accessed by one student at a time. At last
Memorability deals with remembering other features and memorability, learnability and efficiency factors were not
regaining proficiency when not in use of the system for a considered in evaluating the proposed UPM LMS.
long time. So it is not possible for the students to remember

Table 7 Experts’ Usability Result

Factors Usability level Point
Attractiveness High 2.92
Controllability High 2.83
Helpfulness High 3.00
Satisfaction High 2.92
Error prevention High 3.01

IJISRT23JUL458 www.ijisrt.com 821

Volume 8, Issue 7, July – 2023 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165

Fig 5 Experts Usability Result

Figure 5 above shows the post-test evaluation result of the proposed UPM LMS from three usability experts. All the
usability experts navigates through the proposed UPM LMS and found that the proposed UPM LMS was usable based on the
usability factors defined above. Comparing the result with the pre-survey it reveals that there is improvement on the usability
factors of the existing UPM LMS.

Table 8 Post-Test students’ Usability Result

Factors Usability level Point
Attractiveness High 3.33
Controllability High 3.54
Helpfulness High 3.43
Satisfaction High 3.61
Error prevention High 3.51

Fig 6 Students Post-Test Usability Result

IJISRT23JUL458 www.ijisrt.com 822

Volume 8, Issue 7, July – 2023 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
Similarly, figure 6 above reveals that there is great [4]. Caglar E. and Mentes, S. A. (2012), “The Usability of
improvement on the proposed UPM LMS based on the University Websites – A Study on European University
usability factors defined above. 30 students evaluated the of Lefke” International Journal of Business
proposed UPM LMS and majority of the respondents Information Systems, 9 (6) (will be published in the
answered the same usability likert scale. Hence the June 2012 issue)
evaluation result from the 30 students is enough to represent [5]. Boggs, S., Shore, M., & Shore, J. A. (2004). Using e-
the number of students that participated in the pre survey learning platforms for mastery learning in
since most of the students answers were saturated and fall developmental mathematics courses. Mathematics and
within the same likert scale. Computer Education, 38(2), 213-220.
[6]. Epping, R. J. (2010). Innovative use of Blackboard® to
VI. CONCLUSION assess laboratory skills. Journal of Learning Design.
3(3). 32-36.
In this section, we summarized the whole research [7]. Elmubark, A.Y., Cronjé, J.C. & Osman, I.M. (2013).
work in order to show the result obtained from the study. A Framework for Investigating
The usability strength and weaknesses level of UPM LMS [8]. Universities Readiness for On-line Learning: with
was revealed out based on the usability factors of WAMMI Particular Reference to Sudanese
and Nielsen (2012), Attractiveness controllability, [9]. Universities. In IST-Africa 2013 Conference
efficiency, learnability, memorability, helpfulness, error Proceedings, 13 pages.
prevention and satisfaction. The result of the pre survey [10]. George, D., & Mallery, P. (2003). SPSS for Windows
shows that all the usability factors were found to be at high step by step: A simple guide and reference. 11.0 update
level but looking at their point they are not up to the mid- (4th ed.). Boston: Allyn & Bacon.
point of high usability level their point are closer to low [11]. Pituch, K.A., & Lee, Y.K. (2006). The influence of
usability level thus, more attention needs to be given to the system characteristics on e-learning use. Computers &
factors for better students satisfaction especially efficiency Education, 47, 222–244.
and error prevention which their values were found to be at [12]. Falvo, D. A., & Ben F., & Johnson, B. F. (2007). The
the border line of low usability level. However 30 students use of learning management systems in the United
and 3 usability experts evaluated the proposed UPM LMS, States. TechTrends. 51(2). 40-
and found out that, there is significant improvement on all [13]. Frey, B.A. (2005). Enhancing face-to-face courses with
the usability factors. a course management system (ERIC Document
Reproduction Service No. ED 490408).
FUTURE WORK [14]. Inversini, A., Botturi, L., & Triacca, L. (2006).
Evaluating LMS usability for enhanced e-learning
In recommendation for future research, the experts are experience. Paper presented at the EDMEDIA
requested to suggest a way forward to improve the UPM Conference, Orlando, Florida
LMS. The interface of the prototype was found to be good [15]. ISO (1998) ISO 9241-11: Ergonomics Requirements
and usable. However, In order to obtain more usable and for Office Work with Visual Display Terminals
successful UPM LMS the menus and the text have to be (VDTs). Part 11: Guidance on Usability. Geneva:
consistent in all the pages. In addition the experts also International Standards Organisation. Also available
comments on the message page, which lack in from the British Standards Institute, London.
differentiating from the read and un-read message. Also the [16]. Juristo, N., Moreno, A.M., Sanchez-Segura, M.I.,
experts continue to recommend in adding social media in the 2007. Analyzing the impact of Usability on software
prototype for active collaboration and networking. This design. Journal of Systems and Software 80 (9),
study can also be extended to assess the usability levels of 1506e1516.
other Learning Management system beside UPM and [17]. Pituch, K.A., & Lee, Y.K. (2006). The influence of
outside Malaysia. However there is great need to employ system characteristics on e-learning use. Computers &
other users such as lecturers, administrators to participate in Education, 47, 222–244.
the study. [18]. Krejcie, R. V., & Morgan, D. W. (1970). Determining
sample sizes for research activities. Educational and
REFERENCE Psychological Measurement, 607-610.
[19]. Naidu, S. (2006). E-learning a guidebook of principles:
[1]. Abdullah, R., & Wei, K. T. (2008). Usability Procedures and practices (2nd reviseded.). New Delhi,
Measurement of Malaysia Online News Websites. India: Commonwealth Educational Media Center for
International Journal of Computer Science and Asia (CEMCA).
Network Security, 8(5), 159-165. [20]. Nielsen J (nd) Usability 101: Introduction to Usability
[2]. Andersson, A. & Grönlund, Å. (2009) A Conceptual Available at:http://www.useit.com/alertbox/
Framework for E-Learning in Developing Countries: A 20030825.html (accessed 8 September 2012). See also
Critical Review Of Research Challenges. The http://www.nngroup.com/topic/web-usability.
Electronic Journal on Information Systems in [21]. Islam, A., & Tsuji, K. (2011).Evaluation of Usage of
Developing Countries, 38, 8, 1-16. University Websites in Bangladesh. DESIDOC Journal
[3]. Ariola, M. (2006). Principles and methods of research, of Library & Information Technology, 31, 468-478.
slovin's formula, (7), 140-141.

IJISRT23JUL458 www.ijisrt.com 823

Volume 8, Issue 7, July – 2023 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
[22]. Munguatosha, G.M., Muyinda, P B. & Lubega, J.T.
(2011). A Social Networked Learning
[23]. Adoption Model for Higher Education Institutions in
Developing Countries. On the Horizon, 19, 4, 307-320.
[24]. Ssekakubo, G., Suleman, H. & Marsden, G. (2011).
Issues of Adoption: Have E-Learning Management
Systems Fulfilled their Potential in Developing
Countries? In Proceedings of the South African
Institute of Computer Scientists and Information
Technologists Conference on Knowledge, Innovation
and Leadership in a Diverse, Multidisciplinary
Environment (pp. 231–238). Cape Town, South
African
[25]. Unwin, T., Kleessen, B., Hollow, D., Williams, J.,
Oloo, L. M., Alwala, J., Mutimucuio, I.Eduardo, F. and
Muianga, X. (2010). Digital Learning Management
Systems in Africa: Myths and Realities. Open
Learning: The Journal of Open and Distance
Learning, 25, 1, 5-23
[26]. Wang, Y.S., Wang, H.Y., & Shee, D.Y. (2007).
Measuring e-learning systems success in an
organizational context: Scale development and
validation. Computers in Human Behavior, 23, 1792–
1808.

IJISRT23JUL458 www.ijisrt.com 824