Student LMS use and satisfaction in academic institutions: The organizational perspective

This article appeared in a journal published by Elsevier. The attached copy is furnished to the author for internal non-commercial research and education use, including for instruction at the authors institution and sharing with colleagues. Other uses, including reproduction and distribution, or selling or licensing copies, or posting to personal, institutional or third party websites are prohibited. In most cases authors are permitted to post their version of the article (e.g. in Word or Tex form) to their personal website or institutional repository. Authors requiring further information regarding Elsevier’s archiving and manuscript policies are encouraged to visit: http://www.elsevier.com/copyright Author's personal copy Internet and Higher Education 13 (2010) 127–133 Contents lists available at ScienceDirect Internet and Higher Education Student LMS use and satisfaction in academic institutions: The organizational perspective Gali Naveh, Dorit Tubin ⁎, Nava Pliskin Department of Education, Ben-Gurion University of the Negev, P.O. Box 653, Beersheva 84105, Israel a r t i c l e i n f o Article history: Accepted 4 February 2010 Keywords: Learning management system (LMS) LMS use Student satisfaction with LMS Research university Organizational factors a b s t r a c t The present paper examines student use of and satisfaction with the Learning Management System (LMS), and how these dependent variables are correlated with organizational variables at one Israeli university. Data on 1212 course websites was gathered in 2007 from the LMS warehouse, the student-management database, the instructor–management database, and satisfaction questionnaires. The findings indicate varied use of LMS, a high level of satisfaction, and low significant correlation between use and satisfaction. As for the organizational variables, course content was found to significantly correlate with use and satisfaction; course size, instructor status and forum existence showed significant correlation with LMS use; and course discipline had low correlation with satisfaction. Further studies and practical implications are discussed. © 2010 Elsevier Inc. All rights reserved. 1. Introduction A learning management system (LMS) is an information technology (IT) used by instructors to easily build and maintain course websites. Website maintenance includes posting course content, updating events, and managing interactive communication with students via messages, forums, and surveys (OECD, 2005). Academic institutions have invested heavily in LMS implementation to support online teaching (Hawkins & Rudy, 2009). To justify the widespread investment in LMS technology, it is important to study patterns of actual student LMS use and student satisfaction with LMS technology (Delone & McLean, 2003; Lonn & Teasley, 2009), as well as the correlating factors. Past research found that students satisfaction with LMS to be correlated with factors as course content (Selim, 2007), perceived usefulness (Sun, Tsai, Finger, Chen, & Yeh, 2008), communication quality and knowledge transmission (Lonn & Teasley, 2009; Malikowski, Thompson & Theis, 2006), as well as student self efficacy, previous achievements and computer literacy (Liaw, 2008; Hong, 2002). Other studies found that student LMS satisfaction correlates with actual use (Liaw, 2008; Levy, 2008), previous student achievements (Hong, 2002) and course dropouts (Sun et al., 2008). Past research, from an organizational perspective, focused mainly on the technology adoption and diffusion (Rogers, 2003; DeLone & McLean, 2003). Czerniewicz and Brown (2009), for example, found that Structured Corporate institutions enable attainment of an elearning critical mass, while Unstructured Collegium institutions are ⁎ Correspondence author. Tel.: + 972 8 6461864; fax: + 972 8 6472897. E-mail addresses: galin@bgu.ac.il (G. Naveh), dorittu@bgu.ac.il (D. Tubin), pliskinn@bgu.ac.il (N. Pliskin). 1096-7516/$ – see front matter © 2010 Elsevier Inc. All rights reserved. doi:10.1016/j.iheduc.2010.02.004 better at fostering innovation. Malikowski and Thomson et al. (2006) found that traditions and norms affect LMS adoption more than course size or level. Organizational factors, such as course discipline, course type — whether mandatory or elective, class size, staff size, instructor status, timing of the course within the study program, have hardly been studied in the context of LMS use and satisfaction. This is surprising since, unlike student motivation and expectations (McGill & Hobbs, 2008) or instructor proficiency (Selim, 2007), such organizational factors can be controlled by the organization. Understanding how organizational factors correlate with LMS use and satisfaction can help the academic institutions achieve higher returns on investment in LMS. The purpose of the current research is to study how organizational variables impact LMS adoption as manifested by student LMS use and satisfaction. Specifically, two research questions are being raised in this study: first, what is the extent of LMS use by students and to what extent are they satisfied with LMS? And, second, to what extent are these (dependent) variables are correlated with organizational (independent) variables? 2. Background 2.1. Learning management systems According to a report by the Organization for Economic Cooperation and Development (OECD, 2005), LMS technology is used by instructors to build and maintain courses. LMS technology features personal communication via email;, group communication via chatting and forums; posting content including syllabus, papers, presentations and lesson summaries; performance evaluation via question and answer repositories, self-assessment tests, Author's personal copy 128 G. Naveh et al. / Internet and Higher Education 13 (2010) 127–133 assignments, quizzes and exams; instruction management via messaging, grade posting and surveys; and more (Clark, 2002). Blackboard's WebCT is the best known LMS around the world but Desire2Learn and the open-sourced Moodle are also widely used. Britannica's High Learn has the highest market share within academic institutions in Israel, where this study took place. Belief in the potential of LMS systems to improve teaching and learning has led to widespread LMS implementation worldwide. Hawkins and Rudy (2009), for instance, report 97.5% LMS diffusion in 994 academic institutions in the US and around the world by 2007, with even higher diffusion at research universities. Yet, Jones, Johnson-Yale, Millermaier and Perez (2008) found that LMS use for academic purposes is not necessarily associated with student satisfaction. Moreover, teaching and learning processes have remained essentially unchanged following LMS implementation, with most LMS-based course websites used for document transfer and posting administrative information or course content (Blin & Munro 2008; Frank & Barzilai 2004; Malikowski et al., 2006; McGill & Hobbs, 2008). Studies about LMS effectiveness reveal some success stories featuring high student performance, teaching redesign, student satisfaction, monetary savings, and dropout prevention (Bonk, 2004; Joint Information Systems Committee InfoNet, 2008). However, given the widespread LMS implementation, on one hand, and the fact that many students and faculty make only limited formal academic use of LMS (Selwyn, 2007), on the other hand, it is important to understand what constitutes LMS success. 2.2. LMS success IT researchers use two main variables for comparative analysis of IT success: 1) use and 2) user satisfaction (Seddon, Staples, Patnayakuni, & Bowtell, 1999; DeLone & McLean, 2003). Most LMS research, however, has yielded mixed findings about these two variables. A study investigating LMS use by 424 students in Taiwan, for instance, found that efficiency and satisfaction contribute to intention to use (Liaw, 2008). Sun and Tsai et al. (2008), showed a reverse relationship whereby perceived usefulness and ease of use impact satisfaction. Another study, by Hong (2002), found that time invested in a course has no impact on satisfaction, but past experience with computers has a positive impact on satisfaction. Lonn and Teasley (2009) reported that LMS attitudes and preferences are consistent with student LMS use. In most LMS research, so far, use and satisfaction appear as mediating variables in the analysis of LMS impacts and benefits for individuals and organization (Hong, 2002; Liaw, 2008; Lonn & Teasley, 2009). However, considering student satisfaction as an indicator of LMS success and as the ultimate target of LMS use is sensible for several reasons. First, past research (e.g., Arbaugh et al., 2009) showed that LMS use supports, rather than modifies, existing teaching and learning approaches. Thus, it makes sense to redefine LMS success and shift indications of success away from achieving a pedagogic revolution (Harasim, 2000) toward improving student satisfaction. Second, satisfaction is defined as the ability of a service or a product to address customer needs (Smith, Heindel, & Torres-Ayala, 2008). Even if student needs are not fully known, it is reasonable to assume that high student satisfaction is indicative of success in the sense of whether the LMS responds well to their needs (Seddon et al., 1999; DeLone & McLean, 2003). Finally, according to Institutional Theory (Hoy & Miskel, 2001), the expectations of stakeholders in the organizational environment other than students should be taken into account when success is measured. Some of these stakeholders, like IT vendors (Selwyn, 2007), have expectations for student satisfaction with LMS. This study thus considers as dependent variables: student LMS use and student satisfaction with LMS, being the main indicators of LMS success. Since organizational factors are critical success factors in technology adoption and diffusion (Rogers, 2003), the question is which organizational variables are relevant for understanding LMS success due to correlations with use and satisfaction. 2.3. Organizational variables Organizational variables are derived from the organizational structure which can be defined as “the sum total of the ways in which it divides its labor into distinct tasks and then achieves coordination among them” (Mintzberg, 1979, p.2), or as “patterned or regularized aspects of relationships existing among participants in an organization” (Scott, 2003, p.18). The division of labor and pattern of coordination in the academic organizations have been molded for many years as a result of a complex interaction among social, economic, and cultural forces. Despite wide diversity, due for example to location and different history, it is possible to identify several common organizational characteristics that shape and influence teaching and learning processes in academic institutions all over the world (Altbach, 1997). Three such organizational characteristics are particularly of relevance for studying LMS use and student satisfaction with LMS. The first organizational characteristic relevant to LMS use and satisfaction reflects the role definition and the departmental division as derived from orientation of the academic institution according to discipline or goal (Fairweather, 2000). A disciplineoriented institution may emphasize such disciples as arts, business, engineering or natural sciences. A goal-oriented institution may emphasize such goals as scientific research (stressing funded research and graduate research at the doctorate and master levels), academic undergraduate teaching at the college level (stressing bachelor and professional–master education), or academic preparatory and vocational teaching at the communitycollege level (stressing professional two-year programs). The goal orientation of the academic institution affects the two organizational variables: instructor status and course discipline. Instructor status, tenure-track or adjunct, is related to whether the instructor is engaged predominantly in research or in teaching. For tenure-track instructors, research is the major mission and the quality of teaching counts less in recruiting and promotion considerations (Braxton, Eimers & Bayer, 1996). Adjuncts, however, are employed based on their teaching performance. Institutional preference for instructor status depends on discipline and goal orientation, with exact (health, engineering and natural) sciences and research universities preferring tenure-track status. Past research (e.g., Bland et al., 2006; Fairweather, 2005; Mayhew & Grunwald, 2006) found that the instructor status variable affects the various teaching activities and products, such as the time devoted to interacting with students. Although some researchers claim there is no conflict necessarily between research and teaching (Elsen, Visser-Wijnveen, van der Rijst, & van Driel, 2009), it is reasonable to expect attitudes toward teaching in general and LMS technology in particular to vary as a function of instructor status. Course discipline (e.g., humanities, social, management, health, engineering, or natural sciences) influences the teaching style. Smith and Heindel et al. (2008) found that teaching styles in exact sciences courses are different than in management and social sciences courses. In addition, Roca, Chiu and Martinez (2006) found that students in engineering and natural sciences are more computer literate and are able to make better use of IT tools than their peers in arts and social sciences. It is thus reasonable to expect student LMS use and student satisfaction with LMS to vary as a function of course discipline. The second organizational characteristic relevant to LMS use and satisfaction reflects the commitment to hierarchical degree structures and accreditation processes, as derived from the coordinator mechanisms of academic institutions. Graduate academic Author's personal copy G. Naveh et al. / Internet and Higher Education 13 (2010) 127–133 institutions stress conducting research and grooming students to become researchers, while undergraduate institutions emphasize teaching and grooming students to become professionals (Elsen et al., 2009). All academic institutions award academic credit points for studied courses, with comprehensive efforts taken in recent years to promote a uniform credit system and allow student transfer between institutions, as in the case of the Bologna declaration. Launched in 1999, the Bologna declaration was signed by Ministers from 29 European countries who met in Bologna to establish the necessary steps required to create a European Higher Education Area (EHEA) by 2010. The EHEA thrives to remove obstacles to student mobility across Europe, enhance the attractiveness of European higher education worldwide, establish a common structure of higher education systems across Europe, and promote the European system of higher education worldwide (Johnsrud, 1993; Kyvik, Karseth, & Blume, 1999). The commitment to hierarchical degree structures and accreditation processes affects three organizational variables: course year, course size, and staff size. Mandatory introductory courses, given mainly in the first years, are taught to large groups of students and are run by an instructor and a large team of teaching assistants. Trow (1998) suggested that a large course size should motivate instructors to harness IT to help cope with managing the complexities of a large class. In later years, students take specific elective courses which are given by a small staff to small groups of students and may reflect the instructor's area of research. Hong (2002) found that the experience that students accumulate over time influences their attitudes toward teaching and they tend to express more satisfaction with using course websites in later years of study than in early years. The third organizational characteristic relevant to LMS use and satisfaction reflects the regularized aspects of relationships among roles (Scott, 2003) as derived from role autonomy due to academic freedom. The academic freedom affects universities since the establishment of the Humboldt Universität in Berlin at 1810 (Altbach, 2001), allowing researchers to conduct research and teach free of any constrains regarding research areas as well as course content and teaching style. Yet, institutional expectations also create some isomorphism among courses by imitating prestige models or following organizational norms (DiMaggio & Powell, 1983). MIT's OpenCourseWare, for example, with its consortium of 100 universities, 3000 courses and 2 million visitors, serves as such a model by setting some standards for course content (Abelson, 2008). Organizational norms can affect the teaching style via an authoritative managerial decision (Rogers, 2003). For example, applying critical success factors as top management support and involvement, the head of an academic department made an authoritative decision to implement LMS, encouraging LMS use and increasing the number of course websites from 10% before the decision to about 100% after the decision (Naveh, Tubin, & Pliskin, 2006). Thus, although researchers found that ways instructors use LMS depend to a large extent on their LMS perceptions (Nachmias & Ram, 2009), organizational expectations and norms can also play a role. The regularized aspects of relationships among roles affect two organizational variables: content on the course website and existence of forums on the course website. Course content refers to the number of posted items (learning materials as papers, syllabus, and lesson summaries), excluding messages, that students may download (Malikowski, 2008). The existence of forums or surveys on the course website, which reflects an interactive teaching style, refers to whether forums or surveys exist on the course website or not (Phillips et al., 2007). Past studies showed the importance of organizational aspects in the design of academic teaching and, as further explained in Section 3, this study investigates the effect of seven independent variables, instructor status, course discipline, course year, course size, 129 staff size, content on the course website, and existence of forums on the course website, on LMS use and student satisfaction with LMS. 2.4. Research questions Due to complex effects and interactions between the organizational variables and the LMS use and satisfaction, as evident by diverse findings in the past, no hypotheses were raised in advance regarding the correlations among the variables. Rather, two research questions have been raised in this exploratory study: 1) To what extent the LMS is used by students and to what extent students are satisfied with LMS at the studied university? 2) To what extent student LMS use and satisfaction (dependent variables) are correlated with instructor status, course discipline, course year, course size, staff size, content on the course website, and existence of forums on the course website (independent variables). 3. Method 3.1. Setting The study took place at one of seven research universities in Israel, referred to hereinafter as “The University”. Catering to about 17,000 students at the time of data collection, The University has implemented the High Learn LMS several years ago. Like similar tools available on the market, the functionality of this LMS includes message exchange via email, forums, surveys, and bulletin board; posts of syllabi, readings, lecture notes, presentations, and videos in the knowledge base; links to other knowledge resources; and performance evaluation via quizzes and surveys. The diffusion of the High Learn LMS at The University varies among disciplines, ranging from very low to almost 100% (Naveh et al., 2006). Instructors are not obliged to build course websites. Yet, in case they do so, The University strongly encourages them to use the LMS by offering LMS training and help services on a regular basis. 3.2. Variables Section 2.2 presented the two dependent variables (Table 1). To assess Dependent Variable #1, LMS use, the average number of accesses to a course website, per student and per item, served as a proxy. To assess Dependent Variable #2, student satisfaction with LMS, data were collected via a survey (elaborated upon in Section 3.3.2), about the contribution of the whole course website, as well as of each LMS function to student satisfaction, averaging the answers. Seven independent variables, presented in Section 2.3, are listed in Table 1. Independent Variable #1, course size, was reflected by the number of course students (continuous variable). Independent Variable #2, staff size, was reflected by the number of members on the course staff (continuous variable). Independent Variable #3, instructor status, was reflected by whether the course instructor is on a tenure track (coded as 1) or an adjunct (coded as 0). Independent Variable #4, course year, was reflected by whether the course is given in the first year of the study program (coded as 1) or above (coded as 0). Independent Variable #5, course discipline, was reflected by whether the course is taught at the faculty of exact sciences (health, engineering and natural — coded as 1) or not (humanities, social, and management — coded as 0). Independent Variable #6, course teaching materials, was reflected by the number of posted items. Independent Variable #7, existence of forums/surveys, was reflected by whether forums/surveys exist on the course website or not. Author's personal copy 130 G. Naveh et al. / Internet and Higher Education 13 (2010) 127–133 Table 1 Variables, measures and data sources. Variable Measure Data source Dependent variables 1. LMS use 2. Student satisfaction with the LMS PowerData Student satisfaction survey Independent organizational variables 1. 2. 3. 4. 5. Average number of accesses per student and per item Average of answers to questions in the student satisfaction survey Number of course students Size of the course staff Tenure-track or adjunct First or above Exact sciences (health, engineering and natural) or other Number of posted items (learning materials excluding messages) Exist or not Course size Staff size Instructor status Course year Course discipline 6. Content posted on the course website 7. Existence of forums/surveys on the course website 3.3. Data collection Data about the variables (Table 1) were collected from online sources and via a survey, in the second semester of 2007. During the studied semester, 1212 websites were active on HLMS, covering 40% of The University's courses. 3.3.1. Online sources PowerData, the data warehouse of High Learn, was the source of data about: Dependent Variable #1 — LMS use, Independent Variable #1 — course size, Independent Variable #2 - staff size, Independent Variable #6 — content posted on the course website, and Independent Variable #7 — existence of forums/surveys. The student–management database, underlying the student–management information system, was the source of data about Independent Variable #4 — course year, and Independent Variable #5 — course discipline. The instructor–management database, underlying the instructor– management information system, was the source of data about Independent Variable #3 — instructor status. 3.3.2. Survey Student satisfaction was surveyed via a questionnaire designed especially for the study to collect data about Dependent Variable #2 — student satisfaction with the LMS. A preliminary version of the questionnaire was tested in face-to-face interviews with six students. The resulting questionnaire, with 18 statement questions, whose answers are on a Likert Scale (ranging from 1 — strongly disagree, to 5 — strongly agree), was distributed by email to 100 students in 4 courses. The questionnaire was validated via factor analysis, with Oblimin with Kaiser Normalization, resulting in factors consistent with its components. The final questionnaire was distributed by mail to all students. Most students were asked to fill more than one questionnaire, depending on the number of courses with websites that they were registered to during the studied semester, with the course title appearing on top. Due to anonymity, however, there is no data about the number of questionnaires each respondent filled. Of the 63,739 questionnaire copies distributed by mail, 10,583 (17%) were returned. Filled questionnaires were deemed inadequate and eliminated from data analysis if 1) only one respondent per course responded, 2) less than half of the questions were responded to, or 3) the inter-class correlation was under 0.7 (Lindell, Brandt & Whitney, 1999). This elimination scheme left 8245 (13%) adequate responses about 819 (68%) course websites. The number of respondents per course website was between 2 and 77 (10.7 average and 11.94 standard deviation). 4. Results This study was led by two main assumptions. The first assumption is that LMS use and satisfaction are desired outcomes of LMS PowerData PowerData Instructor–management database Student–management database Student–management database PowerData PowerData implementation, and thus served as dependent variables in this study. The second assumption is that organizational variables correlate with the LMS use and satisfaction outcomes, and thus served as independent variables in this study. The results below organized according to the research questions. Section 4.1 is devoted to Research Question 1, providing descriptive statistics for the two dependent variables, student LMS use and satisfaction. After presenting descriptive statistics for the seven independent variables in Section 4.2, Section 4.3 is devoted to the second research question, presenting findings regarding the relationships (correlations) between the dependent and the independent variables. 4.1. Dependent variables 4.1.1. LMS use During the studied semester, 40% of the University's courses featured websites created on the LMS platform. 75 of the websites belonged to the faculty of natural sciences, 173 to the school of management, 237 to the faculty of health sciences, 316 to the faculty of engineering sciences, and 411 to the faculty of arts and social sciences. It is noteworthy that the faculty of natural sciences, the school of management and the faculty of health sciences are of similar size and the faculty of engineering sciences and the faculty of arts and social sciences are larger. The number of items posted on a website varied between 0 and 438, with an average of 45 items. The average number of accesses per student per item in the knowledge base ranged between 0 accesses and 5.8 accesses, averaging 0.42 accesses. The percentage of students posting messages on forums ranged from 0% to 100%, averaging 17.5%. These findings point to a great diversity, with some course websites being not active at all or hardly active, while some websites are being highly active, with hundreds of posted items in the knowledge bases and with many students accessing these items. 4.1.2. Satisfaction with LMS The answers to survey statements averaged between 3.1 and 4.3, with a standard deviation between 1.2 and 1.6 (Table 2). The questions in Table 2 are ordered from the highest to the lowest average answer, with the three statements topping in terms of average answers dealing with the contribution of website to the course, the wishes that other courses would have a website, and the wishes that other courses would post teaching materials (presentations, abstracts, papers, etc.) on the website. It is noteworthy that the average answers regarding forums and surveys (Questions 11 and 14 to 18) were relatively low, but somewhat higher for potential (3.8 average for Questions 11 and 14) than for actual use (3.8 average for Questions 15 to 18), probably because actual use of forums and survey was rather low. Author's personal copy 131 G. Naveh et al. / Internet and Higher Education 13 (2010) 127–133 Table 2 Survey questions: respondents, average and standard deviation. Question Respondents Average Standard deviation 1. The website contributed to the course 2. I wish my other courses had a website 3. I wish other courses would post teaching materials (presentations, abstracts, papers, etc.) on the website 4. Posting teaching materials (presentations, abstracts, papers, etc.) on the website contributes to the course 5. I recommend that in my other courses messages would be posted on the website 6. I am pleased with the course website 7. Posting message on the website contributes to the course 8. I am pleased from posting teaching materials (presentations, abstracts, papers, etc.) on the course website 9. I am pleased with message posting on the course website 10. I am pleased that the course has a website 11. I would recommend making use of forums in websites of other courses 12. In my opinion, every course needs a website 13. I am pleased with posting messages on the course website 14. I would recommend making use of surveys in websites of other courses 15. I am satisfied with the use of forums in the course website 16. I wish there would be more use of forums in the course website 17. I wish there would be more use of surveys in the course website 18. I am pleased with the use of surveys in the course website 8327 8326 8293 4.3 4.3 4.3 1.2 1.2 1.2 8278 4.1 1.3 8291 4.1 1.3 8358 8277 4.1 4.0 1.3 1.4 8316 3.9 1.4 8340 3.9 1.4 8273 1468 3.8 3.8 1.4 1.4 8297 3.8 1.4 8304 3.8 1.4 50 3.8 1.2 1472 3.4 1.5 8024 3.3 1.6 7883 3.1 1.5 50 3.1 1.5 4.2.7. Existence of forums/surveys At least one forum was active in 120 (10%) course websites, with an average of two forums per website. Only eight (0.7%) courses ran surveys and hence surveys were omitted from further analysis. Most forums (77%) served pedagogical (learning) purposes only and 23% served for administrative or social purposes as well. In the faculty of engineering 19% of course websites had forums, in management as well as in arts and social sciences 10% of websites had forums, and in health and natural sciences 2% of websites had forums. 4.3. Correlations 4.2. Independent variables 4.2.1. Course size Course size varied between 2 to 329 students, averaging 46. 26% of the courses were small-sized (20 students or less), 67% were medium-sized (21–100 students) and 7% were large-sized (more than 100 students). 4.2.2. Staff size Staff size varied between 1 and 18 staff members. A staff size of 1 characterized 42% of the courses. In most of the courses (88%) the staff size was small (staff size of 3 or less). In 11% of the courses the staff size was between 4 and 10 and in 0.5% of the courses the staff size was greater than 10. 4.2.3. Instructor status 45% of instructors were senior (tenure track) and 55% were junior (adjuncts). 4.2.4. Course year 50% of the courses were first-year courses and 50% were taught later than in the first year. 4.2.5. Course discipline 53% of the courses belonged to the exact science category (health, engineering and natural faculties) and 47% to the non-exact science category (the faculty of arts and social sciences and the school of management). 4.2.6. Course content The number of posted items ranged from 0 items per course website (empty knowledge base) to 438 items, averaging 45.4 items. The correlations between the (organizational) independent variables and the dependent variables, LMS use and overall student satisfaction with course websites, are displayed in Table 3. Evidently, staff size and course year are correlated with neither dependent variables. Course size and instructor status are correlated with LMS use but not with student satisfaction. The relationship of course discipline with either dependent variable is rather weak, with students in exact sciences slightly less satisfied with course websites compared to their counterparts in non-exact sciences (p = 0.01, β = −0.096). Finally, LMS use is correlated with forum existence and both LMS use and satisfaction are correlated with the number of items in the knowledge base of the website. To understand more thoroughly the relationships between the independent variables, the correlation matrix is displayed in Table 4 for most independent variables but course discipline, for which the link with either dependent variable is rather weak. All correlations between course content and other variables were significant, with most being positive except for instructor status, possibly because adjuncts tended to post more content on their course websites than tenure-track instructors. The correlations between instructor status and other variables were also significant, except for course size. There was less use of websites in courses taught by tenured-track instructors who, according to the other correlations, taught fewer first-year courses with fewer members on the course staff compared to their adjunct counterparts, posted less content, and opened fewer forums. This finding is supported by the positive correlation between staff size and course year, according to which first-year courses were taught by larger teams than courses in later years. It is also noteworthy that forum existence was significantly and positively correlated with staff size, course content, and instructor status. In other words, in courses with forums, students were satisfied when the number of course staff members was high, the number of students was low, and the instructor was an adjunct. Adjuncts and their teams posted more content and created more forms which perhaps encouraged LMS use and promoted student satisfaction with LMS. Finally, there was a significant positive but low correlation between use and satisfaction (p = 0.001, β = 0.18). Given the significant correlations between LMS use, student satisfaction and course content (Table 3), it is reasonable to assume that rich content perhaps increased use, satisfaction, and the correlation between them. Table 3 Use and satisfaction versus independent variables. Independent variables LMS use Student satisfaction Course size Staff size Instructor status Course year Course discipline Course content Forum existence 0.14⁎⁎ −0.023 − 0.113⁎ 0.008 − 0.062 0.025 − 0.058 − 0.096⁎ 0.195⁎⁎ 0.063 ⁎ p b 0.05. ⁎⁎ p b 0.01. − 0.023 −0.021 0.179⁎⁎ 0.235⁎⁎ Author's personal copy 132 G. Naveh et al. / Internet and Higher Education 13 (2010) 127–133 Table 4 Correlations between independent variables. Course size Staff size Instructor status Course year Course content Forum existence Staff size Instructor status Course year Course content Forum existence 0.17 − 0.008 − 0.390⁎⁎ 0.014 0.235⁎⁎ −0.153⁎⁎ 0.183⁎⁎ 0.170⁎⁎ − 0.156⁎⁎ 0.080⁎ − 0.004 0.141⁎⁎ −0.150⁎⁎ 0.053 0.377⁎⁎ ⁎ p b 0.05. ⁎⁎ p b 0.01. To deepen the understanding of the relationships between forum existence and other variables, we also considered under the use variable — the average number of responses to a student in a forum, and under the satisfaction variable — the satisfaction specifically with forums. The results (Fig. 1) revealed several interesting findings. First, the use of forums was related to course characteristics: more forum messages were posted by students who took first-year courses (compared to more advanced ones), or courses with few students (compared to courses with many students). Second, satisfaction was related to course discipline: students in exact sciences were less satisfied with their course websites in general and with forums in particular. Third, satisfaction specifically with forums was related to instructor status and to staff size in the same way that satisfaction in general was related to both. Finally, there was no significant relationship between the use of and satisfaction specifically with forums. 4. Discussion and conclusions This study investigated the use of LMS and student satisfaction with LMS and their relationship with relevant organizational variables. In general, students were satisfied with the status quo under which most course websites included content and messages Fig. 1. Use and satisfaction with sites and forums — correlations mapping. and a minority (10%) incorporated forums. Based on the findings reported in the previous section, it is possible to come up with the following insights. First, in the studied semester, 40% of the courses at The University featured websites. At the same time, students hoped for more websites in courses. The partial LMS diffusion might be related to the unique characteristics of each discipline. Smith and Heindel et al. (2008), for example, found that instructors in mathematics and in natural-science course used LMS functions (e.g., testing and polling tools) more often than their counterparts in social sciences and humanity courses. Against these observed differences, it is noteworthy that appropriate policies and management practices can influence LMS diffusion. Making an authoritative managerial decision to implement an LMS at one academic unit, for example, and securing top management involvement and support and a leading champion, is the most plausible explanation to an increase in the percentage of courses with websites from 10% in 2002 to 98% in 2005 (Naveh et al., 2006). Thus, as Czerniewicz and Brown (2009) confirmed, an affirmative university policy can lead to expanded LMS use as well as to increased student satisfaction with LMS. The partial LMS diffusion might also be related to the size of the course staff. It is noteworthy that 42% of the courses at The University were taught by one instructor (staff size = 1) and, perhaps, building and maintaining a course website, with LMS-aided, is too much for one instructor. Further research is needed for confirm that this is indeed the case. Second, computer literacy (indicated by course year and discipline) does not enhance student satisfaction. In fact, as found in other studies (Levy, 2008; Liaw, 2008; Sun et al., 2008), computer-literate students were dissatisfied when their high technological expectations for a high-quality friendly and easy-to-use system remained unmet. To promote student satisfaction with course websites LMS friendliness must be insured. In addition to diminished satisfaction, an unfriendly LMS might lead computer-literate faculty and students to creation of more convenient alternative course websites. Third, the finding that course content was the most significant organizational factor in relation to student satisfaction, found by other researchers as well (Malikowski, 2008; Sun et al. 2008), highlights the importance of course websites in support of conventional teaching and the student awareness of its contribution to academic learning. Similarly, forums promoted use and satisfaction, especially among first-year students. A plausible explanation for this finding is that, perhaps, forums serve more as platforms for updates and answers to questions, the need for which is highest in the first year, and less for promoting interactive learning processes. These findings reveal that the expectations that IT would revolutionize teaching and learning and lead to significant changes in instructor–student relations (Harasim, 2000), are yet to materialize. In fact, instructors can maintain their conservative teaching habits except for posting their course content on the website. From an organizational perspective, this can be done at low cost, yielding relatively high student satisfaction. Further study however, is needed for find what qualities of the course content are perceived to be most valuable by the students. Despite its wide scope, this study suffers from limitations that call for further research. The data collected did not allow full tracking of actual student activity at the course web site. In addition, the study focused on one particular LMS and left out websites created either without an LMS or by using another one. Thus, the findings do not provide insights to the reasons why the studied LMS is not universally adopted by all, insights that could perhaps shed light on what needs to be done to improve the situation. Finally, the consideration of one case study puts limits on the ability to generalize. This study however, has practical implications for decision makers at higher education institutions. Since LMS use and satisfaction among first-year students was high, there is return on the investment in course websites for them. Building course websites is also recommended in crowded (large-sized) courses, as suggested by Trow Author's personal copy G. Naveh et al. / Internet and Higher Education 13 (2010) 127–133 (1998). In addition, to increase student LMS use and satisfaction with LMS, instructors are advised to post rich content on course websites. To help instructors to do so, a coordinated action by the academic institution is recommended, including top management support (Naveh et al., 2006), creation of a technical-support unit, and offering training not only with regard to LMS technicalities but also with regard to organizing the course website and its content. Such action is likely not only to increase student satisfaction (Nachmias & Ram, 2009), but also to enhance the institution's reputation as advanced and as implementing state-of-the art technology. References Abelson, H. (2008). OpenCourseWare at MIT. Journal of Science Education and Technology, 17(2), 164−174. Altbach, P. G. (1997, Fall). An international academic crisis? The American professoriate in comparative perspective. Daedalus, 100−110. Altbach, P. G. (2001). Academic freedom: International realities and challenges. Higher Education, 41(1–2), 205−209. Arbaugh, J. B., Godfrey, M. R., Johnson, M., Pollack, B. L., Niendorf, B., & Wresch, W. (2009). Research in online and blended learning in the business disciplines: Key findings and possible future directions. Internet and Higher Education, 12, 71−87. Bland, C. J., Center, B. A., Finstad, D. A., Risbey, K. R., & Staples, J. (2006). The impact of appointment type on the productivity and commitment of full-time faculty in research and doctoral institutions. The Journal of Higher Education, 77, 89−123. Blin, F., & Munro, M. (2008). Why hasn't technology disrupted academics' teaching practice? Understanding resistance to change through the lens of activity theory. Computer and Education, 50, 475−490. Bonk, C. J. (2004). The Perfect E-Storm: Emerging technologies, enormous learner demand, enhanced pedagogy, and erased budgets. London: UK: The Observatory on Borderless Higher Education. Braxton, J. M., Eimers, M. T., & Bayer, A. E. (1996). The implications of teaching norms for the improvement of undergraduate education. Journal of Higher Education, 67, 603−625. Clark, J. (2002). A product review of WebCT. The Internet and Higher Education, 5, 79−82. Czerniewicz, L., & Brown, C. (2009). A study of the relationship between institutional policy, organizational culture and e-learning use in four South African universities. Computers & Education, 53(1), 121−131. DeLone, W. H., & McLean, E. R. (2003). The DeLone and McLean model of information system success: A ten-year update. Journal of Management Information Systems, 19, 9−30. DiMaggio, P. J., & Powell, W. W. (1983). The iron cage revisited: Isomorphism and collective rationality in organizational fields. American Sociological Review, 48, 147−160. Elsen, M., Visser-Wijnveen, G. J., van der Rijst, R. M., & van Driel, J. H. (2009). How to strengthen the connection between research and teaching in undergraduate university education. Higher Education Quarterly, 63(1), 64−85. Fairweather, J. S. (2000). Diversification or homogenization: How markets and governments combine to shape American higher education. Higher Education Policy, 13, 79−98. Fairweather, J. S. (2005). Beyond the rhetoric: trends in the relative value of teaching and research in faculty salaries. The Journal of Higher Education, 76, 401−422. Frank, M., & Barzilai, A. (2004). Designing course web sites for supporting lecture-based courses in higher education — Some pedagogical aspects. International Journal of Instructional Technology and Distance Learning, 1(12). Harasim, L. (2000). Shift happens: Online education as a new paradigm in learning. The Internet and Higher Education, 3, 41−61. Hawkins, B.L. & Rudy, J.A. (2009). Fiscal Year 2007 Summary report, EDUCAUSE core data service, May 11, 2009, retrieved October 14, 2009 http://net.educause.edu/ir/ library/pdf/PUB8005.pdf Hong, K. S. (2002). Relationships between students' and instructional variables with satisfaction and learning from a Web-based course. Internet and Higher Education, 5, 267−281. 133 Hoy, W., & Miskel, C. G. (2001). Educational administration. McGraw Hill: U.S.A. Johnsrud, L. K. (1993). Cross-cultural implications of graduate study abroad: The case of Korean academics. Higher Education, 25(2), 207−222. Joint Information Systems Committee InfoNet, 2008. Exploring tangible benefits of elearning, Retrieved on August 11, 2008 form: http://www.jiscinfonet.ac.uk/ publications/camel-tangible-benefits.pdf Jones, S., Johnson-Yale, C., Millermaier, S., & Perez, F. S. (2008). Academic work, the Internet and U.S. college students. The. Internet and Higher Education, 11(3), 165−177. Kyvik, S., Karseth, B., & Blume, S. (1999). International mobility among Nordic doctoral students. Higher Education, 38(4), 379−400. Levy, Y. (2008). An empirical development of critical value factors (CVF) of online learning activities: An application of activity theory and cognitive value theory. Computers & Education, 51, 1664−1675. Liaw, S. (2008). Investigating students' perceived satisfaction, behavioral intention, and effectiveness of e-learning: A case study of the Blackboard system. Computers & Education, 51(2), 864−873. Lindell, M. K., Brandt, C. J., & Whitney, D. J. (1999). A Revised Index of Interrater Agreement for Multi-Item Ratings of a Single Target. Applied Psychological Measurement, 23, 127−135. Lonn, S., & Teasley, S. D. (2009). Saving time or innovating practice: Investigating perceptions and uses of learning management systems. Computers & Education, 53(3), 686−694. Malikowski, S. R. (2008). Factors related to breadth of use in course management systems. The Internet and Higher Education, 11(2), 81−86. Malikowski, S. R., Thompson, M. E., & Theis, J. G. (2006). External factors associated with adopting a cms in resident college courses. The Internet and Higher Education, 9, 163−174. Mayhew, M. J., & Grunwald, H. E. (2006). Factors contributing to faculty incorporation of diversity- related course content. The Journal of Higher Education, 77, 148−168. McGill, T. J., & Hobbs, V. J. (2008). How students and instructors using a virtual learning environment perceive the fit between technology and task. Journal of Computer Assisted Learning, 24, 191−202. Mintzberg, H. (1979). The structuring of organizations. Englewood Cliffs, NJ: Prentice-Hall. Nachmias, R., & Ram, J. (2009). Research insights from a decade of campus-wide implementation of web-supported academic instruction at Tel Aviv University. The International Review of Research in Open and Distance Learning, 10(2). Naveh, G., Tubin, D., & Pliskin, N. (2006). Websites for every department course. Campus-Wide Information Systems, 23(2), 68−75. Organization for Economic Co-operation and Development (OECD) (2005). E-learning in tertiary education, policy brief, Dec. 2005, pp 1–8. [Online], retrieved July 14, 2006, http://www.oecd.org/dataoecd/55/25/35961132.pdf Phillips, P., Wells, J., Ice, P., Curtis, R., & Kennedy, R. (2007). A case study of the relationship between socio-epistemological teaching orientations and instructor perceptions of pedagogy in online environments. Electronic Journal for the Integration of Technology in Education, 6. Roca, J. C., Chiu, C., & Martinez, J. (2006). Understanding e-learning continuance intention: An extension of the technology acceptance model. Homan-Computer Studies, 64, 683−696. Rogers, E. M. (2003). Diffusion of innovation, 5th ed . New York: Free Press. Scott, W. R. (2003). Organizations — Rational, natural, and open system, 5th ed. New Jersey: Prentice Hall. Seddon, P. B, Staples, D. S., Patnayakuni, R., & Bowtell, M. J. (1999). The dimensions of information systems success.Communications of the Association for Information Systems, 2, 20 [online]. Retrieved on July 31, 2006 from http://cais.isworld.org/ articles/2-20/article.pdf Selim, H. M. (2007). Critical success factors for e-learning acceptance: ConWrmatory factor models. Computers & Education, 49, 396−413. Selwyn, N. (2007). The use of computer technology in university teaching and learning: A critical perspective. Journal of Computer Assisted Learning, 23, 83−94. Smith, G. G., Heindel, A. J., & Torres-Ayala, A. T. (2008). E-learning commodity or community: Disciplinary differences between online courses. The Internet and Higher Education, 11(3), 152−159. Sun, P., Tsai, J. R., Finger, G., Chen, Y., & Yeh, D. (2008). What drives a successful eLearning? An empirical investigation of the critical factors influencing learner satisfaction. Computers & Education, 50(4), 1183−1202. Trow, M. (1998). The Dearing report: A transatlantic view. Higher Education Quarterly, 93−117.