Using Student Feedback and Professor-Developed Mul

See discussions, stats, and author profiles for this publication at: https://www.researchgate.
net/publication/287695651
Using student feedback and professor-developed multimedia to improve

instructor presence and student learning
Article in Journal of Teaching and Learning with Technology · November 2014

DOI: 10.14434/jotlt.v3n2.12990
CITATION READS
1 144
4 authors, including:
William Albert Young Teresa Franklin

Ohio University Ohio University
63 PUBLICATIONS 428 CITATIONS 70 PUBLICATIONS 567 CITATIONS
SEE PROFILE SEE PROFILE
Some of the authors of this publication are also working on these related projects:
Amazon Echo and AI View project
Importance of the microbial food web in the Great Lakes View project
All content following this page was uploaded by William Albert Young on 24 March 2016.
The user has requested enhancement of the downloaded file.

Journal of Teaching and Learning with Technology, Vol. 3, No. 2, December 2014, pp. 12 – 30.
doi: 10.14434/jotlt.v3n2.12990
Using student feedback and professor-developed multimedia to

improve instructor presence and student learning
William A. Young II1, Brett H. Hicks2, Danielle Villa-Lobos3, and Teresa J. Franklin4
Abstract: This paper explores the use of Professor-Developed Multimedia

Content (PDMC) in online, distance education to build a community of inquiry
(CoI) through enhanced social presence and real-time, student-driven, adaption
of the learning content. The foundation of higher education has long been,
developing curriculum to meet educational objectives. Most often faculty relies on
assessment information gained at the end of each course. Then assessments,
formative and summative, are re-designed based on student feedback/data from
end of course surveys and educational materials such as textbooks, articles, and
test banks are updated with newer editions. In the distance-learning environment,
PDMC provides a creative, innovative, and interactive ways to engage the student
for real-time learning. Still, the ability to target PDMC materials to the correct
sub-sections of our classroom cohort can produce a richer, more immerse
learning experience and perhaps become the closet recreation of in-seat,
traditional classroom learning in a distance/online environment. By using PDMC
with corresponding surveys, educators can obtain real-time data and metrics to
alter content in the classroom immediately, and develop media content welcoming
sub-sets of learners with desired content based on learning needs, desires, and
feedback.
Keywords: community of inquiry, social, cognitive, and teaching presence,

distance education
Introduction
Based on Dewey’s work (1933; 1967), Garrison, Anderson, and Archer (2000)
introduced Community of Inquiry (CoI) as a framework for collaboration and learning
transactions in computer-mediated higher-education environments. Strongly based in
constructivism, CoI presents presences (social, cognitive and teaching) used to identify learner
and educator behaviors that are widely accepted as central to productive online learning.
Garrison (2007) discusses the power of social, cognitive, and teaching presences in an
online community of inquiry. In addition, Garrison examines some of the early challenges in
creating and maintaining social, cognitive, and teaching presence in an online community of
inquiry as well as the methodological validity associated with the CoI framework. Garrison
refers back to the work of Dewey (1933), which suggests some of the elements of cognitive
presence for learning. The authors of this paper focus on the CoI elements, to create exploration
1
Management, College of Business, Ohio University, Athens, Ohio 45701, youngw1@ohio.edu
2
Department of Social and Public Health, Ohio University, Athens, Ohio 45701
3
Department of Social and Public Health, Ohio University, Athens, Ohio 45701
4
Educational Studies, Patton College of Education, Ohio University, Athens, Ohio 45701
Young, W., Hicks, B., Villa-Lobos, D., & Franklin, T.
and integration, as well as risk-free expression and information exchange (student to

professor/professor to student) for connecting and applying new ideas in an online course.
Using Professor-Developed Multimedia Content (PDMC) along with end of content
surveys, professors can collect data to create a dashboard of student learning experiences. This
dashboard helps professors modify existing content near-real-time; change content; offer content
in a different form (e.g., step-by-step modeling, lecture material, or interactive
practice/application); and target, through time stamps, announcements and triggers to students
about new content available in the classroom. The authors believe that student feedback and
PDMC greatly enhance student learning by providing a dynamic environment; one where
students have control over their learning experience, and are more involved through increased
engagement and investment in the learning process. In addition, by using end-of-content surveys,
instructors can reduce the influence of irrelevant information as well as present a mastery-
approach orientation in an online, graduate course (Adams, et al., 2008; Crippen, Biesinger,
Muis, & Orgill, 2009).
In this paper, the authors provide examples of how teaching presence though PDMC and
student surveys improved student learning outcomes (cognitive presence), as well as increased
student engagement and feelings of connectedness (social presence) to course concepts and
learning objectives. The authors present (a) examples, (b) student feedback, and (c) data analysis
demonstrating the positive influence of PDMC on students’ learning experiences.
Literature Review
The focus of the research presented in this paper is CoI and PDMC. Literature older than
2000 or conducted exclusively on traditional, classroom-based learning was excluded from the
review of literature. With one exception, research older than 2000 was allowed strictly for
reinforcement, background, or historical context of learning research methods and findings. Any
studies researching blended learning, focused specifically on the in-person aspect were
eliminated from consideration because of the author’s belief that PDMC is most effective in
predominately remote teaching arrangements where students are geographically dispersed with
minimal, if any, face-to-face interaction with educators. In addition, by eliminating this aspect of
blended courses, this paper encourages low-bandwidth, and ADA compliant PDMC for
engagement of a larger audience of the student cohort.
Defining Distance Education and Online Learning
Distance education has come to be known by many names, “These include distance
learning, open learning, networked learning, flexible learning, distributed learning, independent
study, learning in connected space and, today, on-line learning is common (Tracey & Richey,
2005, p. 17).” Tabata, and Johnsrud (2008, p. 26) identify distance education as a method that
“uses technology to deliver instruction and learning freed from the geographical and time
constraints associated with face-to-face instruction.”
Kiryakova (2009, p. 29) suggests, “distance education is a form of education in which the
participants in educational process – teacher and learners are physically separated and
communicate by different means and at different times.” This later was incorporated into the
work by Moore’s (2013, p. 68) Transactional Theory in which “transaction in distance education
Journal of Teaching and Learning with Technology, Vol. 3, No. 2, December 2014. 13
jotlt.indiana.edu
is the interplay of teachers and learners in environments that have the special characteristic of
their being spatially separate from one another.”
Online learning readily fits this definition of distance education as the courses use the
Internet as the classroom, all activities including content and collaboration are online, teachers
and student discuss and interact with each other and the content without being in the same place
and same time (Paulsen, 2002). With the increased number of online courses, programs and
degrees, many now consider online learning as a modified version of distance education
(Benson, 2002). This paper focuses on online learning occurring at a distance, in which a course
designed using PDMC is delivering content using a theoretical framework of community of
inquiry in an attempt to meet the needs of students separated from the instructor through distance
and time.
Benefits to Online Learning
There are many reasons why students prefer online courses as these courses have the
advantage of flexibility of learning on a 24/7 schedule, providing diverse learning environments,
communicating with instructors frequently and outside of office hours, and allowing work and
education to co-exist (Vansickle, 2003; Farzaneh, 2011). Large class size can also be used in
online learning to allow for economy of scale when teaching entry level or survey types of
courses (Kiryakova, 2009). One study indicates that online learning “is a suitable means for
learners who have a preference of doing individual work rather than collaborative work, and a
good way to improve critical thinking as well as reducing peer distraction” (Lei & Gupta, 2010).
Li and Irby (2008) note that shy students and limited English speaking students are more able to
participate in discussion online. The lack of face-to-face contact can be a motivator for these
students and reduce the fear of engaging in discussion in face-of-face classes allowing them to
collaborate with their peers and build collaboration skills. Salmon (2004, p. 18) suggests,
“Although many people find the lack of visual clues strange, messages are ‘neutral’ since you
cannot see whether the sender is young or old nor need to consider their appearance or race.”
Students preferring individual work as opposed to collaborative work may also gravitate toward
online learning (Lei & Gupta, 2010).
Community of Inquiry (CoI)
To support the development of a sense of community, being part of a group and

belonging, higher education has become interested in the levels of interaction between and
among students as a means of increasing learning. In an examination of community and
belonging, Garrison et al. (2000) developed the Community of Inquiry framework which has
evolved to focus on cognitive, social, and teaching presence (2007) for building that sense of
belonging in online learning environments. Kupczynski, Ice, Wiesenmayer, and McCluskey
(2010, p. 23) state CoI is “a theoretical framework that explains the online learning experience in
terms of interactions between three overlapping presences: Teaching, Social, and Cognitive.”
Alman, Frey and Tomer (2012) and Annand (2011) state in their research that these three
presences (cognitive, social, teaching) overlap and interact providing a better understanding for
the learning online. Swan and Shih (2005) note a strong association with teaching presence and
social presence and the perceived learning in an online course. Kang, Liew, Kim, & Jung (2011)
in studying the three elements of CoI determined that a “high level of perceived presence should
jotlt.indiana.edu
lead to successful learning.” Nyachae (2011, p. 21) adds “The Community of Inquiry (COI)
framework is a model that shows the process and theory of research behind online learning and
instruction.” Educators can take advantage of the CoI by through course design with
implementation of collaborative activities to build a community of learning that increases student
learning (Jinks, 2009). Also, Jinks (2009, p. 31) suggests that “social presence and teaching
presence are more critical for establishing, supporting, and enhancing the educational
experience.”
Cognitive, Social, and Teaching Presence
Cognitive presence is often defined as “the extent to which online learners are able to
construct meaning and critical thinking through sustained communication” (Ke, 2010, p. 809).
Garrison (2009, p. 355) in his later work defined cognitive presence as “A process of practical
inquiry distinguished by discourse and reflection for the purpose of constructing meaning and
confirming understanding.” Garrison added that assuring appropriate course design and
facilitating activities and discussion could play a role in influencing cognitive presence
Jinks (2009, p. 30) supported this with, “The cognitive presence of a student may be
affected by their peers’ interactions (social presence) or may be affected by the design or
facilitation of the course (teaching presence).”
Social presence and teaching presence are essential concepts in online courses (Picciano,
2002). Social presence is the more frequently of the presences compared with teaching and
cognitive presence. Social presence has a strong impact on student learning outcomes and
satisfaction (Noteboom & Claywell, 2010). The success of an online course and the quality of
the learning experience by a student can be influenced by social presence and increase
collaboration allowing a student to feel connected to others (Sung & Mayer, 2012).
Teaching presence is often stated as, “the ability of a teacher or teachers to support and
enhance social and cognitive presence through instructional management, building
understanding, and direct instruction (Dunlap & Lowenthal, 2009, p. 133).” “Teaching presence
includes subject matter expertise and the design, management and facilitation of learning
(Greyling & Wentzel, 2007, p. 656).” Bangert (2008, p. 40) identifies teaching presence “as the
‘methods’ that instructors use to create quality online instructional experiences that support and
sustain productive communities of inquiry.”. There is a strong correlation between students’
satisfaction and instructors’ presence in providing clear expectations, timely responses, and
engagement (Jackson, Jones, & Rodriguez, 2010) .
The role of the instructor in the instructional design and organization, facilitating
discourse, feedback and evaluation, building the course materials, planning for individual and
group activities, planning the timeframe, and guiding students through net etiquette and
technology use, can build student satisfaction and connection to the content (Anderson, Rourke,
Garrison, & Archer, 2001). "When discussing the benefits of teaching presence, a study by
(Mayne & Wu, 2011) suggested that teaching presence compared with the two other elements of
community of inquiry, social and cognitive presence, is more powerful and its existence has an
influence in facilitating interaction among students (p. 57).” Teacher presence is a better
predictor of the perceived interaction in a course than social presence as teacher presence
explained twice the variance in the outcome variable in a study by Swan & Shih (2005).
While outside factors such as course completers, online, non-completes, and student
demographics carry tremendous influence on student success, this paper focuses on engagement
jotlt.indiana.edu
when the student is ‘present’ and not whether PDMC could be a sole determinant of student
success, but a contributor to a larger process of engagement (Garrison D. , 2007; Traver,
Volchok, Bidjerano, & Shea, 2014).
Student Satisfaction
Student satisfaction is the centerpiece to effective education. This is especially important

in a distance-learning environment because student satisfaction is influenced by relationships and
academic success (Maceli, Fogliasso, & Baack, 2011). An additional concern for online
education is the increased opportunity for students to feel disconnected and isolated from other
members of their cohort and their educator. PDMC helps reduce student isolation by providing
dynamic content through instructional videos, live and recording professor lectures, and how-to
demonstration videos. PDMC increases learner-content interaction which has been found to be
the “. . . largest unique variance in student satisfaction (Kuo, Walker, Belland, & Schroder, 2013,
p. 16).”
Student learning styles vary greatly, therefore creative and interactive methods of
teaching are needed in order to increase theoretical, affective, and perceptual skills required in an
educational program (McDonough & Osterbrink, 2005; Brannan, White, & Bezanson, 2008;
Melrose, 2004; Rassool & Rawaf, 2007; Rothgeb, 2008). Today’s adult learners are digital
natives born after 1980. For these learners, technology has become a lifestyle (Hawranik &
Thorpe, 2008). These learners embrace technology, and as such, distance education should
emulate this environment by providing dynamic, high fidelity content that allows students to
apply theoretical knowledge in real time (Curtin & Dupuis, 2008; Fountain & Alfred, 2009).
Online multimedia
Students believe that multimedia content enhances learning in distance-based courses,

especially when content is linked to specific course needs (Chrisfield, Cosgrove, & Stinson,
2000). PDMC is most effective at building students’ skill using a Do-It-Yourself (DIY) or
follow-along format because it provides the appropriate conditions to understand, remember, and
apply instructions (Brinkmanc, Buil, Cullen, Gobits, & Van Nes, 2001). DIY PMDC is more
effective when instructions are short and clear with four-to-five steps per completion step, which
helps memory when switching between instructional PDMC and the actual task (i.e.,
constructing a gains and loss, financial spreadsheet) required (Brinkmanc, Buil, Cullen, Gobits,
& Van Nes, 2001).
PDMC provides formative evaluation processes to distance education which allows the “
… systematic collection of information for the purpose of informing decisions to design …” thus
creating product improvements (Flagg, 1990, pp. 1-2). PDMC using DIY content provides
gradual improvement of students’ design from rounds of evaluation and feedback (steps in the
process, a ‘completed’ example from the instructor, and grading feedback); in other words, a
heuristic evaluation (Nielsen, 2005). Through this process, PDMC is vital to the establishing of
experiential learning theory where “Knowledge results from the combination of grasping
experience and transforming it (Kolb, 1984, p. 41).”
Coupling PDMC with end of content surveys increases the students’ and the instructor’s
control over content and allows content to be altered within days of feedback being received.
Chen, Moore, & Vo (2012) discussed the need to provide students with incentives to complete
jotlt.indiana.edu
surveys through the awarding of extra points, but that new methods of incentive may be needed
to improve the amount of and criticalness of student feedback. For this study, all student
responses were anonymous, thus challenging the validity of awarding points for responses as
these may skew results creating false-positive responses.
In addition, PDMC helps online learning move towards embracing synchronous
communication, virtual reality, and mobility (Salmon, 2004; Dirckinck-Holmfeld, 2002). Finally,
the move towards mobility may be the single-most important priority for distance education
because hand-held instructional media provides incentives to online students to increase their
willingness to engage in learning activities through method and media effect, without adding
extraneous points to established, tested, and proven grading rubrics and assessments (Sung &
Mayer, 2013).
Case Study
This case study consists of a course offered as part of an online Master of Business
Administration (OMBA) program from a large, mid-west university located in the United States.
The course content relates to data analysis used in business. Although many topics were covered
in the course, the primary learning outcomes of the course included (a) data management; (b)
data modeling; and (c) business applications of probability and statistics. To study the
effectiveness of PDMC, the results of two course offerings were compared within the 2013-2014
academic year. Both course offerings (111 students in the fall and 83 students in the spring) were
held to the same academic rigor.
Four questions were analyzed for this specific research involving PDMC that pertained to
student satisfaction. These questions were:
1. The course materials effectively stimulated my interest in the subject/content.
2. The course materials (e.g. textbook, readings, website links, etc.) enhanced my
learning and helped me to achieve the learning outcomes.
3. The narrated lectures and interactive presentations enhanced my learning and helped
me to achieve the learning outcomes.
4. The virtual classroom sessions enhanced my learning and helped me to achieve the
learning outcomes.
The course and designed learning content, including PDMC was initially designed in
TechSmith’s Camtasia® the summer before the 2013-2014 offerings. The PDMC consisted of
(a) voice-over Microsoft PowerPoint®; and (b) voice-over Microsoft Excel® with add-in
features chosen based on the assignment’s learning objective(s). In general, the course’s
instructional delivery method was ‘learning by doing’ or Do-It-Yourself (DIY). Simply stated,
students watch PDMC (videos) and follow along with instructions narrated by the instructor as
they work to solve business-case problems. In addition, the business-case problems were
described in an accompanying course video; a purely descriptive session covering learning
objective(s) without step-by-step instructions. PDMC videos ranged from 10- to 30-minutes
depending on the complexity of the teaching-topic. Because the case study focuses on students in
a distance, Master of Business Administration (MBA) program, it is worth noting that Evans
(2006, p. 32) stated, "[Business students] must learn the skills of the future, not necessarily the
skills of today.” PDMC allows instructors to tailor content ‘today’ and ensure pedagogy methods
teach current, relevant skills for business professionals.
jotlt.indiana.edu
For example, Figure 1 shows an example of PDMC for a module titled “Visualizing
Data.” For this particular activity, students begin with an Excel® spreadsheet consisting of sales
and demographic information related to the customer transactions of blockbuster movies in the
form of DVDs. The students are introduced to the problem at the beginning of the video, where
the learning objectives/outcomes are stated. For this particular example, the learning activity
video walks a student through the process of turning the raw data into a visual dashboard via
Excel® Pivot Tables, Charts, and Slicers. By the end of the video, students walk through several
scenarios in an attempt to understand the raw data through the use of the dashboard created. As a
result of the exercise, students answer questions related to movies purchased by gender or age
demographics.
Figure 1. Example of PDMC
It is important to note that the spring version of the course started with the content that
was used in the fall offering. However, there was one difference with the addition of relevant
embedded code within video files created with TechSmith’s Camtasia® that after completion of
the video-play, the code directed students to a (free) Google Forum® with three end-of-content
questions. The instructor of the PDMC sought to reduce the number of questions in order to
promote more responses from the students in the course (social presence) and to (a) capture the
value of the PDMC instructionally (cognitive presence); (b) reduce extraneous data that might
cause confusion (cognitive presence), and (c) focus the faculty’s efforts on improving student
engagement (teaching presence).
The survey, which is shown in Figure 2, is presented to the students in their web-
browsers after the video is over. The first question states, “This video taught the identified
learning objectives well,” and the second states, “I will be able to apply what I learned from this
video to my profession.” For these two required questions, a linear, 5-point Likert scale was used,
ranging from Strongly Disagree to Strongly Agree. The final optional question was open-ended,
where students could leave additional, free-text comments, which would be the primary source
of information used by the instructor to modify course content in near-real-time.
jotlt.indiana.edu
Figure 2. Feedback Questionnaire
From the survey shown to students at the end of watching a learning activity, PDMC
video, points of data were retrieved from the Google Forum® and examined in a dashboard that
was created in Excel® by the instructor. For example, Google Forum® data was summarized to
provide a summary of what days students watched PDMC videos. This descriptive statistic was
valuable because of the format and schedule of the course. The OMBA program targets working
professionals with at least three years of professional working experience. Therefore, the
students are not full-time students, and take a single class at a time. The semester is broken down
into two terms (i.e., Term A and Term B); each term is seven weeks in duration. In addition, for
the case-study class, there were two, synchronous online virtual sessions offered to students each
week on Thursday and Saturday using Adobe Connect® a web-conferencing application. Based
on this class section (Figure 3), the data indicates that the students actively participated in
PDMC during times recommended by the instructors. In other words, students were asked to
watch all forms of PDMC before coming to the first virtual session a week, which is indicated by
the lowest participation rate average on Thursdays. This metric is valuable to the instructor as a
measure of effective role-modeling (i.e., appropriate use of time and preparation for the course,
students following the instructor’s directions) which could indicate the effectiveness of PDMC as
participation rates on Thursdays should gradually decline as students respond to and instructors
engage in the tailoring of the PDMC to student needs. This presents an indicator to the instructor
jotlt.indiana.edu
for lesson/discussion topics during Virtual Office sessions. Simply stated, as students realize the
value of PDMC in the course, their engagement (viewing) would gradually increase throughout
course weeks.
25%
20%
Daily Percent
15%
10%
5%
0%
Sunday Monday Tuesday Wednesday Thursday Friday Saturday
Percentage 17.66% 17.68% 19.83% 13.38% 9.07% 8.14% 14.23%
Count 983 984 1104 745 505 453 792
Figure 3. Days when Students would watch PDMC
Similar to Figure 3, the survey results can be summarized in terms of what time of day
students watch learning activity videos. This particular information, which is shown in Figure 4,
is useful for faculty as an indicator of when to dedicate certain periods of their day to respond to
questions via e-mail and discussion boards. The instructors planned their interaction around
student-peak times, increasing social presence; a major indictor for developing a CoI.
10.00%
9.00%
8.00%
7.00%
6.00%
Percent
5.00%
4.00%
3.00%
2.00%
1.00%
0.00%
1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11 12
AM AM AM AM AM AM AM AM AM AM AM AM PM PM PM PM PM PM PM PM PM PM PM PM
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
Percent 2.66 1.17 0.52 0.13 0.09 0.29 1.26 1.56 2.12 2.86 3.54 4.35 5.30 5.23 4.81 6.05 6.59 5.26 6.54 7.29 9.16 10.17 8.12 4.92
% % % % % % % % % % % % % % % % % % % % % % %
count 148 65 29 7 5 16 70 87 118 159 197 242 295 291 268 337 367 293 364 406 510 566 452 274
Figure 4. Time of Day when Students would watch PDMC
As noted by the instructor that developed the PDMC, there were two primary questions
of interest, which were displayed to students after the PDMC video was viewed. It should be
noted that the question “This video taught the identified learning objectives well,” from this point
forward will simply be abbreviated as Q:LO in all proceeding figures. In addition, the second
jotlt.indiana.edu
question, “I will be able to apply what I learned from this video to my profession,” will be
abbreviated to Q:PA from this point forward in all graphs. Figure 5, provides an example of a
grouping of required PDMC videos, called modules, which students were expected to complete
in a weeks’ time. The particular module consisted of an overview titled An Overview of
Probability and Statistics II, and specific PDMC videos were provided on (a) correlation; (b)
regression; (c) an analysis of variances (ANOVA); and (d) decision theory and analysis. The
figure summaries the average results of the 5-point Likert scale for each the two primary
questions asked on the student survey. Though none of the PDMC videos rate below a 3.5
average ranking in either category, the results indicated that there is further refinement needed as
to determine whether students do not perceive the PDMC content as useful or that additional
survey tools are needed to create more specificity on the reasons for ‘low-ranking’ viewing of
certain video content.
Therefore, an instructor viewing this data could choose to either do nothing or act on the
information provided by the rankings. Acting on the feedback, the instructor could revise
learning content placing additional emphasis on why the content is relevant to various business
professions. In addition, the instructor could discuss reasons why the course topics are relevant
through other course-methods such as in e-mail, discussion board, or during weekly, virtual
sessions providing reinforcement of the PDMC through teaching presence and/or social
presence.
5.0
Average Rating (0 - 5)
4.0
3.0
2.0
1.0
0.0
An_Overvi Probability
Probability Probability
ew_of_Pro Correlation Decision_ Decision_ _and_Stati Regression Test_of_Va
ANOVA_E ANOVA_E _and_Stati _and_Stati
bability_an _Example_ Analysis_ETree_Exam stics_II_De _Example_ riance_Exa
xample_A xample_B stics_II_A stics_II_Re
d_Statistics A xample_A ple_A cision_Ana A mple_A
NOVA gression
_II lysis
Q:LO 4.2 4.6 4.5 4.6 4.3 4.6 4.4 4.0 4.0 4.1 4.6
Q:PA 3.5 4.1 4.0 4.3 3.9 4.0 4.0 3.7 3.7 3.8 4.2
Count 13 47 44 47 41 30 53 41 45 51 50
Figure 5. A Module’s Required Learning Activities Example
Another unique point to be made of PDMC for this course is the use of required and
optional learning content. The course developer designed required content that students were
expected to watch and developed optional material to challenge students or provide additional
information to students seeking to gain a deeper understanding of course learning objectives.
jotlt.indiana.edu
This supports adult-learning theory by offering students more control over their learning
environment by choosing only course requirements or choosing a more robust experience
supplemented by additional PDMC content.
Figure 6 shows the average results for optional content, which was built in to the same
module as what summarized in Figure 5. Though the sample sizes are low for many of the
videos, over time, instructors could monitor average rankings of PDMC videos, and decide if a
video needs to be moved from an optional library to the required library. Likewise, an instructor
could remove a video from the required library, and move it to optional material, if the PDMC
content’s ratings were less favorable compared to the optional material. In addition, instructors
and even other students could recommend specific optional content to students seeking help with
assignment problems. In terms of Figure 6, the reason why one video (i.e. Decision Tree
Example B) was evaluated more than the other learning activities within the module were
because the learning objectives covered where related to a particular homework assignment.
While simplistic in description, the ability to shift content based on course-specific cohorts offers
a tremendous opportunity to build a learning experience based on student needs, level of
professional experience, engagement in activities, and comfort-level with multimedia content.
5.0
Average Rating (0 -‐ 5)
4.0
3.0
2.0
1.0
0.0
The_For
The_For The_For
Decision Decision Decision Multi_Re mula_Sh The_For
ANOVA_ ANOVA_ ANOVA_ Decision Regressi mula_Sh mula_Sh
_Analysi _Analysi _Analysi gression eet_Mult mula_Sh
Example Example Example _Tree_Ex on_Exam eet_Deci eet_Line
s_Examp s_Examp s_Examp _Exampl iple_Line eet_pVal
_C _D _E ample_B ple_B sion_Ma ar_Regre
le_B le_C le_E e_B ar_Regre ues
king ssion
ssion
Q:LO 4.0 4.3 4.3 4.3 5.0 5.0 4.4 4.7 4.5 5.0 4.5 5.0 4.0
Q:PA 4.0 3.7 3.7 4.0 5.0 5.0 4.0 4.2 4.0 5.0 4.5 5.0 4.0
Count 1 3 3 9 2 1 21 9 2 1 2 1 2
Figure 6. A Module’s Optional Learning Activities Example
The final question of the student survey included an open-ended, free-text question. At
times, the comments left by students were brief, and reflected feelings or emotions instead of
content-related feedback. For example, the following is a sample of reactions left by students:
• Loved it!
• Awesome!
• Eyes Crossed
• I actually stopped the video and then proceeded to the solution on my own. I was
jotlt.indiana.edu
correct, YaY!
• OH MY!!!
• Tricky!
• I like to see more like this.
• Yay the light bulb had gone off!
• Whew!
• Very Strong video!
• More great stuff, can’t wait for more!
Although brief, the sample expressions above provide an instructor the opportunity to
gauge student’s overall reaction and enthusiasm after viewing the PDMC and lay a foundation to
communicate with the student body. If there are reactions that indicate the students had difficulty
with the content, the instructor could communicate in more compassionately when answering
questions via (a) e-mail, (b) discussion forums, or (c) presenting material during weekly, virtual-
sessions. Likewise, if students react more positively to the PDMC content, perhaps an instructor
can communicate additional applications or professional relevance of the content presented.
Again, use of the PDMC supports a more robust experience for students by allowing tailoring of
content to the student experience.
Besides reaction-based feedback, students often left feedback classified as either
favorable, which might indicate to an instructor that he or she has created content that is thought
of as highly useful by students or perhaps less favorable where an instructor needs to take
immediate action. For example, the comments below were taken from survey results and
classified as favorable outcomes, which also provides the instructor positive reinforcement that
his or her PDMC is well received, thus encouraging more development and creativity.
• Very impressive learning activity video! To date, the most useful tools to apply to my
business. Shows how we can tie the other things learned together to create a powerful
business application.
• Very clear demo of how to use a simple strategy with broad applications that can
save loads of time and effort.
• Immediate knowledge for my profession and current position. Thank you!
• Very helpful for business applications. The use of examples and how this can be
applied to business is very useful in the learning process.
• Straightforward, well-spoken, and easy to follow along.
• The yellow round highlighter very helpful.
• Awesome video and techniques! I can't wait to put them to work.
As demonstrated above, the feedback was positive, and could give an instructor clear
evidence that PDMC was well-received, which might suggest that additional content be
developed in a similar manner. However, there were certainly less-favorable comments. These
comments were reviewed and determined to be critical to the instruction and overall experience
by the students in order to improve the learning content made available to students for this class.
For example, students stated:
• It would be helpful to have the slides available outside of the video for learning
activities that ask students to attempt to solve a problem prior to finishing the video.
• Video cuts out after 7 minutes. Ending is not included.
jotlt.indiana.edu
• Please slow down, it is hard to keep up and understand what you are doing. I am
following the steps but don’t know what it all means.
• Not sure if it was just my computer or not, but there was a blue box on the middle of
the screen the entire time.
• This video was better than in the required version
• It would be nice to be able to print off the sides, so we could take notes on them
instead of writing down everything on the sides.
• Not sure video is complete. I think it's missing the full conclusion as well as the
correct graph.
• This should be the first lecture video for the module! It's making so much more sense!
• The formulas are extremely complex and are not practical in my work environment.
• The length of the video was longer than needed for the content that was presented.
• I felt this was very poorly explained. I've taken stats before, and not too long ago, and
I could not follow this.
From an instructor point of view, there are many good points raised in these comments
concerning the development of PDMC. Specific to individual learning of content these
comments provide indictors for specific actions to take with specific videos. This is unlike end of
the course evaluations where feedback can be more general and instructors do not have enough
information to make informed changes to content such that content remains unchanged.
Students are providing direct feedback on the organization of the content as well as its
value to obtaining learning objectives. In some cases, students pointed out technical issues
unknown to the instructor. Depending on the technical error identified, instructors made
necessary changes to create a more effective learning environment for students who may not
begin learning activities because of technical errors. Finally, comments that were made related to
the content not being satisfactory or related to additional materials needed, allowed the
instructors to make these available to the students to improve their learning experience.
Findings
Students were surveyed after the completion of each course offering and were asked twenty-two,
Likert scale questions about course content, and the instructor. The selection of responses ranged
from Strongly Agree (5) to Strongly Disagree (1) with neutral represented by (3) and N/A or no-
response represented by (0). In terms of the analysis, a two sample, one tailed z-test was chosen
for the analysis. As noted, the two sample groups included a fall and a spring offering of a
graduate-level course in an OMBA program, where the fall student body included 111 students
and the spring offering consisted of 83 registered students. For this analysis, the hypothesis (H1)
was that using PDMC feedback produced improved survey scores on the questions asked
specifically about course content. The null hypothesis (H0) was that there would be no difference
in survey scores on course content questions between a course offered with or without PDMC. A
p-value of 0.05 was used to measure statistical value. Before the two-sample, one-tailed z test
was performed; the descriptive statistics were computed and are shown in
jotlt.indiana.edu
Table 1.
Table 1
Descriptive Statistics Before and After PDMC Feedback

Before PDMC Feedback After PDMC Feedback
Question N Mean Mode Stdev. N Mean Mode Stdev.
1 50 3.90 4 1.07 55 4.45 5 0.69
2 47 4.00 4 1.04 54 4.43 5 0.81
3 50 3.96 4 1.05 54 4.57 5 0.69
4 28 3.71 4 1.12 31 4.45 5 0.77
Of the four questions analyzed, each resulted in a p-value less than the pre-determined alpha
value of 0.05. Thus in each case, there is sufficient evidence against the null hypothesis. In other
words, PDMC feedback was used effectively by the instructor to improve the overall satisfaction
of students within the four categories chosen for analysis. A summary of the statistical findings is
shown in
Table 2.
Table 2
z-Test Results
Question z Test Statistic z Critical Statistic (one-tail) P(Z<=z) one-tail
1 3.096 1.645 0.001
2 2.293 1.645 0.011
3 3.472 1.645 0.000
4 2.927 1.645 0.002
Discussion
This data shows a positive correlation among (a) student satisfaction, (b) knowledge
attainment, and (c) overall success in an online learning environment using PDMC. Moreover,
the data shows active student involvement as well was professor-engagement; further proving the
effectiveness of social and cognitive presence (student-to-professor, and professor-to-student),
cognitive (student-to-professor, and professor-to-student) as well as teaching presence
(professor-to-student, and professor-to-media) in creating a CoI. Altering course content based
on student feedback represents a new element of CoI in online learning as professors can
promote content actively (through e-mails or course announcements) and/or passively (reviewing
survey feedback). The addition of passive engagement helps promote students’ sense of
ownership over content and their learning experience; to a nearly 100% customizable learning
content/environment. Furthermore, survey questions can be developed that ask students directly
jotlt.indiana.edu
if the PDMC helped promote a sense of connectedness to their learning experience (the hallmark
of effective CoI creation).
The limitations of this study are the course content (an MBA course) where some of the
PDMC has a direct influence on student success as they view learning activity videos while
actively working on problems. In this particular course, a major emphasis was dedicated to
developing dynamic spreadsheets to analyze and manage business data, as well as investigating
statistical relationships that help managers augment their business decisions.
Another limitation is the strict exclusion criteria used due to the small repository of
research on direct influences of multimedia content on student satisfaction. Additional research,
controlling for outside variables on student satisfaction is needed to make a stronger, direct
connect between PDMC and student satisfaction; however, a statistically significant connection
suggests that PDMC has a strong, positive influence on knowledge attainment and student
satisfaction.
Conclusions
PDMC and end-of-content feedback is useful towards promoting CoI in a distance-

education/online learning environment by providing instructors with times of high-traffic,
specific student feedback about PDMC relevance, and prompt notification of technical errors.
When instructors can respond in high-traffic times, they capture students when they are ready to
learn. Student responses suggest that by providing pointed feedback and direction during these
times there can be increased knowledge attainment by creating a supportive atmosphere for
teaching presence.
Furthermore, instructors know when a specific learning activity needs revision as well as
knowing when learning content needs to be re-classified as required or optional based on the
cohort’s needs and composition. This increases the value of the student’s learning experience as
they recognize in near-real time content changes based on their feedback, providing students
control and customization of their learning environment.
Overall, this strategy enables faculty to identify student needs in order to improve course-
content for future offerings. In many cases, changes can be made in a short period. This explicit
feedback overcomes the limitation of end of the term evaluations, where students often leave
general or low-level feedback, which is difficult for instructions to react to in a timeframe that
improves the learning experience of students leaving feedback.
References
Adams, W. K., Ried, S., Lemaster, R., McKagen, S. B., Perkins, K. K., Dubson, M., & Wiemen,
C. E. (2008). A Study of Educational Simulations Part II - Interface Design. Journal of
Interactive Learning Research, 19(4), 551-577.
Alman, S. W., Frey, B. A., & Tomer, C. (2012). Social and cognitive presence as factors in
learning and student retention: An investigation of the cohort model in an iSchool setting.
Journal of Education for Library & Information Science, 53(4), 290-302.
Anderson, T., Rourke, L., Garrison, D., & Archer, W. (2001). Assessing teaching presence in a
computer conferencing context. Journal of Asynchronous Learning Networks, 5(2), 1-17.
jotlt.indiana.edu
Annand, D. (2011). Social presence within the community of inquiry framework. The
International Review of Research in Open and Distance Learning, 12(5), 40–56.
Bangert, A. (2008). The influence of social presence and teaching presence on the quality of
online critical inquiry. Journal of Computing in Higher Education, 20(1), 34-61. Doi:
10.1007/BF03033431
Benson, A. (2002). Using online learning to meet workforce demand: A case study of
stakeholder influence. Quarterly Review of Distance Education, 3(4).
Brannan, J., White, A., & Bezanson, J. (2008). Simulator effects on cognitive skills and
confidence levels. Journal of Nursing Education, 47(11), 495-500. Doi: 10.3928/01484834-
20081101-01
Brinkmanc, W., Buil, V., Cullen, R., Gobits, R., & Van Nes, F. L. (2001). Design and evaluation
of online multimedia maintenance manuals. Behaviour & Information Technology, 20(1), 47-52.
Doi: 10.1080/01449290010020639
Chen, W., Moore, J. L., & Vo, N. (2012). Formative evaluation with novice designers: Two case
studies within an online multimedia development course. International Journal of Instructional
Media, 39(2), 95-111.
Chrisfield, T., Cosgrove, R., & Stinson, J. (2000). Building scholarly online multimedia
collections and services. The Electronic Library, 18(5), 328-335. Doi:
10.1108/02640470010354581
Crippen, K. J., Biesinger, K. D., Muis, K. R., & Orgill, M. (2009). The Role of Goal Orientation
and Self-Efficacy in Learning from Web-Based Worked Examples. Journal of Interactive
Learning Research, 20(4), 385-403.
Curtin, M. M., & Dupuis, M. D. (2008). Development of human patient simulation programs:
Achieving big results with a small budget. Journal of Nursing Education, 47(11), 522-523. Doi:
10.3928/01484834-20081101-02
Daspit, J., & D'Souza, D. (2012). Using the Community of Inquiry Framework to Introduce Wiki
Environments in Blended-Learning Pedagogies: Evidence From a Business Capstone Course.
Academy of Management Learning & Education, 11(4), 666-683. Doi: 10.5465/amle.2010.0154
Dewey, J. (1933). How we think. Boston, MA: D. C. Heath.
Dewey, J. (1967). Experience and education (7th ed.). New York, NY: Collier.
Dirckinck-Holmfeld, L. (2002). Designing virtual environments based on problem oriented

project pedagogy. In B. Fibiger, Learning in virtual environments (pp. 31-54). Frederiksberg,
Denmark: Samfundslitteratur Press.
jotlt.indiana.edu
Dunlap, J., & Lowenthal, P. (2009). Tweeting the night away: Using twitter to enhance social
presence. Journal of Information Systems Education, 20(2), 129–135.
Evans, P. (2006). The Wiki factor. In BizEd, 28-32.
Farzaneh, J. (2011). Distance education based on educational philosophy, postmodernism. .

Australian Journal of Basic & Applied Sciences, 5(9), 91–100.
Flagg, B. (1990). Formative evaluation for educational technologies. Hillsdale, NJ: : Lawrence
Erlbaum.
Fountain, R. A., & Alfred, D. (2009). Student satisfaction with high-fidelity simulation: Does it
correlate with learning styles? Nursing Education Perspectives, 30(2), 96-98.
Garrison, D. (2007). Online community of inquiry review: Social, cognitive, and teaching
presence issues. Journal of Asynchronous Learning Networks, 11(1), 61–72.
Garrison, D. R. (2007). Online Community of Inquiry Review: Social, Cognitive, and Teaching
Presence Issues. University of Calgary, Teaching and Learning Centre. Calgary: Sloan
Consortium.
Garrison, D. R. (2009). Online Community of Inquiry Review: Social, Cognitive, and Teaching
Presence Issues. Calgary: Sloan Consortium.: University of Calgary, Teaching and Learning
Centre.
Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical inquiry in a text-based
environment: Computer conferencing in higher education. The Internet and Higher Education,
2(2-3), 87-105. Doi: 10.1016/S1096-7516(00)00016-6
Greyling, F., & Wentzel, A. (2007). Humanising education through technology: Creating social
presence in large classes. South African Journal of Higher Education, 21(4), 654–667.
Hawranik, P., & Thorpe, K. (2008). Helping faculty enhance scholarship. Journal of Continuing
Education in Nursing, 39(4), 155-165. Doi: 10.3928/00220124-20080401-02
Jackson, L., Jones, S., & Rodriguez, R. (2010). Faculty actions that result in student satisfaction
in online courses. Journal of Asynchronous Learning Networks, 14(4), 78–96.
Jinks, S. (2009). An examination of teaching presence and the sense of community on perceived
student learning. ProQuest Dissertations and Theses database.
Kang, M., Liew, B., Kim, J., & Jung, H. (2011). Learning Presence as a Predictor of
Achievement and Satisfaction in an Online Learning Environment. . Paper presented at the
World Conference on Educational Multimedia, Hypermedia and Telecommunications
(EDMEDIA) , (pp. 3015-3024). Lisbon, Portugal.
jotlt.indiana.edu
Ke, F. (2010). Examining online teaching, cognitive, and social presence for adult students.
Computers & Education, 55, 808–820. Doi: 10.1016/j.compedu.2010.03.013
Kiryakova, G. (2009). Review of distance education. Trakia Journal of Sciences, 7(3), 29–34.
Kolb, D. (1984). Experiential learning: Experience as the source of learning and development.
Englewood Cliffs, NI: Prentice-Hall.
Kuo, Y.-C., Walker, A. E., Belland, B. R., & Schroder, K. E. (2013). A predictive study of
student satisfaction in online education programs. The International Review of Research in Open
and Distance Learning, 16-39.
Kupczynski, L., Ice, P., Wiesenmayer, R., & McCluskey, F. (2010). Student perceptions of the
relationship between indicators of teaching presence and success in online courses. Journal of
Interactive Online Learning, 9(1), 23–43.
Lei, S. A., & Gupta, R. K. (2010). College distance education courses: Evaluating benefits and
costs from institutional, faculty and students' perspectives. Education, 130(4), 616–631.
Li, C., & Irby, B. (2008). An overview of online education: Attractiveness, benefits, challenges,
concerns and recommendations. College Student Journal, 42(2), 449-458.
Lyons, T., & Evan, M. M. (2013). Blended learning to increase student satisfaction: An
exploratory study. Internet Reference Services Quarterly, 18, 43-53. Doi:
10.1080/10875301.2013.800626
Maceli, K. M., Fogliasso, C. E., & Baack, D. (2011). Differences of students' satisfaction with
college professors: The impact of student gender on satisfaction. Academy of Educational
Leadership Journal, 15(4), 35-45.
Mayne, L., & Wu, Q. (2011). Creating and measuring social presence in online graduate nursing
courses. Nursing Education Perspectives, 32(2), 110–114. Doi: 10.5480/1536-5026-32.2.110
McDonough, J., & Osterbrink, J. (2005). Learning styles: An issue in clinical education?
American Association of Nurse Anesthetists Journal, 73(2), 89-93.
Melrose, S. (2004). What works? A personal account of clinical teaching strategies in nursing.
Education for Health, 17(2), 236-239. Doi: 10.1080/13576280410001711067
Moore, M. G. (2013). The handbook of distance education. Mahwah, NJ. : Lawrence Erlbaum
Associates.
Nielsen, J. (2005). Ten usability heuristics. Retrieved from

http://intra.iam.hva.nl/content/1112/verdieping1/research_for_design/intro-en-materiaal/RfD-
Heuristic-Evaluation.pdf
jotlt.indiana.edu
Noteboom, J., & Claywell, L. (2010). Student Perceptions of Cognitive, Social, and Teaching
Presence. Annual Conference on Distance Teaching & Learning, (pp. 1-4).
Nyachae, J. (2011). The effect of social presence on students' perceived learning and satisfaction
in online courses. ProQuest Dissertations and Theses database.
Paulsen, M. F. (2002). Online education systems: Discussion and definition of terms. NIK
Distance Education.
Picciano, A. (2002). Beyond student perceptions: Issues of interaction, presence, and course.
Journal of Asynchronous Learning, 6(1), 21–33.
Rassool, G., & Rawaf, S. (2007). Learning style preferences of undergraduate nursing students.
Nursing Standard, 21(32), 35-41. Doi: 10.7748/ns2007.04.21.32.35.c4495
Robertson, I. (2008). Learners' attitudes to wiki technology in problem based, blended learning
for vocational teacher education. Australasian Journal of Educational Technology, 24, 425-441.
Rothgeb, M. (2008). Creating a nursing simulation laboratory: A literature review. Journal of

Nursing Education, 47(11), 489-494. Doi: 10.3928/01484834-20081101-06
Salmon, G. (2004). E-moderating: The key to online teaching and learning. London: Routledge.
Stephens, M., & Hennefer, D. (2013). Intertionalising the nursing curriculum using a Community
of Inquiry Framework and blended learning. Nurse Education in Practice, 13, 170-175. Doi:
10.1016/j.nepr.2012.08.010
Sung, E., & Mayer, R. (2012). Five facets of social presence in online distance education.
Computers in Human Behavior, 28(5), 1738–1747. Doi: 10.1016/j.chb.2012.04.014
Sung, E., & Mayer, R. (2013). Online multimedia learning with mobile devices and desktop
computers: An experimental test of Clark's methods-not-media hypothesis. Computers in Human
Behavior, 29(3), 639-647. Doi: 10.1016/j.chb.2012.10.022
Sung, E., & Mayer, R. E. (2013). Online multimedia learning with mobile devices and desktop
computers: An experimental test of Clark's methods-not-media hypothesis. Computers in Human
Behavior, 29(3), 639-647. Doi: 10.1016/j.chb.2012.10.022
Swan, K., & Shih, L. (2005). On the nature and development of social presence in online course
discussions. Journal of Asynchronous Learning Networks, 9(3), 115–136.
Tabata, L., & Johnsrud, L. (2008). The impact of faculty attitudes toward technology, distance
education, and innovation. Research in Higher Education, 49(7), 625-646. Doi: 10.1007/s11162-
008-9094-7
Tracey, M., & Richey, R. (2005). The Evolution of Distance Education. Distance Learning, 2(6),
17-21.
jotlt.indiana.edu
Traver, A. E., Volchok, E., Bidjerano, T., & Shea, P. (2014, January). Correlating community
college students' perceptions of community of inquiry presences with their completion of
blended courses. The Internet & Higher Education, 20, 1-9. Doi: 10.1016/j.iheduc.2013.09.001
Vansickle, J. (2003). Making the transition to teaching online: Strategies and methods for the
first-time, online instructor. . Morehead, KY: Morehead State University.
jotlt.indiana.edu
View publication stats

Using Student Feedback and Professor-Developed Mul

Uploaded by

Copyright:

Available Formats

Using Student Feedback and Professor-Developed Mul

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Using Student Feedback and Professor-Developed Mul

Uploaded by

Copyright:

Available Formats

See discussions, stats, and author profiles for this publication at: https://www.researchgate.

Using student feedback and professor-developed multimedia to improve

Article in Journal of Teaching and Learning with Technology · November 2014

William Albert Young Teresa Franklin

SEE PROFILE SEE PROFILE

Amazon Echo and AI View project

The user has requested enhancement of the downloaded file.

Using student feedback and professor-developed multimedia to

Abstract: This paper explores the use of Professor-Developed Multimedia

Keywords: community of inquiry, social, cognitive, and teaching presence,

and integration, as well as risk-free expression and information exchange (student to

Defining Distance Education and Online Learning

Benefits to Online Learning

Community of Inquiry (CoI)

To support the development of a sense of community, being part of a group and

Cognitive, Social, and Teaching Presence

Student satisfaction is the centerpiece to effective education. This is especially important

Students believe that multimedia content enhances learning in distance-based courses,

Figure 1. Example of PDMC

Figure 2. Feedback Questionnaire

Figure 4. Time of Day when Students would watch PDMC

Figure 5. A Module’s Required Learning Activities Example

Figure 6. A Module’s Optional Learning Activities Example

Descriptive Statistics Before and After PDMC Feedback

PDMC and end-of-content feedback is useful towards promoting CoI in a distance-

Dewey, J. (1933). How we think. Boston, MA: D. C. Heath.

Dirckinck-Holmfeld, L. (2002). Designing virtual environments based on problem oriented

Evans, P. (2006). The Wiki factor. In BizEd, 28-32.

Farzaneh, J. (2011). Distance education based on educational philosophy, postmodernism. .

Nielsen, J. (2005). Ten usability heuristics. Retrieved from

Rothgeb, M. (2008). Creating a nursing simulation laboratory: A literature review. Journal of

View publication stats

You might also like