POLYTECHNIC UNIVERSITY OF THE PHILIPPINES

A Critique on a Research Article

Instructional Design, Active Learning, and Student Performance:

Using a Trading Room to Teach Strategy

By

Alice C. Stewart, Susan M. Houghton

and Patrick R. Rogers

Presented to
Dr. Dennis O. Dumrique
Polytechnic University of the Philippines

In Partial Fulfillment
of the Requirements for the Course
Curriculum Theories, Principle, and
Instructional Designs

By

Dinnes A. Masubay

October 2016

RESEARCH PROBLEMS
 Leading advocates for teaching and learning in higher education (e.g., Fink,
2003; Whetten, 2007) promise that careful attention to integrated course design
may lead to significant learning experiences. Although the ideas presented by Fink
and others (e.g., Barr & Tagg, 1995) are all relevant nowadays, the researchers
(Stewart et. Al., 2012) believed that it is critical to document the impact of new
instructional design practices for several reasons. The reasons are as follows:

 The students in this generation, known as the millennials, present

challenging issues for educators.

 The institutional environment for higher education is evolving rapidly and

facing competitive pressures from alternative educational models.

 Finally, some portion of faculty will always require strong evidence to

overcome inertia associated with course redesign.

With these provisions, the researchers ask on how will students in course
sections incorporating Fink’s integrated instructional design perform better on a
standardized comprehensive assessment exam than those in a traditionally
designed course?

The research focused on Fink’s (2003) model that describes how significant
learning experiences are created through integrated instructional design. This
model of instructional design has three interrelated components: (a) learning goals,
(b) teaching and learning activities, and (c) feedback and assessment. These
integrated instructional design elements are in service to taxonomy of types of
significant learning experiences (Fink, 2003).

RESEARCH QUESTIONS

One key principle of integrated instructional design is similar to a core

concept stated by Collins and Porras (2000) in the popular management book “Built
to Last”. Simply stated, one must “start with the end in mind.” In this case, the “end
in mind” is creating significant learning experiences for students and creating them
consistently over time and across offered sections.

With this, the researchers (Stewart et. Al., 2012) believed that active
learning, supported by the proper instructional design can lead to significant
learning experiences. Fink (2003) presented survey evidence suggesting that
student respondents report improved learning experiences. Seaton and Boyd (2008)
argue for the use of active learning, specifically simulations in business courses, as
design elements that would create significant learning experiences.

In addition, they see that undergraduate strategy courses should be able to

demonstrate improvement in measurable learning outcomes in circumstances
where Fink’s instructional design model is applied to active learning events within
the classroom. This line of reasoning leads to their hypothesis that students in
strategy courses based on integrated instructional design principles and using
active learning techniques will demonstrate higher performance on end-of-semester
assessments of foundational knowledge than students in courses that do not use
these techniques.

IMPORTANCE OF THE RESEARCH

Active learning using elements of integrated instructional design may be one

way to incorporate consistent, hands-on experiences and maximize their impact on
undergraduate student learning. “The single biggest improvement most teachers
can make is to give students more “doing” and “observing” experiences related to
the subject of the course. Direct experiences are the most powerful” (Fink, 2003, p.
123).

To support this, it said that learning is a consequence of thinking (Perkins,

1992, p.8). Retention, understanding, and active use of knowledge can be brought
about only by learning experiences in which learners think about and think with
what they are learning. The new strand on education today is to let the learners
active and responsible in their own learning. This practice was introduced long
before John Dewey founded progressive education. He envisioned a child-centered
education that took account of children’s interests and abilities (Dewey in Perkins,
1992, p.9)

With these provisions, it can be seen that the research is important for the following
reasons:

 With active learning, students will be given opportunities to be more responsible

with their own learning. They will be given freedom to reflect on what they are
doing in school to make them more motivated in studying. This strategy also
promises retention of knowledge so it will help them to get higher academic
achievements. There is also an active use of knowledge that will encourage
students to be more creative in doing tasks.

 With this research, teachers will have new perspective in their involvement in
teaching-learning process. They will become more aware on the academic
achievements of their students and plan for a more enjoyable and authentic
learning experiences.

 Traditionally, the principal leads the school much as the captain of a ship
commands the crew. Contemporary lessons from the business and school
communities alike suggest that a strongly hierarchical, non-participatory
process of governance misses opportunities. Significant teachers, parents, and
indeed student’s participation in school governance can boost motivation and
 involvement and harvest everyone’s intelligence toward the good of the
enterprise. It means that the school needs to foster a thoughtful involvement
not just for students in their classrooms but for the adults committed to the
school as well. With this research the school administration will become more
aware to energize the whole school community to establish a thoughtful
environment among its members.

Furthermore, since the research was done in a business related subject, it is

also goal of the research to encourage students to gain a deeper foundational
understanding and apply knowledge from a variety of business courses. Students
access their business knowledge from many disciplinary areas and use it within
strategy frameworks to practice analysis and decision-making skills. Thus, although
concepts specific to strategy are indicative of learning, the course is also where
wide variety of foundational knowledge domains in business are reinforced.

METHODS AND PROCEDURES USED

The research used conditions to test the impact of active learning in the
context of integrated instructional design. The control condition was a traditional
approach to teaching an undergraduate strategy capstone class. The intervention
condition was an undergraduate strategy capstone class that was designed based
on Fink’s integrated instructional design principles and that incorporated an active
learning element to teach students financial analysis. As part of the instructional
design, the intervention representing active learning content was a set of financial
trading room assignments.

In this research, the dependent variable was operationalized as performance

on the Educational Testing Service (ETS) Major Field Test in Business. ETS Major Field
Tests are comprehensive assessments designed to measure the core knowledge and
understanding by students in a major field of study. The undergraduate MFT in
Business is a standardized test used by many accredited schools of business in the
United States for program assessment. The Major Field Test (MFT) is an objective
multiple-choice test covering the core areas of business knowledge. Scores are
reported for each individual. Scores are reported as an overall score, and specific
scores for nine disciplinary areas are provided. The MFT was administered at the
very end of the semester and was not directly related to any of previous active
learning elements of the course. The MFT was done as part of the ongoing
assessment activities of the School done at the end of the capstone course.
Consistent with best practices, this assurance of learning activity was embedded in
the course and was a component of the total student grade. The dependent variable
was measured in two ways. The first dependent variable (DV1) was calculated as
the percentage of correct answers on the MFT for each student. This measure was
used to represent the broad domain of topics covered in the context of the course.
The second dependent variable (DV2) was measured by calculating the scores
(percentage of correct answers) for the finance and accounting areas of the MFT.
DV2 is considered to be a more fine-grained dependent variable. These two
disciplinary areas were selected for separate analysis because the active learning
events in the course were so closely tailored to the reinforcement of foundational
knowledge and application in these two disciplinary areas.

The intervention was operationalized by whether a student was a member of

the control section or a member of one of the strategy sections with the integrated
instructional design intervention. Assignment to the intervention was indicated by a
categorical variable where 0 indicated the control group and 1 indicated the
integrated instructional design condition.

Data were collected from all sections at the end of the semester as part of
the standard assessment activities of the School of Business and Economics. Data
on GPA, Major, and student demographics were collected. In addition to testing
students on general business knowledge, students were also surveyed regarding
their perception of the value of learning from the class as well as their perceptions
regarding the quality of teaching in their section.

SAMPLE USED IN THE STUDY

To test the hypothesis and to examine the impact of instructional design and
active learning further, the researchers (Stewart et. Al., 2012) used a
nonrandomized convenience sample of 114 undergraduate strategy students across
four sections of an undergraduate capstone class. All students enrolled in the four
course sections were included in the study. Of the 114 students, 112 provided
complete data for the study, and these 112 students were the subjects used in the
study. The strategy course was the capstone class for all majors in a School of
Business and Economics at an AACSB-accredited institution in the southeastern
United States. The strategy classes used in the study were offered in the same
semester (spring), and the 112 students were spread across four course sections.

The three faculty members teaching these sections have terminal degrees in
the area of strategic management awarded from research-intensive institutions. The
three faculty members each have at least 20 years of teaching experience in
AACSB-accredited institutions and experience in teaching undergraduate, graduate,
and Ph.D. courses. Two instructors taught one section and a third professor taught
two sections of the strategy capstone course. Two professors (responsible for three
sections) revised the capstone strategy course based on integrative instructional
design principles described by Fink (2003). Although they did not team-teach, the
two faculty designers coordinated classroom activities and content coverage. The
third faculty member taught one section of the capstone course using traditional
teaching methodology and coordinated content coverage with the other two
sections.

RELIABILITY AND VALIDITY OF THE INSTRUMENT USED

As mentioned earlier, the dependent variable was operationalized as

performance on the Educational Testing Service (ETS) Major Field Test in Business.

ETS Major Field Tests are described as comprehensive assessments designed

to measure the core knowledge and understanding by students in a major field of
study.

It was cleared in the preview discussions that the Major Field Test (MFT) is an
objective multiple-choice test covering the core areas of business knowledge.
Scores are reported for each individual. The MFT was administered at the very end
of the semester and was not directly related to any of previous active learning
elements of the course.

There were no parts in the research that the validity of the instrument used.
It is actually unnecessary since the researchers (Stewart et. Al., 2012) used an
standardized test administered in majority of the universities and colleges in US.

Furthermore, establishing validity involves examining the logical relationships

that should exist between assessment measures. The researchers expect that the
students in intervention condition will get the higher score than to those in the
control condition. That was exactly what they found out they just used the
instrument to validate their assumptions.

TYPE OF RESEARCH

The research methodology used a quasi-experimental design with two

conditions. A quasi-experimental design is similar to a traditional experimental
design except that assignment to the experimental and control conditions are not
randomized (Cochran, 1983). According to Aussems, Boomsma, and Snijders (2011),
a study is labeled quasi-experimental if two conditions hold. The first is that a
researcher is interested in the effect of an intervention on one or more responses. . .
. The second condition is that an intervention effect is investigated by comparing
groups, which means that control or referent groups are used for the assessment of
a treatment effect. The treatment and control groups are formed or identified before
the treatment is imposed, and it is assumed that external influences are affecting
the groups to the same extent during the experimental period. Furthermore, no
reference is made to the use of randomization in allocating subjects to treatment
conditions. (p. 22)
 In this research, the control condition was a traditional pedagogical approach
to teach an undergraduate strategy capstone class. The intervention condition was
an undergraduate strategy capstone class that was redesigned based on Fink’s
(2003) integrated instructional design principles and that incorporated an
integrated active learning element to teach students financial analysis.

HOW WAS THE DATA ANALYZED?

The hypothesis suggested that the emphasis on integrated instructional

design will have an impact on student learning. The researchers (Stewart et. Al.,
2012) argued that emphasis on the design elements would improve overall
performance on the MFT. To test this hypothesis, they applied a hierarchical multiple
regression analysis and entered the control variables separately in the model prior
to entering the dummy variable representing the intervention. The impact of the
control variables was significant for gender and GPA. Student major did not have a
significant impact on overall student learning. The results for gender suggested that
the male students performed better on the MFT than the female students. As
expected, GPA also was an important predictor of performance on the major field
test. Higher GPAs were associated with better performance on the MFT. After control
variables were entered in the model, the impact of the course design intervention
demonstrated that performance overall on the MFT did not differ across the
sections. There was no significant change in the adjusted R2. This result suggested
that when examining overall business knowledge, there seemed to be a lack of
support for the hypothesis that integrated instructional design improved student
learning outcomes across all business knowledge domains.

To test the Finance and Accounting Scores (DV2) Part of the reason for
including the trading room as the primary active learning component of the
integrative instructional design was to enhance the students’ use and analysis of
financial and accounting information. They decided to examine the impact of this
active learning component more closely. Because the active learning emphasis was
on financial and accounting application in the trading room, they expected that the
intervention might improve performance in this specific knowledge domain. A
subset of questions on the MFT that were associated with accounting and financial
topics was used to examine the impact of the active learning component of the
course (the trading room) on foundational knowledge in finance and accounting.

MAJOR FINDINGS

The results of this research were mixed. On one hand, the emphasis on
integrated instructional design may not have had the measurable impact on
broader student learning that Fink (2003) describes. On the other hand, when the
researchers (Stewart et. Al., 2012) examined the component of the instructional
design that was most closely aligned with the hands-on, active learning component,
a noticeable measurable impact was found.

The results led the authors to conclude that a key element in the integrated
instructional design process may be the type of teaching activities that are chosen.
If this is a valid conclusion, there are significant implications for teaching the
strategy capstone course to millennial students. Unlike studies that found no
relationship between active learning and student outcomes, this study, which
embedded active learning in a larger instructional design context, seemed to find an
effect on student learning outcomes. However, these outcomes seemed to be most
closely related to the knowledge domain of the active learning exercise. These
results suggest that greater emphasis on incorporating targeted active learning may
substantially add to student learning. The capstone strategy class has always been
considered a transition point for business students. However, it must be stressed
that students were not just introduced to the trading room and left to their own
devices. The trading room experience was carefully organized and linked to other
elements in the strategy course. Students were assigned specific exercises with
reflective elements. Students were given feedback on the quality of their analysis
based on the activities in the trading room. Instructional strategies were devised
and implemented, and the sequencing of the course was such that the trading room
featured prominently in more than one element of the class. In short, the professors
created a purposeful experiential component and coordinated their implementation
both within and across the three sections of the course where the trading room was
used.

The most exciting result from this purposeful design was the additional
impact that this activity had over and above the impact of GPA. In 21st century
higher education there is greater emphasis than ever before on teaching students of
all academic backgrounds and levels of talent. Traditional pedagogy, which may
have been well suited to academically talented students, may not work as well for
the average student who now enrolls in college. More active learning and more
attention to the nuts and bolts of design elements in the classroom may be the way
that faculty must respond to this challenge.

Developing assignments that are completed collectively in class or computer

labs with students working together to acquire data and apply it to business
problems in real time is the basis for improved student learning. The environment in
higher education is a challenging one. The characteristics of the millennial student
and the expectations of society regarding the role and responsibility of higher
education demand a response from faculty. Testing new techniques in
undergraduate programs can be a significant contribution to the growing body of
scholarship associated with the teaching and learning role of the Academy.
SUGGESTIONS

It was emphasized that it was the faculty’s first attempt at purposefully

incorporating Fink’s (2003) instructional design principles. Thus, the impact of the
design could become stronger as faculty become more experienced at designing
and implementing this approach to developing the course. It is my thinking that the
researchers must guarantee and prove that the professors teaching in intervention
condition were actually using the Fink’s integrated instructional design. They may
provide checklist or series of observations as evidences to support their
assumptions.

Finally, there were limitations with using the MFT to measure learning in the
capstone course. Significant learning experiences described by Fink (2003) go
beyond cognitive learning. So it is being suggested that aside from measuring the
knowledge of the learners about the content of the subject matter, the researchers
must also include some performance tasks so the student can be assessed by hands
on application of their learnings. The emphasis on critical thinking and knowledge
synthesis was not necessarily captured on an objective, standardized examination.