Curriculum Assessment and Enhancement at Purdue University

Calumet Based on ABET 2000

Nasser Houshangi1

Abstract ? For the first time, in fall 2003, the engineering Throughout the paper different components of the process
department will have an accreditation based on ABET shown in figure 1, will be explained.
(Accreditation Board for Engineering and technology)
Engineering Criteria 2000. ABET 2000 requires: “a system Program Educational objectives
of ongoing evaluation that demonstrates achievement of
program objectives and uses the results to improve the ABET criteria 2 requires each engineering program for
effectiveness of the program,” and “each program must have which an institution seeks accreditation must have a detailed
an assessment process with documented results.” Evidence educational objectives that are consistent with the mission of
must be given that the results are applied to the further the institution and supported by the curriculum. Well-stated
development and improvement of the program. In this paper, program educational objectives identifies the needs of
the assessment plan for the engineering department is constituencies based on document able, quantifiable input
outlined. and will indicate specific program focus [1,4]. Each
engineering program must have a system of ongoing
INTRODUCTION evaluation that demonstrates achievement of these objectives
and uses the results to improve the effectiveness of the
The Purdue Calumet engineering curriculum leads to a
Bachelor of Science in engineering degree and is particularly program.
aimed at students who work in industry. The students Educational objectives shown in table 1 are developed
by the faculty with the feedback from engineering
specialize in Electrical Engineering, Computer Engineering,
department industrial advisory committee. As shown in
or Mechanical Engineering. First generation students count
figure 1, the program objective should be consistent with the
for 74% of student body with 25% minorities based on
department mission. The objectives describe the expected
numbers obtained from the fall 2000 admitted students. Half
accomplishments of the graduates during the first few years
of the students are part-time. Based on the current university
after graduation.
strategic planning, the goal is to improve new freshman
retention rates from 62% to 72% and improve the six year
graduation rate from 22% to 32%, with aspirations of 40%. Program Outcomes and Assessment
The electrical and mechanical engineering programs are
due for reaccreditations. The new computer engineering After stating the program objectives as required in criterion
program will be up for accreditation for the first time in fall 2, the program outcomes need to be stated. A well stated
2003. As mentioned, the goal of ABET is to promote program outcome should encompass ABET “a-k”, should be
continuous quality improvement in engineering education supported by curriculum, and should be linked to program
through faculty guidance and initiative. The new system for educational objectives. The outcomes describe what students
accreditation, ABET 2000, differs from predecessor in are expected to know and able to do by the time of
Criteria 2 and 3 [1]. Work presented in this paper also graduation.
concentrates on Criteria 2 (program objectives) and 3 Table 1 also shows the twelve outcomes for our
(program outcomes and continuous program improvement). program and its link to the educational objectives. The first
During the past few years, the engineering department 11 outcomes “a-k” are the one dictated by ABET. We have
worked on developing an assessment and enhancement added one extra outcome in addition to the required ones
process for the Bachelor of Science programs based on which is specific to our program.
ABET 2000 criteria. Figure 1 shows the developed Assessment process must demonstrate the outcomes and
assessment process for continuous improvement. The the objectives of the program are being measured. Usually
objective of the process is to provide a systematic pursuit of program outcomes provide general information about the
excellence and satisfaction of the needs of constituencies. program and thus are not measurable. In order to measure

1
Nasser Houshangi, Dept. of Engineering, Purdue University Calumet, Hammond, IN 46323-2094, hnasser@calumet.purdue.edu

120
outcomes performance criteria are developed for each From beginning our goal was to design an assessment
outcome. Performance criteria indicate what concrete actions and enhancement process which is effective, simple, and
the students should be able to perform in order to implementable. Steps taken include limiting the number of
demonstrate the outcome. performance criteria for each outcome, selecting primary
Performance criteria for each outcome are shown in courses addressing each outcome, making each faculty
table 2. The performance criteria facilitate the curriculum responsible for an outcome.
delivery strategies, and assessment procedures [2,3]. The In summary the following steps are taken to prepare for
performance criteria should be achievable with the provided ABET visit addressing ABET criterion 2 and 3:
resources of the program. Next, a strategy and an assessment ? Developed an outcome-based engineering program
method for each performance criteria need to be stated. assessment & enhancement process (figure 1)
Strategy will indicate what specifically is done to provide ? Identified relationship between program
the students with the opportunity to achieve the performance educational objectives and outcomes (table1)
criteria. Assessment is a process that identifies, collects, and ? Developed performance criteria for engineering
prepares data that can be used to evaluate achievement. All program outcomes (table 2)
the above information are documented in outcome notebook ? Specified one or two supporting courses for
(table 6). assessing each outcome (table 3)
It is suggested to use more than one assessment method ? Developed and implemented program outcome
for each performance criteria with the mixture of direct and flowchart (table 4)
indirect methods. Example of direct method is like written ? Published educational objectives and program
test items clearly linked to course learning objectives or outcomes on the WEB
observing a student communication skill during project oral ? Developed and conducted undergraduate surveys
presentation. Indirect assessment method may include use of
? Developed and conducted senior exit interviews
surveys. There are number of assessment methods and list of
? Updated list of course responsibility for each
different assessment techniques can be found in [5].
faculty
Table 3 indicates the primary courses supporting the
outcomes. Initially, all the courses in the curriculum ? Developed a standard for syllabi with learning
contributing to outcomes were included. To streamline the objectives for engineering courses
process, it was decided to include one, two, or maximum ? Developed and conducted course assessment
three courses and these are courses that will extensively be reports
assessed and included in the outcome notebook (table 6). Of ? Specified process time schedule
course it is still important to indicate all the courses ? Revised and streamlined the assessment procedure
throughout the curriculum that contribute to a specific based on feedback received
outcome. Table 4 shows the program outcome flowchart for Currently, based on process time schedule the
outcome b for the computer program. The various numbers department is involved with enhancement phase. This phase
inside the parentheses indicate the level of contribution that usually refereed to as closing the process loop as shown in
the course provides to the outcome. Program outcome figure 1.
flowchart indicates where in the curriculum the students are
given the opportunity to learn, apply, and demonstrate the REFERENCES
outcome. [1] ABET Criteria for accrediting engineering programs: Effective for
In order to document the assessment and enhancement evaluations during the 2002-2003 accreditation cycle
process, course journal and outcome notebooks are kept. The .http://www.abet.org/images/Criteria/2002-03EACCriteria.pdf,
accessed February 2, 2003.
contents of course journal and outcome notebooks are shown
in table 5 and 6, respectively. The purpose of course journal [2] http://civeng1.civpitt.edu/~ec2000, accessed February 2, 2003.
is to document course learning objectives and its assessment. [3] Besterfield-Sacre, M.E., et al., “Defining the Outcomes: A Framework
ABET 2000 criterion 3 focuses on outcome based for EC 2000”, IEEE Transactions on Engineering Education, 43(2),
assessment and not courses. ABET evaluators will 2000, pp.100-110.
concentrate on insuring that the listed program outcomes are [4] Carter,M.,R.Brent, and S. Rajala,”EC 2000 Criterion 2: A procedure
achieved and there is an enhancement process in place. for creating, assessing, and documenting program educational
Outcome notebook will provide the needed documentation objectives”, Proceedings of 2001 ASEE Annual Conference, American
Society for Engineering Education, 2001.
for each outcome.
The course contribution to each outcome through its [5] Felder, R., and R. Brent,” Designing and Teaching Courses to Satisfy
learning objectives is indicated in table 7. The program the ABET Engineering Criteria”, Journal of Engineering Education,
Vol.92, No. 1, January 2003, pp. 7-18.
assessment matrix indicates the course level of contribution
to program outcomes. The program outcome flowchart is
constructed based on information received for each course in
table 7.

121
Figure 1 BS Engineering Program Assessment & Enhancement Process

University, School &

Department Mission & Goals
Advisory Board
Industry Students BS Program
Alumni Faculty Objectives

Department Loop

Program Enhancement Outcomes &

Performance Criteria

Evaluation Assessment

Program Loop

Entering
Students

Course Assessment
& Enhancement

BS Course Loop
Program
Each Course:
Learning Objectives Course
Outcomes Journal

Outcome
Undergraduate Notebook
Surveys

Graduating Capstone Course

Seniors Senior Project

Alumni and Employer

Surveys

122
 Table 1 Relationship between Educational Objectives and Outcomes
Computer Option Program

The Computer Option curriculum provides a broad education in the fundamentals of Computer
Engineering. Students may pursue a general program or may choose a specialization in areas such as
Computer Hardware or Computer Software.

Program Educational Objectives Program Outcomes

1. Engineering Competence- Graduates will be Outcome a: Have an ability to apply knowledge of
competent engineers with problem solving and mathematics, science and engineering.
design skills, and the capability to apply
mathematics and science to solve engineering Outcome b: Have an ability to design and conduct
problems. experiments, as well as to analyze and interpret
data.

Outcome c: Have an ability to design a system,

component or process to meet desired needs.

Outcome e: Have an ability to identify, formulate

and solve engineering problems.
2. Foundation in modern technologies - Outcome k: Have an ability to use the techniques,
Graduates will have extensive knowledge about skills, and modern engineering tools necessary for
current technologies. engineering practice.
3. Professional skills - Graduates will have strong Outcome d: Have an ability to function on multi-
communication skills, and the ability to work disciplinary teams.
successfully in teams. They will be well prepared
for work in industry. Outcome g: Have an ability to communicate
effectively.

Outcome l: Have an ability to apply their

engineering knowledge to solve industrial
problems, and enhance industrial profitability
4. Creativity and drive for technical innovation - Outcome i: Have recognition of the need for, and an
Graduates will be self-motivated, creative people ability to engage in life-long learning.
who promote technical innovation and have
enthusiasm for life-long learning.
5. Well-rounded education – Graduates will have Outcome f: Have an understanding of professional
knowledge of contemporary issues, an and ethical responsibility.
understanding of professional and ethical
responsibility, and possess a general education Outcome h: Have the broad education necessary to
necessary to understand the impact of engineering understand the impact of engineering solutions in a
solutions in a global and societal context. global/societal context.

Outcome j: Have knowledge of contemporary

issues.

Table 2: Engineering Program Outcomes and Performance Criteria

Outcome a: Graduates will have the ability to apply knowledge of mathematics, science, and
engineering. Specifically, students will be able to

1. Use concepts from science to solve engineering problems.

2. Formulate analytical models using the laws of physics.
3. Use appropriate mathematical tools to solve equations.

123
Outcome b: Graduates will have the ability to design/conduct experiments and analyze/interpret
data. Specifically, students will be able to

1. Design an experiment, including determining the data to be collected, the range of parameter
values, and the use of statistical analysis.
2. Conduct an experiment and compare experimental with predicted or expected results.
3. Prepare reports that present the data from an experiment, interpret the data/results, and draw
conclusions and make recommendations.

Outcome c: Graduates will have the ability to design a system, component, or process to meet desired
needs. Specifically, students will be able to

1. Determine the necessary constraints and specifications.

2. Design components that meet specifications and constraints
3. Design a system or process that meets specifications and constraints.

Outcome d: Graduates will have the ability to function on multidisciplinary teams. Specifically,
students will be able to

1. Function as a team leader and/or team member in laboratory and problem-solving activities.
2. Function as a team leader and/or member in a senior design project.
3. Effectively participate in team-based oral and written reporting activities.

Outcome e: Graduates will have the ability to identify applicable theories, and formulate and solve
engineering problems. Specifically, students will be able to

1. Demonstrate an ability to formulate engineering problems, to recognize the relevant signals/

parameters, and to identify the governing theories and principles.
2. Create sketches, figures, flow-charts, and free-body diagrams.
3. Show understanding of the applicable theories and principles by demonstrating the use of relevant
formulae and relationships.

Outcome f: Graduates will understand professional and ethical responsibility. Specifically, students
will be able to

1. Demonstrate knowledge of safety factors in the design process.

2. Demonstrate knowledge of professional code of ethics.
3. Evaluate the ethical issues of an engineering problem.

Outcome g: Graduates will have the ability to communicate effectively. Specifically, students will be
able to

1. Develop and present effective oral presentations that integrate appropriate visuals.
2. Write documents that are well organized, properly formatted, and clear.
3. Convey technical information through the use of data plots, graphs, calculations, drawings, and
equations.
4. Communicate effectively with team members.

Outcome h: Graduates will have the broad education necessary to understand the impact of
engineering solutions in a global/social context. Specifically students will be able to

1. Demonstrate knowledge of the impact of the products on society and the environment, including
both production, and use.
2. Demonstrate an understanding of the impact of engineering decisions on society and the
environment.

124
Outcome i: Graduates will recognize of the need, to engage in life-long learning. Specifically, students
will be able to

1. Investigate and gather information on a given engineering issue.

2. Recognize the need for continuing education, and participation in professional societies and
meetings.

Outcome j: Graduates will have knowledge of contemporary issues. Specifically, students will be able
to

1. Identify several contemporary issues.

2. Investigate, gather, and analyze information related to contemporary issues.
3. Describe the impact of social, environmental, legal, and other contemporary issues on
engineering activities.

Outcome k: Graduates will have an ability to use the techniques, skills, and modern engineering tools
necessary for engineering practice. Specifically, students will be able to

1. Use libraries, the internet, and other sources to search for information necessary for engineering
projects.
2. Use computer programs in analysis, simulation, and design of systems and components.
3. Use modern instrumentation to conduct experiments on components and systems.

Outcome l: Graduate will have the ability to apply their engineering knowledge and experience to
solve industrial problems, and to enhance industrial profitability. Specifically, students will be able to

1. Work in cross-discipline teams.

2. Plan and execute projects, and prepare the necessary oral and written reports.
3. Identify engineering solutions within time and budget constraints.

Table 3 Primary Courses Supporting Outcomes

Outcome BSCE BSEE BSME

a. Math & Science Engr. ECE 301, 311 ECE 301, 311 ME 271, 312
(Kozel)
b. Experiments/Data ECE 370, 464 ECE 218, 275, 335 ME 345, 417
(Pai/Gopalan)
c. Design ECE 370 ECE 370 ME 466
(Burridge) ENGR 440, 460 ENGR 440, 460 ENGR 440, 460
d. Multidisciplinary ENGR 190, 440, 460 ENGR 190, 440, 460 ENGR 190, 440, 460
Teams(Pierson)
e. Engr. Problem Solving ECE 201, 301 ECE 201, 301 ME 275, 320, 461
(Mojtahed)
f. Professional & Ethics ENGR 440 ENGR 440 ENGR 440
(Gerber) PHIL 324 PHIL 324 PHIL 324
g. Effective Communication ENGR 460 ENGR 460 ENGR 460
(Abramowitz) MSE 344
h. Global & Societal (Hentea) ENGR 440, 460 ENGR 440,460 ENGR 440, 460
i. Life-Long Learning (Zhou) ENGR 195A, 440, 460 ENGR 195A, 440, 460 ENGR 195A, 440, 460
j. Contemporary Issues ENGR 190, 440, 460 ENGR 190, 440, 460 ME 461
(Kin) ENGR 440, 460
k. Modern Tools ECE, 160, 370, 380 ECE 160, 370, 380 ME 461, 486
(Gray) ENGR 380
l.. Solving Industrial Problems ENGR 440, 460 ENGR 440, 460 ENGR 440, 460
(Pierson/Nnanna)

125
 Table 4 Example of Program Outcome Flowchart

Outcome b: Graduates Must Have an Ability to Design and Conduct

Experiments/Analyze and Interpret Data

Courses ( Level of Contribution)

Semester (Level 1 = Objective Addresses Outcome Slightly, 2 = Moderately, 3 = Substantially)
Freshman CHM 115 (2), ENGR 190 (2)
Fall
Freshman PHYS 152 (2)
Spring
Sophomore ENGR 233 (1), ECE 207(3)
Fall
Sophomore ECE 370 (3), ECE 218(3)
Spring
Junior ECE 330 (1), ECE 275 (3)
Fall
Junior ECE 371 (2), ECE 459 (3)
Spring
Senior ECE 464(3), ECE 476(2), ENGR 380 (2)
Fall
Senior
Spring

Table 5 Example Course Journal Table of Contents

Tab 1: Goals & Objectives

Course Goals
Student Learning Objectives
Outcome Contribution
Tab 2: Homework & Exams
List/Description of Homework Assignments
Exams with Grading Statistics
Quizzes
Tab 3: Labs & Projects
Description & Assignment:
Computer Laboratory Assignments
Physical Laboratory Experiments
Tab 4: Significant Handouts
Tab 5: Course Assessment
Report
Student Statistics
Surveys
Grade Distribution

126
 Table 6 Example Outcome Notebook Table of Contents

Tab 1: Outcome Assessment

Performance Criteria for the Outcome
Strategy
Assessment methods
Assessment & Enhancement Calendar Cycles
Tab 2: Courses Supporting Outcome
Table 7 - Program Assessment Matrix
Support Course Descriptions
Course Goals & Student Learning Objectives
Tab 3: Assessment Results
Support Course Assessment Results
Selections of Student Work at A, B, C Levels
Survey Assessment Results
Sophomore Students Focus Group
Junior Students Alumni
Graduating Seniors Employer
Tab 4: Assessment Summary
Overall Assessment Results
Recommendations
Actions

127
 Table 7 Program Assessment Matrix
Course Number: Course Name:
Evaluator: Date of Evaluation:
(See notes at end of table to explain abbreviations)
Topic Major Program Assessable Level of Assessment Prerequisites
Number Outcomes Topic Component(s) Contribution Method(s) Course(s) Topic(s)
(1, 2 or 3) (EXAM, HW. EXP.
IP, TP, NA, Other)
Outcome Description Level
a Ability to apply knowledge of
mathematics, science and engineering
b Ability to design/conduct experiments
and analyze/interpret data
c Ability to design system, component or
process to meet desired needs
d Ability to function on multi-
disciplinary teams
e Ability to identify, formulate and solve
engineering problems
f Understanding of professional an
ethical responsibility
g Ability to communicate effectively
h Understand the impact of engineering
solutions in a global/societal context
i Recognition of the need for and an
ability to engage in life-long learning
j Knowledge of contemporary issues
k Ability to use the techniques, skills and
modern tools necessary for engineering
practice
l Ability to Apply Engineering
Knowledge to Solve Industrial
Problems and enhance Industrial
Profitability

Notes for Table 8:

Assessable Component(s) NP - Not a Priority in this Class
Otherwise, please list specific activities related to this attribute.
Level of Contribution 1 = Slightly, 2 = Moderately, 3 = substantially
Assessment Method(s) Used to Evaluate Level of Student EXP = Experiment TT = Team Test HW - Graded Homework Problems
Capability In Priority Order TPS = Team Problem Solving IP - Individual Project/Report
TP = Team Project TR = Team Report NA - Not Assessed
Other = Any other method that is applied

128