LECTURE 1

CRITERIA-BASED APPROACH IN ASSESSING OF EDUCATIONAL RESULTS. THE MAIN

DEFINITIONS
An integral part of the education content is the assessment system of students’ learning
outcomes achievements. With the introduction of the updated secondary education the system of
criteria-based assessment of students’ learning outcomes achievements is used. A new for
Kazakhstan system of criteria-based assessment is aimed at developing the learner, increasing his
interest and motivation for learning. This can be applicable if we develop clear and measurable
assessment criteria that are understandable to each student and his parents.
With the help of well-defined criteria-based assessment, the student will understand:
-At what teaching stage the student is
-What his education target is
-What needs to be done to help student achieve the expected learning outcomes.
The purpose of criteria-based assessment is to obtain an objective information on the
learning outcomes of students based on criteria assessment and provision to all stakeholders for
further improvement of the educational process.
The tasks of the system of criteria-based assessment is to:

1. Expand opportunities and evaluation functions in the learning process.

2. Create conditions for the continuous self-development of students by establishing regular

feedback.

3. Promote developing unique standards, qualified mechanisms and tools for assessment.

4. Provide objective, continuous and reliable information:

- students about the quality of their education;

- teachers about students’ progress;

- parents about the level of learning outcomes achievement;

- provide the quality of the educational services to the authority.

CRITERIA-BASED ASSESSMENT PRINCIPLES

Criteria-based assessment is developed according to the following principles:

Interrelation of learning and assessment. Assessment is an integral part of learning and is directly
related to the objectives of the curriculum and the expected results.

Objectivity, reliability and validity. Assessment provides accurate and reliable information. There
is confidence that the criteria and tools used assess the achievement of learning objectives and
expected outcomes.

Continuity. Assessment is a continuous process, allowing timely and systematic monitoring the
progress of students' learning achievements.

Focus on development. The results of evaluation initiate and determine the direction for further
development of the education system, schools, teachers and students.
The content of criteria-based assessment system is regulated by the following instructive and
methodological documents:
The state compulsory standard on primary education;
-Curriculum;
-Syllabus;
-The procedure of criteria-based assessment of students’ educational achievements in educational
organizations that implement general educational curricula on primary education;
-Guidelines on criteria-based assessment for primary school teachers;
-Guidelines on criteria-based assessment for regional and school coordinators;
-Collection of tasks on formative evaluation;
-Methodological recommendations on cumulative assessment.

THEORETICAL FOUNDATIONS OF CRITERIA-BASED ASSESSMENT

The term "criteria-based assessment" was first used by Robert Eugene Glazer (1963) and
characterizes the process that facilitates the definition of a set of typical behavioral models and the
correspondence between the achieved and potential levels of student learning achievements. This
means that the activity of the learner is evaluated through a fixed set of predefined criteria. Glazer
notes that the evaluation of the criteria standards excludes comparison and dependence on the
achievements of other students, and also aims at informing about the level of competence of each
student. The system of criteria-based assessment of students' learning achievements is based on the
fact that teaching, learning and assessment are interrelated and provide a unified approach to the
organization of the learning process (Boyle and Charles, 2010). This involves theoretical
substantiation and the establishment of the relationship between all elements of assessment
(learning objectives, types, tools and evaluation results).
The system of criteria-based assessment of students’ educational achievements is based on the unity
of learning and assessment; is aimed at forming a holistic approach to ensure students’ progress and
achievement; ensures the implementation of teaching objectives and the collection of evidence of
obtaining knowledge and skills development in accordance with teaching programs; includes a
variety of ways and forms of evaluation based on the content of the curriculum for each class.
TERMS AND DEFINITIONS
Descriptor is a characteristic feature that serves to describe or identify the actions when performing
tasks.
Criteria-based assessment is the process of correlating the learning outcomes actually achieved by
students with the expected learning outcomes based on the criteria developed.
Evaluation criterion is a sign on the basis of which the evaluation of the students' learning
achievements is conducted.
Moderation - the process of discussing the work of students on a cumulative assessment for a Mid-
term in order to standardize the scoring to ensure objectivity and transparency of evaluation.
Feedback - feedback, response, response to an action or event.
Expected learning outcomes are a set of competencies that show what the learner will know,
understand, and demonstrate at the end of the learning process.
Reflexion is a thinking process aimed at rethinking and analysis of results of self-education, one's
own learning outcomes.
Rubric is a way of describing the levels of students’ educational achievements in accordance with
the evaluation criteria.
Summative assessment is the type of assessment that is carried out after the completion of
Module / cross-cutting topic of the Curriculum, the specific academic period (mid-term / term,
academic year).
Formative assessment is a type of assessment that is carried out continuously, provides feedback
between the teacher and the learner and allows correct timely the educational process.
Teaching objectives are statements that formulate the expected results on the educational
achievements, awareness and skills obtained during teaching the subject in accordance with the
Curriculum.
Electronic Journal Recording Assessment Results is an electronic document where the results of
the summative assessment (per module / cross-cutting topic, mid-term) are displayed and an
automatic calculation is carried out according to the formula of mid-term, term and final
assessment.
 LECTURE 2
TYPES OF ASSESSMENT IN WORLD EDUCATIONAL SYSTEM

Assessment in American schools

Students do not all learn in the same way; therefore, teachers should use more
than one way to evaluate them. Assessments come in a variety of modes, but all can be
classified as either formative or summative depending on when the assessment takes
place and what its purpose is.
Formative Assessment
Formative assessment is a term for any type of assessment used to gather student
feedback and improve instruction. Formative assessments occur during the learning
process, often while students are engaged in other activities. Anecdotal records, periodic
quizzes or essays, diagnostic tests and in-class or homework assignments are all types of
formative assessment because they provide information about a student's progress.
Summative Assessment
Summative assessment occurs at the end of a unit of study in order to measure the
amount of information the students have learned. Most traditional assessment types are
considered summative. Summative assessments reflect students' learning and the
teacher's ability to communicate information effectively.
Observational Assessment
Observational assessment is the most common form of formative assessment. Teachers
can circulate the room to monitor students' progress. If students are working
independently or in groups, teachers should intervene when the students are not
understanding the material. Teachers can also take note of students' comments and
participation levels during class discussions to gauge their learning.
Selected Response
Selected response assessments are any type of objective exam where there is only one
correct answer for each question. Multiple choice, fill-in-the-blank, matching and
true/false questions are all types of selected response assessments. This type of
assessment allows the teacher to score exams quickly and with a large degree of
reliability in scoring from one exam to another.
Constructed Response
Constructed response assessments require students to generate their own response rather
than selecting a single response from several possible ones. These exams are much more
subjective as there is not a single correct answer. Instead, teachers must grade either
with a rubric or holistically to maintain a fair degree of reliability.
Performance Assessment
Performance assessments require students to perform as a means of showing they
understand class material. The types of performances can include actual performing, as
in a class debate, or performance by creating, as in making a brochure or TV ad. These
assessments evaluate complex cognitive processes as well as attitude and social skills,
and students often find them engaging.
Portfolio Assessment
Portfolio assessments evaluate a student's progress over the course of the semester. It is
more than a one-time picture of what a learner has accomplished. Portfolios include all
of a student's work in a particular area. For example, a student in an English class could
have a portfolio for a research paper that includes note cards, outlines, rough drafts,
revisions and a final draft. The teacher would evaluate the portfolio as a whole, not just
the final draft, to see how the student has grown.

6 Types of assessment of learning in British schools

1. Diagnostic Assessment (as Pre-Assessment)
One way to think about it: Assesses a student’s strengths, weaknesses, knowledge, and skills prior
to instruction.
Another way to think about it: A baseline to work from
2. Formative Assessment
One way to think about it: Assesses a student’s performance during instruction, and usually occurs
regularly throughout the instruction process.
Another way to think about it: Like a doctor’s “check-up” to provide data to revise instruction
3. Summative Assessment
One way to think about it: Measures a student’s achievement at the end of instruction.
Another way to think about it: It’s macabre, but if formative assessment is the check-up, you might
think of summative assessment as the autopsy. What happened? Now that it’s all over, what went
right and what went wrong?
4. Norm-Referenced Assessment
One way to think about it: Compares a student’s performance against other students (a national
group or other “norm”)
Another way to think about it: Group or “Demographic” assessment
5. Criterion-Referenced Assessment
One way to think about it: Measures a student’s performance against a goal, specific objective, or
standard.
Another way to think about it: a bar to measure all students against
6. Interim/Benchmark Assessment
One way to think about it: Evaluates student performance at periodic intervals, frequently at the end
of a grading period. Can predict student performance on end-of-the-year summative assessments.

Advantages & Disadvantages of Traditional Assessment

In the field of education, student assessment is important at all levels. Assessment allows
individual schools to chart their progress working with students. It also helps college admissions
committees weigh individual student performance and aptitudes. Finally, it allows schools to make
decisions regarding teacher hiring and budget allocation. Traditional assessment methods offer
benefits and drawbacks compared to other assessment options available to educators.
Definition and Alternatives
Traditional assessment refers to standardized testing that uses questions with a limited number of
answer choices. It includes multiple choice, true or false and some short answer responses. Other
testing methods, sometimes known as performance-based assessment, alternative assessment or
authentic assessment, focus on the process by which a student arrives at an answer in addition to the
final response. Alternative assessment includes long-form responses and essay questions.
Ease of Analysis
One of the primary benefits of traditional assessment is the ease with which administrators and
admissions professionals can analyze and compare student scores. Standardized testing that relies
entirely on quantifiable responses produces an assessment that is easy to score. Test makers can
categorize questions to determine which areas students excel in and which they have difficulty with.
Students' results are comparable over time and across a large, diverse group of students.
Lack of Context
Traditional assessment methods have the disadvantage of lacking real-world context. Students
answer questions one by one without the need to apply long-term critical reasoning skills. They also
lack chances to demonstrate their reasoning skills despite a lack of knowledge about a question's
specific subject matter. Alternative assessment methods allow students to apply their skills and
knowledge within a context that more closely resembles problem solving and knowledge
application in most jobs or daily tasks.
Impact on Teaching
Traditional assessment also has both advantages and disadvantages when it comes to the way
educators teach. Traditional assessment forces educators to spend time teaching students how to
manage tests, including strategies for selecting correct answers from a listed group. Alternative
assessment allows educators to focus on critical reasoning skills, which students will be able to
apply naturally to long-form test questions. However, in situations where alternative assessment
replaces traditional assessment, this means teachers need to learn new methods of test preparation
and schools need to develop alternative forms of performance assessment for their students and
teachers. These changes to accommodate a new form of assessment mean added cost for schools
and a disruptive change in teaching methods for current students.
 LECTURE 3
THE MODERN SYSTEM OF UPDATED EDUCATIONAL CONTENT IN KAZAKHSTAN

Today's students - the children of the 21st century: age of globalization and integration; age
of mobility of human resources; age of the growing influence of the media and technology; age of
huge changes in information and communication technologies.
How do the world changes (open borders, the volume and rate of communication, economic
development, self-management of people), influence the development and education of our
children? How does Kazakhstan education system respond to these changes?
At the beginning of this century, the UN General Assembly adopted the "Millennium
Declaration", which states "... the central challenge we face today is to ensure that globalization
becomes a positive force for all nations in the world. Globalization will become overall and
equitable only through widespread and sustained efforts for creating a shared future, based upon our
common humanity in all its diversity."
Globalization affects all areas of human activity. It provides the interpenetration of religions,
cultures, traditions, mentality, and familiarizing humankind to the world.
The aims of national education are to provide historical continuity of generations, to
preserve the spiritual values of the people of Kazakhstan building a culture of interpersonal
relations. The basis on resolving these tasks are development of the state language, introduction of
multilingual education, introduction "Mangilik el" values, use of technology to promote the
development of critical thinking and cooperation.
The modern world - a world of increasing information flows. We know that modern man receives
and processes the same amount of information a month, as a person of XVII century - for an entire
lifetime.
According to the sixth IDC research (2012), the ubiquity of technology and access to the
Internet has led to the fact that the amount of information over the past 2 years has doubled. The
study estimated the volume of 2.8 zettabytes generated data in 2012, and predicts increase of up to
40 zettabytes by 2020, which is higher than their previous forecasts by 14%.
According to IDC experts, the amount of data in the world will at least double every two
years, with the proportion of useful information of only 35%.
Accelerating pace of development of society and science has not only led to a sharp increase
in research and production of information, but also led to an intensive "aging" and the depreciation
of some previously accumulated human knowledge and skills.
For example, most of our children do not remember a tube TVs, and our grandchildren have
never heard of pagers and diskettes.
How to maneuver in such information flows? How to select the necessary and useful
information, since the World Wide Web is not responsible for the moral and ethical content and the
information content. Is today's school able to make sense of this for our children?
Recently, it seemed that the Kazakhstan school has retained the best from the former Soviet
system of education, in particular, the fundamental knowledge. However, the results of the
participation of our students in international comparative studies of the quality of education TIMSS-
2011, PISA-2012 showed that having a sufficient level of academic excellence, Kazakhstani
students do not know how to use it effectively, are poorly oriented in the methods and approaches a
critical analysis of the situation for the further application of knowledge.
International experts (OECD, 2014) point out that in most cases today the training of students in
Kazakhstan schools is conducted with an emphasis on the theory, and does not devote enough time
to the possibility of practical application of knowledge. "With the result that, students are not able
to sufficiently and effectively apply and use the acquired knowledge in the context of new
situations. Education does not contribute to the development of higher-order thinking skills.
The question arises, what should be a school of tomorrow to meet the challenges of today
for the future of our children? What needs to be changed? How the best education systems are being
improved?
 At present, almost all developed countries have realized the need to reform their education
systems so that the student really has become the central figure of the educational process and
cognitive activities were in the focus of educators and researchers.
The educational systems of leading world countries, OECD focuses primarily on the
development of competencies. These countries are making great economic success due to the high
competitiveness of human resources (Korea, Japan, Finland and so on).
Kazakhstan aims to improve the quality of education by the transition to the new content of
school education, centered in the development of functional literacy of schoolchildren, the skills of
independent research, critical analysis and assessment, that is the transition from "a man who
knows" to "a man, creatively thinking, acting and self-developing.»
Teachers cannot afford teaching students all the knowledge and achievements of humankind,
but can give "not the fish, but the fishing rod". The teachers can teach students to extract their
knowledge, the skills of creative and critical thinking. Teachers can contribute to the development
of functional literacy of students.
A prominent American businessperson John Grillos at the UNESCO conference very
accurately expressed the importance of this approach. He said that his little worries are about the
strength of the knowledge acquired by students in a particular area, as this knowledge is subject to
change each year, and this knowledge is sometimes obsolete before students will be able to
assimilate them. Much more important, according to the businessman, "...that the economy gained
young people who are able to independently learn to work with the information on their own to
improve their knowledge and skills in various fields, acquiring, if necessary the new knowledge
profession, because that's what they have to deal with their entire adult life".
National Academy of Education named after Altynsarin, in close cooperation with the
Centre for Educational Programs of autonomous organization of education "Nazarbayev Intellectual
Schools", are developing updated education standards and curricula.
State educational standard of primary education has clearly defined requirements for the
content and its main goal: "establishing education area that is conducive to the harmonious
development of personality formation and learning, based on the following range of skills: 1)
functional and creative application of knowledge; 2) critical thinking; 3) conduct research; 4) use of
information and communication technologies; 5) the use of different methods of communication,
including language skills; 6) the ability to work in a group and individually".
Thus, the paradigm of education is changing, we are moving from the concept of "good
education for entire life" to understanding the need for life-long education.
Implementation of activities to update the content of education, relied on supporting 30
schools in Kazakhstan has started current academic year. Next year, the transfer the entire education
system to the updated one is planned.
The result of the renewed education must become an educated man, who has developed a
wide range of skills that allow him to adapt to a rapidly changing environment.
Changing the educational paradigm must ensure Kazakhstan introduction into the international
education space. In 21st century, it is recognized that the level of education of the nation, its ability
to implement advanced technologies will be the determining factors in the global competition.
Updating education is not only the renewal of its content. This includes update of the
educational process, assessment system, the system of relations between teacher and student. That
is, the update of all components of education - content, methods, forms and means. Nevertheless,
this is a topic for the next call.
 LECTURE 4

TYPES OF CRITERIA-BASED ASSESSMENT

Assessment is a process of recording the knowledge, skills and attitudes of learners, and also
helps to study the learner’s understanding of the subject. It can focus on individual learners, a group
of learners or the institution as a whole. It is always best to start any course with an assessment of
student’s prior learning, so that the teacher can start their teaching at the correct level, and ensure an
inclusive teaching method where every learner’s needs are met.
There are many methods of assessments:
Multiple Choice assessment – This includes a set of multiple choices or multiple selections, or true
/ false questions, that help to assess the understanding of the learners about the subject.
Written short answer – This method of assessment includes short answers, small problems and
diagrams, and it gives students a chance to express their thoughts and ideas.
Written long answers – This method includes essay type questions, which gives students a better
chance to express their ideas in detail with lots of creative information.
Research projects – This method involves practical work and written explanation. The work has to
be analyzed and interpreted in different ways, like charts and graphs to prove and explain what was
discovered.
Verbal questioning – This method involves questions and answers on a particular topic, and helps
identify the verbal skills of the learners.
Observing practical work – This method involves the tutor observing the learners performance at
workplace.
Class test – This method also helps assess the written skills of the learners through small tests on a
particular topic.
Quiz activities – These help assess the general knowledge in a subject area, and helps to assess the
level to which students are aware of that particular subject and help to prepare students for the
course.
The initial assessment can take place in the form of question and answers in icebreaker activities or
group discussions of previous knowledge. For a science course, there will be an initial
assessment prior to start of the course in order to ensure that the candidate have a good standard of
Literacy and Numeracy. Also the qualifications and experience of the students will be assessed
against the course needs through a written application form and short interview. Then the first
session can begin with a group discussion and a multiple choice assessment to assess the students’
level of understanding of the subject which will also be part of the initial assessment.
Having assessed the prior understanding of learners, formative assessments can be carried out
which will be ongoing all through to the end of the course. It allows both the teacher and the learner
to develop and improve their knowledge as the course progresses. This assessment will be practical
activities, question and answers, assignments or reports which will be documented throughout the
course. Activity sheets will be handed at the end of each session to assess students’ understanding,
which will be marked and handed in the next session.
At the end of the course, it is necessary to have a summative assessment to ensure that the aims
and objectives of the course have been met. For example when I teach science, the final exam will
be a written paper to assess students’ knowledge of subject, and a practical assessment to observe
the skills that have been mastered. Also a final tutorial will be arranged, to discuss how the learning
experience can be applied in their lives to go further in their career.
All the assessments that were carried out have to be recorded. It helps involve pupils in the learning
process, provide information on learner’s development, identifies special educational needs, assists
in reporting to parents, governing bodies and LEAs and monitor the effectiveness of targeted
actions.
Assessment record is maintained for individual learners. It contains details such as,
 Who was assessed?
 What was assessed?
 Date and time of assessment?
 Where assessment took place?
 Others involved in the assessment?
 Why and how assessment took place?
 Result
 Feedback
Apart from this a tracking sheet is maintained, which has record of all learners. This sheet has
details such as,
· Record of students’ progress.
· Indicates date of assessment.
· Grade achieved – pass or refer.
· Date Unit/Assignment completed
· Can be computerized or hand written
· Ensure relevant legislation is followed.
· Can be used for audit purposes.
Theory of Formative Assessment
Traditionally, the teacher acts as a person who is responsible for teaching and evaluating the
learner. However, it is also necessary to pay attention to the role played by the learner and his peers
in teaching. In most cases, the teacher is only responsible for creating and implementing an
effective learning environment, while the learner is responsible for learning in this environment and
the results achieved by him. Since the responsibility for teaching rests with both the teacher and the
learner (in the language of partnership law - "joint responsibility"), both must make every effort to
mitigate the impact of any failure on the results.
Within the framework of the theory of formative assessment, the basis for the process of supporting
students should be the following three questions: "at what stage of the learning are the students?",
"Where do they seek in their teaching?", "What needs to be done to help them achieve this?".
The data indicate that formative assessment consists of five key strategies:
-Clarification of expected results and evaluation criteria;
-Forming effective discussions and interactions in the class, which will indicate the understanding
of students;
- Providing feedback, which will stimulate students to achieve results;
-Involving learners as sources and resources of mutual learning;
-Student becomes the "creator" of his knowledge.
Formative assessment procedure
-Formative assessment determines the level of knowledge and skills gained in the learning process,
allow to uncover difficulties and help in achieving learning objectives by subject program.
-During formative assessment students are provided with continuous feedback. Marking and
grading always take place.
-Formative assessment is carried out using different ways and methods (questionnaire, practical
work, laboratory work, project, test and etc.) on the different levels of academic activity (process of
topic explaining, accomplishing tasks, checking the tasks, etc.).
-Individual, paired and group forms of work organization is used.
-In the result of formative assessment teacher timely corrects academic process, reduce possible
gaps and errors of students before conducting external summative assessment.
-During formative assessment students achieve learning objectives in accordance with subject
programs by subject.
-In case of absence of student by reasonable excuse (illness, death of close relatives, participation in
conferences, competitions and scientific competitions), missed material should be learned during
two weeks with subsequent achievement of learning objectives after arriving the student to school.
-Results of formative assessment are taken into account during calculating termly and annual
grades.
 Summative Assessment Procedure

Summative assessment is conducted with the aim to determine the level of students’
academic achievement after completing learning sections/cross-cutting themes of subject program
and particular academic period (term, academic year) with marking and grading. As a results of
summative assessment information is provided on the progress of students’ academic achievements
to teachers, students, parents and other legal representatives.
-For subjects with weakly loading of 1-2 hours the summative assessment for term in not exceeding
40 minutes. For subjects with weakly loading of 3 hours and more the summative assessment for
term is not exceeding 80 minutes. This norm of timing is applied for language subjects except for
“speaking” part. Assessment of “speaking” skill should be conducted during the week of summative
assessment.
-The schedule of summative assessment for term by subject shall be approved by the School
Principal. No more than 2 summative works for term should be conducted in a day.
-In case of circumstances outside one's control, i.e. extreme and unavoidable circumstances (acts of
God, warfare, etc.) providing that these circumstances are beyond the control of the parties and
made impossible the conducting and summative assessment for term in set schedule, it is possible to
change the schedule of summative assessment for term on the basis of Order of School Principle.
- Summative assessment for section/cross-cutting theme and term is compulsory for all students.
-The second sitting (rewriting) of summative assessment for section/cross-cutting theme, term and
year shall not be allowed.
-In case of student absence during summative assessment for section/cross-cutting theme and/or
term due to a reasonable excuse (illness, death of close relatives, participation in conferences and
scientific competitions), student should complete it after the appearance at school within two week
time. In this case additional variants of summative assessment task for section/cross-cutting theme
is used.
-In case of student absence during summative assessment for section/cross-cutting theme and/or
term with no reasonable excuse (without medical certificate of established form, Order of School
Principal) the summative work should be assessed with 0 mark.
-In case of breach of academic honesty principles during summative assessment the measurements
should be applied covered by the Rules of Academic Honesty and Rules of internal order for
students.
-Marks for summative assessment for section/cross-cutting theme are given in accordance with
descriptors, summative assessment marks for term are given in accordance with mark schemes.
-To ensure objectiveness and transparency of assessment of students’ results teachers conduct
moderation process of summative work for term.
-Summative works for section/cross-cutting theme with assessment criteria and descriptors,
summative works for term with mark schemes are kept in students’ portfolio.
-Tutors are responsible for maintenance and safety of students’ portfolio.
-Students’ portfolios are individual and must be accessible for all teachers, administration, parents
and other legal representatives.

Moderation Procedure for Summative Assessment for Term

1) Moderation of summative assessment for term is intended to discuss the results of summative
assessment papers to standardize the assessment.
2) Moderation is conducted by teachers of one subject and one parallel.
3) Moderation is carried out at the end of each term. The time between summative assessment for term
and moderation should not exceed 3 working days. The results for summative assessment paper for
term can be changed following the moderation.
4) School administration together with the Heads of Methodical Units shall plan the time and venue
for moderation in advance, approve the Moderation Meetings Chairperson, who will regulate the
 process of discussion. Moderation Meetings Chairperson can be a Head of Methodical Unit or any
other teacher.
5) Before moderation the summative works of students for term (hereinafter - papers) should be
preliminary checked by teachers.
6) Teachers choose three papers from the checked papers, encrypt them and prepare their copies for
discussion during the moderation meeting.
7) Papers with maximum and minimum marks are chosen in accordance with the mark scheme and
paper which cause difficulties in marking. If a teacher teach in several classes in one parallel, papers
are chosen from the total number of papers.
8) On the Moderation Meetings teachers discuss marks and put final results for students’ papers in
accordance with mark scheme.
9) As a result of moderation, marks for students’ summative works which were agreed on the
Moderation Meeting should be put in pen using mark schemes. Summative assessment mark as a
result of moderation can be changed both by increasing or decreasing.
10) After moderation is complete Meeting Minutes should be signed as a result of moderation of
summative assessment for term in accordance with the form shown in Appendix 2 to this Rules.
Minutes should be kept with the Head of Methodical Unit.

LECTURE 5
CRITERIA-BASED ASSESSMENT AS A MEANS OF EDUCATIONAL MOTIVATION

The difference between good students and weak students is that good students are able to
absorb our feedback and use it to create a pathway toward understanding learning targets. Weak
students, however, have trouble understanding feedback and need us to give them specific next
steps in order for them to develop a growth mindset and see a path toward understanding. Unlike
our good students, our weaker students often have given up trying to understand our comments
written in the margins of their papers or don’t know how to find solutions to the test items they
missed.   Because they don’t understand how to improve, they often dismiss school and school work
as “stupid” or simply say, “I don’t care.”
When we teach our students how to use our feedback to analyze their work, it not only gives
meaning to the time and effort we have put into grading and commenting on their work, but also
engages our students in the learning process. Requiring students to think about and apply criteria for
meeting learning targets in the context of their own work encourages students to monitor their own
work and take responsibility for their own learning.
  Getting students involved in analyzing their mistakes on tests helps them to understand the
intended learning, the immediate next steps they need to take in their journey toward learning
targets, and gives them a clearer picture as to just where they are in the journey. Hattie and
Timperely’s (2007) review of the research on feedback determined that analysis of mistakes is one
of the most powerful ways students learn or increase their learning.
 What You Can Do to Make Assessment More Effective
We can help our students’ by teaching them to be more analytic about their own learning, by
giving them class time and a structure to examine their own work in relation to previously explained
criteria, and by clarifying how they can improve their work.
 
 We begin by helping our students to identify their mistakes by providing them with item analyses of
their tests or rubric scored projects.
 We then set up a system to involve our students in thinking about their mistakes.
Give them time to consider why they made the mistake, and
Help them to understand what they will do differently next time. Thinking About Doing
Better (below) is an example of a handout for helping students to analyze their mistakes on a
forced-choice or short answer test. Each student has a form and works in a group of two or three.
  After students analyze their mistakes with a partner, they are asked to set some learning
goals. When students examine what they are doing well and what they need to improve on, they are
beginning the process of setting their own goals for learning. Students should be encouraged to set
small, realistic goals as the most useful goals are those that reflect steps along the way---not just the
final outcome. Taking small steps helps students to self-monitor their way to success. (Davies,
2007)
  Sample Assessment Handout:  Thinking About Doing Better
Directions: Identify three items (questions or problems) you missed on the test. Then with a partner
decide why you missed the question and how you could fix it. Next, with your partner write down
what you will do differently the next time you encounter a similar question or problem.   Budget
your time to eight minutes per item.

Item number Why I got it wrong How I can fix it What I will do next time

 
My Goals
Directions: By yourself write down two learning goals and the activities you will engage in to reach
them. If you need help identifying activities, ask your partner or your teacher.
 
Goal
One:
Activities for Goal
One:

Activities for Goal

Goal
Two:
Two:

Whether we evaluate our students’ work by scanning answer sheets, hand-scoring test items,
putting check marks on rubrics, or commenting directly on students’ written work, our evaluative
feedback needs to provide information for helpful “next steps “ for learning and not just a grade.
Then class time needs to be set aside for students to understand and use our feedback to enhance
their learning.
 
When we define what our students need to know and provide the criteria they need to
successfully learn and meet their learning targets, we help our students believe in their potential for
success. When we build assessment systems that provide valuable information to pinpoint gaps in
learning and show our students the next steps they need to take to eliminate the gaps, we involve
our students in the assessment process, and our students gain sense of ownership and commitment
to learning. Soon they become more focused, motivated and achievement oriented.
 
 
 LECTURE 6
TECHNOLOGIES OF CRITERIA-BASED ASSESSMENT OF STUDENTS. BLOOM`S
TAXONOMY
Criteria for assessing knowledge and skills:
Evaluation of students' knowledge and skills is the final stage of most types of control. At
the same time, objectivity and accuracy should be ensured based on evaluation criteria. For each
discipline, the department should develop uniform criteria, which should reflect the degree of
compliance with the level (objectives) of training for a given course (section, topic) with the
level of its mastering by students. 
The criteria for assessing knowledge and skills are based on the following principles: 

- the content of knowledge and skills is based on the requirements of the discipline, related
disciplines and specialty; 

- the ratio between the volume of knowledge and skills determines the distribution of study time
for lectures, group and practical classes; 

- the amount of skills and abilities is determined by the possibilities of its formation in the
planned time of practical training and self-training; 

- when drafting each control question to the program under the “know” section, the following are
taken into account: 

- knowledge acquired by memory; 

- knowledge realized with the help of educational and visual aids (posters, filmstrips, devices); 

- knowledge realized with the help of lecture notes, textbooks, reference books 

- when compiling each control question for the “be able” section, it is necessary to take into
account the program’s target settings, a list of practical skills and competencies defined by the
methodological developments in the relevant sections and topics of the discipline, as well as the
development of certain command and methodological skills in accordance with the
comprehensive plan of their inoculation to students. In all cases, the level of control must
correspond to the level of targets (level of learning objectives) 

In high school, a four-point system of assessment of knowledge and skills is used. The four-
point grading system represents the four levels of student quality assessed. These levels are
characterized by some unevenness. As a rule, the largest is the interval between “satisfactory”
and “unsatisfactory” ratings. The interval between “excellent” and “good” is less, and the
interval between “good” and “satisfactory” is almost negligible. An individual assessment of the
mastering of a program in a discipline that does not have performance indicators is determined
by: “Excellent” if a student showed deep and firm knowledge of the program material, quickly
makes the right decisions, clearly gives commands, perfectly knows the techniques of working
on equipment and confidently fulfills the established standards; “Good” if a student knows the
program material firmly, sets it up correctly and without errors, correctly applies this knowledge
to solving practical problems, confidently masters the techniques of working with the material
part of equipment, has solid skills in fulfilling the established standards; “Satisfactory” if the
student has knowledge of only the basic material, in some cases requires additional (suggestive)
questions for a complete answer, allows inaccuracies or hesitantly gives commands, hesitantly
performs techniques when working with the equipment and standards  “Unsatisfactory”, if a
student makes gross mistakes in answering the questions posed, cannot apply this knowledge in
practice, has low skills in working on the technique, does not fulfill the established standards for
the “satisfactory” assessment. Individual student assessment is announced immediately after the
exercise, task, standard; in group classes - at the end of the class.
Assessment criteria are statements specifying the standards that must be met and the evidence
that will be gathered to demonstrate the achievement of learning outcomes.
The purpose of assessment criteria is to establish clear and unambiguous standards of
achievement for each learning outcome. They should describe what the learner is expected to do
to show that the learning outcome has been achieved. They should not, however, be confused
with the actual assessment tasks. Rather, the assessment criteria specify how the task will be
evaluated.
There are three broad types of assessment criteria:
1. Threshold standards tell the learner what must be done in order to demonstrate
achievement of the learning outcomes of the unit i.e. what is the minimum requirement for
passing this unit.
2. Grading criteria provide a general description of the standard required for achievement of
each pre-established grade, marking band or degree classification i.e. a first class honors award
requires 70 per cent or more, an upper second requires 60-69 per cent, etc.
3. General criteria provide general outcome descriptors that can be achieved more or less
well. Students’ work will be judged to fall at a point within a performance range and marks are
allocated accordingly. Typically, criteria of this sort are used to evaluate such things as use of
referencing, accuracy of language, use of supporting evidence in drawing conclusions, quality of
critical thinking, etc.
The assessment criteria creation process:
1. Writing assessment criteria starts with a consideration of the learning outcome being
tested.
2. Then this needs to be set alongside the assessment task.
3. Requirements for, or attributes of, successful performance of the task should be listed.
4. If necessary, these requirements can be placed into context of expectations at this level of
learning.
5. The final criteria must focus on what is deemed essential amongst the requirements and
these should be formed into clearly worded criteria.
6. These criteria need to be checked to ensure that they are reliably measurable and clear in
their intention.
7. This process can be refined until a satisfactory set of assessment criteria has been created.
Assessment criteria should reflect the overall, published, aims of the program. If, for
example, the course claims to prepare students for entry into a particular profession, then the
achievement of the entry requirements for that profession should be specified in the assessment
criteria. The criteria must be informed by the published learning outcomes of the module. They
should not, however, merely repeat what has been stated as learning outcomes but must expand
on these to make clear how and to what extent the student is expected to use particular skills or
knowledge in order to meet these outcomes.
Assessment criteria should reflect the level of the module. Higher level modules will
generally require more complex analytical skills and greater depth of knowledge than lower level
ones. This must be reflected in the language used to write the criteria, with more descriptive
verbs such as ‘define’ or ‘describe’, giving way to increasingly sophisticated analytical and
critical ones such as ‘compare’, ‘evaluate’ and ‘critique’.
The criteria must reflect the distinctive epistemological characteristics of the particular
subject or discipline being assessed. Assessment criteria must be comparable to standards set in
other institutions offering the same award. Whilst each course will have, and should retain, its
distinctive individual features, the meaningfulness of any qualification depends on it
representing the same value wherever it has been obtained.
 Assessment criteria need to relate to the specific requirements of the assessment task i.e.
they should describe the performance required for the task set. Oral presentation criteria will be
quite distinct from the criteria set for an essay or portfolio.
Using assessment criteria
Assessment criteria are chiefly of value in so far as they enable students to focus their learning
more effectively and make the assessment process more transparent and fair. For this reason, if
no other, the expected outcomes and assessment criteria for any module should be discussed with
students before they are expected to undertake any assessed work. Such discussions can be
facilitated using the following structure:
 decide on the essential criteria
 make the criteria or checklist simple to use
 allow for brief global impressions
 give the criteria to the students before they do the assignment
 if possible, involve them in the design of the criteria and checklist
 encourage students to use the criteria.
The essentials of good criteria are that they:
 match the assessment task and learning outcome
 enable consistency of marking
 can pinpoint areas of disagreement between assessors
 help students to achieve the learning outcomes
 be used to provide useful feedback to students

Bloom’s Taxonomy of cognitive objectives has been around for a long time. Since 1956,
it has served as a guide for teachers to think about how they can design lessons that will help
their students to think critically. Basically, the taxonomy designed by Benjamin Bloom and his
colleagues provides a way to describe levels of thinking. The taxonomy is essentially a
hierarchy, with knowledge as the first level and evaluation as the sixth level. I’ve listed the six
levels below and included an example of each in parentheses.
 Knowledge – recalling information (e.g. answering comprehension questions from a reading)
 Comprehension – interpreting information (e.g. discussing why a character behaved in a
particular way)
 Application – using knowledge gained to solve problems (e.g. applying information from one
situation to a different situation in a debate activity)
 Analysis – breaking down concepts or ideas to understand the relationship of the parts to the
whole (e.g. analyzing prefixes to see how word meanings change)
 Synthesis – putting together something original from learned information (e.g. writing an essay;
making an oral presentation)
 Evaluation – judging something against specific criteria (e.g. peer editing using a checklist or
rubric)
Bloom’s Taxonomy has had tremendous influence in assisting teachers of any subject
matter to design instructional activities that cover the six levels of the hierarchy. It has also
inspired others to offer their own ‘take’ on critical thinking. Unrau (1977), for example, believes
teachers need to help their students develop a disposition – or inclination – to think
critically. What does it mean to have a disposition to think critically? Some examples are
 Imagine alternative solutions and perspectives
 Make an effort to persevere in acquiring and integrating knowledge
 Play with ideas
 Evaluate the consequences of beliefs, decision, and actions
 Reflect on one’s own thinking and that of others in order to gain knowledge or oneself and
others.
 LECTURE 7
CRITERIA-BASED ASSESSMENT OF READING SKILLS
Reading comprehension offers a tool for judging the level of passage or text
understanding while reading. Effective teaching reading comprehension strategies offer various
modules to enhance this skill combining vocabulary, fluency, phonics and interpretation skills.
Effective reading comprehension is the culmination of mastering vocabulary, phonics, fluency
and reading comprehension skills. Person having good comprehension skills is considered as
active reader, with an ability to interact with the words by understanding its complete meaning
and the concept behind it.
Thus skill of reading comprehension distinguishes an active reader from a passive reader
who just read the text without getting its meaning.
Aims of Teaching Reading Comprehension

 To get better grasping of the context, sequence and the characters narrated in text.
 Certain parts of the text can confuse readers. Reading comprehension skills works on this
aspect to get a clear idea of the meaning of the text.

 Helps to create the questionnaire based on the text about its theme or idea. It often helps in
better understanding of the said paragraph.
 It helps to link the event of narration with our previous experiences and predict the next
probable event in the course based on the information given in the narration.

Testing
 Testing comprehension reading has always proved a great tool in the assessment of the
student's abilities as it provides a feedback on his progress. It also enhances the self-
ability to judge ourselves, provided such tests are carefully designed.
 The carefully designed comprehension test is a cleverly constructed set of questions
targeted at the summary, overall meaning of text including most important meanings of
words. The questionnaire can be of different types like open-ended question, closed
formats or multiple choice questions.
Reading assessments are used for many purposes, but all appropriate uses begin from an
understanding of the reading construct, an awareness of the development of reading abilities, and
an effort to reflect the construct in assessment tasks. The complexity of the construct of reading,
as well as its development, also reveals the potential complexity of reading assessment. Reading
assessments are meant to provide feedback on the skills, processes, and knowledge resources that
represent reading abilities, though it is important to note that different assessment practices may
assume different theories of reading and reading development.
Assessment in general can be categorized in a number of ways, and all assessment frameworks
serve important purposes. Commonly, assessment has been categorized in terms of (a) norm-
reference and criterion- reference testing; (b) formative and summative assessment; (c) formal
and informal (or alternative) assessment; and (d) proficiency, achievement, placement, and
diagnostic assessment.
Five purposes for reading assessment
1. Reading-proficiency assessment (standardized testing)
2. Assessment of classroom learning
3. Assessment for learning (supporting student learning is the purpose)
4. Assessment of curricular effectiveness
5. Assessment for research purposes
There is an inevitable overlap among specific test uses across these categories, but these
categories, nonetheless, serve as a useful framework for organizing reading assessment.
Reading Profiles
A reading profile is a picture of a reader's strengths and needs in the component skills. It can be
presented in several ways.
Here is an example of a reading profile presented as a table:
Reading Skill (Component) Score (Grade Equivalent)

Word Recognition 4.0

Spelling 2.0

Word Meaning (Vocabulary) 10.0

Silent Reading Comprehension 8.0

Oral Reading Rate 120 words per minute

Reading-proficiency assessment. Assessment of reading proficiency is important as a way to

understand students’ overall reading abilities (based on some assumed construct of reading) and
to determine if students are appropriately prepared for further learning and educational
advancement. Commonly, this type of assessment is referred to as standardized testing, although
local groups and researchers also develop proficiency tests of different types. In most respects,
proficiency assessment represents high-stakes testing because decisions are often made about
students’ future educational goals and opportunities. Alternatively, this type of assessment may
lead to special education or reading-disability designations - labels that, once applied, are hard to
remove from a student’s record. Reading proficiency assessment is also sometimes used for
student placement, for policy decisions, for curriculum changes, or for program, teacher, or
institutional evaluations. Assessment of classroom learning Assessment of reading improvement
in classroom settings involves the measurement of skills and knowledge gained over a period of
time and is commonly referred to as summative or achievement testing. Sometimes, proficiency
assessments are used to measure student progress from year to year (as in a final exam), but this
type of reading assessment does not capture ongoing student gains made in reading skills in the
classroom. Year-end testing actually measures growth in proficiency from year to year rather
than measuring gains in reading abilities based on what was taught in class. Much more
commonly, assessment of classroom learning uses tasks that reflect the material taught in class
and the skills practiced. Typically, the teacher, teacher groups, or curriculum groups (or
textbook-materials writers) develop these tests, and they are responsible for deciding what
represents a measure of success, as well as what steps to take as a result of assessment outcomes.
IELTS READING TEST
Test format (60 mins)
There are three sections, each containing one long text. The texts are all real and are
taken from books, magazines and newspapers. They have been written for a non-specialist
audience and are on academic topics of general interest, which means you do not need specialist
knowledge to do well. The texts are appropriate to, and accessible to, candidates entering
undergraduate or postgraduate courses or seeking professional registration. Texts range from the
descriptive and factual to the discursive and analytical. Texts may contain nonverbal materials
such as diagrams, graphs or illustrations. If texts contain technical terms, then a simple glossary
is provided.
Marking
Each correct answer receives one mark. Scores out of 40 are converted to the IELTS 9-band
scale. Scores are reported in whole and half bands.
Reading assessment criteria
Has fully operational command of the language: appropriate, accurate and fluent with
9 40/40
complete understanding.
Has fully operational command of the language with only occasional unsystematic
8 35/40 inaccuracies and inappropriacies. Misunderstandings may occur in unfamiliar
situations. Handles complex, detailed argumentation well.
Has operational command of the language, although with occasional inaccuracies,
7 30/40 inappropriacies and misunderstandings in some situations. Generally handles complex
language well and understands detailed reasoning.
Has generally effective command of the language despite some inaccuracies,
6 23/40 inappropriacies and misunderstandings. Can use and understand fairly complex
language, particularly in familiar situations.
Has partial command of the language, coping with overall meaning in most situations,
5 15/40 although is likely to make many mistakes. Should be able to handle basic
communication in own field
Basic competence is limited to familiar situations. Has frequent problems in
4 10/40
understanding and expression. Is not able to use complex language
Conveys and understands only general meaning in very familiar situations. Frequent
3 6/40
breakdowns in communication occur.
No real communication is possible except for the most basic information using
2 3/40 isolated words or short formulae in familiar situations and to meet immediate needs.
Has great difficulty understanding spoken and written English
1 1/40 Essentially has no ability to use the language beyond possibly a few isolated words.
0 0/40 No assessable information provided.
 LECTURE 8
CRITERIA-BASED ASSESSMENT OF WRITING SKILLS
It is often assumed that the best way to test writing ability is to get the learners to write. 
That may seem obvious but, particularly at lower levels, being asked to write a whole, properly
staged and formed text from scratch is a daunting and challenging task which may demotivate
and depress our learners.  The other issue is that reliably evaluating our learners' ability with a
malformed, disorganized and inaccurately written text as the only data source is almost
impossible.
All assessment starts (or should start) from a consideration of the aims of instruction.
For example, if the (or one) aim of a language course is to enable the learners to do well in an
IELTS academic writing examination then this will be very influential in terms of the types of
assessment tasks we use and the way in which we measure performance.  The backwash (or
washback, if you prefer) from the examination format will almost inevitably have to be reflected
in the task types we set. If, on the other hand, our aim is to enable the learners to operate
successfully in a work environment then we will set different kinds of assessment tasks and
measure performance against different criteria.  In this case, a priority will be to measure how
accurately and with how much communicative success the learners can handle the specific
register and functions required by their work context. Finally, if we are aiming at enabling our
learners to function adequately in an English-speaking environment (perhaps as an immigrant or
temporary resident), then this, too, will fundamentally affect the tasks we set and the benchmarks
against which we measure success.  Here, for example, we might be dealing with form filling,
work getting and accessing services.
Three basic tenets
1. We have to use assessment tasks which properly reflect the kinds of writing tasks the
learners will need to be able to do in 'the real world'.
2. We need to design tasks which accurately show the learners' ability.
3. We need to have a reliable way to score the learners' performance.
The first step is to set out exactly what writing skills are the objects of the teaching
program.  To do this, we have to answer these 4 questions (which come with a few examples of
the kinds of answers we might get):
What sorts of texts will our learners need to be able to produce?  I.e., in which genre(s)
will they need to function?
o Emails / letters (formal or informal, interactional or transactional)?
o Blogs?
o Notes?
o Form filling?
o Announcements?
o Advertisements?
o Academic essays?  In which subject areas?
o Descriptions?
2. What are the purposes of the learners' writing?
o For basic communication with colleagues?
o For academic purposes?
o For social purposes in an English-speaking setting?
o For informal or formal interpersonal communication?
3. What functions do the learners need to be able to perform?
o Expressing needs?
o Stating opinion?
o Apologising?
o Providing information?
o Eliciting information?
o Commenting on opinions?
4. What topics will our learners have to be familiar with?
o Social matters?
o Dealing with officialdom?
o Service providers?
o Entertainment and leisure?
o Travel arrangements?
o The media?
o Medical matters?
o Specific study subjects?
o Specific occupational areas?
Identifying text types
The first step is to find out what sorts of texts the learners will need to be able to
produce.  This is by no means an easy undertaking, especially if the course is one in General
English (also known as ENOP [English for No Obvious Purpose]) when it is almost impossible
to predict what sorts of texts, for what purposes the learners may one day need to write.
On courses for more specific purposes, such as in-company teaching or examination
preparation, it is easier to identify the sorts of texts the learners will write and the purposes to
which they will put them.  For example, if the writing the learners will need to produce is
confined to occupational purposes, it is less likely that the ability to write a narrative, an
exposition or a discussion but the ability to write a recount (as a report), and information report,
a procedure and an explanation may all be important.
Before we can begin to assess our learners' writing skills, therefore, it is important to define
precisely what genre(s) they need to be able to handle.  There is very little point in, for example,
testing the learners' ability to write a narrative if they are never going to be asked to write one.
Designing tasks
Now we know what sorts of thing we want to assess, the purposes of the writing, the
functions the learners need to be able to perform and the topics they need to deal with (i.e., the
register), we can get on and design some assessment procedures. Again, there are some generic
guidelines for all tasks, whatever the answers to the 4 questions above are. If you have followed
the guide to testing, assessment and evaluation (see above), you will know that this is something
of a balancing act because there are three main issues to contend with.
1. Reliability:
A reliable test is one which will produce the same result if it is administered again (and again).
In other words, it is not affected by the learner' mood, level of tiredness, attitude etc.
We can, of course, set a single mega-task and get the learners to write for hours to complete it
but we risk losing reliability.  If any or some of the learners find the topic unappealing, the task
too daunting or the content irrelevant to them, they will almost certainly perform below their
potential.To ensure that a writing test is reliable, then, we need to set as many tasks as we can
over a period of days or even weeks. This will mean that even if a particular learner in the test
group is feeling glum and demotivated when presented with one task, the results will even out
over the whole assessment. It also means that if a particular task is unappealing to some of the
learners for whatever reason, other tasks will appeal more and balance the results. The type of
marking that is required also plays a role here: the more subjective the marking is, the less
reliable will be the test. As a general rule, tests which ask the learners to write in full the sorts of
text that form the target of the teaching program will be less reliable and more difficult to
evaluate than those which break down the skill and test discrete parts of it separately.
2. Validity:
3. Three questions here:
1. does the test measure what we say it measures? For example, if we set out to test someone's
ability to construct a discussion text for academic purposes, do the test items we use actually test
that ability or something else? Again, as a general rule, testing the whole skill, rather than
breaking it down will result in greater validity, at the expense of some reliability.
2. does the test contain a relevant and representative sample of what it is testing?
For example, if we are testing someone's ability to write a formal email, does the task we set get
them to deploy the sorts of language they actually need to do that?
3. do we have enough tasks to target all the skills we want to teach?
For example, if we want to test the ability to summarize information, provide information and
elicit information, do we have tasks focused explicitly and discretely on each area?
4. Practicality:
Against the two main factors, we have to balance practicality.
It may be advisable to set as many different tasks as possible to ensure reliability and to try to
measure as many of the skills as possible in the same assessment procedure to ensure validity but
in the real world, time is often limited and concentration spans are not infinite.
Practicality applies to both learners and assessors:
a. for learners, the issue is often one of test fatigue.
Too many tests over too short a time may result in learners losing commitment to the process.
On shorter courses, in particular, testing too much can be perceived as a waste of learning time.
b. for the assessors, too many open-ended test tasks which need careful marking may put an
impractical load on time and resources.  This may be compounded by task types which require
subjective judgements in marking.  Assessors may become tired and unreliable markers.
Other assessment task types
It may be that your circumstances allow for very simple writing tasks such as those requiring
the learners to write an e-mail explaining a set of given factors, an essay discussing pros and
cons or one advocating a point of view.  Those are all legitimate tasks providing the task type
and content suits the purposes of assessment. There are other ways. No list can be complete, but
here are some other ideas for other ways to set writing tasks for assessment purposes.  The
content of any task will of course, depend on all the factors discussed so far.
1. Form filling tasks
2. Compare and contrast tasks
3. Charts and tables
4. Simple graphics and picture sequences
5. Free writing tasks
6. Free writing tasks
Measuring outcomes
Unless you have a clear way of measuring outcomes, all the work on assessment task
design is wasted.  Ways of getting clean data are discussed in the guide to assessment in general. 
For writing in particular, however, there are some conventional approaches which fall into two
broad categories.
Holistic assessment
This involves a simple scale, perhaps from 1 to 10, on which the product from each learner is
placed on an impression of how well they achieved the task(s).
With a small group and with assessors who have a clear idea of what they are looking for, it can
be quick, reasonably accurate and efficient. There are obvious drawbacks in that judgments can
be subjective (so double or even triple marking can be used) and it lacks clear criteria against
which learners' performances are being measured.
Analytic assessment
Involves breaking down the tasks and being specific about the criteria you are using to judge
success. Large-scale examining boards use this approach as a matter of course.  For example, the
current assessment criteria for the IELTS examination stretch across 10 bands of ability and fall
into 4 assessment criteria groups.  These are:
-Task achievement
-Coherence and cohesion
-Lexical resource
-Grammatical range and accuracy
Here's an example of one level of achievement:
Coherence and Lexical Grammatical range
Band Task achievement
cohesion resource and accuracy
covers the requirements logically organises uses a sufficient uses a variety of
of the task information and range of complex structures
(Academic) presents a ideas; there is clear vocabulary to produces frequent
clear overview of main progression allow some error-free sentences
trends, differences or throughout flexibility and has good control of
stages uses a range of precision grammar and
(General Training) cohesive devices uses less punctuation but
presents a clear purpose, appropriately common lexical may make a few
with the tone consistent although there may items with some errors
7
and appropriate be some under-/over- awareness of
clearly presents and use style and
highlights key collocation
features/bullet points but may produce
could be more fully occasional
extended errors in word
choice, spelling
and/or word
formation
 LECTURE 9

CRITERIA-BASED ASSESSMENT OF SPEAKING SKILLS

Learning how to teach and assess speaking skills is probably one of the biggest challenges
compared to the other three language skills because you have to pay attention to aspects such as:

 Fluency:  This means speaking easily, reasonably quickly and without having to stop and

pause a lot.
 Pronunciation: The act or result of producing the sounds of speech, including
articulation, stress, and intonation.
 Vocabulary: The body of words used in a particular language.
 Accuracy: This refers to how correct learners’ use of the language system is, including
their use of grammar, pronunciation and vocabulary.
 Interaction:  This refers to the ability to interact with others during communicative
tasks.
 Communication: This refers to the students’ ability to transmit her/his ideas.

5 Types of Activities to Assess Speaking Skills

Intensive Speaking
A read aloud Task: Teacher listen to a recording and evaluate the students in a series of
phonological factors and fluency.  
Some variations of this task are:
 reading a scripted dialogue with someone else
 reading  sentences containing minimal pairs
 reading information from a table chart
 Sentence/ dialogue completion task: Students read through the dialogue so he can think
about proper lines to fill in. The teacher produces one part orally and the students
responds
 Picture cued Tasks: The picture-cued requires a description from the test taker
Responsive Speaking
Question  and answer: Students respond questions that the test administrator asks
Giving Instructions and Directions: The test-taker is asked to give directions or instructions
Paraphrasing: The test-taker is asked to paraphrase in two or three sentences what he heard or
read.
Interactive Speaking
Interview: It is a face-to-face exchange between test administrator and test taker. The  stages of
an interview are
 Warm-up
 Level Check
 Probe
 Wind-down
Role play is a common pedagogical activities used in communicative English classes
Discusssions and Conversations: These two speaking tasks provide a level of authenticity and
spontaneity that other assessment techniques may not provide
Games are an informal assessment task but they are not commonly used.
Extensive Speaking
Oral Presentations are the most common task for evaluating extensive speaking, these are
evaluated based on content and delivery.
Picture-cued story telling:  Students describe a story based on series of pictures that they
previously saw.
Re-Telling a story, News Event: Students are asked to tell a story of a new of something they
heard or read.
Imitative speaking:
Imitative speaking tasks are based on repetition. You just need to repeat a sentence you hear.
 LECTURE 10
CRITERIA-BASED ASSESSMENT IN PROJECT ACTIVITY
Project work challenges students to think beyond the boundaries of the classroom,
helping them develop the skills, behaviors, and confidence necessary for success in the 21st-
century. Designing learning environments that help students question, analyze, evaluate, and
extrapolate their plans, conclusions, and ideas, leading them to higher–order thinking, requires
feedback and evaluation that goes beyond a letter or number grade. The term “authentic
assessment” is used to describe assessment that evaluates content knowledge as well as
additional skills like creativity, collaboration, problem-solving, and innovation.
Authentic assessment documents the learning that occurs during the project-building
process and considers the real-world skills of collaboration, problem solving, decision making,
and communication. Since project work requires students to apply knowledge and skills
throughout the project-building process, you will have many opportunities to assess work
quality, understanding, and participation from the moment students begin working.
For example, your evaluation can include tangible documents like the project vision, storyboard,
and rough draft, verbal behaviors such as participation in group discussions and sharing of
resources and ideas, and non-verbal cognitive tasks such as risk taking and evaluation of
information. You can also capture snapshots of learning throughout the process by having
students complete a project journal, a self-assessment, or by making a discussion of the process
one component of the final presentation.
Developing Assessment
As you design the project, it is helpful to begin with the end in mind. What performances
do you want to see? Then, determine exactly how students will demonstrate each performance as
they build a product or solve a problem to complete the task.
Most of our assessment focuses on content mastery. Techniques we are all familiar with include
the evaluation of the final product and having students complete quizzes or tests. Other
benchmarks for content mastery you can use include the number of citations a student references,
amount and quality of research, use of experts, validity and effectiveness of arguments, meeting
the topic, and answering the essential question.
Completing complex authentic projects that require collaboration, creativity, problem-
solving, and innovation helps prepare students for increasingly complex life and work
environments. Effective communication in the 21st-century requires that students can effectively
express themselves in writing, verbally, and visually. Be sure to assess the quality of writing,
including ideas, vocabulary, fluency, organization, and conventions, as well as the use of media
and overall design. Since a project is a collaborative effort that occurs over time, include
evaluation components that consider teamwork, organization, planning, and behavior.
Questions for Students
Content Knowledge
 What new content did you learn while working on this project?
 Did you know more or less than you expected?
 What surprised you?
 What else would you like to know about the topic?
Collaboration & Teamwork
 How did your work and actions contribute to your team’s success?
 What was the hardest part of about working in a team?
 What was the best part?
Technology & Communication
 What new skills did you learn?
 What else do you want to learn how to do?
 Creating Rubrics
Because many performances cannot easily be quantified, you want to be as specific about
your expectations as possible. Creating a rubric for the final product and various components of
project work can ensure a more accurate, specific, and useful assessment.
A rubric is an authentic assessment tool that:
 Provides clear expectations for a project.
 Examines the product as well as the entire project-building process.
 Enumerates the performances on which students will be evaluated.
 Explains what constitutes excellence during the project process.
 Helps students understand what they need to do to excel.
 Helps remove subjectivity and bias from the evaluation process.
Sharing and clarifying the performances that will be assessed during a project removes mystery
from the evaluation process, helping students focus on specific actions they can take to improve
their performance.
Involving Students in Assessment
Involving students in project assessment boosts motivation, improves meta-cognition, and
promotes self-directed learning. Students who are asked to reflect on their own performance
learn to evaluate their strengths and weaknesses and are able to pinpoint where to focus their
efforts to see the greatest results.
You might have students provide feedback and critiques by asking them to keep a project journal
or work log, evaluate themselves using the project rubric, and answer additional self-assessment
questions. An open-ended self-assessment allows students to share learning that occurred during
the process that was not included in the rubric. As they reflect and evaluate, students should
describe their learning and contemplate decisions they have made individually and as a team.
The complexity of student projects makes assessment that captures both the final product
and the learning that occurs along the way an intricate and sometimes difficult task. Summative
assessment can be an effective component of an overall assessment strategy. Authentic
assessment can be used during the project-building process. Rubrics, ideally developed with the
help of the students, can help to evaluate how successfully students address specific goals and
performances. Self-reflection gives students a means to determine what they think they have
learned and how well they have learned it. Crafting assessment strategies that combine all of
these methods helps us gain a much better understanding of the learning that takes place during
the entire process.
 LECTURE 11
CRITERIA-BASED ASSESSMENT OF LISTENING COMPREHENSION SKILLS
Being able to listen well is an important part of communication for everyone. For our
students, guided practice by a teacher is one of the best ways to improve this skill. A student
with good listening comprehension skills will be able to participate more effectively in
communicative situations. What follows is an exploration of areas that language teachers may
find useful in their classroom and when preparing listening materials.
Teaching the skill of listening cannot be emphasized enough in a communicative classroom. For
second language learners, developing the skill of listening comprehension is extremely
important. Students with good listening comprehension skills are better able to participate
effectively in class 
Basic stages of Listening 
(1) You recognize speech sounds and hold a temporary “imprint” of them in short-term memory.
 (2) You simultaneously determine the type of speech event (monologue, interpersonal dialogue,
transactional dialogue) that is being processed and attend to its context (who the speaker is,
location, purpose) and the content of the message.
3) You use (bottom-up) linguistic decoding skills and/or (top-down) background schemata to
bring a plausible interpretation to the message, and assign a literal and intended meaning to the
utterance.
4) In most cases (except for repetition tasks, which involve short-term memory only), you delete
the exact linguistic form in which the message was originally received in favor of conceptually
retaining important or relevant information in long-term memory.
Each of these stages represents a potential assessment objective 
Potential Assessment Objectives:
1) comprehending of surface structure elements such as phonemes, words, intonation, or a
grammatical category
2) understanding of pragmatic context
3) determining meaning of auditory input, and
4) developing the gist, a global or comprehensive understanding
Assessment Tasks and Procedures
Types of Listening Performances
1) Listening for perception of the components (phonemes, words, intonation, discourse markers,
etc.) of a larger stretch of language.
2) Listening to a relatively short stretch of language (a greeting, question, command,
comprehension check, etc.) in order to make an equally short response.
3) Processing stretches of discourse such as short monologues for several minutes in order to
“scan” for certain information. The purpose of such performance is not necessarily to look for
global or general meanings, but to be able to comprehend designated information in a context of
longer stretches of spoken language (such as classroom directions from a teacher, TV or radio
news items, or stories).
Assessment tasks in this type of listening could ask students, for example, to listen for names,
numbers, a grammatical category, directions (in a map exercise), or certain facts and events.
 Listening to develop a top-down, global understanding of spoken language. This kind of
listening performance ranges from listening to lengthy lectures to listening to a conversation and
deriving a comprehensive message or purpose. Listening for the gist, for the main idea, and
making inferences are all part of extensive listening.
 Listening to develop a top-down, global understanding of spoken language. This kind of
listening performance ranges from listening to lengthy lectures to listening to a conversation and
deriving a comprehensive message or purpose. Listening for the gist, for the main idea, and
making inferences are all part of extensive listening.
 Responsive listening testing has a question-and-answer format. The test-taker is required
to find the appropriate response. The test has a multiple-choice format (with answers that
seemingly have similar meanings) or requires a more open-ended framework.
Implies listening to a text with the purpose of scanning for certain details or information
 Extended listening involves the comprehension and reproduction in writing of a
moderately large spoken passage (dictation), generally of about 50 to 100 words. A variant of
this test is answering comprehension questions after listening to the passage several times.
Validity in listening assessment
 measure comprehension (not hearing, spelling, prior knowledge of a topic or reading long
multiple choice questions)
 Base assessment on the learning objectives and listening tasks of the unit/course
Reliability in listening assessment
 Minimize anxiety
 Ensure all learners can hear/see the
text/video equally and that there are no
distracting noises
 Avoid ambiguous or ‘trick’ test items Ensure more than one scorer for correcting open-
ended test items
Authenticity in listening assessment
 Use texts with authentic, real-life speech
 Avoid using texts that are dense and cognitively demanding (meant to be read and not
listened to)
 Choose comprehension tasks that reflect real-life purposes for listening
 Avoid difficult accents and dialects
Steps in designing listening tests
 Identify the purpose of the listening test, keeping in mind learner goals. The listening test
should reflect what students are learning and what they need.
 Decide on the format for test items and create the test form. 
Create or secure listening passages to be used in the test. Many commercial programs come
with audio files. For authentic materials, you could record a radio segment or take advantage of
listening materials available on the Internet. Universities and professional language organizations
often have recorded language samples on their websites.
Steps in designing listening tests
 Be sure that instructions are clear. Do not risk letting students' misunderstanding of test
directions get in the way of assessing listening skill.
Ensure optimal listening comprehension by framing the task to activate background information
that will aid in comprehension. For example, you could introduce a listening segment by saying,
"In the passage you will hear, two friends are having a conversation in a train station." Give step-
by-step instructions for more complex listening tasks
Steps in designing listening tests
Determine scoring criteria. Scoring true/false or multiple choice items is uncomplicated, but
asking students to take notes on a passage or write a summary presents challenges.
Activity types for testing listening comprehension
Information transfer (drawing a route on a sketch/map, labeling diagrams/pictures, completing a
form/a table)
“Picture dictation”
An incomplete picture or a blank page (students listen to the description and draw the picture)
Example:
a. The listeners listen and fill in details on the picture.
b. The learners listen and label parts of a picture or diagram.
 Types of listening test items
 Oral cloze
 Picture ordering
 What is it?
Example: I forgot it when I left home this morning. This made me angry because it is useful. I
don’t like it very much but I need it. Not every person has one, but I think most people do. Some
people like to look at it and now many people play with it. Mine is quite heavy….
 LECTURE 12
CRITERIA-BASED ASSESSMENT OF TESTING
Criterion-referenced tests and assessments are designed to measure student performance
against a fixed set of predetermined criteria or learning standards—i.e., concise, written
descriptions of what students are expected to know and be able to do at a specific stage of their
education. In elementary and secondary education, criterion-referenced tests are used to evaluate
whether students have learned a specific body of knowledge or acquired a specific skill set. For
example, the curriculum taught in a course, academic program, or content area.
If students perform at or above the established expectations—for example, by answering
a certain percentage of questions correctly—they will pass the test, meet the expected standards,
or be deemed “proficient.” On a criterion-referenced test, every student taking the exam could
theoretically fail if they don’t meet the expected standard; alternatively, every student could earn
the highest possible score. On criterion-referenced tests, it is not only possible, but desirable, for
every student to pass the test or earn a perfect score. Criterion-referenced tests have been
compared to driver’s-license exams, which require would-be drivers to achieve a minimum
passing score to earn a license.
Criterion-referenced tests may include multiple-choice questions, true-false questions,
“open-ended” questions (e.g., questions that ask students to write a short response or an essay),
or a combination of question types. Individual teachers may design the tests for use in a specific
course, or they may be created by teams of experts for large companies that have contracts with
state departments of education. Criterion-referenced tests may be high-stakes tests—i.e., tests
that are used to make important decisions about students, educators, schools, or districts—or
they may be “low-stakes tests” used to measure the academic achievement of individual
students, identify learning problems, or inform instructional adjustments.
Well-known examples of criterion-referenced tests include Advanced Placement
exams and the National Assessment of Educational Progress, which are both standardized
tests administered to students throughout the United States. When testing companies develop
criterion-referenced standardized tests for large-scale use, they usually have committees of
experts determine the testing criteria and passing scores, or the number of questions students will
need to answer correctly to pass the test. Scores on these tests are typically expressed as a
percentage.
While criterion-referenced test scores are often expressed as percentages, and many have
minimum passing scores, the test results may also be scored or reported in alternative ways. For
example, results may be grouped into broad achievement categories—such as “below basic,”
“basic,” “proficient,” and “advanced”—or reported on a 1–5 numerical scale, with the numbers
representing different levels of achievement. As with minimum passing scores, proficiency
levels are judgment calls made by individuals or groups that may choose to modify proficiency
levels by raising or lowering them.
The following are a few representative examples of how criterion-referenced tests and
scores may be used:
 To determine whether students have learned expected knowledge and skills. If the criterion-
referenced tests are used to make decisions about grade promotion or diploma eligibility, they
would be considered “high-stakes tests.”
 To determine if students have learning gaps or academic deficits that need to be addressed. For a
related discussion, see formative assessment.
 To evaluate the effectiveness of a course, academic program, or learning experience by using
“pre-tests” and “post-tests” to measure learning progress over the duration of the instructional
period.
 To evaluate the effectiveness of teachers by factoring test results into job-performance
evaluations. For a related discussion, see value-added measures.
 To measure progress toward the goals and objectives described in an “individualized education
plan” for students with disabilities.
 To determine if a student or teacher is qualified to receive a license or certificate.
 To measure the academic achievement of students in a given state, usually for the purposes of
comparing academic performance among schools and districts.
 To measure the academic achievement of students in a given country, usually for the purposes of
comparing academic performance among nations. A few widely used examples of international-
comparison tests include the Program for International Student Assessment (PISA), the Progress
in International Reading Literacy Study (PIRLS), and the Trends in International Mathematics
and Science Study (TIMSS).
 LECTURE 12
CRITERIA-BASED ASSESSMENT IN STUDENTS` PORTFOLIO
Portfolio assessment is a term with many meanings, and it is a process that can serve a
variety of purposes. A portfolio is a collection of student work that can exhibit a student's efforts,
progress, and achievements in various areas of the curriculum. A portfolio assessment can be an
examination of student-selected samples of work experiences and documents related to outcomes
being assessed, and it can address and support progress toward achieving academic goals,
including student efficacy. Portfolio assessments have been used for large-scale assessment and
accountability purposes, for purposes of school-to-work transitions, and for purposes of
certification. 
Types of Portfolios
While portfolios have broad potential and can be useful for the assessments of students'
performance for a variety of purposes in core curriculum areas, the contents and criteria used to
assess portfolios must be designed to serve those purposes. For example, showcase portfolios
exhibit the best of student performance, while working portfolios may contain drafts that
students and teachers use to reflect on process. Progress portfolios contain multiple examples of
the same type of work done over time and are used to assess progress. If cognitive processes are
intended for assessment, content and rubrics must be designed to capture those processes.
How to Design a Classroom Portfolio Assessment
Portfolio assessments are quite varied; however, it is possible to describe the basic steps
for creating a portfolio assessment system in a classroom. For both learning and best-works
portfolios, the first thing to do is identify the purpose for the portfolio. That purpose should
include the classroom learning targets and curriculum or state goals it will serve, whether the
purpose will be formative (learning) or summative (best-works), and the audience (who will be
the users of the portfolio system—who creates and who gets the information and for what uses).
Possible purposes include, but are not limited to, the following:
 formative assessment-as a vehicle for student reflection and teacher feedback, for diagnosing
student needs and planning next steps, for informing instructional planning;
 summative assessment-of student accomplishment of learning goals, including grading;
 program evaluation and/or accountability reporting;
 communication- as a vehicle for communicating with parents, or with future teachers.
At this point, one possible decision is that a portfolio is not the most appropriate
assessment method for the purpose. If it is appropriate, continue with the following steps.
Specify in greater detail exactly what achievement dimensions the portfolio will assess. This
specification includes the content from the learning goals but also extends to the cognitive
complexity, reflection and self-regulation skills, or dispositions or habits of mind desired as the
focus of portfolio evidence. Because portfolios are a big undertaking and require significant
investments of student time and effort, portfolios should assess important, central learning goals
that require complex thinking and involve tasks that are meaningful to students. Portfolios are
not a good way to assess routine knowledge or recall of facts.
When these foundational issues have been decided and written down, more practical
planning for portfolios should follow. Plan the organization of the portfolio, defining for
example how many and what types of entries will be needed to give evidence of student content
knowledge and cognitive process for intended use of the information. Several pieces of
completed writing in different genres would establish student achievement of writing standards,
for summative evaluation and grading. Sets of drafts of individual papers at various stages
throughout the writing process, at the beginning, middle, and end of the year, would be more
useful for student reflection on how he or she was developing as a writer.
 Plan who will decide what goes in the portfolio, the timing of those entries, when and
how individual pieces will be evaluated, how the portfolio will fit into classroom routines,
whether there will be conferences associated with portfolio use, who may see the contents of the
portfolio, who owns the final contents, and so on.
Then plan the scoring or evaluating of the portfolio. For solely formative use, evaluation
may be entirely by feedback and conferencing. For most uses, rubrics of some sort will be used.
Identify or create rubrics that describe performance levels on attributes appropriate to the
portfolio’s purpose. For example, for a writing portfolio designed to furnish evidence of good
use of the writing process, the quality of drafting and editing might be evaluated along with the
quality of the finished pieces. For a writing portfolio designed to furnish evidence of finished-
product writing quality, only qualities of the finished pieces might be evaluated. With or without
rubrics, portfolios are also an excellent vehicle for teachers to give verbal feedback to students.
Teachers can provide written feedback on the portfolio itself, or, especially for younger students,
they can provide oral feedback using the portfolio as the focus of brief student conferences.
Holistic rubrics are used to evaluate the whole portfolio simultaneously. One rubric—one
dimension with a set of quality level descriptions—is used. This type of scoring is quick to use
and works well when the portfolio is used for summative purposes like a final grade. Analytic
rubrics are used to evaluate the portfolio on several dimensions, each with a set of quality level
descriptions. For example, a writing portfolio might have separate rubrics for content/ ideas,
organization, style/voice/word choice, and mechanics. Analytic rubrics take longer to use than
holistic rubrics because several judgments must be made. But they are better for giving feedback
to students because they identify areas of strength and weakness.
Classroom portfolio scoring will often be done by only the teacher. The dependability of
scoring can be checked by having another person cross-check a few portfolios. When double-
scoring is used, there are two approaches to dependability. One is independent scoring, then
calculating the percent of agreement. Another is a consensus approach, sometimes called
moderation, where any disagreements are discussed and resolved. Keeping in mind the question,
Would another person agree with this score? helps focus scoring even if there is no double-
scoring. Use clear rubrics, consistently applied, as if the scoring was going to be checked against
someone else’s.
The results of this planning process will be quite varied. Some portfolios have a cover
sheet or entry log as the first entry that functions as a table of contents. Entry logs sometimes
have space for other information; for example, the rationale for including the piece, the rubric
score for the piece, or the date an entry was put in the portfolio. Teachers sometimes make up
these sheets as a checklist, specifying two pieces of narrative writing and one piece of persuasive
writing and so on, with space for the student to name the selections. Other portfolios are much
less prescribed, with entries put in and removed over time as their usefulness expires. Such fluid
portfolios are better suited to formative assessment.
Some portfolios are in sections; for example, literacy portfolios may have a reading
section that contains a book log and a writing section that contains compositions. Some
portfolios are rather unified, such as math portfolios with entries that demonstrate solution of a
succession of different types of problems, each composed of paper-and-pencil work plus a
reflection about the work’s strengths and challenges. Other portfolios represent a variety of
work, like a science portfolio that includes unit tests, lab reports, photographs and essays about
projects, and reflections.
The reflection methods in portfolios also vary. Some portfolios have student reflections
on each entry, either on a separate sheet or on an attached sticky note. Reflections may be
required responses to teacher questions (e.g., Why did you select this piece? What does this
piece show about your learning?) or may be free-form. Formatted reflections sometimes even
include multiple-choice questions (e.g., How satisfied are you with your work on this piece? with
a list of choices). Usually, the teacher decides what kinds of reflections are required. Other
portfolios have overall reflections, done as an essay written after the work is collected and placed
either at the front or the back of the portfolio.
Implementation Issues, Dilemmas, and Barriers
Portfolios have developed a loyal following, but they have also raised issues. As noted,
scoring reliability is one of these issues. The most frequently reported difficulty is time.
Implementing portfolios well takes both a lot of planning and classroom time. Save portfolio
assessment for occasions when its advantages specifically match the assessment purpose, so the
time is well spent and portfolios are worth the investment.
Another barrier some teachers have reported is that using portfolio assessment has
implications for instruction, so instruction has to be changed. This is particularly problematic if
the instructional changes; for example, giving students control of selection and evaluation of
some or much of their work, go against the teacher’s style of teaching or personal values. Self-
reflection needs to be taught. Time needs to be arranged in classroom lessons for students to
work on their portfolios. Resources and materials need to be arranged.
A final issue for portfolios is that they require teacher professional development. Many
portfolio projects have a professional development component. There is a learning curve for
teachers as they make portfolios part of their teaching repertoire. Teachers go through stages of
development in their ability to use portfolio assessment skillfully. Experience is important for
getting the most out of portfolios, for doing them well, and for not stumbling over the potential
barriers.
Digital portfolios are attracting more and more attention. Electronic storage eliminates
one of the barriers to portfolio implementation—storage space. Digital portfolios, however, share
with other kinds of portfolios the purpose of driving content and plans for assessment. Clear
learning goals are still central. Going digital doesn’t change that. Entries in digital portfolios can
be constructed with the same building blocks that construct any electronic files, depending on
their nature: word processors, digital still or video cameras, spreadsheets, presentation software,
and the like. These files can be simply stored in a folder on a computer or, more and more
frequently, are stored in portfolio software that allows for organizing the artifacts, storing
reflections, and recording scores or teacher comments electronically.
Currently, software developers are marketing products that handle electronic portfolios, or e-
portfolios. Typically, these software products allow for storage of student work in the form of
electronic files and for scoring with rubrics. Some of these programs are Web-based. Some allow
the students to keep their electronic files, some retain ownership for the school, and some allow
access only with a current subscription. Beware of e-portfolio systems that claim to solve
portfolio problems beyond electronic storage and convenience. Setting purpose, ensuring that
portfolios are actually used as intended, identifying and using appropriate rubrics (if scored), and
so on, require human judgments that are the responsibility of the portfolio system users.
Is a Portfolio an Assessment?
Is a portfolio an assessment itself or something else? Some, most notably Stiggins (2005),
originally considered portfolios to be a communication method and not an assessment per se. He
saw a portfolio as a collection of individual assessment information whose purpose was to
communicate information about student achievement to teachers, parents, and students
themselves. Thus he classified portfolios as a communication tool in the same toolbox as grades,
narrative reports, and conferences.
The history of portfolios shows that they have been used as an assessment method, and
most educators now consider portfolios an assessment method in their own right. The failed
experiments with using portfolios for large-scale assessment certainly considered them as a
method in their own right—treating them like tests (for example, standardizing tasks and
directions) and using their results like they would any other assessment results for accountability
and reporting. The classroom uses to which portfolios have been put, whether formative or
summative, have also treated portfolios as an assessment in their own right.
Because of their nature, however, portfolios retain that overlap of instruction and assessment.
They contain real examples of student work that can be reviewed, rediscovered, and reinterpreted
—and those actions constitute a powerful kind of instruction. Thus while the consensus is that
portfolios are an assessment method, they are unusual in that their construction allows them to
float between instruction and assessment more easily than any other kind of assessment. Any
assessment use (for example going over classroom test results) has the potential to inform and
even become part of instruction. But portfolios take to this naturally.
Another difference between portfolios and many other types of assessment is the way they lend
themselves to multiple interpretations. Of course any assessment results do that to some extent.
Because of their history and nature, portfolios easily invite multiple interpretations. The original
uses of portfolios, for artists and others to display their work, were not scored, but rather
interpreted anew by each viewer. An architect reviewing a portfolio of interior designs might be
appraising the designer’s ability to fit into one project or to work in one company. The same
designer might show the same portfolio to a furniture manufacturer who reviews it to appraise
the designer’s ability to fit into an entirely different project or job.
As portfolios migrated into classrooms and were adapted for school use, the purpose of
the portfolio became defined by classroom learning targets. However they are called (objectives,
goals, targets), learning targets are the basis on which classroom instruction and assessment are
planned and are the building blocks for the curriculum that the classroom work serves. Thus
school portfolios, now defined as purposeful collections of work, usually serve the central
purpose of providing evidence and explanations of achievement of a set of learning targets. Even
so, because student work is multidimensional, there are lots of other things to see in it. A set of
papers that show a student has read and understood Hamlet, for example, might also show that
the student has a good sense of humor, or can write particularly moving narrative prose, or
makes a lot of punctuation errors, or any one of a number of things—large and small—that are
not directly related to the stated purposes of the portfolio.
Studies have investigated the effects of portfolio use on instruction. There is some
evidence that portfolios have an effect on instruction. Teachers often report that portfolios
facilitate learning by encouraging students to look back at their work and see where they have
been and how far they have progressed.
Studies have also investigated the effects of portfolio use on learning. There is some
evidence that students who use portfolios regularly—in portfolio assessment systems that are
well conceived and managed—increase their mastery goal orientation. That is, they learn the
value of learning for its own sake. Of course, not every portfolio user becomes a self-regulated,
self-evaluating, self-starter. But on average, portfolios can foster this kind of orientation over
test-driven instruction and assessment.
Some studies have investigated whether students who regularly use portfolio assessment
systems increase their achievement levels as measured by conventional standardized tests. These
results have been mixed. It is not clear whether the reason for the mixed results is that portfolio
use has no real effect on achievement or that standardized tests do not measure the kind of
complex learning that portfolios develop.
The term portfolio culture has been used to describe a classroom environment in which,
because of the use of portfolios, review and reflection about one’s work come to be valued. In
such an environment, it is safe for students to describe both strengths and weaknesses of a piece
of work. Assessment is seen as a repetitive process, with ongoing revision not only allowed but
valued. This contrasts with a classroom environment that values getting good scores for
everything (sometimes called a testing culture). Seeing errors or less than perfect work as an
opportunity for learning or information for improvement is important to a portfolio culture, and it
is also important for developing honest self-evaluation and self-regulation skills. The term is not
used as much in current literature as it was in the 1980s and early 1990s. The formative
assessment literature has picked up this thread for discussion, so the concept is still around.
 LECTURE 14
WAYS TO CREATE CRITERIA AND DESCRIPTORS
How to design rubrics
Rubrics take many different forms. Some rubrics describe only three levels of
performance; others have more. Sometimes the terminology used to describe the various parts of
the rubric will differ. None of this really matters. The important thing is that the purpose of the
rubric remains the same: it is used to make levels of performance explicit for both teacher and
student.
The first step in designing a rubric is to identify the skills, knowledge and understanding that the
teacher wishes the students to demonstrate. These become the success criteria.
For example:
An ability to …
Knowledge of …
Understanding of how/why/the ways in which …
The next step is to decide how many levels are to be described and then to write the
performance descriptors for each of those criteria. Some people find it easiest to start with the
medium level, which should represent a satisfactory level of performance, and proceed from
there to write the higher and lower level descriptors.
Writing performance descriptors
The performance descriptors describe the relative differences between performances at
each level. These are some of the ways that difference can be presented:
 by referring to specific aspects of the performance which will be different at different
levels.
For example, analyses the effect of …/describes the effects of … /lists the effects of …
 
 by using adjectives, adjectival phrases, adverbs and adverbial phrases. These extra
words are used where the aspects of a performance stay the same across the levels, but there is a
qualitative difference to the performance.
For example
- accurately explains/explains with some accuracy/explains with limited accuracy
- provides a complex explanation/provides a detailed explanation/provides a limited explanation
- shows a comprehensive knowledge/shows a sound knowledge/shows a basic knowledge
These descriptions need to be supported by work samples or exemplars in order to make the
differences clear to students.
 
 by using numeric references. Numbers identify quantitative differences between levels.
For example, 
- provides three examples/provides two examples/provides an example
- uses several strategies/uses some strategies/uses few or no strategies*
A word of warning: numeric references on their own can be misleading. They are best teamed
with a qualitative reference (three appropriate and relevant examples) to avoid ignoring quality at
the expense of quantity.
This kind of description needs to be supported by work samples or exemplars in order to
demonstrate exactly what is meant by 'several' etc.
 
 by referring to the degree of assistance needed by the student to complete the task. This
kind of performance descriptor is explicit about the degree of independence shown by the
 student while undertaking the task.
For example,
- correctly and independently uses/ with occasional peer or teacher assistance, uses/ with
teacher guidance, attempts to use
The consistent use of terminology when describing the parts of a rubric will assist in
developing with students a meta language that can later be referred to when providing feedback.
When teachers are uncertain of the terminology, or refer to parts of the rubric differently each
time, clarity is lost and students are confused. A cross-school agreement about the use of
terminology provides further consistency.
Characteristics of standards descriptors
The job of standards descriptors is twofold - firstly, and most importantly, they inform
students of the qualities and elements of their work that are being looked for in order to
determine how well they have performed against each criterion. Secondly, standards descriptors
assist markers in determining student grades by providing information about a typical, mid-level
achievement within each standard for each criterion. Therefore, standards descriptors: describe
evidence in the student's response describe the quality of the student's response in terms of the
criteria suited to the task give meaning to the mid-range or typical standards (HD-NN) use words
which are descriptive and comparative, NOT just comparative contain positive statements about
student achievement, use language that is not derogatory use unambiguous language which
students' understand.
Writing standards descriptors
When starting to write standards descriptors for a criterion, you should start by going
back to the Intended Learning Outcome that is being measured, and then writing a description of
what a student would need to do to meet the criterion sufficiently to demonstrate achievement of
the ILO. You may also wish to start by noting down all the elements that you expect for the
criterion. When you then reflect back on the minimum requirement to demonstrate achievement
of the ILO being measured, to what extent students need to include each element can then be
considered when writing the descriptor. 
Intended Learning Assessment Criterion Standard Descriptor - Pass
Outcome

Contextualise the role Advise a client about the Provides advice about more than one
of courts that deal with options available to them dispute resolution process option that
civil disputes within the civil justice and both accurately explains the process and
dispute resolution landscape applies the client's circumstances to
justify the recommendation.

Use Higher Education Explain how the activity is You described aspects of your teaching
theory, literature and appropriate for your context, student cohort and discipline.
practice to make and context, students and/or
support arguments for discipline You provided some explanation for why
teaching the proposed activity is appropriate for
your context, students and/or discipline.

Monitor and adapt Work with the director, text During rehearsals you adhered to all
performance skills in and production team during theatre rehearsal protocols and worked
response to various cooperatively with the director, other
audiences and non- the rehearsal process actors and the production team by:
theatrical spaces
contributing ideas that related to the
director's vision; and

taking direction

The standards descriptors above demonstrate that it is appropriate to directly address the
students, as well as being appropriate to simply refer to the work itself, without mention of the
student. There is a third option as well, not shown in these examples, where these are combined,
and refer to the students' work (e.g., your website ...). 
It is also equally appropriate to either use bullet points, or to identify key inclusions in
separate sentences, or within a single sentence. The most important thing to remember when
writing a standard descriptor is that it should make clear to students what they need to do within
their assessment, and how well they need to do it. The language, therefore, must be meaningful
to students and not include vague notions with variable interpretations.
Once you have a descriptor for the pass standard, the challenge is to describe three
standards that exceed this standard, at different levels. Often, writing the High Distinction
standard is the easiest place to start, as this is where a description of the ideal performance is
appropriate. This descriptor should include similar elements to the pass standard, albeit at a
much higher expectation of how well they are done. It is also suitable to have additional
elements. If you take the approach of writing the pass and then the high distinction standard
descriptors, it can be helpful to then describe the distinction standard as what is not up to the HD
level, and the credit either as what is not yet a DN, or what demonstrates a higher level of
achievement than a pass. An alternative approach is to start with the credit - describing what a
performance a step up from a pass looks like, then a step up from this to a DN, then a step up
from this, to an HD.
It is also important to describe the sort of performance or work that does not demonstrate
achievement of the ILO being measured - the fail standard. This is best written to describe what
the work does, rather than what it does not do - i.e., it should be written using positives as much
as possible.
There is no 'single' approach to take when writing standards descriptors, but it is
important that you moderate the rubric once it is complete, to be sure that it provides clarity for
students, and for markers as well.
 LECTURE 15
CRITERIA-BASED ASSESSMENT VS. TRADITIONAL
Traditional assessments refer to conventional methods of testing, usually standardized and use
pen and paper with multiple-choice, true or false or matching type test items. Authentic
assessments refer to assessments wherein students are asked to perform real-world tasks that
demonstrate meaningful application of what they have learned.
Traditional Assessment Authentic Assessment

Purpose: to evaluate if the students have Purpose: to measure students’ proficiency by asking
learned the content; to determine whether or them to perform real life-tasks; to provide students
not the students are successful in acquiring many avenues to learn and demonstrate best what
knowledge; to ascribe a grade for them; to they have learned; to guide instruction; to provide
rank and compare them against standards or feedback and help students manage their own
other learners learning; to also evaluate students’ competency

Provides teachers a snapshot of what the Provides teachers a more complete picture of what
students know the students know and what they can do with what
they know

Measures students’ knowledge of the content Measures students’ ability to apply knowledge of
the content in real life situations; ability to
use/apply what they have learned in meaningful
ways

Requires students to demonstrate knowledge Requires students to demonstrate proficiency by

by selecting a response/giving correct performing relevant tasks showing application of
answers; usually tests students’ proficiency what has been learned
through paper and pencil tests

Students are asked to choose an answer from

a set of questions (True or False; multiple
choice) to test knowledge of what has been
taught.

Provides indirect evidence of learning Provides direct evidence of learning/competency;

direct demonstration of knowledge and skills by
performing relevant tasks

Requires students to practice cognitive ability Provides opportunities for students to construct
to recall/recognize/reconstruct body of meaning/new knowledge out of what has been
knowledge that has been taught taught
Tests and strengthens the students’ ability to Tests and strengthens the students’ ability to reason
recall/recognize and comprehend content, but and analyze, synthesize, and apply knowledge
does not reveal the students’ true progress of acquired; Students’ higher level of cognitive skills
what they can do with the knowledge they (from knowledge and comprehension to analysis,
acquired. Only the students’ lower level of synthesis, application, and evaluation) are tapped in
thinking skills, (knowledge and multiple ways.
comprehension), are tapped.

Hides the test Teaches the test

Teachers serve as evaluators and students as Involves and engages the students in the teaching,
the evaluatees: teacher-structured learning and assessment process: student structured

Assessment is separated from teaching and Assessment is integrated with instruction.

learning. Test usually comes after instruction Assessment activities happen all throughout
to evaluate if the students have successfully instruction to help students improve their learning
learned the content. and help teachers improve their teaching.

Provides limited ways for students to Provides multiple avenues for students to
demonstrate what they have learned demonstrate best what they have learned

Rigid and fixed Flexible and provides multiple acceptable ways of

constructing products or performance as evidence of
learning

Standardized; valid and reliable Needs well defined criteria/rubrics and standards to
achieve reliability and validity

Curriculum drives assessment. Assessment drives curriculum and instruction.

Examples: Examples:

True or False; multiple choice tests demonstrations

standardized tests hands-on experiments

achievement tests computer simulations

intelligence tests portfolios

aptitude tests projects

 multi-media presentations

role plays

recitals

stage plays

exhibits

To better compare traditional vs. alternative assessments, there’s a table:

Advantages of Traditional Assessment Over Authentic Assessment:

Traditional Assessment Authentic Assessment

Advantages: Disadvantages:

Easy to score; Teachers can evaluate students Harder to evaluate

more quickly and easily.

Less time and easier to prepare; easy to Time consuming; labor intensive
administer Sometimes, time and effort spent exceed the
benefits.

Objective, reliable and valid Susceptible to unfairness, subjectivity, lacking

objectivity, reliability, and validity if not
properly guided by well-defined/clear criteria
or rubrics/standards