UNIT 12 TECHNIQUES AND TOOLS OF

EVALUATION
Structure
12.0 Introduction
12.1 Objectives
12.2 Types and Techniques of Evaluation
1
12.2.1 Formative and Sumrnative Approaches in Educational Evaluation
12.2.2 Sources of Evaluation

12.3 Criteria for Evaluation

12.4 Tools of Evaluation:Needand Importance
6
12.4.1 .Attributes of a Good Evaluation Tool
12.4.2 Sampl~ngMethods and Procedures

12.5 Types of Evaluation Tools

12.5.1 Questionnaire
12.52 Interview
12.5.3 Observation Schedule
12.5.4 Achievement Tests

12.6 Treatme,~t ;>ah

12.7 Summaj
12.8 Unit End Activities
12.9 References and Suggested Reading
12.10 Clues to Check Your Progress

12.0 INTRODUCTION
In the previousunitsofthis block, we have discussed the concept of evaluation. This
unit will help you in actuallycarrying it out.As a student of Educational Technology,
it is now important for you to understand how you can actually cany out evaluation.
Therefore, in this unit, we will discuss about the various tools and techniques, which
can be used to undertake evaluation. As you know, instruction may be provided
within classroom situations, in the face to face mode or it may be delivered through
technology. Again, technology mediated innstructions, may be impartedwithin the
classroom or through the distance mode. Irrespective of the mode and medium of
imparting instructions, evaluation remains an integral part ofteachingand learning.
Evaluation, as we have discussed in the earlier unit, is essentially a processs of
decision making. Teaching is an activity that requires you to constantly gather
information and make decisions.Teachers often need to make decision9 about their
students at the rate of one every 2 to 3 minutes (Stiggins,1999).To make teaching
effective, it is essential that good evaluationpractices are followed to gather pertinent
informationabout the learners.
Let us recapitulatethe meaning of evaluation.Evaluationis a systematic determination
of merit, worth, and significance of something or someone so as to reach cetrain
52 decisions in an objective manner.
 I I
I

We evaluate the performance of l e s m s , transaction of the Techniques and Tools

academic tasks in the classrooms, etc. I+ daily tasks, ideas, of valuation
facts, and arguments more critically.Thus and may
aim to assess their learning as discussed
cited by Best and Kahn (2006) has
the educational setting as follows:
a It serves as a means for
e It motivates studentsand
control for teachers.

ed out at various

After studyingthis unit, you should be able to:

a analyze the attribute of a good evaluation tool;

a apply various evaluation tools.

12.2 TYPES AND TECNIQUES OF EVALUATION +

YOU now know that'evaluationcan be for learning outc(,mes and also of the activities
or resources of an institution that imparts apart from learning
outcomes, teaching methods, educational such as audio-
video m r d i n g and storing facilities, video es produced, etc.
that are used to achieve the
Therefore, one can say that
to (vhich learning
In this section we
sumnative

Evaluation

1 the programme. It helps in judging the worth or quality of an e d cational

~ program 53
Evaluation Concepts while the various components viz. the instructionalstrategies,curriculum, courseware
( print/othermedia) of the programme are being designed and developed. Formative
evaluationthus facilitatesthe devlopment ofinstructionalproducts, even for wmputer-
based instructions. At various stages of development, courseware designers use
formative evaluation techniques to obtain feedback concerning the product from
clients, subject matter experts, colleagues, and learners. Feedback for formative
evaluation is collected in four ways: self-reporting, observation,tests, and records.
The feedback is used in the subsequent stages of the instmctional design process to
help improve the product (Weston 2004). Formative evaluationof learning, is carried
out while instruction is still being imparted on a particular topic(s). The purpose of
this kind of evaluation is to carry out an ongoing diagnosis and provide the feedback
for modifying or improvingthe product before it is given a final shape.
Formative evaluation of a coursewareusually assesses the following:
1
The authenticity and accuracy of the material, which is usually assessed by one
or more experts in the subjectmatter. Such a review is called an expert review.
The readability of the material, usually assessed through editing. Editors assess
the logic and flow ofthe content, its wnsistencywithin the same leaming program
I
and with related leaming programs, and usage, grammar and punctuation.
Editorial reviews are usually conducted by professionallytrained editors, or by
peers in the department that produced the materials.
The usability of the material by the intended audience, which is usuallyassessed
through an observation of those who aredemogrcphically similarto the intended
learners taking the course, as well as an assessment of scores on draft tests.
Formative evaluationstrategies are used routinely while developing computer-based
instructions or tutorials. For example if you want a lecture to be videotaped and
made available on a Web site, it would be preferable ifyou would conduct a formative
evaluation of the lecture recorded by using different tools such as reports of experts,
learners, etc. and would try to have answers for the following questions:

1) What is the best m e d i d format to be used for lectures?

2) What is the best method to acquire the media?

3) What is the best way to disseminate the courseware? I

4) What would be the benefits of the web based lecture to the learners?
5) What would be the various costs involved ?
Summative evaluation: Another type of evaluation is the summativeevaluation. It
assesses the impact of instruction or an educational programme on learners over a
period of time. It is important as it assesses whether the learning outcomes match
the instructional objectives and if not how much gap is there in between and why.
The term end examinations are in fact summaiive evaluation. Here we jugdge the
quality or worth of a student's acfiz; ement after the instructionalprocess has been
completed.Awarding grad<,ol marks on the report cards is one example of reporting
the findings of the summati .:evlaution of achievementof learners.
Summative evlaution is carried out foiiowing the process of imparting instructions.
Surnmative evaluation is also carried out for coursewares. It ascertains whether a
course that is in use has achieved the intended goals. In another words we can say 1
that it judges the quality or worth of already completed instrusctional procedures,
curricula, educational programmes, etc. Summativeevaluationalso informs us about
 Techniquesand Tools
of Evaluation
in anumber of ways including:
Satisfactionof learners: It is usually assess
of the programme.
criterion-referenced
(discussed in the previous unit) and norm-re
position of a learner in a group as in a class,
as through observations and other means.

behaviour different fiom that when it

learner behaviour is ideally assessed

other tools like, interviews, questionnaire

Assessments of impact It includes long- ofthe educational
programme and projects (discussed in
The basic techniques of educational
schoolsand universities, but also for
and community ed~ication p r o m like the one
centers, etc., and social services.

12.2.2 Sources of Evaluation

Self assessment by learners

and that of teacher's. In order to overcome

e self assessment

assessment:
ess? What was the
least useful area?
Did I achieve my learning objectives in thi
Evaluation Concepb Peer Evaluation
Peer evaluation is another source of information about students. It is particularly
usefid duringdiscussion forums either onlineor in face to face situation or in evaluating
collaborative work. But again it is essential to prepare some guidelines acting upon
which the students can give cornmnets about their peers. But before reaching the
final conclusionit is advisable that you must conrelate students' ratings withteacher's
observation. It will increase the reliability of evaluation.
Teacher as Evaluator
By now you might have understood that there could be different evaluators but
teacher is the main source of evaluation for the learners, as they continuously interact
with the students inside and outside the classroom. Even as a distance learner, you
are being evaluated as your teachersltutorstake into account your performance on
assignments, projects, and participation in discussions in the face to face workshops
or online discussionsin case of online learning situations. Palloff and Pratt (1999)
recommended that the online instructor, before evaluatingthe students should take
into account the needs of the learners, their educational level, their familiaritywith
technology and online learning (and any problems that may have occurred as they
adjusted to the use oftechnology),and issues related to on line communication like
(email and chat) and the learning objectives.All these issues are however determined
before the course is delivered. Besides these sources, the community can also act as
a source of evaluation because ultimately the design of the product or the instructional
materials must be according to the needs of the society at large.
1 Activity I
I Before, you move on to the next section on the various criteriafor the evaluation, I
I evaluate yourself about your understanding of the previous sections. Do you I
need to read them once more?
Check Your Progress 1
Answer the following questiqns briefly.
1) Discuss the importance of evaluation in the classroom.
1 2) How can peer evaluation be useful as a source of evaluation?

12.3 CRITERIA FOR EVALUATION

Now that you have understood the meanilig and importance of the evaluation and
also about the types of evaluation, it is important for you to understand on what
basis you will cany out evaluation. This basis of evaluation is called the 'criterion of
evaluation'. In a classroom situation a criterion helps you to infer the performances
of your learners in a targeted domain. There are basically thee domains - Cognitive,
which involvesknowledge and the development of intellectual skills,Affective, which
includes the manner in which we deal with thingslsituations emotionally, such as
feelings, values, appreciation, enthusiasms, motivations, attitudes and the
Psychomotor domain that includesphysical movement, coordination,and use of the
motor-skill areas. Learners are evaluated on anyla11 of thew domains.
The various criteria that you can use to evaluate students could be as follows:
The curriculum framework
a Instructional objectives\
a Stan* in various domains (cognitive, affective and psycho motor) of learning
56 Age and understanding level ofthe students
This is not an exhaustive list, rather there could be various o@ercriteria as well. Techniques and Tools
I of Evaluation
12.4 TOOLS OF EVALUATION: NEFD A*D
IMPORTANCE ~ I
I

Let us now initiate our discussion on the tools of this section, we will
discuss about the attributes of a good tool. which you choose
must be selected or developed carefully to fit evaluation and the
plan for data analysis.Thiswill help in reliable answers
to your questions for which you
decide the effectiveness of a
of a good evaluation tool
we are going to discuss the same.

12.4.1 Attributes of a Good ~ v a ~ u a t i oTool

b
A good tool for evaluationis characterised by certain
I I about which we shall
discuss now.
Validity
i
he overall function of anyevaluation
or a group of individuals.These inferences are based collected through
assessment, which could be canied out through establish that
student 'A' has a positive attitudetowards using
'B' does not have it. As these inferences are
tests, therefore it is important that

I
BY^ what does the validity of a test mean? Ground1 d (1985 stated that "validity
refers to the extent to which the results of evaluation roced e serve the particular

II
uses for which they are intended".Validity thus refers to the de to which a test is
capable of achieving the objectives formulated fo the test Validity is usaually
expressed in terms of a correlation coefficeientan the val e ranges from - 1.OO
through .OO to + 1.00.Validity is ususally classified in o three s: content-related,
criterion-related, and construct-related.
Content-related validity is the most important the usefulness of
the test. It indicates validity as to the content
example, if you have developed a test with
word processing, the test should assess the
skills. Let us take another example.
on adolescents.The content validity
data collection is really helping in
~dntent
validity is established by the followingmeth s:
thorough search of the relevant literature,
+I I
I
are/ asked to check the
~
from representatives of a relevant populationwho
validity of the tool,
fiom content experts who are asked to judge
I
test represent the defined content domain.
Criterion-related validity is useful when you inferences from
measurement results as to some future behavior validity
evidence) or to a current behavior (ie, concurrent
medical entrance test is used to place students
Evaluation Concepts test should have criterion validity and be able to indicate who would be successful in
medicine. Entrance tests for various professions should have this kind of validity.
Construct validity refers to the extent to which the test results are interpretated in
terms of the known psychological concepts and principles.For example when we
want to interpret the test results as the measures of any psychological construct say
intelligence, then the construct validity would be to what extent the performance on
the particular test is measuring intelligence as defined by the theory on intelligence on
which the test happens to be based.
Reliability of the Test
The concept of validity which we have discussed answers the question of what to
test? But reliability refers to the 'consistency' of the measurement. For instance,
every time the length of a table is measured with aparticular measuring tape, the
result should be the same under identical conditions. Also, whoever measures it
correctly should get the same result with that particular measuring tape. If a student
scores 35 marks out of 50 marks in an objective type achievement test, would that
score indicate the learner's true abilities? The answer may be 'yes' or 'no'. It would
be 'no' if the student has guessed the answers to some of the questions. In such a
case, the score would be different if the test is administered after a reasonable gap
of time. To check this error of measurement, the reliability of the test should be
calculated. Reliability refers to the consistency of measurement results and the extent
to which they are accurate, error h e , and stable. Reliability evidence is often reported
as a correlationcoefficient. Reliability of tests is estimated by different methods:
Using parallel foms (2 equivalent forms of same test) usually estimated through
the use of test-retest or equivalent forms. Areliability coefficient is calculatedon the
scores of the two measures obtained ffom the two parallel forms of the test
administeredto the same group of subjects. High and positive correlationis obtained
when the two forms are parallel. Evidence of test-retest reliabilty indicates that
learners would achieve essentially the same scores if they took the sanie test at
another time (ie, a test-retest procedure). Suppose you have developed a tool to
test the proficiency of learners in aparticualr area, then the learners who will take
this test must score the same scores when the same test is given again.This type of
reliability is evident when the variable (proficiency or expertise or a trait) being
measured is expected to be stable over a period of time.
Equivalence-forms involve the use of two or more forms of the same test. It is also
called parallel or alternativeforms. To find out the reliability through this method, the
items in both the forms are intercorrelated so that both the test are homogeneous in
nature. Both the test forms are administered to the same subjects. Correlation
between the two set of scores provides the estimate of the test reliability. Higher
the reliability coefficient,more the reliability. The major weakness ofthis type of
reliability evidence is that many research instruments are not available in equivalent
forms.
Single administration of a test: We can use split half method for establishingthe
internal consistency. Reliability within the test is established through this method.
Unlike the earlier method that establishesreliabilty through correlations of two sets
of scores obtained fiom two different tests, this method involves calculations of
reliability by administrationof the test only once. One such method is establishing
internal consistency. It establishes the degree to which the individual items are
correlated with each other and thus reflects the homogeneity of the items of the test.
The greater the number of similar items in the test, the greater will be the internal
consistency.
 Another method is the use of split-half method. is a coefficient Techniques and Tools
obtained by dividing a test into two equivalent of Evaluation
the scores on each half, the greater will be the
may be done by methods such as selecting
even numeberd ones in the other, by
Inter-rater Reliability: This type of reliability is two or more
independent evaluators who score the test.
determine the correlationof the scores given
correlation,more the reliablity.
The relaibilty of any test is influenced by various the instrument, the
research subjectsand the administration conditions.
the length of the instrument, i.e. length of a test
measure the same content. In general, the
instrument,the greater the reliability.
items) also tend to produce scores
Usability of The Test
Apart from validity and reliability, for any tool
~ ~ rtant to look into its
practicability like ease of administeringthe of the scores
obtained, cost oftesting, etc. All these
'Usability'. Usabilty refers to the
successfully used by the evaluator.

12.4.2 Sampling Methods and ~roeedbres 1

Whenever aresearch is carried out, we use enables us to
determine apopulation's characteristics.
enumerationof the entire population, as
money. For example, if you want to
the audio
of all the studentsenrolled. For
representativesample ofthe
In this section we will
sampling procedures.

1
Sampling is an act, a process, or a technique of sample, or a
representative part of a population for the
characteristics ofthe whole population.
two types: probability and non- probability sampling
I A probability sampling method
oli the principle of
method, we must
units in the
Evaluation Concepts Simple Random Sampling

It is the simplest form of sampling in which all the units (subjects)of apopuation
have an equal probability of selection or rejection. This technique is suitable for
small populations in which all the units can be listed. For example we may list all the
2000 students of a school and then pick out randomly any 200. There are tables of
random numbers, and also computer random number generator to select the sample.
StratifiedRandom Sampling
It is also known as proportional or quota random sampling. This type of sampling
involves dividing the population into homogeneous subgroups and then taking a
simplerandom sample h m each p u p . Stdfied random samplinghas more statistical
precision and accuracy than simple random sampling. For example a population
can be grouped on the basis of gender, age groups, geographical areas, etc. and
then take a simple random sample from each subgroup. Let us take an example,
suppose you want to evaluate the video programme you have developed for your
students. If there are 900 female students and 600 male students then male and
female are two strata. From these strata, you can select a sample of 800 students
through simplerandom sampling. Or, you may select a sample by having four starta
- students of: east, west, south and north zones of India.

Systematic Random Sampling

Systematic Random Sampling is useful when one has a finite list ofpopulation. For
example, a telephone directory.In the systematicrandom sampling, the selectionof
the subject consi5ts of selecting an nth term. First of all, we have to number the units
in the population from 1 to N and then decide on the sample size that you need.
Also, decide theevery nth unit for which the sample will be drawn. For example, if
you have N as 11 then every 1lthpersonlunit enters the sample. In anothre case,
suppose you want to have feedback on the online programme you have developed,
for your survey you may contact every eighth (n=8) student ( as per enrolment
number) fiom the student population of 4000 students.
Cluster Sampling
Supposeyou want to find out the choice of Indian school going adolescentsregarding
television programmes. You cannot iiiterview all the adolescents. So you may form
clusters with adolescents of schools of each state. Then you may randomply
sample clusters. It is especialy useful when the population is distributed across a
wide geographic region. In cluster sampling, we follow these steps:
a divide population into clusters (usually along geographic boundaries)
a randomly sample clusters
a measure all units within sampled clusters
Non-Probability Sampling
Non- Probability sampling method does not involve random selection procedure.
Hence not every unit ofthe population will have an equal chance of selectionin the
sample. There could be a question as to whether the non-probability samples are
the representative of the population. They are not considered to be very useful for
generalisationsabout the population as the non- probability samples, may or may
not represent the population well. But they have their own importance for data
collection.
In general, probability sampling is considered to be more accurate and rigorous and
60
therefore preferred more to the non-probability sampling. However, in social science
 I
j research there may be circ~stanceswhere it is not f&ble, p&ti~alor theoretically Techniquesand Tools
possible to do mdom sampling.For example, a study to find of Evaluation
1
L
out the social behavior of an autistic child,
I
childrenwith the help of probability sampling,
few. Here, we may consider a
may be done in various ways.
Accidental or Convenience Sampling
In this method of sampling the subjects are
~ ~ an unstructured
manner h m the h e . .Sometimesit is also
Consider a case where an individual is
opinion about the advertisements shown
try and get the views of any person who
in this problem, we don't have any
representativeof the population about
Purposive Sampling I

For selecting a purposive sample, the about a particular

sample with apurpose in mind. For for youth between
20-25 years for fin& out their Then, before stopping
anyone for an i n t e ~ e w
you does meet the
criterion of age for being
usell for situations
proportionality.
Quota sampling
~
1nthis method we continuewith samplinguntil a specificnurn$er ofunits (quotas) for
various sub groups of the population has been attained have
been selected. The quotas may be based on population
ifthere are 3 million people who watch a particular
watch another. You may form samples
sampling is similar to startifid random
The advantagde is that we are forced
a population. But it is different
formation of strata we do not
Snowballsampling
The first respondent refers a fiiend. The fiiend also
~ ~ whom you include
in your sample and it goes on like this till you have You might be
tempted to interviewthose who look most helpll
disadvantageofthistechnique is that the samples
may be subjected to biases ofthe selector. This
w h e s s of the non-probabiltysampling.
Check Your Progress 2 1 ~
I

I 1) Which typeof validity and relabilty measures

Answer the following questions briefly:
~w ~ ould use to standardise
I
1I
the achievemet test and why?
2) Differentialtebetween the content validity and
i
Q"Ontalidity.
~ I
3) ~ifferentiatebetween proability and nonpmbal#lity dP1ing. I
4) From a school disrtict directory if you wish to select a sample of 200 teachers
fiom the list of 4000 teachers, discuss the ways in which a sample could be
selected. - -
Evaluation Concepts
12.5 TYPES OF EVALUATION TOOLS
When you want to collect data, say, for need assessment preceediigthe development
of media courseware or want to go for formative or summative evaluation of the
courseware or you want to evaluate your learners, you have to use tools to collect
data, which you would would later analyse and interprete. Keeping in mind the
issues related to validity and reliabilityjust described, we would now move forward
to understand the commonly used evaluation tools. Understanding of these tools
would help you to select the appropriate tools for carrying out evaluatuion.

12.5.1 Questionnaire
Questionnaire is a widely used tool for collecting factual information. It elicits
structured, often numerical data.Your presence is not needed at the time of
adrninistraion ofthe questionnaireas you can mail it. In comparison to certain other
tools of evaluation,the data obtained is straight forward and easier to analyse. You
should remember that it is different h m an oppinionairewhich elicitsopinions of the
subjects rather than facts.
The questionnaire could have the questions in two forms: the closed form and the
open form. As the name suggests the closed form calls for the questions which has
answers to be given in the restricted form such as 'yes' or 'no'. For example, in
response to the item, 'You like watching talk shows', the answer could be yes or no.
While in the open form the answer are expected to be given in the unrestricted way
i.e free response. For instance, for the item 'you like watching talk shows
because. ......'
Characteristics of a good questionnaire
Some of the characteristicsof a good questionnaireare the following:
The purpose of collecting data should be stated in the questionnaire.
Directions for completing the questionnairesshould be given clearly.
Questions should be arranged in a logical order.
The questions should be unarnbigous
The questionnaire is used vey fi-equently as a tool for data collectionbut the problem
which is often faced by the administrateris the low response rate ofthe respondents
and delayed responses. But now with Internet facilities online questionnairesare
being sent. Let us discuss about this in brief:
Electronic questionnaires
Now a days, with the developmentof the technology the conventional questionnaires
sent by post are being replaced by electronic questionnaires. Electronic
questionnairesare sent through the electronic mail. It has an important advantage
that there is a more favorable response rate compared to the low response rates for
questionnaires sent throughpostal system. Sending questionnaires in this way ensures
economy in terms of both money and time as it is possible to send the same e-mail
to many people at the same time. But the only thing, which you have to think about,
is that the anonymity ofthe respondent cannot be maintained, while using this method
and hence many would not reply honestly or may not respond to the mail.

62
 I
I 12.5.2 Interview Techniques and Tools
I of Evaluation
An interview is basically a que which instead of
1
writing the responses, the subject gives the in okally in a face to face
I situationYou must have experiencedthat in g & more willing to talk
than to write and especially so on certain
has become a very popular way of data corder is commonly
used to record the conversation betwe interviewee. Today
telephone is used to take the interview
we can also see the use of internet for iews. One can thus
have immediateinteraction online, e
by the setting up of electronicfocus
participants. Throughwebconfe can be interviewd
singly or in a group.
Foster (1995) has described the advantages of electro ewing, which are as
follows:
a) Interviewing by electronic mail is not location or
time zone. The interviewer and
1

b) Electronic interviewing data can directly be

I package or computer-based qualitative
I
changes.
I
I c) Saves time and money of the
what exactly the interviewee expressed.
1
I Whether the interview is taken in a conventional
the important point is to develop a rapport wit

i should also be well versed with the content.

interpretingthe mponses of the
likes and dislikes may influence the
carehl to avoid this. But, whe
reduces the influence ofthe
problems caused by dominant and shy participants, p
groups. In this way, electronic interviewinghelps in
problems commonly associated with conventional
whatever be the advantages of an e-mail interaction,
I both the interviewer and the subjects.Also, sometime
of a respondent's thoughts than that through face to
information that can be gathered through non-ve
t
language, facial expressions, etc. and active list
absent when interview is conducted by using the E mail.

12.5.3 Observation Schedule

From the earliest times, observation has been an
i I
and as a technique of evaluating various aspects
in controlled or uncontrolled situations. It
behaviows, events, and the contexts
can observeany settingor physical
nonverbal communications,
be oftwo types- participant
Participant observationinvolves the observer to observe
them as one ofthe members while in the nonparticipant
not the member of the group. In the non- participant
 EvaluationConcepts the noting of the observations by using the one way glass partition or by using a
hidden camera or microphonesor by simply watching and taking notes.
It is essestial to plan the observation systematically so that correct result can be
sought. For planning the observation certain decisions have to be taken before
hand, such as identification of objectives or behaviour, to be observed, tools to be
used for recording and type of observation. It is always advisable to deveop an
observation schedule that lists in a logical sequencethe behaviours to be observed.
As human behaviour is complex and have many traits and characteristics that are
difficult to observe directly and correctly. Therefore, there is a need that they must
be operationallydefined. In other words, we can say that a teacher or any evaluator
must know precisely what s h e is looking for. For example, if you want to observe
the social interactionsamong young children, then you will have to specifj,what kind
of interactions you are looking for. Social interaction may be defined along parameters
like getting on well with peas, playing in group, working in ateam, etc. Observations
could be on these parameters, otherwise the whole excersice will be of no use.
For observationsto be carellly and systematically recorded, we can take the help
of technologies like video recorders. This will help you to observe even minor
details. Concealed microphones and videotape recorders can be used to observe
the behaviour of children and even adults in the natural group activities. This allows
the observer to take ino account the real and natural reactions of the subjects.
However it is very important that we have to make sure that use ofsuch technologies
do not violate privacy and dignity of individuals and is ethical.
Computers not only help in recording of the data in a continuos and uninterrupted
way but also helps in the analysis of the observed data Various computer progmmmes
are there, which can help in this. There are tools available that are easy to use and
I
can be used to collect and manage data for performing observationalstudies. These
tools can streamlinethe process of recording, annotating and transcribing digital
video and audio, allow even hundredsofhom of recording.The coding tools available
with such products can accurately annotate,transcribe and mark up video events,
and facilitate video clips and data for distribution.
So, we see that technology is making the work easier and more precise for the
researcher. But you must choose the right technology to get desired results.
t
12.5.4 Achievement Tests
Achievement test helpsthe teacher to measure what an individual has learnt following
instruction.High achievement scoresusually indicate amastery ofgade-levelmaterial,
and the readiness for advanced instruction. Mostly the achievement tests used by
the teachers in the schools are teacher made tests. Usually we use achievement tests
to gauge the place of the student in the class but these achievemnettests do not help
in the overallassessment of the students.Therefore,alternativeassessment techniques
such as portfolios, are developed.
A Portfolio is a collection of the work performed by a learner in a particular area. It
may include'assignments, projects, etc. It provides an opportunityto the learner to
demonstratehis or her learning towards the achievement of the learning objective in
a given span of time.
An electronic portfolio takes advantage of technology to digitize, organise, store
and tmnsrnit the informationmore effectivelythan a traditional portfolio.Through the
word wide web [ www) electronic portfolio or e-portfolioor digital portfolio bring
64 forward the collection of the work done by a learner over a period of time. The
learner is free to decide which pieces of work to present the Techniquesand Tools
work. They can present their work by using of Evaluation
Frontpage. Besides choosing pieces of work
on what he or she learned while doing the
portfolios resemble in the content, goals
add the advantagesof security, compact
According to Mathur and ~ m ~ ( 2 0 0 6the
), has the following
advantages:
It can be edited, updated, retrieved and instant1 to several
people.
a 1t is designed to accept instant feedback h m teachers, and areaexperts
4
and provide exception1flexibility to the process o learnin and assessment.
4
It helps in making the cross reference kf the stude the content
area.
At this stage we would like to provide you with
integrates the latest technology. The National
working on the concept of On-Demand
This scheme provides flexibility to the
their wish and preparation. When ever a student is ready
and makes a demand for the same, a set
the computer out of the already developed ques
maintaining a huge questionbank, large number of items
developed.For identification, all these
the content area to which the item be1
question, the marks allocated to the
is also planning to initiateit.
The advantages of ODE can be stated as follows:

on the learner and not on the institution.

all subjects at a fixed time and schedule.

a strong motivating factor.

in the examination.
a Malpractices will be reduced, as it is a system where
Evaluation Concepts for collecting it. Unlike tools like questionnaires and interviews with structured
questions, those with open ended questions elicit in-depth and detailed data. Such
data is qualitativei.e. descriptive in nature.A study may involve only quantitative or
qualitative or both types of data.
Qualitative data is classified and organized under different heads. For example,
when data is collected as to the effectiveness of an audio programme, the data
pertaining to the language, sound effects used, duration of the programme, etc. may
be classified and organizedunder these heads. There after the data is described and
content analysis may be carried out to identi@ the key words or concepts, the
6-equencyof their occurrence,their internlationships, etc. This helps us to infer h m
the data, the underlying trends, their effects,causes and even the extent of difference
ofthe prevailing situation h m the standard or ideal one.
In case of quantitative data, as for instance the scores of learners in a test, where the
data is expressed numerically, the data is organized and if necessary tabulated. There
after it is subjected to statisticalprocedures like determination ofcentral tendencies
(mean, median, mode), measures of variation, finding standard scores, correlations,
etc.
Check Your Progress 3
Answer the folIowingquestionsbriefly.
1) How can technology help in observation, as a tool for data collection?
2) Discuss the irr ,ycrknce of On @enlandExamination for the students?

12.7 SUMMARY
Teaching is a process which is intended to bring about changes in the student's
learning.While, teaching a teacher has to make decisions based upon the student's
performance. In this unit, you have understood that it is only through proper
evaluation, one can get the right picture of learning and also about the educational
programmes. Keeping this point in mind, you have been appraised with various
types of evaluation i.e formative amd surnrnative evaluation. The former is done
during the development of the programme or while teaching and the latter is done
when the programme is ready for implementationor when the phase of imparting
instructionsis over. Various kinds of tools are used for data collection for the process
of evaIuation The tools selected must have the necessary attributes namely reliability,
validity and usability. The tools that have been describedin this unit are questionnaks,
observation,achievement tests, interview, etc. Today, technology greatly aids data
collectionthrough the tools described.Technology is being integrated by the teacher
in their class while carrying out evaluationa and also for conducting evaluation of
learners who are at a distance.

12.8 UNIT END ACTIVITIES

1) Develop a portfolio of the activities you are undertaking for the programme,
PGDET.

2) Suppose you have to develop a video film on a topic of your choice for the
young learners, suggest how you will carry out the formativeand summative
evaluationofthe film.
 Techniquesand Tools
12.9 REFERENCES AND SUGGESTED READING . of Evaluation
Best, J.W. & Kahn, J.V. (2006) Research in Education and Bacon.
pages 303- 326
Cohen, L. Manion, L. (1994) Research Routledge
Publication. pages 66-8 1 I
Ebel, R. L., and Frisbie, D. A. (1991). Essentials of nal Measurement
(5th Ed.) Englewood Cliffs, NJ: Prentice Hall
Foster, G. (1995) 'Fishing the Net for Research Journal of
Educational Technology, 25,2, pp. 91-97
Gronlund, N.E. (1 985). Measurement and evaluatio ing (5th ed.). New
York: Collier Macmillan Publishers
JeBrey C. Wayman and Sam Stringfield (2006). Arne ofEducation,
Volume 112, pages 549-57 1
Mathur, S. &Murray,T.(2006). Authentic assessm&nt0nlibe: A practical and
9
theoritical challenge in higher education, Online As essmen Measurement And
Evaluation,page 238-246. f
I
Nitko A.J. (2001) Educational assessment of
Palloff and Pratt (1999) Building
strategies for on line classrooms,
No. 1,Page 102-103
Popham, J. (1 999). Why standardized tests don't quality.
Educational Leadership, 56(6), 8-15.
Selwyn, N. and Robson, K( 1998) Using email rn a besearchltool, Social research
update.
Stiggins, RJ.(1999).Evaluation classroom in teacher education
i programs. Educational
I WeSt0n.T (2004) "FormativeEvaluation for Educational
Technology Applications and Lessons,"
I 25, No. 1, pp. 5 1-63.

i You may visit websites on the topics discussed in thi

i Newlove, Beulah (1969)The Telephone

Research Tool in Teacher Education,

I
Video scribe http://~~~..inhanel.com/video~~ribe.hhnl
retr+xdon 09/09/07
1
I
F
12.10 CLUES TO CHECK YOUR I/ROG@ESS
I I
I
Check Your Progress 1 I

67
I
I
Evaluation Concepts 2) Peer evaluation provides feedbackto fellow studentsduringcollabbrativework.
It also helps in facilitating student p u p s inxsolving their conflicts.
Check Your Progress 2
1) Content Validity and reliabilty through split half method because achievement
test administered in the classroom or to distant learners are uasually not repeated.

2) Content validity refers to extent to which the instrument measures all relevant
dimensionsor domains of the content whereas Criterion-related validity refers
to how well the inferences f?ommeasmment indicatesto some futurebehavior.

3) In probability sampling the sample selection is done by using the random

selection techniques which is not the case with the non probability sampling.
4) Systematicrandom sampling
Check Your Progress 3
1) With the help of hidden camera and microphones but without being unethical
about it.

2) Assesses the studentswhen he/she is ready for the examination, reduces stress,
reduces the chances ofmalpractices,etc.

68
MOTES