DOCUMENT RESUME

ED 385 143 FL 023 141

AUTHOR Cheng, Liying

TITLE How Does Washback Influence Teaching? Implications
for Hong Kong.
PUB DATE 95
NOTE 20p.
PUB TYPE Reports Research/Technical (143) Guides
Classroom Use Teaching Guides (For Teacher) (052)

EDRS PRICE MF01/PC01 Plus Postage.

. DESCRIPTORS *English (Second Language); Evaluation Methods; .

Foreign Countries; *Language Tests; Questionnaires;

Research Methodology; Secondary Education; *Test
Norms
IDENTIFIERS Hong Kong; *Teaching to the Test

ABSTRACT
This paper presents preliminary research findings,
using qualitative and quantitative methods, on the washback effect of
the revised Hong Kong Certificate of Education Examination in English
in Hong Kong secondary schools. The research employed various
methodological techniques such as questionnaires' (one sent out to 42
students; and the other to 48 teachers), interviews, and classroom
observations, which are based on an in-depth case study approach to
sampled schools in Hong Kong. Findings indicate that the washback
effect worked quickly and efficiently to bring about changes in
teaching materials, largely due to the commercial characteristics of
Hong Kong society, but somewhat slowly, reluctantly, and with
difficulty in the methodology that teachers employ. It is suggested
that teaching content has so far received the most intensive washback
effects, although washback effects have also been observed in
teachers' attitudes and behaviors and in the English curriculum.
(Contains 38 references.) (Author/NAV)

***********************************************************************
Reproductions supplied by EDRS are the best that can be made
from the original document.
***********************************************************************
 HOW DOES WASHBACK INFLUENCE TEACHING?
IMPLICATIONS FOR HONG KONG

Liying Cheng
University of Hong Kong

ABSTRACT

There is some evidence to suggest that tests have washback effects on teaching and learning
(Alderson and Wall, 1993). The extensive use of test scores for various educational and social
purposes in society nowadays has made the effect of washback a significant phenomenon. This
paper presents preliminary research findings on the washback effect of. the Hong Kong
Certificate of Education. Examination in English in Hong Kong 'secondary schools by
employing various methodological techniques such as two questionnaires, interviews and
classroom observations, which are based on an in-depth case study approach to sampled
schools in Hong Kong. It further discusses the nature of washback effect, the major teaching
and learning, aspects influenced by it, the different stages of washback effect, and the types of .

washback effects observed. Preliminary results indicate that washback effect works quickly
and efficiently in bringing about changes in teaching materials, which is due largely to the
commercial characteristics of the Hong Kong society, and slowly and reluctantly and with
difficulties in the methodology teachers employ. It is suggested that the latter effect may be
caused by the constraints imposed upon teaching and teachers in our present schools.

INTRODUCTION

The role of school-as-a-selection-device has been particularly evident in the Hong Kong
educational system from early days (Biggs, 1993). Assessment in Hong Kong is still geared
towards providing data to select students: not only in the HKCE (Hong Kong Certificate of
Education) at the end of Secondary 5 and HKAL (Hong Kong Advanced Level) at the end of
Secondary 7 for tertiary selection, but also in primary schools, to provide the data for banding
in the Secondary School Places Allocation exercise. The Hong Kong. educational system is
characterized as an examination-led system where what goes on in the classroom is largely
dictated by what happens in the public zaamination halls (Fullilove, 1992).

Washback or backivash is a term that can be found frequently in official documents in Hong
Kong about general education and language education in particular (See Hong Kong
Government, 1984, 1986 & 1990 and the Education Department, 1989; HKEA, 1993). For

"PERMISSION TO REPRODUCE THIS

U.S. DEPARTMENT OF EDUCATION
MATERIAL HAS BEEN GRANTED BY Othce of Educahonat Research and Improvement
EDUCATIONAL RESOUT INFORMATION
CENTER
\\%"-r T is document has Per ,. reproduced as
vap from the person Or Organization
Ohgonating rt
O Mono( changes have been made to .mprOve
reproduction quality
TO THE EDUCATIONAL RESOURCES Points of mew or oprnrons stated in Ind docu
INFORMATION CENTER (ERIC).- ment do not necessanly reoresent oft.c.at
OERI posthon or policy

2 BEST COPY AVAILABLE

example, it is stressed in Education Commission Report No.4 (1990) that " a project should be
carried out to determine the 'backwash' effects of different forms of assessment".

In Hong Kong, certain washback effects are expected whenever language examinations are
introduced. Some examples of the positive washback effect of public examinations in Hong
Kong are reported in the literature (See Andrews and Fullilove, 1994; Andrews, 1993;
Fullilove, 1992 and Johnson and Wong, 1981). In Education Commission Report No. 4, it is
stated that

"The Hong Kong Examinations Authority is well aware of the fact that the syllabi
for HKCEE affect the curriculum and even the teaching methods in schools,
particularly in Secondary 4 and 5. ... the revised HKCEE syllabi have served .to
improve the clarity of the curriculum objectives which has had a positive influence on
teaching."

Moreover, Morris (1990) states that any change in the Hong Kong educational system must
first involve a change in the examination. However, what 'is still not clear is the nature of
washback and the scope of the effects of public examinations in Hong Kong. A search of the
literature indicates that washback effects are more perceived or assumed than supported by
empirical data.

BACKGROUND

This research was designed to investigate whether or not any washback effect of the revised
Hong Kong Certificate of Education Examination in English (HKCEE) by long Kong
Examinations Authority could be observed in the teaching of English in Hong 'Ong secondary
schools. The revised examination syllabus was used for the first time in classroom teaching in
September, 1994 in Hong Kong secondary schools. The first cohort of students will sit for the
revised examination in May, 1996. The aim of the study was to observe: how the whole
education system would react in the context of the change in its assessment practice and to
attempt to discover the implications of the washback effect on the teaching of English in Hong
Kong secondary schools.

The HKCEE is a public examination sat by the majority of secondary students at the end of the
fifth year of their secondary school. Students will proceed either to further studies at the Sixth
form level, or will leave school and seek employment. Two separate syllabuses, namely the
examination syllabus by HKEA (Hong Kong Examinations Authority) and the teaching
syllabus by the CDC (Curriculum Development Council) coexist in Hong Kong secondary
schools. It is stated in the teaching Syllabus for English Language (Hong Kong Curriculum
Development Committee, 1983) that

2
 "It must be recognized that for both these groups of students, the Hong Kong
Certificate of Education examination assumes critical importance. For almost all of
them the results they achieve in this examination will be the prime factor determining
their future careers. It seems vital, then, that the teaching syllabus at this stage and the
examination syllabus should be in step, ...." (1983:12-13)

In 1993, the HKEA introduced major changes to its existing examination syllabus in English in
accord with the Target Oriented Curriculum (TOC) initiative in Hong Kong.

"The proposed changes of the 1996 HKCEE in English aim to modernize and
improve the examination syllabus as well as to incorporate some TOC principles by
adopting an integrated approach and by being more task-based. It is expected that the
change will narrow the gap between what happens in the exam room and the real
world." (Hong Kong Examinations Authority, 1993: Appendix C1)

This attempt to "narrow the gap" is reflected in the major changes to the present examination
which are mainly in two papers of the examination, namely paper III and Paper VI. Paper III:
Integrated Listening, Reading and Writing, consists of several short tasks and extended tasks
which require students to perform those tasks by integratively employing information through
different skills. Part IV: Oral, has changed greatly from reading aloud and guided conversation
to task-based role play and group discussion. Both examination papers require students to take
an active role and participate fully in language interaction.

As is well known in the field of education, assessment practices are currently undergoing a
major paradigm shift which can be classed as a reaction to the prevailing paradigm - the
emphasis on standardized testing and its perceived shortcomings (Biggs, 1994 and Genesee,
1994,). Hong Kong is no exception as its Target Oriented Curriculum aims at'a target-based
approach to curriculum and assessment. The changes made in the present HKCE public
examination reflect one step towards assessing students' language abilities to carry out real-life
tasks. However, unlike the target-based TOC, this public examination is norm-referenced at
this stage bearing the major function of selecting students rather than educating them.

THEORETICAL FRAMEWORK

A number of research studies have been carried out in this area looking at the relationship
between testing and teaching and learning. Shohamy (1993) comments on some of the common
terms used to refer to the phenomenon. Washback effect refers to the influence of testing on
teaching and learning (Alderson and Wall, 1993; Hughes, 1988; Khaniya, 1990; Kirkland,
1971; Pearson, 1988; Wesdrop, 1982). Measurement-driven instruction refers to the notion

4
that test should drive teaching and hence learning (Bracey, 1987; Frederiksen, 1984;
Haladyna, 1991; Li, 1990, Popham, 1987; Smith, 1991). Popham claims that "measurement-
driven instruction is the most cost-effective way of improving the quality of public education".
(1987:679)

Curriculum alignment focuses on the connection between the testing and teaching syllabus
(Andrews, 1993; Linn, 1983; Madaus, 1988 and Shepard, 1993, 1991 and 1990). Systemic
validity implies the integration of tests into the educational system and the need to demonstrate
that the introduction of a new test can improve learning (Frederiksen and Collins, 1989;
Fullilove, 1992; Johnson and Wong, 1981). Frederiksen and Collins (1989) state that "A
systematically valid test is one that induces in the education system curricular and instructional
changes that foster the development of the cognitive skills that the test is designed to
measure". (1989:27)

Traditionally, tests come at the end of the teaching and learning process. However, with the
advent of high-stake public examinations testing nowadays, the direction seems to be reversed.
Testing usually comes first before the teaching and learning process. Pearson (1988) points out
that examinations are commonly used as levers for change. In those cases, textbooks will be
designed to match the purposes of a new test. Administrative and organization staff, teachers
and students will all work hard to achieve better scores on the test.

In addition, a lot more changes in teaching and learning would happen as a result of a
particular test. Often such consequences are independent of the original intentions of the test
designers. When a test is designed and test results analyzed, it is no longer the end of the job.
Test designers turn to evaluate the consequences their test might have brought about in
teaching and learning, be.it social, psychological, ethical, curricular or educational (Shahomy,
1993). She further points out that "The need to include aspects of test use in construct
validation originates in the fact.that testing is not an isolated event; rather, it is connected to a
whole set of variables that interact in the educational process." (1993:2)

The consequences or the influences a particular test brings are regarded as the extended
construct validity of the test. Messick (1989) recommended a unified validity concept, in
which he shows that when an assessment model is designed to make inferences about a certain
construct, the inferences made from that assessment model should not only derive from test
score interpretation but also from other variables under the social context. Testing is not an
isolated event; it is connected with a whole set of variables that interact in the education
process.

However, as Alderson and Wall (1993) point out, the quality of washback effect might be
independent of the quality of a test. It is possible for good tests to produce good and/or bad

5
effects. Furthermore, bad tests do not necessarily produce bad effects. Morrow (1986) uses the
term "washback validity" to describe the quality of the relationship between testing and
teaching. He claims that "... in essence an examination of washback validity would take testing
researchers into the classroom in order to observe the effect of their tests in action."

It is a1so assumed that if a test is supposed to be good, it would most likely drive teaching and
learning. This might be due to the nature of assessment if it is used for the purpose of
selection. However, whatever changes educators would like to bring into teaching and learning
by whatever. assessment methods, it is worthwhile first exploring the broad educational context
under which an assessment is introduced. Alderson and Wall (1993) point out that

"It is surely conceivable that other forces exist within society, education, and
schools that might prevent washback from appearing, or that might affect the nature of
washback despite the 'communicative' quality of a test." (1993:116)

It is felt that it would be worthwhile investigating first the nature of the examination in
teaching and learning, then the nature of the washback effect and the conditions under which
it operates. However, it should not be only the testers' job to evaluate any consequences of the
tests alone. Instead, it should be the job of those involved in the whole education system to
evaluate the consequences. And if we intend to evaluate the consequences a particular test .

brings about in teaching and learning in an educational context, we ought to first investigate
the educational context in which the test is issued.

Little empirical research has been carried out to investigate this education phenomenon either
in the field of general education or in the specific field of language education. A number of
research studies have been carried out by survey and interview methods collecting data on the
effects of standardized testing (Frederiksen, 1984; Haladyna, Nolen and Haas, 1991; Madaus
and Airasian, 1982;). Smith (1991) investigated the effect of tests on the teacher and
classrooms through qualitative studies. Alderson and Wall (1993) investigated this
phenomenon by detailed classroom observation. A few researchers (Hughes, 1989 and
Khaniya,1990) used language tests to measure the influence of tests on students' learning
outcomes.

METHOD

This research study was designed to combine quantitative and qualitative research methods.
The study started at the same time as the introduction of the revised syllabus in 1994. The
researcher worked as an outsider and observer to record anything that might have occurred
due to the introduction of the revised examination syllabus. Observations were essential and
key in this type of exploratory study. There are two levels of washback phenomena that lend

6
themselves to observation. First, the present- education context related to the teaching and
learning situations in Hong Kong secondary schools can be observed (See Figure 1). The
purpose of observation was to find out which parties reacted most to the examination change.
The current researcher would like to use a term "washback intensity" to refer to the degree of
washback effect in an area or a number of areas that an examination affects most. Secondly,
investigation can be carried out in depth in a particular area or areas of teaching and learning
affected by the examination.

"Different methods such as questionnaires, interviews, school visits and classroom observations
were employed to achieve the above research purpose in depth. While questionnaires give a
general picture as to how teachers and students react in the context of the revised syllabus,
interviews and classroom observations tend to provide detailed information about what
teachers and students actually do in the classroom. All these methods complement each other
in this type of research.

Hong Kong Examinations Authority

(Intended Washback Effect)

1. Textbook publishers 2. Schools

1) Principal
2) Panel chair & Form Coordinator
3) Teachers
4) Students

Figure 1 Flowchart of washback on the education context in Hong Kong

I am grateful to Peter Gu, a research colleague, for suggesting the term.

6
Interview

Semi-structured and free response interviews were carried out throughout the study. Interviews
with HKEA aimed to find out what the intended washback effects were when the authority
made changes to its existing syllabus. Interviews with textbook publishers were used to
investigate how they understood the intended changes and how they integrated the changes into
textbook revision. In-depth interviews as follow up study were carried out among teachers
during school visits to fmd out how they reacted to the change in their classroom teaching.

Survey

1. Teachers' questionnaire2
The questionnaire was designed to obtain preliminary data on teachers' reactions toward the
revised examination, their perceptions and understanding of the examination and what they
would like to prepare their students for the examination. The first part of the questionnaire was
designed in such a way that teachers first responded to particular questions related to possible
changes in those areas of teaching and learning in the context of the revised HKCEE with a
yes or no answer. Then they were asked to explain further their own views with written
answers. The second part of the questionnaire was related to the present teaching situation
such as teachers' teaching arrangements, the choice of textbooks and the teaching methods
they employ in their schools. This part of the questionnaire was designed on a five point likert
scale.

2. Students' questionnaire
Meanwhile, a student questionnaire was issued with the primary aim of investigating the role
of the public examination in students' learning. They were further invited to comment on the
actual teaching and learning of English in their schools. The questionnaire was designed on a
five point likert scale. The part related to students' attitude toward public examinations was
scaled according to the degree of agreement. The part related to their learning activities inside
and outside the classroom was scaled according to frequency.

Classroom observation

At present, there are two versions of the examination syllabus, the old syllabus and the revised
syllabus in Hong Kong secondary schools due to the introduction of the revised HKCEE in
1994. Two cohorts of students are available for study: one group ought to take the old

2
The author would like to thank Lam Hui Por, a research colleague, for his questionnaire on washback effect into the Use of
English in Hong Kong, and for the useful comments he gave on the development of my own questionnaire.

S
examination in 1995; the other ought to take the new exam in 1996. There are some teachers3
who are currently teaching both groups at the same time. Therefore, classroom observations
were carried out to observe the same teachers teaching both groups of students. The
preliminary observations were carried out to observe any change in the classroom activities
employed by the same teachers teaching both groups.

RESULTS

This research typically deals with a life cycle of possible changes that a change in its major
public examination might bring about in the teaching and learning of English in Hong Kong
secondary schools over the lapse of time. A series of changes at the macro level have been
first observed. These changes showed the reactions from different parties in the Hong Kong
education system, and the necessary support they offered to get the change across to schools at
the micro level.

Seminars and workshops

The 1996 HKCEE examination syllabus was implemented in Hong Kong secondary schools in
September, 1994 which aimed to prepare the first cohort of students for two years from
Secondary 4 to Secondary 5 before they actually sit for the examination. Half a year before the
teaching toward the new syllabus began, a series of seminars and workshops were organized to
offer help to teachers.

1. HKEA seminars
Seminars were organized with the aim of providing teachers with information of changes such
as the rationale and contents of the proposed changes, and the test formats to be employed.
Discussion was carried out over the issues related to school and teacher responses, and their
worries over the revised examination syllabus . Sample papers and the new test formats were
also introduced.

2. University seminars
Seminars were organized by tertiary institutes with the major aim to give teachers the
necessary methodological strategies that they urgently required to deal with the proposed
changes in facing the revised examination.

3. Textbook publishers' seminars

3
The usual arrangement of English teaching in Hong Kong secondary schools is to arrange same teachers to teach both
Secondary 4 and 5.

9
Seminars and workshops which received the largest audience of teacher participation were -
those organized by different textbook publishers. In Hong Kong, textbooks are by far the most
direct teaching support teachers can obtain and rely on for their teaching. Textbook publishers
provided their own understanding of the proposed changes and revealed how they had
integrated the changes into their textbooks. They then demonstrated to the teachers how these
language activities could be carried out in the classroom. A lot of sample activities were
demonstrated during those seminars. Suggestions about the time to be devoted to particular
activities were also provided.

Survey results

1. Teachers' questionnaire
A preliminary survey was carried out in April, 1994 among 48 teachers from various kinds of
Hong Kong schools, i.e. government, aided or priVate schools.. Their average teaching
experience ranged from three to six years with the majority having a first degree. 62% were
teaching in English medium schools whereas 29% taught in Chinese medium schools. The
survey was carried out 4 months before teachers began teaching toward the revised
examination syllabus for the first cohort of F4 students. The following results show teachers'
primary reactions to the revised HKCEE and reactions to different aspects of the teaching and
learning in their schools (See Table 1).

1) Reaction to and knowledge of the revised,HKCEE

When teachers were asked about their reaction to the new examination, 37% of them were
skeptical about the changes, 29% were neutral and another 21% welcomed or enthusiastically
endorsed the changes. There were 13% of teachers did not response the question.

Four months before the teaching toward the revised examination, the majority of the
respondents, 79% and 63% respectively knew the year in which the revised HKCEE would be
issued, the major skills to be tested'and the changes made in the test formats (See Table 1).

2) Reaction to teaching methodology and content

84% of the teachers commented that they would change their teaching methodology as a result
of the introduction of the 1996 HKCEE. They mentioned in their verbatim replies to the
questionnaire;

"1 would put emphasis on the integration of the skills."

"Oral practice will mainly base on facilitating discussion among students."
To invite active participation from my students. "

10
While 66% of the teachers mentioned that the proposed changes in the 1996 examination
syllabus do NOT contradict their present teaching methodology, 68% of teachers felt the new
examination would add pressure to their teaching. They believed that they had to do extra
work to cope with it. 73% of the teachers thought that teaching for the new examination would
involve using more teaching materials and aids such as audio & video facilities. At the same
time, teachers showed their concern over the revised examination as they regarded it as much
more demanding for their students than the present examination.

"How can I make students listen and speak more? I have to cut extra materials for my
students."
"Put more effort to train students' listening and speaking ability. But it is hard to do so
since most students' English standard is not good. And their writing ability have not been built
up.
"Students are usually shy and timid. It is difficult to teach them how to speak out."
"It is a bit too harsh for the students. Most of the students will not be able to deal with it. "
Regarding teaching materials, 4 months before teaching towards the 1996 examination syllabus
began, 55% of the schools had already changed textbooks for the teaching of English. By the
time the actual teaching began, nearly every school in Hong Kong whose students would sit
for the 1996 HKCEE employed revised textbooks targeted toward the new examination.
Teachers were provided with new teaching materials in their teaching.

3) Reaction to learning strategy

76% of the teachers mentioned that they would certainly suggest that students should change
their learning strategies because of the new examination. 47% of teachers perceived that the
new examination would motivate students to learn English. However, some teachers still
showed their worry about shy and less outspoken students since the new examination would
involve active participation in learning. In the new syllabus, students have to initiate questions
rather than to answer questions passively.

4) The present teaching situation

The situation in Hong Kong secondary schools is that teaching and learning is usually based on
a major textbook together with a set of workbooks. These workbooks are specifically designed
to prepare students for specific examination papers4 in the HKCEE. When students reach F5
level, teaching and learning become more and more examination-driven. Some of the schools
simply use practice papers right from the beginning of F5 to the time when they sit for the
examinations. Full preparation of their students for the examination is seen in Hong Kong
secondary schools as the responsibility of teachers and principals..
4
The existing HKCEE in English consists of five papers. The revised HKCEE will consist of four papers. They are Paper I
Writing, Paper II Reading Comprehension and Usage, Paper III Integrated Listening, Reading and Writing, Paper IV Oral.
This s kind of teaching arrangement can be found in various schools during school visits.

10

11
No. Item , . yes no
1. Teachers know the year in which the new exam will take 79% 21%
place.
2. Teachers know the major skills to be tested. 63% 26%
3. Teachers would like to change their teaching 84% 16%
methodology.
4. Teachers feel the new exam would add pressure to their 68% 32%
teaching.
5. Teachers regard the new examination as contradictory to 11% 66%
their teaching philosophy.
6. Teachers regard the new examination as an involvement 71% 11%
of more teaching aids. .

7. Teachers would suggest that their students should change 76% 24%
their learning strategies.
8. Teachers perceive that the new exam would motivate their 47% 18%
students to learn English.

Table 1 Teachers' reactions to aspects of teaching toward the revised HKCEE

61% of the respondents stated that the selection of particular textbooks was made by teachers
jointly. 24% and 13% of them stated that the decision was made by panel chairs and principals
respectively. As to the lesson arrangement, decisions are made by teachers according to 60%
of the respondents and panel chairs according to 29% of the respondents. 61% of the English
lessons are arranged for the purpose of teaching separate skills such as listening, reading or
grammar usage. Only 5% of .the lessons are integratedly skill arranged.

In Hong Kong secondary schools, among the factors that influence teaching, examinations play
a 30% role in teachers' teaching. Teaching belief and teaching experience play 35% and 6%
roles in teaching. The rest .of the influencing factors, 19%, 7% and 3% respectively are
attributed to learners', principal's and peers' expectations. Moreover, 68% of teachers
reported tha they had never been negatively affected by their students' past test scores.

2. Students' questionnaire
A preliminary survey was carried out in November 1994. The subjects involved were 42
students in one Secondary 5 class in the New Territory. Among them, only 35% of students
go to private tutorials for preparing for HKCEE.

11

12
1) Opinions on the HKCEE
When students were asked about their motivation to learn English, it was discovered that
HKCEE plays .a 30% role in their learning; future job 27%, parent concern 19%, classmate
competition 8%. Teacher, textbook and learning activities play 7%, 5%, 4% roles respectively
among other learning factors. It can be seen that two factors, public examinations and future
jobs are the main factors that motivate student learning in Hong Kong secondary schools. As is
mentioned above, the major role of the public examination is the role of selection. Passing the
public examination and thus obtaining a good job are closely related. Thus it would be crucial
for students to get a good score on the examination.

Almost all students mentioned that they were influenced in one way or another by public
examinations. When they were asked what aspects of their life might be affected by the
examination results, future job opportunity came highest on the list. The remaining aspects
affected by the examination were in the following order; learning strategies, emotion,
motivation, self-image and teacher-student relationship. According to the students, their exam
results affected parents in the following way; 1) the advice their parents gave them 2) parents
became tense and anxious and 3) parents put more pressure on them.

2) Opinions on current teaching and leaning

When students were asked to rank their language activities according to skills in the
classroom, listening comes as the most frequent activity they do. It is common in Hong Kong
secondary schools that students listen to their teacher talking most of the time in the
classroom. The second most frequent activity they do in class are activities related to language
exercises such as grammar or vocabulary. Reading and writing occupy similar class time
whereas the amount of time spent in speaking is ranked the lowest. In 57% of class time,
teachers talk to the whole class. Orly iii less than 5% of the time, do teachers keep silent
during teaching. The major activities the teachers do in.rank order according to the students
were 1) explanation of language points, 2) explanation of language activities, 3) explanation of
home work and 4) explanation of lesson objectives.

When students were asked to comment on the activities they do in English in class, doing
exercises and finding out answers from exercises were the most frequent activities. These were
followed by inquiring or asking their teacher for further explanation. Regarding their activities
in English outside class, watching TV or reading newspapers and magazines in English were
the most frequent activities they did. Communicating with teachers, classmates or parents were
among the least activities they did outside class.

12

13
classroom observation

Classroom observations were carried out on a small scale among those teachers who were
willing to be observed. As the observation procedure is still in progress, the available changes
that were observed lay in the different activities teachers employed. For example, for the
present F5 students, it is required in the present HKCEE that they have to demonstrate their
ability to read aloud a dialogue. Teachers would teach their students in the way that is required
by the examination - reading aloud, A lot of reading aloud activities were carried by choral
practising reading aloud a dialogue. Whereas with F4, the group of,students for the revised
HKCEE, reading aloud is replaced by role play and group discussion. Teachers no longer
taught reading aloud. More and more time were spent on group discussions and oral
presentations.

DISCUSSION

The above results from the preliminary investigation showed the following degree of washback
effect in the few areas of teaching and learning in Hong Kong secondary schools as a result of
the changes of the present HKCEE.

Washback on teaching contents

The most dramatic change due to the introduction of the 1996 HKCEE lay in the content of
teaching - area of washback intensity. By the time the examination syllabus affected teaching
in Hong Kong secondary 'schools in the 1994 academic year, nearly every school had changed
their textbooks for the students. All of these textbooks are labeled specifically " For The New
Certificate Syllabus"6. Publishers in Hong Kong worked really hard and quickly to get
textbooks ready for the schools. The main reason for this might due to the way in which Hong
Kong society develops quickly especially in commercial matters.

Through interviews with teachers and textbook publishers, it was found out that textbooks
played a very important role in the teaching of English in Hong Kong secondary schools.
Textbook publishers in Hong Kong not only provide teaching materials but also detailed
teaching and learning activities as well as suggested methods. They even suggest how much
time should be devoted to carry out those learning activities. Evidence could be easily seen
from those textbook publishers' seminars and workshops. A textbook publisher said in an
interview;

6
Quoted from the cover of English: A Modern Course published by ARISTO Educational Press LTD.

13

14
 "Anyone who speaks some English would be able to teach English in Hong Kong as we
have provided everything for them. ... Sometimes teachers phone us when they come across
difficulties in teaching a particular unit or task. And we would write a detailed plan for them
thereby."

In Hong Kong secondary schools, there is a general teaching schedule for a whole academic
year called a Scheme of Work which teachers rely on in their teaching. This Scheme of Work
is designed according to the layout of the kind of textbook which the school adopts for the
school cycle or weekly system. Therefore, it could be assumed that if teachers rely on
textbooks a lot, (which was evident through school visits); and if textbooks catering for the
1996 HKCEE have really integrated the underlying theory behind the change of this public
examination and realized this through the language activities in their textbooks, it would be
likely that the 1996 HKCEE would have certain washback effects on the teaching and learning
of English in Hong Kong schools, given the importance of this public examination.

Washback on teachers' attitude and behavior

As can also be seen from the survey results, teachers do have a positive attitude toward the
1996 HKCEE. However, there is not enough evidence yet to prove whether positive attitude
would necessarily bring about positive changes in teaching methodology. Teachers might have
a positive attitude toward the change, yet would still fmd it difficult to put their ideas into
practice. This is evident from the written answers teachers provided when they spoke about
their worries over shy and less outspoken students. Teachers also worried about difficulties
over classroom management and the involvement of teaching facilities as there is an increasing
requirement to carry out more active learning activities by the revised HKCEE.

From the limited data obtained from classroom observations it was clear that those teachers
teaching both the old and the new syllabuses did adopt different types of activities. However,
these obvious change made in teaching lay in the different activities designed in the textbooks
teachers employed: This means that teachers changed their ways of organizing classroom
activities according to the textbook publishers' understanding of the 1996 HKCEE. Those
activities or tasks are designed only on the basis of the simple exam formats provided by the
HKEA.

Sensible teachers were very careful to arrange classroom activities to achieve the requirements
of the revised syllabus. In addition, they also talked about their confusion over the objectives
in the revised syllabus. Virtually, teachers follow the new syllabus simply by adherence to' the
new textbooks at this stage. It would be very much interesting to observe how teachers would
react in their teaching after the first cohort of students sit for the exam. By then, teachers will

14

15
have a clearer idea of the new exam which they can possibly rely on besides the proposed
sample papers and textbooks.

Washback on the English curriculum

Spolsky (1994) points out that backwash effect usually happens irrespective of test designers'
intentions and is out of their control though the reasons might be complicated in different
educational settings. It is unlikely that changes would happen in such a linear and simple
manner as mentioned above. As Alderson and Wall (1993) point out

"What we did not know at the time was just how difficult it would be to determine
whether washback has occurred at all, and to decide, if there were no evidence for it,
whether this was because there was no such thing or because there were conditions in
the educational setting that were preventing it from 'getting through'." (1993:47-48)

Even if there are favorable conditions in the educational setting in Hong Kong (See Results),
we are still not in a position to formulate the nature of washback effect. It is even more
difficult to say whether the effects are positive or negative. What teachers stated that they
would like to change is not necessarily the same as what they actually would do in the
classroom. Though Hong Kong is a fast-changing place where change is common, teachers
still worry about their students. They also showed their confusion over the changes in actual
teaching.

Furthermore, not enough data have been collected to show whether the new examination
would change teachers' behaviors or, if it does, to what extent. From classroom observations,
changes can be seen so far only in the way teachers organize classroom activities. It is difficult
to see whether the washback effect observed in the above teaching activities is simply
cramming for the examination or a difference in the language skills being taught. Cramming is
obviously one level of washback effect. However, it would be difficult to say whether
'cramming of the above kind produces positive o-r negative washback effects on teaching. To
test this, more data based on classroom observations are required. We believe teaching and
learning should include more varied activities than the examination formats alone.

However, it would be natural for teachers to employ activities similar to those activities
required in the examination since a lot of teachers still believe it is their responsibility to
prepare their students well for the examination. If they have not done enough to familiarize
their students with the test formats, they would feel guilty. One teacher said in an interview
that

15

16
 "I am very examination-oriented. And I will do whatever I can to get my students fully
prepared for this public examination."

In Hong Kong secondary schools, the teaching at F5 stops at the end of February or earlier.
March is the month for mock examinations. In May, students will sit for the certificate
examinations. It is stated in the Teaching Syllabus of English (secondary) by the Curriculum
Development Council that the teaching of English at F5 level will be heavily examination-
oriented. Therefore, cramming for the examination would be natural in Hong Kong schools.
With the introduction of the revised examination syllabus toward more task-based skills,
washback effects on the teaching activities might be positive in the way language activities are
carried out.

CONCLUSION

With reference to Figure 1 which investigates how different parties might be influenced by the
1996 HKCEE at the macro level, we could tentatively say that among the different aspects of
teaching and learning, teaching content has so far received the most intensive washback effect
in Hong Kong secondary schools - thus the area of washback intensity. Certain washback
effects on teachers' attitude and behaviors have been observed by school visits and classroom
observations. The role of the public examination in learners' learning has also been
investigated.

However, the data collected so far are not sufficient enough to provide enough evidence to
allow for an analysis of the nature of washback effects, the extent of those effects, and the
quality of them. It would thus be difficult to judge, at the moment, whether those washback
effects are positive or negative in teaching and learning. Further empirical data will be
collected and analyzed later to provide more insights into the nature of this educational
phenomenon.

It is recommended that in research of this kind, investigation should first be carried out to
analyze the nature of the test type, be it a large scale public examination or classroom
assessment since the function and/or the stake a particular test bears determines the degree of
its influences and the area of its washback intensity. Then the investigation should direct
attention to the investigation of that particular education context under which the examination
is issued. Only by analyzing the particular educational context under which the examination
works, would we be likely to find out how a particular examination might influence teaching
and/or learning; how intensive the influence could be and the kind of washback effects,
positive or negative on teaching and learning.

16

17
Moreover, we should explore the degree of the superficiality' of washback effect, 'form' vs.
'substance'. One of major aims of the HKEA in changing the present examination is that " the
new format will have favorable washback on classroom teaching " (HKEA; 1993:5). From the
preliminary results, certain obvious washback effects in aspects of teaching could be seen, for
example, in the changing of textbooks. Textbook publishers have certainly changed the form,
which allowed schools and teachers to have something to rely on in teaching. However, if the
substance was changed, teachers would not like to buy those textbooks. The fact could be
clearly observed in those publishers' seminars and workshops. Teachers turned to textbooks
clearly marked for the revised examination.

Similarly changing the examination is likely to change the kind of exam practice, but not the
fact of the examination practice. Changing the examination test formats does not usually
change the degree of emphasis on examinations nor does it necessarily change teachers'
methods in actual classroom teaching. The intention to bring in positive washback effect in
classroom teaching by changing its major public examination might be ideal, yet it might only
change the form of teaching not too much the substance of teaching.

As Luijten (1991) points out, examinations are not the be-all and end-all of education and the
teaching process. They are not an end in themselves. Research into the role of assessment in
education and the relationship between testing and teaching would contribute to the whole
education purpose assessment serves.

Many people have enlightened me on the impact of the examination change in Hong Kong and
contributed directly or indirectly to this paper. I must, though, thank my supervisor Dr. Peter
Falvey, who worked so hard with me on the whole process of the development of the paper.

The term is recommended by Dr. K.R. Johnson, the University of Hong Kong.

17

is
References;
Alderson, .1C. and Wall, D. (1993). Does washback exist? Applied Linguistics. 14(2), 115-129.

Andrews, S. and Fullitove, J. (1994). Assessing spoken English in public examination-why and how? In J. Boyle and P. Falvey
(eds.) English language testing in Hong Kong. (pp.57-86) Hong Kong: Chinese University.

Andrews, S. (1993). The washback effect of examinations-its impact upon curriculum innovation Paper presented at the 19th
international JALT conference, Sonic City, Omiya, Japan.

Biggs, J.B. (1994)Assumptions underlying new approaches to educational assessment Paper presented at the Hong Kong Educational
Research Association, City University of Hong Kong, November 27, 1994.

Biggs, J.B. (1993) The quality of teaching and learning in an expanding tertiary systemNew Horizon, 34, November, 102-104

Bracey, G.W. (1987). Measurement-driven instruction: catchy phrase, dangero.:.- practice.Phi Delta Kappa, at 683-686.

Education Department, Hong Kong. (1990) Report of the working group set up to review language improvement measures. Education
Department. Hong Kong

Frederiksen, J.R. and Collins, A. (1989). A system approach to educational testingEducational Researcher, 18(9), 27-32.

Frederiksen, N.(1984). The real test bias: influences of testing on teaching and learning.American Psychologiit, 35(3), 193-220.

Fullilove, J. (1992). The tail that wags. Institute of Language in Education Journa1,9 131-147.

Genesee, F. (1994) Assessment alternatives TESOL Matter 3, October/November, 1994.

Gullickson, A.R. (1984) Teacher perspectives of their instructional use of the testslournal of Educational Research, 714)

Haladyna, T.M., Nolen, S.B. & Haas, N.S. (1991). Raising standardized achievement test scores and the origins of test score
pollution. Educational Research, 245), 2-7.

Hong Kong Curriculum Development Committee (1983)Syllabus for English (Forms I-V) Hong Kong

Hong Kong Examinations Authority (1993)Hong Kong Certificate of Education Examination (1996) Proposed English Language
Syllabus Hong Kong

Hong Kong Government

(1984) Education Commission Report No. ! Hong Kong Government Secretariat
(1986)Education Commission Report No.2Hong Kong Government Secretariat
(1990)Education Commission Report No.4Hong Kong Government Secretariat

Hughes, A. (1988). Introducing a needsbased test of English language proficiency into an Englishmedium university in Turkey In A.
Hughes (ed.) (1988). Testing English for university study: ELT Documents /27.(pp.134-153) Modem English Publications.

Johnson, F. and Wong, C.L.K.L. (1981). The interdependence of teaching, testing and instructional materials. In J.A.S. Read (ed.)
Directions in language trsting(pp.277-302) Singapore: Regional Language Center.

Khaniya, T.R. (1990). The washback effect of a textboolibased test. Edinburgh Working Papers in Applied Linguistics, 1 University
of Edinburgh.

Kirkland, M.C. (1971). The effects of tests on students and schools.Review of Educational Research, 41, 303-350.

18

19
Lam, H. P. (1993) Washback - Can it be quantified? A study on the impact of English Examinations in Hong Kong Unpublished
MA dissertation University of Leeds, UK

Li, X. J. (1990) How powerful can a language test be? The MET in China. Journal- of Multilingual and Multicultural
Development, 11(5), 393-404

Linn, R. L. (1983). Testing and instruction: links and d;stinctionsJourna/ of Educational Measurement, 2Q 179-189.

Luijten, J.M.A (1991) Issues in public examinations Lemma

Madaus, G.F. (1988). The influence of testing on the curriculum. In L.N. Tanner (ed.)Critical issues in curriculum: eighty-seventh
yearbook of the national society for the study of education (pp. 83-121). Chicago: University of Chicago Press.

Messick, S. (1989) Validity Educational Measurement (3rd ed.) New York: Macmillan

Morris, P. (1990). Curriculum development in Hong Kong: Education Paper ?Hong Kong: Faculty of Education, University of Hong
Kong

Morrow, K. (1986) The evaluation of tests of communicative performance. In M. Portal (ed.)Inno "ations in language testing
London:NFER/Nelson.

Pearson, I. (1988). Tests as levers for change. In D. Chamberlain and R.J. Baumgardner (eds.)ESP in the classroom: practice and
evaluation (pp.98-107) Modem English Publications.

Popham, W.J. (1987). The Merits of Measurement.driven instruction. Phi Delta Kappa, 68, 679-682.

Shepard, L.A.(1993). The Place of Testing Reform in Educational Reform: A Reply to Cizek.Educational Researcher,244), 10-14.

Shepard, L.A. (1991). Psychometricians' Beliefs about Learning.Educational Researcher, 20(6), .2 -16

Shepard, L.A. (1990). Inflated Test Score Gains: Is the Problem Old Norms or Teaching the Test?Educational Measurement: Issues
and Practice, 9, 15-22.

Shohamy, E. (1993) The Power of Test: The Impact of Language Testing on Teaching and LearningNFLC Occasional Papers, June
1993

Smith, M.L (1991) Put to the Test: The Effects of External Testing on Teachers.Educational Researcher, 2C(5), 8-11.

Spolsky, B. (1994) The examination-classroom backwash cycle: some historical cases. Paper presented at the ILEC'94, University if
Hong Kong.

Wall, D and Alderson, J.C. (1993). Examining Washback: the Sri Lankan Impact StudyLanguage Testing MD.

Wesdorp, H. (1982). Backwash Effects of LanguageTesting in Primary and Secondary Education. Journal of Applied Language
Study, 1(1),40-50

19

20