ISBN: 91-7305-828-9; ISSN: 0281-6768

I CT and formative assessment in the learning society

Akademiska avhandlingar vid pedagogiska institutionen

UME UNIVERSITET
Nr 77 2005

I C T A N D F O R M AT I V E AS S E S S M E NT
IN THE LEARNING SOCIETY

B E RT I L R OO S

Ume universitet
SE-901 87 Ume, Sweden
www.umu.se

BERTIL ROOS

ICT AND FORMATIVE ASSESSMENT

IN THE LEARNING SOCIETY

Department of Education
UME UNIVERSITY

No 77 2 0 0 5

 Bertil Roos, 2005

ICT and formative assessment in the learning society (doctoral thesis)
http://www.onlineassessment.nu

Department of Education, Ume University

Cover Photo: Nils Roos, Landscape of Learning
Graphic Design and Printing: Print & Media, Ume University, March 2005
ISBN 91-7305-828-9
ISSN 0281-6768

Roos, Bertil. ICT and formative assessment in the learning society. Academic
dissertation, Faculty of Social Sciences, Ume university, Sweden, 2005.
ISBN: 91-7305-828-9; ISSN: 0281-6768.

Abstract
In the 1930s and 1940s, less than one percent of the Swedish population were in
higher education. By the beginning of 1990s this proportion had reached 2.4%.
During the 1990s, however, a new economic current flowed in Swedish higher
education. A period of general economic stringency brought the costs of higher
education under scrutiny. Further expansion, therefore, was to be accompanied by
a reduction of unit costs. A discourse of expansion was to be joined by discourse of
efficiency. By the end of the 1990s, however, an efficiency discourse based on quality
assurance was facing difficulties. The educational merits of the efficiency reforms
were not easily discerned.
A new educational or pedagogical emphasis emerged. Quality and
effectiveness were to be augmented via quality enhancement processes and by
mobilising the inner resources of each institution. The emphasis of such thinking
was on development of institutional practices that best favour the development of
activities that, in turn, lead to the best long-term outcomes in teaching and research.
For these reasons, the student body had entered a new world by the start of the
third millennium. The proportion of traditional students was matched by the proportion of non-traditional students. These changes, demographic and economic,
represented a challenge to policy-makers and practitioners in Swedish higher
education. Was it possible to mobilise inner resources to meet the challenge of this
new body of students?
This thesis focuses on one of the responses to this challenge the use of information and communication technologies (ICT) as an integral part of the pedagogics
of higher education. Can ICT, therefore, become an add-in rather than an add-on
to higher education. In particular, this thesis focuses not on teaching in general
but, rather on an add-in issue; that is, can formative assessment be used as an
integral support for learning. Five papers provide perspectives on this response; and
the introduction sets the scene by identifying the key ideas that hold the studies
together; reporting the development projects that were used to clarify these ideas;
clarifying the events and ideas which governed the preparation of the five papers;
and, finally, summarising the conclusions that arise from my research.
The landscape of learning, like the physical landscape, is constantly changing.
But are these changes superficial? Are they the result of ideas and tools that merely
till the surface of the learning landscape? Or do these tools contribute to shaping
the new knowledge that is expected of the learning society? This thesis explores
these overarching questions. It concludes that the distinction between ICT as addon or as an add-in remains central to the organisation of formal education in Sweden.
Key-words: Assessment, internet, ICT, learning, constructivism, formative assessment, constructivist assessment, feedback, examination, on-line, cybernetic, elearning, learning society.

ACKNOWLEDGEMENTS
The telephone rang. I was on a southbound train in the late autumn of 1990.
The caller presented himself as a course co-ordinator from Ume University.
He offered me a position as an assistant lecturer at the Department of Education.
For various reasons, I had to decline; but when the same offer came at a later
date, I gladly accepted. Thanks Lars for starting me on the road to making the
impossible possible.
To become a doctoral candidate and write a thesis did not figure in my previous
view of life. Growing up in the woodlands on the border of Jmtland and
ngermanland was a long way from the academic elite. To become a doctoral
candidate was a strange feeling, far from the everyday life of a one-time gymnasium and adult education teacher. Nevertheless, after several years experience
of university life, I became interested in writing a doctoral thesis. I started in
1999 and remain grateful to Clas-Uno, Widar and Sigbrit who supported me
through those early years. Without them, I would not have started my doctoral
journey.
Finance is an important part of doctoral studies. Thanks are due, therefore, to
the national research programme (LearnIT) of the Knowledge and Competence
Foundation (KK-stiftelsen), the EU commission, the Distance Education
Authority, the Net-University authority, the Foundation for the internationalisation of Higher Education and the Harald and Louise Ekman research foundation, all of whom have financially supported my work. Together,
they not only made it possible for me to conduct my studies but also to gild the
occasionally humdrum life of a doctoral candidate with attendance at international conferences.
I have also been fortunate in being able to work with colleagues from other
research institutions in Sweden and overseas. Many thanks, therefore, to Harry,
John, Barbara, Steven and Hans for all the tricky questions that generated such
constructive discussion in the EU-project; and warmest thanks to Ulf and Anders
for the cooperation, critical comments and constructive proposals we shared at
our working lunches in Ume. Likewise to Inger, Uno, Rolf, Marie-Louise and
Gunnel for their fine cooperation and challenging discussions around the
NetUniversity project.

Thanks, too, to those who participated in the LearnIT courses and seminars,
led by Roger, Ulf, Shirley, Anna, Ove and Gran. Your comments and
constructive viewpoints have been most valuable. Likewise, warm thanks for
the good advice I received from the doctoral seminar group in Ume Elinor,
Jarl, Katarina, Margareta, Mattias, Peter N-n, Peter N-m, Ulrika H and Ulrika
W. And for their readiness to read my articles and provide valuable comments
on my manuscript, I would also like to thank my colleagues Hkan, Ulla and
Martin.
For their willing provision of the information on which this thesis is built, I
would also like to acknowledge the willing assistance of teachers and students
in Ume. Likewise, I must not forget my corridor colleagues with whom I
shared coffee breaks. Many thanks for your indulgence in my ever-lasting
attention to the computer and for maintaining the peace and quiet of my
working environment. Likewise thanks to other colleagues, relatives and friends
who also helped along the way. And thanks, of course, to Nils who photographed
and designed the cover of this thesis.
For his endless patience, his valuable comments, his constant encouragement
and his positive and humble outlook on life, I would like to convey grateful
thanks to my supervisor, Professor David Hamilton. Without his work, this
thesis would never have been finished. I would also like to wish him all the best
when he returns to Scotland for a well-earned retirement after years of hard
work.
What would life be without a family - Kristina, my children Hanna, Linda,
Nils and Ulrika, and my grandson Wille. Without your support, this work
would not have been possible. Thank you for all your patience, for not talking
about this thesis, and for keeping away from the computer when I should have
been sharing my time with you. I hope that in the future, there will less of the
former and more of the latter. Writing a thesis blurs the boundaries between
time and space, working life and private life. It is now time to breathe some
fresh air again; and to take up contact with relatives and friends. Last but not
least, I would like to thank my father, now deceased, and my mother, my
brothers and sisters all of whom have supported me throughout my life.
Thank you all.
Ume, March, 2005

Bertil Roos

ICT and Formative Assessment in the Learning Society

The thesis comprises an introduction and the following papers:
I. Roos, B. (2001). From distance education to on-line learning: Formative
assessment in higher education. Presented at the European Distance and
E-learning Network 10th Anniversary Conference, Stockholm, June
2001.
II. Roos, B. (2002). ICT, Assessment and the Learning Society. Presented at
the European Conference on Educational Research, Lisbon, September 2002.
III. Roos, B. (2005). Examination och det lrande samhllet (assessment
and the learning society). Journal of Research in Teacher Education, Vol.
12, No. 1, pp. 155175. (English translation as an appendix to this
thesis).
IV. Hamilton, D., Dahlgren, E., Hult, A., Roos, B & Sderstrm, T. (2004).
When performance is the product: Problems in the analysis of on-line
distance education. British Educational Research Journal, Vol. 30, No. 6,
pp. 841-854
V. Roos, B., & Hamilton, D. (2005). Formative and Summative Assessment: A cybernetic viewpoint. Assessment in Education, Vol. 12, No. 1,
pp. 7-20.

CONTENT
PROLOGUE ................................................................................................................................................. 1
KEY IDEAS .................................................................................................................................................... 3
University reform ................................................................................................................... 3
Life long, distance, and online education ............................................................... 4
Globalisation of the New Learning ............................................................................. 5
ICT and globalised education ......................................................................................... 6
The Turn to Mind ..................................................................................................................... 8
Mediating artefacts ............................................................................................................ 11
Assessment in Classroom communities .............................................................. 13
Formative assessment ..................................................................................................... 15
Low stakes testing .............................................................................................................. 16
Divergent assessment ..................................................................................................... 17
New Times, New Assessment ..................................................................................... 18
DEVELOPMENT PROJECTS ............................................................................................... 20
Project 1: the Distum project (2000 2001) ...................................................... 21
Project 2: the EU project (2002 2004) ............................................................... 24
Project 3: The Net-university project (2004 2005) ..................................... 30
PREPARATION OF PAPERS .................................................................................................. 33
Paper 1: From Distance education to on-line learning ................................. 33
Paper 2: ICT, Assessment and the Learning Society ...................................... 34
Paper 3: Examination och det lrande samhllet ............................................ 34
Paper 4: When performance is the Product ........................................................ 35
Paper 5: Formative Assessment: A cybernetic viewpoint .......................... 37
EPILOGUE ................................................................................................................................................. 38
REFERENCES ......................................................................................................................................... 40
PAPERS IV ............................................................................................................................................... 45
APPENDIX ............................................................................................................................................. 113

PROLOGUE
In the 1930s and 1940s, less than one percent (11 000 students) of the Swedish population were at universities and institutes of higher education. By the
beginning of 1990s this proportion had reached 2.4% (more than 200 000
students). During the 1990s, however, a new economic current flowed in Swedish higher education. A period of general economic stringency brought the
real and projected costs of higher education under public and political scrutiny.
Further expansion was to be accompanied by a reduction of unit costs. In other
words, a discourse of expansion was joined by discourse of efficiency. Between
1989/90 and 1997/98 the number of students rose by 86%, while the number
of teachers rose by only 17%, a change in student: teacher ratios from 10:1 to
15:1 (Riksdagens revisorer, 2000; Westling et al. 1999). By autumn 2003, 360
000 students were registered in Swedish higher education (Hgskoleverket,
2004, p. 8), 4% of the population.
The introduction of a discourse of efficiency, however, challenged the
advancement of access, effectiveness and quality. In the words of the Swedish
Agency for Higher Education (Hgskoleverket), these changes raised the
possibility that quality was shrinking (Hgskoleverket, 2000, p. 6). Between
1993 and 1998 the number of teaching hours for first year undergraduate
biology courses at Stockholm university fell from 8,3 to 6,4 hours per week
(Riksdagens Revisorer, 2000, p. 25), despite the recommendation of 18 hours
made by an earlier national investigation (SOU 1992:44). Further, only 40%
of this teaching was carried out by university teachers with a doctorate (the socalled gold standard for undergraduate teaching). Hgskoleverkets conclusion
was that if the tendencies noted in the report were valid nationwide, a decline
in the quality of undergraduate education was to be expected (p.6). The Swedish public accountant also suggested that there were signs of a decrease in the
effectiveness of higher education (Riksdagens Revisorer, 2000, p. 6). By the
end of the 1990s, then, an efficiency discourse based on quality assurance was
floundering in Swedish higher education. The educational merits of the
efficiency reforms were not easily discerned.
A new educational or pedagogical emphasis emerged in a second round of
national audits. Quality and effectiveness were to grow, via quality enhancement
processes, by mobilising the inner resources of each institution (Hgskole-

verket, 1998, pp. 6 and 10). The main emphasis of such thinking was the
development of institutional practices that best favour the development of
activities that, in turn, lead to the best long-term outcomes in teaching and
research. The basis suggested for this development was the gathering of information about the work of the institution and the subsequently use of this information in local decision-making (Hgskoleverket, 1998, pp. 17 and 21). In
short, higher education began to incorporate the management maxim: work
smarter, not harder.
In the meantime, however, higher education had not stood still. It had undergone
changes that made it more of a mass than an elite institution. But this change
was not merely quantitative; it was also reflected in the student population, its
age profile, entry routes, composition, and course trajectories.
Age profile: Although there had been a doubling, to 40%, of the proportion
of entrants between the ages of 19 and 25, half of the registered students
were also over 25. Indeed, the increase of students between 19 and 21
was matched by a corresponding increase of entrants over the age of 35.
Entry routes: Fifty percent of the entrants in 2001/02 had studied in municipal adult education compared with 33% in 1995/96. Within medical
subjects, the proportion of entrants from adult education increased from
16% to 43%.
Student composition: Sixty-five percent of the 302 000 applicants in 2003
had already been students in higher education. Moreover, 25% of these
applicants had already completed a degree programme.
Course trajectories: One third of students completing courses in 1995 had
been absent for at least one term. Of the students registered in autumn
2002 only 62% were registered one year later; and, in general, only half
of students who starts at universities complete a degree (SOU 2004:29).
Overall the Swedish student body had entered a new world by the start of the
third millennium. Put simply, the proportion of traditional students was
matched by the proportion of non-traditional students (SOU 2004:29). These
changes, demographic and economic, represented a challenge to policy-makers and practitioners in Swedish higher education. Inherited assumptions were
no longer viable. Was it possible to mobilise inner resources to meet the challenge of this new body of students?
This thesis focuses on one of the responses to this challenge the use of information and communication technologies (ICT) as an integral part of the
pedagogics of higher education. Can ICT, therefore, become an add-in rather
than an add-on to higher education. In particular, this thesis focuses not on
teaching in general but, rather on an add-in issue; that is, can formative assessment be used as an integral support for learning. Five papers provide perspectives

on this response; while this introduction sets the scene. It identifies the key
ideas that hold the studies together; it reports the development projects that
were used to clarify these ideas; it reports the events and ideas which governed
the preparation of the five papers; and, finally, it summarises the conclusions
that arise from my research.

KEY IDEAS
University reform
A new climate arose in European higher education after the Second World
War. Elite education began to retreat in the face of mass higher education; and
resultant images of near-universal post-compulsory education became evident
in many European countries. At the same time, mass higher education became
a very high-stakes affair, an explicitly-regulated gateway to socially prestigious
occupations and, accordingly, the good life (Bakker, 2001, p. 1). Young people
and their parents became aware of the value of continued education; and new
pressures arose around the regulation of entry to higher education.
In a world where university diplomas play an ever-increasing role in determining
peoples lives, demand for tertiary education and continuing pressure on the
secondary examinations used by gatekeepers, will continue for decades to come.
(Bakker, 2001, p. 6)

The eclipse of the norm of the elite university was also accompanied by a parallel
set of political demands that universities should be placed at the heart of the
knowledge economy; and that they should be inclusive institutions where greater
attention is given to gender and ethnic equity and, not least, to democratic
forms of intellectual life. A early consequence of these political demands was
the foundation of Ume University in 1965 (Lane, 1984) and the subsequent
foundation of the Swedish colleges of higher education in 1977.
These reforms were driven by a general wish to increase access to higher
education. In effect, they laid the basis for a national system of higher education
which not only drew on an expanded social and geographic hinterland of
students but also had links with local labour markets. As the then minister of
education (Bertil Zachrisson) stated:
The localisation of resources for higher education are one of the essential factors for
meeting the goal of individual access to education and the maintenance of society

with educated personnel.. ...An obvious starting point is, thus, to broaden and
diversify the provision of higher education, to open higher education to new groups
of students, and to make access easier.... In my view, there are substantial reasons for
extending higher education to new sites outside the existing university towns....The
continued extension of basic higher education should, in this way, and in my view,
be linked to specialist units of higher education. (Prop 1975:9, pp. 488489)

According to the resultant Higher Education Act, which came into force on
1st July 1977, institutes of higher education courses were to foster students:

knowledge and skills;

capacity for critical judgement;
preparation for, and further development within, different occupations; and
personal development. (SFS 1977:218, p. 1)

Life long, distance, and online education

One consequence of the 1970s reforms was that higher education policy
emphasised recurrent education. In turn, colleges of higher education started
to develop short free-standing courses which enabled students to accumulate
academic credit (measured in points). Increased flexibility in the supply of
education also gave students the freedom to devise their own courses, a process
which enabled them to their own learning trajectories, bildningsresor, or (in
Latin) curriculum vitae. One feature of these new trajectories was that many
students elected to blend working life with academic or student life.
This deregulation of courses was formalised in the 1993 reforms for Swedish
higher education. Universities and colleges acquired a measure of autonomy
and self-government as, among other things, an incentive to respond to their
local labour market. This liberalisation also opened higher education to new
markets, in this case constituencies of students who, previously, had received
their training outside higher education (e.g. nursing courses).
Trajectories of life-long and life-wide learning were implicated in this revised
conception of higher education. They were united around a new conception of
learning, notably that learning is not merely the acquisition of knowledge but
also learning the value of intellectual tools relevant to the labour market and
active citizenship. One such tool, drawn into discussions of life-long learning,
adult education and folkbildning in the 1990s, was the technology known,
initially, as IT (information technology) and, later, as ICT (information and
communication technology). This technology which increasingly annexed
new tools or applications was identified as having the educational potential
of sustaining flexible learning. Thus, an intention of the 1990 reforms was that
higher education should be able to offer responsive, flexible courses

commissioned by outside customers and offered as hybrid forms of distance/

local, and synchronous/asynchronous education. In effect, these reforms laid a
foundation for higher education in the learning society.

Globalisation of the New Learning

By the end of the second millennium, Swedish higher education had begun to
focus on the new learning that, in the words of a group of Dutch researchers,
refers to those learning outcomes, processes and instructional methods that are
deemed appropriate to the learning/information society and that are currently
stressed by psychological and educational theory (Kock et. al., 2004, p. 145).
In Swedish law, this new learning differed from the model proposed in the
1970s. It gave less attention to knowledge acquisition and more attention to
intellectual autonomy and research capabilities:
Basic higher education should give students
The ability to make autonomous and critical judgments;
The ability to differentiate, formulate and solve problems autonomously; and
A readiness to engage with changes in their working lives.
Within the educational sphere, [higher education] should not only furnish
students with knowledge and proficiency but also develop their ability to:
seek out and evaluate knowledge in a scientific manner;
follow the development of knowledge; and
Research training should, beyond what applies in basic education, give them
knowledge and skills that are needed for the independent pursuit of scientific
research. (SFS 2001:1263)
In this respect the law was influenced by ideas about the new production of
knowledge that had been described in a report from a study funded by the
Swedish Research Council (Gibbons et al., 1994). Higher education was
redefined as a knowledge-producing industry that also produces human capital professionals and citizens with socially-valued research capabilities. It,
too, is part of the circulation of capital and, accordingly, has its own raw
materials, technologies, sites of production and market places (Bourdieu, P., &
Passeron, J.-C. 1977; and Bourdieu, P. 1988).
While such innovations reflected the discourse of post-1960s expansion, they
also raised efficiency questions. How, for example, will quality be assured when
students are encouraged to compose their own education programmes, including

courses taken outside Sweden? And how will international universities

collaborate and regulate the progression of students? The Swedish deregulation
of courses began, therefore, to take on an global dimension.
The so-called Bologna process creating a common European area for higher
education can be regarded as an example of this internationalisation of higher
education. One of the sub-goals of the process has been to establish common
levels of performance across Europe by 2010. The practical implications of this
proposal are still under the microscope. There is much anxiety over the fact
that there is no common European understanding, for instance, of the differences
between the old and new learning. Nevertheless, the Bologna aspiration is to
regulate performance by reference to a common scale, graded A, B, C, D, E,
Fx, F. The form of this scale, however, conflicts with accepted patterns in European countries (including Sweden). In the words of a Report from the Swedish
Ministry of Education:
Most European countries have more grades than the three grades usually applied in
Sweden. The majority of countries have between five and nine steps, and in many
countries lack a unified national scale. Certain countries use a relative scale while
others operate goal-related grades. Accordingly, the grading systems of European
countries are different and even vary between higher education institutions in the
same country. (Utbildningsdepartementet, 2004:2, p. 113)

As this quotation also indicates, grading systems reflect different views of the
acquisition of knowledge, local as well as national. Further, they are also
problematic insofar as they may embrace different conceptions of knowledge,
new and old. Are students in higher education to be graded according to how
much knowledge they have accumulated and/or the levels of understanding
they have reached? Although modern prophets have suggested, for instance,
that: Students will be able to shop around; that they will be able to take a
course from any institution that offers a good one; and that they will learn
what they want to learn rather than what some faculty committee decided
(Svetcov, 2000, p. 2), such a romantic perspective may not suit impoverished
students who are anxiously looking for appropriate credits or credit-equivalents
as valid tokens of their learning.

ICT and globalised education

As suggested in the previous section, the globalisation of the new learning has
been accompanied by the globalisation of the accreditation of learning. One of
the features of this globalisation has been the spread of information and
communication technology. Just as the harnessing of water and steam power
had, in terms of mechanical power, revolutionary implications for the
industrialisation of the 1800s, information and communications technologies

have been regarded as having equivalent revolutionary implications in terms of

their capacity to manage the information flows that erupted in the latter part
of the twentieth century (Castells, 1999). The British sociologist, Zygmunt
Bauman, has offered a similar, rheological (flow-based) view of adult education.
Humanity lives in liquid-modern times where the mass of accumulated
knowledge is represented as sites of disorder and chaos. As a result, Bauman
suggests, the art of living in a world over-saturated with information is still to
be learned (Bauman, 2003, pp. 4142).
As Castells argues, and Bauman implies, skills in ICT have become essential to
the management of the information flows of modern life and, by the same token,
they are intrinsic to the accomplishment of life-long learning. This, then, has
been the political pressure behind the extension of ICT in higher education. The
political goal has been to make education more accessible, to increase the quality
of educational provision, and to increase the production, distribution and
utilization of information and knowledge. Yet what role should ICT play in this
initiative? Is it merely an information technology, a set of devices for distributing
information, publishing course plans, and communicating with students. Or is
it a tool that, in the terms of the 2001 law, fosters critical and autonomous
knowledge, the ability to differentiate, formulate and solve problems and the
autonomous independent pursuit of scientific research? It may connect with
learners; but does it also communicate with learners, making them not only information-rich but also knowledgeable? Will they drown in such information or
will they learn how, creatively, to go with the flow?
ICT is a product of the work of computer scientists and educational
technologists. But how does it relate to every-day pedagogics? Certain international commentators have discussed this issue in terms of a technology/pedagogy
split. Two Israeli commentators, for instance, have suggested that in spite of
the booming literature on ICT and education, there is almost no discourse on
the subject (Aviram, & Tami, 2004). And a Norwegian writer has suggested
that although distance education is the field which has most vigorously adopted
new technologies it also seems to be the area that is most in need of educational
theories (Nordkvelle, 2004).
The key question is whether there are attributes of ICT that make it qualitatively
different from other sources of power, like steam and water? Put most simply, is
ICT merely an add-on to the practices of higher education; or, in what sense
can it be an add-in changing higher education in the direction of the new
learning? One answer to this pedagogic question is immanent in the field of
informatics and, in particular, the realm of information engineering known as
cybernetics. Cybernetics is sometimes known as the study of feedback. But
cybernetics is a special form of feedback. It is not the kind of feedback that is
discussed in behaviourist psychology, where the human brain is treated rather

like a telephone switchboard. Instead, cybernetics relates to situations where

information is stored, reorganised and subsequently used to steer the course of
a mechanical or human system. For this reason, the analysis of memory is
central to both cybernetics and IT. Indeed, it is this potential that also drives
debates about ICT and education. In effect, can memory-based ICT operate
in the same way as the human brain?
Informatics is based on digital information a series of on/off or 0/1 symbols.
These binary digits (the origin of the word bit) are the raw materials of
informatics. But education is based on something else cultural meanings that
are transmitted from generation to generation. As Hamilton suggests, education
is not an automatic knock-on process transferring information from generation to generation but, rather, one that incorporates intellectual processes of
coding and decoding (1990, p. 22). Such coding and decoding, he claims is a
clarification process:
Teaching and learning occur across a cultural medium. Moreover, many teachers
and learners find this medium to be foggy or cloudy. It hinders satisfactory
communication. Teaching and learning, therefore, are never easy. They always include
an element of demystification. (p. 17)

Thus, the fields of education and informatics overlap, like the parallel fields of
neurology and psychology. But they are also different. Whereas informatics
and neurology operate with a conception of brain, education and psychology
operate with a conception of mind a concept that, together with related ideas
about consciousness and cognition, came to prominence after the Second World
War.

The Turn to Mind

Philosophers and pedagogues have wrestled with learning since Graeco-Roman times. Nevertheless, the advent of the learning society has sharpened these
discussions. What kinds of learning and knowing are appropriate to the learning
society and the pedagogic question how can they be fostered? Further, the
place of mind in education has not only been raised by work in ICT (and
cybernetics), it was raised earlier in a reaction against behaviorism that took
place in the 1950s and 1960s in the USA. It is likely, in fact, that this fusion of
cybernetics and cognitivism (mind science) was fostered by the convergence of
different flows of educational thought. Structuralist or logico-mathematical
ideas about transformation associated with the Swiss psychologist, Jean Piaget
(18961960), were linked to ideas about memory and language raised
respectively among information scientists (e.g. Norbert Wiener) and linguists
(e.g. Noam Chomsky) working at the Massachusetts Institute of Technology
(MIT) and to ideas about mind raised by psychologists (e.g. Jerome Bruner)
working in the nearby institution, Harvard University.

The inter-disciplinary results of these intellectual fusions are discussed in the

chapter on The turn to mind in Bruners autobiography: In Search of Mind
(1983, chapter 7). Bruner reports, for instance, the 1946 split of the psychology
department at Harvard University into social and psycho-physical groupings;
the pioneering symposium on Cognition held at the University of Colorado
(Boulder) in 1955; his own work that led to A Study of Thinking; and the
participation of psychologists (e.g. George Miller) and information scientists
(e.g. Allen Newell & Herbert Simon) at the Second Symposium on Information theory held at MIT in 1956. The work of Bruner and others provided a
link between cybernetic conceptions of memory and psychological conceptions
of metacognition. Both point to the possibility, if not the existence, of reflection
(the re-organization of information); and both imply a vision of what might be
accomplished by such reflection (higher-level understanding).
Another educational consequence of these ideas about the reorganization of
information was that, through reflection, learners would become active
participants in their own learning. The turn to mind, therefore, was one of
the underpinnings of the learning society. Learning and learners were
repositioned at the centre of the educational enterprise, leaving teachers to one
side. In the clich popularised in the USA, teachers underwent a transformation from being sages on the stage to mere guides on the side. Through the
turn to mind, however, education could be re-engineered to produce
autonomous, self-directing and lifelong learners.
One source of ideas for the active participation of learners is the literature
reported by Burbules and Bruce on theory and research on teaching as dialogue
(2001). The importance of their article is that it links dialogue as a longstanding idea in Western education to more recent ideas about dialogue as a
discursive practice. The former, classical perspective is sometimes described as
socratic teaching. Recently, however, doubt has been cast on the pedagogic
merit of such teaching. It has been characterised, for instance, as bullying
(Griffiths 2001, p. 34), on the grounds that it cannot be distinguished from
criminal interrogation. Recently, too, Diana Laurillard (2002, p. 74) has also
commented, in a widely circulated volume on rethinking university teaching,
on the myth of socratic [i.e. dialogic] teaching. Accordingly, she prefers the
more positive tone of teaching through a conversational framework. Indeed,
she adopts the latter perspective in the light of twentieth century research on
language use, including Swedish phenomenographic research (e.g. Marton &
Booth, 1997).
This linguistic perspective on language and dialogue also has a direct link to
the constructivism and cognitivism of the new learning. As Burbules and Bruce
put it, teaching as dialogue:

explores discourses as forms of sociohistorically constituted relations among people,

activities, texts and situations. Participating in a discourse means assuming a role in
a community of practice. (p. 1103)

Further, they define dialogue as: a pedagogical relation characterized by an

ongoing discursive involvement of participants, constituted in a relation of
reciprocity and reflexivity (p. 1117). By their reference to reciprocity and
reflexivity, Burbules and Bruce specifically indicate that dialogue can be linked to a constructivist pedagogy (p. 1117). Learners, that is, are helped into
new ways of thinking by resources e.g. words, ideas, values, symbol systems
mediated in dialogue.
Although Burbeles and Bruce do not take up the challenge of describing such
a pedagogy, the Anglo-Canadian, Gordon Wells, made such an attempt in
Dialogic Inquiry: Toward a sociocultural practice and theory of education (1999).
He claimed, for instance, that an adequate curriculum theory must utilize an
interactive model of teaching and learning and that the teachers role is based
on initiating and guiding this dialogue. Equally, he claimed that teachers must
encourage students to be agentive in directing their own learning and,
accordingly, that teachers should seek to equip them [learners] with socially
valued ways of thinking and acting (p. 119121).
Wells ideas resonate with the arguments about dialogue advanced by Burbeles
and Bruce. This is hardly surprising since they draw from the same well of
pedagogic thought. Wells version of constructivism builds on ideas about
internal and external dialogue advanced by the Russian psychologist, Lev
Vygotsky (18961934; see also Daniels, 2001). In turn, he emphasises three
features of dialogic pedagogy: (1) the essentially dialogic nature of the discourse
in which knowledge is co-constructed; (2) the significance of the kind of activity
in which the knowing is embedded; and (3) the important role played by the
artifacts that mediate the knowing (p. 127). Wells believed, therefore, that an
important role for the teacher is to create a:
classroom community in which the search for understanding, and the dialogue
through which this is accomplished, pervades all areas of the curriculum and is
inclusive of all students, whatever their social, ethnic, or linguistic background.
(Wells, 2001, p. 119120)

To foster dialogue, Wells points to the value of collaboratively produced

knowledge artifacts, psychological tools of various kinds, which are available
for further knowledge building. Further he recognises the cultural dimension
of learning, delegating to each community of inquiry how these principles can
most effectively be realized in different settings (pp. 129130). One of these
psychological tools is feedback which, as Wells acknowledges following Vygotsky,

is both contextual and social. Moreover, the social dimension of dialogue a

word that, etymologically, means through thinking allows that dialogue can
take place within the same person (e.g. reflections over time), as well as between
two or more persons engaged in synchronous speech.
The relationship between feedback, mind and dialogic pedagogy has two
implications for this thesis. First, assessment can be seen as a mediating artifact;
and secondly, that the primary site for the organization of such feedback is, in
Wells terms, the classroom community.

Mediating artefacts
Feedback is a concept that came into everyday use from communication engineering. In the process, however, the nuances that were incorporated by the early
pioneers of cybernetics (e.g. Norbert Wiener) were lost. Wiener and his
collaborators clearly distinguished different kinds of feedback: homeostatic feedback (self-regulating), servo-mechanical feedback (which keeps a system on a
pre-defined course) and self-directing feedback (whereby feedback not only
affects a systems behaviour but also its structure or organisation). Self-direction
has been defined, by the British Standards Institute, as the property of a body,
process, or machines (without closed loop control) of reaching a new steady state
after a sustained disturbance (in Mayr, 1970, p. 134, emphasis added). Indeed,
in this last respect, there is a direct link between cybernetics and constructivism
as noted by one of the pioneers of constructivism, Ernst von Glasersfeld, in his
article on Cybernetics, experience, and the concept of self (1979).
Cybernetic ideas may have been influential in the field of informatics, but their
effect on educational practice seems to have been limited. Glasersfelds intervention, it seems, was too early; its impact only came when the same ideas were
reformulated in his Radical Constructivism: A way of knowing and learning (1995).
In the meantime, feedback retained a restricted, behaviourist meaning, merely
the return flow of information. For instance, university course teams may seek
feedback from students. In turn, it is assumed that they use this flow of information to revise existing practices.
This consumer-oriented sense of feedback is problematic, however, because it
says nothing about how or whether the information is used. In Sadlers
terms, such assembled information is left dangling (1989, p. 121). It may be
gathered; it may even be analysed and reported; but it may or may not have
been utilised for decision-making (as envisaged by Hgskoleverket, 1998).
Return flow has been engineered; but nothing changes. Such feedback, therefore,
is not used cybernetically to create new structures. Thus, popular uses of the
word feedback may not embrace the essence and etymology of the original
Greek word that Norbert Wiener used to invent the word cybernetic, that feedback serves as a governor.

For this reason there are two separate literatures on feedback. One of them
dates back to behaviouristic conceptions of feedback that came to prominence
in the studies of Ivan Pavlov (1849 1936) and the psychological studies of
John Watson (18781958). In that form, behaviourist feedback remained a
feature of educational thinking (e.g. central feature to learning) until the spread
of cognitivism and constructivism via Piagets and Vygotskys formulations
popularised in the 1950s. Indeed, certain reviews of feedback (e.g. BangertDrowns et al. 1991 and Mory, 2002), fail to note the impact of cybernetics on
the revision of feedback theory.
The literature on cybernetic feedback stems from the growth of information
science after the 1930s. By 2001, one of the leading theorists of classroom
assessment and former President of the American Educational Research Association, Lorrie Shepard, was well aware of the feedback ambiguity the fact
that it is central to both behaviourist and constructivist theories of teaching
and learning:
The idea of feedback comes from electronics where the output of a system is
reintroduced as input to moderate the strength of a signal. Correspondingly, both
behaviorist and constructivist learning theories take for granted that providing
information to the learner about performance will lead to self-correction and
improvement. (Shepard, 2002, p. 10911092)

Further, she recognised unlike Bangert-Drowns et al. (1991) and Mory (2002)
that meta-analysis of the feedback literature is of limited value.
For the most part, however, meta-analyses of the feedback literature are of limited
value in re-conceptualizing assessment from a constructivist perspective, because
the great majority of existing studies are based on behaviorist assumptions.... [And]
Relatively few studies have been undertaken in which explicit feedback interventions
have been tried in the context of constructivist instructional settings. (Shepard,
2002, p. 1092)

Moreover Shepard makes a telling observation on the history of behaviourism

in the introduction to her article:
Historically, because of their technical requirements educational tests of any
importance were seen as the province of statisticians and not as that of teachers or
subject matter specialists. Researchers who conceptualized effective teaching did
not assign a significant role to assessment as part of the learning process. (p. 1066)

To this extent, she sought to develop a framework for a reformed view of

feedback, where assessment plays an integral role in teaching and learning (p.
1066). More recently, Laura Hamilton has voiced the same argument:

There is an increasing recognition in the educational practitioner and research

communities that classroom-based assessments can usefully complement large-scale
assessments as instruments to promote educational change and can contribute to enhanced
validity of inferences about student knowledge and skills. (Hamilton, 2003, p. 31)

Assessment in Classroom communities

According to an extensive review by Ginette Delandshere, most educational
measurement specialists are remote from classroom communities. They have
worked from century-old understandings and behaviourist perspectives.
Learning is seen as the accumulation of knowledge and, in turn, teaching is
organized to impart such knowledge in an atomized, sequential, and hierarchical
manner. Assessment, then, is to establish whether an individual can reproduce
delivered knowledge. This view of testing harks back to Skinner, Watson and
Pavlov, since it is based on estimating the probability of recurrence of correct
responses given a particular stimulus (Delandshere, 2002, p. 1463; see also
Callahan, 1962; and Karier, Violas & Spring, 1973).
Although the French psychologist who developed the first IQ tests, Alfred Binet,
believed in the educability of the intelligence, most US psychologists in the
early 1900s saw IQ test results as a scientifically exact measure of a fixed trait
(Shepard, 2002, p. 1069) something that was independent of classroom processes. They saw intelligence as innate; and the way to handle it in education was
by using a differentiated curriculum (see also Kliebard, 1995). Delandsheres
analysis of assessment practices follows the same line of argument. She
maintained that, by the end of the 1900s, educational assessment had, for the
most part, become a relic of the past.
Testing had become weighed down by limited purposes and methods that
generate limited data. It relied on poorly articulated, ad hoc theories and
assumptions of learning and knowing; and examinees were expected to submit to the process without active and equal participation (e.g., critique,
reflection, self-reflection). Moreover, Delandshere suggested that secrecy,
reward, and punishment were still key concepts in the conduct of testing.
Such models might be considered experimental or for use in external assessment, but they were unlikely to be used by teachers in the classroom (Delandshere, 2002, pp. 1466 and 1478).
Earlier, Robert Mislevy had used the turn to mind to make a similar claim
about test theory:
The essential problem is that the view of human abilities implicit in standard test
theory . . . is incompatible with the view rapidly emerging from cognitive and
educational psychology. (Mislevy, 1993, p. 19)

Finally, Delandshere re-iterated the claim, already reported from Shepard and
Wells, that conceptions of assessment that had become dominant in the USA
have little connection with classroom practice or with associated conceptions
of knowledge as product of learning, process of learning, or both.
She continued:
Given the vastly different theoretical perspectives on learning that are currently at
play, how would it even be possible to simply adapt and develop existing assessment
methods and analyses when these were conceived within an incompatible
epistemological tradition? What does the concept of assessment then become when
knowledge is conceived as developing among individuals collectively participating
in an activity rather than as some thing that individuals possess? The value judgments
inherent to assessment certainly do not disappear, but in the latter perspective they
are not external to but a part of the act of learning in that they are located in the
discourse, actions, and transactions of individuals in participation. (pp. 14781479)

Indeed, a stronger version of the same argument has been marshalled by Dylan
Wiliam. Attempts to develop constructivist learning and teaching can, he
suggests, be completely distorted by summative assessments:
Reliance on traditional assessments has so distorted the educational process leading
up to the assessment that we are, in a very real sense, spoiling the ship for a halfpenny-worth of tar. The ten years of learning that students in most countries
undertake in developed countries during the period of compulsory schooling is
completely distorted by the assessments at the end. (Wiliam, 2000, p. 3)

The comments of Shepard, Delandshere, Mislevy, Wells and Wiliam converge

on two ideas: (1) that learning is a developmental and contextual classroom
process; and (2) that the assessment of learning is not the same thing as the
testing of learning. What is preferable, they maintain, is an assessment culture
which is integrated into a learning culture. Moreover, other commentators have
also noted that such integration is even more important to integrated e-learning
(Jochems, Merrinboer, & Koper, 2004, p. 40) or, in Swedish, net learning or
flexible learning.
Assessment in classroom communities, that is, has an integrity and autonomy
of its own. What, then, are the defining elements of this integrated assessment
culture? How can it be characterised? For the sake of this thesis, three aspects of
assessment are considered: formative assessment, low-stakes testing, and divergent assessment.

Formative assessment
The distinction between formative and summative practices has a history that
reaches back to the rise of cognitivism and the beginnings of constructivism in
the middle of the 1900s, as outlined above. Michael Scriven, who popularised
these terms, originally wrote about curriculum evaluation. He recognised that
evaluation, like assessment, could have different roles. He linked the role of
formative evaluation to curriculum development and teacher self-improvement;
while the role of summative evaluation he delegated to evaluation of the final
product. Evaluation, he suggested:
May have a role in the on-going improvement of the curriculum. ... [Or it] may
serve to enable administrators to decide whether the entire finished curriculum ...
represents a sufficiently significant advance on the available alternatives. (Scriven,
1967, pp. 4142)

The relationship between classroom assessment and formative assessment was

pointed out in a seminal paper by Black and Wiliam (1998) when they wrote
that the terms classroom evaluation, curriculum-based assessment, feedback
and formative evaluation can be used as synonyms for formative assessment
(pp. 4445). Elsewhere, Wiliam emphasised that summative and formative
assessment have contrasting purposes:
Summative assessments are best thought of as retrospective. The vast majority of
summative assessments in education are assessments of what the individual has
learnt, knows, understands or can do.... In contrast formative assessments can be
thought of as being prospective. (Wiliam, 2000, p. 14)

Earlier, Sadler had also paid attention to denotations of the word formative.
Etymology and common usage associate the adjective formative with forming
or modelling something, usually to achieve a desired end. (1989, p. 120).
Further, Sadler called attention to the cybernetic, as well as the classroomresearch, dimension of formative assessment. It is concerned with how
judgments about the quality of student responses ... can be used to shape and
improve the students competence by short-circuiting the randomness and
inefficiency of trial-and-error [i.e. behaviourist] learning. (p. 120).
The distinction between formative and summative relates, as Sadler and Wiliam
suggest, not to the assessment information itself but to how that information is
used. If the teacher merely communicates such information to the learner, it is
not legitimate to call the assessment formative. Yet, if the learner understands
and acts on the information then the assessment can legitimately be characterised
as formative (Wiliam, 2000, p. 12). Black and Wiliam also recognise that teachers
have a choice of two complementary options in formative assessment. They
can either develop the capacity of students to recognize and fill the gaps in

their own learning; or teachers can take the responsibility for directing the
appropriate remedial activity. From a constructivist perspective, however, the
former implies the latter since, by developing students capacities in certain
directions, teachers are also implicitly steering the remedial activity.
Black and Wiliam, like other sources used in this thesis, suggest that assessment must be seen as central to learning; that students should be active in their
own assessment; and that they must be able to revise their own learning in the
light of an understanding of what it means to get better (1998, p. 22). If,
therefore, assessment is an occasion for learners to learn, it is important to
induct learners, through dialogue, into a parallel discourse or metacognition
where they are made aware of their understandings. Wells, for example, links
this metacognition not merely to classroom dialogue but also to the provision
of educational scaffolding an image originally articulated by Jerome Bruner
(1975):
one of the chief functions of the use of language in the classroom is to induct
students into modes of discourse that provide them with frames of reference with
which to recontextualize their experience, and that it is this task that gives
educational scaffolding its particular character. (Wells, 1999, p. 127)

Low stakes testing

Assessment is an intervention in someones life. It is designed to foster intellectual
development and, as such, may have significant social consequences. In gambling
terms, assessment is a stake (i.e. an investment) in the outcomes of an action.
But there is an element of chance in all assessments. A high stakes investment
is one whose consequences are socially significant. Candidates who pass the
test receive higher social rewards and benefits than those who fail the test. A
low stakes investment, on the other hand, is one where the social impact of
success and failure is minimal.
Formative assessment, then, is a social strategy for favouring the desired outcome.
Thus, assessment entails both practical implications and ethical risks. When
the stakes are high, for instance, there is a temptation to teach merely to the test;
and this strategy becomes more attractive, if less ethical, as the stakes become
higher (for other high-stakes strategies see, for example, Amrein & Berliner,
2002). The introduction of high-stakes assessment, as Wiliam pointed out,
fosters the unethical over the pedagogical. The assessment procedures shape
the teaching practices. Test developers and teachers start out with the intention of making the important measurable, and ends up making the measurable
important (Wiliam, 2000, p. 1). In a similar way, Shepard identified the ethical
problem associated with high stakes testing in the form WYTIWYG or What
You Test Is What You Get. (2002, p. 1082). As Laura Hamilton pointed out,

the enduring problem with high stakes assessment is that it may cloud teachers
ability to distinguish between ethical and unethical practices (2003, p. 36).
Robert Linn, President of AERA (20034), raised the same issue: the
unintended negative effects of the high-stakes accountability use often outweigh
their intended positive effects (2000, p. 14). Low stakes testing is not troubled
by such ethical problems and Linn went on to make seven suggestions to
reduce the social cost of high stakes testing and, in the process, enhance the
validity, credibility and positive impact of assessment and accountability
systems. These included dont put all of the weight on a single test, recognize,
evaluate, and report the degree of uncertainty in the reported results and put
in place a system for evaluating both the intended and unintended positive
effects and the more likely unintended negative effects of the system (p. 15).
Linns suggestions, that is, were aimed at lowering the assessment risk by raising
the likelihood of desireable educational outcomes. Ideally, low stakes assessment is both ethically and pedagogically defensible. Its side-effects approach
zero and only the desired effects are promoted. In short, the positive effects
that testing is supposed to have on the learner are accomplished without
distraction.

Divergent assessment
Divergent assessment and its opposite, convergent assessment, arise from
teachers differing views of learning and the relationship of assessment to the
process of intervening to support learning. This distinction is known from the
work of Harry Torrance and John Pryor, as reported in Investigating Formative
Assessment (1998, p. 153). Their claim is that two conceptual models of assessment can be identified on the basis of teachers views of learning and the process of intervening to support learning. To clarify these differences they used
the labels convergent and divergent assessment.
The key task in convergent assessment is to find out whether the student has a
predetermined specific kind of knowledge, understanding or skills. It focuses
on students knowledge, understanding and skills in relation to the published
curriculum. It prefers pseudo-open questioning and focuses on contrasting error
responses and correct responses. It employs tick-lists and can-do statements. It
assesses in a linear way; and the assessment is made of the student, and executed
by the teacher. Assessment with these characteristics can also be described as
behaviouristic. Using suitable probes, the teacher elicits the knowledge, understanding or skills of the learner.
Divergent assessment, on the other hand, has students understanding in focus.
It is goal-free. It is not constrained by predetermined knowledge, understanding or skill. It aims to find out what a student knows or can do. The assess-

ment is performed by the teacher and the student working together (cf. dialogue).
Divergent assessment is characterised by flexible planning, open forms of
recording, emphasizing the learners understanding, open tasks, open questioning
and descriptive, qualitative feedback. Divergent assessment strives towards
teaching in the zone of proximal development. It is not claimed, however, that
teachers should always use divergent assessment. Both forms of assessment serve
different purposes. The argument of assessment theorists, like Torrance and
Wiliam, is merely that school teachers and other educationists should be aware
of the differences.

New Times, New Assessment

The turn to mind and the associated awareness that assessment can have different roles has had an impact on test and assessment theory. In particular the
concepts of alternative or authentic assessment have spread throughout the
world of educational research. For example the google search-engine generated
over 5.6 million sites (2004-11-01) while the US data-base, ERIC (Educational
Resources Information Center, pre-1966 to 2003), yielded 6619 references for
the boolean search alternative assessment OR authentic assessment. Moreover, closer scrutiny of the ERIC database suggests that alternative assessment
began to take off as a field of inquiry between 1990 and 1995.
To understand the different elements of alternative assessment, it is useful to
turn to recent reviews. One example is provided by Effie Maclellan from the
University of Strathclyde. She suggests that, broadly speaking alternative assessment is:
characterized as an alternative to standardized, norm-referenced, multiple-choice
testing and typically claims the following features:

Student involvement in setting goals and criteria for assessment

Performing a task, creating an artifact/product
Use of higher level thinking and/or problem solving skills
Measuring metacognitive, collaborative and intrapersonal skills as well as
intellectual products
Measuring meaningful instructional activities
Contextualisation in real world applications
Use of specified criteria, known in advance, which define standards for
good performance. (Maclellan, 2004, p. 312)
The diverse literature of alternative assessment is both a reaction to the past
(e.g. the behaviourism described by Shepard) and a response to the claim
made in the 1970s that the human species is moving towards the kind of new
learning that animates a learning society. One of the earliest comments on the

learning society came from Torsten Husns reflections on the reform of schooling
after the 1960s:
Among all the explosions that have come into use as labels to describe rapidly
changing Western society, the term knowledge explosion is one of the most
appropriate. Reference is often made to the knowledge industry, meaning both
the produces of knowledge, such as research institutes, and its distributors, e.g.
schools, mass media, book publishers, libraries and so on. What we have been
witnessing since the mid-1960s in the field of distribution technology may well
have begun to revolutionize the communication of knowledge within another ten
years. (Husn, 1974, p. 239)

This idea, as noted earlier, subsequently gained considerably official attention.

It was argued that the new economic conditions of human life value learners
who can operate effectively and can go on learning throughout life and who
are capable of coping with uncertainty, diversity and the need for collaboration
with others (Broadfoot, 2002, p. 1).
Shepard made the same argument about the turn to mind that new times
require new assessment practices:
Classroom assessment must change in two fundamentally important ways. First, its
form and content must be changed to better represent important thinking and
problem solving skills in each of the disciplines. Second, the way that assessment is
used in classrooms and how it is regarded by teachers and students must change.
The content of assessments should match challenging subject matter standards and
be connected to contexts of application. (Shepard, 2000, p. 11)

Such a constructivist assessment paradigm will contain self-assessment and feedback as a central part of the social processes that mediate the development of
intellectual abilities, construction of knowledge, and formation of students
identities; it will look to assessment as a source of insight and help instead of
it being the occasion for meting out rewards and punishments; and it will
become integral to Vygotskys idea of a zone of proximal development (Shepard,
2000, pp. 2, 15, 16).
The emergence of ICT since the 1970s has sharpened these questions. Will
these new times incorporate ICT as an add-in rather than an add-on? Does the
current status of e-learning across Europe match up to Shepards blueprint?
Has the promise of ICT been oversold and underused (Cuban, 2001). Does
its presence in higher education merely demonstrate high-level ambitions with
poor implication (Jochems, Merrinboer & Koper, 2004, p. 151)? If ICT is,
indeed, implicated in the information explosion or revolution, what does
this mean for education and the learning society? Are schools and universities

merely channels that provide access to this information, making them scarcely
different from public libraries? What, in these circumstances, has become of
the new learning and the relationship between information, knowledge and
meaning?
Oleson, suggests that there is a fundamentally different perspective on
knowledge built into learning theories. One deals with knowledge as an entity
which can be transacted, transferred and acquired while the other perspective
holds knowledge to be performative, a permanent learning process which is
interwoven into action and social relations (Olesen, 2003, p. 183, see also
Sfard, 1998, Johansson, 1999). Another cautious perspective on ICT in
education is offered by Ewa Olstedt from the Swedish Institute for Studies in
Higher Education (Stockholm). She has asked whether ICT is a burden or
benefit for education? (2003). She feels that if (Swedish) higher education is
to meet the forecasted challenges of the new learning, it must develop a more
sophisticated view of ICT in education. It requires, she suggests, more than the
attention currently given to teaching strategies (cf. net-based delivery systems).
It should also include the promotion of critical reflection. She concludes:
There is an obvious risk with ICT and the great amount of information in the
learning process if the technology comes in the first place and...individual
interpretation and critical ability is underestimated. (Olstedt, 2003, p. 237)

If Olsted and Olesen are correct, Swedish higher education is at a crossroads or,
in Baumans terms, a watershed (2003, p. 42). Is it self-evident that ICT offers
a better way of teaching and learning? Or does it merely operate as an add-on,
offering faster flows of information? In other words, can it meet the reflexive
demands of constructivism as described, variously, by McClellan, Oleson
and Olstedt?
The following section of this introduction portrays various attempts to grapple
with this issue. It reports a series of studies undertaken by the author in
collaboration with colleagues at different departments and institutes of higher
education in Sweden.

DEVELOPMENT PROJECTS
The previous section of this paper has described the key ideas and circumstances
that shaped my work as a doctoral candidate. Throughout, however, I have
been bothered by a parallel question about the viability of these ideas in practice.
This last issue became the focus of three small-scale projects, conducted between

2000 and 2005. The first project was funded between 2000 2001 by the
Swedish Distance Education Authority (DISTUM), with the title Internet
based assessment in distance education. The second project was an extension
of the DISTUM project. It had the same title albeit in English and was
funded jointly, between 20022004, by the European Commission (EU) and
DISTUM. The third project, with the title Net-based assessment (2004
2005), was funded by the Swedish Government as a contribution to the
developmental work of the Swedish Net University.
The first two projects were development initiatives. They were attempts to
insert ideas about formative assessment into the practice of university teachers.
The third initiative was designed to map the form, content and extent of
innovative assessment practice in Swedish higher education. Working on these
projects, I had to grapple with the ideas discussed in the first part of this
introductory essay ideas about examinations, assessment, ICT and the learning
society. The outcomes of these investigations are described, below, in terms of
insights that have arisen from reflection individual and collective over the
results of these investigations. Collectively, they helped in the preparation of
the articles and they have heightened my general awareness about ICT and
formative assessment in the learning society.

Project 1: the Distum project (2000 2001)

(final report available at http://www.rhu.nu/distum/pdf/Distum_2000_H12_
Slutrapport.pdf )

Aims and activities

The aim of the study was to implement internet-based formative assessment,
to identify the advantages and disadvantage of internet-based assessment; and
to establish the impact of frequent assessment on undergraduate students attitudes to their studies. The study also included training teachers in test theory,
question construction, and software systems for internet-based assessment.

Software
On-line assessment was organised in the following way. Teachers could download
software from the internet, install it on their own computers, and operate it
within the aid of Microsoft Windows. The specific software (Questionmark
Perception) was chosen because it met the project specification. That is, it met
the requirement that teachers should be able to create and operate assessments;
and that they should be able to generate reports arising from those assessments.
The software also offered the possibility that teachers could install their assessments on a server (for wider distribution).

In this project, the internet provider also served as an intermediary. Teachers

sent their assessments, via e-mail, to the internet provider who made them
available on the project server. In turn, teachers and students could access the
assessments via their web browser. Finally, the software meant that assessment
reports could also be stored on the server and made available to both students
and teachers. The same software was used in the EU project, and is described
fully in the description of the EU project.

Selection of courses and participants

One department within the area of technical and natural sciences (Department of Applied Physics and Electronics), and another within the area of the
humanities and social science (Modern Languages) were selected to participate
in this study. The selection of these departments was not random. They were
an opportunity sample drawn from earlier contact with teachers in these departments.
The departments chose which courses should take part in the development
project. The Department of Applied Physics and Electronics chose two quartertime courses (25% of a full-time course load):
Medicinsk teknik (5 p)
Analog elektronik (5 p)
and Modern Languages chose two half-time courses
Realia A (3 p)
Sprkhistoria B (2 p)
Four teachers used the software in the Department of Applied Physics and
Electronics, and two in Modern Languages. The teaching groups consisted of
the students in the final phases or upper levels of courses in these areas. Twenty
three students participated in the Department of applied physics and electronics,
and 27 in modern languages. In addition, a random sample of four students
per course was interviewed.
Five men and one woman participated in the study as teachers. Three of them
were between 3040 year while two were older and one younger. Five of the
teachers worked as junior lecturers and one as a senior lecturer. All worked full
time. Three of the teachers had taught fewer than five years, one 515 years, one
1524 year, and the sixth teacher more than 25 years. Five of the teachers described
their experience of the use of computers and internet as extensive or fairly
extensive, while one of the teachers thought that they had limited experience.

Sixteen men and 34 women participated in the study as students. Most of

them were fulltime, between 25 and 35 year, and had previous experience of
distance education. About half of the students also had employment outside
the studies. A majority used their own home-based computers, and almost all
used Internet Explorer as their browser. Between 60 and 80 percent used email
and internet resources on a daily basis; and a majority agreed that it was
important to use IT-tools in their studies.

Data collection and analysis

Data were gathered by means of web based questionnaires and follow-up
interviews. The questionnaire was designed to gather information about student and staff attitudes to internet based assessment; and the interviews, derived from the questionnaires, were designed to illuminate the students and
teachers experience of such forms of assessment.
All teachers completed pre- and post-questionnaires; and the students
completed a single questionnaire after the end of their courses. After providing
background variables, the teachers were asked about assessment and the use of
new technology in general and for assessment in particular. Their responses
were processed using appropriate software the Statistical Package for the Social
Sciences (SPSS). All students also completed their questionnaire.
The interview schedule revisited questions used in the questionnaires. Its purpose was to deepen the information gathered with the questionnaires. The
teacher interviews were carried out partly before start of course and partly after
end of course. Transcripts of the interviews were returned to the teachers for
comment, and a draft of the final report was also sent, for comment, to the
internet provider and the software supplier. The students were interviewed by
telephone. All of these interviews were also transcribed, and examined for themes that seemed relevant to the aims of the project.

Outcomes
This was a small-scale study in an area that has not been widely-researched in
Sweden. The prime outcome of the study was that it raised as many questions
as it solved. Nevertheless, it became clear, for instance, that:
1. Students favoured net-based course-work;
2. On-line assessment technologies used in the study had feedback potential which was not fully exploited.
3. Teachers (perhaps like software developers) found it difficult to distinguish
between testing and assessment.
4. Testing and assessment are cultural practices.

5. Assessment practices raise questions about what, in a particular culture,

counts as knowledge or knowing.
The opportunity to revisit these questions arose when the European Commission office in Stockholm suggested that the work be extended through the
Minerva Action of the EU Socrates programme, a development initiative which
sought to combine ODL (Open and Distance Learning) with ICT (information and communication technology/ies).

Project 2: the EU project (2002 2004)

(final report available a http://www.onlineassessment.nu )

Aims and activities

This development project built, as suggested, on the findings of the original
Distum project. By this time, too, the project proposal was built on the view
that formative assessment can be at the centre of dialogic pedagogic practice. It
asked the question: how can assessment be built into teaching and, especially,
such teaching that takes place among adults in colleges, universities and other
institutions moving from face to face to on-line learning?
Overall, the project had a developmental purpose. Its task was to bridge the
gap between the everyday practices of teachers, the ideals that are associated
with on-line (or distance) learning, and the assumptions that animate so-called
alternative or authentic assessment. Thus, the developmental task of the EU
project was not to identify, recycle and re-affirm the intrinsic features or
abstract merits of internet-based assessment. Rather, it was to detail practices
and problems in three European institutions making the effort to introduce
internet-based assessment.
In common with other universities, Ume University constantly monitors good
practice in teaching and learning and, in the process, seeks to improve the
learning opportunities offered to its students. The 3-year EU project shared
this view. Its development activities were based on two innovations: (1) software
that enables teachers to develop their own tests; and (2) assessment theory that
prioritises supporting learning over measuring learning.
Its goals were to:
1. develop material, methods and tools for internet-based assessment
2. train teachers in assessment theory and internet-based test construction
and test administration.

3. support teachers in the development and implementation of internetbased assessment.

4. develop a web portal with a resource centre to support interested teachers,
and
5. generate new knowledge in this area.

Software, in-service training and support

The Questionmark Perception software utilised in this project, as in the previous
development project, consisted of the following modules:

Question Manager
Session Manager
Perception Server (server based)
Security Manager
Enterprise Reporter
Addition functions, not utilised in the study (e.g. Secure Browser,
registration System and Monitoring)

Question Manager was used to create questions, with the help of a Wizard (a
programmed guide). This module also allowed the creation of questions of the
following type, taken from version 3 of the software manual (Questionmark,
2001):
Drag and drop
Essay
Explanation
Fill in blanks
Matching (of texts)
Matrix (near-matching)
Multiple-choice
Numeric
Pull-down list
Ranking
Select a blank
Text match
This module also allowed teachers to decide, in advance, how questions should
be marked and what kind of feedback should be offered to students. Moreover,
the feedback could also be adjusted to the students specific answer. Further,
teachers could choose whether feedback should be given after each question or
after the respondent had answered all the questions. Finally the teachers could
pre-test each question to see if it worked satisfactorily.

Session Manager was used to assemble the questions into an assessment. Moreover, questions could be randomized from a data-base of questions creating
so-called adaptive tests. Session Manager could also be used variously to decide the maximum time allowed for the assessment, to select the form and layout of the feedback, and to provide students with their overall assessment.
The Perception Server organised distribution and logging-in. Login could be
organised as open login or it could require names and passwords. These log-in
functions could also be linked to other databases, providing access, for instance,
to student records. Thus, the on-line assessment system was, at least in principle,
compatible with other on-line systems used to manage teachers, students and
courses.
Using Security Manager, teachers specified which groups and students would
have access to their assessments. Teachers could password-protect the assessments and schedule them so that students only had access to them at specified
times. It was also possible for the teachers to import names of students from
external files and also, if the teachers so wished, generate automatic passwords
that eased the administration of the assessments. The teachers could also decide whether students could have access to the assessments on more than one
occasion.
In Enterprise Reporter, teachers generated different reports based on different
break-downs of the raw data. Thus, reports could use tables and diagrams to
focus on overall performance, subsets of questions, group results, and so on. As
noted earlier, these reports could also be made accessible to students. Finally,
results from interim assessments could also be amalgamated into a summative
result for a whole course.
Here are some of the variations that could be reported using Enterprise reporter.
Many of them are derived, however, from testing theory rather than assessment theory:
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.

Date and time of the assessment

Start-, stop- and total time
Minimum-, means- and maximum time
Task type
Text answers
Response frequency
Standard deviations
Correlations
Degree of difficulty
Percent points
Number of completed items

12. Item Minimums, means and maximums.

13. Computer name and IP-address
To acquire competence in the software, project teachers were given in-service
training workshops and seminars. The workshops consisted of skill training in
the software; while the seminars consisted, among other things, of discussions
of pedagogy, test theory and question construction. An important concern of
the seminars was to inform the teachers about the aim of the study and to raise
their awareness of conceptions like validity, reliability, and formative assessment. Overall, the seminars had the purpose of helping teachers to appreciate
the difference between testing as the measurement of learning, and assessment
as support for learning. In this last respect, of course, feedback plays a crucial
role.
Besides the workshops and the seminars, teachers were offered technical support via e-mail or phone; and software support was available from the supplier
during office hours, also by e-mail or phone. A list-serve was also created for
the teachers, something that enabled a measure of peer tutoring. Participants
were also encouraged to contact the internet provider and software supplier. In
addition, project teachers had access to the regular support offered across the
university and within their own departments. Students in need of support could
turn to their teachers or to the university support centre.

Selection of courses and participants

While the entire project involved teachers and students from three European
universities, the following discussion is restricted to developmental work
conducted at Ume University for which I was responsible. Project activities
started in 2002 when all teachers in the university were invited to participate in
a meeting which provided information about this development initiative.
Teachers who opted to participate took part in ten workshops that offered the
training described above.
One hundred and four teachers initially participated in one or more than one
occasion. The teachers planned to use internet-based assessment in 25 courses
with 108 separate assessments. In the end, only 10 teachers reached the stage
where they adopted internet-based assessment, using it for 47 different course
assessments. All but one teacher had more than five years of teaching experience;
the age of the teachers ranged from 30 to just over 60; and they comprised
seven lecturers, two senior lecturers and one professor. Thus the group covered
the range of university teaching staff.
Eight hundred and eight students also participated in the project, with their
ages distributed: 21 24 (61%), 25 35 (27%) and >35 (12%). Sixty percent
of the students were male. The faculty distribution of these students was: Medi-

cine (51%), Arts and social science (29%), Science and technology (13%) and
Teacher education (8%).

Data collection and analysis

Data were collected by means of web-based questionnaires and telephone
interviews. The questionnaires were designed to establish how participating
teachers and students felt about this form of assessment. After they had completed
their assessments, teachers and students were invited to fill in a questionnaire on
assessment in general and internet-based assessment in particular. All of the ten
teachers six men and four women who had conducted assessments returned
the questionnaire. They were drawn from the faculties of Arts and social science
(4), Medicine (3), Science and technology (2) and Teacher education (1). A
voluntary questionnaire was also distributed on-line to the students after they
had finished their assessment. Only 20% of the students (160) completed the
survey. The low response rate arose from the voluntary status of the questionnaire
and technical problems associated with retrieving the completed questionnaires.
A conspicuous feature of the project was the contrast between the large initial
interest among teachers and the drop-out encountered when active participation
was encouraged. This was particularly noticeable after the first workshop. Of
the 30 participants in the workshop, only three wanted to be actively involved.
Unstructured telephone interviews were used to clarify this phenomenon. In
February 2003, 19 of the workshop participants were interviewed. It became
apparent that many teachers were curious about internet-based assessment but
had been reluctant to commit themselves to further development work.

Outcomes
The outcomes of the EU project have already been summarised in the final
report submitted to the European Commission (EC). The following is an edited version of the insights gained:
a. Despite the pressure on teachers that have arisen from efficiency gains
engineered in higher/distance education, a core of teachers still exists
who are prepared to reflect on what they are doing and, to a lesser degree,
willing to revise their pedagogic practices.
b. Although the EC and other agencies would like best practice to be
encapsulated in templates or guidelines (eEurope 2005: an information
society for all, 2002, p. 18), national and local variations make this a
difficult task. Indeed, the Bologna process may encounter the same
harmonisation problem when contextual factors are taken into account.
c. One contextual factor, not only evident in Sweden, relates to the legacy
of the university reforms described above. With up to 20 years experience
of recurring waves of reform and associated variants of policy hysteria

d.

e.

f.

g.

(Stronach & Morris, 1994), teachers in higher education have grown

project-wise and project-weary. They take targeted money and adapt it
to other purposes; or more commonly they reject it because they cannot
justify the parallel investment of their own time and effort in a
development project with a limited life.
In an age where efficiency discourses have a powerful influence on
budgetary decisions, the software and support costs of online learning
and assessment are greater than many university departments are prepared
to venture. The availability of open-source software may, however, lower
this barrier to innovation providing the associated modification costs
are marginal?
There is a deep tension between the learning economy and the learning
society. The former is driven by the production and marketing of
commodities (e.g. learning management systems), while the latter is driven by humanist ideas about knowledge as fostering the cultivation of
human beings (e.g. Bildung and its derivatives).
There is a problematic relationship between on-line and liberal education.
The spread of on-line assessment and teaching may prolong the life of
behaviourist conceptions of learning that, ultimately, are incompatible
with the forms of new learning represented in the ideals of the learning
society. There is a risk, therefore, that the spread of on-line assessment
and teaching will serve to redefine centres of public higher education as
sites of training rather than learning.
Such tensions, however, may be reduced where online assessment is
embedded in a blended learning situation where participants recognise
online and face-to-face interaction as complementary and interdependent.
On-line assessment can be a legitimate tool for teachers and learners
engaged in producing alternative assessment regimes that aim to promote
learning through collaborative dialogue. On-line assessment, that is, can
increase opportunities for interaction when contact between teachers and
students is limited. Above all, internetbased assessment can produce texts
which are durable. Moreover, they can be accessed by students repeatedly
in situations of the own choosing, thus enabling the kind of meta-cognitive
reflection that is consonant with the sociocultural, situationist,
transformational and constructivist goals of the new learning.

Collectively, these outcomes have a variety of implications for the learning

opportunities that can be offered to students in higher and adult
education. Further developmental work could be pursued in the light of the
projects findings. Nevertheless, one developmental opportunity quickly came
with an invitation to extend the thinking of the Ume project to a nation-wide
context.

Project 3: The Net-university project (2004 2005)

(final report available at http://www.myndigheten.netuniversity.se )

Aims and activities

Assessment worries both teachers and students. One source of recent worries is
that the context of teaching may have changed (e.g. in favour of the new learning)
while examinations and tests have stayed the same. Indeed, it could be argued
that the multiple choice tests used in national testing programmes (e.g. Hgskoleprovet) are merely linear descendents of testing practices that took their
modern form in the catechisms of the 1500s. The worry is that assessment
practices are still rooted in old forms of learning (e.g. behaviourist). They are
still, that is, based on the measurement of learning rather than notions of assessment for learning.
The aim of this project took account of the worries described above. It was
funded to map net-based assessment practices in different institutes of higher
education and, on the basis of that mapping, identify frameworks for future
inter-institutional co-operation. The project group comprises a representative
each from four different institutes of higher education: Ume university, Malm
university, Karolinska institutet and Gotlands university.

Selection of courses and participants

All teachers at over 30 institutions of higher education in Sweden were defined
as the target group of this investigation.

Data collection and analysis

Information was gathered via web based questionnaires which sought information on how teachers and students perceive assessment and, at the same time,
which made it possible to identify examples of net-based assessment.
A questionnaire was distributed in the form of an e-mail sent to all rectors
which included a link to the questionnaire site: http://www.onlineassessment.nu.
A link was also created on the homepage of the agency responsible for the Net
university. The questionnaire was divided into four sections (background
variables, attitudes to assessment, net based assessment, and space for
respondents comments). Four hundred and eighty seven teachers answered
the questionnaire less than 2% of the teachers in higher education in Sweden.
Note: Less than 5% of the total responses were obtained from 14 (i.e. 45%) of
Swedish institutions of higher education.

University/College

% of respondents
who replied to the
questionnaire

Ume university
Linkpings university
Lule university
Mitthgskolan
Malm hgskola
Karolinska institutet
Vxj university
Hgskolan Dalarna
Uppsala university
Hgskolan Jnkping
Hgskolan Kalmar
Kungliga Tekniska Hgskolan
Stockholm university
Karlstad university
Lund university
Bleking tekniska hgskola
Hgskolan p Gotland
Hgskolan i Trollhttan/Uddevalla

19
14
9
7
6
6
6
6
5
3
3
3
2
2
1
1
1
1

Institutions with a zero (in fact <1%) response rate were Chalmers tekniska
hgskola, Gteborgs universitet, Handelshgskolan i Stockholm, Sveriges
lantbruksuniversitet, rebro universitet, Mlardalens hgskola, Hgskolan i
Bors, Hgskolan i Gvle, Hgskolan i Halmstad, Hgskolan i Kristianstad,
Hgskolan i Skvde, Idrottshgskolan i Stockholm, Lrarhgskolan i Stockholm och Sdertrns hgskola.
The responses came from the following sectors:

Arts and social science (34 percent),

Medicine (24 percent),
Science and technology (33 percent) and
Teacher education (9 percent).

Outcomes
The results of the questionnaire survey must be treated with caution. The response rate was low; and the distribution of responses from different institutions was uneven. Nevertheless, the responses available for analysis can be
summarised as follows:

1. 49% of the respondents had never used net-based assessment; 9% had

used it earlier; and 42 percent were currently using net based assessment.
2. 57% percent of the respondents had the intention of using net-based
assessment in the future; 37% perhaps; and 6% had no such intention.
3. 26% percent of the respondents had used net-based assessment in a
campus course, 74% percent in a distance course.
4. 57 percent of the respondents were very positive/positive to net-based
assessment, 38% were neutral; 4% negative.
5. 84% percent of the respondents felt that their students were very positive/
positive, 14% felt they were neutral, and 4% negative.
It is difficult to assess the validity of these responses for at least two reasons.
First, the most frequent response was non-response and, secondly, it is generally accepted that non-responses are not random. They, too, must be included
in the overall interpretation of the results. To this extent, the net-university
project had a second set of outcomes.
1. The pattern of responses suggests that the issue of internet-based assessment is not a hot potato in Swedish higher education, neither in the
minds of institutional gatekeepers (who, in this study, chose not to
circulate the questionnaire) nor the daily lives of individual members of
staff (who felt no wish to respond).
2. Not only are teachers in higher education project weary (as noted in the
EU project) they also may be weary of completing questionnaires.
3. Informal discussion among the steering group suggests that, in fact, many
of examples of best practice (to use the EU formulation) are either
examples of assessment as an add-on to teaching (e.g. the on-line distribution of assignments) or they are examples of behaviourist testing (e.g.
multiple choice tests with limited feedback).
4. There is a risk that the desire for change among teachers in higher education
is confused with a desire for labour saving technologies. This leads to
increases in efficiency but no change in their practice the phenomenon,
that is, of innovation without change.
5. There is still space for an evaluation of the pedagogic implications of the
new learning in the learning society1.
1

Recent work suggests that the cautionary tone of these comments may be an under-statement
rather than an over-statement. A more provocative commentary on eLearning can be found in a
seven-page policy paper prepared by the On-line Distance Learning (ODL) Liaison Committee
of the European Distance Education Network (2004). The EDEN paper contrasts the vision
and reality of eLearning, echoing the experiences of the Ume projects. It notes, for instance,
that very low quality and simplistic promotional messages and simplified visions and overoptimistic statements provided space for critics to build their case against eLearning; and it
suggests that use of the term blended learning can be a one-size-fits-all panacea concept.
Nevertheless, the EDEN document concludes, like the projects discussed in this thesis, that use
of ICT in learning activities cannot remain the exception in the learning society and that,
therefore, a new vision is needed, one that distinguishes between innovative on the one hand
and merely substitutive use of ICT in different learning contexts, on the other. Evidence of EU
perspectives can be found in the briefing papers prepared for the European Commission (2003).

PREPARATION OF PAPERS
Paper 1: From Distance education to on-line learning
Presented at the European Distance and E-learning Network 10th Anniversary
Conference, Stockholm, June 2001.

I wrote the first two papers as ground-clearing exercises. I began my doctoral

studies with an interest in finding web-based assessment tools. It soon became
apparent that tools are artefacts which attract additional questions: who will use
them? How will they be used? And, what purposes should be served by such
tools? Thus the context of tool-use became a focus of my research. This took me
in two directions: (a) to the current state of higher (and distance) education in
Sweden; and (b) the international literature on examinations and assessment.
I soon realised that the expansion of Swedish higher education has changed
working patterns in universities. Put most simply, there was a return to teaching
students in large classes and large lecture theatres, arrangements that, presumably,
limited discussion and dialogue. At the same time, great international
expectations were claimed for testing and assessment using information and
communication technologies. The marketing of these technologies had become
big business in the slipstream of the IT boom.
I realised, however, that it would be impossible to reproduce the work of multinational testing agencies within the framework of a one-person Swedish doctoral
programme. Accordingly, I turned my attention to a more attractive aspect of
testing the emergent literature on alternative or authentic assessment and the
possibilities that it offered for supporting learning.
I wrote this paper for the conference of the European Distance Education
Network, an organisation whose work is closely linked to the European Commission. My thoughts were speculative. With the knowledge I had gained in
the meantime, I was able to highlight the potential of on-line assessment, even
if I did not fully appreciate the implications of the relationship between teachers
and the technologies that they use. In effect, I argued that there was a need for
further clarification and experimentation in this area. In short, on-line assessment had possibilities; but could they be realised?
This paper also reports the work of the first project described above the
Distum Project. It provided some of the preliminary findings, notably that it
is important that the new technology is made amenable and practical for
interested teachers, and that although online assessment has affected students
learning there remained a need to analyse qualitative changes in students
learning.

Paper 2: ICT, Assessment and the Learning Society

Presented at the European Conference on Educational Research, Lisbon, September 2002.

I wrote this paper early in the life of the EU project. My key insights are included
in the title and the statement: This paper focuses on pedagogy the first
sentence of the abstract. The combination of pedagogy, assessment and
learning society symbolises the difference between assessment as a pedagogic
practice and testing as a measurement practice. Thus discussion of assessment
can, at least in principle, focus on the improvement of teaching and learning.
Indeed, this has been the key claim of writings in the field of alternative or
authentic assessment.
Working on the EU project and writing this paper for the European Conference
on Educational Research deepened my understanding of the difference between
formative and summative assessment, high stakes and low stakes testing and
divergent and convergent assessment. Yet, I was also aware that such dualisms
may overlap and be difficult to sustain in practice. If tools are put into the
hands of teachers, is it possible to ensure that they will use them in the manner
summative or formative intended by the manufacturer of the tools? Moreover, will teachers have the time, energy, inclination and institutional support
to follow the intentions of soft-ware manufacturers or the desires of the European Commission? At this point I became aware of how the work of teachers
may be caught in the tension between the efficiency discourses of the New
Public Management and the quality discourses associated with fostering the
growth of human capital in the learning society. Indeed, this tension accounts
for the last sentence of the paper: is it possible to replace the assessment or
audit society with the learning society?

Paper 3: Examination och det lrande samhllet

Journal of Research in Teacher Education. Vol. 12, No. 1, pp. 155175. English
translation as an appendix to this thesis.

This article builds on the earlier papers. I wrote it in 2002, as I felt that I had
begun to have something that might be of wider interest in Sweden. I also
realised that I had given insufficient attention to a crucial question that the
culture of examinations in Sweden takes a particular form. I had learned this
while taking a course in at the Ume university Enhet fr Pedagogiska mtningar (unit for pedagogical measurements) and in my discussions with the
tutors and the members of the class. For instance, it is difficult to find Swedish
words to distinguish between testing and assessment. Thus, I felt I had to loop
my argument and to go back to a deeper level, not only to issues in assessment
theory raised by protagonists for alternative assessment, but also to issues related

to knowledge and learning of the kind pursued by Anna Sfard and Jean Lave.
Knowledge is not merely delivered it is consciously acquired (Sfards word)
within a community of practice (a concept used by Lave). Indeed, this process
of acquisition can be described in terms of conscious activities of grasping (att
fatta in Swedish).
In terms of conscious activities, a key term in the paper is feedback and my
subsequent work was influenced by Black and Wiliams comment: because of
its centrality in formative assessment, it is important to explore and clarify the
concept [of feedback]. Thus, feedback is more than a concept used in learning
theory, it is also a social practice another key idea in the paper. How, for
instance, should it be organised in teaching? And, as important, how can feedback be seen as a conscious rather than a reactive activity? And, in what sense is
feedback part of the conversations or dialogues that are an everyday element in
the pedagogy of higher education? All these questions took me further into the
fields of cognition, cognitivism and, ultimately, constructivism.

Paper 4: When performance is the Product

British Educational Research Journal, Vol. 30, No. 6, December 2004, pp. 841
854.

This paper was originally drafted, in 2003, for the conference of the Nordic
Association for Pedagogik Research (Copenhagen). Its focus was the analysis of
on-line conversation (i.e. dialogue). It arose from a problem raised in the EU
project the emphasis that the auditing practices of the European Commission places up the development of deliverables. The EU project, run from Ume,
had taken a different position: not only did it follow the European requirement
that it should be developmental but it also chose to develop teachers and learners,
rather than tests and testing protocols. Accordingly, the project team was not
in a position to package re-formed teachers and students in its final report.
Rather, its aspiration became discerning, clarifying and delivering the insights
that had been gleaned from its development experience. To this extent, the EU
project could link with another project in the Ume university department of
education: Folkbildning p distans en samtalsmilj fr lrande (Distance liberal
adult education: a conference environment for learning). This parallel project had
similar problems. Both projects faced the problem of how to characterise processes as products; and all the authors contributed to the drafting of the paper.
The conference version of the paper started with an introduction that seemed
relevant to a Nordic conference:
We began preparing this paper in December 2002. The deadline for proposals had
past; Christmas was approaching; and the outside temperature in Ume was below

freezing. December, therefore, is a good time to defrost the domestic freezer and
repack its contents. This paper serves an analogous purpose. It unpacks educational
ideas that, following the Swedish historian of ideas, Sven-Erik Liedman, we
characterise as frozen educational ideologies.
A frozen ideology unconsciously influences those who work in universities, through
the way that groundrules are established, judgments are made, teaching is enacted
and research is organised. (Liedman, 1997, p. 216, our translation)

The intention of the paper was, therefore, to defrost ideologies that are prevalent
in ICT, distance education and the European Commission. Such ideologies
contain assumptions about the separation of means from ends (or processes from
products), and about learning merely being a process of knowledge acquisition.
Another key idea in the paper is given in the question posed in the abstract: does
a conversation take place within an environment, or by means of the environment?
Writing this paper continued the deepening process started in earlier papers. It
required much reading and much attention to linking ideas from fields as diverse as economic history (technology as a labor saving device), literary theory
(utterances and dialogue) , philosophy (knowledge and doing), information
theory (the content and the meaning of a message), and developmental
psychology (performance art asactivity in context). The preparation of the
paper was a collective effort of the five authors who provided the dialogic
environment for the preparation of the conference version and, after comments
from the editors of the British Journal of Educational Research, for the completion
of the published version.
The net result of this trial and review process was that a more complete account
(or defence) could be given for the work of the EU (and Folkbildning) projects.
That is, both projects were able to fill a gap in their work. The folkbildning
project was able to conceptualise group conversations (using Sfard, Bakhtin,
and Lotman); while the EU project was able to come to terms with the European Commissions unwitting conflation of product development with practice
development (through freezing practices as products). Finally, the problem of
temperature and culture also resulted in a visible, if trivial, change between the
conference version and the published versions of the paper. At the suggestion
of the British journal editors, the Nordic introduction was replaced by an AngloSaxon vignette also an instance where the message of the paper was
strengthened by means of the environment.

Paper 5: Formative Assessment: A cybernetic viewpoint

Assessment in Education, Vol. 12, No. 1, March 2005, pp. 720.

This paper has its origins in a conference paper presented at the 2004 conference
of the Nordic Educational Research Association (Reykjavik). It arose not only
from a wish to have a better understanding of the concept of feedback, but also
from a wish to identify and understand the integrity of formative assessment.
The feedback problem had already been identified in earlier papers (e.g. by
Shepard); namely, that there is a recurrent confusion between behaviourist and
cognitivist senses of feedback. The initial stimulus to write this paper came
from dissatisfaction with two related aspects of my earlier work. First, the fact
that the connection between formative assessment and cybernetics has not
received the close investigation that it seemed to merit; and secondly, a sense
that the such closer investigation would provide some kind of closure on the
unfinished discussion of feedback and cybernetics that appears in paper four.
Key elements in the preparation of the paper were threefold. First, the seminal
writings of Norbert Wiener are available in Ume university library; secondly,
the EU project had gathered enough material for a red thread to be woven through
the history of formative assessment; and thirdly, one of the key actors, D. Royce
Sadler, generously responded to an email about his part in the history. In a sense,
then, this paper restores an element of the narrative that, at the request of journal
editors, Sadler omitted from his seminal paper, Formative assessment and the
design of instructional systems, that appeared in Instructional Design (1989).
With the help of this material it was possible to obtain the closure that was
missing from the fourth paper. It became apparent, for example, that Wiener
had strong neurological (i.e. cognitivist) interests one of the co-authors Arturo
Rosenblueth, was a physiologist at the Harvard Medical School; that cybernetics
had been a fashionable and influential learning tool in the 1950s (e.g. in the
field of management); that Massachusetts Institute of Technology (MIT) had
been a hothouse of ideas in the extension of the information sciences to
neurology, psychology, linguistics and education; and that MIT was one of the
key sites in North American for the infusion of European ideas about
constructivism2.
In turn, it became possible to link the three concepts included in the title of
this thesis, ICT, formative assessment and the learning society. The learning
society is predicated on the assumption that the development of knowledge is
accompanied by qualitative changes in thinking that, in short, there are meta2

Note: the English translation of Piagets Structuralisme (Paris 1970, p. 7) uses the term selfregulation for the French autorglage. Self-direction might have been a better translation since
it is the form conventionally used in English-language cybernetic theory.

levels of cognition. Further, this view of knowledge probably best known in

the form of Piagetian stages can be linked to classroom practice, including
classroom assessment. Thus, assessment operates within, and by means of,
Vygotskys zone of potential development. Dialogue and feedback serve, among
other things, as a basis for formative assessment, a practice which might also be
called developmental or constructivist assessment. Dialogue and feedback mediate
learning the means by which learners construct (with teachers) or self-construct
(with themselves) higher forms of knowing. As Shepard described it, classroom
assessment can foster forms of knowing that, through deliberation, constitute
true understanding.
In this sense alternative assessment has an integrity of its own. It is not, as
paper 5 makes clear at the outset, an extension of summative assessment.

EPILOGUE
I grew up with five brothers and five sisters on a small farm in Krokvg, a
sparsely populated area in the north of Sweden. In the 1960s, government
policies supporting small farmers changed. Assisted migration to the south of
Sweden became the preferred option the so-called vanful of furniture policy.
My parents refused to participate. Nevertheless, the cows had to be sold and
my farther started working as a carpenter on building sites all over the south of
Sweden while my mother worked at home and later in the home-help service.
I move from home when I was 16 to attend the nearest upper secondary school,
90 km away. After I left school, I trained as a youth recreation leader, leisuretime pedagogue, guidance officer and teacher of children and adults; and,
subsequently, I worked in youth recreation, secondary education, gymnasia
and adult secondary education.
During this time, my outlook changed. I became interested in IT as a support
for teaching. I started to read courses in informatics alongside courses in
education, writing essays around the theme of computer-aided instruction and
distance education. In my spare time, I produced study kits for my own teaching
in computer science which later spread all over the country. In the beginning
of the 1990s, I was offered a post teaching education at Ume University. Since
I had just started to teach computer science in a gymnasium, I declined the
offer. When the offer was repeated the following year, I accepted and began
teaching in both computer science and education. This combination proved to
be fruitful.

The opportunity to merge these interests arose in 1997 when I was offered a
position in the recently-founded Centre for Educational Technology at Ume
university. As a university teacher, I had been confronted with the problem of
finding time for examinations in my courses. During the 1990s the number of
students in my groups increased substantially. I had to book bigger lecture
halls, reducing the opportunity for students to ask questions. I also found it
difficult to provide feedback to individual students, even though the university
regulations required that the basic education should be individualised with the
inclusion of individual testing. Since my teaching time was not increased to
compensate for the increased student enrolment, the quality of my teaching
risked being jeopardised. How should I handle this tension? After much thought,
I came to the idea that I could use the new information technology both to
raise the quality of teaching and to lighten the workload of teachers?
I wrote an application for a doctoral scholarship to the Knowledge and
Competence Foundation (KK-stiftelsen) which proved to be successful. The
focus of my application was examinations and the internet. I felt that the
emergence of the internet increasingly made it possible for tools to be standardised, distributed and added to the teachers desktop.
Between 1999 and 2005, I have combined my doctoral scholarship with
participation in the projects described above. These opportunities gave me the
space to extend my horizons, deepen my understanding and develop my research skills. Together, my doctoral and project work have fostered my
educational journey. The journey, however, did not prove to be straightforward.
There were many false starts and pauses as I found my way across the map of
knowledge. I began with a simple road plan; but I finished up travelling across
a complex landscape. Nevertheless, my journey was constructive and positive.
It provided me with a firmer foundation for any future work. Equally, the
journey was also constructive in an intellectual sense: I reached new levels of
understanding about examinations and the internet.
Now that my doctoral journey is coming to a close, what about the landscape
that lies ahead? What features can I discern? And what roads lead to the horizon? To button up this kappa, then, I would like to summarise the truths that
have emerged from my studies the ideas that I carry forward into the next
stage of my professional life.
1. The landscape of learning is complex. Yet it is not complex in itself.
Rather, it is complex because it is difficult to understand.
2. The landscape of learning is also difficult to understand because it is not
easy to appreciate the forms it took in the past, the flows that are
transforming it in the present, and the forms that might emerge in the
future.

3. Refashioning the landscape of learning is not a mechanical process. Success

is rarely guaranteed by the blueprint. Unforeseen external factors typically
intervene to disrupt the innovation process.
4. The landscape of learning also has many names for its features. It is
important to understand the language of learning and the origins of its
dialectic forms.
5. The landscape of learning also supports different forms of life. Students
and pupils do not learn in a vacuum; their learning is evoked, in complex
ways, by the flows that shape the landscape of learning. In short, context
and content are mutually constitutive and, therefore, inseparable.
6. Such differences, like those between medicine and modern languages,
physics and pedagogy are not trivial. They have wide-reaching implications
for the organisation of teaching and learning across all sectors of the
education system.
7. By engaging with the variable landscape of learning, students also begin
to learn differently. They not only acquire new knowledge, they also
acquire new ways of knowing.
8. The landscape of learning is as much a human as a natural artefact. It has
been shaped according to human designs by the tools and hands of human beings. Such designs are as much a part of the tool, as the tool is a
part of the teacher. They cannot be considered apart.
9. The landscape of learning, like the physical landscape, is constantly
changing. But are these changes superficial? Are they the result of ideas
and tools that merely till the surface of the learning landscape?
I began my doctoral work with a question about the teachers desktop, actual or
virtual. I have used add-in and add-on, words taken from the language of
information scientists to ask the question: can assessment be used as an integral
support for learning? And, in the process, I have reached a better understanding of the landscape of learning. The fact that my truths culminate in a new
version of my original question should not be seen as a closed return loop.
Rather, I hope this specific truth can be understood, constructively, as the result
of six years of formative experience.

References
Amrein, A. L., & Berliner, D. C. (2002). High-stakes testing, uncertainty, and student learning.
Educational Policy Anlysis Archives, 10(18). Available at www.epaa.asu.edu/epaa/v10n18.
Aviram, R., & Tami, D. (2004). The Impact of ICT on education: the three opposed paradigms,
the lacking discourse. Unpublished manuscript, Beer-Sheva University, Israel.
Bakhtin, M. (1994). The problem of the text in linguistics, philology and the human sciences:
an experiment in philosophical analysis, in: Speech genres and other late essays (V. W. McGee,
Trans.) (Austin, TX, University of Texas Press), 103131.

Bakker, S. (2001). Editorial: Examinations and entry to university: pressure and change in a
mass system. Assessment in Education, 8 (3), 285290.
Bangert-Drowns, R. L., Kulick, J. A. & Morgan, M. T. (1991). The instructional effect of
feedback in test-like events, Review of Educational Research, 61, 213238.
Bauman, Z. (2003). Educations challenge in the liquid modern era. In Bron, A., &
Schemmann, M. (Eds.). (2003). Knowledge Society, Information Society and Adult Education
(pp. 2344). Munster: LIT Verlag.
Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education:
Principles, Policy & Practice, 5(1), 774.
Bourdieu, P. (1988). Homo Academicus (P. Collier, Trans.). Oxford: Polity Press.
Bourdieu, P., & Passeron, J-C. (1977). Reproduction in Education, Culture and Society (R.
Nice, Trans.). London: Sage.
Broadfoot, P. (2002). Editorial: Assessment for Lifelong Learning: challenges and choices.
Assessment in Education: Principles, Policy & Practice, 8(3), 5-7.
Bruner, J. S. (1975). The ontogenesis of speech acts. Journal of Child Language, 2, 1-40.
Bruner, J. S. (1983). In Search of Mind: Essays in Autobiography. New York: Harper & Row.
Bruner, J., Goodnow, J., & Austin, J. (1956). A Study of Thinking. New York: Wiley.
Burbules, N., & Bruce, B. C. (2001). Theory and research on teaching as dialogue. In V.
Richardson (Ed.), Handbook of Research on Teaching (4th ed., pp. 1102-1121). Washington,
DC: American Educational Research Association.
Callahan, R. C. (1962). Education and the Cult of Efficiency: A study of the social forces that
have shaped the administration of the public schools. Chicago: University of Chicago Press.
Castells, M. (1999). Flows, Networks, and identities: A critical theory of the informational
society In Castells, M., Flecha, R., Freire, P., Giroux, H., Macedo, D., & Willis, P. (Eds).
Critical Education in the New Information Age (pp. 37-64). Lanham, MA: Rowan and
Littlefield.
Cuban, L. (2001). Oversold and Underused: Computers in classrooms. Cambridge, MA: Harvard
University Press.
Daniels, H. (2001). Vygotsky and Pedagogy. London: RoutledgeFalmer.
Delandshere, G. (2002). Assessment as Inquiry, Teachers College Record Volume 104, Number
7, October 2002, pp. 14611484.
European Commission (2002) eEurope 2005: an information society for all (an action plan
to bepresented in view of the Sevilla European Council, 21/22 June 2002). Available online
at: www://u.int/information_society/eeurope/news_library/documents/eeurope2005/
eeurope2005_en.pdf (accessed 10 December 2002).
European Commission, (2003). European Union-supported Educational Research, 1995
2003: Briefing papers for policy makers. Luxembourg: Office for Official Publications of
the European Communities.
European Distance Education Network (EDEN), (2004, November 17th). Distance Learning
and eLearning in European Policy and practice: The vision and the reality. Budapest:
European Distance Education Network (available at www.odl-liaison.org/
pages.php?PN=policy-paper_2004).

Gibbons, M., Limoges, C., Nowotny, H., Schwartzman, S., Scott, P., Trow, M. (1994). The
New Production of Knowledge: The dynamics of science and research in contemporary societies.
London: Sage.
Glasersfeld, E. v. (1979). Cybernetics, Experience, and the Concept of Self. In Mark N. Ozer
(Ed.), A Cybernetic Approach to the Assessment of Children: Toward a More Humane
Use of Humane Beings. Boulder, CO: Westview Press.
Glasersfeld, E. v. (1995). Radical Constructivism: A way of knowing and learning. London:
Falmer Press.
Griffiths, M. (2001). Social justice for education: What kind of theory is needed. The School
Field, 12(1/2), 2541.
Hamilton, D. (1990). Learning about Education: An unfinished curriculum. Buckingham:
Open University Press.
Hamilton, L. (2003). Assessment as a policy tool. In R. E. Floden (Ed.), Review of Research in
Education (Vol. 47, pp. 2568). Washington, DC: American Educational Research
Association.
Hgskoleverket. (1998). Fortsatt granskning och bedmning av kvalitetsarbetet vid universitet
och hgskolor: utgngspunkt samt angrepps- och tillvgagngsstt for Hgskoleverkets
bedmningsarbete (rapportserie 1998: 21 R). Stockholm: Hgskoleverket.
Hgskoleverket. (2000). r grundutbildningens kvalitet i farozonen (arbetsrapport 2000:12
AR). Stockholm: Hgskoleverket.
Hgskoleverket. (2004). Universitet & Hgskolor, Hgskoleverkets rsrapport 2004 (rapportserie
2004: 16 R). Stockholm: Hgskoleverket.
Husn, T. (1974). The Learning Society. London: Methuen.
Jochems, W., Merrinboer, J. v., & Koper, R. (Eds.). (2004). Integrated E-learning: implications
for pedagogy, technology and organisation. New York: Routledge/Falmer.
Johansson, K. (1999). Konstruktivism i distansutbildning Studerandes uppfattning om
konstruktivistiskt lrande. Doktorsavhandling, Pedagogiska institutionen, Ume universitet.
Karier, C. J., Violas, P., & Spring, J. (Eds.). (1973). Roots of Crisis: American education in the
twentieth century. Chicago: Rand McNally.
Kliebard, H. M. (1995). The Struggle for the American Curriculum 18931958 (2nd ed.).
New York: Routledge.
Kock de, A., Sleegers, P., & Voeten, M. J. M. (2004). New learning and the classification of
learning environments in secondary education. Review of Educational Research, 74(2), 141
170.
Lane, J-E (1984). Higher Education Regionalization, Higher Education, 13 347368.
Laurillard, D. (2002). Rethinking University Teaching: A conversational framework for the effective
use of learning technologies (2nd ed.). London: RoutledgeFalmer. [First edition 1993].
Liedman, S-E. (1997). I Skuggan av Framtiden. Stockholm: Bonnier Alba.
Linn, R. L. (2000). Assessments and accountability. Educational Researcher. 29, 416.
Lotman, Y. M. (1988). Text within a text, Soviet Psychology, 26(3), 3251.
Maclellan, E. (2004). How convincing is alternative assessment for uses in higher education.
Assessment and Evaluation in Higher Education, 29(3), 311321.

Marton, F. & Booth, S. (1997). Learning and Awareness. Mahwah, N J: Lawrence Erlbaum.
Mayr, O. (1970). The Origins of Feedback Control. London: The M.I.T Press.
Mislevy, R. J. (1993). Foundations of a new test theory. In N. Frederiksen, R. J. Mislevy, &
I. I. Bejar (Eds.), Test theory for a new generation of tests. Hillsdale, NJ: Erlbaum.
Mory, E, H. (2002). Feedback research revisited. In D. H. Jonassen (Ed.), Handbook of research
for educational communications and technology (pp. 745783). New York: Simon & Schuster.
Nordkvelle, Y. (2004). Technology and didactics: historical mediations of a relation. Journal
of Curriculum Studies, 36(4), 427444.
Olesen, H. S. (2003). Generating knowledge about learning in the knowledge society. In A.
Bron & M. Schemmann (Eds.), Knowledge Society, Information Society and Adult Education
(pp. 183202). Munster: LIT Verlag.
Olstedt, E. (2003). ICT Burden or Benefit For Education. In A. Bron & M. Schemmann
(Eds.), Knowledge Society, Information Society and Adult Education (pp. 222239). Munster:
LIT Verlag.
Prop. (1975:9). Regeringens proposition om reformering av hgskoleutbildningen mm., Stockholm.
Norstedts Tryckeri.
Questionmark (2001). Questionmark Perception (version 3 of Manual). Hillsdale CT:
Question Mark Corporation.
Riksdagens revisorer. (2000). Grundutbildningens hgskolemssighet, (Rapport 2000/01:1).
Stockholm: Riksdagens revisorer.
Sadler, D. R. (1989). Formative assessment and the design of instructional systems.
Instructional Science, 18, 119144.
Scriven, M. (1967). The methodology of evaluation. In R. Tyler, R. Gagn & M. Scriven
(Eds.), Perspectives of Curriculum Evaluation (AERA Monograph Series on Curriculum
Evaluation No. 1) (pp. 3983). Chicago: Rand McNally.
Sfard, A. (1998). On two metaphors for learning and the dangers of choosing just one,
EducationalResearcher, 27(2), 413.
SFS (1977:218). Hgskolelag, Stockholm: Utbildningsdepartementet.
SFS (2001:1263). Lag om ndring i hgskolelagen(1992:1434), Stockholm:
Utbildningsdepartementet.
Shepard, L. (2000). The role of assessment in a learning culture. Educational Researcher,
29(7), 414.
Shepard, L. A. (2002). The role of classroom assessment in teaching and learning. In V.
Richardson (Ed.), Handbook of Research on Teaching (4th ed., pp.10661101 ). Washington,
DC: American Educational Research Association.
SOU (1992:44). Resurser fr hgskolans grundutbildning. Betnkande av Hgskoleutredningen. Stockholm: Utbildningsdepartementet.
SOU (2004:29). Tre vgar till den ppna hgskolan Betnkande av tilltrdesutredningen
Stockholm: Utbildningsdepartementet.
Stronach, I., & Morris, B. (1994). Polemical notes on educational evaluation in the age of
policy hysteria. Evaluation and Research in Education, 8(1 & 2), 519.

Svetcov, D. (2000, September 11th). The Virtual Classroom vs. The Real One. New York:
Forbes Magazine.
Torrance, H. & Pryor, J. (1998). Investigating Formative Assessment. Buckingham, Philadelphia:
Open University Press.
Utbildningsdepartementet. (2004). Hgre utbildning i utveckling Bolognaprocessen i svensk
belysning (Ds 2004:2). Stockholm: Utbildningsdepartementet.
Wells, G. (1999). Dialogic Inquiry: Towards a socio-cultural practice and theory of education.
Cambridge: Cambridge University Press.
Westling, H. (Ed.). (1999). Brjar grundbulten rosta? En debattskrift om grundutbildningen
i hgskolan. Stockholm: Rdet fr hgskoleutbildning.
Wiliam, D. (2000). An overview of the relationship between assessment and the curriculum.
In D Scott (Ed.) (2000) Assessment and the Curriculum(pp. 165181). Greenwich, CT:
JAI Press.

Paper I

FROM DISTANCE EDUCATION

TO ONLINE LEARNING:
FORMATIVE ASSESSMENT
IN HIGHER EDUCATION 1
Bertil Roos, Ume University, Sweden

Introduction
Higher Education is changing. There are:

changes in staff-student ratios

decreasing student/teacher ratio
changes in student profiles
decreasing economic resources
increasing demand
political aspirations to increase participation
changes in student roles (encouraging flexible learning)
communication media to enable students to learn at a distance or
asynchronously
tools which can take over mechanical tasks and free up time for students
to develop higher level skills
changes from distance education to online learning
debates about summative (high stakes) and formative (low stakes) assessment
In the 1930s and 1940s there were less than one percent of the Swedish population at universities and institutes of higher education. Directly after the Second
World War there were about 14 000 students in the Swedish Universities.
Subsequently, the universities in Sweden has been extended to give further
possibilities for higher education. In the beginning of 1990s there were more
1

Paper presented at the European Distance and E-learning Network 10th Anniversary Conference,
Stockholm, June 2001.

than 200 000 students (Srlin & Trnqvist, 2000). During the 1990s the
number of students in higher education has increased by more than 50 percent. During the academic year 1998/99 there were more than 300 000 students
(Hgskoleverket, 1999), and 50 000 employees (SCB, 1999). In Swedish higher
education instruction should build on a foundation of science and be conducted
by teachers who hold a doctorate. The level of teaching that met this requirement
during 1999 was 55 percent (Hgskoleverket, 1999). During the period 1989/
90 1997/98, however, universities student/teacher ratios increased from 10:1
to 15:1 (Westling m.fl. 1999). Whereas the number of students during the
period increased by 86 percent, the number of teachers/researchers increased
by only 17 percent (Riksdagens revisorer, 2000). The character of the universities
has changed from elite to mass institution during the second half of the 1900s.

Consequences for teaching and learning

Such expansion has changed the working patterns in universities. Students are
taught in lecture halls with up to 250 students in each group, such that the
possibility to ask questions is limited (Brownson, 2000).
Research shows that all learning demand thinking and active construction and
development of mental models.
From todays cognitive perspective, meaningful learning is reflective, constructive,
and self-regulated. People are seen not as mere recorders of factual information
but as creators of their own unique knowledge structures. (Miller, 1999)
It is time to move the emphasis from teaching to learning. (Harvey, 1997)

How can resources be reallocated to allows for learning? Use of e-learning, for
example, is one solution that is widely promoted. It is claimed to be 50 - 90
percent cheaper than using real-life teachers and holding formal classes (Svetcov,
2000). Maise (2000), however, suggests that the key to success is to find the
right mix between online learning and classroom based instruction.
Research shows that there are now significant differences in the ability to learn
with different technological tools. Carnevale (2001), however, claim that the
media do not affect learning. The design of the instruction and partly what the
students bring with to the instruction situation that affect learning. Online
education does not differ significantly from traditional classroom-based
education in respect of results and student satisfaction (Vachris, 1997) (Jones,
1997). Harvey (1997) claims that the Web must be brought to life in an
environment of cooperative learning. In online courses students wish to have
frequent feedback and interaction, particularly if they feel cut off from both
teacher and classmates (Hoey, 1998). People tend to lose interest if theres nobody

on the other side who cares if Im here or not, (Svetcov, 2000). The best thing with
online courses is that the students can work when they wish at their own pace
and where they wish (Hoey, 1998).

Much attention give to ICT

Great expectations are tied to the new information and communication technology
(ICT). Computers were, as late as 1985, used only by a little elite for word processing and simple calculation. Fifteen years later more than half of the Swedish
people have access to the Internet (Srlin & Trnqvist, 2000). In November
2000 more than four million person in Sweden between 12 to 79 year old surfed
on the internet. 93 percent of the men in the group surfed some times during the
month (Computer Sweden, 2000). Today students and teachers can communicate
over the Internet with cheap cameras and microphones which gives interaction
to every Internet connected computer (Blotzer, 2000). With access to video- and
sound streaming the Internet will be the obvious choice within areas where there
is access to computers (Dunn, 2000).

New possibilities (ICT), new alternatives

The transformation of higher education is becoming visible. One off the most
important attributes is that the boundary between time and space is being
eliminated. With asynchrony distribution on the Internet the same course can
be taken in South Africa, Australia or in China. Students can receive credits
from different parts of the world and send them to their homeuniversity
(Dunn, 2000).
Students will be able to shop around, taking a course from any institution that
offers a good one. Degree-granting institutions will have to accommodate this.
Students will learn what they want to learn rather than what some faculty
committee decided was the best political compromise. (Svetcov, 2000)

Many of todays universities, especially in the United States, will be examining

and accrediting universities. The Association of Governing Boards in the United States predict that one third of the existing independent schools and
universities will close within the nearest 10 years. Yet, as the numbers of
traditional universities decline, the number of those who offer higher education
will increase. Publishers, companies, and commercial and non-commercial
organizations will share the education trade. These actors will sell courses direct
to the students and eliminate the universities as a middleman. The future virtual
universities will not be an individual institution, but a gateway for an education
organizer who will collectively distribute customised education to students in a
time and place. It is claimed that this will be the dominant form of higher
education in 2025 (Dunn, 2000).

Supporting learning rather than selection

The increasing numbers of students makes it difficult for teachers to mark and
return individual feedback to the students. The use of formative online assessment, however, allows students to test out their knowledge and get immediate
feedback. However there is a danger that the students look at the result and the
feedback as a confirmation of their adequate understanding than like a way to
discover areas of their weakness (Judge, 1999).
How can we use information technology to transfer learning? Teachers can use
systems that provide them with tools for analysing and tracking students
responses. Teachers can use it for helping students with problems or identifying
questions that are more or less bad. The strength with this tool is that it can
promote students learning.
There is an ongoing discussion about learning, just as there are competing
theories of learning. Methods of assessment are also determined by our beliefs
about learning. In this paper we suggest that assessment can support learning
as well as measure learning through internet based processes. An important
purpose of using online assessment is the possibility of giving students immediate
feedback on their understanding of course material (Judge, 1999). It is important
that the student feedback is of a high quality to enhance the learning process.
The students need not only feedback on how well they have done but also on
what they havent understood and help to improve their understanding (Ramsden, 1992).
The use of online assessment has the advantage of enabling student responses
to be marked and analysed with relative ease and speed. Properly designed
online assessment allows students to test their knowledge of a topic and get
immediate feedback. Important questions remain about how and whether
students organize, structure, and use this information in context to solve complex
problems (Miller, 1999).

Low versus high-stakes assessment

When assessment is used for high-stake purposes, it must meet high standards
of reliability and validity. When tests are used for low-stakes assessment, the
teacher considers many pieces of information when making decisions about
students, testing standards can be a bit more relaxed (What is a test, 1997).
High-stakes testing programs frequently result in improved test score, but such
improvement does not necessarily imply a rise in the quality of education or a
better educated student population (Moss, 1992). Test security is also an
important issue in all high-stake assessment. This is not a problem in low-stake
assessment. Low-stake online assessment can be offered to the students at the
place and time that is convenient for them. Students can be assessed at their

desks or from their portables over a mobile phone line. It is not necessary to
drag students into the classroom just to assess them. Less time and money
wasted on travel to the assessment centre (Kleeman, 1998).
Are online students as qualified as campus students? Online students have
showed they are equal successful (Smeaton, 1999) ore more successful measured
in terms of examination results (Redding, 2000). Other studies shows that
online students could perform at least as good as traditional students (Dutton,
1999). Another study has shown that the performance of students examined in
Internet based items was statistical significant higher then in the campusbased
items (Fredda, 2000). Jon Losak, vice president in research and planning at
Nova Southeastern University at Fort Lauderdale has come to the same result.
During the sex last years have his institution carried out more than 24 studies
comparing online students performance with their classroom counterparts. They
analyzed among other things frequency of graduation, time to graduation and
knowledge acquisition. The students performed as well or better in online courses
(Caudron, 2001). Thomas Russell, at North Carolina State University, has
catalogued 365 studies addressing this specific question. He concludes that
there were no significant difference in academic outcomes between the two
groups (Russell, 2000).

Need for exploration of these new possibilities and contexts

The new information and communication technology offer possibilities for
new learning and assessment mode. One of the most important advantages
with online assessment is the function as an aid for the students learning. The
students can be offered assessment independent time and place and the teacher
can integrate multimedia, simulation and graphics into the assessment. Another advantage with online assessment is saving in time and costs. In times
with an increasing number off students and decreasing resources for teacher
jobs means quicker amending and immediate feedback a better economical
resource exploiting and that the teacher can mind more off the time for tutoring the students. Therefore it is important that the new technology is made
feasible for interested teachers. However the question must be asked how this
form of assessment affects learning and the quality of the education. Research
about effects and use of online assessment assists the development and
implementation of new forms of assessment. An interaction between pedagogical
research and practical development of online assessment is therefore important.
The general aim of the project discussed in this paper was to explore the
application of these ideas to teaching and learning in higher education.

The Project
The project has been implemented with support from the Swedish Agency for
Distance Education (Distum) and involves teachers and students from Department of Radiation Sciences at Ume University. In common with other
departments, Department of Radiation Science are constantly monitoring good
practice in teaching and learning, and seeking innovative ways to improve the
learning opportunities offered to students. It was in this climate it was decided
to pilot online assessment during four weeks of a distance course in Medical
Technology. The course aim was to introduce Medical Technology through an
outline of different approaches and technical aids within medical diagnostics,
treatment and evaluation. The name of the online section was Bio potentials
which treated origins and transmission in humans, measurement of bio potentials and their medical benefits. The course target group was 20 online students
of nursing/care in the Medical Technology field.

Activities
Training of the teachers comprised four seminars about question construction
validity/reliability, and different assessment methods. Software training was
covered in a one day of workshop.
The student assessment partly comprised individual-assessment and partly selfassessment. The individual-assessment assignment was to produce a poster to
be presented on the web. The self-assessment used an online question database
that the students accessed over a period of two weeks.
Early in the project a pre-questionnaire was filled in by the teachers in the
course to gather teachers view of assessment in generally and online assessment. Since questionnaires can be limited, a structured pre-interview was held
with the teachers. After the assessment period, and at the end of the course, a
post-questionnaire was circulated and post-interviews were conducted.
To get the student view of using online assessment a questionnaire was
distributed. The response rate was 85 percent. The data was collected between
December 2000 to April 2001.

Outcome
The teachers view was that:
online assessment was positive and that they would do it again
online assessment was time-saving
online assessment afford an ample statistical analyses of the assessment
questions
online assessment empowered generating of a couple of reports for analyse of the students performance
online assessment could be to significantly help in learning and rehearsal

 online assessment make it possible for teachers/tutors to afford more feedback than it would be possible with paper based assessment
online assessment increase the pace of result feedback and comments to
the students
an important advantage with online assessment is that the students can
work at their own rate and as often they need
The Student view was that:
online assessment gave possibilities for quick feedback
online assessment was a good support for learning
a big advantage with online assessment was that they was allowed to take
the time they needed for the assessment
online assessment had good access
they saw many advantages with using internet in assessment
they in the future would prefer online assessment over paper based assessment
that they would be delighted to attend more courses with online assessment

Reflections
One of the most important advantages with online assessment is its function as
an aid for the students learning. Another advantage is that the student can be
offered assessment independent of time and place and that online assessment is
time and cost saving. Therefore it is important that the new technology is
made amenable and practical for interested teachers.
The first part in the assessment process, and the hardest, is to create the questions.
It is important to think carefully about what it is students are required to know,
and what common misconceptions there might be. The content validity in the
assessment is very important and the teachers in the course were successful in
that matter. The overall goal for the assessment was to enhance students learning.
Most of the students also thought that they learnt a lot during the assessment.
Another important matter is what level of knowledge the assessment demand.
An important goal for teachers is to enhance students higher level thinking.
Therefore it is important that teachers have and take time to construct statements that demand students higher level thinking.
The most important thing for teacher is to have time to problemize the assessment concept and discuss with colleagues question construction, validity and
reliability. Therefore, the project started with some discussion seminars where
teachers could reflect together. Next the teachers were trained in the software
package.

The biggest problem in the project however was not to train the teachers in
handling the system but gaining access to the server. The university computer
central had problems to get the server run.
All students in the course considered that the best thing with online assessment
was that they could take the time they needed to realise the assessment. We
have all different learning styles and it take different time for us to learn.
Therefore it is important that the students can take the time they need to
realize the assessment. The students would, if they could choose, have frequent
assessments rather than a single assessment at the end of the course. That is a
challenge for the teachers to meet the students wishes in that way.
It is apparent that the students view is that formative online assessment affected
their learning and that it attracts them to that degree that hardly any students
wanted to go back to paper and pencil. It is apparent that the students view is
that online assessment could increase flexible learning.
The innovation had an impact. The character of this impact can be seen in the
students claim that they prefer online assessment before pencil and paper. What
other effects the innovate had on the students styles of learning cannot be
established from this research. In short, the introduction of online assessment
has affected students learning, but analysis of qualitative changes in students
learning must remain the focus of another study.

References
1.

Blotzer M J (2000) Distance learning. Occupational Hazards; Cleveland,

Mars, 2000

2.

Brownson K & Harriman R L (2000) Distance education in the Twentyfirst Century, Hospital Materiel Management Quarterly; Rockville, Aspen
Publishers, Inc. Nov 2000, vol 22, no. 2, pps 64-72

3.

Carnevale C (2001) What Matters in Judging Distance Teaching? Not How

Much Its Like a Classroom Course, The Chronicle of Higher Education,
February 21 An interview with Barbara B. Lockee, Assistant Professor,
Instructional Technology, Virginia Tech

4.

Caudron S (2001) Evaluating e-degrees. Workforce; Costa Mesa; Feb 2001

5.

Computer Sweden (2000) Fyra miljoner svenskar surfar, Computer Sweden

2000-12-13

6.

Dunn S L (2000) The virtualizing of education. The Futurist, 34(2), pps

34-38

7.

Dutton J Dutton M & Perry J (1999) Do Online Students Perform As Well

As Traditional Students? Submitted for publication, North Carolina State
University

8.

Fredda J V (2000-08) Comparison of Selected Student Outcomes for Internetand Campus-based Instruction at the Fischler Graduate School of Education
and Human Services. Nova Southeastern University

9.

Harvey L (1997) Transforming higher education: Students as key stakeholders.

Quality assurance as support for processes of innovation. Hgskoleverkets
skriftserie 1997:1 S

10. Hoey J (1998) Rising Tide or Tsunami. NC State University. http://

www.ncsu.edu/planning.html
11. Hgskoleverket (1999) Hgskoleverkets rsrapport fr Universitet och
Hgskolor 1998, Hgskoleverkets rapportserie 1999:11 R
12. Jones G R (1997) Cyber schools: An Education Renaissance, Jones Digital
Century, Inc pp. 58 & 128
13. Judge G (1999) The production and use of on-line Web quizzes for economics,
Computers in Higher Education Economics Review, the Virtual Edition,
vol 13, Issue 1
14. Kleeman J (1998) Now is the time to computerize pen and paper tests! A
white paper by John Kleeman MA MBCS C.Eng Managing Director and
Founder of Question Mark Computing Ltd. http://www.qmark.com/
company/1998paper.html
15. Maise E (2000) Balance between classroom and E-learning is vital to success,
Computer Reseller News; Manhasset; Jan 31
16. Miller K (1999) Which assessment type should be encouraged in professional
degree courses - continuous, project-based or final examination-based? In K.
Martin, N. Stanley and N. Davison (Eds), Teaching in the Disciplines/
Learning in Context, 278-281. Proceedings of the 8th Annual Teaching
Learning Forum, The University of Western Australia, February 1999.
Perth: UWA
17. Moss P A & Herter R J (1992) Assessment, Accountability, and Authority in
Urban Schools. http://www.longterm.mslaw.edu/Long%20Term%20View/
Vol%201%20No.%204/mossherter.PDF
18. Ramsden P (1992) Learning to Teach in Higher Education, London:
Routledge
19. Redding T R & Rotzien J (2000) Comparative Analysis of SDL Online
Training with Traditional Classroom Instruction, Presented at the 14th
International Symposium on Self-Directed Learning, http://
teleeducation.nb.ca/nosignificantdifference/
20. Riksdagens revisorer (2000) Grundutbildningens hgskolemssighet ,
Rapport 2000/01:1
21. Russell T (1999) The No Significant Difference Phenomenon, North Carolina
State University

22. SCB (1999) Higher education Employees in Higher Education 1999,

Statistiska meddelanden, UF 23 SM 0001
23. Smeaton A & Keogh G (1999) An Analysis of the Use of Virtual Delivery of
Undergraduate Lectures, Computers and Education - Vol. 32 pp. 83-94
24. Svetcov D (2000) The Virtual Classroom Vs. The Real One, Forbes; New
York; Sep 11, 2000
25. Srlin S & Trnqvist G (2000) Kunskap fr vlstnd. Universiteten och
omvandlingen av Sverige. SNS Frlag, Stockholm
26. Vachris M A (1997) Teaching Economics in a Virtual Classroom, Virginia
Economic Journal - September 1997
27. What is a test (1997) The Center for the Study of Testing, Evaluation, and
Educational Policy, http://wwwcsteep.bc.edu/CTESTWEB/whatistest/
whatistest.html
28. Westling m.fl. (1999) Brjar grundbulten rosta? En debattskrift om
grundutbildningen i hgskolan Rdet fr hgskoleutbildning 1999

Paper II

ICT, ASSESSMENT AND THE

LEARNING SOCIETY 1
Bertil Roos, Ume University, Sweden

Abstract
This paper focuses on pedagogy, and the improvement of teaching, learning
and assessment. It analyses recent developments in test theory, namely the
attention given to assessment as a support for learning by Wiliam, Torrance,
Black and Linn. In the process, it draws attention to the difference between
formative and summative assessment, high stakes and low stakes testing and
divergent and convergent assessment. The paper has three sections: (1) assessment in the recent reform of higher education in Sweden; (2) recent literature
on teaching and assessment; and (3) testing, assessment and examinations as
social practices to promote rather than control learning. Overall the paper explore
possibilities for introducing new ideas about testing into the culture of Swedish higher education.

Assessment and the recent reform of higher education in Sweden.

In the 1930s and 1940s less than one percent of an age cohort studied at
universities and institutes of higher education in Sweden, compared with about
40 percent at the end of the millenium (Srlin & Trnqvist, 2000, p. 34). This
difference meant that there were 11 000 students in the beginning of 1940 and
over 300 000 by the end of the century (Askling, 1997, p. 36).
In the 1990s universities student-staff ratios increased from 10:1 to 15:1 (Westling, 1999). The number of students during the period increased by 86 percent, whereas the number of staff increased by only 17 percent (Riksdagens
revisorer, 2000). Such changes in student-staff ratios have changed the working
patterns in universities. Students have problems getting feedback from the
teachers and Universities report growing problem of stress and fatigue among
staff.
1

Paper presented to the ICT network at the European Conference on Educational Research (ECER),
Lisbon, 11-14 September, 2002.

Some authors claim that there is a need for a paradigm shift in assessment, a
shift from the current assessment paradigm to the problem-solving paradigm,
a shift from a testing culture to an assessment culture. Further, this shift is
associated with a shift from mental measurement to the assessment of learning
(Gipps, 1994, chapter 9, Black & Wiliam, 1998, p. 45).
These paradigm differences can be demonstrated with three distinctions 1)
formative- and summative assessment, high- and low stakes testing and 3) divergent and convergent assessment.

Formative versus summative assessment

We can think about the function of assessment partly, at a methodological
level, as the goals of the assessment and partly, in a sociological or pedagogical
context, as the roles of assessment (Scriven, 1967). The terms formative and
summative first appeared in Scrivens article The Methodology of Evaluation
(1967). Robert Stake has defined the difference in culinary terms: when the
cook tastes the soup, thats formative evaluation; when the guest tastes it, thats
summative evaluation (quoted from Scriven, 1991).
But how does assessment fit the distinction between formative and summative?
Black & Wiliam (1998) define formative assessment as follows:
all those activities undertaken by teachers, and/or by their students, which provide
information to be used as feedback to modify the teaching and learning activities in
which they are engaged. (p. 2)

If teachers make an effort to develop formative assessment it can take them

closer to the students learning and give the students a more active role in their
own learning (Black, 2001). Further, such assessment can be seen as integral to
learning. Students have to be active in their own assessment and picture their
own learning in the light of an understanding of what it means to get better
(Black & Wiliam, 1998, p. 22).
Black & Wiliam (p. 14) assert that the core activity of formative assessment lies
in the perception, by the learner, of a gap between a desired goal and his or her
present state; and in the action taken by the learner to close that gap in order to
attain the desired goal.
The practices of formative assessment and feedback overlap. Because of its
centrality in formative assessment, it is important to explore and clarify the
concept of feedback (Black & Wiliam, 1998, p. 39).
Ramaprasad (1983) defines feedback in terms of two activities: (1) identification
of a gap between the actual level and the reference level of a system parameter;

and (2) utilization of information which is used to alter the gap in some way
(p. 4). For feedback to exist, the information about the gap must be used to
alter the gap. If the information is not actually used in altering the gap, then
feedback has not taken place (Black & Wiliam, 1998, p. 39).
If the term feedback refers to any information that is provided to the performer,
such performance can be evaluated either in its own terms, or by comparing it
with a reference standard. Adopting the definition proposed by Sadler (1998),
we would argue that the feedback in any assessment serves a formative function
only through diagnosis (what do I need to do to get there?). In other words,
assessment is formative only when comparison of actual and reference levels yields
information which is then used to alter the gap. As Sadler remarks, If the information is simply recorded ... or is too deeply coded (for example, as a summary
grade given by the teacher) to lead to appropriate action, the control loop cannot
be closed (p. 121). The assessment might be formative in purpose but it would
not be formative in function. This suggests a basis for distinguishing formative
and summative functions of assessment (Black & Wiliam, 1998, p. 45). Summative
assessment generates coded information; formative assessment utilises referenceor standard-based information.

Low versus high-stakes assessment

Assessment is one of the most potent forces influencing what teacher should
concentrate on in their teaching and what students should concentrate in their
learning. Assessment sends a message to students about what is important to
learn.
High stakes assessment entails a close association between assessment outcomes
and social outcomes. Summative assessment is high stakes if it has important
consequences. The stakes are high because the consequences of failure are
considerable. Typically, high stakes assessment affects the life chances of students.
Moreover, as Amrein and Berliner point out, high stakes assessment is troubled
by this close association of assessment outcomes and social outcomes: the more
important that any quantitative social indicator becomes in social decisionmaking, the more likely it will be to distort and corrupt the social process it is
intended to monitor (2002, p. 1).
On the other hand low-stakes assessment does not have the same implications.
Indeed, its social consequences may be positive, when it is used to guide teaching
and learning, rather than to make possibly irreversible decisions about different educational journeys that students can take in the future.
There is a further problem with high stakes assessment. It tends to inflate
students measured achievement because teachers play the high stakes game

and teach to the test. Such improvement, however, does not necessarily imply
a corresponding rise in the quality of education, or a better educated student
population (Moss, 1992). Through this corruption of teaching and learning,
high-stakes assessment stresses basic skills and a narrowing of the curriculum.
One result of this negative evaluation of high-stakes assessment is that it has
started a wider discussion about assessment. Among other things, it has increased
calls for moving classroom assessment closer to students and their learning
(e.g. Linn, 1998, Shepard, 2000).
To enhance the positive impact of assessment and minimize its negative effects,
Linn (1998) suggest that it is a necessity that have a variety of ways to assess
student. The teachers cannot rely on a single high-stake test when they judge
the students. It is important to use multiple indicators when judging the
students. The key to long term success is to create a culture that accentuates the
intended positive effects and reduces the negative effects of the assessments
that are used.

Divergent versus convergent assessment

Convergent and divergent assessment arise from teachers differing views of
learning and the relationship of assessment to the process of intervening to
support learning (Torrance & Pryor, 1998, p. 153).
The key issue in convergent assessment is to find out whether the student has
a predetermined specific kind of knowledge, understanding or skills. It focuses
on students knowledge, understanding and skills in proportion to the curriculum. It uses tick-lists and performance (or performance) statements. It prefers
pseudo-open questioning and focuses on contrasting error responses and correct
responses. Assessment with these characteristics can be described as
behaviouristic, it assesses in a linear way and the assessment is made of the
student and executed by the teacher (Torrance & Pryor, 1998).
Divergent assessment, on the other hand, has students understanding in focus.
Divergent assessment has an aim to find out what a student know or can do.
The assessment is performed by the teacher and the student together. It is
characterize by flexible planning, open forms of recording, emphasis on the
learners understanding, open tasks, open questioning and descriptive, qualitative
feedback. Divergent assessment strives towards teaching in the zone of proximal
development. The theoretical inference from this is that divergent assessment
reflects a social constructivist view of education (Torrance & Pryor, 1998).

Testing, assessment and examinations as social practices

Assessment, is far from being merely a technical process. Rather, it is deeply
implicated, and may have serious consequences for the lives of those it touches
(Johnston et al., 1995, p. 359 in Black & Wiliam, 1998, p. 12).
This point is clearly made by Messick:
Once it is denied that the intended goals of the proposed test use are the sole basis
for judging worth, the value of the testing must depend on the total set of effects it
achieves, whether intended or not. (Messick, 1989, p. 85)

Linn has made a similar point, arising from his work on high stakes assessment:
As someone who has spent his entire career doing research, writing, and thinking
about educational testing and assessment issues, I would like to conclude by
summarising a compelling case showing that the major uses of tests for student and
school accountability during the last 50 years have improved education and student
learning in dramatic ways. Unfortunately, this is not my conclusion. Instead, I am
led to conclude that in most cases the instruments and technology have not been up
to the demands that have been placed upon them by high stakes accountability.
Assessment systems that are useful monitors lose much of their dependability and
credibility for that purpose when high stakes are attached to them. The unintended
negative affects of the high stakes accountability uses often outweigh the intended
positive effects. (Linn, R. L. 2000, p. 14)

Assessment processes are, at heart, social processes, taking place in social settings,
conducted by, on and for social actors. There are (largely implicit) expectations
and agreements negotiated between students and teachers. A feature of such
contracts is that they serve to delimit legitimate educational activity by the
teacher. For example, in a classroom where the teachers questioning has always
been restricted to lower-order skills, such as the production of correct
procedures, students may well see questions about understanding or
application as unfair, illegitimate or even meaningless (Schoenfeld, in Black
& Wiliam, 1998, p. 47). Thus, all testing has to take account of these social
phenomena in the design and administration of its instruments.

ICT and Assessment in the future

The increasing numbers of students in Sweden makes it difficult for teachers to
provide individual feedback to the students. Formative online assessment,
however be used by students to test out their knowledge and get immediate
feedback. Can we use information technology to support teachers and learners
with this problems? Teachers can use systems that provide them with tools for

analysing and tracking students responses. Teachers can help students with
problems. The use of online assessment has the advantage of enabling student
responses to be marked and analysed with relative ease and speed. Properly
designed online assessment allows students to test their knowledge of a topic
and get immediate feedback. Important questions remain, however, about how
and whether students organize, structure, and use this information in context
to solve more complex problems (Miller, 1999).
It is important that student feedback is of a high enough quality to enhance the
learning process. Students need not only feedback on how well they have done
but also on what they havent understood. They also need help to improve their
understanding (Ramsden, 1992).
Much attention has been given to ICT as a solution to this problem surrounding
teaching and learning in Swedish universities. As late as 1985 computers were
used only by a small elite for word processing and simple calculation. Fifteen
years later more than 50 percent of the Swedish people have access to the Internet
at home or at work. In turn, new forms of on-line assessment have been proposed
as a solution to the problem described above. They are being investigated in a EU
project coordinated for Ume university (see http://www.onlineassessment.nu ).
This paper has pointed to problems in Swedish Higher Education. Sweden is
committed to the Learning Society and the extension of access to education
and knowledge. Historically assessment have been used to separate successful
and unsuccessful students. The question is whether assessment can undergo a
paradigm shift. Can assessment be used to support learning for all students.
International research on assessment, discussed in this paper, has raised the
same question. In short, is it possible to replace the assessment or audit society
(Power, 1999) with the learning society?

References
Askling, B. (1997). Comments on Traditional and Modern Concepts of Institutional. In
Kvalitet- och frbttringsarbete vid universitet och hgskolor, Hgskoleverkets skriftserie 1997:4
S. Stockholm.
Amrein, A. L. & Berliner, D. C. (2002). High-stakes testing, uncertainty, and student learning.
Education Policy Analysis Archives, 10, (18). Retrieved August 5, 2002, from http://
epaa.asu.edu/epaa/v10n18/
Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education:
Principles, Policy & Practice, 5 (1), 774.
Black, P. (2001). Dreams, strategies and systems: Portraits of assessment past, present and
future. Assessment in Education: Principles, Policy & Practice, 8 (1), 6585.
Linn, R. L (1998). Assessments and accountability. CSE Technical Report 490. CREEST/
University of Colorado at Boulder.

Linn, R. L. (2000). Assessments and accountability. Educational Researcher. 29, 416. Retrieved
from (http://www.aera.net/pubs/er/pdf/vol29_02/9403AERA004_016a.pdf )
Messick, S. (1989). Validity. In R. LINN (ed) Educational Measurement (3rd edition, 13
103) (New York, Macmillan).
Miller, K. (1999). Which assessment type should be encouraged in professional degree courses
continuous, project-based or final examination-based? In K. Martin, N. Stanley and N. Davison
(Eds), Teaching in the Disciplines/ Learning in Context, 278281. Proceedings of the 8th
Annual Teaching Learning Forum, The University of Western Australia, February 1999.
Perth: UWA. Retrieved from http://cleo.murdoch.edu.au/asu/pubs/tlf/tlf99/km/miller.html
Moss, P. A. & Herter, R. J. (1992). Assessment, Accountability, and Authority in Urban Schools.
Retrieved from http://www.longterm.mslaw.edu/Long%20Term%20View/
Vol%201%20No.%204/mossherter.PDF
Power, M. (1999). The Audit Society: Rituals of verification. Oxford: Clarendon Press.
Ramsden, P. (1992). Learning to teach in higher education. London: Routledge.
Riksdagens revisorer. (2000). Grundutbildningens hgskolemssighet, Rapport 2000/01:1.
Stockholm.
Sadler, D. R. (1998). Formative assessment: Revisiting the territory. Assessment in Education,
5(1), 7784.
Scriven, M. (1967). The Methodology of Evaluation in Perspectives of Curriculum Evaluation.
Chicago: Rand McNally & Company.
Scriven, M. (1991). Beyond Formative and Summative Evaluation. In Evaluation and
Education: At Quarter Century, pp 1964. The National Society for the Study of Education.
Shepard, L. A. (2000). The Role of Assessment in a Learning Culture. Presented at the annual
meeting of the American Educational Research Association, New Orleans, April 26, 2000.
Retrieved from http://www.aera.net/meeting/am2000/wrap/presadr1.doc
Srlin, S. & Trnqvist, G. (2000). Kunskap fr vlstnd. Universiteten och omvandlingen av
Sverige. SNS Frlag, Stockholm.
Torrance, H. & Pryor, J. (1998). Investigating Formative Assessment. Buckingham, Philadelphia:
Open University Press.
Westling, H. (Ed.). (1999). Brjar grundbulten rosta? En debattskrift om grundutbildningen
i hgskolan. Stockholm: Rdet fr hgskoleutbildning.

ISBN: 91-7305-828-9; ISSN: 0281-6768

I CT and formative assessment in the learning society

Akademiska avhandlingar vid pedagogiska institutionen

UME UNIVERSITET
Nr 77 2005

I C T A N D F O R M AT I V E AS S E S S M E NT
IN THE LEARNING SOCIETY

B E RT I L R OO S

Ume universitet
SE-901 87 Ume, Sweden
www.umu.se

BERTIL ROOS