Wrap Thesis Dixon 2013
Wrap Thesis Dixon 2013
Wrap Thesis Dixon 2013
uk/wrap
http://go.warwick.ac.uk/wrap/58287
David Dixon
March 2011
TABLE OF CONTENTS
I
3.6 Other researchers ......................................................................................... 98
3.7 The research design ................................................................................... 101
3.7.1 The plan for the research .................................................................... 102
3.7.2 Smaller scale data collection .............................................................. 105
3.7.3 Sampling procedures .......................................................................... 106
3.7.4 Subject groups .................................................................................... 111
3.8 Reliability and validity .............................................................................. 111
3.8.1 Reliability ........................................................................................... 111
3.8.2 Validity............................................................................................... 113
3.9 Item selection............................................................................................. 114
3.9.1 The construct of autonomy used in the present research ................... 115
3.10 Questionnaire development ................................................................... 118
3.10.1 Putting items into a draft questionnaire ............................................. 119
3.10.2 The parts of a questionnaire ............................................................... 123
3.11 Administering the questionnaire ............................................................ 125
3.11.1 Groups or individuals ......................................................................... 125
3.11.2 Medium .............................................................................................. 126
3.11.3 Gathering feedback ............................................................................ 127
3.12 Data analysis .......................................................................................... 128
3.12.1 Standardisation ................................................................................... 129
3.12.2 Coding of responses ........................................................................... 130
3.12.3 Factor analysis .................................................................................... 131
3.13 Translation ............................................................................................. 141
3.13.1 Introduction ........................................................................................ 141
3.13.2 Reasons for translating ....................................................................... 142
3.13.3 Translation equivalence ..................................................................... 142
3.13.4 The choice of translation procedure ................................................... 144
3.13.5 Cultural differences in questionnaire response .................................. 149
3.14 Overview of eventual research stages .................................................... 151
3.15 Summary ................................................................................................ 152
4 THE LONG LIST STAGE ...................................................... 155
4.1 Overview ................................................................................................... 155
4.2 Writing items ............................................................................................. 155
4.2.1 Item wording guidelines..................................................................... 155
4.2.2 The choice of closed items ................................................................. 162
4.2.3 Items in the Long List ........................................................................ 166
4.3 Subject groups ........................................................................................... 167
4.3.1 The UAE data gathering..................................................................... 167
4.3.2 English language Internet data gathering ........................................... 171
4.3.3 Mandarin Chinese Internet data gathering ......................................... 173
4.3.4 The BNU Mandarin Chinese data gathering ...................................... 174
4.4 Chapter summary....................................................................................... 176
5 THE SHORT LIST STAGE .................................................... 177
5.1 Item selection process for Short List ......................................................... 178
5.1.1 Respondents‘ feedback....................................................................... 179
5.1.2 Range of response .............................................................................. 180
5.1.3 Standard deviation.............................................................................. 181
5.1.4 Polarisation of response ..................................................................... 181
5.1.5 Selection table .................................................................................... 182
5.1.6 Discrimination indexes....................................................................... 182
II
5.1.7 Combined selection procedure ........................................................... 184
5.1.8 Items in the Short List ........................................................................ 187
5.2 Small scale research .................................................................................. 187
5.2.1 Estimating autonomy levels ............................................................... 194
5.2.2 Student interview ............................................................................... 196
5.2.3 Teacher interview ............................................................................... 197
5.3 ELTCS year 3 data gathering .................................................................... 198
5.3.1 Description ......................................................................................... 198
5.4 Presessional ELTCS group data gathering ................................................ 199
5.5 Other Warwick presessional students data gathering ................................ 200
5.6 Taiwan group data gathering ..................................................................... 201
5.7 Ethics ......................................................................................................... 202
5.8 Limitations................................................................................................. 204
5.9 Conclusion ................................................................................................. 205
6 DATA ANALYSIS ................................................................... 206
6.1 Introduction ............................................................................................... 206
6.2 Larger scale data ........................................................................................ 207
6.2.1 Introduction ........................................................................................ 207
6.2.2 Factor analysis procedure ................................................................... 207
6.2.3 Groupings emerging from the factor analysis process ....................... 220
6.2.4 Comparison with literature................................................................. 223
6.2.5 Comparison with Short List categories .............................................. 226
6.2.6 Instrument performance indications ................................................... 229
6.2.7 Summary results for the full sample .................................................. 232
6.3 Smaller scale data ...................................................................................... 234
6.3.1 Introduction ........................................................................................ 234
6.3.2 Correlating teacher estimates with the questionnaire......................... 234
6.3.3 ELTCS year-3 teacher interview and confidence-related categories . 241
6.3.4 Movement of teacher estimates.......................................................... 244
6.3.5 Two students ...................................................................................... 245
6.4 Validity and reliability............................................................................... 265
6.5 Formative benefits of the questionnaire .................................................... 267
6.6 Conclusion ................................................................................................. 268
7 DISCUSSION ........................................................................ 270
7.1 Introduction ............................................................................................... 270
7.2 Teacher estimates ...................................................................................... 270
7.3 The questionnaire‘s construct .................................................................... 272
7.3.1 Introduction ........................................................................................ 272
7.3.2 Technical factors ................................................................................ 273
7.3.3 Psychological Factors......................................................................... 277
7.3.4 A Model from the questionnaire ........................................................ 284
7.3.5 Autonomy and motivation.................................................................. 289
7.4 The questionnaire and its model‘s relation to themes in the autonomy
literature ............................................................................................................... 291
7.4.1 Autonomy and responsibility ............................................................. 292
7.4.2 Autonomy as capacity and behaviour ................................................ 292
7.4.3 Autonomy and social interaction ....................................................... 293
7.4.4 Autonomy is variable ......................................................................... 294
7.4.5 Self-evaluation in self-report autonomy questionnaires .................... 297
7.4.6 Summary ............................................................................................ 300
III
7.5 What the questionnaire is measuring ......................................................... 300
7.6 Suitable uses for the questionnaire ............................................................ 304
7.7 Translating questionnaires ......................................................................... 308
7.8 Conclusion ................................................................................................. 311
8 CONCLUSIONS .................................................................... 313
8.1 Introduction ............................................................................................... 313
8.2 Answers to research questions................................................................... 313
8.2.1 Research question 1............................................................................ 313
8.2.2 Research question 2............................................................................ 315
8.3 Contributions to knowledge ...................................................................... 317
8.3.1 In the area of language learner autonomy theory ............................... 317
8.3.2 Innovative approach to researching autonomy .................................. 319
8.3.3 Translation of questionnaires ............................................................. 323
8.4 Limitations................................................................................................. 324
8.5 Implications ............................................................................................... 325
8.6 Further research ......................................................................................... 327
8.7 Concluding remarks................................................................................... 329
9 REFERENCES ...................................................................... 332
10 APPENDICES .................................................................... 350
10.1 Full 256 items with back translation ...................................................... 350
10.2 Long List areas covered ......................................................................... 358
10.3 Long List format C with translation ...................................................... 369
10.4 Selection Table ...................................................................................... 371
10.5 Movement of teacher estimates ............................................................. 382
IV
LIST OF TABLES
Table 2.1: Three-stage model of autonomy (Breen & Mann 1997: 143) 31
Table 2.2: Three-stage model of autonomy (Macaro 1997: 170–172) 32
Table 2.3: Levels of implementation of autonomy (Nunan 1997: 195) 33
Table 2.4: Levels of autonomy (Littlewood 1996: 429-430) 34
Table 2.5: Locus of Control (based on Williams and Burden 1997: 101) 43
Table 2.6: The Aspects of Metacognitive Awareness (Sinclair 1999: 102) 73
Table 2.7: Linguistic evidence for metacognitive awareness (Sinclair 1999: 104) 74
Table 2.8: Frequent concepts in descriptions of autonomy, with references 79
Table 3.1: Top 20 countries of origin for non-UK students at UK higher education institutions,
2008-2009 110
Table 3.2: Countries of origin of respondents 110
Table 3.3: The sample groups involved in the research and the format of items used 112
Table 3.4: Questionnaire development stages 118
Table 3.5: The formats of the instrument used in the research 119
Table 3.6: Comparison of Likert item responses 131
Table 3.7: Overview of the actual stages of research as carried out 151
Table 3.8: Original outline plan for the research and subsequent changes 154
Table 4.1: Comparison of item wording guidelines 158
Table 4.2: The general areas for questionnaire items with thesis sections 167
Table 4.3: Participants in HCT UAE data gathering 168
Table 4.4: Participants in English Language Internet group 171
Table 4.5: Participants in Mandarin language Internet group 173
Table 4.6: Participants in BNU group 176
Table 5.1: Summary of all items queried by respondents 180
Table 5.2: Short List’s coverage of areas of autonomy 191
Table 5.3: Comparison of coverage of the areas of autonomy in the Long List and Short List 192
Table 5.4: Participants in ELTCS third year group 198
Table 5.5: Participants in ELTCS presessional group 200
Table 5.6: Participants from other presessional groups 201
Table 5.7: Participants in Taiwan group 202
Table 6.1: The 12 runs of factor analysis 208
Table 6.2: Loadings from factor analysis run eight 210
Table 6.3: The rejected items and scales with reasons for exclusion 211
Table 6.4: Final groups for factor analysis run eight with suggested identifications 212
Table 6.5: Combining the 12 factor analyses 216
Table 6.6: Identification of factors 219
Table 6.7: Possible matches of questionnaire factor groupings with Cotterall (1995) 223
Table 6.8: Comparison of input and output groupings 226
Table 6.9: Comparison of items in Short List and factor analysis groupings 229
Table 6.10: Significant correlations of categories to Categories Average in descending order
(standardised pooled data) 230
Table 6.11: Significant item Correlations with Categories Average in all data combined in
descending order of Pearson’s correlation 231
Table 6.12: Item scores (%) in each category (combined standardised data) 233
Table 6.13: Significant correlations between Teacher Estimate and the categories and
Categories Average (ELTCS year-3 and ELTCS presessional P2) 236
Table 6.14: Significant correlations of categories to Categories Average arranged in descending
order (groups ELTCS year-3 and ELTCS presessional P2) 237
Table 6.15: Significant correlations of individual items to Categories Average arranged in
descending order (groups ELTCS year-3 and ELTCS presessional P2) 238
Table 6.16: Significant correlations of items to Teacher Estimate using Spearman’s r 239
Table 6.17: Comparison of ELTCS year-3 students’ questionnaire results with teacher estimates
of autonomy and confidence 242
Table 6.18: Confidence categories in ELTCS year-3 teacher’s interview 243
Table 6.19: Questionnaire results for students T1-St-b and P-St-a 246
Table 6.20: Average questionnaire results for ELTCS year-3 students with T1-St-b’s results 247
V
Table 6.21: Common terms associated with learning English from T1-St-b’s interview, with
frequencies and frequency order 248
Table 6.22: Internal reliability of scales 266
Table 6.23: Respondent feedback from data gatherings 267
Table 7.1: Inter-Category correlations 272
Table 7.2: Possible equivalences between the questionnaire, Fazey & Fazey, and SDT 291
Table 7.3: Classification of translation problems in the Long List with examples 309
Table 10.1: Full 256 items with back translation 357
Table 10.2: The Long List items with areas covered. 368
Table 10.3: Long List Format C with translation 370
Table 10.4: Selection table 381
LIST OF FIGURES
Figure 2.1: A “spectrum” of autonomy ......................................................................................... 22
Figure 2.2 The major influences on language learning autonomy (Benson 2001)........................ 22
Figure 5.1: Process for combining lists ........................................................................................ 186
Figure 7.1: Self-Concept ............................................................................................................... 281
Figure 7.2: Autonomy seen as an electric circuit ......................................................................... 285
Figure 7.3: A model of Questionnaire Autonomy based on the questionnaire category
correlations (N.B. Arrow thicknesses indicate strength of correlation) ............................ 287
Figure 7.4: Littlewood’s (1996: 430) model of the components and domains of autonomy ....... 288
Figure 7.5: Example of the hypothesised dip in student questionnaire results .......................... 298
Figure 10.1: Categories Average scores plotted against Teacher Estimates .............................. 382
Figure 10.2: Social Comparison scores plotted against Teacher Estimates ............................... 383
Figure 10.3: Linguistic Confidence scores plotted against Teacher Estimates .......................... 384
Figure 10.4: Information Literacy scores plotted against Teacher Estimates............................ 385
Figure 10.5: Locus of Control scores plotted against Teacher Estimates ................................... 386
Figure 10.6: Metacognition scores plotted against Teacher Estimates ....................................... 387
Figure 10.7: Self-Reliance scores plotted against Teacher Estimates ......................................... 388
VI
ACKNOWLEDGEMENTS
Many thanks go to all those who have helped in this research, some of whom are
named below.
My supervisors:
Dr Richard Smith
Dr Ema Ushioda
Professor Hilary Nesi
The translators:
Dr Shi Lijing
Xinjie Guo
Wei Yi-Chun (Sherri)
Dr Jiang Xiaoli
Dr Fei-Yu Chuang
VII
DECLARATION
I declare that this thesis represents my own work, except where due
Signed _____________________
David DIXON
VIII
ABSTRACT
The thesis aims to explore the viability of using a quantitative instrument to measure
language learner autonomy and investigate whether such an instrument has a
function in supporting teachers and learners in the development of learner autonomy.
The research developed into a critical reflexive approach which probed the
theoretical and design issues surrounding the development of a quantitative
autonomy-measurement instrument by actually attempting to produce such an
instrument. This approach means that I could experience and examine first-hand the
theoretical and practical issues which the quantified measurement of autonomy
would involve.
The main conclusions of this research were, firstly, that the aim of measuring learner
autonomy needs to be recast in the light of the research which indicated that it is
necessary to understand autonomy as a quality which has only an abstract existence
if it is not instantiated in a context. This means that the aim of producing an
instrument which measures an abstract universal learner autonomy cannot be
achieved. However, such an instrument can be used to monitor learners in autonomy-
relevant areas and can serve a useful purpose in scaffolding the learners in their
environment in order to facilitate the dialogue which enables a teacher to support the
learners better in the development and maintenance of their autonomous learning.
Secondly, teacher estimates of their learners‘ autonomy can be complemented and
assisted by using the data provided by the quantitative instrument developed in this
research.
Another outcome was that the translation of instruments in second language teaching
research is an issue which needs to be given more serious consideration and should
be carried out in a more principled way than it is currently.
IX
ABBREVIATIONS
BA Bachelor of Arts
CA Cronbach‘s alpha
DI Discrimination Index
HE Higher Education
L1 First Language
L2 Second Language
P Per cent
X
P2 Presessional ELTCS group second administration
R Reverse Coded
UK United Kingdom
XI
1 INTRODUCTION
1.1 Background: Initial motivations
The roots of the initial motivation for this research go back to the 1990s when I was
working for Saudi Development and Training (SDT) as the self-access centre (SAC)
SAC so that it could be given targets and its performance monitored. Gains in the
learning skills and abilities of the students would be recorded using a General
National Vocational Qualification (GNVQ) in the key skill area of Improving Own
time for the training of teachers as assessors and for the paperwork required for a
find an alternative method. Later, working at the Fujairah Women's College, part of
the Higher Colleges of Technology (HCT) in the United Arab Emirates (UAE), I was
called upon to justify the independent learning centre by showing a gain in the
autonomy of the learners. There was not an instrument available to do this in the
quantified way that was requested. For such an instrument to be useable at the HCT
suitable for use in existing courses without upheavals, be quick to complete for
learners, and not make unreasonable demands on teacher time. It would need to be
easy to distribute around the widely scattered campuses of the HCT and would also
Benson (2001: 186) states that ―there is surprisingly little empirical evidence
available for the effectiveness of any particular approach‖ and that there is no
practical tool with which to contribute evidence. Perhaps, I thought, the lack of a tool
1
was due to practical difficulties in measuring autonomy or perhaps it was because the
the possibility of designing such an instrument it became clear that it was not an easy
task as there were many issues around the idea of measuring autonomy, and I became
1.2 Direction
instrument was possible, and in the course of the research as my knowledge and
understanding of the issues developed the nature of the investigation became clearer
and the research became more reflexive and critical. The aims developed into a focus
their learners‘ autonomy. This was fruitful in two main ways: it provided a way of
what teachers were already doing (which was estimating their learners‘ autonomy in
purpose of an autonomy measure should be, in sum, I realised that what was really
valuable in the research was not the autonomy measuring aspect but the aspect of
helping teachers to help learners with the development of their autonomous learning.
Difficulties were encountered with finding sufficient subjects to provide the data
necessary for quantitative analysis, but the more qualitative and small scale aspect of
the research proved fruitful. A difficulty also emerged in the area of translation of the
2
exploratory, with practical problems and ―dead ends‖ along the way. I have tried to
developing the ideas to reflect how my own thinking developed. The researcher at
the outset of this project is not the same person as the researcher at the end.
1.3 Overview
This thesis is presented in primarily chronological order, beginning with the concrete
theory and then to attempting to develop the practical measure and analyse its
performance, followed by the interpretation of the evidence, but with review and
I begin by putting this investigation into its wider context with a review of the
issues connected with the idea of autonomy measurement and discusses how other
order to establish the areas which the autonomy-measuring instrument should cover
and this informs the choice of questionnaire items. I also consider criticisms of the
instrument by attempting to design one and examine its validity. In the chapter I also
address the issue of translation. I present an overview of the stages of the research,
both as initially envisaged and in its final form. I present the methods of statistical
3
analysis which would be necessary to probe the construct represented in the
Chapter 4 concerns the first stages in designing the instrument under investigation in
the research, including writing the items to address the areas of autonomy established
in Chapter 2.
Chapter 5 looks at the item reduction process and why it differed from the originally
intended method (factor analysis). This chapter describes the small scale data
collection which took place, and explains how large scale research was not possible
due to delays caused by slow returns and translation problems. The research had to
adapt to these problems, which meant that gathering statistically significant data was
limited and indications from smaller-scale samples were used for illustrative and
qualitative purposes. The data gathering using the shorter list of items is described.
Chapter 6 presents the data and analyses the patterns emerging the comparison of
questionnaire data with teacher estimates and a more detailed examination of two
the teacher estimates (which involved two teachers, including myself) are discussed,
and the construct embodied in the questionnaire is presented and examined. The
progress of the research is discussed and questions about the function of the
questions are answered, possible future uses of the instrument are suggested, and
directions for future research are proposed. The implications of the research are
discussed.
4
The thesis is exploratory, and describes the journey which I took in developing the
developing an instrument, and gathering data to probe the instrument. It also presents
The thesis is worthwhile because it addresses, in innovative and flexible ways using
area which is discussed theoretically but less often are attempts made to establish
of such an instrument it was hoped to be able to explore the problem from an original
perspective which could shed new light on the question in a fresh way. The account
of my own development in the course of this research will I hope be useful for other
autonomy.
5
2 LITERATURE REVIEW: MEASURING LEARNER
AUTONOMY IN ADULT SECOND LANGUAGE
LEARNING
2.1 Introduction
order to establish clearly where the present research fits in the context of previous
thought and research. It is necessary also for defining what the research will and
should aim to do and to clarify the meanings of the concepts which will be used in
measure autonomy;
6
In order to arrive at a conclusion as to whether measuring autonomy is theoretically
possible arguments both for and against will be presented and evaluated in the
The findings of this Chapter will be used in the decisions regarding the design of the
present research) and also in the initial selection of items to populate a preliminary
long list (see Section 4.2.3) of items intended to represent all the relevant elements of
obtain the most important items to retain for an eventual autonomy measurement
instrument. The findings will also inform the discussion of the eventual results of the
present research and the conclusions about the possible uses and limitations of an
In this Chapter I will not include detailed discussion about measuring possible
research into the effectiveness of self-access centres (e.g. Morrison 2005; Reinders
and Lázaro 2007; Gardener and Miller 1999: 205-240). This is because the aim of the
The answers to four primary questions which are essential for shaping the nature and
direction of the present research need to be found in the literature. The questions are:
7
2. Should autonomy be measured (i.e. is it desirable)?
To provide the answers to these questions it will be necessary to consider the views
At the outset it is necessary to clarify how the term measure (and its derivatives:
―measuring‖, ―measurement‖ etc.) will be used in this thesis. A key point which
needs to be emphasised here is that the term ―measure‖ (and its other forms) has been
very deliberately chosen for use in the present research. I make a distinction between
―measure‖ and other related terms such as: ―test‖, ―assess‖, ―judge‖, and ―evaluate‖.
The word ―measure‖ is the most appropriate for the present research because it
expresses the idea of quantification, but with a more neutral connotation than the
other words which suggest determining value, how good or bad something is, or how
quantification tool as distinct from a test. It is hoped that it will have many useful
functions, but testing is assuredly not one of these. As will be seen in Section 2.3
involving formal high stakes testing where the results will be used to grade students.
8
There are objections that testing autonomy is implicitly un-autonomous (for example,
Thein & Walter 2001) because it opposes learner choice, but with the clarification of
the term ―measure‖ distinguishing it from ―test‖, ―evaluate‖, etc. these do not
actually apply. This is a key point for the present research which is focused on the
measurement of autonomy but not with testing or evaluating it. When the distinction
between measurement, evaluation, testing etc. is made it becomes clear that different
In the field of autonomy in language learning both more qualitative (e.g. Dam 2000)
and more quantitative (e.g. Cotterall 1995) techniques have been used to investigate
Testing and measuring are associated with quantitative techniques (Dörnyei 2007:
someone who makes judgements regarding the subject of the evaluation. In the
are quantitative, I do not see testing and measuring as the same. Measuring is the
collecting of quantitative data, but formal testing is specifically the gathering of data
9
for purposes of judgement. In this thesis the focus is on the measurement of
autonomy and therefore testing and its specific issues are not a central concern. To
avoid any confusion this distinction between the terms test and measure will be
The four key questions introduced in Section 2.1.1 above will underlie the Literature
Review. The first two questions regarding the need for a measure and whether it
should be measured are dealt with in Sections 2.1.3 and 2.3. The conceptual issues
regarding whether autonomy can be measured (question 3) are dealt with in Section
2.4 where the differing concepts of autonomy found in the literature will be
examined through discussion of the key concepts found in the different conceptions
of autonomy. This will serve as the basis for the ―Long List‖ of items which will be
Section 3.9). The more practical challenges regarding the question of how to measure
(question 4) are discussed by presenting previous research which has a direct bearing
on the present project (Section 2.5). Here I will attempt to highlight the lessons to be
autonomy measurement, and how autonomy has been measured previously. The
Discussion (Section 2.6) will summarise the issues and attempt a synthesis which
will lead to my stance being stated and the four questions posed above being
answered. Here I will highlight the points which will be important for the present
10
2.2 Is there a need for a measure of autonomy?
In this section I will review the literature and establish that there is an expressed need
for a measure of autonomy which is not being fulfilled by presently available means
(such as the measurement of learning gains). I will show that an autonomy measure
There is a perception that autonomy lacks evidence to support its claims to provide
Dickinson 1987; Sinclair 1999), and detractors (such as Hand 2006). Benson (2001:
54) makes the point that ―If we aim to help learners to become more autonomous, we
should at least have some way of judging whether we have been successful or not‖.
would make a direct measure unnecessary. A major problem with this, however, is
that achievement tests do not measure autonomy directly and therefore it cannot be
assumed that results of tests are not influenced by other influences unconnected with
outcomes but found problems of attribution. He (La Ganza 2002: 47) makes the point
that control groups cannot be used since if strict rules were laid down for the
(2005) also encountered this problem of securely isolating the reasons for learning
11
Some views of autonomy have it as at least partly a disposition of the learner rather
than entirely a product of a specific context (Little 1997; Carr & Claxton 2002). In
this case autonomy would be transferable (to some degree) between settings and
with methods which focus on learning gains alone is that they do not address the
(2010: 78) it would also potentially allow researchers to investigate how autonomy
interacts with different contexts of teaching and learning, and how it is transferred
over time (2010: 78). This is an important area about which there are a number of
differing theoretical models (e.g. Littlewood 1996; Nunan 1997; Breen & Mann
1997) which have great importance for learners and practitioners, but which have yet
Sinclair (1999: 100) makes the point that the lack of a measure or recognised
framework for autonomy means that ―Teachers, course planners and materials
writers are left to do what they think is best, to rely on their own beliefs about
learning, their values, experience and intuition‖. Evidence for the presence of
autonomy and to what degree it is present in individual learners and classes would
help teachers to make better estimates of their learners‘ autonomy. Benson (2001:
51) also states that ―For the purposes of research and the evaluation of practice, it
12
would indeed be convenient if we had a reliable method of measuring degrees of
Rivers & Melvin (1981: 90) maintain that it can be useful to know the ―average‖
learning style for a class to match instructional activities to fit the strengths,
weaknesses or ―bias‖ of the class, and also the profile of the instructor. This will
allow teachers to become aware of their own biases with respect to mode and style of
presentation, and could prevent a drastic mismatch between the instructional method
and the style of the ‗typical‘ student in the class. While I do not intend to propose
that autonomy is a learning style, the principle that knowing the average or bias of a
class can prevent a ―drastic mismatch‖ is an important idea in support of knowing the
autonomy would therefore provide data which could be used for needs analysis,
potentially providing evidence of learner beliefs and learner readiness for self-study
provide different types of data, from a very basic indication of overall level to a
whether there was cause for concern and would, for example, enable a teacher to
identify at-risk learners and devote time to establishing the precise nature of the
would be indicated thus providing useful indications of areas on which the learner
needs to focus. For example, metacognition is vital for autonomy (Flavell 1987;
Victori & Lockart 1995; Wenden 1995; Vickers & Ene 2006) and hence appropriate
13
remedial learner training activities could be initiated. Likewise, if motivation was
revealed to be a problem area the learner would be counselled and possible solutions
so to autonomy (Champagne et al. 2001; Rivers 2001; Reinders 2007). The feedback
autonomy should not be aimed at a rather abstract concept of autonomy but should if
supporting techniques.
A suggested way of evaluating learner autonomy is the learner diary or logbook Dam
(2009: 139). Nunan sees these as providing ―insights into processes of learning
which would be difficult, if not impossible, to obtain in any other way‖ (Nunan
1992: 123). However, logbooks are not always clear, they depend on the level of the
learner, the learner does not automatically cover all the relevant areas, and indeed
may not be able to (Dam 2009: 139). Tsang (2005) found that learners were divided
about how helpful they found logbooks, and Chan, Spratt & Humphreys (2002: 10)
found that 80.8% of students rarely or never write a diary to help their studies. It
necessary to make them compulsory; Dam (2009: 134) says that her students are
expected to use the logbook. Blin (2005: 101-103) has actually used diaries to assess
learners‘ levels of autonomy, though, again it was compulsory and in addition had
the drawback of being a long-term commitment taking 12 weeks, which would make
it impossible to use on shorter courses and would make it impractical for use in
initial needs analyses. There is, then, a need for some other type of measurement tool
14
which can aid evaluation (or measurement). Dam (2000) also uses questionnaires to
contribute to evaluations, and this would be quicker than the logs. It would also be
formative, i.e. the learners will be thinking about areas they may not have considered
before and this will potentially feed back into their reflections and logbook
considerations.
There is, therefore, evidence from the literature that an autonomy measure would be
useful for researching the nature and development of autonomy, for assisting
The previous section has shown that there is evidence of a need for an autonomy
measure, but some feel that autonomy should not be measured. This can be based on
feelings of the type that ―traditional assessment has been a major force in retarding
educational reform‖ according to Reeves & Okey (1996: 192); or that testing
interrupts learning (Zimmerman 1995); or Benson‘s (2010: 78) more vague ―nagging
feeling that this was not perhaps the ‗right‘ way to think about autonomy‖. In this
feels that learners faced with a test of autonomy would try to achieve high grades.
This would clearly be an external motivation rather than the internal one appropriate
1997) rather than true autonomy. This argument is specifically aimed at autonomy
testing, and as such is not relevant to the present research which (as discussed in
15
excludes testing. Another argument is based on autonomy being seen as
fundamentally involving free choice. Baumann (2007) finds that in his situation the
argument assumes, however, that the measurement will be imposed on the learner. It
would be, in effect, to try to measure something while at the same time taking that
very thing away. However, this is not an argument against measuring per se but
rather against a situation where the learner does not have a free choice to have or not
to have the measurement, as might be the case with an institutional test. As discussed
previously (Section 2.1.2 above), the distinction between measure on the one hand
and test/assess/evaluate on the other is crucial to the aims of the present research.
When the measurement is not a test and where the learner has chosen to
Another objection to measuring autonomy is that it is more for the benefit of the
global economy than for the individual learner (Benson 2010). This is attached to a
fear that autonomy is becoming popular not for its intrinsic benefits but for its
perceived benefits to employers who want a flexible workforce with the ability to
learn new skills rather than merely having a fixed body of knowledge (Pemberton
1996: 1). However, as discussed in the previous section (Section 2.1.3 above) there
16
There is also a suspicion that the desire to measure autonomy is motivated not by the
desire for any real benefits this may offer, but because there is an attitude that if
similar vein, Biesta (2009) feels that values must not be lost sight of with ―The rise
measuring autonomy will differ between individuals, but the present research is
motivated by the potential benefits and hence would count as ―measurement of what
we value‖. Potential benefiters of a measure would be, for example: learners wishing
learners‘ autonomy for needs analysis, and researchers wishing to measure autonomy
In conclusion, I would argue that the key to maintaining autonomy while measuring
A key aim of the present research is to explore the question ―Can autonomy be
measured?‖. In order to proceed with this it has been necessary to clarify the term
measure (Section 2.1.2 above), and in this section I will review the literature
concerning the definition of autonomy and extract from this the basis of a rationale
for measurement which provides the elements a measure will have to cover. This
section of the Literature Review provides the areas which will be used to inform the
This section highlights and discusses nine key recurring ideas and themes found in
the language learner autonomy literature. These areas can be seen as the areas which
17
are most commonly found across the spectrum of the autonomy debate and which are
the most important areas for an understanding of the issues current in the field. The
Autonomy requires
metacognition
In the following subsections each of these areas will be discussed in turn (though
despite numerous attempts (for example Holec 1981; Dickinson 1987; Little 1991;
Littlewood 1999), a single agreed definition remains elusive. Authors still write of it
as being a concept which is ―fuzzy‖ (Dam 2000: 59) or ―seemingly abstract‖ (Smith
2008: 395). Problems of definition led La Ganza (2002: 51) to change his research
focus away from quantifying autonomy, and others have found it necessary to
formulate their own definitions to allow their research to proceed (e.g. Cotterall
1995).
(La Ganza 2002: 47-48) which reflect different stances and theoretical camps, and as
18
yet there is no universally established definition, taxonomy, or terminology for
constructivist approaches to learning) there is the view that all successful learning is
by definition autonomous (Benson & Lor 1998: 12), while at the other end of this
spectrum are ideas that autonomy is an unattainable ideal, which prompts Sinclair
(2000: 6) to speak of teachers being ―put off by what seems to them to be a highly
multidimensional (e.g. Hurd 2004; Benson 2001: 51; Blin 2004). Shaw (2008: 188)
writes of a ―conflation of means and goals‖, where he feels that a set of techniques
has become attached to the pursuit of autonomy which has become confused with
autonomy itself and which leads to what he views as a lazy way of speaking, for
example, ―putting learner autonomy into practice‖. Another reason is that there are
many autonomy-related terms and they have been used inconsistently. For example
Holec essentially defines autonomy as ―the ability to take charge of one‘s own
learning‖ (1981: 3) and he sees autonomy as a capacity of the learner, but for
Dickinson (1987: 11) autonomy is ―the situation in which the learner is totally
responsible for all of the decisions concerned with his learning and the
of autonomy, but this still leaves room for ―a good deal of interpretation‖ (Martinez
2008: 105).
Definitions of autonomy can vary because they reflect the author‘s stance on
attempted to identify the different stances or types of autonomy with labels such as:
19
psychological, technical, sociocultural, political/ideological, liberal progressive,
humanistic, behavioural, economical, etc. Crabbe (1993: 443) for example finds in
autonomy:
Political – learners taking control of learning (Benson 1997; Pemberton et al. 1996)
Humanistic self-initiation or self-direction (Rogers 1961; Kenny 1993; Savage & Storer
1992)
Benson (1997: 25) found three basic versions of definition: the technical version
(where autonomy is the act of learning on one‘s own and the technical ability to do
so); the psychological version (where autonomy is the psychological capacity to self-
direct) and; the political version (where autonomy is control over the content and
Oxford (2003) sees the situation as ―far from coherent‖ and as ―beset by conflicting
response to this she attempts to organise and even integrate the different types of
definition by seeing them not as conflicting definitions but as perspectives which can
coexist. The four classifications she finds are: the technical perspective (the physical
20
In Benson‘s technical variety there is the view that skills, techniques, and strategies
(which can be taught) can be used to pursue autonomy. Oxford (2003) has a similar
on the physical situation being one in which the reins of power have been handed
the learner where motivation, learning styles, and learning strategies are normally
(Bandura 1997) and attribution theory (Weiner 1979). Oxford does not include in this
prevailing political and social ideologies, with issues of power relationships and
rights (see Section 2.4.9). There are strong connections with cultural issues such as
and beliefs about the nature of autonomy may need to be changed in the light of
differing views in different cultural contexts. Oxford (2003) differs from Benson by
distinguishing and giving a separate section to what she calls the ―Sociocultural
Perspective‖, where she places ideas, largely influenced by Vygotsky, that autonomy
21
Political Social Liberal Humanistic Psychological Technical Behavioural Economic
<< >>
In this spectrum political and economic are presented as being on the margins and
humanistic and psychological as being the core ideas, but other ―colours‖ could
equally well be placed more centrally. The positioning of the ―colours‖ will naturally
Gremmo and Riley (1995: 152) stress the eclectic origins of autonomy, and Benson
(2001: 22) gives five major sources of theory which have influenced the thinking on
Figure 2.2 The major influences on language learning autonomy (Benson 2001)
between autonomy and self-directed learning; autonomy is the capacity goal, the
being on the way to having, autonomy (Pemberton 1996: 2-4). However, this
distinction is not uniformly observed in the literature. If there are different versions
of autonomy, what does it mean to use the term ―autonomy‖ as if there were one
unquestioned universal moral good and Shaw (2008: 188) says it is a goal of
education; however, to take only one small example, Holliday (2003) sees autonomy
22
as a pre-existing social phenomenon. It seems that autonomy is just the pedestal on to
which can be placed a large number of different sculptures. Lamb & Reinders (2006:
(2008: 4) believes that it is necessary to stop searching for ―monolithic and stable
answers‖.
Benson (2001: 44) writes that ―People value personal autonomy for its own sake, and
for this reason, it is not simply instrumental in the achievement of well-being, but an
aspect of well-being deserving of protection in its own right‖. Both political and
more scientific advocates will be able to agree with a view that autonomy has
intrinsic value. The differences between the poles would be paradigmatic, i.e. the
more technical side aims to be detached and the political side aims to be involved.
Both sides would, however, be basing their views on values. This might be expressed
as a cline with at one end those whose values lead them to feel that autonomy is more
the other end those who feel autonomy has to be overtly seen as the right to
from Nicolaides (2008), who reports on an ethnographic research project carried out
on future English teachers into learners‘ perceptions of their roles. The example of
one of the subjects, Otávio, illustrates an area of possible difficulty for the present
23
versions of autonomy, Otávio‘s independence is high: he has the ability, will and
motivation to learn well by himself. Viewing autonomy from the social interaction
perspective, i.e. joining in with the group, his autonomy is quite low as he shows a
lack of social responsibility by not getting organised for required class activities.
According to the political view his autonomy is low as he does not adapt to the
context (the class) and therefore does not ―exercise his most important right – the
right of learning also in the classroom‖ (Nicolaides 2009: 154). This appears to have
which they then use to operationalise the concept. For example Cotterall (1995: 195)
use a set of tactics for taking control of their learning‖. This strategy of pre-defining
autonomy reduces the abstraction of the concept and can also focus on the aspects of
particular research design. It has the disadvantage of not fully characterising the
theoretical reasons.
Multidimensionality may mean that autonomy is not one thing and may not be
seen in the example of Otávio means that a learner‘s autonomy may be manifested
24
2.4.2 Autonomy is variable
autonomy may vary with circumstances, such as different tasks. Benson (2001: 47),
for example, has stated that autonomy is ―a multidimensional capacity that will take
different forms for different individuals, and even for the same individual in different
contexts or at different times‖. Little (1991: 4) points out that an individual‘s level of
autonomy can also ―take numerous different forms, depending on their age, how far
they have progressed with their learning, what they perceive their immediate learning
needs to be, and so on‖. Levels of autonomy may also depend on the nature of
different tasks and ―the learner who displays a high degree of autonomy in one area
may be non-autonomous in another‖ (Little 1991: 4). There are many other variables
affective factors (e.g., mood), environment (e.g., noise, temperature), physiological factors
(e.g., tiredness, hunger), motivation (e.g., attitude towards the task, the subject matter, the
A further problem according to Benson (2001: 53) is that we know very little about
the stages in the development of autonomy except that they are highly variable and
uneven. Breen & Mann (1997) for instance theorise that learners react against the
snapshot of the learner‘s performance at any given moment in time may give a
misleading picture‖ (Benson 2001: 54). This is a question relating to the reliability of
tests or assessments, especially high stakes ones, and is not peculiar to attempts to
25
measure autonomy. The reliability of any instrument will affect the conclusions that
can safely be drawn based on its data. Benson seems to be referring to more formal
testing or assessing, but I feel if the instrument is not to be used for testing but, for
example, as a formative spur to reflection, this problem is much less relevant, as long
Learners‘ levels of autonomy may vary with motivation which will affect their
185) a learner may have ample capacity for autonomous learning but not have the
The willingness to take control varies from time to time and task to task, depending on a
headaches) and contextual factors (e.g. too much noise, not enough resources) which can
inconsistent and would misrepresent the learner‘s capacity if it were based on the
observation of the learner when not willing to deploy the capacity (see Section 2.4.4
indicates that measurement of autonomy would require the active volition of the
learner, which would be problematic in the case of a test (see Section 2.1.2 above),
but would appear far more achievable if the measure were voluntary.
Context is an often cited variable affecting autonomous learning and hence can
context the learner‘s level of autonomy may be more closely linked to the learning
context rather than to the qualities of the learner, as Carr & Claxton (2002: 12) note
26
―the manifestation of learning dispositions will be very closely linked to the learning
designed to measure the learner‘s autonomy may in fact be measuring the learning
Holec (1981) sees autonomy as a capacity of the learner, and Carr & Claxton (2002:
has, however, been amplified by others to make it one of the key features, for
example Cotterall (1995) sees it as what she terms ―readiness for autonomy‖ (1995:
Human beings are autonomous in relation to a particular task when they are able to perform
that task (i) without assistance, (ii) beyond the immediate context in which they acquired the
knowledge and skills on which successful task performance depends, and (iii) flexibly, taking
1997: 94)
It implies a responsiveness to one‘s environment and the ability to make creative and unique
responses to situations as they arise rather than patterned and stereotypical responses from
27
It is not likely that students who are dependent on their teachers are going to be as effective
in the world of learning or subsequent employment as those who have developed strategies
which enable them to find and use their own resources for learning. (Boud 1988: 21)
That is, Boud is referring to the likelihood of the transfer of autonomy to subsequent
―knows how to learn and can use this knowledge in any learning situation she/he
may encounter at any stage in her/his life‖ (Dam, Eriksson, Little, Miliander &
Trebbi 1990: 102). This again puts transferability at the centre of autonomy.
Authors often indirectly imply that autonomy is transferable when describing aspects
of autonomy. Dam has written that ―Active involvement facilitates awareness of the
different elements involved in, and when, learning – an awareness to be made use of
in other learning contexts (lifelong learning)‖ (2009: 134) and ―The learners, for
their part, are expected to engage actively in their own learning in order to become
fully aware of the different elements involved in, and when, learning – an awareness
to be made use of in other contexts‖ (2000: 49) (emphases added). Little (2009: 151)
gives two reasons for wanting autonomous learners: they are efficient and effective
because motivated and reflective, and they have knowledge and skills which they
gained in the classroom but which can be applied beyond it. Sinclair (2009: 185)
that it is necessary for transferring learning know how and transcending the
classroom.
situation), but also because it raises the question of the nature of generalisation in
28
autonomy i.e. can autonomy be generalised (transferred), and if so, how and within
what kind of limits. The literature of language learner autonomy (for example
stipulating that it is situation specific, but the boundaries of a situation are not
specified; some situations are similar to others, but at what point does the autonomy
stop being transferable? If autonomy is very specific to certain situations and tasks, is
it still to be seen as autonomy rather than a limited ability in one very specific and
confined area? Lamb (2009: 84) reports on research he carried out with young
teenagers in a UK high school. He found they had clear levels of ability in speaking
about their learning. Those with ―more sophisticated language and a broader
metacognitive knowledge‖ were better able to describe and discuss their learning and
had ―a better chance of feeling more in control of what they are doing‖. The
learners. Language learning may be seen as one broad context or domain (Littlewood
among tasks within this domain. If this is the case then the localised, situated nature
of autonomy may in fact be quite broad and an instrument aimed at the measurement
of autonomy within the domain of language learning may therefore not be as limited
or levels are frequently mentioned in the literature, and this suggests that, whether
can range from low to high. Most authors either state or imply that autonomy is a
matter of degree, and that it has levels. Nunan (1997: 193) for example says that
autonomy is not an absolute but has degrees. Sinclair (2001: 8) also sees ―degrees of
29
autonomy‖ and describes the idea of a continuum from no autonomy at one end, to
various points along this scale. Holec also sees autonomy as a scale from the lower
levels of dependence to the higher levels of autonomy. He (1981: 22) holds that
autonomy has to be acquired, and this is achieved through two parallel processes:
language learning (e.g.: there is only one ideal method and teachers possesses it;
knowledge of the L1 is of no use for learning the L2 and experience gained from
other subjects cannot be transferred; learners cannot assess their own learning).
Through these two processes the learner can gradually proceed from dependence to
22). If the deconditioning process (A) can be observed or gauged in some other way,
perhaps by means of a self-report questionnaire, then this may offer one strand in a
process (B) also indicates a possible avenue to explore for a method of measurement,
i.e. quantifying the knowledge and know-how which the learner has.
The idea, then, that autonomy has degrees is well represented in the literature. A
number of authors have in fact gone further and attempted to describe the levels of
autonomy I will look in more detail at the ideas of a number of authors who have
30
2.4.2.1 Breen & Mann (1997)
Breen & Mann (1997: 143) look at the learner and the group and see three broad
Table 2.1: Three-stage model of autonomy (Breen & Mann 1997: 143)
In Phase 1 the individuals will probably have been socialised to be dependent and the
class will be teacher-led. Both teacher and learners expect and accept this. In Phase 2
the teacher encourages autonomy and so there is a shift in the classroom towards
Phase 3. This apparently worse situation may be a necessary step towards fuller
measure, will need to accommodate, i.e. that the autonomy level given by a measure
may appear to dip – but may still be a sign of progress – before it again rises. In
The simple picture of autonomy levels derived from this is: No autonomy; Exposure
scoring scheme could be applied, e.g. 0 for Phase 1, 1 for Phase 2, and 2 for Phase 3.
This could be a starting point for a more detailed description of levels within the
phases. However, the adjustment level (Phase 2) is a warning that the development of
31
2.4.2.2 Macaro (1997; 2008)
Macaro‘s (1997: 170–172) model divides autonomy into three domains within each
of which there are progressions from lower to higher (see Table 2.2). One of the
domains, Autonomy of choice and action, overtly refers to autonomy beyond the area
The description of the abilities of learners is more specific than in Breen & Mann‘s
model and provides hints of areas that may be measurable, such as learner strategies
area of autonomy suggests that familiar measures of language ability may have a part
target language was safely associated with autonomy then there would be no call for
a separate measure.
Learner strategies
32
2.4.2.3 Nunan (1997)
4 Creation Learners create their own goals and Learners create their own tasks
objectives
Nunan‘s (1997: 195) model is of five levels of ―learner action‖ which increase in
degree of autonomy from level 1 to level 5 (see Table 2.3 above). The ―actions‖ of
the learners and the degree of control they use, whether shown in overt behaviours or
probed by an instrument, perhaps asking learners about aspects of their learning, for
example: ―Do you create your own materials?‖ (Level 4); ―Do you know your
learning style?‖ (Level 1); ―Have you formulated your own goals?‖ (Level 4).
Littlewood‘s levels of autonomy are defined by the choices which are available to the
learner, from low-level choices to high-level ones. He sees any number of levels as
33
Level Degrees of choices
1 Learners are able to make their own choices in grammar and vocabulary (e.g. in controlled
role-plays and simple tasks involving information exchange).
2 Learners choose the meanings they want to express and the communication strategies they
will use in order to achieve their communication goals
3 Learners are able to make more far-reaching decisions about goals, meanings and strategies
(e.g. in creative role-playing, problem-solving and discussion)
4 Learners begin to choose and shape their own learning contexts, e.g. in self-directed learning
and project work
5 Learners become able to make decisions in domains which have traditionally belonged to
the teacher, e.g. about materials and learning tasks
6 Learners participate in determining the nature and progression of their own syllabus
7 Learners are able to use language (for communication and learning) independently in
situations of their choice outside the classroom.
Littlewood believes it is possible to speak of autonomy not only in a global sense but
also in specific domains, such as professional or task specific. For language learning
34
2.4.3 Autonomy is a capacity
potential or ability for self-directed learning which the learner has. Thus, autonomy
is ―the ability to take charge of one‘s own learning‖, and the skills that this involves
including determining objectives, selecting methods, and evaluating what has been
acquired (Holec 1981: 3). Littlewood (1996: 428) agrees that autonomy is a capacity,
but sees two distinct elements in it, ability and willingness. Dickinson has also
agreed with the idea of autonomy as a capacity rather than being defined by action,
and sees this as necessary if the concept is to be applied in teacher-led situations and
autonomy would need to probe beyond the behaviour of the learner and measure
their ability and willingness (Littlewood 1996), and their abilities to control their
learning (Holec 1981). This can be seen as a dimension from the internal or capacity
sees capacity as the more psychological and individual property contrasting with
with the view that autonomy is present only when there is action by the learner which
Benson (2010: 83) feels that autonomy is not considered to be observable. However,
many authors have included forms of behaviour in their definitions, for example
Cotterall (1995: 195) defines autonomy (with my emphases added) as ―the extent to
which learners demonstrate the ability to use a set of tactics for taking control of
35
their learning‖ and goes on to say ―Learners will display these tactics to varying
degrees‖ and speaks of ―learners‘ readiness for the changes in beliefs and behaviour
which autonomy implies‖. The idea that autonomy has to be manifested for it to exist
is, in fact, very much in evidence in definitions of autonomy, often bringing in the
idea of observation of the deployment of learning skills and strategies (see for
However, Sheerin (1997: 57) points out that a learner may be disposed to act
autonomously, but not have the skills to do so, and there is a sense in which the
case – learning, and not the actual behaviour of an individual in that situation‖.
There are possible problems with the observation of autonomy. It is not ―a single,
easily described behaviour‖ Little (1991: 3-4) and it can be manifested in many
different ways (Esch 1996: 37). Sinclair (1999: 95-96) recognises that autonomy is
not the same as behaviour, and makes the point that behaviour can be observed but
not its rationale: ―the tutor cannot see this process, only the outcome‖ (1999: 101).
may in fact have very good autonomous reasons for not manifesting it. Sinclair
concludes that it is not useful to assess learner autonomy on the basis of observation.
Benson (2010: 79) also sees serious problems attached to using behaviour to assess
autonomy. Firstly, there is the problem of determining what the key observable signs
of autonomy are. There is also the probability that autonomy has non-observable
suggests that it would be problematic to determine whether they are, in fact, vital
36
parts or not. Little (1991: 4) admits that in fact ―we recognize autonomous learners
by their behaviour‖, but he is not advocating it. He points out that observable
… can take numerous different forms, depending on their age, how far they have progressed
with their learning, what they perceive their immediate learning needs to be, and so on.
Autonomy, in other words, can manifest itself in very different ways. (Little; 1991: 4)
even though he or she does possess it, the learner may be autonomous but not behave
in an observable way that could reliably be used to measure this (Sinclair 1999: 101;
Benson 2001: 52; Confessore & Park 2004). It is important not to assume that
autonomy or its lack to measure autonomy. Hence a measure of autonomy should not
be based on learner behaviour (see Section 2.5 below for a discussion of the methods
Breen & Mann (1997: 135) say that ―The autonomous person is able to step back
from what they are doing and reflect upon this in order to make decisions about what
they next need to do and experience.‖ Flavell (1979: 908) sees metacognition as
of the learning process‖ which Chan (2001: 508-509) sees as essential, as ―without
such meta-cognitive awareness, the learner will find it difficult to exploit the learning
resources at his/her disposal‖. Lai (2001: 40) sees the alternative to metacognition as
―robot learners who mechanically carry out all designated activities‖ without much
37
awareness of their overall learning process. Gan, Humphreys, & Hamp-Lyons (2004:
knowledge (for example Oxford 1990; Wenden 1991). These skills fall into three
main areas: person, task, and strategy (Flavell 1979: 907). Wenden (1998) describes
learners‘ person knowledge as relating to knowledge and beliefs about for example
aptitude or motivation and their ability as learners, both in general and for particular
tasks. Task knowledge relates to knowledge of the purpose of a task and how it will
serve their language learning needs, the type or purpose of the task, and its demands.
Strategic knowledge is awareness of strategies in general and when and how to use
them (Wenden 1998: 518-519). Sinclair (1999: 102) characterises these as awareness
of the learner him/herself as a learner, awareness of the subject matter (i.e. the
Cotterall (2009: 87-88) maintains that it is only possible for learners to begin to
Bailey & Onwuegbuzie (2002) found that the learners with the poorest performance
in language learning usually had a lack of metacognitive skills shown by: poor note-
taking, not seeking help when needed, not reviewing notes, not being able to manage
38
their moods, losing concentration, and not checking words they do not understand.
Lamb (2009: 84) in his study of high school learners reports that learners with a
what they were doing. The theme of control in relation to metacognition is frequent
in the literature and can be seen as one of the reasons why metacognition is stressed
as being conscious. Benson believes the mastery of learning skills is necessary but
not sufficient for autonomy; he stresses control as being fundamental: learners have
to be free to choose to learn what they want to learn or their learning may not be
―authentically self-directed‖ (Benson 2001: 99). Little, Ridley, & Ushioda (2002: 15)
agree that learners need control so that they can choose their own goals and accept
―In motivational terms the importance of this step can hardly be overestimated‖
It can be seen therefore that there is a prevailing feeling in the literature that
metacognition is essential for autonomy and is necessary for any meaningful taking
of responsibility and thus for controlling learning, though the support it gives the
learner to control learning is not in itself sufficient for truly self-motivated autonomy.
Scharle & Szabó (2000: 4) state that ―in order to foster learner autonomy, we clearly
there are different senses of the words autonomy (see Section 2.4.1 above) and
responsibility. According to Holec (1981: 3), to learn autonomously the learner needs
―to have, and to hold, the responsibility for all the decisions concerning all aspects of
39
this learning‖. Little (1996: 203-204), Boud (1988: 23), and Dickinson (1987: 15) all
highlight the importance of learners taking responsibility for their own learning.
The sense of responsibility is a property of the learner not the situation. Hence, it is
for example Holec‘s (1981) definition. However, Dickinson (1987: 11) defines
autonomy as ―the situation in which the learner is totally responsible for all of the
decisions concerned with his learning and the implementation of those decisions‖
responsibility is also seen as central. Holec and Dickinson may be using different
senses of responsible; Dickinson for where the situation requires it of learners, and
control or influence over what one is responsible for, a sense of agency. There are
implications and connections with the area of motivation since recognising one‘s
(Spratt, Humphreys & Chan 2002). Even if the direction of causation is not agreed,
clearly the two are closely linked. (See Section 2.4.7 which follows for discussion of
broader issue of how the aims of education are implicitly linked to, and in tension
40
with, society‘s values (Biesta 2009), and is therefore not a problem specifically with
Explicit links between autonomy and motivation are frequently mentioned in both
the literatures of language learning (e.g. Dickinson 1995; Ushioda 1996; Benson
2001) and psychology of education (e.g. Deci, Vallerand, Pelletier, & Ryan 1991;
Ryan & Connell 1989). Williams & Burden (1997: 120) give their definition of
motivation as:
A strong link between autonomy and motivation is found in the notion of control,
especially when the learner‘s conscious perception is that he or she is making the
decision to act based on their own intrinsic desires rather than for externally-
controlled reasons. Ryan & Deci (2000: 54) illustrate the difference using the
curiosity and interest or, alternatively, because he or she wants to procure the
because it is ―inherently interesting or enjoyable‖ (Ryan & Deci. 2000: 55) and
consequence (ibid) where the consequence is not the inherent satisfaction of doing
41
something for its own ―reward‖. A key feature of the idea of control is that it be
Deci et al. (1991: 327) maintain that there are three basic human psychological
needs: competence (i.e. knowing how), relatedness (connecting well with other
Motivation will be related to satisfaction of needs in one or more of these three areas.
of the act being self-determined or autonomous (Ryan & Deci. 2000: 58). There is a
Williams & Burden (1997: 101) describe this as concerning one‘s perception of
personal control over events. According to this theory people can be placed on a
continuum between those who see the control of events as internal (―internalisers‖)
and those who see it as external (―externalisers‖). Table 2.5 shows the characteristics
Williams & Burden (1997: 102-103) cite studies which have reviewed ways of
changing a learner‘s locus of control, and they suggest that it can be done, especially
by teaching learners to assume control of their own learning, e.g. by practising and
Weiner (1979) built on the locus of control theory to allow for the fact that a learner
can vary in how he or she makes attributions regarding their successes or failures.
42
Internalisers Externalisers
Feel responsible for everything that happens in Everything that happens in their lives is due to
their lives fate, luck or other people
Active Passive
Exploratory Non-exploratory
Assertive Compliant
Table 2.5: Locus of Control (based on Williams and Burden 1997: 101)
failure to ability or effort (originating inside the individual), and luck and perceived
task difficulty (originating outside the individual). The locus of causality can be
The third dimension is ―Controllability‖. The individual may see success as within
uncontrollable. Relating the theories of locus of control and attributions back to Deci
When a behavior is self-determined, the person perceives that the locus of causality is
internal to his or her self, whereas when it is controlled, the perceived locus of causality is
43
Deci & Ryan‘s theory (Self-Determination Theory) entails that the optimum
motivation is intrinsic, and the best conditions for intrinsic motivation are when the
they have the intention and the competence to take control of their learning.
Fazey & Fazey (2001: 345-346) give a description of the key features of autonomy,
and these have much in common with the picture of motivation given in this section:
decision-making, take responsibility for the outcomes of their actions and have confidence in
themselves.
requires that learners have both the will to make their own choices and the freedom
to exercise that will. As Lamb (2009: 71) points out ―intrinsic motivation can be
implications for measuring autonomy as it may restrict the learners‘ freedom. Benson
generated in response to a task in which the observed behaviours are either explicitly or
implicitly required
―perform‖ some task so that he/she can be observed and assessed for the autonomy
displayed. Importantly, however, he does not look at the possibility that a learner
44
may actively seek such a ―test‖ in order to self-measure (with a view to self-
point, though, is a caveat for the present research, and is applicable to some previous
research where a role has been imposed on learners (as will become apparent when
imposing a test on a learner will inhibit the very autonomy on which observation is
being attempted. However, Lamb (2009: 71), drawing on empirical evidence from
learners he interviewed, found that giving learners a real choice can overcome the
autonomy which a learner has freely chosen to undertake is not subject to the
In this section I consider the view that autonomous learning is essentially social and
interdependent and contrast this with the view that it is primarily concerned with
autonomy. It can be seen as a quality of the individual which is affected by his or her
psychology (e.g. Little 1991) and skills (e.g. Holec 1981), or conversely it is argued
that second language learning is a process situated in a social context (e.g. Pavlenko
45
how individuals learn, which Gremmo & Riley see as a reaction against
behaviourism, which they call ―the sterile hubris of a mechanistic psychology which
dared to extrapolate from dumb animals to human‖ (1995: 152). This reaction
growth of the individual as a complete person, as a human need which is the source
of motivation. It led to the growth of the humanistic curriculum (Dubin & Olshtain
1986: 75). This view of education reacted against traditional ideas where learners
develop rote abilities and depend upon being able to give back what is expected rather than to
make it into something that relates to the rest of their cognitive life (Bruner 1974: 406)
knowledge, did not engage what humanistic psychologists saw as the innate human
desire to learn. Humanistic theories claim that learning should involve the learners
more, making them active participants and having them take on personal
Learning is facilitated when the student participates responsibly in the learning process.
When he chooses his own directions, helps to discover his own learning resources,
formulates his own problems, decides his own course of action, lives with the consequences
The other major psychological strand leading to the notion of individual autonomy in
learning was work in the field of cognitive psychology. Piaget‘s constructivist view
46
(Williams and Burden, 1997: 21-24). In this view an individual learns by
information into the learner‘s mental framework and may necessitate the learner
actively reorganising the way the framework is configured, which involves an active
participation in the process of learning. As Page (1992: 83) puts it ―Every learner in
every situation is, strictly speaking, autonomous because only the learner can learn,
extension of the individual‘s capabilities. Learning was something learners did rather
than something which was done to them and this lead to more learner-centred
from ideas of learner centredness and the view of language as a tool for
Littlewood (1996: 427) saw autonomy as a concept that fitted well with learner-
This was how I saw autonomy in the 1990s, concerned with bringing out and
developing the self-reliance of the individual learner. However, since then there has
been a growing belief that it is better to treat autonomy as socially situated. Esch
(1996: 37) says that autonomy ―is not self-instruction or learning without a teacher‖
nor ―essentially a matter of deciding to learn without a teacher‖. Smith & Ushioda
47
autonomy is now seen to develop out of interaction with others; it benefits from
interdependence, and classrooms and teachers are no longer peripheral but at the centre-stage
of practical concern
there has been a growth in interest in autonomy in the classroom as opposed to the
248) maintain that the social view of autonomy means that it:
is not seen as an abstract set of discrete skills, attitudes or behaviours to be developed, but a
historically and socially situated process that evolves and is mediated and instantiated
A major psychological strand leading to the notion of social autonomy is work in the
field of cognitive psychology by Bruner et al. (1966) using the idea of social
The entire history of the child‘s psychological development shows us that, from the very first
From childhood more competent others help the learner to move to the next level in
what Vygotsky (1978: 86) calls the zone of proximal development (ZPD):
problem solving and the level of potential development as determined through problem
belief is that the pattern continues and is even reflected in adults‘ internal ―dialogue‖
48
(Ohta 2000: 53-54). This model emphasises that autonomous learning should not
learning, arguing for its social nature by attacking the straw man of radically isolated
learning. Little (1990: 27) wrote that ―total detachment is a principal determining
feature not of autonomy but of autism‖; this may be true of ―total detachment‖ but
this does not have the same meaning as the terms ―individual‖ or ―independent‖. The
degree of detachment suggested by Little is not found in the literature, though Little
has been quoted by other authors (e.g. Barnett 1993; Hurd 1998) as if it were. This
type of polarising may lead to an idea that autonomy cannot be found in independent
settings.
al. 1990: 102) which specifies that ―An autonomous learner is an active participant in
definition, an autonomous learner cannot be one who learns outside the classroom.
Empirical, logical, or conceptual grounds for this assertion are not given; it appears,
rather, to be a statement of how the authors will use the term autonomy based on
their own conceptions (and/or those of others). Seen in the context of Gabrielsen
(1990: 96-101), who contributes the section immediately preceding Dam et al.‘s in
the same 1990 Nordic Workshop report (in fact it forms part of the same article as
Dam et al.), the Bergen Definition can be read as a reaction to a prevailing focus on
the individual learner, which Gabrielsen (1990) outlines, and to a role of the teacher
their own learning. If the Bergen Definition is read in this light it appears not so
49
much to be a ―definition‖ of autonomy but rather a statement of how formal
education should be carried out. In the two-part article the first part by Gabrielsen
briefly explains the view of the situation and outlines the policies to improve
education. In the second part of the article Dam et al. provide a kind of executive
how education should be. This has, misleadingly, become labelled the Bergen
Definition. In fact it is not a definition, but should be read as a call for educational
reform, with some specific aims in mind to counter a prevailing situation which is
seen as having negative points (e.g. concentration on individual learners, and the
overly-controlling nature of teaching). Part of the ―definition‖ (Dam et al. 1990: 102)
concept but is emphasising the point that education must in future not only focus on
individuals and their needs but also on the group. It appears, then, that autonomy is
Cooperation with others can be seen as integral to autonomy from social and
Personal decisions are necessarily made with respect to social and moral norms, traditions
and expectations. Autonomy thus includes the notion of interdependence, that is being
responsible for one‘s own conduct in the social context: being able to cooperate with others
capacity:
50
the development of a capacity for reflection and analysis, central to the development of
There appear to be two closely linked but separable concepts regarding the social
nature of learning: (i) that learning is inherently and unavoidably social because that
is the origin (in children) of how humans learn (e.g. Vygotsky 1994; Lantolf 2000),
and (ii) that interaction is beneficial to learning (e.g. Littlewood 1981: 93-94;
measure something which does not exist. Carr & Claxton (2002: 12) see assessment
so, this would mean that a quantitative questionnaire may not be appropriate for
social, however, do not exclude a synthesis and according to Sinclair (2000: 11)
In this section I will focus on the more political end of the autonomy spectrum, as
distinct from the more technical and psychological end. These ends can (crudely) be
51
from the more political end of the spectrum sometimes write of their fears that the
then becoming steadily more ―technologized‖. Pennycook (1997) and Benson (1997)
felt that the political was inherent in autonomy but was being ignored due to the
The idea of autonomy has therefore moved rapidly from a more marginal and politically
engaged concept to one in which questions are less and less commonly asked about the larger
social or educational aims of autonomy. Broader political concerns about autonomy are
increasingly replaced by concerns about how to develop strategies for learner autonomy. The
One suggested reason for this is given by (Crabbe 1993 444), ―the psychological is
Another is given by Benson (2001 46), ―In the context of language education, the
more convincing arguments for autonomy are likely to be pedagogical rather than
that the technologized pole is being favoured at present due to autonomy being taken
up more popularly around the world, but with differing understandings (Esch 2009).
Gremmo & Riley (1995: 152-154) examine the reasons underlying the development
of autonomy in language learning, and they see much of its impetus as being inspired
by the desire to make changes in society. Holec (1981:1) sees in Western countries in
the late 1960s a rise in social awareness regarding improving quality of life (e.g. civil
rights movements). The expectation of greater freedom and equality led to a focus on
bringing ideas of political autonomy into education. Much of the initial impetus for
52
Partly due to these origins, many authors feel that autonomy necessitates learners
being more aware of the power relations implicit in learning so that they can take
more control over their own learning (e.g. Pennycook: 1997; Benson: 1997). Kenny
... it can be said that only when autonomy is being allowed to function is education taking
place at all. For where autonomy is repressed or ignored--in other words where the learner
has no say and no being—then what we have is not education but some sort of conditioning
there is not a strong distinction between learning and life in general. The attitudes
autonomy promotes should result in ―more useful members of society and more
effective participants in the democratic process‖ (Little 1991: 8). For Benson (2001:
46) education is a matter of concern to the whole community, ―authoring the social
realities that constitute our collective lives‖. In this critical pedagogy learner
if we are interested in education for democracy, we must ask critical questions of if and/or
how specific practices, resources and identity roles for teachers and students mirror other
(actual or desired) social arrangements in larger social worlds beyond the classroom
Ideas of democracy are frequently used to distinguish the aim of this view of
autonomy:
Many advocates of autonomy, despite their national and/or cultural situations, seem
interested in promoting the power of individual students and teachers to determine their
as simultaneously involved in learning and critical social practice (Toohey 2007: 242).
53
This democracy and autonomy can only happen, according to Moreira (2007: 70),
when ―learners have their voices heard, are able to participate in pedagogic
decisions, and are able to decide on the course of their learning‖. Autonomy and
democracy are frequently linked with struggle, for example Morieira (2007: 58) says
students‘ autonomy as learners, and, in the process, increase the democratic nature of the
Phillipson (1992) sees the political nature of language learning as due not to the
nature of autonomy specifically but to the nature of education. He says (1992: 67)
―The belief that ELT is non-political [...] assumes that educational concerns can be
divorced from social, political, and economic realities‖ which would be impossible.
language learner should not be seen, to use Atkinson‘s (2002: 525-526) analogy, as a
single cactus in the middle of a lonely desert but rather as a plant in a tropical
―language and its acquisition are not radically disconnected from the rest of the
world‖ and language learning has ―real potential for changing the world‖.
autonomy in that they are pro-social and seek to show what follows from the socially
embedded nature of the learner, learning, and language. There is also, however, a
political strand which takes a more individualistic perspective on the learner. For
example Dearden (1975: 7 quoted in Boud 1981: 22) writes of the qualities of an
54
wondering and asking, with a sense of the right to ask, what the justification is for various
things […] refusing agreement or compliance with what others put to him when this seems
critically unacceptable […] defining what he really wants […] forming purposes and
intentions of his own independently of any pressure to do so from others […]. In short, the
Boud states that the concept of autonomy in education ―refers to the capacity of an
autonomy has been a successful idea because it appeals to two different political
views, i.e. there is the struggle against oppression in order to create a more equal
society, and there is the struggle for the individual‘s freedom within society.
One argument for the political view of autonomy is that all human life is political and
political stance. The argument then is not that autonomy is political or not, it is which
stance is the ―right‖ one for autonomy, the critical (political) or the psychological
what they believe is right despite the absence of one agreed definition of autonomy.
The important debate for Benson can thus be seen as regarding how autonomy
should be seen, rather than trying to establish how it ―really is‖. He says ―it is
other definition through logical or reasoned argument alone‖ (Benson 2009: 21).
Benson uses this to justify the approach of seeing autonomy in terms of what kind of
world we want to live in, of how things should be, and this means seeing autonomy
55
in political terms, particularly as a counter to the negative effects of globalisation on
data regarding the technical considerations from political-critical values. This would
be with the belief that data of this nature will be useful for learners, and may assist
This concludes the section on the key conceptual considerations in autonomy and its
measurement. Some challenges to a measure have been discussed and progress has
dimension.
Each of these concepts has been seen as key to autonomy, but in reality they will
tend to be present together and will interact to varying degrees. For example, taking
motivation provides the necessary energy to take up and wield control; social
tangible end result, from the perspective of others, which can again feed back into
take account of all of these areas and this is reflected in the item selection process for
56
2.5 Previous autonomy measuring research
Measuring autonomy is not a large area of the literature, but there has been much
as necessary for metacognition. There has not, though, been much exploration of the
this section I will first look at a qualitative approach which focuses on evaluation and
discuss its usefulness for the present research. Following this I will look at four
examples of work relevant to measurement which highlight issues for the present
research. I will use lessons learned here to explain and justify my methodological
Dam aims to evaluate a particular type of autonomous learning, which she defines as:
… what takes place in situations in which the teacher is expected to provide a learning
environment where the learners are given the possibility consciously to be involved in their
own learning and thus become autonomous learners. The learners, for their part, are expected
to engage actively in their own learning in order to become fully aware of the different
elements involved in, and when, learning – an awareness to be made use of in other contexts
In an institutional context the aim is not autonomy, maintains Dam, but autonomous
57
Dam divides periods of learning into three phases:
Phase 3. Together session. In this phase both learners and teachers are in
charge.
The evaluations in each phase can be applied to all aspects of the learning process.
and focus on new areas of their learning process. The teacher can collect in
the questionnaires, analyse the data, and present the results to the class for
items in order to help them reflect and to provide a window into how learners
Oral planned. This can be in groups or as a whole class, and are based on the
58
In all phases the same basic procedure is followed: evaluation, followed by
discussion, then revision. This raises awareness and hence has a developmental
function. Dam also carries out peer evaluations based on presentations of learner
projects. She asks individuals to write down their opinions which can then be
Dam provides her students with feedback (evaluation) on the learning process in
three ways: orally, to individuals or groups; via comments in their learner diaries;
and via written evaluation given to the whole class, e.g. at the end of term. To
evaluate the outcome of a period of learning from the learners‘ point of view she
gives them two open ended questions and they have to write their answers. For
Which issues in our English lessons would you describe as being important –
and why?
What have you learned in your English lessons that you feel you might make
Why have some of the ways we have worked in the English lessons been
good?
What are – in your – opinion the main differences between the way we work
now and the way you worked before I took over the class?
learner and teacher) which is Dam‘s focus is a central and familiar part of learner
autonomy, being necessary for the ability to manage one‘s own learning. This is
rather different from the measurement (as opposed to evaluation) of autonomy being
probed in the present research (as discussed in Section 2.1.2 above). In her paper
59
Dam is not addressing the need for a measure of autonomy, but is concentrating on
evaluation. She describes methods of teaching she uses in her autonomous learning
Dam‘s perspective is that of a teacher evaluating her own classroom and practice
instrument. Consequently, she does not focus on a number of issues which would be
important for measuring autonomous learning. She does not for example address
questions of validity or reliability. The feedback which Dam presents from learners
and parents shows that the learners express satisfaction with the classes, but she does
not present this as an indicator of the degree of autonomous learning. Dam describes
classes which appear to have a tight structure controlled by the teacher with the aim
of developing the learners‘ autonomous learning. For example she describes how
periods of learning are divided into phases by the teacher, with all phases following
the same basic procedure of evaluation, then discussion, then revision. An autonomy
measuring instrument on the other hand would need to justify its approach in the
light of the points made by Champagne et al. (2001: 49) and others (discussed in
Section 2.3) regarding the tension between autonomy and the imposition of a
investigation in the present research (see Introduction Chapter), for example speed,
usability in contexts of independent learning and large classes alike, accessibility for
the inexpert user, etc., are not addressed by Dam‘s approach, which requires much
60
There is, in summary, a need for a measure of autonomous learning which is distinct
from a procedure for evaluation. These are not mutually exclusive and could be
In the following four subsections I will consider practical examples from the
which are more quantitative in nature, or have the most potential relevance for the
demonstrate different approaches to the problem. I will assess each relative to the
aims of the present research to find a practical measure. Review of this literature is
also useful for discovering pitfalls and weaknesses which I may face in the present
research. It will be seen that none of them provides a solution which suits the
requirements of the present research, but each provides valuable lessons which can
be incorporated into the hoped for simple yet effective measure of autonomy.
three years, learners studying for the CILL ―learn and demonstrate the skills of
independent learning in a systematic manner‖ and are trained ―to take responsibility
for their own learning using language learning as a vehicle‖ (Ravindran 2000: 64).
Following this learners progress through modules which require them to demonstrate
61
their abilities. The guidance given for the modules is progressively reduced and the
summative project.
The researchers sought to make the CILL a ―possible, workable, reliable and valid‖
assessment method (2000: 65) in the context of the Temasek Polytechnic self-access
centre. Learners were expected to display their levels of awareness and provide
demonstrations of their ability to apply the range of skills which had been selected by
the team (2000: 66) as necessary for autonomy. The criteria for the award of the
CILL consist of twenty items derived from Knowles‘s (1975) ―key skills of self-
learners‘ strengths and weaknesses; observations of the learners were carried out;
CILL helpers read learners' learning logs, learning reviews and contracts, and
assessed the quality of learners‘ reflections on their learning and on the tasks they
had carried out; the quality of learners‘ language in the work they submitted was also
assessed.
For the final assessment, decisions were carried out as a team, as many people had
been involved in judging and observing many students. For this reason the CILL
team engaged in regular training, monitoring, and feedback sessions and carried out
inter- and intra-rater calibration sessions which had at the time of going to press
shown minimal discrepancies (Ravindran 2000: 66). Detailed records had to be kept
at short notice) to aid in reporting and assessment, and for the quality control
62
I will discuss the CILL project and consider what lessons can be learned from it with
relevance to the present research. It is very different in conception from the kind of
CILL is that a sustained effort and commitment were required to set up and maintain
it. A dedicated and trained team was needed, which means this is not something an
individual student, teacher, or researcher could hope to carry out. It is also not quick;
it requires three years of work in a self-access centre, and is therefore not something
which could be achieved quickly, for example, for a needs analysis. It is striking the
training of staff, keeping records, and establishing and maintaining inter- and intra-
rater reliability. A drawback of CILL is its complexity, which means that without the
drive of the individual/s who initiated it there is a danger that it will collapse. My
research, however, aims to investigate the feasibility of a simple, quick, and easy-to-
use measure which will also be potentially usable for self-assessment. CILL is
clearly none of these, but it may present useful lessons to be incorporated into the
present research.
One such lesson is drawn from the impression that the CILL often appears to be un-
and the ability to apply the skill of self-directed learning‖ (Ravindran 2000: 66). The
word ―requires‖ implies that learners do not have the freedom to manage their own
supposed to involve the freedom to choose and control one‘s own learning path (as
63
discussed in Section 2.3). This issue is clearly of relevance to the present research
and will need to be addressed so that the instrument is not imposed on learners.
A further important issue raised by Ravindran‘s paper is that, despite the great efforts
that were made to establish its reliability, CILL is not explicitly construct-validated
research will need to address in relation to its aim of establishing whether such an
instrument is viable. The choice of elements to include in the CILL was based
primarily on one author‘s work, Knowles (1975). Other elements are from the
author‘s and others‘ own experiences, but the paper does not detail the procedure
used in the choice, and this suggests that the present research will need to present this
Cotterall defines autonomy as ―the extent to which learners demonstrate the ability to
use a set of tactics for taking control of their learning‖ (Cotterall 1995: 195). By
―tactics‖ she means to include setting goals, choosing materials and tasks, planning
practice opportunities, and monitoring and evaluating progress. The learners will
have these to different degrees. This is partly because learners have different beliefs
aim is to try to see if learners are ready for these changes because autonomy requires
changes in beliefs and behaviour, and she believes that the learner's beliefs
profoundly influence their learning behaviour. If the learner has mistaken beliefs this
may hinder their learning progress. Cotterall gives the example of ―making mistakes
is bad‖. If learners believe this, they will be inhibited, they will not practise, and
64
therefore they will not learn the language as effectively as they could. She says ―All
behaviours‖ (ibid 196). Cotterall wants to identify areas in students‘ beliefs and
probe how these categories relate to autonomy. She uses the statistical technique of
items which may shed light on students‘ beliefs. (Factor analysis is used in the
present research and is described in detail in Section 3.12.3.) These groups, called
factors, can then be named by the researcher according to what appear to be their
common characteristics. Cotterall wants to show with this technique what the beliefs
are which autonomous language learners hold, and that these can indicate the
Cotterall carried out interviews and used the data to make the questionnaire on
learner beliefs about language learning. The questionnaire had 26 items with a 5-
point Likert scale, and eight items using a forced choice format. The factors Cotterall
found were:
Cotterall‘s Factor 1 is ―Role of the teacher‖. This suggests that preferences about the
degree of teacher control are a key area in describing an autonomous learner. She
describes factor analysis saying how it defines connections between items which
seem to behave in the same way and she sees advantages to ―empirically identifying
65
dimensions underlying the construct of language learning autonomy‖ (ibid 197).
Cotterall looks at the factors revealed by the study and discusses what this may
reveal and the implications. Some of them are areas that tend to support autonomy
Autonomous learners are likely to be individuals who have overcome the obstacles which
educational background, cultural norms, and prior experience may have put in their way. The
degree of independence with which learners feel comfortable will be a key indicator of their
to diagnosing readiness for autonomy and finds in the literature (ibid 201):
general agreement that learner confidence correlates with academic success [which] supports
This concurs with Littlewood (1996) and also with motivation as characterised in
Self-Direction Theory (see Section 2.4.7) and makes confidence one of my choices
Learner beliefs about language learning will profoundly influence their approach to language
learning. Learners need to be aware of the role of cognitive and affective variables in
language learning
Factor 6 is ―Approach to studying‖. Cotterall admits that the link with autonomy is
not clear (ibid 203). By ―approach to studying‖ she means learning style, and she
66
warns that ―It may be unhelpful to suggest that a particular approach to studying
A weakness is the small number of items included in the questionnaire, only 34, and
it seems that this was the total number used in the development, though she says only
that the items ―were developed from a series of interviews with ESL students about
their experience of language learning‖ (ibid 196), without giving the criteria for
selection.
Cotterall does not propose this as a tool to be used by classroom teachers for
measuring autonomy. However, the questionnaire could be used to help the teacher
and learner explore the learner's beliefs and therefore help the learner with the
autonomy.
Cotterall does not explore the construct of autonomy she uses in her questionnaire;
rather, it is presented as a starting point. The use of many more items in the
development of the questionnaire combined with the factor analysis she uses may
have produced more interesting empirical results and this idea has led to the
questionnaire.
Cotterall‘s paper highlighted for me that the interpretation of the underlying concept
what can be concluded, for example in Factors 4 and 5 there are only two items and
it seems unsafe to draw conclusions based on this; I have consequently been cautious
67
It might be a problem that Cotterall assumes that the learners will have beliefs about
example one item reads ―I believe I know how to find my own ways of practising‖
which has a rather awkward sound. The more literal minded student may wonder
whether they believe they know or whether they know they know; in a way it is a
double-barrelled question. The Likert scale should provide the degree to which they
agree or disagree with the statement and therefore it would not be necessary to add
words into the statement which make it stronger or weaker. It may be that Cotterall
included words such as ―I believe‖ in her questions because she wanted them clearly
to relate to beliefs. For the present research the wording of items and their relation to
the Likert scale will be given great attention in an attempt to avoid this type of
drawback which may have an impact on the validity and reliability of the instrument
Interestingly motivation does not overtly appear among the factors. Perhaps this is
been seen in Section 2.4.7) it may be present in items or factor groupings but not
clearly manifested. This is an issue which I will return to at a later stage in the thesis
I gained useful lessons from the analysis of Cotterall‘s work; factor analysis is a very
without imposing one‘s own preconceptions about what factors should emerge from
the data, and this approach was adopted in the present research (see Section 3.12.3).
68
2.5.1.4 Lai (2001)
Lai aims to demonstrate a validated scale for assessing learner autonomy which will
have universal utility in contexts where learner training is being carried out. She
divides autonomy into two areas: process control (which operates at the micro level
of a task), and self-direction (which operates at the macro level). Lai (2001: 35)
a learner's ability to set realistic task aims for her chosen piece of material or activity;
identify problems; employ relevant strategies to tackle the problems; and conduct self-
the learners‘ ability to set realistic goals for their learning, identify scope of learning [...],
relevant materials to work with and related activities to engage in, and skilfully employ them
for monitoring their own learning, set their own pace for learning, and conduct self-
assessment
For process control, Lai decided to evaluate the extent to which learners increased
their control in two areas: ―setting aims‖ and ―carrying out self-assessment‖. Lai
asked them to decide aims and choose tasks to address them. The aims were assessed
regarding: whether they were appropriate to the task chosen; and whether they were
listening course).
For the self-assessment category Lai had two criteria for assessing how well the
aims; and whether the self-assessment is related to the learner‘s listening process
conducting self-directed learning‖ (ibid 40) and this is a central part of her rationale
for the assessment of the learners‘ self-direction. Lai had the learners design a
personal course of self-directed language learning and gave them a list of all the item
headings that they were supposed to cover in their plan, which included setting their
own criteria for self-assessment. Lai gave their plans to raters to assess the learners‘
metacognitive awareness and their planning ability. She did not look at whether they
we can nevertheless infer from a conceptual representation of the course, in the form of a
course design, whether the learner has grasped the rationale behind it, or has the potential
Lai had three raters to assess the plans and she prepared notes for each rater and also
trained them beforehand. The plans were scored by how they compared to 17
statements, using a scale from zero to six. Validation of the measurement scale was
carried out using the internal consistency of the items. Lai checked the inter-rater
reliability of the three independent raters, and she then calculated the reliability
coefficients among the three raters using Spearman rank-order coefficients. Lai
concludes that the two rating scales were both valid and reliable and therefore the
scores which they calculated based on the total mean scores of various raters were
autonomy is given which involves elements which are measurable and an instrument
missing, I feel, is checking the construct validity by, for example, seeing whether
70
other suitably qualified people would formulate the same definition independently.
There is a risk in her approach that the definition may not be acceptable to others,
and this will limit the universality which Lai gives as one of her main aims. As with
Cotterall‘s approach (Section 2.5.1.3) I feel that construct validity is an issue with
autonomy measuring instruments which the present research will need to confront
Lai feels that a course design produced by a learner is in itself sufficient to show
―whether the learner has grasped the rationale behind it, or has the potential and/or
ability to do so‖ (ibid 39). I would agree that it shows potential as the learner is
showing metacognition, but it does not address the different skills, capacities,
measuring autonomy and am, as with Lai‘s course design, not looking at the
by principled selection of items to cover such areas (see Sections 3.9 and 4.2.3).
The instrument used has a very similar approach to the General National Vocational
Qualification (GNVQ) in the Key Skill area of Improving own Learning and
plans, which are then rated by an assessor using a scale of band descriptors. This has
the disadvantage of being a lengthy process making heavy demands on the teacher‘s
time if a whole class has to be assessed (as I know from personal experience) and it
also requires the teacher to be trained before using it. This means that most teachers
are not able to use it without substantial preparation and commitment. Lai‘s approach
71
is similarly not a quick way of achieving the aim of measuring learner autonomy, and
in the present research one of the aims is to explore whether speed and convenience
can be increased.
Finally to assess their autonomy Lai requires the students to act in a certain way as
she gives them explicit instructions and guidelines on what to do and then uses the
data obtained to measure autonomy. This can be criticised for limiting autonomy by
not allowing it to be self-initiated (Benson 2001: 52), and this is one reason why I do
not wish to attempt to test autonomy but find a tool to help teachers support their
which needs to be assessed, and this cannot be done reliably by observing learners
for a short space of time. The key to doing this is metacognition. Sinclair (1999: 102)
says that ―The link between the development of metacognitive awareness and learner
autonomy is clear‖. The three areas which the learner should have metacognitive
awareness of are: the learner him/herself as a learner; the subject matter (i.e. in this
Sinclair (ibid 102) gives aspects of metacognitive awareness for each of these areas
(see Table 2.6), and gives the criteria for assessing metacognitive awareness as
72
Identify their strengths and weaknesses?
Area Aspects
Self Attitudes
Beliefs and expectations
Motivation
Needs
Learning style
Preferred learning environment
Sinclair (ibid 103) proposes that teachers can use these criteria to frame questions
when they are discussing work with the learners. For example:
73
If this type of question is asked systematically and consistently they can be used for
evaluation purposes; Sinclair says ―The extent to which the students are able to
which are shown by the content and the type of language used by the learners (see
method generalisability of results may be an issue as the criteria used would need to
Level 3 Largely aware Confident and competent use of all the above
Can also describe alternative strategies
Table 2.7: Linguistic evidence for metacognitive awareness (Sinclair 1999: 104)
74
Sinclair‘s approach is confined to metacognition, or it looks at autonomy from the
perspective of metacognition, and it may be that the inclusion of other areas would
better reflect the multidimensional nature of autonomy, for example: the learner‘s
knowledge, and skills. It would also require a skilled and knowledgeable teacher,
who may not be available, to carry out the interviews. Being interview-based this
method may also be time consuming and consequently be unsuited to larger class
sizes. Metacognition does figure prominently in autonomy theory (see Section 2.4.5)
2.6 Discussion
In this section I will summarise this Chapter on the literature concerning the
measurement of autonomy, and underline the conclusions which are most important
for shaping the present research. The four questions posed at the start of this chapter
were:
Questions 1 and 2 were addressed in Sections 2.1.3 and 2.3 and appear to support
further research into a measure of autonomy. Clarification of the term measure (in
Section 2.1.2) to distinguish it from testing and evaluation was useful in highlighting
75
The third question regarding whether autonomy can be measured necessitated
considering the conceptual and practical problems (Sections 2.4 and 2.5). The desire
five commonly-found false assumptions (1991: 3-4) about autonomy, i.e. that it is:
If these were true of autonomy, it would make the task of measuring it more simple
than it actually is. This can be shown by imagining them as true of autonomy:
easy to establish whether there was a teacher, and a test of learning gain
3. If autonomy were a new method that was taught by teachers, the description
of the method could be checked against the method used by the teacher and
76
5. If autonomy were a steady state which was equally expressed in all areas of
All of these appear to make measuring autonomy very simple, but they are very
black or white. This simplicity is not matched by the picture of autonomy that has
much of the spectrum of autonomy (Figure 2.1) as possible. The simplistic nature of
the above statements is partly because Little is making a point. In real situations
autonomy is not all or nothing: it may have, for example, some elements which are in
a steady state (No. 5), some behaviours which are revealing while others are not (No.
4), and there may be an element of learning without a teacher (No. 1). Measuring
4. stifled by testing
6. a political concept
7. variable
Yet, we also feel that we know that some learners are more autonomous learners than
others. This suggests that, rather like Little‘s statements these challenges are not
black or white absolutes. I have argued in this chapter that these points leave some
77
The initial specifications for a measure of autonomy which motivated this research
indicate the collection of quantified data. In addition, a measure should provide data
which is most relevant, for example to the learner for self-assessment, and to the
teacher to help in areas such as needs analysis. The belief was that data of this nature
will be useful for learners, and may assist them in their personal and/or collective
struggles for autonomy. For this reason in the present research the political-critical
view of autonomy will not be directly included in the search for an autonomy
measure. This is not to discount it, but to leave the values to be assigned to the data,
i.e. the evaluation, to the learners or teachers themselves. This research does, then,
about learners‘ autonomy, which can be either difficult and time-consuming (e.g.
Ravindran‘s CILL) or be essentially informal estimates. I feel the research will have
Question 4 asked if there were indications from the literature as to how to go about
2.4.2) and is in that sense measurable, what are the practical ways in which
measurement could be carried out? Sinclair (1999) looks for ways of providing
78
Morrison (2005) and Dam & Legenhausen (1996) have looked at learning or
learning gains with autonomy levels has not been done. If autonomy can be
learning‖ (Sinclair 1999: 97) then perhaps a questionnaire could ask learners to rate
their feelings in these areas, a form of feedback, and these might be suggestive of
autonomy level. Using logs or diaries to gauge levels of autonomy has the drawback
possible‖ (Sinclair 1999: 98). The degree of strategy knowledge and use may offer
insights into learners‘ autonomy levels and it may be effective to gather information
autonomous learners key areas emerge which offer the possibility of ways of
quantifying autonomy (which are less indirect than measuring learning gains), such
79
as metacognition (see Table 2.8 above and Section 2.4.5 above). These areas have
been the subject of research and have measures already proposed for them, for
by Sinclair (1999: 102) and; strategies have been the subject of Oxford‘s (1990)
Benson (1997: 25) divides views of autonomy into three types; it is seen as:
Probing these areas in a way that can render a measure may be possible and if so it
would suggest that an autonomy measure may be obtainable. The first classification
offers the possibility of probing the learner‘s technical abilities. The second allows
for measurement of the learner‘s psychological capacities for autonomy, and the
third, while it is necessary to consider that there will be elements of the learner‘s
situation which it may not be practical for him or her to control, still suggests the
avenue of probing the learner‘s attitudes to and perceptions of his/her role and the
amount of control that he/she wants. The learner‘s knowledge and beliefs about
Learners can have mistaken beliefs about learning; Little (1991) and Esch (1996) for
misconceptions will affect how autonomous learners can effectively be, therefore
some items in my questionnaire should address the learner‘s beliefs. For example a
learner may believe that it is better to learn without a teacher. A learner who believes
80
this may neglect to obtain help when necessary and may not be aware of the skills
which are required to learn autonomously. A learner may have concerns which
inhibit his/her autonomy founded on the misapprehension that autonomy means that
Learner beliefs such as this may contribute to their level of autonomy and may be
The areas that may be open to quantification and offer the possibility of more direct
The learner‘s degree of control over the content and processes their own
learning
The practical problems of measuring autonomy which have been presented in this
distinguishing between measurement and testing, and by realising that the problems
of measuring autonomy are not in fact very different from the problems of measuring
other complex concepts such as language ability, which have after much work
81
become generally acceptable (Benson 2010). In fact, the concept of levels of
To sum up, this literature review has considered the need for a measure of autonomy
and has evaluated the possible conceptual and practical problems which need to be
addressed. Possible routes into the quantification of autonomy have been discussed
and it has been shown that there is good reason to believe that autonomy has levels
learning.
Little says it is misguided to measure autonomy (2003), but Benson has written ―It
may simply be the case that the problems that we foresee in the measurement of
autonomy appear more acute because we have, to date, largely failed to address
them‖ (2010: 85). In the following Methodology Chapter I shall show how my
research has addressed the problems of measuring autonomy. The findings of the
literature review will be used to inform my research by guiding the choice and design
82
3 METHODOLOGY
3.1 Overview of chapter
In this chapter I aim to explain and justify the rationale which was adopted in the
researching autonomy, and will provide an overview of the planned research and the
actual stages of the research as carried out and show where this differs from the
original plan.
In the two chapters following this (Chapter 4 Long List stage, and Chapter 5 Short
List stage) I will present the progress of the research chronologically, showing the
actual steps taken with explanations of any changes to the original plan which
The three chapters – Methodology, Long List, and Short List – can be seen as
basically chronological; they build from the theoretical foundations of the research
methodology (Methodology Chapter), to the procedures carried out in the first stages
of the research using 256 items (Long List Chapter), and then to the part of the
research involving the distillation and use of the Short List of 50 items and its
instrument to measure learner autonomy (Short List Chapter). In this way I hope to
show how the research was based on a well thought out rationale and aimed to use
autonomy.
83
3.2 Clarifying the research aims
3.2.1 Aims
quick and accessible instrument to measure autonomy and support learners and
context dependent, and possibly only a Western construct. If I could show that
autonomy was, in a useful sense, measureable I thought it would also then help to
attempt to create such an instrument and test it. It was not a research question which
could be answered only by reference to the literature because opinion is divided (as
intended to use quantitative methods to design it and probe it. However, qualitative
data would be used to investigate whether learners found the instrument was useable
and useful, to find what it meant to respondents, and as Dörnyei (2007) has it, to put
―flesh on the bones‖ of the quantified data. The research can thus be seen as partly
research and support autonomy. In other words, this research should not be seen as
representing a positivist view of autonomy but rather an inquiry into the viability of
such a view in empirical terms. In this respect, the thesis should not be read as a
conventional report on a questionnaire study and its quantitative results, but rather as
84
3.2.2 The broad construct of autonomy to be used in the research
and there are differences in basic theory about what autonomy is, so this makes it
something new; I sampled from a broad range of literature to find what authors
thought autonomy was. I did, however, have to keep in mind the original aim (i.e.
exploring the viability of the instrument) but I did not want to prejudge what the
However, in order to start at all it is necessary and inevitable that options have to be
limited and some general direction established in order not to be pulling in many
different directions at the same time. To make it practical to begin I kept in mind the
original motivation for the research which indicated the nature of the tool, i.e. it
would be simple to use and require little setting up or expertise on the part of the user
questionnaire. The research would also need to address the question of whether such
I would limit the selection of items for the questionnaire, but not more than was
necessary for the aim of trying to find a useful instrument to measure relevant
dimensions of autonomy, i.e. which would be useful for practical formative teaching
and learning purposes yet still be recognisably autonomy – at least as seen by some
of the authors who had published their theories about its nature. It had to fit in to the
85
field in a way that made it recognisably part of the field and so it would need to have
construct validity.
What were these ―relevant dimensions‖ of autonomy? Since the purposes of the
instrument being investigated in this thesis were related to the practical support of
teachers and learners in the development of autonomy the relevant dimensions would
be those which had immediate and pressing relevance to teachers and learners in the
bottom-up support of learning. This would indicate that the relevant dimensions
way, not dependent on teacher control – on ―carrots‖ and ―sticks‖ education – but
having the learners take more control of their learning, and so being able to learn
The questionnaire, being made up of items chosen to reflect the field as found in the
what autonomy was. However, the construct of autonomy intended for any
finished instrument. The statistical technique of factor analysis (see Section 3.12.3
below) can be used to establish whether factor groupings in the data match the
groupings which were intended by the designer (i.e. confirmatory factor analysis, see
approach where the initial construct is broader, and then exploratory factor analysis
(see Section 3.12.3.2.2) is carried to discover the underlying groupings within the
data collected by the questionnaire. This picture can then be compared with the
86
Fullerton & Acton 2002: 184). This would be a check of the construct validity of the
instrument. In the present research this approach was adopted to construct validity.
This check would not look at whether the instrument was actually successfully (or
indeed usefully) measuring learners‘ autonomy (see Section 3.2.4 below for
Based on a review of the relevant literature I would compile a ―Long List‖ of items.
Factor analysis can be used for data reduction (i.e. it could reduce the number of
items) and in the present research this was originally intended as a way of removing
This item selection procedure would have the advantage of not fixing a precise
possible to have a completely open attitude to the construct for practical purposes of
questionnaire length; it would be necessary to limit the number of items in the Long
List to narrow the construct. This would be achieved by focusing on the aims of the
questionnaire and thus gaining an indication of the more relevant dimensions (the
―autonomy for language learning‖ i.e. as a means to the end of learning a language,
rather than ―language learning for autonomy‖ i.e. seeing autonomy as the goal of
learning rather than the target language (Benson & Voller 1997: 2), and one of the
less relevant views of autonomy for this aim is political interpretations of autonomy.
How could I tell if the eventual questionnaire was actually measuring autonomy?
The plethora of autonomy definitions meant that I did not expect the questionnaire to
expected to do that. This is beyond the scope of the present research, which as
87
previously stated is to explore the possibility of a quick and simple autonomy
measure to aid teachers and learners rather than an absolute universal standard
however indirectly, and one source must not depend on or influence the other. The
significant degree. It would be ideal if the other source was a validated autonomy
The most common way in which autonomy levels are judged is self-assessments by
learners of themselves; there are more students than there are teachers, so self-
assessment is likely to be more common than teacher assessments. The next most
carried out in a quite informal way (though, as seen in Section 2.5 there have been
88
The absence of a measure meant that (most) teachers were in effect measuring their
learners‘ autonomy based on their exposure to the learners, but not in a principled
way using a fixed procedure. If the eventual questionnaire could match or exceed
repeatable, and would be quicker since no long exposure to the learners would be
necessary as is the case with teacher estimates. The teacher will in effect estimate the
apparent attitude, conversations, beliefs about the students‘ cultures, etc. (It would be
an interesting project to investigate how teachers make their estimates and how
accurate they are, but this is beyond the scope of the present research.)
shown that it had a net advantage over the currently available methods of informally
advantage for teachers, and especially for learners (who may be less expert in terms
It was thus originally intended that the data produced by the questionnaire would be
compared with the measurements produced by teachers as these estimates are the de
I felt it would be useful to probe more deeply to see what teachers‘ and learners‘
beliefs about autonomy were and how they related to the results of the questionnaire
89
and therefore interviews were planned (see Sections 5.2.2 and 5.2.3). Even if a
data ―the questionnaire data usually reveals [sic] little about the exact nature of the
In addition, I would use the instrument with my own presessional group to gain
experience of its use in class for myself (see Section 5.4). This would be part of a
more qualitative strand in the overall research project (see Section 3.7.2). The project
can thus be described as being of the mixed methods type, utilising as it does both
This part of the research, as it transpired, was to be a key area in the eventual
reappraisal of my aims in this research and a change of emphasis more towards the
utility of the instrument for promoting autonomy and less towards its measurement
As stated in the Introduction I had the creation of a simple yet viable instrument to
measure autonomy as the specific motivation for this research. This would entail
the literature of autonomy to find what had been done in this area and to
problematize the endeavour from the theoretical side; and practically to design and
validate an instrument and, put simply, see if it ―worked‖ by checking its results
against some other acceptable standard. To make the research worthwhile it would be
90
important to show that there were advantages to the use of such an instrument.
measure of autonomy? (What are the issues involved and can they be
autonomy?)
autonomy etc., would also need to be considered in answering the second research
experiences of using the questionnaire with a class; one possible outcome, therefore,
is that I may be able to judge for myself the appropriateness of using the instrument
91
to measure autonomy, and also be in a position to consider the type of use to which it
is suited.
it was necessary to establish more precisely what form it would take and how this
could be justified in terms of the principles of instrument design. In this section I will
discuss these issues and arrive at a more specific description of the desired
instrument.
Oppenheim (1992: 10) warns that ―A questionnaire is not just a list of questions‖; it
has to ―speak for itself‖ and cannot so easily incorporate clarifications and probes as
can an oral interview. This meant that I had to exercise care with the wording of
items (see Section 4.2.1) and as I would be using it with non-native speakers this was
doubly important and influenced my decision to use translation (see Sections 4.3.1
and 3.13).
(Gillham 2000: 6): low cost in time and money; it is easy to get information from a
lot of people very quickly; and the analysis of answers to (especially) closed
questions is straightforward. In addition, Aiken (1997: 46) points out that they ―yield
a great deal of data on numerous variables‖ and Dörnyei (2003: 10) adds that they
are versatile, being able to cover a variety of topics, people, and situations.
respond to. Firstly, data quality may be low as questionnaires do not provide the
opportunity to check with the respondent to correct mistakes, and responses may be
92
superficial or insincere (Gillham 2000: 8; Aiken 1997: 46; Dörnyei 2003: 10-11). I
planned to minimise this by making the items as clear as possible in their wording
and by the use of translation. Where the final version of the questionnaire is being
used in class the teacher may in fact have the chance to respond to the answers and
when respondents are unsure if they agree or not with an item (Dörnyei 2003: 13).
Converse & Presser (1986: 38) however note that acquiescence is more noticeable in
the less educated. I limited acquiescence bias by including a ―don‘t know‖ option
among the possible responses after the first administration of the questionnaire in the
Thirdly, Gillham (2000: 8) and Dörnyei (2003: 10) both note the problems of
which typically they do not enjoy or benefit from in any way‖ (Dörnyei 2003: 11). I
impinging on their free time), and offering incentives (see Section 4.3.4), but the
voluntary Internet data gathering (see Sections 4.3.2 and 4.3.3) did have low
response rates, no doubt partly due to low motivation. When the final version of the
instrument is used in class the learners will experience benefits as the teacher will be
able to respond to what the learners express in their responses and it is therefore
93
Fourthly, people often choose the most desirable answer rather than the truthful
answer, a feature Dörnyei (2003: 12) calls ―Social desirability (or prestige) bias‖ and
the tendency on the part of a person to respond in what he or she judges to be a more socially
I would need to make clear that I would keep the data private and they would not be
In addition, Dörnyei (2003: 14) warns of the ―Fatigue effects‖ of long questionnaires.
To avoid these I would provide a response format which only required ticking a box
(while giving respondents the opportunity to make their own comments at the end of
the questionnaire if they wished) and I would ensure that the final version of the
Oppenheim (1992: 102-103) sorts questionnaires into three varieties using the mode
with groups, whereas a group-targeted instrument would not be useable with solitary
that the questions must be closed-ended (Bourque & Fielder 1995: 17).
94
Questionnaires have also been classified by the level of structuredness (e.g. Gillham
2000: 2-3; Cohen et al. 2000: 247) indicated by the balance of open and closed items.
According to Nunan (1992: 143) ―An open item is one in which the subject can
decide what to say and how to say it‖ whereas closed items, such as multiple-choice
questions, have the ―range of possible responses […] determined by the researcher‖
can be gathered and analysis is rapid. I had to decide which type of questionnaire
would be most suitable, and as I wanted the instrument to be able to gather data in
SACs and via the Internet as well as in classrooms I chose self-administered because
this is the most flexible being useable in or out of the classroom. For ease of data
processing and also for ease of use by teachers with large classes I decided on a
structured questionnaire with items of the closed type (though I also provided space
I now had to consider the structure and formatting of the questionnaire; length, how
items should be grouped, the most effective way to order the questions, the features
relatively appealing are all issues which are seen as important in the literature. I deal
95
questionnaire study and its quantitative results. I use quantitative methods in the
development of the instrument and in the larger scale data gathering and analysis, but
I also employ more qualitative methods such as interviews and use my own
experiences and reflections to explore the use of the questionnaire in a hybrid mixed
253) and Teddlie & Tashakkori (2009: 300) advise that ―research questions dictate
the research design and procedures‖. The aims which underlie my research questions
(2007), and more specifically for autonomy, for example by Riley (1996). In fact
Riley sees mixed methods research as the most appropriate for autonomy research.
Contexts for autonomy, such as self-access centres and classes, have both material
solutions, which are not easy to find or implement. (Riley 1996: 264)
If mixed methods research is indicated for researching autonomy then in the present
research I should aim to combine quantitative and qualitative methods if and when
96
Mixed methods research can be adapted for different reasons. In the present research
for example there are three main considerations. Firstly, mixed methods research is
adopted in order to account for both the researcher‘s and the subjects‘ perspectives
research takes the subject‘s perspective as the point of departure. These emphases may be
quantitative evidence for the measurement validity of the instrument, but such
evidence is often weak when exploring the reasons for relationships and a more
qualitative study can be used to complement it (Punch 2005: 242). Thirdly, mixed
methods research ―may provide a means of bridging the macro-micro gulf‖ (Punch
2005: 242); purely quantitative research is effective at the large scale, and qualitative
Punch (2005: 241) points out that it is necessary to decide whether to combine
methods, data, findings, or all three. They should be integrated in a way such that the
Mixed methods are appropriate for researching autonomy for a number of reasons.
They can help to make the results of the research acceptable to more readers. They
can show that the researcher has awareness of the complexity and/or
the chances of a situation akin to Riley‘s (1996: 251) story of the blind man and the
bubble, where the object of the research is not amenable to the means of research.
97
Certain research questions require a quantitative aspect and a qualitative one. The
present research, for example, concerns a quantitative instrument and its viability as
a support for learners and teachers. At the macro level, therefore, a quantitative
experiences of the instrument and how it applies to them a more qualitative approach
However, mixed methods research is not without problems. Dörnyei (2007: 174)
says the reasons why there is not more mixed method research is because many
researchers do not feel confident that they can do themselves justice in both
quantitative and qualitative areas for various reasons, and also that:
it requires considerable effort to study a phenomenon with two (or more) separate methods,
and in the light of all this it is understandable that many (if not most) researchers may prefer
Mixed methods are indicated for research in autonomy, but mixed methods research
how other researchers have approached the problem of researching the measurement
of autonomy.
In this section I aim to show how my research is informed by the previous autonomy
my research is perhaps closer to Ravindran (2000) and Lai (2001) and less like Dam
98
will have to use quantitative techniques to check it in its own terms in order to
qualitative area to see what it ―means‖ and how users feel about it, so gaining
Adoption of the techniques used in the literature was limited both by my research
aims and by the resources available to me. This also served to remind me that the
target users of the eventual instrument would also have limited resources. It would
not be possible to establish lengthy procedures. Ravindran (2000) had a team which
would, over the course of three years, carry out regular training, monitoring, and
feedback sessions, and keep detailed records for formal certification. To follow Lai
(2001) a teacher would need training and her method would also require much of a
approach would require a skilled and knowledgeable teacher, who may not be
available, to carry out the interviews. Being interview-based this method may also be
required rigorous checks to maintain the necessarily very high levels of reliability for
the three-year course. This would not be possible in the present research, and in fact
the aims of my research required a quick and simple instrument which did not aim at
formal certification but at providing a useful support tool. This governs the levels of
Dam (2000) does not address reliability and validity. Ravindran (2000) does make
great efforts to establish reliability, but rests construct validity on the item selection
99
which is based on only one author. Lai‘s (2001) research does not involve
comparison of results with another measuring method and so here too validity is not
autonomy, and is an area which I will need to address in the present research.
autonomy and then design an instrument to measure the (measurable) elements of the
definition. This is basically the approach adopted by Cotterall (1995), Lai (2001),
I was sensitive to the criticism that Ager (1996: 144) made of performance indicators
that they:
measure outputs rather than outcomes, they reduce complex situations to simple numbers,
they affect ‗performers‘ so that they perform to indicators, they measure only what can be
measured
Or, as Riley (1996: 259) says, ―counting what could be counted instead of what
autonomy and also therefore I decided I would follow an item selection procedure
which was as inclusive as practicable in the time available, was based on the
literature, and would be checked for construct validity (using factor analysis – see
choosing one extant version – for example, Ravindran (2000) chooses elements to
include in the assessment based on Knowles (1975). A large sample of items was
indicated, taken from the range of autonomy ideas that were pertinent to the aims of
the research.
100
Cotterall (1995) uses factor analysis and it appears to be a promising technique for
subjective variations I would seek to establish a procedure which I would follow for
all the factors I would be interpreting. Cotterall (1995) identifies factors with as few
as two items, which is too few to give a clear indication (see Section 3.12.3.3.5
below). This also points to the need for clear standards to be adopted for factor
analysis (see Section 3.12.3.3 below) which are overt and therefore repeatable.
I felt there was an issue in some of the research (Dam 2000; Ravindran 2000) around
making assessment of autonomy part of compulsory course work, and Lai (2001)
also gave models which the learners had to follow. This appears to reduce the learner
choice intrinsic to autonomy and I wanted to avoid this in my research. This was one
reason I attempted to make the questionnaire for the present research ―free-standing‖,
In designing mixed methods research Punch recommends that three questions should
be answered:
101
validate the questionnaire. I can show how it performs quantitatively in one
autonomy levels.
The qualitative part of the research comes in because to understand what the research
means it is necessary to talk to those involved and find what they were thinking,
what the questions meant to them, how they decided their answers, what the teacher
estimates were based, how strongly they were felt, and how confident the teachers
were about them. This is important to know also for understanding what the
instrument was measuring, and so could help to feed back into the cycle of
smaller scale, and so one or two groups of students and their teachers would be
studied, i.e. the questionnaire would be completed and the parties interviewed by me.
In addition I would myself use the instrument with a class I was teaching and use this
The research design involved both large scale more quantified data collection and
At the larger scale the responses from the questionnaire items would be coded and
quantified for analysis and comparison of questionnaire data with teacher estimates
102
autonomy in class: does it serve a useful purpose validly and reliably; does it have
Larger scale data collection was for three purposes: item selection; construct validity
After compiling the initial Long List (I refrain from calling the Long List a
questionnaire as it was not the finished questionnaire), the intention was to make it
available online and to collect a few hundred responses. Originally, the intention had
been to use factor analysis to perform data reduction and thereby form a shorter list
of key items for the questionnaire. However, in fact too few responses came in too
slowly for this and items were selected using other statistical means (see Section
improve the quantity of data available. I also intended to collect feedback from
respondents to find any problem items, e.g. those which were not clearly worded.
After item selection the now much shorter list (the ―Short List‖) of items would
contain the items which correlated most strongly into factors. This list of grouped
The Short List would be used to form a questionnaire and more data would be
gathered. The collection of a large amount of data would enable a check on construct
103
validity to be carried out using factor analysis of the gathered data. The resulting
picture of autonomy would be compared with the literature to see whether it was in
accord.
with the corresponding questionnaire data. The purpose of this was to establish the
comparative validity and reliability of the questionnaire. The estimates and the
expected that the teacher estimates would become better with longer exposure to the
subject class. This was an assumption since it would appear logical that increased
expected to benefit in any way from the intervening time period in the way that the
expected to move towards the level of the learners‘ autonomy over time, but the
This is not to say that the questionnaire results will not vary over time – as seen in
the literature review it is accepted widely that autonomy varies (see Section 2.4.2). If
the results of the questionnaire and the teacher estimates were to move closer over
time, i.e. between the first and second administrations, either viewed for individual
104
questionnaire was unreliable over time. If the results moved but remained equally
separated with no convergence it would suggest that the questionnaire was matching
the estimates but with a bias – this would also be suggestive of a positive result since
the movement would indicate a change in the autonomy level of the class which was
mean that a number of classes and teachers would be necessary for a definite result to
teacher estimates. If the required quantity of classes and teachers was not found any
results could not be shown to be significant, and could be no more than suggestive as
small samples may be idiosyncratic. Problems did mean that only two sets of teacher
estimates could be gathered and therefore statistically significant data would not be
gathered.
This would be more qualitative with concentration on one or two small groups. It
would ―put flesh on the bones‖ and would enable a comparison of questionnaire,
teacher estimates, and teacher and learner interviews in a more detailed way.
Regarding interviews, Dörnyei says, with obvious relevance to the present research,
that conducting interviews with respondents can help to find what they ―really
meant‖ and that ―This design pattern can also be used for validating test results with
105
Delays caused by translation, and difficulties with sourcing respondents meant that
the large scale comparison of teacher estimates and questionnaire data did not take
place and so only the small scale stage of comparison took place. This meant that the
production of statistically significant results was not possible. The small scale stage
could not substitute for the large scale in terms of statistical significance, but it would
still be valuable in two ways. Firstly, it could be a useful rehearsal and trial of the
methods intended for the large scale research which may suggest indications of
would shed light and provide examples of what the data may mean in terms of the
stage (see Section 5.4) so that I would have first-hand experience of estimating levels
and observing students in class which I could relate to their results from the
conjunction with the questionnaire and making estimates would, in fact, vividly
was attempting in this research into the measurement of autonomy in a way which I
had not foreseen when this stage was initially planned, and so this part of the
research was, as it transpired fundamental to the eventual outcome of the thesis (see
Sections 7.4.5 and 7.5, and Chapter 8 especially Sections 8.2 and 8.7).
La Ganza (2002: 48) stresses that there are many varied contexts for learner
advisor, or sitting under a tree with a book. My research was focused on tertiary level
106
learners of English which indicated class groups of learners as one readily available
source, and also the Internet, which would help my sample to be as wide as possible
to enhance generalisability, but it would still be relatively small scale due to limits of
time and resources. The larger scale and smaller scale data collections would have
Punch (2005: 103) gives the three questions which need to be answered in respect of
a sampling strategy: How big will the sample be, and why?; How will it be chosen
and why?; and What claims will be made for its representativeness? These questions
will be addressed here, and where appropriate the large scale and small scale
Punch‘s first question is ―How big will the sample be, and why?‖
The minimum size of the sample for the larger scale data collection (see Section
3.7.1.1 above) was governed by the intention to use factor analysis for data
reduction. (The necessity of carrying out factor analysis for construct validation, and
comparison between the questionnaire and teacher estimates also indicated a large
sample.) In the Long List there would be 256 items and in order to carry out factor
on the size of the ratio varies considerably (see Section 3.12.3.3.1 below) and will
depend on the characteristics of the individual data set. However, it was intended to
107
achieve some hundreds of responses. In reality at the stage of data reduction the
sample was not large enough and alternative methods were used (see Section 5.1).
Sample size would not be governed by the significance level statistics which are
would be acceptable.
specification there is a wide range of qualities the respondents may have in terms of
how it would perform the sampling would therefore be chosen as being one where I
could be the teacher. I would also aim to find another teacher who would use the
questionnaire with their class. I would then interview the teacher and their students.
the available samples would be appropriate to the target of the research as they
108
would consist of tertiary-level Chinese learners of English (Chinese students form
Punch‘s third question is ―What claims will be made for its representativeness?‖
Table 3.1 below shows recent statistics for overseas students in UK higher education,
and Table 3.2 shows the countries of origin of the respondents in the present
reflect the overseas student population in the UK, but rather the availability of
subjects, which will reflect the idiosyncratic situation of the research and contacts
which I had. It is thus a convenience sample and not a random sample of the world‘s
tertiary English language learner population. My department had contacts with China
sessional courses will tend to have more students from countries which do not have
which make English more difficult to learn. This may explain why there are so few
students who have problems with English and this population will be different from
the general overseas student body. The items in the Long List and questionnaire were
in English or translated into Arabic and Chinese. When translation became an issue
(see Section 3.13) it was hoped to provide more languages, but time was not
available. The extent of the questionnaire‘s availability in translation may have had
an influence on the composition of the sample. Any claims which will be made for
109
Region Total
Change since Proportion of all
2008/09 2004/05 % non-UK countries %
Total (all non-UK countries) 368,970 15.9 100
China 47,035 -10.7 12.7
India 34,065 104.2 9.2
Ireland 15,360 -6 4.2
Nigeria 14,380 76.6 3.9
United States 14,345 -0.3 3.9
Germany 14,130 12.5 3.8
France 13,090 12 3.5
Malaysia 12,695 10.6 3.4
Greece 12,035 -38.9 3.3
Cyprus (European Union) 10,370 82.7 2.8
Pakistan 9,610 46.8 2.6
Hong Kong 9,600 -10.9 2.6
Poland 9,145 318.5 2.5
Italy 6,035 13.5 1.6
Spain 5,690 -5.2 1.5
Canada 5,350 27.7 1.4
Taiwan 5,235 -11 1.4
Saudi Arabia 5,205 113.3 1.4
Thailand 4,675 18.7 1.3
South Korea 4,275 11.2 1.2
Adapted from HESA 2010, table 6a. Cited in UK Higher Education International Unit (2010: 8)
Table 3.1: Top 20 countries of origin for non-UK students at UK higher education
institutions, 2008-2009
110
3.7.4 Subject groups
Table 3.3 below shows the composition of the subject groups in the total combined
sample used in the present research. Also indicated in the table are the sections where
more description can be found of the data gathering for each group.
The reliability and validity of the eventual instrument will govern what it can
justifiably be used for as, for example, the standards for a formal test are much
higher than those for a formative class activity. Similarly, if the eventual instrument
were to be used as a research instrument the conclusions which could be drawn from
its data would have to respect the demonstrated limits of the reliability and validity.
3.8.1 Reliability
Dörnyei says ―reliability indicates the extent to which our measurement instruments
instrument but rather ―is a property of the scores on a test for a particular population
of testtakers‖ (2007: 50). At the larger scale level in the present research the internal
reliability of the scores will be found by means of the Cronbach‘s alpha of the
component groupings produced by the factor analysis of the data after the final data
figures above .7 and not below .6. Cronbach‘s alphas tend to increase with the
number of items in the scale (Field 2005: 668) so this will be taken into account with
111
Sample Sample Sex Av. Av. time Countries of origin (N) Av. time Data Section
name size age studying taken gathered
(years) English (minutes) using the 50
M/F (years) items in:
HCT 54 0/54 NA NA UAE (53), UK (1*) NA Format A 4.3.1
English 8 0/8 38 15.6 UK (1**), Colombia (1), Sweden (1), KSA (1), UAE 31 Format B 4.3.2
Internet (1), China (1), Switzerland (1), Canada (1)
Table 3.3: The sample groups involved in the research and the format of items used
At the smaller scale, data from individuals or small groups, such as those obtained
from the English Language, Translation and Cultural Studies (ELTCS) presessional
group (see Table 3.3 above), can be examined for consistency with the ―treatment‖
which the student would undergo, in this case a five-week presessional course. If
3.8.2 Validity
The routes to validation which were chosen as being practicable within the timescale
The construct validity check would help to ensure that the construct was not
representing a view of autonomy chosen only for its ability to be measured. It would
have to be a construct which was recognisably autonomy (Miller et al. 2002: 184).
Therefore, it was important to carry out a construct validity check on the concept of
achieve this, the statistical technique of factor analysis would be used to reveal the
underlying model. This could be different from the expected model, i.e. the areas on
113
which the item selection was based, but the picture which was revealed by factor
analysis will be compared with the literature to establish whether the questionnaire is
In this section the rationale for the choice of the Long List of items is explained and
justified. The details of this stage of the research are reported in Section 4.2.3.
As it was intended to have learners respond to all the items, practicalities of length
meant that judgement would have to be used to decide how many items were
acceptable to cover each area (see Section 3.10.1.1 for discussion of the issues
have some items covering more than one area to economise on the number of items
(Table 10.2 shows the items which were selected and the areas they were intended to
cover).
The literature would be surveyed in order to provide the rationale for the choice of
the Long List of candidate items for the eventual questionnaire. Items would be
autonomous learning. These areas of autonomy have been presented and discussed in
The items would be chosen with regard to the aims of the research, that is, to
measure autonomy (see Introduction Chapter) and therefore the items should
represent the relevant views of autonomy and the relevant purpose of autonomy for
114
language learning, rather than political conceptions of autonomy (see Literature
Review Section 2.4.1 and Section 3.2.3). The construct of autonomy which was used
would be necessary to establish an overt, detailed, and firm basis for the construct for
use in the selection of items for the Long List. When research into autonomy requires
description
Gather an expert group and agree together on a formulation, e.g. Dam et al.
(1991)
Each of these methods will result in a definition which is approved of by some, but
will inevitably not be acceptable to all involved in the field. I did not wish to alienate
readers by choosing a definition of autonomy at the very outset of the research. I felt
it would be preferable to initiate a clear and overt procedure which would reassure
the reader that I was open to empirical evidence in the formation of the construct. I
would allow the construct to emerge from the research, but with clear parameters
permitting a focus on the original aims of the research. I would not have a fully-
115
formed and final definition of autonomy as an initial assumption. (This attention to
the area of making the research acceptable to as wide an audience as possible is also
It would therefore be necessary to have a principled procedure for item selection and
for validation of the selection. Item selection would be achieved by reviewing the
literature of autonomy for language learning and establishing the main areas then
covering the areas with the items. Validation would be with a robust process for
checking the construct validity of the construct of autonomy embodied in the items.
Choosing items will inevitably involve the researcher in thinking about the subject,
but this should not be the only method employed. Gillham (2000: 17) warns that it is
dangerous to rely only on your own experience, as it ―may lead you into the
assumption that you know what the issues are because you are familiar with that kind
of context‖. One way to find ideas for items is to review the literature (Aiken 1997:
The Literature Review (Section 2.4) has shown the areas which would be used as a
basis for the selection of items. As discussed in the Literature Review it is not a
would be difficult to achieve for practical reasons of length, and (b) because it would
not be necessary for the aims of the questionnaire. The aims, as previously stated,
autonomy for language learning rather than autonomy for life, or language learning
116
for autonomy (Benson & Voller 1997: 2). These specifications would define the
Once the main areas have been identified, they can be used in generating an item
pool (Dörnyei 2003: 32). Gillham (2000: 41) observes that ―A typical weakness of
the novice researcher is to try to include […] too many topics‖. In the present
research it was considered that at the initial stages of instrument development it was
dimensions. The dimensions chosen are presented below; however, one feature of
feature rather than a defining characteristic. The area of transferability is, for the
reasons given in the Literature Review (see Section 2.4.2), an important aspect which
is seen as making autonomy a worthwhile goal and so the Long List would include
items which focused on different areas of language learning skill, for example there
would be items which address listening, speaking, reading, writing, grammar, and
vocabulary. As a result of the Literature Review the following areas were highlighted
Control Metacognition
Confidence Actions/Behaviours
Motivation Responsibility
117
With the areas defined it would be necessary to populate them with items which
There is agreement that a questionnaire is not an ad hoc list of items, but has to go
design, though the exact contents and techniques recommended vary (Fowler 1993:
94; Dörnyei 2003: 16-17; Sudman & Bradburn 1982: 281-282). The stages are
shown in Table 3.4 below, which also shows the sections which deal with them.
118
3.10.1 Putting items into a draft questionnaire
The items have been arranged in six different formats during the course of the
research as shown in Table 3.5 below. At each of the first four stages feedback was
gathered and changes were made, including removing items, which are detailed in
the reports of data gatherings. The different formats were used for data gathering and
contributed to the data used for factor analysis of the full data set. Only the 50 items
selected for the questionnaire were used in the factor analysis at the end of this
research.
Items grouped
into “face valid”
sections
Gillham (2000: 41) notes that it is common for ―the novice researcher […] to try to
include […] too many questions‖. Dörnyei (2003: 18) recommends thinking of the
119
slowest reader in a sample as a guide to limiting the length of a questionnaire, and
points out that one cannot include everything. Respondent fatigue is another reason
to keep questionnaires short and Gillham feels that this is particularly relevant with
rating scale questions (which is the type of response format chosen for this
questions; and people stop thinking about what they are doing‖ (Gillham 2000: 39).
The figure of thirty minutes is often mentioned, for example Dörnyei (2003: 18),
Aiken (1997: 38), and Fowler (1993: 103). The initial list of items was rather long,
but this was necessary in order to deal with all the areas. The latest version of the
Sudman & Bradburn (1982: 207) see the order of the questionnaire as requiring the
same care as the wording of the items. Cohen et al. (2000: 258) consider that the
configuration of the questionnaire indicates ―the overall logic and coherence of the
questionnaire to the respondents‖. The order of the questions and the groups into
which they are placed contribute to this impression and lead to better data (Sudman
Aiken (1997: 38) recommends that the items should follow a ―logical conversational
sequence‖ and the questionnaire should not ―jump around‖. It is important not to
120
are more likely to be careful in the responses they give‖ (Sudman & Bradburn 1982:
228). Gillham (2000: 25) also recommends that questions should lead logically from
one to the next for two reasons: it makes it easier for the respondent to work through,
and it prevents ―dotting around‖, i.e. not answering the questions in sequence. This is
important because questions are not ―stand alone‖ says Gillham (2000: 25) and if
they are not answered in the intended order the contexts of the questions will be
different for each respondent. This context effect is defined by Tourangeau, Rips, &
Rasinski (2000: 200) as ―the effects of earlier questions on the responses to later
ones‖. This arises because questions can stimulate memories and with the memories
activated, the responses to following questions can be influenced. The answer the
respondent gives will be in the context of those activated memories (Converse &
Presser 1986: 38). This is an effect which can make such a great difference that
(Gillham 2000: 12). However, Tourangeau et al. (2000: 216) maintain that context
effects are greater when the questions are perceived as being related and ―When
questions shift from one topic to the next without warning, respondents are no longer
likely to see earlier questions as carrying implications about the meaning of later
questions‖. This suggests that ―dotting around‖ may not be so undesirable. There
appears to be some disagreement in the literature, about the nature of this effect and I
feel that it is so difficult to account for, especially when it is not known whether the
effect is present in the questionnaire or not, that it should be looked at only in the
I put the open ended questions at the end of the questionnaire as they are more
difficult to answer, take up more time (Dörnyei 2003: 48, 62), and are perceived as
threatening if they occur at the beginning (Sudman & Bradburn 1982: 218, 262).
121
Demographic questions were also placed at the end in line with advice from Sudman
& Bradburn (1982: 218) and Dörnyei (2003: 61). However, Converse & Presser
(1986: 39) admit that ―It is frequently unclear that one order is better than another‖ as
―each order may reveal a different facet of the issue being studied‖. As with question
―break up‖ the 256 item questionnaire and make it appear shorter. I sequenced these
sections were given headings to orientate the respondents to the focus and help to
―The appearance of the questionnaire is vitally important‖ (Cohen, et al. 2000: 258);
the respondents‘ perception of the difficulty of the task can be affected by its
appearance (Sudman & Bradburn 1982: 243). Consequently ―It must look easy,
attractive and interesting rather than complicated, unclear, forbidding and boring‖
(Cohen, et al. 2000: 258). This can be helped by aiming for a clean uncluttered look
(Gillham 2000: 39; Dörnyei 2003: 19; Cohen et al. 2000: 258). It is a common
together and using a small type face (Sudman & Bradburn 1982: 244). All this advice
was kept in mind for all versions of the questionnaire, and following a suggestion by
Dörnyei (2003: 21), different colours were used for the different sections in the
Microsoft Word-based version and, rather than numbering the questions from 1-256,
122
they were numbered within the sections to help make it appear shorter (Cohen et al.
2000: 258-259). Format F (the latest version) has only 50 items and it was not
Apart from the items themselves, which make up its bulk, Dörnyei (2003: 25-30)
gives the component parts of a questionnaire as: Title; Introduction and general
thanks. A questionnaire should speak for itself (Gillham 2000: 38) and this is
Respondents are not trained and are unlikely to be motivated (Fowler 1993: 100), so
attention has to be given to introducing them to the questionnaire, giving them clear
instructions, and making them feel positive about doing the questionnaire.
3.10.2.1 Title
Dörnyei (2003: 25) gives the title‘s functions as: identifying the domain of
investigation, providing the respondent with initial orientation, and activating content
schemata.
Aiken (1997: 40) specifically recommends that words such as ―questionnaire‖ should
not be included in the title. He does not say why, but Dörnyei (2003: 25) thinks it is
because such words are uninformative. I decided not to put the word ―autonomy‖ in
the title of the questionnaire because it is a technical term and it was sufficient that
respondents understood that it was a questionnaire about how they learned English.
123
3.10.2.2 Introduction and general instructions
There are some basic areas to include in the introduction and general instructions
how long it will take to complete, and what to do with it when finished. Dörnyei
(2003: 26) advises naming the organization responsible for conducting the study,
respondents that there are no right or wrong answers in order to encourage honesty.
To this should be added the assurance that the results will not go towards grades if
included about the purpose of the questionnaire (e.g., Aiken 1997: 37; Dörnyei 2003:
autonomy partly because it is a technical term, and I also felt that respondents might
It is important that respondents know exactly how to indicate their answers to the
different types of question contained in the questionnaire (Cohen et al. 2000: 258;
Aiken 1997: 37) and instructions must be clear (Dörnyei 2003: 27). Oppenheim
(1992: 142) warns that ―Serious loss of data can result from ambiguous or inadequate
must be distinct from the questions to avoid confusion. The present questionnaire has
the advantage of using only one question type for the closed questions, and for this
124
3.10.2.4 Ending the questionnaire
A space at the end inviting optional comments and suggestions on the questionnaire
and its content is recommended (Sudman & Bradburn 1982:218; Bourque & Fielder
1995: 104). I did this for all versions of the questionnaire apart from the first where
Questions for demographic information should come at the end of the questionnaire
(Oppenheim 1992: 132; Aiken 1997: 38; Sudman & Bradburn 1982: 218-219).
Aiken points out that placed at the beginning they would appear to be inconsistent
with the questionnaire‘s title and stated purpose, which will not mention collecting
the time the respondents have completed the questions they are more likely to be
convinced that the inquiry is a genuine one and so be more willing to give personal
information. In Format A of the Long List no demographic questions were asked, but
these were included in subsequent versions placed at the end of the questionnaire.
In this section I will explain my choices regarding how I would administer the
instrument for data gathering purposes. I will consider whether to target groups or
individuals, the medium to be used (paper, online), and how I would gather feedback
supervisory person‖ (Bourque & Fielder 1995: 4) is present to give instructions and
125
monitor the respondents, and the precise way that the administration is handled
should be decided in advance, and the administrator should have clear instructions
(ibid 1995: 5). Ideally the researcher would always be present in person to ensure
that procedures and conditions are comparable with other administrations. With a
stand-alone questionnaire such as mine (see Section 3.4.1) this problem is reduced as
it is less necessary for the researcher to be present and there are fewer of the
uncertainties resulting from how the class teacher handles the administration.
As resources were limited and with a large number of responses necessary, the most
3.11.2 Medium
There is a choice of media for questionnaires, online or paper-based, and I would aim
to have online data gathering as much as possible as I have had experience with both
of these modes in the past, and have come to prefer the digital option over the paper-
disproportionate amount of time to transfer the answers from the sheets to the
computer for analysis, and there was also the strong possibility of making
The price of postage for this can be quite high, and it is a problem to transfer funds to
pay for the return postage. Thirdly, respondents sometimes do not answer an item,
126
The use of computer technology rather than paper helps to avoid mistakes in
transcribing, as data can be entered into software such as Excel and SPSS with a few
mouse clicks without ever having to physically type the numbers into the computer.
In addition electronic formats make the questionnaire more quickly and easily
take time to set up, but this was considered worthwhile because of the advantages
already mentioned.
I gathered feedback on both the items and the questionnaire to, as Cohen et al. (2000:
260) recommend, check clarity, eliminate ambiguities, see how the questions
perform, get feedback on the appearance and layout, see how long it takes, see if
on the items in a number of ways and at different stages of the research. An initial
translate the questionnaire had been made. Changes were made to the items based on
the feedback received, and changes were also made to the format of the
questionnaire, for example I decided to use words to describe the Likert options
rather than pictures or symbols. For the class-based data gatherings I asked the
teachers to gather feedback from their students about the questions. For the online
data gatherings I made provision for feedback on the items at the end of the
questionnaire.
127
Another angle of feedback was the translation process which proved to be an
excellent way of improving the clarity of the items (see Section 3.13). I found it
speaker respondents.
I gathered feedback on the items and the instrument at the following chronological
stages:
In this section I will cover issues of data analysis such as coding, standardisation of
data from different data gatherings, and factor analysis of the data for checking
construct validity.
128
3.12.1 Standardisation
I standardised all data before analysis using SPSS, which is essential if data from
different data gatherings are to be combined. For the purposes of factor analysis,
large amounts of data are required, specifically the ratio of the number of
respondents to the number of items in the questionnaire must be high (see Section
3.12.3 below). Without standardisation it would not be possible to combine the data
from different data gatherings. Each data gathering has its own unique context, for
example in the present research the format of the questionnaire, the medium, the
context of an item within the list of items, the place (home, class etc.), the length of
time needed for completion, etc. would all vary. Therefore in order for these data to
standardisation would have to be followed. Dörnyei (2007: 205) explains that ―The
sample in a way that the mean will be 0 and the standard deviation 1‖. According to
Dörnyei the standard scores express each raw value in terms of its difference from
the group mean. The means from the different samples are equalised making the
The data were combined for the factor analysis necessary at the construct validation
stage. I had intended to use one format of the questionnaire for this, but in the event
different formats were combined. I could combine the data because I would not be
of learners from many backgrounds that would be built up. Most importantly, the 50
129
3.12.2 Coding of responses
Responses to the questionnaire items would be coded so that I could process them
extent, as I would need to decide (based on the literature) which items indicated high
or low capacity for autonomy. The factor analysis would also tend to counteract this
subjectivity since it would group items by their statistical interaction, not by high or
low codings.
Each of the 256 items making up the Long List would be a statement. For purposes
of data gathering each item would be given a Likert scale for participants to make
their responses. Initially these were 5-point (―Format A‖), worded ―Strongly agree‖,
―Agree‖, ―Neither agree nor disagree‖, ―Disagree‖, and ―Strongly disagree‖. The
For the purposes of data analysis it would be necessary to reverse code those items
which were worded negatively (or had a negative connotation as regards autonomy).
73 of the 256 items were reverse coded. The inclusion of negative items is advisable
to discourage respondents from marking only one side of the rating scale and also to
counter the effects of acquiescence bias (Dörnyei 2007: 205). In this thesis items
which are reverse coded are indicated (where appropriate) with an ―R‖. Normally-
coded and reverse-coded items all had ―Neither agree nor disagree‖ = 3.
After the UAE data gathering the 5-point Likert scale was widened for all subsequent
formats to increase the definition of the responses. Table 3.6 below shows this and
the corresponding changes in the coding of responses. The description of the Formats
130
Format A Coding Formats B, C, D, E, F Coding
Very strongly agree 7
Strongly agree 5 Strongly agree 6
Agree 4 Agree 5
Neither agree nor disagree 3 Neither agree nor disagree 4
Disagree 2 Disagree 3
Strongly disagree 1 Strongly disagree 2
Very strongly disagree 1
Don‘t know 0
Table 3.6: Comparison of Likert item responses
When all the data gatherings had been completed the data would be standardised (see
Section 3.12.1 above) to form the complete data set for the 50 items to be factor
analysed. The purpose of this would be to check the construct validity of the
Though I would have carefully reviewed the literature to find areas for inclusion in
the Long List, no one item could be expected to isolate one area of autonomy, and
the area which it was intended to represent may not be how it would be perceived by
the respondents and so it may be placed by factor analysis into an unexpected factor.
Exploratory factor analysis may put two items together in a factor which come from
different areas, but in doing so reveal a hitherto unforeseen area of autonomy, or put
an unexpectedly stronger emphasis on one aspect rather than another. The strength of
the correlations between factor groupings suggests how areas of the construct may be
linked together. This is potentially of great value to learners, teachers, and possibly
131
Since factor analysis groups items together into what are, statistically speaking,
similarly behaving sets (factors) of items, it is a procedure which can be used, for
example, to show how questionnaire items may be interrelated. As Jones (1998: 386-
388) explains the ―Relative strength of the Factors is shown by the percentage of the
total data-set variance which each one accounts for‖ which means that it is possible
to identify which of the factors found in the analysis account for responses to a
questionnaire (Schommer 1990: 499). Factor analysis can be used for data reduction
as it can ―reduce a large number of variables to a small number of values that will
still represent the information found in the original variables‖ (Nakatani 2006: 153).
Factor analysis, then, produces groupings of items which are behaving similarly in a
statistical way. It also shows how the items are more strongly or weakly associated
with a particular factor, for example some items will be ―cross loading‖, i.e. they are
strongly associated with more than one factor. The results of a factor analysis are
formed by two axes, the items (or variables) on one, and the factors on the other. As
Jones explains, ―The figures in the matrix show how well each raw variable
correlates with the Factor as a whole in other words, its relative contribution to the
Firstly, factor analysis is a powerful tool for finding the concepts which can be said
to underlie a set of items (Regan 1994; Tremblay 2001) such as those found in a
questionnaire.
132
Secondly it can be used for data reduction. The procedure can be used to form multi-
item scales, which can then be statistically checked for internal consistency and
therefore indicate items which are suspect and can be rejected. This was the original
intention in the present research as it would allow the Long List to be shortened
while still retaining its necessary coverage of the significant elements of autonomy.
Thirdly, as Green & Oxford (1995) note, it can be used a means of supplying
evidence for the construct validity of an instrument. For example, a factor analysis of
the data obtained from a questionnaire may provide a ―picture‖ which is ―consistent
with theory‖ (Miller et al. 2002: 184) and this means that it is more likely to be a
There are two basic types of factor analysis, confirmatory and exploratory, based on
groupings which have already been established by other means, such as expert
Schmitt (2006: 90) adopted this in their research into self-regulation in vocabulary
acquisition as they ―assessed the hypothesized model for its fit to the observed data‖.
Confirmatory factory analysis (CFA) is designed to assess how well a hypothesized factor
structure ―fits‖ the observed data. Unlike exploratory factor analysis, the researcher has an
explicit prediction concerning both the number of factors that underlie a set of measures and
133
In the present research factor analysis would be used not to confirm a prediction or
discover ―the patterns that underlie the correlations between a number of variables‖
(Miller et al. 2002: 174). Researchers have made much use of EFA ―to generate
not exist on the surface of the observed data‖ (Nakatani 2006 153).
EFA produces ―a set of empirically-derived categories‖ (Sakui & Gaies 1999: 475)
which are valuable because they provide an alternative source of ideas about what
the components of autonomy are and how they may be related (see Section 7.3.4 for
Naturally it does not provide a new model for autonomy as it is dependent on many
that the coverage of the set of items is comprehensive (Dörnyei 2007: 234) and it can
thus measure all relevant areas. It is also important for maintaining the construct
EFA can also be used for the purpose of data reduction, for example to eliminate
items from a questionnaire which are only weakly correlated with the factors
(Russell 2002: 1636). The technique of factor analysis gives a figure for the loading
of individual items into factor groups and these figures can be used in selection, so
134
items which load strongly are candidates for selection and items which load weakly
or on to more than one group are candidates for rejection. As previously stated one
aim of factor analysis was originally data reduction. This was not possible due to the
low ratio of respondents to items. This was because the Long List was too long
which meant that there were not enough willing volunteers to complete it.
analysis was not reached. Factor analysis would have to wait until the Short List of
50 items had been selected from the original 256 of the Long List by other statistical
means (see Section 5.1). A drawback of this could have been that it reduced the
coverage of the field of autonomy before the factor analysis had been carried out. It
was therefore important that the Short List would continue to represent the range of
areas covered by the Long List and thus maintain its links with the literature on
autonomy which had been originally reviewed. This was achieved, though the
proportions (i.e. the ratio of items in one area to the total number of items in the
Lists) were not identical (see Table 5.3) but they were very similar. The precise ratio
was not critical, though it was vital that at least three items were present in each area
as this is the minimum for a factor to be recognised (see Section 3.12.3.3.5 below).
This and other important considerations to be aware of when carrying out and
EFA is a complex procedure with few absolute guidelines and many options
(Costello & Osborne 2005: 1) which could make the process of deciding which to
use relatively subjective; and it is tempting to try as many as possible and then,
135
retrospectively, justify the choice which fits with one‘s expected outcomes. I avoided
with data from a sample of adequate size. Field (2005: 638-640) reviews the
literature on sample size for factor analysis, which offers a number of conflicting
indications. He reports (2005: 638) that there is a common rule of thumb which
(2002: 1632) however, has found that ―Minimums of 5 or 10 cases per measure have
Field‘s literature review also suggested that 300 cases is a good sample size. He
reports (Field: 2005: 640) that Guadagnoli & Velicer (1988) argue that if a factor has
four or more loadings greater than 0.6 then it is reliable regardless of sample size. He
reports that MacCallum et al. (1999) find that relatively small samples of less than
100 may be perfectly adequate if all the factor loadings are above 0.6. It was
therefore decided to aim for 300 participants for the questionnaire research, but
additionally, to process the data and inspect the factor loadings to see if it would be
possible to stop before reaching 300. Another alternative is to see whether enough
adequacy (KMO). If SPSS indicates that the value is 0.5 or above this is acceptable,
and becomes more acceptable if it is nearer to 1.0. If it is below 0.5 more data should
be collected (Field: 2005: 648-650). It is also, states Field, (2005: 650) important to
examine the individual KMO statistics for all variables. These should likewise be
above 0.5. If any are not the analysis should be run without them and the results
136
compared. The off-diagonal data in the anti-image matrix should be very small. In
addition Field (2005: 652) states that Bartlett's Test of Sphericity should have
Field (648) says that ―to do a factor analysis we need to have variables that correlate
fairly well, but not perfectly‖ and, ―Any variables that correlate with no others
should be eliminated.‖ The significance values of any variable should be mostly less
than 0.05. The correlation coefficients themselves should be below 0.9. If necessary
one of the two variables causing the problem should be eliminated. This is a way of
its accuracy. He also recommends (2005: 648) checking that the determinant is
bigger than 0.00001. According to Miller et al. (2002: 184) the criterion of successful
The first issue faced by any investigator planning an exploratory factor analysis
concerns how to extract factors from the data (Russell 2002: 1630) because there is
no single best procedure which has been established for factor analysis; for example,
ascertaining the correct number of factor groupings to extract from the data is an area
which is much debated (Dörnyei 2007; Costello & Osborne 2005). I chose to base
my decision on the technique of using indications from the scree plot of eigenvalues.
The break of slope in the scree plot indicates the number of factors to extract
137
Dörnyei recommends (2007: 233-236) using either maximum likelihood or principal
component analysis as the extraction method, and either oblimin or varimax as the
rotation method, giving four combinations for the factor analysis procedure and three
alternatives for the number of factors to extract, and this is the procedure which I
used. This resulted in 12 separate factor analyses which I then checked for ―low-
loading‖ and ―cross-loading‖ items, and for groupings with insufficient items.
check that the items are strongly associated with the factor.
Low loading: If an item does not load on to a factor above a certain level it can be
ignored. The loading of an item onto a group can be strong or weak, ranging from the
maximum 1.000 to 0. The loadings should be below 0.9 (Field 2005: 648) and at the
lower level the figure is debatable and depends on the strength of one‘s data. For
example Nakatani (2006: 154) used the figure of 0.4 or less, while Yang (1999: 520)
chose 0.3. I follow Dörnyei (2007: 235) with a figure of 0.3. Items loading below this
Cross loading: Often a single item will load onto two different groups at the
significant level or above. This ―cross loading‖ indicates that an item is shared
between two or more factors. This would be a criterion for rejecting an item from my
than one factor above the 0.3 level it would be ignored. If this cross loading item
were retained it would confuse the identification of the factor groupings (see Section
3.12.3.3.4 below).
138
Cronbach’s alpha: Individual items in factors can detract from the overall reliability
of the grouping, and the Cronbach‘s alpha for reliability of scales can be used to find
such items. There are two criteria for accepting or rejecting items based on the
Cronbach‘s alpha: firstly, the corrected item-total correlation figure should be above
.3 and certainly above .2 (some leeway can be allowed for if an item is otherwise
strong); secondly, if deleting an item from the scale improves the Cronbach‘s alpha
for that scale then the item should be deleted (Field 2005: 666-675).
For a factor grouping to exist it must have three items or more which load on to it at
the significant level. In the case of a grouping with only two items it will not be
possible to identify reliably the underlying common feature and for this reason such
Gan et al. (2004: 407) explains how he identified factors, ―The name or label for a
factor was based on a unifying concept (or unifying concepts) embedded within a
pool of items‖. The factor groupings produced by EFA need to be interpreted by the
researcher (this has not been pre-defined as in CFA) and it is beyond the capabilities
experienced independent parties. In the present research this would be done in the
supervision process.
139
The process of identification is seen as a subjective process (Field 2005: 666;
Dörnyei 2007:236) and detailed procedures for carrying it out seem to be absent from
the literature. To find the underlying connection in a principled way a procedure was
2. Find the key words in each of the items in the scale and highlight
5. Think of words or short descriptions which represent the most key words,
6. Compare the descriptions for all the scales to ensure they are distinct.
It is clear that this is not an objective method. It does, however, have the advantage
aware of any other researcher in the autonomy field who has specified an overt
procedure for labelling factors. It has the advantage of being a set of guidelines
which can easily be referred to. Point 1, for instance, addresses the fact that it is easy
investigation. The procedure was carried out and resulted in themes or areas that
Russell (2002: 1632) quotes a figure of four or more items per factor for ensuring
140
groupings produced by factor analysis would be treated as forming provisional multi-
item scales. The Cronbach‘s alpha measure of internal consistency reliability for
these scales would be calculated and these figures used to check the composition of
scales. According to Dörnyei (2003:112) short scales of 3 or 4 items should aim for a
reliability of 0.70, and should at least reach 0.60. However, Field (2005: 668) points
out that the number of items in the scale is a major influence and longer scales will
always tend to have better Cronbach‘s alphas. He cites a particular case where a
figure of 0.57 is ―respectable‖ because there were only three items in the scale.
3.13 Translation
3.13.1 Introduction
Very often interesting things come out of research which were not foreseen at the
planning stage. This was the case with my research as issues (e.g. feedback to items
37 and 105) emerged regarding the Arabic translation of the Long List for the UAE
administration (see Section 4.3.1). This raised the question of how to translate, and
whether to present parallel English and translated items (see Section 3.13.4.6). The
importance of all respondents having the same understanding of the items is clear
respondents available for my research Mandarin Chinese was chosen as the first
language for the new more considered translation process (further languages were
In this section I will first consider the reasons for translating the questionnaire, then
possible ways of conducting my translation and finally I explain and describe the
procedure I adopted.
141
3.13.2 Reasons for translating
are good reasons for conducting English language teaching/learning research in the
language, and therefore the data obtained are more likely to be reliable
long questionnaire)
own language
translations are not usually appropriate for questionnaires. Harkness & Schoua-
Glusberg (1998: 93), for instance, point out that ―Languages are not isomorphic and
142
Rather, the notion of equivalence is seen as key, an idea which has its roots in the
literature of translation studies. Nida (1964: 129), for example, says ―equivalence
aims at complete naturalness of expression, and tries to relate the receptor to codes of
behaviour relevant within the context of his own culture‖. The relevance of this to
the autonomy measuring instrument is that where items refer to situations which are
would term this ―covert translation‖, that is, the translation process ―produces a
target language text which reads like an original text […] in the target language and
thus does not signal that it is a translation‖ (1998: 104). Newmark (1981 1991)
translation. A semantic translation will be one which aims to remain close to the
However, when back translation (see Section 3.13.4.2) was carried out on the initial
Chinese version of the questionnaire (see Appendix 10.1), it raised doubts about the
communicative nature of the translation. It was not clear whether the problem lay in
Chinese native speaker (a fellow student) who was fluent in English and who
confirmed that a semantic style translation had been carried out. It was therefore
necessary to do the translation again with much closer liaison with the translators.
143
achieving it; as Munday (2001: 43) points out ―equivalence inevitably entails
subjective judgement‖.
In the translation process I would need to include checks for ―ethnocentric bias‖ i.e.
(Van de Vijver & Hambleton 1996: 90), in this case Chinese. Smith (2004)
recommends the use of more than one item to probe a point; the same question can
be asked using different wordings to make sure that the concept is equivalent
between the languages: ―Three linguistically distinct measures of the same construct
are desirable‖ maintains Smith, and ―if all three agree, one has a clear, robust
finding‖ (2004: 434). Compare this with the discussion (Section 4.2.1.8) on multi-
item scales, where more than one item probes the same area.
Staying close to the original text may appear to promote equivalence, for example,
by using the same terms in source and target versions. However, this is not advisable
when the term is culture-specific such as, for example, the word ―Parliament‖. It
would probably be better to change this to an equivalent term in the target language
These points indicated that I should use procedures involving close liaison between
myself and the translators to minimise translation problems; the choice of translation
There is little hard evidence to guide in the selection of translation procedure (Smith
2004: 446). However, in this section I will give my rationale for the selection of
144
translation method from among the available options, and describe the process that I
chose. The methods I considered, which are the most common methods used in
Harkness & Schoua-Glusberg (1998: 100) describe this as ―one translator producing
one translation in a traditional manner‖. This was how the Arabic (see Section 4.3.1)
and the first (problematic) Chinese translations were done. Smith (2004: 447) says
that this approach is frequently used, being ―quick, easy, and inexpensive‖, but he
adds that it is not usually recommended because it relies too much on one person‘s
Harkness & Schoua-Glusberg (1998: 97) define back translation as ―the translation
of a translation back into the source language‖. The purpose for doing this is ―to
compare/contrast the back translation with the source text, usually with a view to
assessing the quality of a translation‖ (1998: 111). The forward and back translations
are carried out by separate translators. It is useful for detecting errors in translation,
with the added advantage that the researcher does not have to have a command of the
I used back translation, as previously stated, on the initial Chinese translation of the
questionnaire and it was useful in highlighting difficulties with it, showing that the
145
3.13.4.3 Committee translation
This is where a team of translators and researchers discuss the meaning of the source
items and assess possible translations. This method places emphasis on writing good
items ―not just on translating words‖ (Smith 2004: 448). It has the advantage of the
combined expertise of the committee members, for example some may have
knowledge of the target culture and others may be familiar with the construct under
should ―make independent translations of the same questionnaire‖ then ―compare the
Schoua-Glusberg 1998: 101). I thought this a promising technique, but its major
drawback was the difficulty of assembling a committee which was able to meet on
This is where translators work independently then come together to compare results.
When the results differ the translators work with the developers of the questionnaire
to find out why (Smith 2004: 448). It did prove possible to have different translators
working independently, however it was not possible to bring the translators together
to compare their work and discuss it with me. However, parallel translation did
Harkness & Schoua-Glusberg support the idea that translation should begin during
the drafting of the questionnaire, and not be left until after the items have been
finalised. Although they do not cite research that provides empirical evidence for
146
this, they are of the opinion that this is an important procedure which ―is often
particularly relevant for the languages and cultures furthest removed from the models
underlying the source text‖ (Harkness & Schoua-Glusberg 1998: 105). This makes it
particularly relevant for the present Chinese translation, but not as a technique for
which is important.
Making both the source and translated texts available to respondents appears to be a
bilingual, can check both versions to work out the correct interpretation. Stansfield
reports that ―there is a growing belief that it ‗does no harm‘‖ (2003: 201). However,
he does not mention the possibility that test takers may be confused by possibly
conflicting versions where the translation and the source have different connotations
or interpretations. The data gathering in the UAE (see Section 4.3.1) received
feedback from one student who was confused because there was a slight difference
between the English and the Arabic translation. I feel that if the translation is perfect
then it is not necessary to have a bilingual questionnaire, but if it is not perfect and
there are differences between the two language versions, it seems to me that some
doubt or confusion may result. In fact when discussing this point with a Chinese
confusing. On this basis I decided to aim for a good monolingual Chinese translation.
3.13.4.7 Decentering
Johnson (1998: 18-19), ―may involve multiple iterations of translation and back-
147
translation, with each language version being continually refined to bring them into
closer concordance of meaning‖. There are various procedures for doing this. In one
questionnaires, one in the source and one in the target language. It is seen as a way of
a specific culture and language (Harkness & Schoua-Glusberg 1998: 98). Van de
Vijver & Leung (1997: 39) describe decentering as ―the removal of words and
and in this way it seeks to remove item bias and increase the equivalence.
about individual items and if necessary the changing to alternative phrasing of the
original text so that they worked in both languages as equivalently as possible. One
translator would make a translation and then another translator would check it and
add an alternative Chinese translation if they thought it necessary. I then worked with
a translator and discussed which the best translation was, asking how it translated
back into English, and if it would be easier to translate into natural sounding Chinese
if the English text was changed. The resulting translation was then checked by
independent Chinese L1 speakers who also made suggestions which I sent to other
There were some items where I went against the advice of the translators because I
felt my instincts were possibly right and I needed to await feedback from the
administrations to see how the items performed. For example regarding item 251 ―I
use real English texts (i.e. not made for students) in my learning‖, different
translators thought I should use the term ―authentic English texts‖. However, I was
148
not convinced that this would be a term in everyday speech. The translators I was
using were all in the English language teaching field which meant that this term was
familiar to them, but I did not think it would be familiar to the Chinese respondents,
questionnaires may be subject to cultural differences. In this section I will discuss the
In order to avoid using words and so reduce problems of translation and cross-
cultural equivalence, the item response options can be expressed as numerical scales
nations
149
3.13.5.2 Response styles
questionnaire items; this makes the use of reverse coding a useful technique. This is
where two questions cover the same construct but one is worded positively and the
other has the statement worded negatively and reverse-coded (Tourangeau at al.
Some people will tend to favour the middle responses and others the extremes,
regardless of their true attitude to an item (Smith 2004: 440). This is often given as a
difference between East Asians and Westerners. However a study by Chen, Lee &
Stevenson (1995) did not find evidence for East Asians preferring the midpoint, a
to include the middle options in Likert scales. Smith says ―Research from several
countries finds that providing ambivalent respondents with a clear response option
produces more reliable results‖ (Smith 2004: 441), but he does not specify which
countries. However, Dörnyei (2001: 207) says ―this […] appears to be a relatively
unimportant question that is not expected to modify the results significantly‖. In the
light of this, I chose to include a middle response to avoid forced choices (see
Section 3.12.2)
After being translated and checked the administration of the Long List of 256 items
continued, though much time had been lost due to the unforeseen need for a lengthy
process to convert the list for use with Chinese respondents. However, I feel my
research benefitted through working with the native Chinese speaking translators
150
will return to highlight the issues and implications of translation in language learning
Though there was an original guiding plan it was soon modified. Table 3.7 provides
Stages See
1 Researching the field, literature review. 3.2.2
2 Writing items for the Long List of 256 items. 4.2
3 Data gathering in UAE. Parallel English/Arabic translation. 5-point Likert Scale. 256 4.3.1
items. (Format A)
4 Online data gathering of the English items. 7-point Likert Scale introduced (Format B) 4.3.2
5 Translation into Mandarin Chinese 3.13
6 Online data gathering with the Mandarin items (Format C) 4.3.3
7 Questionnaire designing
8 Email data gathering with the Mandarin items in a Microsoft Word form at Beijing 4.3.4
Normal University (BNU). (Format D)
9 Item selection process reducing number of items to 50 5.1
10 Web-based data gathering with 50-item Mandarin questionnaire (Format E) with 3rd 5.2,
year ELTCS BA students at Warwick University (―T1‖) 5.3
11 Interview with one 3rd year ELTCS student 5.2
12 Interview with 3rd year ELTCS teacher. Teacher‘s estimates of students‘ autonomy 5.2
levels are made
13 Presessional course with ELTCS group starts at Warwick University. I am their 5.2,
teacher. I make initial estimates of their autonomy. (―P1‖) 5.4
14 First Web-based data gathering with 50-item Mandarin questionnaire with my 5.2
presessional ELTCS BA students. (Format E)
15 I teach presessional course with ELTCS students. I observe and make notes regarding 5.2
students‘ autonomy.
16 I make post-course estimates of ELTCS students‘ autonomy. 5.2
17 Second Web-based data gathering with my presessional ELTCS group. (―P2‖) (Format 5.2
E)
18 Data was obtained from students in other presessional classes at Warwick University, 5.5
Internet based, in English. (Format F)
19 Data obtained from 14 students in Taiwan. (Format E) 5.6
20 Standardisation of all data. Factor analysis of combined standardised data. Analysis of 3.12
results of factor analysis and comparison with literature. Comparison of quantitative
and qualitative data.
Format (A-F) refers to the presentation format of the items – see Table 3.5 for details
151
Table 3.8 (page 153) shows the original outline plan for the research as envisaged at
the outset and next to this are notes of problems and issues which emerged which
3.15 Summary
In this chapter I have presented and explained the aims and rationale for the research
and presented the plan which was based on those considerations. I have also, for
In the following two chapters, each covering a distinct period in the research, I
describe in chronological order how the research actually progressed. In the first of
them I describe the part of the research based around the Long List of items, which
includes showing how the items were designed to cover a broad range of areas of
autonomy, and how care was taken with the wording of the items. This Chapter also
deals with the unforeseen issue of translation (which had far-reaching consequences
on the timescale of the research). In the subsequent chapter I deal with the Short List,
including how it was selected from the Long List, and I look at the issues involved in
presenting it as a questionnaire.
152
Step Description of original step Notes Section
1 Survey the literature of autonomy to find the main The range of areas to be covered was very large and this meant that selection had to be more 3.2.2,
areas. focused on those which appeared to relate to the practical aims of the eventual questionnaire. 3.2.3,
This meant deciding which areas were peripheral to these aims and eliminating them. This 3.7.1.1.1,
meant that the ―objectivity‖ was reduced. 3.9.1.1
2 Compose items which cover the areas of autonomy. Scales to cover each area and sub-area were found to be impractical for the present research 4.2
Cover each area with multiple items worded due to the great number of items which would be necessary. Respondent fatigue would be a
differently so that a very large number of items, major limitation. Items had therefore to be chosen more subjectively with the aims of the
perhaps as many as 200, would result. eventual questionnaire in mind.
3 Compose a Likert scale; upload the list of items to the Access to the Internet was an issue in China. In other places students did not have access to 4.3.3
Internet as a form in a webpage with each response computers in class. This meant that paper versions had to be printed and distributed and the 4.3.4
coded with a ―score‖ to gather responses to the items. papers returned by post and the data digitised. This severely slowed down the research.
Respondents would give their feedback on the items The issue of translation emerged as a major issue. The research was intended to be
to highlight any which they found unclear. international, but items were not being understood as intended, and consequently it was
decided to translate the items into a number of different languages. Quality of translation 3.13
became a major issue. This became time-consuming as the items were translated into
Mandarin Chinese and checked. Most of the respondents were Chinese, but more translations
would have been done if time had been available.
4 Having amassed at least 200 responses from a wide The number of respondents fell short of the numbers hoped for, which slowed down the 5.1
range of respondents, items which had caused research further. It meant that the numbers of responses required for factor analysis would be
confusion would be removed. The data would then be reached much later than envisaged. In order to proceed with the research it was necessary to
factor analysed. The results of this would be used to reduce the number of items to make the ratio of respondents to items acceptable for factor
eliminate items. analysis. This meant reducing the number of items to 50. This item reduction was carried out
by statistical means
5 The remaining items would be used to form a well-
designed questionnaire.
6 The questionnaire would then be administered over
the Internet to as broad a sample of respondents as
possible, numbering at least 200.
153
Step Description of original step Problems which emerged during the research Section
7 Items which had caused confusion for 5.1
respondents would be eliminated and the data
would be factor analysed to eliminate items
which did not contribute to the factors found.
10 Small scale data gathering in tandem with the Qualitative data gathering in tandem with the questionnaire was carried out on a 5.2
questionnaire. Learners would be observed, presessional course and on a third year undergraduate course. The presessional was
interviewed, and feedback sought from their my own class and the undergraduate class was that of one of the lecturers at Warwick
teacher to gather data on the respondents. I University.
would teach one class and find another class and Only one student who had agreed to be interviewed actually came for interview. This
teacher at Warwick so that I could interview was caused by the earlier delays in the research caused by translation and by the
students who were not from my own class. difficulty of finding sufficient respondents to the Long List. The delay meant that
interviews clashed with end of year assessments.
11 Comparison of the questionnaire and the Comparison with the literature: the data so far gathered were standardised so that they 3.12.1
qualitative data to gain insights on its viability could be combined to carry out factor analysis. The results of the factor analysis
and validity. would be used for construct validity.
Comparison was made of the small scale qualitative and quantitative data which had
been gathered to see whether the questionnaire was functioning well for individual
learners and classes to provide a qualitative indication of validity.
Table 3.8: Original outline plan for the research and subsequent changes
154
4 THE LONG LIST STAGE
4.1 Overview
In this chapter I describe the compilation of the Long List and give details of the data
gathering which was carried out using it. The purpose of this stage was to gather data
previously stated, factor analysis had originally been envisaged for this, but
insufficient returns were obtained (the actual procedure used is given in Section 5.1).
The rationale for item selection is given in Section 3.9 above. The Long List was a
list of items from which it was intended to select items for the autonomy-measuring
self-administered. As Cohen et al. (2000: 251) point out, words are inherently
ambiguous and great care needs to be taken to minimise this. In order for the present
research to maximise the clarity of the items and to avoid item quality issues I
decided to aim for the highest standards possible and consequently I will discuss item
design in depth in this section. Guidelines for item writing are plentiful (e.g.
Tourangeau et al. 2000; Oppenheim 1992; Dörnyei 2003; Cohen et al. 2000;
In Table 4.1 a number of authors‘ guidelines on item wording are summarised. This
reveals the main areas of concern, which are vocabulary, length of items, simplicity
155
of grammar, leading or loaded questions, ambiguity, vagueness, double-barrelled
items, the middle option in multiple choice answers, and multi-item scales. These
will be discussed here as they relate to the item-authoring for the Long List.
4.2.1.1 Vocabulary
It was important not to make assumptions about which words respondents knew or
gathering feedback on the questions. However, rather than relying entirely on this,
which may not reveal all difficulties with all populations, it was prudent to choose
vocabulary which is not overly technical. Aiken (1997: 37) recommends that
researchers ―always choose the simplest way to say something‖. Converse & Presser
(1986: 15) agree that common concepts rather than abstract ones from the academic
field should be used. They add (1986: 11) that chatty, over familiar language or
4.2.1.2 Shortness
Shorter items are more likely to be understood (Converse & Presser 1986: 11-12;
Dörnyei 2003: 52-53; Gillham 2000: 25). Oppenheim (1992: 128), Dörnyei (2003:
52), and Aiken (1997: 40) recommend no more than 20 words per question. The
average length of the items in the Long List is 9.3 words, a figure well within this
166. I know which sense is best for me to use when learning (i.e. sight, or hearing, or touch,
or physical movement)
167. I select learning techniques (i.e. taking notes, or drawing diagrams, or by listening, etc.)
156
Author Tourangeau et al. Oppenheim (1992: 128-130) Dörnyei (2003: 52-56) Ellard & Rogers Cohen et al. (2000: Converse & Presser
(2000: 61) (1993: 17, from 248-249) (1986)
Dörnyei 2003)
Vocabulary Use simple words, avoid Thou shalt match the Avoid highbrow Use standard English,
acronyms, abbreviations, jargon vocabulary used in questions
and technical terms items to the Use common concepts,
vocabulary of those not abstract concepts
who will respond to from the academic field.
them.
Grammar Avoid complicated Use simple and natural Thou shalt not use
syntax language complex grammatical
forms
Shortness Questions should not be too Aim for short […] items Use short questions when
long possible
Simplicity Keep questions Aim for […] simple Avoid complex Better to ask two or three
simple items questions simple questions rather
than one complex
question
157
Author Tourangeau et al. (2000: 61) Oppenheim (1992: Dörnyei (2003: 52- Ellard & Rogers (1993: 17, Cohen et al. (2000: Converse & Presser
128-130) 56) from Dörnyei 2003) 248-249) (1986)
would not necessarily make them any clearer. The length is due to the provision of
clarification (within brackets) which fits with other guidelines recommending less
vague or ambiguous language. In fact, Bourque & Fielder (1995: 46) make the point
Gillham (2000: 25) recommends avoiding complex constructions, and Converse &
Presser (1986: 16) recommend asking two or three simple questions instead. I used
the first person as much as possible to make the items simple and immediate.
Converse & Presser (1986: 23) and Aiken (1997: 37) recommend asking about actual
experiences rather than using hypothetical items. I aimed to use the present simple
It is frequently stated (Dörnyei 2003: 54; Oppenheim 1992: 128; Converse & Presser
1986: 13) that negative constructions should be avoided as they can confuse the
respondent and are difficult for the researcher to interpret. However, I have retained
some for reasons of clarity, such as item 147. ―I worry if I don‘t understand all the
feedback from the administrations to see whether negative wording did emerge as an
4.2.1.4 Leading/Loading
Leading or loaded questions and vocabulary mean that the item ―indicates the ‗good‘
159
own point of view‖ (Oppenheim 1992: 137). Dörnyei (2003: 54) makes the point that
apparently innocuous words such as ―merely‖, ―modern‖, and ―natural‖ can also
influence a respondent. There were seven items in the questionnaire which may
are shared with the respondent (Oppenheim 1992: 128; Tourangeau et al. 2000: 42).
in natural language questions‖. It becomes a problem when the assumptions are not
shared. The situation here is similar to that with negatives, discussed above, and I
took the same measures to ensure that they did not have a significant
disadvantageous effect.
4.2.1.5 Ambiguity
Definitions are not always shared (Converse & Presser 1986: 18) and in addition
many common words (such as ―week‖, ―dinner‖ etc.) have alternative meanings
(Oppenheim 1992: 129). In fact Cohen et al. (2000: 249) feel that ambiguity cannot
be avoided, rather we can only attempt to minimise it. This is another indication of
the importance of collecting feedback about the items in the Long List. One item
(221) was changed: ―I want to learn in a more Western way‖ was adapted to ―I want
160
to learn in a more learner-centred way‖, though this was due to feedback from tutors
and my fellow students rather than any negative feedback from respondents.
4.2.1.6 Vague/Specific
Converse & Presser (1986: 31) say ―The more general the question, the wider the
range of interpretations it may be given‖. Tourangeau et al. (2000: 61) advocate the
avoidance of vague concepts and the provision of examples when such concepts
must be mentioned. Dörnyei (2003: 54) says words to be avoided are non-specific
frequencies and values (such as ―good‖ and ―often‖), and universals (such as ―all‖, or
―never‖). In the Long List there are some items with such ―vague‖ concepts, for
unnatural and unduly pedantic, I feel, to specify exactly what ―good‖ means; what
matters is that the respondents are clear on the meaning, and that they are able to
Double-barrelled items ask two questions at the same time so that respondents may
not be able to agree or disagree to both parts at the same time. This can result in a
―don‘t know‖ or non-response, or if they do put an answer the researcher will not
know if it applies to one ―barrel‖ or both (Oppenheim 1992: 128; Dörnyei 2003;
Aiken 1997: 37). There are some double-barrelled items in the questionnaire:
44. I know my strong points and weak points (How to respond if I know my weak points but
49. I have changed the way I learn after thinking about it (What if I thought about it but did
not change?)
161
55. I know why I did well or did badly (What if I know why I did well, but not why I did
badly?)
If they are unanswerable or too open to randomness in the responses chosen by the
them since the situations which may lead to difficulty in responding (given in
Dörnyei (2003: 33) recommends four or more items to cover a single point, forming
a multi-item scale, but Oppenheim (1992: 143) argues that ―it is simply not possible
to develop multiple-item scales for everything‖. This is the case with the Long List
stage of the questionnaire. However, (and to conclude this section on item wording
guidelines), Converse & Presser (1986: 10) distinguish four key concepts in item
information. All of these are reflected in the advice which has been discussed in this
section, and so the emphasis of the items in the Long List is on the immediate and
tangible, rather than the vague and hypothetical. There is a degree of judgement and
subjectivity involved at this stage in the development of the questionnaire (and any
questionnaire), but I am well aware of the possible pitfalls, and the use of respondent
In order for the autonomy measuring instrument to be able to gather data from
162
quickly and simply a closed-response questionnaire was indicated. There is a choice
to be made as there are basically four types of closed questions: rank ordering,
multiple choice, dichotomous, and rating scale. I will look at each of these options in
responses (Cohen et al. 2000: 252). This type of question shows the respondents‘
ideas of the relative values of different items (Gillham 2000: 31). There are two
major drawbacks for the present research: it is more difficult to answer than
individual items and respondents may not be able to make the distinctions necessary
(Cohen et al. 2000: 252; Dörnyei 2003: 44; Aiken 1997: 46); and also the results are
not easy to process statistically (Dörnyei 2003: 45). For this reason I decided against
Multiple choice questions are a familiar question type which has a statement
answers. The individual responses do not overlap (Cohen et al. 2000: 251) and they
have the advantage of producing easy to process data (Cohen et al. 2000: 251).
Dichotomous questions are items limited to two possible responses, such as yes/no or
agree/disagree, and do not offer middle options (see Section 4.7.1.1.8). As I hoped to
obtain gradations of response beyond the black or white this method appears
unsuited for a putative autonomy-measuring instrument, and for this reason I decided
The remaining choice was rating scales. The most familiar manifestation of rating
scales is the Likert scale, which is widely used and popular for questionnaires
163
because it is ―simple, versatile, and reliable‖ (Dörnyei 2003: 36). It is a very common
response format, and therefore it should be familiar to the respondents meaning they
are more likely to answer in a reliable way. It also allows for more subtle responses
than dichotomous questions (Cohen et al. 2000: 253). Another advantage is that the
data produced are easy to process. For these reasons I chose the Likert scale for this
Firstly, as with all closed response formats, a disadvantage is that there is no way of
telling if respondents might have wished to add something to their response (Cohen
et al. 2000: 254), and the reasons why a certain option was chosen are not recorded
respondents to find out more about why they answered in the ways they did, and a
teacher using the questionnaire with a class would also be able to do this.
Secondly, there is the question of how many points to include in the scale, and
whether to include middle and ―don't know‖ options. Tourangeau et al. (2000: 248-
249) talk of scale range effects, where the available range of response options can
change beliefs about the question. The presence of an option, such as neutral, can
true that the more options an item contains the more accurate it will be (Dörnyei
2003: 42) it is also the case that respondents often avoid the extremes of the scale
(Tourangeau et al. 2000: 248-249; Gillham 2000: 32; Cohen et al. 2000: 254).
Gillham feels that this makes seven-point scales redundant, while Cohen et al.
actually see larger scales as a way of reducing the effect of avoidance of extremes.
For the present purposes I considered that a wide range of responses should be
offered in order to find gradations which would make it easier to distinguish between
164
responses. At first, a five-point Likert scale was chosen. This was used in the United
Arab Emirates, but then for the subsequent versions I used a seven-point scale to
Having an equal number of options produces a scale with no middle. Converse &
Presser (1986: 36) ask whether offering the middle alternative encourages a non-
committal response which is the easiest for the respondent or provides an additional
gradation of opinion in the data. Cohen et al. (2000: 254) consider that having an odd
number of items is better, as in not forcing a choice one does not hide the
option.
There is much of agreement in the literature that the ―don‘t know‖ option should be
included in rating scales (Muijs 2004:48; Aiken 1997: 45; Converse & Presser 1986:
35; Tourangeau et al. 2000: 43; Oppenheim 1992: 129). Authors point out that in any
questions‖ (Tourangeau et al. 2000: 43). Presuppositions can take the form of
―assumptions not only about the nature of what is to be measured, but also about its
very existence‖ (Converse & Presser 1986: 35). Consequently, it is perfectly possible
that respondents may not be familiar with an element of the question, and so may
really be unable to give a definite answer. It would be tempting to omit the ―don‘t
know‖ choice to constrain a decisive response, but as Oppenheim (1992: 129) asks
―do we really want to obtain ‗forced‘ responses which are virtually meaningless?‖. In
the light of this, I decided to offer a ―don't know‖ option, though this decision was
165
In this section a variety of question/answer types have been discussed. Gillham
(2000: 34) thinks that to maintain respondent interest question types should be
varied. Cohen et al. (2000: 258) recommend including ―questions that are likely to be
of general interest‖ and by mixing attitude and behaviour questions. This was done in
the Long List of items, and feedback from respondents indicated that they found it
Table 10.2 in the Appendix shows all the 256 items in the Long List and gives the
areas relevant to autonomy which they were intended to cover. These are the areas
It is not clear that it is possible to write one item which covers one discrete area, and
it is not clear that the intention of the author can precisely match the understanding of
the reader (see Section 4.2.1). It is also apparent that it is not possible to have four or
more items for each possible dimension of autonomy (as discussed in Section
4.2.1.8). With the help of my supervisors I decided to use a grid (which developed
into Table 10.2) in which I could place items and control the coverage of all the areas
– though it was not possible to show how the items would objectively (i.e. without
I populated the grid with items to form the Long List. To help me with ideas for
Dörnyei (2001), Cotterall‘s (1995) learner beliefs questionnaire, and many others. I
166
also looked at learner training materials, and discussed ideas for items with
supervisors. I then wrote, adapted or borrowed items making sure they followed the
development process from start to finish went from more subjective to more
objective. Compiling the Long List was the most subjective stage, and I therefore
took care to ensure (using the grid) that the items represented the wide range of
specified areas which had been identified in the Literature Review as central to
autonomy for learning. In this way I intended that the subsequent more objective
steps (Short List and then final factor analysis) would be selecting from the widest
possible range of items. Table 4.2 shows the main sections of the thesis where each
Area Section
Social Interaction 2.4.8
Motivation 2.4.7
Responsibility 2.4.6
Actions/Behaviours 2.4.4
Metacognition 2.4.5, 2.5.1.5
Control (capacity for) 2.4.3
Attitudes to learning (beliefs) 2.5.1.3
Confidence 2.5.1.3
Strategies (knowledge and use of strategies) 2.5.1.4
Skills (Areas of Critical thinking, Reading, Writing, Listening, Speaking, 2.5.1.2
Information, Grammar, Vocabulary etc.)
Table 4.2: The general areas for questionnaire items with thesis sections
At the stage of the Long List there were two data gatherings, one in the United Arab
Emirates (UAE), and one via the Internet which was open to all volunteers.
The respondents were 53 Emirati women in their second year at the Higher Colleges
of Technology (HCT) at two campuses, Fujairah Women‘s College and Abu Dhabi
167
Men‘s College (though female students) in the United Arab Emirates (UAE). HCT
is an English-medium tertiary institution. All the students had attended state high
(CEPA test) with a score of 150 or more, which is the level required for university or
Higher Diploma courses at the HCT (UAE Ministry of Higher Education and
Scientific Research 2006). It was assumed that regarding culture and education the
Sample size 54
Sex Male 0
Female 54
The respondents were presented with the full 256 items of the Long List (―Format
A‖). Each was a statement with 5-point Likert scale response, worded ―Strongly
agree‖, ―Agree‖, ―Neither agree nor disagree‖, ―Disagree‖, and ―Strongly disagree‖.
disagree‖. 73 of the 256 items were reverse coded, i.e. where items were thought to
bear a negative relation to autonomy the coding of the responses would be reversed.
Normally-scored and reverse-scored items all had ―Neither agree nor disagree‖ = 3.
Use of symbols instead of words was rejected as it was not certain that they would be
168
The questionnaire was administered in paper format in three classes by the class
teachers during 55 minute English lessons. It was not possible to be present in the
classes, but the teachers informed the class that the questionnaire was a survey and
not a test, and that the results were not going to be part of their course, and that it was
completely anonymous. This was also written at the top of the questionnaire. The
items in the questionnaire were written in both English and Arabic. Most participants
I will now examine all the feedback which I received from respondents. The length
of the questionnaire caused problems: “Can we take it home?”, “It‟s boring”, “We
need breaks”, “It‟s too long”, and queries arose regarding some items:
37 is ―If I find a word that I don‘t know, I always ask a teacher first‖ and 38 is ―If I
find a word that I don‘t know, I look it up in a dictionary first‖. This indicates either
a problem with the translation of these items, or that respondents are not be
- 16
This item was queried: it reads ―When I learn something new I feel good because I
can stop learning it‖. The problem with this was not specified, but it may be a
translation problem.
- 26 “lucky”?
Item 26 is ―I feel lucky when I get good marks‖. I judge this to be a translation
questionnaire.
169
- 62 “What‟s the point of this question? Is it linked to the previous one?
Item 61 is ―I find it difficult to tell facts from opinions when reading‖ and item 62 is
―Sara eats a lot of sweets so she must be fat‖. These are both items aimed at probing
the skill area of critical thinking and are not connected in any other way.
-88 “Did Mr David write the questions himself? I think there should be a comma
here.”
Item 88 is ―If I am not sure about something it bothers me‖. Only one respondent
- 90
Item 90 is ―I learn exclusively about college subjects‖. This was a candidate for
rewording.
Item 105 reads ―When I read in English I think about what the source of the text is‖.
This is probably a subjunctive construction in Arabic. The meaning does not appear
Items 184 ―I like negotiating with other students in class‖ and 185 ―I like class
discussions‖. The distinction is pair or small group work versus whole class
discussions.
The two lowest-scoring items, 26 and 88, are also ones which received negative
feedback. Five of the lowest scoring items were near the end of the instrument where
170
4.3.2 English language Internet data gathering
This version of the questionnaire (Format B) is a web page form. It is essentially the
same as the UAE version, though the previous 5-point Likert scale response format
―don‘t know‖ box was also added. Spaces were provided at the end for item-specific
and general feedback, and a section also asked for background information on age,
postgraduate), years learning English, and time taken to complete the questionnaire.
Sample size 8
Country of origin UK 1*
Colombia 1
Sweden 1
KSA 1
UAE 1
China 1
Switzerland 1
Canada 1
Sex Male 0
Female 8
171
198. I hate to study with less than my best effort. ―It is unclear.‖
17. If I must finish a job at a certain time I finish early. ―The word ‗job‘‖.
62. Sara eats a lot of sweets so she must be fat. ―A trick question?‖.
183. I am happy to use different worksheets from the rest of the class.
The Chinese respondent said ―I think there are some repetitions in the questionnaires.
That is, some questions reflect the same aspect of learning English‖. The Swedish
respondent said:
A general comment: Maybe a comment in the beginning about keeping an eye open for the
tricky questions. It took time to scroll through again to find the ones I hesitated on. Maybe
The first point was not repeated by other respondents, but the second shows
One of the respondents (from the UK, learning Japanese) gave more feedback via
email:
just did your survey! Really made me think as I am just wondering whether to start serious
study of Japanese again. I tend to be plunged into language using situations so far above my
ability level (at work) that it feels like whatever I do wouldn't show any results for a long
time, so I need a lot of energy and consistency but lack it. The survey made me realise I am
illiterate as regards libraries and reference works in Japanese. It was a stage in taking myself
172
This was interesting feedback as it concurs with the idea that an autonomy measuring
Prior to this data gathering the items had been translated into Chinese due to issues
which had emerged with the UAE data gathering (see Section 3.13). This was the
Long List with 7-point Likert scale for responses delivered online (Format C). It can
Sample size 6
Sex Male 2
Female 4
In the feedback section there were comments about perceived repetitions: ―something
has repeated for several times!!‖ and ―There are some repetitions of these
questions‖. One of the respondents contacted me via email and specified items 200,
202, and 219 as having ―similar connotations with the previous questions‖. He also
commented on how long the questionnaire was, and reported a mistake with the
Chinese characters in item 97. He queried item 251 ―I use real English texts (i.e. not
173
made for students) in my learning‖ suggesting it be reworded to ―authentic texts‖.
There were a small number of returns for this data gathering. This can be ascribed to
its length (also reported as a problem in feedback from the other Long List data
Compulsory item completion. When the ―submit‖ button was pressed the
page was automatically checked for unanswered items and if there were any
it showed a message asking the user to complete the questions marked with
web page authoring software). This message was not very prominent and it
is possible that the message was not being noticed and users thought they
had successfully finished and left the page without it actually being
submitted.
The questionnaire took the form of one long page, and this may have caused
All respondents were English majors at Beijing Normal University (BNU) and
The Long List of items translated into Chinese was again used. However, following
the problems with the Internet delivery of the questionnaire in China (see Section
174
4.3.3) the questionnaire was reformatted as a form in a Microsoft Word document.
Another variation on the previous data gatherings was that the items were grouped
with the intention of ―breaking up‖ the list to make the 256-items seem less daunting
to respondents. There were two feedback sections, the first to report individual items
which had caused problems, and the second for general reflections.
The Long List was distributed by email via two local administrators, one of whom
was briefed by me and the other by the first. An incentive was offered (free
umbrellas) to reward the respondents for their time and effort. The administrators
informed the respondents that the questionnaire was a survey and not a test, that the
results were not going to be part of their course, and that it was completely
anonymous.
There was much feedback from this group. 11 respondents said it was repetitious, for
example one said ―I think some of the questions are just repeating one another. There
are some overlapping‖. Three respondents referred to its length, for example:
I think the questionnaire is a little bit long. Anyone who is facing this questionnaire should
have plenty of patience. Yet, it is quite convenient for us to finish, especially using a
Four said it had made them reflect on their learning, for example:
I think the questionnaire is very useful and helpful for me to rethink my learning methords,
I find some questions are too absolute, which makes me hard to decide, such as the use of
always"and so on.
175
and
some of the questions are not so clear and the choices are very general, so it is a little bit
It is interesting that more respondents did not make these points as the questions
were written or selected with an aim of not assuming a specific context. It appears
that most respondents did not have great difficulties answering the items.
Sample size 48
Sex Male 5
Female 42
Not given 1
In this chapter I have examined in some depth the item wording guidelines from the
literature which were followed in order to obtain well-formed items for the Long
List. I have given details of the data gathering which was carried out using the Long
List. In the next chapter I will describe how the data were processed to reduce the
176
5 THE SHORT LIST STAGE
In this chapter I will describe the stage of the research involving the selection of the
Short List of items and its subsequent use for data gathering, combined with the
The Long List stage had not proceeded entirely according to plan. There had been
problems and delays due to difficulties obtaining enough responses to requests for
participants. There had also been at the Long List stage the unforeseen and lengthy
translation process. Time constraints meant that it would not be possible to wait for
more participants. The original timetable and the plan for the research had to be
reconsidered to make something that would still be worthwhile and achievable, but
would be modified. As a result the Short List stage described here is not how it was
1. Item selection by factor analysis was not possible and an alternative method
2. Construct validity checks using factor analysis (see Section 3.8.2) would still
be carried out but would be delayed until the ratio of respondents to items had
been improved by reducing the number of items, and also by amassing more
data.
participate. This would now only take place at the small scale (see Section
5.2).
177
5.1 Item selection process for Short List
The item selection process was based on multiple parallel techniques rather than
relying on one procedure. It was hoped that this would help to preserve the construct
and reduce the chances of an idiosyncratic result. The techniques used where:
Respondents‘ feedback
Range of response
Standard deviation
Polarisation of response
Discrimination index
As previously stated it had been intended to use factor analysis and Cronbach‘s alpha
reliability coefficients, but this resulted in a selection of only 20 items as there were
not enough respondents at this point in the data gathering for these procedures to
function. In meetings with my supervisor it was decided that 20 items were probably
decided to find a different selection procedure which would produce a longer list of
still robust items, but not so long that it would produce respondent fatigue and so I
(for the slower respondents). A figure of 50 items was thought to be a good balance
Item analysis using the Rasch technique (Hughes 2003: 228-233) was also
considered as a way of reducing the number of items, but was rejected as it involved
deciding on an order of ―difficulty‖ of questions which had at this stage not been
empirically established. I left the question of whether any subjective choice of items
178
would be necessary until after the selection procedure had given its results, and then
This selection process did not look at whether items were good indicators of
autonomy or not. The purpose of this process was to look at the qualities of the
coverage of autonomy by the 50 items was important and this was checked, as shown
in Table 5.2, and found to be satisfactory. In Sections 5.1.1 to 5.1.7 which follow I
In all, respondents were unclear or gave negative feedback on 19 items, and these are
shown in Table 5.1. More details of the feedback can be found in the individual
reports of the data gatherings in Sections 4.3.1, 4.3.2, 4.3.3, and 4.3.4. Numbers or
question marks in the group columns indicate how many individuals queried the
item. In the HCT column question marks indicate that the class teachers did not
record how many respondents queried the item. The BNU column is blank because,
although two respondents did say that two pairs of items seemed too similar, it was
not possible to identify clearly to which items they were referring. Most respondents
in all data gatherings did not mention any items as problematic. There were two
items, 62 and 198, which received negative feedback from respondents in different
groups which makes them the stronger candidates for rejection by the respondents‘
feedback criterion.
179
No. Item Respondent Groups
HCT English Chinese BNU
Internet Internet
16 When I learn something new I feel good ?
because I can stop learning it?
17 If I must finish a job at a certain time I 1
finish early?
26 I feel lucky when I get good marks. ?
desirable for the instrument as the measurement of autonomy requires that an item
can detect differences among respondents. The range of items was expressed as the
180
number of different options on the Likert scale which had been selected by more than
x per cent of the respondents, where x is a figure that enables a ranking order of
items to be produced. Initially it was set at x=2.1%, but this did not produce a useful
ranking and it was raised to 10%. The results were added to an Excel spread sheet of
all the items, the table was sorted by this criterion, and then the top 50 items were
indicated with shaded boxes. Appendix 10.4 shows a table covering the complete
selection process. The Range of Response criterion figures can be seen in the column
labelled A.
Questionnaire items with a larger standard deviation will be more useful for the
responses for an item. A wider spread of responses will permit more discrimination
between the individuals in the group answering the questionnaire. The standard
deviations for each item were added to the selection table (Appendix 10.4) in the
column labelled C, and the table was then sorted by this criterion and the top 50
respondents. The normal bell curve distribution has most respondents in the middle
of the range, whereas a polarised distribution has most respondents occupying the
extremes of the range with few in the middle. This is desirable for the instrument as
its items need be able to detect differences among respondents in order to measure
autonomy. Column B in Appendix 10.4 shows the polarities in the selection table.
The polarity of the items was calculated by finding items where the middle Likert
181
option had fewer responses than the surrounding options. This difference was
quantified as the difference between the middle option and the closest of the
neighbouring options. The table was sorted in Excel by this criterion and the top 50
items were indicated with a shaded box. The selection table shows that the
respondents had not favoured the extremes of the Likert scale. This indicates that
they had not treated it as a test but rather, as advised, they had treated it as a survey.
The sum of the figures for each item in columns A, B, and C was added together and
the total added to the table in column D. The table was sorted by this criterion and
again the top 50 were indicated with a shaded box. Finally, the number of shaded
boxes for each item was put in column E and the table sorted by this figure and,
secondarily, by standard deviation as many items had the same number of shaded
boxes. The resulting table had 75 items with at least one shaded box. This ranking
section.
between higher and lower performing respondents in tests (Hughes 2003: 226-228).
If an item does not correlate with the overall result indicated by the instrument it is
While the autonomy measuring instrument was not envisaged as a test this technique
182
since the DI looks at the relation of a single item to all the items. The procedure for
All respondents were given a figure by calculating the total coded score for all their
answers. The respondents were then ranked high to low. This set was divided into
two sets, the respondents with higher overall scores and those with lower overall
scores. Next individual items were addressed: the total for each item was summed
within the higher group and separately within the lower group. Item totals within the
higher group should be greater than the same items totalled in the lower group. The
larger the difference the more effectively the item has discriminated between the
groups. If an item has a similar score in both groups it has not performed well in
addresses a point which does not vary with levels of autonomy, or because it was in
―best‖ answer). Items which had a large difference between the scores in the higher
and lower groups were selected for the new questionnaire. Items which had not
discriminated would be less effective items and would be candidates for removal. I
now had two lists of items ranked in order of decreasing effectiveness by their
response.
In the following section I will show how these were combined and explain my
rationale.
183
5.1.7 Combined selection procedure
The two lists were now combined, taking the highest ranking items from each list so
as to obtain a final selection of 50 items. The rationale for this was that the two lists
1. All of the top 50 items are identical between the lists (the selection would
be straightforward)
2. None of the top 50 items are shared between the lists (it would be simply
3. The two lists share some of their top 50 items (the actual situation)
Case 3 was the actual situation as many of the items appeared in both lists (which
was encouraging because it confirmed their selection), but there were also many
One option which was considered was to take only the common items and to do this
look further down the rankings beyond the top 50, perhaps to 75 or 100 or beyond.
This would ensure that all items had been selected by both criteria. However, the
problem which was found with this approach was that since all the items are
common in the full lists (since they are both composed of the same items though in
different orders) it meant that an item high in list A may be very low in list B and
item it may match at a much lower level in list A. It would therefore be necessary to
find a way to balance the two lists in a manner which, as much as possible, favoured
the selection of items which were highly ranked in both lists. For this reason I
decided to favour the highly ranked items rather than relying solely on the matching
184
items. This meant choosing the top x items from each list, then combining these two
lists. The items which occurred in both lists therefore had two instances in the
combined list, one from each of the initial two lists. For each item which had two
instances one instance was removed, and the total number of remaining items in the
combined list was summed. If the total was less than 50 the process was repeated
with the top x+1 items from both lists (i.e. moving one step down the ranking). If the
total was more than 50 the process was repeated with the top x-1 items from both
lists (i.e. moving one step up the ranking). The process was repeated until a value of
x was found which produced the desired 50 items. The diagram below (Figure 5.1)
The 50 items which resulted from this process were checked to see whether they
were items which had undergone any rewording as a result of respondent feedback
which may have changed their response qualities due to changes of meaning or
connotation. If there had been such changes it would have made it problematic to
combine the data from the item responses in all the data gatherings. All the 50 items
were found to be identical except for item 76 which had originally read ―I rely on the
teacher‖ but had been changed to ―I rely on the teacher when learning‖. Item 76 had
been changed as a result of feedback from one respondent in the English language
Internet data gathering (see Section 4.3.2), but this change was not regarded as
was taken.
185
LIST A LIST B
N = x Items N = x Items
1 1
6 4
21 21
..... .....
Combine Lists
A+B
1
1
4
6
21
21
.....
Remove repetitions
A+B
1
4
6
21
.....
If N = 50 If N =>50 If N = <50
186
5.1.8 Items in the Short List
Table 5.2 (overleaf) shows the items selected for the Short List with the areas of
autonomy which they cover. In order to show (in terms of representative coverage of
the autonomy areas) that the Long List and the Short List were broadly similar, Table
This new version of the questionnaire, Format E (in Chinese) and Format F (in
http://www2.warwick.ac.uk/fac/soc/al/research/groups/llp/circal/omall
The 50 items are arranged into face valid groupings to ―break up‖ the questionnaire
and to give a sense of logical organisation (see Appendices Section 10.3). The
rationale for breaking up and grouping items in a questionnaire has been discussed
It had been intended (as described in Section 3.7.1.1.3) to conduct large scale
List and observations and estimates of autonomy made by the class teachers,
followed by interviews. The rationale for this had been the quantitative validation of
be investigated. In the event, the large scale research was not possible because of
187
Attitudes to learning
Actions/Behaviours
Social Interaction
Metacognition
Responsibility
Vocabulary
Confidence
Motivation
Grammar
Strategies
Listening
Speaking
Reading
Writing
Control
Skills
Item
049. I have changed the way I learn after thinking about it. x x x x x x x x x
188
Attitudes to learning
Actions/Behaviours
Social Interaction
Metacognition
Responsibility
Vocabulary
Confidence
Motivation
Grammar
Strategies
Listening
Speaking
Reading
Writing
Control
Skills
Item
189
Attitudes to learning
Actions/Behaviours
Social Interaction
Metacognition
Responsibility
Vocabulary
Confidence
Motivation
Grammar
Strategies
Listening
Speaking
Reading
Writing
Control
Skills
Item
190
Attitudes to learning
Actions/Behaviours
Social Interaction
Metacognition
Responsibility
Vocabulary
Confidence
Motivation
Grammar
Strategies
Listening
Speaking
Reading
Writing
Control
Skills
Item
191
Reading
Writing
Listening
Speaking
Grammar
Vocabulary
Attitudes to learning
Social Interaction
Motivation
Confidence
Responsibility
Actions/Behaviours
Strategies
Metacognition
Control
Skills
Long List N 24 14 6 15 22 35 194 47 127 157 114 82 142 144 152 90
(*256) % 9% 5% 2% 6% 9% 14% 76% 18% 50% 61% 45% 32% 55% 56% 59% 35%
Short List N 9 3 4 3 3 9 35 8 22 37 16 16 27 27 31 14
(*50) % 18% 6% 8% 6% 6% 18% 70% 16% 44% 74% 32% 32% 54% 54% 62% 28%
*= Number of items in list. N= the number of items which refer to the area. % = N expressed as a percentage
Table 5.3: Comparison of coverage of the areas of autonomy in the Long List and Short List
192
The smaller scale research took place with two classes. Both classes were for the
one was at third year undergraduate level (see Section 5.3 for a description of the
class) and one was a presessional course (see Section 5.4 for a description of the
class).
The procedures for gathering the data in the two classes were as follows. For the
3. Estimates of autonomy
Ideally for the research there would have been stages prior to this where the students
answered the questionnaire and the teacher, with only first impressions of the
students, made estimates of their autonomy. This was not possible as the class came
to my research only after the course had already started. At step 4 volunteers were
requested but (despite incentives) only one volunteer attended for interview.
For the ELTCS presessional group the data gathering procedure was:
1. The teacher (myself) estimates the levels of autonomy of the students based
2. Students complete the questionnaire on-line (but the data are stored only and
not analysed)
193
4. Second estimates of autonomy at the end of the course. On the same day …
A positive result in the comparison of questionnaire data and estimates gained from
instrument such as the one used in the present research was viable and could
potentially serve a useful function in similar contexts. In the event the necessary
level of response did not materialise and the data gathering was limited to the small
scale. At this small scale, if the questionnaire did match the estimates it would be
suggestive only, because significant correlations are hard to demonstrate with small
samples. The reasons why comparison was nevertheless carried out at this small
scale were to gather qualitative data, and for the purpose of gathering quantitative
data to demonstrate the principle and techniques which had been envisaged for the
large scale research. A good result for this part of the research would be a strong
suggestion that the instrument (or the principle of such an instrument) merited further
research.
I decided to use teacher estimates of autonomy for comparison with the questionnaire
data because estimates are what I have always used to gauge my students‘ levels of
autonomy. Estimates are the de facto way of making low stakes decisions because
they do not require the commitment of large amounts of time and resources. They are
what the autonomy measuring instrument seeks to match or improve upon and are
194
therefore a source of data which is appropriate for the purposes of the present
research.
normal course. In fact both classes were normal courses and not especially set up for
research purposes; this was so that data gathering would stay close to the aim of the
research i.e. to reflect how estimates are actually made. For this reason I did not
specify in advance what should be observed – it was clear to me that this would have
defeated the idea of using estimates; it would have been, in effect, another untried
and untested procedure for measuring autonomy and would therefore have been
in terms of experience, will probably make their estimates based on their own
understandings of autonomy. The two teachers (myself and the other teacher)
would not be investigating how teachers make estimates (though this would be an
interesting area for separate research). It was not necessary to know this to find
whether the questionnaire produced results which correlated well with teacher
estimates. However, for qualitative putting of flesh on bones I intended to ask the
ELTCS year-3 teacher during interview how she decided on the estimates with the
question ―Are there any features which identify the more and less autonomous
students?‖.
195
5.2.2 Student interview
There were two aims for carrying out student interviews in the small-scale data
collection stage of the research. Firstly, I hoped to be able to probe more deeply the
respondents‘ questionnaire answers and so gain insights beyond the limitations of the
Likert scale. I hoped that interviewees would be able to give more reasons for their
answers and so illustrate to me whether and how the questionnaire was functioning.
questionnaire and the wording of the items. I was interested in how they had
items had confused them and whether they had answered in a general way or with a
specific personal context in mind. Additionally, I wanted to find more about how the
Likert scale had functioned, for example why they had respond ―strongly agree‖
The above aims indicated using the questionnaire itself as the basis of the interview,
instrument, item by item, with the interviewee and probing more deeply where
interesting points arose. The questionnaire is designed for the general context of
tertiary level learners of English and is not aimed specifically at a single specific
group. This means that it would be useful to obtain responses from the interviewees
which provided more detailed, particular, and concrete information than the abstract
and decontextualized data from the autonomy measuring instrument (Richards 2003:
53).
196
There were time constraints as potential respondents were in the final term of the
year with exams due shortly. I therefore proposed a maximum of 1 hour for each
interview.
The interview guide (below) I drew up was intended to allow for opportunities for
comparisons between the questionnaire and the interview. A key element of the
In general how would you characterise the level of autonomy of the ELTCS
students?
Would you say that some students are more autonomous than others?
Are there any features which identify the more and less autonomous
students?
estimate their levels of autonomy so that I can see if there is any correlation
In general do you think they are socially confident? (For example, whether
students worry what other students will think about their accent, etc.)
or bottom-up way?
In general are they able to find the information they need either in books
197
Are they good at making their own choices?
Do you think they analyse input, i.e. do they notice the way people speak or
The questionnaire was the 50-item web page version in Mandarin Chinese with 7-
Sample size 10
Sex Male 2
Female 8
This respondent group was involved in the BA in English Language, Translation and
Cultural Studies course (ELTCS). This is a four-year course, the first two of which
are at Renmin University in China where the students study English, translation, and
198
Chinese and British culture. Then the following two years are spent at Warwick
University where they do cultural studies, translation, and English language and
linguistics. The participants were 12 third year students on the ELTCS BA nearing
the end of their first year at Warwick University. Their English language level was
the equivalent of the University of Warwick English Language Test grade BBB.
language problems in their essays. The introduction to the questionnaire informed the
students that the results were not part of their course and that it would be
confidential, and the class teacher informed the respondents that the questionnaire
some of the respondents and all were invited though only one student (known here as
This group, as the previous one, was involved in the ELTCS BA course. The students
first year in the UK. I was their teacher for the 5-week presessional course. I made a
more detailed data gathering with this group. On day 1 I initially estimated the
students‘ levels of autonomy and noted them down. They then completed the 50-item
questionnaire was a survey and not a test, that the results were not going to be part of
their course, and that it was completely anonymous. Details of the sample are given
in Table 5.5.
199
Sample size 10
Sex Male 7
Female 3
For the next five weeks I taught the students and was also observing and making
notes on them for their end of course reports, including notes on their apparent
autonomy. In week 5 I estimated the students‘ levels of autonomy for a second time
and noted them down, had the students complete the questionnaire a second time,
I took the opportunity while teaching on the ELTCS presessional to gather data from
standardised and added to the total data set to improve the factor analysis process. As
can be seen from Table 5.6 most were East Asian learners in tertiary education. Their
teachers informed the respondents that the questionnaire was a survey and not a test,
that the results were not going to be part of their course, and that it was completely
anonymous.
200
Sample size 35
Sex Male 15
Female 20
The 14 respondents in this data gathering were all studying English at Fu-Jen
Catholic University, Taiwan. As can be seen from Table 5.7 all were of Taiwanese
background. The sample consisted of three males, nine females and two were not
specified. Respondents completed the questionnaire online in class and the average
time for completion was 27.5 minutes. The questionnaire was the 50-item web-based
Mandarin Chinese version with 7-point Likert scale and feedback section (i.e. Format
E). The introduction to the questionnaire informed the students that the results were
not part of their course and that it would be confidential, and the class teacher
informed the respondents that the questionnaire was a survey and not a test. It was
not possible to interview any of the respondents. The data were standardised and
added to the total data set to improve the factor analysis process.
201
Sample size 14
Sex Male 3
Female 9
Unspecified 2
5.7 Ethics
Punch (2005: 277-278) gives 11 relevant questions to ask oneself, based on Miles &
Huberman (1994: 290-297), before, during, and after research, which can be
confidentiality of data‖ Punch (2005: 277). Warwick University has published ethics
www2.warwick.ac.uk/fac/soc/pais/staff/page/administration/phdguidelinesweb1.doc.
The issues of ethics which were dealt with in my research were addressed in four
interviewees) from any harm as a result of their research‖. No harm was expected as
actual benefits to the respondents in the form of increased awareness and reflection
on learning.
202
Regarding consent, the Warwick University ethics guidelines say ―Researchers
should carry out investigations, interviews etc. with the informed, and prior, consent
of participants‖ and Punch (2005: 277) recommends asking oneself the question ―Do
the people I am studying have full information about what the study will involve?‖.
learners learn and of learner attitudes to learning and did not discuss concepts of
autonomy, or use the word autonomy within the items. I was aware that, as Richards
(2003: 140) says, there is a ―fine dividing line between limited description and
deliberate deception‖ but I felt it was important to elicit data from respondents who
had not been influenced by me so that they would remain in that respect more
institutions were involved informed consent was sought from the representative of
the institution, in this case the data gatherings at the Higher Colleges of Technology,
―Confidentiality of participants and their data must be respected. Details that would
not involved in the research project unless explicit consent is given‖. All data were
anonymised and the respondents were informed that their contributions would be
were also informed that the questionnaire and its data were not part of their course.
203
5.8 Limitations
In this section I will summarise the constraints which I have identified in the course
of the three methodology-related chapters (see especially Table 3.8). There were
three unforeseen sources of delay which forced the original plan for the research to
be modified. Firstly, there was the necessity of translating the items, and then the
discovery that the casual translation method which had been adopted was not
adequate and would require much time and organisation to put right. Secondly, the
response rate to the questionnaire was low. This was partly due to the number of
items to be responded to (as indicated by feedback such as that in Section 4.3.1), and
partly due to Internet access problems in China (see Sections 4.3.3 and 4.3.4). The
insufficient ratio of respondents to items for factor analysis (see Section 5.1).
2. It was not possible to organise large scale data gathering, and so the
results was not possible, though small scale comparison went ahead (see
Section 5.2).
3. Delayed interviews now coincided with end of year assignments so only one
the questionnaire to permit factor analysis for construct validity checking (see
Section 3.12.1).
204
5.9 Conclusion
At the conclusion of this stage in the research a total of 185 useable responses to the
50 items had been gathered. Observations and autonomy estimates had been carried
out for the two ELTCS groups, and one student and one teacher had been
interviewed. In the following chapter I present the findings of the analysis of the data
and provide a discussion of the issues and questions which arise such as how
significant the data are, and the implications of the restricted data gathering.
205
6 DATA ANALYSIS
6.1 Introduction
In this chapter I will present the different data analyses and results. I will distinguish
large scale and small scale data. I will use the larger scale data to look at the validity
of the instrument using factor analysis to find the autonomy-related model which is
embodied in the questionnaire items and which it is necessary to examine to see how
much validity and reliability it has. At this larger scale (standardised accumulated
statistically significant data which makes this an important step. In order to do this I
Checking to establish whether the model reflects the areas found in the Short
List or is novel.
At the smaller scale I will continue to examine the questionnaire using quantitative
Seeing whether there are correlations between the questionnaire results and
the two teachers‘ estimates (i.e. mine and the ELTCS year-3 teacher‘s) in
Seeing whether there are correlations between the questionnaire and the
206
Comparing two students‘ questionnaire results with more qualitative data to
see whether the questionnaire corresponds with the qualitative data, and to
The data gathered and analysed here will be useful for exploring the questionnaire‘s
viability and credibility at the level of individual learners and teachers. At this level
the quantified data will not be treated as statistically significant but have a purpose in
In the light of the above small and larger scale investigations, I will consider the
validity and reliability of the instrument‘s data and consider how much it is
In this section I will present the analysis of the larger scale data gathered using 185
pooled and standardised responses to the questionnaire. I will describe the process of
factor analysis which was adopted and the groupings which emerged. This picture of
After all data gatherings were complete the questionnaire data for the 50 items were
207
idiosyncratic results. The procedure I adopted called for 12 separate factor analyses
as I would use the two recommended extraction methods (maximum likelihood and
principal component analysis) and the two recommended rotation methods (oblimin
and varimax) with the three possible alternatives for the number of factors to extract
which had been found by examining the scree plot of eigenvalues. The break of slope
extract. I would run the analysis for each of these. Table 6.1 below shows these 12
procedures and gives each a reference number. These were then checked for low-
loading and cross-loading items, and for factors with insufficient items for
An example of one of these 12 runs (number eight in Table 6.1) is included here to
illustrate the stages in the process. The factor matrix with loadings is given in Table
6.2.
208
Scale Items 1 2 3 4 5 6 7 8 9
(*CA)
1. 046. I can describe the learning strategies I use. .624
(.773) 109. I predict the content before I listen. .608
049. I have changed the way I learn after thinking about it. .535
138. I know techniques to help me remember vocabulary. .456 -.325
237. I am an active dynamic person. .455 -.311
130. My writing is better now than it was a year ago. .424 .398
142. I fix my problems in vocabulary. .419 -.388
212. I talk to others about how I feel about learning English. .403
234. It is my job to check my work for mistakes. .403
055. I know why I did well or did badly.
2. 147. I worry if I don‘t understand all the words in a text. (R) .768
(.702) 112. I worry if I don‘t understand everything when I listen. (R) .653 -.305
150. I worry if I don‘t understand all the grammar in a text. (R) .612
086. When I read an English text I need to understand every word in it. (R) .603
110. Every word is important for understanding a listening text. (R) .375
026. I feel lucky when I get good marks. (R) .349 -.348
3. 256. I know how to find the information I need on the Internet. .710
(.731) 196. I am confident I can learn English well. .674
254. I know the parts of a book (index, glossary, contents, chapters). .628
253. I know how to use English language reference books (encyclopedias, dictionaries,
-.388 .450 .444
etc.).
175. I look at causes and effects logically. .428 -.349
252. I know how to find information in a library. .411 .367
125. I change the way I write according to who will read it. .405
209
Scale
(*CA) Items 1 2 3 4 5 6 7 8 9
5. 017. If I must finish a job at a certain time I finish early -.581
(.100) 111. The last time I listened to English I tried to predict the content. .560 .312
203. I organise my time for studying. -.358 -.449
100. Last time I read an English text I predicted the content of it. .403
023. Students should always do what their teacher says. (R). .325
076. I rely on the teacher when learning. (R)
7. 194. The other students are more confident than me at speaking English. (R) -.764
(.718) 193. The other students know English better than me. (R) -.733
187. I think learning English is more difficult for me than for the average learner. (R) .328 -.466
195. I worry that other students will laugh at me when I speak English. (R) -.342 .377 -.443
8. 153. Learning grammar is a talent some people have, but not all. (R) .588
(.099) 021. All lessons are equally valuable (R) .458
210
Summarised in Table 6.3 below is the procedure followed for accepting/rejecting
items (showing the reasons) both by Cronbach‘s alpha and from the indications
5 All items are below Items 111 and 100 Item 76 is low- Reject whole scale
.3 would improve scale loading
if deleted
6 All items are below - - Keep all items
.3, but above .2
7 - Item 195 would - Item 195 rejected
improve scale if
deleted
8 Both items are well - Only two items Reject whole scale
below .3 loading on this factor
9 Items 30 and 189 are - Item 246 is cross- Keep items 30 and
both below .3 but are loading 189
otherwise strong Item 246 rejected
Table 6.3: The rejected items and scales with reasons for exclusion
The groupings which remained are shown in Table 6.4, below with suggested factor
names. Similar processing was carried out for the other 11 runs. Since all the 12 runs
did not agree precisely on the composition of the final factors they were combined.
This was achieved by selecting the most frequently occurring items in each factor
211
Scale Items Loading Factors
1 046. I can describe the learning strategies I use. .624 Metacognition
109. I predict the content before I listen. .608
049. I have changed the way I learn after thinking about it. .535
138. I know techniques to help me remember vocab‘. .456
237. I am an active dynamic person. .455
130. My writing is better now than it was a year ago. .424
142. I fix my problems in vocabulary. .419
212. I talk to others about how I feel about learning English .403
234. It is my job to check my work for mistakes. .403
2 147. I worry if I don‘t und‘ all the words in a text. (R) .768 Linguistic
112. I worry if I don‘t und‘ everything when I listen. (R) .653 Confidence
150. I worry if I don‘t understand all the grammar in a text. .612
(R)
086. When I read an English text I need to understand every .603
word in it. (R)
110. Every word is important for understanding a listening .375
text. (R)
026. I feel lucky when I get good marks. (R) .349
3 256. I know how to find the information I need on the .710 Information
Internet. Literacy
196. I am confident I can learn English well. .674
254. I know the parts of a book (index, glossary, contents, .628
chapters).
253. I know how to use English language reference books .450
(encyclopedias, dictionaries, etc.).
175. I look at causes and effects logically. .428
252. I know how to find information in a library. .411
125. I change the way I write according to who will read it. .405
6 194. The other students are more confident than me at -.764 Social
speaking English. (R) Comparison
193. The other students know English better than me. (R) -.733
187. I think learning English is more difficult for me than for -.466
the average learner. (R)
212
Group Items 1 2 3 4 5 6 7 8 9 10 11 12 Selection (N)
1 150. I worry if I don‘t understand all the grammar in a text. (R) x x x x x x x x x x x 11
147. I worry if I don‘t understand all the words in a text. (R) x x x x x x x x x x x 11
112. I worry if I don‘t understand everything when I listen. (R) x x x x x x x x x x x 11
086. When I read an English text I need to understand every word in it. (R) x x x x x x x 7
110. Every word is important for understanding a listening text. (R) x x x x 4
153. Learning grammar is a talent some people have, but not all. (R) x x 2
3 194. The other students are more confident than me at speaking English. (R) x x x x x x x x x 9
193. The other students know English better than me. (R) x x x x x x x x x 9
187. I think learning English is more difficult for me than for the average learner. (R) x x x x x x x x x 9
195. I worry that other students will laugh at me when I speak English. (R) x x 2
189. I learn English because I have to. (R) x x 2
213
Group Items 1 2 3 4 5 6 7 8 9 10 11 12 Selection (N)
5 237. I am an active dynamic person. x x x x x x 6
234. It is my job to check my work for mistakes. x x x x x x 6
212. I talk to others about how I feel about learning English. x x x x x x 6
138. I know techniques to help me remember vocabulary. x x x x x x 6
109. I predict the content before I listen. x x x x x x 6
049. I have changed the way I learn after thinking about it. x x x x x x 6
046. I can describe the learning strategies I use. x x x x x x 6
142. I fix my problems in vocabulary. x x x x 4
130. My writing is better now than it was a year ago. x x x x 4
238. I choose the exercises I work on. x x x 3
236. I am good at making choices. x x x 3
196. I am confident I can learn English well. x x x 3
175. I look at causes and effects logically. x x x 3
125. I change the way I write according to who will read it. x x x 3
251. I use real English texts (i.e. not made for students) in my learning. x x 2
231. I can study independently. x 1
229. I can choose the method of learning that suits me best. x x x 3
203. I organise my time for studying. x x 2
055. I know why I did well or did badly. x x 2
214
Group Items 1 2 3 4 5 6 7 8 9 10 11 12 Selection (N)
6 cont. 142. I fix my problems in vocabulary. x 1
138. I know techniques to help me remember vocabulary. x 1
109. I predict the content before I listen. x 1
095. To read you must proceed word by word. (R) x 1
049. I have changed the way I learn after thinking about it. x 1
215
Group Items 1 2 3 4 5 6 7 8 9 10 11 12 Selection (N)
9 021. All lessons are equally valuable (R) x x 2
153. Learning grammar is a talent some people have, but not all. (R) x 1
140. To remember vocabulary you need to be talented. (R) x 1
11 251. I use real English texts (i.e. not made for students) in my learning. x x 2
243. I decide what I need to read. x x 2
238. I choose the exercises I work on. x x 2
12 195. I worry that other students will laugh at me when I speak English. (R) x 1
111. The last time I listened to English I tried to predict the content. x 1
017. If I must finish a job at a certain time I finish early x 1
216
In the 12 analyses some groupings occurred repeatedly, composed of substantially
the same items. The groupings in the table (Table 6.5) are arranged in order of their
analyses. Within the groupings the individual items are arranged in descending order
of frequency. This arrangement made it possible to find the most prevalent groupings
with the most robust compositions. In total it can be seen that there were 13
groupings, but below group 6 the groupings are composed of items which had
already occurred more frequently in preceding groups (items indicated in Table 6.5
with the ―Not acceptable‖ hatching). This analysis therefore produced six groups
which were selected. The reliability of these scales was checked by Cronbach‘s
alpha. Two items were removed: item 153 (to improve the Cronbach‘s alpha of
group 1, and because its corrected item total correlation was below .3), and item 195
In Table 6.6 below I summarise the process of identifying the underlying connection
between the items in each group. The steps in the process have already been
presented in Section 3.12.3.3.4. Firstly, as shown in the column ―Key Words‖, I took
or interpolated the key words or concepts contained in the items. Then, as seen in the
interpretation which would fit with the key words and the sense of the items. Finally,
as seen in the column ―Factor‖, I attempted to choose one idea which would
217
Group Items Key Words Interpretations Factor
(*CA)
1 150. I worry if I don‘t understand all the grammar in a text. (R) worry (confidence) Confidence in approach Linguistic
(.712) 147. I worry if I don‘t understand all the words in a text. (R) text to texts Confidence
112. I worry if I don‘t understand everything when I listen. (R) understand Tolerance of ambiguity
086. When I read an English text I need to understand every word in it. (R) every word, everything, all
110. Every word is important for understanding a listening text. (R) grammar
receptive skills
2 254. I know the parts of a book (index, glossary, contents, chapters). know Familiar with how Information
(.749) 253. I know how to use English language reference books [..] book parts information is stored Literacy
256. I know how to find the information I need on the Internet. find information and able to retrieve
252. I know how to find information in a library. internet information
251. I use real English texts (i.e. not made for students) in my learning. library
175. I look at causes and effects logically. authentic text
196. I am confident I can learn English well.
125. I change the way I write according to who will read it.
3 194. The other students are more confident than me at speaking English. (R) other students Learners‘ comparing Social
(.718) 193. The other students know English better than me. (R) English themselves with Comparison
187. I think learning English is more difficult for me than for the average learner. (R) confident perceptions of other
the average learner students‘
4 230. My way of learning will never change. (R) never Being in control; having Locus of
(.642) 140. To remember vocabulary you need to be talented. (R) change power , or being Control
246. Memorizing answers is the best way to learn. (R) need to powerless
189. I learn English because I have to. (R) talented
095. To read you must proceed word by word. (R) memorizing
have to
must
word by word
218
Group Items Key Words Interpretations Factor
(*CA)
5 237. I am an active dynamic person. Active dynamic Actively and Metacognition
(.764) 234. It is my job to check my work for mistakes. check work consciously using
212. I talk to others about how I feel about learning English. talk to others techniques and
138. I know techniques to help me remember vocabulary. feel strategies to aid
109. I predict the content before I listen. techniques learning,
049. I have changed the way I learn after thinking about it. predict
046. I can describe the learning strategies I use. thinking about [learning]
142. I fix my problems in vocabulary. learning strategies
130. My writing is better now than it was a year ago. fix problems
238. I choose the exercises I work on. better now
choose exercises
219
6.2.3 Groupings emerging from the factor analysis process
I shall look at the individual factor groupings in the following paragraphs, and
explain my analysis of each one. These groupings will be discussed further in the
All the items involve the idea of approaching the ―understanding‖ of a text from a
building-block or atomistic level. The first three items all use the word ―worry‖,
target language text. It suggests an anxious learner who is not tolerating ambiguity
However, it must be remembered that the Likert scale registers from agreement to
disagreement. The poles are security-insecurity, and this suggests that the underlying
connection is (level of) confidence with regard to target language text, hence the
label Linguistic Confidence. It is the strongest category to emerge from the selection
process (see Table 6.5) in the sense that it is the most consistently represented across
This is the second strongest category to emerge from the selection process, and it
relates to information literacy skills, i.e. the ability to retrieve information from
online and printed sources. The items when arranged in order of frequency (see
Table 6.5) show that the first four items all deal directly with using sources to find
information. The fifth-ranked item (item 251) deals with using authentic texts, a
subject which has a superordinate connection with the idea of finding information in
220
English language sources. The items lower in the category have less specific
relevance to the central idea, but could still have a link in that the logic of cause and
effect (item 175), and confidence with regard to learning (item 196) are both, if not
absolutely necessary, at least useful supports for information literacy. The final item
(125) shows an awareness of different types of text which, again, is a useful skill to
It was immediately clear that this group involved a relation or comparison to other
people, i.e. ―other students‖ (194, 193) and ―the average learner‖ (187). One of the
items, 194, specifically uses the word ―confident‖; it is the highest loading item and
so suggests that this factor involves self-belief based on comparisons with others.
I see this category as being linked by ideas of power and feeling in control. The
bestowed on a lucky few (item 140), but perhaps as a potential for success which can
feeling at liberty to try. Item 246 does not specifically follow in the theme of power,
and inflexibility in learning, and therefore of the opposing ideas of trying (and being
able to try) new ways of learning, of being able to be flexible, and of being in control
rather than a passive subject. Item 95 is akin to 246 in the sense that the respondent
who agrees with the item will see the text as being in control; the text has to be
followed in an externally preordained linear way rather than in the way one chooses
to approach it for oneself. The expressions ―need to‖, ―have to‖, and ―must‖ in items
221
140, 189, and 95 all indicate a control which, though not necessarily external, is
This category seems to be showing an element which deals with learners being
empowered and having a consequent broader range and outlook on what is possible,
and so gives them the sense of learning being within their own control. For this
reason I have labelled this category Locus of Control. It corresponds well with the
The items in this group (see Table 6.6) appear to be united by learners making active
and conscious use of techniques and strategies to aid their learning. Learning
strategies and techniques are evident in items 138, 46, 212, and 109. Reflecting on
learning is shown in items 49, 46 and 130. Awareness of reflection can be seen in
items 49 and 46. The active engagement with learning combined with reflection
which is clear in this grouping suggests that this category is associated with learner
metacognition.
Items 231 and 8 both deal with independent study, while the use of ―I can‖ (items
231 and 229) indicates ability, and the expressions ―I am good at‖ (items 8 and 236)
combined with item 220 ―I am ready to learn in unfamiliar ways‖ indicate a positive
assessment of the learners own abilities regarding learning. Items 236 and 229 both
deal with making choices, and possibly also item 203 suggests decision making. Item
220 underlines readiness to learn in different ways, and similarly 205 suggests active
222
characterised by qualities related to the individual‘s sense of ability and self-reliance
independence.
its factors with the groupings found in Cotterall‘s (1995) work, hers being the most
readily comparable approach available. In the Discussion Chapter (Section 7.4) I will
discuss the questionnaire model with the wider literature of autonomy for language
learning.
Factor Groupings
Cotterall 1995 Present Research
A 1. Role of the teacher 4. Locus of Control
B 2. Role of feedback 4. Locus of Control?
C 3. Learner independence 6. Self-Reliance
D 4. Learner confidence in study ability 1. Linguistic Confidence
E 5. Experience of language learning 5. Metacognition
F (6. Approach to studying) -
G - 2. Information Literacy
H - 3. Social Comparison
Table 6.7: Possible matches of questionnaire factor groupings with Cotterall (1995)
Cotterall‘s (1995) study (see Section 2.5.1.3) produced six factor groupings.
Comparison of these groupings with those produced by the present research reveals a
reasonably close match (see Table 6.7 above). Common areas are control (rows A
and B), independence (row C), and learning confidence (row D). Cotterall‘s
―Approach to studying‖, and in fact in a later paper (Cotterall 1999) she discards this
(Cotterall 1999: 498). This leaves the questionnaire‘s categories 2 and 3 without
223
matches. I will look at each of Cotterall‘s five factors and compare them with the
questionnaire‘s equivalents.
teacher‖ factor. Her items are specifically couched in terms of the role of the teacher
rather than a wider spectrum of possible sources of perceived control. The items in
the questionnaire‘s Locus of Control category do not address teacher role but are a
more varied collection of control-related items. This difference may reflect differing
approaches to item selection between those used for the present questionnaire and for
Cotterall‘s. Cotterall selected all the 34 items herself (she does not give details of any
using a large initial selection of items (Cotterall 1995: 256) which were then reduced
in number statistically using data from trials and not by using judgements of the
researcher or experts. This process has, I feel, produced a more interesting ―data-
driven‖ collection of items for the autonomy measuring questionnaire which is less
likely to reflect preconceptions. In Category 4 there are items which (like Cotterall‘s
Role of the Teacher) accommodate possibly external loci (e.g. 189. I learn English
because I have to) as well as items which can potentially reflect more internal
attributions (e.g. 230. My way of learning will never change). This broadens the
coverage of the control area from the narrow focus on the role of the teacher.
Factor 2 Role of feedback. Her respondents like to get feedback. The items seem to
than self-evaluation. This seems to suggest that her grouping is not finding out about
autonomy in general but about her specific students. Perhaps this grouping could be
labelled ―Reflection on own performance‖, and then her respondents‘ answers can be
224
interpreted as evidence of a lack of this. Reflection would imply that this is a form of
Metacognition.
connection. However, both the items are worded such that confidence in one‘s ability
is a possible interpretation: ―I know how to study languages well‖ and ―I know how
Factor 5 Experience of language learning. Again, this group is composed of only two
items (―I have been successful in language learning in the past‖, and ―I have my own
ways of testing how much I have learned‖), but Cotterall believes that they can be
(1995: 201). The autonomy measuring instrument has ten items to cover this area,
and is much more specific and more clearly identifiable with metacognition.
Social comparison and information literacy are not reported in Cotterall's research
which raises questions about the selection process used for her items – only 34 were
used compared with 256 for the present questionnaire‘s development. The
comparison with Cotterall‘s groupings adds to the evidence for the autonomy
225
measuring questionnaire having construct validity in that it broadly agrees with
Cotterall‘s findings.
In Section 6.3.2 I will use the factors which have been preliminarily established here
to investigate which areas teachers use in estimates, and whether this is a balanced
range of the components of autonomy. I will also use these components to evaluate
This section looks at whether the questionnaire‘s factor analysis groupings were new
or aligned closely with the criteria originally used for the item selection. In Table 6.8
the areas represented in the 50-item Short List are presented for comparison with the
six factors emerging from the factor analysis of the combined standardised data for
the 50 items.
Both are arranged in order of strength, i.e. for the Short List areas by the proportion
of items covering that area, and for the factors by the strength of the factors from
high to low (factor number). There are parallels between the two grouping schemes,
226
but there has been a re-sorting of items. As can be seen in Table 6.9 below, the items
have not been organised by the factor analysis into the same groupings as before.
The Long List categories and how the items were judged to cover them (shown in
Table 10.2) were based on the areas identified in the Literature Review (see Section
2.4) and on personal judgement and discussion with supervisors. Following this the
Short List was based on a number of statistical analyses aimed at selecting those
items with a stronger ability to produce a wider spread of responses. With the factor
structure produced by factor analysis the remaining items have been grouped
statistically without the necessity of the subjective judgement found in the first stages
of item selection. The final groupings, as can be seen in Table 6.9 do not exactly
match the initial ones. This indicates to me that it was correct not to rely on
judgement alone in the item selection process, but to go through the stages from
more subjective but also broad, to less subjective and more focused. Correlations
between the factor analysis results and the literature are more suggestive for purposes
of construct validity checking when the narrower final selection of items has been
227
Attitudes to learning
Actions/Behaviours
Social Interaction
Metacognition
Responsibility
Confidence
Motivation
Strategies
Control
Factor
Skills
Item
228
Attitudes to learning
Actions/Behaviours
Social Interaction
Metacognition
Responsibility
Confidence
Motivation
Strategies
Control
Factor
Skills
Item
Table 6.9: Comparison of items in Short List and factor analysis groupings
In this section I will consider how the items and factor groupings in the questionnaire
contributed to its overall result. This overall result will be expressed as a ―Categories
respondent. It is found by summing the respondent‘s scores for each scale and then
3.12.2 items were coded either negatively or positively. Examination of the final list
229
of items in the factor structure reveals that all items have a clear positive
(autonomous) or negative (un-autonomous) value and that any ambiguous items have
Table 6.10 below shows the significant correlations of the factor groupings (or
―Categories‖) with the Categories Average (using combined and standardised data
from all respondent groups). This shows how the categories were predictive of the
aggregate result. The Social Comparison category has the highest correlation to the
as expressed by the sample. It shows that Social Comparison was the strongest
level then this category would be the most important for a teacher or learner to
consider. It may also be that this category has an underlying influence on all the
other categories, i.e. it supports aspects of the autonomy construct present in the
other categories, and hence appears in the statistics as that most significant to the
overall autonomy performance. This suggests that the construct measured by this
instrument tends to be related with positive social comparisons and less with the
230
G Items Categories Average
Pearson Sig. (2- N
r tailed)
3 193. The other students know English better than me. (R) .452** 0 161
3 187. I think learning English is more difficult for me than for the .400** 0 161
average learner. (R)
- 055. I know why I did well or did badly. .381** 0 161
6 231. I can study independently. .379** 0 161
6 229. I can choose the method of learning that suits me best. .378** 0 161
- 195. I worry that other students will laugh at me when I speak .370** 0 161
English. (R)
4 189. I learn English because I have to. (R) .363** 0 161
2 175. I look at causes and effects logically. .356** 0 161
2 251. I use real English texts [..] in my learning. .355** 0 161
2 196. I am confident I can learn English well. .348** 0 161
3 194. The other students are more confident than me at speaking .338** 0 161
English. (R)
2 254. I know the parts of a book [..] .308** 0 161
6 236. I am good at making choices. .301** 0 161
5 234. It is my job to check my work for mistakes. .293** 0 161
2 253. I know how to use English language reference books [..] .293** 0 160
5 237. I am an active dynamic person. .287** 0 161
5 046. I can describe the learning strategies I use. .272** 0 162
5 142. I fix my problems in vocabulary. .269** 0.001 162
4 246. Memorizing answers is the best way to learn. (R) .268** 0.001 160
6 220. I am ready to learn in unfamiliar ways. .267** 0.001 161
5 138. I know techniques to help me remember vocabulary. .266** 0.001 162
6 205. I notice how other people use English. .260** 0.001 161
4 095. To read you must proceed word by word. (R) .257** 0.001 162
1 147. I worry if I don‘t understand all the words in a text. (R) .239** 0.002 161
2 125. I change the way I write according to who will read it. .236** 0.003 160
6 008. I am good at studying on my own. .230** 0.003 162
1 112. I worry if I don‘t understand everything when I listen. (R) .226** 0.004 162
6 203. I organise my time for studying. .225** 0.004 161
4 230. My way of learning will never change. (R) .224** 0.004 161
5 212. I talk to others about how I feel about learning English. .215** 0.006 161
4 140. To remember vocabulary you need to be talented. (R) .211** 0.007 162
5 049. I have changed the way I learn after thinking about it. .199* 0.011 162
- 026. I feel lucky when I get good marks. (R) .193* 0.014 161
1 110. Every word is important for understanding a listening text. (R) .189* 0.016 162
2 256. I know how to find the information I need on the Internet. .187* 0.018 160
5 130. My writing is better now than it was a year ago. .178* 0.024 162
1 086. When I read an English text I need to understand every word in .166* 0.035 162
it. (R)
- 030. The teacher‘s job is to give me all the information. (R) .165* 0.036 162
2 252. I know how to find information in a library. .165* 0.036 161
*Correlation is significant at the 0.05 level. **Correlation is significant at the 0.01 level. G = Group
1. Linguistic Confidence. 2. Information Literacy. 3. Social Comparison. 5. Metacognition.
6. Self-Reliance
Table 6.11: Significant item Correlations with Categories Average in all data combined
in descending order of Pearson’s correlation
with caution as this result would also be found when the top categories are the
highest-scoring due to having items which are too generally true of most of the
231
respondents in the sample, or which most students prefer to answer in a positive way
Table 6.11 (above) shows all the individual items which correlated significantly with
the Categories Average. These items can be seen as the most important ones in the
The two highest items (193 and 187) are from the Social Comparison category.
When ranked in order of the size of correlation, in the top third of the items five are
related to confidence (items: 193, 187, 195, 196, and 194), compared to four from
Information Literacy, three from Self-Reliance, and one from Locus of Control.
Confidence is also highlighted by teacher estimates (see Section 6.3.2 for discussion
of this).
As can be seen in Table 6.12 Linguistic Confidence has the second lowest average
item score (57.72%) of all the categories, just behind Social Comparison (56.77%).
This suggests that confidence is a weakness for these respondents. The highest
scoring category is Information Literacy with an average score of 74.19 per item.
average scores. Data of this type indicate how the results of the questionnaire could
autonomous learning. I would suggest that the aggregate Categories Average figure
232
Category and Items Min Max Mean Ave
1. Linguistic Confidence 57.72
086. When I read an English text I need to understand every word in 0 100 61.20
it. (R)
110. Every word is important for understanding a listening text. (R) 0 100 61.64
112. I worry if I don‘t understand everything when I listen. (R) 0 100 54.56
147. I worry if I don‘t understand all the words in a text. (R) 14.29 100 56.29
150. I worry if I don‘t understand all the grammar in a text. (R) 14.29 100 54.92
2. Information Literacy 74.19
125. I change the way I write according to who will read it. 0 100 70.10
175. I look at causes and effects logically. 0 100 70.14
196. I am confident I can learn English well. 0 100 78.23
251. I use real English texts (i.e. not made for students) in my 0 100 74.04
learning.
252. I know how to find information in a library. 0 100 72.45
253. I know how to use English language reference books [..] 20 100 74.59
254. I know the parts of a book (index, glossary, contents, chapters). 0 100 75.94
256. I know how to find the information I need on the Internet. 0 100 78.04
3. Social Comparison 56.77
187. I think learning English is more difficult for me than for the 0 100 64.24
average learner. (R)
193. The other students know English better than me. (R) 0 100 52.80
194. The other students are more confident than me at speaking 0 100 53.26
English. (R)
4. Locus of Control 65.51
095. To read you must proceed word by word. (R) 0 100 61.97
140. To remember vocabulary you need to be talented. (R) 0 100 57.06
189. I learn English because I have to. (R) 14.29 100 63.70
230. My way of learning will never change. (R) 0 100 72.71
246. Memorizing answers is the best way to learn. (R) 0 100 72.12
5. Metacognition 70.17
046. I can describe the learning strategies I use. 0 100 66.70
049. I have changed the way I learn after thinking about it. 0 100 69.70
109. I predict the content before I listen. 0 100 68.97
130. My writing is better now than it was a year ago. 0 100 76.00
138. I know techniques to help me remember vocabulary. 0 100 68.60
142. I fix my problems in vocabulary. 0 100 66.90
212. I talk to others about how I feel about learning English. 0 100 72.66
234. It is my job to check my work for mistakes. 0 100 72.25
237. I am an active dynamic person. 0 100 69.56
238. I choose the exercises I work on. 0 100 70.36
6. Self-Reliance 72.73
008. I am good at studying on my own. 14.29 100 74.40
203. I organise my time for studying. 0 100 70.08
205. I notice how other people use English. 40 100 80.77
220. I am ready to learn in unfamiliar ways. 0 100 74.09
229. I can choose the method of learning that suits me best. 0 100 71.83
231. I can study independently. 20 100 74.76
236. I am good at making choices. 0 100 63.16
R = Reverse coded
Table 6.12: Item scores (%) in each category (combined standardised data)
areas of strength or weakness, but can be a quick summary to be noted and then
expected result or an indicator of something which the teacher is not aware of. An
233
example of its use is given in Section 6.3.5.2. The categories average is also a way of
examining how the instrument performs, that is, examining the weightings of the
understand how the questionnaire‘s results really relate to individual learners, and
In this section I will continue to examine the questionnaire, now looking at how its
data and construct correspond with smaller scale or more qualitative sources of data.
The data used were gathered from the two ELTCS classes (20 students), an interview
with one of the ELTCS teachers, and an interview with one of the students, therefore
in this section the evidence will be of a different nature to the larger scale data
analysed in the previous section. Conclusions will not go beyond the limits of the
data but interesting questions can be prompted by the qualitative data here reviewed.
It had originally been the intention to compare large numbers of teacher estimates
with the questionnaire data and so establish whether the instrument was giving
similar results to those obtained from teachers. If correlations had been found then it
would have suggested that the questionnaire could be used as a tool to substitute for
At the smaller scale now being considered, correlating teacher estimates with the
234
However, it is worthwhile and interesting to emulate the procedure at a micro-level
because it was an opportunity to gather evidence of the types of issue which could be
As well as looking at the correlation between the total questionnaire score and the
teacher estimates, it was also decided to look at the individual categories and see
whether these correlated and, if so, which gave the closest match. This was extended
to looking for correlations between the individual items and the teacher estimates.
The purpose of this was to continue the quality control of items in order to find
ELTCS year-3 teacher‘s interview will also be examined to find if her verbal
The two groups of students have been described in Sections 5.2, 5.3, and 5.4. In both
the ELTCS year-3 group and the ELTCS presessional group there were 10 students. I
myself taught the latter class. This group did the questionnaire twice, once at the
beginning and once at the end of the five-week presessional, though only 6 of the
Both teachers (i.e. the ELTCS year-3 teacher and myself) provided estimates of their
students‘ autonomy levels on a scale of 1 to 10. In the case of the ELTCS year-3
group the teacher gave her estimates at the same time as she was being interviewed
estimates on the first day of the presessional course and also administered the
questionnaire (this stage will be referred to as P1). The students completed the
questionnaire a second time at the end of the course, and I made my second set of
235
estimates of their autonomy levels, now based on five weeks‘ experience of teaching
It was hoped to shed light on which areas teachers actually use in everyday situations
to evaluate their students‘ autonomy, and therefore these three sets of estimates were
Looking first at the categories, the significant correlations with teacher estimates are
shown in Table 6.13. (As the data from the questionnaire are from a 7-point Likert
scale but the data for the teacher estimates are from a 10-point ordinal scale it was
Teacher Estimate
Spearman‘s Correlation Sig. (1-tailed) N
1. Linguistic Confidence .523(*) 0.019 16
4. Locus of Control .519(*) 0.02 16
Table 6.13: Significant correlations between Teacher Estimate and the categories and
Categories Average (ELTCS year-3 and ELTCS presessional P2)
Linguistic Confidence has the strongest correlation. It should be remembered that the
teacher estimates are based on observation of the learners and the teacher‘s own
judgement, which are both subjective and will almost inevitably favour some
students with better English are be judged as more autonomous. This kind of
overemphasis on one area can be expected to show in the correlations figures such as
236
The ―Categories Average‖ is an overall score given to each individual questionnaire
respondent. It is calculated by adding the score for each category and then dividing
by the number of categories (six) to produce an average. This produces a score which
Average correlations are made within the questionnaire (Table 6.14) it is possible to
compare the balance of components used by teachers with that predominating in the
questionnaire.
Metacognition. This is a similar pattern to that found with item correlations. In Table
6.16 below, only two items correlated significantly with the teacher estimates (110
and 150). When item to Categories Average correlations are calculated for the
questionnaire (see Table 6.15 below) there are 23 items which significantly correlate,
two of which are not in the six categories, but including the two items found to
correlate significantly with Teacher Estimates. Information Literacy does not appear,
and (as with the significant correlations with the items) Locus of Control is not
present.
237
Group Item Categories Average
Pearson Sig. (1-
Correlation tailed)
3 193. The other students know English better than me. (R) .803(**) 0
3 194. The other students are more confident than me at .788(**) 0
speaking English. (R)
1 112. I worry if I don‘t understand everything when I listen. (R) .770(**) 0
6 231. I can study independently. .713(**) 0.001
6 229. I can choose the method of learning that suits me best. .690(**) 0.002
1 150. I worry if I don‘t understand all the grammar in a text. .678(**) 0.003
(R)
3 187. I think learning English is more difficult for me than for .655(**) 0.004
the average learner. (R)
6 008. I am good at studying on my own. .642(**) 0.005
- 026. I feel lucky when I get good marks. (R) .642(**) 0.005
2 256. I know how to find the information I need on the Internet. .633(**) 0.006
5 130. My writing is better now than it was a year ago. .595(**) 0.01
1 110. Every word is important for understanding a listening .572(*) 0.013
text. (R)
5 138. I know techniques to help me remember vocabulary. .543(*) 0.018
1 086. When I read an English text I need to understand every .539(*) 0.019
word in it. (R)
6 220. I am ready to learn in unfamiliar ways .523(*) 0.023
5 142. I fix my problems in vocabulary .511(*) 0.026
- 017. If I must finish a job at a certain time I finish early .510(*) 0.026
1 147. I worry if I don‘t understand all the words in a text. (R) .507(*) 0.027
6 236. I am good at making choices. -.484(*) 0.034
5 234. It is my job to check my work for mistakes .481(*) 0.035
2 253. I know how to use English language reference books .475(*) 0.037
(encyclopedias, dictionaries, etc.).
6 205. I notice how other people use English .468(*) 0.039
2 196. I am confident I can learn English well. .467(*) 0.04
N=15. ** Correlation is significant at the 0.01 level (1-tailed). * Correlation is significant at the 0.05
level (1-tailed).
I will now look at correlations at the items level. The data from the two groups
ELTCS year-3 and ELTCS presessional P2 were combined to provide a larger set of
data. It was expected that these groups would provide better correlations than ELTCS
presessional P1 since the two teachers (myself and the year-3 teacher) had had longer
to familiarise themselves with the students. Table 6.16 shows the significant
correlations which were found between the average scores for items and our
238
Group Items Teacher Estimate
Spearman‘s Correlation Sig.
(2-tailed)
1 110. Every word is important for understanding a .658(**) 0.006
listening text. (R)
Both of the items belong to the Linguistic Confidence group, which suggests that
students‘ linguistic confidence has a part to play in the process which the year-3
teacher and myself went through to estimate their students‘ autonomy. This was also
the indication of the analysis at the Categories level (see Table 6.14). It is possible
that the learners with the best English are those who we score more highly because
we have assumed that these learners will have the greater autonomy. If this is the
case, then it suggests that we were overly influenced by a limited section of the
autonomy spectrum. It therefore suggests that the questionnaire could have a place in
correlation at the .05 level between Teacher Estimate and Categories Average, which
suggests that we two teachers and the questionnaire are, for the two groups ELTCS
overview judgments of the autonomy levels of the learners. This on a larger scale
In order to investigate further how the ELTCS year-3 teacher made her estimates I
239
6.3.2.3 Interview
In the interview she often stated that she had some difficulties in assessing the
now, [T1-St-c], I‘ve got to try and make sure I‘ve got the right name [unclear] oh yes [T1-St-
c] yes she is, she‘s very kind of difficult to know because she‘s always smiling and pleasant
... she probably, I mean I don‘t really know because I don‘t really know her well enough
Of the same student she says ―she‘s so inscrutable‖ (line 85) and ―I can‘t really make
up my mind about her (line 88). Speaking of another student she says:
I wouldn‘t be surprised if there were quite a few homeworks missing from him [T1-St-g] and
things like that, although you know I would have to check that (Line 195-196)
About one particular student she says: ―anyway, [T1-St-d] I should know quite well
she‘s my tutee‖ (line 90), which suggests that she is not as familiar as this with the
other students.
The teacher makes a point which is particularly important in larger classes ―[T1-St-j]
now difficult to analyse or assess her because she is so quiet‖ (line 299), and, in an
email communication she writes ―when there are 33 you can't know each one very
In view of these difficulties, the fully validated questionnaire could have a role to
play in teacher training by raising their awareness of how they make their
assessments and what other considerations they should be looking for. The issue also
highlights that teachers can teach students for a substantial time and yet not be able
to become familiar with them. This suggests that a valid and reliable instrument of
the type being analysed here could have a use in helping teachers to know their
240
6.3.3 ELTCS year-3 teacher interview and confidence-related
categories
In analysing the interview with the ELTCS year-3 teacher I thought I discerned two
she says ―confident members of the group and confident language users in the
group...‖ (lines 364-365), which implies that she sees a distinction between two types
of confidence, one perhaps more general and the other more specifically relating to
using language.
In order to investigate whether the teacher‘s division between the confident and the
Table 6.18 the two uses of confidence which occur in the interview. One type is
confidence concerning relating to the others in the group which I have labelled
―people confidence‖; the second type is where the teacher talks about ―self-belief‖
and ―self-esteem‖ and is referring to the learner‘s abilities in English, and this I have
predominantly positive or negative. These can be compared to her estimates, and the
involved confidence in its highest-loading item), and Categories Average, which are
The ELTCS year-3 teacher says in her interview (Lines 366-367) that:
... looking at the ones I put down low [i.e. in her estimates of autonomy level] ... I would say
yes all of these I would say lack confidence, you know they are socially probably shy
Table 6.17 shows that she in fact gave good autonomy marks to some students who
in the interview she said were lacking confidence (see transcript excerpts in Table
241
6.18). For example students T1-St-a and T1-St-d have the highest autonomy
estimates (see Table 6.17), but do have negative reports about their people
confidence from the teacher who describes student T1-St-d as ―very shy‖, but
estimates her autonomy at 7.5 out of ten. However, they also have positive-sounding
reports of their learning confidence; she for example describes student T1-St-d as
having ―self-belief in her abilities‖, which would compensate for her shyness. The
two students with the lowest autonomy estimates both have negative reports of their
confidence: T1-St-e for learning confidence and T1-St-j for her people confidence,
and neither of these are balanced by positive reports, which corresponds with the
teacher giving them lower autonomy estimates. In Sections 6.3.2.1 and 6.3.2.2 it also
emerged that the year 3 teacher‘s estimates and mine correlated most closely with the
Linguistic Confidence scale, both when viewing items and categories. This still
seems to be the case when considering individual students, as can be seen in Table
6.17. The suggestion is that the teacher will base estimates of autonomy on a readily
observable quality such as Linguistic Confidence. This is the kind of effect which the
242
People confidence Learning-confidence
T1-St-a right, starting with [T1-St-a], well the major thing to say about her is that she is her, you know, self-esteem is quite high I think (line 12)
a bit different from the rest of the group and in class is quiet and withdrawn (POSITIVE)
(lines 1-3)
(NEGATIVE)
T1-St-d ...she‘s just shy ... in terms of other people (lines 136-139) [T1-St-d] has self-belief in her abilities she knows that she is quite good at
she‘s not an outgoing person (line 123) English (line 136-137)
she‘s very shy and she‘s finding this quite hard in her, in her flat because it although she knows that she is good, she‘s not good enough for herself (line
means that she‘s very shy to talk to the other people in her flat (lines 111-113 140)
I think she comes apart in social situations so she, her presentation was awful it (POSITIVE)
was the worst of the group (lines 109-111)
(NEGATIVE)
T1-St-e -not commented on Then we come to [T1-St-e], now she‘s somebody who is hard-working and
extremely weak and you know I mean there is this kind of sense of where is
autonomy linked to confidence and ability because she‘s got very little
confidence and very little ability but lots and lots of effort (lines 129-133)
[little] self-belief in her abilities (line 136)
she‘s got a little bit of self-belief in the sense that she knows that she can work
hard (lines 153-154)
(NEGATIVE)
T1-St-f And then we have [T1-St-f] and yes she comes over as someone who is very -not commented on
together, very, yes quite confident socially and doesn‘t seem to be sort of you
know addicted to her Chinese group (lines 163-166)
(POSITIVE)
T1-St-h whereas the others [i.e. the ones she rated more highly in autonomy]are yes -not commented on
probably quite confident although [T1-St-h] comes over as shy but she is, you
know by dint of her you know actions she‘s obviously broken away from the
Chinese group which is very very brave (lines 369-371)
(NEGATIVE?)
T1-St-j she is so quiet (line 300) -not commented on
(NEGATIVE)
Table 6.18: Confidence categories in ELTCS year-3 teacher’s interview
243
6.3.4 Movement of teacher estimates
This section looks at some smaller scale data drawn from a sample which due to its
size will not give statistically significant results. However, it illustrates in principle a
way in which the teacher estimates and questionnaire results can be compared which
reveals if they are converging over time, and is therefore a further assessment of the
questionnaire was providing useful information more quickly than were teacher
estimates.
In this analysis scatterplots were made showing each student‘s results from the six
from the ELTCS presessional class. As there were two sets of teacher estimates, (i.e.
beginning (P1) and end (P2) of the ELTCS presessional this resulted in 14
scatterplots which are shown below in Figure 10.1 to Figure 10.7 in Appendix 10.5.
In each scatterplot a linear fit line has been plotted automatically by SPSS to show
the trend of the data points. It can be seen that in all cases the fit line in the
correlation between my estimates and the questionnaire data in the second round of
questionnaire and estimates (P2) than in the first (P1). This is very much what would
be expected when a teacher (in this case myself) has had some time to increase his or
her knowledge of a class. The hypothesis suggested by these figures is that increased
exposure to a group of students will increase the accuracy of estimates made about
those students. This will result in movement of the estimates, and if that movement is
towards the questionnaire result then the questionnaire has been shown to be faster at
244
finding a level than the teacher. If this result were achieved at a larger scale it would
In this section I will explore the information I have gathered on two of the ELTCS
students. The purpose of this is to look at the questionnaire data with reference to
individual learners to gauge whether the different sources of data converged and to
see what indications they provide on the validity and reliability of the questionnaire
and its categories. At this small scale, dealing as it does with data from only two
questionnaire data reflect the indications from other sources to explore whether
questions are raised about the suitability of the questionnaire for use with learners.
Two students are involved in this stage of the analysis, T1-St-b and P1-St-a. T1-St-b
was in the ELTCS year-3 group (which I did not teach) and she was the only one of
She is also described by the ELTCS year-3 teacher in her interview with me, which
means that for this student I have more material than for any of the others. P1-St-a
was one of my students in the presessional group. She completed both the initial
administration (P1) and the end of course administration (P2) of the questionnaire.
Table 6.19 below summarises the questionnaire results of these two students.
245
ID T1-St-b P-St-a
Group EY3 P1 P2
Teacher Estimate 75.00 90.00 60.00
Linguistic Confidence 74.29 62.86 57.14
150. I worry if I don‘t understand all the grammar in a text. R 5 3 3
147. I worry if I don‘t understand all the words in a text. R 5 5 2
112. I worry if I don‘t understand everything when I listen. R 5 3 5
086. When I read an Eng‘ text I need to und‘ every word in it. R 6 5 5
110. Every word is important for understanding a listening text. R 5 6 5
Information Literacy 78.57 89.29 69.64
254. I know the parts of a book [..]. 6 7 5
253. I know how to use English language reference books [..]. 5 3 5
256. I know how to find the information I need on the Internet. 5 7 5
252. I know how to find information in a library. 5 6 5
251. I use real English texts [..] in my learning. 7 6 5
175. I look at causes and effects logically. 5 7 2
196. I am confident I can learn English well. 6 7 7
125. I change the way I write according to who will read it. 5 7 5
Social Comparison 71.43 90.48 66.67
194. The other students are more confident [..] at speaking Eng‘. R 5 7 4
193. The other students know English better than me. R 5 5 3
187. [..] learning Eng‘ is more diffi‘ for me than for the ave‘ l‘rner. R
5 7 7
Locus of Control 68.57 77.14 82.86
230. My way of learning will never change. R 5 7 7
140. To remember vocabulary you need to be talented. R 4 7 7
246. Memorizing answers is the best way to learn. R 6 7 7
189. I learn English because I have to. R 4 1 1
095. To read you must proceed word by word. R 5 5 7
Metacognition 74.29 60.00 52.86
237. I am an active dynamic person. 6 6 2
234. It is my job to check my work for mistakes. 6 7 5
212. I talk to others about how I feel about learning English. 4 7 1
138. I know techniques to help me remember vocabulary. 6 3 4
109. I predict the content before I listen. 5 3 6
049. I have changed the way I learn after thinking about it. 5 7 2
046. I can describe the learning strategies I use. 5 3 5
142. I fix my problems in vocabulary. 5 2 4
130. My writing is better now than it was a year ago. 6 3 5
238. I choose the exercises I work on. 4 1 3
Self-Reliance 75.51 79.59 51.02
231. I can study independently. 5 6 4
236. I am good at making choices. 5 3 3
229. I can choose the method of learning that suits me best. 6 7 4
220. I am ready to learn in unfamiliar ways. 4 7 2
205. I notice how other people use English. 6 7 7
203. I organise my time for studying. 5 3 2
008. I am good at studying on my own. 6 6 3
Categories Average 73.78 76.56 59.75
Other items
021. All lessons are equally valuable. R 4 1 1
023. Students should always do what their teacher says. R 4 5 7
026. I feel lucky when I get good marks. R 5 5 6
030. The teacher‘s job is to give me all the information. R 4 1 7
055. I know why I did well or did badly. 5 5 4
076. I rely on the teacher when learning. R 6 7 7
195. I worry that other students will laugh [..] when I speak Eng‘. R 5 7 7
243. I decide what I need to read. 4 1 5
R= Reverse Coded. EY3= ELTCS year-3. Items scored 0-7. Category scores are %.
Table 6.19: Questionnaire results for students T1-St-b and P-St-a
246
6.3.5.1 Student T1-St-b
The table below (Table 6.20) shows this student‘s questionnaire scores in the context
of her group. She is above the mean in all categories and has the highest score in two,
A prominent feature of my interview with this student was the importance she gave
to what she herself referred to as confidence. I will discuss this and then move on to
6.3.5.1.1 Confidence
The word ―confidence‖ occurs ten times in the 5,160 words she speaks, which is over
five times the frequency of this word in spoken English according to the British
INTERVIEWER: Yes. Okay, 22 [i.e. item 187] ―I think learning English is more difficult for
me than for the average learner‖ disagree
INTERVIEWEE: I think I have the confidence
INTERVIEWER: Confidence, now that‘s interesting.
INTERVIEWEE: Yes
INTERVIEWER: Do you think confidence is...
INTERVIEWEE: I think it‘s really important (Lines 605-612)
247
This was particularly interesting as her first use of the word was unprompted. In fact,
she uses the word more frequently in her interview than some other common terms
These data suggest that she was thinking of confidence as an important element in
learning English. Her questionnaire results (in Table 6.19 above) seem to confirm her
beliefs about confidence, as she has scored well in the two categories which appear
to relate to confidence (i.e. Linguistic Confidence and Social Comparison), and she
in fact has the highest score for Linguistic Confidence in her group. The interview
In the second instance of the word she attributes confidence at school to obtaining
good marks:
248
INTERVIEWER: Where does it [i.e. confidence] come from? Does it come from being
successful, or are you confident and then because you are confident you become successful?
INTERVIEWEE: I think if you are in school I think the confidence just comes from your
score, the English mark (Lines 612-615)
INTERVIEWEE: ... one of my friends [T1-St-d] also came here [Warwick University] I
think she she is a very strange person I think she lacks lots of confidence [unclear] she never
talks to her flatmates and sometimes she is so hungry but she doesn‘t go downstairs to cook
and we say why and she says lots of people in the kitchen
...
INTERVIEWER: So you think is very important to have this, do you think confidence with
other people is the same as confidence about learning English?
INTERVIEWEE: Yes (Lines 615-623)
Here she is clearly referring to confidence with other people, which seems to equate
to the idea of people confidence which was found in the ELTCS year-3 teacher‘s
interview (see Section 6.3.3). It also appears to have a relation to the questionnaire
in relation to others, and possibly not being inhibited by worries about the judgement
In the above quote from the interview, student T1-St-b appears to feel that the
―people confidence‖ is the same as the confidence gained from successful work,
though she is not suggesting, I feel, that her friend is lacking in confidence because
INTERVIEWER: .... [item 193] ―the other students know English better than me‖. I think
INTERVIEWEE: [unclear] difficult. This is still about confidence I think
INTERVIEWER: Yes
249
INTERVIEWEE: I think there must be some students better than me
INTERVIEWER: Well yes
INTERVIEWEE: I just want to have some confidence about myself
INTERVIEWER: I think I agree with you, it is confidence isn‘t it. In a way it doesn‘t matter
whether they are better than you or not it‘s that you feel...
INTERVIEWEE: Yes, exactly (Lines 642-651)
Here she seems to be saying that for her it is important to maintain her inner sense of
confidence and not think about whether other students are better at English than her.
This suggests a conscious management of her reactions to her context to maintain her
I think no matter good or bad [unclear] they will speak English but I think I have the
confidence. I don‘t care about the result I just want understand me (Lines 669-670)
She is here saying that concern for accuracy in speech should not inhibit one from
speaking, even if other students might laugh. Her seventh mention of confidence is in
I think I got the confidence and I speak out and I just try my best to explain my English to
you (Lines 671-672)
In her eighth and ninth uses she goes back to her idea of the good students being the
confident ones:
I think in my secondary school the good students always had confidence, only if you have
confidence you can get the very high mark (Lines 679-680)
However, here she implies that confidence precedes good marks, which is apparently
250
when [unclear] my classmates also have sometimes the speaking [unclear] someone else will
laugh at [unclear] I think most my classmates still have the confidence to keep them alert,
don‘t stop (Lines 687-689)
She is making the point that her classmates, like her, have sufficiently high levels of
confidence to overcome adverse reactions from their peers. This appears again to
relate to Social Comparison. Her use of the term, however, appears to vary and she
appears to contradict herself. This might be because she has not distinguished
questionnaire were to put a learner at very different levels in the two confidence-
them; it could also be a sign that a learner is below the language level for the course
warning sign that a student is responding to the questionnaire in a biased way i.e.
putting answers which appear to be the ―best‖ ones rather than ones which are true of
him or her. Thirdly, the answers could be ones which the respondent believes,
wrongly, to be true of him- or herself. The questionnaire may not reveal some types
of problem when the learner does not see them. For example, an inadequate ability to
misleading. If a respondent gained high scores in the questions ―I notice how other
people use English‖ (item 205) and ―I organise my time for studying‖ (item 203) due
to a mistaken ability to assess themselves this could lead to a higher than warranted
score for Self-Reliance. This suggests that a single questionnaire result viewed in
251
knowledge of the individual concerned. This possible issue with the reliability of
self-report questionnaires is not demonstrated here, but is the type of problem which
remaining questionnaire categories and examine what she said in the interview about
her answers.
She scored 78.57% in this making it her highest category. I asked her, regarding item
253 ―I know how to use English language reference books‖ (her questionnaire
response was Agree), how she knew how to use reference books, and she said that
Item 252 ―I know how to find information in a library‖ (Agree). This part of the
interview is a little unclear, but she appears to say that she has used the university
library a lot and has learned how to find things so that for her it is now easy, but this
is not the case for everyone (lines 501-549). She had used libraries in China, but
mainly for leisure, preferring to use Internet sites such as Wikipedia for reference.
She only responded to the item with an ―agree‖, which either indicates her
Item 256 ―I know how to find the information I need on the Internet‖ (Agree). She
says ―I think it‘s everybody can find the information on the Internet‖ (line 592) and
―I think it‘s [unclear] an easy skill. Even my mum can use the Internet‖ (lines 594-
595). I wondered whether this was a little complacent or naïve and asked her why
she had only said agree to this, not strongly or very strongly agree and she thought
252
Item 251 ―I use real English texts (i.e. not made for students) in my learning‖ (Very
strongly agree). She read English websites in China such as the BBC because she
wanted to improve her English, and that this had originally been suggested by a
teacher. She thinks that ―everyone does that‖ (line 53). This is the first marked
response in this category, indicating that she is more comfortable with this item.
Item 125 ―I‘ve changed the way I write according to who will read it‖ (Agree):
If I‘d just writes the academic writing I really need to focus on the word vocabulary and the
register ... but if I just write for my friends, just some MSN talk and then I think I can
[unclear] whatever I want. (Lines 310-314)
She says that she thinks it‘s ―natural‖ (line 318) to do this, though she has again only
gone as high as ―agree‖ in her answer, which suggests either more doubt than she is
Item 175 ―I look at causes and effects logically‖ (Agree). Verbally, she essentially
agrees, but adds that ―Sometimes a little luck is good for me‖ (line 996).
Item 254 ―I know the parts of a book (index, glossary, contents, chapters)‖. She
strongly agrees in her questionnaire response. She says that she was not taught how
... the first time I need to write an assignment finish assignment so I went to the library and I
found a lot of books about the topic ... but I don‘t know how to, and I can‘t read it in the
library ... for the whole book so I just go back the book and find the index I want to find
―education‖ so I find ―E‖ ... and I just go back and I go back go back which page and find
[unclear] me or not, I just go skim (Lines 574-584)
253
6.3.5.1.3 Locus of Control
Item 95 ―To read you must proceed word by word‖ (Disagree). She demonstrates
that she understands the point of the question, for example she says ―I think if in
exams I must understand almost all and not every one word ... And if it‘s everyday
life I just need to understand the general idea (lines 57-60). She says that she does
stop and check in a dictionary if ―I encounter a word it is very familiar but I can‘t
remember the meaning so I check this with my e-dictionary‖ (lines 63-64). If she
sees a word that she does not know she ―Sometimes ignore oftentimes just to guess‖
(line 75).
Item 246 ―Memorising answers is the best way to learn‖ (Strongly disagree). In the
interview she seems to be clear in her mind about memorising answers not being a
I think sometimes when we do the about mathematics [unclear] teacher will always tell you
to learn a method not just for this question not just for this answer [unclear] you need to learn
a method how to work out this mathematic problem and not just learn the answers (Lines
955-958)
She appears to be substituting one kind of memorisation for another, i.e. memorising
a method rather than understanding or finding out for oneself. This illustrates to me
that a short or closed answer to a single item cannot reliably probe all the details of a
respondent‘s thinking. However, I also feel that an appropriate degree of her thinking
has been recorded for the purposes of a general overview of the respondent.
Item 140 ―To remember vocabulary you need to be talented‖ (Neither agree nor
254
I think everyone can remember the vocabulary but some maybe someone is with the talented
[unclear] remember it faster (Lines 435-436)
I think her response reflects accurately her feelings about the question, though
another person with the same idea but a stricter interpretation of the question could
have put one of the disagree responses (varying individual response styles like this
Item 230 ―My way of learning will never change‖ (Disagree). She is able to reflect
I think the way of learning in my primary school and now is really different I think everyone
will change and we encounter new things [unclear] really one day we will find the best way
of learning (Lines 937-939)
Again her questionnaire response is more muted than her ideas expressed in the
6.3.5.1.4 Self-Reliance
I think in China I think my only job is to study and I don‘t need to worry about what I‘m
going to eat today and what I‘m going to worry about ... but I came here a lot of life
problems I need to find what to eat today. I need to go to supermarkets, so it‘s a lot of
problems so I need to organise my time for study (Lines 716-721)
She has been pushed into planning her time now, so perhaps her ―agree‖ is not a sign
of being a pro-active learner. On the other hand in this area she seems to be
responding to a new situation for herself and has worked out a way to cope. This
Item 205 ―I notice how other people use English‖ (Strongly agree).
255
Yes I think especially the native speaker and you need to try to learn their, copy their into-,
intonations ... and the tone and the way of their speaking (Lines 1075-1078)
She is not just saying strongly agree to please me – her response style has been to
agree – and she mentions the specific area of intonation which gives credence to her
answer.
Item 220 ―I‘m ready to learn in unfamiliar ways‖ (Neither agree nor disagree). This
item was her lowest scoring in this category. She was initially hesitant in the
interview, reflecting her questionnaire response ―I think it‘s a little difficult with
unfamiliar ways‖ (line 765), but when asked if she would try a new method that was
recommended to her she said ―I think I would try it, but if it‘s not suitable for me I
would just give it down‖ (lines 769-770). She tells a story about a method she tried in
China called ―Crazy English‖, which involved shouting out in public and did not suit
her at all so she stopped. Perhaps this experience made her cautious about trying new
ways of learning and so prevented her from agreeing with the item. The
Item 236 ―I am good at making choices‖ (Agree). She was not sure how to
understand the item, wondering whether spelling could be the kind of matter of
choice intended by the question, and this doubt made her want to respond with
―disagree‖. This illustrates the difficulty of maintaining clarity across language and
I think I can make the choices, but if it is the homework I have to do it but it is not homework
I can decide which one I want to learn today and [unclear] tomorrow (Lines 753-755)
256
She was still doubtful about her answer and thought of changing it to disagree. I
think, in the light of her above quote, that she was right to put agree for this item.
6.3.5.1.5 Metacognition
Her questionnaire score for this category was 74.29%, her third highest behind
She gives some examples in the interview, such as ―... sometimes the teacher told us
a lot of technical skills for example D-I-S means [unclear] is a negative way‖ (lines
386-387). So, she is remembering the techniques from school. She also thinks that it
is helpful to try to guess new words. I asked her if I gave her a list of new words to
learn, how would she go about it, and her reply was ―I think I just study, try to
recite‖. She says that in China everyone learns English in this way. It involves
Item 49 ―I have changed the way I learn after thinking about it‖ (Agree).
I think sometimes my way just I think usual way it‘s really a little bit wasting time it takes a
long time to take the notes or something and just to pick out the keywords it really takes time
so sometimes I will learn from everyone else and to try to highlight something in the
handouts (Lines 985-988)
She shows that she is reflecting on her way of learning, though it is not perhaps an
example of a fundamental change in her way of learning; she did respond to the item
Item 212. ―I talk to others about how I feel about learning English‖ (Neither agree
nor disagree). She said ―Sometimes if somebody ask me sometimes I will tell them,
257
but actually I never talk about this topic with someone else‖ (lines 1010-1011). This,
she says, is the case generally, the only exceptions being when class activities call for
it, an interview with other students for example. She has responded accurately, in
that if she was to decide herself she would never talk about her feelings about
learning English, though technically she does occasionally in class, so she was forced
to answer with ―neither agree nor disagree‖. This is another case of an item being
hard to answer for certain respondents. I think it is impossible to cater for all contexts
at the same time within an item of manageable length. I think it is a limitation that
Item 142. ―I fix my problems in vocabulary‖ (Agree). She says that ―Every time I
encounter a word I‘m very familiar with but I can‘t remember I will check it‖ (lines
471-472). She uses an electronic dictionary for this. This appears to be another
example of quite a limited thought behind the questionnaire answer, though it is fully
accurate.
Item 109 ―I predict the content before I listen‖ (Agree). She mentions exams where
the instructions are to read the questions before listening (lines 241-242), and she
refers to television trailers (lines 244-246) which help her to predict the content.
Item 234 ―It is my job to check my work for mistakes‖ (Strongly agree). She admits
that ―it‘s really boring I think assignment will take a long time and when you finish
you don‘t want to touch it again I don‘t want to open the Word again‖ (Lines 911-
912). However, she says that she always does check her work. This suggests that she
is a diligent student, perhaps with a developed sense of her own responsibility for her
learning, though the source of the motivation for this does not come out in the
interview.
258
Item 130 ―My writing is better now than it was a year ago‖ (Strongly agree). When
asked how she knew that she had improved she said that the marks her teachers are
giving now are better. She also gave examples of how she judges for herself that her
writing is better:
And now the thing [unclear] writing when I‘m in China we always write 200 words or 500
words passages [unclear] just kind of writing academic writing but I really can [unclear] I
will write for one day a whole day, this takes a long time. [unclear] I came here sometimes
it‘s a rush to write an essay before the deadline so sometimes I can write 2000 words in one
day (Lines 348-352)
She is aware of how she feels, saying ―I think the speed can also reflect something
about my writing I think sometimes it‘s natural to write some things academically‖
(Lines 356-357). She also reflects on her mental process ―I don‘t need to ... write the
sentence in my head before I type it into the computer ... In China I think sometimes
I need to think of the sentence‖ (Lines 359-362). Finally, she even mentions
ownership of her writing ―...it‘s MY English ... It‘s not THEIR English ... It‘s not for
my father mother or my English teacher...‖ (Lines 371-380). This all indicates that
her ―strongly agree‖ response is well founded and that her answer correctly reflects
Item 238 ―I choose the exercises I work on‖ (Neither agree nor disagree). She talks
about this in the interview as if she has interpreted the question as applying only to a
situation where the teacher has given a choice of exercises. This was not what was
intended by the item. I was looking for a higher level of control of learning, which
could be shown by a motivated student working for him- or herself. I think the
wording is too loose here; it should be ―I choose some of the exercises I work on‖, or
259
Item 237 ―I am an active dynamic person‖ (Strongly agree). She interprets the item
This question was included to find the more pro-active learners. In this case her
This category was her joint third ranking in terms of questionnaire score. It leaves me
foreign language for her, and one should not read too much into a sometimes very
unclear interview. It does not seem to me that she is quite 74.29 per cent of the way
to perfect metacognition, though she does have some nice insights, especially
regarding her reflections on her writing such as her feeling of ownership. If I were to
estimate her level I would say about 55 to 65 based on the interview. It is true that
the questionnaire does have a quite narrow range of scores, and here I feel that a
misleading score. This also indicates an issue with the type and wording of the items,
6.3.5.1.6 Autonomy
I avoided using the word ―autonomy‖ both in the questionnaire and in the interview.
She does not speak about autonomy, but she does refer to working on her own, or
260
I think lots of people like teamwork I think I prefer to do the homework or do the study by
myself ... and lots of people [unclear] it‘s not easy to concentrate [unclear] it‘s only chat, chat
about other things ... and although I know that teamwork is really good and it‘s a lot of
different ideas and I think it is waste [unclear] time (Lines 865-878)
She seems to be aware of learning habits and learning styles. She often alludes to
I think for example I try so I deserve to get the mark if I‘ve not tried of course I can‘t get the
good marks (Lines 992-995)
She seems to have rather conservative ideas about learning. She describes learning at
school in China:
... you need to recite every word we can‘t guess any word so we checked every word and we
know every word‘s meaning ... I think it‘s a good way ... You remember lots of English
lesson it‘s easy for you to use (Lines 94-100)
However, she does strongly disagree that she relies on the teacher when learning
(item 76). She makes a distinction between following the teacher and relying on the
teacher, saying:
Yes I think I really follow the teacher‘s way, but I don‘t need to rely on the teacher ... I don‘t
need to follow everything he told me (Lines 851-861)
The interview with the ELTCS year-3 teacher had a number of points which also
came up in the interview with T1-St-b. She is hard working and diligent, but is not
adapting as well as she could to the new environment in the UK. She prefers to work
on her own and perhaps this is related to staying inside the ―Chinese bubble‖, and
also not finding her place in the academic field (both of which the ELTCS year-3
teacher mentions). The ELTCS year-3 teacher says that she does not think that T1-
St-b is naturally autonomous, and this is not at odds with the impression I obtained
from my interview with her. The questionnaire results are mostly in line with my
261
interviews with the ELTCS year-3 teacher and T1-St-b (though necessarily less rich
in details).
I made observations of all the students during the ELTCS presessional course and
wrote them down. I wrote about this student‘s confidence and how she had
impressed me with her spoken English and ―Western‖ air. This had led me to think
of her as very autonomous. For the initial teacher estimate of her autonomy I gave
her 9/10, but this had fallen to 6/10 when I wrote up her entry in my observations of
the students in the fifth week of the course. I think the main reasons why I marked
her down were her disappointing level of written English and her passive in-class
behaviour; she did not seem to have a dynamic attitude to making progress, though
she was always one of the more diligent and cooperative students. For the initial
assessment I had only used her spoken English which seemed very fluent and
natural, with appropriately-used colloquial phrases. Then over the course of the
Self-Reliance went down from 79.59 to 51.02. She had large reductions in score in
three items. The fall in item 229 suggests that she was no longer agreeing that she
could choose the learning method that suited her best. In item 220, she went from
very strongly agreeing that she could learn in unfamiliar ways, to strongly
disagreeing, and in item 8 she no longer felt that she was good at studying on her
own. It appears that she had reconsidered her self-evaluations, perhaps as a result of
her experiences. This probably does not mean that she was less autonomous at the
end of the presessional course. It probably indicates that she is more autonomous, in
the sense that she has a better idea of herself and a better ability to assess her learning
262
at the time of the second administration (P2) than at the first (P1). Her P1 self-
assessment was too high and was adjusted to a more realistic position in P2. This
process could have affected her morale and caused her to lose confidence in herself
as a learner, and in relation to her classmates. An increase in honesty for the second
administration due to feeling more relaxed about the marks not contributing to her
She also seems to be realising that she cannot rely on the teacher and will be
expected to be able to put more emphasis on study on her own (items 23, 30, 229,
and 231) and this has made her worried about her capabilities and how she compares
to other students (items 8, 55, 193, 194, and 229). Her Locus of Control score has
fallen from 77.14 to 54.29, suggesting that she feels less confident about her abilities
to be in control of her learning, perhaps because she is becoming aware that more
responsibility is being transferred from the teacher to her. (This kind of reorientation
is part of the purpose of the presessional and is why I have interpreted the result in
this way.)
Her Linguistic Confidence score was only down slightly, going from 62.86% at the
start to 57.14% at the end. Her Social Comparison though went down more, from
90.48% to 66.67%. This at face value would be because she had changed her view of
the other students, deciding they were more confident speakers and knew English
better than her. Her Metacognition score fell from 60.00 to 52.86 which will
Literacy score has also dropped, from 89.29 to 69.64, possibly this is related to her
263
The score reductions after a period of new input suggest to me that she is rethinking
her assessments of herself in the light of a period of change and challenges (see
Section 7.4.5 in the Discussion Chapter). What she considered to be high abilities at
the start of the course have become less adequate in her assessment by the time she
has reached the end of the course. All this resulted in her Categories Average falling
At the start of the presessional I asked each of the students to write me a letter saying
what they hoped to learn from the course. Student P-St-a wrote that she wanted to
study grammar and vocabulary to improve her reading, and that she thought that
speaking was not a problem for her. Verbally, she told me she thought I would be
disappointed with her writing, because she felt herself to be weak in written
expression. After seeing some of her writing I felt she was right in her assessment. In
her end of course revision test she got 53/90, doing badly at combining sentences,
using linking words, and using the passive, all of which are important for academic
writing.
I am happier with the assessments of her which came at the end of the course, and
these were very different from those at the beginning, including the questionnaire
scores. However, the teacher estimate at P1 was 90% compared to the questionnaire
result of 76.56%, and the teacher estimate at P2 was 60% compared to 59.75% from
the questionnaire which shows that the teacher estimate has been adjusted down by
30% whereas the questionnaire result has moved down by the smaller figure of
16.81% suggesting that the questionnaire had given a better idea of the student than
264
The interview with T1-St-b clearly revealed more details and meant that speculation
about what scores mean can be reduced. However, I feel that Student P-St-a‘s
questionnaire results have led me to notice more and ask myself more questions
about this learner than I would have been able to without the questionnaire. My
experience of using the questionnaire has therefore been a very useful and positive
one. If I had continued to be her teacher I feel I would have been able to support her
autonomous learning in a much more effective way due to the questionnaire data.
The purpose of this section is to summarise the data which have been presented to
examine the construct validity of the questionnaire. The groupings which have
emerged from factor analysis are not the same as the groupings by which items were
initially chosen (Section 6.2.5). The questionnaire thus embodies a different picture
The factor analysis produced six clear factors which were identified and comparison
of these with related literature in Section 6.2.4 showed marked similarities. Section
6.2.5 showed that the groupings were not the same as the Short List‘s item area
coverage. This suggests that these data have produced findings which are worthy of
further consideration, and this will be addressed in the Discussion Chapter (Section
7.3).
instrument are longitudinally consistent with the sample‘s treatment (Dörnyei 2007:
50), and internal reliability is shown by the Cronbach‘s alpha of the factor groupings,
265
which should be above .7 and not below .6 (Dörnyei 2007: 207). According to Field
(2005: 668) Cronbach‘s alpha figures are usually higher with increasing numbers of
items in a scale. For the large scale data the internal reliability is demonstrated in
Table 6.22 which shows the Cronbach‘s alpha figures for the scales, and it can be
seen that they are all above .7 even when there are only three items in the scale.
At the smaller scale, data from individuals or small groups, such as those obtained
from the ELTCS presessional group, can be examined for consistency with the
―treatment‖ which the student has undergone. In the case of the ELTCS presessional
What emerged from my consideration of Student P-St-a in Section 6.3.5.2 was that
the questionnaire reliability was dependent on the purposes to which it would be put.
As I was not testing the learners and as autonomous learning is dynamic and variable
the questionnaire results were not written in stone but demonstrated how changeable
with its focus on reliability over extended periods appears, after this experience with
the present questionnaire, to be unnecessary. I feel now that a lighter more nimble
instrument could be more appropriate for a more ―real-time‖ picture of the changes
in autonomous learning.
266
6.5 Formative benefits of the questionnaire
In Table 6.23 below I present the feedback concerning how respondents felt about
the questionnaire. The vast majority of the feedback concerned the questionnaire‘s
length (for the Long List), or comments about individual items which I have not
included here. The feedback indicates that a closed-item questionnaire of this type
can help learners to reflect on their learning, and this clearly has benefits in relation
able to fulfil a formative function quite apart from any value it has as a measure of
autonomy and as a source of information for teachers. The formative aspect will be
independently.
C -
D BNU ―it really helps me to think carefully about my own studying habits.‖
―It is very detailed, and it makes me think over my leraning style.I think it
is very good.‖
267
6.6 Conclusion
In this chapter I have presented the findings from the analysis of the questionnaire
data, interviews with a teacher and a student, and observations of students. I have
questionnaire.
Analysis of the questionnaire data at the larger scale was used to look at the construct
validity of the instrument using factor analysis to find the model embodied in the
questionnaire items. This had encouraging results as a model (which was not the
same as the areas used in the initial item selection) was found which compared
related construct, and with this the emergence of the unexpected factor Social
learner‘s morale or psychology, or perceptions of their place in the group, and that
this could make autonomous learning highly variable in level over time. Both
At the small scale, data analysis has suggested (see Section 6.3.2) that the teachers
estimated the autonomy of their students mainly based on language ability and
268
confidence, but that the teacher estimates and questionnaire average are close. Some
and techniques used which show promise of being effective for investigating the
In the next chapter I will explore in greater depth the important issues arising here
such as the use of teacher estimates in the research and the questionnaire‘s
conceptual model. I will return to reconsider my original aims in the light of the
research, and I will also look at how the questionnaire may be used. The issue of
269
7 DISCUSSION
7.1 Introduction
The chapter can be broadly divided into four parts. Firstly, in Section 7.2 I look at the
7.3 and 7.4 I explore the questionnaire‘s conceptual model, compare it to the
Thirdly, in Section 7.5 I reassess the original aims of the research in the light of the
data and the experience of using the questionnaire with learners. In Section 7.6 I look
at how the present questionnaire may be used. Finally in Section 7.7 I consider the
As part of the validity check of the questionnaire I compared its results with teacher
estimates of autonomy. This involved only two teachers (including myself) and so
the evidence is, statistically, suggestive only. However, it is worth examining the
relevant to the aims of the present research which involve exploring the possibility of
For the two groups ELTCS year-3 and ELTCS presessional P2 there was a
significant correlation (p = < .05) between Teacher Estimates and the Categories
Average (see Section 6.3.2.1), which suggests that the questionnaire could match
270
Teacher estimates correlated most with the areas of Linguistic Confidence, Social
Comparison, and Locus of Control (see Section 6.3.2). Within the questionnaire
Social Comparison and Locus of Control are also the categories which correlate most
with the Categories Average (see Table 6.7). This correlation is an indication of a
by producing reliable and valid findings earlier than can a teacher and potentially
The idea of assessing the questionnaire‘s validity by comparing its results with
teacher estimates may be called into question since teacher estimates may not be
acceptable as a valid standard for assessing autonomy and therefore they would not
teachers, or a little quicker, then it would be of practical use for a teacher. This shows
that it is not necessary to compare the questionnaire more directly (in some way)
with autonomy for it to be a useful tool. The question is not whether the measure is
therefore help teachers to know their students more quickly and support their
questionnaire research can be recast in terms of this function; rather than being
271
can be seen as a teacher-estimate emulator or a way of improving on and
students better (the latter is an issue which is mentioned by the ELTCS year-3
the picture so that in the process I can further probe its construct validity.
1 2 3 4 5 6
1 Pearson
Sig. (2-tailed)
2 Pearson -.023
Sig. (2-tailed) .764
3 Pearson .346(**) .326(**)
Sig. (2-tailed) .000 .000
4 Pearson .423(**) .309(**) .415(**)
Sig. (2-tailed) .000 .000 .000
5 Pearson -.005 .614(**) .322(**) .265(**)
Sig. (2-tailed) .949 .000 .000 .000
6 Pearson -.012 .492(**) .248(**) .166(*) .593(**)
Sig. (2-tailed) .880 .000 .001 .030 .000
CA Pearson .376(**) .661(**) .751(**) .709(**) .665(**) .569(**)
Sig. (2-tailed) .000 .000 .000 .000 .000 .000
** Correlation significant at 0.01 level (2-tailed). * Correlation significant at 0.05 level (2-tailed).
1. = Linguistic Confidence. 2. = Information Literacy. 3. = Social Comparison. 4. = Locus of Control
5. = Metacognition. 6. = Self-Reliance. CA = Categories Average
Table 7.1: Inter-Category correlations
The six categories found (Table 7.1) appear to fall into two broad areas, Technical
(knowledge and skills), and Psychological (including affect, which can block or
promote the actualisation of autonomy). The first can be seen as what the learner
knows about learning, and so would be the potential for autonomy given the lack of
any blocks to its actualisation. This area is composed of two factor groups,
272
composed of the factors named Linguistic Confidence, Social Comparison, Locus of
In this interpretation of the picture emerging from the questionnaire, the technical
area would be the knowledge and skills which are necessary for autonomy.
The category was identified in Section 6.2.3.2 and is composed of the following
knowledge of the major current resources available in at least one field of study
273
Candy et al. did not obtain this characterisation from factor analysis of data, but from
Their characterisation matches well with some of the items in the questionnaire
category, covering much the same ground. Critical evaluation of information is not
overtly stated in the questionnaire category, but two items, 125 and 175, do seem to
embody this idea. Item 125, which addresses writing rather than finding information,
can be seen as evidence of a critical skill, albeit more productive than receptive. Its
interpretation in the context of information literacy is that the writer can be seen as
mentally putting him or herself in the position of the reader and evaluating what is
being written (or will have been written), and so in order to empathise it is necessary
to be aware of how communication can be made effective. Item 175 fits in with the
the knowledge and ability to find and interpret information for one‘s own learning
7.3.2.2 Metacognition
274
130. My writing is better now than it was a year ago
212. I talk to others about how I feel about learning English.
138. I know techniques to help me remember vocabulary.
109. I predict the content before I listen.
049. I have changed the way I learn after thinking about it.
046. I can describe the learning strategies I use.
142. I fix my problems in vocabulary.
130. My writing is better now than it was a year ago.
238. I choose the exercises I work on.
but mainly it is the skill of reflecting. It is much emphasised as being a key area of
knowledge is generally categorised into three types (Wenden 1998: 518-519), person
knowledge, task knowledge, and strategic knowledge (see Section 2.4.5). Person
knowledge, i.e. the knowledge and beliefs learners have about themselves and their
ability as learners (in general and for particular tasks) appears to be addressed by two
items in this category, 237 and 130. Task knowledge, i.e. knowledge of the purpose
and demands of the task, appears to be addressed by item 238. Strategic knowledge,
i.e. the awareness of strategies and how and when to apply them, appears to be
involved in five of the items: 212, 138, 109, 49, and 46. The remaining items also
involve the use of metacognition: item 142 is person knowledge and also implies task
and strategy knowledge; and item 234 implies a belief that the learner has sufficient
extent. This tends to support the idea that metacognition is a concept that reflects a
class of beliefs or behaviour that is present in learners. Reciprocally, this means that
the literature provides a level of support for the idea that the questionnaire is reliable
in that it has independently grouped items which are also grouped in the literature.
275
This is the area of overlap between the technical area of skills and knowledge with
the psychological area. It is the conscious awareness of skills, and outcomes, and
strengths and weaknesses, and awareness of one‘s own psychology which permits
…to have, and to hold, the responsibility for all the decisions concerning all aspects of this
learning, i.e.:
- monitoring the procedure of acquisition properly speaking (rhythm, time, place, etc.);
This requires self-awareness combined with the knowledge and skills necessary.
psychological states and the context in which one finds oneself. Metacognition, then,
appears to be the keystone which holds together all the other areas and is an essential
component in autonomy. Without it the other areas are merely mechanical and not
directed with conscious understanding (Lai 2001); it is what makes the learner a
7.3.2.3 Self-Reliance
The category was identified as ―Self-Reliance‖ in Section 6.2.3.6. There are no items
which mention working with others or place the learner in a social setting, so this
This seems to have an equivalent in Candy et al.‘s ―sense of personal agency‖, which
Self-concept, self-efficacy, and self-determination are combined with the skills that
support and justify the individual‘s sense of capability. Candy et al. are specifically
addressing the idea of lifelong learning, which is a concept that comes to autonomy
277
110. Every word is important for understanding a listening text.
194. The other students are more confident than me at speaking English.
193. The other students know English better than me.
187. I think learning English is more difficult for me than for the average
learner.
These two categories will be discussed together as they were both linked with
Linguistic Confidence) more closely to see if the two types of confidence which
came out of the interview with the ELTCS year-3 teacher (see Section 6.3.3) match.
The analysis of the interview suggested that there were two kinds of confidence
being talked about: (i) confidence about the ability with the language, and (ii)
confidence regarding relations with other people. Do these two types of confidence
Confidence, but a match is not safe between the interview type (ii) confidence and
Category 3 Social Comparison. In the interview with the ELTCS year-3 teacher she
was not about introversion or extroversion; it was about the confidence or insecurity
which can result from making comparisons of oneself with peers. This could still
learner having confidence and being less inhibited in learning in a class situation.
James (2001/1892) divided the self into two main components, the ―Me‖ and the ―I‖.
He had ―Me‖ as, among other constituents, the social self (2001/1892: 46) compared
278
with ―I‖ for the volitional self. This division is followed by Damon and Hart (1982)
in their review of research into child psychological development which showed that a
child gains a more sophisticated sense of a social self as he/she develops. The word
―me‖ occurs five times among the full set of items in all the questionnaire categories
(compared to ―I‖ occurring 57 times). Of the five instances of ―me‖, three are in
Category 3, which is also the only category to have more than one instance,
suggesting that this category is the individual more objectified, that is, seen in a
and it does not seem to have been fully explored in many models of autonomy (e.g.
Oxford 1990; Cotterall 1995; Victori and Lockhart 1995; Wenden 1995; Murray
1999). Littlewood (1996), who sees it as one of the four main components of
autonomy, does not discuss it in any depth. The connection between confidence and
that actualised autonomy requires ability and willingness, the latter being composed
The analysis of the categories in the questionnaire is very similar and it tends to
279
Cotterall (1995) identified one of the factors which she found in her research as
―learner confidence in study ability‖. It was composed of two items: ―I know how to
study languages well‖ and ―I know how to study other subjects well‖ (see discussion
identification, and confidence is not overt in the wording of the items, but it does
mean that at least one other researcher has identified a confidence factor which,
while not corroborating the findings of the present research, does at least make it not
entirely unprecedented.
Stanton (1988: 127) points out that ―...if a person believes he is anxious or lacking
true.‖ If they can be helped to improve their confidence they will feel ―more
competent to transcend the limits they have been placing on themselves‖ (1988:
131). However, it seems that for learning confidence to be able to enter the beneficial
cycle there has to be a recognition on the part of the learner that it is their
responsibility that they have succeeded, and that they do not see it as luck or the
Crick & Wilson (2005) maintain that confidence is required to accept the
situation would lead to ―Fragility and dependence‖ (Deakin-Crick & Wilson 2005:
372). The relationship with others brings us back to the idea of social comparison
being related to confidence, a link which can be seen in the correlations in Table 7.1
above, and is also presented visually in Figure 7.3 (which can be found in Section
7.3.4 below).
280
Williams and Burden (1997: 97) review the multifaceted make-up of self-concept,
and their description is summarised in Figure 7.1 below. From the perspective of
others, and therefore social comparison will affect confidence. Failures are more
likely to result in lack of confidence when learners are motivated by the (external)
goal of gaining peer approval, rather than having a more internal focus on effort and
strategy use (Dweck 1986: 1046) which is perceived as being their responsibility.
This is in line with other research into the influence of the learner‘s group on the
to be in harmony with the others in the learning environment this ―promotes student
The level of confidence one feels in self-comparisons with the group will affect the
quantity and quality of interaction. This is important because, as Arthur (2001: 43)
indicates:
...by taking part in pair or small group work learners can [...] develop the confidence to ‗let
go‘, to make mistakes in front of others, to take the initiative and to experiment with new
281
This will potentially lead to a virtuous cycle of confidence feeding in to achievement
and in to confidence again. Without engaging with the group, as may be the case
with learners who perceive themselves as comparing negatively with their peers,
learners who give such comparisons too much importance may be held back. This
can work in two ways, both of which will take away from the sense of control and
therefore be harmful to learners‘ perceptions of autonomy. They are either (a) that
the inhibitions tend to make the learner perceive the locus of causality as being
external; or that (b) these negative external comparisons may result in an internal
There are strong correlations between Locus of Control and both Social Comparison
and Linguistic Confidence (see Table 7.1 and Figure 7.3). An autonomous learner
will need to be able to control his/her learning (to the extent it is practicable). Social
Comparison, which can be useful, could also be harmful to confidence and control as
one‘s perceived status in relation to others is linked with one‘s feelings of self-worth
and will have an effect on decision making. Learning involves accommodating new
ideas into one‘s own matrix or schemata which therefore requires that an individual
plays an active role in learning. If this part is insufficiently played and inhibited by
There is in the literature a link between confidence and motivation, for example
Vandergrift (2005: 83) says that a lack of self-confidence and self-efficacy leads to
lowered motivation to act, and Burt (2004: 7) says that self confidence in their ability
and their perception of control of their learning play an important role in learners‘
motivation.
282
Motivation has not overtly figured in the questionnaire analysis, yet it should play a
sustainable (Ushioda 1996: 22). An attempt to account for the apparent absence of
Section 2.4.7). Autonomy involves making one‘s own choices and taking
responsibility for managing one‘s own learning. An internal locus, where the learner
feels able to control, is preferable for autonomy to an external locus, where the
Looking at each item it is clear that they do address an area which is at least closely
comparable to the idea of locus of control (all the items were reverse coded for the
questionnaire). For example, learners agreeing with items 95, 230, and 246 would
appear to be non-exploratory; and learners agreeing with items 140 and 189 would
appear to see learning as beyond their control; both of these are areas specifically
283
attributions can be improved by training, and it is therefore a component of
autonomy which can be said to have levels. This has implications for teaching, and
Williams and Burden (1997: 101-103) cite studies which have reviewed ways of
control of their learning through, for example, practice in planning or finding and
organising information.
technical ability sufficient for the task is not enough in itself to motivate the learner
to act autonomously at any given time. There are helps and hindrances which filter or
block the learner‘s potential, which are such psychological influences as the learner‘s
the task and actualisation of potential for autonomy may be stronger or weaker
depending on how much potential there is and how much it is blocked or not. This is
analysis.
battery, resistor, and bulb (see Figure 7.2 below). The potential comes from the
battery, this is passed or blocked to varying degrees by the resistor which represents
the psychological (motivation, confidence, etc.) and other influences which may
interfere with the potential. The bulb represents the task, and the completion of the
circuit represents autonomy with all the areas, necessarily, involved. The
questionnaire probes the potential, the resistance and the task completion (though all
284
from the perspective of the respondent‘s self-report, not by observation of the
the ―circuit‖ is here the feedback to the metacognitive knowledge, which can, as it
were, charge the battery with new (process, metacognitive) knowledge, which would
mean enhanced ability to overcome resistance, can make the light brighter, or can
light a higher wattage bulb in the future, that is, the learner learns from the fact of
having done a task which gives the learner enhanced abilities to complete more, or
A = Battery/Potential.
B = Resistor (or dimmer switch)/areas affecting potential.
C = Bulb/Task
Teachers can promote autonomy by increasing the ―power‖ in the battery (learner
training), by reducing the resistance (for example by making the context more
supportive of autonomy), and by having a ―light bulb‖ or task that is not too high a
―wattage‖ for the potential in the battery and the level of resistance, i.e. the teacher
can grade the task appropriately for the level of development of the learner.
The circuit can be looked at in different ways, according to the point of view:
285
as basically two areas, capacity (skills) and action (with feedback from
as three areas (potential (A), psychological boosts or blocks (B), and task
Figure 7.3 below represents the relations between the categories found by factor
analysis of the questionnaire data. The full questionnaire data were used for this
found. The model shows that Social Comparison, technical skills (Information
Literacy), Metacognition, and control (Locus of Control) are key areas, having the
Since this is a product of the questionnaire data it can be seen as a model of what the
political autonomy, etc.), and it will need to be investigated with reference to the
Confidence (see Figure 7.2 above). Metacognition is the conscious reflection which
allows autonomy to be intentional, and this intentionality is necessary for the learner
286
Confidence
(Linguistic
Confidence)
Social Control
Comparison (Locus of Control)
Metacognition Independence
(Self-Reliance)
Technical Skills
(Information
Literacy)
In the model above the strength of correlation arrows indicate two areas of groupings
bound together with strong correlations, one around Confidence and one around
Technical Skills. The correlation arrows link all the areas in the model together, but
the arrows indicate weaker correlations between these two nodes and stronger ones
within them. The Confidence grouping is linked to Social Comparison and Control,
while the Technical Skills area is linked to Metacognition and Independence. This
reflects well the division between technical skills and more psychological areas in
287
In this model, Littlewood (1996) sees two main components of autonomy,
responsibility for a task, and Ability which covers the knowledge and skills required.
MOTIVATION KNOWLEDGE
CONFIDENCE SKILLS
WILLINGNESS ABILITY
TO
MAKE AND CARRY OUT CHOICES
IN
Figure 7.4: Littlewood’s (1996: 430) model of the components and domains of
autonomy
In this section an arrangement of the categories has been suggested, and how they
may logically be seen to interact has been discussed. The suggested model has the
autonomy. Areas which are low, comparatively, can be hypothesised as causing the
288
―bulb‖ to burn less brightly. In a classroom situation these areas could be targeted by
the teacher in support of autonomous learning. Alternatively, the task can be adjusted
A notable absence from the picture from the questionnaire data is the area of
motivation. This is notable, especially since it was highlighted in the review of the
literature (Section 2.4.7) as an important area, and as such was included in the areas
efficacy, control etc. (see Section 2.4.7), and is therefore in a sense hinted at if not
actually overtly present. The lack of a clear motivation element could appear to be a
example:
289
Most of these items were eliminated during the selection process (see Section 5.1),
but even in the preliminary factor analyses there was no evidence of motivation
emerging as a factor. Perhaps it is a ―can‘t see the wood for the trees‖ situation.
Fazey & Fazey (2001: 345-346) describe the key features of autonomous people as:
Intrinsic motivation
Perception as at the locus of control
Take responsibility for their actions
Have confidence in themselves
In the questionnaire model (Figure 7.3) categories are there which match with Fazey
& Fazey‘s description: Locus of Control could match with Fazey & Fazey‘s locus of
control and taking responsibility (since the two are logically connected); Linguistic
Confidence could match with confidence; motivation does not have a direct match.
The presence of these in both the literature and in the questionnaire model is
promising. What is missing from the otherwise rather good fit between the
Section 2.4.7) may be discernible. In SDT terms motivation involves the three broad
reasons for action. Matches for these appear to be present, or at least suggested, in
the questionnaire model in the form of the categories named respectively Information
290
The questionnaire Fazey & Fazey (2001) SDT
(model) (autonomy) (motivation)
Information Literacy Competence
(skill)
Metacognition
Intrinsic motivation
Self-Reliance
Table 7.2: Possible equivalences between the questionnaire, Fazey & Fazey, and SDT
questionnaire, as with the Fazey & Fazey (2001) and Cotterall (1995) pictures of
autonomy, is not explicit about motivation, but the SDT elements of motivation, i.e.
with views in the literature (e.g. Dickinson 1995; Ushioda 1996; Benson 2001) that
In the previous section I related the areas embodied in the questionnaire with the
literature. In this section I will consider important areas in the literature and
291
7.4.1 Autonomy and responsibility
(Holec 1981: 3; Boud 1988: 23; Dickinson 1987: 15; Little 1996: 203-204; Little et
al. 2002: 17). It is therefore an area which would be expected to appear in a model of
Locus of Control. In the model this area is most strongly linked to Linguistic
Skills (Figure 7.3). In the model, then, taking responsibility is associated with the
supporting areas of Technical Skills and Confidence. Motivation has recognised links
with the sense of being an agent and so to taking responsibility (Ushioda 2003; Spratt
et al. 2002). This was discussed earlier in Section 7.3.5. In the area of responsibility
then, the model compares well when viewed in relation to the literature.
―the ability to take charge of one‘s own learning‖, and the skills that this involves
including determining objectives, selecting methods, and evaluating what has been
acquired). Littlewood (1996) analyses capacity into two distinct elements, ability and
willingness (Littlewood 1996: 428). Ability is the technical skills, and willingness is
the motivation and confidence. This is a picture which maps onto the questionnaire
model very well. For a person to be successful in acting autonomously all the four
292
A measure of autonomy should not be based on learner behaviour alone (see Section
2.4.4) because behaviour can be misinterpreted, and does not reliably indicate the
underlying intentionality, and may not reveal the levels of knowledge, skills,
motivation, and confidence. A self-report questionnaire should help the teacher to see
beyond the behaviour, by probing it, not by direct observation but based on
respondent perceptions which may be accurate but may also be misleading. Self-
autonomy. However, the questionnaire model which has emerged from the data
model, and therefore it is suggestive that the self-report format is producing data
which are related to autonomy. This is turn indicates a degree of reliability in the
There are seven items in the Short List which refer to interactions with others, but
they did not form a scale. This raises questions about both the usefulness and validity
current concern in the autonomy literature then the construct validity is called into
question. If this area is not represented in the questionnaire then the questionnaire
will not be useful for diagnosing problems in this area. However, the Social
Comparison category does look at one aspect of social interaction, and the
Metacognition category contains item 212 ―I talk to others about how I feel about
learning English‖. The Social Comparison category and item 212 both indicate levels
293
7.4.4 Autonomy is variable
There are four reasons which have been given for autonomy being seen as variable
it is multidimensional
it is sensitive to context
In one sense autonomy can be seen as variable because it has stages in development,
expressed in the literature with band descriptors (Breen & Mann 1997: 143; Nunan
1997; Littlewood 1996). Different approaches are adopted: Breen & Mann (1997:
143) use levels of dependency; Nunan‘s (1997: 195) model is expressed in terms of
levels of ―learner action‖; and Littlewood‘s (1996) levels of autonomy are defined by
the choices which are possible for the learner. These are not practical measures of
autonomy; they are theories about how autonomy may develop in different
dimensions.
After having studied the data from the questionnaire the band descriptors seem to be
breakdown of specific areas which affect the autonomous learning of the individual,
but has not suggested levels. The six components in the questionnaire model were
seen (in the case of the ELTCS presessional class) to rise and fall individually over
time which suggests that using broad phases to describe levels of autonomy will not
classroom situations. I had seen these abstract models as a potential route into
294
operationalizing a measure of autonomy (see Section 2.4.2) but I now see, based on
my experiences with learners and with the data provided by the questionnaire, that an
and that they have limited use for practical teaching purposes.
2001: 47). This is a concern which I raised in Section 2.4.1 and which implies that
Linking its validity to that of teacher estimates (see Sections 6.3.2 and 7.2).
With this clarification of the initial aims it has not been necessary to confront the
In a third way, autonomy can be seen as variable due to its sensitivity to short term
individuals and their willingness to engage in autonomous learning tasks (Carr &
Claxton 2002: 12; Sinclair 2009: 185). These influences can be very short term, such
learning.
Longer term influences can be seen as forming part of the context, and this leads to
the idea that autonomy is necessarily situated. This situated nature of autonomy
would imply that it is inherently variable as it is not separable from all relevant
295
contextual (and contingent) influences. Seen in relation to this, a measure of
autonomy would need to look either at autonomy over a long time period and not
Section 2.5.1.2, takes three years to complete; or be quick and simple enough to
capture short term ―snapshots‖ (Benson 2001: 54) which can then be interpreted in
relation to other current information regarding the learner and context, and would
This type of data has great potential, for example, repeated applications of the
instrument may reveal tendencies in individual students or classes which can then be
Changes in the context (different tasks, times, places, etc.) will influence the level of
autonomy, but the variability will depend on the transferability of autonomy (see
that an instrument such as a version of the one under investigation here may have a
investigation in the present research when fully developed should be appropriate for
this due to its ease of application and its division of autonomy into categories,
assuming that its potential in the present research can be corroborated in the field.
In the following section I will look at an aspect of variability which concerns how
probing autonomy.
296
7.4.5 Self-evaluation in self-report autonomy questionnaires
It is not always clear what has caused a change in a questionnaire result: in item 237
strongly disagree between the two administrations of the questionnaire. In this case
Firstly (a), the questionnaire is, in part, a self-assessment of one‘s ability to self-
assess, and a product of this reflexivity is that, should a student‘s ability to self-
assess improve over time, s/he can be expected to give him/herself a lower, but a
more accurate, assessment than previously. Alternatively (b), at the beginning of the
course she felt ready for new learning experiences, but after five weeks‘ exposure to
unfamiliar ways of learning (in a new country and in a new learning culture) she felt
less enthusiastic and as a result of this she felt less motivation. Thirdly (c), she
misread the question as a negative thus transposing strongly agree and strongly
disagree.
If I were still her teacher I would talk to her; in a normal classroom situation this
would be the response, and it would not be necessary to speculate as I have here. A
use of the questionnaire would thus be to indicate possible problems and enable an
This discussion raises two important points: firstly, the questionnaire will not be, and
monitoring changes should be the primary purpose for using such a questionnaire.
Secondly, the questionnaire will not in fact measure an abstract autonomy, but
297
aspects related to autonomous learning as found in the questionnaire model. The
questionnaire looks at six areas, plus the overall score (Categories Average) and with
Possibility (a) above was a change in the learner‘s ability to self-assess. A pattern
which may illustrate whether this is occurring would be when the questionnaire
scores start relatively high, then dip, and later start a rising trend, as illustrated in
standards
298
By this hypothesis, student P-St-a is entering Phase 2. A prediction from this would
be that a third administration would show changes in her scores which fit in to the
The concept of a ―dip‖ as indicating progress does have a precedent in the literature;
something analogous is proposed by Breen & Mann (1997: 142-144) concerning the
introduced (see Section 2.4.2.1). In the wider language learning literature it also not
... the acquisition of forms such as ‗went‘ follows a U-shaped pattern of development, with
children first using it correctly (for example, ‗went‘) and then incorrectly (for example,
‗goed‘) before they finally once again produce the correct form (‗went‘). (Ellis 1994: 77)
It indicates the use of the rule which is evidence of progress beyond the level of
The parallel with the ―autonomy dip‖ is that something that has been learned
that it appears to have relapsed but a more important advance has in fact been
achieved. This underlines a point made above that the questionnaire results should be
seen in context with other indications (e.g. homework, participation etc.), and with
299
7.4.6 Summary
In this section I have looked at important areas in the literature and investigated
whether the questionnaire and its model have accounted for them. The questionnaire
model was investigated with reference to standards from the literature and was found
social interaction were all found to be present in the model. Littlewood‘s (1996)
framework was shown to correspond well to the questionnaire model. The variability
of autonomy which has been viewed as a threat to measurement was accounted for
abstract autonomy, but as a source of data which are related to autonomy for use in a
and it was underlined that the questionnaire should be primarily comparative and
used in conjunction with other sources of data, not least the learners themselves.
As a result of the research I have been led to question and clarify the original
concepts of measurement and autonomy which were used in formulating the aims.
Regarding the first question, Dam (2000: 48-49) was careful to specify that she was
abstract, more practical and more ―in-class‖ concept fits well with my initial
300
motivation for this project, a practical tool to help teachers support their students. It
seems clear now after engaging in the research that the questionnaire should be
maintain that the model is a new model of autonomy, and that the questionnaire is
change of emphasis which clarifies the original expression of the purpose of the
notion of autonomy. The instrument then would be aimed at classroom use, and
would need to have advantages over the present methods of measuring autonomy,
but it would not need to be a measure of the abstract notion of autonomy, but of areas
relevant to autonomous learning in class. This meant that it would then be possible to
compare the questionnaire results with teacher estimates based on observations and
My research has tried to deal seriously with checking the construct validity of the
questionnaire. The model does appear to match areas of the literature of autonomy,
The analysis of the data has found that the questionnaire does have potential for
helping teachers to judge their learners‘ autonomy. Cofield (2002: 40) has pointed
out that:
301
... the inability to define a concept precisely does not have to act as a barrier to measuring it;
indeed empirical research into a complex concept may lead in time to a more precise
definition or definitions
but the factors which have been found are variable and hence the construct which has
been found is of the measurable kind, and has been given the name Questionnaire
of autonomy-related statements.
questionnaire?), the idea of a ―measure‖ has been clarified and become more
nuanced in the course of the research. The questionnaire provides information on the
six areas in its model which can be monitored separately rather than as one
relatively in a context and with a more holistic attitude to the individual learner. The
system rather than being similar to an IQ score. When the context changes it should
not be assumed that this will have no influence on the individual‘s autonomy. It is
not expected that the questionnaire will give absolute measures of autonomy.
However, it may be useful for comparative measures, i.e. within a group, and for
302
Benson says that snapshots are misleading (2001: 54). In the questionnaire this can
with other knowledge which the teacher or researcher has about the respondent.
The components of autonomy may be stable or unstable in learners; if they are stable
then a snapshot will be less limited. What will be important is that the snapshot is
quick enough to provide many such snapshots to build up a more informative picture
not to link the questionnaire scores to an independent standard, but look at them in
comparison with others from the same or a comparable class or with the same
individual student if the scores are seen in context. Comparison with the respondent‘s
previous results could potentially indicate when a student is not progressing well and
act as a warning signal to the teacher who can then communicate with the individual
learner and ascertain whether it is necessary to consider taking some remedial action.
way of raising questions to be asked of the learner. This means that the questionnaire
can provide indications about how the learner is placed regarding their levels in the
categories, but that these are better seen as initiating or contributing to a more
303
The development and use of this closed response quantitative questionnaire has
more detail in the one interview carried out, and then the interaction of these
experiences with the questionnaire data has shown me that the questionnaire adds a
questionnaire results (see Section 6.3.5.2), but in a normal teaching context a teacher
would be able to interact with the learner. This points to the importance of the
questionnaire being used by a teacher who knows the class and is more aware
therefore of what has been happening in class. It also points to the importance of
knowing one‘s class, that one cannot support autonomous learning only in an abstract
way but one has to be involved with the class and individual learners in order to be
The use made of the questionnaire by a teacher would need to be similar to the use
that is made of teacher estimates. By this I mean that if the questionnaire can be
about teaching methods or contents which go beyond what teacher estimates may
304
A number of roles for the questionnaire present themselves. The instrument is quick
and can be administered in class, so that teachers will be able to see with only a small
investment of time whether their assumptions are correct. They may see that their
students have more potential than they imagined, and may feel, as a result of having
this evidence, more able to modify their teaching to help learners actualise their
potentials for autonomous learning. Viewed in this light, the questionnaire is as much
a tool for teacher support as it is for the learners‘ benefit. Eventually, it could be an
important tool for the review of teacher estimates and provide empirical evidence
which could help to influence institutional beliefs and practices which may have
with repeated applications over long periods to establish which categories are quicker
to change and which tend to be more stable. This will help with the understanding of
how autonomy develops and so indicate ways to enhance its promotion. As Benson
To date, however, research does not provide conclusive evidence on the mutability of
individual variables in learning, their interrelationships, or the role of experience, training and
feedback received (see Section 6.5), i.e. the learners will be thinking about areas they
may not have considered before, and this will potentially feed back into their
305
As the questionnaire may reveal comparative weaknesses in the respondents‘
component areas, this information could be used for diagnostic purposes to see
whether learner training or other interventions are required. Appropriate steps can
then be taken at individual or class level to help learners improve in the indicated
areas and so enhance their autonomous learning ability. Take, for example, weakness
learners‘ confidence (Gan et al. 2004: 401), and it is widely seen as important (Ellis
& Sinclair 1989:3; Oxford 1990: 1; Victori & Lockhart 1995: 223; Wenden 1995:
192). One way of doing this is for the teacher to arrange ―study buddies‖ or peer
tutoring schemes; Mynard & Almarzouqi (2006: 14) point out that peer tutoring
programmes are beneficial as they lead to ―making friends; building confidence and
esteem and self-confidence, even for the more inhibited learners (Walker 2001: 92).
different context, for example self-access learning might be indicated for a low
allows those who are shy to work at their own pace without having to be judged
boon to those whose questionnaire scores indicate that they are inhibited in relation
306
The questionnaire could be used to augment teacher evaluations or to enrich the
information from learners‘ logbooks. Logbooks are not always clear, they depend on
the level of the learner, and the learner may not automatically cover all the
component areas.
The questionnaire could be a tool for teacher development. Chan (2003: 43) found
among teachers ―a strong preference for a relatively dominant teacher role‖ which
suggested that teachers did not expect learners to take on an autonomous role.
Teachers did not believe that students were ready to accept responsibility for their
learning, and it was also thought that learners saw it as the teacher‘s role to make the
decisions about learning. In addition, teachers also thought that it saved time if the
teacher made the decisions (Chan 2003: 49).Such attitudes sound very familiar and I
have encountered them in my teaching in the Middle East. There is a clear mismatch
between what teachers understand as autonomy and what they see as realistically
achievable in their cultural and institutional contexts. If such views are as widespread
relatively low. Davis (2003: 212) reports that ―Findings suggest teachers‘ beliefs can
shape both the quality of their interactions with students as well as the quality of their
instruction‖. It may be that the questionnaire could be used to change teacher beliefs
about their learners and so potentially improve the environment for autonomy in their
classrooms.
The questionnaire was not intended, and is not designed, to be appropriate for use as
a high stakes test, in contrast to Ravindran‘s (2000) CILL (see Section 2.5.1.2). The
characteristics of the present questionnaire mean that it can be used in different ways
from such instruments as CILL. The present questionnaire differs from the methods
307
adopted by Ravindran (2000), Lai (2001), and Dam (2000) in that it can be used
within a pre-existing course without the need for a new curriculum. This is an
informally in his or her own classes. The questionnaire can be administered in a short
time and can provide more immediate information than the long term methods and
without the need for a well-trained team. The questionnaire could provide data, in a
quick and economical way, which could support and complement teacher estimates.
The questionnaire will also not depend on the commitment of dedicated and expert
In Section 3.13 I explained my reasons for translating the questionnaire and gave the
translation procedure which was used. Checking the translation using the technique
of back translation proved to be an important stage. In Table 7.3 I have classified the
types of problems found in the back translation of the Long List from Chinese, and
give some examples (for the full list of items and back translations see Appendix
10.1). Sometimes there is one clear source of error, in other cases there are
combinations of problems, for example item 203 shows two problem areas,
Feedback suggests that the translation was a success; prior to translation there was a
with the rate after translation of only 1.85 per 100 respondents. The fact that there are
fewer queries strongly suggests that respondents were better able to understand the
questionnaire. Translation clearly had benefits for the reliability of the instrument
308
and the quality of the data gathered compared to the untranslated instrument, and
therefore the data indicate that translation should be considered for questionnaire-
based research where the original questionnaire is not in the respondents‘ first
language.
False 071. If I learn something well, it is If I learn well, that is because I work
assumption because I studied well hard.
004. Learning continues all your life It is never too old to learn.
Superficial 024. The student‘s job is to develop as a The task of the students is to develop
similarity person individually.
Trans- 220. I am ready to learn in unfamiliar I have made preparations for learning by
position ways using unfamiliar methods.
Emphasis 203. I organise my time for studying I arrange my learning time soundly.
added
Over- 233. I know how to check my own work I know how to examine my homework
specified for mistakes and find the errors out.
Table 7.3: Classification of translation problems in the Long List with examples
considered, and the translation process into Chinese revealed some assumptions in
items, which were then changed. At this level of attention to clarity across cultures
309
There is another level to be considered, that of the appropriateness of Western
very much based on ideas of learner autonomy which are thought to originate in the
West. No feedback on the questionnaire has been received from respondents which
criticises it in this area, which is an indication that the questionnaire is not causing
respondents, consciously, to feel imposed upon. The suggestion from this is that the
books on research methods in general (for example Nunan 1992; Punch 2005) or in
volumes focusing on questionnaires (e.g. Dörnyei 2003). I had to look beyond the
language teaching field to find most of the information I used for guidance (see
Section 3.13).
Most papers in the language teaching-related field which use questionnaires in the
research do not mention translation (e.g. Cotterall 1995), but a minority of them do
refer to the translation of the instrument, and some describe the procedure and
rationale. Mynard & Almarzouqi (2006: 22) briefly state in a footnote that ―Tutees
were given […] this questionnaire with an Arabic translation on the reverse side‖ and
Prodromou (1992: 43) mentions parenthetically that ―The questionnaire was given to
For this study, SILL (Oxford, 1990) was translated into Chinese by the author first, and then
was checked by a professional translator from the New Zealand Translation Centre
Spencer-Oatey & Xiong (2006: 41) and Tseng et al. (2006: 87-88) are moving
towards committee translation (see Section 3.13.4.3) as both pieces of research use
310
several well-qualified individuals to translate and check the translation. Gan (2004:
407) describes using back translation (see Section 3.13.4.2), describing how:
The questionnaire was issued in Chinese. A preliminary version of the questionnaire items
was initially formulated in English. These items were then professionally translated into
Chinese. To further ensure the validity of the questionnaire, the Chinese version was
Sometimes bilingual instruments are used (Spencer-Oatey & Xiong 2006; Mynard &
Almarzouqi 2006) which have both languages present but this is not the
3.13.4.6).
The approaches used in the examples above are all well-intentioned but give the
rather than being informed by an awareness of the issues and procedures which have
been discussed earlier in this thesis (see Section 3.13). I maintain that there is a need
should be an integrated part of the plan for the research and not a late addition
because it can be an involved process to organise and carry out, especially if the
7.8 Conclusion
In this chapter I began by discussing and justifying the use of teacher estimates for
validation. I then examined the questionnaire‘s construct and presented and probed
close links. I have also developed and clarified the initial aims of this research with
311
uses for the questionnaire. Finally I assessed the advantages of translating the present
questionnaire.
In the next chapter I will return to the research questions to consider whether they
have been answered and I will assess the contribution made by this research.
Limitations will be discussed and areas for further research will be considered.
312
8 CONCLUSIONS
8.1 Introduction
In this chapter I will answer the research questions posed in Section 3.3 and look at
some of the implications of the research. I will consider what contribution has been
made, will consider its limitations, and suggest further research which may develop
what has been started here. Finally I will give my concluding remarks.
measure of autonomy? (What are the issues involved and can they be
autonomy?)
promise in matching well with previous published work in the field, such as
that autonomous learning has multiple dimensions which vary, and that indications
about these can be obtained using a questionnaire which can thus serve a useful
purpose in the classroom when the data it provides are viewed in context and in
313
Comparison of the limited data from teacher estimates with the questionnaire results
has shown correlations between them (see Section 6.3). It has therefore been
relevant to what the two teachers (myself and the year-3 ELTCS teacher) involved
understand as autonomy, and therefore, seen in these terms, the questionnaire shows
The variability of autonomy is reflected in the instrument‘s results (see for example
Section 6.3.5.2) which means that the data the questionnaire provides regarding a
learner must be seen as a snapshot which may soon be out of date; but such
informing a current and relevant engagement with learners which can enhance the
teacher‘s interaction with the learner and so enable the teacher to better support the
learners‘ autonomy (see Research question 2 in Section 8.2.2 below). This shows that
learning and cannot make any claim (and nor is it necessary to) to measure autonomy
I set out to see whether it was possible to have a measure of autonomy which was
quick and practical. I now have discovered that expressing this aim with the
indeterminate use of the word autonomy is too abstract to be useful for the practical
314
but ―autonomy‖ is not, it is a ―Platonic Idea‖, a concept without substance until given
a shape in a context such as a classroom where it can inform events, and be part of
environment.
My research has highlighted that the published techniques for measuring autonomy
much quicker than the alternatives found in the literature and requires no
can even be used informally by individual teachers as it takes only a few minutes of
class time. The literature describes methods which require weeks, or even years. The
main competitor with the present questionnaire in terms of speed and low investment
the same quantitative principle, may (based on the limited data so far) have the
potential to offer general indications about learners more immediately than the
teacher estimates of their autonomy (see especially Sections 7.2 and 6.3.4). The
questionnaire has advantages over initial teacher estimates, providing more detail of
315
category, and teacher estimates appeared to move towards the questionnaire results
The questionnaire could also act as a standard procedure which would not have the
which is repeatable for a teacher in his or her context and so would help teachers and
A closed-item questionnaire could inform the dialogue between teacher and learner,
raising questions and indicating possible problems which can then be followed up.
with a time-limited answer, but an answer which is still important and useful for
purposes of building and maintaining the quality of interaction between teacher and
learner, and this would be particularly advantageous with a large class, as was
indicated by the interview with the year-3 teacher (see Section 6.3.2.3).
A limitation which teachers must be aware of is that the questionnaire cannot ask and
answer all the questions about an individual‘s autonomy and the questionnaire results
should not be looked at in isolation. However, the data suggest that it does have the
Section 7.6).
Feedback from respondents (see Section 6.5).strongly suggests that the questionnaire
has a potential to help individual learners reflect on their learning in ways they have
not done before and it could therefore be useful as a formative aid in developing their
316
The use of the questionnaire will not be to measure autonomy (still less to test it), but
periodically and which can add to the information you obtain by looking out of the
none the less, with the advantage of being within the means of an individual teacher
or learner. This fits in well with my earlier argument that autonomy should not be
choose to use themselves to self-diagnose and so it can support autonomy from the
bottom up.
I have divided the contributions to knowledge into three sections, dealing with
This thesis has highlighted transferability as an issue in autonomy (see Section 2.4.2)
since authors have claimed on the one hand that autonomy is necessarily situated,
context dependent, or task dependent (e.g. Dickinson 1987; Carr & Claxton 2002),
but on the other hand it is claimed that a major part of the importance of autonomy is
that it is a valuable transferable quality to have for life, which is expressed for
317
example in the Bergen Definition (Dam et al. 1990: 102). Thus a question that needs
transferable, and what is the extent of a ―situation‖, i.e. when does it cease to be the
The word ―snapshot‖ suggests something rapid and casual (Oxford Concise
Dictionary, 1982) and instruments to measure autonomy will, says Benson, (2001:
54), provide only misleading snapshots. It would appear from this that such
of learner autonomy. However, I have argued (see Section 7.4.4) that they are a
highly important source of current information which can be used in the classroom
provided that their limits are observed. A learner‘s mood, recent experiences, and
feelings, which all vary, will have an influence on the choice of response to
questionnaire items. The questionnaire need not be criticised for reflecting these
transient states, but the use that is made of the data and the conclusions drawn from
them must be appropriate to the nature of what is being measured, in particular one
still useful information for a teacher to help them engage with learners‘ autonomy
and so can be added to the range of information which can be used by a teacher to
318
8.3.1.3 Confidence
autonomy which makes it less interesting in the context of current thinking about
research has links to confidence which indicate that confidence is socially situated
and forms part of the social context relevant to learner autonomy (see Sections
The approach adopted in this thesis involved pursuing one line of research (the
appropriate to researching autonomy. This was combined with the use of more
which I have explored how autonomy can be investigated, and in particular whether
approach was an appropriate means of researching autonomy, but rather was a means
for me to investigate for myself the limits of positivism in the field of promoting
319
learner autonomy. This critical reflexive approach has allowed me to experience in a
practical, vivid, and first hand way, from the bottom up, what had hitherto seemed to
approach has allowed me to see, in a way that has been very effective, that positivist
tools used appropriately and sensitively can complement what should be a primarily
This research has shown that in principle it is not necessary to compare questionnaire
results against some objective standard of autonomy which does not exist. The lack
establish the construct validity of measurements of autonomy in the past, and I have
highlighted this in relation to Ravindran‘s (see Section 2.5.1.2) and Lai‘s (see
Section 2.5.1.4) schemes. In my method the idea of comparing with the current de
facto method (i.e. teacher estimates) has been proposed as it is more relevant to the
realities of actual class teaching, and because estimation is the existing method of
It is not necessary to prove that the other measure (in this case teacher estimates) is
actually measuring autonomy, only that it is the accepted way, even if it is accepted
only for want of a better method. Since I was looking for an instrument for practical
use I was able to justify using comparison with teacher estimates of autonomy in the
classroom and did not need to find an objective measure of abstract autonomy. The
320
it is accurately emulating teachers, and can therefore help teachers to know more
quickly their students and interact more effectively with them. If the new measure
can be shown to have advantages (such as providing equivalent results but delivered
more quickly) over the estimates the analysis will have been productive, even though
it has not proved that the new instrument is actually measuring autonomy.
It has been illustrated in this thesis that there is a useful distinction to be made
between measuring and testing (see Sections 2.1.2 and 2.3); one can measure without
testing and this can remove apparent objections and inhibitions to the use of
quantitative instruments to obtain useful information about learners which can help
constrained (Champagne et al. 2001: 49) and clearly distinguishing the present
quantitative measure from a test has avoided this type of objection. It may be a useful
transferable.
I have illustrated that the idea that it is necessary to define the construct before it can
be measured is not necessarily universally true; one can measure and then find out
what it is that is being measured. Furthermore, one can use a measure even if one
does not formally identify its subject; it can be identified in terms of its functions or
usefulness. This is important with autonomy because it is still so much debated what
it is. This difficulty with defining autonomy in advance does not need to inhibit us
because it is not necessary to the function of the measure. In fact, giving a definition
321
in advance can close off the possibilities of discovery of new dimensions or
techniques: as Cofield (2002: 40) points out ―empirical research into a complex
This illustrates another point made by Cofield (2002: 40), i.e. that a definition is not
required for measurement. This view appears to challenge ideas of the type that
―before developing a test of any construct, one should clearly and explicitly express
what one wants to test‖ (Most & Zeidner 1995). It may be that the clear
disassociation I make between the concepts of ―testing‖ and ―measuring‖ will allow
not predefining a construct of autonomy, and thus going against what has been the
norm for previous authors, is important to the more general acceptance of the
early in the development of an instrument and then using exploratory factor analysis
after data has been collected can reveal the construct which the instrument is
The use of factor analysis in my research meant that I studied the statistical literature
regarding its procedures and also papers in applied linguistics which made use of it.
This reading has made me realise that the stage in factor analysis where each scale is
named by the researcher (see Section 3.12.3.3.4) based on its unifying concept did
not yet have a set procedure and was thus susceptible to subjective interpretation by
322
the researcher. In Section 3.12.3.3.4 I presented a procedure for this interpretation of
factor groupings, and this can contribute to research in future because it highlights
both the lack of and the necessity for a standardised procedure for this step in factor
development which are seldom discussed in the field of language teaching research.
Translation is little used in research in our field, perhaps due to the ethic or habit of
using the target language as much as possible in language teaching. However, this
study argues for increased use of translation in questionnaire research in our field.
3.13.1). The benefits of the correct use of translation (see Section 3.13.2) in the
satisfaction‖, and more reliable responses as respondents will understand their own
experienced were: firstly I found that translation is useful in considering the wording
and clarity of one‘s items, even for the original language version; secondly,
translation forces engagement with the subjects‘ world, culture, way of thinking, and
enriches the research itself by informing one‘s understanding of what one is trying to
research.
I have looked beyond the language teaching field and surveyed the literature of
323
review (see Section 3.13) should be a useful resource for future researchers in the
field of language teaching research. The implications are explained in Section 8.5.
8.4 Limitations
As has been noted throughout the thesis and in particular in Sections 3.13.5 and 5.8
the research encountered a number of obstacles and it was not possible to follow the
original plan. This has had an impact on the nature and quantity of data collected and
consequently has had a significant effect on what can be concluded. In this section I
The results of the factor analysis must be treated with caution. Factor groups cannot
be formed of items which were not included in the initial Long List selection and
items which were de-selected in the data reduction process (Dörnyei 2007: 234). This
is a feature of factor analyses and means that conclusions about the questionnaire
model should not be treated as final. The ratio of respondents to items of a little over
3:1 could be improved. Longer multi item scales would also be preferable (see
Section 4.2.1.8 above), but were not achievable as the very large number of items
would be impractical. With the quantity of data at present available the sample (see
the language learner population. The items used in the factor were the same 50 items
for all respondents, though some are drawn from the Long List and some were
translated into Chinese. This was a necessary compromise to achieve factor analysis
of the data. This means that data for specific situations and nationalities has not been
available.
324
The small scale data do not allow statistical significance to be demonstrated. This
affects what can be concluded quantitatively from the comparison which took place
between the questionnaire data and the year-3 teacher and myself (see Section 5.2).
regard the year-3 teacher and myself only. Further research with larger samples
significant results.
The two interviews (Sections 5.2.2 and 5.2.3) which were carried out are insufficient
for drawing wide conclusions. They serve to illustrate the use of the questionnaire
and show possible areas meriting further investigation. However, they served to raise
The quantity of data from the small scale research is therefore insufficient to confirm
or otherwise other empirical research or theory, and I have not done this. The larger
scale data from the standardised and pooled questionnaire responses which was used
for factor analysis is more acceptable and has produced scales with acceptable
Cronbach‘s alpha figures and construct validity has been suggested by the
comparison of the questionnaire model with the autonomy literature. However, these
data must also be treated with caution and will need to be confirmed with larger scale
8.5 Implications
325
They are quick to administer and can supply relevant information which can act as a
administered can aid reflection on learning. These benefits are likely to aid the
too ambitious and a cause of upheavals. Teachers often wish to introduce more
autonomy into their classes, but the institutional environment is not conducive to
this; an unobtrusive instrument will be one small way for teachers to introduce
autonomy into their interaction with their students. An instrument such as the one
being developed for this thesis should be able to assist teachers with their
This research provides some empirical evidence to suggest that confidence plays a
role in autonomy, and this is an idea which has been found in other reports of
teacher can indirectly support autonomy. This however would need to be carried out
appropriately so as not to adversely influence the other five areas found in the
present research. The model of the electric circuit (Section 7.3.4) may be useful here.
This thesis has highlighted the area of questionnaire translation in second language
research and has proposed that the issues which it presents are neglected in books on
research methods and that researchers often overlook the importance of translating
instruments in their projects, and when they are translated it is often quite informally
326
done. The reasons for and benefits of translating an instrument have been presented
in Sections 3.13 and 7.7. Table 7.3 showed some of the problems which came to light
in my own questionnaire after it had been translated informally without the level of
argues for the increased use of translation in questionnaire-based research and a more
principled and consistent approach to it. This means that more time and resources
will need to be dedicated to it. For this to be achieved the profile of translation needs
with caution, and it is advisable to interpret results with a team which represents all
Further research can be broadly divided into two areas: (a) consolidation, by
remedying the limitations of the present research (see Sections 5.8 and 8.3.3); and (b)
Firstly, as regards consolidation, in the present research only two teachers made
estimates and only one of them was properly independent of the research (as the
other was myself). This means that there is a need for further and more extensive
gathering of teacher estimates (from those not directly involved in the research) with
327
At the micro scale of comparing my own estimates with the questionnaire results
there was over time a convergence that suggests that estimates move towards the
questionnaire‘s reading (see Section 6.3.4). If this analysis could be carried out with
a larger statistically significant sample it would confirm or deny that estimates and
It would be very useful to involve other researchers and experts in the research. The
choice of items was carried out fairly independently, as was the identification and
naming of the factors. The contribution of others would have obvious benefits in the
views of autonomy. A panel of experts with varied positions to make the initial
choice of items and to identify the scales produced by factor analysis would
For the questionnaire it would be important to gather new data from large specific
samples, perhaps with as many as ten individuals for each item, so as to enable factor
analysis of the Long List of items (which was beyond the resources of the present
research). Interviews with respondents should also be carried out. This is necessary
to confirm the results of the factor analysis which was carried out with a pooled
sample in the present research due to the low number of respondents to any one
Secondly, regarding extension, the present research has explored the methodology
gather data to shed light on how practically useful the instrument is for teachers.
328
In terms of researching the development of autonomy, the fully validated instrument
construct to ascertain whether, as seems likely, some are more dispositional and
some more variable over time. Repeated administrations of the questionnaire may
help to distinguish which of the categories are more variable, and which are more
stable.
they initially become better able to reflect on their own learning. This is a testable
hypothesis and so the data gathered for consolidation purposes could also be used to
research this area of the development of learner autonomy and so start to identify the
developmental processes over time as hoped for by Benson (2001: 51; 2010: 78).
the items in order to answer them, but it would also be enhanced if it offered
ambition for future research and development to produce a version of the instrument
Five years ago at the beginning of this research I wanted (I thought) a simple solution
management in a way that they could readily understand and which would relate to
the stated goals of the ILC and the college. In the course of my research this
329
aspiration has been considerably modified and the question quickly became whether
The critical reflexive aspect of my investigation was very interesting. I feel that I
went about it in the right way, in principle, though in the event the project was too
ambitious for the resources available and when unforeseen difficulties arose there
was little margin of safety. Qualitative research requires rich data and much time,
and quantitative research needs substantial resources and guaranteed access to large
making it a very ambitious choice for a PhD student with limited time and resources.
I still, however, think it is the best way to conduct research where possible as it
combines the advantages of both. Valuable lessons have been learned which I can
I hope I have indicated that, though the problems raised regarding quantitative
methods in the area of autonomy research are challenges and do restrict what can be
usefully investigated by quantitative methods alone, they are not conceptual barriers
to it. There is a place for the contributions of quantitative research allied with
qualitative methods, especially if they can result in a practical and viable tool for use
I did not find the autonomy measuring instrument, but this is not a negative result as
I have learned in the process that I did not want it, and that there is a far more
satisfying and useful function to be had from an instrument, that of enhancing the
teacher, and helping to clarify problems by initiating a dialogue rather than being a
way of labelling a learner as being autonomous at a certain level. I have been able to
330
understand the impact of the emphasis on the social and situated nature of autonomy
because I have worked it through for myself in a way that I hope comes across in this
thesis. I have also learned that understanding autonomy is a long and open-ended
process.
331
9 REFERENCES
Ager, D. E. (1996) Language policy in Britain and France: The processes of policy.
London: Continuum
Open University
Benson, P. (1997) The philosophy and politics of learner autonomy. In Benson, P. &
Longman
332
Benson, P. (2009) Making sense of autonomy in language learning. In Pemberton R.,
Centre
Harlow: Pearson
Blais, A. & Gidengil, E. (1993) Things are not always what they seem: French-
Breen, P. & Mann, S. (1997) Shooting arrows at the Sun: Perspectives on a pedagogy
333
Bruner, J. S. (1966) Toward a theory of instruction. Cambridge MA: Harvard
University Press
Bruner, J. S. (1974). Beyond the information given. London: Allen and Unwin.
Candy, P.C., Crebert, G. & O‘Leary, J. (1994) Developing lifelong learning through
http://www.dest.gov.au/sectors/training_skills/publications_resources/indexes/documents/94_21_pdf.htm
Champagne, M-F., Clayton, T., Dimmitt, N., Laszewski, M., Savage, W., Shaw, J.,
Chan, V. (2001) Readiness for learner autonomy: What do our learners tell us?
Chan, V., Spratt, M. & Humphreys, G. (2002) Autonomous language learning: Hong
education. 16/1
Chen, C., Lee, S. & Stevenson, H. W. (1995) Response style and cross-cultural
comparisons of rating scales among East Asian and North American students.
334
Cohen, L., Manion, L. & Morrison, K. (2000) Research methods in education.
London: RoutledgeFalmer
Four recommendations for getting the most from your analysis. Practical
23/2 195-205
Crabbe, D. (1993) Fostering autonomy from within the classroom: The teacher's
Longman
Dam, L (2009) The use of logbooks – A tool for developing learner autonomy. In
335
Dam, L. & Legenhausen, L. (1996) The acquisition of vocabulary in an autonomous
al. (eds.) Taking control: Autonomy in language learning. Hong Kong: HKUP
Dam, L., Eriksson, R., Little, D., Miliander, J. & Trebbi, T. (1990) Towards a
Deakin-Crick, R. & Wilson, K. (2005) Being a learner: A virtue for the 21st century.
Dublin: Authentik
teaching. 31 117–35
336
Dörnyei, Z. (2007) Research methods in applied linguistics. Oxford: OUP
17/1 17-20
Esch, E. (2009) Crash or clash? Autonomy 10 years on. In Pemberton R., Toogood,
Field, A. (2000) discovering statistics using SPSS for windows: Advanced techniques
337
Gan, Z., Humphreys, G. & Hamp–Lyons, L. (2004) Understanding successful and
88/2
language teaching and learning: The history of an idea. System. 23/2 151–164
Grice, H. P. (1975) Logic and conversation. In Cole, P. & Morgan, J. L. (eds.) Syntax
Council of Europe
Holec, H. (2008) Foreword. In Lamb, T. & Reinders, H. (eds.) Learner and teacher
Hughes, A. (2003) Testing for language teachers – Second edition. Cambridge: CUP
338
Hurd, S. (1998) ―Too carefully led or too carelessly left alone‖? Language learning
journal. 17 70-74
Hurd, S (2004) Autonomy and the distance language learner. In Holmberg, B.,
Shelley, M. & White, C. (eds.) Distance education and languages: Evolution and
http://oro.open.ac.uk/622/1/Hurd_Autonomy_chapter.pdf
Karlsson, L., Kjisik, F. & Nordlund, J. (2005) Language counselling: A critical and
35/46-65
Authentik.
339
Lai, J. (2001) Towards an analytic approach to assessing learner autonomy. AILA
(eds.) Maintaining control: Autonomy and language learning. Hong Kong: HKUP
Leech, G., Rayson, P. & Wilson, A. (2001) Word frequencies in written and spoken
Li, A. (2005) A look at Chinese ESL students‘ use of learning strategies in relation to
quantitative study. In Anderson, H. Hobbs, M., Jones-Parry, J., Logan, S., &
Oceania.
Authentik
340
Pemberton, R. et al. (eds.) Taking control: Autonomy in language learning. Hong
Kong: HKUP
centre for languages, linguistics and area studies. Retrieved May 2007 from:
http://www.llas.ac.uk/resources/goodpractice.aspx?resourceid=1409
Little, D. (2005) The common European framework and the European Language
Little, D. (2009) Learner autonomy, the European Language Portfolio, and teacher
Little, D., Ridley, J. & Ushioda, E. (2002) Towards greater learner autonomy in the
Cambridge: CUP
Multilingual Matters
341
Macaro, E. (2008) The shifting dimensions of language learner autonomy. In Lamb,
T. & Reinders, H. (eds.) Learner and teacher autonomy: Concepts, realities and
Messick, S. (1975) The standard problem: Meaning and values in measurement and
Miller, R. L., Maltby, J., Fullerton, D.A. & Acton, C. (2002) SPSS for social
Abingdon: Routledge
In Barfield A. & Nix, M. (eds.) Autonomy you ask! 1–10. Tokyo: JALT Learner
Mynard, J. & Almarzouqi, I. (2006) Investigating peer tutoring. ELTJ. 60/1 13-22
342
Nakatani, Y. (2006) Developing an oral communication strategy inventory. The
Oxford, R. L. (1990) Language learning strategies: What every teacher should know.
343
Pavlenko, A. & Lantolf, J. P. (2000) Second language learning as participation and
their needs, wants, and learning styles. In Alatis, J. E., Altman, H. B. & Alatis,
P.M. (eds.) The second language classroom: Directions for the 1980‟s. New York:
OUP
344
Rogers, C. (1969) Freedom to learn. Columbus, Ohio: Charles E. Merrill
Ryan, R. M & Connell, J.P. (1989) Perceived locus of causality and internalization:
Examining reasons for acting in two domains. Journal of personality and social
Owens, R. (ed.) The translator‟s handbook 3rd edition. The Association for
Shaw, J. (2008) Teachers working together: What do we talk about when we talk
345
Sinclair, B. (1999) More than an act of faith? Evaluating learner autonomy. In
Sinclair, B. (2000) Learner autonomy: The next phase. In Sinclair, B., McGrath, I. &
Pearson
Smith, R. (2008) Key concepts in ELT: Learner autonomy. ELTJ Vol. 62 No. 4
Smith, R. & Ushioda, E. (2009) Autonomy: Under whose control? In Pemberton R.,
Presser, S., Rothgeb, J. M., Couper, M. P., Lessler, J. T., Martin, E., Martin, J. &
Singer, E. (eds.) Methods for testing and evaluating survey questionnaires. 431-
Spratt, M., Humphreys, G. & Chan, V. (2002) Autonomy and motivation: Which
346
Stansfield, C. W. (2003) Test translation and adaptation in public education in the
Kogan Page
Tourangeau, R., Rips, L. J. & Rasinski K. (2000) The psychology of survey response.
Cambridge: CUP
Tseng, W-T., Dörnyei, Z., & Schmitt, N. (2006) A new approach to assessing
347
UK Higher Education International Unit (2010) International higher education in
Van de Vijver, F. R. & Leung, K. (1997) Methods and data analysis for cross-
Van de Vijver, F. R., & Hambleton, R. K. (1996) Translating tests: Some practical
Vygotsky, L. (1994) Academic concepts in school aged children. In van der Veer, R.
school Latin i students. Ph.D. Thesis, Capella University. Retrieved June 2009
from http://www.eric.ed.gov/ERICDocs/data/ericdocs2sql/content_storage_01/0000019b/80/1b/bf/16.pdf
348
Weiner, B. (1979) A theory of motivation for some classroom experiences. Journal
Prentice Hall
CUP
Yang, N-D. (1999) The relationship between EFL learners‘ beliefs and learning
349
10 APPENDICES
10.1 Full 256 items with back translation
350
035 If I am not sure what I have to do I ask If I am not clear about what I am to do, I ask
somebody other people.
036 If I find a word that I don‘t know, I always When I don’t know the words in the reading
ignore it and continue reading materials, I put them aside and go on reading.
037 If I find a word that I don‘t know, I always When I don’t know the words in the reading
ask a teacher first materials, I ask teacher at once.
038 If I find a word that I don‘t know, I look it up When I don’t know the words in the reading
in a dictionary first materials, I look them up in the dictionary at
once.
039 If I find a word that I don‘t know, I try to When I don’t know the words in the reading
guess it materials, I try to guess their meanings.
040 I usually need the teacher to help me with my I usually need teacher to help me learn.
learning
041 I choose my own ways of studying I choose my own learning method.
042 I think about different ways of studying I have thought about different kinds of
learning methods.
043 When I have something to learn I try to think When I learn new thins, I think about different
of different ways of doing it methods can be employed.
044 I know my strong points and weak points I know my merits and shortcomings.
045 I think about how I study best I wonder how I can learn best.
046 I can describe the learning strategies I use I can describe my learning methods.
047 When I finish something I think about the When I have finished a task, I think about the
ways I worked methods I have employed.
048 When I finish something I think about ways When I have finished a task, I think about how
to do it differently in the future to do it by using different methods in future.
049 I have changed the way I learn after thinking After I thought about these methods, my
about it. learning methods have changed.
050 I have tried different ways of learning I have tried various kinds of learning methods.
051 I know why I have problems learning I know the reason why I have difficulties in
learning.
052 I try to fix problems I have in learning I try to solve the problems existing in my
learning process.
053 I know some different ways of learning I know some different kinds of learning
methods.
054 I choose the best way to learn something When I am learning, I choose the best learning
method.
055 I know why I did well or did badly I know the reason why I do well or badly.
056 I have made my own plans for my learning I have already made a plan for my learning of
next week next week.
057 It is important to finish an exercise before my It is important for me to finish the exercise
classmates earlier than my classmates.
058 I use the teacher‘s comments and corrections I use the teachers’ comments and
in my written work to improve my English explanations to improve my English writing.
059 When I like a learning activity, I know why I When I like one learning activity, I know the
like it reason why I like it.
060 I always agree with what a teacher says I always approve of what the teacher says.
061 I find it difficult to tell facts from opinions It is difficult for me to distinguish the facts
when reading from the author’s opinion when I am
reading.
062 Sara eats a lot of sweets so she must be fat Sally eats a lot of sugar everyday, so she is
definitely fat.
063 My dictionary is always right with its The explanations in the dictionary are always
definitions. right.
064 Science books contain only facts The science books only consist of facts.
065 I learn about all kinds of different things I learn various kinds of things out of class.
outside class
066 Knowledge is something a teacher gives me The knowledge is given by the teacher.
067 Knowledge is something I construct for I cumulate the knowledge by myself.
myself
351
068 I trust the Internet I trust the internet.
069 In the last 4 months I have disagreed with During the last four months, I have voiced
something a teacher told the class different opinions for the teachers’ utterance.
070 The Great Wall of China can be seen from The Great Wall can be seen from the Space.
space
071 If I learn something well, it is because I If I learn well, that is because I work hard.
studied well
072 If I learn something well, it is because my If I learn well, that is because the teacher
teacher taught well teaches well.
073 The teacher is responsible for my learning The teacher should be responsible for my
learning.
074 Memorization is the best way to learn Reciting is the best learning method.
075 I am responsible for my learning I must be responsible for my learning.
076 I rely on the teacher I rely on my teachers.
077 I am self-reliant I trust myself.
078 I enjoy making my own choices about I like to make decision on my learning by
learning myself.
079 I want to make my own choices about I want to make decision on my learning by
learning myself.
080 I make my own choices about learning I make decision on my learning by myself.
081 It is good to make your own choices about It is good to make decision on learning by
learning myself.
082 I like the teacher to make the choices about I like the teacher to make decision on my
learning learning.
083 My work is my own, not my teacher‘s My homework is mine, not the teacher‘s.
084 My work is my teacher‘s, not mine My homework is teacher‘s, not mine.
085 I have a mature attitude to learning My attitude towards learning is mature.
086 When I read an English text I need to When I read an English text, I need to make
understand every word in it clear the meaning of every word.
087 I need to be sure about instructions The demands for me must be clear.
088 If I am not sure about something it bothers If I am not sure about something, I will be
me puzzled.
089 There is no one correct way to write an essay There is not only one correct writing method.
090 I learn exclusively about college subjects I only learn the subjects provided by school.
091 Learning well is a talent that some people Learning well is the endowment for some
have and others do not have people, but some other people do not possess
this natural gift.
092 I can learn how to learn better I can learn to how to learn better.
093 Reading is a passive activity; the information Reading is a passive activity. The information
passes from the page to you transfers from the book to you.
094 Reading is an active activity Reading is an active activity.
095 To read you must proceed word by word When we are reading, we have to read word
by word.
096 There is one correct way of reading There is only one correct reading method.
097 I predict the content of a text (using pictures, I predict the content of a passage (by pictures,
headings, the context etc.) titles or context)
098 I read newspapers in a different way to books I use different method to read books and
newspapers.
099 I sometimes look up words on the internet or Sometimes, I look up the unknown words in
in reference books net or in the reference books.
100 Last time I read an English text I predicted When I read the English text last time, I
the content of it predicted the content of it.
101 Different types of text (novel, newspaper, We should use different methods to read
web site etc.) are read in different ways different kinds of articles (novels, newspapers,
websites, etc.).
102 My general knowledge helps me to My basic common sense helps me with the
understand texts I read comprehension of the articles.
103 It is best to read by starting at the beginning When reading an article, it is better to read
and reading line by line to the end from the very beginning, and come to the end
word by word.
352
104 I read in English outside class I also read the English articles in my spare
time.
105 When I read in English I think about what the When I am reading English articles, I consider
source of the text is their sources.
106 I know the sources of the texts I read I know the sources of my reading articles.
107 When I read I think about the motives of the When I am reading, I consider the author’s
writer writing purpose.
108 When I read I start at the beginning and read When I am reading, I read from the beginning
line by line to the end and then read word by word to the end.
109 I predict the content before I listen Before I listen to one passage, I predict its
content.
110 Every word is important for understanding a Every word is important for comprehending a
listening text listening text.
111 The last time I listened to English I tried to When I listen to English last time, I tried to
predict the content predict the content of the passage.
112 I worry if I don‘t understand everything When I listen to English, I will be worried if I
when I listen cannot comprehend every word.
113 I look for opportunities to speak English I find opportunities to practice my oral English
outside class in the spare time.
114 I have looked for opportunities to speak I have tried to find opportunities to practice
English recently my oral English recently.
115 I enjoy speaking English I really like speaking English.
116 Accuracy is very important in speaking When speaking English, accuracy is very
important.
117 Making mistakes is OK when speaking When speaking English, making errors is
nothing important.
118 I know different ways of practicing speaking I know different ways of practicing oral
English.
119 I use different ways to practice speaking I use different ways to practice oral English.
120 Speaking well is a talent some people have, Speaking English well is the endowment for
but not all some people, and not all the people possess
this natural gift.
121 I can help myself to improve my level of I can help myself with the improvement of the
speaking level of oral English.
122 It is important to check one‘s writing Re-examining my own writing is very
important.
123 I check my writing I examine my writing.
124 Last time I wrote in English I checked it When I wrote in English last time, I examined
myself my article.
125 I change the way I write according to who I change my writing methods according to
will read it different readers.
126 There are different types of writing There are many kinds of articles.
127 I know my problems in writing I know the problems existing in my writing.
128 I can help myself to improve my writing I can help myself with the improvement of
writing.
129 If I try my writing will get better I can improve my writing if only I work hard.
130 My writing is better now than it was a year My writing ability have improved compared
ago with one year ago.
131 I enjoy writing in English I like to write in English.
132 I guess the meaning of new words I guess the meaning of new words.
133 I like learning new words I like to learn new words.
134 I keep a record of new words I make record of the new words.
135 I choose the best ways for me to learn new I choose the most appropriate way for me to
words learn new words.
136 I try to use new words outside class I try to use new words out of class.
137 I try to use newly learned words in my essays I try to use new words in my compositions.
138 I know techniques to help me remember I know the strategies to help me memorize
vocabulary English vocabulary.
139 I use techniques to help me remember I use some strategies to help me memorize
vocabulary English vocabulary.
353
140 To remember vocabulary you need to be It needs some talent to memorize English
talented vocabulary.
141 I know my problem areas in vocabulary I know my difficulties in learning English
vocabulary.
142 I fix my problems in vocabulary I can solve the problems in my vocabulary
learning.
143 I only learn words that a teacher recommends I only learn the words recommended by the
teacher.
144 I am able to decide myself which words are I can decide which words are important and
important to learn need to learn.
145 My vocabulary is better now than it was a My vocabulary is richer than one year ago.
year ago
146 Different types of text (magazine, letter, Different kinds of articles use different kinds
recipe etc.) have different vocabulary of vocabulary (magazines, letters, recipes, etc.)
147 I worry if I don‘t understand all the words in I will be worried if I cannot understand the
a text meaning of all the words in the articles.
148 I can help myself to improve my level of I can help myself improve the level of
vocabulary vocabulary.
149 If I try my vocabulary will get better I will learn vocabulary better if only I work
hard.
150 I worry if I don‘t understand all the grammar I will be worried if I cannot understand all the
in a text grammar in the text.
151 I try to find ways of practising grammar I try to find opportunities to practice English
outside class grammar out of class.
152 I know different ways of practising grammar I know different methods to practice grammar.
153 Learning grammar is a talent some people Some people have the inborn gift to learn
have, but not all grammar, and not all the people have.
154 I know my problem areas in grammar I know the problems existing in my grammar.
155 I can help myself to improve my level of I can help myself improve grammatical level.
grammar
156 If I try my grammar will get better My grammatical level will improve if only I
work hard.
157 My grammar is better now than it was a year My grammatical level has made progress
ago compared with one year ago.
158 I guess the meaning of new grammatical I guess the meaning of new grammatical
structures structures.
159 I like learning new grammar I like to learn new grammar.
160 I keep a record of new grammar I make record of the new-learned grammar.
161 I choose the best ways for me to learn new I choose the best way for me to learn grammar
grammar
162 I am able to decide which grammar is I can decide what grammar is important and
important to learn needs to learn.
163 I only learn grammar that a teacher I only learn grammar recommended by the
recommends teacher.
164 Different types of text (magazine, letter, Different kinds of articles use different
recipe etc.) have different grammar grammar (magazines, letters, recipes, etc.)
165 Different people have different ways of Different people use different kinds of
learning learning methods.
166 I know which sense to use to learn best (i.e. I know which sense can best help me learn
sight, or hearing, or touch, or physical (e.g. visual, listening, touch or body
movement) movements)
167 I select learning techniques (i.e. taking notes, I choose the most appropriate learning
or drawing diagrams, or by listening, etc.) strategies for my learning methods (e.g. taking
that suit my best way of learning notes, making diagrams, listening, etc.)
168 I think about the context for something new I I think about the linkage between the new-
am learning learned content and the context, or its
background.
169 When I am learning something new I look When I learn new thing, I find the similarities
for similarities with things I already know existing between it and the things I have
already known.
354
170 I use my knowledge from other subjects When I am learning English, I employ the
when I study English knowledge from other subjects.
171 I use my background knowledge when I do I connect my background knowledge with the
something new new-learned things.
172 I relate new things to my own personal I connect the new-learned things with my own
experiences experience.
173 I approach a topic in a careful, step by step I learn a subject step by step.
manner.
174 I consider facts and come to objective I get the objective conclusion by considering
conclusions the facts.
175 I look at causes and effects logically I check the relations of cause and result by
using logical methods.
176 I prefer a structured plan when I study I am inclined to make an ordered learning plan
when I am learning.
177 I follow textbooks as closely as possible I use my textbook as closely as possible.
178 I collect all necessary information before I I collect all the information before I act.
start
179 I have a general idea for studying, then I have a general idea about learning, and then I
organise the details later organize the details.
180 When studying, I don‘t plan first I do not make a plan before I start learning.
181 When I study I only use the textbooks I only use textbook when I am learning.
182 I need time for personal reflection when I When I am learning, I need time to do self-
study reflection.
183 I am happy to use different worksheets from It does no matter that the teacher asks me to do
the rest of the class homework different from other classmates’.
184 I like negotiating with other students in class I like to discuss with my classmates.
185 I like class discussions I like the discussions in class.
186 I like working in pairs or small groups in I like pair work or group work in class.
class
187 I think learning English is more difficult for I feel that learning English is more difficult for
me than for the average learner me than for other students.
188 I am motivated to learn English I am motivated to learn English.
189 I learn English because I have to I learn English because I have to.
190 I do extra work I do extra homework or learning out of class.
191 I think about what I have studied in class I think about the things learned from the class.
192 I do my English homework I finish my English homework.
193 The other students know English better than Other students learn English better than me.
me
194 The other students are more confident than Other students speak English more confidently
me at speaking English than me.
195 I worry that other students will laugh at me I am worried that when I speak English, some
when I speak English classmates will laugh at me.
196 I am confident I can learn English well I am confident that I will learn English well.
197 I am determined about learning English I am determined to learn English.
198 I hate to study with less than my best effort I think that one should do his/her best to learn.
199 I always notice my mistakes I can always be aware of my errors.
200 I try to find out how to learn better I try to find out the method about how to learn
better.
201 I have clear goals for improving my English I have a very clear goal for improving my
English.
202 I look for opportunities to practice English I look for every opportunity to practice my
English.
203 I organise my time for studying I arrange my learning time soundly.
204 I use my mistakes to help me do better I learn from the errors in order to do better in
future.
205 I notice how other people use English I pay attention to how other people use
English.
206 I try to find the best environment for I try to find the best learning environment.
studying
355
207 I know the aim of the learning tasks I do I know the purpose of my homework and
exercise.
208 I know how much improvement I have made I know how much progress I have made in the
in the last six months last six months.
209 I think about my progress in learning English I think about the progress of my English
learning.
210 I try to relax when I am nervous about When I feel nervous to speak English, I try to
speaking English relax.
211 I avoid situations where there is a chance of I avoid making errors.
making mistakes
212 I talk to others about how I feel about I talk with other people about my feeling of
learning English learning English.
213 If someone is speaking English too fast I ask If someone speaks English too fast, I will ask
him/her to slow down or repeat her/him to speak more slowly or repeat.
214 I practice English with other students I practice English with other students.
215 I ask for help from English speakers I look for help from the English speakers.
216 I am aware of the feelings of others I am aware of others’ feelings.
217 I make learning plans I make learning plans.
218 I join in with classroom discussions I participate in the discussion in class.
219 I reflect on my learning I summarize and reflect on my learning.
220 I am ready to learn in unfamiliar ways I have made preparations for learning by using
unfamiliar methods.
221 I want to learn in a more Western way I want to learn in a more western-style way.
222 Repetition is important for learning Repetition is very important for learning.
223 Errors must always be corrected Errors must be corrected.
224 In learning it is important to work Learning independently is very important.
independently
225 Praise from the teacher is important to me The approval from teacher is important for me.
226 I need tests to motivate me I need quiz to motivate myself.
227 I need praise to motivate me I need approval to motivate myself.
228 I motivate myself I can motivate myself.
229 I can choose the method of learning that suits I can choose the most appropriate learning
me best method for myself.
230 My way of learning will never change My learning method will always be
unchanged.
231 I can study independently I can learn independently.
232 My own needs are important to the way I My own desire is important for my learning
learn method.
233 I know how to check my own work for I know how to examine my homework and
mistakes find the errors out.
234 It is my job to check my work for mistakes It is my responsibility to find errors from my
homework.
235 Making mistakes is bad for language learning Making errors is bad for learning language.
236 I am good at making choices I am good at making choice.
237 I am an active dynamic person I am enthusiastic and energetic.
238 I choose the exercises I work on I choose my exercise.
239 I like to work at my own pace I like learning in my own speed.
240 If I am not sure about an answer I go to the If I am not sure about the answer to a question,
next question I skip it and come to next one.
241 I do not go on to the next question in an If I am not sure about the answer to a question,
exercise until I am sure about the answer I will not continue to do the next one.
242 I guess answers if I don‘t know them for sure If I am not sure about the answer to a question,
I guess the answer.
243 I decide what I need to read I choose my reading materials.
244 I like myself I like myself.
245 I need the teacher to check my answers I need teacher to examine my answers.
246 Memorizing answers is the best way to learn Memorizing the answers is the best learning
method.
247 If I do badly in a test I know why I know the reason why I have not done well in
the test.
356
248 I am motivated by making progress in The progress in my learning motivates me to
learning continue learning.
249 It is necessary to practice using English It is necessary to practice English out of class.
outside the classroom
250 Students can help the teacher choose the Students can help teacher with the choice of
subject of lessons course’s topics and contents.
251 I use real English texts (i.e. not made for I use the authentic English articles when I am
students) in my learning learning (viz. non-specifically written articles
for the learners )
252 I know how to find information in a library I know how to find materials in library.
253 I know how to use English language I know how to make use of English reference
reference books (encyclopedias, dictionaries, books (encyclopaedia, dictionary, etc.)
etc.)
254 I know the parts of a book (index, glossary, I know each part of the book (index, table,
contents, chapters) content, chapter)
255 I keep a learning diary I write learning diary.
256 I know how to find the information I need on I know how to search information I need in the
the Internet internet
357
10.2 Long List areas covered
Reading
Writing
Listening
Speaking
Grammar
Vocabulary
Attitudes to
Social
Motivation
Confidence
Responsibility
Actions/
Strategies
Meta-
Control
Skills
learning
Interaction
Behaviours
cognition
Item
358
Reading
Writing
Listening
Speaking
Grammar
Vocabulary
Attitudes to
Social
Motivation
Confidence
Responsibility
Actions/
Strategies
Meta-
Control
Skills
learning
Interaction
Behaviours
cognition
Item
359
Reading
Writing
Listening
Speaking
Grammar
Vocabulary
Attitudes to
Social
Motivation
Confidence
Responsibility
Actions/
Strategies
Meta-
Control
Skills
learning
Interaction
Behaviours
cognition
Item
360
Reading
Writing
Listening
Speaking
Grammar
Vocabulary
Attitudes to
Social
Motivation
Confidence
Responsibility
Actions/
Strategies
Meta-
Control
Skills
learning
Interaction
Behaviours
cognition
Item
361
Reading
Writing
Listening
Speaking
Grammar
Vocabulary
Attitudes to
Social
Motivation
Confidence
Responsibility
Actions/
Strategies
Meta-
Control
Skills
learning
Interaction
Behaviours
cognition
Item
097. I predict the content of a text (using pictures, headings, the context x x x x x
etc.)
098. I read newspapers in a different way to books x x x x
099. I sometimes look up words on the internet or in reference books x x x x x x x x
100. Last time I read an English text I predicted the content of it x x x x x x x x
101. Different types of text (novel, newspaper, web site etc.) are read in x x x
different ways
102. My general knowledge helps me to understand texts I read x x x x
103. It is best to read by starting at the beginning and reading line by line x x x x x x
to the end
104. I read in English outside class x x x x x x
105. When I read in English I think about what the source of the text is x x x x x x
106. I know the sources of the texts I read x x x x x
107. When I read I think about the motives of the writer x x x x x
108. When I read I start at the beginning and read line by line to the end x x x x x x x
109. I predict the content before I listen x x x x x x x
110. Every word is important for understanding a listening text x x x x x x
111. The last time I listened to English I tried to predict the content x x x x x x x
112. I worry if I don‘t understand everything when I listen x x x x x x
113. I look for opportunities to speak English outside class x x x x x x x x x
114. I have looked for opportunities to speak English recently x x x x x x x x x
115. I enjoy speaking English x x x x x
116. Accuracy is very important in speaking x x x x x x x
117. Making mistakes is OK when speaking x x x x x x x x x
118. I know different ways of practising speaking x x x x x
119. I use different ways to practise speaking x x x x x x x x x x
120. Speaking well is a talent some people have, but not all x x x
362
Reading
Writing
Listening
Speaking
Grammar
Vocabulary
Attitudes to
Social
Motivation
Confidence
Responsibility
Actions/
Strategies
Meta-
Control
Skills
learning
Interaction
Behaviours
cognition
Item
363
Reading
Writing
Listening
Speaking
Grammar
Vocabulary
Attitudes to
Social
Motivation
Confidence
Responsibility
Actions/
Strategies
Meta-
Control
Skills
learning
Interaction
Behaviours
cognition
Item
146. Different types of text (magazine, letter, recipe etc.) have different x x x
vocabulary
147. I worry if I don‘t understand all the words in a text x x x x x x x
148. I can help myself to improve my level of vocabulary x x x x x x x x x
149. If I try my vocabulary will get better x x x x x x x
150. I worry if I don‘t understand all the grammar in a text x x x x x
151. I try to find ways of practising grammar outside class x x x x x x x x x
152. I know different ways of practising grammar x x x x x x
153. Learning grammar is a talent some people have, but not all x x x
154. I know my problem areas in grammar x x x
155. I can help myself to improve my level of grammar x x x x x x x x x
156. If I try my grammar will get better x x x x x x x
157. My grammar is better now than it was a year ago x x x
158. I guess the meaning of new grammatical structures x x x x x x x
159. I like learning new grammar x x x
160. I keep a record of new grammar x x x x x x x
161. I choose the best ways for me to learn new grammar x x x x x x x x x
162. I am able to decide which grammar is important to learn x x x x x x x
163. I only learn grammar that a teacher recommends x x x x x x x x x x x
164. Different types of text (magazine, letter, recipe etc.) have different x x x
grammar
165. Different people have different ways of learning x x
166. I know which sense to use to learn best (i.e. sight, or hearing, or x x x x
touch, or physical movement)
167. I select learning techniques (i.e. taking notes, or drawing diagrams, or x x x x x
by listening, etc.) that suit my best way of learning
168. I think about the context for something new I am learning x x x x
364
Reading
Writing
Listening
Speaking
Grammar
Vocabulary
Attitudes to
Social
Motivation
Confidence
Responsibility
Actions/
Strategies
Meta-
Control
Skills
learning
Interaction
Behaviours
cognition
Item
169. When I am learning something new I look for similarities with things x x x
I already know
170. I use my knowledge from other subjects when I study English x x x x
171. I use my background knowledge when I do something new x x x x x
172. I relate new things to my own personal experiences x x x x
173. I approach a topic in a careful, step by step manner. x x x x x x
174. I consider facts and come to objective conclusions x x x x
175. I look at causes and effects logically x x
176. I prefer a structured plan when I study x x x x
177. I follow textbooks as closely as possible x x x x x x
178. I collect all necessary information before I start x x x
179. I have a general idea for studying, then organise the details later x x x x
180. When studying, I don‘t plan first x x x x x
181. When I study I only use the textbooks x x x x x x
182. I need time for personal reflection when I study x x x x
183. I am happy to use different worksheets from the rest of the class x x x x x x
184. I like negotiating with other students in class x x x x x
185. I like class discussions x x x x x x
186. I like working in pairs or small groups in class x x x x x x
187. I think learning English is more difficult for me than for the average x x x x
learner
188. I am motivated to learn English x x x x
189. I learn English because I have to x x x x x
190. I do extra work x x x x x x x
191. I think about what I have studied in class x x x x x x
192. I do my English homework x x x x x x
365
Reading
Writing
Listening
Speaking
Grammar
Vocabulary
Attitudes to
Social
Motivation
Confidence
Responsibility
Actions/
Strategies
Meta-
Control
Skills
learning
Interaction
Behaviours
cognition
Item
366
Reading
Writing
Listening
Speaking
Grammar
Vocabulary
Attitudes to
Social
Motivation
Confidence
Responsibility
Actions/
Strategies
Meta-
Control
Skills
learning
Interaction
Behaviours
cognition
Item
367
Reading
Writing
Listening
Speaking
Grammar
Vocabulary
Attitudes to
Social
Motivation
Confidence
Responsibility
Actions/
Strategies
Meta-
Control
Skills
learning
Interaction
Behaviours
cognition
Item
368
10.3 Long List format C with translation
No. Item Translation
1 251. I use real English texts (i.e. not made for 我在学习时,使用‗真实‘的英语文章(即:非为
students) in my learning. 学生专门写的)
2 095. To read you must proceed word by word. 在阅读时,我们得一个字一个字地读
3 086. When I read an English text I need to 当我读一篇英语课文时,我需要弄懂其中每一个
understand every word in it. 词的意思
4 100. Last time I read an English text I predicted 我上次读英语文章的时候,我预测了它是在讲什
the content of it. 么内容
5 147. I worry if I don‘t understand all the words in 我如果不能理解文中所有词的意思,就会觉得担
a text. 心
6 243. I decide what I need to read. 我决定我要读什么
7 109. I predict the content before I listen. 我在听一段课文之前会预测它的内容
8 110. Every word is important for understanding a 要听懂一段文章的内容,每一个字都很重要
listening text.
9 111. The last time I listened to English I tried to 我上次听英语的时候,我尝试了预测文章的内容
predict the content.
10 112. I worry if I don‘t understand everything 我在听英语的时候,如果不能听懂每个词的意思
when I listen. ,我就会很担心
11 125. I change the way I write according to who 我会根据读者的不同而改变我的写作方法
will read it.
12 130. My writing is better now than it was a year 我的写作比一年以前有进步
ago.
13 138. I know techniques to help me remember 我知道如何帮我记住英语词汇的學习技巧
vocabulary.
14 140. To remember vocabulary you need to be 要记住英语词汇,你得有些天赋
talented.
15 142. I fix my problems in vocabulary. 我解决我在词汇方面的问题
16 153. Learning grammar is a talent some people 有的人有学语法的天赋,但不是所有人都有
have, but not all.
17 150. I worry if I don‘t understand all the grammar 我如果不能理解文中所有的语法,就会觉得担心
in a text.
18 252. I know how to find information in a library. 我知道如何在图书馆找资料
19 253. I know how to use English language 我知道如何使用英语参考书(百科全书,字典等
reference books (encyclopedias, dictionaries, )
etc.).
20 254. I know the parts of a book (index, glossary, 我知道书的各部分(索引,术语表,目录,章节
contents, chapters). )
21 256. I know how to find the information I need 我知道如何在网上搜索我要的信息
on the Internet.
22 187. I think learning English is more difficult for 我觉得学英语对我来说,比一般的学生要难
me than for the average learner.
23 193. The other students know English better than 其他同学英语比我学得好
me.
24 194. The other students are more confident than 别的同学讲英语时比我更加有自信
me at speaking English.
25 195. I worry that other students will laugh at me 我担心我讲英语时,别的同学会笑我
when I speak English.
26 196. I am confident I can learn English well. 我自信我能学好英语
27 017. If I must finish a job at a certain time I finish 如果我必须在规定时间内完成一个任务,我会尽
early 早完成
28 203. I organise my time for studying. 我合理安排学习的时间
29 238. I choose the exercises I work on. 我选择我要做的练习
30 229. I can choose the method of learning that 我能选择最适合我的学习方法
suits me best.
31 236. I am good at making choices. 我很善于做选择
32 220. I am ready to learn in unfamiliar ways. 我愿意尝试新的方法学习
33 023. Students should always do what their 学生应该总根据老师说的做
teacher says.
369
34 030. The teacher‘s job is to give me all the 老师的任务是给我所有的相关信息
information.
35 076. I rely on the teacher when learning. 我的学习靠老师
36 008. I am good at studying on my own. 我善于独立学习
37 231. I can study independently. 我能独立地学习
38 234. It is my job to check my work for mistakes. 从我的作业中找出错误是我的责任
39 055. I know why I did well or did badly. 我知道我为什么做得好或做得差
40 230. My way of learning will never change. 我的学习方法将永远不会改变
41 026. I feel lucky when I get good marks. 我得到好成绩时,觉得自己很幸运.
42 246. Memorizing answers is the best way to 记住答案是最好的学习方法
learn.
43 046. I can describe the learning strategies I use. 我能描述我使用的学习策略
44 049. I have changed the way I learn after thinking 在我思考了学习方法之后我改变了我的学习方法
about it.
45 175. I look at causes and effects logically. 我用逻辑的方法来看原因和结果的关系
46 237. I am an active dynamic person. 我是一个积极主动,充满活力的人
47 212. I talk to others about how I feel about 我与他人谈论对于英语学习的感受
learning English.
48 189. I learn English because I have to. 我学英语,因为我不得不学
49 021. All lessons are equally valuable. 所有的课都一样有价值
50 205. I notice how other people use English. 我留意别人是怎样使用英语的
370
10.4 Selection Table
Likert Responses as Percentage Selection Criteria
Item 1 2 3 4 5 6 7 A B C D E
93. Reading is a passive activity; the information passes 3.0 7.9 20.8 6.9 35.6 12.9 12.9 4 13.9 1.597 19.497 4
from the page to you.
195. I worry that other students will laugh at me when I 2.0 5.1 26.3 17.2 27.3 10.1 12.1 5 10.1 1.613 16.713 4
speak English.
189. I learn English because I have to. 5.8 7.8 27.2 19.4 27.2 1.9 10.7 4 7.6 1.562 13.162 4
194. The other students are more confident than me at 4.3 10.0 32.9 17.1 24.3 7.1 4.3 4 7.2 1.545 12.745 4
speaking English.
112. I worry if I don’t understand everything when I listen. 0 14.1 24.2 17.2 32.3 7.1 5.1 4 7 1.424 12.424 4
120. Speaking well (in English) is a talent some people have, 10.7 12.0 34.7 16.0 20.0 1.3 5.3 5 4 1.510 10.510 4
but not all.
21. All lessons are equally valuable 0 2.7 13.3 12.0 38.7 10.7 22.7 5 1.3 1.387 7.687 4
91. Learning well is a talent that some people have and 1.4 6.8 32.4 17.6 18.9 13.5 9.5 4 1.3 1.506 6.806 4
others do not have.
164. Different types of text (magazine, letter, recipe etc.) 0 1.0 10.2 9.2 50.0 15.3 14.3 4 1 1.503 6.503 4
have different grammar.
96. There is one correct way of reading. 0 2.7 10.8 2.7 47.3 14.9 21.6 4 8.1 1.293 13.393 3
140. To remember vocabulary you need to be talented. 2.8 8.3 36.1 19.4 22.2 4.2 6.9 3 2.8 1.406 7.206 3
130. My writing is better now than it was a year ago. 0 0 12.7 9.9 40.8 28.2 8.5 3 2.8 1.379 7.179 3
61. I find it difficult to tell facts from opinions when reading. 0 0 23.3 20.5 38.4 9.6 8.2 3 2.8 1.295 7.095 3
243. I decide what I need to read. 3.1 7.1 11.2 14.3 39.8 13.3 11.2 5 1.716 6.716 3
251. I use real English texts (i.e. not made for students) in 0 0 10.5 23.2 38.9 15.8 11.6 5 1.666 6.666 3
my learning.
254. I know the parts of a book (index, glossary, contents, 0 0 12.7 19.7 36.6 12.7 18.3 5 1.384 6.384 3
chapters).
187. I think learning English is more difficult for me than for 1.4 2.7 20.3 20.3 33.8 10.8 10.8 5 1.365 6.365 3
the average learner.
70. The Great Wall of China can be seen from space. 7.6 1.5 12.1 13.6 34.8 9.1 21.2 4 1.998 5.998 3
180. When studying, I don’t plan first 1.3 6.7 21.3 20.0 37.3 5.3 8.0 3 1.3 1.349 5.649 3
371
Likert Responses as Percentage Selection Criteria
Item 1 2 3 4 5 6 7 A B C D E
30. The teacher’s job is to give me all the information. 5.0 14.0 13.0 26.0 28.0 10.0 4.0 4 1.530 5.530 3
111. The last time I listened to English I tried to predict the 0 0 11.7 11.7 56.4 14.9 5.3 4 1.503 5.503 3
content.
175. I look at causes and effects logically. 0 0 1.4 26.8 45.1 15.5 11.3 4 1.486 5.486 3
14. When I learn something new I feel good because the 3.9 13.2 15.8 25.0 28.9 9.2 3.9 4 1.441 5.441 3
teacher is happy
230. My way of learning will never change. 1.0 1.0 5.0 16.0 44.0 13.0 20.0 4 1.418 5.418 3
17. If I must finish a job at a certain time I finish early 2.7 2.7 18.9 24.3 27.0 14.9 9.5 4 1.416 5.416 3
90. I learn exclusively about college subjects, and nothing 1.9 1.9 11.4 13.3 21.9 14.3 5.7 4 1.397 5.397 3
else.
100. Last time I read an English text I predicted the content 0 1.0 13.3 15.3 52.0 12.2 6.1 4 1.394 5.394 3
of it.
153. Learning grammar is a talent some people have, but not 5.4 10.8 27.0 25.7 23.0 4.1 4.1 4 1.388 5.388 3
all.
236. I am good at making choices. 2.8 5.6 20.8 27.8 27.8 12.5 2.8 4 1.381 5.381 3
32. The teacher’s job is to control students in the classroom. 0 1.3 13.2 19.7 30.3 13.2 22.4 4 1.374 5.374 3
26. I feel lucky when I get good marks. 6.7 17.3 24.0 24.0 22.7 4.0 1.3 4 1.368 5.368 3
166. I know which sense is best for me to use when learning 0 0 4.2 18.1 40.3 20.8 16.7 4 1.368 5.368 3
(i.e. sight, or hearing, or touch, or physical movement).
116. Accuracy is very important in speaking English. 3.9 9.7 33.0 24.3 15.5 11.7 1.9 4 1.358 5.358 3
205. I notice how other people use English. 0 1.4 4.1 17.8 45.2 20.5 11.0 4 1.330 5.330 3
246. Memorizing answers is the best way to learn. 0 2.7 8.2 12.3 35.6 19.2 21.9 4 1.313 5.313 3
1. I will be happy when I can stop learning 0 2.6 5.3 10.5 36.8 14.5 30.3 4 1.311 5.311 3
237. I am an active dynamic person. 0 0 8.3 26.4 34.7 15.3 15.3 4 1.306 5.306 3
150. I worry if I don’t understand all the grammar in a text. 1.0 9.7 39.8 16.5 26.2 5.8 1.0 3 9.7 1.194 13.894 2
124. Last time I wrote in English I checked it myself. 0 0 12.0 5.3 41.3 24.0 17.3 3 6.7 1.183 10.883 2
34. When I study English, if I am not sure what I have to do, I 2.0 1.0 15.7 10.8 53.9 8.8 7.8 3 4.9 1.294 9.194 2
don’t worry about it.
252. I know how to find information in a library. 1.4 1.4 6.8 4.1 50.7 21.9 13.7 3 2.7 1.181 6.881 2
372
Likert Responses as Percentage Selection Criteria
Item 1 2 3 4 5 6 7 A B C D E
256. I know how to find the information I need on the 1.4 0 6.8 4.1 50.0 20.3 17.6 3 2.7 1.160 6.860 2
Internet.
74. Memorization is the best way to learn. 2.9 11.7 26.2 30.1 23.3 2.9 2.9 5 1.255 6.255 2
23. Students should always do what their teacher says. 0 0 13.2 23.7 36.8 11.8 14.5 5 1.213 6.213 2
8. I am good at studying on my own. 0 0 7.8 11.7 39.0 23.4 18.2 5 1.141 6.141 2
2. One day I will stop learning. 0 1.3 9.3 13.3 26.7 24.0 25.3 4 1.566 5.566 2
33. Everybody can make progress if they try. 0 0 5.3 3.9 27.6 25.0 38.2 3 1.4 1.135 5.535 2
125. I change the way I write according to who will read it. 0 1.4 14.9 24.3 36.5 14.9 8.1 4 1.288 5.288 2
138. I know techniques to help me remember vocabulary. 0 0 15.1 23.3 43.8 5.5 12.3 4 1.279 5.279 2
117. Making mistakes is OK when speaking English. 0 0 7.9 12.9 46.5 19.8 12.9 4 1.278 5.278 2
234. It is my job to check my work for mistakes. 0 0 3.1 21.6 50.5 14.4 10.3 4 1.277 5.277 1
226. I need tests to motivate me. 6.9 22.5 35.3 20.6 12.7 0.0 2.0 4 1.263 5.263 1
38. If I find an English word that I don’t know, I look it up in a 3.8 17.3 26.0 28.8 20.2 1.9 1.9 4 1.259 5.259 1
dictionary first.
16. When I learn something new I feel good because I can 0 1.3 4.0 12.0 38.7 28.0 16.0 4 1.252 5.252 1
stop learning it
36. If I find an English word that I don’t know, I always ignore 1.3 14.5 21.1 22.4 35.5 3.9 1.3 4 1.247 5.247 1
it and continue reading.
86. When I read an English text I need to understand every 2.9 2.9 12.6 21.4 44.7 10.7 4.9 4 1.243 5.243 1
word in it.
244. I like myself. 0 0 1.4 11.3 32.4 26.8 28.2 4 1.240 5.240 1
114. I have looked for opportunities to speak English 0 0 9.0 14.0 55.0 11.0 11.0 4 1.236 5.236 1
recently.
143. I only learn words that a teacher recommends. 1.0 0 7.8 18.6 43.1 18.6 10.8 4 1.232 5.232 1
136. I try to use new words outside class. 0 0 6.0 27.0 43.0 14.0 10.0 4 1.230 5.230 1
75. I am responsible for my learning. 0 0 1.3 0 21.3 36.0 41.3 3 1.3 0.855 5.155 1
12. I watch TV or videos in English in my own time 0 1.0 3.8 2.9 31.7 36.5 24.0 3 0.9 1.049 4.949 1
92. I can learn how to learn better. 8.9 6.9 9.9 5.0 31.7 21.8 15.8 3 1.868 4.868 1
162. I am able to decide which grammar is important to 1.1 1.1 7.4 16.8 52.6 18.9 2.1 3 1.559 4.559 1
learn.
373
Likert Responses as Percentage Selection Criteria
Item 1 2 3 4 5 6 7 A B C D E
229. I can choose the method of learning that suits me best. 0 0 3.2 21.1 50.5 15.8 9.5 3 1.509 4.509 1
163. I only learn grammar that a teacher recommends. 0 2.8 9.9 28.2 38.0 4.2 16.9 3 1.490 4.490 1
73. The teacher is responsible for my learning. 0 5.5 23.3 32.9 24.7 6.8 6.8 3 1.408 4.408 1
82. I like the teacher to make the choices about learning. 2.0 5.1 14.3 27.6 38.8 9.2 3.1 3 1.399 4.399 1
57. It is important to finish an exercise before my classmates. 4.1 6.8 21.9 35.6 21.9 4.1 5.5 3 1.398 4.398 1
154. I know my problem areas in grammar. 0 0 8.2 15.3 48.0 21.4 7.1 3 1.384 4.384 1
56. I have made my own plans for my learning next week. 0 4.0 24.0 26.0 32.0 6.0 8.0 3 1.380 4.380 1
144. I am able to decide myself which words are important to 0 2.0 8.0 25.0 47.0 10.0 8.0 3 1.326 4.326 1
learn.
247. If I do badly in a test I know why. 0 0 5.6 9.7 54.2 16.7 13.9 3 1.305 4.305 1
66. Knowledge is something a teacher gives me. 13.3 9.3 30.7 34.7 8.0 2.7 1.3 3 1.300 4.300 1
119. I use different ways to practise speaking English. 0 1.0 9.2 35.7 35.7 9.2 9.2 2 1.345 3.345 1
240. If I am not sure about an answer I go to the next 0 0 5.0 13.0 55.0 14.0 13.0 4 1.213 5.213 0
question.
178. I collect all necessary information before I start. 1.4 0 12.2 31.1 37.8 13.5 4.1 4 1.200 5.200 0
176. I prefer a structured plan when I study. 1.3 0 9.3 16.0 38.7 24.0 10.7 4 1.196 5.196 0
10. I give myself targets for studying. 0 0 5.3 17.1 40.8 25.0 11.8 4 1.189 5.189 0
11. I am good at planning my learning. 0 1.3 14.7 18.7 38.7 18.7 8.0 4 1.178 5.178 0
197. I am determined about learning English. 1.0 0 4.0 16.8 29.7 30.7 17.8 4 1.173 5.173 0
135. I choose the best ways for me to learn new words. 0 0 1.0 17.2 56.6 11.1 14.1 4 1.171 5.171 0
48. When I finish something I think about ways to do it 1.3 0 17.3 26.7 33.3 17.3 4.0 4 1.164 5.164 0
differently in the future.
83. The work I do for my course is mine, not my teachers’. 0 1.3 3.9 13.2 43.4 15.8 22.4 4 1.163 5.163 0
218. I join in with classroom discussions. 0 0 5.0 19.8 40.6 22.8 11.9 4 1.154 5.154 0
18. I meet deadlines 0 1.3 6.6 19.7 43.4 14.5 14.5 4 1.147 5.147 0
185. I like class discussions. 0 1.0 8.7 23.3 39.8 15.5 11.7 4 1.141 5.141 0
68. I always trust the information I find on the Internet. 0 0 9.5 27.0 32.4 20.3 10.8 4 1.140 5.140 0
145. My vocabulary is better now than it was a year ago. 0 0 1.4 12.3 47.9 21.9 16.4 4 1.136 5.136 0
374
Likert Responses as Percentage Selection Criteria
Item 1 2 3 4 5 6 7 A B C D E
110. Every word is important for understanding a listening 0 5.9 12.7 12.7 52.9 12.7 2.9 4 1.134 5.134 0
text.
210. I try to relax when I am nervous about speaking English. 1.4 1.4 4.1 10.8 56.8 12.2 13.5 4 1.130 5.130 0
71. If I learn something well, it is because I studied well. 0 1.4 5.4 18.9 40.5 20.3 13.5 4 1.127 5.127 0
198. I hate to study with less than my best effort. 0 0 2.7 12.3 32.9 21.9 30.1 4 1.123 5.123 0
239. I like to work at my own pace. 0 0 5.9 16.8 34.7 25.7 16.8 4 1.120 5.120 0
235. Making mistakes is bad for language learning. 16.4 17.8 47.9 13.7 2.7 0.0 1.4 4 1.118 5.118 0
225. Praise from the teacher is important to me. 17.2 25.3 39.4 12.1 5.1 1.0 0.0 4 1.117 5.117 0
29. I know which is my best subject. 0 0 4.0 14.7 40.0 18.7 22.7 4 1.116 5.116 0
213. If someone is speaking English too fast I ask him/her to 1.0 0 5.9 17.6 47.1 15.7 12.7 4 1.114 5.114 0
slow down or repeat.
129. If I try my writing will get better. 0 0 5.4 10.8 40.5 21.6 21.6 4 1.111 5.111 0
227. I need praise to motivate me. 19.4 22.3 36.9 18.4 1.9 1.0 0.0 4 1.110 5.110 0
95. To read you must proceed word by word. 0 0 12.7 15.7 52.0 7.8 11.8 4 1.104 5.104 0
121. I can help myself to improve my level of speaking. 0 0 10.7 17.5 52.4 5.8 13.6 4 1.101 5.101 0
183. I am happy to use different worksheets from the rest of 0 1.4 15.1 23.3 42.5 12.3 5.5 4 1.096 5.096 0
the class.
115. I enjoy speaking English. 0 0 5.8 19.4 39.8 20.4 14.6 4 1.091 5.091 0
88. If I am not sure about something it bothers me. 14.9 27.0 39.2 12.2 6.8 0.0 0.0 4 1.084 5.084 0
200. I try to find out how to learn better. 0 0 3.0 17.0 48.0 21.0 11.0 4 1.081 5.081 0
47. When I finish something I think about the ways I worked. 0 1.3 16.0 28.0 37.3 13.3 4.0 4 1.080 5.080 0
7. When I study, I am an organised learner. 0 1.0 10.5 28.6 39.0 14.3 6.7 4 1.072 5.072 0
31. The teacher’s job is to help me learn 0 0 9.2 15.8 46.1 18.4 10.5 4 1.070 5.070 0
202. I look for opportunities to practice English. 0 0 6.8 14.6 47.6 16.5 14.6 4 1.070 5.070 0
37. If I find an English word that I don’t know, I always ask a 0 0 1.0 10.1 49.5 23.2 16.2 4 1.062 5.062 0
teacher first.
212. I talk to others about how I feel about learning English. 0 0 6.8 11.0 53.4 16.4 12.3 4 1.014 5.014 0
133. I like learning new words. 0 0 4.9 17.5 50.5 15.5 11.7 4 0.993 4.993 0
182. I need time for personal reflection when I study. 0 0 5.0 11.9 51.5 18.8 12.9 4 0.989 4.989 0
375
Likert Responses as Percentage Selection Criteria
Item 1 2 3 4 5 6 7 A B C D E
9. I know how to find the information I want. 0 0 3.9 15.8 53.9 13.2 13.2 4 0.981 4.981 0
188. I am motivated to learn English. 0 0 1.0 16.5 42.7 23.3 16.5 4 0.981 4.981 0
25. I know my own ability. 0 0 5.3 11.8 47.4 25.0 10.5 4 0.978 4.978 0
148. I can help myself to improve my level of vocabulary. 0 0 3.9 14.7 55.9 11.8 13.7 4 0.976 4.976 0
221. I want to learn in a more learner-centred way. 0 0 4.2 13.9 50.0 20.8 11.1 4 0.963 4.963 0
27. I need a teacher to help me. 0 0 3.0 15.0 47.0 23.0 12.0 4 0.960 4.960 0
65. I learn about all kinds of different things outside class. 0 0 1.3 14.5 50.0 18.4 15.8 4 0.958 4.958 0
181. When I study I only use the textbooks 0 0 4.1 10.8 55.4 17.6 12.2 4 0.944 4.944 0
172. I relate new things to my own personal experiences. 0 0 1.4 12.3 39.7 31.5 15.1 4 0.944 4.944 0
6. When I study I take breaks in order to maintain my 0 0 1.9 10.5 45.7 26.7 15.2 4 0.939 4.939 0
concentration.
41. I choose my own ways of studying English. 0 0 0 15.5 41.7 28.2 14.6 4 0.924 4.924 0
214. I practice English with other students. 0 0 1.0 18.4 53.4 14.6 12.6 4 0.919 4.919 0
255. I keep a learning diary. 5.9 6.9 40.6 26.7 13.9 5.0 1.0 3 1.290 4.290 0
193. The other students know English better than me. 0 5.8 37.7 26.1 21.7 7.2 1.4 3 1.283 4.283 0
108. When I read I start at the beginning and read line by line 1.4 4.1 24.3 21.6 35.1 8.1 5.4 3 1.271 4.271 0
to the end.
55. I know why I did well or did badly. 0 0 5.5 15.1 50.7 19.2 9.6 3 1.268 4.268 0
157. My grammar is better now than it was a year ago. 0 1.4 19.2 35.6 27.4 9.6 6.8 3 1.259 4.259 0
208. I know how much improvement I have made in the last 0 4.2 5.6 20.8 47.2 13.9 8.3 3 1.258 4.258 0
six months.
24. The student’s job is to develop as a person. 0 1.9 8.7 28.2 31.1 21.4 8.7 3 1.258 4.258 0
64. Science books contain only facts. 0 4.1 15.1 17.8 49.3 8.2 5.5 3 1.252 4.252 0
161. I choose the best ways for me to learn new grammar. 1.0 0 7.1 17.2 48.5 19.2 7.1 3 1.251 4.251 0
105. When I read in English I think about what the source of 0 1.0 27.7 28.7 29.7 5.9 6.9 3 1.246 4.246 0
the text is.
49. I have changed the way I learn after thinking about it. 0 1.4 5.4 24.3 47.3 16.2 5.4 3 1.245 4.245 0
223. Errors must always be corrected. 21.8 28.7 34.7 6.9 5.0 3.0 0.0 3 1.241 4.241 0
42. I think about different ways of studying English. 0 0 3.0 24.0 50.0 16.0 7.0 3 1.224 4.224 0
376
Likert Responses as Percentage Selection Criteria
Item 1 2 3 4 5 6 7 A B C D E
46. I can describe the learning strategies I use. 0 0 9.1 19.2 53.5 14.1 4.0 3 1.223 4.223 0
179. I have a general idea for studying, then organise the 0 0 4.1 11.0 53.4 23.3 8.2 3 1.223 4.223 0
details later.
211. I avoid situations where there is a chance of making 6.1 12.1 54.5 16.2 5.1 5.1 1.0 3 1.220 4.220 0
mistakes.
173. I approach a topic in a careful, step by step manner. 1.4 1.4 5.4 14.9 52.7 17.6 6.8 3 1.214 4.214 0
159. I like learning new grammar. 0 0 16.8 24.8 43.6 9.9 5.0 3 1.211 4.211 0
72. If I learn something well, it is because my teacher taught 2.7 1.4 16.4 46.6 27.4 4.1 1.4 3 1.202 4.202 0
well.
241. I do not go on to the next question in an exercise until I 11.0 9.6 47.9 19.2 9.6 1.4 1.4 3 1.202 4.202 0
am sure about the answer.
45. I think about how I study best. 0 0 4.1 9.5 37.8 31.1 17.6 3 1.198 4.198 0
20. All teachers are equally good 0 1.3 3.9 15.8 40.8 14.5 23.7 3 1.195 4.195 0
147. I worry if I don’t understand all the words in a text. 0 8.1 23.0 23.0 36.5 6.8 2.7 3 1.190 4.190 0
242. I guess answers if I don’t know them for sure. 1.0 0 5.9 17.8 53.5 16.8 5.0 3 1.181 4.181 0
106. I know the sources of the texts I read. 2.8 1.4 44.4 31.9 15.3 0.0 4.2 3 1.179 4.179 0
40. I usually need the teacher to help me with my English 1.0 8.9 30.7 28.7 24.8 3.0 3.0 3 1.177 4.177 0
language learning.
5. I work hard to learn English. 1.0 1.0 3.8 5.7 37.1 27.6 23.8 3 1.177 4.177 0
203. I organise my time for studying. 0 2.7 6.8 21.6 51.4 10.8 6.8 3 1.175 4.175 0
199. I always notice my mistakes. 0 4.0 15.8 27.7 39.6 9.9 3.0 3 1.171 4.171 0
245. I need the teacher to check my answers. 2.0 6.1 31.3 34.3 19.2 4.0 3.0 3 1.166 4.166 0
215. I ask for help from English speakers. 0 0 5.9 14.9 56.4 13.9 8.9 3 1.166 4.166 0
142. I fix my problems in vocabulary. 0 1.4 8.3 29.2 41.7 13.9 5.6 3 1.165 4.165 0
201. I have clear goals for improving my English. 0 0 4.0 25.0 45.0 19.0 7.0 3 1.165 4.165 0
171. I use my background knowledge when I do something 0 0 2.0 7.9 52.5 23.8 13.9 3 1.160 4.160 0
new.
158. I guess the meaning of new grammatical structures. 1.0 0 9.0 20.0 55.0 12.0 3.0 3 1.155 4.155 0
15. When I learn something new I don’t feel good 0 0 1.4 2.7 39.2 27.0 29.7 3 1.155 4.155 0
233. I know how to check my own work for mistakes. 0 0 11.1 23.2 52.5 8.1 5.1 3 1.146 4.146 0
377
Likert Responses as Percentage Selection Criteria
Item 1 2 3 4 5 6 7 A B C D E
167. I select learning techniques (i.e. taking notes, or 0 0 1.4 9.7 54.2 15.3 19.4 3 1.145 4.145 0
drawing diagrams, or by listening, etc.) that suit my best way
of learning.
186. I like working in pairs or small groups in class. 0 3.0 5.0 27.7 38.6 15.8 9.9 3 1.139 4.139 0
156. If I try my grammar will get better. 0 0 2.7 9.6 47.9 24.7 15.1 3 1.136 4.136 0
19. The student’s job is to remember the content of all 1.0 1.0 20.4 26.2 42.7 1.9 6.8 3 1.133 4.133 0
lessons
151. I try to find ways of practising grammar outside class. 0 0 10.9 27.7 48.5 8.9 4.0 3 1.125 4.125 0
160. I keep a record of new grammar. 2.0 1.0 6.9 18.6 49.0 16.7 5.9 3 1.120 4.120 0
109. I predict the content before I listen. 0 0 9.0 21.0 49.0 13.0 8.0 3 1.117 4.117 0
141. I know my problem areas in vocabulary. 0 1.4 6.8 16.4 52.1 19.2 4.1 3 1.114 4.114 0
128. I can help myself to improve my writing. 0 0 8.0 30.0 48.0 10.0 4.0 3 1.107 4.107 0
177. I follow textbooks as closely as possible. 5.4 8.1 50.0 23.0 10.8 1.4 1.4 3 1.091 4.091 0
168. I think about the context for something new I am 0 0 2.7 9.6 52.1 24.7 11.0 3 1.083 4.083 0
learning.
89. There is no one correct way to write an essay. 1.3 0 1.3 6.7 46.7 22.7 21.3 3 1.083 4.083 0
248. I am motivated by making progress in learning. 0 0 1.0 2.0 29.7 31.7 35.6 3 1.083 4.083 0
63. My dictionary is always right with its definitions. 2.7 0 20.0 24.0 48.0 2.7 2.7 3 1.082 4.082 0
122. It is important to check one’s writing. 0 0 8.0 8.0 42.7 26.7 14.7 3 1.080 4.080 0
76. I rely on the teacher when learning. 0 1.4 8.1 17.6 48.6 14.9 9.5 3 1.078 4.078 0
3. I want to learn something new every day. 0 0 2.6 6.5 22.1 26.0 42.9 3 1.076 4.076 0
59. When I like a learning activity, I know why I like it. 0 0 1.4 6.8 52.7 24.3 14.9 3 1.075 4.075 0
113. I look for opportunities to speak English outside class. 0 0 4.9 21.6 52.9 10.8 9.8 3 1.074 4.074 0
35. When I study English, if I am not sure what I have to do I 0 0 5.9 16.7 53.9 14.7 8.8 3 1.071 4.071 0
ask somebody.
51. I know why I have problems learning. 0 0 6.8 17.6 58.1 10.8 6.8 3 1.070 4.070 0
170. I use my knowledge from other subjects when I study 0 0 3.0 8.9 46.5 28.7 12.9 3 1.067 4.067 0
English.
101. Different types of text (novel, newspaper, web site etc.) 0 0 1.0 9.8 40.2 32.4 16.7 3 1.065 4.065 0
are read in different ways.
378
Likert Responses as Percentage Selection Criteria
Item 1 2 3 4 5 6 7 A B C D E
87. I need to be sure about the instructions for learning 6.8 14.9 47.3 23.0 6.8 0.0 1.4 3 1.064 4.064 0
activities.
192. I do my English homework. 0 0 4.9 7.8 45.6 20.4 21.4 3 1.064 4.064 0
81. It is good to make your own choices about learning. 0 0 2.7 9.3 36.0 25.3 26.7 3 1.061 4.061 0
217. I make learning plans. 0 1.9 8.7 23.3 41.7 19.4 4.9 3 1.061 4.061 0
149. If I try my vocabulary will get better. 0 0 0 0 39.2 33.8 27.0 3 1.053 4.053 0
80. I make my own choices about learning. 0 0 3.9 9.2 43.4 22.4 21.1 3 1.052 4.052 0
232. My own needs are important to the way I learn. 0 0 0 2.7 46.6 31.5 19.2 3 1.046 4.046 0
44. I know my strong points and weak points related to 0 0 1.0 4.9 46.1 29.4 18.6 3 1.036 4.036 0
learning English.
54. I choose the best way to learn something. 0 0 8.0 20.0 46.7 16.0 9.3 3 1.033 4.033 0
84. The work I do for my course is my teacher’s, not mine. 0 0 1.3 8.0 34.7 25.3 30.7 3 1.025 4.025 0
196. I am confident I can learn English well. 0 0 2.0 7.9 34.7 28.7 26.7 3 1.015 4.015 0
43. When I have something to learn I try to think of different 0 0 3.9 21.6 48.0 21.6 4.9 3 1.014 4.014 0
ways of doing it.
253. I know how to use English language reference books 0 1.4 4.1 9.5 54.1 18.9 12.2 3 1.010 4.010 0
(encyclopedias, dictionaries, etc.).
103. It is best to read by starting at the beginning and 0 1.9 13.6 34.0 36.9 11.7 1.9 3 0.999 3.999 0
reading line by line to the end.
107. When I read I think about the motives of the writer. 0 0 13.7 20.6 50.0 10.8 4.9 3 0.997 3.997 0
184. I like negotiating with other students in class. 0 0 7.0 22.0 50.0 12.0 9.0 3 0.993 3.993 0
104. I read in English outside class. 0 0 3.9 7.8 50.0 21.6 16.7 3 0.987 3.987 0
134. I keep a record of new words. 1.0 0 5.8 27.2 49.5 10.7 5.8 3 0.984 3.984 0
250. Students can help the teacher choose the subject of 0 0 3.0 22.0 52.0 18.0 5.0 3 0.984 3.984 0
lessons.
62. Sara eats a lot of sweets so she must be fat. 0 0 6.8 14.9 58.1 9.5 10.8 3 0.979 3.979 0
131. I enjoy writing in English. 0 1.0 12.9 38.6 34.7 9.9 3.0 3 0.976 3.976 0
78. I enjoy making my own choices about learning. 0 0 1.3 7.9 39.5 27.6 23.7 3 0.976 3.976 0
58. I use the teacher’s comments and corrections in my 0 0 2.9 9.7 45.6 26.2 15.5 3 0.965 3.965 0
written work to improve my English.
379
Likert Responses as Percentage Selection Criteria
Item 1 2 3 4 5 6 7 A B C D E
123. I check my writing. 0 0 4.0 8.0 49.3 25.3 13.3 3 0.954 3.954 0
99. I sometimes look up words on the Internet or in reference 0 0 5.4 2.7 60.8 16.2 14.9 3 0.952 3.952 0
books.
190. I do extra work for learning English. 0 0 3.9 9.7 52.4 21.4 12.6 3 0.946 3.946 0
224. In learning it is important to work independently. 0 0 1.0 7.0 36.0 32.0 24.0 3 0.946 3.946 0
67. Knowledge is something I construct for myself. 0 0 2.6 2.6 57.9 15.8 21.1 3 0.945 3.945 0
222. Repetition is important for learning. 22.5 25.5 44.1 6.9 1.0 0.0 0.0 3 0.944 3.944 0
220. I am ready to learn in unfamiliar ways. 0 0 2.7 9.3 49.3 24.0 14.7 3 0.943 3.943 0
77. I rely on myself when learning. 0 0 0 8.0 45.3 22.7 24.0 3 0.941 3.941 0
94. Reading is an active activity. 0 0 1.0 6.9 42.2 28.4 21.6 3 0.933 3.933 0
139. I use techniques to help me remember vocabulary. 0 0 6.9 13.7 55.9 16.7 6.9 3 0.928 3.928 0
50. I have tried different ways of learning. 0 0 9.2 10.5 59.2 15.8 5.3 3 0.923 3.923 0
79. I want to make my own choices about learning. 0 0 0 3.9 38.2 26.3 31.6 3 0.919 3.919 0
249. It is necessary to practice using English outside the 0 0 0 2.7 26.7 22.7 48.0 3 0.916 3.916 0
classroom.
216. I am aware of the feelings of others. 0 1.3 2.7 2.7 64.0 17.3 12.0 3 0.912 3.912 0
127. I know my problems in writing. 0 0 5.4 10.8 51.4 27.0 5.4 3 0.892 3.892 0
39. If I find an English word that I don’t know, I try to guess it. 0 0 1.9 20.4 56.3 11.7 9.7 3 0.889 3.889 0
191. I think about what I have studied in class. 0 0 3.9 16.7 39.2 37.3 2.9 3 0.887 3.887 0
228. I motivate myself. 0 0 2.0 14.7 43.1 32.4 7.8 3 0.885 3.885 0
238. I choose the exercises I work on. 0 0 4.0 23.8 46.5 21.8 4.0 3 0.883 3.883 0
4. In general, learning continues all of a person’s life. 0 0 0 1.3 23.4 15.6 59.7 3 0.883 3.883 0
206. I try to find the best environment for studying. 0 0 0 9.3 44.0 30.7 16.0 3 0.875 3.875 0
152. I know different ways of practising grammar. 0 0 12.5 30.6 50.0 4.2 2.8 3 0.871 3.871 0
146. Different types of text (magazine, letter, recipe etc.) 0 0 0 2.7 38.7 30.7 28.0 3 0.871 3.871 0
have different vocabulary.
137. I try to use newly learned words in my essays. 0 0 2.9 19.6 57.8 11.8 7.8 3 0.867 3.867 0
204. I use my mistakes to help me do better. 0 0 1.0 5.8 63.1 14.6 15.5 3 0.853 3.853 0
22. I know how to study 0 0 4.0 17.3 57.3 16.0 5.3 3 0.846 3.846 0
380
Likert Responses as Percentage Selection Criteria
Item 1 2 3 4 5 6 7 A B C D E
169. When I am learning something new I look for similarities 0 0 0 9.7 54.4 22.3 13.6 3 0.844 3.844 0
with things I already know.
231. I can study independently. 0 0 1.0 5.9 46.1 33.3 13.7 3 0.841 3.841 0
126. There are different types of writing. 0 0 0 1.3 21.3 32.0 45.3 3 0.827 3.827 0
219. I reflect on my learning. 0 0 1.0 10.8 56.9 21.6 9.8 3 0.825 3.825 0
97. When I read in English, I predict the content of a text 0 0 1.0 11.8 57.8 20.6 8.8 3 0.814 3.814 0
(using pictures, headings, the context, etc.).
174. I consider facts and come to objective conclusions. 0 1.4 0 18.9 59.5 17.6 2.7 3 0.776 3.776 0
52. I try to fix problems I have in learning. 0 0 0 5.3 50.7 33.3 10.7 3 0.760 3.760 0
13. When I learn something new I feel satisfaction in myself 0 0 0 0 14.7 34.7 50.7 3 0.729 3.729 0
165. Different people have different ways of learning. 0 0 0 0 13.3 17.3 69.3 3 0.721 3.721 0
85. I have a mature attitude to learning. 0 0 9.7 30.6 44.4 9.7 5.6 2 1.228 3.228 0
155. I can help myself to improve my level of grammar. 0 0 8.1 25.3 53.5 9.1 4.0 2 1.186 3.186 0
98. I read newspapers in a different way to books. 0 1.4 6.8 9.6 54.8 17.8 9.6 2 1.182 3.182 0
53. I know some different ways of learning. 0 0 4.1 5.5 72.6 11.0 6.8 2 1.127 3.127 0
28. I feel unlucky when I get bad marks. 1.3 3.9 9.2 23.7 48.7 9.2 3.9 2 1.123 3.123 0
207. I know the aim of the learning tasks I do. 0 0 1.0 8.1 55.6 27.3 8.1 2 1.076 3.076 0
209. I think about my progress in learning English. 0 0 2.0 7.1 65.7 16.2 9.1 2 1.074 3.074 0
118. I know different ways of practising speaking English. 0 1.4 9.6 28.8 46.6 6.8 6.8 2 1.026 3.026 0
69. In the last 4 months, I have thought that something a 0 0 8.0 22.7 53.3 8.0 8.0 2 0.968 2.968 0
teacher told the class was wrong.
102. My general knowledge helps me to understand texts I 0 0 2.0 4.0 53.5 30.7 9.9 2 0.964 2.964 0
read.
60. I always agree with what a teacher says. 0 0 9.2 30.3 48.7 9.2 2.6 2 0.873 2.873 0
132. I guess the meaning of new words. 0 0 1.0 7.8 68.9 18.4 3.9 2 0.658 2.658 0
A = Responses at 10+. B =Polarity (the smallest difference). C = Standard Deviation. D = Sum Score. E = No. of shaded.
Indicates a top-50 item in that category (there are 75 questions with at least one shaded)
381
10.5 Movement of teacher estimates
P1 P2
100.00 100.00
90.00 90.00
80.00 80.00
70.00 70.00
Categories Average
Categories Average
60.00 60.00
50.00 50.00
40.00 40.00
30.00 30.00
20.00 20.00
0.00 0.00
0.00 1.00 2.00 3.00 4.00 5.00 6.00 7.00 8.00 9.00 10.00 0.00 1.00 2.00 3.00 4.00 5.00 6.00 7.00 8.00 9.00 10.00
T Est T Est
382
P1 P2
100.00 100.00
90.00 90.00
80.00 80.00
70.00 70.00
Social Confidence
Social Confidence
60.00 60.00
50.00 50.00
40.00 40.00
30.00 30.00
20.00 20.00
R Sq Linear = 0.323
R Sq Linear = 0.224 10.00
10.00
0.00
0.00
0.00 1.00 2.00 3.00 4.00 5.00 6.00 7.00 8.00 9.00 10.00
0.00 1.00 2.00 3.00 4.00 5.00 6.00 7.00 8.00 9.00 10.00
T Est T Est
383
P1 P2
100.00 100.00
90.00 90.00
80.00 80.00
70.00 70.00
Linguistic Confidence
Linguistic Confidence
60.00 60.00
50.00 50.00
40.00 40.00
30.00 30.00
20.00 20.00
0.00 0.00
0.00 1.00 2.00 3.00 4.00 5.00 6.00 7.00 8.00 9.00 10.00 0.00 1.00 2.00 3.00 4.00 5.00 6.00 7.00 8.00 9.00 10.00
T Est T Est
384
P1 P2
100.00 100.00
90.00 90.00
80.00 80.00
70.00 70.00
Information Literacy
Information Literacy
60.00 60.00
50.00 50.00
40.00 40.00
30.00 30.00
20.00 20.00
0.00 0.00
0.00 1.00 2.00 3.00 4.00 5.00 6.00 7.00 8.00 9.00 10.00 0.00 1.00 2.00 3.00 4.00 5.00 6.00 7.00 8.00 9.00 10.00
T Est T Est
385
P1 P2
100.00 100.00
90.00 90.00
80.00 80.00
70.00 70.00
Locus of Control
Locus of Control
60.00 60.00
50.00 50.00
40.00 40.00
30.00 30.00
20.00 20.00
0.00 0.00
0.00 1.00 2.00 3.00 4.00 5.00 6.00 7.00 8.00 9.00 10.00 0.00 1.00 2.00 3.00 4.00 5.00 6.00 7.00 8.00 9.00 10.00
T Est T Est
386
P1 P2
100.00 100.00
90.00 90.00
80.00 80.00
70.00 70.00
Metacognition
Metacognition
60.00 60.00
50.00 50.00
40.00 40.00
30.00 30.00
20.00 20.00
0.00 0.00
0.00 1.00 2.00 3.00 4.00 5.00 6.00 7.00 8.00 9.00 10.00 0.00 1.00 2.00 3.00 4.00 5.00 6.00 7.00 8.00 9.00 10.00
T Est T Est
387
P1 P2
100.00 100.00
90.00 90.00
80.00 80.00
Empowered Self-Reliance
Empowered Self-Reliance
70.00 70.00
60.00 60.00
50.00 50.00
40.00 40.00
30.00 30.00
20.00 20.00
0.00 0.00
0.00 1.00 2.00 3.00 4.00 5.00 6.00 7.00 8.00 9.00 10.00 0.00 1.00 2.00 3.00 4.00 5.00 6.00 7.00 8.00 9.00 10.00
T Est T Est
388