RELIABILITY

AND VALIDITY
IN QUALITATIVE
RESEARCH

JEROME KIRK
University of California, Irvine
MARC L. MILLER
University of Washington

Qualitative Research Methods,

Volume 1

SAGE PUBLICATIONS ~
-~~~-~--~----------------------T_h_e_Pubhs~rs o! P~~!1ess110!:'!!!~~~ \._~
 Copyright c 1986 by Sage Publications, Inc.

Printed in the United States of America

All rights reserved. No part of this book may be reproduced

or utilized in any form or by any means, electronic or mechanical,
including photocopying, recording, or by any
information storage and retrieval system. without permission in writing CONTENTS
from the publisher.

Series Introduction s
For information address: Editors' Introduction 7
1. Objectivity in Qualitative Research 9
SAGE Publications, Inc.
275 South Beverly Drive Objectivity IO
Beverly Hills, California 90212 Plan of This Book 12

SAGE Publications India Pvt. Ltd.

M-32 Market
Greater Kailash I
@ SAGE Publications Ltd
28 Banner Street
London ECIY 8QE
l. Reliability and Validity
The "Positivist" View
13
14
The Discovery of the New 16
New Delhi 110 048. India England
Components of Objectivity 18

International Standard Book Number 0-8039-2560-3

3. The Problem of Validity 21
0-8039-2470-4 (pbk.)
Calling Things by the Right Names 23
Three Illustrations 24
Library of Congress Catalog Card No. 85-2412
Field Research as a Validity Check 29
4. Towud Theoretical Validity 32
FOURTH PRINTING, 1988
The Ancestors 33
When citing a University Paper, please use the proper form. Remember to cite the correct Papa Franz 35
Sage University Paper series title and include the paper number. One of the following Malinowski 37
formats can be adapted (depending on the style manual used): The Chicago School 38
(I) AGAR, MICHAEL H. (1985) Speaking of Ethnography. Sage University Paper Stages and Phases 40
series on Qualitative Research Methods, Volume 2. Beverly Hills, CA: Sage.
S. The Problem of Reliability 41
or
(2) Aga.r, Michael H. 1985. Speaking ofethnography. Sage University Paper series on Three Illustrations 43
Qualitative Research Methods (Vol. 2). Beverly Hills, CA: Sage. The Reporting of "Raw" Data 49
6

dicta issued by traditional field researchers, they are also being

shaped more distinctly by explicit philosophical and moral
positions. This series seeks to elaborate both qualitative tech-
niques and the intellectual grounds on which they stand.
The series is designed for the novice eager to learn about
specific modes of social inquiry as well as for the veteran EDITORS' INTRODUCTION
researcher curious about the widening range of social science
methods. Each contribution extends the boundaries of method-
ological discourse, but not at the expense of losing the un- The most venerable tradition among qualitative methods is
initiated. The aim is to minimize jargon, make analytic premises unquestionably participant observation. Strictly speaking, this
visible. provide concrete examples, and limit the scope of each stiff but precise phrase refers more to the oscillating situation of
volume with precision and restraint. These are, to be sure, researchers as they move in, through, and out of the field than it
introductory monographs, but each allows for the development does to a particular research technique. Jerome Kirk and Marc
of a lively research theme with subtlety, detail, and illustration. Miller, in this first volume of the Sage Series on Qualitative
To a large extent, each monograph deals with the specific ways Research Methods, classify fieldwork situations in terms of a
qualitative researchers establish norms and justify their craft. We highly general process model of participant-observation research.
think the time is right to display the rather remarkable growth of They do so well within the conventional wisdom of what
qualitative methods in both number and reflective consideration. constitutes science and. as their title suggests, concern themselves
We are confident that readers of this series will agree. largely with issues surrounding the scientific status of field data.
A sort of flowchart of social research, from discovery to analysis,
-John Van Maanen emerges in this monograph along with a guide to the critical issues
Peter K. Manning and themes that characteristically mark each designated phase of
Marc L Miller the process. A nondoctrinaire but distinctly pragmatic research
philosophy accompanies their efforts to bring rhyme and reason
to the often chaotic circumstances surrounding participant
observation.

7
RELIABILITY AND VALIDITY
IN QUALITATIVE RESEARCH
JEROME KIRK
University of California, Irvine
MARC L. MILLER
University of Washington

1. OBJECTIVITY IN QUALITATIVE RESEARCH

Qualitative research is a particular tradition in social science that

fundamentally depends on watching people in their own territory
and interacting with them in their own language, on their
own terms. As identified with sociology, cultural anthropology,
and political science, among other disciplines, qualitative re-
search has been seen to be "naturalistic," "ethnographic," and
..participatory."
It should be remarked at the outset that the term "qualitative"
in reference to this tradition has led to a variety of misunder-
standings. Technically, a "qualitative observation" identifies the
presence or absence of something, in contrast to "quantitative
observation,·· which involves measuring the degree to which some
feature is present. To identify something, the observer must know
what qualifies as that thing, or that kind of thing. This entails
counting to one. It follows from this narrow consideration that
qualitative research would denote any research based on percent-
ages, means, chi-squares, and other statistics appropriate to cardi-
nal, or counting, numbers.
On the other hand, "quality" connotes the nature, as opposed
to the "quantity," or amount, of a thing. According to this equally
limited consideration, qualitative research would denote any
research distinguished by the absence of counting.
AUTHORS' NOTE: We relish this opportunity to apologize to all those we fear to
aclcnowledgc.

9
10 11

These two plausible definitions directly contradict one another. the hypothetico-deductive method exemplifies this connotation.
Neither suits the present purpose very well. Whether or not a According to Popper (1959: 42), the scientist prepares to test
number gets used in the process of recording and analyzing theories by deriving from them hypotheses that can in principle
observations is an entirely abstract issue. By our pragmatic view, break down when applied in the real world:
qualitative research does imply a commitment to field activities.
It does not imply a commitment to innumeracy. Qualitative What characterizes the empirical method is its manner of exposing
research is an empirical, socially located phenomenon, defined by to falsification, in every conceivable way, the system to be tested.
its own history, not simply a residual grab-bag comprising all Its aim is not to save the lives of untenable systems but, on the
things that are "not quantitative." Its diverse expressions include contrary, to select the one which is by comparison the fittest, by
exposing them all to the fiercest struggle for survival.
analytic induction, content analysis, semiotics, hermeneutics,
elite interviewing, the study of life histories, and certain archival,
Even in Popper's sophisticated formulation, the hypothetico-
computer, and statistical manipulations. One purpose of the deductive model is rather an inaccurate and schoolmarmish
series of which this volume is a part is to elaborate on these and
description of what scientists do, but it properly contrasts the
other possibilities.
scientific enterprise with others (such as art or ethics) in which
The accumulated wisdom of the academic tradition of qual-
practitioners do not routinely subject their theories to that sort of
itative research is largely a formal distillation of sophisticated
empirical risk, or their egos to the potential of battery not only by
techniques employed by all sorts of professionals- adventurers,
the arguments of intellectual adversaries but also by the de-
detectives, journalists, spies- to find out things about people.
monstrative refutation of the empirical world.
Necessarily, the formal tradition has been accompanied by
It is in this latter sense that qualitative researchers have always
certain distinctive orientations. Qualitative research is socially
celebrated objectivity. A commitment to objectivity does not
concerned, cosmopolitan, and, above all, objective.
imply a desire to "objectify" the subject matter by .. over-
Objectivity measurement" (Etzioni, 1964), or to facilitate authoritarian social
relationships by treating human beings as though they were
"Objectivity," too. is an ambiguous concept. In one sense, it 1
certain features they may happen to have. It does not presuppose
refers to the heuristic assumption, common in the natural any radically positivist view of the world; it emphatically eschews
sciences, that everything in the universe can, in principle, be the search for final, absolute "truth,,. preferring to leave such an
explained in terms of causality. In the social sciences, this enterprise to philosophers and theologians. . . .
assumption often seems to miss the point, for much of what social The assumptions underlying the search for objectivity are
scientist~ try to explain is the consequence of inner existential simple. There is a world of empirical reality out there. The way we
choices made by people. In ordinary language, when we ask perceive and understand that world is largely up to us , but the
"why" a person acts as he or she does, we are generally inquiring world does not tolerate all understandings of it equally (so that
teleologically about his or her purposes. Indeed, if knowledge the individual who believes he or she can halt a speeding train
itself is taken to be merely the inevitable consequence of some with his or her bare hands may be punished by the world for
mechanistic chain of cause and effect, its logical status would acting on that understanding). There is a lo~g-standing intel-
seem to be compromised. lectual community for which it seems worthwhile to try to figure
In another sense, ..objectivity" refers to taking an intellectual out collectively how best to talk about the e~pirical world'. by
risk- the risk of being demonstrably wrong. Popper's model of means of incremental, partial improvements m understandmg.
12 13

Often, these improvements come about by identifying ambiguity deavor, whether it is of the natural or social variety, entails an
in prior, apparently clear, views, or by showing that the~e are appreciation of its objectivity. By this convention, the objectivity
cases in which some alternative view works better. Previously of a piece of qualitative research is evaluated in terms of the
held views are not in general taken to be refuted by such reliability and validity of its observations- the two concepts to
contributions, but complemented by them. "Truth" (or what which this monograph is devoted.
provisionally passes for truth at a particular time) is thus bounded Chapter 2 introduces the role of reliability and validity in ~he
both by the tolerance of empirical reality and by the consensus of unfolding of science. Chapter 3 more fully explores the meamng
the scholarly community (Blumer, 1968). of validity, and points out that much research (particularly
~ N at\lfal science is strongly identified with a commitment to
nonqualitative research) lacks not only validity but also any
objectivity. Like natural science, qualitative social research is means of appraising its validity. In Chapter 4, the history of
pluralistic. A variety of models may be applied to the same object
qualitative research is seen as a cumulative effort to correct this
for different purposes. A man may be an object of a certain mass flaw: Were it otherwise valueless, qualitative research would be
and size to an engineer, a bundle of neuroses to the psychologist, a
justified solely as a validity check. Yet, as is pointed out in
walking pharmacy to the biochemist, and a bank account with Chapter 5, much of the validity of qualitative research has been
desires to an economist. Light may have a frequency or (in this gained at the expense of reliability in the "discovery," or data-
case, by a describable transformation) consist of photons. Water
collection phase of research. Finally, Cha~ter 6 pre~ents a ~o?el
is the canonical acid and the ultimate primitive base. Natural of the fieldwork activity that constitutes discovery m qualitative
human vision is binocular, for seeing the same thing simul- research, and provides some detailed instructions for maintaining
taneously from more than one perspective gives a fuller under-
reliability in the process.
standing of its depth. The reason Einstein originally called his
theory of relativity the Theory of Invariance is because though
everything displays different aspects to different viewpoints, 2. RELIABILITY AND VALIDITY
some features remain the same.
Despite the prestige and success of natural science in recent
Plan of This Book years, application of science as a model for social "science·: is ~ot
inevitable. Many have argued that social science has an mtrm-
The several points in this orientation are easily reviewed. sically different set of goals that call for an altogether separate
Qualitative research is a sociological and anthropological tradi-
collection of methods. Others (nonscientists, it should be noted)
tion of inquiry. Most critically, qualitative research involves
contend that recent developments in the natural sciences entirely
sustained interaction with the people being studied in their own
discredit the fundamental notions (such as objectivity) of an
language, and on their own turf. Less important is whether or not,
earlier and outdated science.
or at what level of sophistication, numbers are employed to reveal
Yet whatever their detailed goals, the natural and social
patterns of social life. To see qualitative research as strictly scienc~s share an aspiration to cumulative collective knowledge
disengaged from any form of counting is to miss the point that its
that is of interest on its own merits to those other than the friends
basic strategy depends on the reconciliation of diverse research
and admirers ofits creators. This goal is exactly objectivity. In the
tactics.
natural sciences, objectivity is obtained in two ways. First,
It is our view that qualitative research can be performed as
experience is reported in such a way that it is accessible to. others,
social science. Understanding the workings of a scientific en- for example, when reporting an experiment every effort is made
14 15

to describe the way the experiment was carried out, just in case Such an assertion obviously concerns the investigator's theoret-
somebody else would like to try the same thing. Second, the ical view of the world as much as it does the psychic organization
results of the experiment are reported in terms of theoretically of the interviewees. The investigator's theory contains categories
meaningful variables, measured in ways that are themselves not imposed by the structure of empirical reality. Elements such
justifiable in terms of the relevant theories. as "attitudes" and ..public opinion" serve rather to organize
Since Wilhelm Dilthey and George Herbert Mead, the vast understanding of the world. Certainly, political and psycholog-
majority of social scientists have agreed that objectivity, in this ical theories that do not use these constructs (or even deny their
sense, is an admirable goal. Yet, the description of reliability and meaningfulness) are possible, and treating analytic devices as
validit~ ordinarily provided by nonqualitative social scientists though they are facts is the well-known fallacy of reification.
rarely seems appropriate or relevant to the way in which In response to the propensity of so many nonqualitative
qualitative researchers conduct their work. research traditions to use such hidden positivist assumptions,
It is the purpose of this book to reconcile the means-ends some social scientists have tended to overreact by stressing the
discrepancy. The remainder of this chapter will pursue the possibility of alternative interpretations of everything to the
argument that, subject to clearly specifiable differences in goals exclusion of any effort to choose among them. This extreme
and practice, social science is in every sense of the word fully as relativism ignores the other side of objectivity- that there is an
"scientific" as physics, and has fully as much need for reliability external world at all. It ignores the important distinction between
and validity as any other science. knowledge and opinion, and results in everyone having a separate
insight that cannot be reconciled with anyone else's.
The "Positivist" View Metaphysical polemics, often directed against caricatures of
In recent decades, the social science literature has incorporated the opposing views, largely miss the point. As is shown in the next
a great deal of discussion of an epistemology called "positivism." chapter, the problem is not so much one of metaphysics as it is a
(The term is generally employed by those advocating some pragmatic question of the validity of measurements. The survey
alternative view of knowledge, and often amounts to a straw researcher who discusses attitudes is not wrong to do so. Rather,
man.) In its strongest form, positivism denies objectivity as the researcher is wrong if he or she fails to acknowledge the
defined here by assuming not only that there is an external world, theoretical basis on which it is meaningful to make measurements
but that the external world itself determines absolutely the one of such entities and to do so with survey questions addressed to a
and only correct view that can be taken of it, independent of the probability sample of voters.
process or circumstances of viewing. No one seriously defends For any observation (or measurement) to yield discovery, it
such an ontology, but scholars attentive to the social and cultural must generate data that is (a) not already known and (b)
2
construction of social things (including social science) point out identifiable as "new" by the theory already in place. Most of the
that much research (particularly nonqualitative research) makes technology of''confirmatory"nonqualitative research in both the
sense only in terms of a set of unexamined positivist assumptions. social and natural sciences is aimed at preventing discovery.
Most often, these assumptions pertain to the "naturalness" of When confirmatory research goes smoothly, everything comes
the measurement procedure employed. Thus a survey researcher out precisely as expected. Received theory is supported by one
may interview a large number of people about their political more example of its usefulness, and requires no change. As in
attitudes, and conclude that "public opinion" says something. everyday social life, confirmation is exactly the absence of insight.
18 19

to subject predictions to empirical test. Each time Chauncey examine methodological formulations from other traditions to
greets his old friend Ricky, he does expose himself to the unlikely assess their adaptability to qualitative research.
possibility that he has mistaken a perfect stranger for Ricky. One appropriate and useful device first used in psychometrics
Much social research deliberately seeks out such "embarrassina" (th~ ~el_d ~f tests and measurements) is the partitioning of
interaction; Agar ( 1982) bas applied the hermeneutic term obJ~vity_!Bl~ t":~ ~~ponents: reliability and validity. Loosely
"breakdown" to these informative gaffes. The general commit- speaktng, reliability 11 the extent to which a measurement
ment of qualitative researchers to interacting with their objects of procedure yields the same answer however and whenever it is
study on the latter's home ground strongly encourages the carried :>ut; "validity" is the extent to which it gives the correct
discovery that what the researcher takes for granted at his home answer. These concepts apply equally well to qualitative obser-
does not apply in the new situation. The anthropologist who vations.'
returns alive from some exotic place must know something A standard physical example of reliability and validity involves
nontrivial about it. the use of thermometers to measure temperature. A thermometer
Relaxing certain of the narrow definitions of the hypothetico- that shows the same reading of 82 degrees each time it is lun
deductive model, then, facilitates discovery of the new and
unexpected. It would be an error, however, to drop the scientific
concern for objectivity. The scientific credo is one good way to
permit the resolution of a conflict of opinion. It is not the only
way; the scholastic solution, still prevalent in many disciplines
called "humanities," relies on argument and rhetoric rather than
on argument and demonstration. Another alternative is argu-
..
mentum ad ~rium-"migbt makes right." One attractive
feature of the scientific solution is that it is an extension of the
ordinary processes of inference that people use in everyday life
(Piaget, l 9S4). As Wilhelm Dilthey pointed out, it is impossible to
account for the observed reality of human interaction without
acknowledging that human beings have an innate capacity to
understand one another. Thus striving for ever-greater objectivity
is as much a part of people's everyday social inference as it is of
their everyday physical inference.
Components of Objectivity
The analogy between qualitative research and other scientific
methods and traditions has its limitations. Yct the ability of
practitioners of certain kinds of scientific endeavor to talk about
what it is they do is much more advanced than that of qualitative
researchers (Van Maanen, 1979). Indeed, a primary purpose of
this monograph is to remedy that situation. It is often useful to
18 19

to subject predictions to empirical test. Each time Chauncey examine methodological formulations from other traditions to
greets his old friend Ricky, he does expose himself to the unlikely assess their adaptability to qualitative research.
possibility that he has mistaken a perfect stranger for Ricky. One appropriate and useful device first used in psychometrics
Much social research deliberately seeks out such "embarrassing" (th.e ~el.d ~f tests and measurements) is the partitioning of
interaction; Agar (I 982) has applied the hermeneutic term object1VIty mto two components: reliability and validity. Loosely
"breakdown" to these informative gaffes. The general commit- speaking, "reliability" is the extent to which a measurement
ment of qualitative researchers to interacting with their objects of procedure yields the same answer however and whenever it is
study on the latter's home ground strongly encourages the carried out; "validity" is the extent to which it gives the correct
4
discovery that what the researcher takes for granted at his home answer. These concepts apply equally well to qualitative obser-
does not apply in the new situation. The anthropologist who vations.5
returns alive from some exotic place must know something A standard physical example of reliability and validity involves
nontrivial about it. the use of thermometers to measure temperature. A thermometer
Relaxing certain of the narrow definitions of the hypothetico- that shows the same reading of 82 degrees each time it is plunged
deductive model, then, facilitates discovery of the new and into boiling water gives a reliable measurement. A second
unexpected. It would be an error, however, to drop the scientific thermometer might give readings over a series of measurements
concern for objectivity. The scientific credo is one good way to that vary from around 100 degrees. The second thermometer
permit the resolution of a conflict of opinion. It is not the only would be unreliable but relatively valid, whereas the first would
way; the scholastic solution, still prevalent in many disciplines be invalid but perfectly reliable.
called "humanities," relies on argument and rhetoric rather than The standard example of the thermometer is neither very
on argument and demonstration. Another alternative is argu- qualitative nor very familiar to social scientists. A rather homier
mentum ad imperium-"might makes right." One attractive (if artificial) example occurs when Chauncey sees a blond man
feature of the scientific solution is that it is an extension of the across the room at a large cocktail party, and has the uncertain
ordinary processes of inference that people use in everyday life feeling that he knows him from somewhere. He looks again, sees
(Piaget, 1954). As Wilhelm Dilthey pointed out, it is impossible to the same thing, and continues to have the feelings of uncertainty.
account for the observed reality of human interaction without Chauncey has perfectly reliable data, and it is of no use. Is his
acknowledging that human beings have an innate capacity to feeling valid? (As in ordinary language, the technical use of the
understand one another. Thus striving for ever-greater objectivity term "valid" is as a properly hedged weak synonym for "true. j
is as much a part of people's everyday social inference as it is of Chauncey might ask himself whether it seems he would know a
their everyday physical inference. person who looks like that, moves like that, dresses like that, and
so on. ls the blond, in other words, apparently the sort of person
Components of Objectivity Chauncey would know? Or he might ask himself subtler ques-
The analogy between qualitative research and other scientific tions, such as whether people who look like that frequent the
methods and traditions has its limitations. Yet the ability of places he does. At a cocktail party, such a search for validity will
practitioners of certain kinds of scientific endeavor to talk about probably fail because the guest list is deliberately socially
what it is they do is much more advanced than that of qualitative homogeneous, and any two members are likely to have been in the
researchers (Van Maanen, 1979). Indeed, a primary purpose of same other places. So Chauncey must resort to empirical research
this monograph is to remedy that situation. It is often useful to if he is to discover whether his feeling is useful.
 21
20

Perhaps Chauncey's least costly pilot project would be to ask As a m~ans to the truth, social science has relied almost entirely
the host what the blond man's name is, or whether in fact the host on .te.c~~ques for assurin~ reliabilit~, in part because "perfect
has relevant information (e.g., that the blond has just arrived in vahd1ty is not even theoretically attamable. Most nonqualitative
the country for the first time from a place Chauncey has never resear~h ~ethodologies come complete with a variety of checks
been, or that the three of them had a conversation last week). on reltab1hty, and none on validity.
Another strategy would be to make ambiguous eye contact with
the blond, in such a way as to assign to the other responsibility for
acknowledging the acquaintance. Ultimately, it may prove nec- 3. THE PROBLEM OF VALIDITY
essary fo confront him and ask, "Don't I know you?"
If Chauncey devotes as much time to worrying about his . No experiment can be perfectly controlled, and no measuring
problem as it requires to read about it, we would conclude that he instrument can be perfectly calibrated. All measurement, there-
is socially inept, or at least painfully shy. This is one of the fore, is to some degree suspect. When the measurement is
problems of methodological discussion: detailing the inferential nonqualitative, this reservation may amount to no more than the
steps in getting the job done looks picky and absurd. If we sup- acknowledgment that "accuracy" is limited. 6 More generally,
pose this computation passes very quickly through Chauncey's however, the issue of validity is a fundamental problem of theory.
mind as he gives the blond a second glance, we might better To discuss the validity of a thermometer reading, a physical
empathize with him. When discussing the validity checks of social theory is necessary. The theory must posit not only that mercury
research, it is useful to remember that a careful description of expands linearly with temperature, but that water in fact boils at
what is done generally tends to suggest an obsessive preoccupa- 100°. With such a theory, a themometer that reads 82° when the
tion with detail on the part of the researcher. This is an artifact of water breaks into a boil can be reckoned inaccurate. Yet if the
the fact of description, not a recommendation for compulsive theory asserts that water boils at different temperatures under
behavior. different ambient pressures, the same measurement may be valid
Objectivity, though the term has been ta.ken by some to suggest under different circumstances-say, at one-half an atmosphere.
a naive and inhumane version of vulgar positivism, is the essential In the case of qualitative observations, the issue of validity is not a
basis of all good research. Without it, the only reason the reader matter of methodological hair-splitting about the fifth decimal
of the research might have for accepting the conclusions of the point, but a question of whether the researcher sees what he or she
investigator would be an authoritarian respect for the person of thinks he or she sees.
the author. Objectivity is the simultaneous realization of as much In the real world, validity is the issue of much contention over
rehability and validity as possible. Reliability is the degree to the organization of actions and events. In the scientific world,
which the finding is independent of accidental circumstances of validity is a common denomination in cause and effect discussions
the research, and validity is the degree to which the finding is of "pragmatic utility," "fruitfulness," "felicity of notation," and
interpreted in a correct way. "spuriousness." (These terms correspond to the terms "generality
Reliability and validity are by no means symmetrical. It is easy of scope," "robustness," '"replicability," and "insignificance"
to obtain perfect reliability with no validity at all (if, say, the applied to the sister issue of reliability in more abstract contexts.)
thermometer is broken, or it is plunged into the wrong flask). To focus on the validity of an observation or an instrument is to
Perfect validity, on the other hand, would assure perfect reli- care about whether measurements have currency (what do the
ability, for every observation would yield the complete and exact observations buy?), and about whether phenomena are properly
truth. labeled (what are the right names for variables?). The notions of
22
23

apparent validity, instrumental validity, and theoretical validity behavior (Durkheim, 1951), giving the same name to a mea-
are helpful in addressing these problems. surement of feelings of powerlessness (Srole, 1956) can be
In the best of worlds, a measuring instrument is so closely questioned-unless feelings of powerlessness can be indepen-
linked to the phenomena under observation that it is "obviously" dently shown related both to environmental disruption and to
providing valid data. Formal examinations of competence and deviant behavior.7 Theoretical validity is a hedge against concepts
achievement (e.g., academic, civil service, professional tests) are that are virtually defined as puns. For example, those who refer to
based on this kind of apparent validity: correct answers are an employer's furnishing tools to his or her workers as "aliena-
preferred to incorrect ones. Unfortunately, the validity of mea- tion,. from the tools of production beg the crucial theoretical
surements is too seldom evident "on the face of things." Con- question of whether that arrangement inevitably produces neg-
clusions of apparent validity are not entirely out of order, but they ative feelings on the part of the worker.
can be illusory. Apparent validity suggests or assumes instru- Theoretical validity underlies discussions of both apparent and
mental or theoretical validity; it can exist without them. instrumental validity. If the perverse examination on which good
A measurement procedure is said to have instrumental validity students differentially give the wrong answers were backed by a
(also referred to as "pragmatic" and "criterion" validity) if it can theoretical reason why it worked, its use could be justified
be shown that observations match those generated by an without resorting to apparent validity. Thermometers are not
alternative procedure that is itself accepted as valid. In most ordinarily calibrated by comparison with a standard thermo-
practical applications, demonstrating the validity of a measure- meter kept in the Bureau of Standards. Instead, they are
ment against a criterion is essentially unproblematic (Nunnally, calibrated by direct reference to the "boiling point of water"-a
1959). The expansion of a column of mercury can be shown to notion heavily burdened with a theory that says that under
correspond to other criteria of temperature such as vapor controlled circumstances water boils at a constant temperature.
pressure or electrical conductivity ("concurrent validity"); scores
on the Graduate Record Examinations correlate with the success Calling Things by the Right Names
of candidates in school and in their profession ("predictive Of course, definitions are made by people, and can be made any
validity''). The distinction between apparent and instrumental way the definer chooses. Consider this well-known exchange:
validity can be illustrated by imagining a Graduate Record
Examination on which those students who do well in graduate ..There's glory for you!"
school get all the questions wrong, whereas those who do poorly "I don't know what you mean by 'glory,'" Alice said.
in graduate school answer many of them correctly. For the
Humpty Dumpty smiled contemptuously. ·•or course you don't-
instrumental purpose of selecting graduate students, such an
till I telJ you. I meant, 'there's a nice knock-down argument.'"
exam might be excellent, but since it would have no apparent
validity, it would doubtless be illegal. "But 'glory' doesn't mean 'a nice knock-down argument,'" Alice
Finally, measurement procedures are seen to exhibit theoreti- objected.
cal validity ("construct validity") if there is substantial evidence "When I use a word," Humpty Dumpty said, in rather a scornful
that the theoretical paradigm rightly corresponds to observations tone, "it means just what I choose it to mean- neither more nor
(Cronbach and Meehl, 1955). For example, if the construct less."
"anomie" is taken as the subjective cultural state that associates "The question is," said Alice, "whether you can make words mean
sudden disruptions of the environment with an increase in deviant so many different things."
24 25

"The question is," said Humpty Dumpty, "which is to be master, qualitative research. Testing hypotheses against explicit alter-
that's all" [Lewis Carroll, 1960: 268-269]. natives cannot guard against unanticipated sources of invalidity.
The fieldworker, on the other hand, is continuously engaged in
In fact, Dodgson (Lewis Carroll) took a strongly nominalist something very like hypothesis testing, but that effectively checks
position on the issue of definitions, arguing (according to perception and understanding against the whole range of possible
Gardner, in Carroll, 1960) that if a writer chooses to produce a sources of error. He or she draws tentative conclusions from his
whole book in which "black" means white and the reverse, it is the or her current understanding of the situation as a whole, and acts
responsibility of the reader to adapt. Yet a definition that violates upon them. Where, for unanticipated reasons, this understanding
the ass.ociations given a familiar term by theory or by ordinary is invalid, the qualitative researcher will sooner or later (often to
language is useful only as Veblenian irony. 8 Goldschmidt (1982: the researcher's intense dismay) find out about it.
642), arguing against the use of the same term for voluntary Three examples illustrate some of the pitfalls in unthinking
human behavior and some other behavior on the part of another acceptance of apparently valid data. The first shows the dif-
species, puts it in the following way: ference between reliability and validity. The second and third
come from the literature of nonqualitative research, and in
The very fact that terms must be supplied with arbitrary meanings different ways display the relative disadvantage under which
requires that words be used with a great sense of responsibility. experimental and survey researchers labor.
This responsibility is twofold: first, to established usage; second,
to the limitations that the definitions selected impose on the user. "WHEN DO YOU GIVE COCA TO ANIMALS?"

Jn other words, we are under considerable constraint to keep Jn 1976, we jointly conducted field research in several South
our thinking clear by calling things by their right names. Graff American countries with the goal of understanding prevailing
(1979: 90), calling down relativistic "deconstructionist" literary cognitions and usages of coca, the organic source of the alkaloid
critics who disparage any concern for validity on the grounds that cocaine. Part of this work focused on the coca knowledge of the
external reality is just a matter of opinion, emphasizes that Peruvian urban lower-middle class, as exemplified by taxi
drivers, merchants, ~d restaurant personnel (all cosmopolitan
our ability to identify a perverse use of terms as perverse depends and accessible roles).
on the assumption that there is such a thing as calling things by In modern Peru, coca is perhaps less centrally located within
their right names, and this in turn depends on the assumption that the cultural milieu of Quechua-speaking indios and mestizos than
there is a common world and that language's relation to it is not it was in the postconquest era, but coca is emphatically not a
wholly arbitrary. predominantly displayed element of non-Indian life. Nonethe-
less, coca is widely available and legal throughout the country. At
Three Illustrations the outset, we were acquainted with the social problems of coca as
Questions concerning all three kinds of validity-apparent they were identified by the Peruvian government and the tourist
validity, instrumental validity, and theoretical validity-arise in brochures. The twin foci were a notoriously successful illicit
ethnographic field research. Apparent validity can be chimerical, cocaine industry, and the persistence of traditional Quechua
and may not signify theoretical validity. Instrumental validity is reliance on coca leaves.
ultimately circular, and cannot assure theoretical validity unless We proceeded by engaging the range of urban Peruvians at our
the criterion itself is theoretically valid. Theoretical validity, disposal in informal conversation about coca. As it turned out,
unfortunately, is diffi~ult to determine by methods other than our Mexican Spanish was sufficient to explore a domain loaded
28 29

the reluctance of her profession to look impartially at the facts in Tambiah and his co-workers also noted the remarkable
this way: It is "obvious''that women conform more than men, and difference in yield of the land held by rich and poor members of
those studies supporting this common-sense view are the ones the community. Leach points out that wealthier landowners
that tend to be cited and believed. Evidently, social psychologists, generally own the lands developed since purchase from the crown
like all people, are not immune to what Ross et al. ( 1976) call the around the turn of the century, whereas less prosperous land-
"fundamental attribution error": explaining others' behavior on holders are more likely to own older lands measured by tradi-
the grounds of personal disposition to behave in such ways across tional methods. He also points out that the standardized con-
a variety of situations, rather than (as we interpret our own version factor used by the social surveyers to render the size of
behavi~r) as a response to circumstantial and contextual traditional plots in acres was wrong. This factor systematically
pressures. overestimates the size of traditional plots by 50 percent. Un-
The error made in the early studies of gender and conformity critical acceptance of these nonsensical "acreage'' figures ap-
was an error of theoretical validity. Clearly, many of the studies parently led the Tambiah team to conclude that the traditional
measured not conformity but the variable of familiarity with land was unproductive because it yielded less per "acre."
stimuli. A proper conclusion would have been that women had Leach's objection follows directly from a sensitivity to theo-
less interest in and familiarity with spatial judgments and retical validity. He points out merely that the survey researcher
politicoeconomic issues than men. Unfortunately, the variable (who Leach labels "the sociologist") often does not really know
was given the wrong label, and the spurious, but socially what his or her variables are, and so risks entirely invalid
acceptable, conclusion (that women in general have less courage inferences:
of their convictions) was drawn.
What impresses me here is that when the sociologists encounter an
THE DISTRIBUTION Of WEALTH IN SRI LANKA
unexpected discrepancy of this sort they accept the validity ?f their
Leach ( 1967) has presented a classical critique of a social survey questionnaire data and simply analyse the figures so as to d~scover
(Sarkar and Tambiah, 1957) that, according to Leach, reached all their statistical significance. In contrast , the anthropologist sus-
the wrong conclusions about the distribution of wealth in a part pects the validity of the original data as such and looks for a source
of Sri Lanka. The survey, for example, defined a "household" as of error [Leach, 1967: 81].
"persons who cook their rice from the same pot." By this
definition, they found the striking statistic that some 66 percent of Leach was able to display these insights so easily because he
households owned no paddy land. Leach points out from his field had previously done qualitative field rese~rc? in ~ °:e~by,
experience in a culturally similar nearby village that it is very culturally similar area. Even when the quanhtattve rehab~~ty of
common for a young married couple to live in the compound of survey research is essential to the research goal, the add1t1onal
the husband's father and work the land that is the husband's own perspective of qualitative research is useful as a rule for the
in all but formal title. As every married woman has her own purpose of assuring validity.
cooking pot, such a couple would be considered "landless,"
independent of the size or value of the holdings the husband Field Research as a Validity Check
manages and expects to inherit. Indeed, such a son's relationship Believing a principle to be true when it is not (i.e .• mistakenly
would be called anda (sharecropping). The frequency of anda was .. rejecting the null hypothesis'') is called ·:!~pe ~ne ~rror." I~ is .n ot
seen by Tambiah et al. to reflect unequal land distribution and the only possible kind. "Type two error ts reJectmg a pnnc1ple
contribute to the disintegration of the village. when in fact it is true. "Type three error" is asking the wrong
30 31

question (Raiffa, 1968, credits John Tukey with this insight.) tions, where it is quite apparent that the investigator makes
Asking the wrong question actually is the source of most validity assumptions about meanings, situations, and attributions at his
errors. Devices to guard against asking the wrong question are or her own risk. Because of this built-in sensitivity, field research
critically important to the researcher. intrinsically possesses certain kinds of validities not ordinarily
Diversity of method is a strong candidate for such a device. possessed by nonqualitative methods.
Webb et al. (1966: 174) advocate this kind of (theoretical) validity The "automatic" validity of qualitative field research has
check: contributed to the romantic image of the anthropologist strug-
gling to survive under maximally difficult social and physical
The .most fertile search for validity comes from a combined series circumstances. Anthropologists do not struggle alone. The
of difference measures, each with its idiosyncratic weaknesses, historical record, outlined in the next chapter, indicates that a
each pointed to a single hypothesis. When a hypothesis can survive high proportion of field research technique has been developed in
the confrontation of a series of complementary methods of testing, the streets of Chicago and other urban and industrial contexts
it contains a degree of validity unattainable by one tested within by people who called themselves sociologists (and sometimes
the more constricted framework of a single method. political scientists or social psychologists). Nonetheless, simply
by virtue of being "in the field"-in territory controlled by the
Webb et al. agree, in other words, with the moral we draw from investigatees rather than the investigator-qualitative research
the three war stories above-that the various errors were partakes of this virtue of adversity.
avoidable by multiple exposures of differing kinds to the problem To the extent that confirmatory methods are used, they are
area. It is also true that the more diffuse and less focused the used quickly and informally, rather than constituting entire
method, the wider net it casts. This, too, is a basic argument for research projects of themselves, and they belong to the data-
the value of qualitative research. collection rather than the analysis phase of the research. The field
Typically, the qualitative researcher arrives on the scene with investigator simply does not have the resources to "control all
considerable theoretical baggage but very little idea of what will relevant variables"; when he or she takes time out for statistical
happen next. Using theory, common sense, and any resources at analysis, the researcher's tools are essentially restricted to those
hand, the researcher attempts first, to survive in the field that can be written on the back of an envelope. Above all, the field
situation, and second, to work him- or herself into a position researcher is at the mercy of the world view of his or her subjects.
where both observation and interviewing of locals will be This is not to take an entirely idealist view. Very often, the
possible. determining factor in human life is material reality. The ratio of
Face-to-face, routine contact with people continues through- population to available food resources in a particular time and
out the period of fieldwork, and unless the fieldworker is place, for example, is an ecological fact with immense conse-
unusually craven or complacent, his or her emerging hypotheses quences for the organization of everyday life and thought. As
are continually tested in stronger and stronger ways in the Harris ( 1979) points out, cultures that do not somehow manage to
pragmatic routine of everyday life. This "method" is unusually adapt the cognitive structures of their members to the physical
~ensiti.ve to discrepancies between the meanings presumed by environment do not last long.
investigators and those understood by the target population. Yet the study of human groups and cultures must take
Indeed, this is one reason that qualitative research has been such a members' meanings into account. First, social evolution of
dominant method in the anthropological study of exotic popula- whatever sort is a very slow process compared with environ-
32 33

mental change. Even in the large-scale examples of human particularly associated with the contributions of Franz Boas,
evolution that concern some anthropologists, emplaced cognitive Bronislaw Malinowski. and Robert Park. 10
structures have substantial effects in two ways. It takes genera-
tions for certain kinds of cultural entities to change noticeably, The Anceston
and it is a truism that for the individual in the short run, behavior Most cultures probably define a role that includes anthro-
is governed by norms, beliefs, and expectations. It is also true that pology (or more properly, xenology, the study of foreigners). We
the measurement of material reality is often intrinsically prob- have records of ethnographic fieldwork in the European tradition
lematic, particularly when some interaction with subjects is dating at least to the unsung sources of Herodotus of Heli-
necessary. It may be relatively easy, in practically any culture, to carnassus (c. 484-425 B.C.). In 1800, Joseph-Marie Degerando, a
count t~he number of people living in a village. But the demog- member of the pioneering Societe des Observateurs de /'Homme,
rapher may need other information, such as the number of live published a field manual for a scientific expedition, remarking,
births over some period of time. It is typical of field demog-
raphers' experience around the world that this information is The first fault that we notice in the observations of explorers on
difficult to elicit with straightforward questions. Having large savages is their incompleteness; it was only to be expected, given
numbers of children often has positive (or negative) status the shortness of their stay, the division of their attention, and the
connotations, so that members are motivated, even when they are absence of any regular tabulation of their findings [Degerando,
trying to be honest, to miscount. Often it is difficult to elicit an J969; 65].' I
exhaustive list of the children a woman has ever had, because
after the enumeration is complete it turns out that, oh yes, there In 1843, Gustave Klemm published a compilation of data on
was another one, but it broke. other cultures comparable in intent to Murdock's (1967) Ethno-
We have no other technology for making this kind of validity graphic Atlas. These data were collected in large part by means of
check than long-run personal interaction. We can never be watching and interviewing native peoples, often in their natural
absolutely sure that we understand all the idiosyncratic cultural environment. Among Klemm 's sources were Ynca Garcilaso de la
implications of anything, but the sensitive, intelligent field worker Vega's Peruvian history, Joseph Banks' observations in Polynesia
armed with a good theoretical orientation and good rapport over from Captain James Cook's 1768-1781 voyages, and Henry Rowe
a long period of time is the best check we can make. Schoolcraft's reports of decades spent among the Algonquians.
Also in 1843, the Royal Anthropological Institute of Great
Britain and Ireland commissioned J . C. Prichard to chair a
4. TOW ARD THEORETICAL VALIDITY committee that compiled a field guide (Notes and Queries on
Anthropology, first edition, 1874) for those engaged in collecting
Qualitative research finds its formal and intertwined roots in primary data.
the traditions of cultural anthropology and American sociology. Jn the meantime, the evolutionary theories of Charles Lyell,
Implicitly oriented to the question of validity, generations of field Herbert Spencer, and Charles Darwin had laid sufficient
researchers have for over a hundred years worked and reworked groundwork for the consolidation of a professional role by Lewis
the particulars of ethnographic inquiry. This has involved several Henry Morgan (1818-1881), Edward B. Tylor (1832-1917). and
important breakthroughs in method. In a generic way, these Sir James G. Frazer (1854-1941).1 2 These ..founding fathers" of
refinements on the work of the earliest of researchers are anthropology compiled, transcribed, and evaluated the reports of
 34 35

colonial officials, missionaries, merchants, and others who had Papa Franz
described and commented upon the customs and culture of non-
Much of what was called "social anthropology" in Britain was
Western societies.
called "sociology" on the continent, and it was under this latter
Contemporary anthropologists react with a certain ambiva-
name that it was first taught in the United States, notably at the
lence to the achievements of these men. On the one hand, their
new universities of Johns Hopkins and Chicago. William G.
contributions to the theory of social evolution, to the "compar-
Sumner taught a course in sociology at Yale in 1876, and by 1889
ative method," and to accurate knowledge of the substantive
the University of Kansas, seeking a trained sociologist, solicited
details of cultural variation are gigantic. Further, there is a record
F. W. Blackmar from the well-known program at the Hopkins. In
of considerable intercultural travel, dabbling, and casual mixing
1892, the new UDiversity of Chicago opened with a department of
with the natives, though only Morgan seems to have spent any
"social science and anthropology" consisting of sociologists
considerable time actually interacting with members of other
Albion Small and C. W. Henderson (Small, 1916).
cultures. On the other hand, modern anthropologists are not
The term "anthropology" continued to carry a certain implica-
willing to dignify what these figures did with the respectful term
tion of gentlemanly amateurism until 1896, when Franz Boas,
"fieldwork." As Evans-Pritchard ( 1951: 7 I-72) insists,
armed with a German doctorate in physics and field experience in
It is indeed surprising that, with the exception of Morgan's study Greenland, came to Columbia University (where Franklin
of the Iroquois ... not a single anthropologist conducted field Giddings had been teaching sociology for some years). At
studies till the end of the nineteenth century . . .. Williams James tells Columbia, Boas established the profession of cultural anthro-
16
us that when he asked Sir James Frazer about natives he had pology that he was to dominate for the rest of his life. His
known, Frazer exclaimed, "But heaven forbid!" 13 mission, according to Harris (1968), was "to rid anthropology of
its amateurs and armchair specialists" by making ethnographic
We are left with rather a paradox. Although in the iconography research in the field the central experience and minimum attribute
of both sociology and anthropology the discipline of anthro- of professional status. To the present day, "field experience"
pology (and implicitly the technology of qualitative research) constitutes such an attribute for anthropology, and the implicit
grew "from nothing to maturity" in the period 1860-1890 (Tax, requirement of fieldwork virtually suffices to distinguish the
17
I 955), its legendary practitioners at that time obviously failed to anthropologist from the sociologist.
match the quality of the field research of nonprofessionals Both as a charismatic teacher and as a collector of ethno-
centuries earlier. 14 graphic materials, Boas was tireless. Virtually every major
This paradox is resolved when we recognize that these anthropologist working in the United States during the first half
practitioners of "verandah" (or "armchair'') anthropology did of the century was trained, directly or indirectly, by Boas, and
part of the job extraordinarily well. Their sin of omission, by every professional organization in the discipline felt his impact.
· cmitemporary standards, was to do exclusively secondary analy- Boas was revolted by the wildly speculative and amateurish
sis, and to lack any commitment to the fieldwork experience. The pronouncements that passed at the time for anthropology, and
kind of work they did would today be seen as a reasonable model was equally offended by the crude racism into whose service these
for a student paper, such as a "library research" master's thesis, "theories" were typically pressed. In response, Boas insisted not
rather than for genuinely "professional" work. 15 only that the analyst collect his or her own data, but that it be
reported as nearly without comment or interpretation as possible.
36 37

Clearly, he overdid it. Boas in effect discarded the possibility of Malinowski

any sort of explicit theory. He seems not to have realized that a At the outbreak of World War I, a young Polish anthro-
thousand pages of Kwakiutl routines and recipes do not, by pologist named Bronislaw Malinowski was arrested in Australia
themselves, produce understandings of a culture. Not only did as an enemy alien and confined to the South Pacific by the British.
Boas fail to produce a single overall description of any culture Upon his return to England, he published several monographs
during his long career, he did not even organize his thousands of about the lives of the Trobriand Islanders that have remained
pages of materials in such a way that anyone else has ever been exemplars of what anthropological fieldwork can do. Reading
able to summarize them. Shortly after Boas's death, Murdock Malinowski's reports gives insight into the life of a foreign people
( 1949: xiv) characterized him critically as that makes them seem people like ourselves doing their best under
ecological and historical circumstances different from ours,
extravagantly overrated by his disciples ... the most unsystematic
rather than incomprehensibly savage primitives. For many,
of theorists, his numerous kernels of genuine insight being
scattered amongst much pedantic chaff. He was not even a good Malinowski has represented the ethnographer as hero-the
field worker [and in a footnote]. Despite Boas '"five-foot-shetr' of outstanding individual with the courage to move alone into a
monographs on the Kwakiutl, this tribe falls into the quartile of ..savage"village and the perceptiveness to understand and explain
those whose social structure and related practices are least what the people of that village were up to (Sontag, 1963).
adequately described among the 250 covered in the present study. The posthumous publication of Malinowski's Diary In The
Strict Sen.se Of The Term in 1967 came as a rude shock to the
Murdock's strong statement has been disputed, but it is certain profession. The Diary, never intended for publication. reveals
that Boas's fieldwork was by contemporary standards inadequate Malinowski as a bitchy, neurotic, and self-centered individual
in a number of respects. He seems never to have spent more than a (though one who in these pages is capable of considerable
few weeks in any one field site, and his data are devoid of any candor). This revelation compromises not the credibility or
detailed observation or interviewing. Nor did he successfully quality of Malinowski's professional achievement but the myth
communicate the qualitative techniques at which he was expert to that good field research arises from the saintlike sympathy of an
his students. Mead (1972: 151-152) remarks, "I really did not extraordinary virtuoso of humanity. Geertz (1974: 27) has
know much about fieldwork. The course on methods that perceptively analyzed the insight we obtain from the Diary:
Professor Boas taught was not about fieldwork. It was about
theory- how material could be organized to support or call in If anthropological understanding does not stem. as we have been
question some theoretical point." taught to believe, from some sort of extraordinary sensi.bilit~, an
It is patently unfair, however, to judge the man who single- almost preternatural capacity to think, feel, and perceive hke a
handedly created and disseminated the "ethos" of fieldwork for native (a word. I should hurry to say, I use here "in the strict sense
failing to match the standards that later developed largely out of of the term"), then how is anthropological knowledge of the way
natives think, feel, and perceive possible?
his own work. A more appropriate criticism would be that Boas
and his students, fundamentally unsympathetic to theory of any
Geertz was correct in taking the scandalized response of the
kind, rarely supported any theoretical point, but exclusively
profession to the publication of the Diary as evidence t~at fiel~
employed their evidence to call in question those of others. In researchers have been emulating a myth. Apparently, Malmowsk1
rejecting "bad" theory, Boas neglected to replace it with good
did not himself employ the methods his talented and worshipful
theory.
 40 41

complished in a socially responsible and timely manner concealed Malinowski's analysis and interpretation certainly presupposes
a neglect of any standardization of method. As perhaps sociol- good data- which for Malinowski required immense investment
ogy's original participant observer, Nels Anderson ( 196 J) re- of time in the field to collect. And even as a city editor, Park
vealed later, .. the only instruction I recall from Park was, 'Write demanded accurate data and adequate analysis prior to writing a
down only what you see, hear, and know, like a newspaper story.
reporter.' ''20 Each of these three advances over the earliest professional
Douglas (1976) goes so far as to assert that Park's inclination ethnography has improved our confidence in the validity of our
toward cultural anthropology ultimately diluted the contribution work. Yet the tradition of qualitative research that has been
he was able to make toward "investigative" styles of research. Jn outlined here has grown up almost independently of the more
comparing a 1928 field research manual by Vivien Palmer with quantitative research methods (such as experiments and social
Buford Junker's 1960 manual (both from the Chicago School), surveys) that have emphasized the issue of reliability.
Douglas notes that the 1928 manuscript "contains no significant
reference to anthropology," while Junker "uses anthropological
5. THE PROBLEM OF RELIABILITY
and sociological sources indiscriminately." At the same time,
Douglas finds newer sociological work lacking in the suspicious, Comparison of findings is a basic process of scientific, as well
expose-seeking investigative intent he feels Park's journalistic as everyday life. Knowing what conclusions to draw when
background brought to the work of the Chicago School in its findings differ across studies (or even when they agree~ d~~ends
heyday. Douglas attributes the loss of the "conflict model" largely upon evaluations of the validity (see Chapter 3) and reliability of
to the influence of anthropologists recruited by Park such as
observations.
Robert Redfield and W. Lloyd Warner. Douglas's statement is Observations entail the recording of the reaction of some entity
rather strong, and certainly exaggerates the insensitivity of to some stimulus, even if the only stimulus is th~ . act of
anthropologists to diversity and conflict even in small, non- measurement. Reliability depends essentially on exp~ic~tly ~e­
Westem communities. Whether or not Park was able to sustain as scribed observational procedures. It is useful to distmguish
aggressive a skeptical position as a muckraker would want, several kinds of reliability. These are quixotic reliability, dia-
Douglas properly emphasizes the power of the journalistic ideals chronic reliability, and synchronic reliability. . .
that nourished Park's Chicago School. "Quixotic reliability" refers t~ the cir~umstances m _which a
Stages and Phases
single method of observation cont1~ua~l~ yields ~n unva.rymg ~e~­
surement. 21 The problem with rehab1hty of this sort is that it ts
Obviously, Frazer, Boas, Malinowski, and Park were each trivial and misleading. The absurd case of the broken thermo~­
more skilled at certain aspects, or phases, of the qualitative eter is an instance of this kind of reliability. In ethnographic
research process than at others. Yet, while their careers largely research quixotic reliability frequently proves only that the
overlapped, it is clear that they built on one another's achieve- investig~tor has managed to observe or elicit "p.arty line" or
ments in much the same order as the sequence of phases of a rehearsed information. America.?s, ~or example,_reh~~l~ res.~ond
particular research project. Boas's emphasis on accurate col- to the question, "How are you? with t~e knee-Jerk Fme. The
lection of data presupposes (perhaps wrongly, in Boas's case) reliability of this answer does not make it useful data about how
sufficient theoretical orientation to know what data is important. Americans are.
 42 43

"Diachronic reliability" refers to the stability of an observation Three IDustrations

through time. In the social sciences, the concept is manifest in
test-retest paradigms of experimental psychology and survey COMMUNITY POWER STRUCTURE
research. Diachronic reliability is conventionally demonstrated
Hunter's (1953) description of the workings of oligarchic
by similarity of measurements, or findings, taken at different
authority in "Regional City" (Atlanta) went somewhat beyond
times. The general applicability of diachronic reliability is
the sociological/ anthropological tradition of community studies
somewhat diminished by the fact that it is only appropriate to
typified by the Lynds' (1929) study of "Middletown" (Muncie)
measurements offeatures and entities that remain unchanged in a
and Warner's (1941) "Yankee City" (Newburyport) series. Hunter
changi_ng world. In the study of sociocultural phenomena, it is
( 1953: 233) concluded that power in Atlanta is demonstrably in
often dangerous to assume that configurations of data would be
isomorphic across substantial intervals of time. To make such an the hands of a small clique whose
assumption is to deny history.
leaders are interested in maintaining their own positions which
"Synchronic reliability" refers to the similarity of observations give them such things as wealth, power, and prestige. They are
within the same time period. Unlike quixotic reliability, syn- fearful that any swaying of the balance of power may destroy the
chronic reliability rarely involves identical observations, but positions they now hold, and of course they c~uld ~ right,
rather observations that are consistent with respect to the although it is felt that a case could be made for allaying their fears.
particular features of interest to the observer. In the apocryphal
story of the Tower of Pisa, Galileo's observation that unlike Hunter's study received a good deal of praise, but it also
objects took the same length of time to reach the ground was provoked a number of rather irritable attacks from both outr_a~ed
reliable despite the unlikeness of the objects. This kind of internal residents of Atlanta and social scientists, particularly political
reliability can be evaluated by comparisons of data elicited by scientists. These disagreements culminated in an elaborate study
alternate. forms (e.g., split-half testing, interrater correlation). of the political workings of New Haven (Dahl: 1961) that r~ached
Paradoxically, synchronic reliability can be most useful to field a very different conclusion. Dahl found no smgle group i_n New
res.ear.c?ers when it fails because a disconfirmation of synchronic Haven with overwhelming power, though the dynamic and
rehab1hty forces the ethnographer to imagine how multiple, but popular mayor occupied a central ~osi~ion as "c?i_ef nego~iator."
somehow different, qualitative measurements might simultane- Because the studies were done m different cities at different
ously be true. times the difference in their findings could be attributed either to
The main thrust of methodological development in qualitative diach~onic or synchronic unreliability- that is, to the simple fact
research during the last century has been toward greater validity. that the workings of power were different in Dahl's New Haven
In contrast to the concerns of many nonqualitative traditions and Hunter's Atlanta. But while the partisans of both Hunter and
~ssues o~ reliability have received little attention. The followin~ Dahl would probably be willing to stipulate such a difference,
1Uustrat1ons from qualitative research concern trade-offs in they agree that the main reason for t~e diff~rence in res~lts w~ a
attaining kinds of reliability, and also hint again at the delicate difference in method. Hunter obtamed hsts of promment .in-
links between reliability and validity issues. dividuals, and employed a panel of judges to select fr~m the li~ts
those who were "most influential ... The people on this short list
 44 45

were then interviewed about how they interacted with one obviously has the ability to exercise pressure if and when it
another and about who else were powerful members of the
chooses to do so.
community.
The debate that continued for the next decade or so ~as at
Dahl used an entirely different method, selecting several times quite ill-tempered, for each side saw the other as fanahcally
important community decisions (e.g., urban renewal, education) preoccupied with an epiphenomenon falsel~ labeled as p~~er.
on which official decisions had recently been made, and exam- The issue of reliability, then, became over time one of validity.
ining the history of public negotiations leading to those decisions. The real disagreement between the two tradit~ons w~s a~out the
"Pluralist" theorists, such as Dahl (1961) and Polsby (1980), definition of power. To the pluralists, power.is the hkehh~od of
fault t})e "reputational'' method for guaranteeing its own find- prevailing over another who is actively seekmg a contrad~ctory
ings. True, they concede, members of a community find the goal. To partisans of the reputational method, _power is the
question "who has power around here" intelligible, and can even
likelihood of getting one's way. When the commum~y.beh~ves by
come to a consensus about the answer. But pluralists argue that default (rather than as a consequence of a public dec1s1on) m su:h
this is just rumor or hearsay, and has no necessary bearing on the a way as to benefit some person or group, these latter see power m
actual functioning of power. They prefer the scientifically ascetic action· the former do not. To those who study power by
method of restricting their conclusions to those that may be
reputa~ion. the status quo and prevailing definitions o_f the
drawn from the public record rather than focusing on unverifi- appropriate topics for political debate are power resources m the
able myths about pulling the strings "behind the scenes." When hands of those who benefit from them. To those who study
this is done. investigators quite reliably see competition among olitical debate, power is only one way of obtaining benefit from
ihe community, and the individual or members of a grou~ who
groups, each of which has sufficient resources to keep the others
from having their way all the time, and all of whom are "kept
get goodies simply because they have always gotten them ts not
honest" by the underlying fear of an outraged mass electorate
pulling the plug on them. employing power. . . .
Both the reliability issue and the subsequent vahd1ty issue
Adherents of the reputational method retort that the pluralists,
proved highly productive, and after another d~cade or .so. t~e
too, guarantee their findings by choice of method. By restricting 22
sharpness of the debate has tempered. The question of rel~ab1lity
their attention to publicly debated governmental decisions, it is
led to studies like Freeman's ( 1968), i~ which a :ep~tation for
held, pluralists see only those issues on which the clique holding
power was seen to coincide wit~ multi?le orgamz~llon~ _con-
real power is either split or indifferent. Other issues, including nections and participation in a wide variety of public dec1s1ons,
most of the important ones, are "settled out of court'' without and Clark's (1971) comparative studies of the ~onsequences of
either public display or public participation. The "potential" The validity issue bas profoundly clarified conceptual
power of the mass electorate, they insist, is even more of a power. h. . t d dels
issues about power, and led to highly sop istica e mo
mythical entity than the power of economic notables operating
(Emerson, 1962~ March, 1966; Coleman, 1973).
through their publicly visible representatives, for patently the
electorate lacks information and other resources (most people, DAUGHTERS AND FATHERS
after all, do not vote), whereas the group in power just as
Recently, Freeman ( 1983) has confirmed the rele~ance of
diachronic reliability to study and restudy comparisons in
 46 47

qualitative research. Freeman impeaches Mead's (1928) early asking a British aristocrat whether adulterous hanky-panky go~s
ethnographic conclusions, most notably that anger, violence, and on in the Commonwealth.) It is possible that Samoan culture is
competition are neutralized in Samoan culture. 23 Freeman and not so different from contemporary American culture on the
Mead also differ substantially in their respective evaluations of score of adolescent sex. By analogy, an American daughter might
adolescent sexual behavior. well tell (to a young and female adult ethnographer) some pretty
By Mead's report, virginity at marriage is nominally important interesting stories based on considerable sophistication; at the
in Samoan society, but teenagers systematically engage in same time, an American male parent might assert (to an adult
considerable expert and playful sex. According to Freeman, this male ethnographer) that his daughters are virgin. By the s~e
misrepresents the facts. The cult of virginity, he argues, is taken token the father's description of his work life as a ..rat race"m1ght
very seriously by traditional and contemporary Samoans. Female seem 'inconsistent with the daughter's characterization of her
virgins are highly valued, eagerly sought after, and zealously Daddy and his friends as "nice men." .
protected by male kin. The point here is that Freeman's findings do not necessarily
The discrepancy between the observations of Mead and refute Mead's or vice-versa. Mead and Freeman were on to
Freeman compel a discussion of reliability. Indeed, Freeman different aspe~ts of a very large and complex subject. The partial
effectively finesses the issue of diachronic reliability, presenting understandings they achieved are different for good reason, and
historical and survey evidence to indicate that the Samoa he we are better off with both sets of findings than only one.
studied had not been radically transfigured during the decades
after Mead's visit. Freeman, then, sees this as an issue of EXTRAS
synchronic reliability. Naturally, he regards his own conclusions
One of the things that Rasmussen (Douglas, 1976) wanted to
as internaJly reliable and those of Mead as counterfeit.
know about the local massage parlors was whether they w~re
There is an alternative to choosing between Mead and
Freeman. The possibility of accepting both sets of findings, sex-for-money shops. To investigate, Rasmussen ~tar.ted h~ngmg
out in one near his home, talking, drinking, and JOkmg with the
however, requires a theory of how Mead and Freeman obtained
the different results that they did. Mead talked with female employees. Evidently, he achieved a very .high level of frien~ly
adolescents at a time she herself was a young woman. Freeman rapport. In answer to his casual que~t1?ns about sex with
conducted much of his study of Samoa with male parents at a customers, he was consistently told by his informants that they
gave no "extras." They conceded that some masseuses at some
time he himself was a high-ranking adult. Mead's and Freeman's
conclusions are together based on the interplay of the field parlors might be out-and-out prostitutes, but poin~ed out a
variety of reasons why they preferred to sell the fantasy instead of
researcher and the studied culture, but this case study may boil
down to different investigators observing different parts of the the reality. .
same Samoan scene. Douglas, supervising the researc~, argues m fa~?r of two rul~s
of thumb for investigative research m general: (a) Where theres
fndeed, the sexual behavior of adolescents is a touchy subject •
smoke, there's fire,"(and, ''Where there's some fire, th~re's bound
and Samoans are inclined to be insulted by the suggestion that
Samoan teenagers are promiscuous. (This reminds one of to be more fue"), and (b) "There's always far more immoral or
shady stuff going on than meets the eye." He recomm~nded
Malinowski's famous passage about a Martian anthropologist
greater skepticism to Rasmussen (Douglas, 1976). Accordmgly,
 46 47

qualitative research. Freeman impeaches Mead's (1928) early asking a British aristocrat whether adulterous hanky-panky go~s
ethnographic conclusions, most notably that anger, violence, and on in the Commonwealth.) It is possible that Samoan culture is
competition are neutralized in Samoan culture. 23 Freeman and not so different from contemporary American culture on the
Mead also differ substantially in their respective evaluations of score of adolescent sex. By analogy, an American daughter might
adolescent sexual behavior. well tell (to a young and female adult ethnographer) some pretty
By Mead's report, virginity at marriage is nominally important interesting stories based on considerable sophistication; at the
in Samoan society, but teenagers systematically engage in same time, an American male parent might assert (to an adult
considerable expert and playful sex. According to Freeman, this male ethnographer) that his daughters are virgin. By the s~e
misrepresents the facts. The cult of virginity, he argues, is taken token the father's description of his work life as a ..rat race"m1ght
very seriously by traditional and contemporary Samoans. Female seem 'inconsistent with the daughter's characterization of her
virgins are highly valued, eagerly sought after, and zealously Daddy and his friends as "nice men." .
protected by male kin. The point here is that Freeman's findings do not necessarily
The discrepancy between the observations of Mead and refute Mead's or vice-versa. Mead and Freeman were on to
Freeman compel a discussion of reliability. Indeed, Freeman different aspe~ts of a very large and complex subject. The partial
effectively finesses the issue of diachronic reliability, presenting understandings they achieved are different for good reason, and
historical and survey evidence to indicate that the Samoa he we are better off with both sets of findings than only one.
studied had not been radically transfigured during the decades
after Mead's visit. Freeman, then, sees this as an issue of EXTRAS
synchronic reliability. Naturally, he regards his own conclusions
One of the things that Rasmussen (Douglas, 1976) wanted to
as internaJly reliable and those of Mead as counterfeit.
know about the local massage parlors was whether they w~re
There is an alternative to choosing between Mead and
Freeman. The possibility of accepting both sets of findings, sex-for-money shops. To investigate, Rasmussen ~tar.ted h~ngmg
out in one near his home, talking, drinking, and JOkmg with the
however, requires a theory of how Mead and Freeman obtained
the different results that they did. Mead talked with female employees. Evidently, he achieved a very .high level of frien~ly
adolescents at a time she herself was a young woman. Freeman rapport. In answer to his casual que~t1?ns about sex with
conducted much of his study of Samoa with male parents at a customers, he was consistently told by his informants that they
gave no "extras." They conceded that some masseuses at some
time he himself was a high-ranking adult. Mead's and Freeman's
conclusions are together based on the interplay of the field parlors might be out-and-out prostitutes, but poin~ed out a
variety of reasons why they preferred to sell the fantasy instead of
researcher and the studied culture, but this case study may boil
down to different investigators observing different parts of the the reality. .
same Samoan scene. Douglas, supervising the researc~, argues m fa~?r of two rul~s
of thumb for investigative research m general: (a) Where theres
fndeed, the sexual behavior of adolescents is a touchy subject •
smoke, there's fire,"(and, ''Where there's some fire, th~re's bound
and Samoans are inclined to be insulted by the suggestion that
Samoan teenagers are promiscuous. (This reminds one of to be more fue"), and (b) "There's always far more immoral or
shady stuff going on than meets the eye." He recomm~nded
Malinowski's famous passage about a Martian anthropologist
greater skepticism to Rasmussen (Douglas, 1976). Accordmgly,
50 51

interpretations (perspectives) as equally valid, and expl~citly experimental situations. Observational sciences generally rely on
denies the possibility of mediating among them. As Hirsch the contrast between things that change (as planets) and those
suggests, there is no such thing as "raw data" in the purest sense. that stay relatively the same (as "fixed" stars).
Human beings do not simply perceive, then interpret, but rather Even within psychometrics, where the issue of reliability was
go through a process called cognition. The normal adult human is fust made explicit, the demand for quixotic reliability creates
not ordinarily fooled by his or her visual perspective into thinking irresolvable theoretical paradoxes. Both intelligence and per-
people walking toward him or her are growing taller, or that a disc sonality tests were originally designed as therapeutic tools; to
seen from an angle is elliptical. Prior to interpreting cognitive measure (or bring about) change in a variable designed to reflect
experi~nce, people match visual and other input with stored unchanging dispositions is futile by definition.
percepts in particular ways. This is to say they actually require a
theory (e.g., of stimulus constancy) in order to be able to see an FIELDNOTES AS A RELIABILITY CHECK
object as approaching rather than growing. (For a discussion of To place an observation in perspective in a theoretical context,
current psychological views of this process, see Anderson, 1980.) the analyst wishes to know as much as possible about the
Data, then, can only be reported in terms of some explicit or cognitive idiosyncracies of the observer- which is to say about
implicit theory. That the theories people use to perceive are not his or her theories. These theories include not only academic
altogether culture-free is shown by a variety of studies (see Segall commitments but also values, behavioral style, and experience-
et al., 1966). These studies show also that some theories are better features that are often classified as part of "personality." In
for perceiving some things, whereas other theories work better for everyday life, in historiography, in legal proceedings, and in
perceiving others. With a theory that the elephant is so large and journalistic reportage, much is made of the nature of the source.
complex that no single observation can encompass it, the various Too often, in blind imitation of the reporting style of natural
blind men's reports can be integrated without the necessity of science, a pretense is made by social scientists of being "neutral
special occult vision. With no interpretive theory, we may forget observers." Of course, this constitutes the arrogant claim to be a
that there is an elephant out there and simply marvel that the sighted person in a world of blind men, but worse, it fails to reflect
blind men can report anything at all. is For present purposes, the feature of natural-science reporting it is designed to emulate.
however, it is more important that if the blind men themselves Laboratory experiments are intended to display the effects of
(the field researchers) have an initial notion that each feels only a interaction among a very small number of variables, and all the
part of some huge object, their reports will not even be apparently "relevant" variables are reported. 26 The field observations of
contradictory. As Maquet (1964) has observed," A perspectivistic qualitative research intrinsically involve the observer, whereas
knowledge is not as such nonobjective; it is partial. . . . the observations made in a chemistry lab do so minimally if at all.
N onobjectivity creeps in when the partial aspect is considered as The theory we all share about chemical observations is that the
the global one." same observation will be made whether or not the observer is
Reliability, then-like validity-is meaningful only by refer- suffering from insect bites or malnutrition; extending that
ence to some theory. The implicit theory that requires all assumption to the ethnographic observer of a tropical village is
observations to be identical is rarely appropriate. In the natural doubtful. When the observation is presented stripped of in-
sciences, this quixotic reliability is only expected in artificial formation about how it was collected, the reader is unable to
52 53

place any meaningful interpretation on it because the status of field (Bowen, 1954~ Dumont, 1978; Levi-Strauss, 1961; Van
relevant variables is unspecified. The chemist who reports that he Maanen, 1982, and many others}, which serve not only to orient
or she obtained an explosion has little trouble knowing that what the reader both to the interpersonal and cognitive style of the
reagents he or she used is relevant; the participant observer who resear:her but also to his or her theoretical and methodological
commitments. Toward the end of distinguished careers auto-
begins without a clearly specified list of relevant variables may in . .
b1ograph1cal essays are frequently published (Powdermaker,
'
a misguided attempt at scientific modesty and "objectivity" refuse
to report equally relevant context. To do so is as arrogant as the 1966; Mead, 1972; Anderson, 1980). None of these confessions,
chemisi refusing to report his or her ingredients. however, seems to help us understand the context of observation
Qualitative research has developed a number of conventions sufficiently to shed new light on the observations actually made.
that aggravate the problem. Typically, the same individual They are ordinarily relegated to separate appendices or (worse) to
collects the data and presents an analysis of it (and, unfortunate- separate volumes. Divorced once more from the data and from
ly, is not always aware of which he or she is doing at a particular the situation in which it was collected, these self-interpretations of
point). Finished ethnographies are professionally circulated; the investigator tend to suffer from the same uncritical cognition
once in a while they are supplemented by discussions of field and leaps of interpretive logic as do the interpretations of the
methods (because competing methods are more often used in the target of study by the same researchers. This will inevitably be the
field, qualitative researchers from the discipline of sociology have case, even for those who can heroically resist the temptation to
traditionally been rather better at this than those from anthro- launder their own autobiographies.
pology); rarely has any researcher (or student) actually seen Frake (1964) has pointed out that whereas the ethnographic
another's field notes. Fieldworkers studying weak or oppressed record of qualitative researchers has traditionally consisted of
groups are professionally bound to take the role of protector of lists of questions and answers, "the tradition in modern anthro-
those groups: This not only leads to an inclination to "launder" pology . . . is not to make such a record public but to publish an
reports (Douglas, 1976), but discourages "poaching" on a col- essay about it." The ethnographer, according to Black and
league's "territory." Metzger (1965), "needs to know what question people are
Working ethnographers know they can report only what answering in their every act. He needs to know which questions are
happened to them in the field, not what life there was like before being taken for granted because they are what 'everybody knows'
they arrived. Yet to raise questions about the reliability of without thinking."
another's observations is taboo, as though it were an accusation From Mead to Whiting et al. ( 1966), efforts have been made to
of incompetence, bias, or dishonesty. "standardize" questions and the recording of observations. These
efforts are designed to introduce into qualitative observation
THE NATURE OF FIELDNOTES some of the reliability characteristic of laboratory and survey
methods. Unfortunately, as Labov and Fanshel (1977) comment
The contemporary search for reliability in qualitative obser-
about psychotherapeutic interviews, reports of such standardized
vation revolves around detailing the relevant context of obser-
interviews and observations sometimes provide so little of the
vation. W. F. Whyte ( 1955) discussed in revealing detail his
broader ethnographic context that the relevance of their reliable
experiences and the way he arrived at the methods he used. Others
findings to their conclusions is suspect. Excessive standardization
have issued personal statements about their experiences in the
54 55

deliberately abandons the attempt to discover things more for granted the same things as did the fieldworker at the time the
accessible to some observers than to others. notes were recorded. Indeed, qualitative researchers commonly
A casual example of Mead's (1965) technique displays the find their own earlier notes ambiguous or incomprehensible,
limits of standardization. In explaining to high school students because they have forgotten what it was that they knew or felt
the nature of fieldwork, she asked her colleagues similar when the notes were taken. Recording the questions that were
questions: asked contributes a great deal to the meaningfulness of notes. (As
Black and Metzger point out, the ultimate aid in understanding
"Professor Arens berg, you ·re a social anthropologist, aren•t you?" the answers would be a record not of what the fieldworker
"Yes." thought she was asking, but of what the informant heard.
Unfortunately, this is rarely possible.)
For many purposes (particularly for studying relatively famil-
.. Dr. Bunzel, you are Adjunct Professor of Anthropology at iar groups), recording questions, and to some degree standard-
Columbia, is that right?" izing them, is not only a necessary step, but a sufficient one. For
"That's right." other purposes, emulation of lahoratory protocols, closed-end
interview schedules, or linguistic eliciting techniques is not
sufficient. (Berreman lists these other purposes as "understanding
.. Professor Solecki, you're an archaeologist, aren't you?" how people relate to one another and to their environment, what
"Yes." is the nature of their social interaction. and how it relates to their
values, emotions, attitudes, and self-conceptions; their hopes and
The ..standardization" of questions here goes part way toward fears.") For those purposes, ..extensive, explicit, and perceptive
the reinvention of the social survey, a retrogressive move that field notes, self-analytical reporting of research procedures and
seeks quixotic reliability at the expense of validity. Mead, research contexts, documentation of sources, documentation of
however. was too good a fieldworker to be able to ask identical the bases for inferences, and documentation of the ethnogra-
questions of everyone. Professor Arensberg is not addressed as pher's theories of society and his biases" (Berreman, 1966) are
"Dr. Bunzel," or accused of being an archaeologist. Like all important.
competent fieldworkers, Mead used what she already knew about Spradley (1979) recommends four separate "kinds of field
the target culture to phrase her questions. (Whether any of her notes": the condensed (verbatim) account, an expanded account
high school audience could have chanced upon the correct recorded as soon as possible after each field session, a •'field work
questions is a different issue.) The reader, however, can place an journal" that contains "experiences, ideas, fears, mistakes, con-
interpretation on the otherwise uninformative answers ("Yes," fusions, breakthroughs, and problems that arise during field
and "That's right.") if and only if the reader knows the question. work," and a provisional running record of analysis and in-
terpretation. Both paradigms demand that certain minimal
CONVENTIONALIZED FIELONOTES
requirements be met. Entries must be legible and chronologically
One reason field notes are rarely published or distributed is that identified. They must clearly differentiate among various kinds of
they tend to be entirely unintelligible to anyone who does not take entry, and must record native-language utterances as nearly
56 57

verbatim as possible. (This last requirement entails c~~ain Diacritical Convention Use
necessary syntactic and diacritical deviations from prose wntmg.) double quotat10n marks contain verbatim quotes
The emerging conventionalization of field-note format per- single quotation marks contain paraphrases
forms a variety of services beyond making reliability possible. It parenthese~ encloses contextual data and/or
fieldworker'~ interpretation
encourages the incorporation of socially unde~irable but reve~­
ing content (selfish thoughts, obscene or racist remarks, wild < > angle brackets denotes elements of emic
lexicon
speculations, theoretically unpalatable inter~retatio~s, and other solid line partitions time
.. irresponsible" material of the sort that Ma~n?wsk1 r~legated to I $lash denotes emic contrast
his Diary). That the researcher might be unw1llmg to display such
Fi9ure 5 .1 Basic Fieldnot• Conventions
idiosyncratic passing thoughts to his or her informants, to
professional adversaries, or to his or her close friends should not,
as it bas tended to in the past, inhibit their recording. Freed from supportive colleagues actually encourages the verbalization of
the necessity to be "responsible" for the interpretive content of shameful (but often analytically useful) notions.
the notes, the researcher is also at liberty to record obvious errors.
As suggested in the first chapter of this essay, the role played by A GUIDE TO FIELDNOTE STYLE
stupid mistakes in the history of science is impressive; Agar ( 1983)
Fieldnotes, however unique the fieldworker, must conform. to
advocates an active search by qualitative researchers for "anti-
several minimal requirements. First, field entries must be legible
coherence ... These mistakes are of particular importance in the
and chronologically ordered. Second, entries must differentiate
study of groups initially unfamiliar to the researcher. As Mead
among categories of data (e.g., what people lite~ally say, the
(1973) remarks, "when the field worker arrives in his field, work
unobtrusively measured "context .. of social interactions, what the
begins immediately; there are first impressions that will not be
ethnographer preliminarily hypothesizes about the situation).
repeated and so must be recorded." The more the fieldworker
The general strategy for the recording of cul~urally ~eaningful
"goes native" by understanding and identifying with the target
utterances (i.e., the things that people say) is to build from a
group, the less accessible such "naive" impressions become, and
foundation of modem orthography, and to depart from general
the researcher, if he or she has failed to record them, may lose
editorial style as necessary to ensure faithfulness to the original
access to his or her own first impressions and responses.
comment. For example, capital letters and periods may be used
The ostensible purpose of fieldnote conventionalization, facil-
when complete sentences are recorded. Similarly, quotation
itating exchange of notes among colleagues, seems to be en-
marks including those for quotes within quotes, may also be
hanced rather than inhibited by the decreased laundering that
occurs. Not only is it always possible to withhold or censor field
emplo~ed. However, fieldnote punctuation should definitely
facilitate the recording of incomplete thoughts.
notes, but when the sharing of notes is reciprocal a highly
Figure 5.1 presents a basic inventory of diacritical conventions
cooperative, rather than adversarial, relationship among col-
for taking ethnographic fieldnotes.
leagues develops. Many of those currently experimenting with the
To illustrate the use of these conventions, Figure 5.2 displays a
exchange of field notes of the informal and uncensored variety
portion of field notes recorded by the authors in beginning stages
find that the prospect of sharing such confessional notes with
58 59

Sunday 24 June 1984 4:27 p.m.

of the study of water leisure. The fieldnotes refer to the cultural
WINDSURHNG CLUB scene at an elite club in Seattle.
inside:
walk in to "Surfin' USA"
"Daddy, you have to gob>' the hot tub to get to the deck!"
(not to mention equipment and fashion section, sauna, locker rooms, weight 6. ETHNOGRAPHIC DECISION MAKING:
room, deli, tanning statiom .•• ,) THE FOUR PHASES OF QUALITATIVE RESEARCH
video repeats Columbia Gorge windsurfing
Writing a communicable account of a complex event in a social
out.side: ~
now there arc three decks (seasonal fill·ins between slips). situation requires not only a high degree of literacy but at least
south (newest) deck: instruction. two traming units on deck some comprehension of the social science purposes for which the
middle deck: " ... now it's the restaurant. The new deck is for instruction ... record may have its uses. It is a task in which the beginner and
and the old deck is for members (to sunbathe)."
north deck: sunning on blue outdoor carpet (?)but no lounge chairs.
professional social scientist, if they were to compare notes, might
find that they have more in common than either would suppose.
9 people 2 toking lessons; 2 sunning not crowded
6 women 3 tables
The experienced observer encounters phenomena and deals with
3 men 2 employees(?) a social rhetoric that he or she aims to record in its natural
<short boom I ('regular boom')>
freshness-not as something to be assimilated immediately to a
theoretical system that usefully compares analogues but that, if
'What does this place resemble to you?'
''Well, you know, I wiv. raised in Europe . . Lake Geneva . ..."
imposed upon the situation of observation, may rob it of the
(he insist~ he approves of the "chaos" of Seattle's urban waterfront) opportunity to reveal something new. The task is not greatly
Beginners are on strings different for the beginner. Despite any social science theoretical
system to which be or she has been exposed, the novice has not
<duck tack/ helicopter tack>
<duck Jibe> had so many direct experiences with textbook analogues that they
threaten to intervene between his or her recording and his or her
1984 HWly 300
$600 observation or between his or her perception and the event. In
their place, the novice has the handicaps of at least his or her
by surfing standard~. even the statc-of-the·:irt [production boards I
(e.g., HiHy 320) arc heavy cultural training to overcome if the novice is to learn to do
fieldwork for a social science.
ATTENTION SEATTLE HARBOR PATROL REQUIRES ALL WINDSURI fRS TO
WEAR A HARNESS OR PERSONAL fl.OTATION DEVICcS
(posted: "itJUSt happened, it's the law" ... even for instructors • . .') For expert or student, therefore, the whole point of devoting time
to recording is not merely to make sure he will have materials
can't tell employees from the members/guests. employees may be younger.
down in black and white upon which to base his final report, but
harbor pohce. cruise .slowly 20' from the dock between 6 windsurfers. None o 1· w hom also to insure that he has the opportunity, while in the field or fresh
have fl otatton anythmg. no action. from it, to relate insightful experience to theoretical analysis,
Figure 5.2 Fieldnotes re: Lake Union Leisure percept to concept, back and forth, in a kind of weaving of the
fabric of knowledge [Junker, 1960: 13].
 62 63

rap her moves from the state of research preparation to the state of understood to be ex-Fools. If the fieldworker can provide data to
research analysis. the academic community suggesting he or she was once a Fool,
The four phases of qualitative research map easily to the the fieldworker improves his or her chances of making Sage.
fieldwork period. The phases directly correspond to problem For these reasons, the scenario of the noble fieldworker first
solving associated with ..finding," "working," "reading," and arriving on the scene and handily demonstrating ignorance to
"leaving" the field. local culture is universally acted out by ethnographers. The
offshoot of this is that all ethnographers (for their own protection
INVENTION: "GETIING IN" AND "GETTING ALONG" and aspiration) have a favorite and well-rehearsed "getting-in"
If there is one binding common experience to be found across war story.
more than one hundred years of ethnographic inquiry, it en- Generally speaking, the first several waves of ethnographers
compasses the moments of finding the field. Contemplation of who insisted, as a matter of professional birthright, on the
what it means to confront an exotic culture is a time-honored opportunity to conduct fieldwork reported little difficulty in
meditation that not only figures prominently in the recruitment of physically locating societies to study. Whether the cultures of
students to the study of human patterns but also inspires and concern were organized by Native Americans in the Pacific
consolidates the culture of field researchers. The great empathy Northwest, European immigrants in greater Chicago, island
ethnographers demonstrate for one another when the war stories peoples in Oceania, or colonized British subjects elsewhere, eager
of "getting in" and "getting along" are recounted underscores a fieldworkers established expectations that the major problems of
shared awareness of the personal tribulations, symptomatic locating a tribe would be logistical (i.e., not methodological or
vulnerability, and odd courage of the field researcher species. theoretical). The original algorithm for Finding the Field under
Unless the planned research is basically surreptitious in nature, early circumstances was something on the order of
at no time are ethnographers so aware of their innocence as at the
time of first contact. There are, no doubt, many examples of (I) pick a continent, or set of islands, or a suburb;
fieldworkers who never attained a status accorded by people (2) find someone connected in some way (e.g., social service,
ecclesiastic, commercial) to both the Western culture of orienta-
studied of anyone more complicated than a hopeless fool or a dull
tion and the people to be studied;
stranger. Nonetheless, fieldworkers in their own eyes grow wise
(3) accept a ride (or at least directions) in the field.
through fieldwork.
Fieldwork and intellectual progress are made most evident by Roughly speaking, Finding the Field starts when the ethno-
the staggering qualitative difference between what the ethno- grapher leaves his hometown and ends just before the ethno-
grapher knew at the outset of field research and what he or she grapher arranges the first systematic collection of data. Three
came to know later. In academia, the assertion by an ethno- aspects of this "Ethnography at the periphery" are defined by the
grapher that he or she qualifies as a Field Sage is not always securing of first directions, first views, and first conclusions.
followed quickly by community congratulations. However, when
one claims to have been a Field Fool, collegial annointment is Copping Directions. Copping Directions is the first aspect of
considerably less problematic. The ranks of Sage and Fool Finding the Field. It pertains to the moments of networking
depend on one another in the strategic sense that some Sages are across social systems to the express end of arriving at the culture
64 65

to be studied. It is the popular image of field research that principles of detached observation. Of course, history has shown
ethnographers, whether aided by supreme luck, guile, or money, this has not universally been the case.
gravitate effortlessly to the hubs of culture, action, and influence.
Somehow, the imperative "Take me to your leader" is thought to Copping a Taste. Copping a Taste is the third aspect of Finding
engender quick results, to be intrinsically meaningful, and to be the Field. This very American terminology for the moments of
an appeal worthy of serious attention by anyone in the world. first cultural readings suggestively hints at the elements of
This romantic script of how ethnographers establish contact surprise, invasion, strategy, pleasure, and excitement that char-
with societies they study ignores the role of intermediaries who acterize the motivation and response of both parties in the coming
arrange the Grand Entrance of the heroic field researcher. together of native and ethnographer. The period of Copping a
Intermediaries (e.g., missionaries, merchants, administrators) Taste consists of the early episodes of unobtrusive (or at least
operate at the fringes and borders of cultures and are among the "free-form'") assessment of host and guest cultures. It is a time for
first in any society to denounce, proselytize, and straddle cultural preconceptions to be shattered, unanticipated constraints to
systems. Often stigmatized as eccentric, deviant, and mission- emerge, and bases for bias to be identified.
oriented, the intermediaries are the prototypical marginal citi-
DISCOVERY: ..GETTING DATA..
zens. They are innovative in their interaction with air-dropped
field researchers for the basic motives of curiosity, profit, and, in The discovery phase of fieldwork is the ethnographic process of
some cases, altruism. collecting data. Working the Field roughly begins when the field
researcher instrumentally concerns him- or herself with identi-
Copping a Look. Copping a Look is the second aspect of fying a specific time and place to conduct an observation or
Finding the Field. It concerns the moments of first viewing a inquiry. The phase ends when the ethnographer has obtained an
culture. As simple as this experience sounds, it is a nagging appropriate quantity and quality of data over the course of
paradox of human studies that social scenes and cultural settings multiple exposures to, and interactions with, the people under
(to scramble Spradley and Lofland terminologies) evaporate as study.
the well-intentioned ethnographer comes up close. Amassing of data in the field is a ritualistic test of great
lt is a naive notion that sociological concepts such as culture, significance within the culture of qualitative social science-
society, community, and tribe cleanly map onto the legally ethnographic data in hand is worth twenty times that amount in
calculated physical territories of the world. Ironically, anthro- the bush. Once the field worker has command over the data, he or
pologists have substantiated this view of a Disney-partitioned she has jumped the midpoint hurdle in the research process. The
cultural universe with their remarkable record of locating "lost" fieldworker has faced the unknown native in his or her own
peoples. The ethnographic literature seems replete with expedi- language and on his or her own turf, and has secured data that has
tion sagas detailing the formidable obstructions presented by meaning for him or her.
jungle, mountain, and other nefarious no-man's-lands. Victori- Working the Field pertains to the settling down of the
ous over the elements, and first viewing the culture of study, the ethnographer to the actual business of recording information.
level-headed ethnographer is seen in the classic view to be Three aspects of Working the Field are defined by the discovery
unswayed by the seductions of sentiment and to be true to the of opportunities, data, and closure.
66 67

Scoring a Chance. Field research conducted without attention defined by the location of hypothesis-testing act1v1ty in the
to the native perception and local cultural context of ethnography discovery, rather than the interpretation, phase. Scoring the
is a contradiction in terms. Presumably. such abuses are indicated Facts begins with the formal effort to gather information,
in the varied ways ethnographers pursue and maintain kinds of inquiries. on-the-scene modification of procedure, and termi-
rapport. The first aspect of Working the Field concerns prepara- nates with data in hand.
tions and the securing of an opening for data collection. These
logistics and exercises in rapport are termed Scoring a Chance. Scoring-an Ending. The third aspect of Working the Field has
Preparation alludes to topics that include the strategic presenta- to do with the circumstances by which the process of Scoring the
tion oft he ethnographer's persona, the development of research Facts is terminated. Scoring an Ending pertains to the final
instruments, the location of key and other informants, and the moments of individual interviews, observation sessions, and the
coordination of a research situation (i.e., a task environment). like. Importantly. Scoring an Ending is not always desired by the
Success in this endeavor is no mean feat of cultural choreography. ethnographer. Endings of data gathering can be caused by acts of
God, inadvertent insult, depletion of funds, and social fatigue, as
Scoring the Facts. Scoring the Facts denotes the very essence of well as by attainment of research objectives. The reality of field
Working the Field -the hands-on real work of gathering ethno- research is that, once engaged, the two parties of the data-
graphic data. At first impressionistic glance, it would seem that collecting encounter cannot be expected to separate gratefully
the Scoring of Facts can be approached and conducted in a with equivalent satisfaction about the exchange.
straightforward manner. To this. two comments are in order. More often than not, field researchers have attended less to the
First, any possibility of a smoothly running fact collecting field winding down of fieldwork than they would care to admit.
operation is critically dependent on a plan of research action. It is Certainly, ethnographers exhibit concern when they are victims
unscientific as well as maddening to initiate data collection of breakdowns in Scoring the Facts (for example, scoring a
without a language (paradigm) that precisely contrasts data and brush-off; scoring a lie; scoring wrong data). But there are also
noise. The ethnographer who gathers without knowing what he lessons in understanding to externalities of "successful" data
or she wants (at the logical level) will find no happiness in the recovery efforts.
process.
Second, the Scoring of Facts in the field research application is INTERPRETATION: "GETTING IT STRAIGHT"
a multifaceted affair. Clearly, the focal activity must be the Reading the Fieldhas its locus in interpretive moments follow-
systematic (i.e., research design specified) amassing of informa- ing the discovery of ethnographic data. It is in this subphase that
tion. But, even as this goes on, the field research has the opportuni- the field researcher begins to ponder the validity, reliability, and
ty to consider tangential issues of any order. That is, the ethnogra- overall meaning of materials. It is also a time of readjusting
pher is, at one and the same time, Scoring the Facts and discover- rapport, recalibrating tools, and redesigning field strategems.
ing new kinds of facts (those not being collected). Reading the Field begins as the field researcher accumulates
Scoring the Facts primarily involves the gathering of data, but ethnographic evidence, continues through the (sometimes radi-
extends to considerable hypothesizing from that activity. Though cal) bargaining of research objectives, and winds down as the field
perhaps too glib, it is almost fair to say that qualitative research is researcher is wedded to a complete data set.
66 67

Scoring a Chance. Field research conducted without attention defined by the location of hypothesis-testing act1v1ty in the
to the native perception and local cultural context of ethnography discovery, rather than the interpretation, phase. Scoring the
is a contradiction in terms. Presumably. such abuses are indicated Facts begins with the formal effort to gather information,
in the varied ways ethnographers pursue and maintain kinds of inquiries. on-the-scene modification of procedure, and termi-
rapport. The first aspect of Working the Field concerns prepara- nates with data in hand.
tions and the securing of an opening for data collection. These
logistics and exercises in rapport are termed Scoring a Chance. Scoring-an Ending. The third aspect of Working the Field has
Preparation alludes to topics that include the strategic presenta- to do with the circumstances by which the process of Scoring the
tion oft he ethnographer's persona, the development of research Facts is terminated. Scoring an Ending pertains to the final
instruments, the location of key and other informants, and the moments of individual interviews, observation sessions, and the
coordination of a research situation (i.e., a task environment). like. Importantly. Scoring an Ending is not always desired by the
Success in this endeavor is no mean feat of cultural choreography. ethnographer. Endings of data gathering can be caused by acts of
God, inadvertent insult, depletion of funds, and social fatigue, as
Scoring the Facts. Scoring the Facts denotes the very essence of well as by attainment of research objectives. The reality of field
Working the Field -the hands-on real work of gathering ethno- research is that, once engaged, the two parties of the data-
graphic data. At first impressionistic glance, it would seem that collecting encounter cannot be expected to separate gratefully
the Scoring of Facts can be approached and conducted in a with equivalent satisfaction about the exchange.
straightforward manner. To this. two comments are in order. More often than not, field researchers have attended less to the
First, any possibility of a smoothly running fact collecting field winding down of fieldwork than they would care to admit.
operation is critically dependent on a plan of research action. It is Certainly, ethnographers exhibit concern when they are victims
unscientific as well as maddening to initiate data collection of breakdowns in Scoring the Facts (for example, scoring a
without a language (paradigm) that precisely contrasts data and brush-off; scoring a lie; scoring wrong data). But there are also
noise. The ethnographer who gathers without knowing what he lessons in understanding to externalities of "successful" data
or she wants (at the logical level) will find no happiness in the recovery efforts.
process.
Second, the Scoring of Facts in the field research application is INTERPRETATION: "GETTING IT STRAIGHT"
a multifaceted affair. Clearly, the focal activity must be the Reading the Fieldhas its locus in interpretive moments follow-
systematic (i.e., research design specified) amassing of informa- ing the discovery of ethnographic data. It is in this subphase that
tion. But, even as this goes on, the field research has the opportuni- the field researcher begins to ponder the validity, reliability, and
ty to consider tangential issues of any order. That is, the ethnogra- overall meaning of materials. It is also a time of readjusting
pher is, at one and the same time, Scoring the Facts and discover- rapport, recalibrating tools, and redesigning field strategems.
ing new kinds of facts (those not being collected). Reading the Field begins as the field researcher accumulates
Scoring the Facts primarily involves the gathering of data, but ethnographic evidence, continues through the (sometimes radi-
extends to considerable hypothesizing from that activity. Though cal) bargaining of research objectives, and winds down as the field
perhaps too glib, it is almost fair to say that qualitative research is researcher is wedded to a complete data set.
68 69

A distinguishing and favorite feature of the ethnographic Checking the Validity. The first aspect of Reading the Field is
process commonly cited by practitioners is that the ethnographer labeled, "Checking the Validity." This is primarily an evaluation
finds him- or herself in a position to chronicle cultural per- of Working the Field. As such, it calls for a consideration of the
formances primarily as a consequence of remaining on the scene components of the research situation (place, time, informant) and
long enough to witness the full cycles of cultural routines, as well the research problem and tools. At issue is the validity of
as long enough to dispel native anxieties concerning the fate of observations (i.e., whether or not the researcher is calling what is
collected information. What this means is that the field researcher measured by the right name).
is exposed to generically similar human situations, roles, be-
haviors, beliefs, and so on from a variety of vantage points. This Checking the Reliability. The second aspect of Reading the
occurs with the interpretative assistance of informants and local Field involves checking the strength of the data. This is purely and
observers of the scenes. When things go smoothly, the ethno- simply the exercise of investigating the reliability of qualitative
grapher watches his or her confusion tum to tentative hypothesis. research. The issue is one of whether or not (or under what
He or she probably also notices that his or her hunches require conditions) the ethnographer would expect to obtain the same
reformulation, and his or her pet theories may crumble apart. funding if he or she tried again in the same way.
Such is the scientific endeavor.
The field researcher engaged in Reading the Field is struggling EXPLANATION: ..GETTING OUT" AND "GETTING EVEN"
to understand how the data he or she has amassed qualify as The explanation phase of fieldwork concerns the ethnographic
information (rather than noise), and how they are amenable to process of Leaving the Field. This process begins with the
analysis. When the process proceeds smoothly, there is a gradual realization by the ethnographer that an adequate qualitative data
verification of a hypothesized relation between the research base has been secured, and ends when the fieldworker returns
problem, tools, and data. The ethnographer becomes alert to the home.
reaction he or she has prompted, keen to the fact that ethnog- The field exit is much more than a matter of packing and
raphy is not a report on a people but a report on the encounter perfunctory good-byes. It is a phase of research in which foreign
between the researcher and the tribe. When things do not go and native parties to the ethnographic contract settle accounts (as
smoothly-when there is a breakdown-the ethnographer must well as establish groundrules for future communication and
adjust the problem, the tools, or his or her reading of the data. interaction). Thus the professional ethic requires attention to the
The work of Reading the Field correctly is a process part of the rights and obligations of "getting even" in the course of "getting
test tradition of science. There is always the chance that the out."
blunders, mistakes, and errors that constitute "getting it wrong" Leaving the Field is a phase based on closure and departure.
and that are so integral to "getting it right" may become the Three aspects of Leaving the Field are defined by separations
anomalies that herald Discovery. from relationships, costs and benefits, and the field.
We isolate two aspects of Reading the Field, both of which
incorporate the themes of reappraisal, iteration, and conver- Splitting-Up. The first aspect of Leaving the Field centers
gence. These concern questioning the meaning and strength of the around the severing of professional working relationships. It is at
facts. this time that the qualitative researcher disassociates him- or
72 73

and Robert Park demanded timeliness in reporting out of respect (analysis), or Explanation (documentation). The simple rules of
for the research contract. The "bad news" is that little progress the model (complete a phase before moving to the next phase,
was made over the same period in mediating the limitations of complete all phases) do not in themselves guarantee a respectable
qualitative methodologies vis-a-vis reliability. research project, but they do provide structure and direction
Qualitative researchers can no longer afford to beg the issue of pertinent to this objective. Knowing how to code the research
reliability. While the forte of field research will always lie in its activity (or subactivity) at hand, and knowing what other
capability to sort out the validity of propositions, its results will activities bracket it, alert the ethnographer to the phase require-
(reasonably) go ignored minus attention to reliability. For ments of science. Thinking about the sequencing of research in
reliability to be calculated, it is incumbent on the scientific terms of moving through phases (or shifting gears or operating in
investigator to document his or her procedure. This must be qualitatively different modes) is the necessary first step in
accomplished at such a level of abstraction that the loci of reporting about procedure- the topic of most discourse in
decisions internal to the research project are made apparent. The matters of reliability. The model of science is the only defense a
curious public (or peer reviewer or funding source) deserves to qualitative researcher needs.
know exactly how the qualitative researcher prepares him- or 1t is our argument that qualitative research can be performed as
herself for the endeavor, and how data is collected and analyzed. social science and can be evaluated in terms of objectivity. The
But the researcher also needs to be able to isolate the conditions fundamental gist of this book is that the problem of validity is
under which he or she best "goes to risk"-the time at which he or handJed by field research and the problem of reliability is handled
she is organized to learn something. by documented ethnographic decision making.
The solution proposed here to the scientific problem of talking
about reliability and the pragmatic problem of efficiently doing
science lies in the adoption of a language for coding the scientific
behavior of the researcher. Specifically, qualitative researchers
need to know where they are in the research process at different
points in time. The novelty of the field and the ambition of
researchers to understand the totality of social facts create a
"no-win" situation in which the fieldworker must resist the
temptation to study all things at once. In short, the qualitative
researcher must plan on asking him- or herself "where am I?" and
"when am I done?" many times. Not to do so is to risk the research
project, as well as the mental health of the researcher.
The four-phase model of the ethnographic process presented in
this chapter helps the qualitative researcher make decisions. In
the application to work in the field, the model sharply categorizes
activities as falling within the purview of either Invention
(research design), Discovery (data collection), Interpretation
 NOTES

I. Mosl such errors arc "hermeneutic," in the sense that they represent misunder·
standings of the relationships of parts to wholes.
2. Following lbe oral tradition ofscience. we regard "data" as a mass noun like "love"
or ..jewelry," and assign it a singular verb.
3. Sec Tukey (1977) and Hartwig and Dearing (1979) for a discussion of a recent
trend by statisticians to construct quantitative techniques more appropriate to qualitative
research.
4. For well-known introductions to these concepts in the social science and
part1cipant observation applications, see. for example, Cronbach and Meehl (1955),
Selltiz et al. (1963). Webb et al (1966), Sjoberg and Nett (1968), and Becker (1970).
S. Beanng in mind that the term "measurement" in many contexts implies the
assessment of degree-that is, nonqualitauve observation- it will be convenient here to
regard qualitative observations as a special case of measurement.
6. The terms reliability and validity are reminiscent of the physical-i;cience notions of
precision and accuracy. ln physics. "precision" refers to a feature of reporting a
meuurement- roughly speaking, to the amount of accuracy being claimed. Spurious
precision consists of reporting a measurement in such detail that it has neither reliability
nor validity. To avoid spurious precision, one restricts the report to the level of specificity
at which an accurate statement can be made. Thus both reliability and validity are
subsumed under the concept of accuracy.
7. According to Bonjean ct al. (1967), Srole's five-question Anomia scale, with 28
independent citations, was the seventh moi;t frequently used measure in liOciology during
the period 1954-1965.
8. "In making use of lhe term 'invidious,' it may perhaps be unnecessary to remark,
there is no intention 10 exlol or depreciare, or to commend or deplore any of the
phenomena which the work is used 10 characterize. The term is used in a technical sense"
(Veblen, 1931 : 34).
9. Spradley (1979) used the term "translation competence" lo refer to the ethno-
graphically undesirable tendency of informants to provide prepackaged, partyline, and
extra-emic answers to questions.
10. Historical treatments of anthropological fieldwork are found in Penniman ( 1974).
Hodgen (1964), and Stocking (1983)
11 . A member of the research team, Francois Peron, was perhaps the first paid
ethnographic fieldworker. Unfonunately, Peron ignored mu~h of his assigned rask.
largely because he was a "self-appointed spy" intent on studying British settlements in the
spirit of French colonial expansion (Moore, 1969).
12. At Oxford. Tylor was the first Instructor in Anrhropology (1883) and first
Profes~or and Lecturer of Anthropology (31 December 1898) in the Brirish Isles. For

75
76 77

reports on early worldwide distributions of the anthropological teaching force, see Doney whereas the one who intenicw~ and identifies Y.ith the powerful will discover that the
(1894)and MacCurdy(l899, 1902). untrammeled exercise of po\\cr is only a myth .
13. No\\ here is Tylor cited as actually having conducted fieldwork; however, the 23. Certain anthropologists (e.g., Brady, 1983) have takC'n F rccman to be challenging
circumstances of poor health and a wealthy family resulted ma restorative trip to the New the competence and integrity of one of their culture heroes and favorite people, and have
World and a fit st-hand appreciation of Mexican and tropical cultures. Harris's ( 1968) entirely dismissed hb worl as crude and intended only to ~hock . This does not appear to
opinion is that Morgan's worl.. "would not be considered a bonafide field expencncc by ha\C been his intent. Modem ethnography was in its very beginnings at the time of Mead's
modern standards, '>ince it did not involve continuou) or prolonged contact with the daily research, and to criticize her findings out of historical context would be equivalent to
routanc of a given local community." critici1J ng Edison's onginal light bulb for burning out after a few seconds.
14. Freilich ( 1977) prefers to say anthropology grew "from 'infancy' to 'childhood'" 24. Bridgman (1927) 1~ one ~tatcment of the position most vulnerable to this kind of
during the period 1860-1900. Kluckhohn (1949: 4)contends that "It would be going too far error.
to call the nineteenth-century anthropology 'the investigation of oddments by the 25. This extreme is approached, for example, by Mchan and Wood (1976).
eccentric.' " 26. Friedman (1967) doubts even this m ~oci al psychology.
15. We have done disservice m this section to Adolf Bastian (1826-1905). Bastian 27. These terms, Invention, Discovery, Interpretation, and Explanation, are used as
tra\elcd the globe as a ship'ssurgeon in the mid-l800s; he returned to Berlin to publish his technical term~ in the ways defined here. In order to avoid proliferation of jargon, rather
ethnographic findings, became Curator of Ethnography, founded the Konigliches more vulgar labels have been used for activittes subsumed under these majo r phases.
Museum fur Volkerkunde, helped organize the Berlin Society for Anthropology,
Ethnography, and Prehistory, and coedited the journal Zeitschri/tfur Ethnologie. Lowic
( 1937) devote~ an entire chapter to Bastian, who must have invented the image of
anthropologists •~connoisseurs of foreign cultures while terrorizing in their own.
16 Boas's anthropological research among the Baffin Island Eskimo( 1883-1884) and
Pacific Northwest peoples (Kwakiutl work beginning 1886; the Jesup Nonh Pacific
expedition 1897-1902) ranks among the earliest fieldwork on record . Other nineteenth·
century fieldworkers include Karl von den Steinen (Brazilian expeditions in 1884 and
18~7J : the Torres Strait expedition team (A. C. Haddon and associates, 1888 and 1889),
and certainly Sir Walter Baldwin Spencer and Frank J . Gillen. By 1899, these last two
researchers had spent more than two decades among the Austrahan Arunta with the
con)equencc that ~both of us are regarded as full>• initiated members of the same tribe..
(1904 : x).
17. That, as will be shown, the discipline of sociology has contributed its full share to
the technology officldwork coun~ for little in the common as~umption of both dbciplincs
that cultural anthropologists arc somehow the custodians of qaalitauvc methods.
18. Descnptaons of the Chicago School as fortified by Park arc found in Stein ( 1960),
Madge (1962), and Faris ( 1967).
19. The rise of the Chicago School wcu greatly facilitated by the fact that many citiLens
of Chicago wanted their city understood and were willing to underwrite social research.
20. The terms "participant observation" and "objective observation" are attributed to
Lindeman ( 1924). Interestingly, Anderson ( 1961) repons that he was unfamiliar with these
labels at the time he conducted fieldwork for The Hobo. The two men later collaborated
(Anderson and Lindeman, 1930).
21 . This terminology arises from a Jorge Luis Borges ( 1964) account of an author who
labored to compose the Quixote in precisely the same words as Miguel de Cervantes,
22. Perhaps the best resolution of the original debate is that power is something
experienced from the bottom, not from the top. It is a common experience to have one's
whim frustrated by some per~on or agency with the power to do so, but a rare experience
indeed to be able to impose one's whim on others. Power is indeed exercised, but those
who e.xerci~e it rarely have the choice about whether or how to do so, for the power only
exists by virtue of accommodating those other interests that suppon it. Naturally, the
investigator who intenicws and identifies with the powerless will sec power in action,
 GLOSSARY

APPARE!'T VALIDITY:"facc Validity"- Theobviousness of the relationship between

an observational procedure and what it 1s mtended to observe. (Chapter 3)
CHECKl:-.IG THE VALIDITY: An a!>pect of INTERPRETATION. At issue is whether
or not Lhe researcher is calling things by their right name!>c. (Chapter 6)
CHECKING THE RELIABILITY: An aspect of INTERPRETATION. At issue is under
"hat condition~ r~ults might be replicated . (Chapter 6)
COPP11'iG DIRECTIO~S An aspect of INVENTION. At is~ue is the problem of
amvmg at the field. (Chapter 6)
COPPING ATASTE; At issue is the preliminary and "free-form" exchange between the
observer and observed. (Chapter 6)
COPPI~G A LOOK: An aspect of INVENTION. At issue is the fir~t exposure to a studied
culture. (Chapter 6)
DIACHRO~IC RELIABILITY: ''flltemal Rehab1lity"- The extent 10 which the same
ob:.cenation made at different limes yields the same information. (Chapter 5)
DISCOVERY: "Data Collection"; "Worlung the field"- The second, ..field phase" of
qualitative research that diStinguishes it from nonqualitative research. (Chapter 6)
F.X Pl.A~ATlO'S': "Report writing"; "Leaving the field"- The fourth and final phase of
cientific research, beginning after data has been collected and analyzed; presentation
of a thematic point to a particular audience. (Chapter 6)
FIELD :\'OTES. Timely and verbatim records kept by a field researcher, may include
diary-like material and tentative interpretation. (Chapter 5)
11"STRUME:\IAL VALlDITY: "Criterion Validity"- The correspondence between an
observauon and a different and accepted observation of the ~amc thing. (Chapter 3)
11'TERPRETATION: ..Analysis"; ..Reading the field"- The third phase of scientific
research, beginning after an appropriate amount of data has been collected from a
particular time and place; organization and summary of data, drawing conclusions
from the data. {Chapter 6)
INVENTIO};: "Researeh design"; "Finding the field"- The first phase of scientific
research, comprising all the activity leading up to the collection of data. (Chapter 6)
OBJECTIVITY A commitment to integrating new findings into the cumulative body of
collective knowledge and confronting ideas with data as well as argument. (Chapter I)
PHENOMENOLOGY: Emphasis on the process of observation, sometimes to the
exclus10n of concern for external reality. (Chapter I)
POSrT IVlSM : Emphasis on external reality, sometimes to the exclusion of concern for
the process of observation. (Chapter 2)
PRAGMATISM: Emphasis on a continuing concern with the ob~erver, the ob~erved , and
the activity of observation; manife~t m social science under the name ..symbolic
interactionism. ~(Chapter I)

79
80

QUALITATIVE RESEARCH: I. Research involving counting. 2. Research not m-

voh1ng counting J . Observation and interaction with the target of study on its own
home ground : ethnography, fieldwork, naturalistic research, panicipant observation
(Chapter I)
QUIXOTIC RELlABIUTY: Multiple observations yielding identical information; m
field research often a signal of problems in ,·aJictity. (Chapter 5)
RELIABILll Y: The extent to '1.hich the same observational procedure in the same
context yields the same information; for the implications of the term Msame," see REFERENCES
diachronic, qu1xouc, synchronic reliability (Chapter 5)
SCORING A CHA~CE: An ~pect of DISCOVERY. At issue is the securing of the ADAMS, R . :-;. and J . J. PREISS (eds.] ( 1960) Human Organization Research
situation to collect data. (Chapter 6) Homewood, IL. Dorsey.
SCORING AN ENDING. An aspect of DISCOVERY. Atissuebknowtngwhen and how AGAR, M. H. ( 1983) .. Ethnographic e\idencc." Urban life 12: 32-48.
to ~top collecting data (Chapter 6) {1982) "Toward an ethnographic language." Amencan Anthropologist 84:
SCORING THE FACTS: fhecentral aspect of DISCOVERY. At issue is the gathering of 179-195.
data. (Chapter 6) - - -{ 1980) The Profess1onal Stranger. New York: Academic.
SPLITTING THE SCENE: An aspect of EXPLANATION. At issue is the con~cquence ANDERSO~ . J . R . (1980) Cognitive P~ychology and Its Implications. San Francisco:
of the ob~erver leaving the field . (Chapter 6) Freeman.
SPLITTING 'I HE TAKE: An aspect of EXPLANATION. At issue is the dividing of ANDERSON.~- (1961) The Hobo. Chicago: Chicago (orig. pub 1923) .
rc~i:arch co5ts and bcnclits between the observer and the observed . (Chapter 6) - -and E. C. LINDEMAN (1930) Urban Sociology. New York: Knopf.
SPl.ll"l ING-UP; An aspect of EXPLANATION. At issue is the stopping of data BADASH. L. (1965) "Chance favors the prepared mind: Henn Becquerel and the
collection activities. (Chapter 6) discovery of radioactivity." Archives of the International Historical Society 18: 55-66.
SYNCHRO?llIC RELIABILITY: "Internal Reliability"' The extent to which two BECKER, H. S. (1970) Sociological Work. Chicago: Aldme.
simultaneous ob!>Crvations, or two observations of an unchanging target, )ield the BERREMAN, G. D. (1966) "Anemic and emetic analysis in social anthro pology."
'amc information. (Chapter 5) Amcncan Anthropologist 68: 346-354
THEORE1 JC Al. VALIDITY: "Construct Validity..- The quality of the relationship BEVERIDGE, W.I.B. (1950) The Art of Sc1enufic Investigation. !lie\\ York: W. W.
between an observation and the clement of a model that represents i1. (Chapter 3) >'onon.
VALIDITY: The quality of fit between an observation and the basis on which it is BLACK. M. and D . METZGER(l965)"Ethnographicdescnption and the study of law,"
made-see apparent. instrumental, theoretical validity, often the issue b the naming of in L. Nader (ed.) The Ethnography of Law. American Anthropolo~t 67, 6 (Special
variable~. (Chapter 3) Publication. Pan 2): 141-165
BLUMER. H. (1968) Symbolic lnteractionism ~cw York: Prentice-Hall.
BLUMER. M. (1983) MThe society for social research," in J. Thomas (ed.) The Chicago
School. Urban Life 11, 4 (S~iaJ Issue): 421-439.
BONJEAN. C. M., R . J. HILL. and S. D. McLEMORE (1967) Sociological Measure-
ment. San Francisco: Chandler.
BORG ES, J L. ( 1964rPierrc Menard, author of the Quixott," in Labyrinths. New York:
1'ew Directions 36-44.
BOWEl'i, E. S. [Bohannan. L.J (1954) Return to Laughter. New York· Harper.
BRADY, I. {cd.](1983)"Spea.kinginthe name of the real: Freeman and Mead on Samoa."
American Anthropologist 85. 4 (Special Scctton): 908-944 .
BRIDGMAN, P. W. (1927) The Logic of Modern Physics. New York: Macmillan.
BRIM, J . A. and D. H. SPAIN (1974) Research Design in Anthropology. New York:
Holl Rinehart & Winston.
CARROLL, L. (1960) The Annotated Alice. M. Gardner (ed.) New York: Clarkson N.
Potter. (Carroll's book pub. originally in 1865)
CLARK . T. N. (1971) Community Politics. New York: Free Press.
COLEMAN. J . S. (1973) The Mathematics of Collective Action, Chicago: Aldine

81
84 85

MOORE, F.C. T. ( 1969) "Translator·s introduction," in J . Degerando, The Observation of STOCKING, G. W.• Jr. ( 1983) Ob)ervers Ob)crved. Madison· Wi~consin University
Savage Peoples. Berkeley: University of California Press. Pre~~ .
MURDOCK, G. P. ( 1967) Ethnographic Atlas. Pittsburgh: University of Pittsburgh TAX. S . ( 1955) "From Lafitan to Radcliffe-Brown: a )hort history or the study of social
Press. organization," in F. Eggen (ed .) Social Anthropology of 1\'orth American Tribes.
--( 1949) Social Structure. New York: Macmillan. Chicago: Chicago University Pres~ .
NUNNALLY, J. (1959) Tests and Measurements. New York: McGraw-Hill TUKEY, J. W. (1977). Exploratory Data Analysis. Reading, MA: Addison-Wesley.
PALMER, V. (1928) Field Studies in Sociology. Chicago: University of Chicago Press. VAN MAAr-.;E~, J . (1982) .. Fieldwork on the beat," in J. Van Maanen ct al. (eds.)
PELTO, P. J . and G. H. PELTO (1978) Anthropological Research. New York· Varieties of Qualitative Re!>tarch. Bevcrl}' Hill~. CA: Sage. 103-151.
Cambridge University Press. -{ed.) ( 1979) Qualitative Methodology. Adminbtrativc Science Quarterly 24
PE1':'1.'IMAN, T. K. (1974) A Hundred Years of Anthropology. New York: Morrow. (Special Issue): 4.
PENZIAS, A. A. and R W, WILSON (1965) "A measurement of excess antenna VEBLEN. T. B. (1931) The Theory of The Leisure Class. New York· Modem Library.
temperature at 4080 Mc/ s." Astrophyi;ical Journal 142: 419-421 . (orig. pub. 1899) .
PIAGET, J . ( 1954) IheCon~truction or Reality m the Child. (M. Cook, tram.) New York: WARNER. W. Land P. S. LUNT(l941)TheSocial Lireofa ModernCommuruty. New
Basic. Haven, CT; Yale Umver~ily Press.
POLS BY, N. W. (1980) Community Power and Political Theory. New Haven, CT; Yale. WEBB, E. J. , D. T. CAMPBELL, R. 0 . SCHWARTZ. and L. SECHREST (1966)
POPPER, K. R. ( 1959) The Logic or Scientific Discovery. New York: Basic. Unobtrusive Measures. Chicago: Rand McNally.
POWDERMAKER, H. (1966) Stranger and Friend. New York: W.W. Norton. WHITING. J .W.M. et al. (1966) Field Guide for a Study of Socialization. Six Cultures
RABINOW, P. (1977) Reflections on Fieldwork in Morocco. Berkeley: University of Series, Vol. I. New York. Wiley.
Caltfornia Prc~s WHYTE, W. F. (1955) Street Corner Society Chica1w Chicago University Press.
RAif FA, H. (1968) Decision Analysis. Reading, MA: Addison-Wesley.
ROSS, L., G. BIERBRAUER, and S . HOFFMAN (1976) "The role of attribution
processes in conformity and dissent." American Psychologist 31 : 148-157.
Royal Anthropological ln~titute (1951) Notes and Quenes on Anthropology. London :
Routledge & Kegan Paul (6th edition).
- --( 1929) Notes and Queries on Anthropology London: Author (5th edition).
- (1874) Notes and Qucnes on Anthropology. London: Bntish Association for the
Advancement of Science. (ht edition)
SARKAR,!'\. K. and S . J. TAMBIAH (1957) The Disintegrating Village. Colombo, Sri
Lanka: Ceylon University Socio-Economic Survey or Pata Dumbara.
SEGALL. M. H., D. T. CAMPBEi.i., and M. H. HERSKOVITS (1966) The Influence of
Cuhurc on Visual Perception. Indianapolis: Bobbs-Merrill.
SHLTIZ., C .• M JAHODA, M. DEUTSCH, and S . W. COOK (1963) Research
Methods m Social Relations. New York: Holt, Rinehart & Winston.
SISTRUNK, F. and J . W. McDAVID (1971) ..Sex variable in conforming behavior."
Journal of Pcn.onahty and Soda! Psychology 17: 200-207.
SJOBERG, G. and R. f';E:.Tl (1968) A Methodology for Social Research. New York:
Harper & Row.
SMALL, A W, (1916) "Fifty years or sociology in the United State~." American Journal
or Sociology 21, 6: 721-864.
SONTAG. S. (1963) "A hero of our time." The New York Review of Books I, 7: 6-8 .
SPENCER, W. B. and F. J . GILLEN {1904) The Northern Tnbes of Central Australia.
New York: Macmillan .
SPRADLEY, J . P. ( 1979) The Ethnographic Interview. New York: Holt, Rinehart &
Winston.
SR OLE, L. 0956) ''Social integration and certain corollaries," ASR 21 : 709-716.
STEIN, M. R. C1960) The Ed1pse cf Community. Pnnceton, l'\J : Pnnceton Unl\ers1ty
Press.
 ABOUT THE AUTHORS

JEROME KIRK is Associate Professor of Comparative Soci-

ology and Urban Anthropology at the University of California,
Irvine. He received a B.A. in mathematics from Reed College,
and a Ph.D. in sociology from The Johns Hopkins University. In
pursuit of his interest in collective innovation and social change,
he has conducted field research in Polynesia and South America
as well as in a wide variety of North American sites. His articles
have been published in such journals as American Anthro-
pologist. American Journal of Sociology. American Political
Science Review, American Sociological Review, Anthropologi-
cal Linguistics, Public Opinion Quarterly, and Social Forces.

MARC L. MILLER is Assistant Professor of Marine Studies and

Adjunct Assistant Professor of Anthropology at the University of
Washington. He received his Ph.D. from the University of
California, Irvine, in 1974. He specializes in the sociology of work
and occupations, formal organizations, recreation, leisure, and
tourism, and has conducted field research in Mexico, South
America, and on the East and West coasts of the United States.
He has published in Coastal Zone Management Journal, Human
Organization, New Scholar, Urban Life, Ocean Development
and International Law, Social Networks, Work and Occupa-
tions, and Environment and Behavior, among other professional
journals.

87
 ~ARVGROUC COLLE.GE LIBQA!IV
Rel1ab1l1ty and val1d1ty 1n qu
306.01 K63
I II l l lllllllll lllll llllll llll llllll lll llllll lllllllllllll
3 1 92 7 0 0 01 8 41 9 9

DATE DUE

6.01 K63
K rk, Jerome.
ability and validity ln
alita ve research

04YL.O•o _.,UNYCO•Hv.,,,.,