Nothing Special   »   [go: up one dir, main page]

Teamwork in Multi-Person Systems: A Review and Analysis

Download as pdf or txt
Download as pdf or txt
You are on page 1of 24

ERGONOMICS, 2000, VOL. 43, NO.

8, 1052± 1075

Teamwork in multi-person systems: a review and analysis

CAROL R. PARIS² , EDUARDO SALAS³ * and JANIS A. CANNON -BOWERS²


² Naval Air Warfare Center Training Systems Division, Code 4961 12350
Research Parkway, Orlando, FL 32826-3224 , USA
³ Department of Psychology, University of Central Florida, PO Box 161390,
Orlando, FL 32816-1350 , USA

Keywords: Teamwork; Team performance; Team training; Team eVectiveness;


Teams.

As the scope and complexity of modern task demands exceed the capability of
individuals to perform, teams are emerging to shoulder the burgeoning
requirements. Accordingly, researchers have striven to understand and enhance
human performance in team settings. The purpose of this review is to summarize
that research, from the theoretical underpinnings that drive it, to the
identi® cation of team-level elements of success, to the methodologies and
instruments that capture and measure those characteristics. Further speci® ed are
three important avenues to creating successful teams: team selection, task design
and team training. In other words, one can select the right people, provide them
with a task engineered for superior performance and train them in the appropriate
skills to accomplish that task. Under task design, new technologies and
automation are examined that both support and impede team functioning.
Finally, throughout are provided critical remarks about what is known about
teamwork and what is needed to be known to move the science and practice of
team performance forward. The paper concludes with the identi® cation of team
issues that require further investigation.

1. Introduction . What are teams? What is teamwork?


Transforming teams of experts into expert teams necessarily begins with an
understanding of what characteristics uniquely de® ne the team and set it apart from
small groups. Teams are more than collections of individuals and teamwork is more
than the aggregate of their individual behaviours. Moreover, one cannot simply label
a group of individuals a `team’ and expect that they will perform as a team (Bass
1980). Speci® cally, one may conceive a team to be a `distinguishabl e set of two or
more people who interact dynamically, interdependently, and adaptively toward a
common and valued goal/objective/mission, who have each been assigned speci® c
roles or functions to perform, and who have a limited life-span membership’ (Salas et
al. 1992: 4). Characteristics that distinguish teams from small groups include the
following: multiple sources of information, task interdependencies, coordination
among members, common and valued goals, specialized member roles and
responsibilities, task-relevant knowledge, intensive communication, and adaptive

*Author for correspondence; e-mail: esalas@pegasus.cc.ucf.edu

Ergonomics ISSN 0014-013 9 print/ISSN 1366-584 7 online Ó 2000 Taylor & Francis Ltd
http://www.tandf.co.uk/journals
Teamwork in multi-person systems 1053

strategies to help respond to change (Dyer 1984, Modrick 1986, Morgan et al. 1986,
Salas and Cannon-Bowers 1997). To understand eVective team performance or
`teamwork’, one must understand how groups of individuals function to produce
eVectual synchronized output, rather than just summed or aggregated responses
(Steiner 1972, Hackman and Morris 1975, Nieva et al. 1978, Meister 1985,
Fleishman and Zaccaro 1992). But, what precisely is `teamwork’ ?

2. Theoretical developments
The ® rst serious attempts to study team processes began in the 1950s and 1960s, with
a focus largely on military teams and team processes that enabled them to function
more eVectively under conditions of extreme time pressure, high stress, ambiguous
and incomplete information, and severe consequences for actions taken. Much of the
impetus for team research over the years has been tied to team failures, particularly
those associated with high visibility (e.g. aircraft accidents, military accidents) (Ilgen
1999).

2.1. Evolving perspectives


The last half-century has produced theories that encompass many diVerent
perspectives. Although team theories began as descriptive eVorts, many have
evolved over time to provide more normative guidance for improving teams. Most
theories have incorporated a general input-process-outpu t approach, whereby
certain variables are fed into the system (e.g. environmental and organizational
variables, individual input variables, and team input variables), team processes
follow (e.g. orientation, communication, adaptation, etc.), and team output
(productivity) results. Some models incorporate dynamic change through feedback
loops, emanating primarily from the output side and feeding back to the input or
process components (Ilgen 1999).
Although detailed descriptions of the many theories, models, and taxonomies are
beyond the scope of this eVort, representative theories contributing to our
understanding of the teamwork fall into eight primary categories. These are
captured in table 1.

2.2. Validation of competencies


In addition to developing theories and models, team researchers have struggled to
identify those critical skills or traits that enable teams to coordinate, communicate,
strategize, adapt, and synchronize task relevant information so that they can ful® l
their goals and missions. In the 1970s, research focused on orientation, resource
distribution, timing, response coordination, motivation (Nivea et al. 1978), and team
morale (RuVell-Smith 1979).
In the 1980s, emphasis was given to collective self-eYcacy (Bandura 1986),
implicit or explicit coordination activities (Kleinman and Serfaty 1989), and
providing tasks and motivational reinforcement (Oser et al. 1989). Morgan et al.
(1986) highlighted seven speci® c skill dimensions Ð giving suggestions or criticisms,
cooperation, communication, team spirit and morale, adaptability, coordination,
acceptance of suggestions or criticism.
Finally, in the 1990s, the most signi® cant teamwork competencies were thought
to be mutual performance monitoring (Hackman 1990), belief in the importance of
teamwork (Gregorich et al. 1990), collective orientation (Driskell and Salas 1992),
adapting to novel and unpredictable situations (Prince and Salas 1993), exhibiting
1054 C. R. Paris et al.

Table 1. Theoretical contributions to the teamwork concept.


Representative theories

Social psychological approach: pertaining to the social and psychological implications of team
members’ relationships/interactions with one another
Interaction Processes (Hackman and Morris 1975)
Normative Model of Group EVectiveness (Hackman 1983, 1987)

Sociotechnical approach: concerning the technical and work-related implications of team


members’ relationships and interactions with one another
Dynamic Interactions (Kolodny and Kiggundu 1980)
Self-Regulating Work Groups (Pearce and Ravlin 1987)

Ecological approach: relating to the spacing and interdependence of team members and
institutions, or the relationships of team members with their organizational or working
environment
Group and Organizational Boundaries (Sundstrom and Altman 1989, Sundstrom et al.
1990)

Human resource approach: focusing on the utilization of human capabilities and talents
Human Resource Management (Shea and Guzzo 1987)

Technological approach: relating to industry or applied science, or to technological progress


Technological and Organizational Variables (Goodman et al. 1987)

Lifecycle approach: pertaining to changes within a team that result from its maturation or
evolution over a lifecycle
Time and Transition Model (Gersick 1985, 1988)
Team Evolution and Maturation Model (TEAM) (Morgan et al. 1986, Morgan et al. 1994)

Functional (taxonomic) or task-oriented approach: concerning team roles, functions, or


tasking
Team Performance Model (Nieva et al. 1978; others = Dickinson 1969, Dieterly 1988,
Fleishman and Zaccaro 1992, Naylor and Dickinson 1969, Shi¯ ett et al. 1982

Integrative approach: inclusive of multiple approaches or models


Composition, Structure, Resources and Process, or Task Group EVectiveness Model
(Gladstein 1984)
Input, Throughput, Output or Team EVectiveness Model (TEM) (Tannenbaum et al. 1992)
Work Team Design (Campion et al. 1993, Campion et al. 1996)

¯ exibility (Prince and Salas 1993), potency (Guzzo et al. 1994), cohesion (Mullen and
Copper 1994), performing self-correction (McIntyre and Salas 1995), using closed-
loop communication (McIntyre and Salas 1995), exhibiting assertiveness (Smith-
Jentsch et al. 1996b), predicting each others’ behaviour (Volpe et al. 1996), and four
speci® c skill dimensions (information exchange, communication, supporting
behaviours, and team initiative/leadership) (Smith-Jentsch et al. 1998a,b). Recently,
Cannon-Bowers et al. (1995) and Salas and Cannon-Bowers (2000) summarized
teamwork dimensions into three primary categories: cognitions, skills and attitudes.
Cognitions (or knowledge) include cue strategy associations, task speci® c team-mate
characteristics, shared task models, team mission, objectives, norms, and resources,
task sequencing, accurate task models and problem models, team role interaction
patterns, teamwork skills, boundary spanning roles, and team orientation.
Behaviours (or skills) consist of adaptability, shared situational awareness, mutual
Teamwork in multi-person systems 1055

performance monitoring, motivating team members/team leadership, mission


analysis, communication, decision-making, assertiveness, interpersonal coordina-
tion, and con¯ ict resolution. Last, attitudes embody motivation, collective eYcacy/
potency, shared vision, team cohesion, mutual trust, collective orientation and
importance of teamwork.
Although to date, much of the theoretical development and research has rested
largely within the central process component of team models, a new trend is
emerging. One is witnessing an increasing interest in issues facing organizational
teams, and with that interest comes a shift in research and theoretical development to
the inputs and outputs that bound, constrain, and impact team processes within
organizations (Ilgen 1999).

2.3. Hallmark of the nineties: the shared mental model


Team research in the 1990s has also evolved to a great extent around another
theoretical development Ð one which constitutes a signi® cant and unifying thread
underlying much of the current work in this ® eld. This is the concept of the shared
mental model (Cannon-Bowers et al. 1993). Mental models are knowledge
structures, cognitive representations or mechanisms (e.g. mental simulations) which
humans use to organize new information, to describe, explain and predict events
(Rouse and Morris 1986), as well as to guide their interaction with others
(Rumelhart and Ortony 1977, Gentner and Stevens 1983). A team mental model
would re¯ ect the dependencies or interrelationships between team objectives, team
mechanisms, temporal patterns of activity, individual roles, individual functions, and
relationships among individuals. Shared mental models allow team members to
implicitly and more eVectively coordinate their behaviours, i.e. they are better able to
recognize the individual responsibilities and information needs of team-mates,
monitor their activities, diagnose de® ciencies, and provide support, guidance, and
information as needed (Orasanu 1990, Entin et al. 1994, Duncan et al. 1996). More
about mental models will be detailed below.
Because theoretical foundations ground our understanding of team variables and
processes, they enable us to target critical skills to be measured. This is a signi® cant
® rst step in improving team performance. Theories of team performance abound in
the literature. And our understanding of what variables aVect team performance has
greatly improved over the years. However, it is suggested that at this point in time
more theories are not necessarily better. That is, it is believed that there are probably
enough theories and models. The ® eld needs richer, deeper and better speci® ed
models (and validated!) of team performance to more precisely guide measurement
and training eVorts.

3. Measurement of team performance


Without the measurement of team performance, one could not ascertain the success
of any intervention to improve it, whether that intervention is training, automation,
or other. Team-level measurement, however, presents a signi® cant challenge.

3.1. The challenge


To assess teamwork skills, one must capture the dynamic, multilevel nature of
teamwork, a process which is neither simple nor straightforward (Salas and Cannon-
Bowers 1997). Teamwork (or process) skills are not readily quanti® able, as are team
inputs and outputs (Baker and Salas 1992). Moreover, team behaviours evolve over
1056 C. R. Paris et al.

the life-cycle of a team. Teamwork behaviours , resulting from team member


interactions, need to be diVerentiated from taskwork behaviours , which are the
position-speci® c behaviours of individual team members (Morgan et al. 1986, 1994).

3.2. Team task analysis: a prerequisite


Prerequisite to the measurement of team performance is the team task analysis. A
team task analysis provides information about team learning objectives and team
competencies needed. It identi® es the cues, events, actions, coordination demands,
and communication ¯ ows needed for eVective teamwork. It enables the under-
standing of the nature of task interdependency in teamwork and to distinguish
collective team tasks from individual tasks (Salas and Cannon-Bowers 2000).
However, not yet available is a sound, validated and systematic methodology for
analysing team tasks. Some work is now going on and still under development
(Annett 1997, Swezey et al. 1998).

3.3. Instrument criteria


To measure teamwork eVectively, the chosen instrument should be theoretically
based, psychometrically sound, and a practically useful indicator of teamwork
(Baker and Salas 1992). Team assessment tools should: (1) identify processes linked
to key team outcomes, (2) distinguish between individual and team level de® ciencies,
(3) describe interactions among team members so as to capture the moment-to-
moment changes that occur, (4) produce assessments that can be used to deliver
speci® c performance feedback, (5) produce evaluations that are reliable and
defensible and (6) support operational use. Metrics that ful® l these criteria will
assess, diagnose and remediate important skill de® ciencies. To review speci® c
principles guiding team performance measurement, see Baker and Salas (1992) and
Salas and Cannon-Bowers (2000).

3.4. Selecting the appropriate measures/methodologies


No single measure of performance will be appropriate for all purposes. The primary
types of measures are: (1) descriptive measures, which describe what is happening at
any given time and seek to document individual and team behaviours by highlighting
crucial points of interaction and moment-to-moment changes in team functioning;
(2) evaluative measures, which judge performance against identi® able standards and
serve to answer questions of eVectiveness; and (3) diagnostic measures, which seek to
identify the causes of behaviour and question how and why things occurred as they
did. Diagnostic measures contribute inputs to the feedback process necessary to
improve subsequent performance (Cannon-Bowers and Salas 1997).
The measures for assessing team skills generally fall into two categories:

· Team versus individual measures: individual (taskwork) measures assess the


knowledge/skills required to perform individual tasks and meet individual
responsibilities. Teamwork measures focus on coordination requirements
between team members, such as backup, mutual performance monitoring/
error correction, information ¯ ow, etc.
· Outcome versus process measures: although successful outcomes demonstrate
that training objectives have been met, a focus on how the task was
accomplished (i.e. process measurement) is important for diagnosing
performance problems that may someday obstruct or prevent desired
Teamwork in multi-person systems 1057

outcomes. Because ¯ awed processes occasionally result in successful


outcomes, it becomes necessary to measure both processes and outcomes if
one is to ensure consistently eVective performance. Process measures
generally depend on observation by skilled raters. They might include
analysis of team communications or information ¯ ow, analysis of interac-
tions or strategies employed by team members, interviews, ® eld observation
techniques, task analysis, modelling or computerized simulation of teams,
and mathematical indices of team processes (Dyer 1984, Meister 1985). When
outcome measures are used, they typically rely on expert opinion or
automated performance recording. They might include ratings of pro® ciency
with respect to timeliness (latency), accuracy/errors, number and type of
omitted or incomplete tasks, frequency counts, and measures of knowledge
(Dyer 1984). Processes and outcome measures complement one another, and
when provided at both the individual and team levels, provide the most
complete picture of team performance (table 2).

Measurement approaches for team evaluation range from developing/applying


critical events or event-based techniques to modelling human performance via expert
systems, neural networks, fuzzy sets, or mathematical models (e.g. Petri nets). Data
may be captured online by observers or via automated systems (the team outcome).
Instruments include rating or event-based scales, observational checklists, critical
incident analysis, communication analysis, employee surveys and debrie® ng
procedures. While progress has been made in designing measurement techniques
and tools, more work is needed. And maybe a shift in focus is more appropriate. One
needs to develop more dynamic measurement systems that allow for on-line
assessment of teamwork. The technology (e.g. human performance modelling) is
maturing, but more work is required. The current focus on checklists, while useful,
does not capture fully the dynamic nature of teamwork.

4. Approaches for enhancing teamwork: variables that shape our interventions


There are theoretical models to explain the teamwork process. One has a sense of the
most important teamwork competencies. Finally, there are measurement techniques
and tools to diagnose a team’ s de® ciencies. So how can one apply this knowledge to
enhance the way teams perform? The remaining discussion will focus on three
interventions Ð team selection, task design and training Ð that can be utilized to

Table 2. Taxonomy of human performance measures.


Individual Team

Process
· Cognitive processes · Information exchange
· Position-speci® c taskwork skills · Communication
· Supporting behavior

Outcome
· Team leadership

· Accuracy · Mission eVectiveness


· Latency · Aggregate latency and accuracy
From Canon-Bowes and Salas (1997), 56 Copyright 1997 by Lawrence Erlbaum
Associates. Adapted with permission.
1058 C. R. Paris et al.

increase a team’s functioning. Before one can plan methods for improving team
performance, however, one must ® rst evaluate those factors (input and process
variables) that impinge on it. Table 3, adapted from the taxonomies of Morgeson et
al. (1997) and Meister (1985), enumerates the most signi® cant of these.
Con® guration of table 3 factors serves to in¯ uence, positively or negatively, the
manner in which a team performs. With these determinants in mind, the ® rst
intervention Ð team selection Ð is presented.

4.1. First approach: team selection


One of the ® rst things to be done to ensure successful team performance is to select
the right team members. If stable individual characteristics associated with superior
abilities for team coordination and performance can be identi® ed, then steps to select
the right people can be made.

4.1.1. Individual traits: Traditional staYng processes, which normally determine


position requirements, recruit applicants, then assess and select those most quali® ed
to perform the job, are not entirely adequate for populating teams. Teams require
special staYng considerations. Initially, one would need to consider how staYng
requirements might vary according to team type and function. For example, consider

Table 3. Taxonomy of variables in¯ uencing team performance.


Applicable
Factor(s) Description Examples intervention(s)
Contextual Variables that pertain to the Culture Team selection
factors environment in which the Climate Task design
team activity is embedded Training/education Training
systems
Reward systems
Information systems
Structural Variables impinging primarily Physical environment Task design
factors from sources external to the Organizational Training
team, but may include some arrangements
internal to the team (e.g. Technological systems
team organization)
Team design Variables inherent to the team Work design Team selection
factors itself Task interdependence Task design
Team size/composition Training
Leadership
Process Variables inherent to the team Boundary management Team selection
factors itself and the way in which it Task cohesion Task design
functions Performance norms Training
Communication
Team interactions
Potency/team self-eYcacy
Team spirit
Contingency Variables impinging from Team application/mission Task design
factors sources internal and external Resource availability Training
to the team Procedural requirements
Rules of operation,
managing, or decision-
making
Teamwork in multi-person systems 1059

the following types of teams: command and control teams, production teams,
customer service teams, professional/technical decision-making teams, and executive
teams. What types of traits or skills are needed for each Ð functional ¯ exibility or
highly specialized skills? Ability to deal with people and handle personal dynamics?
Ability to deal with poorly de® ned problems? Once a determination is made with
respect to basic abilities, team staYng requires that one goes further and address how
selection choices may in¯ uence performance outcomes of the team as a whole. There
is a need, therefore, to understand not only individual eVectiveness, but also team
eVectiveness as well, and to evaluate probable interactions between group-and
individual-level factors. This is where the theories and models of team eVectiveness
presented earlier provide some basis for action.
Success depends not only on knowledge, skills and attitudes (KSA) required for
individual task performance, but also on those traits of individual team members
that facilitate team interaction and functioning (e.g. learning ability, initiative, risk-
taking propensities, adaptability, tolerance for stress, etc.). These characteristics are
critical for teams that require more coordination, such as command and control
teams and traditional assembly line (production) teams. They are less important in
collaborative tasks where team interaction is less restricted. Similarly, service teams
would look on the ability to adapt team tasks to environmental demands as a
desirable attribute. Leadership qualities in¯ uence the performance of all teams,
whether they are exhibited by formally appointed leaders, or by non-appointed,
emerging leaders (Klimoski and Jones 1995).

4.1.2. Team size: Team selection involves establishing appropriate team size. This
becomes problematic when the task has not been performed before, or when it is
arti® cially constrained by such factors as leader preferences, available resources, or
the number of people free to participate. If too few people are chosen, undue stress
will be placed on team members; on the other hand, if too many are chosen,
resources are wasted (Klimoski and Jones 1995). Size is largely ® xed by the nature of
the tasks to be performed. It limits the manner in which the team can be organized
and how it can interact. Empirical evidence suggests that group productivity is not a
linear function of group size, but that a negatively accelerated function generally
exists between the two, that is, larger teams tend to be detrimental to eVectiveness,
typically as a result of heightened coordination needs (Kidd 1961, George et al. 1963,
Steiner 1972, Gladstein 1984, Sundstrom et al. 1990). As a general rule, teams should
be staVed to the smallest number needed to do the work (Hackman 1987, Sundstrom
et al. 1990).

4.1.3. Team composition: Not only size, but aspects of team composition must also
be determined. Speci® cally, team composition may vary along several dimensions:
(1) attributes such as age, gender, race, intelligence, aptitude, training, experience,
personality, etc., (2) distribution of these attributes within the team and (3) the
stability of team composition over time. Among the issues to be raised are: (1) the
eVects of team homogeneity/heterogeneity, compatibility and turnover, (2) the
degree of redundancy that exists between team functions and (3) the extent to which
team output is related to individual performances. With respect to the latter, the
more individual performance determines team output, the less important become the
team processes, and the less necessary it becomes to provide special team training
(Meister 1985).
1060 C. R. Paris et al.

4.1.4. Team stability: Team stability was mentioned above. Because individual skill
is a major determinant of team performance, the eVects of adding untrained
personnel as replacements are likely to be adverse. While adding skilled replacements
may show a positive eVect, that eVect tends to be smaller than the negative impact
resulting from fewer skilled replacements. As a general rule, there is little disruption
of team performance from turnover, as long as only one team member is replaced at
a time and that replacement is as skilled as the person he replaces. Disruption is
increased if additional team members are replaced (Naylor and Briggs 1965, Meister
1985). Teams composed of 40% or more untrained individuals demonstrate
declining performance (Morgan et al. 1978), and the greatest eVect on performance
is likely to come from changes in key or central positions (Ziller 1963, Trow 1964).

4.1.5. Select or train?: Selection becomes important in areas where training is


unlikely to have an eVect. For example, training may in¯ uence an individual team
member’s attitudes, but not his personality traits. Attitudes, or learned patterns of
behaviour, are more or fewer modi® able through training. Personality traits, on the
other hand, are `by de® nition’ stable, deeply rooted predispositions to respond in
predictable ways. As such, they may be resistant to change through training and
hence become an issue for selection. In short, personality traits limit training eVects
and constrain what may be accomplished with a particular individual (Prince et al.
1992).

4.1.6. Predicting team productivity: It is important to establish the mix, or level, of


the particular KSA that may be appropriate for a team. For some tasks (e.g.
production or manufacturing tasks), team performance is in¯ uenced by the ability of
the least capable person. In these situations, homogeneous teams may be a better
arrangement than heterogeneous teams. Although teams with lower ability may
produce less, their lower output can be compensated by the greater output of higher
ability teams, provided the latter are not adversely aVected by a single member of less
ability. The assessment of key attributes, such as team orientation, responsibility,
task focus, commitment, and interpersonal compatibility are hard to de® ne and
measure. Much work remains to be done in the area of developing construct-valid
and practical instruments that can measure these traits or qualities. There is hope
that through the alignment of team member similarities and diVerences, one may
begin to predict team productivity (Prince et al. 1992, Klimoski and Jones 1995).
Again, traditional staYng processes are not adequate for team member selection.
One cannot simply identify necessary KSA at the individual level, choose candidates
that meet those criteria, then hope that the team will perform as planned. Teams
present new challenges. And these challenges have to be met with new approaches to
select individuals to function in teams. This is evident, not only in team selection, but
also for task design.

4.2. Second approach: task design


If characteristics of the task can be found to facilitate or inhibit team coordination,
then tasks can be redesigned. Task design variables include such things as
automation, workload, time pressure, governmental regulations, organizational
policies, established procedures, team structure, etc. Campion et al. (1993, 1996)
determined that job design ranked second, below team processes, but above
interdependence, context, and composition, as important variables in¯ uencing team
Teamwork in multi-person systems 1061

eVectiveness. These authors proposed speci® c advice for task design. First,
management should make jobs motivating by encouraging autonomy, wide
participation in team decisions, a variety of task assignments, and interdependence
between team members. Management should create a supportive context for the
team in terms of training, resources, information, and encouragement, and should
monitor and encourage positive team processes. Some of the key design targets that
have been investigated to date are discussed below.

4.2.1. Workload/time constraints: Workload is a consideration because even one


overloaded team member, by neglecting his obligations to fellow team members,
could impact the performance of the team as a whole (Roby and Lanzetta 1957a,b,
Dyer 1984). Level of workload may also determine which team processes are needed
for team performance and which training interventions will be most useful for those
situations (Kleinman and Serfaty 1989, Bowers et al. 1992). Time pressure, which is
closely tied to workload, is negatively correlated with the reliability of decision-
making. As time pressure increases, the reliability of team decision-making drops
(Adelman et al. 1986).

4.2.2. Team architecture/structure: Team architecture (TA) refers to those system


or task variables that de® ne or in¯ uence how team members interact. At least three
variables are hypothesized to constitute TA: member proximity, communication
modality, and the allocation of functions to individual team members (Urban et al.
1995).
Member proximity involves both physical and psychological distance. Team
cohesiveness and communication ¯ uctuate as a function of physical distance.
Generally, when physical distance is smaller, cohesiveness and communication are
positively aVected. Greater distances tend to exert negative in¯ uences. Team
decision-making and coordination are ultimately aVected. Psychological distance is a
perceptual phenomenon brought about by chain-of-comman d or status diVerences,
both of which may also be related to physical distance. Psychological distance
operates in much the same manner as physical distance, resulting in decreased
communications as the perceived distance grows (Urban et al. 1995).
Communication modality refers to the nature of the medium through which team
members engage in their interactions. The most prevalent modality is face-to-face
verbal interaction. Other modalities include paper-mediated, audio-mediated, com-
puter-mediated, and video-mediated communication. Team processes, team decision-
making, and team coordination may all be diVerentially impacted by these modalities.
For example, interacting via computers tends: (1) to limit severely the social context cues
(visual feedback and status cues) that are available in face-to-face communication, (2) to
constrain the depth of discussion and analysis that is likely to occur and (3) to increase
the time needed for groups to make a decision. On the other hand, computer-mediated
interaction tends to obscure status diVerences, resulting in greater participation by
members and possibly better team coordination (Urban et al. 1995).
Team structure or organization refers to the assignment of particular
components of the team’s collective task to individual team members and to the
nature of the interactions that must ensue for the team to eVectively coordinate and
execute those tasks. A team’s organization, as well as its ability to adapt to various
system loads, is largely a function of the dependency relationships between tasks and
operators. Tasks may be performed independently, sequentially, or in parallel. If
1062 C. R. Paris et al.

task/operator dependency is low, organization is relatively unimportant, i.e.


inadequate performance at one position will have little eVect on other operator
positions. If dependency is high, inadequate performance at one operator station will
seriously aVect another (dependent) station, and this eVect may be ampli® ed by the
nature of the organization. In this sense, team organization determines team output.
Serially structuring a team will expose it to the distinct possibility of system overload
since its performance is determined by the weakest link in the chain (Meister 1985).
A team’s organization and task dependencies in¯ uence how team members interact.
Research has investigated whether those interactions can be objectively speci® ed or
trained, and has spawned practical guidelines (Swezey and Salas 1992a,b).
Teams communicate diVerentially in response to the demands imposed by team
structure. Consider, for example, how distributed team structures might have
important implications for member coordination and communication (Kleinman
and Serfaty 1989). Similarly, consider, non-hierarchical teams, where there is at least
some degree of overlap in responsibilities. Team members in those types of teams
communicate more than those in hierarchical teams, where there is no overlap in
team responsibilities. Additionally, non-hierarchical teams perform more eVectively
than hierarchical teams, at all workload levels. These results suggest that non-
hierarchical structures facilitate more eVective team coordination and decision-
making than hierarchical structures (Bowers et al. 1992, Urban et al. 1995).

4.2.3. Technology: Of all the variables to be investigated in the human engineering


of teams, particularly of those related to task design, one of the most in¯ uential is
that of technology. Consider just a few of the advances that have become part of
everyday life for many teams: telecommunications (where geographically separated
team members can communicate electronically to solve problems or to perform
simulated tasks in virtual environments), networked simulations, distance learning,
multimedia technology, interactive courseware, and even intelligent systems. In spite
of the tremendous promise of technology and automation, in team settings these
advancements have ironically yielded much disappointment (Wiener et al. 1991,
Thornton et al. 1992, Morgan et al. 1993, Bowers et al. 1996).
Team tasks are at signi® cant risk when automated technologies are introduced.
Automation eVects operating at the individual level may become compounded when
distributed across teams. As automation wholly or partially replaces team functions,
team structure and composition change, team roles are unavoidably rede® ned, and
interaction and communication patterns are altered (Bowers et al. 1993, Weiner
1993, Jentsch et al. 1995, Bowers et al. 1996, Mosier and Skitka 1996). While it has
been assumed that reductions in workload would accompany automation, this
bene® t has not been wholly realised. Automation frequently replaces physical
activity with cognitive and perceptual activity, leaving workload levels unchanged.
Additionally, situational awareness (SA) may decline. SA is a decision-maker’ s
moment-by-momen t ability to monitor and understand the state of a complex system
and its environment (Adams et al. 1995, Morrison et al. 1998). SA may decline as a
result of (1) monitoring demands and subsequent vigilance decrements, (2)
complacency due to over-reliance on automation, (3) system complexity, (4) poor
interface design, (5) inadequate training, or even (6) lack of trust in automation
(Endsley 1997). If operators lose SA, they may fail to notice system anomalies, may
intervene prematurely, may use improper procedures, and may not supply the
appropriate intervention should automation fail. Given that automation may not
Teamwork in multi-person systems 1063

reduce workload and may compromise SA, it may not be desirable to implement full
automation in team tasks, even if that is technically possible. Endsley (1997)
recommends that intermediate levels of automation may be preferable for certain
types of tasks or functions to keep SA at a higher level and allow human operators to
perform critical functions.
Automation also makes possible interactive training, which may now be
provided through distributed simulations linked via local or wide area networks.
Salas and Cannon-Bowers (1997) caution, however, that such simulations need to
link speci® c task requirements to team training needs to create eVective learning
environments (explanation of event-based approaches to training in Dwyer et al.
1997). Training simulators may also be embedded in operational equipment so that
they can provide instructional opportunities when operators are not busy performing
their tasks. Sophisticated systems, capable of capturing multiple forms of
performance data (eye movements, communications, keypress actions), diagnosing
performance, and providing on-line feedback, now exist (e.g. Zachary and Cannon-
Bowers 1997, Zachary et al. 1997a,b).
Besides their training value, computer systems may expedite decision-making in
team settings. Decision-making at the team level multiplies those limitations inherent
in human decision-making at the individual level. Decision support systems (DSS) can
aid the decision-maker by reducing problem complexity and associated memory load,
and by enhancing SA. An example of current work in this area may be found in the
TADMUS (Tactical Decision-Making Under Stress) DSS being designed for Naval
command and control environments. This DSS organizes and presents tactical data in
a form that is consistent with experts’ usage, and guides decision-makers through the
huge amount of tactical data which they need to process (see Morrison et al. 1998, for a
description of this system). In business settings, DSS (called group decision support
systems, or GDSS) support tasks with pooled interdependencies that require
discussion and communication. They are typically utilized in decision rooms, local
decision networks, teleconferencing, and remote decision-making. While most
evidence supports the use of decision support in group settings, i.e. decision quality
is increased, some research suggests that decision aids may isolate operators from each
other’s SA and decision-making processes, thereby reducing redundancy and human
input, and increasing the possibility of errors (Mosier and Shitka 1996). Bowers et al.
(1996) suggest that team decision-making imposes additional requirements on
decision-makers due to the distributed nature of the information, and that making
eVective use of that information requires that one develops techniques to integrate
what is necessary and vital to the decision-making task.
For better or for ill, the requirements for teams to operate with automated
systems will likely increase with time. Clearly, that technology can guarantee neither
greater eVectiveness nor eYciency in the way teams perform. Given that automation
eVects are both compounded and confounded at the team level, it becomes
imperative that engineering eVorts to optimize team performance account for their
potential impact. Taking the time to address this, as well as the impact of other task
design variables, may enable us to ameliorate many foreseeable de® ciencies in the
way teams are likely to conduct their tasks.

4.3. Third approach Ð team training


If the knowledge, skills, and strategies used to facilitate team coordination or
eVective performance can be identi® ed, then teams can be trained. Training serves to
1064 C. R. Paris et al.

automate controlled behavioural processes. Automating these behaviours makes


them more resilient to the eVects of stress, thus bolstering performance under
stressful conditions. It is easy to understand, then, why the demand for team training
has skyrocketed over the last 15 years, given the exponential growth in the number of
teams over that same period. Fortunately, great strides have been made since the
mid-1980s , when team training suVered many de® ciencies (e.g. too few training
methods/strategies that had been evaluated for their applicability to teams, few, if
any, quantitative performance standards for team tasks and skills, lack of speci® city
in feedback to teams and individuals) (Modrick 1986). Researchers are now more
able to de® ne what competencies need to be trained, as well as how to train them
(e.g. Dyer 1984, Swezey and Salas 1992a,b, Wiener et al. 1993, Guzzo and Salas
1995, Salas et al. 1995, West 1996, Cannon-Bowers and Salas 1998).

4.3.1. What to train: Until recently, team performance has been considered to be a
function of the average skill level of the individuals who comprise the team. While
individual skills are unquestionably necessary to team success, they are not suYcient
(Salas et al. 1992). Process losses, resulting from interruptions in the ¯ ow of
communication or coordination behaviours, are likely to threaten performance if
teamwork skills are not adequately developed (Steiner 1972). One can maximize
training eYciency by combining individual and team skills training into a single
training design and by allocating the appropriate amount of individual skills training
relative to team skills training. Additionally, one can tailor the sequence in which
each type of training is be presented (i.e. train individual skills ® rst), as well as the
time that should be allowed to elapse between individual and team skills training
(Salas et al. 1992). Finally, the skills that are targeted for training should meet
speci® c criteria, namely: (1) they have been empirically demonstrated to have an
eVect on team success, (2) they are diYcult to learn, (3) they require more than
simple repetition for development, (4) they require practice to prevent their loss and
(5) they may be infrequently required, but are essential for survival (Dyer 1984).
Cannon-Bowers et al. (1995) proposed a framework that contributes some useful
insights into selecting the appropriate competencies for training. The competencies
may be speci® c to a particular task or team, or they may be generic in that they apply
across teams or tasks. To ascertain what competencies must be trained, one would
begin by asking certain questions: Are the members of this team the same individuals
over time? Is the membership fairly stable? Does the team perform the same or
similar tasks over time? Are the tasks fairly stable? The answers to these questions
will align the required competencies into one of four categories. Table 4 provides an
explanation of each competency type, as well as the four possible alignments of the
various competency types.
This framework establishes the idea that team membership and task character-
istics prescribe the type of competencies that will lead to its success. Skills targeted
for training will be focused on the ability to work with speci® c or diVerent
individuals, or to work on speci® c or diVerent types of tasks. Once the appropriate
competencies and focus for training have been identi® ed, then it becomes important
to determine how best to train those skills.

4.3.2. What is team training?: In spite of signi® cant progress regarding what to
train, many present-day assumptions regarding how to train have no basis in
research. Team training is more than just team building. Clearly the data imply that
Teamwork in multi-person systems 1065

Table 4. Task and team competency types.


Competency types
Task Team
Speci® c Task-speci® c competencies involve Team-speci® c competencies relate to
performing teamwork behaviours a speci® c team and in¯ uence the
for a speci® c task or situation (e.g. performance of that team only (e.g.
interaction required for a task, knowledge of teammates’
knowledge of the speci® c role characteristics, team cohesion, etc.)
responsibilities for a particular
team, etc.)
Generic Task-generic competencies are Team-generic competencies are, in
transportable and can be used for principle, transportable from one
other tasks (e.g. planning skills, team to another and can in¯ uence
interpersonal skills, etc.) the performance of any team that an
individual serves on (e.g.
communication skills, attitudes
toward teamwork, etc.)

Competency alignments

Task speci® c/team speci® c: Needed when team membership is stable and the number of tasks
is small. Examples: sports teams and some air crews/combat teams
Task speci® c/team generic: Needed when team members perform a speci® c team task, but do
not work consistently with the same teammates. Examples: ® re® ghting team, air crews, or
medical teams
Task generic/team speci® c: Needed when team membership is stable, but the tasks vary.
Examples: self-managed work teams or quality circles
Task generic/team generic: Needed where team members work on a variety of teams, as well
as on a variety of tasks. Examples: task forces or project teams

it is not enough to train individual responsibilities and simply hope that the team
members will magically ® gure out how to operate as a team (Stout et al. 1994,
McIntyre and Salas 1995, Salas and Cannon-Bowers 2000).
Training strategies should be grounded in theory and sound instructional
principles. Salas and his colleagues have undertaken to formulate and empirically
validate principles, guidelines, and speci® cations for team training (Swezey and Salas
1992a,b, Salas and Cannon-Bowers 1997, 2000). Team training is essentially `a set of
tools and methods that, in combination with required competencies and training
objectives, form an instructional strategy’ (Salas and Cannon-Bowers 2000: 5).
Cognitive, behavioural, and aVective competencies necessary for eVective teamwork
drive the training objectives. Those objectives combine with available resources
(tools and methods) to shape the development of speci® c instructional strategies.
Training tools include, but are not limited to, team task analysis, task simulation and
exercises, and performance measurement and feedback. Methods for delivery may be
information-based , demonstration-based , or practice-based, and may include
lectures, video or multimedia presentations, demonstrations, guided practice, and
role-playing (Salas and Cannon-Bowers 1997, 2000). The goal is to design and
develop from these tools and methods speci® c instructional strategies for in¯ uencing
team processes and outcomes. Representative instructional strategies that have been
1066 C. R. Paris et al.

formulated in recent years are presented in table 5 (note that each of these strategies
has demonstrated its eVectiveness in the areas indicated).

4.3.2.1. Train the part or the whole?: Particularly problematic in the develop-
ment of training strategies is the selection of `part’ versus `whole’ training methods.
In part training, team members learn speci® c components of the task individually
and sequentially, and then gradually integrate those skills until the entire task is
mastered. In whole training, they are exposed to the entire task during all segments
of the training procedure. Instructors must decide whether to (1) provide individual
skills training that includes a focus on the interactive requirements of the team task
(individual-whole training), (2) provide individual skills training that ignores the
team context (individual-part training), (3) provide team skills training with speci® c
focus on each individual’s assigned subtask (team-part training) or (4) provide team
skills training within the context of the team and its coordination and communica-
tion requirements (team-whole training). If high complexity and high organization
characterize the team task, then the best strategy may be to combine the individual-
whole and the team-whole approaches. Unfortunately, no standard exists for
sequencing these two training types or for the establishing the appropriate ratio of
each. It is hoped that future research will shed more light on when individual/team
and part/whole training paradigms are most bene® cial, and will demonstrate how
task characteristics aVect these relationships (Salas et al. 1992).

4.3.2.2. Performance feedback: Once training is implemented, performance


feedback can become in the hands of team leaders one of the most in¯ uential tools
for shaping the development of their teams. Team leaders should consider the
amount, timeliness, focus, speci® city, and sequencing of their feedback to team
members. Ideally, that feedback is not delayed, but is given frequently and
immediately after each signi® cant team response, whether correct or incorrect. It
focuses on the task and is directed to both the individual and to the team as a whole.
It avoids emphasizing one characteristic of team performance at the expense of
others. It focuses on only one aspect of task performance during early training

Table 5. Representative instructional strategies.


Targeted skills Representative strategies References

Leadership/ Team leader training Dyer (1984)


assertiveness Tannenbaum et al. (1998)
Performance Guided team self-correction Blickensderfer et al. (1997a,b),
monitoring/ Smith-Jentsch et al. (1998),
feedback Salas and Cannon-Bowers (2000)
Communication/ Crew resource management (CRM) Wiener et al (1993), Prince and Salas
coordination/ Aircrew coordination training (1993), Serfaty et al. (1998), Volpe et
adaptability/ (ACT) al. (1996), Blickensderfer et al.
interpersonal Team adaptation and coordination (1998), Smith-Jentsch et al. (1998a,b,
skills training 1996a), Johnston et al. (1997)
Cross training
Team dimensional training (TDT)
Stress Stress exposure training Driskell and Johnston (1998)
management
Decision-making Critical thinking training Cohen et al. (1998)
Teamwork in multi-person systems 1067

sessions, but increases its scope to cover many aspects during later sessions. It
increases in speci® city with training, such that it reinforces gross aspects of
performance during early phases of team training, then more speci® c aspects later on
to `® ne tune’ performance. Finally, it is sequenced so that individual feedback is
given in initial training sessions, while feedback regarding overall performance is
provided in later phases of training. As the number of interdependencies among
team members increases, so does the importance of team feedback (Salas et al. 1992).

4.3.2.3. What are the instructional strategies?: Certain teamwork skills have
presented considerable challenge to the development of instructional strategies. One
such skill is the ability to learn on a continuous basis. Since learning is meant to be a
continuous process, teams should be trained in techniques that stimulate continuous
team growth. One of the strategies illustrated earlier Ð `Guided Team Self-
Correction’ Ð has successfully structured this type of training and made signi® cant
progress in meeting this particular challenge (Blickensderfer et al. 1997a,b, Smith-
Jentsch et al. 1998b).
Progress has been slower in other areas, notably in instructional development for
two very complex teamwork skills Ð shared SA and decision-making. If one accepts
the idea that teams are information processing units, a notion advanced by Hinsz et
al. (1997), then it becomes apparent that member interactions can enhance or
degrade a group’s ability to attend to, encode, store, retrieve, feedback, and
ultimately learn from crucial task information. These interactions become the
foundation for training in the areas of SA and decision-making.
SA relies on both individual and team mental models (Cannon-Bowers et al.
1993, Duncan et al. 1996, Stout et al. 1996). Each team member perceives one or
more parts of the complex environment, and these perceptions must be integrated
with those of fellow team-mates. The objective is for the team to share a common
picture of the environment. To accomplish this, team members must voice
communications that promote collective awareness of the surrounding environment,
both internal and external to the team, and make timely and accurate reports of
deviations from the norm or potential problems. Team leaders can continually
update team members during times of stress to keep them abreast of rapidly
changing priorities and performance objectives. Training that supports these
abilities, as well as the development of shared perceptions of critical task and
interpersonal dimensions will improve a team’s SA ability.
Decision-makers must ® rst recognize the problem and the need for action.
Depending on their task environment, teams rely on controlled, analytic decision-
making (e.g. problem-solving teams in an industrial environment) or on more
automatic perceptual-cognitiv e processes (e.g. tactical decision-making teams, law
enforcement or ® re-® ghting teams, emergency medical teams). The latter form of
decision-making, called `naturalistic decision-making’ (Zsambok 1997) or `recogni-
tion-primed decision-making’ (Klein 1989, 1993), is prevalent in environments where
performance is both resource and data-limited. In those environments, decision-
makers must maintain large amounts of information in memory under conditions of
high workload and stress, and their decisions may be skewed by the lack of complete,
error-free, unambiguous data. Because the situation itself either determines or
constrains the response options, decision-makers in these environments typically
make up to 95% of all decisions without considering alternatives (Klein 1989,
Kaempf et al. 1996). If the situation appears similar to one that the decision-maker
1068 C. R. Paris et al.

has previously experienced, the pattern will be recognized and the course of action is
usually immediately obvious. Due to notable team failures in these types of
environments (e.g. the 1988 Vincennes incident, in which the decision was made to
launch a defensive missile against an Iranian airbus carrying 290 passengers, whose
radar signal only moments before was misclassi® ed as that of a hostile attacking
® ghter aircraft), eVorts are currently underway to improve decision-making in these
types of environments.
Because this type of decision-making relies heavily on the ability of decision-
makers to recognize important features/patterns within contexts, naturalistic
approaches would promote the viability of training perceptual processes (Orasanu
1995). In fact, this approach parallels what researchers have discovered concerning
the development of expertise. In short, novices tend to use more controlled, analytic
approaches to reach their decisions. As they accumulate experience, they begin to
switch between controlled and more automatic, perceptual-cognitiv e processes. By
the time they become experts, they almost exclusively utilize the automatic
perceptual processes (Klein and HoVman 1993). Training for team members who
must perform in naturalistic settings might focus on key novice/expert diVerences
that have been identi® ed, namely: (1) recognizing patterns, (2) making ® ne
perceptual discriminations, (3) recognizing typicality and detecting anomalies, (4)
mentally simulating future states and past states, (5) improvising and (6) adapting to
events (Shanteau 1988, Klein and HoVman 1993, Means et al. 1993).
An earlier referenced training strategy, `Critical Thinking Training’, represents a
less common form of naturalistic decision-making called `explanation-base d
reasoning’ or story-generatio n (Cohen et al. 1998). In story-generation , the
decision-maker attempts to evaluate the discrepancies between expectations and
what actually happens by creating a reasonable hypothesis or story to explain it. As
with recognition-primed decision-making, however, this type of reasoning is not
exhaustive in the analytic tradition, but is fairly short and concise. It is usually
implemented when pattern recognition fails or when time permits.
These two processes Ð feature-matching and story-generatio n Ð have been
demonstrated to account for the majority of situation assessments and subsequent
decisions made by decision-makers in environments which preclude the use of more
controlled, analytic types of decision-making. In fact, Kaempf et al. (1993)
demonstrated that in a sample of 183 decisions made by the Navy’s command-
level decision-makers in the Combat Information Centre of an Aegis cruiser, 87% of
the situation assessments were derived through feature matching and the remaining
13% were derived through story generation.

5. Where does one go from here?


While team research, especially in the last decade, has been fruitful, considerable
work remains to be done. Several lines of investigation may be particularly bene® cial
(Salas et al. 1995, Salas and Cannon-Bowers 2000). Foremost, perhaps, are the
implications of cognitive theory, for team performance. One needs to understand
how teams function as information-processin g units, that is, how knowledge is
acquired, shared and acted on. How do individual team members contribute vital
pieces to the problem-solving puzzle, and how do those contributions build shared
mental models and promote SA? If one can understand this process, team
performance measurement tools training can be shaped to capitalize on it. The
next frontier in team measurement is to develop team assessment tools to capture
Teamwork in multi-person systems 1069

cognitive phenomena. This is a must if progress is to be made understanding team


functioning in complex systems. Related, and as stated above, is the need to improve
task analysis for teams so that one may evaluate interdependence better among team
members and the cognitive requirement. For a speci® c team, one needs to
understand how much shared knowledge is required by that team’s task
interdependence, and what factors potentially moderate the relation between their
shared knowledge and performance. Until one understands this interdependence,
and the contributions both expected and required, it is impossible eVectively to assess
or train team performance. Improved team task analysis will facilitate the
development of expert models needed for diagnostic purposes. Furthermore, task
analysis, especially cognitive task analysis, is extremely time-consuming and
painstaking in nature. One needs better tools to facilitate this process. One needs
to improve the elicitation of knowledge from experts so that one can understand the
subtleties, relationships, and interdependencies present in team tasks, and provide
insights necessary to identify important training objectives.
Finally, how can one improve the diagnosticity of team performance measures?
Interpreting the mass quantities of performance data (e.g. communications, keypress
actions, eye movements, other physiological indicators) that are captured in some
team settings, in some meaningful way is a daunting task at best. What knowledge
requirements and behavioural skills lie behind those measures? It is vitally important
that solid links are established between knowledge requirements, training objectives
and performance indicators. This is particularly true for training simulations, both
stand-alone and networked. Given the cost drivers in many modern operational
settings, it is even more important that training resources be allocated eYciently.
Automating the synthesis and interpretation of performance measurement data, as
well as its collection, is also vital to many operational environments (e.g. military
training situations). Solutions are needed to alleviate the labour intensive monitoring
and evaluation so often necessary in team settings. Whether or not automation is
present or feasible, one needs better measurement tools, particularly for unique
team-related skills (e.g. its dynamic nature). If one considers that such tools would
need to elicit and measure shared knowledge, then one can understand how
seemingly formidable such a task would be.
Undoubtedly, the ® eld of team research will present enormous challenges and
opportunities in the years that lie ahead. Clearly, the answers to these and many
other questions will show us the way to transform teams of experts into expert teams.
Most importantly, they will empower teams to respond to the scope and complexity
of the task demands that society has enlisted them to shoulder.

6. Concluding remarks
The progression of this concept has been traced from its inception 50 years ago to
current thinking. It has been learnt that teamwork is the seamless integration of
speci® c cognitive, behavioural and aVective competencies that allow team members
to adapt and optimize their performance. Researchers have made great strides in
de® ning teams, in ascertaining how they mature and develop, and in diVerentiating
their performance from that of individuals and groups. They have identi® ed
competencies that enable teams to meet their goals, and have developed eVective
techniques for capturing, measuring and teaching those skills. In terms of training, a
considerable repertoire has been acquired from which to draw principles, guidelines
and speci® cations to maximize success. Finally, researchers have considered ways to
1070 C. R. Paris et al.

select better teams, to design better team tasks and to adapt automation to team
settings.

Acknowledgements
The views expressed herein are those of the authors and do not re¯ ect the oYcial
position of the organization to which they are aYliated.

References
ADAMS , M. J., TENNEY, Y. J. and PEW , R. W. 1995, Situation awareness and the cognitive
management of complex systems, Human Factors, 37, 85 ± 104.
ADELMAN, L. ZIRK, D. A., LEHNER, P. E., MOFFETT, R. J. and HALL, R. 1986, Distributed
tactical decision-making: Conceptual framework and empirical results, IEEE Transac-
tions on Systems, Man, and Cybernetics, SMC-16, 794 ± 805.
ANNETT, J. 1997, Analysing team skills, in R. Flin, E. Salas, M. Strub and L. Martin (eds),
Decision-making Under Stress: Emerging Themes and Applications (Aldershot: Ashgate),
315 ± 325.
BAKER , D. P. and SALAS, E. 1992, Principles for measuring teamwork skills, Human Factors, 34,
469 ± 475.
BANDURA , A. 1986, Social Foundations of Thought and Action (Englewood CliVs: Prentice-
Hall).
BASS, B. M. 1980, Individual capability, team performance, and team productivity, in E. A.
Fleischman and M. D. Dunnette (eds), Human Performance and Productivity (Hillsdale:
Lawrence Erlbaum), 179 ± 232.
BLICKENSDERFER , E. L., CANNON -BOWERS , J. A. and SALAS , E. 1997a, Fostering shared mental
models through team self-correction: theoretical bases and propositions, in M.
Beyerlein, D. Johnson and S. Beyerlein (eds), Advances in Interdisciplinary Studies in
Work Team, vol. 4 (Greenwich: JAI Press).
BLICKENSDERFER , E. L., CANNON -BOWERS, J. A. and SALAS, E. 1997b, Training teams to self-
correct: an empirical investigation. Paper presented at the 12th Annual Conference of
the Society for Industrial and Organizational Psychology, St Louis, MO.
BLICKENSDERFER , E. L., CANNON -BOWERS , J. A. and SALAS , E. 1998, Cross-training and team
performance, in J. A. Cannon-Bowers and E. Salas (eds), Making Decisions Under
Stress: Implications for Individual and Team Training (Washington, DC: American
Psychological Association Press), 299 ± 311.
BOWERS, C. A., DEATON , J., OSER, R., PRINCE, C. and KOLB , M. 1993, The impact of automation
on crew communication and performance, in Proceedings of the Seventh International
Symposium on Aviation Psychology (Columbus: Ohio State University), 758 ± 761.
BOWERS, C. A., OSER, R. L., SALAS , E. and CANNON-BOWERS , J. A. 1996, Team performance in
automated systems, in R. Parasuraman and M. Mouloua (eds), Automation and Human
Performance: Theory and Applications (Mahwah: Lawrence Erlbaum), 243 ± 263.
BOWERS, C. A., URBAN, J. M. and MORGAN , B. B. JR 1992, The Study of Crew Coordination and
Performance in Hierarchical Team Decision Making. Report no. TR-92-01 (Orlando:
University of Central Florida, Team Performance Laboratory).
CAMPION, M. A., MEDSKER, G. J. and HIGGS, A. C. 1993, Relations between work group
characteristics and eVectiveness: implications for designing eVective work groups,
Personnel Psychology, 46, 823 ± 850.
CAMPION, M. A., PAPPER, E. M. and MEDSKER , G. J. 1996, Relations between work team
characteristics and eVectiveness: a replication and extension, Personnel Psychology, 49,
429 ± 452.
CANNON -BOWERS, J. A. and SALAS, E. (eds) 1998, Making Decisions Under Stress: Implications for
Individual and Team Training (Washington, DC: American Psychological Association).
CANNON -BOWERS, J. A., SALAS , E. and CONVERSE, S. A. 1993, Shared mental models in expert
team decision-making, in N. J. Castellan, Jr (ed.), Individual and Group Decision Making
(Hillsdale: Lawrence Erlbaum), 221 ± 246.
CANNON -BOWERS, J. A., SALAS, E. and GROSSMAN , J. D. 1991, Improving tactical decision-
making under stress: research directions and applied implications. Paper presented at
the International Applied Military Psychology Symposium, Stockholm.
Teamwork in multi-person systems 1071

CANNON -BOWERS , J. A., TANNENBAUM , S. I., SALAS , E. and VOLPE, C. E. 1995, De® ning team
competencies: Implications for training requirements and strategies, in R. Guzzo and E.
Salas (eds), Team EVectiveness and Decision Making in Organizations (San Francisco:
Jossey Bass).
COHEN, M. S., FREEMAN, J. T. and THOMPSON , B. 1998, Critical thinking skills in tactical
decision-making: a model and a training strategy, in J. A. Cannon-Bowers and E. Salas
(eds), Making Decisions Under Stress: Implications for Individual and Team Training
(Washington, DC: American Psychological Association Press), 155 ± 189.
DICKINSON, T. L. 1969, The eVects of work interaction and its interplay with task organization
on team and member performance. Doctoral dissertation, Ohio State University,
Dissertation Abstracts International, 30, 1937 ± 1938.
DIETERLY, D. L. 1988, Team performance requirements, in S. Gael (ed.), The Job Analysis
Handbook for Business, Industry, and Government, vol. 1 (New York: Wiley), 486 ± 492.
DRISKELL, J. E. and JOHNSTON, J. H. 1998, Stress exposure training, in J. A. Cannon-Bowers
and E. Salas (eds), Making Decisions Under Stress: Implications for Individual and Team
Training (Washington, DC: American Psychological Association Press), 191 ± 217.
DRISKELL, J. E. and SALAS , E. 1992, Collective behaviour and team performance, Human
Factors, 34, 277 ± 288.
DUNCAN, P. C., ROUSE, W. B., JOHNSTON, J. H., CANNON -BOWERS, J. A., SALAS , E. and BURNS, J.
J. 1996, Training teams working in complex systems: a mental model-based approach,
in W. B. Rouse (ed.), Human/Technology Interaction in Complex Systems, vol. 8
(Greenwich: JAI Press), 173 ± 231.
DWYER, D. J., FOWLKES, J. E., OSER, R. L., SALAS, E. and LANE , N. E. 1997, Team performance
measurement in distributed environments: the TARGETs methodology, in M. T.
Brannick, E. Salas and C. Prince (eds), Assessment and Measurement of Team
Performance: Theory, Research, and Application (Hillsdale: Lawrence Erlbaum), 137 ±
154.
DYER , D. J. 1984, Team research and team training: a state-of-the-art review, in F. A. Muckler
(ed.), Human Factors Review: 1984 (Santa Monica: Human Factors Society), 285 ± 323.
ENDSLEY , M. R. 1997, Level of automation: integrating humans and automated systems, in
Proceedings of the Human Factors and Ergonomic s Society 41st Annual Meeting (Santa
Monica: Human Factors Society).
ENTIN, E., SERFATY, D. and DECKERT, J. C. 1994, Team Adaptation and Coordination Training.
Technical report, NAWCTSD Contract N61339-91-C-0142 .
FLEISHMAN, E. A. and ZACCARO, S. J. 1992, Toward a taxonomy of team performance
functions, in R. W. Swezey and E. Salas (eds), Teams: Their Training and Performance
(Norwood: Ablex), 31 ± 56.
GENTNER, D. and STEVENS, A. L. 1983, Mental Models (Hillsdale: Lawrence Erlbaum).
GEORGE, C. E., HOAK , G. and BOUTWELL , J. 1963, Pilot Studies of Team EVectiveness. Research
memorandum no. 28, AD-627, 217 (US Army Infantry Human Research Unit, Human
Resources Research OYce, Ft Benning, GA).
GERSICK, C. J. G. 1988, Time and transition in work teams: towards a new model of group
development, Academy of Management Review, 31, 941.
GLADSTEIN, D. L. 1984, Groups in context: A model of task group eVectiveness, Administrative
Science Quarterly, 29, 499 ± 517.
GOODMAN , P. S., RAVLIN, E. and SCHMINKE, M. 1987, Understanding groups in organizations,
in L. L. Cummings and B. M. Staw (eds), Research in Organizational Behaviour, vol. 9
(Greenwich: JAI Press), 121 ± 173.
GREGORICH, S. E., HELMREICH, R. L. and WILHELM, J. A. 1990, The structure of cockpit
management attitudes, Journal of Applied Psychology, 75, 682 ± 690.
GUZZO, R. A. and SALAS, E. (eds) 1995, Team EVectiveness and Decision Making in
Organizations (San Francisco: Jossey Bass).
GUZZO, R. A., YOST, P. R., CAMPBELL, R. J. and SHEA, J. P. 1994, Potency in groups:
articulating the construct, British Journal of Social Psychology, 32, 87 ± 106.
HACKMAN, J. R. 1983, A Normative Model of Work Team EVectiveness. Technical report no. 2
(New Haven: Yale University).
HACKMAN, J. R. 1987, The design of work teams, in J. W. Lorsch (ed.), Handbook of
Organizational Behaviour (Englewood CliVs: Prentice-Hall), 315 ± 342.
1072 C. R. Paris et al.

HACKMAN , J. R. (ed.) 1990, Groups That Work (And Those That Do not): Creating Conditions
for EVective Teamwork (San Francisco: Jossey-Bass).
HACKMAN , J. R. and MORRIS , C. G. 1975, Group tasks, group interaction process, and group
performance eVectiveness: a review and proposed integration, in L. Berkowitz (ed.),
Advances in Experimental Social Psychology, vol. 8 (New York: Academic Press), 45 ±
99.
HINSZ, V. B., TINDALE , R. S. and VOLLRATH , D. A. 1997, The emerging conceptualisation of
groups as information processors, Psychological Bulletin, 121, 43 ± 64.
ILGEN , D. R. 1999, Teams embedded in organizations: some implications, American
Psychologist, 54, 129 ± 139.
JENTSCH, F., BOWERS, C., REDSHAW , B., BERGEN, H., HENNING, J. and HOLMES, B. 1995,
DiVerential EVects of Automation on Team Performance, Workload, and Communica-
tions. Technical report (Orlando: Naval Air Warfare Centre Training Systems Division,
University of Central Florida).
JOHNSTON , J. H., SMITH-JENTSCH, K. A. and CANNON -BOWERS, J. A. 1997, Performance
measurement tools for enhancing team decision-making, in M. T. Brannick, E. Salas
and C. Prince (eds), Team Performance Assessment and Measurement: Theory, Methods,
and Applications (Hillsdale: Lawrence Erlbaum).
KAEMP F, G. L., KLEIN, G., THORDSEN, M. L. and WOLF, S. 1996, Decision-making in complex
Naval command-and-control environments, Human Factors, 38, 220 ± 231.
KAEMP F, G. L., WOLF, S. and MILLER, T. E. 1993, Decision-making in the Aegis combat
information center, in Proceedings of the Human Factors and Ergonomics Society 37th
Annual Meeting (Santa Monica, CA), 1107 ± 1111.
KIDD, J. S. 1961, A comparison of one-, two-, and three-man work units under various
conditions of workload, Journal of Applied Psychology, 45, 195 ± 200.
KLEIN, G. A. 1989, Recognition-primed decisions, in W. Rouse (ed.), Advances in Man ±
Machine Systems Research, vol. 5 (Greenwich: JAI Press), 47 ± 92.
KLEIN, G. A. 1993, A recognition primed decision (RPD) model of rapid decision-making, in
G. A. Klein, J. Orasanu, R. Calderwood and C. E. Zsambok (eds), Decision Making in
Action: Models and Methods (Norwood: Ablex), 138 ± 147.
KLEIN, G. A. and HOFFMAN, R. 1993, Seeing the invisible: perceptual ± cognitive aspects of
expertise, in M. Rabinowitz (ed.), Cognitive Science Foundations of Instruction
(Hillsdale: Lawrence Erlbaum) 203 ± 226.
KLEINMAN, D. L. and SERFATY, D. 1989, Team performance assessment in distributed decision-
making, in Proceedings of the Symposium on Interactive Networked Simulation for
Training (Orlando), 22 ± 27.
KLIMOSKI, R. and JONES , R. G. 1995, StaYng for eVective group decision-making: key issues in
matching people and teams, in R. A. Guzzo, E. Salas and Associates (eds), Team
EVectiveness and Decision Making in Organizations (San Francisco: Jossey-Bass), 291 ±
332.
KOLODNY , H. F. and KIGGUNDU , M. N. 1980, Towards the development of a sociotechnical
systems model in woodlands mechanical harvesting, Human Relations, 33, 623 ± 645.
MCINTYRE, R. M. and SALAS , E. 1995, Measuring and managing for team performance: emerging
principles from complex environments, in R. Guzzo and E. Salas (eds), Team EVectiveness
and Decision Making in Organizations (San Francisco: Jossey-Bass), 149 ± 203.
MEANS , B., SALAS, E., CRANDALL , B. and JACOBS, O. 1993, Training decision-makers for the real
world, in G. A. Klein, J. Orasanu, R. Calderwood and C. E. Zsambok (eds), Decision
Making in Action: Models and Methods (Norwood: Ablex).
MEISTER, D. 1985, Behavioural Foundations of System Development. 2nd edn (Malabar: Robert
E. Krieger).
MODRICK, J. A. 1986, Team performance and training, in J. Zeidner (ed.), Human Productivity
Enhancement: Training and Human Factors in Systems Design, vol. 1 (New York:
Praeger), 130 ± 166.
MORGAN , B. B. JR, COATES , G. D., ALLUIZI, E. A. and KIRBY , R. H. 1978, The Team-Training
Load as a Parameter of EVectiveness for Collective Training in Units. ITR-78-14, DTIC
no. AD A063 135 (Norfolk: Old Dominion University).
Teamwork in multi-person systems 1073

MORGAN , B. B. JR, GLICKMAN, A. S., WOODWARD , E. A., BLAIWES, A. S. and SALAS, E. 1986,
Measurement of Team Behaviours in a Navy Environment. NTSC technical report TR-
86-014 (Orlando: Naval Training Systems Centre).
MORGAN , B. B. JR, HERSCHLER, D. A., WIENER, E. L. and SALAS , E. 1993, Implications of
automation technology for aircrew coordination performance, in W. B. Rouse (ed.),
Human/Technology Interaction in Complex Systems, vol. 6 (Greenwich: JAI Press), 105 ±
136.
MORGAN , B. B. JR, SALAS , E. and GLICKMAN, A. S. 1994, An analysis of team evolution and
maturation. Journal of General Psychology, 120, 277 ± 291.
MORGESON , F. P., AIMAN-SMITH, L. D. and CAMPION, M. A. 1997, Implementing work teams:
Recommendations from organizational behaviour and development theories, in M. M.
Beyerlein, D. A. Johnson and S. T. Beyerlein (eds), Advances in Interdisciplinary Studies
of Work Teams, 4, 1 ± 44.
MORRISON , J. G., KELLY, R. T., MOORE , R. A. and HUTCHINS, S. G. 1998, Implications of
decision-making research for decision support and displays, in J. A. Cannon-Bowers
and E. Salas (eds), Making Decisions Under Stress: Implications for Individual and Team
Training (Washington, DC: American Psychological Association Press), 375 ± 406.
MOSIER, K. L. and SKITKA, L. J. 1996, Human decision-making and automated decision aids:
made for each other?, in R. Parasuraman and M. Mouloua (eds), Automation and
Human Performance: Theory and Applications (Mahwah: Lawrence Erlbaum), 201 ± 220.
MULLEN, B. and COPPER, C. 1994, The relation between group cohesiveness and performance:
an integration, Psychological Bulletin, 115, 210 ± 217.
NAYLOR , J. C. and BRIGGS, G. E. 1965, Team-training eVectiveness under various conditions,
Journal of Applied Psychology, 49, 223 ± 229.
NAYLOR , J. C. and DICKINSON, T. L. 1969, Task structure, work structure, and team
performance, Journal of Applied Psychology, 53, 167 ± 177.
NIVEA, F., FLEISHMAN, E. A. and REICK, A. 1978, Team Dimensions: Their Identity, Their
Measurement, and Relationships. Final technical report, Contract DAH19-78-C-0001
(Washington, DC: Advance Resources Research Organization).
ORASANU , J. 1990, Shared mental models and crew performance. Paper presented at the 34th
Annual Meeting of the Human Factors Society, Orlando.
ORASANU , J. 1995, Training for aviation decision-making: the naturalistic decision-making
perspective, in Proceedings of the Human Factors and Ergonomics Society 39th Annual
Meeting (Santa Monica, CA).
ORASANU , J. and SALAS , E. 1993, Team decision-making in complex environments, in G. A.
Klein, J. Orasanu, R. Calderwood and C. Zsambok (eds), Decision Making in Action:
Models and Methods (Norwood: Ablex), 327 ± 345.
OSER, R. L., MCCALLUM, G. A., SALAS, E. and MORGAN , B. JR 1989, Toward a De® nition of
Teamwork: An Analysis of Critical Team Behaviour. NTSC technical report no. 90-009
(Orlando: Naval Training Systems Centre).
PEARCE , J. A. and RAVLIN, E. C. 1987, The design and activation of self-regulating work
groups, Human Relations, 40, 751 ± 782.
PRINCE, C., CHIDESTER, T. R., BOWERS , C. and CANNON -BOWERS, J. 1992, Aircrew coordination:
achieving teamwork in the cockpit, in R. W. Swezey and E. Salas (eds), Teams: Their
Training and Performance (Norwood: Ablex), 329 ± 353.
PRINCE, C. and SALAS , E. 1993, Training and research for teamwork in the military aircrew, in
E. L. Wiener, B. G. Kanki and R. L. Helmreich (eds), Cockpit Resource Management
(Orlando: Academic Press), 337 ± 366.
ROBY, T. B. and LANZETTA, J. T. 1957a, A Replication Study of Work Group Structure and Task
Performance. DTIC no. AD 134 205 (Air Force Personnel and Training Research
Centre, Lackland Air Force Base, TX).
ROBY, T. B. and LANZETTA, J. T. 1957b, Con¯ icting principles in man ± machine system design,
Journal of Applied Psychology, 41, 170 ± 178.
ROUSE , W. B. and MORRIS , N. M. 1986, On looking into the black box: prospects and limits in
the search for mental models, Psychological Bulletin, 100, 349 ± 363.
RUFFELL-SMITH, H. P. 1979, A Simulator Study of the Interaction of Pilot Workload with Errors.
NASA technical report no. TM-78482 (MoVett Field: National Aeronautics and Space
Administration, Ames Research Centre).
1074 C. R. Paris et al.

RUMELHART , D. D. and ORTONY , A. 1977, The representation of knowledge in memory, in R.


C. Anderson and R. J. Spiro (eds), Schooling and the Acquisition of Knowledge
(Hillsdale: Lawrence Erlbaum), 99 ± 135.
SALAS , E., BOWERS , C. A., CANNON -BOWERS , J. A. 1995, Military team research: 10 years of
progress, Military Psychology, 7, 55 ± 75.
SALAS , E. and CANNON -BOWERS, J. A. 1997, Methods, tools, and strategies for team training, in
M. A. Quinones and A. Ehrenstein (eds), Training for a Rapidly Changing Workplace:
Applications of Psychological Research (Washington, DC: American Psychological
Association), 249 ± 279.
SALAS , E. and CANNON -BOWERS, J. A. 2000, The anatomy of team training, in L. Tobias and D.
Fletcher (eds), Handbook on Research in Training (New York: Macmillan) (in press).
SALAS , E., DICKINSON, T. L., CONVERSE, S. and TANNENBAUM , S. I. 1992, Toward an
understanding of team performance and training, in R. W. Swezey and E. Salas
(eds), Teams: Their Training and Performance (Norwood: Ablex), 3 ± 29.
SERFATY, D., ENTIN, E. and JOHNSTON, J. H. 1998, Team coordination training, in J. A. Cannon-
Bowers and E. Salas (eds), Making Decisions Under Stress: Implications for Individual
and Team Training (Washington, DC: American Psychological Association Press), 221 ±
245.
SHANTEAU , J. 1988, Psychological characteristics and strategies of expert decision-makers, Acta
Psychologica, 68, 203 ± 215.
SHEA, G. P. and GUZZO, R. A. 1987, Groups as human resources, in K. M. Rowland and G. R.
Ferris (eds), Research in Personnel and Human Resources Management, vol. 5
(Greenwich: JAI Press), 323 ± 356.
SHIFLETT, S. C., EISNER, E. J., PRICE, S. J. and SCHEMMER, F. M. 1982, The De® nition and
Measurement of Team Functions. Final report (Bethesda: Advanced Research Resources
Organization).
SMITH, P. C. and KENDALL, L. M. 1963, Retranslation of expectations: an approach for the
construction of unambiguous anchors for rating scales, Journal of Applied Psychology,
47, 149 ± 155.
SMITH-JENTSCH, K. A., JOHNSTON , J. H. and PAYNE, S. C. 1998a, Measuring team-related
expertise in complex environments, in J. A. Cannon-Bowers and E. Salas (eds), Making
Decisions Under Stress: Implications for Individual and Team Training (Washington, DC:
American Psychological Association), 61 ± 87.
SMITH-JENTSCH, K. A., PAYNE , S. C. and JOHNSTON, J. H. 1996a, Guided team self-correction: a
methodology for enhancing experiential team training, in K. A. Smith-Jentsch (Chair),
When, how and why does practice make perfect? Paper presented at the 11th Annual
Conference of the Society for Industrial and Organizational Psychology, San Diego.
SMITH-JENTSCH, K. A., SALAS, E. and BAKER, D. 1996b, Training team performance-related
assertiveness, Personnel Psychology, 49, 909 ± 936.
SMITH-JENTSCH, K. A., ZEIZIG, R. L., ACTON, B. and MCPHERSON , J. A. 1998b, Team
dimensional training: a strategy for guided team self-correction, in J. A. Cannon-Bowers
and E. Salas (eds), Making Decisions Under Stress: Implications for Individual and Team
Training (Washington, DC: American Psychological Association), 271 ± 297.
STEINER, I. D. 1972, Group Processes and Productivity (New York: Academic Press).
STOUT , R. J., CANNON -BOWERS , J. A. and SALAS , E. 1996, The role of shared mental models in
developing team situational awareness: implications for training, Training Research
Journal, 2, 85 ± 116.
STOUT , R. J., SALAS , E. and CARSON, R. 1994, Individual task pro® ciency and team process:
what’s important for team functioning, Military Psychology, 6, 177 ± 192.
SUNDSTROM , E. and ALTMAN , I. 1989, Physical environments and work-group eVectiveness, in
L. L. Cummings and B. M. Shaw (eds), Research in Organizational Behaviour, vol. 11
(Greenwich: JAI Press), 175 ± 209.
SUNDSTROM , E., DE MEUSE, K. P. and FUTRELL, D. 1990, Work teams: applications and
eVectiveness, American Psychologist, 45, 120 ± 133.
SWEZEY, R. W., LLANERAS , R. E. and SALAS , E. 1992, Ensuring teamwork: a checklist for use in
designing team training programs, Performance and Instruction, 31, 33 ± 37.
Teamwork in multi-person systems 1075

SWEZEY, R. W., OWENS, J. M. BERGONDY , M. L. and SALAS , E. 1998, Task and training
requirements analysis methodology (TTRAM): an analytic methodology for identifying
potential training uses of simulator networks in teamwork-intensive task environments,
Ergonomics, 41, 1678 ± 1697.
SWEZEY, R. W. and SALAS , E. 1992a, Guidelines for use in team-training development, in R. W.
Swezey and E. Salas (eds), Teams: Their Training and Performance (Norwood: Ablex),
219 ± 245.
SWEZEY, R. W. and SALAS , E. (eds) 1992b, Teams: Their Training and Performance (Norwood:
Ablex).
TANNENBAUM , S. I., BEARD , R. L. and SALAS , E. 1992, Team building and its in¯ uence on team
eVectiveness: an examination of conceptual and empirical developments, in K. Kelley
(ed.), Issues, Theory, and Research in Industrial/Organization Psychology (Amsterdam:
Elsevier), 117 ± 153.
TANNENBAUM , S. I., SMITH-JENTSCH, K. A. and BEHSON , S. J. 1998, Training team leaders to
facilitate team learning and performance, in J. A. Cannon-Bowers and E. Salas (eds),
Making Decisions Under Stress: Implications for Individual and Team Training
(Washington, DC: American Psychological Association Press), 247 ± 270.
THORNTON , C. BRAUN , C., BOWERS, C. A. and MORGAN , B. JR 1992, Automation eVects in the
cockpit: a low-® delity investigation, in Proceedings of the 36th Annual Meeting of the
Human Factors Society (Santa Monica, CA), 30 ± 34.
TROW , D. B. 1964, Teamwork Under Turnover and Succession. Technical report no. 2, OYce of
Naval Research contract no. 3679 (00), project NR 170-331, DTIC no. AD 601 816
(Endicott: Harpur College).
URBAN , J. M., BOWERS, C. A., CANNON -BOWERS , J. A. and SALAS , E. 1995, The importance of
team architecture in understanding team processes, in M. Beyerlein (ed.), Advances in
Interdisciplinary Studies in Work Teams, vol. 2 (Greenwich: JAI Press), 205 ± 228.
VOLP E, C. E., CANNON -BOWERS , J. A., SALAS , E. and SPECTOR, P. 1996, The impact of cross
training on team functioning, Human Factors, 38, 87 ± 100.
WEST, M. A. 1996, Handbook of Work Group Psychology (New York: Wiley).
WIENER, E. L. 1993, Crew coordination and training in the advanced-technology cockpit, in E.
L. Wiener, B. G. Kanki and R. L. Helmreich (eds), Cockpit Resource Management (San
Diego: Academic Press), 199 ± 229.
WIENER, E. L., CHIDESTER, T. R., KANKI , B. G., PALMER, E. A., CURRY , R. E. and GREGORICH, S.
E. 1991, The Impact of Cockpit Automation on Crew Coordination and Communication :
I. Overview, LOFT Evaluations, Error Severity, and Questionnaire Data. Technical
report no. NASA-CR-177587 (MoVett Field: NASA Ames Research Centre).
WIENER, E. L., KANKI, B. J. and HELMREICH, R. L. (eds) 1993, Cockpit Resource Management
(San Diego: Academic).
ZACHARY , W., BILAZARIAN, P., BURNS, J. and CANNON -BOWERS , J. A. 1997a, Advanced embedded
training concepts for shipboard systems (CD-ROM), in Proceedings of the 19th Annual
Interservice/Industry Training, Simulation, and Education Conference (Orlando: National
Training Systems Association), 670 ± 679.
ZACHARY , W. and CANNON -BOWERS , J. A. 1997, Guided practice Ð a new vision for intelligent
embedded training, in Proceedings of the Human Factors and Ergonomics Society 41st
Annual Meeting (Santa Monica, CA).
ZACHARY , W., RYDER, J., HICINBOTHOM, J. and BRACKEN, K. 1997b, The use of executable
cognitive models in simulation-based intelligent embedded training, in Proceedings of
the Human Factors and Ergonomics Society 41st Annual Meeting (Santa Monica, CA).
ZILLER, R. C. 1963, The EVects of Changes in Group Composition on Group Performance. Final
report, grant no. AFOSR 62-95, DTIC no. AD 413 965 (Newark: University of
Delaware).
ZSAMBOK, C. E. 1997, Naturalistic decision-making: where are we now?, in C. E. Zsambok and
G. Klein (eds), Naturalistic Decision Making (Mahwah: Lawrence Erlbaum).

You might also like