DEVELOPMENT

AND IMPLEMENTATION
OF AN
ADVANCED QUALIFICATION PROGRAM
(AQP)

May 2010
 INTRODUCTION
The purpose of this document is to inform air operators of the procedures required to develop and
implement an Advanced Qualification Program (AQP). It also establishes the conditions under
which an AQP will be approved.

When the Canadian Aviation Regulations (CARs) were initially drafted, provision was made to
include training approved and conducted under the auspices of AQP.

Standards for AQP are currently being developed. In the mean time, it has been decided to publish
the information and requirements in this document, under the cover of a Policy Letter (PL), to allow
air operators to take immediate advantage of the program.

This document is intended to serve both as a guide for air operators in the development of their
AQP, and for the use of the Principal Operations Inspectors (POI) in the review and approval
process of an applicant’s program.

The future AQP standard will be developed based on the material contained herein.

2
 TABLE OF CONTENTS

INTRODUCTION ................................................................................................2

ACRONYMS .................................................................................................8

DEFINITIONS ...............................................................................................10

CHAPTER 1 - AQP INTRODUCTION AND OVERVIEW ...................................19

1.1 Introduction......................................................................................................... 19
1.1.1 Statement of Purpose .................................................................................... 19
1.1.2 History........................................................................................................... 19
1.1.3 Background ................................................................................................... 20
1.1.4 Benefits of AQP............................................................................................ 21

1.2 AQP Overview..................................................................................................... 22

1.2.1 General.......................................................................................................... 22
1.2.2 Objectives of AQP ........................................................................................ 23
1.2.3 Characteristics of AQP ................................................................................. 23
1.2.4 Requirements of AQP ................................................................................... 24
1.2.5 Developmental Assistance and Support........................................................ 26
1.2.6 Transport Canada Support ............................................................................ 27
1.2.7 Evaluation Review Team (ERT)................................................................... 27

CHAPTER 2 - AQP DEVELOPMENT PHASES.................................................28

2.1 Introduction......................................................................................................... 28
2.1.1 Phases............................................................................................................ 28
Fig 2-1: AQP Development Process ............................................................................ 29
2.1.2 Instructional System Development (ISD)..................................................... 29
2.1.2 Instructional System Development (ISD)..................................................... 30
2.1.3 Training Systems .......................................................................................... 30

2.2 Phase I - Application........................................................................................... 31

2.2.1 Joint Meeting ................................................................................................ 31
2.2.2 The AQP Application/Administration Document ........................................ 31
2.2.3 Authorization ................................................................................................ 36

2.3 Phase II - Curriculum Development ................................................................. 36

2.3.1 Overview....................................................................................................... 36
2.3.2 Job Task Analysis (JTA)............................................................................... 37
2.3.3 Job Task List ................................................................................................. 38
Fig 2–2: Sample Pilot Job Task Listing....................................................................... 40

3
 2.3.4 Learning Analysis ......................................................................................... 41
2.3.5 Job Task Analysis Document........................................................................ 41
2.3.6 Qualification Standards................................................................................. 42
2.3.7 Proficiency Objectives .................................................................................. 42
2.3.8 Task Factors Analysis:.................................................................................. 43
Fig 2-3: AQP Continuing Qualification Critical/Currency Chart................................ 45
2.3.9 Evaluation Strategy....................................................................................... 45
2.3.10 Consolidation of Objectives.......................................................................... 46
2.3.11 Conditions, Contingencies, and Media ......................................................... 46
2.3.12 Qualification and Continuing Qualification Curricula ................................. 46
2.3.13 Qualification Standards Document ............................................................... 47
Fig 2-4: RATING SCALE EXAMPLE - First-Look, Manoeuvres Validation, Line
Operational Evaluation, or Online Evaluation.............................................................. 49
2.3.14 Instructional Systems Development (ISD) Methodology Document ........... 52
Fig 2-5: Curriculum Development............................................................................... 53
2.3.15 Curriculum Outline Document ..................................................................... 54
Fig 2-6: Sample AQP Curriculum Outline .................................................................. 56
Fig 2-7: EXAMPLE - Performance Difference Rating Scale...................................... 57
2.3.16 Implementation and Operations Plan (I&O) Document ............................... 58
2.3.17 Approval ........................................................Error! Bookmark not defined.

2.4 Phase III: Implementation ................................................................................. 64

2.4.1 Overview....................................................................................................... 64
2.4.2 Authorization ................................................................................................ 65

2.5 Phase IV: Initial Operations .............................................................................. 65

2.5.1 Overview....................................................................................................... 65
2.5.2 Phase IV Activities ....................................................................................... 66
2.5.3 Revisions to Training Programs and AQP Documents................................. 66
2.5.4 Authorization ................................................................................................ 66

2.6 Phase V: Continuing Operations...................................................................... 67

2.6.1 Overview....................................................................................................... 67
2.6.2 Data’s Impact on Continuing Operations ..................................................... 67
2.6.3 Quality Assurance......................................................................................... 67

2.7 Document and Reporting Requirements .......................................................... 68

2.7.1 Overview....................................................................................................... 68
Fig 2–8: Document Structures Chart ........................................................................... 70

CHAPTER 3 - AQP DOCUMENTATION APPROVAL PROCESS ....................71

3.1 Documentation Review and Approval ............................................................. 71

3.1.1 Overview....................................................................................................... 71
3.1.2 Review Process ............................................................................................. 71
3.1.3 Approval Process .......................................................................................... 71

4
3.2 Transport Canada Approval Procedures ......................................................... 72
3.2.1 Phase I - AQP Application............................................................................ 72
3.2.2 Phase II – Curriculum Development............................................................. 72
3.2.3 Phase III - Implementation............................................................................ 73
3.2.4 Phase IV- Initial Operations.......................................................................... 76
3.2.5 Phase V - Continuous Operations ................................................................. 78

3.3 Withdrawal Of Authorization .......................................................................... 78

CHAPTER 4 - CURRICULA AND CERTIFICATION..........................................79

4.1 Curricula.............................................................................................................. 79
4.1.1 Background ................................................................................................... 79

4.2 Qualification Curriculum................................................................................... 81

4.2.1 Ground Qualification Training Activities..................................................... 81
4.2.2 Simulation/Flight Qualification Training Activities..................................... 81
4.2.3 Initial Operating Experience (IOE)............................................................... 81
4.2.4 Special Qualification Training...................................................................... 82
4.2.5 Special Purpose Operations Training (SPOT) .............................................. 82
4.2.6 Validation/Evaluation/Remediation.............................................................. 82
Fig 4-2: AQP Training, Validation, & Evaluation....................................................... 83
Fig 4-3: AQP Validation/Evaluation Table ................................................................. 84
4.2.7 Planned Hours............................................................................................... 87

4.3 Continuing Qualification Curriculum .............................................................. 89

4.3.1 First-Look Manoeuvres................................................................................. 89
4.3.2 Training Activity........................................................................................... 90
4.3.3 Validation/Evaluation/Remediation.............................................................. 91
4.3.4 Flight Crewmember Recent Experience ....................................................... 92
4.3.5 Cycles and Evaluation Period ....................................................................... 92

4.4 Secondary Curricula........................................................................................... 96

4.4.1 Transition Curriculum................................................................................... 96
4.4.2 Upgrade Curriculum ..................................................................................... 97
4.4.3 Requalification Curriculum .......................................................................... 97
4.4.4 Difference Curriculum .................................................................................. 98

CHAPTER 5 - INSTRUCTORS AND EVALUATORS ........................................99

5.1 Instructor/Evaluator Curricula......................................................................... 99

5.1.1 General.......................................................................................................... 99
5.1.2 Instructor/Evaluator Curricula ...................................................................... 99
5.1.3 Definitions................................................................................................... 100
5.1.4 Qualification Training................................................................................. 100
5.1.5 Qualification – Evaluation and Observation............................................... 103
5.1.6 Continuing Qualification – Training, Evaluation and Observation ............ 103

5
 5.1.7 Instructor and Evaluator CRM Training and Evaluation............................ 104

5.2 Type, Qualification and Currency Requirements for Evaluators................ 104

5.3 Quality Assurance and Standardization......................................................... 104

5.3.1 Quality Assurance....................................................................................... 104
5.3.2 Standardization ........................................................................................... 105

CHAPTER 6 - AQP DATA MANAGEMENT.....................................................106

6.1 Introduction............................................................................................................. 106

6.1.1 Background ................................................................................................. 106
6.1.2 Definitions................................................................................................... 106

6.2 Data Management................................................................................................... 107

6.2.1 Overview..................................................................................................... 107
6.2.2 Data Collection ........................................................................................... 107
6.2.3 Data Entry ................................................................................................... 108
6.2.4 Data Submission ......................................................................................... 108
6.2.5 Data Analysis ............................................................................................. 117

CHAPTER 7 - CREW RESOURCE MANAGEMENT (CRM)............................118

7.1 Introduction....................................................................................................... 118

7.1.1 Overview..................................................................................................... 118

7.2 Integrating CRM into an AQP ....................................................................... 118

7.2.1 Scope of Integration.................................................................................... 118
Fig 7-1: Sample Phase Independent CRM Skills....................................................... 119
7.2.2 CRM and the AQP Task List ...................................................................... 121
7.2.3 CRM Knowledge and Skills ....................................................................... 121
7.2.4 CRM and Proficiency Objectives ............................................................... 122
7.2.5 Training Events........................................................................................... 122
7.2.6 Curriculum Design...................................................................................... 125
7.2.7 Line Operations and Proceduralized CRM ................................................. 126

7.3 Evaluating CRM .............................................................................................. 126

7.3.1 Overview..................................................................................................... 126
7.3.2 Observable Behaviours ............................................................................... 126

7.4 CRM in Line Operations.................................................................................. 127

7.4.1 Supplemental Feedback .............................................................................. 127
7.4.2 Corporate Culture........................................................................................ 127

APPENDIX A - AQP DOCUMENTATION .................................................129

AQP Document Requirements..................................................................................... 129

6
Minimum AQP Documentation Requirements for the Program Audit Database . 130

APPENDIX B - AQP DOCUMENTATION CHECKLIST ............................132

Application/Administration Document....................................................................... 132

Job Task Analysis (JTA) .............................................................................................. 136

Qualification Standards................................................................................................ 143

Instructional Systems Development Methodology..................................................... 148

AQP Curriculum Outline............................................................................................. 150

Implementation and Operations Plan......................................................................... 152

APPENDIX C - AQP DOCUMENTATION CHECKLIST AND REVIEW JOB

AID TABLES ............................................................................................155

7
 ACRONYMS
(Used throughout this document)

ACI: Air Carrier Inspector

ACP: Approved Check Pilot
AFM: Aircraft Flight Manual
AOM: Aircraft Operating Manual
AQP: Advanced Qualification Program
ATA: Air Transport Association
CARs: Canadian Aviation Regulations
CASS: Commercial Aviation Services Standards
CBA: Commercial and Business Aviation
CBT: Computer Based Training Device
COM: Company Operations Manual
EPO: Enabling Performance Objectives
CQ: Continuing Qualification
CRM: Crew Resource Management
CRP: Cruise Relief Pilot
CS: Cognitive Skills
EO: Enabling Objective
ETOPS: Extended Twin Engine Operations
FAA: Federal Aviation Administration
FBS: Fixed Based Simulator
FCOM: Flight Crew Operations Manual
FCTM: Flight Crew Training Manual
FFS: Full Flight Simulator
FL(M): First-Look (Manoeuvres)
FMS: Flight Management System
FOQA: Flight Operations Quality Assurance
FTD: Flight Training Device
I/E: Instructor/Evaluator
I&O: Implementation and Operation
IOE: Initial Operating Experience
IOETC: Initial Operating Experience Training Captain
ILS: Instrument Landing System
IRR: Inter Rater Reliability
ISD: Instructional System Development
JTA: Job Task Analysis
KSA: Knowledge, Skills and Attitudes
LOE: Line Operational Evaluation
LOFT: Line Oriented Flight Training
LOS: Line Operational Simulations
MATS: Master AQP Transition Schedule

8
MPV: Manoeuvres Proficiency Validation (for Qualification Curriculum)
MT: Manoeuvres Training
MTV: Manoeuvres Training and Validation (for Continuing Qualification)
MV: Manoeuvres Validation (means the same as MPV/MTV but is abbreviated
for data entry purposes)
NAVAID: Navigational Aid
NDB: Non-Directional Beacon
NTSB: National Transportation Safety Board
ODR: Operator Difference Requirement
OE: Online Evaluation
PADB: Program Audit Database
PF: Pilot Flying
PIC: Pilot-in-Command
PM AQP: Program Manager, AQP
PNF: Pilot Not Flying
POI: Principle Operations Inspector
PPC: Pilot Proficiency Check
PPDB: Performance/Proficiency Data Base
PS: Psychomotor Skills
PV: Procedures Validation
QA: Quality Assurance
QAE: Quality Assurance Evaluator
QAI: Quality Assurance Instructor
QC: Qualification Curriculum
RMCBA: Regional Manager, Commercial and Business Aviation
RRLOE: Rapid Reconfigurable Line Operational Evaluation
RRR: Referent Rater Reliability
SGT: Small Group Try-Outs
SIC: Second - in – Command
SKV: Systems Knowledge Validation
SME: Subject Matter Expert
SMS: Safety Management System
SOP: Standard Operating Procedures
SV*: System Validation (note: means the same as SKV but shortened for data
entry purposes)
SPO: Supporting Proficiency Objectives
SPOT: Special Purpose Operations Training
TC: Transport Canada
TEM: Threat and Error Management
TPO: Terminal Proficiency Objectives
VOR: Very High Frequency Omni-directional Radio

9
 DEFINITIONS

The following terms are used throughout this document and are defined as follows:

ADVANCED QUALIFICATION PROGRAM (AQP): A voluntary program and

alternative method of training, evaluating and qualifying flight crewmembers, instructors
and evaluators, that uses a systematic methodology for developing proficiency-based
training and evaluation programs in lieu of traditional training programs.

ANONYMOUS DATA: Data that cannot be identified with a named individual. Also
referred to as DE-IDENTIFIED DATA.

APPLICANT: An air operator that applies to conduct training and evaluation under an
AQP.

ATTITUDE: A persisting internal mental state that influences an individual’s choice of

personal action toward some object, person or event.

COGNITIVE SKILLS (CS): Those intellectual skills that are prerequisite to the
performance of a task, sub-task, element or sub-element. The three primary categories of
cognitive skill are discrimination, concept learning and rule using.

CONDITION: One of the three primary components of a proficiency objective:

performance, condition and standard. The conditions describe the range of circumstances
under which student performance will be measured and evaluated. Conditions may
include the natural environment (ceiling, visibility, wind, turbulence, etc.), the
operational environment (unserviceable navigational aid (NAVAID), birds, conflicting
air traffic, gate change, passengers not seated, etc.) and operational contingencies
(abnormal situations and emergencies).

COMPUTER-BASED TRAINING: Classroom instruction that is performed

individually by trainees at a computer stations.

CONTINUING QUALIFICATION (TRAINING/PROGRAM): Training that follows

initial qualification on a recurrent basis.

CONTINUING QUALIFICATION CYCLE: The time period during which training

and evaluation on all proficiency objectives have been accomplished by all flight
crewmembers, instructors or evaluators as applicable.

COURSEWARE: All instructional material that a candidate requires to complete a

curriculum, in whatever media required, including manuals, visual aids, lesson plans,
flight event descriptions, computer software programs, audio-visual programs,
workbooks, handouts, etc.

10
CREW RESOURCE MANAGEMENT (CRM): The effective use of all available
resources - human resources, hardware, and information - to achieve safe and efficient
flight.

CRITICALITY: A terminal proficiency or supporting objective for which the

substandard task performance would adversely affect safety. The relative need for
awareness, care, exactness, accuracy or correctness during task performance. Critical
tasks must be accomplished more frequently in training and evaluation than non-critical
tasks. All critical tasks must be accomplished during each evaluation period.

CURRENCY: A terminal proficiency or supporting objective for which individuals

and/or crews can maintain proficiency by repeated performance of the item in normal line
operations. For pilots, most currency items may be validated during Online Evaluations
(OE), while most non-currency items must be demonstrated during training, validation
and evaluation events in a simulator or Flight Training Device (FTD).

CURRICULUM: A portion of an AQP that covers one of two program areas:

Qualification and Continuing Qualification. Qualification and Continuing Qualification
program areas may include upgrade, transition and re-qualification curricula. The
Qualification and Continuing Qualification Programs address the required training and
qualification activities for each specific make, model, and series aircraft (or variant) and
for a specific duty position.

CURRICULUM DESIGN: The activities involved in organizing, clustering,

sequencing and otherwise structuring the elements of instruction (objectives, lessons,
evaluations, etc.) into an orderly flow of learning experiences to facilitate student
performance.

CURRICULUM OUTLINE: The document that organizes training objectives into

curricula, segments, modules, lessons, lesson elements, etc.

CURRICULUM SEGMENT: An integral part of a curriculum, which can be separately

evaluated and individually authorized, but by itself does not qualify a person for a duty
position. (e.g., ground training segment, flight training segment and evaluation segment).
The first level of curriculum detail (Segment, Module, Lesson, Lesson Element).

DE-IDENTIFIED DATA: Data that cannot be identified with a named individual. Also
referred to as ANONYMOUS DATA.

DUTY: All the actions (tasks, sub-tasks, etc.) required by one's position or occupation.

DUTY POSITION: The operating position of a flight crewmember, or other person.

Duty positions include Captain, First Officer, Cruise Relief Pilot, Second Officer, Flight
Engineer, Instructor or Evaluator.

11
ELEMENT: A component of training analysis or design. In the case of task analysis,
the element may be used as a level of analysis: phase of flight, task, sub-task, element,
sub-element, etc. In the case of curriculum design, the element may be used as a level of
curriculum organization: curriculum, segment, module, lesson, lesson element, etc.

ENABLING OBJECTIVE (EO): An instructional objective created at the level of an

element, skill, knowledge, or attitude. Describing the functions of the hydraulic system
would be an example. EOs are lower level learning objective that helps students master a
higher level objective, such as a Supporting or Terminal Level Objective. The
knowledge and skill prerequisites of manoeuvres and procedures are usually trained as
Enabling Proficiency Objectives (EPO).

EVALUATION: Careful appraisal of an individual’s performance by an evaluator to

ascertain whether the standards required for a specified level of proficiency have been
demonstrated.

EVALUATION OF PROFICIENCY: In AQP, this is called a Line Operational

Evaluation (LOE).

EVALUATION PERIOD: A period within the Continuing Qualification Cycle during

which each person must receive at least one training session and an LOE. All Critical
Terminal Proficiency Objectives (TPO) are trained and evaluated.

EVALUATOR: A person delegated by the Minister, who has satisfactorily met

approved AQP evaluator training and evaluation requirements that qualify that person to
evaluate the performance of flight crewmembers, instructors, or other evaluators.

EVENT: A training or evaluation situation comprised of a task or sub-task to be

performed by the crew under a specified set of conditions.

EVENT SET: A relatively independent segment of a scenario made up of several

events, including an event trigger, possible distracters, and supporting events.

FILL-IN or “SEAT FILLER”: A qualified crewmember who substitutes for a

candidate who is unable to attend an evaluation session, thus allowing the rest of that
candidate’s crew to complete their evaluation with a full crew complement.

FIRST-LOOK: The performance and assessment of specific tasks, procedures or flight

manoeuvres in accordance with approved program documentation, as a means of
assessing performance and proficiency on designated tasks, procedures or flight
manoeuvres before any briefing or training has taken place, in order to determine trends
of degraded proficiency, if any, within the fleet’s flight crewmember group as a whole.

FLIGHT OPERATIONS QUALITY ASSURANCE (FOQA): A program that

receives and analyzes input of flight operations and noted problems in order to identify
and to reduce future occurrences of those problems.

12
FLIGHT TRAINING: Training given in the aircraft, flight simulator, Flight Training
Device (FTD), or other cockpit environment. See GROUND TRAINING.

FLIGHT TRAINING DEVICE (FTD): A full-scale replica of an airplane cockpit that

may not have the motion or visual systems associated with flight simulators.

FLIGHT TRAINING EQUIPMENT: Aircraft and those FTDs or flight simulators that
are used for any of the following purposes: (1) required evaluation of individual or crew
proficiency; (2) training activities that determine if an individual is ready for an
evaluation; (3) activities used to meet recency of experience requirements; and (4) Line
Operational Simulations (LOS).

FORMATIVE EVALUATION: Process of reviewing courseware for technical

accuracy, instructional soundness, and suitability for use by instructors, evaluator and
students. Dry run of the total curriculum with a small group of students to test the
effectiveness and efficiency of the training (e.g., small group try-out).

FREQUENCY: Number of occurrences of a task/sub-task in a specific period of duty

(one flight, one trip, one month, one year, etc.). How often a task/sub-task is performed.
Frequency may be used to determine currency (see CURRENCY) by comparing the
frequency with which activities occur on the line, with the frequency required to maintain
proficiency without additional training.

FRONT END ANALYSIS: A generic term for any process used to identify the learning
needs of a student population. May include needs analysis, job analysis, task analysis,
student entry behaviour analysis, performance analysis, competency analysis, etc.

GROUND TRAINING: Aviation/aircraft specific training provided in a classroom,

learning centre, lecture hall or other traditional educational setting that occurs outside the
cockpit environment.

INSTRUCTIONAL ANALYSIS: A process conducted during the design of instruction

to identify the presentational components, or learning events, necessary for the student to
master the complete range of skills, knowledge, attitudes, abilities, and CRM factors
required for proficient performance.

INSTRUCTIONAL SYSTEMS DEVELOPMENT: A systematic methodology for

deriving and maintaining qualification standards and associated curriculum content based
on a documented analysis of the job tasks, skills, and knowledge required for job
proficiency.

INTER-RATER RELIABILITY: A program that is conducted periodically to

“calibrate” instructors and evaluators, so that they will rate performance as closely to the
same standard as possible.

JOB: A job is the summation of the functions, identified as tasks and sub-tasks,
performed by an individual.

13
KNOWLEDGE: Specific information required enabling a student to develop the skills
and attitudes to effectively recall facts, identify concepts, apply rules or principles, solve
problems, and think creatively. Because knowledge is covert, students must be assigned
overt activities to demonstrate their knowledge base.

LESSON: A meaningful division of learning consistent with the method of study,

learning, or testing of performance (proficiency) objectives. The third level of
curriculum definition (Segment, Module, Lesson, Element). Lessons usually contain
objectives, training events, student materials, instructor materials, and an evaluation
scheme or form.

LESSON ELEMENT OR TOPIC: A subgroup of activities within a lesson. It is the

fourth level of curriculum detail (Segment, Module, Lesson and Lesson Element).

LICENSING EVENT: An event required for licensing action during a Qualification

Curriculum. The Qualification Standards for all pilot programs will designate those
manoeuvres, procedures and events that must be trained and evaluated as a pre-condition
for pilot licensing.

LINE OPERATIONAL EVALUATION (LOE): The LOE is the primary proficiency

evaluation. This evaluation addresses the individual’s ability to demonstrate technical
and CRM skills appropriate to fulfilling job requirements in a full mission scenario
environment. The intent of an LOE is to evaluate and verify that an individual’s job
knowledge, technical skills and CRM skills are commensurate with AQP Qualification
Standards. The LOE is conducted in a simulation device approved for its intended use in
the AQP.

LINE ORIENTED FLIGHT TRAINING (LOFT): A Line Operational Simulation

(LOS) flight scenario designed for training purposes to provide practice in the integration
of technical and CRM skills. LOFT is conducted using a complete cockpit flight crew to
the maximum extent feasible and is accomplished in a Transport Canada approved
simulation device. A LOFT training session is not interrupted by the instructor, unless
negative learning begins to occur.

LINE OPERATIONAL SIMULATION (LOS): LOS is a simulator or FTD training

session conducted in a “line environment” setting. LOS includes LOFT, LOE and
Special Purpose Operational Training (SPOT). Instruction and training is based on
learning objectives, behavioural observation, assessment of performance progress and
instructor debriefing or critique (feedback). LOS implies that flight crewmembers are
trained to proficiency. However, in the LOE, crew performance and CRM are formally
evaluated.

MANOEUVRE VALIDATION: A simulator session in which specific manoeuvres are

performed and evaluated to proficiency.

14
MEDIA: Physical means for providing the instructional content and experience to the
student. Includes entire set of instructional presentation materials; e.g., workbook,
videotape, overheads, Computer Based Training Device (CBT), mock-ups, FTDs,
simulators, etc.

MODULE: A group of subject matter under a specific curriculum segment. Second of

four curriculum levels of detail (Segment, Module, Lesson, Element). Often corresponds
to a day of training or a device event, such as FTD #3 or simulator #6.

MOTOR SKILL: Physical actions required to perform a specific task (sub-task or

element). Students have acquired a motor skill not when they can simply perform a
prescribed procedure, but when their movements are smooth, regular and precisely timed.
Those hands-on skills that are prerequisite to the performance of a task, sub-task, element
or sub-element.

OBSERVABLE BEHAVIOUR: A behaviour whose occurrence during the

performance of an event is an indicator that the crew is handling the event properly.
Observable behaviours are one part of the performance standards identified for each
event. See PERFORMANCE STANDARD.

ONLINE EVALUATION: An evaluation conducted by a qualified evaluator during

normal flight operations that assesses the candidate’s proficiency with respect to the
particular aircraft, crew position and type of operations, and his or her skill and ability to
operate effectively as part of a crew.

PERFORMANCE/PROFICIENCY DATABASE (PPDB): A database that collects

results of performance and proficiency evaluations, and which is used to analyze training
programs, to spot developing trends, and to correct any problems that may be noted.

PERFORMANCE STATEMENT: One of the three components of an objective. A

statement of physical and/or cognitive activities which, when executed or carried out, will
complete the work required for a specific portion of a job (in the case of a proficiency
objective), or the activities required of a learning goal (in the case of a learning
objective). See PROFICIENCY OBJECTIVE.

PHASE OF FLIGHT: The standard high-level set of activities performed by pilots on

all operational flights. For example: Pre-flight, Engine Start, Pushback, Taxi, Take-off,
Climb, Cruise, Descent, Holding, Approach, Landing, Taxi and Post Flight Operations.

PROFICIENCY OBJECTIVE: A statement describing the behaviour that the

candidate must be able to demonstrate on the job. Each objective must specify precisely
what behaviour must be exhibited (the performance statement), the conditions under
which the behaviour will be accomplished (condition statement), and the minimum
standard of acceptable behaviour (standard or criterion statement). A learning objective
(usually an EO) can be demonstrated in a classroom or academic type setting, while a

15
performance objective (usually a terminal or supporting proficiency objective (SPO))
must be demonstrated in an environment equivalent to the operational environment.

PROGRAM AUDIT DATABASE (PADB): A database that is used to analyze the

elements of a training program and the supporting task analysis that must be
accomplished during any training cycle. It may develop lesson plans and be used to
address deficiencies found in performance and proficiency by the PPDB
(performance/proficiency database).

QUALIFICATION STANDARDS: The terminal and SPOs coupled with test and
evaluation strategies (where, how and by whom qualification is measured). Qualification
Standards and previous experience provide the baseline of mastery for the duty position.
Demonstration that an individual has met certain or all of these standards may lead to
certification.

QUALITY ASSURANCE EVALUATOR (QAE): A Quality Assurance Evaluator

(QAE) is an experienced AQP Evaluator, who has been designated by the air operator to
perform quality assurance functions for the air operator’s AQP evaluation programs.
Their duties include monitoring AQP Evaluator nominees and AQP Evaluators. The air
operator may utilize other terms such as “Evaluator Mentor” for individuals acting in this
role.

QUALITY ASSURANCE INSTRUCTOR (QAI): A Quality Assurance Instructor

(QAI) is an experienced AQP Instructor, who has been designated by the air operator to
perform quality assurance functions for the air operator’s AQP training programs. Their
duties include monitoring AQP instructor candidates and AQP instructors. The air
operator may utilize other terms such as “Instructor Mentor” for individuals acting in this
role.

SEAT FILLER: See FILL-IN above

SIMULATOR: A full sized replica of a specific type of airplane cockpit, including both
visual and motion systems.

SKILL: An ability to perform an activity or action. Often divided into motor/hands-on

and cognitive categories.

SPECIAL PURPOSE OPERATIONAL TRAINING (SPOT): A portion of a Line

Operational Simulation (LOS) training scenario consisting of flight tasks selected from
any phase or phases of flight to provide practice in the integration of technical and CRM
skills appropriate to the selected flight tasks. SPOT is conducted using a complete
cockpit flight crew to the maximum extent feasible and is accomplished in a simulation
device.

16
SPECIAL TRACKING: A system of monitoring the proficiency of an individual at
scheduled intervals. It may be applied to individuals that have failed to demonstrate
proficiency during an evaluation (LOE) or as required.

STANDARD OF PERFORMANCE: Observable, measurable parameters of

performance with tolerances; e.g., course deviation degrees, + or -. Includes procedures,
manoeuvres, and observable behaviours.

SUB-ELEMENT: A subcomponent of an element. See ELEMENT.

SUB-TASK: Specific separate step or activity required in the accomplishment of a task.

May also refer to categories of a task (e.g., Non-precision approach –Very High
Frequency Omni-directional Radio (VOR), Non-directional Beacon (NDB), Localizer
(LOC) etc.).

SUMMATIVE EVALUATION: Training program evaluation accomplished in a full

operational setting. Usually accomplished during the first full increment of classes with a
full student complement.

SUPPORTING PROFICIENCY OBJECTIVE (SPO): A proficiency objective

created at the sub-task level. For example: Perform Engine-Out Precision Approach
Preparation Procedures.

TASK: A task is a unit of work within a function having an identifiable beginning and
ending point, which results in a measurable product. An example of a task applicable to
AQP: perform a normal take-off.

TECHNICAL SKILLS: Within an AQP, technical skills refer to those manoeuvres,

procedures and other behaviours that have a high psychomotor component, while CRM
skills refer to those communication, decision-making and workload management
behaviours that have a high cognitive component.

TERMINAL PROFICIENCY OBJECTIVE (TPO): A proficiency objective created

at the task level. For example: perform an engine-out precision approach.

TPO/SPO HIERARCHY: The hierarchy of all TPOs and SPOs organized by phase of
flight in the Transport Canada Model AQP database.

TRAINING SESSION: A contiguously scheduled period of time devoted to training

activities at a facility acceptable to Transport Canada for that purpose.

TRAINING TO PROFICIENCY: Training to a performance level that meets or

exceeds a Qualification Standard. This concept must include enough repetition and
practice to ensure that each individual can perform at the Qualification Standard level
over the entire evaluation period or Continuing Qualification Cycle.

17
TRIGGERING CONDITIONS: The conditions whose occurrence defines the
beginning of an event.

VALIDATION: A determination that required results with regards to performance

objectives or the methods and procedures for development, implementation and
maintenance of training systems were produced.

VARIANT: An aeroplane or a group of aeroplanes sharing similar characteristics but

having pertinent differences from a base aeroplane. Pertinent differences are those that
require different or additional flight crewmember knowledge, skills and/or abilities that
affect flight safety.

18
Chapter 1 - AQP Introduction and Overview

1.1 Introduction

1.1.1 Statement of Purpose

This document provides guidance for authorization of an Advanced Qualification

Program (AQP). An AQP is an alternate method of training, evaluating, qualifying, and
certifying, to ensure the competency of pilots, flight crewmembers, instructors, and
evaluators subject to the training and evaluation requirements of Subparts 702, 703, 704
and 705 of the Canadian Aviation Regulations (CARs).

AQP employs a systematic methodology for developing the content of training programs
for air operator. It replaces the traditional training program with a proficiency-based
training and evaluation program. This proficiency-based program is derived from a
detailed task analysis that includes Crew Resource Management (CRM). AQP
encourages innovation in the methods and technology that are used during instruction and
evaluation. It also encourages the efficient management of training systems. A leading
objective of AQP is to provide effective training that will enhance professional
qualifications to a level above the present standards. The goal of AQP is to achieve the
highest possible standards of individual and crew performance.

This document has been developed as a Canadian equivalent to the Federal Aviation
Administration (FAA) Advisory Circular (AC) 120-54A, Advanced Qualification
Program.

1.1.2 History

In 1975, the FAA began to deal with two issues: Hardware requirements needed for total
simulation and the redesign of training programs to deal with increasingly complex
human factors problems. At the request of the air transportation industry, the FAA
addressed the hardware issue first. This effort culminated in 1980 in the development of
the Advanced Simulation Program. Since then, the FAA has continued to pursue
approaches for the redesign of training programs to increase the benefits of advanced
simulation and to address the increasing complexity of cockpit human factors.

In 1987 a Join Government-Industry Task Force on Flight Crew Performance was

created. The task force met at the Air Transport Association’s (ATA) headquarters to
identify and discuss flight crewmember performance issues. Working groups in three
major areas were formed: (1) man/machine interface; (2) flight crewmember training; and
(3) operating environment. Each working group submitted a report and recommendations
to the joint task force.

19
In June of 1988, the National Transportation Safety Board (NTSB) issued a Safety
Recommendation (A-88-71) on the subject of CRM training. The recommendation was
that all part 121 carriers review initial and recurrent flight crew training programs. The
purpose of this review was to ensure that the training programs include simulator or
aircraft training exercises that involve cockpit resource management and active
coordination of all crewmembers under training. It should also permit evaluation of crew
performance and adherence to proper crew coordination procedures.

In response to the recommendations from the joint task force and from the NTSB, the
FAA, on October 2, 1990, published SFAR 58 - AQP, which addressed the majority of
the above recommendations. AQP was also established to permit a greater degree of
regulatory flexibility in the approval of innovative pilot training programs. Based on a
documented analysis of operational requirements, an air operator under AQP may
propose to depart from traditional practices with respect to what, how, when, and where
training and testing is conducted. This is subject to FAA approval of the specific content
of each proposed program. SFAR 58 required that all departures from traditional
regulatory requirements be documented and based upon an approved continuing data
collection process sufficient to establish at least an equivalent level of safety. AQP
provides a systematic basis for matching technology to training requirements and for
approving a training program with content based on relevance to operational
performance.

When the CARs were initially drafted, a provision was made to include training programs
approved as AQP. A similar provision has been incorporated into the Commercial Air
Service Standards (CASS).

The Canadian AQP regulatory framework generally conforms to the accepted concept of
AQP, as outlined in FAR Part 121, Subpart Y, which supersedes SFAR 58 and AC 120-
54A. These FAA AQP standards have been used as the basic model for the Canadian
AQP standards. Canadian AQP standards also have particular requirements based on the
Canadian operational requirement and regulatory framework.

1.1.3 Background

The capabilities and use of simulators and other computer-based training devices in
training and qualification activities have changed dramatically. AQP regulatory
requirements and this document allow an air operator to develop innovative training and
qualification programs that incorporate the most recent advances in training methods and
techniques. These training and evaluation applications are now grouped under the
general term of Line Operational Simulation (LOS). These include Line Oriented Flight
Training (LOFT), Special Purpose Operational Training (SPOT), and Line Operational
Evaluation (LOE).

Due to the role of CRM in preventing fatal accidents, it has become evident that training
curricula should develop and evaluate both technical and CRM skills. In AQP, a
structured LOS design process is employed to specify and integrate the required CRM
and technical skills into LOS scenarios. The design methodology used to design LOS

20
scenarios must be approved by the Transport Canada division responsible for operational
oversight, such as National Operations – Airlines Division.

1.1.4 Benefits of AQP

Although AQP is a voluntary program, Transport Canada encourages air operators to

participate. AQP provides for enhanced curriculum development and a data-driven
approach to quality assurance along with the flexibility to target critical tasks during
aircrew training. The AQP methodology directly supports the Transport Canada’s goals
for safety enhancement. The primary goal of AQP is to achieve the highest possible
standard of individual and crew performance. In order to achieve this goal, AQP seeks to
reduce the probability of crew-related errors by aligning training and evaluation
requirements more closely with the known causes of human error. For example:

A. Crew Performance

Most accidents are caused by crew issues. Traditional training programs focus on
individual training and evaluation. Under AQP, the focus on crew performance in
both training and evaluation is significantly enhanced.

B. Crew Resource Management

Most accidents are caused by errors of judgement, communication and crew co-
ordination, while traditional training programs focus primarily on flying skills and
systems knowledge. Under AQP, competence in flying skills and systems
knowledge are integrated with CRM skills in training and evaluation throughout
the curriculum.

C. Scenario-Based Training And Evaluation

Most accidents are caused by a chain of errors that build up over the course of a
flight and which, if undetected or unresolved, result in a final, fatal error.
Traditional training programs, with their manoeuvre-based training and
evaluation, artificially segment simulation events and prevent the realistic build-
up of the error chain. Under AQP, both training and evaluation are scenario-
based, simulating more closely the actual flight conditions known to cause most
fatal carrier accidents.

D. Additional Benefits

Added benefits that are expected for individual air operators will vary, but may
include:

a) The ability to modify training curricula, media and intervals.

b) Crew evaluation as well as individual assessment.

21
 c) Improved standardization across fleets and flight personnel.

d) Shift from programmed hours to proficiency-based training.

e) Access to innovative training ideas and research.

f) Opportunity to achieve more efficient training.

1.2 AQP Overview

1.2.1 General

AQP integrates a number of training and evaluation features that are aimed at improving
performance relative to traditional training programs. An AQP is a systematically
developed, continuously maintained, and empirically validated proficiency-based training
system. AQP allows an air operator to systematically develop, implement, evaluate and
maintain a training program that will be self-correcting.

A. Phases of Development

There are five phases for developing, implementing and maintaining an AQP. A
description of each phase is located in Chapter 2 of this document. Each phase
must be reviewed by Transport Canada before the applicant may be authorized to
proceed to the next phase. Each phase consists of specific activities, including the
documentation of those activities, which must also be reviewed and authorized by
Transport Canada.

B. AQP Documentation

In addition to the supporting documents and manuals which must be provided to

Transport Canada in traditional training programs, there are six documents and an
annual report requirement that are unique to AQP. These documents are
instrumental in managing the AQP and must be maintained throughout the life of
the program. These documents are categorized by their function and
interrelationship into two groups: “management” and “database” documents. All
of the documents, combined with the annual reports comprise the Program Audit
Data Base (PADB). A brief description of the AQP documents appears below:

a) Management Documents: These documents include the

Application/Administration Document, Instructional Systems
Development (ISD) Methodology, and Implementation and Operations
(I&O) Plan. These documents are stand-alone in that a change in one will
not necessitate a change in another.

22
 b) Database Documents: These documents include the Task Analysis,
Qualification Standards and Curriculum Outlines. Because a change to
one document often requires a change in the others, they should be
maintained in an interactive database.

c) Annual Report: The purpose of the report is to identify changes to the

curricula, training equipment upgrades, and the AQP Maintenance
Strategy resulting from feedback and analysis of the information in the
Proficiency/Performance Data Base (PPDB).

Additional details of these AQP documents can be found in Appendices A and B.

1.2.2 Objectives of AQP

The following is a list of general objectives of AQP:

a) To improve safety by continuously improving training and evaluation.

b) To be responsive to continuing changes in industry, including new aircraft

technology, changing operational environments, and new training methods and
equipment.

c) To be responsive to continuing changes and best practices relative to training and

evaluation.

1.2.3 Characteristics of AQP

The following is a list of the general characteristics of AQP:

a) Participation is voluntary.

b) An AQP will employ innovative training and qualification concepts with the
regulatory flexibility to tailor training to individual company circumstances.

c) An AQP may build upon an existing training program or be completely new using
empirical performance data to drive curriculum changes.

d) Qualification is based on individual and team performance. Under AQP,

qualification is determined using progressive evaluations of proficiency
objectives. It is dependent on the structure and maintenance of all elements of the
program. These include: curriculum, facilities, training equipment, instructors,
evaluators, courseware and quality assurance.

e) Individual and team proficiency, and the AQP itself, will be empirically validated
by data collection and analysis.

23
f) Training will be systematically developed with an audit trail for all training and
data requirements.

g) The methods used for development, implementation and maintenance of program

operations will be continued throughout the life of the program.

1.2.4 Requirements of AQP

AQP is a voluntary program that requires a strong commitment from the air operator to
exceed minimum training standards in the greater interest of safety. To determine an
equivalent level of safety compared with a traditional training program, the entire AQP
must be examined as a comprehensive whole rather than considering any one component
in isolation. In order to assure that the AQP does establish an initial justification and a
continuing process to show an equivalent level of safety, mandatory requirements have
been established.

Under an AQP, Transport Canada monitors the process as well as the product. Instead of
basing curricula on prescribed generic manoeuvres, procedures and knowledge items,
AQP curricula are based on a detailed analysis of the specific job tasks, knowledge and
skill requirements of each duty position for the individual airline. Compared to
traditional training programs, the AQP process provides a systematic basis for
establishing an audit trail between training requirements and training methodologies.

AQPs have the following requirements:

a) All aspects of the program, as authorized by Transport Canada, must be complied

with.

b) The processes used for development, implementation and maintenance of

program operations will be maintained throughout the life of the program.

c) AQPs shall accommodate each specific make, model, and series aircraft (or
variant).

d) AQPs may build upon existing training programs or may be completely new.

e) AQPs must provide two basic types of curricula for every duty position. These
include:

i) Qualification; and

ii) Continuing Qualification.

Secondary curricula (transition, difference, upgrade, re-qualification, etc.) will

be derivatives from these basic types.

24
f) Duty positions covered must include all flight crewmember positions, instructors
and evaluators.

g) Air operators must provide justification, in a manner acceptable to Transport

Canada, that the proficiency-based qualification of personnel under AQP meets or
exceeds the requirements in Parts IV, VI and VII of the CARs as applicable.

h) Each of the training, evaluation and qualification requirements of Parts IV, VI and
VII of the CARs, which are not specifically addressed in the AQP, continue to
apply to the air operator.

i) Under AQP, the air operator must document the requirements of Parts IV, VI and
VII of the CARs, which would be met or replaced by an AQP curriculum.

j) Under AQP, the air operator must establish an initial justification and a
continuing process to show how the AQP curriculum provides an equivalent level
of safety for each requirement to be replaced.

k) AQP requires the use of a LOS methodology for both training and evaluation,
including LOS scripts reviewed and accepted by Transport Canada.

Note: The design methodology used to design LOS scenarios must be approved
by Transport Canada. Individual LOFT, SPOT and LOE scripts require review
and acceptance by Transport Canada. Alternatively, individual LOE scripts
developed through an approved system of script design methodology require
event set review. Once these event sets have been accepted by Transport Canada,
the entire script does not require acceptance. Among the possible script design
methodologies available are rapid reconfigurable LOS methodology (i.e., Rapid
Reconfigurable Line Operational Evaluations (RRLOEs).

l) An AQP must contain provisions for the training and evaluation of instructors and
evaluators.

m) Air operators must provide Transport Canada with access to performance data.
This will allow Transport Canada to validate training methods and the training
program.

n) Curricula must be based on an Instructional System Development (ISD)

methodology. This methodology must include a thorough analysis of the air
operator’s operations, aircraft, line environment and job functions.

o) AQPs must include a list of, and text that describes the knowledge requirements,
subject materials, job skills and Qualifications Standards of each task to be trained
and evaluated. These are contained in the Job Task Analysis (JTA) and
Qualification Standards.

25
p) AQPs must include a list of, and text that describes the supervised operating
experience, evaluation/remediation strategies, provisions for special tracking, and
how recency of experience will be accomplished. The details are contained in
Chapter 4.

q) All curricula will include planned hours for ground training, flight training,
evaluation, and operating experience. These planned hours will be derived from a
detailed task analysis that includes CRM.

r) Training and evaluation in Qualification and Continuing Qualification Curricula.

s) Curricula must be conducted in a crew or team environment. CRM must be

trained and evaluated throughout the program.

t) AQP must include LOS.

u) Flight crewmember proficiency evaluation will be accomplished though Line

Operational Evaluation (LOE).

v) Training and evaluation under AQP will integrate appropriate advanced flight
training equipment. FTDs and simulators will be used to support scenario-based
training as appropriate.

w) Air operators will develop data collection and analysis processes in order to
obtain performance information on crewmembers, instructors and evaluators.
This data will enable the air operator and Transport Canada to determine whether
the form and content of training and evaluation activities are satisfactorily
accomplishing the overall objectives of the curriculum.

x) Upon request, performance data will be provided to Transport Canada

electronically or in a paper report format that is acceptable to Transport Canada.

y) Air operators will provide Transport Canada with a Master AQP Transition
Schedule (MATS) that provides a plan to transition from a traditional program to
an AQP. In addition, the MATS should include a plan on how the applicant
would return to a traditional program should that become necessary or desirable at
some later date. This is not to imply that the traditional program must be kept
current once AQP is entered, but only how the applicant would update the
program and the time frame needed for implementation.

1.2.5 Developmental Assistance and Support

AQP applicants may avail themselves of a wide range of support organizations,

documents, and services in the development of their AQPs. The total inventory of such

26
support is constantly increasing, and the latest versions of all such support resources are
always available by contacting the air operator’s Principal Operations Inspector (POI).

1.2.6 Transport Canada Support

Support is available from the following Transport Canada units:

A. Transport Canada, Standards Branch, Certification and Operational

Standards Division (AARTF)

The Certification and Operational Standards Division (AARTF) is the office

responsible for the development of regulations, standards and guidance material
for AQP.

B. Transport Canada, National Operations Branch, Airlines Division

(AAROA) or regional Commercial & Business Aviation operational
oversight divisions

The Regional Offices of C&BA and the Airline Division (AAROA) (as
applicable) are responsible for ensuring regulatory requirements are met and
standards are maintained. In addition, these offices will provide oversight of the
entire operator’s AQP process and documentation.

C. Transport Canada, Principal Operations Inspector (POI)

The POI is responsible for ensuring that regulatory requirements are met and that
the AQP provides a satisfactory level of safety and pilot proficiency. The POI
will recommend and coordinate the approval and provide oversight of the AQP
documentation and program. The POI will recommend and coordinate the
authorization of any modifications and subsequent authorizations throughout the
life of the AQP. The POI will provide oversight of the instructor and evaluator
program and adherence to approved documentation for all AQP curricula.

1.2.7 Evaluation Review Team (ERT)

There are fives phases in AQP that are described in Chapter 2. Authorization to proceed
from one phase to the next is subject to the assessment of an Evaluation Review Team
(ERT). The ERT will include (but not necessarily be limited to) the following:

a) The POI (or a designated representative)

b) An AQP Specialist from National Operations – Airlines Division or from the

Regional Office responsible for operational oversight.

27
Chapter 2 - AQP Development Phases

2.1 Introduction

2.1.1 Phases

The development and implementation of an AQP is a five-phase process. This process

provides a structured building-block approach to program development. It results in a
program with fully documented curricula, supporting rationale and development
methodology. The use of this standardized AQP development process, along with its
documentation requirements, allows an AQP applicant to develop a training and
evaluation program based on analyzed and justifiable requirements. The documentation
from this five-phase process will then serve as the regulatory basis for promulgating the
AQP-developed training and evaluation programs.

Each phase is authorized sequentially and consists of specific activities and includes the
documentation of those activities. These documents must be submitted to Transport
Canada for approval. An applicant cannot exercise any training and evaluation
provisions permitted in Phases III, IV, or V until it has met all requirements of the
preceding phases to the satisfaction of Transport Canada. See figure 2-1 for the AQP
Development Process.

28
Fig 2-1: AQP Development Process

Phase I • Joint AQP Airline/TC Meeting

AQP Application • Develop AQP
Application/Administration

• Develop JTA
Phase II • Develop Qualification Standards
Curriculum Development • Develop Instructional Systems
Development (ISD) Methodology
• Develop Curricula
• Develop Implementation and Operations
Plan

Phase III • Train Instructors And Evaluators

Implementation • Conduct Small Group Try-outs
• Initiate Performance/Proficiency Data
Process
• Revise Program & Documents

• Conduct Initial Program Evaluation

Phase IV • Analyze Performance/Proficiency Data
Initial Operations Regularly – Submit Required Reports
• Revise Training Program and AQP
Documents
• Submit Annual Report

Phase V • Continual Training Program Evaluation

Continuous Operations • Analyze Performance/Proficiency Data
Regularly – Submit Required Reports
• Revise Program and AQP Documents
• Submit Annual Report
• TC Reviews & Authorizes

29
2.1.2 Instructional System Development (ISD)

The use of a systematic curriculum development methodology known as ISD has been
incorporated into the AQP process. Applicants may employ any of a wide range of
current ISD models. Alternatively, they may customize their own approach to curriculum
development. This chapter provides one acceptable methodology, but applicants are
encouraged to be creative in tailoring their own ISD approach to their requirements,
subject to Transport Canada approval. Innovation and practical application may result in
an equally acceptable AQP.

Because some ISD models are far more complex than others, Transport Canada has found
it useful to define its minimal ISD requirements. These are listed below:

a) Develop a job task listing.

b) Analyze the job task listing to determine essential skill and knowledge
requirements (either directly or by reference).

c) Determine which skill and knowledge requirements must be trained/tested.

d) Develop Proficiency Objectives that capture all training requirements.

e) Develop Qualification Standards that define acceptable operational performance

levels.

f) Develop tests that measure proficiency in skill and knowledge areas.

g) Provide instructional programs that teach and test training requirements.

h) Establish and maintain an audit trail of explicit links between task requirements,
training requirements, training and evaluation activities, and evaluation results.

i) Measure student performance against proficiency objectives and Qualification

Standards for all curricula.

j) Revise the training program based on student performance levels on an ongoing

basis. This de-identified data (stored in the PPDB) will be collected and reported
to Transport Canada on a regular basis.

2.1.3 Training Systems

While all applicants must meet the minimum requirements of the ISD approach, the
amount of effort each applicant should put into each step of their training analysis and
development depends on a number of factors. Applicants may use the ISD process to
build complete training systems from the ground up, to build a proficiency-based quality
control shell around an existing training system, or to make minor modifications to

30
existing proficiency-based curricula. Building a training system from the ground up for a
new aircraft will require far more depth of analysis and development than merely
modifying a current curriculum for an existing aircraft.

2.2 Phase I - Application

2.2.1 Joint Meeting

The applicant’s first step in establishing an AQP is sending a letter to the Principal
Operations Inspector (POI), stating their intent to develop an AQP. Transport Canada
will respond to the air operator with a letter, which will request a meeting with
representatives of the air operator’s training department. The meeting shall include the
POI, a representative from National Operations or from the Regional Office responsible
for operational oversight (as applicable), a fleet typed Subject Matter Expert (SME)
instructor (as applicable), and the air operator.

The focus of this meeting is to ensure that all parties understand AQP, specifically the
responsibilities and the commitment needed to complete the project. Discussions will
include the benefits of AQP, the development process requirements, AQP entry and exit
strategies, possible problem areas, and available tools.

2.2.2 The AQP Application/Administration Document

The purpose of the Application/Administration Document is to establish the applicant's

methodology and procedures for developing an AQP for some or all of its fleets, and for
its instructors and evaluators. The Application/Administration Document provides the
terms of reference under which the AQP will operate.

The Application/Administration Document is submitted when the air operator initially

applies for AQP program authorization. It is then updated as information warrants
changing (e.g., a changing transition schedule or adding new aircraft, etc.). The
application is one of the six documents unique to AQP that is maintained in a current
status throughout the life of the program and must have an acceptable revision control
methodology.

In order to establish the applicant's intent and approach for developing an AQP, the
Application/Administration Document shall thoroughly discuss the following topics:

A. Statement of Intent

The application should clearly state the air operator’s intent to develop,
implement, and operate an AQP. In addition, the statement of intent should
include:

31
 a) The specific concept, approach and methodology to be used for
developing the AQP. This will include the specific methods and
procedures for all steps;

Note: Applicants may either cite acceptance of the methodology proposed

in this document, or describe alternative approaches.

b) The specific concept, approach and methodology for implementing the

AQP;

c) How the AQP will be operated and maintained;

d) How CRM will be integrated and measured;

e) How the Transportation of Dangerous Goods training will be addressed

(as applicable);

f) How Security Training will be addressed; and

g) How LOS concepts will be integrated into both evaluation and training.

B. Applicant AQP Staff Organization

An applicant’s AQP staff may be comprised of personnel resources already

existing within the airline. These may include regular employees as well as
contractors. Generally, the staffing and expertise of an AQP applicant’s staff
should support the following functions:

a) AQP Manager - This individual is usually a management or supervisory

level person who is the company focal point for its AQP development
effort. This person is not only responsible for AQP leadership, but is the
primary contact with Transport Canada and any other external
organizations that may impact the applicant’s AQP. Experience has
shown that this position should be filled with an individual who
understands Job Tasks Analysis (JTA) and the CARs/CASS training
requirements the AQP is to address.

b) Subject Matter Expert (SME) - These are people that are current and
qualified on the applicable fleet type with varying levels of expertise that
represent the population of professionals the AQP will address. The
individuals may be called upon to act as liaison with operational support
personnel.

c) Document and Curriculum Developers - These individuals interface

with the AQP Manager and the SMEs to develop the requisite AQP
process, curriculum and instructor and evaluator documents.

32
 d) Document Managers - These individuals ensure AQP document control
and compliance with Transport Canada approval.

e) Computer Specialist/Database Manager - This individual provides

oversight to the development and management of the
performance/proficiency data acquisition and analysis system. In addition,
this individual could be used for other functions related to the facilitation
of an AQP.

C. Program Audit Database Master List (PADB)

This is a list of the six applicant-developed documents and the annual report that
are unique to AQP. All documents should be listed by title and have a brief
corresponding summary description. The six PADB documents are listed below:

a) Application/Administration Document;

b) Job Task Analysis (JTA);

c) Qualification Standards;

d) Instructional Systems Development (ISD) Methodology;

e) Curriculum Outlines; and

f) Implementation and Operations (I&O) Plan.

See Appendix B for further details.

D. Documentation Procedures

Management of an AQP depends on an organized, co-ordinated and well-

maintained documentation control system. The AQP application must describe
the air operator’s proposed AQP documentation design PADB, including revision
methodology and how AQP documentation will integrate with, or replace the
applicant's current traditional training program documentation, as the case may
be.

NOTE: An applicant may wish to consolidate all of the company’s AQP policies
and methodologies into the Application/Administration Document. This will
provide a “one stop shopping” approach. This document would be considered as
the applicant’s AQP Standard Operating Procedures (SOP).

E. Data Submission

33
 The AQP applicant must acknowledge their understanding and acceptance of the
AQP performance data requirements.

F. Supporting Documents and Manuals

For each type, model, series or variant, the applicant must provide Transport
Canada with the following documents or manuals:

a) Company Operations Manual (COM);

b) Aircraft Operating Manual (AOM)/Flight Crew Operations Manual

(FCOM) or Aircraft Flight Manual (AFM) and/or SOPs (as applicable);

c) Flight Crew Training Manual (FCTM);

d) General descriptive summary of each aircraft type, model, series or

variant.

G. Operating Environment Description

Applicants should describe their operating environment, including the general

physical environmental factors expected to be encountered in operations.
Environmental factors are critical to development of Line Operational Simulation
(LOS) scenarios and meaningful proficiency objectives. Environmental factors
include:

a) Weather norms and extremes;

b) Normal, abnormal and emergency equipment operation; and

c) Geographic areas of operations.

H. Trainee Demographics

The application should provide a summary of demographic data on the experience

level of the flight crewmember population that will be trained under AQP. This
data includes the current ground and flight instructors and evaluators that are
expected to continue these functions under AQP.

For each fleet, this information would identify the:

a) Entry requirements for ground and flight instructors and evaluators;

b) Entry requirements for new hires;

34
 c) Students should be identified as a group in terms of previous experience;
and

d) The current and anticipated need for replacement crew members by duty
position.

I. Training Equipment Description and Location

The application should describe the training equipment, its location and the
organization responsible for its security and maintenance. If the type of training
equipment for the AQP is not known at the time the application is submitted, the
application will be updated when the training equipment requirements are
identified. Flight simulators and/or FTDs are identified (including the level of
qualification) by the Transport Canada identification number assigned by the
Simulator Program Manager (AAROP). Other training devices will be listed and
if qualification is required, the applicant should state when it intends to submit a
test guide and a request for equipment qualification. Qualification requests are
processed in accordance with the TP 9658 - Aeroplane and Rotorcraft Simulator
Manual.

J. Facilities Description

Each AQP submission should describe the facilities the applicant intends to use.
The description should include the location, type of facility, classrooms, training
aids, courseware, and other features that contribute to creating and maintaining a
positive learning environment.

K. Master AQP Transition Schedule (MATS)

The MATS will depict the projected transition/development schedule for all AQP
curricula. Since these schedules usually change through the course of events, this
section of the application must be updated accordingly. Transition from one type
training program to another (i.e., traditional to AQP) may include a period of
overlap as one program is phased in and the other phased out. The following
guidelines for transition are applicable:

a) Currently qualified personnel may transition between traditional recurrent

training curricula and Continuing Qualification Curricula.

b) Personnel who have completed a traditional training program initial,

transition, difference, or upgrade curricula may enter a Continuing
Qualification Curriculum.

c) Partial MATS (incomplete) are not acceptable.

35
 d) The MATS may provide for incremental implementation of Qualification
and Continuing Qualification Curricula.

e) The MATS must provide the time frame necessary to withdraw from AQP
if it becomes necessary to revert to the traditional training program as
outlined in the CARs/CASS.

2.2.3 Authorization

Transport Canada’s authorization to proceed with the curriculum development phase,

following its satisfactory review of the air operator’s Application/Administration
Document, will mark the end of Phase I and the beginning of Phase II.

2.3 Phase II - Curriculum Development

2.3.1 Overview

Phase II is the development phase of the pilot training curriculum described in the Phase
I, Application/Administration Document. There are five general stages in this
developmental process of this phase, with associated documents:

a) Job Task Analysis (JTA);

b) Qualification Standards;

c) Instructional Systems Development (ISD) Methodology;

d) Curriculum Outlines; and

e) Implementation and Operations (I&O) Plan.

These steps are all inter-related. Each step builds on the previous step. The JTA supports
the development of the Qualification Standards. ISD Methodology clarifies how JTA
and Qualifications Standards will be used to support the development of the Curriculum
Outlines. The end product is presented through the Implementation and Operations Plan
which is used in Phases III and IV.

The applicant must establish, demonstrate and maintain a clear linkage between each of
these steps. This linkage is provided by a systematic approach to the development of a
complete instructional system. This section recommends a systematic approach and a
methodology that is acceptable to Transport Canada, but innovation and practical
application may result in equally acceptable variations.

36
NOTE: Instructors and evaluators shall be the first group to be transitioned from a
traditional training program to an AQP.

A. Rationale

Under a traditional training program, Transport Canada evaluates the finished

training curricula by comparing their contents, as described in each curriculum
outline, to the requirements listed within the applicable CARs and the Air Carrier
Inspector Manual (ACI). Under an AQP, Transport Canada evaluates a
curriculum by monitoring its development, and by approving a series of required
documents. These documents include the Job Task Analysis, the Qualification
Standards and the ISD Methodology documents. This process affords Transport
Canada greater insight into the rationale used by the applicant to develop each
component of each curriculum.

B. Analysis Based Approach

Under an AQP, Transport Canada monitors both the process and the product.
Instead of basing curricula on Transport Canada prescribed manoeuvres,
procedures and knowledge items, AQP curricula are based on a detailed analysis
of the requirements of each duty position at each organization. To receive
Transport Canada authorization, AQP curricula must be accepted to be safer than,
or at least as safe as, traditional training programs.

The analysis-based approach allows each applicant the opportunity to develop air
operator specific training programs. Consequently, AQP curricula will be more
sensitive to differences in aircraft, operating conditions, emergency and abnormal
contingencies, student skill levels and other operational variables, than are
traditional programs.

2.3.2 Job Task Analysis (JTA)

In order to understand task analysis as it applies to AQP it is necessary to build upon a

few definitions:

A. Job

A job is the summation of functions performed by an individual at work.

B. Function

One of the major subdivisions of work activities performed by one individual.

One or more functions constitute a job. Examples of functions applicable to AQP
would be a phases of flight such as “Take-off”, “Climb”, “Cruise”, etc.

C. Task

37
 A task is a unit of work within a function having identifiable beginning and
ending points which results in a measurable product. An example of a task
applicable to AQP would be to “perform a normal take-off”.

D. Sub-task

Specific separate step or activity required in the accomplishment of a task. An

example of a sub-task applicable to AQP would be to “perform rotation and lift-
off”.

E. Element

A further component of training analysis necessary in the accomplishment of a

sub-task. An example of an element would be to “rotate the aircraft at VR to 12
degrees pitch”.

JTA is the method or procedure used to reduce a unit of work to its base components.
The JTA document consists of a detailed, sequential listing of tasks, sub-tasks, and
elements (if required) with the knowledge and skills (technical and CRM) that clearly
define and completely describe the job. This includes the Knowledge, Skill, and
Attitudes (KSA) characteristics that clearly define and completely describe the job. A
JTA provides consideration for conditions surrounding the job both in the environment
and in the equipment used. It establishes standards (parameters and tolerances) that
provide safe and effective job accomplishment. It also identifies characteristics such as
consequence of error, relative difficulty, frequency of occurrence in specific operations,
and the time needed to accomplish the task. As a document the JTA has three parts: a
Job Task List, a Task Factors Analysis, and a Learning Analysis.

2.3.3 Job Task List

The task analysis process begins with the development of a job task list that includes all
of the major tasks performed by those who hold a particular duty position. For example,
a pilot-in-command (PIC) job task list includes all major activities involved in operating
an aircraft. These would include conducting ground operations, performing take-offs,
etc. An instructor job task list includes all the major activities involved in teaching
students. These would include preparing training materials, managing the learning
environment, operating training equipment, etc.

A. Structure

For complex jobs, it may be best to divide a job into several functions, which can
be then divided into tasks. Each task is subsequently divided into sub-tasks.
Finally, sub-tasks are then divided into elements. Each of these divisions is
identified with a corresponding number code. Figure 2-2 shows an example of
this relationship.

38
 For our discussion, we will consider the job of a PIC. We will divide this job into
functions as defined by the various phases of flight. Each function (phase of
flight) will be identified with a number code. For example, “2.0” will represent
the second phase of flight, “Take-off”.

Each function (phase of flight) can then divided into job tasks. For example, “2.0
Take-off”, can be subdivided into several tasks, each with a corresponding
number code. Since all of the tasks in our example are derived from the same
function, “2.0 Take-off”, they will all have number codes that begin with “2”.
The first digit identifies the function and the second digit identifies the individual
task. These will include: “2.1 Perform Normal Take-off”, “2.2 Perform
Instrument Take-off”, “2.3 Perform Engine Failure After V1 Take-off” and “2.4
Perform Rejected Take-off”.

Each of these job tasks can then be further divided into sub-tasks. For example
the task,“2.1 Perform Normal Take-off”, can be subdivided into numerous sub-
tasks. For “2.1 Perform Normal Take-off”, the sub-tasks will include: “2.1.1
Assess Performance and Environmental Factors”, “2.1.2 Perform Take-off Roll”,
“2.1.3 Perform Rotation and Lift-off”, and so on. Here, the first two digits
represent the task and the third digit defines the individual sub-task.

Where necessary, these sub-tasks can be further subdivided into elements. For
example, “2.1.3 Perform Rotation and Lift-off”, can be subdivided into “2.1.3.1
Rotate aircraft at VR to target pitch angle”, “2.1.3.2 Observe barometric/ADC
altimeter increase [PF]”, “2.1.3.3 Call out positive rate [PNF]”, etc. In this case, the
first three digits represent the sub-task, and the final digit defines the individual
element.

B. Hierarchy

By dividing tasks into sub-tasks and elements, each Job Task Listing produces a
numbered hierarchy of job requirements for each duty position. These job
requirements are essentially the graduation requirements for the courses that are
developed from them. Through a series of additional analyses, these job
requirements are translated into the training requirements of the various AQP
curricula: Qualification and Continuing Qualification.

The tasks are translated into Terminal Proficiency Objectives (TPOs) and the sub-
tasks into Supporting Proficiency Objectives (SPOs). Elements are translated into
Enabling Performance Objectives (EPOs). The hierarchical numbering system is
retained as the basis of the audit trail that connects job requirements and
performance with curriculum requirements and performance. Figure 2-2 illustrates
this hierarchy and serves as an example of building a task listing.

C. Rationale

39
 Proficiency-based training systems always begin with the development of a detailed
task listing. This means that required job proficiency is the basis for designing,
developing, operating and maintaining the training system. Task 1.1.1 will be
taught in lesson 1.1.1, topic 1.1.1, assessed using test item 1.1.1, and those test
results used to validate that the individual/crew can perform task 1.1.1. This audit
trail links job performance to training performance throughout every component of
the training curriculum.

Fig 2–2: Sample Pilot Job Task Listing

Function Job Tasks Sub Tasks Elements References

(TPO) (SPO) (EO) Source/Page
1.0: Ground COM /2-12
Operations
2.0: Take-off COM /3-1
2.1: Perform
Normal COM /3-2, 3-3
Take-off

2.1.1: Assess Airport

Performance Analysis
and Chart/Specific
Environmental City
Factors
2.1.2: Perform COM/ 3-2, 3-3
Take-off Roll
2.1.3 Perform 2.1.3.1: Call out COM/3-3
Rotation and V speeds [PNF]
Lift-off
2.1.3.2: Rotate COM 3-3
aircraft at VR
to target pitch
angle [PF]
2.1.3.3: COM 3-3
Observe
barometric /
ADC altimeter
increase [PNF]
2.1.3.4: Call out COM 3-3
positive rate
[PNF]
2.2: Perform
Instrument
Take-off

40
 2.3: Perform
Engine
Failure After
V1
Take-off
2.4: Perform
Rejected
Take-off
3.0: Climb
4.0: Cruise
5.0: Descent
6.0: Approach
7.0: Landing
8.0: After
Landing
9.0: Abnormal
Procedures
10.0:
Supplementary
Procedures

2.3.4 Learning Analysis

The second part of the JTA is sometimes called competency analysis, skill analysis,
Knowledge Skills and Attitudes (KSA) analysis, or hierarchical analysis. Here, those
tasks, sub-tasks or elements that were selected for training as part of the task factors
analysis are further analyzed into their more basic knowledge and skill level (attitudes are
optional). This learning analysis will determine to a finer level of detail, exactly what
should be learned, and the best approaches for teaching and testing what is to be learned.
While the task factors analysis adds greater specificity to the performance and training
requirements of the tasks, the learning analysis defines in greater detail exactly what
should be taught and tested, and how it should be taught and tested, to assure that the
students acquire those job performance requirements.

2.3.5 Job Task Analysis Document

The JTA document is the second of the six documents unique to AQP that must be
maintained in a current status throughout the life of the program and must have an
acceptable revision control methodology. Not all of the results of the JTA need to be
reported to Transport Canada in the JTA document itself. While Transport Canada
requires that the results of the learning analysis to be reported in the JTA document, it
may be more convenient to report the results of some of the Task Factors Analysis in the
Qualification Standards document.

41
2.3.6 Qualification Standards

A Qualification Standard is a job task proficiency objective (TPO or SPO) linked to an

evaluation strategy. Qualification Standards define the requirements of mastery for the
duty position. Demonstration that an individual has met the required standards will lead
to certification. As a document the Qualification Standards is the single most important
part of any AQP. It provides the complete proficiency baseline for all duty positions and
serves as the basis of development for the Qualification Curriculum and Continuing
Qualification Curriculum. The first step in the development of Qualification Standards is
the development of proficiency objectives.

2.3.7 Proficiency Objectives

A Proficiency Objective is the result of applying a performance statement, conditions(s),

and proficiency standard(s) to a task or sub-task. It is a statement describing the
behaviour that the candidate must be able to demonstrate to successfully perform a task.
For each duty position, there are two types of Proficiency Objectives, both of which are
developed from the Job Task Analysis (JTA). These are: Terminal Proficiency
Objectives (TPOs), which are developed from tasks; and Supporting Proficiency
Objectives (SPOs), which are developed from sub-tasks.

Each proficiency objective statement has three parts:

a) A performance statement that specifies precisely what behaviour must be

exhibited.

b) A condition statement identifying the operational and equipment contingencies as

well as the environmental conditions under which the behaviour will be
accomplished.

c) A standard or criterion statement establishing the parameters and tolerances that

define satisfactory performance.

Note: All document references used in defining the performance, conditions, and
standards for each proficiency objective must be listed by title and chapter in the
documentation of the proficiency objectives in the Qualification Standards Document.

A. Terminal Proficiency Objectives (TPOs)

TPOs are statements of performance, conditions and standards established at the

task level. A complete set of TPOs will fully describe a particular job in the
applicant's flight operation. TPOs are classified by the air operator as either
critical or currency items based on an operational assessment in the Task
Analysis process. This classification will determine the frequency with which
these tasks are evaluated during the Continuing Qualification Cycle. TPOs

42
 include the range of flight training equipment and the abnormal and emergency
contingencies to be considered for training and evaluation.

B. Supporting Proficiency Objectives (SPOs)

SPOs are statements of performance, conditions, and standards established at the

sub-task level. SPOs are used to develop training and evaluation curriculum
lessons, modules, and segments. SPOs include a list of flight training equipment
and the abnormal and emergency contingencies to be considered for training and
evaluation.

C. Enabling Objectives (EOs)

EOs are used to prepare individuals and crews for subsequent training in an
operational cockpit environment. An applicant may identify a certain knowledge
factor, cognitive skill, motor skill, or CRM factor as an Enabling Objective.
These are normally not carried forward in the supporting performance objective
statement and, therefore, are not normally found in the Qualification Standards
Document. However, performance of a SPO would depend on a student acquiring
the particular knowledge, skill, attitude or CRM factor.

Note: A learning objective (usually an EO), which doesn't have condition, can be
demonstrated in a classroom or academic type setting. A performance objective
(usually a terminal or supporting proficiency objective) must be demonstrated in
an environment equivalent to the operational environment.

2.3.8 Task Factors Analysis:

This process rates TPOs and SPOs using the following factors: criticality, currency, need
for training, applicable conditions, and applicable standards. The determination of
criticality and currency determines ‘when’ and ‘how’ the objective is trained, validated or
evaluated (see figure 2-3). To make this determination, the analyst answers a series of
questions about each TPO and SPO to describe its performance requirements, both on the
line and in the training setting.

Criticality is a determination of the relative impact of substandard task

performance on overall safety. It indicates an increased need for awareness, care,
exactness, accuracy, or correctness during task performance. Critical tasks are
proficiency objectives that are trained, validated or evaluated more frequently during an
AQP evaluation period in a simulator or FTD.

A currency task is a proficiency objective for which individuals and/or crews

maintain proficiency by repeated performance of the item in normal line, duty or
work operations. Most currency items are validated during online evaluations (OE) and
may be sampled in the Continuing Qualification cycle.

43
Tasks that are determined to be critical and not current are trained, validated or evaluated
each evaluation period. Tasks that are determined to be neither critical nor current are
trained, validated or evaluated each Continuing Qualification Cycle.

Note: Many flight crew job task SPOs do not fit the classic definition of a sub-task, a
specific separate step or activity required in the accomplishment of a task. In
recognition, non critical/non current SPOs under a common TPO that differ only in
knowledge requirements may be trained, validated or evaluated in a simulator/FTD,
during OEs, using oral, written or electronic exams, class room briefings or distributed
material. However, it is recommended that these SPOs, as appropriate, be demonstrated
in a simulator/FTD on a recurring cycle authorized by Transport Canada.

Transport Canada recommends that the applicant examine each task, sub-task, and
element, as appropriate, for the following factors:

Primary factors to be considered:

a) Statement of performance;
b) Environmental conditions affecting difficulty/success;

c) Performance standards (parameters with tolerances);

d) Abnormal and emergency procedure contingencies;

e) Student entry-level performance evaluated against proficiency objective;

f) Document references (title and section) governing or specifying the operation;

g) Consequence of error to safety; and

h) Relative difficulty.

Additional Factors:

a) Equipment and system operation dependencies (if used for establishing learning
sequences for curriculum development);

b) Criterion for success upon which performance standards are based. If new
performance standards are created, this criterion should be established for each
task and sub-tasks (e.g., the tracking standards for VHF omni-directional radio
(VOR) approaches are based on navigation requirements). The navigation
requirements are the criteria for success. Success criteria are developed in those
cases where current standards are missing or thought to be inadequate.

44
Fig 2-3: AQP Continuing Qualification Critical/Currency Chart

Training Critical Currency Terminal Proficiency Objective/Supporting

Priority (Y/N) (Y/N) Proficiency Objective

1 YES NO Train, validate and evaluate each Evaluation Period. (e.g.,

Engine failure after take-off, CAT I and CAT II
Approaches, Wind shear, Engine-out precision approach)
2 NO NO Train, validate and evaluate each Continuing
Qualification cycle. (e.g., pilot operation of passenger
doors, unpressurized flight procedures, alternate landing
gear procedures)
3 YES YES Sample at First-Look (FL)/Manoeuvres Validation
(MV)/Line Operational Evaluation (LOE) and/or Online
Evaluation (OE) for each Evaluation Period. (e.g.,
Perform high altitude airport operations, perform adverse
weather (icing) procedures)
NOTE: For AQP, the sample size has to be large enough
to provide reasonable assurance that the population is
remaining proficient.
4 NO YES Sample at First-Look/Manoeuvres Validation (MV)/Line
Operational Evaluation (LOE) and/or Online Evaluation
(OE) each Continuing Qualification cycle (e.g., Perform
Normal Landing, Perform Cruise Operations)
NOTE: For AQP, the sample size has to be large enough
to provide reasonable assurance that the population is
remaining proficient.

Training Critical Current Month Sample

Priority
1 Yes No 12 No
2 No No 24 No
3 Yes Yes 12 Yes
4 No Yes 24 Yes

NOTE: The sample size must be large enough to provide reasonable

assurance that the population is remaining proficient.

NOTE: The Qualification Standards Document for instructors/evaluators

does not need to include conditions or a criticality/currency analysis.

2.3.9 Evaluation Strategy

45
The Qualification Standards document will identify the curriculum (Qualification and/or
Continuing Qualification) in which specific proficiency objectives will be met. The
applicant should consider student entry level in determining this allocation. All TPOs
must be included in a Qualification Curriculum (Qualification Course) regardless of
entry-level analysis. For SPOs, an entry-level analysis determines what objectives will
be taught under each curriculum. All objectives should also be covered in Continuing
Qualification Curriculum test and evaluation strategies.

2.3.10 Consolidation of Objectives

In the Qualification Standards document, qualification standards are developed at the task
and sub-task level only and at no lower level. Tasks become TPOs and sub-tasks become
SPOs by combining performance statements, conditions and standards. TPOs and SPOs
having common knowledge, skill, attitude, and/or CRM factors may be consolidated to
avoid duplication. The consolidated tasks are translated into TPOs, and a terminal level
qualification standard is developed for each one. The consolidated sub-tasks are
translated into SPO, and a supporting level qualification standard is developed for each
one of those as well. An example of consolidation would be non-precision approaches.
VOR and Non-Directional Beacon (NDB) approaches may be consolidated as a single
proficiency objective if the performance statement and standards are the same.

2.3.11 Conditions, Contingencies, and Media

The Qualification Standards will include a listing of relevant operational and

environmental factors along with equipment contingencies to be addressed in training.
They will also identify the media that will be employed to test, validate, or evaluate the
specific training objective. There is a difference between the Qualification Standard for a
Qualification Curriculum and one for a Continuing Qualification Curriculum. The TPOs
and SPOs in a Qualification Standard for a Qualification Curriculum must identify the
specific set of conditions and contingencies to be employed in training and evaluating a
task (usually with an asterisk). In addition, the media description will describe the
specific media in which the task will receive final evaluation. In contrast, in a
Continuing Qualification curriculum, the TPOs and SPOs may identify a selectable menu
of conditions and contingencies to be employed in training or evaluation, and a media
range that will specify the minimum media level in which the task may be trained,
validated, or evaluated.

2.3.12 Qualification and Continuing Qualification Curricula

The Qualification Standards for both Qualification and Continuing Qualification

Curricula can be combined into one document as long as the differences in conditions and
media are addressed and the tasks are annotated for applicable curricula. However, if
applicants find it more convenient to submit separate Qualification Standards documents
for these curricula (e.g., because of differences in conditions, contingencies, and media
between the Qualification and Continuing Qualification curricula), they may do so. In
addition, some Qualification Standards may be “aircraft generic” in that they may apply

46
to more than one aircraft type. For ease of use and definition of “fleet common”
curricula, applicants may also find it convenient to specify these Qualification Standards
separately from those Qualification Standards that apply to only one aircraft fleet type.

2.3.13 Qualification Standards Document

The Qualification Standards document is the third of the six documents unique to AQP
that is maintained in a current status throughout the life of the program and must have a
revision control methodology. It is the central AQP document because it provides the
regulatory basis for all deviations from current regulations, and identifies the basic
training and evaluation requirements. The Qualification Standards document has four
parts: the Prologue, a Regulatory Comparison, the Evaluation/Remediation Methodology,
and the specific TPO or SPO Qualification Standard information.

A. Prologue

The Qualification Standards document requires an introductory section that

explains the methodology, format, and terminology of the standards to the broad
range of company and Transport Canada personnel who will need an in-depth
understanding of the document to perform their job functions.

B. Regulatory Requirements Comparison

The Qualification Standards document must also include regulatory comparison

information. The comparison must indicate specifically the requirements of Parts
IV, VI and VII of the CARs as applicable that would be replaced by an AQP
curriculum. Once approved, the Qualification Standards document becomes the
regulatory foundation for the operator’s AQP.

Note: The purpose of this regulatory comparison is not to justify differences

from current traditional practices, but merely to document them once they have
been justified. It is the quality control processes of AQP that form the basis for
establishing an equivalent level of safety.

C. Testing/Validation/Evaluation & Remediation Methodology

Before implementing an AQP curriculum, the applicant must decide how, when,
where, and who will assess a student’s proficiency on each terminal and
supporting objective. The guidelines in figure 2-3 may be used to translate TPOs’
and SPOs’ criticality and currency ratings into testing strategies for the
Continuing Qualification Cycle. This testing strategy may include a discussion
describing how similar SPOs will be addressed. For example, the TPO non-
precision approach may have many SPOs, such as VOR, Non Directional Beacon
(NDB), localizer (LOC), etc. Depending on the criticality/currency rating, all of
these approaches do not have to be evaluated each evaluation period. This section
of the document is where the applicant can describe how these approaches will be

47
alternated/sampled over multiple evaluation periods or Continuing Qualification
Cycles.

The testing/validation/evaluation methodology also addresses the applicant’s

approach to documenting the different qualifications of the instructor and
evaluator, in terms of who will conduct the testing, validation, LOE, and OEs. In
this section, the applicant can describe the level of device appropriate for
conducting the evaluation. It also identifies the point in the curriculum when the
testing/validation/evaluation methodology will be applied, and identifies what
constitutes a failure and/or unsatisfactory performance. In addition, the applicant
must specify the strategy for remedying unsatisfactory performance.

a) General Strategies: For each terminal and supporting proficiency

objective, the applicant will designate the testing/validation/evaluation
strategy. These strategies could include, and are not limited to:

i) Train to Proficiency;

ii) Systems/Knowledge Validation (SKV);

iii) Procedures Validation (PV);

iv) Manœuvres Validation (MV);

v) Line Operational Evaluation (LOE);

vi) Operating Experience;

vii) Online Evaluation (OE).

b) Training Media: The applicant also will designate the level of training
devices, simulators, or aircraft to be used to evaluate the proficiency
objective.

c) Rating Scale/Scoring: The applicant will identify the rating methodology

that will be used to grade the performance of the proficiency objectives
against the Qualification Standards. Typically, the measurement codes
associated with performance events are ratings, repeat counts, and reason
codes or skill categories.

Ratings are used to define different quality levels of performance. Rating

codes usually are air operator specific and Transport Canada requires that
something more sensitive to performance differences than a binary code is
used (i.e. some rating method that provides more performance
differentiation than pass/fail for individual items being evaluated.

48
 Note: For Manoeuvres Validations (MV), Line Operational Evaluation
(LOE) and Online Evaluation (OE) a minimum four point grading scale
shall be used.

Each carrier should ensure that the grades established on the rating scale
are clearly defined, meaningful to the instructor and evaluator, and easily
used for performance assessment. Although consistency among fleets and
across different types of evaluations - Online Evaluation (OE),
Manoeuvres Validation (MV), and Line Operational Evaluation (LOE) - is
important and generally desirable, rating scales may be slightly different
when used for different purposes such as training vs. evaluation. Figure 2-
4 provides an example of a rating scale that discriminates among
performance levels.

Fig 2-4: RATING SCALE EXAMPLE - First-Look, Manoeuvres Validation, Line

Operational Evaluation, or Online Evaluation

GRADE CRITERIA
Major deviations from the prescribed qualification
1 Unsatisfactory standards occur that are not recognized or corrected
in a timely manner commensurate with safety.
Individual or crew performance could result in hull
loss or loss of life. CRM skills are not effective.

Deviations from the prescribed qualification

2 Satisfactory standards occur and are corrected in a timely manner.
Individual or crew performance is safe but would be
unsatisfactory if diminished by any amount. CRM
skills are not completely effective.

No deviations occur from the prescribed qualification

3 Standard standards. Individual or crew performance meets
expectations. CRM skills are clearly effective.

Performance remains well within the prescribed

4 Excellent qualification standards. Individual or crew
performance, management and CRM skills are
exemplary.

NOTE: This example should not be taken as limiting possible intervals to a four-point
scale. With appropriate scale construction and instructor and evaluator training,
carriers may elect to define other scales that maximize the quality (sensitivity, reliability,
validity) of the collected data.

49
 d) Remediation Strategy: This section of the Qualification Standards
document should describe the methodology that will be used to re-mediate
unsuccessful testing, validation, or evaluation sessions. This remediation
strategy must detail when and what may be repeated and whether or not
additional training is warranted. Remediation strategy must also specify
when no more training will be offered to the individual and the resulting
actions such as “Referred to Committee,” returned to previous position,
etc. The AQP Evaluator Manual includes expanded discussions on
validation, evaluation, and remediation in the Qualification and
Continuing Qualification Curricula. This strategy may be presented in
narrative text or flowchart format.

e) Special Tracking: This is a program for monitoring the proficiency of an

individual at scheduled intervals. It is applied to individuals who have
failed to demonstrate proficiency during an evaluation event (e.g., LOE).
There may be other criteria that the air operator may use to place an
individual on special tracking. These could include continuing difficulty
in completing the Manoeuvres Validation (MV) or a failure of an OE.
This section should discuss the following:

i) The situation(s) that requires an individual to be placed in special

tracking

ii) The strategy to be used.

iii) When special tracking is no longer required.

D. Qualification Standard Information:

The information contained in the specific qualification standard is the basis for
determining proficiency and evaluation criteria. The qualification standard is
constructed by applying a performance statement, conditions, and standards to a
task or sub-task, thereby creating a TPO or an SPO.

Although each air operator will determine the format and content of its
Qualification Standards, component fields have developed out of practice and are
illustrated in Figure 2-2. In this example, the phase of operations is 2.0: Takeoff.
The TPO is 2.1: Perform Normal Takeoff. The SPOs are 2.1.1: Assess
Performance and Environmental Factors, 2.1.2: Perform Takeoff Roll and 2.1.3:
Perform Rotation and Lift-off. EPOs are 2.1.3.1: Call out V Speeds, 2.1.3.2:
Rotate Aircraft at VR to Target Pitch Angle, 2.1.3.3: Observe Barometric/ADC
Altimeter Increase and 2.1.3.4: Call Out Positive Rate.

Variation in the format of a given air operator’s Qualification Standards is

permissible if all of the categories of information in the example are addressed.

50
a) A header identifies the airline and the document.

b) Page revision control dates and revision numbers.

c) Consecutive page numbers.

d) Phase of Operations. Number and title from task listing.

e) Qualification Standard Title. Either TPO(s) or SPO(s).

f) Hierarchical Task or Sub-task. Identifier and title from task listing.

g) Crew Duty Position(s). This identifies which crew member(s) will be

evaluated performing the task.

i) Pilot-in-Command = PIC

ii) Second-in-Command = SIC

iii) Flight Engineer = FE

iv) Captain, First Officer, Second Officer

h) Criticality/Currency Rating. From the task factors analysis of the job task
listing. This may be the first place that the task factors analysis is tied to
the tasks.

NOTE: The Qualification Standards document for instructors and

evaluators does not need to include conditions or a criticality/currency
analysis.

i) Curriculum. This field identifies the curriculum(s) in which the task will
be trained and evaluated.

j) Evaluation Strategy. The evaluation point for a particular qualification

standard (e.g., (1) train to proficiency; (2) systems validation;
(3) Procedures Validation (PV); (4) Manoeuvres Validation (MV);
(5) LOE; or (6) OE).

k) Media. The specific media in which training and/or evaluation will be

conducted. For qualification, the media is the lowest media used for final
evaluation. For continuing qualification, the media includes the range of
media used for training, validation, and evaluation.

51
 l) Performance Statement. An expanded statement of expected behaviour
that, when executed, will complete the work required for a specific portion
of a job. A performance statement specifies precisely what behaviour
must be exhibited, and may include the knowledge and skill issues that
comprise the EO supporting that performance.

m) Operational and Environmental Conditions. Conditions describe the range

of circumstances under which student performance will be measured and
evaluated. Conditions include the operational environment (unserviceable
navigational aid (NAVAID), different aircraft weight, passengers not
seated, aircraft configuration, etc.) and natural environment (ceiling,
visibility, wind, turbulence, etc.). The qualification standard should:
(1) indicate those specific conditions to be trained and tested as part of the
Qualification Curriculum; and (2) provide a more exhaustive listing of
conditions over which crew members will be trained and tested during the
course of successive Continuing Qualification Cycles.

n) Contingencies. Contingencies include abnormal situations, MEL/CDL,

and emergencies. The qualification standard should (1) indicate those
specific contingencies to be trained and tested as part of the
Qualification Curriculum; and (2) provide a more exhaustive listing of
contingencies over which flight crewmembers will be trained and tested
during the course of successive Continuing Qualification Cycles.

o) Manœuvre Standards. Observable, measurable parameters of performance

with tolerances (e.g., course deviation degrees (+ or -)). Standards include
manoeuvres, procedures, and CRM considerations.

p) References. Identify the primary references from which performance

statements and associated standards were derived. Cite documents by title
and where applicable, chapter or section. Page numbers are not required.

2.3.14 Instructional Systems Development (ISD) Methodology Document

This is the fourth of the six documents unique to AQP and is maintained throughout the
life of the program. It must have an acceptable revision control methodology.
Applicants with established curriculum development guidelines may submit these for
consideration. Others should describe a systematic approach for developing a
proficiency-based training system that is organized around the teaching and testing of
terminal, supporting and enabling proficiency objectives. The methodology identifies the
rationale, justification, and subsequent documentation to be used in the applicant’s
curriculum development process. The instructional systems development methodology
document describes the approach to be used by applicant airlines to develop and maintain
all AQP curricula.

52
ISD Methodology document should be finalized before constructing curricula for each
duty position. It applies to pilot, instructor, and evaluator programs. This document is
divided into two sections. The first section, Curriculum Development Procedures,
describes the applicant’s approach for using the JTA and Qualification Standards as
baseline documents to construct their general training curricula across all AQP courses.
The second section, Line Operational Simulation (LOS) Methodology, describes the
approach for developing LOS scenarios.

A. Curricula Development Process/Methodology

Applicants should describe the process they will use to build their curricula based
on the JTA, Qualification Standards and proficiency objectives they develop for
each duty position. This document should discuss how:

a) TPOs and SPOs are allocated to curricula.

b) Learning and evaluation activities are developed to support these

objectives.

c) Instructional media and methods are assigned to objectives.

d) Objectives are clustered and sequenced into lessons, modules, segments,

and curricula (see figure 2-5 and figure 2-6 for examples).

e) An audit trail (hierarchical numbering or a matrix) will be maintained to

link job tasks, proficiency objectives, lesson activities/content, and test
items.

The resulting curricula are translated into a course footprint and are documented
in each curriculum outline. These curricula are expanded in more detail in the
student and instructor syllabi and in individual lessons and tests.

Fig 2-5: Curriculum Development

Curricula

Segments

Modules

Lessons

TPO TPO TPO

SPO SPO SPO

EO EO 53 EO
B. Develop Line Operational Simulation (LOS) Development Methodology

Transport Canada will approve the methodology by which LOS scenarios (i.e.,
SPOT, LOFT and LOE) are generated and will review and accept individual LOS
scenarios. The LOS approach divides the typical LOFT scenario into a series of
relatively independent segments, called event sets. A typical scenario might have
for example eight event sets, relating or not to phases of flight (e.g., pre-departure,
take-off, climb, cruise, descent, approach, landing, and taxi-in). Each event set
consists of a series of training or evaluation events (graded events/tasks), which
include both technical and CRM activities. An LOE shall contain a minimum of
eight to a maximum of eleven event sets.

The above technique enables scenarios to be constructed in a building block

approach, assuring that each event set is carefully scripted, sequenced and
considered in relation to the other event sets in any given scenario.

The criteria used by Transport Canada when validating LOS scripts are located in
Chapter 3 – AQP Authorization Process and Documentation.

Applicants with established curriculum development guidelines may submit these

for consideration. Others should describe a systematic approach for developing a
proficiency-based training system that is organized around the teaching and
testing of TPOs, SPOs and EOs.

This document should discuss the following:

a) Proficiency objectives are developed and organized into curricula;

b) Learning and evaluation activities are developed to support these

objectives; and

c) An audit trail will be maintained to link objectives, lesson

activities/content, and test items.

2.3.15 Curriculum Outline Document

This is the fifth of the six documents unique to AQP that must be maintained in a current
status throughout the life of the program. It must have an acceptable revision control
methodology. The curriculum outline provides the footprint, which is a high level
description of the training and evaluation activities and planned time allotment for each
day in the curriculum.

54
The Curriculum Outline Document contains a listing of course material divided into
segments. Typical segments would be Ground School or Flight Training. These
segments are then divided into modules. For example, within the Ground School
segment, there could be several modules including: Aircraft Systems, SOPs, and Long
Range Navigation. The Flight Training Segment would typically include FTD, Fixed
Base Simulator (FBS) and Full Flight Simulator (FFS) modules. These modules are
further divided into lessons. The first lesson in the FTD module might focus on Pre-
Flight Operations and Normal Checklists. Finally, lessons are divided into elements or
topics. The FTD module on Pre-Flight Operations and Normal Checklists would cover
elements or topics such as Flight Compartment Inspection, Flows, Before Starting
Engines Checklist, etc. Figure 2-6 provides an example of a curriculum outline showing
portions of ground training and flight training segments which have been divided into
modules, lessons and elements (topics).

Curriculum outlines are developed and submitted with the understanding that application
of the course material may require some flexibility regarding the actual day on which
each activity is accomplished. Each part of the curriculum outline must clearly indicate
the subject matter to be taught and correspond directly to the hierarchical system of the
task analysis. While the curriculum outline document need only go down to the level of
the element under each lesson title, the applicant will be required to show the terminal,
supporting and EOs associated with each lesson. Refer to Figure 2-5 for an illustration of
this association. This is part of the necessary audit trail that links the job requirements
(contained in the JTA) to the training requirements (contained in the Qualification
Standards) to the training activities (contained in the Curriculum Outline).

A curriculum outline provides the basis for the curriculum footprint, which is a high level
graphical overview of the curriculum content depicting the training and evaluation
activities and the planned duration of each day of the curriculum (see figure 2-5). The
curriculum outline document should reference the results of the student entry analysis, if
one was conducted, and will include a curriculum footprint.

55
Fig 2-6: Sample AQP Curriculum Outline

B 737 Qualification Curriculum Outline

Segment: Ground Training

Module: Introduction to Aircraft

Lesson #: Aircraft Overview 9.1.4

Topic: Fuselage 9.1.4.1
Topic: Wings 9.1.4.2
Topic: Flight Controls 9.1.4.4, 9.1.4.5, 9.1.4.6
Topic: Landing Gear 9.1.4.7
Topic: Powerplant 9.1.4.3
Topic: Fuel System 9.1.4.8
Topic: Hydraulic System 9.1.4.9
Topic: Electrical System 9.1.4.10, 9.1.4.10.1, 9.1.4.10.2,
9.1.4.10.3

Lesson #: Aircraft Lighting 9.1.9

Topic: Exterior Lighting 9.1.9.1
Topic: Cockpit Lighting 9.1.9.2
Topic: Cabin Signs and Lights 9.1.9.9, 9.1.9.4, 9.1.9.6
Topic: Lighting Power Sources 9.1.9.5
Topic: Emergency Lighting 9.1.9.5, 9.1.9.5.1, 9.1.4.10.2

B 737 Qualification Curriculum Outline

Segment: Flight Training (FT)

Module: Flight Training Device (FTD)

Lesson: FTD #1 Pre-Flight Normal Checklists

Topic: Flight Compartment Safety Inspection 1.2.5.1
Topic: Flows 1.2.7.1
Topic: Acceptance Checklist 1.2.8.1
Topic: Before Starting Engines Checklist 1.2.11.1
Topic: Normal APU First Engine Start 1.9.1.1
Topic: Normal APU Second Engine Start 1.9.1.4
Topic: Pushback 1.9.6.1
Topic: Before Taxi Checklist 1.4.1.1
Topic: Normal Taxi 1.4.2.1
Topic: Before Take-off Checklist 1.4.9.1
Topic: Line-Up Checklist 1.4.4.1

56
A. Entry Level Analysis

As an option, the applicant may develop and document a student entry-level

performance analysis for TPO and SPO. This analysis compares the Knowledge,
Skills and Attitudes (KSAs) of the student population against the TPOs and SPOs
in the JTA in order to tailor the instruction to the student. A four-point
performance difference rating scale is suggested (Figure 2-7). Highly skilled
instructors who are familiar with the experience and background of the student
population and knowledgeable of the TPOs and SPOs should make the rating.
This analysis provides guidance to determine efficient teaching strategies for the
Qualification Curriculum. This analysis can also identify where training is not
needed, where basic “enabling” skills must be taught, and what number of trials is
expected for an applicant to reach TPO standards. More than one population
group may be used in conducting the student entry analysis for a single duty
position. The results of such an analysis may be used, for example, to justify
alternative curriculum tracks or modules targeted at expected differences in entry-
level background, and in order to achieve the most efficient use of training
resources.

Fig 2-7: EXAMPLE - Performance Difference Rating Scale

Performance Discrimination Scale Scale Description

4 Meets or exceeds the required performance.

3 Can accomplish tasks with minor errors or omissions.

May take longer than expected or allowed.

2 Cannot accomplish tasks. Does not demonstrate basic

background skills and knowledge.

1 Does not demonstrate basic background experience,

skills or knowledge. Unfamiliar with simplest
elements of a task.

B. Trainee and Instructor/Evaluator’s Curricula

The instructor and evaluator curricula must be developed in the same systematic
manner as those developed for the duty positions with which they are associated.
This requires the development of separate JTA, Qualification Standards,
Curriculum Outlines and other documents for these instructor and evaluator
positions. These instructor and evaluator curricula may share common modules
or lessons. Instructors and evaluators also require separate Qualification and
Continuing Qualification curricula.

57
C. Link Qualification Standards to Curricula

Both the Qualification and Continuing Qualification curricula for a given duty
position are derived from the same set of Qualification Standards. The link
between Qualification Standards and a curriculum is the Proficiency Objective
(TPO or SPO). As a rule of thumb, Qualification Standards developed from
TPOs focus more on the higher-level evaluation activities than Qualification
Standards developed from SPOs. Qualification Standards developed from SPOs
focus more on the lower level learning activities that may be validated and
represent the components of those higher level standards. Therefore, a
Qualification Curriculum will focus equally on Qualification Standards developed
from TPOs and SPOs, while the Continuing Qualification Curriculum will focus
more heavily on the Qualification Standards developed from TPOs.

2.3.16 Implementation and Operations Plan (I&O) Document

This document, like the other five unique AQP documents, must be maintained in a
current status throughout the life of the program. It must have an acceptable revision
control methodology and must be updated as necessary to accurately reflect the status of
the AQP applicant’s plan for implementing and operating each of the AQP curricula.
This document is a milestone schedule detailing the transition to an AQP for flight
crewmembers, instructors and evaluators and a blueprint describing provisions for
maintenance, administration, data management, and continuing quality control of
curricula. The Implementation and Operations Plan Document can be sectioned into two
major parts.

The first part of the Implementation and Operations Plan spells out how the operator
proposes to implement the AQP. Included in this proposal is the schedule for phase III,
training and evaluation to include instructor and evaluator training and small group try-
outs. It should also include provisions for evaluating the effectiveness of performance
measurement tools, and provisions for evaluating facilities, courseware, and equipment
before starting the plans for the small group try-outs.

The second part of the Implementation and Operations Plan contains an explanation of
how the Operator intends to operate the AQP in phases IV and V. Included in this
section are strategies for maintaining the program, crew pairing policy, First-Look
administration, instructor and evaluator requirements. The operations plan should also
include the data management plan, a statement of understanding addressing the collection
and analysis of performance/proficiency data, a description of the PPDB, the data
management collection process, and the Transport Canada data submission, analysis, and
reporting requirements.

58
A. Implementation

This section provides the schedule for evaluating curricula in the small group try-
out, as well as instructor and evaluator training, and the strategies for evaluating
facilities, courseware, and equipment. It also includes the proposal for evaluating
instructors, evaluators, and performance measurement tools such as the rating
scale and grade sheets. The schedule for initiating phase III should correlate with
the MATS as submitted in the application. This part of the I&O Plan is updated
each time a new curriculum is added to the AQP and a small group try-out is
planned.

NOTE: If the air operator is requesting no-jeopardy credit for the students in the
small group try-out, it must be indicated in the I&O Plan and requested in writing
to the POI.

B. Operations

This section of the I&O Plan describes the guidance and policies that will be used
to provide for AQP maintenance, First-Look administration, crew pairing policy,
instructor and evaluator requirements and data management. Once established,
the operations section remains reasonably stable and does not necessarily change
with the addition of a new curriculum.

a) AQP Maintenance Strategy for Phases IV and V. This section

describes quality control procedures (i.e., plans for acquiring and
measuring data for tracking curricula, students, instructors, and evaluator
performance). It also includes the strategy employed for curriculum
maintenance and update. Maintenance includes the methodology for
maintaining control of the AQP. It includes approval documents,
maintaining curriculum currency, upgrading equipment, monitoring and
responding to demographic changes, and for using training/evaluation
feedback from the PPDB and other forms of surveillance to maintain and
improve the AQP.

b) First-Look Activity Administration. First-Look performance items are

graded procedures/manoeuvres performed for the first time since the
previous training cycle. The First-Look grades are analyzed to determine
trends of degraded proficiency due to numerous factors, including the
length of the training interval. To maintain the validity of the performance
proficiency data obtained from the performance of the First-Look
activities, this section must describe the strategy employed for those
activities. This strategy must state that information or techniques that
would unduly interfere with the First-Look performance validity will not
be briefed before the first execution of these items. In addition, this
strategy should discuss how First-Look manoeuvres are selected and how
they will be administered.

59
 Note: Phase V of AQP may allow for modified training and evaluation
cycles. First-Look requirements are an integral part of an AQP’s
development process and are useful to air operators who wish to modify
their training and evaluation cycles from current regulatory requirements.
In such cases, the air operator must have previously implemented First-
Look Manoeuvres and collected sufficient data through one full
Continuing Qualification Cycle in order to establish a base line by which
to measure the effect of modified intervals.

c) Identify Line Operational Simulation (LOS) Crew Scheduling and

Pairing Strategy. A basic requirement of AQP is to train and evaluate
flight crewmembers in a crew configuration identical to line operations.
Therefore, LOS should take place in a line operational environment with a
complete crew. A complete crew concept allows crewmembers to use their
full resources and creativity to create a complete learning experience. A
complete crew consists of flight crewmembers for each seat position who
are line-qualified or in qualification training for their respective seat
positions.

In AQP, line crewmembers must be scheduled and paired together, as

much as practical, in a standard crew configuration (e.g., line Captain with
line First Officer). Transport Canada recognizes that circumstances will
occur where the initial composition of the schedule cannot be maintained.
Hiring requirements, illness, high First Officer to Captain ratios, or failure
of a crewmember to progress, are all situations that would necessitate
providing a seat substitute to complete the training. This section must
address the decision rules that will apply to the selection of seat substitutes
for those circumstances. In all cases, the seat substitute must be task-
familiar with the duty position. Task familiar describes a flight
crewmember who is familiar with and can satisfactorily accomplish the
duties of a particular crew duty position, though not qualified for that duty
position. For example, an SIC candidate who performs the duties of the
PIC during simulator training.

The following paragraphs provide examples of substitution rules that

could be applied when addressing selection of seat substitutes for LOFT,
SPOT and LOE. Transport Canada strongly encourages air operators to
develop such decision rules for all types of AQP training and
validation/evaluation events.

Recurrent LOFT stresses scheduling of a complete crew who should be

line-qualified. The use of substitutes is discouraged, and substitution
should be rare. When the composition of the scheduled line-qualified crew
cannot be maintained, the operator may use substitutions based on the
guidelines in the table below. However, the air operator will attempt, first,

60
 to substitute with another line-qualified flight crewmember. This table
should be used only as a last resort to prevent interruption of scheduled
training.

Recurrent LOFT Substitution Table

________________________________________________________________________
_____
Pilot-in-Command Second-in-Command Flight Engineer
Position Position Position
1 1 1
1. PIC SIC FE
2 1 3
2. SIC PIC FE Instructor
3 3
3. Instructor Instructor
1
Includes those who are either line-qualified, or in training, and are line and task familiar
with the position in which they are substituting.
2
An SIC may be substituted for this position if the pilot has received a type rating in the
aircraft the simulator replicates.
3
An instructor (aircraft or simulator) as provided for under part VII of the CARs. The
instructor should not have previous knowledge of the scenario; however, when this is
unavoidable, the instructor should not use that knowledge to influence or direct the
scenario.

NOTE: The instructor conducting the LOS session will not act as a
substitute crewmember.

Qualification LOFT requires a complete crew complement. It is preferable

to schedule a flight crewmember who is qualifying with other flight
crewmembers who are fully line-qualified. As a minimum, LOFT flight
crewmembers will be task familiar with their assigned duty position. The
use of substitutes is highly discouraged and substitution should be
implemented rarely. When the composition of the scheduled crew cannot
be maintained, the operator may substitute flight crewmembers using the
table below.

Qualification LOFT Substitution Table

________________________________________________________________________
_____
Pilot-in-Command Second-in-Command Flight Engineer
Position Position Position
1 1 1
1. PIC SIC FE
1 1 2
2. SIC PIC FE Instructor
2 2
3. Instructor Instructor

61
1
Includes those who are either line-qualified, or in training, and are line and task familiar
with the position in which they are substituting.
2
An instructor (aircraft or simulator) as provided for under Part VII of the CARs. The
instructor should not have previous knowledge of the scenario; however, when this is
unavoidable, the instructor should not use that knowledge to influence or direct the
scenario.

NOTE: The instructor conducting the LOS session will not act as a
substitute crewmember.

Crew composition for SPOT may include the use of a complete or partial
crew, depending upon the training objectives. The use of crew substitutes
in SPOT depends upon the type of training being provided.

With respect to an LOE, a complete crew complement should be

scheduled and maintained. Flight crewmember substitution is highly
discouraged. If crew substitutions are necessary, the substitute flight
crewmember will be either another line-qualified flight crewmember or a
task familiar flight crewmember in a training status comparable to the
person being evaluated. Evaluators conducting the LOE may not serve as a
substitute flight crewmember. The LOE substitution table/matrix will be
part of the air operator’s approved AQP documentation.

d) Instructor and evaluator Requirements. This part will address the

specific job functions, training, validation, or evaluation that instructors
and evaluators are authorized to perform. It will identify the title of each
position and describe the training that each receives in order to perform
the job function associated with that event.

e) Data Plan. Before an applicant can proceed with data collection and
analysis, it must establish the intended purpose and method for the
collection, entry, reporting, and analysis of AQP training/evaluation data
for each curriculum. The plan must be thorough and accurately reflect the
airline’s PPDB system. The AQP applicant must also acknowledge its
responsibility to collect and analyze more data than required for
submission to Transport Canada in order to adequately identify
performance trends and requisite changes to factors that impact the
performance.

(i) Data Collection. This part of the Data Plan should address the
methods used to collect performance/proficiency data for all
curricula. These methods will include the rationale for employing
the method as well as providing the data input medium (e.g., grade
sheets, computer-input screens, etc.) as examples that exemplify
the data acquisition rationale. In addition, the data collection

62
 method should explain data input quality control, security, and
usability.

(ii) Data Base and User Interface Management. This part of the
Data Plan should explain the means and strategy the air operator
intends to employ to enter, access and utilize the AQP
performance/proficiency data that is collected. Included in this
explanation should be:

• The type of software data management system

employed (e.g., relational database, spreadsheet etc.);

• The organization of the information in the electronic

medium (e.g., database definition, database table
relationships, spreadsheet description, etc.);

• A description of the user interface to this data

management system.

(iii) Data Analysis. This part of the Data Plan should discuss the type
of analysis that will be employed to facilitate the AQP
performance information needs of the air operator and Transport
Canada. This discussion of the data analysis must address how
each type of AQP data will be analyzed, including training and
evaluation feedback as part of determining the effectiveness of the
program. This discussion should be used as a preamble to the
annual AQP report.

(iv) Data Reporting. This part of the Data Plan must discuss the AQP
data reporting requirements that it must meet for Transport Canada
inspection and audit purposes to include format and frequency. In
addition, it should discuss the type of data reporting it will employ,
to include report example types (e.g., tabular reports, graphs),
frequency, and the air operator personnel for whom the reports are
intended.

2.3.17 Authorization

Transport Canada’s authorization to proceed with the implementation phase, following

the approval of all documents required under Phase II, marks the end of Phase II and the
beginning of Phase III.

63
2.4 Phase III: Implementation

2.4.1 Overview

In Phase III, the AQP applicant will acquire and test the resources required to support one
or both of the Qualification or Continuing Qualification curricula. These activities
include developing courseware, qualifying instructors and evaluators, Small Group Try-
outs (SGT), program revisions, and data submission. Included is the initial establishment
of an AQP Quality Assurance team that will provide quality assurance activities
throughout the program.

A. Resources

During this phase, the applicant will secure the training facility and equip
classrooms with training aids, courseware, and other features that contribute to
creating and maintaining a positive learning environment.

B. Training of Instructors and Evaluators

It is important for the air operator to have all instructor and evaluator AQP
documentation approved prior to implementing Phase III. This is necessary
because the applicant must train, evaluate, and qualify their instructors and
evaluators before beginning the SGT during this phase. Refer to Chapter 5 and
the AQP Evaluator Manual for more information on the training and quality
control of instructors and evaluators.

Note: It is important to note that the AQP Evaluator Course will replace the
traditional ACP Course. It will require specific authorization from
Transport Canada in order to ensure that it meets all of the requirements within
the AQP Evaluator Manual.

C. Conduct of the Authorized Curriculum Small Group Try-outs (SGT)

This training and evaluation will consist of SGT of each lesson using actual
students and instructor/evaluators. Evaluation of the SGT Formative Evaluation
will normally involve no jeopardy or credit for students, since its primary purpose
is to determine lesson suitability and effectiveness. The applicant may choose,
however, to give student credit for part or all training and qualification achieved
in the formative evaluation. The decision to give credit must be authorized by
Transport Canada before conducting the curriculum evaluation and the
air operator’s request to do so should be documented in the Implementation and
Operations Plan.

D. Revisions to Training Programs and AQP Documents

64
 Lessons learned to the program as a result of this step will be incorporated as
changes to the approved AQP documents from Phase I and Phase II. This would
include any required adjustments/changes to the JTA, Qualification Standards,
development methodologies (i.e., Line Operational Simulation (LOS), Curriculum
Design), curriculum outlines, footprints, and scripts. In addition, the Data Plan of
the Implementation and Operations Plan will be implemented for the curriculum
evaluated in this phase.

E. Performance/Proficiency Data Maintenance

The applicant will maintain and analyse the performance and proficiency data as
described in the applicant’s Data Plan. This crewmember data may be in
electronic, digitized format or other format as authorized by Transport Canada.
Any changes to format or procedures must be authorized by Transport Canada.
The maintenance and analysis of the performance/proficiency data will continue
for the life of this curriculum (i.e., through Phase III, IV and V).

F. Periodic Program Revisions

Periodic program revisions will be made to the courseware, hardware, software,

personnel, organization, and data collection and reporting system of the AQP,
based on system performance and the examination of performance and
proficiency data. These improvements will be implemented using the processes
and the procedures described in the applicant’s AQP Maintenance Strategy. This
plan is part of the Implementation and Operations Plan.

2.4.2 Authorization

Transport Canada AQP authorization process for Phase III, SGT is located in Chapter 3:
AQP Documentation Approval Process.

Transport Canada’s authorization to proceed with the implementation of initial

operations, following a satisfactory assessment of Phase III activities and outcomes,
marks the end of Phase III and the beginning of Phase IV.

2.5 Phase IV: Initial Operations

2.5.1 Overview

In this phase the applicant implements the Qualification or Continuing Qualification

curriculum, in accordance with the approved AQP documents. This phase of the
curriculum calls for the initial operation of the AQP in a “probationary” status for a
minimum of two years.

65
2.5.2 Phase IV Activities

During Phase IV, the applicant will implement and complete a full evaluation of the
AQP. This will include collecting program audit data and individual
performance/proficiency data, analysing the results and producing reports. In addition,
the applicant will continue to enhance and expand their quality assurance program to
maintain instructor and evaluator performance (i.e., mentorship program), curriculum and
courseware concurrence, suitability, and adequacy. Collected data will be used by:

a) The air operator for its internal quality control program to maintain curriculum
and courseware concurrence, suitability, and adequacy.

b) The air operator to analyze and validate flight crewmember performance.

c) The air operator and Transport Canada to analyze and validate instructor and
evaluator performance.

d) The air operator and Transport Canada to support analysis for special subjects,
such as CRM performance factors.

e) Transport Canada to analyze and validate curriculum performance.

f) Transport Canada to analyze and validate program development, implementation

and maintenance procedures.

2.5.3 Revisions to Training Programs and AQP Documents

The focus of this phase is the validation of the AQP curriculum by acquiring
performance/proficiency data and lessons learned in the course of conducting the
curriculum for 24 months. Applicants will summarize the lessons learned and
adjustments made to the curricula in an annual AQP report described in 2.7.1 as well as
in Appendix C. In addition, adjustments made to the AQP will be reflected in revisions
to the approved AQP documents. Performance/proficiency data must be of sufficient
reliability and validity to reasonably enable conclusions concerning the effectiveness of
the curriculum. Transport Canada approval of these revisions to the AQP will qualify an
applicant for entry into the final phase of the AQP process: Phase V, Continuing
Operations.

2.5.4 Authorization

Transport Canada’s authorization to proceed with the implementation of continuing

operations, following a satisfactory assessment of Phase IV activities and outcomes,
marks the end of Phase IV and the entrance into Phase V.

66
2.6 Phase V: Continuing Operations

2.6.1 Overview

In this phase, the applicant continues operation of the AQP unless Transport Canada
withdraws authorization or unless the applicant withdraws or modifies the AQP. This
phase requires the maintenance of the AQP approved documentation and updated
documentation of the data requirements for all curricula.

2.6.2 Data’s Impact on Continuing Operations

Data will continue to be collected and analyzed by the applicant and regularly audited by
Transport Canada for verification of student, instructor and evaluator proficiency. Data
will also be collected and analyzed by the applicant for:

a) Continued validation of the AQP;

b) Identification of requirements for curriculum changes;

c) Program maintenance.

2.6.3 Quality Assurance

For AQP to succeed, each applicant must pay particular attention to overall program
quality assurance. Continued validation of performance/proficiency data of the
individual and team, as achieved and maintained by all personnel, is particularly
important. This quality assurance would also include the performance of all instructors
and evaluators.

Continued validation of overall program completeness, accuracy, and currency, as

provided by the Performance Audit Data Base (PADB), is also very important. Elements
of program control should ensure that quality in proficiency is maintained throughout.
The air operator’s continued commitment to identify and execute required changes is
essential to a successful AQP. Transport Canada will expect any AQP quality assurance
program to identify needed changes in instructor and evaluator performance, curriculum,
courseware and equipment, and to make these changes before unwanted trends in reduced
proficiency manifest themselves. A proven and well-established quality assurance
program will allow the air operator to take advantage of several benefits. These benefits
include, among others, the application of data derived matrices, extended training and
evaluation cycles and biennial OEs.

67
2.7 Document and Reporting Requirements

2.7.1 Overview

The approved AQP documentation establishes the applicant’s regulatory requirements for
the particular training program. This fact makes it imperative that the applicant develops
a document structure that is organized and standardized. The structure must ensure that
the information contained within the AQP documents will be used and translated down to
the lowest level of training guidance. It must allow the applicant’s entire organization to
easily access and use the contents. Transport Canada has established a list of the
minimum documentation required for AQP. The applicant may want to develop
additional materials in support of their AQP.

A. Documentation Required by Transport Canada

Each document contains unique information that is integral to the AQP that will
be used when developing subsequent curricula. A majority of the documents,
once developed, will require periodic updates and therefore are subject to the
revision control process.

These documents should be submitted to Transport Canada in electronic format:

a) Application/Administration Document;

b) Job Task Analysis (JTA);

c) Qualification Standards;

d) Instructional Systems Development (ISD) Methodology;

e) Curriculum Outlines;

f) Implementation and Operations (I&O) Plan; and

g) Annual AQP Report.

B. Document Structures

Figure 2-8 provides an example of an AQP document structure. It is not required that an
applicant follow this structure. However, the document structure that is adopted must
easily identify the location of the AQP documents and more specifically, the required
AQP information. If the AQP documentation is part of a subset of other manuals, a
method should be developed to identify the manual and the specific AQP document that
it contains.

68
C. Annual AQP Report

AQP requires that each AQP certificate holder prepare an annual report for TC (see
Appendix C for specific requirements). This report is based on the certificate holder’s
analysis of the data that is collected during training and at strategic points
(validation/evaluation gates) in each curriculum and maintained in the PPDB. AQP
requires data collection and analysis in order to establish and maintain quality control of
curricula for flight crewmembers, instructors, and evaluators. Actual adjustments made to
the AQP are reflected in revisions to the approved AQP documents. The report should be
submitted to TC no later than 60 days past the end of the report period. The reporting
period is usually based on the authorization date for a particular curriculum in either
phase IV or V. During AQP development, particularly for multiple fleet operators, with
different authorization dates for multiple curricula, the reporting period may be modified
(as agreed upon by TC and the certificate holder). Once the certificate holder has all its
fleets and curricula into phase V, the reporting period can be fixed into a particular cycle.
Copies of the report should be distributed to the POI at least 2 weeks prior to the annual
AQP review meeting.

The annual AQP report should summarize the lessons learned and adjustments made to
the curriculum(s) during the reporting period. The report should also include projected or
proposed changes to the curriculum(s) based on the certificate holder’s current analysis.
As a minimum, the specific content of the annual report should consist of the following:

• An analysis of the information contained in the Performance and

Proficiency Database,
• Identification of any negative trends and potential deficiencies that
could result in decreased proficiency,
• A description of corrective measures taken and any resulting changes
to curricula,
• Any changes to AQP maintenance strategy,
• A comparison of data between equivalent periods in preceding
reporting periods,
• Any future operational changes that will affect the AQP, and
• Any additional safety data or information relating to flight crew
performance and procedures which were considered by the air operator
in applicable AQP training and evaluation strategies.

D. Annual AQP Review – Phases IV and V

An annual AQP review meeting between TC and the certificate holder should be held to
coincide with the submission of the annual AQP report in phases IV and V. The purpose
of the meeting is to formally discuss the results of the certificate holder’s data analysis,
program revisions, future revisions and the analysis of the data that is submitted to TC.
Additional information is available from Appendix C.

69
Fig 2–8: Document Structures Chart

Title
List of Effective Pages

Section I – All Organizational Standardized Documents:

Application/Administration Document
Instructional Systems Development Methodology
Implementation and Operations Plan

Section II - Qualification Training Curricula

Task Analysis
Qualification Standards
Curriculum Outline
Data Acquisition/Grading Forms

Section III - Continuing Qualification Training Curricula

Curriculum Outline
Data Acquisition/Grading Forms

Appendices A and B provide a more comprehensive listing of AQP documents.

70
Chapter 3 - AQP Documentation Approval Process

3.1 Documentation Review and Approval

3.1.1 Overview

This section establishes how Transport Canada reviews, grants or withdraws an

authorization for all or part of an AQP. Specific document content is discussed in
Chapter 2.

3.1.2 Review Process

Chapter 1 of this document discusses the roles and responsibility of the Transport Canada
personnel involved with AQP. Transport Canada prefers to interact closely with
applicants as AQP documentation is developed. Experience has shown that it is more
efficient for Transport Canada to discuss plans and review rough drafts of document
sections early in the development process. The submission of finished documents then
becomes more of a formality, with minimal changes.

3.1.3 Approval Process

Applicants develop, implement and operate the AQP in five sequential phases.
Transport Canada authorizes entry into the subsequent phase once it is satisfied that all
requirements of the previous phases have been met.

A. Approval Authority

The air operator’s designated POI will co-ordinate the review and approval of all
AQP document revisions in each phase of an AQP. Upon written
recommendation of the POI, Transport Canada will issue approval of specific
documents and revisions.

B. AQP Quality Assurance Team

Developing and implementing an AQP requires a commitment of resources from

both Transport Canada and the air operator. Transport Canada and the air
operator must use their resources in an effective and efficient manner to fulfil this
commitment. In order to provide a clear delineation of Transport Canada
expectations regarding reasonable progress towards AQP implementation,
Transport Canada requires that the operator establish an internal AQP Quality
Assurance team from the start of the program. This team should, at least, be
composed of AQP curriculum design, data, instructor and evaluator specialists
and fleet specific Subject Matter Experts (SMEs).

71
 Note: Further information can be found in Section 5.3 Quality Assurance and
Standardization

The AQP Quality Assurance team will report to the AQP manager. Roles and
responsibilities of this team will involve the conduct of quality assurance
activities throughout the development phases of the program. The team will make
recommendations to ensure that the program is in compliance with the CARs (as
applicable), AQP philosophies and methodologies as well as company approved
AQP policies and procedures. The role of this team is especially important during
Phase III (Small Group Try-outs (SGT)) where the initial transition from the
traditional training program to AQP takes place.

The AQP manager shall be responsible for ensuring that recommendations

originating from the team are properly addressed and implemented as deemed
appropriate. The implementation of the team’s recommendations shall be made
prior to Transport Canada’s program validation exercises. Transport Canada shall
be notified in the form of a report from the team, confirming that the program is
ready for a regulatory validation.

Note: Any conflict within the team shall be resolved by the AQP manager in
conjunction with the manager responsible for the operator’s overall training
program as applicable (i.e., Director of Training, and/or Chief Pilot, Training).

3.2 Transport Canada Approval Procedures

3.2.1 Phase I - AQP Application

Initial Application: Transport Canada’s review and acceptance of the initial application
marks the air operator’s formal entry into the AQP. The steps for the applicant are:

a) Submit the application to the applicable Transport Canada POI as appropriate;

b) Address comments and recommendations from Transport Canada.

Once Transport Canada accepts the application and approves the Application /
Administration document, it will issue a letter of authorization to the applicant, with
permission to begin Phase II.

3.2.2 Phase II – Curriculum Development

Curriculum Development: Phase II consists of five stages with corresponding

documents. Each of these stages is inter-related and must be completed in a specific
order. Because of this linking, Curriculum Development follows a sequential order of
document development: JTA, Qualification Standards, ISD Methodology, Curriculum
Outlines, and finally the Implementation and Operations Plan. After all Phase II required

72
documents have been approved, Transport Canada will issue a letter of authorization to
the applicant, with permission to begin Phase III.

3.2.3 Phase III - Implementation

As mentioned in 3.1.3 B. above, Transport Canada requires the establishment of a

Quality Assurance (QA) team. This team will conduct quality assurance on the SGT
activities in order to ensure that the program develops properly. Numerous challenges
are normally encountered during this phase. These will require constant vigilance and
guidance from the QA team. The team shall ensure that all regulatory requirements are
met during the conduct of the SGT. This is especially important for a credited SGT.

Transport Canada’s involvement at this stage is kept to a minimum in order to allow the
operator to adjust and amend their program as deemed appropriate. Once Transport
Canada receives a satisfactory report from the QA Team, the regulatory validation
activities commence.

Note: The duration of a SGT (before a regulatory validation) should not exceed 3 runs of
the curriculum. However, this may be extended at the discretion of the POI.

The following criteria are used by Transport Canada while validating Phase III events
(i.e., Small Group Try-out scripts).

A. AQP Philosophy

The AQP philosophy must be applied throughout the design of the program. A
Qualification Curriculum should guide the candidates through the program in a
progressive and systematic way. The introduction of new material (KSAs) that
focuses on manoeuvres training should have been completed at the end of the
Manoeuvres Training (MT) segment. MT is then validated in the Manoeuvres
Proficiency Validation (MPV).

In the LOFT segment of the training, the primary focus switches to CRM skills.
During LOFTs, the candidates should not be forced to concentrate on acquiring
new technical knowledge and new technical skills. To help flight crews to
develop their CRM skills, direct involvement from the facilitator should be kept
to a minimum.

Through careful analysis and planning, AQP creates a training program that
addresses both technical topics and crew resource issues. In so doing, AQP
prepares candidates for a seamless transition to line flying duties.

B. Line Operational Simulation (LOS) Methodology

The design of LOFT/LOE/SPOT must be done in accordance with the operator's

approved LOS methodology.

73
C. Approved Matrix

All items within the approved Matrix must be covered.

D. Realism

When designing LOEs and LOFTs, the objective is to realistically simulate "line"
operations. As a guideline, LOE and LOFT scripts should focus equally on
technical topics and CRM issues. Introducing too many technical topics may
create an imbalance. This is a common fault seen during script development.

E. Flow of Scripts

LOE and LOFT should be constructed to realistically simulate line operations.

The event sets should be sequenced in a chronological, logical order. No
repositioning is allowed during LOEs and LOFTs.

A logical flow in Manoeuvres Training (MTs) and Manoeuvres Validation (MV)

is also encouraged as much as possible. Repositioning is encouraged during MTs
and MVs. This allows repetition of exercises, reinforcement of learning and also
permits more efficient use of the simulator. MT sessions should be constructed
logically, using the building block approach.

F. Complexity (Level of Difficulty) and Scope of Scripts

In AQP the level of difficulty should not overwhelm and result in negative
training. In addition to level of difficulty, the scope (number of exercises) of each
training session should be carefully considered to prevent an overload of
candidates and instructors.

The demographics of the pilots should be considered and when required, the
program must be adjusted accordingly. For example, if a large group of pilots
transitioned from aircraft with traditional instruments (electro-mechanical “round
dials”) to new technology (“glass cockpit”) aircraft, a special training segment
within the overall program might be required. The introduction of such a segment
must be identified within the database and segregated in order to avoid
contaminating the general database.

G. Duration of Scripts

Scripts should be constructed so the all events can be completed in the allotted
time, assuming there are no simulator malfunctions or other mitigating
circumstances.

74
 Transport Canada uses the following guidelines when assessing the duration of
scripts:

A typical 4.0 hr simulator session would include the following:

a) Line Operational Evaluation (LOE) / Line Oriented Flight Training

(LOFT): (including SPOT if integrated within the simulator period):
These scripts must include time for at least 2 repeats (7.5 minutes each for
a total of 15 minutes) and a 10 minutes break between legs. Total time for
the script (excluding the time for a break and repeats) should not exceed
3hrs and 35 minutes.

b) Manoeuvres Procedures Validation (MPV) / Manoeuvres Training

Validation (MTV): These scripts must include time for a 10 minutes
break and an allowance for at least 2 repeats (5.0 minutes each). Total
time for the script (excluding the time for a break and repeats) should not
exceed 3hrs and 40 minutes.

Note: If the operator has included a warm-up period prior to the validation, then
the time allowed for the warm-up must also be included within the allotted
time for the script.

c) Manoeuvres Training (MT): These scripts must include time for a 10

minutes break and an allowance for at least 2 repeats (5.0 minutes each).
Thus total events should be designed for no more than 3 hrs and 40
minutes.

H. Functionality of Scripts in Applicable Simulators

All scripts must be tested and found to be acceptable in all simulators used by the
operator for a particular aircraft type.

I. Conduct of Scripts

The scripts must be conducted in accordance to the operator's authorized

instructor and evaluator course. The following issues are of particular interest:

a) Instructor and evaluator's preparedness.

b) Adherence to scripts.

c) Role playing by the instructor and evaluator as applicable to the type of

script.

d) Time management as appropriate.

75
 e) Intervention by the instructor and evaluator as appropriate and where
applicable.

f) Instructional techniques as appropriate.

g) Handling of mitigating circumstances (i.e., what to do when simulator

malfunctions or when the crew takes an “unforeseen” plan of action).

h) Ability to operate the simulator correctly.

i) Ability to brief and de-brief candidates in accordance with the authorized

instructor and evaluator course.

j) Knowledge of the AQP program with particular emphasis on their roles

and responsibilities (i.e., ensuring that the proper level of proficiency is
reached, proper grading assessment, proper conducts of repeats).

k) Attitude towards the AQP program.

J. Program Operation

Proficiency data, maintenance plan, quality assurance, data collection, analysis

and reporting systems will be evaluated for consistency and accuracy.

K. Implementation and Operations Plan Review

Once Phase III is concluded, the applicant will provide Transport Canada with an
updated Implementation and Operations Plan. Subject to the recommendation of
the POI, Transport Canada will be in a position to acknowledge the completion of
Phase III. This constitutes authorization for using the individual curricula and
will authorize the applicant to enter Phase IV. The applicant may now execute
the updated Implementation and Operations Plan.

3.2.4 Phase IV- Initial Operations

In this phase the applicant will implement the AQP as defined in the updated
Implementation and Operations plan through one complete cycle of the Continuing
Qualification curriculum.

A. Review and Surveillance

Transport Canada activities in this phase will consist of surveillance and audit of
AQP operations as well as analysis of data collection results. Joint Transport
Canada and applicant reviews will be conducted periodically. These meetings
will provide both parties the chance to analyze results and discuss program

76
concerns. A brief report of the findings and solutions will be submitted to
Transport Canada by the air operator.

A final joint review will be accomplished prior to the end of Phase IV initial
authorization (typically following 24 months of Phase IV activities). The review
will identify any changes engendered by initial data analysis reports and
demonstrate that the data collection and analysis process is still adequate and
sufficient. In addition, this review should verify that the current proficiency
measures for the Qualification Standards are reliable and valid and that the AQP
is being maintained in accordance with the Implementation and Operations Plan.

Focus areas for these reviews are:

a) Data Management:

i) Collection;

ii) Analysis;

iii) Standardization;

iv) Observations;

v) Additional training;

vi) First-Look;

vii) Program feedback.

b) Record Keeping.

c) Adherence to Implementation Plan.

d) Modifications to the program.

e) Qualification Standards reliability and validity.

f) Maintenance of the AQP.

g) Currency of the PADB.

h) Instructor and Evaluator Program.

i) Special Tracking.

j) Transition to AQP for Non-AQP Fleets.

77
 Once the final joint review has been completed, the applicant will update the
Implementation and Operations Plan to include the changes recommended by
Transport Canada. The applicant will submit a copy of the updated
Implementation and Operations Plan to the POI.

B. Quality Assurance Program

Transport Canada will validate the operator’s QA program to determine if all

appropriate processes are being satisfactorily implemented. A proven and
established QA Program (including mentorship program for instructor and
evaluators) is essential for advancement into Phase V.

3.2.5 Phase V - Continuous Operations

In this phase, oversight of the AQP is maintained through ongoing Transport Canada
surveillance and audit programs. The air operator will report on the maintenance of the
AQP through documentation changes and program revisions. In addition, the air operator
will continue to provide a copy of its Annual AQP Report to Transport Canada and hold
an annual AQP review meeting with TC.

The operator will have a proven and fully established Quality Assurance (QA) Program.
Subject to the authorization of Transport Canada, innovative approaches and deviations
from the “traditional” requirements of the CARs may be considered at this stage of the
program. All changes of this nature must be supported by data analysis. Among the
possibilities which could potentially be considered are changes to the Online Evaluation
(OE) training and evaluation cycles and the introduction of data derived methodologies.

3.3 Withdrawal Of Authorization

Transport Canada may withdraw interim, conditional or final authorization any time the
AQP is not in regulatory compliance, does not provide for safe operations, does not
effectively prepare personnel to meet qualification objectives, or when the required data
is not being provided or maintained. Before withdrawing authorization, Transport
Canada will make every reasonable effort to work with an applicant to correct its
program deficiencies.

78
Chapter 4 - Curricula and Certification

4.1 Curricula

4.1.1 Background

The AQP requires two primary curricula for each type, model, and series of aircraft (or
variant), and each duty position. These curricula are Qualification and Continuing
Qualification. Figure 4–1 illustrates how traditional curricula correlate to the AQP
primary curricula. In addition to primary curricula, operational necessities may require
secondary curricula to fulfil particular needs. Secondary curricula are Transition,
Upgrade, and Requalification.

Transport Canada continues to recognize, in its regulations, a need for indoctrination

training for flight crewmembers new to the air operator. AQP however is first and
foremost a vehicle to train and qualify flight crewmembers on a specific type of aircraft
and in a particular duty position. Indoctrination-type knowledge and information must
still be imparted to new flight crews, but are not required to be developed and delivered
using an AQP methodology. Indoctrination topics of interest continue to rest with
company policies, practices and general operational knowledge, and contain elements
that pertain to the air operator’s methods of compliance with regulations and safe
operating practices.

79
Fig 4-1: AQP Curricula

Traditional AQP
Indoctrination Traditional
Indoctrination

Initial Equipment
Qualification

Transition

Certification
Upgrade If not previously
qualified

Requalification

Recurrent
Continuing
Qualification
Upgrade If previously
qualified

80
4.2 Qualification Curriculum

AQP requires a Qualification Curriculum for each duty position in each type, model, and
series aircraft (or variant). To this end, the air operator shall develop Operator Difference
Requirement (ODR) Tables for each applicable aircraft in its fleet. Each Qualification
Curriculum will include training, validation, evaluation and pilot licensing action, as
applicable. The training activities include ground and flight training, operational
experience, and may include special qualification.

If the training is to result in licensing action for issuance of an Instrument and/or Type
Rating, curriculum segments must explicitly identify the training and evaluation strategy
to be used in place of the prescribed practical test requirements outlined in Part IV and
VII of the CARs. The applicant must demonstrate, to the satisfaction of Transport
Canada, that the proposed AQP training and evaluation strategy will ensure individual
competency that equals or exceeds CARs requirements. The applicant must also
demonstrate that each person qualified through an AQP has demonstrated satisfactory
proficiency in the integration of technical and CRM skills.

4.2.1 Ground Qualification Training Activities

To be qualified for a particular duty position, a person will receive job-specific ground
training. This training typically includes general operational subjects, technical systems,
system and procedures integration as well as emergency-type training. Ground
Qualification is confirmed by a Systems Knowledge Validation (SKV). This may take
the form of a traditional written examination. It may also incorporate other means of
systems knowledge validation (e.g., computer-based), as authorized by Transport Canada.

4.2.2 Simulation/Flight Qualification Training Activities

Each AQP includes curriculum segments for training, validation, or evaluations as

appropriate in FTD and simulators. The use of FTDs or higher-level training devices in
training is encouraged. Manoeuvres Validation, LOFT and the LOE must be
accomplished in a full flight simulator that has been approved for this purpose. However,
upon satisfactory small group try-out demonstrations, LOFT may be conducted in an
FTD with authorization from Transport Canada. Training and evaluation in an aircraft is
discouraged, but may be authorized by Transport Canada on a case-by-case basis.

4.2.3 Initial Operating Experience (IOE)

IOE replaces traditional Line Indoctrination. IOE curriculum segments are integral to the
Qualification Curriculum. IOE provides hands-on training and actual experience in
performing the duties of a newly assigned flight crew. An Initial Operating Experience
Training Captain (IOETC) conducts IOE during actual line flying operations. Validation
is accomplished when the individual is trained to proficiency and recommended for an
Online Evaluation (OE).

81
4.2.4 Special Qualification Training

Curriculum segments may include special purpose training. These are portions of ground
or flight training that has a specific application to flight crewmembers who are in
international operations or for the introduction of new flight operations. Examples
include: international operations, Extended Twin Engine Operations (ETOPS) and
special airport operations. Special qualification training may initially be a separate
curriculum segment that is later integrated into ground or flight training segments.

4.2.5 Special Purpose Operations Training (SPOT)

This is a portion of a Line Operational Simulation (LOS) training scenario consisting of

flight tasks selected from any phase or phases of flight to provide practice in the
integration of technical and CRM skills appropriate to the selected flight tasks. SPOT is
conducted using a complete cockpit flight crew to the maximum extent feasible and is
accomplished in a full flight simulator. However, in some cases, SPOT may be
conducted in an FTD with authorization from Transport Canada.

4.2.6 Validation/Evaluation/Remediation

In AQP, a “validation” is a determination that the training produces the required results
as identified in the Qualification Standards and that the individual has met the
performance objectives of the training module. An “evaluation” is an appraisal of an
individual to ascertain whether the standards required for a specified level of proficiency
has been successfully demonstrated. Interrupting the evaluation session for training is
not permitted. Both the validation and evaluation are assessments that the proficiency
objectives of the training module have been met and the individual can proceed to the
next level of training or line operations. Figure 4-2 contains a table that summarizes the
relationship between the various training activities and their associated
validation/evaluation gates. Figure 4-3 provides a more detailed look at the
validation/evaluation gates and the associated remediation.

Note: These validations and evaluations, including the associated remediation

strategies, are fully described in the AQP Evaluator Manual.

82
Fig 4-2: AQP Training, Validation, & Evaluation

Training/
Activity Device Purpose Can it be Validation or Sequencing of
Interrupted? Evaluation Events

Classroom or
Ground Computer Based Training
Instruction, and Systems Training Yes & Syllabus
Training Training Devices Validation

Manoeuvres Isolated
and FTD & Simulator Aircraft Training Manoeuvres and
Manoeuvres and Yes & Procedures.
Procedures Operational Validation Logical, Specific
Training Procedures Sequencing of
Events
Special Purpose Focuses on CRM Isolated
Operational FTD & Simulator Skills, Differences Manoeuvres and
Tng., Wind shear Yes Training Procedures.
Training Tng., Special Logical, Specific
(SPOT) Qualification Sequencing of
Events
Line Oriented No Logical Sequencing
Flight FTD & Crew Oriented of Events Within
Simulator Training (CRM) in Except To Begin Training Varying Length
Training Preparation for Different Scenarios Scenarios
(LOFT) LOE
Line May be Segmented
Operational FTD & Evaluate the to Condense Specific Flight
Simulator training and Distances as in Evaluation Scenario from
Evaluation qualifications of the International Take-off to
(LOE) crewmembers Scenarios Landing
Consolidation of
Operating Aircraft knowledge & skills Routine Flight
Experience in a operational Yes Experience Operations
environment
To verify a
Qualification Aircraft crewmember’s Routine Flight
Online ability to No Evaluation Operations
satisfactorily
Evaluation perform duties &
(OE) responsibilities

To evaluate crew
Continuing Aircraft proficiency and Routine Flight
their knowledge, No Evaluation Operations
Qualification skills, and ability to
Online operate effectively
Evaluation as part of a crew.
(OE)

83
Fig 4-3: AQP Validation/Evaluation Table

Gate Qualification Curriculum Continuing Qualification Evaluation

Curriculum Media

System 80% or Greater 80% or Greater Written, Oral, or

Knowledge corrected to 100%, corrected to 100%, Electronic Testing
Validation System (ETS)
(SKV) < 80% retest < 80% retest

Procedures Train to As Authorized

Validation Proficiency
(PV)

Manœuvres 2 Repeats of One Repeats Allowed Approved

Validation Manoeuvre, Within Training Simulation Device
(MV) or Period
1 Repeat Of Any 2
Manoeuvres*

Line Operational Grading By Event Set Grading By Event Set Approved

Evaluation Simulation Device
(LOE) 8 or more Event Sets 8 or more Event Sets
Repeat - 2 Event Sets** Repeat - 2 Event Sets **

Online Any Unsatisfactory Task Any Unsatisfactory Task must Aircraft

Evaluation must be Repeated. be Repeated. (Actual Line
(OE) Operations)
If Required, Remedial If Required, Remedial
Training, Additional Initial Training, Additional Initial
Operating Experience, and Operating Experience, and
Another Operational Another Operational
Evaluation as Evaluation as Recommended
Recommended by the by the evaluator
evaluator

* For any repeat in a Qualification Manoeuvres Validation (MV), Qualification LOE or

Continuing Qualification LOE, no training, practice, or coaching is permitted.

** Failure to meet the required level of proficiency during an LOE or OE will result in retraining, re-
evaluation, and will require special tracking.

** Regardless of number of events sets, unsafe individual or crew performance that would result in
significant damage, hull loss or loss of life (e.g., crash) during an LOE constitutes a failure of the LOE.

A. Systems Knowledge Validation (SKV)

84
 This is an assessment of an individual’s technical systems knowledge. The intent
of the Systems Knowledge Validation session is to ensure an individual’s systems
knowledge is at an appropriate level before progressing into the next training
phase. Systems Knowledge Validation may be accomplished by a written,
electronic or oral exam. An overall score of 80 percent or better, corrected to 100
percent, would be an acceptable validation. An overall score of less than 80
percent will require retraining and another complete test. A failure of an
individual test module or sub-section, with an 80 percent or better overall score,
only requires retraining and retesting of the specific module. Consideration
should be given to establishing a maximum number of modules or sub-sections
that if failed constitute an overall failure of the validation.

B. Procedures Validation (PV)

This is an assessment of an individual’s systems integration knowledge and skill.

This validation addresses the individual’s ability to assimilate system and
procedural knowledge into the appropriate execution of procedures. This
validation session should take place in an FTD or a simulator. The intent of the
PV session is to ensure an individual’s systems and procedural knowledge and
skills are at an appropriate level before progressing into the flight simulator
training phase. There is no established requirement that the systems and PV be
accomplished sequentially. It is the prerogative of each air operator to develop an
evaluation system that works for them. For example, the systems validation could
take place in two parts. The first part may be a written exam administered at the
conclusion of ground school. The second stage could be an oral examination
conducted at the conclusion of procedures training, before the Manoeuvres
Validation (MV) or before the LOE. Validation is accomplished when it is
verified that the individual is trained to proficiency.

C. Manœuvres Validation (MV)

This validation addresses the individual’s proficiency in the execution of

manoeuvres. It must take place in a full flight simulator. In order to differentiate
between the Manoeuvres Validation (MV) conducted in a Qualification
Curriculum and a Continuing Qualification Curriculum, the following terms have
been established:

a) Manoeuvres Proficiency Validation (MPV) for the Qualification

Curriculum

b) Manoeuvres Training and Validation (MTV) for the Continuing

Qualification Curriculum.

The essential difference between the MPV for Qualification Curricula and the
MTV for Continuing Qualification Curricula is the manner in which repeats of
unsuccessful exercises are addressed.

85
 For a MPV (Qualification Curriculum), flight crewmembers are expected to have
reached a satisfactory level of proficiency in the manoeuvres prior to the
validation event. An MPV should not allow more than two repeats of any one
manoeuvre or one repeat of any two manoeuvres. A debriefing of why the
manoeuvre(s) was unsatisfactory is allowed, but the repeats must occur with no
training, practice, or coaching. If the flight crewmember fails to demonstrate
proficiency in the time constraints of the simulator session, an additional training
session is required. After additional training, the individual need only repeat the
manoeuvres that were unsatisfactory.

D. Line Operational Evaluation (LOE)

This evaluation addresses the individual’s ability to demonstrate technical and

CRM skills appropriate to fulfilling job requirements in a full mission scenario
environment. The intent of a LOE is to evaluate and verify that an individual’s
job knowledge, technical skills, and CRM skills are commensurate with AQP
Qualification Standards and that the individual is qualified to begin the Initial
Operating Experience portion of the Qualification Curriculum. LOE is
considered a jeopardy event and a failure is reported to Transport Canada.

LOEs are graded at the event set level. A LOE consists of a minimum of 8 events
sets. During the LOE, two events sets can be repeated. No single event set can be
repeated more than once. A debriefing of why the event set(s) is unsatisfactory is
allowed, but the repeat must occur with no training, practice, or coaching. If any
repeated event is unsatisfactory, remedial training and another complete LOE
evaluation is required.

NOTE: Regardless of number of events sets, unsafe individual or crew

performance that would result in significant damage, hull loss or loss of life (e.g.,
crash) during an LOE constitutes a failure of the LOE.

E. Online Evaluation (OE)

OE replaces the traditional Line Check. Flight crewmembers receiving this

evaluation are assessed for their proficiency in the duty position. Successful
completion of the OE verifies that the individual is adequately trained and is
capable of performing their duties and responsibilities. If any task is
unsatisfactory, the individual must receive remedial training on that task,
additional operating experience if necessary, and possibly another OE. If a pilot
receives an unsatisfactory overall performance rating on an OE, the pilot cannot
progress to line operations until the accepted means of remediation (additional
training as required) has been successfully completed.

86
4.2.7 Planned Hours

All curricula will include planned hours for ground training, flight training, evaluation,
and operating experience. Planned hours represent the estimated amount of time, as
specified in a curriculum outline, that it takes an average candidate to complete a segment
of instruction. This will include all instruction, demonstration, practice and evaluation, as
appropriate, to reach proficiency. Planned hours enable Transport Canada and the air
operator to schedule their personnel resources more efficiently and provide a baseline for
curriculum adjustments. Planned hours are not used by Transport Canada as a basis for
program authorization, review, or compliance assessment. Planned hours can be shown
on the course footprint that is part of the curriculum outline. Figure 4-4 provides an
example of a course footprint with planned hours.

87
 Figure 4-4: Example of AQP Qualification Curriculum Footprint, Transport
Category Airplane operated under Subpart 705 of the CARs

Day 1 Day 2 Day 3 Day 4 Day 5

Welcome :30 CBT 5:00 CBT 5:00 CBT 5:00 Evacuation 3:00
Intro :30 Review 1.00 Review 1.00 Performance 1.00 Ditching 3:00
CBT 5:00 Day Off Day Off

TT 6:00 TT 6.00 TT 6.00 TT 6.00 TT 6:00

Day 6 Day 7 Day 8 Day 9 Day 10
CBT 5:00 CBT 3:00 CBT 5:00 CBT 3:00 SYSTM VAL
Setup Lect. 1.00 FTD Brief 1.00 Review 1.00 FTD Brief 1.00 Testing 1:00
FTD # 1 2.00 FTD # 2 2.00 Flt Ops Brf. 4:00 Day Off Day Off
Debrief :30
TT 6.00 TT 6.00 TT 6.00 TT 6.00
TT 6:30
Day 11 Day 12 Day 13 Day 14 Day 15
Sys Rev 1:00 FTD Brief 1:30 FTD Brief 1:30 FTD Brief 1:30 Proc Val
FTD Brief 1.00 FTD # 5 4.00 FTD # 6 4.00 FTD # 7 4.00 FTD Brief 1:30
FTD # 4 4.00 Debrief 1:00 Debrief 1:00 Debrief 1:00 FTD # 8 4.00 Day Off Day Off
Debrief 1:00

TT 6.00 TT 6.30 TT 6.30 TT 6.30 TT 6.30

Day 16 Day 17 Day 18 Day 19 Day 20
Sim Brief 1:30 Sim Brief 1:30 Sim Brief 1:30 Sim Brief 1:30 MAN VAL
Sim # 1 4.00 Sim # 2 4.00 Sim # 3 4.00 Sim # 4 4.00 Sim Brief 1:30 Day Off
Debrief 1:00 Debrief 1:00 Debrief 1:00 Debrief 1:00 Sim # 5 4.00 Day Off
Debrief 1:00

TT 6.30 TT 6.30 TT 6.30 TT 6.30 TT 6.30

Day 21 Day 22 Day 23 Day 24 Day 25
LOFT Brief LOFT Brief LOFT Brief LOFT Brief LOE
1:30 1:30 1:30 1:30 Brief 1:30 Day Off Day Off
LOFT # 1/SPOT LOFT # 2/ SPOT LOFT # 3/ SPOT LOFT # 4/ SPOT LOE 4:00
4.00 4.00 4.00 4.00 Debrief 1:00
Debrief 1:00 Debrief 1:00 Debrief 1:00 Debrief 1:00
TT 6:30

TT 6.30 TT 6.30 TT 6.30 TT 6.30

Day 26 Day 27 Day 28 Day 29 Day 30

IOE IOE IOE IOE IOE

Day 31 Day 32 SYST VAL - Systems Validation CBT – Computer-based
PROC VAL - PV training
IOE OE MAN VAL - Manoeuvres Validation FTD - FTD
LOE - LOE Sim - FFS
SPOT - Special Purpose Operations LOFT - LOFT
Training IOE - Initial operations
experience
OE –Online Evaluation

88
4.3 Continuing Qualification Curriculum

A Continuing Qualification curriculum provides the means for fully qualified individuals
to maintain their proficiency in their duty positions and aircraft assignments. Continuing
Qualification applies to all persons subject to an AQP, including instructors and
evaluators. AQP requires a Continuing Qualification curriculum for each duty position in
each type, model, and series aircraft (or variant). Similar to the Qualification
Curriculum, each Continuing Qualification Curriculum will include training, validation,
and evaluation.

4.3.1 First-Look Manoeuvres

Analysis of AQP data may allow for modified or extended training and evaluation cycles
once the program moves into Phase V. In order to substantiate modifications to training
and evaluation intervals, the air operator must have previously implemented First-Look
Manoeuvres (FLM) and collected sufficient data through one full Continuing
Qualification Cycle in order to establish a base line by which to measure the effect of
modified intervals. FLM are those manoeuvres, procedures or tasks that are identified as
likely to be sensitive to loss of proficiency due to infrequent practice or exposure.

The principal purpose of FLM is to test the retention of the flight crews in performing
these manoeuvres over the evaluation cycle. FLM are an AQP requirement as soon as
flight crewmembers are subject to a Continuing Qualification Curriculum. FLM are also
a valuable tool that can be employed as a means of validating that currency items are
performed in line operations with sufficient frequency that proficiency is being
maintained.

An AQP-qualified instructor may conduct first look proficiency assessment in a Level C

or higher FFS. However, if an applicant proposes to request manoeuvres validation
credit for critical first-look manoeuvres, the applicant must ensure that the first-look
proficiency assessment is accomplished by an AQP-qualified evaluator, rather than by an
instructor. During First-Look, evaluators must employ the same measurement
methodology and rating criteria as used in Manoeuvres Validation. The First-Look
grades are analyzed to determine if trends of degraded proficiency exist.

There are four considerations for First-Look Manoeuvres proficiency assessment:

a) Composition of the manoeuvres list,

b) Strategy for testing the manoeuvres,

c) Administering the test, and

d) Remediation.

89
A. List of Manoeuvres

The list of First-Look Manoeuvres is developed by the applicant. This list may be
data derived provided that the methodology has been accepted by Transport
Canada. First-Look items are performed, graded and analyzed to validate that
flight crews can maintain proficiency in these items between training intervals.
These may also include certain items given a designation of “Currency” in the
Qualification Standard, in order to facilitate initial validation that these items are
being performed outside of training with sufficient frequency that proficiency is
being maintained.

B. Testing Strategy

The testing strategy the applicant develops for First-Look is part of the
Implementation and Operations Plan (see Chapter 2). An ideal approach would
be to develop a list of several critical and/or currency items that will be sampled
using a controlled sampling technique. This would ensure that each of the items
is adequately and evenly assessed during the evaluation period. It is important to
remember that First-Look testing is not as much an assessment of an individual's
skills, as it is a measure of the collective retention of proficiency by flight crews.
Individual assessment occurs in MV and LOE only. The data that is collected
from First-Look testing is used for trend analysis and as a tool to validate the
AQP's overall effectiveness.

C. Administration

First-Look items must not be briefed in advance of the first execution of such
manoeuvres. Proficiency data must be collected before the repeated execution of
any such First-Look item during training in a flight simulator. There are several
options as to when the First-Look Manoeuvres testing should be conducted. For
example, First-Look Manoeuvres could be introduced as the first event of a
simulator training session addressing manoeuvres. Another option would be to
allow the flight crew an opportunity to “warm up” to the simulator by doing other
pre-briefed manoeuvres prior to First-Look. Other options would be to make it
part of an event in a LOFT or SPOT. The common element in all such options is
that proficiency is assessed the first time the First-Look item occurs in training.

D. Remediation

First-Look Manoeuvres proficiency assessment is considered a no jeopardy event,

subject to the requirement that any manoeuvres unsuccessfully accomplished be
trained to proficiency prior to the LOE.

4.3.2 Training Activity

90
Continuing Qualification curricula should achieve a proper balance between training and
evaluation.

Continuing Qualification curricula should typically outline a uniform timetable for the
following activities:

A. Continuing Qualification Ground Training Activities

Continuing Qualification training includes ground instruction and evaluation for

flight crewmembers, instructors and evaluators. This training includes a review
of the information covered in Qualification training, updated as appropriate.

B. Continuing Qualification Flight Proficiency Training

Flight crews and those instructors and evaluators who conduct flight training or
flight evaluations will complete proficiency training designed for their respective
duty position. This training may be achieved in an aircraft, flight-training device,
or flight simulator. Flight proficiency training permits flight crews to experience
and practice the procedures and manoeuvres that are not normally encountered in
day-to-day flight operations such as alternate, abnormal, and emergency flight
events. For instructors and evaluators whose duties are limited to flight
simulators and FTDs, flight proficiency training may be conducted in flight
simulators and FTDs.

C. Special Purpose Operational Training (SPOT)

These training segments in Continuing Qualification curricula are used for the
same purposes as in qualification curricula.

4.3.3 Validation/Evaluation/Remediation

Continuing Qualification must include validation/evaluation in all events and major

subjects required for original qualification. This requirement is met through proficiency
evaluations and OEs.

A. Manoeuvres Validation (MV)

The MV session in the Continuing Qualification curriculum allows assessment and

attainment of technical proficiency in the training program prior to evaluation in the
LOE. In Continuing Qualification curriculum training, repeats are allowed and are
not counted as an evaluation repeat. In a Continuing Qualification curriculum, MV
must be successfully completed within the time limits of the standard company
scheduled simulator session (national norm is approximately 2 hours per flight
crewmember) or an additional training period is required. If an individual requires
additional training periods to be able to demonstrate proficiency, consideration
should be given to placing the individual in special tracking.

91
B. Line Operational Evaluation (LOE)

The LOE is the primary proficiency evaluation. The LOE is conducted in a

simulation device approved for its intended use in the AQP. Under extenuating
circumstances, the AQP proficiency evaluation may be accomplished in an
aircraft, subject to Transport Canada authorization. The purpose, administration,
and remediation strategy for the Continuing Qualification Curriculum LOE is the
same as for a Qualification Curriculum.

C. Online Evaluation (OE)

Irrespective of the length of the CQC, an OE must be scheduled on an annual

basis following initial qualification.

Note: These validations and evaluations, including the associated remediation

strategies, are fully described in the AQP Evaluator Manual.

4.3.4 Flight Crewmember Recent Experience

The applicant’s AQP should show compliance with the currency experience requirements
as outlined in the CARs/CASS. These currency requirements, if not met during line
operations, may be satisfied through a flight currency reestablishment activity specified
in the Continuing Qualification Curriculum. Currency activities for instructors and
evaluators who are not regular line flight crewmembers will be specified in each AQP.
These instructor and evaluator activities should enable each instructor or evaluator to
maintain proficiency in teaching and evaluating the events he/she is authorized to
perform.

4.3.5 Cycles and Evaluation Period

The time period during which all proficiency objectives are trained, validated, or
evaluated for all crewmembers is called a Continuing Qualification Cycle (CQC). Figure
4-5 illustrates an example of a Continuing Qualification Cycle (following initial
qualification). A CQC is initially based on a 2-year matrix (24 months). This CQC is
typically divided into two 12-month Evaluation Periods. All Critical Proficiency
Objectives must be evaluated during each Evaluation Period. All Currency Proficiency
Objectives must be accomplished during each Continuing Qualification Cycle. It is
important to remember that Criticality and Currency does not pertain solely to TPOs, but
can also apply to SPOs, dependent on the air operator's Job Task Analysis. CQC
intervals can be modified when the program reaches Phase V given adequate justification
based on program data analysis.

A. Schedule

92
 The Continuing Qualification Cycle footprint must provide sufficient detail to
show compliance with the CASS. Elements of ground training activities, flight
training activities, validation, evaluation and currency activities are specifically
identified. The schedule for the cycle should specify the period between each
type of activity such as Manoeuvres Training (MT), LOFT, MTV and LOE. It
should also specify the order in which each activity is to be performed.

Developing a Continuing Qualification activity schedule involves selecting and

arranging modules (with related proficiency objectives) from the Qualification
Curriculum. These modules are regularly revisited to maintain both individual and
crew proficiency. Each Continuing Qualification Curriculum will identify the
frequency of training sessions for each person qualified under an AQP.

B. Training and Evaluation Periods

The Continuing Qualification Cycle must be divided into Evaluation and Training
Periods. All critical proficiency objectives shall be evaluated during each
Evaluation Period. A typical, twelve-month Evaluation Period will be valid until
the first day of the thirteenth month following the month in which the evaluation
was completed. Once in Phase V however, Evaluation Periods may be longer or
shorter than twelve months. For illustration purposes, a sixteen-month Evaluation
Period, for example, would be valid until the first day of the seventeenth month
following the month in which the evaluation was completed.

Each Evaluation Period shall have one or more Training Periods during which a
training activity occurs. A typical, six-month Training Period will be valid until
the first day of the seventh month following the month in which the training was
completed. Once in Phase V however, Training Periods may be longer or shorter
than six months. For illustration purposes, an eight-month Training Period, for
example, would be valid until the first day of the ninth month following the
month in which the training was completed.

Within the traditional program, when a pilot proficiency check (PPC) or training
is renewed within the last 90 days of its validity period, its original anniversary
date can be maintained. A similar provision exists for air operators using AQP
that are maintaining 6/12 month training and evaluation periods: If the flight
crewmember’s evaluation or training is renewed within 90 days of its validity
period, then the original anniversary date can be maintained.

However, for AQP air operators that are authorized for training and evaluation
periods greater than 6/12 months, the original anniversary date can only be
maintained if the training or evaluation occurs within the last 60 days of the
validity period.

93
Figure 4-5: Continuing Qualification Cycle (Example Using a Phase V, 32-Month
Matrix)

8 Months 16 Months 24 Months 32 Months

Continuing Qualification Cycle (1)

Evaluation Period (2) Evaluation Period (2)

Training Training Training Training

Period Period Period Period

Initial Training Training & Training Training &

Qualification Evaluation Evaluation

MPV/LOE MT/LOFT MTV/LOE MT/LOFT MTV/LOE

Note 1: All Currency Proficiency Objectives must be evaluated during each Continuing Qualification
Cycle.

Note 2: All Critical Proficiency Objectives must be evaluated during each Evaluation Period.

C. Extensions

94
 For air operators that are maintaining 6/12 month training and evaluation periods,
a 60-day extension may be granted, if the Minister is of the opinion that aviation
safety is not likely to be affected. When an air operator is authorized to maintain
training and evaluation periods longer than 6/12 months, a 30-day extension may
be granted, if the Minister is of the opinion that aviation safety is not likely to be
affected.

Extensions are only considered for unforeseen circumstances that are beyond the
air operator’s control. These unforeseen circumstances could include such things
as illness and simulator breakdown. Extensions will not be granted due to poor
planning, scheduling conflicts or lack of proper preparation.

D. Validation

The Continuing Qualification Cycles and Evaluation Periods are subject to

continued demonstration of overall effectiveness. The demonstration will be
dependent on the data submitted by the applicant for program validation and
Transport Canada surveillance. To ensure adequate individual and crew
qualification, an applicant must show that its AQP has the capability to monitor
each individual’s demonstrated proficiency. Included within this validation is the
introduction of First-Look Manoeuvres.

E. Dual Qualification

For the purposes of AQP, an individual is deemed to be “dual qualified” if, during
the Continuing Qualification Cycle following an AQP proficiency evaluation
(LOE), the individual performs flight crew duties in an additional aircraft type. If
maintaining qualification in more than one aircraft type in accordance with the
definition of “dual qualification” above, the individual will have one aircraft type
designated as the “primary” type. The other aircraft type on which they are
maintaining qualification will be designated as the “secondary” type.

F. Training Cycle

A person, who is qualified on more than one aircraft type or in more than one
duty position on different aircraft types, should be simultaneously enrolled in a
separate Continuing Qualification Curriculum for each assigned aircraft and duty
position. For each aircraft type on which he/she is maintaining qualification, the
individual flight crewmember must accomplish each of the relevant aircraft’s
Continuing Qualification Curriculum in its entirety. Those training items that are
not “fleet specific” in nature need only be addressed in the primary aircraft’s
Continuing Qualification Cycle.

G. Online Evaluation

95
 In addition, the individual must accomplish at least one OE during the Continuing
Qualification Cycle of the "primary" aircraft. The flight crewmember should be
subject to OEs on each additional type prior to repeating the OE on any single
type.

H. Multiple Duty Positions

A person assigned simultaneously as a flight crewmember, instructor, and/or

evaluator on the same aircraft, may be enrolled in a Continuing Qualification
Curriculum which combines the activities necessary to maintain skill and
proficiency in all duty positions.

4.4 Secondary Curricula

Under a traditional training program, an air operator may develop secondary curricula by
using their approved training program as a basis. Under AQP, an air operator may
proceed in much the same manner. The authorized AQP curricula can be used as a
reference to generate secondary curricula. Developing a secondary curriculum entails
selecting, revising, and arranging modules with related Proficiency Objectives from both
primary curricula. In all cases, the TPOs, SPOs, and EOs must include CRM principles
and include the use of Line Operational Simulation (LOS) for training and evaluation.
The differences between the traditional training/checking regulatory requirements and
those specified in an air operator's AQP will be identified by the applicant.

Validation and evaluation strategies (i.e., MPV versus MTV) for secondary curricula are
based on the strategy employed under the primary curriculum (Qualification or
Continuing Qualification) from which they were developed. Therefore, it is essential that
all secondary curricula be classified under a primary curriculum as authorized by
Transport Canada. For example, a Transition Curriculum is classified as a Qualification
Curriculum.

4.4.1 Transition Curriculum

This curriculum is applicable for a candidate who has been previously trained and
qualified in a specific duty position by the air operator and is now being assigned to the
same duty position on a different aircraft. As required by Part VII of the CARs, the new
aircraft must be in the same aircraft group or the Qualification Curriculum must be used.
For the purpose of AQP, an aircraft of the same group means reciprocating engine, turbo-
propeller engine or turbo-jet engine airplanes. A Transition Curriculum utilizes the same
Qualification Standards as the Qualification Curriculum. Candidates must meet all the
same testing, validation and evaluation points contained in the Qualification Curriculum.

However, a Transition Curriculum may be somewhat abbreviated compared to the normal

Qualification Curriculum. This abbreviation will be based on an analysis of the

96
training/validation/evaluation requirements of the Qualification Curriculum compared to
an assessment of the currency, knowledge, skills and qualifications of the individual. In
most cases, the Transition Curriculum will consist of modules extracted from the
Qualification Curriculum. For example, flight crewmembers may already be qualified on
an aircraft with a particular type of Flight Management System (FMS). If they then
transition to an aircraft with the same FMS, the FMS training might only address the
specific differences such as aircraft weight and fuel burn. Proficiency could be validated
through testing, and it would not be necessary for the candidates to attend the entire FMS
curriculum segment.

4.4.2 Upgrade Curriculum

This curriculum is for a candidate who has been previously trained and qualified as either
a First Officer, Cruise Relief Pilot, Flight Engineer or Second Officer for the air operator
and is being assigned as either a Captain or First Officer, respectively, for the same
aircraft type in which he or she was previously trained and qualified. Elements or
training modules for this curriculum may be found in both primary curricula. In upgrade
training, the same Qualification Standards apply as found in the Qualification
Curriculum. The individual must meet all the same testing, validation and evaluation
points for the applicable Qualification Curriculum. However, the training received may
be abbreviated, based on an analysis of the training/validation/evaluation requirements of
the Qualification and Continuing Qualification Curricula compared to an assessment of
the currency, knowledge, skills, and qualifications of the individual. For example, if a
First Officer is already qualified and current on a specific type, proficiency in training
modules such as systems and FMS may be validated through testing. Training on topics
such as command authority and CRM, which may be unique to the individual
crewmember’s duty position, may be trained and evaluated using a combination of
classroom and LOS methodology.

4.4.3 Requalification Curriculum

This curriculum is for individuals who no longer meet the recency requirements and/or
the validity date of their LOE has expired. These individuals would not meet the
requirements of a Continuing Qualification Curriculum, because they have become
unqualified for their duty position. The individual must be re-qualified under a secondary
curriculum to resume serving in that duty position.

Note: Requalification training is applicable for Captains reassigned as First Officers in

the same type aircraft. In this case, duty position dependent training is required.

Note: Requalification Curriculum is required for Captains and First Officers who are
reassigned as Flight Engineers or Second Officers on the same aircraft type provided
they were previously qualified as Flight Engineers or Second Officers on this type
aircraft.

97
4.4.4 Difference Curriculum

This curriculum is applicable for an employee who has previously been trained and
qualified in a specific duty position by the air operator and is being assigned the same
duty position on an aircraft of the same family (i.e., A330 to A340, B757 to B767, and
CRJ-200 to CRJ-700). The new aircraft must be in the same aircraft family or a
Qualification Curriculum must be used. In a Difference Curriculum, an analysis of the
Qualification Curriculum of both aircraft must be made to identify the differences. Each
Qualification Curriculum will include training, validation, evaluation and pilot licensing
action, as applicable. These curriculum differences are used to develop the Difference
Curriculum.

98
Chapter 5 - Instructors and Evaluators

5.1 Instructor/Evaluator Curricula

5.1.1 General

Instructors and evaluators are the backbone of the AQP. Without adequately trained and
qualified instructors and evaluators, AQP will not be successful. The applicant must
devote the appropriate time and resources to the qualification and continuing
qualification of these key personnel.

5.1.2 Instructor/Evaluator Curricula

Each AQP must contain instructor and evaluator Qualification and Continuing
Qualification Curricula. These requirements include a separate JTA, Qualification
Standards, curricula, and curriculum outlines focusing on the instructor/evaluator duty
positions.

Note: The Qualification Standards document for instructor/evaluators does not need to
include conditions or a criticality/currency analysis.

An AQP is an alternate method of training, evaluating, qualifying, and certifying, to

ensure the competency of pilots, flight crewmembers, instructors, and evaluators subject
to the training and evaluation requirements under the applicable Subpart of Part VII of
the CARs.

AQP provides for authorization of an alternate method for qualifying, training, certifying
and ensuring the competency of instructors and evaluators required to be trained or
qualified under Part VII of the CARs. AQP has entailed a detailed re-examination of the
existing policies and procedures as they apply to the eligibility, nomination and
surveillance requirements for instructors and evaluators. The specific qualification and
currency requirements of AQP evaluators are located in the AQP Evaluator Manual.

Historically, ACPs checked pilots using a manoeuvre-based evaluation scenario based on

the ACP Manual and the PPC schedules located in Part VII of the CASS. AQP relies on
a progressive validation process. In addition to a manoeuvre-based evaluation, AQP also
features a LOE. The Qualification Standards used for pilot checking and licensing under
AQP will reflect the complete set of standards under which the pilots will operate
throughout their operational line flying. Due to the unique characteristics and detailed
scripting of LOEs, all personnel authorized to conduct licensing action, are required to
complete AQP evaluator training. This training can consist of the Qualification
Curriculum either for new personnel or for a differences module for instructors and ACPs
that are already qualified under the air operator’s traditional program.

99
5.1.3 Definitions

Evaluator: An “evaluator” is a person delegated by the Minister who has satisfactorily

completed training and evaluation that qualifies that person to evaluate the performance
of flight crewmembers, instructors, and other evaluators, as appropriate. In an effort to
remain aligned with traditional ACP policy and procedure, the AQP term “Evaluator” is
considered synonymous with “Check Pilot” or “ACP” in a traditional program.

5.1.4 Qualification Training

The training requirements for each curriculum are derived by a thorough and accurate
task analysis. In order to provide an example of what might be included in each
curriculum, Transport Canada reviewed several task analyses and extracted some subject
areas of interest which are listed below:

A. Instructor Qualification Curriculum

1. General overview of:

a) The learning process.

b) Elements of effective teaching.

c) Student evaluation, quizzing, and testing.

d) Overview of AQP program development, implementation, and operation

policy.

e) Lesson preparation and application.

f) Classroom instructing techniques.

g) Techniques for instructing in the cockpit environment.

h) Standardization and Rater/Referent Reliability.

i) CRM and human factors training.

j) How to conduct training modules for students with varying backgrounds

and varying levels of experience and ability.

k) Instructor responsibilities.

2. Specific training in:

100
 a) Effective use of and qualification in specific FTDs, flight simulators and
aircraft used in the AQP.

b) Limitations on use of training equipment.

c) Evaluation of performance against objective standards.

d) Effective pre-flight and post-flight instruction.

e) Effective analysis and correction of common errors.

f) Teaching/facilitation of CRM skills.

g) Performance and analysis of standard flight events and procedures.

h) Safety considerations in the training environment.

i) Data gathering procedures.

j) Standardization and Rater/Referent Reliability

B. Differences Between Traditional and AQP for Existing Instructors

a) Overview of AQP program development, implementation, and operation

policy.

b) CRM and human factors training.

c) Standardization and Rater/Referent Reliability.

d) Data gathering procedures.

e) Effective use of and qualification in specific FTDs, flight simulators and

aircraft used in the AQP.

f) Limitations on use of training equipment used in the AQP.

g) Evaluation of performance against objective standards.

C. Evaluator Qualification Curriculum

1. General overview of:

a) Evaluation policies and techniques.

b) The role of the evaluator.

101
c) Administrative procedures.

d) General safety considerations.

e) Evaluating human factors and CRM skills.

f) Standardization and Rater/Referent Reliability.

2. Specific training in:

a) For each crewmember position requiring a particular evaluation the

method of conducting:

i) Manoeuvres Validation

ii) First-Look Manoeuvres

iii) Online Evaluations

iv) In-flight proficiency evaluations if required

v) LOE in flight simulators and/or FTDs

vi) Special purpose evaluations (e.g., long range navigation)

b) The standards for the evaluations in a).

c) The methods and standards associated with licensing requirements for the
issuance of an Instrument and Type Rating.

d) How to conduct evaluations while simultaneously serving as PIC, SIC, or

safety pilot.

e) Safety considerations for the various types of evaluations.

f) Safety considerations particular to the make, model, and series aircraft (or
variant).

g) How to evaluate instructors and evaluators.

h) Company/Transport Canada policies with regard to the conduct of

evaluations.

i) Administrative requirements particular to evaluations.

j) Evaluating CRM skills.

102
 k) Briefing and debriefing techniques.

l) Data gathering procedures.

D. Differences between Traditional and AQP for Existing Evaluators

a) Overview of AQP program development, implementation, and operation

policy.

b) CRM and human factors training/evaluating.

c) Standardization and Rater reliability.

d) Data gathering procedures.

e) Effective use of and qualification in specific FTDs, flight simulators and

aircraft used in the AQP.

f) Limitations on use of training equipment.

g) Evaluation of performance against objective standards.

h) Briefing and debriefing techniques.

i) How to evaluate instructors and evaluators.

j) Company/Transport Canada policies with regard to the conduct of

evaluations.

k) Administrative requirements particular to evaluations.

5.1.5 Qualification – Evaluation and Observation

Every instructor/evaluator must receive a validation/evaluation at the end of each

curriculum. These methodologies must be described in the test and evaluation strategy
within the Instructor/Evaluator Qualification Standards.

5.1.6 Continuing Qualification – Training, Evaluation and Observation

Instructors/evaluators should be given an opportunity to operate in all positions in which

they are authorized to perform duties. Flight instructors/evaluators should demonstrate
the ability to brief and debrief, fly and instruct in both pilot seats and operate the
simulator, if appropriate. If possible, instructors/evaluators should focus on the training,
validations and evaluations that are under development for the next evaluation period.
This will allow the instructors/evaluators to become familiar with all aspects of the

103
sessions they will be expected to teach/evaluate during the following year. This will help
ensure the development of a polished Continuing Qualification program and give the
instructors/evaluators an opportunity to maintain their own proficiency using scenarios
that they have not seen. Each training period should be conducted by one of a small core
of the most experienced instructors/evaluators.

Each Instructor/Evaluator’s Continuing Qualification curriculum segment should include

the requirements listed below. The requirements of paragraphs c. and d. below should be
defined in the Test and Evaluation Strategy in the Instructor/Evaluator Qualification
Standards:

a) Basic crew position Continuing Qualification.

b) Ground and flight training to enhance, upgrade, and maintain each

instructor/evaluator’s knowledge, skills, and abilities.

c) A schedule for recency of instructor/evaluator’s experience. The program must

define the minimum requirements each category of instructor/evaluator, as
defined in the Evaluator Manual, will have to meet in order to stay current in their
position. Instructors must maintain a high level of competency, so the minimum
standard must be sufficient to maintain their proficiency. Evaluators will perform
an adequate number of each of the validation events that they are qualified to
administer, as specified in the AQP Evaluator Manual.

d) A schedule for critical examination of each instructor/evaluator’s abilities and

adherence to prescribed standards.

5.1.7 Instructor and Evaluator CRM Training and Evaluation

All instructors and evaluators will receive instruction and evaluation in CRM objectives
and training methods.

5.2 Type, Qualification and Currency Requirements for Evaluators

Type, qualification and currency requirements for AQP evaluators are located within the
AQP Evaluator Manual (TP 14672).

5.3 Quality Assurance and Standardization

5.3.1 Quality Assurance

104
In order to continuously improve the quality of AQP training, an AQP applicant shall
develop a quality assurance program (i.e., mentorship program) to continually evaluate
the training programs, the instructors, and evaluators. The most experienced personnel
available in the organization will be appointed by the air operator as Quality Assurance
Instructors (QAI) and Quality Assurance Evaluators (QAE) to carry out the quality
assurance duties. Air operators may also use other terms such as “mentors” to refer to
these individuals. Quality Assurance Instructor and Evaluator duties would include
observing training, validation, and evaluation events in order to:

a) Critique performance.

b) Recommend change.

c) Provide feedback to the entire training organization at regular intervals.

A Job Task Analysis (JTA), Qualification Standards, and Qualification Curriculum will
be required for QAIs and QAEs. All QAIs and QAEs will be required to attend an initial
course. The air operator shall establish currency requirements as deemed appropriate.

5.3.2 Standardization

A standardization program for instructor and evaluator is necessary to establish uniform

grading criteria, address reliability between instructor and evaluator, and develop
remediation procedures. The AQP data collection and analysis is incumbent upon
reliable and valid grading assessment from instructors and evaluators. The program must
provide Inter-Rater Reliability (IRR) and/or Referent Rater Reliability (RRR) training
during the Qualification, Continuing Qualification and Secondary Curricula.

105
Chapter 6 - AQP Data Management

6.1 Introduction

6.1.1 Background

This chapter provides general guidance for the management of performance/proficiency

data within an AQP. Background information and principles for collecting, entering,
reporting, and analyzing performance data are addressed in a publication titled “Data
Management Guide”. This reference document was developed by the Data Management
Focus Group AQP Subcommittee, which is sponsored by the Air Transport Association
(ATA). The “Data Management Guide” is available from Transport Canada.

A. Regulatory Requirement

In AQP, the air operator is required to collect and analyze performance

information on its flight crewmembers, instructors and evaluators. The data
collection, analysis and reporting processes employed by the air operator must be
acceptable to Transport Canada. This data will enable the air operator and
Transport Canada to determine whether the form and content of training and
evaluation activities are satisfactorily accomplishing the overall objectives of the
curriculum. Good data management practices are necessary to determine whether
an AQP is meeting its objectives.

B. Validation

The principal goal of the AQP is true proficiency-based training and qualification.
Performance Objectives are systematically developed and maintained, then
continuously validated through the collection and evaluation of empirical
performance data. Data collection and analysis (data management) is, therefore,
an integral part of AQP.

6.1.2 Definitions

Data management can be classified into two broad categories:

a) Individual Qualification Records; and

b) Performance/Proficiency Data.

These broad categories are described below:

106
A. Individual Qualification Records

These are identifiable, detailed records that are maintained on each individual
flight crewmember, instructor and evaluator who is qualifying or has qualified
under an AQP. These records show how and when the individual satisfied the
requirements of the curriculum required for their assigned duty position. They
may also include demographic and work history information, as well as
completion information on the modules and lessons. Air operators may maintain
a manual record keeping system, or a computerized record keeping system. The
record keeping process in AQP does not differ from traditional record keeping
requirements.

B. Performance/Proficiency Data

In addition to the traditional record keeping requirements described above, AQP

also requires the establishment of a separate PPDB. Performance/proficiency data
records are de-identified and are maintained separately from the normal
qualification records.

This de-identified information represents the results of an individual’s ability to

successfully demonstrate the performance objectives of each curriculum. This
information is captured during validation and evaluation gates as a flight
crewmember progresses through an AQP curriculum. This data is obtained from
each flight crewmember’s performance and is stored in a collective form in the
PPDB.

This data is used to analyze training programs and/or groups of participants, not
for tracking individual accomplishment. Successful collection and analysis of this
data will allow the air operator to identify and correct problems, validate AQP
curricula, and identify developing trends.

6.2 Data Management

6.2.1 Overview

In AQP, data management is a continual process of data collection, entry, submission and
analysis.

6.2.2 Data Collection

AQP data collection is required in all curricula. The specifics are detailed in the air
operator’s AQP Data Collection and Analysis Plan, which are contained in the
Implementation and Operations Plan (I & O Plan). Data is collected at each validation or
evaluation gate. This data consists of graded proficiency objectives using a rating scale
with associated reason codes (if applicable). Data collection requirements for the AQP
will vary with the curriculum, the type of curriculum activity (training, validation, or

107
evaluation), the type of participant (flight crewmember, instructor, or evaluator), and the
overall management objectives for use of the data. Figure 6-1 illustrates the AQP data
collection fields required for submission to Transport Canada. All performance data
collected on each proficiency objective must be relative to the applicable Qualification
Standards defined for the training and evaluation activities.

For each flight crewmember in a Qualification or Continuing Qualification Curriculum,

Transport Canada must be able to associate the data records applicable for that flight
crewmember in that curriculum through logical grouping of the records, or linkage by a
common de-identified index number (but not by name).

6.2.3 Data Entry

All performance/proficiency data collected throughout the AQP is entered into the air
operator’s PPDB. Typically, this is an electronic database for ease in analysis,
comparison and reporting purposes. Considerations for data entry include the method,
the hardware/software required for data input, and the hardware/software required for
data storage and utilization. Distinct advantages, disadvantages and costs are associated
with any method of data entry. Database design is at the discretion of the air operator,
provided that the design can generate the required report table specified, in a manner
acceptable to Transport Canada.

6.2.4 Data Submission

For the purposes of program oversight, Transport Canada has established the minimal
requirements for the submission of de-identified data by curriculum. Figures 6-1 through
6-4 summarize the submission requirements in three tables. These tables are the
Performance Data Report Table (PDRT), the Skill Reason Table (SklRsn) and the
Training Objectives Report Table (TORT). The information in these tables is
downloaded from the air operator’s PPDB. The air operator is responsible to prepare de-
identified data reports and check them for data integrity, and then to submit them to
Transport Canada in 1 calendar-month blocks within 2 months of collection. Reports can
be forwarded electronically to Transport Canada or made available by convenient means
such as through web access. Transport Canada will analyze the de-identified data using
standard automated queries and reports to identify AQP performance trends. Figures 6-2,
6-3, and 6-4 contain specifications for each field in the three tables.

a) PDRT

The PDRT contains a listing of 23 fields that are reported for every measured item,
providing a separate record for each. A measured item is a manoeuvre, task, procedure,
or event set, and is the main component for data analysis. These fields provide a record
of the results of the performance of each measured item along with supporting data for
reporting and analysis. Certain supporting data fields (air operator designator,
curriculum, etc.) repeat for each record and can be automatically generated from a
query/software routine. Each field in the PDRT must contain either an alpha/numeric,

108
numeric, or text entry. Figure 6-2 contains the specifications for each field, defines the
meaning of each field item, and provides examples of the field values.

b) SklRsn

With respect to the SklRsn table, an unsatisfactory rating (Mrate) of a measured item
requires that a reason be entered to explain the rating. All reasons are entered in the
SklRsn table in field No. 24, Skill Reason Text. This table allows the carriers to report
more than one Skill Reason Text for an unsatisfactory rating of a measured item without
having to enter multiple records. There are only two fields in this table.

(1) The first field is Field No. 5 (SklRsn), which is the same as in the PDRT and
provides the linkage between the two tables. For each record in the PDRT where
one or more Skill Reasons need to be reported, an identifier is entered in the
SklRsn field that is unique to the measured item. This identifier can be a
concatenation of the fields that make it unique. For instance:
Fleet+EvalType+CmID+Mitem. As long as the identifier is unique to the record,
it is the carrier’s discretion to determine which fields are used.

NOTE: If this approach is used, care must be taken to ensure that none of the
component field contains an NA value.

(2) The second field in the SklRsn Table is the Skill Reason Text, the field that
will contain descriptive text for each reason relevant to a particular measured
item. For example: Systems Knowledge, Procedural, CRM, Technical
Knowledge, etc.

c) TORT

The TORT table is comprised of training objective(s) associated with each measured
item. It provides a means of auditing the relationship between the measured item and the
task analysis, qualification standards, and the curriculum. There are four fields in this
table. The TORT links to the PDRT through the first field, Measured Item ID (MItemID).
The second field, Objective ID (ObjID) is an identifier assigned to each terminal
proficiency objective (TPO) or supporting proficiency objective. The Objective ID may
be the same as the Measured ID if it is a single TPO or SPO. However, in the case of a
measured item that is an event set, there may be several training objective identifiers
associated to the Measured Item ID. The third field, Objective Title (ObjTitle) is a text
description of the training objective. The last field in this table is the Objective Type
(ObjType) that identifies the training objective as a TPO, SPO, EO or as required by the
air operator’s qualification standards structure.

Due to the unique features of each operator’s AQP, TC in consultation with the air
operator may require the collection of additional data as deemed appropriate.

d) File Naming Convention

109
Data submissions must use the file-naming format (xyzmmyya.mdb) as shown in the
example below:

XYZ – is the identifier

assigned to the air operator. A for AQP
Usually it is the first three
letters from the air operator’s
designator code. xyz 0909 A.mdb

mmyy - Is the month and year . mdb - Is the proper

for the reported training extension for the
month. In this case, September Access formatted file.
2009.

On the following pages, figure 6-1 lists the minimum AQP data submission requirements.
Figure 6-2 provides the specification for each field in the PDRT. Figure 6-3 discusses the
skill reason codes and supporting text applicable to the SklRsn table while figure 6-4 on
the topic of the TORT links training objectives with measured items.

110
 FIGURE 6-1: DATA SUBMISSION REQUIREMENTS

Training Program Advanced Qualification Program

Curriculum Qualification Continuing Qualification
Table # Field Short Name SV PV MV LOE OE FL MV LOE OE
1 Measured Item MItem Des Des Req Req Req Req Req Req Req
2 Measured Item ID MitemID Des Des Req Req Req Req Req Req Req
3 Measured Item Rating* Mrate Des Des Req Req Req Req Req Req Req
4 Rating Meaning Rmean Des Des Req Req Req Req Req Req Req
5 Skill/Reason SklRsn N/A N/A Req Req Req Req Req Req Req
6 PF/PNF PFPNF N/A N/A DES** Req Req Req** DES** Req Req
7 Is Currency Item? Currcy N/A N/A Req N/A N/A Req Req N/A N/A
8 Is Critical Item Crit N/A N/A Req N/A N/A Req Req N/A N/A
Proficiency 9 Crew ID CrewID Req*** Req.*** Req Req Req Req Req Req Req
Data
10 Evaluation Date EvalDate Req Req Req Req Req Req Req Req Req
Report
Table 11 Air operator Designator AirDsgn Req Req Req Req Req Req Req Req Req
12 TC Fleet Designator Fleet Req Req Req Req Req Req Req Req Req
13 Training Program TrPgm Req Req Req Req Req Req Req Req Req
14 Curriculum Curr Req Req Req Req Req Req Req Req Req
15 Evaluation Type EvalType Req Req Req Req Req Req Req Req Req
16 TC Simulator ID SimID N/A N/A Req Req N/A Req Req Req N/A
17 Evaluator ID EvaltrID Req Req Req Req Req Req Req Req Req
18 TC Inspector ID TCID Req Req Req Req Req Req Req Req Req
19 OE Geo. Area GeoArea N/A N/A N/A N/A N/A N/A N/A N/A Req
20 Comments Comment Des Des Des Des Des Des Des Des Des
21 Crewmember ID CmID Req Req Req Req Req Req Req Req Req
22 Crew Position CrewPos Req Req Req Req Req Req Req Req Req
23 Evaluation Rating Type EvalRtg Req Req Req Req Req N/A Req Req Req
Skill Reason 5 Skill/Reason SklRsn N/A N/A Req Req Req Req N/A Req Req
Table 24 Skill Reason Text SklRsnText N/A N/A Req Req Req Req N/A Req Req

Training 2 Measured Item ID MItemID Des Des Req Req Req Req Req Req Req
Objectives 25 Objective ID ObjID Des Des Req Req Req Req Req Req Req
Report 26 Objective Title ObjTitle Des Des Req Req Req Req Req Req Req
Table
27 Objective Type ObjType Des Des Req Req Req Req Req Req Req
Req - A required entry. Qual - AQP Qualification Curriculum PV - Procedures
N/A - If no value is to be entered, use N/A. Leave no blank fields. CQ - AQP Continuing Qualification Curriculum Validation
Des - Desired. This is optional information that is not required, but desired for submission. If FL - First Look Manoeuvres MV - Manoeuvres
not submitted, then N/A must be entered as the value. SV - Systems Validation Validation
*/** - See Notes in Data Reporting Format - Figure 6-2. LOE - Line
*** - If there is no crew pairing for Systems or Procedures Val, submit the single CmID (No. Operational
21) for the crewmember. Evaluation
OE - Online
Evaluation
 FIGURE 6–2: PERFORMANCE DATA REPORT TABLE (PDRT)

Table No Field Description Short Type Field Example(s) Notes

Name Size
1 Measured Item The task, manoeuvre, Mitem Text 80 Perform Engine 1) This field will be N/A for Qual. SV and PV because these validations are
procedure, or event set Failure usually graded pass/fail and recorded in field No. 23.
statement. Procedure

2 Measured Item An alphanumeric used to MitemID Text 12 1.2.1.3 1) This value is associated with a TPO, SPO, or event set. When it is a TPO or
ID identify the task, manoeuvre, or SPO, use the related number from the hierarchical numbering system. When it
procedure or event set being KK73456 is an event set, enter the event set identifier.
graded. 2) This field will be N/A for Qual. SV and PV because these validations are
normally graded in aggregate (pass/fail), and recorded in field No. 23.
3 Measured Item The numeric rating assigned Mrate Numeric 1 3 1) Values depend on rating scale.
Rating on the first attempt for each 2) Do not provide records that are missing MRate: Provide only performance
measured item. related information. Do not provide “Incomplete”.
3) Reported rating must refer only to the first attempt of the measured Item, not
the terminal performance.
4) * Use 9 instead of N/A for Qual. SV and PV. This is a numeric only field.
4 Rating Meaning A description of the Rmean Text 40 Satisfactory 1) The text meaning of each number in the rating scale.
Measured Item Rating

5 Skill/Reason Link between the PDRT and SklRsn Text 90 B-737-LOE- 1)This field is a unique record identifier that must be provided for Unsatisfactory
SKLRSN tables 5555-Taxi Measured Item ratings (MRate) to provide a link to the Skill Reason table for
single or multiple reasons for a failure.
PDRT 2) This identifier can be a combination of the fields that make it unique. For
example: Fleet+EvalType+CmID+MItem. As long as the identifier is unique
to the record, it is the carrier’s discretion to determine which fields are used.
3) If Mitem is satisfactory, insert N/A in this field.
6 PF/PNF Indicates whether the pilot PFPNF Text 3 PF or PNF 1) No entry requirement for FE.
performing the Measured 2) N/A for Qualification Curriculum Systems and Procedures Validation
Item was flying or not 3) ** For FL, it is desired and encouraged that PNF data be submitted along with
flying. PF data. However, if significant grade sheet changes are necessary to
accommodate this, only the standard PF items need be identified. For these air
operators, the value entered for all FL Measured Items will be “PF”.
4) ** This field is desired for Qual. and CQ Manoeuvres Validation; however, if
PNF is not submitted then the value will be “PF”.
7 Is currency Indicates whether the Currcy Text 3 Yes, No or N/A 1) N/A is used when the Measured Item is an Event Set.
Item? Measured Item is a currency
item.
Table No Field Description Short Type Field Example(s) Notes
Name Size
8 Is Critical Item Indicates whether the Crit Text 3 Yes, No or N/A 1) N/A is used when the Measured Item is an Event Set.
Measured Item is a critical
item.

9 Crew ID The de-identified CrewID Text 40 P21234 SIC 1) CrewID is the concatenation of the CmID’s (#21) of all crewmembers present.
alphanumeric assigned to a F12312 Order of ID’s: PIC SIC RP FE.
specific pairing of 2) If two PICs, SICs or RPs are trained together use the structure PIC PIC FE, or
crewmembers that will be SIC SIC FE or RP RP, as appropriate, separated by spaces.
maintained for the duration 3) If there is no crew pairing for Systems or Procedures Val, submit the single
of a curriculum, unless the CmID (#21) for the crewmember.
crew make-up changes or a 4) Maintain CrewID in exact form and order, except for seat substitutes. For seat
seat substitute is present. subs, use Crew Position (No. 22) values relative to the seat position in the
CrewID sequence and add the CmID number. For example, P21234 SIC
S17521 F12312 would indicate that the first officer position was occupied by a
First Officer seat substitute with CmID number S17521.
5) Do not submit data collected on non-employees (e.g., contract instructors)
performing their required validations/evaluations.
10 (MM/YYYY) The month and the year the Date Date 7 10/2001 1) Set the date field format in Access to month/year; use full century: MM/YYYY.
Measured Item data is 2) The day value will default to 01.
collected.

11 Air operator The air operator’s four- AirDsgn Text 4 XYZA

Designator character designator
PDRT reporting the measured
items.
12 TC Fleet The TC designation of the Fleet Text 20 B-737 1) See Air operator/Fleet Designator Menu
Designator fleet or equipment type
relevant to the Measured
Item.
13 Training The training program TrPgm Text 4 AQP
Program relevant to the Measured
Item.
14 Curriculum The AQP curriculum in Curr Text 4 Qual, CQ or
which the Measured Item is N/A
being validated or evaluated.

15 Evaluation Type The evaluation type in which EvalType Text 4 FL, SV, PV,
the Measured Item is MV, LOE, OE
accomplished.
Table No Field Description Short Type Field Example(s) Notes
Name Size
16 TC Simulator The TC simulator ID SimID Text 4 1234 or N/A 1) If a simulator is not used, enter N/A
ID number of the simulator
where the Measured Item is
performed.
17 Evaluator ID The identification number of EvaltrID Text 15 123456 1) If an TC inspector is the evaluator, use the TCID in this field and in the TC
the evaluator who graded the Inspector ID field, No. 18.
Measured Item.

18 TC Inspector ID The Form 110A number of TCID Text 4 1234 1) Report the TC ID Number (110A) of a TC inspector who is there as an
the aviation safety inspector observer and/or evaluator (#17). If no TC inspector is present, enter N/A.
who observed the Measured
Item.
19 OE Geographic The geographic area where GeoArea Text 30 Pacific 1) Use geographic descriptors as defined by the air operator.
Area the Measured Item is 2) This field is only required for AQP operators intending to apply or already
collected for OE. authorized for deviations from regulatory OE interval requirements; otherwise,
enter N/A.

20 Comments Additional comments Comment Memo 1) The comment field provides further explanation of unsatisfactory or
submitted by evaluator. outstanding measured Item Rating.
PDRT 2) Evaluator comments are provided at the discretion of the air operator, unless
otherwise required in accordance with the air operator’s AQP.
3) If there are no comments, enter N/A.
21 Crewmember The de-identified CmID Text 10 P123456 1) Begin all PIC IDs with P, SIC IDs with S, Cruise Relief Pilot (CRP) IDs with
ID alphanumeric assigned for R and FE IDs with F.
the duration of a curriculum 2) The P, S, R or F refer to the position for which the pilot in training is being
to the crewmember qualified, not necessarily the seat occupied.
performing the measured 3) Do not submit data collected on non-employees (e.g., contract instructors)
item. performing their required validations/evaluations.
22 Crew Position The “seat” occupied by the CrewPos Text 4 PIC PIC = Pilot in Command
person performing the SIC = Second in Command
Measured Item. FE = Flight Engineer
E.g., a SIC or CRP in the left seat would be entered as PIC.

23 Evaluation Type The overall rating the pilot EvalRtg Text 5 Sat or 1) Rating must refer to the first execution of the evaluation type. Repeated
Rating performing the Measured Unsat sessions are not reported in this table.
Item received for the
evaluation type.
 FIGURE 6–3: SKILL/REASON TABLE (SKLRSN)

Table # Field Description Short Type Field Example(s) Notes

Name Size
SklRsn 5 Skill/Reason Link between the PDRT and SklRsn Text 90 B-737-LOE- 1) This field is a unique record identifier that must be provided for Unsatisfactory
SKLRSN tables 5555-Taxi Measured Item ratings (MRate) to provide a link to the Skill Reason table for
single or multiple reasons for a failure.
2) This identifier can be a combination of the fields that make it unique. For
example: Fleet+EvalType+CmID+ MItem.
3) As long as the identifier is unique to the record, it is the carrier’s discretion to
determine which fields are used.
4) None of the component fields can contain a N/A value.
24 Skill reason A description of the skill or SklRsn Text 40 Technical 1) Provided for Unsatisfactory Measured Item Ratings from the carrier’s Reason
Text reason for assigning an Text CRM Codes.
unsatisfactory Measured Procedural 2) If more than one skill or reason is relevant, allow one field for each skill or
Item rating. reason.
3) Enabling Objectives (EOs) are acceptable as skill descriptors.

FIGURE 6–4: TRAINING OBJECTIVES REPORT TABLE (TORT)

Table # Field Description Short Type Field Example(s) Notes

Name Size
TORT 2 Measured An alphanumeric used to MitemID Text 12 1.2.1.3 or 1) This value is associated with a TPO, SPO, or event set. When it is a TPO or
Item ID identify the task, manoeuvre, SPO, use the related number from the hierarchical numbering system. When it
procedure or event set being KK73456 is an event set, enter the event set identifier.
graded. 2) This field will be N/A for Qual. SV and PV because these validations are
normally graded in aggregate (pass/fail), and recorded in field No. 23.
25 Objective ID An alphanumeric assigned to ObjID Text 12 3.2 1) List all the TPOs, SPOs or other high level objective grouping that apply to the
each terminal or supporting Measured Item.
objective tested by the
Measured Item.
26 Objective A description of the ObjTitle Text 80 Perform 1) A description of the objective ID in field No. 25.
Title objective ID Engine Fire
Procedures

27 Objective A description of the ObjType Text 5 TPO or 1) Identify the training objective as a TPO, SPO, or EO.
Type objective type SPO or
EO
6.2.5 Data Analysis

The primary users of data reports are air operator personnel and Transport Canada.

A. Air Operator

AQP requires that the data collection conducted by the air operator for its own use in
monitoring curricula will support more analytical detail and diagnostic functions than the
data collected for submission to Transport Canada. Transport Canada expects the air
operator to do an in-depth analysis of the effectiveness of the training. Reporting of data
is based on the analysis of the PPDB to provide information on the curriculum and
participant groups (flight crewmembers, instructors/evaluators). Once the data is
collected and entered into the data management system (PPDB), an analysis should be
performed on the aggregate information. Statistical analysis of the proficiency data
enables air operators to make an internal assessment of their performance. Air operators
should tailor these processes and techniques to suit their own requirements. Each air
operator’s data collection and performance assessment processes should be refined over
time, based on their own practical experience. That is, the measures and processes
should be optimized on an iterative basis to provide the degree of discrimination in flight
crewmember performance needed to establish effective quality control over AQP
curricula.

B. Transport Canada

The data submissions to Transport Canada are primarily ratings and reason codes
associated with performance measures taken at validation and evaluation gates and
supporting data. The data, presented to Transport Canada in the tables previously
discussed is analyzed using the Canadian Data Analysis Reporting Tool (CDART) to
allow a POI and other Transport Canada personnel to conduct trend analysis to monitor
overall program effectiveness.

117
Chapter 7 - Crew Resource Management (CRM)

7.1 Introduction

7.1.1 Overview

CRM training develops skills that enhance flight safety through the effective use of all available
resources including human, hardware and information resources. CRM training increases
awareness of human error and systemic and organizational threats, and discusses techniques,
skills and attitudes that will minimize their effects. Meanwhile, recent developments in
assessment techniques focus on threat and error management strategies and performance where it
is recognized that from time to time and although not desirable, errors or deviations from
standard practices will occur. Effective risk mitigation therefore includes an awareness of flight
crewmember attitudes and behaviors as well as the use of practical flight management skills.

CRM training achieves a greater degree of integration through an AQP than that offered through
a traditional training program. This chapter describes the integration of CRM training into pilot
training curricula using the instructional systems development methodology that forms the
foundations of the AQP process. The approach demonstrates how the analytical methodologies
in AQP have the potential to produce more rigorous CRM training, including the appropriate
integration of CRM and technical training.

7.2 Integrating CRM into an AQP

7.2.1 Scope of Integration

One of the major objectives of AQP is the seamless integration of CRM and technical training,
such that CRM becomes an integral part of the flying job. Where appropriate, CRM procedures
are identified, documented, integrated and accorded the same weight as the technical procedures
required for the execution of a given phase of flight and its associated flight tasks. Seamless
integration does not, however, mean that only those aspects of CRM that can be proceduralized
are addressed in flight operations technical training and evaluation. Comprehensive CRM
training in AQP requires that two aspects of CRM be addressed.

A. Phase Specific

Some aspects of CRM are inherent to manoeuvre performance. For example,

communication procedures for co-ordinating callouts during take-offs, approaches, and
other manoeuvres are clearly specified in terms of “what” should be said and “when” it
should be said. These callouts take place during most flight phases and are performed,
for the most part, at fairly fixed points in the flight-phase sequence of task activities.
Similarly, some aspects of communication during the management of non-normal
conditions can be easily positioned within the sequence of activities performed to manage

118
 the non-normal condition. These aspects of CRM are phase specific or condition
specific.

B. Phase Independent

Other equally important CRM activities, in contrast, are performed on an as-needed basis,
in order to manage the flight, work well as a team, or respond to unique situations.
Recognizing the need for, and effectively executing these activities, is critical to
coordinating the various duties the crew must perform during the flight. For example,
certain communications should be performed in order to maintain crew awareness of
flight status. Regardless of the phase of flight, it is critical that the crew recognizes this
communication requirement and effectively acts upon it in a timely manner to maintain
crew situational awareness.

These global activities do not fit neatly within a hierarchical list of technical activities,
organized by phase of flight. Instead, they act as a “shell” or “template” that overlays
and organizes the activities that may need to be performed during any phase of flight.
These phase-of-flight independent skills as a whole constitute a management strategy that
represents a critically important part of the inventory of defences against error by flight
crews and threats surfacing from the organization or the operational environment. In
particular, they can provide the crew with the tools needed to resolve problematic
situations to which they may never have been previously exposed in training or in flight
operations. It is important, therefore, that training activities seek to fully instil these
skills in crews, in order to provide a basis for generalization to a potentially wide variety
of situations. Figure 7-1 contains samples of some of these phase-of-flight independent
CRM skills.

Fig 7-1: Sample Phase Independent CRM Skills

Exercise Distribute workload and prioritize between primary and distracting duties.
Captain’s
Authority
or
Responsibility

Communicate plans and decisions to the crew.

Enforce standardization, policies, and procedure.
Set expectations for maintaining vigilance and avoiding complacency.
Respond to any safety-related concern raised by any crewmember.
Develop and enhance the aviation skill and knowledge of junior
crewmembers.
Review operational irregularities and establish bottom lines.
Communicate intentions, “bottom lines,” and decisions to all
crewmembers.

119
 Fulfil Cross-check and back the Captain up. This requires maintaining vigilance
First Officer and flying proficiency. It also includes effective monitoring of the
or situation.
Flight Engineer
Responsibilities

Report to the Captain any safety-related concern and request a plan or

decision if none is articulated.
Support decisions articulated by the Captain within the limits of safety,
legality, and procedure.
Develop individual proficiency and take the best from each Captain.

Maintain Prepare, plan and maintain vigilance—be prepared for what you can
Situational reasonably expect.
Awareness
Carry out actions or decisions based on priorities and crew workload
established by the Captain.
Identify systemic traps.
Be aware of the limits of human performance and the nature of human
error.

Establish Conduct or contribute to briefings (keep your head in the game and work
Effective to get ahead of it).
Communications
Maintain a communications “loop”—acknowledge commands, statements,
and questions of crewmembers.
Use resources appropriately to make informed decisions.
Resolve disagreements or differences in expectations—ensure that all
crewmembers are working from the “same page.”
Manage errors appropriately to mitigate consequences.
Continuously review the appropriateness of decisions made and actions
taken
Debrief critical flight events—take the opportunity to learn from unusual
events by reviewing the expectations and actions of all flight
crewmembers at the end of the flight.

Develop Establish appropriate duties and responsibilities by crew position.

And
Maintain
Teamwork
Back each other up through effective cross-check and acknowledgment.
Demonstrate motivation appropriate to the situation—transition between
casual conversation and focused flight communication based on the need
to prepare and execute your flight. This entire range is appropriate at
different points in flight.

120
 Protect crewmembers from the consequences of work overload.
Effectively coordinate with other groups: flight attendant’s, gate agents,
dispatch, and ground crew.
Apply judgment in use of automated systems and modes.
Operate the airplane using different levels of automation as appropriate to
the situation.
Verify that automation is doing what you expect and act to control it when
it does not.
Intervene to control autoflight.
When using automation, back each other up (verify settings, state
intentions, establish roles).

7.2.2 CRM and the AQP Task List

The AQP task list reflects the air operator’s definition of the flight crew’s job, including the role
that CRM is expected to play in the performance of that job. This job definition specifies the
task activities, the knowledge and the skills that must be trained in order to achieve and maintain
pilot and crew proficiency. Because of the fundamental role played by the task list, it should
provide a comprehensive specification of the various task activities that constitute the job, and
the knowledge and skills required to perform those task activities, both technical and CRM.

In terms of the task list, this means that there are CRM task activities just as there are technical
task activities. All of the technical and CRM task activities that must be performed to support
these high-level task activities can now be identified. The high-level task activities, which also
act as objectives, serve as a type of template that will overlay the specific procedures for
handling each condition. High-level CRM task activities act as the framework within which
technical activities are positioned to support management objectives.

7.2.3 CRM Knowledge and Skills

A CRM skill represents the ability of a person to apply specific CRM knowledge across a broad
range of flight related situations. In AQP, these CRM skills are combined to develop proficiency
objectives. Once the air operator has identified the set of task activities appropriate for defining
the flight crew’s job (from the task list), these activities then provide the framework for
identifying the CRM knowledge and skills that a pilot or crew must possess in order to
effectively perform each task activity. Two approaches are commonly used to identify CRM
knowledge and skills:

A. Top-down

This approach uses the CRM categories chosen by the air operator, to identify the set of
CRM knowledge and skills. The resulting knowledge and skills can then be attached to
those task activities whose performance they support.

121
B. Bottom-up

This approach identifies CRM knowledge and skills by analyzing each individual task
activity. The structure of the task activity determines the knowledge and skill
requirements. This approach has the advantage that it defines an inherent link between
the task activity and its knowledge and skills.

7.2.4 CRM and Proficiency Objectives

Once the task list has been completed, the proficiency objectives can be identified for that job.
Two types of proficiency objectives are used. These are categorized according to whether a
flight crewmember requires specific knowledge or whether the flight crewmember is required to
perform an activity. Ground training EOs reflect the subject matter that a flight crew must
“know”. Flight training TPOs, SPOs and EOs reflect the activities a flight crew must be able to
“do”. Appropriate integration of CRM into both ground and flight proficiency objectives ensures
that the range of CRM issues important to the air operator will be addressed both in training and
in evaluation. This is accomplished by including CRM performance standards in the proficiency
objectives.

7.2.5 Training Events

The complete set of proficiency objectives defines the end result of training: the task activities
the crew must be able to perform, the set of conditions under which they must be able to perform
them, the performance standards that must be met, and the evaluation strategy that will be used
to evaluate proficiency. They do not, however, describe the specific training situations and
activities that will be used to achieve this end result, especially in terms of flight training. One
means for specifying the set of flight training situations to be included in a curriculum is by
means of events.

The suggestion that flight training and testing activities should be developed around a set of
events was formalized in 1994 by an industry group tasked to recommend a systematic approach
for developing LOS scenarios under AQP. The event set methodology has achieved wide
acceptance because of its analytical approach to scenario design and its reinforcement of the use
of realistic line conditions that enable crews to practice the full range of flight management
skills.

The effectiveness of the event set methodology for integrating technical and CRM training
objectives suggests that an event orientation throughout the curriculum, rather than just in LOFT
or LOE, could offer important advantages.

A. Event Sets

The primary unit of both LOS design and CRM assessment is the event set. The event set
is made up of one or more events, including an event trigger, distracters, and supporting
events. The event trigger is the condition or group of conditions under which the event is
fully activated. The distracters are conditions inserted within the event set time frame
that are designed to divert the crew’s attention from other events that are occurring or are

122
 about to occur. Finally, supporting events are other events taking place within the event
set designed to further CRM and technical training objectives. In LOS scenario design,
the CRM and technical training objectives should be integrated into the event sets.

This event set framework allows the design team to present the appropriate degree of
realism in the LOS. Instead of focusing on a specific technical issue, the event set
integrates the entire complex line environment (e.g., terrain, Air Traffic Control (ATC),
weather issues, etc.) to facilitate and maximize the crew’s performance in response to
specified CRM and technical issues. The event set tends to follow the phase of flight and
may extend beyond a single phase. This event set framework provides a logical
breakdown for terrain, ATC, and weather issues as they interact with LOS events. With
the LOS scenario now defined by event sets, scenario validation is performed at the event
set level rather than limiting validation to the overall LOS. A sample event-set
development worksheet is shown in Figure 7-2.

B. Sources of Events and Event Sets

The air operator’s safety information system (incident reports, flight crew reports, FOQA
data, OE data, etc.) is an important source for events and event sets. The conditions that
encouraged the occurrence of an incident can be replicated in flight training or discussed
in ground training. The purpose is to educate pilots about the types of conditions that can
increase the likelihood of an error, to present strategies for avoiding these errors, and
techniques for recovering from them, should they occur.

123
Figure 7-2: Sample Event Set Worksheet (aircraft operated under Subpart 705 of the
CARs)

A340 EVENT SET NUMBER 101 WORKSHEET

(FROM A340 R L 03-02 LEG 1)

OVERVIEW. Low Visibility take-off and climb

with a reroute and a TCAS event SUCCESS CRITERIA

Phase of Flight: Take-off through Climb

TPOs and Conditions Technical CRM Skills and Observables
SPOs Skills and
Observables

Trigger: Low Take Off Thrust - Proficient in use Crew coordinates for airspeed
Visibility Normal of FMS and and altitude changes. (SA 3.4)
Departure take-off Autopilot Flight
weather, operations IMC weather Director Crew verbalizes and
200 overcast, (2.1) System. acknowledges changes in the
RVR 1500. (9.1.11) (9.1.13) altitude selector window. (AT
Proper 6.4)
cleanup Accomplishes
Distracters: profile. take-off/climb PF coordinates with PM in the
(2.1.4) procedures use of automation. (AT 6.6)
TCAS RA,
IAW
shortly after Perform SOP. (2.1.1)
take-off. TCAS RA (2.1.2)
avoidance (2.1.3) (2.1.4)
Supporting (9.1.28) (3.1.1)
Events: (3.1.2)
Perform
Reroute and climb Appropriate
climb operations. response to
restriction (3.1) TCAS Alert
(9.1.28)

Difficulty
Equivalency
Rating:

Low Visibility
take-off - IMC
-4
FMS – 1
TCAS – 3

Total - 8

124
7.2.6 Curriculum Design

Curriculum design is the final product of the AQP analyses performed to this point: what is the
job, what is proficiency on the job and how is it measured, and what type of training should be
provided to achieve proficiency. The curriculum layout reflects the products of these analyses.
Much of the work involved in designing a curriculum has been accomplished through the
preparation of the task list, proficiency objectives and event sets. If a series of objectives and
events have been developed, the design of the curriculum is largely complete, except for
choosing the specific locations within the syllabus for the individual elements.

A. Qualification Curriculum

CRM training should progress from general information to specific application. First, a
separate CRM portion might be appropriate to address the philosophical issues pertaining
to Captain’s and First Officer’s authority and corporate expectations concerning
professionalism and individual responsibility. In addition, CRM is also likely to play a
supporting role in other portions, such as flight management during conditions of severe
weather. The decision processes involved in managing severe weather conditions
provides an appropriate flight management context for addressing operational issues
pertaining to weather.

Specific qualification training will likely use a different set of training topics. These
topics could reflect the transition from knowledge to skill acquisition and, finally, to skill
application. If so, the sections will reflect the following learning stages:

a) Knowledge: This includes basic awareness training concerning the nature of the
skills, their value, strategies for using them, and ways to assess the effectiveness
of skill use. Presenting the different roles that could be played by each
crewmember sets the stage for later events where the crew must actually assume
the appropriate roles for that situation.

b) Procedures: This training includes the proceduralized aspects of CRM which are
typically reflected in an air operator’s SOPs.

c) Manoeuvres: This training can extend beyond simply practicing individual

manoeuvres so as to encompass situation assessment, planning, workload
distribution, and other critical CRM skills.

d) Flight management: This training requires the strategic use of multiple skills
adapted to the requirements of challenging flight situations. It also requires the
accurate assessment of skill effectiveness in management of such situations.
Effectively accomplishing such training requires a systematic approach to the
development of scenario events designed to elicit complex crew skills.

125
B. Continuing Qualification Curriculum

This curriculum has two goals: To evaluate pilot and crew proficiency, and to provide
supplemental training. Because of the severe time constraints imposed on this
curriculum, only “snapshot” samples of pilot and crew performance are possible. If a
flight management framework has been used to prepare the task list and proficiency
objectives, the performance samples could utilize an event-based approach that gauges
pilot and crew performance for procedures, manoeuvres, and flight management.

7.2.7 Line Operations and Proceduralized CRM

Developing and teaching specific observable actions that would be required in the execution of
specific activities at designated points in normal flight operations, as well as during abnormal or
emergency conditions, can enhance the crew’s ability to communicate effectively, plan and
manage their workload, and solve problems during flight operations. A procedural approach
may raise key aspects of CRM to the level of SOP, which increases CRM’s operational
significance and provides crews with a standard form of CRM. CRM procedures may be
embedded in a range of crew activities through the different phases of flight, reducing
distractions to the pilot flying (PF) in both normal and abnormal situations. Also, providing
structure to briefings with a checklist format can enhance the crew’s performance and improve
the transfer of critical information.

7.3 Evaluating CRM

7.3.1 Overview

The training developed for AQP reflects the corporate philosophy as to how the job is to be
performed, including the identification of observable behaviours that serve as the basis for
evaluation. The use of a flight management philosophy during the development of training
curricula and during the actual training supports an outcome-oriented assessment of skill use.
Effective task activity performance is defined within the context of achieving predefined flight
objectives. This approach supports the definition of objective standards that can serve as the
basis for evaluation, and provides a consistent foundation on which to evaluate both CRM and
technical skills.

7.3.2 Observable Behaviours

Observable behaviour is a specific action employing a CRM skill in a given situation.

Evaluation of individual pilot and crew CRM proficiency is possible through the inclusion of
observable behaviours in the evaluation criteria that reflect performance on CRM-related task
activities. These observable behaviours are defined for both the proficiency objectives and the
events. Evaluation of CRM skills is possible if the event sets used in the evaluation process
address these skills and incorporate appropriate observable behaviours in the performance
standards.

126
7.4 CRM in Line Operations

7.4.1 Supplemental Feedback

Pilots should be able to recognize when flight management task activities need to be performed,
implement strategies for performing these activities, and utilize techniques for assessing their
effectiveness in the achievement of flight objectives. The AQP process offers a formal analytical
methodology for developing and maintaining effective training programs towards that end. It
provides a concrete approach to defining, training and evaluating CRM. In particular, it supports
skill-oriented training intended to provide crews with realistic strategies for effectively managing
flight situations in accordance with corporate standards for safety and efficiency. Further, it
encourages the use of a comprehensive strategy for training pilots, instructors, and evaluators to
help ensure that a common corporate standard is followed by all.

However, even with the best of training and evaluation strategies, the extent to which skills are
regularly applied during normal operations must be determined. The ability to demonstrate the
successful application of CRM skills in training does not ensure their use outside this setting. It
is therefore important that air operator develops mechanisms to assess CRM practices during line
operations. AQP requires full crew OEs as one source of information on CRM practices during
actual line operations. Additional sources of feedback are strongly recommended. The following
are among some of the vehicles that have been effectively employed for that purpose:

a) Anonymous surveys to measure acceptance of CRM concepts;

b) Periodic non-jeopardy audits of line operations to observe CRM practices and Threat and
Error Management techniques;

c) Pilot non-punitive self-reporting programs;

d) Company-based incident reporting systems that can be supplemented by data from other
outcome-based systems; and

e) Formalized programs (such as the routine analysis of data from a Flight Operations
Quality Assurance – FOQA program and/or a Line Operations Safety Audit – LOSA
program).

7.4.2 Corporate Culture

Ultimately, the effectiveness of CRM in the operational environment depends upon the extent to
which an air operator treats CRM as an integral part of its culture. The commitment of corporate
management to this integration is critical to its achievement. A corporate decision to implement
AQP is a clear indication of that commitment. It is expected that good CRM will be adopted as a
cultural norm and seen as important as good stick and rudder skills. A measure of the successful
achievement of that norm is the extent to which a pilot’s competence, as perceived by peers, is

127
determined by flight management skills as well as by technical skills. AQP provides a highly
effective vehicle for achieving such an organizational culture.

128
Appendix A - AQP DOCUMENTATION

AQP Document Requirements

There are six documents and an annual report required for each AQP air operator. The six AQP
documents, and the required number of each document, are listed below:

1. Application/Administration - One per AQP air operator.

2. Job Task Analysis - One for each trainee type and one for each trainee type’s
instructors, evaluators, QAIs and QAEs.

3. Qualification Standards - One for each trainee type and one for each trainee type’s
instructors, evaluators, QAIs and QAEs.

4. Instructional Systems Development (ISD) Methodology - One per AQP air

operator.

5. Curriculum Outline - One per curriculum for each type, model, series, variant and
instructors, evaluators, QAIs and QAEs.

6. Implementation and Operations Plan - One per AQP air operator but must be
organized to address issues specific to each curriculum

Each of the above documents must remain current throughout the life of the AQP and must have
a revision control process acceptable to Transport Canada.

In addition to the six documents described above, the air operator is also required to submit to
TC an annual report once any curricula has reached Phases IV and V. Details can be found in
paragraph 2.7.1 and Appendix C.

129
Minimum AQP Documentation Requirements for the Program Audit
Database

Listed below are the topics that must be addressed in each document. If an air operator adopts a
different document configuration than the one suggested below, then the air operator should
provide clear and specific guidance in the applicable document(s), as to the location of the
information for each of these document topics.

Phase Document Type Document Topics

I Application/Administration 1. Application Statement of Intent

2. Applicant Staff Organization
3. Documentation Organization and Revision
Procedures
4. Reporting Requirements
5. Aircraft Configuration and Performance
Baseline
6. Operating Environment Description
7. Trainee Demographics
8. Principle Operator Document References
Governing Operations
9. Training Equipment Description and
Location
10. Facilities Description
11. Transition Plan/Update
12. PADB Master List

II Job Task Analysis 1. General Explanation

(JTA) 2. Task Hierarchy
3. Knowledge and Skills
4. CARs Training Requirements

II Qualification Standards 1. Prologue

2. Training/Checking Requirements
compared to CARs Requirements
3. Curriculum Test and
Evaluation/Remediation Strategy
4. Qualification Standards

130
 II Instructional System Design 1. Curriculum Development Methodology
(ISD) Methodology 2. Line Operational Simulation (LOS)
Development Methodology

II Curriculum Outlines 1. Curriculum Entry/Prerequisite Analysis

2. Curriculum Footprint
3. Curriculum Outline

II Implementation and Operations 1. Implementation and Operations Guidance

Plan and Policy
2. Data Plan

IV, V Annual Reports (to TC) 1. Performance/Proficiency Data Analysis

Reports
2. Curriculum Maintenance Reports

131
Appendix B - AQP DOCUMENTATION CHECKLIST

This section provides detailed information for use by both Transport Canada and the air operator
for the development and review of required AQP documents. Subject to the authorization of
Transport Canada, other document configurations may be utilized. If an air operator adopts a
different document configuration, other than the one suggested here, the air operator should
provide clear and specific guidance in the applicable document(s), as to the location of the
information for each of these document topics.

Application/Administration Document

(Phase I Step 1)

The purpose of the Application/Administration Document is to establish the applicant’s

methodology for developing an AQP for all or part of its fleets, and for instructors and
evaluators. The application is submitted once and is updated as information in the document
changes (e.g., a change in the transition schedule, adding new aircraft types, etc.). In order to
establish the applicant’s intent and approach for developing an AQP, the application should
thoroughly discuss the topics in the following checklist.

A. Statement of Intent

This section of the application should address the following topics:

a) The applicant’s intent to develop, implement and operate an AQP.

b) The general concept, approach and methodology for developing the AQP
(specific methods and procedures for all steps).

c) The general concept, approach and methodology for implementing the AQP.

d) How and to what extent the AQP is expected to differ from a traditional training
program.

e) How the AQP will be operated and maintained.

f) How CRM will be integrated and measured.

g) How LOS concepts will be integrated into both evaluation and training.

h) How existing levels of performance and safety will be met or exceeded.

132
B. Applicant Staff Organization

The size of the air operator’s organization and the level of effort required by the air
operator in developing its AQP will dictate the level of personnel man-hours required.
The AQP staff can be comprised of personnel resources already existing within the
organization, or with contractor support. These personnel can be used to develop an AQP
in concert with their normal duties. As a general rule, the staffing and expertise of an
AQP applicant’s staff should consist of the following:

a) AQP Co-ordinator: Management or supervisory level person who is the

company focal point for its AQP development effort. This person is not only
responsible for AQP leadership but will also act as the primary contact with
Transport Canada and any other external organizations that may impact the air
operator’s AQP. Experience has shown that this position should be filled with an
individual who understands the job tasks and training requirements the AQP is to
address.

b) Subject Matter Experts (SME): Current and qualified individuals who have
varying levels of expertise that fairly represent the population of professionals the
AQP will affect. The individuals must act as liaison with operational support
personnel.

c) Document and Curriculum Developer(s): Individual(s) who interface with the

AQP Coordinator and the SMEs to develop the requisite AQP process,
curriculum, and instructor and evaluator documents.

d) Document Managers: Individual(s) who ensure AQP document control and

congruence with Transport Canada approvals.

e) Computer Specialist/Database Manager: Individual(s) who develop and

manage the performance/proficiency data acquisition and analysis system. In
addition, this individual could be used for other computer-related issues related to
the facilitation of an AQP such as electronic document review.

C. Documentation Procedures

The prudent management of an AQP depends on an organized, coordinated and well-

maintained documentation system.

The application must describe the air operator’s proposed AQP documentation scheme,
how changes to the documentation will be managed, and how this documentation will
relate to the air operator’s current traditional training program documentation.

133
D. Reporting Requirements

The applicant must discuss their responsibility in providing Transport Canada with
performance/proficiency data, on demand, for inspection and audit purposes. The
applicant must also discuss the need to provide an annual report to Transport Canada.
Finally, the air operator must discuss their responsibility to respond to Transport Canada
requests for information that may fall outside the auspices of an inspection/audit or
annual reporting.

E. Aircraft Configuration and Performance Baseline

For each type, model, and series of aircraft (or variant) the following information shall be
provided:

a) General descriptive summary of each aircraft.

b) Training and qualification recommendations included in the standards.

c) Flight Crew Training Manual (FCTM)

d) Aircraft Operations Manuals or FCOM (identify by title).

F. Operating Environment Description

Applicants should describe their operating environment, including the general physical
environmental factors expected to be encountered in operations. Environmental factors
are critical to the development of LOS scenarios and serve as meaningful proficiency
objectives. Environmental factors include:

a) Weather norms and extremes (e.g., minimum extremes of expected weather

conditions)

b) Normal, abnormal and emergency equipment operation.

G. Trainee Demographics

Demographic data will be part of the supporting data. This data should be updated as
demographic changes warrant. This data may be provided in tabular format and must
include the following:

a) General summary data for trainee experience and entry level should be provided,
by aircraft type, model, series (or variant). Entry requirements for ground and
flight instructors and evaluators should be stated; e.g., previous experience
working for the air operator. Students should be identified as a group in terms of
previous experience (e.g., with high, low and mean experience).

134
 NOTE: It may be desirable to create curriculum modules for more than one
student entry-level population for a single duty qualification.

b) The current and anticipated need for replacement crewmembers by duty position
(throughput) should be provided.

H. Documents Governing the Applicant’s Operations

A list of air operator’s principal references which govern its operations. (e.g., COM,
AOM, etc.)

I. Training Equipment Description

This document should list the training equipment to be used and identify the organization
responsible for its security and maintenance. Flight simulators and/or FTDs should be
identified by type, model, serial number, and manufacturer; or by the identification
number assigned by Transport Canada. Specifically, the applicant will identify any new
training equipment to be used, if known at the time of application. If qualification is
required, applicants should indicate when they intend to submit a request for equipment
qualification. Qualification requests will be processed in accordance with existing
procedures.

J. Facilities Description

Each AQP submission should describe the facilities the applicant intends to use. These
facilities may belong to an air operator or may be operated by a training centre. In either
case, the applicant should briefly describe the location, general type of facility,
classrooms, training aids, and other resources to be used to support AQP training.

K. Transition Plan

Each applicant will include a separate transition plan (containing a calendar of events) to
accompany the cover letter. Transition from one program to another (traditional to AQP
or AQP to traditional) may include a period of overlap as one program is phased in and
the other phased out. This plan must cover all aircraft, flight crewmembers, instructors
and evaluators. The following are guidelines for appropriate transition:

a) Currently qualified personnel may transition between traditional recurrent training

curriculum and Continuing Qualification Curriculum.

b) Personnel who have completed Initial, Transition or Upgrade Curricula may enter
a Continuing Qualification Curriculum. Personnel who have received traditional
indoctrination training, but have not completed an Initial, Transition or Upgrade
Curriculum may enter an AQP Qualification Curriculum.

135
 c) Personnel who are qualified and current instructors or evaluators may transition
via an AQP Differences Course.

d) Partial transition plans are not acceptable.

e) The transition plan may provide for incremental implementation of Qualification

and Continuing Qualification curricula in Phase IV (Initial Operations), and
incremental final authorization into Phase V.

L. Program Audit Data Base (PADB) Master List

A master list of all documents in the database is required. All documents should be listed
by title. The applicant may tailor the listing to include only applicable documents, or
combine documents, provided prior Transport Canada authorization is obtained.

Job Task Analysis (JTA)

(Phase II Step 1)

A JTA is the method or procedure used to reduce a unit of work to its base components. The
JTA provides a detailed, sequential listing of tasks, sub-tasks, and elements with the knowledge,
skill, and attitude characteristics (KSAs) that clearly define and completely describe the job. An
applicant will provide a JTA for each type, model, and series aircraft (or variant). These may be
submitted as individual listings, or a single-higher level listing with appendices for each aircraft,
showing its unique lower-level features. It also identifies characteristics such as consequence of
error (Criticality), relative difficulty, and frequency of occurrence in specific operations
(Currency). As a complete document, the JTA has four components: a job task list, a learning
analysis (KSA), identified crew positions, and references.

A. General Hierarchy

The JTA should provide a general introduction explaining the development of the task
listing and the subsequent task analysis and how it is to be used to form the basis for the
Qualification Standards, and the AQP curricula upon which they are built.

An applicant will provide a Task Listing for each type, model, and series aircraft (or
variant). These may be submitted as individual listings, or a single-higher level listing
with appendices for each aircraft, showing the unique lower-level features.

A Task Listing is developed to include tasks, sub-tasks, and (where desired), attitudes.
The Task Listing should be of sufficient scope to identify all prerequisite knowledge and
skills. This can be accomplished by reference to applicable documents at the element
level. In addition, the Task Listing should incorporate all knowledge and skill
requirements currently specified in the regulations (e.g., Hazardous Material, Security,

136
 Emergency equipment, etc.). The identification of Knowledge, Skills and Attitudes is
referred to as KSA development.

The task listing includes the listing of phases of flight, tasks and sub-tasks only. Because
this basic structure forms the backbone of the much more detailed JTA, it should be
reviewed and authorized by Transport Canada before the development of the JTA.

B. Task Hierarchy

The task listing should be organized in a hierarchical fashion, with the flight phases at the
top level, higher-order tasks at the next level, lower-order component sub-tasks at the
next level, elements (where desired) at the next level, and applicable knowledge, skills,
CRM markers, and (where desired) attitudes, in subsequent descending order. The Task
Listing should include an internally consistent numbering system that represents the
hierarchical order. One acceptable format is listed below:

2.1 Take-off

2.1.1 Perform Normal Take-off [Captain or First Officer].

2.1.2 Perform take-off roll [Captain or First Officer]

2.1.3 Apply appropriate take-off guidance IAW AOM Chapter X, Section XX (K)

2.1.4 Release brakes (Psychomotor Skill - PS)

2.1.5 Use nosewheel steering to remain on centreline (PS)

2.1.6 Set appropriate take-off power (PS)

2.1.7 Monitor instruments for appropriate operations (Cognitive Skill - CS)

Note: The following Acronyms are used in this JTA:

CA Captain
FO First Officer
K Knowledge
PS Psychomotor Skill
CS Cognitive Skill
IAW In accordance with

137
 Another acceptable format of the Task listing, exhibiting KSAs by reference, follows:

Task Number Task Source Page Number

2 TAKE-OFF
2.1 Assess Take-off Airport Analysis as required for city
Environment Charts
2.1 Assess Take-off Take-Off and
Environment Landing
Speeds
2.1 Assess Take-off Pilot Handbook 4-13 to 4-18
Environment
2.2 Perform Normal Pilot Handbook 2-22, 2-35 to 2-37, 3-
(Instrument) Take-off 8,
6-18 to 6-20
2.3 Perform Rejected Pilot Handbook 3-13, 6-54
Take-off
2.3.1 Perform engine fire Abnormal Emergency 1-B
procedures before V1 Checklist
2.3.1 Perform engine fire Pilot Handbook 1-6, 6-55
procedures before V1
2.3.2 Perform engine Abnormal Emergency 3-B
failure procedures Checklist
before V1
2.3.2 Perform engine Pilot Handbook 1-10, 6-54
failure procedures
before V1
2.4 Perform Take-off Abnormal Emergency 3-B
with Engine Failure Checklist
after V1
2.4 Perform Take-off Pilot Handbook 6-49 to 6-51
with Engine Failure
after V1

The JTA must denote the specific aircrew position, where applicable. In the example
above, [Captain or First Officer] is annotated by "Perform Normal Take-off" and
"Perform Take-off Roll".

C. Flight Crew Training Requirements

The Task Listing must provide task detail to the Knowledge and Skill level (Attitudes are
optional). CRM factors must be included, either as part of the main Task Analysis, or as
a separate CRM Task Analysis that is later integrated into the Qualification Standards
and AQP curricula. The knowledge and skill descriptions may be specified by reference
to one or more documents, at the element level or below. The document reference must

138
be of sufficient specificity to provide precise guidance in discerning the knowledge or
skill description (i.e., document name, chapter, section, paragraph number, page number).

NOTE: It is only for the initial iteration of the JTA that references are made all the way
down to the page number. For the Qualification Standards, and updates to the JTA,
references need be made only down to the document title and section or chapter title
level.

CARs Training Requirements must be addressed. The Task Analysis/Listing is the basis
for Qualification Standards and Curriculum courseware development. Since AQP
programs must meet or exceed the current regulatory standard of training quality, the
training subjects listed below should be addressed in the Task Analysis/Listing.
Deviations will be entertained on a case-by-case basis. They should be specified either as
specific Knowledge and Skills required for the execution of sub-tasks or elements, or by
reference at the element or KSA level.

Aircraft Systems' Subjects (as applicable to aircraft type):

• Aircraft General.

• Equipment and Furnishings.

• Emergency Equipment.

• Powerplants.

• Electrical.

• Pneumatic.

• Air-conditioning and Pressurization.

• Ice and Rain Protection.

• APU.

• Hydraulics.

• Landing Gear and Brakes.

• Flight Controls.

• Fuel.

• Communications Equipment.

139
 • Flight Instruments.

• Navigation Equipment.

• Autoflight.

• Warning and Detection Systems (to include TCAS, GPWS, and WX Radar).

• Fire and Overheat Protection.

• Oxygen.

• Aircraft Performance General System-Operations Integration Training.

• AOM or FCOM or AFM Content and/or SOPs.

System-Operations Integration Training:

• Pre-flight Visual Inspection.

• Pre-start Checklist and Procedures.

• Powerplant Start.

• Taxi to include lowest visibility allowed by OP SECS.

• Pre-Take-off Checks and Procedures.

• Normal Take-off.

• Crosswind Take-off.

• Instrument Take-off (Low Visibility).

• Powerplant Failure on Take-off (at or near V1).

• Powerplant Failure After V2.

• Rejected Take-off.

• Area Departure.

• Cruise Procedures.

• Holding.

140
Area Arrival.

Normal ILS.

Engine-out ILS.

Autopilot-Coupled ILS.

Non-precision Approach.

ILS missed Approach.

Second Missed Approach.

Precision Radar Monitored Approaches/Missed.

Circling Approach.

No-flap Landing.

Crosswind Landing.

Landing With Engine Out.

Landing From Circling Approach.

Rejected Landing.

Landing With 50% Power Loss.

Approaches to Stalls.

Steep Turns.

Powerplant Failure.

Wind shear Training.

Mis-trimmed Situations/Trim Runaway.

Selected Events - Unusual Attitudes.

TCAS and GPWS –Escape.

Normal and Abnormal Procedures.

141
 Emergency Procedures.

COM Content, as applicable:

• Company Policy or Procedures (dispatch and flight release requirements).

• Regulations, Operations Specifications, and SOPs.

• Weather Requirements (seasonal changes, flight into various geographic locations

and temperatures).

• Hazardous Material.

• Security (to include Hijack Procedures).

• Special Operations (special airports, special approaches and departures).

• Emergency crew assigned duties and procedures.

• Operation of emergency equipment/systems.

• Operation of ditching and evacuation equipment/systems.

• CRM.

• Emergency Situation Training - Rapid Decompression, Fire in Flight or on the

surface, and smoke control procedures.

• Assistance of Persons to Exits during Emergency.

• Illness, Injury, or Other Abnormal Situations Involving Passengers or Crew (to

include use of medical kit).

• Flight Physiology (i.e., Hypoxia, Respiration, etc.).

• Use of Checklist.

• Cockpit Familiarization.

• Pre-flight Planning and FMS.

• In-flight Planning LNAV, VNAV, and GPS.

• Required Navigation Procedures.

142
 • Navigation Systems Integration.

• Autoflight and Flight Director Integration.

• Use of Radar/CRTs.

• TCAS/ACAS.

• GPWS/TAWS.

• Communication Systems Integration (ACARS/FMS/CPDLC).

• Surface Movement Guidance Systems (SMGS).

• Runway Incursion Prevention Strategy.

• Stabilized Approach Strategy.

• Precision Radar Monitoring (PRM) Procedures.

• Land and Hold Short (LAHSO) / Simultaneous Intersecting Runway Operations

(SIRO) Procedures.

• CAT II/III operations.

Qualification Standards

(Phase II Step 2)

The Qualification Standards document is the single most important document of any AQP. It
provides the complete curriculum baseline. As such, it will be all-inclusive in specifying the
foundational aspects of the content and execution of the AQP curriculum.

The Qualification Standards document has four parts. These are:

a) Prologue.

b) Regulatory Requirements Comparison.

c) Testing/Validation/Evaluation & Remediation Methodology.

d) The Qualification Standard.

143
Each part is described in detail below:

A. Prologue

This is an introductory section that explains the methodology, format, and terminology of
the document.

The intent of the Qualification Standards Prologue is to provide the background

information so that a reader of the Qualification Standard, who is not intimately familiar
with its development, could comprehend how the AQP compares to existing training
requirements and how the development and implementation of the AQP curricula relate
to the Qualification Standards. It is important to note that the Qualification Standards
serves as the definitive basis for the training program. Therefore, the Prologue to the
Qualification Standards should be comprehensive and understandable so that a reader can
discern the scope and appropriateness of the training. In addition, any aspect of the
curriculum, from curriculum outline to lesson topics or grade sheet items should be
traceable to some item in the Qualification Standard.

The following are topics that the Prologue should address:

a) The Prologue must discuss the methodology used to develop the Qualification
Standards document.

b) The explanation of methodology must show how any aspect, from curriculum
outline to lessons topics or grade sheets, is to be traceable to an item in the
Qualification Standard.

c) The Prologue must discuss the format (structure) that is used for the Qualification
Standards.

d) The Prologue must define terms used for the Qualification Standards.

B. Regulatory Requirements Comparison

The Qualification Standards document must also include regulatory comparison

information. The comparison must show how the AQP will address the requirements of
the CARs/CASS, as applicable. The comparison should be both comprehensive and
clearly understandable. It must allow the reader to easily discern the scope and
appropriateness of the training and checking activities.

The following individual issues must be addressed:

a) The Regulatory Requirements Comparison must specify the requirements of the

CARs/CASS and show how they will be addressed in an AQP.

b) All departures from traditional requirements must be identified and justified.

144
 c) Any Qualification Standards specifications that are at variance from Schedule I, II
or III as applicable must be explained.

C. Testing/Validation/Evaluation & Remediation Methodology

This section is a detailed plan describing the point in the curriculum when a test,
validation, or evaluation will be applied. It must identify what constitutes a failure and/or
unsatisfactory performance. This section will also describe the remediation strategy to be
used to correct unsatisfactory performance and special tracking provisions.

Most objectives will be validated using one of the following methods:

a) In a train to proficiency context;

b) As part of MV and/or LOE; or

c) During OEs.

One common variant of this approach is to divide the Qualification Curriculum into four
evaluation gates: SKV, PV, MV, and LOE. While the specific strategy assignment for
each individual Qualification Standard will be spelled out in the body of the text, this
strategy provides a general discussion of the types of evaluations and the rationale for
selecting and assigning evaluation strategies to Qualification Standards.

The following individual issues must be addressed:

a) A detailed plan for describing how evaluation is accomplished throughout each

curriculum is contained in this section. It describes how, when, where and who
will assess a candidate’s proficiency on each Terminal and Supporting Objective.

b) The points in the curriculum where the testing, validation or evaluation will be
applied are identified. These will include: SKV, PV, MV, LOE, and OE.

c) Clear definitions of the different strategies that will be used to test, validate or
evaluate performance will be provided. These will include: First Look, Train to
Proficiency, SKV, PV, MV, LOE, and OE.

d) A description of how the “criticality” and “currency” ratings translate into testing
strategies for TPOs and SPOs in the Continuing Qualification Cycle will be
included.

e) This section will describe how a TPO with several SPOs may be alternatively
sampled over multiple evaluation periods of Continuing Qualification Cycles.
e.g., TPO (Non-Precision Approaches) SPOs (Non Directional Beacon (NDB),
VOR, BC, etc.).

145
 f) A clear description of the rating scale that will be used by instructor and evaluator
to score performance will be included. Rating scale definitions must clearly
discriminate various performance levels.

g) The defining criteria for a failure and/or unsatisfactory performance for each
validation/evaluation point will be identified.

h) A discussion of the commensurate remediation strategy that would be employed

(i.e., special tracking, etc.) in the event of a failure will be included.

i) The description of remediation strategy must detail ‘when’ and ‘what’ must be
repeated and whether or not additional training is warranted.

j) The description of remediation strategy will also detail the methodology that will
be used to remediate unsuccessful test, validation or evaluation sessions.

k) For circumstances when no more training will be offered to the individual, the
remediation strategy must specify and the resulting actions such as “Referred to
Director of Training”. “Returned to previous position”, etc.

l) Remediation strategy must describe the criteria for placing an individual on

“Special Tracking”. It must also detail the strategy used for these individuals
while they have this status. Finally, it will describe what must take place for an
individual to be removed from Special Tracking.

m) The level of training devices, simulator or aircraft to be used to evaluate the

proficiency objectives at each point in the curriculum will be described.

D. Qualification Standards

The Qualification Standard is constructed by applying a performance statement,

conditions, and standards to a task or sub-task, thereby creating a TPO or an SPO.

The following items will be included for each individual Qualification Standard.

a) Header identifying the air operator and the document.

b) Page revision control dates and revision numbers.

c) Consecutive page numbers.

d) Phase of Flight: Number and Title from Task Listing.

e) Qualification Standard Title: Either TPOs or SPOs.

146
f) Task or Sub-task: Number and Title From Task Listing.

g) Crew Duty Positions.

h) Criticality/Currency Rating: From Task Factors Analysis.

i) Curriculum: This field identifies the curriculum in which the task will be trained
and evaluated.

j) Evaluation Strategy: The validation or evaluation point for this particular

Qualification Standard: e.g. train to proficiency, PV, MV, LOE, or OE.

k) Media: The specific media in which training and/or evaluation will be conducted.
For Qualification, the media is the lowest media used for final evaluation. For
Continuing Qualification, the media is the range of media used for training.

l) Performance Statement: An expanded statement of expected behaviour, which,

when executed, will complete the work required for a specific portion of a job.

m) Conditions: The applicable conditions and combinations thereof that will be

addressed in training and/or evaluation. If this Qualification Standard is
applicable to both the Qualification Curriculum and Continuing Qualification
Curriculum, annotate those conditions that the task will be evaluated with for
Qualification/Certification. The Qualification Standard should (1) indicate those
specific conditions to be trained and tested as part of the Qualification
Curriculum, and (2) provide a more exhaustive listing of conditions over which
flight crewmembers will be trained and tested during the course of successive
Continuing Qualification Cycles.

n) Contingencies: The specific contingencies to be used for the Qualification

Curriculum.

o) Standards: The standards that will apply for training to proficiency and/or
evaluation of proficiency. These standards are usually organized under the
headings of Manoeuvre Standards and Procedural Standards. Manoeuvre
Standards should be relatively specific and resemble the standards listed in the
Schedule I, II or III as applicable. The Procedural Standards are usually more
general and may reference information in a document or manual. The reference
should be specific to chapter or section but does not require page number.

p) References: Identify the primary references from which performance statements

and associated standards were derived. Cite documents by title and where
applicable, chapter or section. Page numbers are not required.

There is a difference between the Qualification Standard requirements for a Qualification

Curriculum and the requirements for a Continuing Qualification curriculum. The

147
 Terminal Level and Supporting Level Tasks in a Qualification Standard for a
Qualification Curriculum must identify the specific set of conditions to be employed in
training and evaluating a task. In addition, the media description will describe the
specific media in which the task will receive final evaluation. In contrast, the Terminal
Level and Supporting Level Tasks in a Qualification Standard for a Continuing
Qualification Curriculum may identify a selectable menu of conditions to be employed in
training or evaluation, and a media range that will specify the minimum media level in
which the task may be trained. Because of this difference, applicants may find it more
convenient to submit separate Qualification Standards documents for Qualification and
Continuing Qualification. However, the Qualification Standards for both of these
Curricula can be combined in one document as long as the differences in conditions and
media are addressed, and the tasks are annotated for applicable curricula.

Some Qualification Standards may be “aircraft generic” in that they may apply to more
than one aircraft type. For ease of use and definition of “fleet common” curricula,
applicants may also find it convenient to specify these Qualification Standards separately
from those Qualification Standards that apply to only one aircraft fleet type.

For the instructor and evaluator Qualification Standards, the document must adequately
address the role of instructor and evaluator. This means that the instructor and evaluator
process must be specified in adequate detail to include a description of who will
administer the instructor and evaluator evaluations, how often they are administered, and
what is evaluated. Applicants must provide sample-grading forms.

Instructional Systems Development Methodology

(Phase II Step 3)

The ISD Methodology document describes the approach to be used by applicant air operators to
develop and maintain all AQP curricula. This document is divided into two sections:

a) Development Procedures; and

b) Line Operational Simulation (LOS) Methodology.

A. Development Procedures

This section describes the procedure for using the JTA and Qualification Standards as
baseline documents to construct their training curricula. This involves allocating
objectives to lessons, selecting media and methods, and developing the curricula. It
explains the basis for grouping lessons into modules, modules into segments, and
segments into a curriculum.

148
 The following individual issues must be addressed:

a) The procedure for allocating TPOs and SPOs to lessons, selecting media and
methods, as well as developing curricula is described here.

b) A description of how EOs are developed to support their higher end objectives is
included.

c) This section contains a description of how learning and evaluation activities are
developed to support these objectives.

d) The assignment of instructional media and methods is described.

e) The method for clustering and sequencing objectives into lessons, modules,
segments and curricula is described.

f) A method for developing an audit trail will be described. This will link
proficiency objectives, lesson activities/content and test items.

B. Line Operational Simulation (LOS) Development Methodology

This document describes the approach for developing LOS scenarios. This includes the
methodology used for LOFT, LOE and SPOT development.

The following individual aspects must be included:

a) The method for constructing a typical scenario.

b) An explanation of how each event set relates to a phase of operation.

c) A description of how each event set consists of a series of proficiency objectives

that include both technical and CRM activities.

d) A description of the use of event conditions, triggers and distracters, as well as

supporting events.

e) An explanation of how the applicant will identify possible sources of incidents

that will elicit the behaviour required by the proficiency objectives for the
scenario.

f) Definitions of the basic success criteria for the LOS, and each event set within it,
must be included.

149
 g) The applicant must define the scenario development process in terms of the
following issues:

i) An explanation of who will do the work of drafting the scenarios.

ii) How grade sheets will be used.

iii) Who will be involved with testing the scenarios.

iv) How instructors and evaluators will be trained to administer LOS

scenarios.

AQP Curriculum Outline

(Phase II Step 4)

Curriculum Outlines are required for each of the two primary curricula (Qualification and
Continuing Qualification) and any specialized secondary curricula (Requalification, Transition,
Upgrade, etc.) for every duty position in each type, model, and series/variant aircraft (refer to
Chapter 3).

The Curriculum Outline is a listing of course content. It should be arranged from curriculum
into segments, segments into modules, modules into lessons, and lessons into elements or topics.
Each part of the Curriculum Outline must clearly indicate the subject matter to be taught and
correspond directly to the hierarchical system of the task analysis. A Curriculum Outline
provides the basis for the Curriculum Footprint, which is a high level graphical overview of the
curriculum content depicting the training and evaluation activities and the proposed hours for
each day of the curriculum.

To maintain appropriate oversight of the curriculum, the Curriculum Outline provides a

sufficient level of detail and will allow the AQP applicant to make changes to the syllabus
without submitting a new syllabus document for each syllabus change.

A. Curriculum Outline

The following aspects must be addressed:

a) Qualification and Continuing Qualification curricula are required for each duty
position in each aircraft type, model and series/variant.

b) Separate Qualification and Continuing Qualification curricula are required for

instructors and evaluators.

c) Special curricula are required for Transition, Upgrade, Requalification and

Refresher.

150
d) Each curriculum must be constructed in the following order: Curriculum,
segment, module, lesson, and lesson element or topic.

e) The curriculum outline must provide a level of detail that will allow the AQP
applicant to make changes to the syllabus without submitting a new document for
each syllabus change.

f) The Curriculum Outline must be part of a revision control system. This will be
indicated by page format.

g) Curriculum Outlines must provide a hierarchical link (proficiency objectives)

between the Qualification Standards and a curriculum.

h) Each part of the Curriculum Outline must clearly indicate the subject matter to be
taught and correspond directly to the hierarchical numbering system of the Task
Analysis.

i) Each Curriculum Outline must include the following:

i) Air Operator's Name.

ii) Type of Aircraft.

iii) Duty Position(s).

iv) Title of curriculum and/or curriculum segment.

v) A listing of numbered (coded) objectives organized into lessons, modules

and segments. Numbers (codes) must allow Transport Canada to track
objectives back to Qualification Standards and JTA.

vi) An outline of each training module within each curriculum segment. Each
module should contain sufficient detail to ensure that the main features of
the principal elements or events will be addressed during instruction.

j) The checking and qualification modules of the Qualification Curriculum segment

used to determine successful course completion, including any additional CARs
qualification requirements for flight crewmembers to be employed in their
respective CARs Part VII operations (e.g., line indoctrination, OEs, route and
aerodrome qualifications, etc.).

k) The applicant will document the analysis results of curriculum entry and
minimum prerequisite requirements for each curriculum. Examples include:
prior glass cockpit experience, prior instructor experience, flight experience, time
away from performing duties, etc.

151
B. Curriculum Footprint

This section provides a graphical depiction of the curriculum content overview in relation
to the number of days and proposed hours on a daily basis. For a sample AQP
Qualification footprint, see chapter 3.

The following issues must be addressed:

a) The Curriculum Footprint must describe the training and evaluation activities
conducted each day of the curriculum.

b) The Curriculum Footprint must include planned hours.

Implementation and Operations Plan

(Phase II Step 4)

This document summarizes the analysis of the training requirements for implementing and
operating the AQP. The Implementation and Operations Plan now includes the following
sections:

A. The Implementation and Operations Guidance and Policy

This section will include the following topics:

a) Curriculum Schedule: This section includes proposed schedules for the

implementation of the AQP curricula.

b) Phase III Curriculum Evaluation Strategy: This section describes the plan and
schedule for the evaluation of facilities, courseware, equipment, students,
instructors, evaluators, and performance measurement techniques. The plan
normally includes provisions for small group try-outs of all new courseware,
software, and equipment. Applicants who wish to provide no-jeopardy credit for
their small group try-out course graduates must request authorization to do so in a
separate letter addressed to the National Operations Branch, Airlines Division
(AAROA) or the regional Commercial & Business Aviation operational oversight
division, as appropriate. The air operator’s Quality Assurance team will be
assembled in order to provide Quality Assurance activities and oversight of the
program.

c) AQP Maintenance Strategy for Phases IV and V: This section describes

quality control procedures; i.e., plans for acquiring and measuring data for
tracking curricula, students, instructors, and Evaluator performance. It also
includes the strategy to be employed for curriculum maintenance and update.
Maintenance includes the methodology for maintaining control of the AQP

152
 approval documents, maintaining curriculum currency, upgrading equipment,
monitoring and responding to demographic changes, and for using
training/evaluation feedback to maintain and improve the AQP.

d) First-Look Activity Administration Strategy: To maintain the validity of the

performance and proficiency data obtained from the performance of the First-
Look activities, this section must describe the strategy employed for those
activities. This strategy must describe how the candidates will not be provided
with information or techniques on these activities prior to the first execution of
these activities that would unduly improve their performance if provided prior
assistance. In addition, this strategy should discuss when in the training program
the activities would occur and under what circumstances the activities would be
updated.

e) Crew Scheduling and Pairing Strategy: A basic tenant of AQP is to maximize

the training and evaluation of professionals in a pairing or crew configuration as
they would perform those duties in their typical job environment. Although the
AQP applicant must make every effort to schedule and pair professionals in a
typical crew configuration or pairing (e.g., a Captain with a First Officer),
situations will arise where non-standard professional pairings will arise. This
section must address the procedures used to ensure appropriate training and
evaluation will occur with a non-standard crew composition.

f) Instructor and Evaluator/Mentor Requirements: The description of

Instructor, Evaluator and Mentor requirements must include a schedule for AQP
Instructor, Evaluator and Mentor qualification. This schedule should identify
both differences training for existing instructors and evaluators, and the time
frame by which a full AQP course will be completed for future Instructors and
Evaluators.

B. Data Plan

Before an air operator can proceed with data collection and analysis, they must establish
a plan. The plan must address the intended purpose, collection methods, management,
analysis and reporting of AQP training/evaluation data. This plan must be thorough and
maintained to accurately reflect the air operator’s PPDB system. The air operator must
also realize its responsibility to collect and analyze sufficient data in order to adequately
identify performance trends and required changes. Refer to Chapter 6 for more specific
guidance on AQP Data Collection, Management, Analysis and Reporting.

The Data Plan will include the following topics:

a) Data Acquisition: This plan should address the methods used to collect
performance/proficiency data for all curricula. The rationale for employing a
particular method will be described. Examples of the data input medium (i.e.,

153
 grade sheets, computer-input screens, etc.) will be provided. In addition, the data
collection method should address data input quality control, security and usability.

b) Data Base and User Interface Management: This plan should discuss how the
air operator intends to store, access, and assimilate the performance/proficiency
data that is collected. This discussion will include:

i) The type of software data management system employed (i.e. relational

database, spreadsheet, etc.);

ii) The organization of the information in the electronic medium (i.e.,

database definition, database table relationships, spreadsheet description,
etc.); and

iii) A description of the user interface to this data management system.

c) Data Analysis: The air operator must discuss the type of analysis it will employ
to facilitate their own performance information needs as well as the performance
information needs of Transport Canada. This discussion must address how each
type of data will be analyzed.

d) Data Reporting: This part of the Data Plan must discuss Transport Canada’s
data reporting requirements. Format and frequency must be specified. In
addition, it should discuss the type of data reporting that will be employed. It will
include examples of the various report types (e.g. tabular reports, graphs). An
explanation of duties of various air operator personnel must be provided. Finally,
the requirement to provide Transport Canada with access to pilot
performance/proficiency data for inspection and audit purposes must be
highlighted.

e) Data Acquisition/Grading Forms: This section will include examples of all

forms used for data acquisition and grading.

f) Database Structure: This section will include a description of data field types
and a graphical depiction of the database table relationships.

g) Data Quality Assurance Plan: This section will describe the quality assurance
plan for data acquisition. The plan to assure data integrity and reliability and the
plan for instructor and evaluator grading calibration will be described.

154
Appendix C - AQP DOCUMENTATION CHECKLIST AND REVIEW JOB
AID TABLES

This AQP tool contains seven job aids for both TC and the certificate holder to use as
simplified checklists for the development and review of the documentation requirements of
AQP. Other document configurations may be appropriate to a specific certificate holder. If a
certificate holder adopts a different document configuration, other than the one suggested in
the attachment to Policy Letter 169, then the applicant should provide clear and specific
guidance as to the location of the information for each of these document topics. Certificate
holders should include specific references where information can be found which addresses
each item. The comment section may be used to record any remarks relative to the review and
approval status of the document.

There are six document types and one annual report required for each AQP certificate holder.
Each document will have its own job aid:

1. Application – One per AQP air operator

2. Job Task Analysis – One for each trainee type and one for each trainee type’s
instructor/evaluators
3. Qualification Standards – One for each trainee type and one for each trainee type’s
instructor/evaluators
4. Instructional Systems Development Methodology – One per AQP air operator
5. Curriculum Outline – One per curriculum/make, model, series, variant and
instructor/evaluator
6. Implementation and Operations Plan (I&O Plan) – One per AQP air operator

NOTE: Each of the above documents must remain current throughout the life of the AQP. Each
of the documents must utilize a revision control process.

Annual AQP Report – Certificate holders will monitor the status of all AQP curricula and the
performance/proficiency Data Base and will summarize their findings annually in a report to
TC. Although there is no established format for the report, the associated job aid highlights
areas that should be addressed.

155
Application, Phase I. The purpose of the application is to establish the applicant’s
methodology for developing an AQP for all of its fleets, their instructors and evaluators. The
application is submitted once and is updated as information in the application changes (e.g., a
change in the transition schedule, adding new aircraft, etc.). In order to establish the
applicant’s intent and approach for developing an AQP, the application should thoroughly
discuss the following topics numbered 1 – 9 in this job aid.

1 Statement of Intent Y N Comments

a. Does the Statement of Intent specify
the applicant’s intent to develop,
implement, and operate an AQP?
b. Does the Statement of Intent address
all fleets?
c. Does the Statement of Intent address
how and to what extent the AQP will be
operated and maintained?
d. Does the Statement of Intent address
how CRM will be integrated and
measured?
e. Does the Statement of Intent address
how LOS concepts will be integrated
into both training and evaluation?
2 The Applicant’s Staff Organization Y N Comments
a. AQP Coordination: Is a person
identified that will be the focal point for
the applicant’s AQP development and
contact with TC?
b. Subject Matter Expertise (SME): Are
current and qualified individuals
identified by name or position, who
have varying levels of expertise that
fairly represent the population of
professionals the AQP will address?
c. Document and Curriculum
Development: Are individual(s)
identified by name or position, who
interface with the AQP coordinator and
the SMEs to develop the requisite AQP
process, curriculum, and
instructor/evaluator documents?

156
 d. Document Management: Is an
individual identified, who ensures AQP
document control and congruence with
TC approvals?
e. Computer Specialist/Database
Management: Is an individual identified
who will develop and manage the
performance/proficiency data
acquisition and analysis system?
f. In addition, will the computer
specialist/database manager be used for
other computer-related issues related to
the facilitation of an AQP, such as
electronic document review?
3 Data Collection, Submission and Y N Comments
Analysis Reporting
a. Does the applicant acknowledge their
understanding and acceptance of the
AQP performance data requirements by
stating the intended purpose for the
collection, management, analysis, and
reporting of AQP training/evaluation
data for each curriculum?
b. Does the applicant defer to the I&O
plan for describing the process and
methodology for AQP data collection
and analysis?
c. Does the applicant acknowledge that
an electronic data management system
will be developed prior to entering
phase III of any AQP curriculum?
d. Does the applicant acknowledge that
requirement for submitting de-
identified data to TC no later than 2
months after collecting the data?
e. Does the applicant acknowledge the
requirement of a more stringent
collection and analysis of the data than
that submitted to TC?
f. Does the applicant describe the
purpose of the data analysis and how it
will be used?

157
 g. Does the applicant acknowledge the
requirement for submitting an annual
AQP report summarizing their data
analysis and any resulting changes that
ensued in their AQP program?
4 Supporting Documents and Manuals Y N Comments
a. Do the supporting documents and
manuals list each make, model, and
series aircraft or variant?
b. Has the applicant provided TC with
the following documents or manuals:
1. A current listing of company
and manufacturers manuals that
govern company operations?
2. General descriptive summary
of each aircraft type, including
aircraft configuration and the
performance baseline?
3. Flight Crew Operations
Manual (FCOM)?
4. Company Operations Manual
(COM)?
5. Manufacturer’s Aircraft
Flight Manual (AFM)?
6. Master Equipment List
(MEL/CDL)?
c. Does the applicant have available the
training and qualification
recommendations in the Flight
Standards Board reports where
applicable?
5 Operating Environment Description Y N Comments
a. Does the applicant describe the
operating environment, including the
general meteorological and geographic
factors expected to be encountered
during operations?
b. Does this description include the
weather norms and extremes expected
to be encountered in operations?
c. Does this description include normal,
abnormal, and emergency equipment
operation in geographic areas that
require special procedures (e.g., engine
failures in mountainous terrain)?

158
 d. Does this description include
terminal and enroute operating areas
such as controlled and uncontrolled
airfields?
6 Trainee Demographics Y N Comments
a. Does the applicant provide a general
summary of trainee experience and
entry level by aircraft make, model,
series or variant?
b. Does the applicant identify entry
requirements for ground and flight
instructors and evaluators?
c. Does the applicant group students in
terms of previous experience (e.g. with
high, low and mean experience
included)?
d. Does the applicant identify the
current and anticipated need for
replacement crewmembers by duty
position? This information is necessary
to determine priority on curriculum
development.
7 Training Equipment – Description Y N Comments
and Location
a. Does the applicant identify the
training equipment to be used, its
location, and identify the organization
(vendor or applicant) responsible for its
security and maintenance?
b. Does the applicant identify flight
simulators and/or flight-training devices
by make, model, serial number, and/or
TC identification number?
8 Facilities Description Y N Comments
a. Does the applicant describe the
location, general type of facility,
classrooms, training aids, course
software, and other resources to be used
to support AQP training?
9 Master AQP Transition Schedule Y N Comments
(MATS)
a. Does the MATS include all aircraft,
aircrew, instructors and evaluators that
the applicant intends to transition to
AQP?

159
b. Is the MATS complete? A partial
MATS is not acceptable.
c. Does the MATS address how A simple spreadsheet, provided by the
currently qualified personnel may applicant, may be helpful to highlight the
transition between traditional recurrent crew pairing requirements during the first
training & continuing qualification year of AQP operations.
curricula?
d. Does the MATS address how
personnel who have completed initial,
transition or upgrade curricula may
enter a continuing qualification
curriculum?
e. Does the MATS address personnel
who have completed a traditional basic
indoctrination curriculum, but have not
completed an initial, transition or
upgrade curriculum?
f. Does the MATS address personnel
who are current instructors or
evaluators and how they may transition
to AQP via a differences course?
g. Does the MATS address the
incremental implementation of the
curricula as opposed to all at once?
h. Does the MATS provide the time
frame necessary to withdraw from AQP
if it becomes necessary to revert to the
applicable CARS Part VII training
program?

160
Job Task Analysis (JTA), Phase II. A JTA is the method or procedure used to reduce a unit
of work to its base components. The JTA provides a detailed, sequential listing of tasks,
subtasks, and elements with the knowledge, skill, and attitude characteristics (KSAs) that
clearly define and completely describe the job. An applicant will provide a JTA for each
make, model, and series aircraft (or variant). These may be submitted as individual listings, or
a single-higher level listing with appendices for each aircraft, showing its unique lower-level
features. As a complete document, the JTA has four components: a job task list, a learning
analysis (KSA), identified crew positions, and references.

1 JTA Structure Y N Comments

a. Does the JTA provide a general
introduction explaining the development
of the task listing and the subsequent
task analysis and how it is to be used to
form the basis for the qualification
standards, and the AQP curricula upon
which they are built?
b. Is the JTA organized using a
hierarchical system with the flight
phases at the top level, tasks at the next
level, component subtasks at the next
level, elements at the next level?
c. Is the JTA complete with tasks,
subtasks, elements, and crew positions?
d. Has the applicant completed full
development of the JTA to the element
level?
e. In the JTA, are applicable knowledge,
skills, CRM markers, and (where
desired) attitudes applied at the element
level?
2 Flight Crew Training Requirements Y N Comments
Does the JTA incorporate all knowledge
and skill requirements currently
specified in the regulations?
a. Aircraft Systems’ Subjects
Aircraft General
Equipment and Furnishings
Emergency Equipment
Powerplants
Electrical
Pneumatic

161
 Air conditioning and Pressurization
APU
Hydraulics
Landing Gear and Brakes
Flight Controls
Fuel
Communications Equipment
Flight Instruments
Navigation Equipment
Autoflight
Warning and Detection Systems (to
include TCAS/ACAS,
GPWS/TAWS, and Weather Radar)
Fire and Overheat Protection
Oxygen
Aircraft Performance & Limitations
MEL/CDL
b. System-Operations Integration
Training
Pre-flight Visual Inspection
Pre-start Checklist and Procedures
Powerplant Start
Taxi to include lowest visibility
allowed by OPS SPECS
Pre-Takeoff Checks and Procedures
Normal Takeoff
Crosswind Takeoff
Instrument Takeoff (Low Visibility)
Powerplant Failure on Takeoff (at or
near V1)
Powerplant Failure After V2
Rejected Takeoff
Area Departure
Cruise Procedures
Holding
Area Arrival
RNAV, RNP, SAAAR
Normal ILS
Engine-out ILS
Autopilot-Coupled ILS – CAT II/III
Non-precision Approach
ILS missed Approach
Second Missed Approach

162
 Precision Radar Monitored
Approaches/Missed
Circling Approach
No-flap Landing
Crosswind Landing
Landing With Engine Out
Landing From Circling Approach
Rejected Landing
Landing With 50% Power Loss
Approaches to Stalls
Steep Turns
Powerplant Failure
Windshear Training
Mis-Trim Situations / Trim Runaway
Selected Events - Unusual Attitudes
TCAS and GPWS - Escape
Normal and Abnormal Procedures
Emergency Procedures
c. Company Operations Manual
Content
Company Policy or Procedures
(dispatch and flight release
requirements)
Regulations, Operations
Specifications, and Standards
Operating Procedures (SOP)
Weather Requirements (seasonal
changes, flight into various
geographic locations and
temperature-related requirements)
Hazardous Material
Security
Special Operations (special airports,
special approaches and departures)
Emergency crew-assigned duties and
procedures
Operation of emergency
equipment/systems
Operation of ditching/evacuation
equipment/systems
Crew Resource Management (CRM)
Emergency Situation Training –
Rapid Depressurization, Fire (in
flight/on ground), and Smoke Control
Procedures

163
Assistance of Persons to Exits during
an Emergency
Illness, Injury, or Other Abnormal
Situations Involving Passengers or
Crew (use of medical kit)
Flight Physiology (i.e., Hypoxia,
Respiration, etc.)
Use of Checklist (SOP)
Cockpit Familiarization
Preflight Planning and FMS
In-flight Planning: LNAV, VNAV,
RNAV and GPS
Required Navigation Procedures
Navigation Systems Integration
Autoflight and Flight Director
Integration
Use of Radar/CRTs
TCAS/ACAS
GPWS/TAWS
Communication Systems Integration
(ACARS/FMS/CPDLC)
Surface Movement Guidance Systems
(SMGS)
Runway Incursion Prevention
Strategy
Stabilized Approach Strategy
Precision Radar Monitoring (PRM)
Procedures
Land and Hold Short (LAHSO)
Procedures / Simultaneous
Intersecting Runway operations
(SIRO)
CAT II/III

164
Qualification Standards (QS), Phase II. The qualification standards document has four
parts: The prologue, a regulatory comparison, the evaluation/remediation methodology, and
the specific terminal proficiency objectives (TPO)/supporting proficiency objectives (SPO)
qualification standard.

1. Prologue: This is an introductory section that explains the methodology, format, and
terminology of the document.

2. Regulatory Requirements Comparison: The qualification standards document must

also include regulatory comparison information. The comparison must meet the
requirement of AQP regulatory guidance, which states that the AQP program must
indicate specifically the requirements of CARS Parts IV, VI and VII, as applicable,
that would be replaced by an AQP curriculum. The comparison should be
comprehensive and understandable so that a reader can discern the scope and
appropriateness of the training.

3. Testing/Validation/Evaluation & Remediation Methodology: This section is a detailed

plan describing the point in the curriculum when a test, validation, or evaluation will
be applied. It must identify what constitutes a failure and/or unsatisfactory
performance. This section will also describe the remediation strategy to be used to
correct unsatisfactory performance and special tracking provisions.

4. The Qualification Standard: The qualification standard is constructed by applying a

performance statement, conditions, and standards to a task or subtask, thereby creating
a TPO or an SPO.

1 Qualification Standards Prologue Y N Comments

a. Does the prologue discuss the
methodology used to develop the
qualification standards document?
b. Does the methodology explain how
any aspect, from curriculum outline to
lesson elements or grade sheet items, is
to be traceable to an item in the
qualification standard?
c. Does the prologue discuss the format
(structure) that is used for the
qualification standards?
d. Does the prologue define terms used
for the qualification standards?

165
2 Regulatory Comparison Y N Comments
a. Does the regulatory comparison
specify the requirements of the
applicable flight check standards and
CARS Parts IV, VI and VII that would
be replaced by an AQP curriculum and
how they will be addressed?
b. Are departures from those
requirements identified and justified?
c. Are any standards specifications
used that differ from flight check
standards?
3 Testing/Validation/Evaluation & Y N Comments
Remediation Methodology
a. Does the applicant describe where to
ascertain how, when, where, and who
will assess a student’s proficiency on
each terminal and supporting
objective?
b. Does this section identify the points
in the curriculum where the testing,
validation, or evaluation will be
applied?
• Systems Knowledge Validation
• Procedures Validation
• Manoeuvres Validation
• LOE
• OE
c. Does the applicant clearly define the
different strategies that will be used to
test, validate, or evaluate performance?
• First Look
• Train To Proficiency
• Systems Knowledge Validation
• Procedures Validation
• Manoeuvres Validation
• LOE
• Initial OE
• OE
d. Does this section describe how the
criticality and currency ratings translate
into testing strategies for TPOs and
SPOs in the continuing qualification
cycle?

166
e. Does this section describe how a
TPO with several SPOs may be
alternatively sampled over multiple
evaluation periods or continuing
qualification cycles? e.g., TPO (non-
Precision Approaches) and
corresponding SPOs (NDB, VOR, BC,
etc.)
f. Does the applicant specify and
clearly describe the rating scale that
will be used by instructors/evaluators to
score performance?
g. Do the rating scale definitions
clearly discriminate performance
levels? Are they clear?
h. Does the applicant identify what
constitutes a failure and/or
unsatisfactory performance for each
validation/evaluation point?
i. Does the applicant specify the
strategy for remedying unsatisfactory
performance?
j. Does this remediation strategy detail
when and what may be repeated and
whether or not additional training is
warranted?
k. Does the applicant describe the
methodology that will be used to
remediate unsuccessful testing,
validation, or evaluation sessions?
l. Does the applicant describe the level
of training devices, simulators, or
aircraft to be used to evaluate the
proficiency objective at each point in
the curriculum?
m. Does the remediation strategy
specify when no more training will be
offered to the individual and the
resulting actions such as “Referred to
Director of Training,” “returned to
previous position,” etc.?
n. Does the remediation strategy
describe the criteria for placing an
individual on special tracking?

167
 o. Does the remediation strategy
describe the strategy that will be used
for an individual on special tracking?
p. Does the remediation strategy
describe what must take place for an
individual to be removed from special
tracking?
4 Qualification Standards Y N Comments
a. Does each individual qualification
standard contain the following:
• A header identifying the air
operator and the document?
• Page revision control dates and
revision numbers?
• Consecutive page numbers?
• Phase of Operations: Number
and title from task listing?
• Qualification Standard Title:
Either TPOs or SPOs?
• Task or Subtask: Number and
title from task listing?
• Crew duty positions?
• Criticality/Currency Rating:
From the task factors analysis?
• Curriculum: This field
identifies the curriculum(s) in
which the task will be trained
and evaluated.
• Evaluation Strategy: The
evaluation point for this
particular qualification
standard: e.g., train to
proficiency, procedures
validation, manoeuvres
validation, LOE, or OE?
• Media: The specific media in
which training and/or
evaluation will be conducted?
(For qualification, the media is
the lowest media used for final
evaluation. For continuing
qualification, the media is the
range of media used for
training.)

168
 • Performance Statement: An
expanded statement of expected
behaviour, which, when
executed, will complete the
work required for a specific
portion of a job?
• Conditions operational and
environmental? Are the
specific conditions to be used
for the qualification curriculum
specified?
• Contingencies: Are the specific
contingencies to be used for the
qualification curriculum
specified?
• Manoeuvre Standards: Are they
specific and do they correspond
to the standards listed in the
flight check standards?
• Procedural Standards: They
may be specific or general. If
they are specific, do they
correspond to the standards
listed in the manuals listed in
the reference block? If general,
do they reference information in
a document or manual to
chapter or section (page
numbers are not required)?
• References: Do they identify
the primary references from
which performance statements
and associated standards were
derived? Do they cite
documents by title and where
applicable, chapter or section
(page numbers are not
required)?
b. Are there any operations
specifications requirements other than
those listed above?

169
The Instructional Systems Development Methodology. This document describes the
approach to be used by applicant air operators to develop and maintain all Advanced
Qualification Program (AQP) curricula. This document is divided into two sections. The first
section, Development Procedures, describes the applicant’s approach for using the Job Task
Analyses and Qualification Standards as baseline documents to construct their general training
curricula across all AQP courses. The second section, Line Operational Simulation
Methodology, describes the approach for developing line operational simulation (LOS)
scenarios.

1 Development Procedures Y N Comments

a. Is the procedure for allocating TPOs
and SPOs to lessons, selecting media
and methods, and developing the
curricula described?
b. Does the applicant describe how
enabling objectives (EO) are developed
to support their higher-level objectives?
c. Does the applicant describe how
learning and evaluation activities are
developed to support these objectives?
d. Does the applicant describe how
instructional media and methods are
assigned to objectives?
e. Does the applicant describe how
objectives are clustered and sequenced
into lessons, modules, segments, and
curricula?
f. Does the applicant describe how an
audit trail will be maintained to link
proficiency objectives, lesson
activities/content, and test items?
2 Line Operational Simulation (LOS) Y N Comments
Methodology
a. Does the applicant describe how the
typical scenario is constructed?
b. Does the applicant describe how each
event set relates to a phase of
operation?
c. Does the applicant describe how each
event set consists of a series of
proficiency objectives that include both
technical and CRM activities?

170
d. Does the applicant describe the use
of event conditions, triggers, and
distracters, as well as supporting
events?
e. Does the applicant identify possible
sources of incidents that will elicit the
behaviour required by the proficiency
objectives selected for the scenario?
f. Does the applicant define the basic
success criteria for the LOS, and each
event set within it?
g. Does the applicant describe the
scenario development process?
• Drafting - who will do the
work?
• Use of grade sheets?
• Testing - who will be involved?
• Training instructors or
evaluators to administer specific
LOS scenarios?

171
The curriculum outline. This is a listing of course content. It should be arranged from
curriculum into segments, segments into modules, modules into lessons, and lessons into
elements. Each part of the curriculum outline must clearly indicate the subject matter to be
taught and correspond directly to the hierarchical system of the task analysis. A curriculum
outline provides the basis for the curriculum footprint, which is a high level graphical
overview of the curriculum content depicting the training and evaluation activities and the
proposed hours for each day of the curriculum.

1 Curriculum Outline Y N Comments

a. Does the certificate holder have
qualification and continuing
qualification curricula for each duty
position in each make, model, and
series/variant of aircraft?
b. Does the certificate holder have
separate qualification and continuing
qualification curricula for the
instructors and evaluators?
c. Does the certificate holder have any
special curricula (transition, upgrade,
requalification or refresher)?
d. Is each curriculum constructed in the
following order: curriculum, segment,
module, lesson, and lesson element?
e. Does the curriculum outline provide
a level of detail that will allow the AQP
applicant to make changes to the
syllabus without submitting a new
document for each syllabus change?
f. Does the curriculum outline include
the following:
• Operator’s name?
• Type of aircraft?
• Duty position(s)?
• Title of curriculum and/or
curriculum segment?
• A listing of numbered (coded) Numbers (codes) must allow TC to track
objectives organized into objectives back to qualification standards
lessons, modules, and and Job Task Analysis
segments?

172
 • An outline of each training Each module should contain sufficient
module within each curriculum detail to ensure that the main features of
segment? the principal elements or events will be
addressed during instruction.
• The checking and qualification
modules of the qualification
curriculum segment used to
determine successful course
completion, including any
regulatory qualification
requirements for crewmembers
to serve in CARS Part VII
operations (such as initial
operating experience, OE, route
and aerodrome familiarization)?
g. Does the curriculum outline indicate
that it is part of the revision control
system by page format?
h. Do the curriculum outlines provide a
hierarchical link (proficiency
objectives) between the qualification
standards and a curriculum?
i. Does each part of the curriculum
outline clearly indicate the subject
matter to be taught and correspond
directly to the hierarchical numbering
system of the task analysis?
2 Curriculum Footprint Y N Comments
a. Does the curriculum footprint
describe the training and evaluation
activities conducted each day of the
curriculum?
b. Does the curriculum footprint
include the planned hours?

173
Implementation and Operations Plan (I&O). This document is a milestone schedule
detailing the transition to an AQP for flight crewmembers, instructors and evaluators, and a
blueprint describing provisions for maintenance, administration, data management, and
continuing quality control of curricula. The I&O Plan can be sectioned into two parts. The
first part spells out how the operator proposes to implement the AQP. Included in this
proposal is the schedule for the phase III training evaluation to include instructor/evaluator
training and small group tryouts. It should also include provisos for evaluating the
effectiveness of performance measurement tools, and provisions for evaluating facilities,
courseware, and equipment before starting the plans for the small group tryouts. The second
part explains how the certificate holder intends to operate the AQP in phases IV and V.
Included in this section are strategies for maintaining the program, crew pairing policy, first-
look administration, and instructor/evaluator requirements. The operations plan should also
describe in detail the data management plan. This plan includes a statement of understanding
addressing the collection and analysis of performance/proficiency data and a description of the
performance/proficiency database (PPDB), the data management collection process, and the
TC data submission, analysis, and reporting requirements.

1 Implementation – Phase III Y N Comments

a. Does this section include schedules
for the implementation of each of the
AQP curricula?
b. Do the schedules correlate to the
MATS?
c. Do the schedules include dates for
training the instructors/evaluators?
d. Does the instructor/evaluator training
include difference training for those
previously qualified (if applicable)?
e. Does this section include provisions
for evaluating facilities, courseware,
and equipment before starting the phase
III training?
f. Does this section describe a plan for
evaluating and observing the instructors
and evaluators during the phase III
small group tryout?
g. Does the instructor/evaluator training
include provisions for evaluating the
effectiveness of performance
measurements such as the application
of the rating scale, use of grade sheets
and student feedback?

174
 h. Does this section indicate that the
applicant will request no-jeopardy
credits for the small group tryout course
graduates in a separate letter addressed
to TC?
2 Operations Phases IV & V – AQP Y N Comments
Maintenance
a. Does this section describe the
methodology for maintaining control of
the AQP approval documents?
b. Does this methodology include a
procedure for providing document
copies to the POI after receiving the
approval stamp from the POI?
c. Does this section describe the
strategy to be employed for curriculum
maintenance and update?
d. Does the strategy for curriculum
maintenance and update include plans
for acquiring and measuring data for
tracking curricula?
e. Does the strategy for curriculum
maintenance and update identify the
person(s) responsible for making
changes in the AQP?
f. Does this section describe the
strategy for monitoring and responding
to demographic changes?
g. Does this section describe using
training and evaluation feedback to
maintain and improve the AQP?
h. Does this section describe how
student and instructor feedback will be
obtained?
i. Does this section describe any plans
to upgrade training equipment?
3 Operations Phases IV & V – First Y N Comments
Look Manoeuvres Administration
a. Does the applicant define first-look
manoeuvres, their purpose, and the
strategy that will be used to administer
them?
b. Does this strategy indicate who will
administer the first-look manoeuvres
and at what point in the curriculum?

175
 c. Does this strategy state that first-look
will not be briefed prior to the first
execution of these items?
d. Does this section describe how first-
look manoeuvres are selected?
e. Does this section describe how the
first-look manoeuvres would be
updated?
f. Does the applicant describe how first-
look manoeuvres will be analyzed to
determine trends of degraded
proficiency?
4 Operations Phases IV & V – Y N Comments
LOFT/LOE Crew Scheduling and
Pairing Policy
a. Does this section describe the
circumstances that would require a seat
substitute?
b. Does this section describe the
decision rules that would apply in
obtaining a seat substitute?
c. Do the decision rules that would
apply in obtaining a seat substitute
ensure that in all cases, the seat
substitute must be task familiar with the
duty position?
d. Does this section describe at what
point in the curriculum that a seat
substitute would be used?
e. Does the applicant acknowledge that
all occurrences of seat position
substitution including the qualification
of the seat substitute must be reported?
5 Operations Phases IV & V – Y N Comments
Instructor/Evaluator Requirements
a. Does the applicant identify by title
each instructor or evaluator position?
b. Does the applicant describe the job
function(s) that each instructor or
evaluator is authorized to perform?
6 Data Plan Y N Comments

176
a. Does the data plan have a prologue
that establishes the intended purpose and
methods for the collection, management,
analysis, and reporting of AQP
training/evaluation data for each
curriculum?
b. Does the prologue specify how the
data plan will be maintained and
updated?
c. Does the prologue acknowledge the
air operator’s responsibility to collect
and analyze more data than is required
to be submitted to TC in order to
adequately identify performance trends
and make changes to factors that impact
crewmember performance?
d. Does the data plan address the
methods (e.g., grade sheets, computer-
input screens, etc.) used to collect
performance/proficiency data for all
curricula?
e. Does the data plan address data input
quality control, security, and usability?
f. Does the data plan address data
management as the means and strategy
the AQP air operator intends to employ
to store, access, and assimilate the AQP
performance/proficiency data that is
collected?
g. Does the data plan address the type
of software the data management
system employs (e.g., relational
database, spreadsheet etc.), the
organization of the information in the
electronic medium (e.g., database
definition, database table relationships,
spreadsheet description, etc.) and a
description of the user interface to this
data management system?
h. Does the data plan address the type
of analysis it will employ to facilitate
the AQP performance information
needs of the air operator and TC? This
discussion of the data analysis must
address how each type of AQP data will
be analyzed.

177
i. Does the data plan address TC data
submission requirements including
format and frequency?
j. Does the data plan address the type of
data format it will employ for the
reports, (e.g., tabular reports, graphs)?
k. Does the data plan address the
frequency of the reports, both internal
and TC?
l. Does the data plan identify the
internal air operator personnel that will
receive the reports?
m. Does the data plan include copies of
all forms used for data acquisition and
grading?
n. Does the data plan include a database
description of data field types and a
graphical depiction of the database
table relationships?
o. Does the data plan address a quality
assurance strategy for ensuring data
integrity? Does this strategy include
instructor and evaluator grading
calibration?

178
Annual AQP Report – Phases IV & V. AQP requires that each AQP certificate holder
prepare an annual AQP report. This report is based on the certificate holder’s analysis of the
data that is collected during training and at strategic points (validation/evaluation gates) in
each curriculum and maintained in the performance/proficiency database (PPDB). AQP
requires data collection and analysis in order to establish and maintain quality control of
curricula for flight crewmembers, instructors, and evaluators. The annual AQP report should
summarize the lessons learned and adjustments made to the curriculum(s) during the reporting
period. The report should also include projected or proposed changes to the curriculum(s)
based on the certificate holder’s current analysis. The actual adjustments made to the AQP are
reflected in revisions to the approved AQP documents. The report should be submitted to TC
no later than 60 days past the end of the report period. The reporting period is usually based
on the authorization date for a particular curriculum in either phase IV or V. During AQP
development, particularly for multiple fleet operators, with different authorization dates for
multiple curricula, the reporting period may be modifiable as agreed upon by TC and the
certificate holder. Copies of the report should be distributed the principal operations inspector
(POI) at least 2 weeks prior to the annual AQP review meeting.

Annual AQP Report Y N Comments

a. Is there a prologue or introduction to
the report that addresses how each type
of AQP data was analyzed? This
prologue should corroborate the
information in the I&O Plan.
b. Does the report discuss data
reliability and consistency?
c. Does the report summarize the
internal quality assurance reports as
addressed in the I&O Plan?
d. Does the report validate the
effectiveness of the AQP with
supporting evidence of successful
training and evaluation?
e. Is the report supported by TC
analysis of the submitted data?

179
f. Does the report identify any trends,
problem areas, and potential
deficiencies that could result in
decreased proficiency?
g. Does the report include a description
of corrective measures taken and any
resulting changes to curricula?
h. Does the report include any projected
corrective measures to be taken and
provide rationale for these changes?
i. Does the report indicate a need for
changes to the AQP maintenance
strategy as described in the I&O Plan?
j. Does the report indicate a need for
changes to the certificate holders data
plan as described in the I&O Plan?
k. Does the report provide an analytical
comparison of data between equivalent
periods in preceding years?
l. Does the report identify any future
operational changes that will affect the
AQP (operational changes or trainee
demographics)?
m. Does the report analyze training and
evaluation feedback as part of the
collected data to determine the
effectiveness of the training program?
n. Does the report analyze on-time
completion rates for training curricula
and initial operating experience?
o. Does the report analyze special
tracking rates?
p. Does the report analyze
instructor/evaluator (I/E) rater
reliability training results?
q. Does the report analyze instructor
comments as part of the collected data
to determine the effectiveness of the
training program?
r. Does the report analyze first look
data?
s. Does the report analyze manoeuvres
data?
t. Does the report analyze LOE data by
technical topics, and CRM elements?

180
u. Does the report analyze OE data
(exclusive of initial OE)?
v. Does the report address progress
towards phases III, IV, & V in other
fleets (as applicable)?
w. Does the report address seat
substitution rates?
x. Does the report address
recordkeeping?
y. Does the report address adherence to
the I&O Plan?
z. Does the report address the validity
and usefulness of the qualification
standards?
aa. Does the report address internal
audit or TC surveillance findings?
bb. Does the report address related
safety programs implemented by the air
operator (i.e., flight data monitoring /
operational safety auditing)?

181