Guidelines For OSCEs
Guidelines For OSCEs
Guidelines For OSCEs
Table of Contents
Preface ....................................................................................................................... 3 Glossary ..................................................................................................................... 3 Assessment Meeting .................................................................................................. 4 Room Bookings .......................................................................................................... 4 Recruit Item Writers.................................................................................................... 4 Submission of Items ................................................................................................... 5 OSCE Matrix for Students, Role Players & Examiners............................................... 5 Recruiting Role Players & Examiners......................................................................... 5 Video in MyLO ............................................................................................................ 6 Examiner & Role Player Training Meeting.................................................................. 6 Amendments to Questions ......................................................................................... 6 Catering...................................................................................................................... 6 Exam Information to Students .................................................................................... 7 Station Requirements ................................................................................................. 7 Final Meeting prior to OSCE ...................................................................................... 7 Finalised OSCEs in KB .............................................................................................. 7 Sign Off on Exam ....................................................................................................... 7 OSCE Day Requirements .......................................................................................... 8 Collation of Exam Papers ........................................................................................... 8 Entry of Students Results .......................................................................................... 8 Sign off on Exam Results ........................................................................................... 8 Examiner and Role Player Feedback Forms .............................................................. 8 Post Exam Analysis & Report .................................................................................... 9 Accountability Matrix for OSCEs .............................................................................. 10 Timelines .................................................................................................................. 11 2011 Unit, Rotation Coordinators and Relevant Staff ............................................... 12 Appendices .............................................................................................................. 13 Appendix 1 OSCE Sample .................................................................................. 13 Appendix 2 Sample OMR OSCE Sheet ............................................................. 23 Appendix 3 Sample Assessment Feedback to Students .................................... 25
Preface
This is the first edition of the Guidelines for Objective Structured Clinical Examinations (OSCEs) primarily written as a guide to provide important information to academic and professional staff of the Tasmanian School of Medicine (TSoM) in relation to the procedures required for organising OSCEs in the School of Medicine. It is also hoped that the manual will be useful to all staff members regarding expectations and responsibilities. The manual will be revised and updated as regularly as new changes come into effect. Comments on how to improve the usefulness of the manual are welcome and will be appreciated in revising subsequent editions. I would like to acknowledge the valuable contribution of Ms Leslee Wells in the development of the manual.
Bunmi Malau-Aduli, PhD Lecturer in Medical Education (Assessment), Medical Education Unit, School of Medicine University of Tasmania, February 2011 Glossary
MBBS MEU PDA EL IDEAL KB EOS DO Supp MyLO SOMO HCS LCS RCS Bachelor of Medicine, Bachelor of Surgery Medical Education Unit Program Delivery & Assessment Team External Liaison Team International Database for Enhanced Assessment & Learning Knowledge Base End of Semester Deferred Ordinary Exam Supplementary Exam My Learning On-Line School of Medicine shared folders Hobart Clinical School Launceston Clinical School Rural Clinical School
Assessment Meeting
Years 1-3: Before the beginning of the new academic year, the chair of the MBBS Years 1-3 Assessment Working Party will call for a meeting between the Unit Coordinators, OSCE Coordinators (Years 1-2), Rotation Coordinators (Year 3), MEU Assessment Coordinator, the PDA and EL Team Leaders. EL will book a room for this meeting. At this meeting OSCE blueprints and exam protocols (e.g. number of stations, competencies to be assessed, and potential examiners) will be confirmed. The person in charge of the OSCE will be determined at this meeting and will become the main point of contact for any OSCE decisions for the semester. Years 4-5: Before the beginning of the new academic year, the chair of the MBBS Years 4-5 Assessment Working Party will call for a meeting between the Associate Heads of the three Clinical Schools, MEU Assessment Coordinator, the PDA and EL Team Leaders. EL will book a room for this meeting. At this meeting OSCE blueprints and exam protocols (e.g. number of stations and competencies to be assessed) will be confirmed. Item writing workshop dates will be determined at this meeting.
Room Bookings
Years 1-3: EL will book rooms for training/review workshops and venue/rooms for OSCE day. Years 4-5: MEU will liaise with relevant professional staff from each clinical school to organise venue and invitations to prospective participants.
Submission of Items
Years 1-3: OSCE/Rotation Coordinator/MEU will ensure that all items are received from writers at the required times. Items are checked for consistency by MEU and formatted by PDA (if required). MEU will upload first draft of OSCEs in KB. Please note: The University regulations specify that exam questions should NOT be transmitted via emails. OSCE/Rotation coordinators will be given access to KB. Other item writers can drop off their questions in person to the OSCE/Rotation coordinators or the MEU. Years 4-5: Associate Heads will upload items that they have developed into KB. Items developed at item writing workshops will be formatted and uploaded into KB by MEU.
Video in MyLO
Years 1-5: MEU will liaise with Ms Jo Hanuszewicz, and clinical tutors regarding dates for video recording. MEU, in collaboration with OSCE/Rotation Coordinator and EL, will organise video recording of scenarios with role players and clinical tutors (who will role play examiner and student). MEU will upload videos into the OSCE Examiner Training site in MyLO and notify examiners.
Amendments to Questions
Years 1-3: Amendments will be made by item writers/examiners (if required) and edited by MEU. Years 4-5: Associate Heads meet regularly to review and edit assessment items in KB.
Catering
In Hobart, EL will organise catering for OSCE day. In Launceston and Burnie, relevant professional staff will organise catering.
Station Requirements
Station requirements (in Appendix 1) are sourced for, by PDA (Hobart) and relevant professional staff (Launceston and Burnie).
Finalised OSCEs in KB
Years 1-5: MEU will generate identification numbers for OSCE items in the IDEAL database and upload them into KB.
TASK
Assessment Team Meeting Assessment timelines Initial contacting of question writers MEU provides templates and orientation to resources Follow-up with question writers First draft of questions prepared Format questions First check of questions Draft questions in KB Recruit Role Players Recruit Examiners Recruit QA Examiner Exam Matrix Organise video recordings & upload into MyLO 1st Examiner & Role Player Training Meeting Casual Authority etc for examiners & role players Amendments to questions Format final exam questions Organise Catering Advise all of exam dates and times Final Examiner and role player meeting prior to OSCE Finalised questions in KB Signoff on Exam OSCE Day coordination Results collation Entry & double checking of students results Signoff on exam results Post OSCE analysis and report File papers
3 1 3 1 1 1 1
1 1
1 3 3 3 3 1
1 3 3 1 1 1
2 3
3 1 1 1
1 1 2 3 2 1 1 1 2 3 1
2 1 2 1 2 2 1
10
Timelines
Task Assessment Meeting Room Bookings Recruit Item Writers Submission of Items Timeline January - February January - February Mid February March - April Semester One July - August Semester Two See Assessment Date Schedule for exact dates 6 8 weeks prior to exam day 6 weeks prior 4 5 weeks prior Training 3 weeks prior 4 weeks prior 1 week prior 1 week prior 1 week prior 1 week prior 1 week prior OSCE/unit 1 week prior 1 - 3 days prior 2 3 days after 2 4 days after 2 4 days after 1 2 weeks after
OSCE Matrix Recruiting Role Players & Examiners Video in MyLO Examiner & Role Player Meeting Amendments to Questions Catering Exam Information to Students Station Requirements Final Meeting prior to OSCE Finalised Questions in KB Sign off on coordinator Exam by
OSCE Day Requirements Collation of Exam Papers Entry of Students Results Collation of Feedback Forms Post OSCE Analysis & Report
11
Staff Member
Dwight Assenheimer Christine Clifford Derek Choi-Lundberg John Beadle
Role
Unit Coordinator OSCE Coordinator Unit Coordinator & Chair of Years 1-3 Assessment Working Party OSCE Coordinator Unit Coordinator OSCE Coordinator OSCE Coordinator Unit Coordinator OSCE Coordinator OSCE Coordinator CAM304 Unit Coordinator CAM305 Unit Coordinator Primary Care Coordinator Clinical Specialties Coordinator Surgery Coordinator Medicine Coordinator Associate Head HCS Chair of Assessment Committee (4/5) Associate Head LCS Deputy Associate Head LCS Associate Head RCS Deputy Associate Head RCS Theme 1 Coordinators Theme 2 Coordinator Theme 3 Coordinators Theme 4 Coordinators Theme 5 Coordinator Director, Medical Education Unit Assessment Coordinator - MEU Team Leader, PDA Team Leader, EL
CAM201
Anne-Marie Williams Judi Errey Helen Lord Lisa Foa Judi Errey Helen Lord Matthew Kirkcaldie Richard Turner Emma Warnecke Christine Clifford Mary Self Kwang Yee Janet Vial Richard Turner Kym Rooney Jan Radford Peter Arvier Lizzi Shires Justin Walls, Ros Malley Christine Clifford Jan Radford, Alison Miles Michael Ashby, Bruce Wilson Lindsay Edwards Craig Zimitat Bunmi Malau-Aduli Lisa Ames Anne Barker
CAM202
CAM304/CAM305 (Year 3)
12
Appendices
Appendix 1 OSCE Sample
IDEAL Item ID: MBBS Year Level: 3 Length of Station: 8 minutes Station Writer: Richard Turner Discipline/Topic: Surgery, Medicine - Diabetic Foot Overview of station (incl aim): Daphne McDonald, aged 62, recently moved from interstate, has been diagnosed with Type II diabetes and needs to be examined. Year objective/s being assessed: This station aims to assess the students ability to perform a focused physical examination of the feet of a newly diagnosed diabetic patient. They require integrative reasoning to decide what relevant systems must be examined in this clinical context.
13
14
Student Information:
Scenario:
Daphne McDonald, aged 62, has recently moved from interstate. As her new GP, you have run a battery of screening tests, and have diagnosed her with Type 2 diabetes mellitus. She was previously unaware of this diagnosis, and considered herself to be generally quite healthy. You have given her some advice about weight loss and diet, and will be commencing her on an oral hypoglycaemic agent. As part of the further screening process, you also wish to examine her for any possible diabetic complications. This includes an examination of her lower limbs and feet. A full history is NOT required, but questions relevant to the proposed examination may be asked. Tasks: Perform a physical examination of the feet and lower limbs, based on what complications might be expected to arise from previously undiagnosed diabetes. Explain what you are doing to the patient as you go. Summarise your findings for the patient at the end of the examination. Give the patient some advice about diabetic foot care if any time remains.
15
Examiner Information
This station aims to assess the students ability to perform a focused physical examination of the feet of a newly diagnosed diabetic patient. They require integrative reasoning to decide what relevant systems must be examined in this clinical context.
16
MARKING GUIDE:
Please give a detailed description of the tasks to be performed. Please describe student performance at expected and above expected standards. This document will be provided to the examiner as additional information to refer to during the exam.
Comments on the presence / absence of most of the following: ulcers, gangrene, calluses, muscle wasting, trophic changes, deformity.
Important
Essential
Feels for temperature changes or difference on both feet / legs. Feels for pulses on both feet / lower limbs: dorsalis pedis, posterior tibial, popliteal, femoral. Comments on strength or quality of pulses. Asks if any areas are tender. Tests for light touch (protective sensation), pain, proprioception.
Comments on the presence / absence of all of the following: ulcers, gangrene, calluses, muscle wasting, trophic changes, deformity. Describes the above in a systematic fashion. Also checks for capillary return. Palpates correct site for all pulses. Examines in a systematic and efficient sequence.
Desirable
Tests for lower limb muscle tone and power. Tests for knee, ankle and plantar reflexes.
Systematically examines for light touch (protective sensation), pain, proprioception Also tests (or offers to test) for vibration and temperature Works from distal to proximal Also comments on the quality of lower limb muscle tone and power. Good technique for reflex testing using reinforcement if necessary. Also explains rationale for the examination.
Essential
17
18
Marking Sheet
Core Mark:
Marking Criteria Not Competent Minimally Competent Fully Competent
Examination Skills (ES) 1. General inspection of feet and legs 2. Vascular examination of feet and legs 3. Sensory neurological examination of feet 4. Motor examination of feet and legs Patient Centred Care (PCC 2) 5. Summarisation and explanation
Out of 13 ___________
0 0 0 0 0
Global Score:
Key: 0 Critical error made 1 Very poor performance 2 Well short of expected standard 3 Short of expected standard 4 Expected standard 5 Better than expected 6 Much better than expected 7 Exceptional performance
Out of 7 ___________
......................... .........................
19
Station Requirements
Additional Items Cotton wool / cotton buds Monofilament skin sensation tester Thin skin lancets / needles Container for sharps disposal Tuning fork Chair Examiner Folder Normal Reading Pen, Paper, Folder, Chair Examiner Folder Quantity 1 per student 1 per station 1 per student 1 per station 1 per station 4 1 for each normal examiner 1 1 for external Examiner (only 1 of these) Available from /provided by
Patient Details
Real Patient / Simulated Patient / Role Player: Role Player Approximate Age: 50 70 years Gender: Female Patient name: Daphne McDonald Presenting complaint: Newly diagnosed Type II diabetes mellitus
20
21
Competency
Knowledge of Basic Sciences History Taking Skills Examination Skills Management Plan
Description
Ability to demonstrate understanding of underlying basic sciences concepts in relation to the situation of a patient. Ability to take an appropriate systematic history matched to the situation of a patient. Ability to take an appropriate systematic examination matched to the situation of a patient. Ability to synthesise a plan based on the differential diagnosis/problem list to safely and appropriately address the situation - includes physical treatments, medications, surgery or other specific interventions in a holistic manner (physical, emotional, social, cultural, spiritual, family concerns considered) and with respect to any specific ethical or legal aspects. Ability to collate data from history and examination, using evidence based medicine, knowledge of the disease , to synthesize differential diagnoses or a problem list for complex problems. Synthesis of results of investigations (if appropriate). Ability to concisely and accurately record notes, compile drug charts, write referral letters and medico legal documents. Ability to succinctly and coherently present a case to a colleague (senior), using appropriate medical terminology and in a systematic manner. Ability to set up and perform tasks appropriate for the job (e.g. functional assessment, suturing, ECG, IV cannulation, venepuncture, urinalysis, etc.). Ability to develop and maintain a therapeutic relationship with patients and families Ability to develop and maintain a therapeutic relationship with patients and families - ongoing consideration of patient care, excellent education and counselling of patients, families, and colleagues. Ability to introduce, use non-verbal cues during and successfully conclude a consultation in an appropriate manner, demonstrates excellent relationship building through listening, narrative and nonverbal skills. Ability to use information obtained in collating clinical data to develop an appropriate list of investigations (if needed). Appreciation of the necessity to follow-up any result ordered. Understands complex issues, interrelates patient problems, thorough understanding of differential approach. Ability to document key information, with focused, comprehensive reporting approach. Ability to recognise and explain the ethical dimensions of medical practice and understands the legal demands on the medical profession. Ability to recognise the holistic nature of professional commitment, and appreciate the changing nature of medical professionalism. Ability to demonstrate an awareness of the principles of self-care and the importance of personal development for medical professionals. Ability to demonstrate procedures related to delivering a particular aspect of clinical care based on current research and best clinical practice.
Year
Years 1-5 Years 1-5 Years 1-5 Years 3-5
DS
Years 2-5
RK CP
PS
Procedural Skills
Years 2-5
PCC 1 PCC 2
CS
Communication Skills
Years 1-5
IP
Investigation Plan
Years 3-5
IV OCS
EL
Years 2-5
Professionalism
Years 2-5
SC
Self-Care
Years 2-5
SOP
Scope of Practice
Years 3-5
22
23
24
25
26