Lex Rock Heemstra
Human Factors Specialist,
Independent Human Factors
Evaluating Pilots – A Flaw in Judgement
When pilots conduct their annual medical checks regarding their medical status they expect an objective and rational evaluation. The evaluation is conducted by a medical doctor with at least 7 years of academic study, plus x amount of experience and is based on objective measurements regarding blood pressure, blood count, physical condition and any potential or existing medical ailments. The inability to meet the required objective standards could result in a loss of license.
Annually, pilots are also subjected to two assessments a year based on their performance and behaviour during a Pilot Proficiency Check (PPC) in the simulator. In this case they also expect a realistic and objective assessment of their competencies regarding their performance and behaviour. Their inability to display the required standards could also lead to their loss of license.
Daniel Kahneman, in his latest book: “Noise – A Flaw in Human Judgement” – 2021, discusses the case of a study by Judge Marvin Frankel (1973) where two individuals without a criminal record, where found to be guilty of cashing fraudulent checks in the amount of US$58.40 and US$35.20, respectively. The first man was sentenced to 15 years and the second 30 days, by two different judges. Pilots are very aware of this and know that their evaluation is very much based on who the examiner is. In the worst case scenario, they have even call in sick to avoid the “hanging judge”.
This presentation, based on 12 years of developing standardisation workshops for Instructors and Examiners in two major airlines regarding bias in judgement and noise in the methodology of assessment, will discuss the training than can be conducted to reduce the bias of examiners and adjust the noise in the evaluation tools to ensure that pilots receive a more objective evaluation of their performance and behaviour
Pilot Conference
Day 2 – Wednesday 9th November 2022
Session 5 – 0900-1030
