An Approach to Automatic Flight Deviation Detection

Aircraft pilots are constantly undergoing situations where they must process significant amounts of data in very small periods of time, which may lead to mistakes. We have created a deviation detection system that is capable of auditing the cockpit in real time to detect actions that have been incorrectly added, omitted, or done out of sequence. This model assesses deviations based on hierarchical task data found in the Ontological Reference Model for Piloting Procedures, which contains knowledge-based reference procedures assembled by experts in the domain. Pilot actions are compared to ontology reference sequences using the Needleman-Wunsch algorithm for global alignment, as well as a Siamese LSTM network. An API that can be expanded to several Aerospace Simulators, as well as a Runner, was created to enable the Deviation Framework to connect to the XPlane simulator for real-time system monitoring. Synthetically created data containing sequence mutations were analyzed for testing. The results show that this framework is capable of detecting added, omitted, and out of sequence errors. Furthermore, the capabilities of Siamese networks are leveraged to understand the relation of certain sequence chain anomalies so that they can correctly be ignored (such as certain tasks that can be performed out of order from the reference sequence). These deviation assessments are capable of being run Real-Time (10 Hz) and have been clocked at 0.0179 s for one Takeoff sequence containing 23 actions. The evaluation results suggest that the proposed approach could be applied in aviation settings to help catch errors before harm is done.