Open AccessReview

A Survey on Artificial Intelligence (AI) and eXplainable AI in Air Traffic Management: Current Trends and Development with Future Research Trajectory

Augustin Degas

^1,*

Mir Riyanul Islam

^2,*

Christophe Hurter

Shaibal Barua

Hamidur Rahman

Minesh Poudel

¹,

Daniele Ruscio

Mobyen Uddin Ahmed

Shahina Begum

Md Aquif Rahman

²,

Stefano Bonelli

Giulia Cartocci

⁴

Gianluca Di Flumeri

⁴

Gianluca Borghini

⁴

Fabio Babiloni

⁴

and

Pietro Aricó

⁴

Ecole Nationale de l’aviation Civile, 7 Avenue Edouard Belin, CS 54005, CEDEX 4, 31055 Toulouse, France

Artificial Intelligence and Intelligent Systems Research Group, School of Innovation, Design and Engineering, Mälardalen University, Högskoleplan 1, 722 20 Västerås, Sweden

Deep Blue s.r.l., Via Manin 53, 00185 Rome, Italy

⁴

Department of Molecular Medicine, Sapienza University of Rome, Piazzale Aldo Moro 5, 00185 Rome, Italy

Authors to whom correspondence should be addressed.

Appl. Sci. 2022, 12(3), 1295; https://doi.org/10.3390/app12031295

Submission received: 9 December 2021 / Revised: 12 January 2022 / Accepted: 17 January 2022 / Published: 26 January 2022

(This article belongs to the Special Issue Explainable Artificial Intelligence (XAI))

Download

Browse Figures

Graphical abstract
"> Figure 1
Sector division of the upper airspace of France [<a href="#B4-applsci-12-01295" class="html-bibr">4</a>]. "> Figure 2
Excerpt of a map of the route network in the south of France. Routes are defined by waypoints—black and grey dots. Source: EUROCONTROL. "> Figure 3
Separation zone of an aircraft. From Degas et al. [<a href="#B7-applsci-12-01295" class="html-bibr">7</a>]. "> Figure 4
PRISMA flow of the review made on International Conference on Research in Air Transportation (ICRAT), Air Traffic Management Research and Development Seminar (ATM seminar), Transportation Research Part C: Emerging Technologies (TR_C), Journal of Air Transport Management (JATM), and IEEE Transactions on Intelligent Transportation Systems (IEEE Trans. on ITS). "> Figure 5
Publication distribution on the different object feature from <a href="#applsci-12-01295-t003" class="html-table">Table 3</a>. "> Figure 6
Tree map of the distribution of the articles in function of on the secondary extracted features from <a href="#applsci-12-01295-t004" class="html-table">Table 4</a>. Each rectangle represent a leaf of the table—the lowest level of description between object, complement, and sub-complement—, with the number of article focusing on this feature. "> Figure 7
Distribution of the publications per year in function of the Source Type. "> Figure 8
Evolution of the number of publication per year of the Prediction, Optimisation/Automation, Modelling/Simulation, and Analysis categories. "> Figure 9
Distribution of the publications in function of the categories. "> Figure 10
Synthesis of EXplainable Artificial Intelligence (XAI) conceptual framework. "> Figure 11
Conflict scenario involving two Aircrafts, <math display="inline"><semantics> <msub> <mi>A</mi> <mn>1</mn> </msub> </semantics></math> and <math display="inline"><semantics> <msub> <mi>A</mi> <mn>2</mn> </msub> </semantics></math>. ">

Versions Notes

Abstract

Air Traffic Management (ATM) will be more complex in the coming decades due to the growth and increased complexity of aviation and has to be improved in order to maintain aviation safety. It is agreed that without significant improvement in this domain, the safety objectives defined by international organisations cannot be achieved and a risk of more incidents/accidents is envisaged. Nowadays, computer science plays a major role in data management and decisions made in ATM. Nonetheless, despite this, Artificial Intelligence (AI), which is one of the most researched topics in computer science, has not quite reached end users in ATM domain. In this paper, we analyse the state of the art with regards to usefulness of AI within aviation/ATM domain. It includes research work of the last decade of AI in ATM, the extraction of relevant trends and features, and the extraction of representative dimensions. We analysed how the general and ATM eXplainable Artificial Intelligence (XAI) works, analysing where and why XAI is needed, how it is currently provided, and the limitations, then synthesise the findings into a conceptual framework, named the DPP (Descriptive, Predictive, Prescriptive) model, and provide an example of its application in a scenario in 2030. It concludes that AI systems within ATM need further research for their acceptance by end-users. The development of appropriate XAI methods including the validation by appropriate authorities and end-users are key issues that needs to be addressed.

Keywords:

Air Traffic Management (ATM); Artificial Intelligence (AI); eXplainable Artificial Intelligence (XAI); user-centric XAI (UCXAI)

Graphical Abstract

1. Introduction

1.1. Air Traffic Management

Air Traffic Management (ATM) is a vast and complex domain [1] encompassing all activities carried out to ensure the safety and fluidity of air traffic. In a nutshell, ATM aims at efficiently managing and maximising the use of the different resources available to it—e.g., the airspace and its subdivisions such as the sectors (see Figure 1), the air routes (see Figure 2), the airport, the runways—by the users of the resources—e.g., aircrafts, airlines—, in any time-frame of their use of the resources—i.e., in the taxi phase in the airport, or any flight phase simplified by the triplet climb, cruise, descent—while ensuring flight safety [2]. This task is achieved by managing resources, i.e., AirSpace Management (ASM), managing the global demand before and while aircrafts are flying, i.e., Air Traffic Flow and Capacity Management (ATFCM), and managing—avoiding separation losses between aircrafts, i.e., avoiding one to be in the separation zone of another (see Figure 3)—and providing local information to the flying aircraft, i.e., Air Traffic Control (ATC) [3].

Nowadays, computer science plays a major role in data management and decisions made in ATM, and even if humans remain as the main agents, computer science remains important, and is more likely to have a more relevant part in the future with increasing air traffic—notwithstanding actual COVID situation [5]—and its complexity—notably with the insertion of new aerial vehicles such as drones, e-VTOL into the airspace [6].

Artificial Intelligence (AI), being one of the most researched topic in computer science, should be part of the picture.

Figure 3. Separation zone of an aircraft. From Degas et al. [7].

1.2. AI and XAI for ATM

‘Artificial Intelligence’ term was first used in 1956 for the first “Dartmouth Summer Research Project on Artificial Intelligence”, and is generally refers to any machine that exhibits traits associated with a human mind, such as learning and problem-solving. Since then, the discipline has known several ‘summer’ with important interest, and ‘winter’, disinterest from the field, associated with scepticism [8]. In particular, AI in general has experienced a new bloom during the 2010s, boosted by the increasing access to massive volumes of data, and the discovery of the very high efficiency of computer graphics card processors to accelerate the calculation of learning algorithms [9]. This bloom has been materialised by some significant public successes and has boosted funding, such as Watson—IBM’s IA—winning the television game show Jeopardy against two of its champions [10], Google X being able to have an AI recognise cats on videos [11], or later in the decade, AlphaGo—and its successor AlphaGo Zero—beating one of the world players of Go [12]. EXplainable Artificial Intelligence (XAI), methods and techniques enabling humans to understand (i) the AI algorithm (i.e., global explanation or interpretability), or (ii) its solutions(i.e., local explanation or justification), being strongly linked to the systems it explains, followed the same tendency, and is actually in its third generation—according to Muller et al. [13]. Artificial Intelligence in Air Traffic Management roughly followed the same tendencies, with some delay. As the following shows, AI in ATM roughly evolved from AI systems used to optimise the traffic, to AI systems to predict various objects—like predicting 4D trajectories—during the last decade.

Despite historical research work in AI for ATM, a researcher facing a problem in the ATM domain—to our knowledge—will find no general guide presenting how to resolve this problem (or similar ones), nor the limitation of current work, which is detrimental to the domain and its evolution. Some review exist in the domain, but they are specialised into a category of AI algorithms—e.g., meta-heuristics [14,15], multi-agent systems [16]—, focused on other aspects, e.g., communications [17], or are outdated [18], and focus more on the techniques than the integration for the end users.

Unfortunately, despite several research work already carried in AI for ATM domain, it has not been ‘fully operational’ nor has it brought any benefits to end users. Slow progress within the use of AI in the ATM domain is explainable by the fact that the ATM domain is a critical domain with life at stake, and that safety is the top most priority. Historically, safety has been achieved in ATM with human-in-the-loop—in particular but not restricted to, Air Traffic Controller (ATCO)—, and will most likely, as contend by the authors, evolve by designing tightly human-centered systems, requiring those systems to be understandable by the end-user, and to adapt to its characteristics—mental and physical—and to its psychological state. For example, if the operator’s workload is exceeds their cognitive capacity, or if some kind of incapacitation is occurring, their cognitive state could be automatically detected by the system, and used by this assessment to execute actions autonomously along an escalating scale of automation (i.e., adaptive automation) [19,20]. In other domains such as healthcare and criminal justice, among others, the increasing interest in AI to support high-consequence human decisions has spurred the field of XAI and User-Centric eXplainable Artificial Intelligence (UCXAI) [21]—User Centered Design (UCD) refers to the methods employed when designing systems for end-users to validate novel algorithms/working methods/interaction techniques [22,23,24,25]. This primordial aspect is yet to be fully assessed in ATM, but the interest is growing [26,27].

Based on the two previous observations, the goal of this article is to depict the trends of AI and XAI, and set the trajectory that these works must take in order to reach end-users. Our main research questions stem from the two previous observations:

RQ1: What are the current trends of AI and XAI in ATM tasks?
RQ2: What are the limitation that arise from the use of AI and XAI in ATM tasks?
RQ3: How could the general XAI field benefit AI and XAI in ATM?
RQ4: What limitation may arise from the use of general XAI in ATM?
RQ5: What should the trajectory of AI and XAI be for this domain?

To answer these questions, this article is divided into two parts: (i) The first part of this article is dedicated to the review of research work of the last decade of AI in ATM, presenting the methodology employed (Section 2), the extraction of relevant trends and features, the clustering of these work into representative groups (Section 3), and the extraction of representative dimensions, allowing us to create a design space representing those works, used then to analyse the publications (Section 4); (ii) the second part of this article is based on the dimensions extracted in the first part, to analyse general and ATM XAI work, analysing where and why XAI is needed, how it is currently provided, and the limitations (Section 5 and Section 6), then synthesise the findings into a conceptual framework (Section 7), that is then applied to different scenarios (Section 8). Finally, we conclude the different findings of this article (Section 9).

2. Paper Selection

This section provides details for the procedure involved in the selection, inclusion, and exclusion of research articles. The review was conducted using the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines [28]. Overall, the review was performed using different well-ranked conferences and journals, judge representative of the domain, namely, Transportation Research Part C: Emerging Technologies (TR_C) and IEEE Transactions on Intelligent Transportation Systems (IEEE Trans. on ITS) (top two journal on Transportation according to Google Scholar metrics [29]), Journal of Air Transport Management (JATM) (the first ATM journal [29]), and International Conference on Research in Air Transportation (ICRAT) and Air Traffic Management Research and Development Seminar (ATM seminar) (the two ATM conferences supported by EUROCONTROL and the Federal Aviation Administration). Figure 4 describes the complete pipeline of the paper selection.

2.1. Identification

The focus of this research is on English published articles from 2010 until the end of December 2021. The year 2010 has been determined as a starting point of the searching process so as to be able to determine trends and evolution of the vast field from the beginning of the new bloom of AI interest—see Section 1.2—, and be sure to fully capture the essence of the research space. Apart from this, the inclusion and exclusion criteria of this review are shown in Table 1.

To be firstly selected (Identification), articles of the different journals and conferences required: (i) To be in the ATM domain, a characteristic that is either obtained by the database used—i.e., ATM specific conferences or journals, or using keywords to filter, in detail, “air traffic” OR “airplane” OR “aircraft”, named in the following “ATM_Filter”; and (ii) to work with the AI algorithm, characteristic that was filtered using general regular expression “Predict*” OR “Estimat*” OR “Optimi*” OR “Cluster*” OR “Analy*” OR “Visu*” OR “Learn*” OR “Explain*” OR “Model*” OR “Plan*” OR “Conflict” OR “Classif*”. The number of articles identified per source are represented in Figure 4, and details about the keyword filters catches are represented in Table 2. The keywords used were first refined in a preliminary study performed to plan a systematic review, using different keyword-extraction techniques, ATM domain insights from interviews performed, and general author domain knowledge. In a nutshell, these keywords represent main tasks for which AI can be employed in an ATM, along with some specific AI keywords, such as the theory or framework employed—e.g., neural network, genetic algorithm.

2.2. Screening

The review process used conferences and journal databases, duplicates came only from external databases from previous work, and forward backward reference searching, hence the low number of exclusion for duplication (29 excluded).

Selected papers were then manually screened for relevance, based on a first superficial reading using an empirical keyword list—containing mostly primary keywords and methods employed—resulting in the exclusion of 383 papers.

2.3. Eligibility

The remaining 366 articles were selected for a full-text review and content analysis. For inclusion in the final list, articles must be related with ATM—excluding example articles where drones are used for surveillance—, and AI or XAI. These inclusion criteria resulted in 257 relevant articles. In regard to exclusion criteria, the articles about passenger experience and security were not included in this review, as this was not the core focus of this study. Furthermore, only the articles written in English were considered in this review.

2.4. Inclusion

Inclusion rate was highly different depending on the sources of the papers. On one side, papers coming from ATM domain conferences—ICRAT and ATM seminar—only needed to be consistent with the AI field, hence the low exclusion rate at the identification phase—285 out of 383 + 343 = 726, roughly 39 percent, see Figure 4.

On the other side, papers coming from more general journals—TR_C and IEEE Trans. on ITS—needed to be consistent with both the AI and ATM domain, hence the high exclusion rate at the identification phase—(n = 7186), around 98.5 percent. In between, papers coming from the Journal of Air Transport Management targeting ATM and other Aeronautical issues, the exclusion rate was medium-high—(n = 870), 70 percent.

Past this phase, exclusions were mostly in the screening and in the eligibility phases, due to the fact that AI techniques were not used: In the screening phase—n = 383, roughly 51.1 percent—clearly defined in the title/abstract the use of techniques not from the AI field; and in the eligibility phase—n = 109, around 29.8 percent did not meet the criteria. As a result, the overall exclusion rate is a bit low compared to some systematic review, but still high in general—around 97.2 percent.

3. Paper Clustering

This section provides details about data extraction from the publications, the different statistics on the extracted data, and the clustering of the publications into representative groups.

3.1. Data Extraction

The aim of this section is to create an information extraction form to derive out the accurate data from the selected articles. In this step, the relevant data were derived from selected articles through the use of spreadsheets and reference management software.

The following primary features were extracted in this systematic review: Author(s), Publication, Year, Title of the Study, Source Type, Theory or Framework, Objective, and Factors. The description of these items is presented in Table 3.

The extracted factors were then refined in a preliminary study performed to plan the systematic review—and of course adapted if needed in the rest of the study—, using different keyword extraction techniques, ATM domain insights from interviews performed, and general author domain knowledge. The refined extracted factors are presented in Table 4.

3.2. First Data Clustering on Additional Extracted Data

The clustering of the paper was performed in two steps. The first step was performed in the preliminary study, and resulted in the additional extracted feature presented in Table 4.

In detail, while extracting the different features of Table 3, it seemed promising, at first, to categorise the publications by the specific part of the ATM world every publication was benefiting to. However, although interesting, the feature was not selective enough to assess any trends or categories in the different works. Nonetheless, it seemed that the main Objective from Table 3, was a promising feature to fully categorise the Design Space of AI in ATM. This clustering was performed by refining the Objective feature by extracting the subject, the time-frame, and any complementary information pieces about the subject from the feature. For example the following Objective “Predict the future location of a general aviation aircraft” from [30], can be cut into “Predict” “the future” “location of a general aircraft”. After this quite simple phase, the different extracted data—i.e., subject, time-frame, complementary information pieces—have analysed and clustered into more general categories. In a nutshell, the categories are regrouped into “Object”, subdivided into “Complement”, and, if any, subdivided into “Sub-Complement”. All Object categories except Time-Frame are named in the following “Material Object” and represent the general subject of the objective, namely, “Aircraft”, “Traffic”, “Airport/Controlled Traffic Region (CTR)”, “Airspace”, “ATCO”, and “Pilot” (the distribution of the publications in function of the object feature is represented in Figure 5). These categories have been created as: (i) They represented a distinct part for practitioners; and (ii) the publications across those categories appeared to have a partial distinction of Objective and employed algorithms and methodologies.

Most publications targeting the ATCO or the Pilot focused on predicting or analysing their behaviour and their decisions—the command for ATCO and the flight decisions for the Pilot—, or analysing their audio transmissions. Publications targeting those two categories are less represented than the others (see Figure 6), this most likely comes from the global interest of this category, and possibly from the databases used that are not focusing on this area.

The Airspace category regroups publications that target: (i) The analysis or the prediction of the “State” of the airspace; (ii) optimisation of the airspace “Structure” in function of different criteria; or (iii) the optimisation of the capacity/demand balance.

The Airport/CTR Traffic category regroups any publication about the Airport, that either: (i) Predict its state (e.g., opened runway, or their configuration); (ii) model, optimise, automate, or predict the “Ground Traffic”, i.e., aircraft(s) taxing; and (iii) model, optimise, or automate the CTR traffic, i.e., Aircraft taking off or landing. While both the Airport and CTR Traffic could be separated, lots of publication are mixing the two traffic, most of the time to optimise the area.

The Aircraft category regroups any publication focusing only on one flying Aircraft, with the publications focused on taxiing aircraft being already regrouped in the Airport/CTR Traffic category. This category regroup publication focusing on: (i) Predicting the state of an Aircraft (e.g., its bearing); and (ii) modelling, predicting, analysing, or optimising its Trajectory. Sub-categories have been created in the Trajectory category, namely, Indicators, Route/Flight Plan, 4D Trajectory, as the intent behind the work—especially for indicators where its is mostly to foresee the said indicator—, or the employed algorithms and methodology change between them.

The Traffic category regroup any publication focusing only on a set of more than one flying Aircraft, any publication focused on a set of more than one aircraft on the ground being already regrouped in the Airport/CTR Traffic category. These publications either focus on: (i) Predicting indicators of the traffic; and (ii) analysing, predicting, modelling, automating, or optimising all traffic.

Within the Aircraft, Traffic, and Airport/CTR categories, two type of clusters have been created regarding publications working on (i) Indicators, measures on intangible object (like a trajectory), such as indicators of the complexity of the traffic, or on (ii) a State, measures on tangible object (like an aircraft), such as the mass of an Aircraft.

The distribution of the article in function on the secondary extracted features from Table 4 is represented in detail in Figure 6.

3.3. Second Data Clustering on All Extracted Data

The second clustering of the publications have been performed from an analysis on all the extracted features—listed in Table 3 and Table 4—and it allowed the extraction of additional knowledge on the general AI in ATM work.

This clustering resulted on the following four categories tightly connected with AI in general, that globally define the purpose of the application:

Prediction.
Optimisation/Automation.
Analysis.
Modelling/Simulation.

This last clustering was performed using the previously extracted feature. Among the different extracted features, the Time Frame feature—Table 3—and Objective features— Table 4—proved to be very valuable while categorising the different work—Table A1 in Appendix A details these two features per article. Based on these two features, the different group could be clustered into 3 categories, further refined into 4.

The papers from the first and second categories, namely Prediction and Optimisation/Automation categories, contain paper from the “Pre-Flying” or the “Flying” parts of the Time Frame feature—based on the actual state of the considered “Material Object”.

The

Prediction category contains paper seeking to foresee the future behaviour of a “Material Object”, answering “what if” questions about premises—e.g., “what will be the trajectory of this aircraft, performed by this company, between this city pair”. As an example, this category contains papers about the prediction of traffic or prediction of the runway occupancy time. This category is about foreseeing an event, not to be confused with the “prediction” of an AI algorithms on a subject, such as its “prediction” of the labels of an image.

The

Optimisation/Automation category contains papers seeking to enhance the behaviour of a “Material Object” according to different criteria, the most represented being “avoiding separation losses between aircraft”, and “optimising the traffic”. As an example, this category contains papers about sequencing arrival aircraft, avoiding conflict, or optimising a trajectory.

The

Analysis category, contains papers focusing mainly on the “Post-Analysis” part and on the “Flying” part of the Time Frame feature. These papers are seeking to understand the observed behaviour of a “Material Object”—see Table 4—answering “why” and “why not” questions about the facts—i.e., what happened—and/or the foil—what is expected or plausible to happen. As an example, this category contains papers about assessing the workload of an Air Traffic Controller (ATCO) in a sector, or evaluating the important factors influencing the arrival of an aircraft.

The

Modelling/ Simulation category contains papers that did not apply to any part of the Time Frame feature. Those papers are modelling the behaviour of “Material Object” in order to simulate it, which in the long run could lead to answer “why”, “what if”, “why not”, and “how to” questions depending on the model constructed. As an example, this category contains paper about simulating the air traffic of an airspace, or modelling the arrival of aircraft.

4. Design Space: Trends of AI in ATM

This section analyses the trends of AI trough the lens of the previously presented Design Space—composed of four categories: Applsci 12 01295 i001

Prediction, Applsci 12 01295 i002

Optimisation/Automation, Applsci 12 01295 i003

Analysis, and Applsci 12 01295 i004

Modelling/Simulation—, first from a general point of view, and then from the category point of view.

4.1. General Insights

From a global point of view, AI in ATM is a growing domain: The number of publications of AI in ATM—as shown in Figure 7 and Figure 8—has almost doubled between the 2014 and 2018 pikes, and has more than tripled since 2010 The total distribution of the publication in function of the categories can be visualised in Figure 9.

Through the lens of our categorisation, the growth of the last four years of publications originate from the growing work in the Applsci 12 01295 i001

Prediction category—publications about Prediction tripled between 2013 and 2020—and the Applsci 12 01295 i002

Optimisation category—doubled between 2013 and 2020. It seems that AI in ATM—and the number of publications—highly benefited from AI community work—researcher work or developer community, e.g., scikit [31], tensorflow [32], keras [33]—of the last decade, that democratised and made AI model generation far more accessible, and in some areas, more effective, in particular in prediction and optimisation. The time window of the review does not allow any previous trend in AI for ATM before 2010. However, previous work and other reviews [14] suggest that publications of AI in ATM has grown in the last decade but had a strong core base for many years, in particular in Applsci 12 01295 i002

Optimisation—collision avoidance and traffic flows being arguably one of the most consistent subjects of AI in ATM.

The following section analyses the trends of AI through the lens of the Design Space from the category point of view.

4.2. Categorisation Insights

The primary Objective of the model already categorise successfully the different AI models used in ATM. The different models used in the different categories are instead presented in the following, using selected references—please refer to Table 4 for the description of the descriptors.

AI Prediction in ATM is performed using a vast range of AI models, with most used utilised being: (i) Multi-Agent Systems (MAS); (ii) Neural Network (NN); (iii) Random Forest (RF); (iv) Gradient Boosting Machine (GBM); (v) Support Vector Machine (SVM); and (vi) Linear Regression. The five later models—NN, RF, GBM, SVM, and linear regression—are mostly used to predict: (a) A descriptor of the State or an Indicator of the Trajectory of an aircraft, e.g., mass estimation [34], descent length [35], phase of flight [36]; and (b) a Ground Traffic Indicator or a State descriptor of an Airport, e.g., the estimated take-off time [37], or taxi speed [38]. Authors using these models are mostly capitalising on framework availability of the past years and are often used jointly for comparison, with the linear regression often used as a baseline. Nonetheless, (ii)–(v) AI models have also been used for other type of predictions such as Route choice [39], the Structure of a Sector configuration [40], the Environmental State of the Airspace, ATCO action prediction [41], or—short-term—4D Trajectory prediction [42,43]. Multi-Agent Systems on their side have been used to model and predict more complex tasks, like Indicators of the Traffic such as delay propagation on networks [44], 4D Trajectory, and to a certain extent, 5D Traffic prediction [45] and CTR Traffic—CTR Traffic being easier to predict due to the important amount of constraints.

AI Optimisation and Automation works use mostly a more restricted range of AI models: (i) Multi-Agent Systems (MAS); (ii) Evolutionary Algorithm (EA), mostly genetic algorithms; (iii) Simulated Annealing (SA); and (iv) Reinforcement Learning (RL) (e.g., RL, multi-agent RL, Deep RL, and DQN, see Table 5). A majority of these works focus on optimising the traffic and/or avoiding collisions, from the point of view of the trajectories. 5D Traffic Optimisation works focus in general on one flight phase, such as optimising (a) En-Route traffic, using centralised, i.e., SA [46] or EA [47], or decentralised, i.e., MAS [48,49]; (b) arrival traffic [50]; (c) departure traffic [51]; (d) Ground Traffic (“5D” Traffic) Optimisation—although this traffic is not really using altitude, it is still noted 5D Traffic in the categorisation to avoid confusion between traffic and trajectory—; or (e) the whole CTR Traffic [52]. Notable other focus of AI model for optimisation are optimising Airspace Structure, such as Route network [53], Sectors [54], and Optimising a 4D Trajectory [55], optimising the Demand/Capacity Balance, or the Route of an Aircraft.

Analysis of ATM activities using an AI model is mostly composed of: (i) Techniques that clusterise—e.g., DBSCAN, BIRCH, or auto-encoder NN—4D Trajectories in order to analyse the different factor influencing the 4D Trajectory, the Route choice [56], the CTR Traffic, in particular the arrivals [57], but also Traffic Indicators such as delays [58], in order to understand the different influencing factors and/or as a first analysis to latter use in a predictive model; or (ii) more precise analysis, such as trajectory analysis to detect ATCOs action [59], or speech recognition and analysis or utterance of ATCOs [60] or a Pilot.

AI Modelling in ATM is mostly performed using multi-agent systems, which is not a surprise considering their importance in simulation in many domains, such as the simulation of car traffic. Nonetheless, AI modelling is not that prevalent in ATM compared to other domains since in ATM, modelling and simulation are mostly made using records, and mathematical models [61]—e.g., BADA [62], ASTOR. Multi-Agent modelling is quite broad, going from modelling CTR Traffic, notably arrivals, to assess risks in TMA [63], simulate network delay [44], simulation of 5D Traffic [64], or simulate all ATM environments [65]. Other AI models found focused on more simple tasks, such as modelling a particular pilot decision [66] with NN. The latest work on a graph neural network [67] could possibly become important in modelling in the future, such as modelling actual ATCOs [68].

Table 5. Publication distribution with the design space.

AI/ML Models	Prediction	Optimisation/Automation	Analysis	Modelling/Simulation
Multi-Agent System (MAS)	[44,45,64,69,70,71,72,73]	[7,49,53,61,74,75,76,77,78,79]		[44,45,61,63,64,65,70,71,73,80,81,82,83,84,85,86,87,88]
Evolutionary Algorithm (EA) (e.g., Genetic Algorithm (GA), Differential Evolution (DE))	[89]	[54,90,91,92,93,94,95,96,97,98,99,100]	[101]	[102,103]
Simulated Annealing (SA)		[46,50,51,52,55,97,104,105,106,107,108,109,110]
Particle Swarm Optimisation		[98,111]
Tabu Search		[112,113,114]
Ant Colony Algorithm		[115]		[116]
Bee Colony Algorithm		[117]
Neural Network (NN)	[34,35,36,37,40,66,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134]	[130,135]	[57,59,60,66,136,137]	[66]
Random Forest (RF)	[38,39,41,42,129,130,131,138,139,140,141,142,143,144,145,146,147,148,149,150,151,152]		[137,149]
Support Vector Machine (SVM)	[36,39,89,142,145,150,153,154,155]	[135,156]
Quantile Regression Forest	[139,140,157]
Gradient Boosting Machine (GBM)	[27,35,37,39,123,131,138,141,142,143,148,149,158,159,160,161,162]
Decision Tree	[138,143,145,149,150,163]		[137]	[164]
Linear Regression	[35,36,39,56,119,125,130,145,155,163,165,166,167,168,169]	[135,170]	[56,137,171,172,173,174,175,176,177,178]	[166]
Linear Functional Regression	[168,179,180]
Deep Deterministic Policy Gradient (DDPG)		[79,181,182]
Convolutional Neural Networks (CNN)	[183,184,185]		[186]
Deep Q-Network (DQN)		[78,187]
Binary Logistic Regression Models			[188]
Dynamic Bayesian Belief Network	[189]			[189]
Linear Discriminant Analysis (LDA)	[123]
Quadratic Discriminant Analysis (QDA)	[123]
Gaussian Mixture Model (GMM)	[153,190,191,192,193,194,195]		[196,197,198]
Autoencoder	[199]
Fuzzy Logic	[200]	[117,201,202]	[203,204,205,206,207]
Logistic Regression	[131,150]		[126,208]
Bayesian Network	[30,209,210,211]			[212]
Recursive Bayesian estimation	[213]	[214]
Hierarchical clustering				[215]
k-nearest neighbours (kNN)	[130,216]
BIRCH			[200]
OPTIC			[58,217]
DBSCAN	[127,151,195,218]	[219]	[200,220,221,222,223,224]
K-means	[153,225,226,227]		[146,224,228,229]	[230,231]
Principal Component Analysis (PCA)	[127]		[178]
Non Negative Matrix Factorisation (NMF)		[232]
A* & Derivated Methods		[92,108,233,234,235]
Reinforcement Learning	[236,237]	[238,239]		[84]
Deep Reinforcement Learning		[68,78,240,241,242]		[68]
Recurrent Neural Network (RNN) & Long Short Term Memory (LSTM)	[43,152,156,184,185,243,244,245,246,247,248,249,250,251]		[186]	[252]
Multi-Layer Perceptron (MLP)	[125,163,251]
Graph-Theoretic Clustering	[253]
Hidden Markov Model (HMM)				[254]
Markov Decision Process (MDP)		[78,235,240,255]
Not referenced	[256,257]	[258]	[259]	[260]

4.3. Validating the Design Space

While the Design Space presented previously seems to seize the global structure of the articles of AI in ATM from our data set, one may wonder if:

(i): This Design Space is covering all the AI in ATM work.
(ii): This Design Space contains overlap.
(iii): This Design Space is perennial and how would it evolve otherwise.

As the Design Space was generated from an analysis of a representative data set of articles, the Design Space should cover all the AI in ATM work of the last decade. Furthermore, as the Design Space focuses more on the abstract task which make, in the end, quite general—i.e., predict the future state of an “object” Applsci 12 01295 i001

, understand the actual state of an “object” Applsci 12 01295 i003

, automate or optimise Applsci 12 01295 i002

the behaviour or the state of an “object”, and simulate Applsci 12 01295 i004

an “object”.

In terms of overlapping, some boundaries between the categories might seem blurry, but they are in fact quite distinct. The Optimisation/Automation Applsci 12 01295 i002

category is quite distinct from the other 3 categories, and this distinction is also noticeable from the different algorithms used—in particular the use of Meta-Heuristics. The boundary between the Prediction Applsci 12 01295 i001

category with the Modelling/Simulation Applsci 12 01295 i004

category might be seen as blurry as the historic of a realistic simulation—e.g., the trajectory performed by the aircraft during the simulation—can be used as a prediction. Nonetheless, the distinction comes from the fact that the historic is only a side-effect of the technique, and not its main purpose, which is to simulate an “object” in a virtual environment. Similarly, the borders of the Analysis Applsci 12 01295 i003

category with the Modelling/Simulation Applsci 12 01295 i004

category could seem permeable, as a simulation can help to understand a past situation(e.g., understanding a collision alert afterward), but this would be the result of the analysis of an end-user. The analysis would result easier for the end-user with AI, but it would still be made by the human and not by the system. Finally, the Analysis Applsci 12 01295 i003

algorithms are often used as an input for Prediction Applsci 12 01295 i001

algorithms, but both fulfil very distinct ATM tasks.

Looking into the future and the sustainability of the Design Space, as the category focus on abstract tasks on “object”, the Design Space should be valid for at least the next decade. The “object” might change, e.g., the aircraft feature from Table 4 might be divided into different new sub features but the abstract task will not remain. Nonetheless, new categories could appear, and sub-categories could be created.

5. Analysing General XAI

The concept of XAI is comparatively contemporary to other concepts of AI and the term “explainability” is deliberately used in various research domains including AI. Thus, it is expedient to develop a general understanding of the term explainability from the perspective of XAI. The prime hindrance towards developing the ground knowledge of explainability concerning AI, is the interchangeable use of several terms in the literature, such as: Interpretability, transparency, explainability, etc. Before proceeding to the literature review, the commonly used terms are presented briefly according to the definitions compiled by Barredo Arrieta et al. [261].

“Understandability”, often termed as “Intelligibility”, is the characteristics of a model that helps a user realise its functions. In other words, how the model works without any requirement of further explanation for the model’s internal operations on the data. Another similar term is “Comprehensibility”, which has been used to define the ability of an ML model to represent its learned knowledge to humans in an understandable way. Clearly the prior terms differ from the second on representing the internal operations on the data and the knowledge acquired from the data. In addition, the terms “Interpretability” and “Transparency” are mostly used in describing similar concepts to explainability and they refer to a model’s ability to provide meaning or explain in an understandable way to human beings. Nonetheless, model transparency also indicates the ability to be understandable to humans. There are three types of transparent models [262]:

Simulatable models have the capacity to make humans understand their structure and functioning entirely.
Decomposable models can be decomposed into individual components, i.e., input, parameters and output, and their respective intuitions.
Algorithmically Transparent models behave “sensibly” in general with some degree of confidence.

Above all, the term “Explainability” affiliates the interface between humans and decision-makers, which is concurrently comprehensible to humans and an accurate representation of the decision-maker [263]. In XAI, explainability is the interface between the models and the end-users through which an end-user gains clarifications on the decisions from an AI/ML model.

The AI/ML models learn the underlying characteristics of the available data and subsequently try to classify, predict, or cluster new data. The stage of explainability refers to the period in the process mentioned above in which a model generates the explanation for the decision it provides. The stages are found to be ante-hoc and post-hoc [264]. Brief descriptions of the categorised methods based on these stages are:

Ante-hoc methods generally consider generating the explanation for the decision from the very beginning of the training on the data while aiming to achieve the optimal performance. Mostly, explanations are generated using these methods for transparent models, such as, Fuzzy models, Tree-based models, etc.
Post-hoc methods comprise an external or surrogate model and the base model. The base model remain unchanged, and the external model mimics the base model’s behaviour to generate an explanation for the users. Generally, these methods are associated with the models where the inference mechanism remains unknown to users, e.g., Support Vector Machines, Neural Networks, etc. Moreover, the post-hoc methods are again divided into two categories: Model-agnostic and model-specific. The model-agnostic methods are applicable to any AI/ML model, whereas the model-specific methods are confined to particular models.

The scope of explainability defines the extent of an explanation produced by some explainable methods. Vilone and Longo deduced after scanning more than 200 scientific articles published on XAI that the scope of explainability can be either global or local [264]. That is, the whole inferential technique of a model can be made transparent or comprehensible to the user by a full decision tree (global) vs. only a single instance of inference can be explicitly presented to the user (local) e.g., a single branch can be termed as a local explanation.

XAI in Terms of Design Space

The prime reason of adding explanation to the dimensions of the design space defined in Section 4 ( Applsci 12 01295 i001

Prediction, Applsci 12 01295 i002

Optimisation/Automation, Applsci 12 01295 i003

Analysis, and Applsci 12 01295 i004

Modelling/Simulation), is to increase trust of humans on the decision making process of AI/ML models [265] and improve human decisions or predictions based on the models [266]. Moreover, human decisions can be often driven by biases and heuristics that may present some limitations under certain working conditions [267]. However, abundant relevant information also does not necessarily assist people in making proper decisions. In human decision-making for the stated dimensions, the heuristics and biases are mostly controlled by statistical constraints [268]. Regardless of the domain expertise, people generally can not produce fully optimal decisions [269]. On the contrary, XAI can be beneficial to human decision-making by increasing the trust on the automated systems. This can be investigated through human experiments testing AI systems with explanations compared with traditional AI systems alone for different dimensions of the design space.

Literature indicates that mostly four different forms of explanations are generated to “explain” to the human the decisions of the AI/ML models as well as the process of deducing a decision for the tasks from different dimensions of the design space. Those four forms of explanations have the following format: Numeric, rules, textual, and visual explanations. Some researches [270,271,272] investigated the use of a combination of formats to make the explanation more understandable and user friendly. The available methods for adding explainability of these forms to the existing and proposed AI/ML models are clustered on the basis of the categories of tasks; prediction, optimisation/automation, analysis, and modelling/simulation from the publications on AI in ATM. In addition, the available explainable methods are grouped based on the explanation generation scope and stages. The summary of the clustering is represented in Table 6. The bottom row of the table presents the total number of articles that contribute to each of the components of explainability and categories of tasks from the design space. It is evident that most of the methods were deployed to add explainability to prediction tasks whereas the tasks of modelling/simulation were investigated least with a view to add explanations. On the other hand, for specific methods, the Adaptive Neuro-Fuzzy Inference System (ANFIS) [273] was observed to be utilised to generate all four types of explanations in the tasks containing optimisation/automation and analysis. However, Sequential Rule Mining (SRM) [274] algorithms were found to be exploited in all task categories of design space except prediction. Among other methods available for adding explainability to intelligent systems, Local Interpretable Model-Agnostic Explanations (LIME) [270,272,275,276] and Shapley Additive Explanations (SHAP) [21,271,272,277] are worth mentioning due to their wide acceptability among researchers.

6. Analysing XAI in ATM

Globally, the explainability of the AI algorithms used by the different works reviewed is barely addressed.

In our review, the algorithm belonging to the Applsci 12 01295 i001

Prediction category is the only one where explainability is approached clearly even if it is secondary to the papers main goals and it was restricted mainly to predicting an indicator of the trajectory, i.e., the landing time [138] and take-off time [37]. The main goal of Xie et al. [27] was to explain the results of their risk of incidents and accidents predictive model. Although those AI algorithms are already useful to the ATM community, fully understanding the underlining reasons of congestion, trajectory routes, and delay, i.e., answering “why” and “why not” questions, is more than required to better enhance latter traffic, or more simply better predict it. Predicting the traffic and its delays is one key to enhance the general traffic, its congestion, and better balance the demand and resources, so more research should focus on this topic. Additionally, another example of the need for explainability for those algorithms would be to predict the future behaviour of those algorithm with other features, or in a new environment—e.g., transposing the landing time prediction model from one airport to another. Moreover, being able to efficiently and effectively alert the ATM users on the prediction result and explain the reasons behind the alert, has proven to be key for not only building trust in the system. For example, alerting the user that the prediction result might be false (because of the class-imbalanced data set for that class) or alerting a possible lack of feature provided to perform the analysis (e.g., an over simplification of the trajectories), are all “explanations” that need to be communicated in a certain way, during a certain time to the final human operator, otherwise the advantages of automation support would not be considered [327].

Unfortunately—as shown in the previous section—AI algorithms used to Optimise and Automate Applsci 12 01295 i002

are not the main target of the general explainable AI community, and has not been further studied in the ATM field. Nonetheless, optimising and/or automating the general traffic and avoiding collision is one key—arguably the most important—to enhance general traffic and its safety. Due to the upmost importance of safety in ATM, fully understanding the underlining reasons of conflict avoidance procedures(e.g., explaining why one aircraft is moved away from its planned trajectory and not another), sequencing, or any other optimisation result, is more than required to be accepted and used by human operators such as ATCO. Going further, it would be interesting to predict how the AI algorithm would perform in a different environment—e.g., how it would organise the traffic when a part of the airspace changes and becomes not available (e.g., military zones). Furthermore, alerting on a possible bad solution, for example alerting on a possible bias from the training, could also prove to be essential to the acceptance of the algorithm.

In our review, AI algorithms used to Analyse Applsci 12 01295 i003

did not receive any effort to explain the result or the algorithm, despite some algorithms being strongly addressed by the general XAI community—see Section 5. Adding explainability could help the end-user understand the classification performed by the algorithm (e.g., the different factors influencing this analysis), and help one to understand also how this analysis could evolve with new parameters or data, and how to modify the analysed “Material Object” to obtain the desired behaviour. Additionally, it would be interesting to be able to predict the change of the classes if a new feature is added, or predict the evolution of classes with a modification of the problem structure—e.g., predict the evolution of the flows of trajectories with a new route structure.

Similarly, Modelisation and Simulation Applsci 12 01295 i004

AI algorithms were not found to be explained either. In a sense, AI modelling most of the time start from the underlying reasons motivating actions of the different actors taking part in the simulated world. This is particularly true with MAS systems that are dominant in this category, where MAS designers try to represent the different entities in the system, their actions and reasoning, allowing a global state to emerge for agent interactions. Nonetheless, explainability could be added to explain some emerging behaviours instead of the local behaviour(e.g., delay propagation), with a valuable impact to global understanding.

In the current state, the effectiveness and acceptability of AI algorithms in ATM will be limited by the machine’s inability to explain its thoughts and actions to human users in these critical situations, and fully understand the needs and desires of the end-user. Finally, the following section formalises the lack of explainability using a conceptual framework to further detail the need and direction that must be taken in order to tackle the explainability problem in ATM.

7. XAI in ATM Synthesis

The ATM domain being a critical one, with life at stake, the effectiveness of AI systems will be limited by the machine’s inability to explain its thoughts and actions to human users in these critical situations, and fully understand the needs and desire of the end-user. As seen in previous sections, the third wave of XAI is yet to be at a general XAI level. Nonetheless, as contended by the authors, actual general XAI will not be totally sufficient for AI to reach end-users as ATCO.

Indeed, developed XAI systems are more directed to the developer or the debugger than the final end-user [13]. Although some XAI are focused on an explanation that might be presented to a non-developer [270,328], little justification is provided for choosing different explanation types or representations, and it is unclear why these explanations will be feasibly useful to actual users or simply understood [329]. Already existing formal psychological theories that are greatly summarized for XAI in [330,331,332,333], are poorly used to guide explanations facilities, as argued in [13,21]. The last concern is essential to move towards human-centric AI since it is essential to understand how humans think as well as being able to adapt to different ways of thinking. A second or parallel step can be to understand what information they seek and the biases that impair their reasoning, so as to understand what reasoning method triggers actual XAI facilities [334], and how XAI can be leveraged to mitigate decision biases.

Furthermore, explanation is both a product and a process, in particular a social process [335]. XAI systems are required to fully understand the user, which means to adapt to the one that receives the explanation [21,336]. This is crucial to determine the explanation requirements for a given problem, and understand the ‘why’ behind user actions [337]. Furthermore, understanding is required to adapt to its socio-technical environment since the AI user will interact with other humans outside of the 1-1 human-computer interaction, and thus trust should be transitive to them [334]. Lastly, the systems also need to understand the user, which means it has to be able to interact with the user, which here is beneficial in both ways, i.e., human understanding the machine and machine understanding the human. In order to adapt while the XAI systems is in use and not only during the development process to enhance explainability, the XAI systems must be able to adapt to the user and provide information which is not only about the internal state of the AI [338]. The previous paragraphs about general XAI—and thus Section 5—and Section 6, let emerge user-centric XAI requirements for ATM that can be synthesised by the three following XAI:

Descriptive XAI, any XAI that describe an AI algorithm, or its outputs.
Predictive XAI, any XAI that predict the behaviour of an AI algorithm to a certain input or system modification.
Prescriptive XAI, any XAI that detect errors or an unwanted behaviour of an AI algorithm and prescribe a way to overcome it.

Descriptive XAI is particularly covered by actual general XAI state of the art, and is required by the end-user to understand the machine. Predictive XAI is required by the end-user to interact and to ask counterfactual ‘what if’ or ‘why not’ questions to the AI system—must it be on its internal behaviour or its output—, on a more semantically accessible level. This level requires the description level—as shown in Figure 10—to be intelligible by the end user. Finally, the prescriptive XAI is required by the end-user expressing to the machine, allowing ‘how to’ questions to be asked, and overcoming errors and unwanted behaviours. The prescriptive XAI requires the predictive level to analyse different outcomes and prescribe adequate modifications to the AI system, and the descriptive level to be intelligible by the end user.

We illustrate these different types of XAI in the following section on a selected scenario about conflict avoidance.

8. Scenarios XAI in ATM

In this section, we present scenarios and benefits of our XAI framework within aviation concrete examples. It is assumed that safety within the context of civil aviation is taken by default of high importance during our scenario presentation.

Let us assume that the fully functional XAI is deployed in 2030 within the aviation domain and is being used by all stakeholders including an Air Traffic Controller and Management. We assume further that at point C (see Figure 11), aircraft

A_{1}

and

A_{2}

will go below the minimum safety distance in X minutes if no actions are taken by the pilots. ATC gets a notification that actions need to be taken—e.g., safety warning with blinking notifications on the radar screen.

Let us further assume that the controller in charge changed the flight level of the Aircrafts such that this dangerous situation is avoided. However, due to change in the flight level or flight path, Aircraft

A_{1}

will reach the final destination during peak landing hour, which increases the congestion and holding time of several aircraft, which will drastically impact the take-off/landings sequences.

The situation leads to congested airspace inducing stress and a high workload of several stakeholders (e.g., ATC, Pilots, Airline’s ground staff, or other stakeholders), degrading the landing and take-off performance of several aircraft—e.g., delay, cost.

In the above scenario, based on the inputs from all interconnected complex systems, any safety events are identified well in advance (SAFETY PREDICTION) such that all actors—including ATC/pilots—can take appropriate actions.

In 2030, when AI modules are induced within the complex Air Traffic Management System, algorithm transparency and explanation should be able to appropriately provide all stakeholders, including ATC retrieve, the following three major pieces of information:

(1): Descriptive XAI: The system should be able to provide to all users the detailed description and rational of the action to be taken. In the above example, the XAI should be able to provide information on why there is need of change in the flight plan which is due to a possible collision risk. Or it should be able to provide information on potential congestion in airspace, during take-off or landing, which will help to optimise among other things the efficiency of the whole systems and stakeholders in addition to the avoidance of safety catastrophe.
(2): Predictive XAI: In the above example, the XAI should be able to determine the ‘what if’ conditions or in other words, provide information to all stakeholders what will be the consequences of the actions that will be taken. The XAI in the above case should be able to provide information to ATC so that if the ATC performs certain actions to avoid collision, then it will lead to congestion in the airport due to their actions. This will help and support the ATC including other stakeholders understand the consequences of certain actions, ‘what if I perform this action’.
(3): Prescriptive XAI: The induced AI functions will, in addition to the above information, be able to suggest/propose the appropriate actions and options along with an appropriate explanation such that stakeholders can decide on the next course of actions. This next course of action will be based on safety criteria as a main contributor but will also take other appropriate considerations like congestion, weather information, induced workload to ATC, pilots taking into account human factors, cost benefits, and environmental benefits to name the few. In the above scenario, the user can use the XAI prediction to assess the efficiency of potential actions—‘what if’. XAI prescription will provide sufficient information to enable the user with immediate action to perform without testing them. For instance, the ‘what if’ function shows that aircraft A will induce high delay in the landing sequence, and XAI prescription will provide the immediate solution which will address this issue—i.e., providing a different solution for conflict resolution with a change of path rather than a change of altitude.

9. Setting the Trajectory of AI and XAI in ATM

TThis paper presented a systematic review of the literature regarding Artificial Intelligence (AI) and EXplainable Artificial Intelligence (XAI) in Air Traffic Management (ATM). The review found that there is a wide range of ATM tasks that have been considered (Section 2 and Section 3). Some of these applications include, but are not limited to, tasks related to AirSpace Management (ASM) (e.g., optimising the structure of the sectors), tasks related to Air Traffic Flow and Capacity Management (ATFCM) (e.g., predicting the network delay), and tasks related to Air Traffic Controller (ATCO) (e.g., conflict detection and avoidance).

This application space has been clustered by a iterative process into four categories (Section 3 and Section 4), namely (i) the “Prediction” Applsci 12 01295 i001

category, papers seeking to foresee the future behaviour of a “Material Object”, see Table 4, (ii) the ”Optimisation/Automation” Applsci 12 01295 i002

category, papers seeking to enhance the behaviour of a “Material Object”, (iii) the “Analysis” Applsci 12 01295 i003

category, papers seeking to understand the observed behaviour of a “Material Object”, and finally (iv) the “Modelling/Simulation” Applsci 12 01295 i004

category, paper modelling the behaviour of a “Material Object” to simulate it.

These four categories that constitute the author Design Space of AI in ATM have then be used to analyse current AI in ATM work (Section 4). As a summary, the main challenges for the Optimisation/Automation Applsci 12 01295 i002

category is to reach end users that are mostly performing high risk/time pressure tasks (since, most of those works optimise actual traffic). MAS and RL could particularly be of great interest for potential UAV uses. Modelling Applsci 12 01295 i004

could also greatly benefit from using MAS more often in order to enhance the reality of the simulations. As for the Prediction Applsci 12 01295 i001

category, the main opportunity lies in enhancing their models and embracing the complexity of air traffic. Finally, works from the Analysis Applsci 12 01295 i003

category would benefit from more transparency toward its users.

The Design Space has then be used to analyse XAI in general and XAI in ATM (Section 5 and Section 6). It has shown that AI in ATM still needs to apply XAI to its algorithm in order to reach end users, as nearly no article were focusing and explaining their results, and that XAI for ATM should move toward a more user-centric design, where both the AI system and the end-user can understand and interact with each other thanks to this user-centric XAI (UCXAI).

This analysis of XAI in ATM through the lens of the Design Space has presented some common points between the four categories and have then been summarised into a conceptual framework composed of three interconnected XAI levels required in ATM, namely, (i) “Descriptive” XAI, XAI system that describes an AI algorithm, or its outputs, (ii) “Predictive” XAI, that predict the behaviour of an AI algorithm to a certain input or AI system modification, and (iii) “Prescriptive“XAI, that detect errors or an unwanted behaviour of an AI algorithm and prescribes a way to overcome it.

The current investigation highlights that general XAI are focused on the Descriptive level, although some research provides Predictive characteristics—mostly the sensibility of the prediction to some variables. However, more research on XAI for the predictive and prescriptive levels within ATM are required to grab the potential value that XAI in general can bring to the aviation community.

Author Contributions

Conceptualisation, A.D., M.R.I., C.H., S.B. (Shaibal Barua), D.R., M.U.A., S.B. (Shahina Begum), S.B. (Stefano Bonelli), P.A., F.B., G.D.F. and G.B.; methodology, A.D., M.R.I. and M.U.A.; design space modelling, A.D., C.H. and M.U.A.; validation, A.D. and M.R.I.; formal analysis, A.D. and M.R.I.; investigation, A.D., M.R.I., S.B. (Shaibal Barua) and H.R.; data curation, A.D., M.R.I., S.B. (Shaibal Barua) and H.R.; writing—original draft preparation, A.D., M.R.I., C.H. and M.P.; writing—review and editing, A.D., M.R.I., C.H., S.B. (Shaibal Barua), H.R., M.P., D.R., M.U.A., S.B. (Shahina Begum), M.A.R., S.B. (Stefano Bonelli), G.C., P.A., G.D.F. and G.B.; visualisation, A.D. and M.R.I.; supervision, A.D., C.H., S.B. (Shaibal Barua), M.P., D.R., M.U.A., S.B. (Shahina Begum), S.B. (Stefano Bonelli), G.C., P.A. and F.B.; project administration, A.D., C.H., S.B. (Shaibal Barua), D.R., M.U.A., S.B. (Stefano Bonelli), G.C., P.A. and F.B.; funding acquisition, C.H., S.B. (Shaibal Barua), M.U.A., S.B. (Shahina Begum), S.B. (Stefano Bonelli) and F.B. All authors have read and agreed to the published version of the manuscript.

Funding

This work was financed by the European Union’s Horizon 2020 within the framework SESAR 2020 research and innovation program under grant agreement N. 894238, project Transparent Artificial Intelligence and Automation to Air Traffic Management Systems, ARTIMATION.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A. Principal Extracted Features from Articles

We present in this annex the principal extracted features from all publications (i.e., the material object and time frame feature from Table 4), along with their title and reference.

Table A1. Principal extracted features from articles.

Reference	Title	Material Object	Time Frame
[339]	Parameterised framework for the analysis of probabilities of aircraft delay at an airport	Airport, Analysis, Airport State	Post-Analysis
[340]	A survival model for flight delay propagation	Aircraft, Analysis, Aircraft Trajectory, Aircraft Trajectory Indicator	Post-Analysis
[341]	Optimisation of the waiting time and makespan in aircraft departures: A real time non-iterative sequencing model	Optimisation, Airport, Ground Traffic, Ground Traffic 5D Traffic,	Flying
[342]	An empirical analysis of delays in the Turkish Airlines network	Analysis, Traffic, Traffic Indicator	Post-Analysis
[343]	Aircraft line maintenance scheduling and optimisation	Optimisation, Aircraft, Aircraft State	Pre-Flying
[215]	Aircraft grouping based on improved divisive hierarchical clustering algorithm	Analysis, Traffic, 5D Traffic Analysis, Traffic 5D Traffic	Post-Analysis
[344]	A novel heuristic approach for solving aircraft landing problem with single runway	Modelling, CTR, CTR Traffic	Not Referenced
[345]	Analysis of bilateral air passenger flows: A non-parametric multivariate adaptive regression spline approach	Modelling, Airspace, Airspace Structure, Airspace Structure Route	Post-Analysis
[346]	Airport and route classification by modelling flight delay propagation	Analysis, Traffic, 5D Traffic Analysis, Traffic 5D Traffic	Post-Analysis
[131]	Predicting demand for air taxi urban aviation services using machine learning algorithms	Prediction, Airspace, Demand Capacity Balancing	Pre-Flying
[210]	Using causal machine learning for predicting the risk of flight delays in air transportation	Prediction, Traffic, Traffic Indicator	Pre-Flying
[208]	Analysis of airport weather impact on on-time performance of arrival flights for the Brazilian domestic air transportation system	Airport, Analysis, Airport State	Post-Analysis
[347]	Using machine learning algorithms to predict the risk of small Unmanned Aircraft System violations in the National Airspace System	Prediction, CTR, CTR Traffic	Pre-Flying
[130]	Automated data-driven prediction on aircraft Estimated Time of Arrival	Aircraft, Prediction, Aircraft Trajectory, Aircraft Trajectory Indicator	Flying
[205]	Airport selection criteria of low-cost carriers: A fuzzy analytical hierarchy process	Airport, Analysis, Airport State	Post-Analysis
[129]	Assessing strategic flight schedules at an airport using machine learning-based flight delay and cancellation predictions	Prediction, CTR, CTR Traffic	Flying
[348]	Forecasting air passenger demand with a new hybrid ensemble approach	Prediction, Airspace, Demand Capacity Balancing	Pre-Flying
[349]	Towards a maturity model for big data analytics in airline network planning	Analysis, Traffic, 5D Traffic Analysis, Traffic 5D Traffic	Post-Analysis
[350]	Improving aircraft approach operations taking into account noise and fuel consumption	Optimisation, CTR, CTR Traffic	Pre-Flying
[203]	A hybrid approach based on the fuzzy AHP and HFACS framework for identifying and analyzing gross navigation errors during transatlantic flights	Analysis, Traffic, 5D Traffic Analysis, Traffic 5D Traffic	Post-Analysis
[100]	An efficient hybrid approach for resolving the aircraft routing and rescheduling problem	Optimisation, CTR, CTR Traffic	Flying
[204]	A novel hybrid fuzzy DEA-Fuzzy MADM method for airlines safety evaluation	Analysis, Traffic, 5D Traffic Analysis, Traffic 5D Traffic	Post-Analysis
[351]	Collaborative air traffic flow management: Incorporating airline preferences in rerouting decisions	Optimisation, Airspace, Demand Capacity Balancing	Not Referenced
[352]	An intelligent decision making approach for identifying and analysing airport risks	Airport, Analysis, Ground Traffic, Ground Traffic 5D Traffic	Post-Analysis
[206]	Multi-attribute sustainability evaluation of alternative aviation fuels based on fuzzy ANP and fuzzy grey relational analysis	Aircraft, Analysis, Aircraft State	Post-Analysis
[353]	Statistical characterisation of deviations from planned flight trajectories in air traffic management	Analysis, Traffic, 5D Traffic Analysis, Traffic 5D Traffic	Post-Analysis
[176]	Causal Analysis of En Route Flight Inefficiency—the US Experience	Analysis, Airspace, Airspace Structure, Airspace Structure Route	Post-Analysis
[160]	Predicting & Quantifying Risk in Airport Capacity Profile Selection for Air Traffic Management	Prediction, Airport, Airport State	Pre-Flying
[125]	A Novel Machine Learning Model to Predict Abnormal Runway Occupancy Times and Observe Related Precursors	Prediction, Airport, Ground Traffic, Ground Traffic Indicators	Pre-Flying
[212]	A Bayesian Network Model of Pilot Response to TCAS Resolution Advisories	Prediction, Modelling, Pilot	Pre-Flying, Flying
[123]	Learning Air Traffic Controller Workload from Past Sector Operations	Prediction, Air Traffic Controller (ATCo)	Flying
[148]	Predicting Performance of Ground Delay Programs	Prediction, Airport, Ground Traffic, Ground Traffic Indicators	Pre-Flying
[163]	A Comparative Analysis of Models for Predicting Delays in Air Traffic Networks	Prediction, Traffic, Traffic Indicator	Pre-Flying
[147]	Using Machine-Learning to Dynamically Generate Operationally Acceptable Strategic Reroute Options	Aircraft, Prediction, Aircraft Trajectory, Aircraft Trajectory Route	Flying
[149]	Predicting and Analysing US Air Traffic Delays using Passenger-centric Data-sources	Prediction, Analysis, Airspace, Static Structural State, Airspace State	Flying, Post-Analysis
[234]	Optimising Successive Airspace Configurations with a Sequential A∗ Algorithm	Optimisation, Airspace, Airspace Structure, Airspace Structure Sector	Pre-Flying
[78]	Optimising Collision Avoidance in Dense Airspace using Deep Reinforcement Learning	Optimisation, Conflict Avoidance, Traffic, Traffic 5D Traffic, 5D Traffic Optimisation	Pre-Flying
[124]	Cost Reductions Enabled by Machine Learning in ATM	Air Traffic Controller (ATCo), Analysis	Post-Analysis
[182]	A Machine Learning Approach for Conflict Resolution in Dense Traffic Scenarios with Uncertainties	Optimisation, Conflict Avoidance, Traffic, Traffic 5D Traffic, 5D Traffic Optimisation	Pre-Flying
[170]	Regression Analysis of Top of Descent Location for Idle-thrust Descents	Aircraft, Prediction, Aircraft Trajectory, Aircraft Trajectory Indicator	Flying
[178]	Analysis of Airspace Complexity Factors’ Capability to Predict Workload and Safety Levels in the TMA	Prediction, Airspace, Static Structural State, Airspace State	Flying
[164]	Modelling Delay Propagation Trees for Scheduled Flights	Prediction, Modelling, Traffic, Traffic Indicator	Pre-Flying
[159]	Machine Learning Applied to Airspeed Prediction During Climb	Aircraft, Prediction, Aircraft Trajectory, Aircraft Trajectory Indicator Pre-Flying
[258]	Machine Learning of Controller Command Prediction Models from Recorded Radar Data and Controller Speech Utterances	Air Traffic Controller (ATCo), Analysis	Post-Analysis
[56]	Combining Visual Analytics and Machine Learning for Route Choice Prediction Application to Pre-Tactical Traffic Forecast	Aircraft, Prediction, Analysis, Aircraft Trajectory, Aircraft Trajectory Route	Pre-Flying, Post-Analysis
[60]	A context-aware speech recognition and understanding system for air traffic control domain	Air Traffic Controller (ATCo), Analysis	Post-Analysis
[57]	Deep Trajectory Clustering with Autoencoders	Analysis, Traffic, 5D Traffic Analysis, Traffic 5D Traffic	Post-Analysis
[171]	Identification of Significant Impact Factors on Arrival Flight Efficiency within TMA	Analysis, CTR, CTR Traffic	Post-Analysis
[138]	Predicting Aircraft Landing Time in Extended-TMA Using Machine Learning Methods	Aircraft, Prediction, Aircraft Trajectory, Aircraft Trajectory Indicator	Flying
[66]	Analysing Pilot Decision-Making Using Predictive Modelling	Prediction, Modelling, Analysis, Pilot	Not Referenced
[200]	Large-scale flight phase identification from ads-b data using machine learning methods	Aircraft, Prediction, Analysis, Aircraft Trajectory, Aircraft Trajectory Indicator	Pre-Flying, Post-Analysis
[232]	Classification of Air Traffic Controller Utterance Transcripts via Warm-Start Non-Negative Matrix Factorisation	Air Traffic Controller (ATCo), Analysis	Post-Analysis
[59]	Detecting Controllers’ Actions in Past Mode S Data by Autoencoder-Based Anomaly Detection	Air Traffic Controller (ATCo), Analysis	Post-Analysis
[58]	Clustering Trajectories by Relevant Parts for Air Traffic Analysis	Analysis, Traffic, 5D Traffic Analysis, Traffic 5D Traffic	Post-Analysis
[188]	Analysis of Airspace Infringements in European Airspace	Analysis, Airspace, Static Structural State, Airspace State	Post-Analysis
[189]	A Dynamic Bayesian Belief Network Approach for Modelling the ATM Network Delays	Analysis, Traffic, Traffic Indicator	Post-Analysis
[173]	A linear programming approach for capacity estimation and robustness analysis of the European air traffic network	Analysis, Airspace, Static Structural State, Airspace State	Post-Analysis
[174]	A linear programming approach to maximum flow estimation on the European air traffic network	Analysis, Airspace, Static Structural State, Airspace State	Post-Analysis
[175]	Topological properties of the air navigation route system using complex network theory	Analysis, Airspace, Static Structural State, Airspace State	Post-Analysis
[61]	Auto-structuration de trafic temps-réel multi-objectif et multi-critère dans un monde virtuel	Optimisation, Aircraft, Conflict Avoidance, Modelling, Traffic, Aircraft Trajectory, Aircraft 4D Trajectory, Traffic 5D Traffic, 5D Traffic Modelling	Flying, Not Referenced
[45]	Simulation massive de monde virtuel par système multi-agent auto-adaptatif	Aircraft, Prediction, Traffic, Aircraft Trajectory, Aircraft 4D Trajectory, Traffic 5D Traffic, 5D Traffic Prediction	Not Referenced
[64]	Adaptive Air Traffic with Big Data Analysis	Aircraft, Prediction, Traffic, Aircraft Trajectory, Aircraft 4D Trajectory, Traffic 5D Traffic, 5D Traffic Prediction	Not Referenced
[70]	Learning aircraft behaviour from real air traffic	Aircraft, Prediction, Traffic, Aircraft Trajectory, Aircraft 4D Trajectory, Traffic 5D Traffic, 5D Traffic Prediction	Not Referenced
[68]	An Autonomous Free Airspace En-route Controller using Deep Reinforcement Learning Techniques	Optimisation, Modelling, Air Traffic Controller (ATCo)	Flying
[63]	Using agent-based modelling to determine collision risk in complex TMA environments: the turn-onto-ILS-final safety case	Modelling, CTR, CTR Traffic	Not Referenced
[80]	Using Agent-Based Modelling to Determine Collision Risk in Complex TMA Environments	Modelling, CTR, CTR Traffic	Not Referenced
[81]	Bluesky ATC simulator project: an open data and open source approach	Modelling, Traffic, Traffic 5D Traffic, 5D Traffic Modelling	Not Referenced
[82]	An Agent Based Model of the Air Traffic Management	Aircraft, Modelling, Air Traffic Controller (ATCo), Traffic, Airspace	Not Referenced
[65]	Agent-based simulation framework for airport collaborative decision making	Airport, Modelling, Ground Traffic, CTR, CTR Traffic, Ground Traffic 5D Traffic	Not Referenced
[85]	Agent-based safety risk analysis of Trajectory Based Operation in the Terminal Manoeuvring Area	Modelling, CTR, CTR Traffic	Not Referenced
[83]	Risk-capacity tradeoff analysis of an en-route corridor model	Modelling, Airspace, Airspace Structure, Airspace Structure Route	Not Referenced
[91]	Aircraft conflict resolution by genetic algorithm and B-spline approximation	Optimisation, Conflict Avoidance, Traffic, Traffic 5D Traffic, 5D Traffic Optimisation	Pre-Flying
[76]	Decentralised cooperative conflict resolution among multiple autonomous mobile agents	Optimisation, Conflict Avoidance, Traffic, Traffic 5D Traffic, 5D Traffic Optimisation	Pre-Flying
[96]	Hybridisation of interval methods and evolutionary algorithms for solving difficult optimisation problems	Optimisation, Conflict Avoidance, Traffic, Traffic 5D Traffic, 5D Traffic Optimisation	Pre-Flying
[53]	Multi-agent systems to help managing air traffic structure	Optimisation, Conflict Avoidance, Airspace, Airspace Structure, Airspace Structure Route	Flying
[61]	Auto-structuration de trafic temps-réel multi-objectif et multi-critère dans un monde virtuel	Optimisation, Aircraft, Conflict Avoidance, Modelling, Traffic, Aircraft Trajectory, Aircraft 4D Trajectory, Traffic 5D Traffic, 5D Traffic Modelling	Pre-Flying, Flying, Not Referenced
[48]	Dynamic Collision Avoidance using Local Cooperative Airplanes Decisions	Optimisation, Conflict Avoidance, Traffic, Traffic 5D Traffic, 5D Traffic Optimisation	Pre-Flying, Flying
[115]	Ant colony optimisation for air traffic conflict resolution	Optimisation, Conflict Avoidance, Traffic, Traffic 5D Traffic, 5D Traffic Optimisation	Pre-Flying, Flying
[95]	Modélisation mathématique et résolution automatique de conflits par algorithmes génétiques et par optimisation locale continue	Optimisation, Conflict Avoidance, Traffic, Traffic 5D Traffic, 5D Traffic Optimisation	Pre-Flying
[104]	Strategic deconfliction of aircraft trajectories	Optimisation, Conflict Avoidance, Traffic, Traffic 5D Traffic, 5D Traffic Optimisation	Pre-Flying
[74]	Multiagent Reinforcement Learning Methods for Resolving Demand—-Capacity Imbalances	Optimisation, Airspace, Demand Capacity Balancing	Pre-Flying
[93]	Multiobjective Pre-tactical Planning Under Uncertainty for Air Traffic Control	Optimisation, Traffic, Traffic 5D Traffic, 5D Traffic Optimisation	Pre-Flying
[77]	The Effect of Intent on Conflict Detection and Resolution at High Traffic Densities	Optimisation, Conflict Avoidance, Traffic, Traffic 5D Traffic, 5D Traffic Optimisation	Pre-Flying
[68]	An Autonomous Free Airspace En-route Controller using Deep Reinforcement Learning Techniques	Optimisation, Modelling, Air Traffic Controller (ATCo)	Flying
[55]	FPCA applied to flight paths optimisation	Optimisation, Aircraft, Aircraft Trajectory, Aircraft 4D Trajectory	Pre-Flying
[135]	Coupling Mathematical Optimisation and Machine Learning for the Aircraft Landing Problem	Optimisation, CTR, CTR Traffic	Pre-Flying
[181]	Improvement of Conflict Detection and Resolution at High Densities Through Reinforcement Learning	Optimisation, Conflict Avoidance, Traffic, Traffic 5D Traffic, 5D Traffic Optimisation	Flying
[94]	Merging Flows and Optimising Aircraft Scheduling in Terminal Manoeuvring Area Based on GA	Optimisation, CTR, CTR Traffic	Flying
[92]	Aircraft Ground Traffic Optimisation	Optimisation, Airport, Ground Traffic, Ground Traffic 5D Traffic	Pre-Flying
[214]	Airport ground traffic optimisation	Optimisation, Airport, Ground Traffic, Ground Traffic 5D Traffic	Pre-Flying
[54]	A multi-objective evolutionary method for Dynamic Airspace Re-sectorization using sectors clipping and similarities	Optimisation, Airspace, Airspace Structure, Airspace Structure Sector	Pre-Flying
[187]	Autonomous Aircraft Sequencing and Separation with Hierarchical Deep Reinforcement Learning	Optimisation, Conflict Avoidance, CTR, CTR Traffic	Flying
[52]	Integrated Optimisation of Arrival, Departure, and Surface Operations	Optimisation, Airport, Ground Traffic, CTR, CTR Traffic, Ground Traffic 5D Traffic	Pre-Flying
[50]	Merging flows in terminal manoeuvring area using time decomposition approach	Optimisation, CTR, CTR Traffic	Pre-Flying
[49]	Multi-agent Systems for Air Traffic Conflicts Resolution by Local Speed Regulation	Optimisation, Conflict Avoidance, Traffic, Traffic 5D Traffic, 5D Traffic Optimisation	Flying
[46]	Large Scale Adaptive 4D Trajectory Planning	Optimisation, Aircraft, Conflict Avoidance, Aircraft Trajectory, Aircraft 4D Trajectory	Pre-Flying
[90]	A new framework for solving en-routes conflicts	Optimisation, Conflict Avoidance, Traffic, Traffic 5D Traffic, 5D Traffic Optimisation	Pre-Flying
[98]	Benchmarking conflict resolution algorithms	Optimisation, Conflict Avoidance, Traffic, Traffic 5D Traffic, 5D Traffic Optimisation	Pre-Flying
[105]	Trafic aérien: détermination optimale et globale des trajectoires d’avion en présence de vent	Optimisation, Conflict Avoidance, Traffic, Traffic 5D Traffic, 5D Traffic Optimisation	Pre-Flying
[75]	The effect of swarming on a voltage potential-based conflict resolution algorithm	Optimisation, Conflict Avoidance, Traffic, Traffic 5D Traffic, 5D Traffic Optimisation	Pre-Flying
[51]	Optimisation of arrival and departure routes in terminal manoeuvring area	Optimisation, Airspace, Airspace Structure, Airspace Structure Route	Pre-Flying
[97]	The use of meta-heuristics for airport gate assignment	Optimisation, Airport, Ground Traffic, Ground Traffic 5D Traffic	Pre-Flying,
[14]	Metaheuristics for air traffic management	Optimisation, Traffic, Airspace, Airspace Structure, Airspace Structure Route, Traffic 5D Traffic, 5D Traffic Optimisation, Airspace Structure Sector	Pre-Flying, Flying
[145]	Evaluation of algorithms for a miles-in-trail decision support tool	Optimisation, Conflict Avoidance, Traffic, Traffic 5D Traffic, 5D Traffic Optimisation	Pre-Flying
[112]	Optimised Flight Level Allocation at the Continental Scale	Optimisation, Conflict Avoidance, Traffic, Traffic 5D Traffic, 5D Traffic Optimisation	Pre-Flying
[113]	Air transportation network robustness optimisation under limited legs itinerary constraint	Optimisation, Airspace, Airspace Structure, Airspace Structure Route	Pre-Flying
[106]	Optimisation of aircraft trajectories in North Atlantic oceanic airspace	Optimisation, Conflict Avoidance, Traffic, Traffic 5D Traffic, 5D Traffic Optimisation	Pre-Flying
[233]	Iterative planning of airport ground movements	Optimisation, Airport, Ground Traffic, Ground Traffic 5D Traffic	Flying
[7]	Cooperative multi-agent model for collision avoidance applied to air traffic management	Optimisation, Conflict Avoidance, Traffic, Traffic 5D Traffic, 5D Traffic Optimisation	Flying
[45]	Simulation massive de monde virtuel par système multi-agent auto-adaptatif	Aircraft, Prediction, Traffic, Aircraft Trajectory, Aircraft 4D Trajectory, Traffic 5D Traffic, 5D Traffic Prediction	Pre-Flying, Not Referenced
[64]	Adaptive Air Traffic with Big Data Analysis	Aircraft, Prediction, Traffic, Aircraft Trajectory, Aircraft 4D Trajectory, Traffic 5D Traffic, 5D Traffic Prediction	Not Referenced
[70]	Learning aircraft behaviour from real air traffic	Aircraft, Prediction, Traffic, Aircraft Trajectory, Aircraft 4D Trajectory, Traffic 5D Traffic, 5D Traffic Prediction	Not Referenced
[56]	Combining Visual Analytics and Machine Learning for Route Choice Prediction Application to Pre-Tactical Traffic Forecast	Aircraft, Prediction, Analysis, Aircraft Trajectory, Aircraft Trajectory Route	Pre-Flying, Post-Analysis
[44]	A Multi-Agent Approach for Reactionary Delay Prediction of Flights	Prediction, Traffic, Traffic Indicator	Flying
[141]	Data-driven Aircraft Trajectory Predictions using Ensemble Meta-Estimators	Aircraft, Prediction, Aircraft Trajectory, Aircraft 4D Trajectory	Pre-Flying
[122]	Prediction of the Propagation of Trajectory Uncertainty for Climbing Aircraft	Aircraft, Prediction, Aircraft Trajectory, Aircraft 4D Trajectory	Flying
[138]	Predicting Aircraft Landing Time in Extended-TMA Using Machine Learning Methods	Aircraft, Prediction, Aircraft Trajectory, Aircraft Trajectory Indicator	Flying
[142]	Full-scale pre-tactical route prediction Machine Learning to increase pre-tactical demand forecast accuracy	Aircraft, Prediction, Aircraft Trajectory, Aircraft Trajectory Route	Pre-Flying
[42]	Short-Term Trajectory Prediction Using Generative Machine Learning Methods	Aircraft, Prediction, Aircraft Trajectory, Aircraft 4D Trajectory	Flying
[38]	Taxi-speed Prediction by Spatio-Temporal Graph-based Trajectory Representation and Its Applications	Prediction, Airport, Ground Traffic, Ground Traffic Indicators	Pre-Flying
[36]	Comparative Analysis of Machine Learning and Statistical Methods for Aircraft Phase of Flight Prediction	Aircraft, Prediction, Aircraft Trajectory, Aircraft Trajectory Indicator	Flying
[158]	A Boosted Tree Framework for Runway Occupancy and Exit Prediction	Prediction, Airport, Ground Traffic, Ground Traffic Indicators	Pre-Flying
[121]	Data-driven predictive analytics of runway occupancy time for improved capacity at airports	Prediction, Airport, Ground Traffic, Ground Traffic Indicators	Flying
[143]	Model generalisation in arrival runway occupancy time prediction by feature equivalences	Prediction, Airport, Ground Traffic, Ground Traffic Indicators	Flying
[34]	Predictive Distribution of the Mass and Speed Profile to Improve Aircraft Climb Prediction	Aircraft, Prediction, Aircraft State	Pre-Flying
[37]	Improving the predictability of take-off times with Machine Learning: a case study for the Maastricht upper area control centre area of responsibility	Prediction, Airport, Ground Traffic, Ground Traffic Indicators	Pre-Flying
[140]	Probabilistic Prediction of Separation Buffer to Compensate for the Closing Effect on Final Approach	Aircraft, Prediction, Aircraft Trajectory, Aircraft Trajectory Indicator	Flying
[139]	Probabilistic Prediction of Time To Fly using Quantile Regression Forest	Aircraft, Prediction, Aircraft Trajectory, Aircraft Trajectory Indicator	Pre-Flying
[190]	En-route Arrival Time Prediction using Gaussian Mixture Model	Aircraft, Prediction, Aircraft Trajectory, Aircraft Trajectory Indicator	Flying
[120]	Neural Network based Convection Indicator for Pre-Tactical Air Traffic Flow Management	Prediction, Airspace, Airspace State, Environmental State	Pre-Flying
[118]	A Multi-Layer Artificial Neural Network Approach for Runway Configuration Prediction	Prediction, Airport, Airport State	Pre-Flying
[40]	Predicting sector configuration transitions with autoencoder-based anomaly detection	Prediction, Airspace, Airspace Structure, Airspace Structure Sector	Flying
[213]	Aircraft Mass and Thrust Estimation Using Recursive Bayesian Method	Aircraft, Prediction, Aircraft State	Pre-Flying
[41]	A machine learning apporach on past ads-b data to predict planning controller’s actions	Prediction, Air Traffic Controller (ATCo)	Flying
[66]	Analysing Pilot Decision-Making Using Predictive Modelling	Prediction, Modelling, Analysis, Pilot	Flying, Post-Analysis
[89]	Research on Civil Aircraft Fuel Consumption in Cruise Phase Based on Least Square Support Vector Regression with Genetic Algorithm	Aircraft, Prediction, Aircraft State	Flying
[39]	Predicting Aircraft Trajectory Choice – A Nominal Route Approach	Aircraft, Prediction, Aircraft Trajectory, Aircraft Trajectory Route	Pre-Flying
[119]	Evaluation of feasible machine learning techniques for predicting the time to fly and aircraft speed profile on final approach: Predictive dynamic support tool on final approach	Aircraft, Prediction, Aircraft Trajectory, Aircraft Trajectory Indicator	Pre-Flying
[35]	Predicting aircraft descent length with machine learning	Aircraft, Prediction, Aircraft Trajectory, Aircraft Trajectory Indicator	Flying
[69]	Wind and Temperature Networking Applied to Aircraft Trajectory Prediction	Prediction, Airspace, Airspace State, Environmental State	Flying
[200]	Large-scale flight phase identification from ads-b data using machine learning methods	Aircraft, Analysis, Aircraft Trajectory, Aircraft Trajectory Indicator	Post-Analysis
[144]	Visual Analytics for Exploring Local Impact of Air Traffic	Prediction, Airspace, Static Structural State, Airspace State	Pre-Flying
[166]	Machine Learning Model for Aircraft Performances	Aircraft, Prediction, Aircraft State	Flying
[72]	Data-driven modelling of the Tree of Reactionary Delays	Prediction, Traffic, Traffic Indicator	Flying
[73]	Systemic delay propagation in the US airport network	Prediction, Modelling, Traffic, Traffic Indicator	Flying, Not Referenced
[30]	Predicting the future location of a general aviation aircraft	Aircraft, Prediction, Aircraft Trajectory, Aircraft 4D Trajectory	Flying
[146]	A network-based model for predicting air traffic delays	Prediction, Traffic, Traffic Indicator	Flying
[256]	Accuracy of Reinforcement Learning Algorithms for Predicting Aircraft Taxi-out Times	Prediction, Airport, Airport State	Pre-Flying
[179]	Trajectory prediction by functional regression in sobolev space	Aircraft, Prediction, Aircraft Trajectory, Aircraft 4D Trajectory	Pre-Flying
[180]	Trajectory prediction by functional regression in Sobolev space	Aircraft, Prediction, Aircraft Trajectory, Aircraft 4D Trajectory	Pre-Flying
[165]	Estimation and Comparison of the Impact of Single Airport Delay to the National Airspace System Using Multivariate Simultaneous Models	Prediction, Airspace, Static Structural State, Airspace State	Flying
[225]	Generating day-of-operation probabilistic capacity profiles from weather forecasts	Prediction, Airspace, Static Structural State, Airspace State	Pre-Flying
[172]	Analysing and Decomposing Taxi Times and Predicting Taxi out Times	Prediction, Airport, Airport State	Pre-Flying
[254]	Learning Traffic Patterns at Small Airports From Flight Tracks	Airport, Modelling, Ground Traffic, Ground Traffic 5D Traffic	Not Referenced
[116]	An Efficient Ant Colony System Based on Receding Horizon Control for the Aircraft Arrival Sequencing and Scheduling Problem	Optimisation, CTR, CTR Traffic	Pre-Flying
[111]	Two-Stage Stochastic Programming Based on Particle Swarm Optimisation for Aircraft Sequencing and Scheduling	Optimisation, CTR, CTR Traffic	Pre-Flying
[228]	Airspace Collision Risk Hot-Spot Identification using Clustering Models	Analysis, Airspace, Static Structural State, Airspace State	Post-Analysis
[220]	Trajectory Clustering and an Application to Airspace Monitoring	Analysis, Traffic, 5D Traffic Analysis, Traffic 5D Traffic	Post-Analysis
[253]	Forecasting Flight Delays Using Clustered Models Based on Airport Networks	Aircraft, Prediction, Aircraft Trajectory, Aircraft Trajectory Indicator	Pre-Flying
[193]	Learning Probabilistic Trajectory Models of Aircraft in Terminal Airspace From Position Data	Aircraft, Prediction, Aircraft Trajectory, Aircraft 4D Trajectory	Pre-Flying
[157]	High Confidence Intervals Applied to Aircraft Altitude Prediction	Aircraft, Prediction, Aircraft Trajectory, Aircraft Trajectory Indicator	Pre-Flying
[192]	Efficient Road Detection and Tracking for Unmanned Aerial Vehicle	Aircraft, Analysis, Aircraft Trajectory, Aircraft 4D Trajectory	Post-Analysis
[153]	A Two-Phase Method of Detecting Abnormalities in Aircraft Flight Data and Ranking Their Impact on Individual Flights	Aircraft, Analysis, Aircraft Trajectory, Aircraft 4D Trajectory	Post-Analysis
[191]	Anomaly detection via a Gaussian Mixture Model for flight operation and safety monitoring	Aircraft, Analysis, Aircraft Trajectory, Aircraft Trajectory Indicator	Post-Analysis
[151]	Flight trajectory data analytics for characterisation of air traffic flows: A comparative analysis of terminal area operations between New York, Hong Kong and Sao Paulo	Analysis, Traffic, 5D Traffic Analysis, Traffic 5D Traffic	Post-Analysis
[127]	A hybrid machine learning model for short-term estimated time of arrival prediction in terminal manoeuvring area	Aircraft, Prediction, Analysis, Aircraft Trajectory, Aircraft 4D Trajectory	Flying, Post-Analysis
[154]	A knowledge-transfer-based learning framework for airspace operation complexity evaluation	Analysis, Airspace, Static Structural State, Airspace State	Post-Analysis
[218]	Temporal logic learning-based anomaly detection in metroplex terminal airspace operations	Analysis, Traffic, Traffic Indicator	Post-Analysis
[199]	Detection and identification of significant events in historical aircraft trajectory data	Aircraft, Analysis, Aircraft Trajectory, Aircraft 4D Trajectory	Post-Analysis
[217]	A data-driven methodology for characterisation of a terminal manoeuvring area in multi-airport systems	Analysis, Traffic, Traffic 5D Traffic, 5D Traffic Analysis	Post-Analysis
[259]	Interactive image-based information visualisation for aircraft trajectory analysis	Analysis, Traffic, 5D Traffic Analysis, Traffic 5D Traffic	Post-Analysis
[226]	Characterisation and prediction of air traffic delays	Prediction, Traffic, Traffic Indicator	Pre-Flying
[177]	Predictability impacts of airport surface automation	Airport, Analysis, Ground Traffic, Ground Traffic Indicators	Post-Analysis
[102]	Evaluating ground–air network vulnerabilities in an integrated terminal manoeuvring area using co-evolutionary computational red teaming	Analysis, CTR, CTR Traffic	Post-Analysis
[354]	Modelling delay propagation within an airport network	Modelling, Traffic, Traffic Indicator	Not Referenced
[201]	Multi-objective optimisation for aircraft departure trajectories minimising noise annoyance	Optimisation, Traffic, Traffic 5D Traffic, 5D Traffic Optimisation	Pre-Flying
[237]	Reward functions for learning to control in air traffic flow management	Optimisation, Conflict Avoidance, Traffic, Traffic 5D Traffic, 5D Traffic Optimisation	Pre-Flying
[99]	Systemic identification of airspace collision risk tipping points using an evolutionary multi-objective scenario-based methodology	Optimisation, Traffic, Traffic 5D Traffic, 5D Traffic Optimisation	Pre-Flying
[260]	A CD&CR causal model based on path shortening/path stretching techniques	Optimisation, Conflict Avoidance, CTR, CTR Traffic	Flying
[202]	A fuzzy approach to addressing uncertainty in Airport Ground Movement optimisation	Optimisation, Airport, Ground Traffic, Ground Traffic 5D Traffic	Pre-Flying
[101]	Preference-based evolutionary algorithm for airport surface operations	Optimisation, Airport, Ground Traffic, Ground Traffic 5D Traffic	Pre-Flying
[107]	Integrated sequencing and merging aircraft to parallel runways with automated conflict resolution and advanced avionics capabilities	Optimisation, CTR, CTR Traffic	Flying
[117]	Solving the gate assignment problem through the Fuzzy Bee Colony Optimisation	Optimisation, Airport, Airport State	Pre-Flying
[114]	Metaheuristics for efficient aircraft scheduling and re-routing at busy terminal control areas	Optimisation, CTR, CTR Traffic	Flying
[355]	Increasing stability of crew and aircraft schedules	Optimisation, Traffic, Traffic Indicator	Pre-Flying
[84]	Intelligent computing methods in Air Traffic Flow Management	Optimisation, Conflict Avoidance, Traffic, Traffic 5D Traffic, 5D Traffic Optimisation	Pre-Flying
[216]	Prediction of aircraft performances based on data collected by air traffic control centers	Aircraft, Prediction, Aircraft State	Pre-Flying
[126]	Development of network restructuring models for improved air traffic forecasts	Prediction, Traffic, Traffic Indicator	Pre-Flying
[236]	Accuracy of reinforcement learning algorithms for predicting aircraft taxi-out times: A case-study of Tampa Bay departures	Prediction, Airport, Ground Traffic, Ground Traffic Indicators	Pre-Flying
[167]	Learning the aircraft mass and thrust to improve the ground-based trajectory prediction of climbing flights	Aircraft, Prediction, Aircraft Trajectory, Aircraft Trajectory Indicator	Pre-Flying
[230]	Generating day-of-operation probabilistic capacity scenarios from weather forecasts	Prediction, Airspace, Static Structural State, Airspace State	Pre-Flying
[150]	Predicting the operational acceptance of airborne flight reroute requests using data mining	Aircraft, Prediction, Aircraft Trajectory, Aircraft Trajectory Route	Flying
[161]	Learning aircraft operational factors to improve aircraft climb prediction: A large scale multi-airport study	Aircraft, Prediction, Aircraft Trajectory, Aircraft Trajectory Indicator	Flying
[154]	A knowledge-transfer-based learning framework for airspace operation complexity evaluation	Analysis, Airspace, Static Structural State, Airspace State	Post-Analysis
[136]	Analysis of air traffic control operational impact on aircraft vertical profiles supported by machine learning	Aircraft, Prediction, Aircraft Trajectory, Aircraft Trajectory Indicator	Pre-Flying
[209]	Estimating runway veer-off risk using a Bayesian network with flight data	Prediction, Airport, Ground Traffic, Ground Traffic Indicators	Pre-Flying
[247]	Flight time prediction for fuel loading decisions with a deep learning approach	Prediction, Airport, Ground Traffic, Ground Traffic Indicators	Flying
[183]	Prediction of runway configurations and airport acceptance rates for multi-airport system using gridded weather forecast	Prediction, Airport, Airport State	Flying
[184]	Short-term prediction of airway congestion index using machine learning methods	Prediction, Airspace, Static Structural State, Airspace State	Flying
[218]	Temporal logic learning-based anomaly detection in metroplex terminal airspace operations	Analysis, Traffic, Traffic Indicator	Post-Analysis
[128]	A machine learning approach to air traffic interdependency modelling and its application to trajectory prediction	Aircraft, Prediction, Aircraft Trajectory, Aircraft 4D Trajectory	Flying
[243]	Machine learning approach to predict aircraft boarding	Aircraft, Prediction, Aircraft State	Flying
[226]	Characterisation and prediction of air traffic delays	Prediction, Traffic, Traffic Indicator	Pre-Flying
[168]	Aircraft trajectory forecasting using local functional regression in Sobolev space	Aircraft, Prediction, Aircraft Trajectory, Aircraft 4D Trajectory	Pre-Flying
[244]	A Deep Graph-Embedded LSTM Neural Network Approach for Airport Delay Prediction	Prediction, Traffic, Traffic Indicator	Pre-Flying
[132]	A New Multilevel Input Layer Artificial Neural Network for Predicting Flight Delays at JFK Airport	Prediction, Airport, Airport State	Pre-Flying
[43]	Multi-Aircraft Trajectory Collaborative Prediction Based on Social Long Short-Term Memory Network	Aircraft, Prediction, Aircraft Trajectory, Aircraft 4D Trajectory	Flying
[246]	A Deep Learning Approach for Aircraft Trajectory Prediction in Terminal Airspace	Aircraft, Prediction, Aircraft Trajectory, Aircraft 4D Trajectory	Flying
[194]	Hybrid Machine Learning and Estimation-Based Flight Trajectory Prediction in Terminal Airspace	Aircraft, Prediction, Aircraft Trajectory, Aircraft 4D Trajectory	Flying
[235]	3D path planning and real-time collision resolution of multirotor drone operations in complex urban low-altitude airspace	Optimisation, Conflict Avoidance, Traffic, Traffic 5D Traffic, 5D Traffic Optimisation	Pre-Flying, Flying
[255]	Multi-Rotor Aircraft Collision Avoidance using Partially Observable Markov Decision Processes	Optimisation, Conflict Avoidance, Traffic, Traffic 5D Traffic, 5D Traffic Optimisation	Flying
[195]	Identification and prediction of urban airspace availability for emerging air mobility operations	Prediction, CTR, CTR Traffic	Pre-Flying
[108]	Trajectory-based flight scheduling for AirMetro in urban environments by conflict resolution	Optimisation, Conflict Avoidance, Traffic, Traffic 5D Traffic, 5D Traffic Optimisation	Pre-Flying, Flying
[185]	Predictive classification and understanding of weather impact on airport performance through machine learning	Prediction, Airport, Ground Traffic, Ground Traffic Indicators	Pre-Flying, Flying
[356]	Computation of Air Traffic Flow Management Performance with Long Short-Term Memories Considering Weather Impact	Prediction, Airport, Ground Traffic, Ground Traffic Indicators	Pre-Flying, Flying
[221]	Characterising air traffic networks via large-scale aircraft tracking data: A comparison between China and the US networks	Analysis, Traffic, 5D Traffic Analysis, Traffic 5D Traffic	Post-Analysis
[86]	Comparing the modelling of delay propagation in the US and European air traffic networks	Modelling, Traffic, Traffic Indicator	Not Referenced
[222]	From aircraft tracking data to network delay model: A data-driven approach considering en-route congestion	Traffic, 5D Traffic Analysis, Traffic 5D Traffic	Post-Analysis
[109]	A probabilistic model based optimisation for aircraft scheduling in terminal area under uncertainty	CTR, CTR Traffic	Flying
[357]	Terminal area control rules and eVTOL adaptive scheduling model for multi-vertiport system in urban air Mobility	CTR, CTR Traffic	Pre-Flying
[71]	An agent-based model for air transportation to capture network effects in assessing delay management mechanisms	Traffic, Traffic Indicator	Pre-Flying, Post-Analysis
[240]	UAV-Assisted Content Delivery in Intelligent Transportation Systems-Joint Trajectory Planning and Cache Management	5D Traffic Optimisation, Traffic, Traffic 5D Traffic	Pre-Flying
[110]	An Iterative Two-Phase Optimisation Method Based on Divide and Conquer Framework for Integrated Scheduling of Multiple UAVs	Traffic, Traffic Indicator	Pre-Flying
[241]	Data Freshness and Energy-Efficient UAV Navigation Optimisation: A Deep Reinforcement Learning Approach	5D Traffic Optimisation, Conflict Avoidance, Traffic, Traffic 5D Traffic	Pre-Flying
[229]	A Traffic Demand Analysis Method for Urban Air Mobility	Analysis, Traffic, Traffic Indicator	Post-Analysis
[156]	4-D Flight Trajectory Prediction With Constrained LSTM Network	Aircraft, Aircraft Trajectory, Aircraft 4D Trajectory	Pre-Flying, Flying
[248]	A Machine Learning Approach to Predict the Evolution of Air Traffic Flow Management Delay	Traffic, Traffic Indicator	Pre-Flying, Flying
[249]	Predicting Air Traffic Congested Areas with Long Short-Term Memory Networks	Traffic, Traffic Indicator	Pre-Flying, Flying
[250]	Early Detection of Night Curfew Infringements by Delay Propagation with Neural Networks	Traffic, Traffic Indicator	Pre-Flying
[155]	Having a Bad Day? Predicting High Delay Days in the National Airspace System	Traffic, Traffic Indicator	Pre-Flying
[227]	Modelling of Flight Time Prediction Uncertainty for Four-Dimensional Descent Trajectory Management	Aircraft, Aircraft Trajectory, Aircraft Trajectory Indicator	Flying
[223]	Leveraging local ADS-B transmissions to assess the performance of air traffic at general aviation airports	Traffic, Traffic Indicator	Post-Analysis
[196]	Modelling and Detecting Anomalous Safety Events in Approach Flights Using ADS-B Data	CTR, CTR Traffic	Post-Analysis
[197]	Modelling and Detecting Anomalous Safety Events in Approach Flights Using ADS-B Data	CTR, CTR Traffic	Post-Analysis
[198]	Wind Profile Estimation from Aircraft Derived Data Using Kalman Filters and Gaussian Process Regression	Airspace, Airspace State, Environmental State,	Post-Analysis
[238]	Recommending Strategic Air Traffic Management Initiatives in Convective Weather	Traffic, Traffic 5D Traffic, 5D Traffic Optimisation	Pre-Flying
[79]	Towards Conflict Resolution with Deep Multi-Agent Reinforcement Learning	Conflict Avoidance, Traffic, Traffic 5D Traffic, 5D Traffic Optimisation	Flying
[103]	Learning Uncertainty Parameters for Tactical Conflict Resolution	ATCO	Not Referenced
[186]	Readback Error Detection by Automatic Speech Recognition to Increase ATM Safety	ATCO, Pilot	Flying
[358]	Feasibility Study of Free routing Airspace Operation over the North Pacific Airspace	Traffic, Traffic 5D Traffic, 5D Traffic Modelling	Not Referenced
[224]	A machine learning-based framework for aircraft manoeuvre detection and classification	Traffic, 5D Traffic Analysis, Traffic 5D Traffic	Post-Analysis
[219]	Aircraft Performance-optimised Departure Flights Using Traffic Flow Funnels	Aircraft, Aircraft Trajectory, Aircraft 4D Trajectory	Pre-Flying
[242]	Real-time departure slotting in mixed-mode operations using deep reinforcement learning: a case study of Zurich airport	CTR, CTR Traffic	Pre-Flying
[87]	Evaluation of UTM Strategic Deconfliction Through End-to-End Simulation	Traffic, Traffic 5D Traffic, 5D Traffic Modelling	Flying
[88]	Mid-air collisions with drones Assessment of collision scenarios and of drone operation risks in urban areas	Traffic, Traffic 5D Traffic, 5D Traffic Modelling	Flying
[239]	Reinforcement Learning for Traffic Flow Management Decision Support	Traffic, Traffic Indicator	Pre-Flying
[152]	Predicting arrival delays in the terminal area five hours in advance with machine learning	Aircraft, Aircraft Trajectory, Aircraft Trajectory Indicator	Pre-Flying
[231]	Spatial Modelling of Airport Surface Fuel Burn for Environmental Impact Analyses	Airport, Ground Traffic, Ground Traffic Indicators	Not Referenced
[252]	A data-driven operational model for traffic at the Dallas Fort Worth International Airport	CTR, CTR Traffic	Not Referenced
[211]	Characterisation of delay propagation in the air traffic network	Traffic, Traffic Indicator	Flying
[133]	Air passenger forecasting using Neural Granger causal Google trend queries	Traffic, Traffic Indicator	Pre-Flying
[162]	An explainable machine learning approach to improve take-off time predictions	Airport, Ground Traffic, Ground Traffic Indicators	Pre-Flying
[137]	Estimating entry counts and ATFM regulations during adverse weather conditions using machine learning	Airspace, Airspace State, Static Structural State	Post-Analysis
[257]	Prediction and extraction of tower controller commands for speech recognition applications	ATCO	Flying
[134]	Estimating the impact of COVID-19 on air travel in the medium and long term using neural network and Monte Carlo simulation	Traffic, Traffic Indicator	Pre-Flying
[169]	Ratio-based design hour determination for airport passenger terminal facilities	Airport, Airport State	Pre-Flying
[207]	An integrated SWOT-based fuzzy AHP and fuzzy MARCOS methodology for digital transformation strategy analysis in airline industry	Traffic, Traffic Indicator	Pre-Flying
[251]	Artificial neural network models for airport capacity prediction	Airport, Airport State	Pre-Flying

References

Erdi, P. Complex systems: The intellectual landscape. In Complexity Explained; Springer: Berlin/Heidelberg, Germany, 2008; pp. 1–23. ISBN 3540357777. [Google Scholar]
Allignol, C.; Barnier, N.; Flener, P.; Pearson, J. Constraint programming for air traffic management: A survey1: In memory of Pascal Brisset. Knowl. Eng. Rev. 2012, 27, 361–392. [Google Scholar] [CrossRef] [Green Version]
International Civil Aviation Organization. Air Traffic Management: Procedures for Air Navigation Services; International Civil Aviation Organization: Montréal, QC, Canada, 2007. [Google Scholar]
Pouzenc, C. Cartes des Secteurs de Contrôle de L’espace Aérien Civil Français. 2017. Available online: https://www.cypouz.com/article/150203/cartes-des-secteurs-controle-espace-aerien-civil-francais (accessed on 2 December 2021).
EUROCONTROL. EUROCONTROL Forecast Update 2021–2024. 2021. Available online: https://www.eurocontrol.int/publication/eurocontrol-forecast-update-2021-2024 (accessed on 1 June 2021).
Davies, L.; Vagapov, Y.; Grout, V.; Cunningham, S.; Anuchin, A. Review of Air Traffic Management Systems for UAV Integration into Urban Airspace. In Proceedings of the 2021 28th International Workshop on Electric Drives: Improving Reliability of Electric Drives (IWED), Moscow, Russia, 27–29 January 2021; pp. 1–6. [Google Scholar] [CrossRef]
Degas, A.; Kaddoum, E.; Gleizes, M.P.; Adreit, F.; Rantrua, A. Cooperative multi-agent model for collision avoidance applied to air traffic management. Eng. Appl. Artif. Intell. 2021, 102, 104286. [Google Scholar] [CrossRef]
Marquis, P.; Papini, O.; Prade, H. A Guided Tour of Artificial Intelligence Research: Volume I: Knowledge Representation, Reasoning and Learning; Springer Nature: Cham, Switzerland, 2020. [Google Scholar]
Council of Europe. History of Artificial Intelligence; Council of Europe: Strasbourg, France, 2020. [Google Scholar]
Ferrucci, D.A. Introduction to “this is watson”. IBM J. Res. Dev. 2012, 56, 1–15. [Google Scholar] [CrossRef]
Markoff, J. How many computers to identify a cat? 16,000. New York Times, 26 June 2012. [Google Scholar]
Silver, D.; Schrittwieser, J.; Simonyan, K.; Antonoglou, I.; Huang, A.; Guez, A.; Hubert, T.; Baker, L.; Lai, M.; Bolton, A. Mastering the game of go without human knowledge. Nature 2017, 550, 354–359. [Google Scholar] [CrossRef] [PubMed]
Mueller, S.T.; Hoffman, R.R.; Clancey, W.; Emrey, A.; Klein, G. Explanation in human-AI systems: A literature meta-review, synopsis of key ideas and publications, and bibliography for explainable AI. arXiv 2019, arXiv:1902.01876. [Google Scholar]
Durand, N.; Gianazza, D.; Gotteland, J.B.; Alliot, J.M. Metaheuristics for Air Traffic Management; John Wiley & Sons: Hoboken, NJ, USA, 2015. [Google Scholar]
Shone, R.; Glazebrook, K.; Zografos, K.G. Applications of stochastic modeling in air traffic management: Methods, challenges and opportunities for solving air traffic problems under uncertainty. Eur. J. Oper. Res. 2021, 292, 1–26. [Google Scholar] [CrossRef]
Emami, H.; Derakhshan, F. An overview on conflict detection and resolution methods in air traffic management using multi agent systems. In Proceedings of the 16th CSI International Symposium on Artificial Intelligence and Signal Processing (AISP 2012), Shiraz, Iran, , 2–3 May 2012; pp. 293–298. [Google Scholar] [CrossRef]
Weigang, L.; Leite, A.F.; Ribeiro, V.F.; Fregnani, J.A.; de Oliveira, I.R. Towards Intelligent System Wide Information Management for Air Traffic Management. In Security, Privacy, and Anonymity in Computation, Communication, and Storage; Lecture Notes in Computer Science, Wang, G., Atiquzzaman, M., Yan, Z., Choo, K.K.R., Eds.; Springer International Publishing: Cham, Switzerland, 2017; pp. 584–593. [Google Scholar] [CrossRef]
Wu, C.L.; Caves, R.E. Research review of air traffic management. Transp. Rev. 2002, 22, 115–132. [Google Scholar] [CrossRef]
Aricò, P.; Reynal, M.; Di Flumeri, G.; Borghini, G.; Sciaraffa, N.; Imbert, J.P.; Hurter, C.; Terenzi, M.; Ferreira, A.; Pozzi, S.; et al. How Neurophysiological Measures Can be Used to Enhance the Evaluation of Remote Tower Solutions. Front. Hum. Neurosci. 2019, 13, 303. [Google Scholar] [CrossRef]
Aricò, P.; Borghini, G.; Di Flumeri, G.; Colosimo, A.; Bonelli, S.; Golfetti, A.; Pozzi, S.; Imbert, J.P.; Granger, G.; Benhacene, R.; et al. Adaptive Automation Triggered by EEG-Based Mental Workload Index: A Passive Brain-Computer Interface Application in Realistic Air Traffic Control Environment. Front. Hum. Neurosci. 2016, 10, 539. [Google Scholar] [CrossRef] [Green Version]
Wang, D.; Yang, Q.; Abdul, A.; Lim, B.Y. Designing Theory-Driven User-Centric Explainable AI. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems—CHI ’19, Glasgow, Scotland, UK, 4–9 May 2019; ACM Press: New York, NY, USA, 2019; pp. 1–15. [Google Scholar] [CrossRef]
Sedlmair, M.; Meyer, M.; Munzner, T. Design study methodology: Reflections from the trenches and the stacks. IEEE Trans. Vis. Comput. Graph. 2012, 18, 2431–2440. [Google Scholar] [CrossRef] [Green Version]
Munzner, T. Visualization Analysis and Design; CRC Press: Boca Raton, FL, USA, 2014. [Google Scholar]
Beaudouin-Lafon, M.; Mackay, W.E. Prototyping tools and techniques. In Human-Computer Interaction; CRC Press: Boca Raton, FL, USA, 2009; pp. 137–160. [Google Scholar]
Vredenburg, K.; Mao, J.Y.; Smith, P.W.; Carey, T. A survey of user-centered design practice. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems–CHI02, Minneapolis, MN, USA, 20–25 April 2002; ACM Press: New York, NY, USA, 2002; pp. 471–478. [Google Scholar]
Kistan, T.; Gardi, A.; Sabatini, R. Machine Learning and Cognitive Ergonomics in Air Traffic Management: Recent Developments and Considerations for Certification. Aerospace 2018, 5, 103. [Google Scholar] [CrossRef] [Green Version]
Xie, Y.; Pongsakornsathien, N.; Gardi, A.; Sabatini, R. Explanation of Machine-Learning Solutions in Air-Traffic Management. Aerospace 2021, 8, 224. [Google Scholar] [CrossRef]
McInnes, M.D.; Moher, D.; Thombs, B.D.; McGrath, T.A.; Bossuyt, P.M.; Clifford, T.; Cohen, J.F.; Deeks, J.J.; Gatsonis, C.; Hooft, L.; et al. Preferred reporting items for a systematic review and meta-analysis of diagnostic test accuracy studies: The PRISMA-DTA statement. JAMA 2018, 319, 388–396. [Google Scholar] [CrossRef] [PubMed]
Google Scholar. Top Publications: Categories, Engineering & Computer Science, Transportation. Available online: https://scholar.google.com/citations?view_op=top_venues&hl=fr&vq=eng_transportation (accessed on 10 April 2021).
Le Tallec, C.; Taurino, D.; Lancia, C.; Verstraeten, J. Predicting the future location of a general aviation aircraft. In Proceedings of the ICRAT 2014, 6th International Conference on Research in Air Transportation, Istanbul, Turkey, 26–20 May 2014. [Google Scholar]
Pedregosa, F.; Varoquaux, G.; Gramfort, A.; Michel, V.; Thirion, B.; Grisel, O.; Blondel, M.; Prettenhofer, P.; Weiss, R.; Dubourg, V. Scikit-learn: Machine learning in Python. J. Mach. Learn. Res. 2011, 12, 2825–2830. [Google Scholar]
Abadi, M.; Barham, P.; Chen, J.; Chen, Z.; Davis, A.; Dean, J.; Devin, M.; Ghemawat, S.; Irving, G.; Isard, M. Tensorflow: A system for large-scale machine learning. In Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16), Savannah, GA, USA, 2–4 November 2016; pp. 265–283. [Google Scholar]
Chollet, F. Deep learning with Python; Apress: Berkeley, CA, USA, 2018; Volume 361. [Google Scholar]
Alligier, R. Predictive Distribution of the Mass and Speed Profile to Improve Aircraft Climb Prediction. J. Air Transp. 2020, 28, 114–123. [Google Scholar] [CrossRef] [Green Version]
Alligier, R.; Gianazza, D.; Durand, N. Predicting aircraft descent length with machine learning. In Proceedings of the ICRAT 2016, 7th International Conference on Research in Air Transportation, Philadelphia, PA, USA, 20–24 June 2016. [Google Scholar]
Kovarik, S.; Doherty, L.; Korah, K.; Mulligan, B.; Rasool, G.; Mehta, Y.; Bhavsar, P.; Paglione, M. Comparative Analysis of Machine Learning and Statistical Methods for Aircraft Phase of Flight Prediction. In Proceedings of the ICRAT 2020, 9th International Conference on Research in Air Transportation, Virtual Event, 15 September 2020. [Google Scholar]
Dalmau Codina, R.; Belkoura, S.; Naessens, H.; Ballerini, F.; Wagnick, S. Improving the predictability of take-off times with Machine Learning: A case study for the Maastricht upper area control centre area of responsibility. In Proceedings of the SID 2019, 9th SESAR Innovation Days, Athens, Greece, 2–6 December 2019; pp. 1–8. [Google Scholar]
Tran, T.N.; Pham, D.T.; Alam, S.; Duong, V. Taxi-Speed Prediction by Spatio-Temporal Graph-Based Trajectory Representation and Its Applications. In Proceedings of the ICRAT 2020, 9th International Conference on Research in Air Transportation, Virtual Event, 15 September 2020. [Google Scholar]
Liu, Y.; Hansen, M.; Ball, M.; Lovell, D. Predicting Aircraft Trajectory Choice—A Nominal Route Approach. In Proceedings of the ICRAT 2018, 8th International Conference on Research in Air Transportation, Castelldefels, Catalonia, Spain, 26–29 June 2018. [Google Scholar]
Dubot, T. Predicting sector configuration transitions with autoencoder-based anomaly detection. In Proceedings of the International Conference on Research in Air Transportation, Castelldefels, Catalonia, Spain, 26–29 June 2018. [Google Scholar]
Pham, D.T.; Alam, S.; Yi-Lin, S.; Duong, V.N. A machine learning approach on past ads-b data to predict planning controller’s actions. In Proceedings of the ICRAT 2018, 8th International Conference on Research in Air Transportation, Castelldefels, Catalonia, Spain, 26–29 June 2018. [Google Scholar]
Le, T.H.; Tran, P.; Pham, D.T.; Schultz, M.; Alam, S. Short-Term Trajectory Prediction Using Generative Machine Learning Methods. In Proceedings of the ICRAT 2020, 9th International Conference on Research in Air Transportation, Virtual Event, 15 September 2020. [Google Scholar]
Xu, Z.; Zeng, W.; Chu, X.; Cao, P. Multi-Aircraft Trajectory Collaborative Prediction Based on Social Long Short-Term Memory Network. Aerospace 2021, 8, 115. [Google Scholar] [CrossRef]
Guleria, Y.; Cai, Q.; Alam, S.; Li, L. A Multi-Agent Approach for Reactionary Delay Prediction of Flights. IEEE Access 2019, 7, 181565–181579. [Google Scholar] [CrossRef]
Rantrua, A. Simulation Massive de Monde Virtuel par Système Multi-Agent Auto-Adaptatif. Ph.D. Thesis, Université de Toulouse, Université Toulouse III, Paul Sabatier, Toulouse, France, 2017. [Google Scholar]
Dieumegard, P.; Chaimatanan, S.; Delahaye, D. Large Scale Adaptive 4D Trajectory Planning. In Proceedings of the 2018 IEEE/AIAA 37th Digital Avionics Systems Conference (DASC), London, UK, 23–27 September 2018; pp. 1–9, ISSN 2155-7209. [Google Scholar] [CrossRef] [Green Version]
Durand, N.; Gotteland, J.B. Genetic Algorithms Applied to Air Traffic Management. In Metaheuristics for Hard Optimization Methods and Case Studies; Springer: Berlin/Heidelberg, Germany, 2006; Chapter 9; pp. 277–306. [Google Scholar] [CrossRef]
Degas, A.; Kaddoum, E.; Gleizes, M.P.; Adreit, F.; Rantrua, A. Dynamic Collision Avoidance using Local Cooperative Airplanes Decisions. CEAS Aeronaut. J. 2018, 11, 309–320. [Google Scholar] [CrossRef] [Green Version]
Breil, R.; Delahaye, D.; Lapasset, L.; Féron, E. Multi-agent Systems for Air Traffic Conflicts Resolution by Local Speed Regulation. In Proceedings of the ICRAT 2016, 7th International Conference on Research in Air Transportation, Philadelphia, PA, USA, 20–24 June 2016. [Google Scholar]
Ma, J.; Delahaye, D.; Sbihi, M.; Mongeau, M. Merging flows in terminal maneuvering area using time decomposition approach. In Proceedings of the ICRAT 2016, 7th International Conference on Research in Air Transportation, Philadelphia, PA, USA, 20–24 June 2016. [Google Scholar]
Zhou, J.; Cafieri, S.; Delahaye, D.; Sbihi, M. Optimization of arrival and departure routes in terminal maneuvering area. In Proceedings of the ICRAT 2014, 6th International Conference on Research in Air Transportation, Istanbul, Turkey, 26–20 May 2014. [Google Scholar]
Ma, J.; Delahaye, D.; Sbihi, M.; Scala, P.M. Integrated Optimization of Arrival, Departure, and Surface Operations. In Proceedings of the ICRAT 2018, 8th International Conference on Research in Air Transportation, Castelldefels, Catalonia, Spain, 26–29 June 2018. [Google Scholar]
Breil, R.; Delahaye, D.; Lapasset, L.; Féron, E. Multi-agent systems to help managing air traffic structure. J. Aerosp. Oper. 2017, 5, 119–148. [Google Scholar] [CrossRef]
Tang, J.; Alam, S.; Lokan, C.; Abbass, H.A. A multi-objective evolutionary method for Dynamic Airspace Re-sectorization using sectors clipping and similarities. In Proceedings of the 2012 IEEE Congress on Evolutionary Computation, Brisbane, QLD, Australia, 10–15 June 2012; pp. 1–8, ISSN 1941-0026. [Google Scholar] [CrossRef] [Green Version]
Ligny, L. FPCA applied to flight paths optimization. In Proceedings of the ICRAT 2020, 9th International Conference on Research in Air Transportation, Virtual Event, 15 September 2020. [Google Scholar]
Marcos, R.; Ros, O.G.C.; Herranz, R. Combining Visual Analytics and Machine Learning for Route Choice Prediction Application to Pre-Tactical Traffic Forecast. In Proceedings of the SID 2017, 7th SESAR Innovation Days, Belgrade, Serbia, 28–30 November 2017. [Google Scholar]
Olive, X.; Basora, L.; Viry, B.; Alligier, R. Deep Trajectory Clustering with Autoencoders. In Proceedings of the ICRAT 2020, 9th International Conference on Research in Air Transportation, Virtual Event, 15 September 2020. [Google Scholar]
Andrienko, G.; Andrienko, N.; Fuchs, G.; Garcia, J.M.C. Clustering Trajectories by Relevant Parts for Air Traffic Analysis. IEEE Trans. Vis. Comput. Graph. 2018, 24, 34–44. [Google Scholar] [CrossRef] [Green Version]
Olive, X.; Grignard, J.; Dubot, T.; Saint-Lot, J. Detecting Controllers’ Actions in Past Mode S Data by Autoencoder-Based Anomaly Detection. In Proceedings of the SID 2018, 8th SESAR Innovation Days, Salzburg, Austria, 3–7 December 2018. [Google Scholar]
Oualil, Y.; Klakow, D.; Szaszák, G.; Srinivasamurthy, A.; Helmke, H.; Motlicek, P. A context-aware speech recognition and understanding system for air traffic control domain. In Proceedings of the 2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), Okinawa, Japan, 16–20 December 2017; pp. 404–408. [Google Scholar]
Degas, A. Auto-Structuration de Trafic Temps-Réel Multi-Objectif et Multi-Critère Dans un Monde Virtuel. Ph.D. Thesis, Université de Toulouse, Université Toulouse III, Paul Sabatier, Toulouse, France, 2020. [Google Scholar]
Mouillet, V. User Manual for the Base of Aircraft Data (BADA) Revision 3.14; Technical Report; EUROCONTROL Experimental Centre: Brétigny, France, 2017. [Google Scholar]
Fricke, H.; Förster, S.; Vogel, M. Using agent-based modeling to determine collision risk in complex TMA environments: The turn-onto-ILS-final safety case. Aeronaut. Aerosp. Open Access J. 2018, 2, 155–164. [Google Scholar] [CrossRef] [Green Version]
Rantrua, A.; Chabrier, S.; Gleizes, M.P. Adaptive Air Traffic with Big Data Analysis. In Proceedings of the 1st EWG-DSS International Conference on Decision Support System Technology: Big data analytics for decision-making (ICDSST 2015), Belgrade, Serbia, 27–29 May 2015; Volume 1, pp. 46–52. [Google Scholar]
García Vasco, I.; Valero García, M.; Prats Menéndez, X.; Delgado, L. Agent-based simulation framework for airport collaborative decision making. In Proceedings of the 6th International Congress on Research in Air Transportation (ICRAT), Istanbul, Turkey, 26–20 May 2014. [Google Scholar]
Shah, S.R.; Campbell, A.; Campbell, A. Analyzing Pilot Decision-Making Using Predictive Modeling. In Proceedings of the ICRAT 2018, 8th International Conference on Research in Air Transportation, Castelldefels, Catalonia, Spain, 26–29 June 2018. [Google Scholar]
Scarselli, F.; Gori, M.; Tsoi, A.C.; Hagenbuchner, M.; Monfardini, G. The graph neural network model. IEEE Trans. Neural Netw. 2008, 20, 61–80. [Google Scholar] [CrossRef] [PubMed] [Green Version]
Mollinga, J.; Hoof, H. An Autonomous Free Airspace En-route Controller using Deep Reinforcement Learning Techniques. In Proceedings of the ICRAT 2020, 9th International Conference on Research in Air Transportation, Virtual Event, 15 September 2020. [Google Scholar]
Legrand, K.; Delahaye, D.; Rabut, C. Wind and Temperature Networking Applied to Aircraft Trajectory Prediction. In Proceedings of the ICRAT 2016, 7th Edition of the International Conference on Research in Air Transportation, Philadelphia, PA, USA, 20–24 June 2016. [Google Scholar]
Rantrua, A.; Maesen, E.; Chabrier, S.; Gleizes, M.P. Learning aircraft behavior from real air traffic. In Proceedings of the ATCA 2015, 60th ATCA Annual Conference & Exposition (2015), Washington, DC, USA, 8–9 February 2015. [Google Scholar]
Gurtner, G.; Delgado, L.; Valput, D. An agent-based model for air transportation to capture network effects in assessing delay management mechanisms. Transp. Res. Part C Emerg. Technol. 2021, 133, 103358. [Google Scholar] [CrossRef]
Campanelli, B.; Ramasco, J.J.; Fleurquin, P.; Eguíluz, V.M.; Etxebarria, I.; Arranz, A. Data-driven modelling of the Tree of Reactionary Delays. In Proceedings of the ICRAT 2014, 6th International Conference on Research in Air Transportation, Istanbul, Turkey, 26–20 May 2014. [Google Scholar]
Fleurquin, P.; Ramasco, J.J.; Eguiluz, V.M. Systemic delay propagation in the US airport network. Sci. Rep. 2013, 3, 1159. [Google Scholar] [CrossRef] [Green Version]
Kravaris, T.; Spatharis, C.; Blekas, K.; Vouros, G.A.; Garcia, J.M.C. Multiagent Reinforcement Learning Methods for Resolving Demand—Capacity Imbalances. In Proceedings of the 2018 IEEE/AIAA 37th Digital Avionics Systems Conference (DASC), London, UK, 23–27 September 2018; pp. 1–10. [Google Scholar] [CrossRef]
Maas, J.; Sunil, E.; Ellerbroek, J.; Hoekstra, J. The effect of swarming on a voltage potential-based conflict resolution algorithm. In Proceedings of the ICRAT 2016, 7th International Conference on Research in Air Transportation, Philadelphia, PA, USA, 20–24 June 2016. [Google Scholar]
Pallottino, L.; Scordio, V.G.; Bicchi, A. Decentralized cooperative conflict resolution among multiple autonomous mobile agents. In Proceedings of the 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601), Paradise Island, Bahamas, 14–17 December 2004; Volume 5, pp. 4758–4763, ISSN 0191-2216. [Google Scholar] [CrossRef]
Ribeiro, M.; Ellerbroek, J.; Hoekstra, J. The Effect of Intent on Conflict Detection and Resolution at High Traffic Densities. In Proceedings of the ICRAT 2020, 9th International Conference on Research in Air Transportation, Virtual Event, 15 September 2020. [Google Scholar]
Li, S.; Egorov, M.; Kochenderfer, M.J. Optimizing Collision Avoidance in Dense Airspace using DeepReinforcement Learning. In Proceedings of the Thirteenth USA/Europe Air Traffic Management Research and Development Seminar (ATM2019), Vienna, Austria, 17–21 June 2019. [Google Scholar]
Isufaj, R.; Aranega Sebastia, D.; Piera, M.A. Towards Conflict Resolution with Deep Multi-Agent Reinforcement Learning. In Proceedings of the Fourteenth USA/Europe Air Traffic Management Research and Development Seminar (ATM2021), Virtual Event, 20–23 September 2021; pp. 20–23. [Google Scholar]
Förster, S.; Fricke, H.; Vogel, M. Using Agent-Based Modeling to Determine Collision Risk in Complex TMA Environments. In Proceedings of the ICRAT 2018, 8th International Conference on Research in Air Transportation, Castelldefels, Catalonia, Spain, 26–29 June 2018. [Google Scholar]
Hoekstra, J.M.; Ellerbroek, J. Bluesky ATC simulator project: An open data and open source approach. In Proceedings of the ICRAT 2016, 7th Edition of the International Conference on Research in Air Transportation, Philadelphia, PA, USA, 20–24 June 2016; Volume 131, p. 132. [Google Scholar]
Micciche, S.; Mantegna, R.; Gurtner, G.; Lillo, F.; Valori, L.; Ducci, M.; Monechi, B.; Pozzi, S. An Agent Based Model of the Air Traffic Management. In Proceedings of the SID 2013, SESAR Innovation Days, Stockholm, Sweden, 26–28 November 2013. [Google Scholar]
Ye, B.; Hu, M.; Shortle, J. Risk-capacity tradeoff analysis of an en-route corridor model. In Proceedings of the 5th International Conference on Research in Air Transportation (ICRAT), Berkeley, CA, USA, 22–25 May 2012. [Google Scholar]
Weigang, L.; Dib, M.V.P.; Alves, D.P.; Crespo, A.M.F. Intelligent computing methods in Air Traffic Flow Management. Transp. Res. Part C Emerg. Technol. 2010, 18, 781–793. [Google Scholar] [CrossRef]
Teuwen, R.J.; Blom, H.A.; Everdij, M.H.; Bakker, G.B. Agent-based safety risk analysis of Trajectory Based Operation in the Terminal Manoevring Area. In Proceedings of the ICRAT 2014, 6th International Conference on Research in Air Transportation, Istanbul, Turkey, 26–20 May 2014. [Google Scholar]
Campanelli, B.; Fleurquin, P.; Arranz, A.; Etxebarria, I.; Ciruelos, C.; Eguíluz, V.M.; Ramasco, J.J. Comparing the modeling of delay propagation in the US and European air traffic networks. J. Air Transp. Manag. 2016, 56, 12–18. [Google Scholar] [CrossRef]
Egorov, M.; Evans, A.; Campbell, S.; Evans, T.; Zanlongo, S.; Young, T. Evaluation of UTM Strategic Deconfliction Through End-to-End Simulation. In Proceedings of the Fourteenth USA/Europe Air Traffic Management Research and Development Seminar (ATM2021), Virtual Event, 20–23 September 2021. [Google Scholar]
Fricke, H.; Förster, S.; Bruehl, R.; Austen, W.; Thiel, C. Mid-air collisions with drones Assessment of collision scenarios and of drone operation risks in urban areas. In Proceedings of the Fourteenth USA/Europe Air Traffic Management Research and Development Seminar (ATM2021), Virtual Event, 20–23 September 2021. [Google Scholar]
Gong, S. Research on Civil Aircraft Fuel Consumption in Cruise Phase Based on Least Square Support Vector Regression with Genetic Algorithm. In Proceedings of the ICRAT 2018, 8th International Conference on Research in Air Transportation, Castelldefels, Catalonia, Spain, 26–29 June 2018. [Google Scholar]
Allignol, C.; Barnier, N.; Durand, N.; Alliot, J.M. A new framework for solving en-routes conflicts. In Proceedings of the ATM 2013, 10th USA/Europe Air Traffic Management Research and Development Seminar, Chicago, IL, USA, 10–13 June 2013. [Google Scholar]
Delahaye, D.; Peyronne, C.; Mongeau, M.; Puechmorel, S. Aircraft conflict resolution by genetic algorithm and B-spline approximation. In Proceedings of the EIWAC 2010, 2nd ENRI International Workshop on ATM/CNS, Tokyo, Japan, 10–12 November 2010; p. 71. [Google Scholar]
Gotteland, J.B.; Durand, N.; Alliot, J.M. Aircraft Ground Traffic Optimization. In Proceedings of the ATM 2001, 4th USA/Europe Air Traffic Management Research and Development Seminar, Santa Fe, NM, USA, 4–7 December 2001. [Google Scholar]
Marceau, G.; Savéant, P.; Schoenauer, M. Multiobjective Pre-Tactical Planning Under Uncertainty for Air Traffic Control. In Proceedings of the ATACCS 2013, 3rd International Conference on Application and Theory of Automation in Command and Control Systems, Naples, Italy, 28–30 May 2013. [Google Scholar]
Murata, A.; Delahaye, D.; Takadama, K. Merging Flows and Optimizing Aircraft Scheduling in Terminal Maneuvering Area Based on GA. In Proceedings of the ICRAT 2018, 8th International Conference on Research in Air Transportation, Castelldefels, Catalonia, Spain, 26–29 June 2018. [Google Scholar]
Peyronne, C. Modélisation Mathématique et Résolution Automatique de Conflits par Algorithmes Génétiques et par Optimisation Locale Continue. Ph.D. Thesis, Université Paul Sabatier-Toulouse III, Toulouse, France, 2012. [Google Scholar]
Vanaret, C. Hybridization of Interval Methods and Evolutionary Algorithms for Solving Difficult Optimization Problems. Ph.D. Thesis, Université Paul Sabatier-Toulouse III, Toulouse, France, 2014. [Google Scholar]
Cheng, C.H.; Ho, S.C.; Kwan, C.L. The use of meta-heuristics for airport gate assignment. Expert Syst. Appl. 2012, 39, 12430–12437. [Google Scholar] [CrossRef]
Vanaret, C.; Gianazza, D.; Durand, N.; Gotteland, J.B. Benchmarking conflict resolution algorithms. In Proceedings of the ICRAT 2012, 5th International Conference on Research in Air Transportation, Berkeley, CA, USA, 22–25 May 2012. [Google Scholar]
Alam, S.; Lokan, C.; Aldis, G.; Barry, S.; Butcher, R.; Abbass, H. Systemic identification of airspace collision risk tipping points using an evolutionary multi-objective scenario-based methodology. Transp. Res. Part C Emerg. Technol. 2013, 35, 57–84. [Google Scholar] [CrossRef]
Kammoun, M.A.; Rezg, N. An efficient hybrid approach for resolving the aircraft routing and rescheduling problem. J. Air Transp. Manag. 2018, 71, 73–87. [Google Scholar] [CrossRef]
Weiszer, M.; Chen, J.; Stewart, P.; Zhang, X. Preference-based evolutionary algorithm for airport surface operations. Transp. Res. Part C Emerg. Technol. 2018, 91, 296–316. [Google Scholar] [CrossRef]
Zhao, W.; Alam, S.; Abbass, H.A. Evaluating ground–air network vulnerabilities in an integrated terminal maneuvering area using co-evolutionary computational red teaming. Transp. Res. Part C Emerg. Technol. 2013, 29, 32–54. [Google Scholar] [CrossRef]
Degaugue, S.; Gotteland, J.B.; Durand, N. Learning Uncertainty Parameters for Tactical Conflict Resolution. In Proceedings of the Fourteenth USA/Europe Air Traffic Management Research and Development Seminar (ATM2021), Virtual Event, 20–23 September 2021. [Google Scholar]
Chaimatanan, S.; Delahaye, D.; Mongeau, M. Strategic deconfliction of aircraft trajectories. In Proceedings of the ISIATM 2013, 2nd International Conference on Interdisciplinary Science for Innovative Air Traffic Management, Toulouse, France, 8–10 July 2013. [Google Scholar]
Girardet, B. Trafic Aérien: Détermination Optimale et Globale des Trajectoires D’avion en Présence de Vent. Ph.D. Thesis, Université de Toulouse, Toulouse, France, 2014. [Google Scholar]
Rodionova, O.; Sbihi, M.; Delahaye, D.; Mongeau, M. Optimization of aicraft trajectories in North Atlantic oceanic airspace. In Proceedings of the ICRAT 2012, 5th International Conference on Research in Air Transportation, Berkeley, CA, USA, 22–25 May 2012. [Google Scholar]
Liang, M.; Delahaye, D.; Maréchal, P. Integrated sequencing and merging aircraft to parallel runways with automated conflict resolution and advanced avionics capabilities. Transp. Res. Part C Emerg. Technol. 2017, 85, 268–291. [Google Scholar] [CrossRef]
Wu, Y.; Low, K.H.; Hu, X. Trajectory-based flight scheduling for AirMetro in urban environments by conflict resolution. Transp. Res. Part C Emerg. Technol. 2021, 131, 103355. [Google Scholar] [CrossRef]
Huo, Y.; Delahaye, D.; Sbihi, M. A probabilistic model based optimization for aircraft scheduling in terminal area under uncertainty. Transp. Res. Part C Emerg. Technol. 2021, 132, 103374. [Google Scholar] [CrossRef]
Liu, H.; Li, X.; Wu, G.; Fan, M.; Wang, R.; Gao, L.; Pedrycz, W. An Iterative Two-Phase Optimization Method Based on Divide and Conquer Framework for Integrated Scheduling of Multiple UAVs. IEEE Trans. Intell. Transp. Syst. 2021, 22, 5926–5938. [Google Scholar] [CrossRef]
Hong, Y.; Choi, B.; Kim, Y. Two-Stage Stochastic Programming Based on Particle Swarm Optimization for Aircraft Sequencing and Scheduling. IEEE Trans. Intell. Transp. Syst. 2019, 20, 1365–1377. [Google Scholar] [CrossRef]
Allignol, C.; Barnier, N.; Gondran, A. Optimized Flight Level Allocation at the Continental Scale. In Proceedings of the ICRAT 2012, 5th International Conference on Research in Air Transportation, Berkeley, CA, USA, 22–25 May 2012. [Google Scholar]
Nagarajan, H.; Wei, P.; Rathinam, S.; Sun, D. Air transportation network robustness optimization under limited legs itinerary constraint. In Proceedings of the ICRAT 2012, 5th International Conference on Research in Air Transportation, Berkeley, CA, USA, 22–25 May 2012. [Google Scholar]
Samà, M.; D’Ariano, A.; Corman, F.; Pacciarelli, D. Metaheuristics for efficient aircraft scheduling and re-routing at busy terminal control areas. Transp. Res. Part C Emerg. Technol. 2017, 80, 485–511. [Google Scholar] [CrossRef]
Durand, N.; Alliot, J.M. Ant colony optimization for air traffic conflict resolution. In Proceedings of the ATM Seminar 2009, 8th USA/Europe Air Traffic Management Research and Developpment Seminar, Napa, CA, USA, 29 June–2 July 2009. [Google Scholar]
Zhan, Z.H.; Zhang, J.; Li, Y.; Liu, O.; Kwok, S.K.; Ip, W.H.; Kaynak, O. An Efficient Ant Colony System Based on Receding Horizon Control for the Aircraft Arrival Sequencing and Scheduling Problem. IEEE Trans. Intell. Transp. Syst. 2010, 11, 399–412. [Google Scholar] [CrossRef]
Dell’Orco, M.; Marinelli, M.; Altieri, M.G. Solving the gate assignment problem through the Fuzzy Bee Colony Optimization. Transp. Res. Part C Emerg. Technol. 2017, 80, 424–438. [Google Scholar] [CrossRef]
Ahmed, M.S.; Alam, S.; Barlow, M. A Multi-Layer Artificial Neural Network Approach for Runway Configuration Prediction. In Proceedings of the ICRAT 2018, 8th International Conference on Research in Air Transportation, Castelldefels, Catalonia, Spain, 26–29 June 2018. [Google Scholar]
Herrema, F.F.; Treve, V.; Curran, R.; Visser, H.G. Evaluation of feasible machine learning techniques for predicting the time to fly and aircraft speed profile on final approach: Predictive dynamic support tool on final approach. In Proceedings of the ICRAT 2016, 7th Edition of the International Conference on Research in Air Transportation, Philadelphia, PA, USA, 20–24 June 2016. [Google Scholar]
Jardines, A.; Soler, M.; García-Heras Carretero, J.; Rovira, A. Neural Network based Convection Indicator for Pre-Tactical Air Traffic Flow Management. In Proceedings of the ICRAT 2020, 9th International Conference on Research in Air Transportation, Virtual Event, 15 September 2020. [Google Scholar]
Meijers, N.P. Data-Driven Predictive Analytics of Runway Occupancy Time for Improved Capacity at Airports. Ph.D. Thesis, Massachusetts Institute of Technology, Cambridge, MA, USA, 2019. [Google Scholar]
Zeh, T.; Rosenow, J.; Alligier, R.; Fricke, H. Prediction of the Propagation of Trajectory Uncertainty for Climbing Aircraft. In Proceedings of the DASC 2020 IEEE/AIAA 39th Digital Avionics Systems Conference, San Antonio, TX, USA, 11–15 October 2020. [Google Scholar] [CrossRef]
Gianazza, D. Learning Air Traffic Controller Workload from Past Sector Operations. In Proceedings of the Twelfth USA/Europe Air Traffic Management Research and Development Seminar (ATM2017), Seattle, WA, USA, 26–30 June 2017. [Google Scholar]
Helmke, H.; Kleinert, M.; Rataj, J.; Motlicek, P.; Klakow, D.; Kern, C.; Hlousek, P. Cost Reductions Enabled by Machine Learning in ATM. In Proceedings of the Thirteenth USA/Europe Air Traffic Management Research and Development Seminar (ATM2019), Vienna, Austria, 17–21 June 2019. [Google Scholar]
Herrema, F.; Treve, V.; Desart, B.; Curran, R.; Visser, D.; Delft, T. A Novel Machine Learning Model to Predict Abnormal Runway Occupancy Times and Observe Related Precursors. In Proceedings of the Twelfth USA/Europe Air Traffic Management Research and Development Seminar (ATM2017), Seattle, WA, USA, 26–30 June 2017. [Google Scholar]
Kotegawa, T.; DeLaurentis, D.A.; Sengstacken, A. Development of network restructuring models for improved air traffic forecasts. Spec. Issue Transp. Simul. 2010, 18, 937–949. [Google Scholar] [CrossRef]
Wang, Z.; Liang, M.; Delahaye, D. A hybrid machine learning model for short-term estimated time of arrival prediction in terminal manoeuvring area. Transp. Res. Part C Emerg. Technol. 2018, 95, 280–294. [Google Scholar] [CrossRef]
Verdonk Gallego, C.E.; Gómez Comendador, V.F.; Amaro Carmona, M.A.; Arnaldo Valdés, R.M.; Sáez Nieto, F.J.; García Martínez, M. A machine learning approach to air traffic interdependency modelling and its application to trajectory prediction. Transp. Res. Part C Emerg. Technol. 2019, 107, 356–386. [Google Scholar] [CrossRef]
Lambelho, M.; Mitici, M.; Pickup, S.; Marsden, A. Assessing strategic flight schedules at an airport using machine learning-based flight delay and cancellation predictions. J. Air Transp. Manag. 2020, 82, 101737. [Google Scholar] [CrossRef]
Wang, Z.; Liang, M.; Delahaye, D. Automated data-driven prediction on aircraft Estimated Time of Arrival. J. Air Transp. Manag. 2020, 88, 101840. [Google Scholar] [CrossRef]
Rajendran, S.; Srinivas, S.; Grimshaw, T. Predicting demand for air taxi urban aviation services using machine learning algorithms. J. Air Transp. Manag. 2021, 92, 102043. [Google Scholar] [CrossRef]
Khanmohammadi, S.; Tutun, S.; Kucuk, Y. A New Multilevel Input Layer Artificial Neural Network for Predicting Flight Delays at JFK Airport. Procedia Comput. Sci. 2016, 95, 237–244. [Google Scholar] [CrossRef] [Green Version]
Li Long, C.; Guleria, Y.; Alam, S. Air passenger forecasting using Neural Granger causal Google trend queries. J. Air Transp. Manag. 2021, 95, 102083. [Google Scholar] [CrossRef]
Truong, D. Estimating the impact of COVID-19 on air travel in the medium and long term using neural network and Monte Carlo simulation. J. Air Transp. Manag. 2021, 96, 102126. [Google Scholar] [CrossRef]
Ikli, S.; Mancel, C.; Mongeau, M.; Olive, X.; Rachelson, E. Coupling Mathematical Optimization and Machine Learning for the Aircraft Landing Problem. In Proceedings of the ICRAT 2020, 9th International Conference for Research in Air Transportation, ICRAT 2020, Virtual Event, 15 September 2020. [Google Scholar]
Verdonk Gallego, C.E.; Gómez Comendador, V.F.; Sáez Nieto, F.J.; Orenga Imaz, G.; Arnaldo Valdés, R.M. Analysis of air traffic control operational impact on aircraft vertical profiles supported by machine learning. Transp. Res. Part C Emerg. Technol. 2018, 95, 883–903. [Google Scholar] [CrossRef]
Jardines, A.; Soler, M.; García-Heras, J. Estimating entry counts and ATFM regulations during adverse weather conditions using machine learning. J. Air Transp. Manag. 2021, 95, 102109. [Google Scholar] [CrossRef]
Dhief, I.; Wang, Z.; Liang, M.; Alam, S.; Schultz, M.; Delahaye, D. Predicting Aircraft Landing Time in Extended-TMA Using Machine Learning Methods. In Proceedings of the ICRAT 2020, 9th International Conference for Research in Air Transportation, ICRAT 2020, Virtual Event, 15 September 2020. [Google Scholar]
Forster, S.; Schultz, M.; Fricke, H. Probabilistic Prediction of Time to Fly using Quantile Regression Forest. In Proceedings of the ICRAT 2020, 9th International Conference for Research in Air Transportation, ICRAT 2020, Virtual Event, 15 September 2020. [Google Scholar]
Förster, S.; Schultz, M.; Fricke, H. Probabilistic Prediction of Separation Buffer to Compensate for the Closing Effect on Final Approach. Aerospace 2021, 8, 29. [Google Scholar] [CrossRef]
Hernández, A.M.; Magaña, E.J.C.; Berna, A.G. Data-driven Aircraft Trajectory Predictions using Ensemble Meta-Estimators. In Proceedings of the 2018 IEEE/AIAA 37th Digital Avionics Systems Conference (DASC), London, UK, 23–27 September 2018; pp. 1–10, ISSN 2155-7209. [Google Scholar] [CrossRef]
Martín Martínez, I.; Mateos Villar, M.; García, P.; Herranz, R.; García Cantú-Ros, O.; Prats, X. Full-Scale Pre-Tactical Route Prediction Machine Learning to Increase Pre-Tactical Demand Forecast Accuracy. In Proceedings of the ICRAT 2020, 9th International Conference for Research in Air Transportation, ICRAT 2020, Virtual Event, 15 September 2020. [Google Scholar]
Nguyen, A.D.; Pham, D.T.; Lilith, N.; Alam, S. Model generalization in arrival runway occupancy time prediction by feature equivalences. In Proceedings of the ICRAT 2020, 9th International Conference for Research in Air Transportation, ICRAT 2020, Virtual Event, 15 September 2020. [Google Scholar]
Buchmüller, J.; Janetzko, H.; Andrienko, G.; Andrienko, N.; Fuchs, G.; Keim, D.A. Visual Analytics for Exploring Local Impact of Air Traffic. Comput. Graph. Forum 2015, 34, 181–190. [Google Scholar] [CrossRef] [Green Version]
Bloem, M.; Hattaway, D.; Bambos, N. Evaluation of algorithms for a miles-in-trail decision support tool. In Proceedings of the ICRAT 2012, 5th International Conference on Research in Air Transportation, Berkeley, CA, USA, 22–25 May 2012. [Google Scholar]
Rebollo, J.J.; Balakrishnan, H. A network-based model for predicting air traffic delays. In Proceedings of the ICRAT 2012, 5th International Conference on Research in Air Transportation, Berkeley, CA, USA, 22–25 May 2012; pp. 22–25. [Google Scholar]
Evans, A.; Lee, P. Using Machine-Learning to Dynamically Generate Operationally Acceptable Strategic Reroute Options. In Proceedings of the Thirteenth USA/Europe Air Traffic Management Research and Development Seminar (ATM2019), Vienna, Austria, 17–21 June 2019. [Google Scholar]
Estes, A.; Ball, M.O.; Lovell, D.J. Predicting Performance of Ground Delay Programs. In Proceedings of the Twelfth USA/Europe Air Traffic Management Research and Development Seminar (ATM2017), Seattle, WA, USA, 26–30 June 2017. [Google Scholar]
Monmousseau, P.; Marzuoli, A.; Feron, E.; Delahaye, D. Predicting and analyzing US air traffic delays using passenger-centric data-sources. In Proceedings of the Thirteenth USA/Europe Air Traffic Management Research and Development Seminar (ATM2019), Vienna, Austria, 17–21 June 2019. [Google Scholar]
Evans, A.D.; Lee, P.; Sridhar, B. Predicting the operational acceptance of airborne flight reroute requests using data mining. Transp. Res. Part C Emerg. Technol. 2018, 96, 270–289. [Google Scholar] [CrossRef] [Green Version]
Murça, M.C.R.; Hansman, R.J.; Li, L.; Ren, P. Flight trajectory data analytics for characterization of air traffic flows: A comparative analysis of terminal area operations between New York, Hong Kong and Sao Paulo. Transp. Res. Part C Emerg. Technol. 2018, 97, 324–347. [Google Scholar] [CrossRef]
Christien, R.; Favennec, B.; Pasutto, P.; Trzmiel, A.; Weiss, J.; Zeghal, K. Predicting arrival delays in the terminal area five hours in advance with machine learning. In Proceedings of the Fourteenth USA/Europe Air Traffic Management Research and Development Seminar (ATM2021), Virtual Event, 20–23 September 2021. [Google Scholar]
Smart, E.; Brown, D.; Denman, J. A Two-Phase Method of Detecting Abnormalities in Aircraft Flight Data and Ranking Their Impact on Individual Flights. IEEE Trans. Intell. Transp. Syst. 2012, 13, 1253–1265. [Google Scholar] [CrossRef]
Cao, X.; Zhu, X.; Tian, Z.; Chen, J.; Wu, D.; Du, W. A knowledge-transfer-based learning framework for airspace operation complexity evaluation. Transp. Res. Part C Emerg. Technol. 2018, 95, 61–81. [Google Scholar] [CrossRef]
Dai, L.; Hansen, M.; Ball, M.O.; Lovell, D.J. Having a Bad Day? Predicting High Delay Days in the National Airspace System. In Proceedings of the Fourteenth USA/Europe Air Traffic Management Research and Development Seminar (ATM2021), Virtual Event, 20–23 September 2021. [Google Scholar]
Shi, Z.; Xu, M.; Pan, Q. 4-D Flight Trajectory Prediction With Constrained LSTM Network. IEEE Trans. Intell. Transp. Syst. 2021, 22, 7242–7255. [Google Scholar] [CrossRef]
Hamed, M.G.; Alligier, R.; Gianazza, D. High Confidence Intervals Applied to Aircraft Altitude Prediction. IEEE Trans. Intell. Transp. Syst. 2016, 17, 2515–2527. [Google Scholar] [CrossRef] [Green Version]
Martínez, D.; Belkoura, S.; Cristobal, S.; Wachter, P.; Herrema, F. A Boosted Tree Framework for Runway Occupancy and Exit Prediction. In Proceedings of the 8th SESAR Innovation Days 2018, SID 2018, Salzburg, Austria, 3–7 September 2018. [Google Scholar]
Alligier, R.; Gianazza, D.; Durand, N. Machine Learning Applied to Airspeed Prediction During Climb. In Proceedings of the Eleventh USA/Europe Air Traffic Management Research and Development Seminar (ATM2015), Lisbon, Portugal, 23–26 June 2015. [Google Scholar]
Jones, J.C.; DeLaura, R.; Pawlak, M.; Troxel, S.; Underhill, N. Predicting & Quantifying Risk in Airport Capacity Profile Selection for Air Traffic Management. In Proceedings of the Twelfth USA/Europe Air Traffic Management Research and Development Seminar (ATM2017), Seattle, WA, USA, 26–30 June 2017. [Google Scholar]
Alligier, R.; Gianazza, D. Learning aircraft operational factors to improve aircraft climb prediction: A large scale multi-airport study. Transp. Res. Part C Emerg. Technol. 2018, 96, 72–95. [Google Scholar] [CrossRef] [Green Version]
Dalmau, R.; Ballerini, F.; Naessens, H.; Belkoura, S.; Wangnick, S. An explainable machine learning approach to improve take-off time predictions. J. Air Transp. Manag. 2021, 95, 102090. [Google Scholar] [CrossRef]
Gopalakrishnan, K.; Balakrishnan, H. A comparative analysis of models for predicting delays in air traffic networks. In Proceedings of the Twelfth USA/Europe Air Traffic Management Research and Development Seminar (ATM2017), Seattle, WA, USA, 26–30 June 2017. [Google Scholar]
Ciruelos, C.; Arranz, A.; Etxebarria, I.; Peces, S.; Campanelli, B.; Fleurquin, P.; Eguiluz, V.M.; Ramasco, J.J. Modelling delay propagation trees for scheduled flights. In Proceedings of the 11th USA/EUROPE Air Traffic Management R&D Seminar, Lisbon, Portugal, 23–26 June 2015; pp. 23–26. [Google Scholar]
Zhang, Y.; Nayak, N.; Diana, T. Estimation and Comparison of the Impact of Single Airport Delay to the National Airspace System Using Multivariate Simultaneous Models. In Proceedings of the ICRAT 2010, 4th International Conference on Research in Air Transportation, Budapest, Hungary, 1–4 June 2010. [Google Scholar]
Hrastovec, M.; Solina, F. Machine Learning Model for Aircraft Performances. In Proceedings of the 33rd Digital Avionics Systems Conference (DASC), Colorado Springs, CO, USA, 5–9 October 2014. [Google Scholar] [CrossRef] [Green Version]
Alligier, R.; Gianazza, D.; Durand, N. Learning the aircraft mass and thrust to improve the ground-based trajectory prediction of climbing flights. Transp. Res. Part C Emerg. Technol. 2013, 36, 45–60. [Google Scholar] [CrossRef] [Green Version]
Tastambekov, K.; Puechmorel, S.; Delahaye, D.; Rabut, C. Aircraft trajectory forecasting using local functional regression in Sobolev space. Transp. Res. Part C Emerg. Technol. 2014, 39, 1–22. [Google Scholar] [CrossRef] [Green Version]
Waltert, M.; Wicki, J.; Jimenez Perez, E.; Pagliari, R. Ratio-based design hour determination for airport passenger terminal facilities. J. Air Transp. Manag. 2021, 96, 102125. [Google Scholar] [CrossRef]
Stell, L.; Bronsvoort, J.; McDonald, G. Regression Analysis of Top of Descent Location for Idle-thrust Descents. In Proceedings of the ATM 2013, 10th USA/Europe Air Traffic Management Research and Development Seminar, Chicago, IL, USA, 10–13 June 2013. [Google Scholar]
Lemetti, A.; Polishchuk, T.; Polishchuk, V.; Sáez, R.; Prats, X. Identification of Significant Impact Factors on Arrival Flight Efficiency within TMA. In Proceedings of the ICRAT 2020, 9th International Conference for Research in Air Transportation, ICRAT 2020, Virtual Event, 15 September 2020. [Google Scholar]
Zhang, Y.; Chauhan, A.; Chen, X. Analyzing and Decomposing Taxi Times and Predicting Taxi out Times. In Proceedings of the ICRAT 2010, 4th International Conference on Research in Air Transportation, Budapest, Hungary, 1–4 June 2010. [Google Scholar]
Pien, K.C. A Linear Programming Approach for Capacity Estimation and Robustness Analysis of the European Air Traffic Network; Imperial College London: London, UK, 2015. [Google Scholar]
Pien, K.C.; Majumdar, A.; Han, K.; Ochieng, W.Y. A linear programming approach to maximum flow estimation on the European air traffic network. In Proceedings of the ICRAT 2014, 6th International Conference on Research in Air Transportation, Istanbul, Turkey, 26–20 May 2014. [Google Scholar]
Sun, X.; Wandelt, S.; Linke, F. Topological properties of the air navigation route system using complex network theory. In Proceedings of the ICRAT 2014, 6th International Conference on Research in Air Transportation, Istanbul, Turkey, 26–20 May 2014. [Google Scholar]
Liu, Y.; Hansen, M.; Lovell, D.J.; Chuang, C.; Ball, M.O.; Gulding, J.M. Causal Analysis of En Route Flight Inefficiency—The US Experience. In Proceedings of the Twelfth USA/Europe Air Traffic Management Research and Development Seminar (ATM2017), Seattle, WA, USA, 26–30 June 2017. [Google Scholar]
Liu, Y.; Hansen, M.; Gupta, G.; Malik, W.; Jung, Y. Predictability impacts of airport surface automation. Transp. Res. Part C Emerg. Technol. 2014, 44, 128–145. [Google Scholar] [CrossRef]
Vogel, M.; Schelbert, K.; Fricke, H.; Kistan, T.; Australia, T. Analysis of Airspace Complexity Factors’ Capability to Predict Workload and Safety Levels in the TMA. In Proceedings of the ATM 2013, 10th USA/Europe Air Traffic Management Research and Development Seminar, Chicago, IL, USA, 10–13 June 2013. [Google Scholar]
Dougui, N.; Delahaye, D.; Puechmorel, S.; Mongeau, M. A new method for generating optimal conflict free 4D trajectory. In Proceedings of the ICRAT 2010, 4th International Conference on Research in Air Transportation, Budapest, Hungary, 1–4 June 2010. [Google Scholar]
Tastambekov, K.; Puechmorel, S.; Delahaye, D.; Rabut, C. Trajectory prediction by functional regression in sobolev space. In Proceedings of the SFDS 2010, 42èmes Journées de Statistique, Marseille, France, 25–29 May 2010. [Google Scholar]
Ribeiro, M.J.; Ellerbroek, J.; Hoekstra, J.M. Improvement of Conflict Detection and Resolution at High Densities Through Reinforcement Learning. In Proceedings of the ICRAT 2020, 9th International Conference on Research in Air Transportation, Virtual Event, 15 September 2020. [Google Scholar]
Pham, D.T.; Tran, N.P.; Alam, S.; Duong, V.; Delahaye, D. A Machine Learning Approach for Conflict Resolution in Dense Traffic Scenarios with Uncertainties. In Proceedings of the Thirteenth USA/Europe Air Traffic Management Research and Development Seminar (ATM2019), Vienna, Austria, 17–21 June 2019. [Google Scholar]
Wang, Y.; Zhang, Y. Prediction of runway configurations and airport acceptance rates for multi-airport system using gridded weather forecast. Transp. Res. Part C Emerg. Technol. 2021, 125, 103049. [Google Scholar] [CrossRef]
Yang, Z.; Tang, R.; Zeng, W.; Lu, J.; Zhang, Z. Short-term prediction of airway congestion index using machine learning methods. Transp. Res. Part C Emerg. Technol. 2021, 125, 103040. [Google Scholar] [CrossRef]
Schultz, M.; Reitmann, S.; Alam, S. Predictive classification and understanding of weather impact on airport performance through machine learning. Transp. Res. Part C Emerg. Technol. 2021, 131, 103119. [Google Scholar] [CrossRef]
Helmke, H.; Kleinert, M.; Shetty, S.; Ohneiser, O.; Ehr, H.; Arilíusson, H.; Simiganoschi, T.S.; Prasad, A.; Motlicek, P.; Veselý, K.; et al. Readback Error Detection by Automatic Speech Recognition to Increase ATM Safety. In Proceedings of the Fourteenth USA/Europe Air Traffic Management Research and Development Seminar (ATM2021), Virtual Event, 20–23 September 2021. [Google Scholar]
Brittain, M.; Wei, P. Autonomous Aircraft Sequencing and Separation with Hierarchical Deep Reinforcement Learning. In Proceedings of the ICRAT 2018, 8th International Conference on Research in Air Transportation, Castelldefels, Catalonia, Spain, 26–29 June 2018. [Google Scholar]
Psyllou, E.; Majumdar, A.; Ochieng, W. Analysis of Airspace Infringements in European Airspace. In Proceedings of the ICRAT 2014, 6th International Conference on Research in Air Transportation, Istanbul, Turkey, 26–20 May 2014. [Google Scholar]
Kaya, Y.B.; Inalhan, G. A Dynamic Bayesian Belief Network Approach for Modelling the ATM Network Delays. In Proceedings of the ICRAT 2014, 6th International Conference on Research in Air Transportation, Istanbul, Turkey, 26–20 May 2014; pp. 26–30. [Google Scholar]
Paek, H.; Lee, K.; Vela, A.E. En-Route Arrival Time Prediction Using Gaussian Mixture Model. In Proceedings of the ICRAT 2020, 9th International Conference for Research in Air Transportation, ICRAT 2020, Virtual Event, 15 September 2020. [Google Scholar]
Li, L.; Hansman, R.J.; Palacios, R.; Welsch, R. Anomaly detection via a Gaussian Mixture Model for flight operation and safety monitoring. Transp. Res. Part C Emerg. Technol. 2016, 64, 45–57. [Google Scholar] [CrossRef]
Zhou, H.; Kong, H.; Wei, L.; Creighton, D.; Nahavandi, S. Efficient Road Detection and Tracking for Unmanned Aerial Vehicle. IEEE Trans. Intell. Transp. Syst. 2015, 16, 297–309. [Google Scholar] [CrossRef]
Barratt, S.T.; Kochenderfer, M.J.; Boyd, S.P. Learning Probabilistic Trajectory Models of Aircraft in Terminal Airspace From Position Data. IEEE Trans. Intell. Transp. Syst. 2019, 20, 3536–3545. [Google Scholar] [CrossRef] [Green Version]
Choi, H.C.; Deng, C.; Hwang, I. Hybrid Machine Learning and Estimation-Based Flight Trajectory Prediction in Terminal Airspace. IEEE Access 2021, 9, 151186–151197. [Google Scholar] [CrossRef]
Murça, M.C.R. Identification and prediction of urban airspace availability for emerging air mobility operations. Transp. Res. Part C Emerg. Technol. 2021, 131, 103274. [Google Scholar] [CrossRef]
Bonifazi, A. Modeling and Detecting Anomalous Safety Events in Approach Flights Using ADS-B Data. Master’s Thesis, Delft University of Technology, Delft, The Netherlands, 2020. [Google Scholar]
Bonifazi, A.; Sun, J.; van Baren, G.; Hoekstra, J. Modeling and Detecting Anomalous Safety Events in Approach Flights Using ADS-B Data. In Proceedings of the Fourteenth USA/Europe Air Traffic Management Research and Development Seminar (ATM2021), Virtual Event, 20–23 September 2021. [Google Scholar]
Marinescu, M.; Olivares, A.; Staffetti, E.; Sun, J. Wind Profile Estimation from Aircraft Derived Data Using Kalman Filters and Gaussian Process Regression. In Proceedings of the Fourteenth USA/Europe Air Traffic Management Research and Development Seminar (ATM2021), Virtual Event, 20–23 September 2021. [Google Scholar]
Olive, X.; Basora, L. Detection and identification of significant events in historical aircraft trajectory data. Transp. Res. Part C Emerg. Technol. 2020, 119, 102737. [Google Scholar] [CrossRef]
Sun, J.; Ellerbroek, J.; Hoekstra, J. Large-scale flight phase identification from ads-b data using machine learning methods. In Proceedings of the ICRAT 2016, 7th Edition of the International Conference on Research in Air Transportation, Philadelphia, PA, USA, 20–24 June 2016. [Google Scholar]
Prats, X.; Puig, V.; Quevedo, J.; Nejjari, F. Multi-objective optimisation for aircraft departure trajectories minimising noise annoyance. Spec. Issue Transp. Simul. 2010, 18, 975–989. [Google Scholar] [CrossRef]
Brownlee, A.E.; Weiszer, M.; Chen, J.; Ravizza, S.; Woodward, J.R.; Burke, E.K. A fuzzy approach to addressing uncertainty in Airport Ground Movement optimisation. Transp. Res. Part C Emerg. Technol. 2018, 92, 150–175. [Google Scholar] [CrossRef]
Havle, C.A.; Kılıç, B. A hybrid approach based on the fuzzy AHP and HFACS framework for identifying and analyzing gross navigation errors during transatlantic flights. J. Air Transp. Manag. 2019, 76, 21–30. [Google Scholar] [CrossRef]
Barak, S.; Dahooei, J.H. A novel hybrid fuzzy DEA-Fuzzy MADM method for airlines safety evaluation. J. Air Transp. Manag. 2018, 73, 134–149. [Google Scholar] [CrossRef] [Green Version]
Loh, H.S.; Yuen, K.F.; Wang, X.; Surucu-Balci, E.; Balci, G.; Zhou, Q. Airport selection criteria of low-cost carriers: A fuzzy analytical hierarchy process. J. Air Transp. Manag. 2020, 83, 101759. [Google Scholar] [CrossRef]
Chen, L.; Ren, J. Multi-attribute sustainability evaluation of alternative aviation fuels based on fuzzy ANP and fuzzy grey relational analysis. J. Air Transp. Manag. 2018, 68, 176–186. [Google Scholar] [CrossRef]
Büyüközkan, G.; Havle, C.A.; Feyzioğlu, O. An integrated SWOT based fuzzy AHP and fuzzy MARCOS methodology for digital transformation strategy analysis in airline industry. J. Air Transp. Manag. 2021, 97, 102142. [Google Scholar] [CrossRef]
de Oliveira, M.; Eufrásio, A.B.R.; Guterres, M.X.; Murça, M.C.R.; Gomes, R.d.A. Analysis of airport weather impact on on-time performance of arrival flights for the Brazilian domestic air transportation system. J. Air Transp. Manag. 2021, 91, 101974. [Google Scholar] [CrossRef]
Barry, D.J. Estimating runway veer-off risk using a Bayesian network with flight data. Transp. Res. Part C Emerg. Technol. 2021, 128, 103180. [Google Scholar] [CrossRef]
Truong, D. Using causal machine learning for predicting the risk of flight delays in air transportation. J. Air Transp. Manag. 2021, 91, 101993. [Google Scholar] [CrossRef]
Li, Q.; Jing, R. Characterization of delay propagation in the air traffic network. J. Air Transp. Manag. 2021, 94, 102075. [Google Scholar] [CrossRef]
Londner, E.H.; Moss, R.J. A Bayesian Network Model of Pilot Response to TCAS Resolution Advisories. In Proceedings of the Twelfth USA/Europe Air Traffic Management Research and Development Seminar (ATM2017), Seattle, WA, USA, 26–30 June 2017. [Google Scholar]
Sun, J.; Blom, H.A.P.; Ellerbroek, J.; Hoekstra, J.M. Aircraft mass and thrust estimation using recursive bayesian method. In Proceedings of the ICRAT 2018, 8th International Conference on Research in Air Transportation, Castelldefels, Catalonia, Spain, 26–29 June 2018. [Google Scholar]
Taheri, H.; Lyhyaoui, A. Airport Ground Traffic Optimisation. In Proceedings of the ICRAT 2018, 8th International Conference on Research in Air Transportation, Castelldefels, Catalonia, Spain, 26–29 June 2018. [Google Scholar]
Xia, Q.; Li, X.; Song, Y.; Zhang, B. Aircraft grouping based on improved divisive hierarchical clustering algorithm. J. Air Transp. Manag. 2014, 40, 157–162. [Google Scholar] [CrossRef]
Hrastovec, M.; Solina, F. Prediction of aircraft performances based on data collected by air traffic control centers. Transp. Res. Part C Emerg. Technol. 2016, 73, 167–182. [Google Scholar] [CrossRef]
Amaro Carmona, M.A.; Saez Nieto, F.; Verdonk Gallego, C.E. A data-driven methodology for characterization of a terminal manoeuvring area in multi-airport systems. Transp. Res. Part C Emerg. Technol. 2020, 111, 185–209. [Google Scholar] [CrossRef]
Deshmukh, R.; Sun, D.; Kim, K.; Hwang, I. Temporal logic learning-based anomaly detection in metroplex terminal airspace operations. Transp. Res. Part C Emerg. Technol. 2021, 126, 103036. [Google Scholar] [CrossRef]
Lindner, M.; Zeh, T.; Braßel, H.; Fricke, H. Aircraft Performance-optimized Departure Flights Using Traffic Flow Funnels. In Proceedings of the Fourteenth USA/Europe Air Traffic Management Research and Development Seminar (ATM2021), Virtual Event, 20–23 September 2021. [Google Scholar]
Gariel, M.; Srivastava, A.N.; Feron, E. Trajectory Clustering and an Application to Airspace Monitoring. IEEE Trans. Intell. Transp. Syst. 2011, 12, 1511–1524. [Google Scholar] [CrossRef] [Green Version]
Ren, P.; Li, L. Characterizing air traffic networks via large-scale aircraft tracking data: A comparison between China and the US networks. J. Air Transp. Manag. 2018, 67, 181–196. [Google Scholar] [CrossRef]
Lin, Y.; Li, L.; Ren, P.; Wang, Y.; Szeto, W.Y. From aircraft tracking data to network delay model: A data-driven approach considering en-route congestion. Transp. Res. Part C Emerg. Technol. 2021, 131, 103329. [Google Scholar] [CrossRef]
Mitkas, D.Z.; Lovell, D.J.; Venkatesh, S.; Young, S.B. Leveraging local ADS-B transmissions to assess the performance of air traffic at general aviation airports. In Proceedings of the Fourteenth USA/Europe Air Traffic Management Research and Development Seminar (ATM2021), Virtual Event, 20–23 September 2021. [Google Scholar]
Dang, P.H.; Tran, P.N.; Alam, S.; Duong, V.N. A machine learning-based framework for aircraft maneuver detection and classification. In Proceedings of the Fourteenth USA/Europe Air Traffic Management Research and Development Seminar (ATM2021), Virtual Event, 20–23 September 2021. [Google Scholar]
Buxi, G.S.; Hansen, M. Generating day-of-operation probabilistic capacity profiles from weather forecasts. In Proceedings of the ICRAT 2010, 4th International Conference on Research in Air Transportation, Budapest, Hungary, 1–4 June 2010. [Google Scholar]
Rebollo, J.J.; Balakrishnan, H. Characterization and prediction of air traffic delays. Transp. Res. Part C Emerg. Technol. 2014, 44, 231–241. [Google Scholar] [CrossRef]
Takeichi, N.; Hashimoto, R. Modeling of Flight Time Prediction Uncertainty for Four-Dimensional Descent Trajectory Management. In Proceedings of the Fourteenth USA/Europe Air Traffic Management Research and Development Seminar (ATM2021), Virtual Event, 20–23 September 2021. [Google Scholar]
Nguyen, M.; Alam, S. Airspace Collision Risk Hot-Spot Identification using Clustering Models. IEEE Trans. Intell. Transp. Syst. 2018, 19, 48–57. [Google Scholar] [CrossRef]
Bulusu, V.; Onat, E.B.; Sengupta, R.; Yedavalli, P.; Macfarlane, J. A Traffic Demand Analysis Method for Urban Air Mobility. IEEE Trans. Intell. Transp. Syst. 2021, 22, 6039–6047. [Google Scholar] [CrossRef]
Buxi, G.; Hansen, M. Generating day-of-operation probabilistic capacity scenarios from weather forecasts. Transp. Res. Part C Emerg. Technol. 2013, 33, 153–166. [Google Scholar] [CrossRef] [Green Version]
Badrinath, S.; Abel, J.; Balakrishnan, H.; Joback, E.; Reynolds, T. Spatial Modeling of Airport Surface Fuel Burn for Environmental Impact Analyses. In Proceedings of the Fourteenth USA/Europe Air Traffic Management Research and Development Seminar (ATM2021), Virtual Event, 20–23 September 2021. [Google Scholar]
Enriquez, M. Classification of Air Traffic Controller Utterance Transcripts via Warm-Start Non-Negative Matrix Factorization. In Proceedings of the ICRAT 2014, 6th International Conference on Research in Air Transportation, Istanbul, Turkey, 26–20 May 2014. [Google Scholar] [CrossRef]
Lesire, C. Iterative planning of airport ground movements. In Proceedings of the ICRAT 2010, 4th International Conference on Research in Air Transportation, Budapest, Hungary, 1–4 June 2010. [Google Scholar]
Gianazza, D. Optimizing Successive Airspace Configurations with a Sequential A∗ Algorithm. In Proceedings of the Thirteenth USA/Europe Air Traffic Management Research and Development Seminar (ATM2019), Vienna, Austria, 17–21 June 2019. [Google Scholar]
Zhang, N.; Zhang, M.; Low, K.H. 3D path planning and real-time collision resolution of multirotor drone operations in complex urban low-altitude airspace. Transp. Res. Part C Emerg. Technol. 2021, 129, 103123. [Google Scholar] [CrossRef]
Balakrishna, P.; Ganesan, R.; Sherry, L. Accuracy of reinforcement learning algorithms for predicting aircraft taxi-out times: A case-study of Tampa Bay departures. Spec. Issue Transp. Simul. 2010, 18, 950–962. [Google Scholar] [CrossRef]
Cruciol, L.L.; de Arruda, A.C.; Weigang, L.; Li, L.; Crespo, A.M. Reward functions for learning to control in air traffic flow management. Transp. Res. Part C Emerg. Technol. 2013, 35, 141–155. [Google Scholar] [CrossRef]
Jones, J.; Ellenbogen, Z.; Glina, Y. Recommending Strategic Air Traffic Management Initiatives in Convective Weather. In Proceedings of the Fourteenth USA/Europe Air Traffic Management Research and Development Seminar (ATM2021), Virtual Event, 20–23 September 2021. [Google Scholar]
Taylor, C.; Vargo, E.; Bromberg, E.; Carson, E. Reinforcement Learning for Traffic Flow Management Decision Support. In Proceedings of the Fourteenth USA/Europe Air Traffic Management Research and Development Seminar (ATM2021), Virtual Event, 20–23 September 2021. [Google Scholar]
Al-Hilo, A.; Samir, M.; Assi, C.; Sharafeddine, S.; Ebrahimi, D. UAV-Assisted Content Delivery in Intelligent Transportation Systems-Joint Trajectory Planning and Cache Management. IEEE Trans. Intell. Transp. Syst. 2021, 22, 5155–5167. [Google Scholar] [CrossRef]
Abedin, S.F.; Munir, M.S.; Tran, N.H.; Han, Z.; Hong, C.S. Data Freshness and Energy-Efficient UAV Navigation Optimization: A Deep Reinforcement Learning Approach. IEEE Trans. Intell. Transp. Syst. 2021, 22, 5994–6006. [Google Scholar] [CrossRef]
Pham, D.T.; Chan, L.L.; Alam, S.; Koelle, R. Real-time departure slotting in mixed-mode operations using deep reinforcement learning: A case study of Zurich airport. In Proceedings of the Fourteenth USA/Europe Air Traffic Management Research and Development Seminar (ATM2021), Virtual Event, 20–23 September 2021. [Google Scholar]
Schultz, M.; Reitmann, S. Machine learning approach to predict aircraft boarding. Transp. Res. Part C Emerg. Technol. 2019, 98, 391–408. [Google Scholar] [CrossRef]
Zeng, W.; Li, J.; Quan, Z.; Lu, X. A Deep Graph-Embedded LSTM Neural Network Approach for Airport Delay Prediction. J. Adv. Transp. 2021. [Google Scholar] [CrossRef]
Kim, Y.J.; Choi, S.; Briceno, S.; Mavris, D. A deep learning approach to flight delay prediction. In Proceedings of the 2016 IEEE/AIAA 35th Digital Avionics Systems Conference (DASC), Sacramento, CA, USA, 25–30 September 2016; pp. 1–6, ISSN 2155-7209. [Google Scholar] [CrossRef]
Zeng, W.; Quan, Z.; Zhao, Z.; Xie, C.; Lu, X. A Deep Learning Approach for Aircraft Trajectory Prediction in Terminal Airspace. IEEE Access 2020, 8, 151250–151266. [Google Scholar] [CrossRef]
Zhu, X.; Li, L. Flight time prediction for fuel loading decisions with a deep learning approach. Transp. Res. Part C Emerg. Technol. 2021, 128, 103179. [Google Scholar] [CrossRef]
Dalmau-Codina, R.; Genestier, B.; Anoraud, C.; Choroba, P.; Smith, D. A Machine Learning Approach to Predict the Evolution of Air Traffic Flow Management Delay. In Proceedings of the ATM Seminar 2021, 14th USA/Europe Air Traffic Management Research and Developpment Seminar, Virtual Event, 20–23 September 2021. [Google Scholar]
Shi-Garrier, L.; Delahaye, D.; Bouaynaya, N. Predicting Air Traffic Congested Areas with Long Short-Term Memory Networks. In Proceedings of the Fourteenth USA/Europe Air Traffic Management Research and Development Seminar (ATM2021), Virtual Event, 20–23 September 2021. [Google Scholar]
Dalmau-Codina, R.; Murgese, G.; Wandeler, Y.; Correira, R.; Marsden, A. Early Detection of Night Curfew Infringements by Delay Propagation with Neural Networks. In Proceedings of the Fourteenth USA/Europe Air Traffic Management Research and Development Seminar (ATM2021), Virtual Event, 20–23 September 2021. [Google Scholar]
Choi, S.; Kim, Y.J. Artificial neural network models for airport capacity prediction. J. Air Transp. Manag. 2021, 97, 102146. [Google Scholar] [CrossRef]
Lunacek, M.; Williams, L.; Severino, J.; Ficenec, K.; Ugirumurera, J.; Eash, M.; Ge, Y.; Phillips, C. A data-driven operational model for traffic at the Dallas Fort Worth International Airport. J. Air Transp. Manag. 2021, 94, 102061. [Google Scholar] [CrossRef]
Güvercin, M.; Ferhatosmanoglu, N.; Gedik, B. Forecasting Flight Delays Using Clustered Models Based on Airport Networks. IEEE Trans. Intell. Transp. Syst. 2021, 22, 3179–3189. [Google Scholar] [CrossRef]
Mahboubi, Z.; Kochenderfer, M.J. Learning Traffic Patterns at Small Airports From Flight Tracks. IEEE Trans. Intell. Transp. Syst. 2017, 18, 917–926. [Google Scholar] [CrossRef]
Mueller, E.R.; Kochenderfer, M. Multi-Rotor Aircraft Collision Avoidance using Partially Observable Markov Decision Processes. In Proceedings of the AIAA Modeling and Simulation Technologies Conference, American Institute of Aeronautics and Astronautics, Washington, DC, USA, 13–17 June 2016. [Google Scholar] [CrossRef] [Green Version]
Balakrishna, P.; Ganesan, R.; Sherry, L. Accuracy of Reinforcement Learning Algorithms for Predicting Aircraft Taxi-out Times. In Proceedings of the ICRAT 2008, 3rd International Conference on Research in Air Transportation, Fairfax, VA, USA, 1–4 June 2008. [Google Scholar]
Ohneiser, O.; Helmke, H.; Shetty, S.; Kleinert, M.; Ehr, H.; Šarūnas, M.; Pagirys, T. Prediction and extraction of tower controller commands for speech recognition applications. J. Air Transp. Manag. 2021, 95, 102089. [Google Scholar] [CrossRef]
Kleinert, M.; Helmke, H.; Siol, G.; Ehr, H.; Finke, M.; Oualil, Y.; Srinivasamurthy, A. Proceedings of the 7th SESAR Innovation Days, SID 2017, Belgrade, Serbia, 28–30 November 2017.
Hurter, C.; Conversy, S.; Gianazza, D.; Telea, A.C. Interactive image-based information visualization for aircraft trajectory analysis. Transp. Res. Part C Emerg. Technol. 2014, 47, 207–227. [Google Scholar] [CrossRef]
Zúñiga, C.; Piera, M.; Ruiz, S.; Del Pozo, I. A CD&CR causal model based on path shortening/path stretching techniques. Transp. Res. Part C Emerg. Technol. 2013, 33, 238–256. [Google Scholar]
Barredo Arrieta, A.; Díaz-Rodríguez, N.; Del Ser, J.; Bennetot, A.; Tabik, S.; Barbado, A.; Garcia, S.; Gil-Lopez, S.; Molina, D.; Benjamins, R.; et al. Explainable Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI. Inf. Fusion 2020, 58, 82–115. [Google Scholar] [CrossRef] [Green Version]
Lipton, Z.C. The Mythos of Model Interpretability. Queue 2018, 16, 31–57. [Google Scholar] [CrossRef]
Guidotti, R.; Monreale, A.; Ruggieri, S.; Turini, F.; Giannotti, F.; Pedreschi, D. A Survey of Methods for Explaining Black Box Models. ACM Comput. Surv. 2018, 51, 1–42. [Google Scholar] [CrossRef] [Green Version]
Vilone, G.; Longo, L. Explainable Artificial Intelligence: A Systematic Review. arXiv 2006, arXiv:2006.00093. [Google Scholar]
Glikson, E.; Woolley, A.W. Human trust in artificial intelligence: Review of empirical research. Acad. Manag. Ann. 2020, 14, 627–660. [Google Scholar] [CrossRef]
Keil, F.C. Explanation and understanding. Annu. Rev. Psychol. 2006, 57, 227–254. [Google Scholar] [CrossRef]
Kahneman, D. Thinking, Fast and Slow; Macmillan: London, UK, 2011; ISBN 0141033576. [Google Scholar]
Gigerenzer, G.; Brighton, H. Homo heuristicus: Why biased minds make better inferences. Top. Cogn. Sci. 2009, 1, 107–143. [Google Scholar] [CrossRef] [Green Version]
Dawes, R.M.; Faust, D.; Meehl, P.E. Clinical versus actuarial judgment. Science 1989, 243, 1668–1674. [Google Scholar] [CrossRef]
Ribeiro, M.T.; Singh, S.; Guestrin, C. “Why Should I Trust You?” Explaining the Predictions of Any Classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, 13–17 August 2016; ACM: New York, NY, USA, 2016; pp. 1135–1144. [Google Scholar] [CrossRef]
Lundberg, S.M.; Lee, S.I. A Unified Approach to Interpreting Model Predictions. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, Long Beach, CA, USA, 4–9 December 2017; pp. 4766–4775. [Google Scholar]
Serradilla, O.; Zugasti, E.; Cernuda, C.; Aranburu, A.; de Okariz, J.R.; Zurutuza, U. Interpreting Remaining Useful Life Estimations Combining Explainable Artificial Intelligence and Domain Knowledge in Industrial Machinery. In Proceedings of the 2020 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), Virtual Event, 20–24 July 2020; Volume 2020, pp. 1–8. [Google Scholar] [CrossRef]
Chen, H.Y.; Lee, C.H. Vibration Signals Analysis by Explainable Artificial Intelligence (XAI) Approach: Application on Bearing Faults Diagnosis. IEEE Access 2020, 8, 134246–134256. [Google Scholar] [CrossRef]
Anguita-Ruiz, A.; Segura-Delgado, A.; Alcalá, R.; Aguilera, C.M.; Alcalá-Fdez, J. eXplainable Artificial Intelligence (XAI) for the Identification of Biologically Relevant Gene Expression Patterns in Longitudinal Human Studies, Insights from Obesity Research. PLoS Comput. Biol. 2020, 16, e1007792. [Google Scholar] [CrossRef]
Palatnik de Sousa, I.; Maria Bernardes Rebuzzi Vellasco, M.; Costa da Silva, E. Local Interpretable Model-Agnostic Explanations for Classification of Lymph Node Metastases. Sensors 2019, 19, 2969. [Google Scholar] [CrossRef] [PubMed] [Green Version]
Dindorf, C.; Teufl, W.; Taetz, B.; Bleser, G.; Fröhlich, M. Interpretability of Input Representations for Gait Classification in Patients after Total Hip Arthroplasty. Sensors 2020, 20, 4385. [Google Scholar] [CrossRef] [PubMed]
Ponn, T.; Kröger, T.; Diermeyer, F. Identification and Explanation of Challenging Conditions for Camera-Based Object Detection of Automated Vehicles. Sensors 2020, 20, 3699. [Google Scholar] [CrossRef] [PubMed]
Bonanno, D.; Nock, K.; Smith, L.; Elmore, P.; Petry, F. An Apporach to Explainable Deep Learning using Fuzzy Inference. In Next-Generation Analyst V; Hanratty, T.P., Llinas, J., Eds.; SPIE: Bellingham, WA, USA, 2017; Volume 10207. [Google Scholar] [CrossRef]
Aghamohammadi, M.; Madan, M.; Hong, J.K.; Watson, I. Predicting Heart Attack Through Explainable Artificial Intelligence. In Computational Science—ICCS 2019; Rodrigues, J.M.F., Cardoso, P.J.S., Monteiro, J., Lam, R., Krzhizhanovskaya, V.V., Lees, M.H., Dongarra, J.J., Sloot, P.M., Eds.; Springer International Publishing: Berlin/Heidelberg, Germany, 2019; Volume 11537. [Google Scholar] [CrossRef]
Keneni, B.M.; Kaur, D.; Al Bataineh, A.; Devabhaktuni, V.K.; Javaid, A.Y.; Zaientz, J.D.; Marinier, R.P. Evolving Rule-Based Explainable Artificial Intelligence for Unmanned Aerial Vehicles. IEEE Access 2019, 7, 17001–17016. [Google Scholar] [CrossRef]
Ribeiro, M.T.; Singh, S.; Guestrin, C. Anchors: High-precision Model-agnostic Explanations. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence, AAAI 2018, New Orleans, LA, USA, 2–7 February 2018; pp. 1527–1535. [Google Scholar]
Anysz, H.; Brzozowski, Ł.; Kretowicz, W.; Narloch, P. Feature Importance of Stabilised Rammed Earth Components Affecting the Compressive Strength Calculated with Explainable Artificial Intelligence Tools. Materials 2020, 13, 2317. [Google Scholar] [CrossRef] [PubMed]
Pintelas, E.; Liaskos, M.; Livieris, I.E.; Kotsiantis, S.; Pintelas, P. Explainable Machine Learning Framework for Image Classification Problems: Case Study on Glioma Cancer Prediction. J. Imaging 2020, 6, 37. [Google Scholar] [CrossRef]
Rehse, J.R.; Mehdiyev, N.; Fettke, P. Towards Explainable Process Predictions for Industry 4.0 in the DFKI-Smart-Lego-Factory. Künstliche Intell. 2019, 33, 181–187. [Google Scholar] [CrossRef]
Štrumbelj, E.; Kononenko, I. Explaining Prediction Models and Individual Predictions with Feature Contributions. Knowl. Inf. Syst. 2014, 41, 647–665. [Google Scholar] [CrossRef]
Kwon, B.C.; Choi, M.J.; Kim, J.T.; Choi, E.; Kim, Y.B.; Kwon, S.; Sun, J.; Choo, J. RetainVis: Visual Analytics with Interpretable and Interactive Recurrent Neural Networks on Electronic Medical Records. IEEE Trans. Vis. Comput. Graph. 2019, 25, 299–309. [Google Scholar] [CrossRef] [Green Version]
Kovalev, M.S.; Utkin, L.V. A Robust Algorithm for Explaining Unreliable Machine Learning Survival Models using Kolmogorov-Smirnov Bounds. Neural Netw. 2020, 132, 1–18. [Google Scholar] [CrossRef]
Ferreyra, E.; Hagras, H.; Kern, M.; Owusu, G. Depicting Decision-Making: A Type-2 Fuzzy Logic Based Explainable Artificial Intelligence System for Goal-Driven Simulation in the Workforce Allocation Domain. In Proceedings of the 2019 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), New Orleans, LA, USA, 23–26 June 2019; Volume 2019, pp. 1–6. [Google Scholar] [CrossRef]
Moradi, M.; Samwald, M. Post-hoc Explanation of Black-box Classifiers using Confident Itemsets. Expert Syst. Appl. 2021, 165, 113941. [Google Scholar] [CrossRef]
Yang, Z.; Zhang, A.; Sudjianto, A. Enhancing Explainability of Neural Networks Through Architecture Constraints. IEEE Trans. Neural Netw. Learn. Syst. 2020, 32, 2610–2621. [Google Scholar] [CrossRef] [PubMed]
Ten Zeldam, S.; De Jong, A.; Loendersloot, R.; Tinga, T.; ten Zeldam, S.; de Jong, A.; Loendersloot, R.; Tinga, T. Automated Failure Diagnosis in Aviation Maintenance Using Explainable Artificial Intelligence (XAI). In Proceedings of the European Conference of the PHM Society 2018, Utrecht, The Netherlands, 3–6 July 2018; pp. 1–11. [Google Scholar]
Plumb, G.; Molitor, D.; Talwalkar, A. Model Agnostic Supervised Local Explanations. In Proceedings of the 32nd International Conference on Neural Information Processing Systems (NIPS ’18), Montréal, QC, Canada, 3–8 December 2018; pp. 2520–2529. [Google Scholar]
Zhao, G.; Fu, H.; Song, R.; Sakai, T.; Chen, Z.; Xie, X.; Qian, X. Personalized Reason Generation for Explainable Song Recommendation. ACM Trans. Intell. Syst. Technol. 2019, 10, 1–21. [Google Scholar] [CrossRef]
Le, T.; Wang, S.; Lee, D. GRACE: Generating Concise and Informative Contrastive Sample to Explain Neural Network Model’s Prediction. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Virtual Event, 6–10 July 2020. [Google Scholar] [CrossRef]
Magdalena, L. Designing Interpretable Hierarchical Fuzzy Systems. In Proceedings of the 2018 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), Rio de Janeiro, Brazil, 8–13 July 2018; Volume 2018, pp. 1–8. [Google Scholar] [CrossRef]
Lauritsen, S.M.; Kristensen, M.; Olsen, M.V.; Larsen, M.S.; Lauritsen, K.M.; Jørgensen, M.J.; Lange, J.; Thiesson, B. Explainable Artificial Intelligence Model to Predict Acute Critical Illness from Electronic Health Records. Nat. Commun. 2020, 11, 1–11. [Google Scholar] [CrossRef] [PubMed]
Alonso, J.M.; Toja-Alamancos, J.; Bugarin, A. Experimental Study on Generating Multi-modal Explanations of Black-box Classifiers in terms of Gray-box Classifiers. In Proceedings of the 2020 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), Virtual Event, 20–24 July 2020; Volume 2020, pp. 1–8. [Google Scholar] [CrossRef]
Lindsay, L.; Coleman, S.; Kerr, D.; Taylor, B.; Moorhead, A. Explainable Artificial Intelligence for Falls Prediction. In Advances in Computing and Data Sciences; Communications in Computer and Information Science; Singh, M., Gupta, P.K., Tyagi, V., Flusser, J., Ören, T., Valentino, G., Eds.; Springer: Singapore, 2020; Volume 1244, pp. 76–84. [Google Scholar] [CrossRef]
Hatwell, J.; Gaber, M.M.; Atif Azad, R.M. Ada-WHIPS: Explaining AdaBoost Classification with Applications in the Health Sciences. BMC Med Inform. Decis. Mak. 2020, 20, 1–25. [Google Scholar] [CrossRef] [PubMed]
Vlek, C.S.; Prakken, H.; Renooij, S.; Verheij, B. A Method for Explaining Bayesian Networks for Legal Evidence with Scenarios. Artif. Intell. Law 2016, 24, 285–324. [Google Scholar] [CrossRef] [Green Version]
Letham, B.; Rudin, C.; McCormick, T.H.; Madigan, D. Interpretable Classifiers using Rules and Bayesian Analysis: Building a Better Stroke Prediction Model. Ann. Appl. Stat. 2015, 9, 1350–1371. [Google Scholar] [CrossRef]
Assaf, R.; Schumann, A. Explainable Deep Neural Networks for Multivariate Time Series Predictions. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19, Macao, China, 10–16 August 2019; Number 2. pp. 6488–6490. [Google Scholar] [CrossRef] [Green Version]
Sun, K.H.; Huh, H.; Tama, B.A.; Lee, S.Y.; Jung, J.H.; Lee, S. Vision-Based Fault Diagnostics Using Explainable Deep Learning With Class Activation Maps. IEEE Access 2020, 8, 129169–129179. [Google Scholar] [CrossRef]
Sabol, P.; Sinčák, P.; Magyar, J.; Hartono, P. Semantically Explainable Fuzzy Classifier. Int. J. Pattern Recognit. Artif. Intell. 2019, 33, 2051006. [Google Scholar] [CrossRef]
D’Alterio, P.; Garibaldi, J.M.; John, R.I. Constrained Interval Type-2 Fuzzy Classification Systems for Explainable AI (XAI). In Proceedings of the 2020 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), Virtual Event, 20–24 July 2020; Volume 2020, pp. 1–8. [Google Scholar] [CrossRef]
Wachter, S.; Mittelstadt, B.; Russell, C. Counterfactual Explanations Without Opening the Black Box: Automated Decisions and the GDPR. Harv. J. Law Technol. 2018, 31, 841–887. [Google Scholar] [CrossRef] [Green Version]
Fernández, R.R.; Martín de Diego, I.; Aceña, V.; Fernández-Isabel, A.; Moguerza, J.M. Random Forest Explainability using Counterfactual Sets. Inf. Fusion 2020, 63, 196–207. [Google Scholar] [CrossRef]
Loyola-González, O.; Gutierrez-Rodriguez, A.E.; Medina-Perez, M.A.; Monroy, R.; Martinez-Trinidad, J.F.; Carrasco-Ochoa, J.A.; Garcia-Borroto, M. An Explainable Artificial Intelligence Model for Clustering Numerical Databases. IEEE Access 2020, 8, 52370–52384. [Google Scholar] [CrossRef]
Alonso, J.M.; Ducange, P.; Pecori, R.; Vilas, R. Building Explanations for Fuzzy Decision Trees with the ExpliClas Software. In Proceedings of the 2020 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), Virtual Event, 20–24 July 2020; Volume 2020, pp. 1–8. [Google Scholar] [CrossRef]
Massie, S.; Craw, S.; Wiratunga, N. A Visualisation Tool to Explain Case-Base Reasoning Solutions for Tablet Formulation. In Applications and Innovations in Intelligent Systems XII; Springer: London, UK, 2004; pp. 222–234. [Google Scholar] [CrossRef]
Panigutti, C.; Perotti, A.; Pedreschi, D. Doctor XAI An Ontology-based Approach to Black-box Sequential Data Classification Explanations. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency FAT* 2020, Barcelona, Spain, 27–30 January 2020; pp. 629–639. [Google Scholar] [CrossRef]
Lamy, J.B.; Sekar, B.; Guezennec, G.; Bouaud, J.; Séroussi, B. Explainable Artificial Intelligence for Breast Cancer: A Visual Case-Based Reasoning Approach. Artif. Intell. Med. 2019, 94, 42–53. [Google Scholar] [CrossRef] [PubMed]
Lamy, J.B.; Sedki, K.; Tsopra, R. Explainable Decision Support through the Learning and Visualization of Preferences from a Formal Ontology of Antibiotic Treatments. J. Biomed. Inform. 2020, 104, 103407. [Google Scholar] [CrossRef] [PubMed]
Ming, Y.; Qu, H.; Bertini, E. RuleMatrix: Visualizing and Understanding Classifiers with Rules. IEEE Trans. Vis. Comput. Graph. 2019, 25, 342–352. [Google Scholar] [CrossRef] [Green Version]
Waa, J.V.D.; Schoonderwoerd, T.; Diggelen, J.V.; Neerincx, M. Interpretable Confidence Measures for Decision Support Systems. Int. J. Hum.-Comput. Stud. 2020, 144, 102493. [Google Scholar] [CrossRef]
Guidotti, R.; Monreale, A.; Giannotti, F.; Pedreschi, D.; Ruggieri, S.; Turini, F. Factual and Counterfactual Explanations for Black Box Decision Making. IEEE Intell. Syst. 2019, 34, 14–23. [Google Scholar] [CrossRef]
Shalaeva, V.; Alkhoury, S.; Marinescu, J.; Amblard, C.; Bisson, G. Multi-operator Decision Trees for Explainable Time-Series Classification. In Information Processing and Management of Uncertainty in Knowledge-Based Systems. Theory and Foundations; Medina, J., Ojeda-Aciego, M., Verdegay, J.L., Pelta, D.A., Cabrera, I.P., Bouchon-Meunier, B., Yager, R.R., Eds.; Springer: Cham, Switzerland, 2018; Volume 853, pp. 86–99. [Google Scholar] [CrossRef]
Jung, A.; Nardelli, P.H.J. An Information-Theoretic Approach to Personalized Explainable Machine Learning. IEEE Signal Process. Lett. 2020, 27, 825–829. [Google Scholar] [CrossRef]
Itani, S.; Lecron, F.; Fortemps, P. A One-class Classification Decision Tree based on Kernel Density Estimation. Appl. Soft Comput. J. 2020, 91, 106250. [Google Scholar] [CrossRef] [Green Version]
Kim, J.; Canny, J. Interpretable Learning for Self-Driving Cars by Visualizing Causal Attention. In Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22–29 October 2017; Volume 2017, pp. 2961–2969. [Google Scholar] [CrossRef] [Green Version]
Riquelme, F.; De Goyeneche, A.; Zhang, Y.; Niebles, J.C.; Soto, A. Explaining VQA Predictions using Visual Grounding and a Knowledge Base. Image Vis. Comput. 2020, 101, 103968. [Google Scholar] [CrossRef]
Féraud, R.; Clérot, F. A Methodology to Explain Neural Network Classification. Neural Netw. 2002, 15, 237–246. [Google Scholar] [CrossRef]
Weber, R.O.; Johs, A.J.; Li, J.; Huang, K. Investigating Textual Case-Based XAI. In Case-Based Reasoning Research and Development; Cox, M.T., Funk, P., Begum, S., Eds.; Springer International Publishing: Berlin/Heidelberg, Germany, 2018; Volume 11156, pp. 431–447. [Google Scholar] [CrossRef]
Zhong, Q.; Fan, X.; Luo, X.; Toni, F. An Explainable Multi-attribute Decision Model based on Argumentation. Expert Syst. Appl. 2019, 117, 42–61. [Google Scholar] [CrossRef] [Green Version]
De, T.; Giri, P.; Mevawala, A.; Nemani, R.; Deo, A. Explainable AI: A Hybrid Approach to Generate Human-Interpretable Explanation for Deep Learning Prediction. Procedia Comput. Sci. 2020, 168, 40–48. [Google Scholar] [CrossRef]
Rutkowski, T.; Łapa, K.; Nielek, R. On Explainable Fuzzy Recommenders and their Performance Evaluation. Int. J. Appl. Math. Comput. Sci. 2019, 29, 595–610. [Google Scholar] [CrossRef] [Green Version]
Parasuraman, R.; Manzey, D. Complacency and Bias in Human Use of Automation: An Attentional Integration. Hum. Factors 2010, 52, 381–410. [Google Scholar] [CrossRef]
Rosenthal, S.; Selvaraj, S.P.; Veloso, M.M. Verbalization: Narration of Autonomous Robot Experience. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI 2016, New York, NY, USA, 2 February 2016; Volume 16, pp. 862–868. [Google Scholar]
Wilkenfeld, D.A.; Plunkett, D.; Lombrozo, T. Depth and deference: When and why we attribute understanding. Philos. Stud. 2016, 173, 373–393. [Google Scholar] [CrossRef] [Green Version]
Hoffman, R.R.; Klein, G. Explaining Explanation, Part 1: Theoretical Foundations. IEEE Intell. Syst. 2017, 32, 68–73. [Google Scholar] [CrossRef]
Klein, G. Explaining Explanation, Part 3: The Causal Landscape. IEEE Intell. Syst. 2018, 33, 83–88. [Google Scholar] [CrossRef]
Hoffman, R.R.; Mueller, S.T.; Klein, G. Explaining Explanation, Part 2: Empirical Foundations. IEEE Intell. Syst. 2017, 32, 78–86. [Google Scholar] [CrossRef]
Hoffman, R.; Miller, T.; Mueller, S.T.; Klein, G.; Clancey, W.J. Explaining Explanation, Part 4: A Deep Dive on Deep Nets. IEEE Intell. Syst. 2018, 33, 87–95. [Google Scholar] [CrossRef]
Hong, S.R.; Hullman, J.; Bertini, E. Human factors in model interpretability: Industry practices, challenges, and needs. Proc. Acm Hum.-Comput. Interact. 2020, 4, 1–26. [Google Scholar] [CrossRef]
Zhu, H.; Yu, B.; Halfaker, A.; Terveen, L. Value-sensitive algorithm design: Method, case study, and lessons. Proc. Acm Hum.-Comput. Interact. 2018, 2, 1–23. [Google Scholar] [CrossRef]
Miller, T. Explanation in artificial intelligence: Insights from the social sciences. Artif. Intell. 2019, 267, 1–38. [Google Scholar] [CrossRef]
Ehsan, U.; Riedl, M.O. Human-Centered Explainable AI: Towards a Reflective Sociotechnical Approach. In HCI International 2020—Late Breaking Papers: Multimodality and Intelligence; Lecture Notes in Computer Science; Stephanidis, C., Kurosu, M., Degen, H., Reinerman-Jones, L., Eds.; Springer International Publishing: Cham, Switzerland, 2020; pp. 449–466. [Google Scholar] [CrossRef]
Spinner, T.; Schlegel, U.; Schafer, H.; El-Assady, M. explAIner: A Visual Analytics Framework for Interactive and Explainable Machine Learning. IEEE Trans. Vis. Comput. Graph. 2019, 26, 1064–1074. [Google Scholar] [CrossRef] [PubMed] [Green Version]
Wesonga, R.; Nabugoomu, F.; Jehopio, P. Parameterized framework for the analysis of probabilities of aircraft delay at an airport. J. Air Transp. Manag. 2012, 23, 1–4. [Google Scholar] [CrossRef]
Wong, J.T.; Tsai, S.C. A survival model for flight delay propagation. J. Air Transp. Manag. 2012, 23, 5–11. [Google Scholar] [CrossRef]
Feuser Fernandes, H.; Müller, C. Optimization of the waiting time and makespan in aircraft departures: A real time non-iterative sequencing model. J. Air Transp. Manag. 2019, 79, 101686. [Google Scholar] [CrossRef]
Aydemir, R.; Seymour, D.T.; Buyukdagli, A.; Guloglu, B. An empirical analysis of delays in the Turkish Airlines network. J. Air Transp. Manag. 2017, 65, 76–87. [Google Scholar] [CrossRef]
Shaukat, S.; Katscher, M.; Wu, C.L.; Delgado, F.; Larrain, H. Aircraft line maintenance scheduling and optimisation. J. Air Transp. Manag. 2020, 89, 101914. [Google Scholar] [CrossRef]
Vadlamani, S.; Hosseini, S. A novel heuristic approach for solving aircraft landing problem with single runway. J. Air Transp. Manag. 2014, 40, 144–148. [Google Scholar] [CrossRef]
Chang, L.Y. Analysis of bilateral air passenger flows: A non-parametric multivariate adaptive regression spline approach. J. Air Transp. Manag. 2014, 34, 123–130. [Google Scholar] [CrossRef]
Kim, M.; Park, S. Airport and route classification by modelling flight delay propagation. J. Air Transp. Manag. 2021, 93, 102045. [Google Scholar] [CrossRef]
Truong, D.; Choi, W. Using machine learning algorithms to predict the risk of small Unmanned Aircraft System violations in the National Airspace System. J. Air Transp. Manag. 2020, 86, 101822. [Google Scholar] [CrossRef]
Jin, F.; Li, Y.; Sun, S.; Li, H. Forecasting air passenger demand with a new hybrid ensemble approach. J. Air Transp. Manag. 2020, 83, 101744. [Google Scholar] [CrossRef]
Hausladen, I.; Schosser, M. Towards a maturity model for big data analytics in airline network planning. J. Air Transp. Manag. 2020, 82, 101721. [Google Scholar] [CrossRef]
Rodríguez-Díaz, A.; Adenso-Díaz, B.; González-Torre, P. Improving aircraft approach operations taking into account noise and fuel consumption. J. Air Transp. Manag. 2019, 77, 46–56. [Google Scholar] [CrossRef]
Murça, M.C.R. Collaborative air traffic flow management: Incorporating airline preferences in rerouting decisions. J. Air Transp. Manag. 2018, 71, 97–107. [Google Scholar] [CrossRef]
Jahangoshai Rezaee, M.; Yousefi, S. An intelligent decision making approach for identifying and analyzing airport risks. J. Air Transp. Manag. 2018, 68, 14–27. [Google Scholar] [CrossRef]
Bongiorno, C.; Gurtner, G.; Lillo, F.; Mantegna, R.; Miccichè, S. Statistical characterization of deviations from planned flight trajectories in air traffic management. J. Air Transp. Manag. 2017, 58, 152–163. [Google Scholar] [CrossRef] [Green Version]
Pyrgiotis, N.; Malone, K.M.; Odoni, A. Modelling delay propagation within an airport network. Transp. Res. Part C Emerg. Technol. 2013, 27, 60–75. [Google Scholar] [CrossRef]
Dück, V.; Ionescu, L.; Kliewer, N.; Suhl, L. Increasing stability of crew and aircraft schedules. Transp. Res. Part C Emerg. Technol. 2012, 20, 47–61. [Google Scholar] [CrossRef]
Reitmann, S.; Schultz, M. Computation of Air Traffic Flow Management Performance with Long Short-Term Memories Considering Weather Impact. In Proceedings of the Artificial Neural Networks and Machine Learning—ICANN, Rhodes, Greece, 4–7 October 2018; Lecture Notes in Computer Science. Kůrková, V., Manolopoulos, Y., Hammer, B., Iliadis, L., Maglogiannis, I., Eds.; Springer International Publishing: Cham, Switzerland, 2018; pp. 532–541. [Google Scholar] [CrossRef]
Shao, Q.; Shao, M.; Lu, Y. Terminal area control rules and eVTOL adaptive scheduling model for multi-vertiport system in urban air Mobility. Transp. Res. Part C Emerg. Technol. 2021, 132, 103385. [Google Scholar] [CrossRef]
Hirabayashi, H.; Brown, M.; Takeichi, N. Feasibility Study of Free routing Airspace Operation over the North Pacific Airspace. In Proceedings of the Fourteenth USA/Europe Air Traffic Management Research and Development Seminar (ATM2021), Virtual Event, 20–23 September 2021. [Google Scholar]

Figure 1. Sector division of the upper airspace of France [4].

Figure 2. Excerpt of a map of the route network in the south of France. Routes are defined by waypoints—black and grey dots. Source: EUROCONTROL.

Figure 4. PRISMA flow of the review made on International Conference on Research in Air Transportation (ICRAT), Air Traffic Management Research and Development Seminar (ATM seminar), Transportation Research Part C: Emerging Technologies (TR_C), Journal of Air Transport Management (JATM), and IEEE Transactions on Intelligent Transportation Systems (IEEE Trans. on ITS).

Figure 5. Publication distribution on the different object feature from Table 3.

Figure 6. Tree map of the distribution of the articles in function of on the secondary extracted features from Table 4. Each rectangle represent a leaf of the table—the lowest level of description between object, complement, and sub-complement—, with the number of article focusing on this feature.

Figure 7. Distribution of the publications per year in function of the Source Type.

Figure 8. Evolution of the number of publication per year of the Prediction, Optimisation/Automation, Modelling/Simulation, and Analysis categories.

Figure 9. Distribution of the publications in function of the categories.

Figure 10. Synthesis of EXplainable Artificial Intelligence (XAI) conceptual framework.

Figure 11. Conflict scenario involving two Aircrafts,

A_{1}

and

A_{2}

Figure 11. Conflict scenario involving two Aircrafts,

A_{1}

and

A_{2}

Table 1. Exclusion and inclusion criteria.

Criteria	Principles
Inclusion	Papers published from 2010 to end of 2021. Full text. Peer-reviewed studies. Paper in the ATM domain. Paper in the AI domain. Paper answers the defined research questions.
Exclusion	Papers not in the English language. Review papers. Papers less than 4 pages. Not peer reviewed studies.

Table 2. Paper selection result at the identification phase.

Keyword	ICRAT	ATM Seminar	TranC	JATM	IEEE Trans. on ITS	Total
“Predict*”	39	23	15	18	5	100
“Estimat*”	17	12	6	20	2	57
“Optim*”	26	27	24	26	5	108
“Cluster*”	5	3	0	7	0	15
“Analy*”	33	38	12	171	1	255
“Visual*”	2	2	1	1	0	6
“Learn*”	13	11	10	14	4	52
“Explain*”	0	0	0	3	0	3
“Model*“	38	43	30	134	4	249
“Plan*”	9	9	12	37	0	67
“Conflict”	17	13	9	2	5	46
“Classif*”	1	1	0	8	0	10
Total (without duplication)	160	141	88	371	18	778

Table 3. Extracted features from primary papers.

Data	Description
Author(s)	Name of the author(s).
Publication Year	The year of publishing the paper.
Title of the Study	The title of each paper that is visible in the searching step.
Source Type	Journal, book chapter, and conference proceeding.
AI Theory or Framework	The AI theory or framework that the study had adopted, e.g., Neural Network (NN).
XAI Theory or Framework	The XAI theory or framework that the study had adopted, e.g., LIME.
Objective	The main objective of papers.
Factors	The examined factors of the studies, detailed in Table 4.

Table 4. Additional extracted features.

	Object	Complement	Sub-Complement	Description
	Time Frame	Pre-Flying		A publication focused on before the Aircrafts are flying/moving; contains the ATM time-frame called Stategic, Pre-Tactical, and part of Tactical phase.
		Flying		A publication focused on while implicated Aircrafts are flying.
		Post-Analysis		A publication focused on after implicated Aircrafts have landed.
		Not time-framed		The publication is not time framed
“Material Object”	Aircraft	State		The publication deals with any descriptor of the actual physical state of the aircraft, such as mass or True Air Speed (TAS).
		Trajectory	Indicators	The publication deals with any descriptor of the trajectory, not related to the direct physical state of the Aircraft, such as Phase of Flight (PoF), or Descent Length.
			Route/Flight Plan	The publication focuses on the description of the intended flight.
			4D Trajectory	The publication focuses on the description of the actual flight.
	Traffic	Indicators		The publication deals with any descriptor of the traffic, such as time buffer separation, or delay.
		5D Traffic	Conflict avoidance	The publication deals with the avoidance of separation losses between Aircrafts.
			Optimisation	The publication deals with the optimisation of Aircraft trajectories.
			Prediction	The publication deals with the prediction of Aircraft trajectories, and their potential interactions.
			Simulation	The publication deals with the simulation of Aircraft trajectories, and their potential interactions.
			Analysis	The publication deals with the analysis of Aircraft trajectories, and their potential interactions.
	Airport/ CTR Traffic	State		The publication deals with any descriptor of the state of the airport, e.g., the runway configuration.
		Ground Traffic	Indicators	The publication deals with any descriptor of the ground traffic, e.g., Taxi-Speed, or Estimated Take-Off Time (ESOT), Arrival Runway Occupancy.
		Ground Traffic	“5D” Traffic	The publication focuses on the trajectories of the taxiing aircrafts
		CTR Traffic		The publication deals with the arrival of aircrafts(e.g., the sequencing of arriving aircraft), the departure, or both (e.g., optimisation of departure and arrival).
	Airspace	State	Static Structural State	The publication deals with any descriptor of the state of all or a part of the airspace, e.g., the capacity of a sector, without modifying it.
		State	State of Environment	The publication focuses on the weather, the wind or any other environmental descriptor.
		Structure	Sector	The publication deals with the structure of the sector(s), e.g., the configuration of the sectors, or their geometrical structure.
		Structure	Route	The publication deals with the route network structure.
		Demand/ Capacity Balancing		The publication focuses on the balancing of the demand and capacity.
	ATCO			The publications focuses on the Air Traffic COntroller (ATCO).
	Pilot			The publication focuses on the Pilot.

Table 6. XAI methods with associated types of explanations (N: Numeric, R: Rules, T: Textual, V: Visual), stage (Ah: Ante-hoc, Ph: Post-hoc), scope (L: Local, G: Global) of explainability, and the design spaces (P: Prediction, O/A: Optimisation/Automation, A: Analysis, M/S: Modelling/Simulation).

Methods for Explainability	References	Explanations				Stage		Scope		Design Space				Total Count
Methods for Explainability	References	N	R	T	V	Ah	Ph	L	G	P	O/A	A	M/S	Total Count
ANFIS	[273,278,279,280]	✔	✔	✔	✔		✔	✔	✔		✔	✔		9
Anchors	[281]	✔	✔	✔	✔		✔	✔	✔	✔				8
Feature Importance	[282,283,284,285]	✔		✔	✔		✔	✔	✔	✔				7
LIME	[270,272,275,276]	✔			✔		✔	✔	✔	✔		✔		7
RetainVis	[286]	✔		✔	✔		✔	✔	✔			✔		7
SHAP	[21,271,272,277]	✔			✔		✔	✔	✔	✔		✔		7
SRM	[274]				✔	✔		✔	✔		✔	✔	✔	7
SurvLIME-KS	[287]	✔			✔		✔	✔	✔	✔		✔		7
TreeExplainer	[271]	✔			✔		✔	✔	✔	✔			✔	7
BB-BC IT2FLS	[288]	✔	✔		✔	✔		✔					✔	6
CIE	[289]	✔		✔			✔	✔	✔	✔				6
ExNN	[290]				✔	✔		✔	✔	✔	✔			6
FDE	[291]	✔					✔	✔	✔		✔	✔		6
MAPLE	[292]	✔			✔		✔	✔	✔	✔				6
Generation	[293]			✔			✔	✔				✔	✔	5
GRACE	[294]			✔			✔	✔		✔		✔		5
HFS	[295]				✔	✔		✔	✔		✔			5
iNNvestigate	[296]				✔		✔	✔	✔	✔				5
J48	[297,298]		✔	✔		✔		✔	✔					5
Ada-WHIPS	[299]		✔				✔	✔		✔	✔			5
BN	[300]			✔	✔	✔		✔				✔		5
BRL	[301]		✔			✔			✔	✔		✔		5
CAM	[302,303]				✔		✔		✔	✔			✔	5
CFCMC	[304]			✔		✔			✔	✔		✔		5
CIT2FS	[305]			✔	✔	✔		✔			✔			5
Counterfactual Sets	[306,307]			✔			✔	✔		✔	✔			5
eUD3.5	[308]		✔			✔		✔	✔			✔		5
FINGRAM	[309]				✔	✔		✔		✔		✔		5
FormuCaseViz	[310]				✔		✔	✔				✔	✔	5
FURIA	[297]		✔			✔		✔		✔	✔			5
Ontological Perturbation	[311]		✔				✔	✔			✔	✔		5
RBIA	[312,313]				✔		✔	✔	✔		✔			5
RuleMatrix	[314]				✔		✔		✔			✔	✔	5
FFT	[298]		✔			✔		✔		✔				4
ICM	[315]			✔			✔	✔		✔				4
LORE	[21,316]		✔				✔	✔		✔				4
MTDT	[317]				✔	✔			✔	✔				4
Mutual Importance	[318]	✔					✔	✔		✔				4
OC-Tree	[319]		✔			✔			✔	✔				4
Attention Maps	[320,321]				✔		✔	✔		✔				4
Causal Importance	[322]	✔					✔		✔			✔		4
CTree	[298]		✔			✔		✔		✔				4
TCBR	[323]			✔			✔	✔				✔		4
Template-based Natural Language Generation	[324]			✔		✔		✔		✔				4
TREPAN	[325]		✔				✔	✔				✔		4
WM Algorithm	[326]		✔			✔			✔				✔	4
Total Count		14	15	15	23	18	28	38	26	27	11	20	8	–

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

Share and Cite

MDPI and ACS Style

Degas, A.; Islam, M.R.; Hurter, C.; Barua, S.; Rahman, H.; Poudel, M.; Ruscio, D.; Ahmed, M.U.; Begum, S.; Rahman, M.A.; et al. A Survey on Artificial Intelligence (AI) and eXplainable AI in Air Traffic Management: Current Trends and Development with Future Research Trajectory. Appl. Sci. 2022, 12, 1295. https://doi.org/10.3390/app12031295

AMA Style

Degas A, Islam MR, Hurter C, Barua S, Rahman H, Poudel M, Ruscio D, Ahmed MU, Begum S, Rahman MA, et al. A Survey on Artificial Intelligence (AI) and eXplainable AI in Air Traffic Management: Current Trends and Development with Future Research Trajectory. Applied Sciences. 2022; 12(3):1295. https://doi.org/10.3390/app12031295

Chicago/Turabian Style

Degas, Augustin, Mir Riyanul Islam, Christophe Hurter, Shaibal Barua, Hamidur Rahman, Minesh Poudel, Daniele Ruscio, Mobyen Uddin Ahmed, Shahina Begum, Md Aquif Rahman, and et al. 2022. "A Survey on Artificial Intelligence (AI) and eXplainable AI in Air Traffic Management: Current Trends and Development with Future Research Trajectory" Applied Sciences 12, no. 3: 1295. https://doi.org/10.3390/app12031295

APA Style

Degas, A., Islam, M. R., Hurter, C., Barua, S., Rahman, H., Poudel, M., Ruscio, D., Ahmed, M. U., Begum, S., Rahman, M. A., Bonelli, S., Cartocci, G., Di Flumeri, G., Borghini, G., Babiloni, F., & Aricó, P. (2022). A Survey on Artificial Intelligence (AI) and eXplainable AI in Air Traffic Management: Current Trends and Development with Future Research Trajectory. Applied Sciences, 12(3), 1295. https://doi.org/10.3390/app12031295

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Menu

A Survey on Artificial Intelligence (AI) and eXplainable AI in Air Traffic Management: Current Trends and Development with Future Research Trajectory

Abstract

1. Introduction

1.1. Air Traffic Management

1.2. AI and XAI for ATM

2. Paper Selection

2.1. Identification

2.2. Screening

2.3. Eligibility

2.4. Inclusion

3. Paper Clustering

3.1. Data Extraction

3.2. First Data Clustering on Additional Extracted Data

3.3. Second Data Clustering on All Extracted Data

4. Design Space: Trends of AI in ATM

4.1. General Insights

4.2. Categorisation Insights

4.3. Validating the Design Space

5. Analysing General XAI

XAI in Terms of Design Space

6. Analysing XAI in ATM

7. XAI in ATM Synthesis

8. Scenarios XAI in ATM

9. Setting the Trajectory of AI and XAI in ATM

Author Contributions

Funding

Institutional Review Board Statement

Informed Consent Statement

Data Availability Statement

Conflicts of Interest

Appendix A. Principal Extracted Features from Articles

References

Share and Cite

Article Metrics

Article Access Statistics

Further Information

Guidelines

MDPI Initiatives

Follow MDPI