About the Aviation Safety Monitor

Did Safety Degrade in the National Airspace System in the Winter of 2022-2023?

​

A series of well-publicized runway incursions and near midair collisions from October 2022 through February 2023 raised concerns throughout the aviation community and spurred the unprecedented call by the FAA Administrator for an industry Safety Summit. Shortly thereafter, the Administrator established the National Airspace System Safety Review Team that issued its findings in November 2023. That study made several recommendations for FAA actions but left unanswered the question of whether safety margins in the NAS had decreased. Robust Analytics has been engaged for the past several years in developing methods and data to measure safety margins in terminal airspace. We apply those methods in this article to examine changes in safety margins in terminal airspace from May 2022 through June 2023 and investigate whether the reported close calls were a random set of events or an indication of an underlying decrease in safety margins. Our conclusion is that safety margins degraded from August 2022 through June 2023, with the NAS exhibiting signs of stress that are reflected in various measures of safety-related events. The causes of that stress deserve further investigation.

 

To learn more, please keep reading.

​

A series of well-publicized runway incursions and near midair collisions from October 2022 through February 2023 raised concerns throughout the aviation community and spurred the unprecedented call by the FAA Administrator for an industry Safety Summit. Shortly thereafter, the Administrator established the National Airspace System Safety Review Team that issued its findings in November 2023. That study made several recommendations for FAA actions but left unanswered the question of whether safety margins in the NAS had decreased. Robust Analytics has been engaged for the past several years in developing methods and data to measure safety margins in terminal airspace. We apply those methods in this note to examine changes in estimated safety margins in terminal airspace from May 2022 through June 2023 and investigate whether the reported close calls were a random set of events or an indication of an underlying decrease in safety margins.

 

We measure safety margins by estimating the frequency, duration, and severity of buffer encroachments. A buffer encroachment occurs when an aircraft pair is predicted to be closer than the minimum separation standard – both lateral and vertical – for that airspace. The encroachment algorithms account for aircraft type (heavy and other aircraft), airspace rules, horizontal distance between successive aircraft, and vertical separation. FAA has defined minimum separation standards when aircraft are under Instrument Flight Rules (IFR). In this research, the buffer encroachments use those separation standards as a baseline to define the adequate safety margin levels. 

 

The buffer encroachment algorithms do not identify loss of separation events. Instead, they evaluate what would occur if aircraft continued on their current speed and heading. Avoidance of a loss of separation when a buffer encroachment occurs requires action by flight crews and controllers. In many cases, these actions call for the aircraft to follow well-defined procedures accurately and in a timely fashion. In that sense, when the algorithm identifies a buffer encroachment it highlights a situation in which pilots and controllers must operate with a shared awareness of what all aircraft in the airspace will be doing. It is with this understanding that we interpret buffer encroachments as a measure of safety margins in the airspace.

 

The buffer encroachment algorithms use flight track messages from the FAA’s System Wide Information Management (SWIM) data service. The track data from SWIM consists of asynchronous position reports for all aircraft in the terminal airspace. To calculate the buffer encroachments, the actual aircraft trajectory must be linearly interpolated between any two given position reports. For a given pair of aircraft with piece-wise linear trajectories, a buffer encroachment can occur if any pair of line-segments that constitute the trajectories violate the buffers. The algorithms identify both the occurrence of an encroachment and its duration. The encroachment duration calculation is applied to all pairs of line segments for all aircraft pairs. The sum of all segment-wise durations gives the total encroachment duration for the airspace. The Aviation Safety Monitor calculates updates of the buffer encroachments and other hazards and metrics every minute. The results are output in five-minute epochs, or time periods.

 

 

 

 

 

 

 

 

The flight track data include all aircraft within a 50-mile radius of the airport center. All flights for the 26 airports are included. Airport area weather conditions are obtained from the Aviation Weather Center (AWC) METAR service that reports airport ceiling and visibility conditions approximately every hour. The ceiling and visibility values are used to identify the meteorological conditions in each five-minute epoch, using the airport specific thresholds reported by the FAA Operational Information System (OIS) website.

 

The meteorological condition categories used are VMC, MVMC, IMC, and LIMC. The buffer encroachment calculations adjust thresholds for different meteorological conditions, allowing for aircraft to get closer during VMC and MVMC. Many of these terminal airspaces have more than one commercial airport inside the 50-mile radius. To ensure that encroachment events are counted accurately, all aircraft within the metroplex are analyzed simultaneously and reported together. For example, in the Dallas-Fort Worth area, buffer encroachments for DFW and DAL are analyzed jointly and reported out as a single metroplex. Therefore, the 26 airports used in the study are combined into 17 metroplexes. 

 

Findings

In the larger study from which we draw results for this note, we calculate number, duration, and severity of every buffer encroachment in these 17 terminal areas. To focus this article on trends over time, we report daily average encroachment durations and event rates. 

 

The daily mean duration and event counts per epoch exhibit similar patterns over the 14-months, with significant day-to-day variation and large spikes on several days. To assist in detecting broader trends, the next two plots display 7-day moving averages for the duration and event daily epoch series. 

​

​

​

​

 

 

 

 

 

 

 

 

 

 

 

 

 

 

​

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Figure 1 shows the daily totals for the 17 metroplexes for encroachment durations and event counts. The increase in encroachment durations and event rates in the last four months of 2022 is striking. The monthly duration metric almost triples from June 2022 to its peak in December 2022, then briefly decreases before returning to the levels of November-December 2022 before declining slightly in April 2023. The June 2023 total durations are more than twice those of June 2022. The adjusted event rates for June 2023 are 70 percent above the rates for June 2022. 

​

These findings suggest that safety margins decreased substantially during the second half of 2022. While the metrics have since declined, they remain well above the levels from the first half of 2022.  Another perspective is provided by calculating the average encroachment duration per aircraft as shown in Figure 2. The duration per aircraft estimate divides the total daily duration numbers from Figure 1  by the total number of aircraft in the terminal airspace during the five-minute period. This provides a rough estimate of the per aircraft exposure to the reduced safety margin. To illustrate the trend more sharply we display the 7-day moving average of the daily means in Figure 2. The moving average smooths the day-of-week fluctuations and shows the overall changes more sharply.

 

​

​

​

​

​

​

​

​

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Figure 2. Daily Duration Per Aircraft 7-Day Moving Average

 

Do these safety margin estimates reflect underlying changes in aviation safety? In our larger study we examine several publicly available safety sources, but Figure 3 highlights the relationship to the events of the fall and winter of 2022-2023. That graphic superimposes the ten events highlighted by the NASA Safety Review Team as drivers for the formation of their review team on the chart of Figure 2. The clustering of events at the peak increase in encroachments is quite clear and the results suggest that there is some predictive value as the decrease in safety margins precedes the highlighted events. A more complete analysis should be performed as the Safety Review Team only looked at events for a four-month period preceding its formation and events have continued to occur since February 2023.

 

​

​

​

​

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Figure 3. Safety Margins and Safety-Related Events

 

One persistent feature of aviation is that it varies across locations, as operations at one airport (for example, DEN) will differ significantly from another (ATL). We would not be surprised to observe differences in safety margins across terminal areas as well and this is reflected in the data. Is the event rate increase shown in Figure 1 common throughout the NAS, or limited to a handful of airports that drive the change? Examination of monthly encroachment durations and events at 17 metroplexes shows that the trend was common across most airports. Although there is substantial variation over the series, every airspace has a higher average duration in June 2023 than in June 2022. Moreover, the increases tend to be clustered in roughly the same time period, from August to December of 2022. They generally remain elevated above the early 2022 values into June of 2023.

 

Table 1 reports the mean per period encroachment duration by month and terminal area. In Table 1, monthly cells that increase one second or more above the previous month are colored orange, and cells with a decrease of one second or more from the previous month are colored green. Visual inspection of the table shows the clustering of orange cells in September through December 2022. Out of 68 cells in these four months, 35 are orange during this time, only 4 months are green, and 29 have a neutral change. Although there is substantial variation over the series, every airspace has a higher average duration in June 2023 than in June 2022. Moreover, the increases cluster in roughly the same time period, from August to December of 2022. They remain elevated above the early 2022 values into June of 2023. Every terminal airspace we analyzed experienced an increase in monthly encroachment durations from June 2022 to June 2023. Even though the monthly duration peak occurred in December 2022 with a value of 11.9 seconds, it only decreased to 9.2 seconds in June 2023. That value is 2.25 times greater than the value 12 months earlier.

 

Table 1. Monthly Average Per Epoch Duration By Metroplex

​

​

​

​

​

​

​

​

​

 

 

 

 

 

 

 

 

 

 

This study summarizes recent research into safety margins in terminal airspace in the National Airspace System. Flight track data for all aircraft operating in the terminal airspace at 26 airports from April 2022 through June 2023 were used to identify buffer encroachment events and estimate their frequency and duration. The findings show that terminal airspace safety margins, measured by buffer encroachments, decreased significantly through the last four months of 2022, then declined slightly before stabilizing at levels much higher than occurred in early 2022. Across the 26 airports analyzed, encroachment duration monthly averages increased 2.2 times from May 2022 to December 2022.  These findings and preliminary analysis of safety-related event data from the FAA and other sources indicate that our safety margin metric may serve as a leading indicator of degraded safety margins.