Millisecond software glitch caused more than 2,000 UK flight cancellations

Last week, passengers at UK airports faced mass flight cancellations and hours-long delays. The reason is now coming to light – a seemingly tiny software error was to blame.

A preliminary report into the September 8th incident was released by National Air Traffic Services (NATS) on Friday and disclosed that the issue was caused by a software defect in one of the systems used to manage UK air traffic.

“This was a software issue in a specific part of our flight data system, that we have traced to a small subsection of coding,” NATS chief executive officer Martin Rolfe said. “The issue has been identified and mitigation is in place while a permanent fix is safety tested and deployed.”

The affected software assigns identification codes to aircraft, which are used to identify flights on radar. While it was processing a request to assign a code manually, it received “a message for a higher priority activity” and temporarily paused the task.

When the system went back to processing the original request, a previously unknown software defect prevented it from resuming correctly and corrupted the resulting output, affecting subsequent flight data updates.

“This happened in the space of a millisecond,” the report said.

The error first occurred at 10 a.m. Controllers and engineers received a system error notification two minutes later informing them that the link between the LAC system and the NAS had dropped. The error disappeared 45 seconds later, indicating that the link had been successfully re-established, according to NATS.

However, at 12:30 p.m., controllers and engineers received new warnings about link failures, forcing controllers to switch to manual handling of some tasks. What they did not know at the time was that the link was automatically dropping as a safety measure after repeated attempts to process flight data timed out.

A major incident was declared at the same time.

Restrictions put in place to limit air traffic had to extend beyond the affected London Area Control Center and across the UK so that the NAS could be restarted and flight data reloaded.

Although the restrictions were only in place for six hours, the resulting disruption took more than two days to clear.

"I would like to apologise again, very sincerely, to everyone who was affected last week. It’s our job to get people where they want to go, quickly and without delay and we are devastated when that goes wrong. However, our primary role is to keep our skies safe, and everyone who flies through them. At no point last week was safety in question."

Martin Rolfe

The UK air traffic control provider stressed that last week’s failure was unrelated to the 2023 outage, which also caused widespread disruption across UK airports. It also said that it found no evidence of a cyberattack or malicious activity.

Hundreds of thousands hit by a massive disruption

Last week, roughly 330,000 passengers had their travel disrupted, with more than 155,000 passengers on cancelled flights and more than 180,000 on delayed flights, according to industry body Airlines UK. On Tuesday, about 30% of the day's scheduled flights were cancelled.

Airlines have demanded tens of millions of pounds in compensation. Airlines UK, which represents carriers including British Airways and Virgin Atlantic, called for “immediate compensation for the costs incurred and a credible plan to improve resilience”.

In turn, easyJet chief Kenton Jarvis said there had to be a “fair solution to ensure that airlines are not left to pick up the bill for NATS' failures”.

Ryanair called for Rolfe to be fired, with Ryanair chief operating officer Neal McMahon saying: “The ‘software ate my homework’ is not an acceptable explanation.”

In the public news statement, Rolfe stressed that “over the past 10 years, NATS has invested well over one billion pounds in systems and technology” and “will soon submit plans to the Civil Aviation Authority to invest a further billion pounds by the end of 2033.”