Southwest Airlines incidents prove companies need to deal with technical debt – now

Thousands of US organizations are running on outdated software. This kind of failure to address technical debt is a recipe for a disaster – and it’s already costing trillions.

In December 2022, a faulty router took Southwest Airlines’ reservation system offline and stranded thousands of passengers during the holiday rush. More than 16,000 flights were canceled and the airline lost north of $1 billion.

This week, lightning struck for the second time. The very same airline again experienced a major technical issue with one of their internal systems – according to the US Federal Aviation Administration – and had to ground all flights nationwide.

These kinds of outages can hit companies and organizations extremely hard, experts say. They stress that it’s vital to prioritize dealing with technical debt, which is what the tech industry calls the implied cost incurred when businesses fail to fix problems that will affect them in the future.

“Southwest’s system meltdown is an example of the detrimental consequences of technical debt going unaddressed for years, which resulted in an IT disaster and loss of customers and revenue,” SonarSource, a company that develops open source software for continuous code quality and security, recently said.

Southwest Airlines was even sued for outdated technology – but it’s not only businesses that are suffering. A fresh US Government Accountability Office study has shown that around one-third of the digital applications run by the Internal Revenue Service are out of date.

The report raises concerns over cybersecurity issues, as some of the applications in use range from 25 to 64 years old. This, combined with some software being up to 15 versions behind up-to-date versions, highlights the extent of technical debt in play.

The very fact that technical debt is accumulating shouldn’t be a surprise. In this day and age, speed is king. Firms and organizations prioritize fast delivery over perfect and glitch-free code.

Besides, most executives or other decision-makers shy away from adopting new and updated technology, especially within their most important or risk-heavy operations. They’re afraid that by switching to something new, fresh problems might arise. With this in mind, sticking with the tried-and-trusted can often feel like the safer option.

According to Fabrice Bellingard, Vice President of Products at SonarSource, this is the wrong approach. In the interview below, he tells Cybernews that mitigating risks requires constant work.

Be ready to lose precious time

Recent crew-scheduling challenges for Southwest Airlines have shown that it pays off for companies to fix and constantly update old software. Shortcuts are cheaper, though, and a lot of firms or organizations live on the financial edge anyhow. What are the numbers behind technical debt, as in why does modernizing software make sense in the end?

“Data from the Consortium for Information and Software Quality (CISQ) revealed that the expenses resulting from deficient software quality in the US has reached a minimum of $2.41 trillion, and the total amount of technical debt related to software has accumulated to approximately $1.52 trillion.

Technical debt also consumes a significant amount of time, resources, and 10-24% of a developer’s time. That said, it’s not accurate to attribute technical debt as a whole to shortcuts taken by developers.

Many organizations lack the systems and approach to development required to minimize technical debt – and that rolls downhill to developers.

Expenses resulting from deficient software quality in the US has reached a minimum of $2.41 trillion.

Consortium for Information and Software Quality.

Given the negative impact that too much technical debt can have, it’s imperative for organizations to adopt clean code practices to ensure their software is high quality, stay competitive, and meet evolving business needs. Modernizing software using a clean code approach means a reduction in new technical debt introduced to the codebase, helps remediate existing technical debt over time, and makes code fit for development and production.

By minimizing the accumulation of technical debt by focusing on delivering new quality code, organizations reduce the likelihood of an outage or system failure. Also, developers are happy because they can focus on innovative projects like delivering new features and functionality to customers, rather than fixing bugs and sifting through spaghetti code.”

How does technical debt actually accumulate? What does it stem from?

“Technical debt arises due to many different factors, and depends on the priorities of the organization. For instance, business pressures from leadership to meet tight deadlines, launch products faster than competitors, and deliver new features quickly can all lead to technical debt if development teams are not enabled with a clean code approach.

Other causes include lack of developer resources, insufficient training, and unforeseen challenges during development. It’s worth noting that while developers are tasked with writing code, they cannot be solely responsible for the management of technical debt.

Technical debt buildup is normally the result of a systematic issue that spans across the organization from developers to product owners to the executive team. Companies can address this issue by shifting to a ‘Clean as You Code’ approach, providing resources for maintenance and improvement, and establishing processes to manage technical debt over the long term.”

Cheaper to write clean code

Why do some say technical debt is not necessarily a bad thing as a proof-of-concept? Are there any specific examples of companies implementing necessary changes at a huge cost first, but then, after having learned the lesson, successfully avoiding accumulating technical debt going forward?

“The way technical debt is typically explained is misleading because it implies that developers deliberately choose to take shortcuts and suffer the consequences later. However, this is not usually the case.

In reality, technical debt is often introduced unintentionally due to a lack of awareness of potential problems or the complexity of certain issues, which can be difficult for humans to identify.

In other words, introducing technical debt is more of a side effect of poor DevOps systems and practices rather than a conscious one. Training and education, in addition to using clean code tools help prevent technical debt from accumulating and are beneficial in flagging problems and enabling developers and DevOps practitioners to make the right decisions.

Finally, it is also important to note that aiming for zero technical debt may be unrealistic.”

Still, even if technical debt is vital, you don’t have the cash if you don’t have the cash, right? Is avoiding technical debt expensive, or can it be done on the cheap?

“Writing clean code does not cost more than writing bad code. So the short answer is ‘no,’ avoiding technical debt is not expensive. If you ‘Clean as You Code,’ which consists of making sure that the code that gets added to or changed in your application is clean, you can actually remediate the past and improve the overall state of the application.

If you look at the bottom line, this is actually much cheaper to write clean code as you will not have to pay the interest and do reworks.”

So this is in essence a shortcut to avoid shortcuts, isn’t it?

“We need to enable developers to achieve the ideal state for their code by ensuring that all new and altered code is suitable for development and production.

Implementing clean code practices is beneficial to development teams, departments, operations teams and security teams throughout the software lifecycle. The goal is to empower organizations to achieve a clean code state that saves developers time, conserves money, and reduces security risks.

We can make clean code easy for development teams by shortening feedback loops and enforcing quality gates to prevent new problems or vulnerabilities from arising from shipping to production. Then, developers can sit back and enjoy the progress.”