Key takeaways:
- Recent outages aren’t flukes. Complex systems break without constant upkeep.
- That upkeep is shrinking while cyberattacks grow.
- AI makes attacks faster, cheaper and easier.
- The fix may be fewer connections, decided before we depend on them.
Tens of thousands of air travelers found themselves stranded last week as the Federal Aviation Administration ordered a nationwide ground stop; fears of war or terrorism were quickly replaced with annoyance as it was revealed that the cause was a glitch during a routine router upgrade that quickly threatened air traffic control’s telecommunications. Then, a couple of days later, many major banks, including here in New York City, saw significant outages that made it harder for customers to use routine services. And all that came after Microsoft tools and apps were disrupted worldwide for a day and a half. So, what gives?
Let’s start with the actual groundwork that has gotten us here. The natural tendency of complex systems is toward not working unless that’s offset by careful and complicated maintenance. That’s because they are in fact made up of many separate little systems that not only have to work individually but have to be in sync at all times. A breakdown in one part of such a system can often take the whole thing down.
A lot of people found it hard to believe that a single malfunctioning router installed as part of the FAA’s system modernization could have grounded all flights in the United States, theorizing instead that the router was a scapegoat for the administration’s ineptitude. I say: ¿por qué no los dos? It seems entirely plausible to me that a misfiring router could take down a nationwide system and that the odds of that happening would have been increased by the DOGE-thinned ranks of the FAA failing to build in the redundancies and safeguards that would have stopped this chain reaction. There’s a famous comic by the roboticist and xkcd creator Randall Munroe that gets to the heart of this, which depicts the whole of the modern digital infrastructure propped up by one tiny block representing “a project some random person in Nebraska has been thanklessly maintaining since 2003.”
More demands, fewer defenders
That’s already where we stood, and then a few developments came in and really kicked the legs out from under us. An implication of that Munroe cartoon is that this thankless developer is doing that work as part of an open-source project. For the uninitiated, open-source software projects are nonproprietary, typically involving groups of developers collaboratively volunteering to create products that are free to use and adapt, with the expectation that users donate or themselves continue to contribute to the project’s development. You may think that this is the province of small, quirky hobbyist projects, but you’d be wrong. To give just one example, WordPress, the dominant platform for web publishing, is an open source project, as are many other elements that form the basis for the modern internet. As the understanding between the open source community and the companies that profit from their work has broken down, software has become siloed in corporate domains and a lot of the shared understanding of systems has frayed.
We’re also just asking more and more of software systems. Updates come more often and try to pack in more features to demonstrate that the companies are “innovating” and growing, even when customers themselves don’t seem to want that. People have gotten used to there being apps for everything, accounts for everything, and what can seems to be a single app or tool is often a series of connected systems. A recent exposure of tens of thousands of personnel records at the Department of Homeland Security, for example, did not require hacking the government directly; the records seem to have been extracted at least in part from caches of stolen Salesforce data.
Simultaneously, the Trump administration has seen fit to massively cut back on the government entities that are meant to monitor for and prevent cyberattacks. The DOGE project led by Elon Musk — which most infamously killed off the global foreign aid agency, USAID, but also let young, inexperienced political operatives slash workforces and budgets across the federal government — cut about a third of the personnel at the Cybersecurity and Infrastructure Security Agency (CISA) and other agencies with similar jobs, partly because Trump remained irate that federal intelligence agencies had once concluded that he had been helped by Russian interference in the 2016 election. These cuts have coincided with a massive growth in cyberattacks as a tool of crime, of war, of provocation, even just of trolling.
Even largely conventional conflicts such as Russia’s war in Ukraine have seen increasing use of cyberattacks. I saw some recent discussions by experts on Bluesky pointing out that most firms specializing in cybersecurity and incident response are basically at capacity and unable to take on new clients. Most cyberattacks are ones that you’ll never hear about — ransomware and data theft attacks against institutions ranging from Fortune 500 corporations to hospital systems to small, local businesses. Just this month, commercial freezers in multiple European countries were remotely commandeered and turned to heat their contents instead, ruining tens of millions of dollars’ worth of food in an attack the authorities can’t even attribute, further demonstrating that it’s not just ephemeral cloud systems that can be hit but our real, actual critical infrastructure.
AI lights the match
Of course, if all of these vulnerable systems are like a pile of firewood, the most worrisome potential match is without a doubt the proliferation of increasingly sophisticated and accessible AI systems. Intrusions like the Hugging Face incident have demonstrated a few things, among them that AI tools can independently decide that pursuing their instructions should include such steps as exploiting vulnerabilities in existing systems, even coordinating among themselves to do so. Whether or not they are “conscious” is in my view a bit of an inane and philosophical debate that’s beside the point in terms of their documented capacities. Human-directed AI hackers can find, figure out how to exploit and devise tools to attack specific software vulnerabilities in practically no time, even against defenses that not very long ago could have required a team of dedicated hackers weeks or months to breach.
Attackers no longer even really need to have much in the way of technical skill themselves; the machines are plenty capable of doing as directed, and tools custom-built to engage in malfeasance are now on offer in ready-to-use form in Telegram forums and on the dark web. Executives and government officials have maintained that some of the recent outages have not been caused by AI, but instead were caused by attempts to fix vulnerabilities identified by AI. Setting aside how absurd that sounds, it’s still the case that this is an arms race — AI tools make hacking easier, so other AI tools are used to ward off attacks, and all of this makes systems less stable and subject to breakage. Even “regular” AI operations like web scraping can break systems and cause outages.
The case for unplugging
So what’s the upshot of all of this? To be frank, I think it’s worth considering what a less interconnected world might look like. For years now, people have pushed back against the “internet of things” trend, calling it both unnecessary and something presenting a huge surface area for attacks. Why, the question went, does your dishwasher need internet access? Maybe the swarms of travelers, email users and bank customers who were out of luck recently can provide a nudge toward unplugging from the internet what’s possible to unplug. In this light, New York City’s recent decision to keep AI and some other technologies out of K-8 classrooms, along with other school limitations, may end up looking prescient. And maybe the news this week of a study that tied school cellphone bans to improved academic performance might add to that tech-skeptical nudging.
None of these tools are holding up a classroom yet, and that’s the point. The time to decide on whether you rely on something is before you have to rely on it. Imagine that one day the systems that power this tech go down. Would most school districts in the country even still be able to operate? NYC might, and that’s worth something.
