Skip to content

The maintenance bill nobody budgeted for: how AI is raising the cost of keeping software running

This was originally published at https://confusedtechguy.substack.com/p/the-maintenance-bill-nobody-budgeted

Maintenance has always dominated the true cost of software: most of a system’s lifetime cost arrives after it launches. What has changed in the past two years is what makes up that bill. AI, in the hands of both software teams and criminals, has added costs that did not exist in 2023: defending against attacks that move at machine speed, checking and repairing AI-written code, and managing the risks in the shared building blocks nearly all modern software is made from. Here is what those overheads look like in mid-2026.

Attackers automated first

The most direct new cost is defence. IBM’s 2025 Cost of a Data Breach Report found that 16% of the breaches it studied involved attackers using AI tools [1], and that only counts cases where AI use could be identified. The bigger shift is speed. In November 2025, the AI company Anthropic disclosed the first documented spying campaign run largely by AI: a state-sponsored group tricked an AI assistant into doing the attacking for them, with the AI carrying out an estimated 80 to 90 per cent of the work across roughly thirty target organisations at a pace no human team could match [2]. The cost of mounting a sophisticated attack has fallen sharply; the cost of defending against one has not.

The numbers on software flaws tell the same story. More than 46,000 software security flaws were publicly reported in 2025, a record, averaging around 127 every day [3]. Criminals actively used 883 of them in real attacks, far more than the 245 added to the US government’s official warning list [3]. And the gap between a flaw becoming known and criminals using it has collapsed, from a median of 771 days in 2018 to roughly four hours in 2024; by 2025, most flaws used in attacks were being used before they were even publicly announced [4]. Waiting weeks to apply fixes is no longer safe. Round-the-clock monitoring, automatic updating and the ability to release an emergency fix at any time have gone from good practice to a baseline cost in tools and people on call.

AI-written code is cheap to produce and expensive to keep

The second cost arrives with the software itself. AI assistants have made writing code dramatically faster; mounting evidence says they have also made the result more expensive to look after. An analysis by GitClear of 211 million lines of code changes between 2020 and 2024 found that the amount of duplicated code increased eightfold during 2024, with copying and pasting overtaking careful reorganisation for the first time on record [5]. Duplication matters: when the same instructions exist in several places, every fix must be made in every copy, and a missed copy becomes a fault waiting to surface.

Security is starker. Veracode tested more than 100 AI models on 80 realistic programming tasks and found the code they produced contained security weaknesses in 45% of cases [6]. Its 2026 follow-up found that while AI-written code now almost always runs correctly, the share that is actually safe has been stuck near 55% for two years [7]. People are absorbing the difference: in research by Harness, 67% of software developers say they now spend more time fixing problems in AI-written code, 68% more time on AI-related security weaknesses, and 60% of organisations have no formal process for checking such code at all [8]. A meaningful share of the speed gained up front is paid back later in checking, repair and rework: maintenance by another name.

The shared building blocks of software are under attack

Modern software is mostly other people’s software. A typical business application is assembled largely from free, publicly shared blocks of code, written and looked after by people the company has never met and rarely paid; using one block usually pulls in dozens of others it relies on. That has always carried some risk; AI has turned it into a fast-moving liability, in three ways.

First, AI invents building blocks that do not exist, and criminals are making them real. A 2025 study examined 2.23 million samples of AI-written code from 16 popular models and found nearly one in five recommended at least one component that does not exist, more than 205,000 made-up names in total [9]. The mistakes are predictable too: 58% recurred when the same requests were repeated, so criminals can watch what the AI recommends, publish malicious software under those exact names, and wait for AI-assisted developers to install it [9]. This turns the economics of defence upside down: the attacker publishes a poisoned name once, cheaply; the defender must check every component an AI suggests, forever.

Second, tampering with genuine building blocks has become industrialised. A self-spreading malicious program known as Shai-Hulud, first detected in September 2025, stole the publishing credentials of volunteers who look after popular components and pushed out infected versions, compromising more than 500 in its first wave [10]. Investigators assessed that parts of the program were probably written with AI [11]. A second wave in November tampered with components used in major products including Zapier, PostHog and Postman, touching around 700 in total [12]; successor campaigns were still compromising hundreds more well into 2026 [13]. A single tampered component, live for only a few hours, can hand attackers the digital keys of every business whose systems downloaded it in that window.

Third, and least visible, the volunteers who look after these building blocks are being buried. In January 2026 the curl project, a small piece of free software that runs inside billions of devices, shut down its program of paying rewards for reported security flaws after years of fake, AI-written reports: confident, professional-looking, frequently fabricated, each demanding human time to read, test and reject [14]. By mid-2025 only around 5% of reports received were genuine, roughly 20% appeared to be AI-generated, and weekly volumes at times surged to eight times the normal rate [15]. Other major projects have tightened their reporting rules for the same reason, and the Open Source Security Foundation is now writing guidance for volunteers drowning in convincing fakes [16]. The risk for businesses downstream is twofold: exhausted volunteers ship fewer fixes, more slowly; and, as curl’s founder has warned, a constant flood of fakes raises the odds that a real flaw in widely used software slips through unnoticed.

Looking after these building blocks can therefore no longer be occasional housekeeping. An accurate inventory of every component, checks on where each came from, monitoring for tampering, careful protection of publishing credentials, and contractual clarity over who carries the risk of third-party code are now permanent tasks. Regulators in financial services and critical infrastructure increasingly treat them as the responsibility of the business running the software, not the volunteers who wrote its parts.

What this means for budgets

Pulled together, the 2026 maintenance ledger has at least four new or sharply inflated entries: constant, automated updating; extra capacity to check and repair AI-written code; permanent controls that treat every borrowed building block as untrusted until proven otherwise; and the people and tools needed to sort genuine problem reports from convincing machine-made fakes.

None of this argues against using AI to build software; the gains are real and the direction is one-way. It argues for honest accounting. Organisations that budget for AI purely as a cost reducer will find the savings quietly consumed by line items they never planned for. Those that treat checking, security and the care of their software’s building blocks as first-class costs of adopting AI, and fund them accordingly, will be the ones who keep the dividend.

If you’d like to chat about how SGY can help with your security needs, get in touch today:

Contact us

References

  1. IBM Newsroom, IBM Report: 13% of Organizations Reported Breaches of AI Models or Applications, 97% of Which Lacked Proper AI Access Controls (Cost of a Data Breach Report 2025), 30 July 2025. IBM Report: 13% Of Organizations Reported Breaches Of AI Models Or Applications, 97% Of Which Reported Lacking Proper AI Access Controls
  2. Anthropic, Disrupting the first reported AI-orchestrated cyber espionage campaign (full report), November 2025. https://assets.anthropic.com/m/ec212e6566a0d47/original/Disrupting-the-first-reported-AI-orchestrated-cyber-espionage-campaign.pdf
  3. Zafran Security, The 2025 Spike in Vulnerabilities Isn’t the Full Story, January 2026. https://www.zafran.io/resources/the-2025-spike-in-vulnerabilities-isnt-the-full-story
  4. Resilient Cyber, The Zero Day Clock Is Ticking: Why the Collapse of Exploitation Timelines Changes Everything, March 2026. The Zero Day Clock Is Ticking: Why the Collapse of Exploitation Timelines Changes Everything
  5. GitClear, AI Copilot Code Quality: 2025 Data Suggests 4x Growth in Code Clones (research report analysing 211M changed lines, 2020–2024). AI Copilot Code Quality: 2025 Data Suggests 4x Growth in Code Clones – GitClear
  6. Veracode, 2025 GenAI Code Security Report. Veracode October 2025 Update: GenAI Code Security Report |
  7. Veracode, Spring 2026 GenAI Code Security Update: Despite Claims, AI Models Are Still Failing Security, March 2026. Spring 2026 GenAI Code Security Update: Despite Claims, AI Models Are Still Failing Security | Veracode
  8. Harness, The State of Software Delivery Report 2025: The Role of AI in the SDLC. The State of Software Delivery Report 2025: The Role of AI in the SDLC (summary of key findings: Harness Releases its State of Software Delivery Report: Developers Excited by Promise of AI to Combat Burnout, But Security and Governance Gaps Persist )
  9. SecurityWeek, AI Hallucinations Create a New Software Supply Chain Threat, April 2025 (covering Spracklen et al., “We Have a Package for You!”, USENIX Security 2025). AI Hallucinations Create a New Software Supply Chain Threat
  10. CISA, Widespread Supply Chain Compromise Impacting npm Ecosystem (Alert), 23 September 2025. Widespread Supply Chain Compromise Impacting npm Ecosystem | CISA
  11. Palo Alto Networks Unit 42, “Shai-Hulud” Worm Compromises npm Ecosystem in Supply Chain Attack. “Shai-Hulud” Worm Compromises npm Ecosystem in Supply Chain Attack (Updated November 26)
  12. Wiz Research, Shai-Hulud 2.0: Ongoing Supply Chain Attack: 25K+ Repos Exposed, November 2025. Sha1-Hulud 2.0 Supply Chain Attack: 25K+ Repos Exposed | Wiz Blog
  13. Microsoft Security Blog, Shai-Hulud 2.0: Guidance for detecting, investigating, and defending against the supply chain attack. Shai-Hulud 2.0: Guidance for detecting, investigating, and defending against the supply chain attack | Microsoft Security Blog
  14. The Register, Curl shutters bug bounty program to remove incentive for submitting AI slop, January 2026. Curl shutters bug bounty program to stop AI slop
  15. The Register, Curl creator mulls nixing bug bounty awards to stop AI slop, 15 July 2025. Curl creator mulls nixing bug bounty awards to stop AI slop
  16. OpenSSF Vulnerability Disclosures Working Group, AI-SLOP: Develop best current practices for Open Source maintainers (Issue #178). AI-SLOP: Develop best current practises for Open Source maintainers · Issue #178 · ossf/wg-vulnerability-disclosures