The Refactoring Paradox: How LLMs Are Killing the Monoliths They Were Supposed to Live Inside

For most of the last decade, the dominant enterprise AI narrative was one of careful containment. Large language models would be layered on top of legacy systems — a chatbot here, a document summarizer there — while the monolithic core remained untouched, its COBOL subroutines and mainframe batch jobs humming along beneath a veneer of modernity. The strategy had a name: “leave and layer.” By 2026, it has hit a wall.

The Modernization Trap

The leave-and-layer approach was always a deferral, not a solution. Enterprises that spent 2019–2024 building API wrappers around 40-year-old systems have discovered that the architectural debt doesn’t compound linearly — it compounds exponentially. Today, the average Fortune 500 financial institution runs an estimated 35–50 million lines of legacy COBOL, much of it undocumented, written by engineers who retired before the iPhone existed. These systems handle trillions in daily transactions and encode decades of regulatory logic that exists nowhere else — not in wikis, not in runbooks, not in the memory of any living employee.

The promise of microservices modernization — decompose the monolith, containerize the components, iterate independently — broke on the rocks of this opacity. You cannot decompose what you cannot read. And until recently, reading a million-line COBOL codebase with sufficient fidelity to extract its business logic required armies of specialist consultants and multi-year timelines that boards simply wouldn’t fund.

The irony arrives here: the technology enterprises were most cautious about adopting has become the only practical tool for breaking the logjam.

How LLMs Parse What Humans Can’t

Modern LLM-based code analysis platforms — tools from vendors including IBM, Moderne, and a growing cohort of startups — do something that grep and static analysis never could: they reason about code. Feeding a COBOL program to a well-tuned model doesn’t just produce a call graph. It produces annotated natural-language explanations of business intent, flags regulatory compliance logic embedded in conditional branches, identifies implicit service boundaries based on data ownership patterns, and generates candidate API contracts for extracted modules.

The workflow typically proceeds in three phases:

• Discovery: The LLM ingests the full codebase and produces a domain map — identifying clusters of functionality (claims processing, premium calculation, customer identity) even when they are physically entangled in the same program files.
• Decomposition planning: The model proposes extraction sequences, surfacing dependencies and suggesting which modules can be peeled away first without destabilizing the core.
• Contract generation: For each candidate microservice, the tool generates interface definitions, data schemas, and — critically — the undocumented business rules that must be preserved.

What took a team of 20 COBOL specialists 18 months now takes a team of 5 engineers and a focused LLM pipeline roughly 6–10 weeks for initial mapping. The human specialists don’t disappear; their role shifts from reading code to validating the model’s interpretations — a far more tractable task.

Where It’s Working: Banking, Insurance, and Government

The evidence is no longer theoretical. A major Scandinavian bank publicly disclosed in late 2025 that it had used AI-assisted decomposition to migrate its core lending platform from a 1980s mainframe architecture to a containerized microservices stack in 14 months — a project it had previously estimated would require five years and $200M using conventional approaches. The AI tooling reduced the discovery and mapping phase from projected years to weeks.

In the U.S. insurance sector, several carriers have begun using LLM pipelines to extract actuarial logic embedded in legacy policy administration systems — logic that had effectively become unauditable because no living underwriter understood its full scope. The extracted rules, now expressed as versioned, testable business logic modules, are not only easier to modernize but easier to comply with under evolving state regulations.

Government adoption is perhaps the most significant signal. Agencies in the UK and Australia have initiated pilot programs applying LLM-based analysis to citizen-facing benefit calculation systems, many of which run on COBOL stacks that predate the public internet. The goal is not just modernization but explainability — regulators increasingly require that automated benefit determinations be auditable, and the AI-generated documentation these tools produce is becoming a compliance artifact in its own right.

The Irony, Explained

Why did enterprises fear LLMs in the first place? Largely because of governance concerns: unpredictability, hallucination, regulatory exposure, and the risk of encoding AI outputs into production systems. These concerns were legitimate — and they remain legitimate for high-stakes inference tasks.

But code analysis is a different use case. The LLM isn’t making real-time decisions about a loan application or a benefits claim. It’s producing a map — a structured hypothesis about what legacy code does, which human engineers then verify. The risk profile is fundamentally different. A hallucinated business rule caught in review is a bug in a document, not a production incident.

This is the core of the refactoring paradox: the governance frameworks enterprises built to keep LLMs out of their critical systems turn out to be precisely what makes LLMs safe for this use case. Human-in-the-loop validation, which feels like a limitation in autonomous AI workflows, is the appropriate architecture for legacy code interpretation.

Risks, Limitations, and What Success Actually Requires

None of this is without friction. Organizations pursuing AI-assisted modernization should account for several hard realities:

• Model limitations on truly archaic dialects: Some COBOL variants and assembly-level modules challenge even frontier models. Hybrid approaches combining LLM analysis with traditional static analysis tools remain necessary.
• The validation bottleneck: Generating maps fast is only valuable if the organization has engineers capable of validating them. Skills gaps in COBOL literacy don’t disappear — they get reframed. Retraining and specialist retention remain critical investments.
• Scope creep risk: AI tooling that makes discovery cheap can tempt organizations into over-ambitious decomposition plans. Incremental migration discipline — extracting one bounded context at a time, proving stability before proceeding — remains the difference between successful programs and expensive failures.
• Data migration complexity: Code decomposition is often the easier half. Disentangling the monolithic data stores that legacy systems share requires careful domain modeling that AI tools assist but cannot replace.

The enterprises that will capture the most value from this shift are those that treat AI-assisted decomposition as a program capability, not a one-time project — embedding the tooling into ongoing engineering practice rather than deploying it as a point solution.

The Bottom Line

The technology that enterprises spent years carefully walling off from their legacy cores has become the key to unlocking them. That’s not a failure of governance strategy — it’s a maturation of it. Understanding where AI judgment can be trusted, and building workflows that leverage it accordingly, is what separates organizations that modernize from those that continue to defer. The monolith had a good run. Its obituary is being written in the language that built it.