Reading Legacy Code with Generative AI

Most discussion of Generative AI (GenAI) in software development centers on its ability to write new code. But for organizations running large, long-lived legacy systems, the more valuable application may be in reading and understanding the code that already exists. Thoughtworks has been testing this premise with an internal tool called CodeConcise, which pairs a Large Language Model (LLM) with a knowledge graph built from the abstract syntax trees of a codebase. The early results suggest that the approach can help extract both low-level requirements and high-level system explanations, which could meaningfully change the economics of legacy modernization.

Since the release of ChatGPT in November 2022, GenAI tools have largely focused on forward engineering: autocomplete, snippet generation, and test writing. Yet developers and architects spend considerably more time reading code than writing it, and the complexity of legacy systems often resists the same prompt-driven approaches that work for greenfield development. Thoughtworks’ own experience modernizing client systems points to an evolutionary approach — incremental displacement rather than a risky "Big Bang" cutover — but the cost and time to value for large systems still frequently becomes prohibitive. GenAI, the company believes, can bend that curve.

CodeConcise is the third generation of an internal accelerator designed around a simple hypothesis: that the modernization challenges most clients face are similar, and that a tool can codify the collective learnings of previous engagements. As an accelerator, it is meant to be used by Thoughtworks consultants, not packaged as a standalone product. The company is clear that CodeConcise is not the only valid way to apply GenAI to modernization — it is simply the approach its teams have had hands-on experience with so far.

Why Modernization Is Expensive

The exploration of GenAI for modernization began by identifying the problems that make legacy displacement costly or slow. The questions that emerged are practical and recurring:

  • How can we understand the implementation details of an existing system?
  • How can we understand its design without a human expert available to guide us?
  • Can we gather domain knowledge from the codebase?
  • Can we translate code idiomatically and at scale to a new technology stack?
  • How can we add automated tests as a safety net before and during the migration?
  • Can we extract the domains, subdomains, and capabilities embedded in the code?
  • How do we make behavior differences between the old and new systems explicit and intentional — so cut-overs are less painful?

Not every question applies to every engagement. The team intentionally framed these around mainframes, the most significant legacy systems in both size and complexity. The logic is that if an approach can handle the hardest cases, it can certainly apply to other stacks.

Reverse Engineering with an LLM and Knowledge Graph

The first practical goal for CodeConcise was low-level reverse engineering: drawing out the implicit requirements buried in existing code. A useful generated artifact here is a set of unit tests, but the tool’s architecture is broader than test generation. CodeConcise combines an LLM with a knowledge graph — a structured representation of relationships (data, calls, dependencies) derived from the system’s abstract syntax trees. This gives the LLM a queryable, code-backed context beyond what a raw prompt with source snippets could provide.

On the architecture of CodeConcise, the approach is to use foundational LLMs for reasoning and generation tasks, while framing their output with the knowledge graph to ground responses in the actual system under analysis. This helps avoid hallucinated explanations and provides a verifiable connection to the codebase. The team reports positive results in using this as a capability to support human analysts and architects, supplementing the work of people rather than replacing their judgment.

High-Level Explanation and Capability Discovery

At a higher level, CodeConcise aims to provide a system-level explanation: what does this application do, in plain terms, and why? For systems that have grown organically over decades with little documentation, even a rough functional overview can be a significant accelerator. Rather than presenting a sprawling block of generated prose, the tool progressively builds up the narrative with the knowledge graph ensuring each claim in the overview traces back to the code structure that supports it.

A related focus is discovering a capability map: the domains and subdomains represented by the system. Understanding and identifying the platform capabilities that the legacy system holds — and how they interact with one another — is a necessary precondition for splitting a monolith or planning its gradual replacement.

Another target is proactively identifying unused, dead, and duplicate code. Removing such code before translation reduces the scope of the modernization effort and ensures the team isn’t paying to carry technical debt forward.

Because CodeConcise combines LLMs with a knowledge graph from a codebase’s abstract syntax trees, it is able to produce explanations at the right level of granularity for eliciting requirements from stakeholders, speeding up requirements gathering. The next steps include exploring idiomatic translation of technology paradigms — for instance, translating from one language not merely syntactically, but in a way that respects the idioms and practices of the destination platform to avoid moving debt between ecosystems.

Editor’s note

This is new ground and caution is appropriate — Thoughtworks itself repeats the belief that GenAI’s power is not limited to code generation, and senior staff are careful to share their learnings in public. The immediate conclusion is practical: GenAI tools are available today for the primary, and often least glamorous, need of modernization programs: understanding the code that businesses rely on.

The immediate uses with demonstrated client value (drawing out low-level requirements and high-level explanation) are described in further detail in the follow-on pieces of this series on CodeConcise’s application.

Code as Data: The CodeConcise Pipeline

The CodeConcise approach is rooted in the idea of treating code as data. Instead of analyzing source files as raw text, the tool uses language-specific parsers to extract intrinsic structural information and map relationships between code entities. The parsing process converts code into a forest of Abstract Syntax Trees (ASTs), which are then stored in a graph database.

Edges between nodes represent relationships such as “the code in this node transfers control to the code in that node.” This granular mapping goes beyond file-level connections to capture, for example, which specific conditional branch in one file invokes code in another. This precision reduces noise in the context provided to LLMs, which is a significant advantage for files lacking high cohesion. The reduction in noise yields two benefits: the LLM remains more focused on the prompt, and the limited context window is used efficiently, allowing more information in a single prompt. This enables LLM analysis that is not constrained by how developers originally organized the code. This deterministic process is called the ingestion pipeline.

A subsequent comprehension pipeline traverses this graph using algorithms such as Depth-first Search with backtracking in post-order traversal. This enriches the graph with LLM-generated explanations at varying depths, including methods, classes, and packages. While some techniques are common across legacy stacks, the prompts used during this stage are engineered for specific languages and frameworks. As CodeConcise was applied to real production code, it became clear that the comprehension pipeline needed to be extensible to capture knowledge most relevant to a client’s domain. In one case, a query to a specific database table was better understood by Business Analysts when explained in the client’s own business terminology, particularly where no Ubiquitous Language unifies technical and business teams. Although the enriched knowledge graph is the primary output, some enrichments—such as automatically generated documentation—are valuable independently and can supplement or replace existing system documentation.

Neo4j serves as the graph database and also supports vector search, which allows the integration of the Knowledge Graph into a RAG-based frontend. Rather than relying solely on indexed retrieval, the LLM can traverse neighboring nodes to access explanations at various levels of abstraction. For a prompt like “how does authorization work when viewing card details?”, the index alone might only return results referencing user-role validation. However, with both behavioral and structural edges, the graph can pull in relevant called methods, surrounding packages, and the data structures passed into the code, leading to a more informed answer.

The quality of context gained through such traversal depends wholly on the criteria used to build and enrich the graph initially. There is no universal solution here; the right weighting of edges like INHERITS_FROM versus COMPOSED_OF depends on the architecture and development principles of the original codebase.

Reverse Engineering for Low-Level Requirements

For modernization paths like Rewrite or Replace, generating a comprehensive list of low-level requirements for the target system requires deep examination of the legacy source. These requirements guide forward-engineering teams, though not everything in the legacy system may deserve migration—some behavior may be obsolete given today’s business context. Understanding existing behavior is essential to making informed decisions about what to keep, cut, or add.

Reverse engineering the legacy codebase is time-consuming and demands both technical and business expertise. Several activities form part of this process; GenAI can accelerate each one.

Manual Code Reviews

This encompasses static and dynamic analysis. Static analysis inspects the source directly, often with the aid of stack-specific tooling, to derive dependency diagrams, CRUD (Create Read Update Delete) reports, and low-level flowcharts. Dynamic analysis observes runtime behavior in a controlled environment, with logs offering further insights into system components. GenAI enhances code review helpfully for engineers unfamiliar with the legacy tech stack. CodeConcise was used at a client site to have an LLM extract low-level requirements directly from code, with the comprehension pipeline extended to generate reports that Business Analysts found sufficient to derive requirements. This showed GenAI made non-technical staff effective participants in the reverse engineering process.

Abstracted Program Flowcharts

Low-level flowcharts tend to obscure the overall intent of the code and overwhelm analysts with detail. Collaborating with Subject Matter Experts (SMEs) is the way to produce abstracted flowcharts that convey the essential flows and intentions clear of technical jargon. Manually creating an abstract flowchart for a system might be cost-effective, but doing so for every module—at that client, roughly 1,500 modules close to 10,000 lines of code each—was inefficient. GenAI allowed Business Analysts to receive visual abstractions revealing code intentions without the clutter of conventions.

Extracting Business Rules

Model-based frameworks like BREX provide systematic methods for extracting, visualizing, and describing business variables and rules inside COBOL systems. Experience with prompt engineering at one site covered extraction of these variables and rules, though integrating BREX with AI is promising for enhancing how rules are represented and explained.

SME Validation

SMEs validate specific parts of the system and reviewed artifacts during reverse engineering cycles, offering both technical and business clarity on unresolved questions. Their scarcity usually means they are stretched thin. At the referenced client, the CodeConcise chatbot allowed analysts to clarify ambiguities and request additional explanations. That chatbot drew on an LLM and Knowledge Graph to deliver answers approximating an SME’s—easing the bottleneck without replacing the expert entirely.

Measured Acceleration

Working with this client’s COBOL/IDMS code, CodeConcise was extended to support the stack and used in a proof of concept. Prior to the PoC, reverse engineering a 10,000-line module required about six weeks: two full-time engineers for four weeks linked to wait time and SME review. Post-PoC estimates cut that down to two weeks per module—a reduction of two-thirds. For the entire modernization program, that projects to a potential saving of 240 FTE years.

From Code to Concepts: Using LLMs for High-Level System Understanding

Understanding a legacy system at a high level is often harder than understanding its individual parts. Experts hold mental models of how a system fits together, but these are rarely written down. To surface these models, we run workshops like Event Storming, where technical and business experts can map out domain language and system interactions on digital boards. This creates a shared, visual representation of complex domains and highlights integrations with external systems.

The challenge is that this discovery process depends on access to subject matter experts (SMEs), which is often a bottleneck. Many of the same constraints that apply to low-level code understanding apply here, too. So we are exploring whether LLMs can help us build these abstract models more quickly.

Our accelerator, CodeConcise, builds on Code as Data techniques, using a graph representation of a codebase to generate LLM explanations at different levels of abstraction:

  • Graph traversal strategy: The entire codebase is represented as a graph. We use traversal algorithms to enrich the graph with LLM-generated explanations at various depths.
  • Contextual knowledge: We are also processing available system documentation and connecting it to code nodes on the graph. This provides valuable business terminology and context that can enhance LLM responses to user queries.

The goal is to let CodeConcise's chatbot answer questions that would typically require an SME, even when the answer isn't directly visible in the code. Early results are promising: on an open-source COBOL repository, we successfully asked high-level questions like "what are the system's features and user interactions?" The answers were stronger when the repo included documentation. Similarly, our GenAI-powered assistant, Haiven, has shown how system-specific context lets an LLM tailor responses to a particular client environment.

Automating Capability Map Discovery

At the start of any modernization project, we catalog existing technology, processes, and people. This defines the scope and builds the business case for change, along with the technology and business roadmaps. Without this, there's no way to agree on what to include, the steps to get there, or when the work is done.

Traditionally, teams create these maps through a combination of Event Storming, Process Mapping, "following the data," and targeted code reviews. This manual effort is valuable: it aligns stakeholders on scope and builds a shared vision for delivery. Capability maps are also crucial input for a capability-driven modernization, letting you deliver features incrementally in the target system without breaking up the monolith's clean separation of concerns.

GenAI changes this picture considerably. Its strength in summarizing large text bodies can be applied to existing documentation to extract a starting point for conversation, or even the end knowledge. Technologies like Microsoft's GraphRAG can extract knowledge from documents via graph algorithm analysis.

We've also been trialing GenAI on top of the knowledge graph we build from legacy code. By asking it to summarize key capabilities of modules and then clustering these through hierarchical summarization, we can generate a map of capabilities, each linked back to the source code modules that implement them. This gives us a high-level and detailed view of the system for faster scoping and planning.

Importantly, we don't see this LLM-generated map as a replacement for the traditional methods. Manual mapping connects stakeholders using the organization's Ubiquitous Language. When a capability map already exists, it's useful to cluster code based on those known capabilities, producing results that resonate better with SMEs. Comparing the two approaches will likely yield valuable cross-insights.

Dead Code and Duplication: Static, Dynamic, and GenAI

Another part of the discovery process is understanding what's still in use and where capabilities are duplicated.

Static analysis can identify unused method calls and unreachable paragraphs, for example in COBOL, but can't tell whether an entire API endpoint or batch job is still invoked. Dynamic analysis fills that gap using observability and runtime data to determine what should stay in the modernization backlog.

For duplication detection, static analysis checks chunk-by-chunk text similarity, but it only works within the same programming language. Technology-agnostic descriptions of code abstractions—which we get from our capability extraction approach—could enable estate-wide duplication analysis. That would be a significant improvement to roadmap planning.

However, when it comes to unused code, we see little benefit in applying GenAI. Existing static analysis tools like IntelliJ or Sonar are mature and leverage code's structure effectively. Dynamic analysis from APM tools is likewise powerful for extracting this information.

Where GenAI might help is elsewhere in these workflows: they produce vast amounts of data to understand and interrogate. GenAI's knowledge of these tools could help operators use them correctly, for instance by suggesting queries for observability stacks like New Relic, which has already integrated LLMs to speed up onboarding and error resolution—an advantage you can turn toward modernization.

Why Naive Translation Falls Short

Language-to-language translation tools have existed for decades, mostly relying on static analysis with Abstract Syntax Trees (ASTs) as the intermediary. They produce syntactically correct output, but the result rarely reads like something a developer would author if starting fresh today. The shortcomings cluster into a few distinct problem areas.

Quality and Idiom Gaps

The algorithmic translation approach typically yields code that is correct but unsophisticated. It rarely matches the idiomatic conventions of the target stack. Naming is inherited from the source language, and even good legacy names rarely survive the jump cleanly — translating a procedural file-transfer routine into an object-oriented hierarchy inevitably produces awkward class and method names.

Missing Ecosystem Integration

Modern applications lean heavily on open-source libraries and frameworks, but generated code rarely leverages them. Enterprise environments compound the problem: internal libraries that tooling cannot anticipate are simply ignored. Generated code thus tends toward isolation rather than integration.

Data Precision and History Loss

  • Primitive type precision varies across languages, opening the door to silent data loss.
  • Version control history, invaluable for understanding how code evolved to its current state, loses relevance once the code has been machine-translated.

Organizations that push through these issues then face lengthy verification cycles to prove behavioral equivalence, a challenge that stings most where testing safety nets are thin. Still, code conversion retains appeal as the lowest-effort path for leapfrogging between technology paradigms.

GenAI in the Gap

GenAI cannot fix every one of these deficiencies, but it can meaningfully address the first several: code quality, idiom alignment, naming, and library usage. The more promising direction borrows from Refactoring principles — making the paradigm jump safely and incrementally rather than as a one-shot translation. Success stories already exist in the field, including OpenRewrite and Slack's AI-powered conversion from Enzyme to React Testing Library.

Outlook

The expansion of GenAI across software roles is well underway, and modernization of the vast installed base of critical systems is one of its more consequential applications. Those systems tend to be large, layered with years of patches, and understood fully by only a handful of experts. Evolution, not replacement, has been the safest route, but the cost and timeline have often been prohibitive. GenAI has the potential to shift that cost/time/value equation, and ongoing experimentation — including work on the CodeConcise accelerator — aims to identify what works, what GenAI might be able to solve, and where it still fails. There is no universal fix; each environment has its own nuances, but shared patterns can guide the effort. Wider industry experimentation with "GenAI for Modernization" and published results would benefit everyone involved in legacy displacement.