Sooner or later, someone in a planning meeting says the words: “Honestly, we should just rewrite it.” Everyone nods. The engineers look relieved, product looks nervous, and finance hasn’t heard yet. I’ve been in that room more times than I can count, as the person proposing the rewrite, the person arguing against it, and lately as the person leadership asks to make the final call. After many years as a Principal Engineer, I have one firm belief about rewrite vs refactor legacy code: the choice is rarely about the code. It’s about risk, knowledge, money, and how much patience the business has.
This article explains how I decide whether to rewrite or refactor legacy code. It isn’t a template. Think of it as what I’ve learned from rewrites that worked, rewrites that nearly killed products, and refactors that dragged on so long nobody remembered why they started.
First, Let’s Agree on What It Means to Refactor Legacy Code
People use “refactor” to mean anything from renaming a variable to replacing the whole persistence layer. That vagueness causes real problems, because two people can agree to refactor legacy code while picturing completely different budgets.
Martin Fowler has the cleanest definition, and I still use it. Refactoring means to restructure software by applying a series of refactorings without changing its observable behavior. The key part is “without changing its observable behavior.” If users or calling systems can see a difference, you aren’t refactoring. You’re changing the product, and that’s a separate conversation. Fowler also points out that the system is kept fully working after each refactoring, reducing the chances that a system can get seriously broken during the restructuring.
A rewrite means building a new implementation that replaces the old one. You write fresh code, often with a new architecture, sometimes in a new language, and eventually you switch traffic from the old system to the new one. The old code stops being the thing you change and becomes the thing you copy.
Between those two sits a third option that most good teams end up choosing: incremental replacement, where you rewrite pieces while the old system keeps running. I’ll spend a lot of time on that, because it’s usually the right answer once you get past the false binary.
Why the Rewrite Is So Tempting
Let’s be honest about why engineers want to rewrite. Some of the reasons are good and some are about how we feel.
The good reasons are real. The legacy system is slow to change. Every feature costs three times what it should. Onboarding a new engineer takes months because the codebase has no structure a reasonable person could learn. The framework is past end of life, and the security team is uneasy after the last application security assessment.
The emotional reasons are just as real, and we rarely say them out loud. Old code is unpleasant to work in. Nobody gets promoted for patching a ten year old billing module. A rewrite feels like a fresh start, and new code always looks cleaner than old code, partly because it hasn’t been through production yet.
Joel Spolsky captured this better than anyone in his famous essay. He wrote that programmers are, in their hearts, architects, and the first thing they want to do when they get to a site is to bulldoze the place flat and build something grand. That was in 2000 and it’s still true. I’ve watched brilliant engineers argue against the slow work to refactor legacy code, with excellent slides, when the real driver was that they were tired of it.
Being tired of the code is a valid feeling. It just isn’t a business case.
The Cost Nobody Puts in the Estimate
Every rewrite estimate I’ve seen leaves out the same things, whether it’s an internal plan or a vendor’s software development cost estimate. When a rewrite fails, it almost always fails for one of these reasons.
The old code knows things you don’t
That strange conditional on line 400 of the order service, the one that checks whether a customer ID starts with a particular prefix? It’s there because of a migration from an acquired company eight years ago. Nobody wrote it down. The engineer who added it left long ago. If you leave it out of the rewrite, a few hundred enterprise customers lose their invoices the week you cut over.
Legacy code is ugly partly because it has absorbed years of edge cases, incident fixes, and customer workarounds. Spolsky’s main example was Netscape. He noted that there never was a version 5.0. The last major release, version 4.0, was released almost three years ago when version 6.0 finally reached beta. Three years without a major release is a long time to stand still while competitors keep shipping.
Your system has more contracts than you think
Hyrum Wright, an engineer at Google, put words to something every platform engineer learns the hard way. Hyrum’s Law says that with a large enough number of users, all observable behaviors of your system will be depended on by somebody. The order your API returns results in, the exact wording of an error message, a timeout that happens to be 30 seconds: someone downstream depends on all of it.
That’s why rewrites so often lead to “bug for bug compatibility” work, where the team carefully rebuilds behavior they know is wrong because a partner integration breaks without it. That work doesn’t show up in the first estimate. It shows up in month fourteen.
The target keeps moving
While you build the new system, the business doesn’t stop. Sales still signs deals. Product still needs features. Regulators still publish new rules. Every one of those changes has to land in the legacy system, because that’s what customers use, and then be built again in the new system. I’ve seen teams spend more than half their rewrite capacity just keeping up with changes to the system they were trying to replace.
You run two systems for longer than you planned
Nobody plans to run both systems for two years, but plenty of teams end up doing it. That means two sets of on call rotations, two deployment pipelines, two sets of monitoring dashboards, and data synchronization between them that becomes its own fragile product. It also doubles the work of securing legacy applications that were supposed to be retired by now. Doubled operating cost is one of the most underestimated parts of any rewrite, and it’s a cost you avoid entirely when you refactor legacy code in place.
When to Refactor Legacy Code Instead of Rewriting
In my experience, the decision to refactor legacy code is the right default for most systems, as long as a few conditions hold.
The architecture is basically sound. If the problem is messy code inside a reasonable structure, such as huge classes, unclear names, duplicated logic, or missing tests, refactoring will get you a long way. You don’t need to tear down a house because the kitchen is a mess.
The system is core to revenue and can’t pause. If this platform pays the bills and customers expect regular improvements, you can’t afford a long period with no visible progress. Refactoring lets you ship value while you improve the foundation.
The knowledge lives in the code. When the original authors are gone and the documentation is thin, the code is the only reliable specification you have. Choosing to refactor legacy code keeps that specification intact while you make it readable.
You can make changes in small, safe steps. If you can get parts of the system under test and deploy often, refactoring compounds. Each small improvement makes the next one cheaper.
There’s a useful heuristic from Naresh Jain that InfoQ reported years ago. He suggested teams should refactor when the code is difficult to understand and the team is not sure of what it does. Rewrite when it is clear what the code does but it is difficult to understand. I like it because it reverses most people’s instinct. Confusing code you don’t understand is exactly the code you shouldn’t rewrite, because you’d be rebuilding something you can’t describe.
When a Rewrite Is Actually Justified
I’m not against rewrites. I’ve led a few that were the right call and paid off well. Over time I’ve found a rewrite is justified when one or more of these is true.
The platform itself is dying. If the language runtime, framework, or vendor product is unsupported and you can’t get security patches, the question isn’t whether to move but how. The risks of end-of-support software only grow with time, and trying to refactor legacy code inside a dead platform just polishes something that’s going away. This comes up often when teams need to migrate legacy Windows applications off desktop frameworks that no longer get updates.
The architecture can’t meet a hard requirement. Sometimes the core design can’t support what the business needs now. A single tenant system that has to become multi tenant. A batch system that has to become real time. A monolith with latency limits it can’t physically meet. When the constraint is structural, refactoring hits a ceiling.
The domain has changed so much that the old model is wrong. If the business has pivoted and the legacy system’s core concepts no longer match how the company works, rebuilding around a correct domain model can be cheaper than bending the old one indefinitely.
The system is small and well understood. Rewriting a 5,000 line service with clear behavior and good test coverage is a very different bet from rewriting a 2 million line platform. Small rewrites with clear boundaries are often the fastest path. Smaller companies facing this choice should also ask whether custom software development for small business pays off at all, or whether an off the shelf tool now does the job.
Even then, I almost never approve a full cutover rewrite done in one shot. I approve a rewrite carried out incrementally.
The Middle Path: Rewrite and Refactor Legacy Code One Slice at a Time
The best answer to rewrite vs refactor legacy code is often “both, in sequence, one slice at a time.” The pattern most people point to is the Strangler Fig, named by Martin Fowler after the vines he saw in the Queensland rainforest that grow around a host tree and eventually replace it. It has become one of the most proven patterns in legacy application modernization.
The idea is simple. A routing layer goes in front of the legacy system. New functionality, or rebuilt existing functionality, lives in a new system beside the old one. Traffic moves over one capability at a time. Over time the new system grows, the old one shrinks, and eventually the old one gets switched off. In his original write up, Fowler argued that the most important reason to consider a strangler fig application over a cut-over rewrite is reduced risk. Microsoft’s architecture guidance describes the same benefit: it allows the existing application to continue functioning during the modernization effort.
What I value most about this approach isn’t the technical elegance. It’s that you can stop. If priorities change halfway through, which they will, you still have a working system and real value already delivered. A big bang rewrite that stops halfway leaves you with nothing.
A real example worth studying
Shopify’s storefront rewrite is one of the better public case studies, because it was a true rewrite carried out with incremental discipline. Their engineering team wrote that in January 2019, we set out to rewrite the critical software that powers all online storefronts on Shopify’s platform to offer the fastest online shopping experience possible, entirely from scratch and without downtime.
The detail I always point engineers to is the verification work. In a follow up post, the team explained that they built our storefront verification system with Lua modules and used that system to ensure our new storefront achieved parity with our legacy storefront. They also adjusted load balancers to shift traffic between the old and new storefronts gradually. That’s what a responsible rewrite looks like: new code, yes, but proven against the old system with real traffic before anyone trusts it.
A Framework to Decide Whether to Rewrite or Refactor Legacy Code
When an executive asks me whether we should rewrite something, I don’t answer straight away. I go through these questions with the team, and I write the answers down, because writing forces honesty.
Start with the outcome and the problem
- What business outcome are we trying to unlock? “Cleaner code” isn’t an outcome. “Ship pricing changes in days instead of weeks” is. “Pass the SOC 2 audit” is. If nobody can name the outcome, we stop here.
- Is the problem in the code or in the architecture? Code problems tell you to refactor legacy code. Architecture problems may call for replacing components, sometimes with a product you buy rather than build. A clear build vs. buy software decision belongs here. Be specific about which parts are which.
Check what you actually know before you refactor legacy code
- How well do we understand what the system does today? If we can’t describe current behavior in detail, including edge cases, we aren’t ready to rewrite anything. The first job is discovery.
- What’s our test coverage around the behavior that matters? Without a safety net, both paths are dangerous. With one, both become much safer.
Test the plan against reality
- Can we slice it? Can we find a seam, such as a module, an endpoint, a workflow, or a customer segment, where we can replace one piece and route traffic to it? If yes, incremental replacement is available. If no, find out why.
- How long can the business tolerate reduced feature delivery? Be realistic. A two year rewrite isn’t an option when leadership will only accept a quarter of slower delivery, no matter how attractive the plan looks.
- Who holds the domain knowledge, and will they still be here? A rewrite without the people who understand the domain is just guessing with better syntax. If you’re bringing in outside help, choosing a software development partner who will work alongside those people matters more than the hourly rate.
- What does stopping halfway look like? For every plan, ask what happens if funding is cut at the midpoint. If the answer is “we have two broken systems,” redesign the plan.
The Dell engineering blog made a point years ago that I repeat often: the decision to rewrite or not rewrite is a business decision first. Engineers inform the choice. The business owns the risk.
Characterization Tests: How to Refactor Legacy Code Safely
Whether you rewrite or refactor legacy code, you need to know what the system actually does, not what the documentation claims. This is where Michael Feathers’ book Working Effectively with Legacy Code is still essential reading. Feathers idiosyncratically defines as legacy code, code that is lacking tests, and once you’ve worked on enough of these systems, that definition feels exactly right.
His most useful contribution is the characterization test. Unlike a normal test, which checks what code should do, a characterization test records what the code does now, bugs and all. As one summary puts it, the conservative path is to assume that the old behavior is the required behavior.
Here’s how I apply it in practice. Before touching a legacy module, the team writes tests that capture its current outputs for a wide range of inputs. When a result looks wrong, we don’t fix it in the test. We flag it and ask the business whether it’s a bug or a feature someone depends on. You’d be surprised how often it’s the second.
For rewrites, the same idea scales up. Record production requests and responses, replay them against the new system, and compare. Shadow traffic, where the new system processes real requests without its responses reaching users, is one of the most effective tools I know for building confidence before cutover.
How to Pitch a Plan to Refactor Legacy Code to the Business
Engineers often lose this argument, in both directions, because they pitch it in engineering terms. Executives don’t care about cyclomatic complexity. They care about speed, risk, and cost.
When I recommend that we refactor legacy code, I frame it as protecting delivery: “We’ll keep shipping features at the current pace while spending about 20 percent of capacity on making the next features cheaper. Within two quarters, the changes your team asks for most often will be noticeably faster.”
For an incremental rewrite, I frame it as a sequence of bets that each pay off: “First we move checkout, which is where most of our incidents come from. That alone cuts our on call load in half. If it goes well, we move inventory next. At every stage we can stop and still be better off.”
I almost never pitch a big bang rewrite. On the rare occasions I’ve had to, I’ve insisted on clear milestones, a parallel run, and a named executive sponsor who understands the risk. A rewrite without executive cover gets cancelled the first time revenue dips.
Mistakes Teams Make When They Rewrite or Refactor Legacy Code
A few patterns come up again and again, regardless of company size or tech stack.
Process and scope mistakes
Rewriting to escape a team problem. If the legacy system is a mess because the team had no code review, no tests, and constant deadline pressure, a rewrite built under the same conditions will end up just as messy, just faster. Fix the process first.
Adding features to the rewrite. “While we’re at it” kills rewrites. The first goal is parity. New features come after cutover, or they go into the legacy system and get ported later. Mixing the two makes it impossible to tell whether the new system is correct.
Data, direction, and people mistakes
Treating the data migration as an afterthought. Code is the easy part. Data with fifteen years of inconsistent formats, orphaned records, and undocumented meaning is where the time goes. Start the data work on day one.
Refactoring with no destination. The opposite mistake is just as common. Teams refactor legacy code for years with no clear target architecture, making local improvements that never add up. Refactoring needs a direction, written down and agreed on.
Ignoring the people. The engineers who know the legacy system are often the least excited about the new one, and the most valuable during the transition. Bring them in early. Give them ownership. Their knowledge is the difference between a smooth cutover and a painful one.
Final Thoughts on When to Refactor Legacy Code
After all these years, I don’t think rewrite vs refactor legacy code is really a choice between two options. It’s a spectrum, and most of the time the right answer is to refactor what you can, replace what you must, and do both in small, reversible steps while the business keeps running.
Rewrites aren’t evil and refactoring isn’t always safe. What separates the teams that succeed from the ones that don’t is discipline: knowing exactly what the system does, having tests to prove it, picking a seam, and moving one piece at a time with real traffic as the final judge.
If you take one thing away, make it this. Before anyone writes a line of new code, make sure you can describe, in detail and in writing, everything the old code does. If you can’t, you aren’t ready to make the call yet. And that’s fine. Finding out is the first step.
Frequently Asked Questions
When should you rewrite vs refactor legacy code?
Refactoring improves the internal structure of existing code without changing what users or other systems can observe. Rewriting builds a new implementation to replace the old one. Refactor legacy code when the architecture is sound, and consider a rewrite only when the platform or architecture is the real blocker. Martin Fowler’s Definition of Refactoring is the clearest explanation of where that line sits.
Is it better to rewrite software from scratch or refactor legacy code?
In most cases it’s safer to refactor, but there are exceptions. A rewrite makes sense when the platform is unsupported, the architecture can’t meet a hard requirement, or the system is small and well understood. Joel Spolsky’s essay Things You Should Never Do, Part I is the classic argument for caution.
Can the Strangler Fig pattern help you refactor legacy code?
Yes. It replaces a legacy system gradually by routing traffic through a layer that sends some requests to new code and the rest to the old system, until the old system can be retired. See Fowler’s Strangler Fig Application and the Microsoft Azure Strangler Fig pattern guide for implementation detail.
How do I refactor legacy code that has no tests?
Start with characterization tests that record what the code does today, then refactor in small steps behind that safety net. Michael Feathers covers this in depth in Working Effectively with Legacy Code, and the characterization test overview is a good quick introduction.
How long does a legacy code rewrite usually take?
It depends on size and complexity, but almost always longer than the first estimate, because of hidden behavior, ongoing feature work, and data migration. Shopify’s storefront rewrite case study shows a realistic timeline for a large production rewrite.
Why do so many software rewrites fail?
Mostly because teams underestimate how much undocumented behavior the old system has, and how much downstream systems rely on it. Hyrum’s Law explains why every observable behavior eventually becomes a contract someone depends on.
References
- Fowler, Martin. Definition Of Refactoring. martinfowler.com.
- Fowler, Martin. Strangler Fig Application. martinfowler.com.
- Fowler, Martin. Original Strangler Fig Application. martinfowler.com.
- Fowler, Martin. Refactoring.com.
- Spolsky, Joel. Things You Should Never Do, Part I. Joel on Software.
- Wright, Hyrum. Hyrum’s Law.
- Feathers, Michael. Working Effectively with Legacy Code. O’Reilly Media.
- Characterization Test. Wikipedia.
- Microsoft. Strangler Fig Pattern. Azure Architecture Center.
- Amazon Web Services. Strangler Fig Pattern. AWS Prescriptive Guidance.
- Shopify Engineering. How Shopify Reduced Storefront Response Times with a Rewrite.
- Shopify Engineering. How to Dynamically Route Storefront Traffic.
- InfoQ. Refactor or Rewrite.
- Dell Technologies. The Rewrite Dilemma: Transforming Legacy Applications.
- TechTarget. Refactor vs. Rewrite: Deciding How to Fix Problem Software.