The Greenfield Project Trap

Most engineers want to prove themselves on a greenfield project. It's not a great idea.

A green field.

At some stage in their career, every developer will convince themselves that they want to work on a greenfield project.

When developers say greenfield, what they imagine is them being in charge of designing an entire feature or product from beginning to end. To build a robust, reliable architecture that will work well no matter what else gets thrown at it. To build just the right set of features to make every user happy with no messy code or awkward compromises. Do it right, do it once once - it’s intellectual bliss for them and it’ll surely print money too. That doesn’t sound too bad.

The desire to be technically challenged is great, but as we’ll see, a greenfields project is not a great way to get that. What you will get is confusion and complexity.

Projects without challenges

It’s not how they’d phrase it, but in my experience, what people imagine when they say greenfield is a project without any challenges outside of their control.

Advocates will make two big arguments for greenfield projects. They’ll say working in the current codebase (“brownfield”) is hard because of “legacy code” and “legacy requirements”. Therefore, we need a fresh new codebase (“greenfield”) where these challenges don’t exist. Unfortunately these challenges are what makes software development interesting (and useful). Ignore them, and your project will fail.

Legacy code means connecting to other pieces of software. Very few bits of software that live in total isolation get used. It might sound nice that a greenfield project doesn’t need to interact with “legacy systems”, use a crappy old language or framework, or work with a database schema that doesn’t quite fit the solution. But by not doing any of that you’re moving the problem elsewhere. Eventually the new code will need to connect to the old database schema; if not now, then later through an even more complex API layer.

Legacy requirements means ensuring the software does useful things. engineers will sometimes describe a requirement as legacy as a polite way of saying “we don’t want to implement it because it’s annoying”. But most requirements exist for good reasons. After all, requirement just means “things people pay us to have the product do”. Most requirements are ugly and complex, that’s why we get paid to solve them! Declaring that some requirements are too annoying to be worth getting paid for is sometimes a good idea, but it’s not a decision you should make lightly because it’ll make your code more elegant.

To better understand why greenfield projects fail, we need to look at the two types of project: greenfield products, and greenfield features.

Greenfield products

This looks like someone asking if as part of their current job they can build a new product, in a new codebase or repository. (This is different to quitting their job to start a new business.)

Sometimes this is done to test a new hypothesis, validate a market, or try some new product idea. Sometimes it’s just to scratch that greenfields itch and prove one’s engineering prowess. The only thing that’s certain is that it’ll be using the most cutting edge technology and that the code will be slick, performant, and well crafted.

I’ve fallen into this trap a bunch of the times, and I eventually learned that the grass isn’t greener in a new codebase. The most likely outcome is that it won’t get any users, which means nobody will want to keep investing in you and you’ll have to stop working on it. But if you are lucky enough to get users, then you have to re-do all the annoying bits that your current codebase already has. I hope you like writing billing system integrations, creating authentication systems, and configuring rate limiting.

If the goal of the new product was to test a hypothesis or a market, it’s better to do that in an existing codebase, or to find a similar way to prototype the idea without any code. That way if it works you can keep building on the foundations you have rather than having to rebuild them every time you learn anything new.

Don’t launch new products only so you can play with new technologies.

Greenfield features

This looks like someone asking if they can build the next feature using a different technical stack or architecture to all the other ones.

This tends to happen when developers have issues with the existing stack. Sometimes this happens because the stack is totally shit to work with, sometimes it comes because a new technology offers legitimate new opportunities. Sometimes they’re just bored.

When developers ask for this, the risk is them assuming that their new feature will be able to live in a vacuum, untainted by any of the other shit code near it. In reality the only code that lives in a vacuum is code that’s not used. Eventually the new feature will still need to connect to the existing stack, either at a feature level (eg. wiring it up with the billing or authentication systems) or lower (eg. it’s probably in the same database, so uses the same schema).

This is fine if you intend to eventually upgrade all your existing code to also use the new technology. In that case, you have to start somewhere, and it may as well be here. But you have to be really diligent about this. In my experience, when folks ask for a greenfield feature, they aren’t very interested in also rewriting lots of existing code afterwards.

A better way to approach this is to upgrade existing features to the stack you want, if doing that will make you 5x more productive. Or, just embracing the stack and learning to master it, which is its own highly challenging journey filled with intrinsic rewards.

Technical challenges that are actually challenging

I think developers are attracted to greenfield projects because they seem like really great technical challenges. But actually, they are the easiest code in the world to write! There’s no real constraints, no compromises you have to make, no head-scratching as you read through archives. No tests you need to get passing. Instead you can just write whatever you like and deal with the consequences later (if ever).

I’ve seen plenty of developers move through this phase and out the other side, and seek out problems that are technically challenging because they come with real constraints. These problems are everywhere. Common ones in SaaS include deployments and infrastructure at scale, product / feature performance, database optimisation, platform functionality (particularly at a framework level, eg. contributing to Rails), as well as the entire field of product management. Other folks decide that all those things are too far away from the user and they just want to build really great features and master their stack in the process.

All these are great outcomes that create real value for customers. And they’re real technical challenges, too.