Is TDD/unit testing compatible with Lean Startup methodologies?

Question

We're a small team mostly building MVPs following the Lean Startup Methodologygies and lean development.

Up until now, we've been shipping code without unit tests, in order to get our products or features MVPs out rapidly and get validated learning by having real people use them.

However, one of the MVP is starting to grow in popularity and we're shipping more and more either new features or incrementally improving existing ones to better fit customer's needs.

On one hand, we don't want to waste time delivering a perfectly coded/tested feature that turns out to be useless/wrong assumption (we could have figured that out without "wasting" time testing the code) but on the other hand, we're also conscious that shipping something quickly/untested might create bugs that keep us from getting our learnings by degrading the experience.

I feel like our MVP is actually turning into a real product, which might be the origin of the issue here but I was wondering if you had any advice on how we can keep experimenting quickly without compromising the quality of the production environment. Is a TDD approach compatible with quick experimentation? (Especially since we're not used to using this approach)

Thanks

Answer 1

TDD is more about how you write production code. If you need to write a large pice of code TDD allows you to split the work into small pieces. And tests guide you all the way. There are many cases when this approach leads to faster development, but there are also some cases when it doesn't (if you have too much doubts about the design of your code you may end up refactoring both production and test code all the time).

So right now you're at the stage when you want the quality to go up, but not too much. You're afraid that you'll spend too much time on a feature that you'll throw out. My recommendation would be:

1. Try using TDD everywhere where possible for a month and then compare the results. Note that developers should believe that this may help. Otherwise they may sabotage the process. But at some point they should notice themselves where tests help the most and where they drag them down.
2. Recognize which tests are complicated and take a lot of time and which aren't. Try to write simple tests first.
3. Don't treat every feature equally. Some features you know for sure will be useful. Some features require lots of calculations (this is where automated tests shine the most). These will benefit from tests the most. Other features are vague and unclear, and it's too hard to write tests for them (e.g. for PDF generation), so it may be worth to delay testing there.

And note that while TDD helps building code faster, the primary goal of tests is to make future development safer. Every time a new code is introduced it can potentially break something that worked before. This annoys users and really slows down the project.

PS: personally more often I see slow projects because they have poor quality, not the opposite.

Answer 2

Advantages of TDD:

• It should reduce the number of bugs and hence save on bug-fixing time
• It gives developers more confidence to refactor
• It can help get a better design (thinking about testability helps with design)

Disadvantages of TDD:

• Extra effort in writing the tests
• When refactoring, the tests may also need to be refactored, complicating the process

There are things you can do to mitigate the disadvantages. For example, the team could target their test coverage towards the more critical parts of the product.

The balance of these advantages and disadvantages will vary depending on many factors, including:

• The domain
• The experience of the team members
• The type of code being written
• Complexity of the code base

My suggestion would be to first decide what you think will improve by introducing TDD and then think of ways of measuring it. Then, run an experiment where you use TDD for a period of time and track the impact it has.

I would caution that TDD is a skill, so it is likely that developers will struggle with it at first. Your experiment needs to be long enough to give TDD a fair chance.

Answer 3

The idea of TDD is to code only what you really need and is a form of Test First. It's really a way of nailing down acceptance criteria for a user story more than it is a way to unit test everything that goes out.

It doesn't really sound like this is what you're interested in to me, at least not at this point. Rather you may be looking at some more traditional form of testing. Either way, an embedded testing methodology with testers in your dev teams/scrums/etc may give you a way to start incorporating some testing without holding up release and allow you to easily transition to TDD later if it looks like the way you want to go.

You might look at variations/evolutions of TDD, like Behavior Driven Development (BDD), for additional ideas.