I am in the process of hiring resources for testing team. I have none currently. What should be the number of testers if my developers are 10?

Is there any rule of thumb which specifies how many testers should be there for given number of developers?

Please let me know.

  • Can you clarify the type of testing you expect the tester(s) to do - Functional, Non-functional (security, resilience, operational acceptance, performance, etc), and also whether you plan to use any automated scripting and testing tools? Will the developers also carry out any testing? – Iain9688 Apr 21 '12 at 12:27
up vote 9 down vote accepted

I worked in teams that had anything from 1 tester per 1 developer through 1 tester per 10+ developers to no testers at all. Each of these was able to release software of expected quality (not each did but that's a different story).

The short answer is of course it depends. However looking through the answers that are already here I found much about cost, type of software and methods of testing while nothing about practices related to building software.

If we think of Quality Assurance the responsibility for quality starts far earlier than testers kick in. Or even better, if your team understand quality assurance you should start talking about a role of tester and not a tester's job. In such environment everyone in the team, especially developers, should fulfill the role of tester.

With such attitude one of key factors would be a set of engineering practices you use to build software:

  • Methods like TDD, ATDD, BDD help you to automate testing during development, basing on test scenarios that are possibly close to real usage of the app
  • Continuous deployment meaning that you release to production (or possibly close to production) environment as often as possible on one hand forces the team to automate as much important stuff as possible (not only testing) but also make increments small enough that risk and cost of releasing with a bug is typically very small.
  • Code reviews and/or pair programming (see a nice comparison of them) help to improve the code quality significantly before it even goes to functional testing.
  • Automated testing with tools like TestComplete, Selenium or similar helps you to cover gaps that can't be easily covered with test written in the code and serves similar purpose: fewer regressions, higher quality, less need to test manually.

I know teams that were able to go to 0 testers using such approach. Note: I say about 0 tester jobs, not that no one ever fulfills this role. Pretty much the opposite everyone does.

There is another factor which is crucial, which is expected quality. Not every piece of software is expected to be of the same quality. Another social website can fail with little to no consequences while we definitely don't want to see bugs in military software.

Talking about this it is worth to ask how fast we can fix the problem and the answer may be in continuous delivery I've already mentioned. If we are prepared to update the software instantly the potential consequences of bug sliding through testing won't be as big.

Having said all that, I personally feel safer when someone takes a look at software before releasing it so no matter how automated process we have I prefer to have at least on tester in a team.

There are WAY too many unknowns here, but I'll give some food for thought....

Things you should ask yourself:

How experienced are the developers?

  • How well defined are the requirements?
  • What's the cost of failure? (Website vs Space Shuttle)
  • How easy is it to automate running the program?
  • Is it more important to test usability or functionality?
  • Are you custom developing, or writing one size fits all software?
  • What are the current quality problems?

Even answering these won't get to a rule of thumb, but at least it's a start.

The best way to find out the optimal # is:

  • Hire 1, and come up with metrics on where defects are found, and come up with a rough cost of finding them at that level. (Example: A user finds a problem is a $10,000 mistake. Fixing it after finding it in system test is $5,000, catching a problem during a design discussion is almost free) The #s don't need to be exact as long as there is some basis behind them.

  • Over a few months, watch how the metrics change, and see how the tester is pushing the identification of issues earlier and saving money.

  • Base testing decisions on predictions on what will happen. "If we automate tests, how much do we think it will save?" And then measure it. "If we hire another person, how many user defects can we find in system test?" And then measure it. "If it takes 2 weeks to do system testing with one person, how much can we cut off our release cycle if we add another tester, and what's the benefit to the company?" And then measure it. "If we hire a usability tester, how many issues can we push to design discussions?" And then measure it.

The idea isn't to get too crazy into measurement, just to realize that testing should be thought of as any other business decision. For the low hanging fruit, quality is free. Hiring someone usually will save costs in other areas if done right. Measuring things helps justify cost, because there's always pressure to just hire one more developer.

  • I can't agree with "the best way of finding out." You assume that value of each bug can be easily estimated which just isn't true in vast majority of teams. If we base on model that early fix is way cheaper, why wait until functional testing? – Pawel Brodzinski Apr 22 '12 at 9:27
  • @PawelBrodzinksi - It helps to have some metric so that you're not guessing. Again it doesn't have to be precise - even an average by phase that is +/- 50% is better than nothing. In general it is better to find problems earlier if possible. If you can catch a defect in unit testing, a lot less time is wasted than if it's found later. – MathAttack Apr 22 '12 at 15:14

There can be a heuristic for this and I would find it terribly helpful in estimating my work. I do not know the answer but it can be determined by simply analyzing past projects. No heuristic is a magic number. But it does serve to ground your thinking and as a great basis of estimate.

Out of curiosity, I looked at my current program and found my ratio to be 10:1, a datum to begin this discussion.

  • Let's see where this takes us! 13:1 – CodeWorks Apr 21 '12 at 14:01
  • 4:1 right now, but we're a small shop. :) I won't hire another tester until I have a second whole scrum team. – jcmeloni Apr 21 '12 at 17:45
  • 7:0 here, QA tasks are distributed among the team along with at least 2 different code reviewers. Working nice, for now. – eMgz Apr 27 '12 at 12:44

I think a good rule of thumb is 1 tester for every 2-3 programmers, though that number can vary wildly depending on how critical the software is, how experienced the programmers are, and even what methodology you use.

If I had 10 developers, as a baseline I'd probably put them in two scrum teams and give each 2 testers. If I were doing software for medical devices or nuclear reactors, I'd add more testers. If I were writing a casual game, I'd probably use fewer.

There is no magic ratio, and as other have said there are a lot of factors that go into determining how many testers to hire (and when).

If your software development methodology happens to be Agile, in my experience one UAT (user acceptance tester) per Scrum Team (and ON the Scrum Team), where the team size is 7 +/- 2, works out pretty well.

The downside to a UAT on a Scrum Team is that you sometimes run into a testing bottleneck or other impediments such as testing at inaccurate stages of story development and completion, but these are all issues that a good Scrum Master can work through.

So, is one tester for 10 developers a good ratio? Maybe, maybe not, but it's a fine starting point to determine that for yourself based on metrics you can clearly measure.

Edited to add, prompted by Pawel's answer, that my scrum team example is one of a group that strives for good build practices using a number of automated and integrated testing tools. Since that's been my experience for so long, I momentarily forgot that's not a "no-brainer" for everyone, & should've been included.

  • one for ten? That sounds way too light IMO. – Bryan Oakley Apr 21 '12 at 17:02
  • Like I & others have said, there are a lot of factors that go into determining this for the individual. However, speaking entirely from my personal experience, I've never had more than 1 UA tester to every 8-10 developers, so again as I and others said, it's 1 for every 10 is a reasonable starting point, in an environment which doesn't have one at all (& has a lot to figure out). – jcmeloni Apr 21 '12 at 17:35
  • I guess I've been lucky. My best gigs had an almost 1:1 or 2:1 ratio. My current is approximately 2 testers for every 5-6 developers which is sufficient, though we would like more testers if only we could find good ones. I guess to balance that out, I worked at one company where there were 5 developers and no testers. Admittedly, those were 5 absolutely stellar developers who diligently cared for an impressive regression suite themselves. Looking back, that was probably the highest quality code base I've ever worked on. – Bryan Oakley Apr 21 '12 at 18:06

I assume that we are talking about functional (acceptance) testing here, therefore integration and unit testing are not applicable here as they are carried out by developers, rather than dedicated functional testers.

Based on what I have seen, I strongly believe that there isn't a rule of a thumb, but I would consider hiring an additional tester when:

  • You are continuously releasing software. If you have a single tester and they are off-work for some reason, then your continuous delivery will be impeded by that tester
  • Quality of released software starts to decline. This would require you to have quality metrics.
  • Tester gives you a solid argument that they need more resources. Once again, this requires metrics.
  • You know there is a large project in a pipeline that would require significantly more resources. Requires product and team knowledge. This decision can be normally made by software managers.

Are you using something like Kanban to visualize your flow? If you aren't, you should consider finding ways to make the processes and bottlenecks more apparent.

Once you have some visibility it's a matter of adding one then shuffling things around before deciding if you need another. In a pinch you can schedule some coders to help test as long as it's work that was done by someone else and the impact on your velocity isn't significant. Once again, this depends on whether or not you are measuring or visualizing the impact of these assignments. If you aren't then there really isn't any basis for comparison.

If the project is important and budget isn't an issue, consider getting two since there really is no way to tell if you are getting a competent tester or if they can even integrate with the team properly.

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.