5

Background

We have 50 employees who are working on different projects in a team of 3-4 people. In my firm the testing is done by the developers. We have 2 or 3 products with a large code base that we constantly modify while we add new functionality.

Question

Should the same person do the testing who did the development, or should another team member do the testing?

  • 2
    After thinking a bit more about this question, you might get better responses at programmer SE. – Zsolt May 31 '13 at 15:22
  • As written, this isn't a project management question. The question should probably be refactored to reflect a project management perspective, or migrated elsewhere. – Todd A. Jacobs May 31 '13 at 15:55
  • This might be relevant to PM if the question is directed at user acceptance testing, then you get into questions about user/supplier coordination. – Doug B May 31 '13 at 17:13
  • Why can't you use and end user for testing? Them that use it are most vested in making sure it works. – James Jenkins Jun 1 '13 at 13:28
  • I edited the question heavily to make it less of an opinion-based NC question. I also removed the off-topic testing methodology question because: a) it's a separate question, and b) as written it wasn't on-topic for PMSE. Feel free to re-edit the question if you think the edits fail to capture the spirit of your original question, but keep in mind that off-topic or polling questions may be closed. – Todd A. Jacobs Jun 3 '13 at 18:02
8

I am not deep into code testing but, from a QC perspective, having the same person check his/her own work is like having a prisoner keeping the key to his cell.

The QC objective is to find defects. The more defects found, the better the QC capability. That objective is inconsistent with the objective of the developer/builder, who is being measured by how well (s)he is developing/building. There is no possible way that someone can objectively and critically evaluate his/her own work in the manner which is expected in QC.

  • 2
    From experience, the QC objective can kill collaboration, because their goal is not to deliver a good product, but find as much errors as possible, and these two things are rarely the same. For example, twitter is a buggy, but great product. A QC objective would have killed it a long time ago. Anyway, I liked your prisoner metaphor ;-) – Zsolt May 31 '13 at 15:33
  • That's an interesting perspective. – David Espina May 31 '13 at 15:41
  • 2
    Even if the person checking their own work, is 100% dedicated to doing in correctly, they are only going to check from their perspective. You need someone who was not the developer to test, otherwise you won't find all those things that users find on day one of go live. – James Jenkins Jun 1 '13 at 13:26
6

The Project Management Perspective

Should the same person to do the testing that did the development, or should another team member to do the testing?

There is no canonical answer for this from a project management perspective because it's a false dichotomy. Quality assurance (QA) or user acceptance testing (UAT) often have different objectives and perspectives than developer-oriented tests, although there's certainly overlap.

Generally, developer testing falls into the categories of test-driven development (TDD) or behavior-driven development (BDD). Perhaps more importantly, they constitute a sort of white-box testing where there's a clear understanding of the internal workings of the application. UAT, on the other hand, focuses on the user experience and generally treats the system itself as a black box so that the focus is on user-visible functionality rather than internal implementation.

Having dedicated QA/UAT resources on a project team provides the following benefits:

  1. Separation of duties. This may be important for security products or in certain regulatory environments.
  2. Role-specific expertise. While programmers need to write tests, and testers often need to write code, good testing is as much an art as a science. A great tester is an invaluable addition to a cross-functional team.
  3. Management of testing infrastructure. Someone will certainly need to manage the testing and continuous deployment infrastructure. A good tester is often the logical person to do that, as they are generally the most familiar with that aspect of the toolchain.

Think "Cross-Functional"

Unless you are in a regulatory environment that precludes it, everyone on the team should be involved in testing on some level. How they are involved will depend on your particular project and organization. However, as a generic example, many of my agile teams look like this:

  1. Programmers mostly write unit tests, but work closely with the QA/UAT people to support integration tests and to write code that is highly testable.
  2. QA folks manage the test infrastructure, monitor continuous integration and continuous deployment systems, and perform integration tests. They also work closely with the programmers to identify edge cases and boundary conditions that should be tested.
  3. Programmers and QA folks work closely with the technical writers, ensuring that functionality is properly documented and that technical documentation is tested from an end-user perspective in much the same way that code should be tested.

The idea here is that everyone has a role to play in testing, but the inclusion of team members with deep expertise in testing adds skills and experience to the team that are often invaluable to the success of a project. QA team members shouldn't be treated as second-class citizens or optional members for a team.

Great testers have skills and talents that can make your project more successful. Don't skimp on the things that lead to success.

3

If the same person does the development and test, the feedback loop of doing things right is short (verification). The test driven development method builds on this idea. On the other hand, it is good to have another view on the problem just to know if the right thing is built (validation), and that's the case when somebody else tests the developers' code.

Nowadays, developers are responsible for writing test cases for their code. This covers unit, function and integration testing. Additionally, UX or test responsibles are doing user tests and exploratory testing in order to know whether the right product is built.

3

Props to @Zsolt for a very good answer. However this is a plan for the future, not a response to the situation in which you find yourself.

I believe that the relationship between coder and tester should not be adversarial - they're both partners in trying to produce finished code of a known quality. I want accurate measurements of code quality, and I want code that will threaten the business pulled back. Code with flaws that don't threaten the business is just technical debt.

My time as a tester was very brief, but I think there are two advantages to independent testing. First, an independent tester is less likely to share assumptions about the code. IF the coder tests the code, it is very difficult to detect the assumptions. For example, I currently confront a situation where the coder assumed that all names were Firstname, Lastname, and that the combination {Firstname, Lastname} was relatively unique. The truth is that there are many many exceptions to that rule. (All the women in my family have the same firstname, and 2/3rd of them have the same lastname. All have more than four names each - First, Middle, Middle, Middle, Last). Independent testers are less likely to have the same assumption. (in this case I think the error would have slipped by the tester as well). Frequently I find that the coder included instructions or directions that are .... incomplete... When I write prose, I always ask a colleague to review it for the same reason - it is likely to reveal invalid assumptions I've made.

Second there is the two mind problem. Study after study reveals that diverse teams are better teams, not because they fulfill some administrative diversity goal, but because diverse minds working on a problem are more effective at solving the problem. Mea culpa for not citing any of those studies If the coder and developer are the same person, then the team is about a homogenous as it is possible to get. Psychologically people tend to take the stance of "loyal opposition" even when working collaboratively (especially when they have separate roles - coder and tester). They know that they can improve the quality of the project by intentionally distancing themself from the partner and introducing artificial diversity.

edited in response to a comment by @David Espina The relationship between coder and tester shouldn't be agnostic; if the relationship is agnostic, that indicates something has gone seriously wrong. If QC is ruining code, then the manager of the QC department needs to have a serious talk with the business manager/sponsor/whatever. QC's goal is to detect unacceptable technical debt. If QC strives for perfect code, then QC is working for your competitors. sorry for the digression, but I think it is important in context.

  • Are you suggesting that QC, with an objective of finding defects, has an objective of creating perfect code or product? Is this why the relationship can be perceived as antagonistic? – David Espina May 31 '13 at 16:21
  • Ahh - that's the opposite of what I meant to say. I need to edit. QC should detect unacceptable technical debt. IF QC strives for perfection, then it benefits only your competitors. IF QC strives for perfection, then the relationship will be as antagonistic as a bad marriage. Antagonism indicates that something has failed. – Mark C. Wallace May 31 '13 at 16:56
  • Yes, would agree with that completely! Well stated. – David Espina May 31 '13 at 16:58
3

People are prone to cognitive bias, and developers are not exempt. A person who tests their own work will tend to miss problems that other people have a high probability of finding, because their thinking has been constrained by their understanding of how things are supposed to work. Put another way, a person can't think of using something in a different way unless they can first imagine using it that way.

You can mitigate this by have people test each other's work.

2

As a developer myself (now working in Programme Management) I can say with absolute honesty that in all projects in which the assigned developer/development team (myself included) were tasked with completing all testing there were increased post-release bugs and errors identified, and a decrease in client satisfaction.

From a project management perspective saving time/resources etc. by combining the testing with development creates something of a false economy. Especially if the project is delivering to an external client, this is likely to result in either financial penalties (if there are delivery clauses within the contract), lack of potential future business/income and a poor perception of the delivering company and likely, although perhaps misplaced, the Project Manager themselves.

As a developer I would expect to complete component, integration, basic functional and deployment testing. In addition to producing functional test scripts for use by testing resources. If appropriate to the development I would also expect to complete volume/stress, availability, and performance testing.

I've worked in organisations where we support the client and the Project Manager to set the acceptance criteria for these and others where it's established independently of the development team and neither seems to have any particular impact upon the outcome.

I would always advocate that full functional testing and user acceptance testing (which will likely also include non-functional requirements e.g. branding, UI and usability, other emotional factors, documentation, etc. should always be completed by a designated Quality Assurance resource provided by the client (whether the client is internal or external). If an external client is unable to resource this role then it can be fulfilled by an independent (unrelated to the development work) resource from within the delivering organisation. Similarly regression testing if appropriate should be completed by an independent QA resource.

If penetration/security testing is required (financial/healthcare/other public sector or high risk organisations will often require this) then an independent company can be contracted to provide assurance (and mitigate any potential liability on the delivering organisation).

1

In my experience, if the developer is testing his own code, it needs to be automated and that too using TDD or BDD. Having him test manually wouldn't work since a developer is an eternal optimist (my code can't break) and hence wouldn't test it thoroughly. This is why you need a QC to validate what has been developed.

There have been instances where TDD also doesn't help (an empty unit test case would never fail), so the role of QC needs to be there and is an important one. I recommend both to co-exist although there needs to be a balance between the two (~80% unit testing via developers since it cuts the feedback loop and rest 20% testing via QC).

1

In (at least) agile team context i don't see any problem with this. IMHO when planning the user story the tests should be first things to sketch, and never alone, so in a sense BDD approach.

This approach with continuous integration and good review tools & practices will ensure that the tests are not just testing what the implementation does, rather what it should do.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.