How to set properly aligned KPIs for engineers and QA staff

Question

Intro

I like to setup KPIs for my engineering/QA teams. I use that to evaluate performance and help determine bonus/raise each quarter.

This is one KPI I set for my engineers:

• % of tasks that passed QA with no bugs

  • ie depending on the QA flow (per issue or per sprint), we see how many bugs were reported against all the issues that were completed by 1. the engineer that fixed the issue 2. the other engineer that peer reviewed it.
    • ie in a given sprint engineer X resolved 20 issues, which got reviewed by engineer Y. Assuming the QA does their work on a sprint basis, and that the QA found 5 bugs in those 20 issues, the tasks that passed QA with no bugs percentage is 75%

• % of tasks that functioned properly post deployment Same idea as above, but this one will be treated for code that actually gets released live and is used by users

This is one KPI I have for my QA staff:

• number of bugs found per issue
  • ie the more bugs a QA person finds in an issue the better.

Problem

The KPIs for the engineers and QA personnel seem contradictory, or said another way: they seem like a zero sum game. Which obviously isn't good for employee morale and would cause rivalry/resentment.

How can I change this so that both engineers and QA staff have KPIs that make them work together as opposed to against each other?

---

update: a concrete case

this is in response to novigt's answer below as well as in reference to this comment on a related post:

Are the testers involved before a build? Meaning are they involved in developing the requirements or use cases or user stories, reviewing design documentation, or participating in code reviews?..

I figured that proper and sound documentation of scenarios is a huge part of evaluating QA personnel (keep in mind here I'm talking about non-technical QA who simply do black box testing and cannot do automated tests etc).

I agree that a metric that calculates total number of bugs is bogus, and i agree with the incidents in production metric.

Let me explain my new metric with an example:

Suppose I have a single login screen (forgot password/remember me etc not included for brevity) that I would like to build and deploy. Using Gherkin Syntax the QA engineer writes the following scenarios:

Scenario 1: Happy path login
given user enters correct username/pwd
when user clicks on login
then user should be directed to dashboard

Scenario 2: login failure
given user enters false username/pwd
when user clicks on login
then error message shows: incorrect username/pwd

this documentation is given to the engineers (along with the designs) on sprint one , who develop it, peer review it, send it back to QA for testing and then is deployed in version 0.0.01 of the app.

Once on prod, the following bug happens

given user enters correct username/pwd
and the server is down when user clicks on login
then
expected
user should get an error message: service is currently unavailable
actual
a spinner spins for ever

---

Here are the KPIs/metrics we capture for this scenario:

• number of scenarios: 2
• number of incidents in production: 1

---

metrics as a snapshot don't give the whole picture, so let's go on:

so on sprint two, the QA staff updates the documentation with scenario 3:

scenario 3: handle login when server is down
given user enters correct username/pwd
and the server is down
when user clicks on login
then user should get an error message: service is currently unavailable

the engineers fix that bug, and deploy it on release 0.0.02, on the release this bug happens:

given user enters correct username/pwd
and the user is not connected to the internet when user clicks on login
then
expected
user should get an error message: user is offline
actual
a spinner spins for ever

---

Here are the KPIs/metrics we capture for this scenario:

• number of scenarios: 3

• number of incidents in production: 1

• rate of increase of scenarios: 33%

• rate of decrease of incidents in production: 0%

---

so we use the last two KPIs (increase in scenarios and decrease of incidents in production) together to evaluate the performance of a QA person. The final evaluation should use both KPIs (in addition to other metrics that can be discussed elsewhere) to evaluate an engineer. This way increasing useless case scenarios alone won't be positive alone, and also an increase in incidents in production without a corresponding increase in scenarios should be a red flag for the engineer. I know one can also achieve this by having comprehensive unit tests that have >95% code coverage etc.. but let's assume we're not there yet.

Is there a problem with this?

Answer 1

So right now, as an Engineer, I take a single task, mull over it for 4 weeks, make sure it contains no bugs at all and I am the best worker you ever had. Despite the fact that it took me 4 weeks to deliver such a simple task.

On the other hand, you have QA, who is wildly dependent on getting a crappy Engineer assigned. The crappier the engineer, the more rewards the QA will get. What happens if I have to test a product from a good Engineer? Will I be put on a performance improvement plan for not finding enough bugs?

Your metrics are arbitrary and less than useful. Your people are not worker drones, they use their intelligence to get their job done. So don't insult them with metrics that can easily be gamed.

At least start with what really matters: incidents in production. Bugs that escaped both Engineers and QA. Try to minimize those and you will see that Engineers and QA are actually a team working towards the same end goal.

However, it's debatable whether having metrics is worth it for intelligence workers. Nobody has found the right ones yet and the wrong ones can backfire easily. If you base anything on your KPIs (like praise or salary) be prepared that people will be clever enough to outsmart you on any of them. Then you are stuck with people that did their jobs, but look average on the metrics and people that gamed the system instead of doing a good job, but look great on your system.