Is it sound project management practice to make software engineers fix bugs "off the clock"?

Question

I work in a largish, multi-team software development shop that builds a vertical B2B solution suite for a highly regulated domain. We are trying to become more Agile; we are iterating better than we have in the past, but there is still a lot of waterfalling going on.

Over the last year, we have become pretty hardcore in adopting the principle that engineers must fix their own defects (those found internally and those that escape to the end users). Not only are they supposed to correct them, they are supposed to correct them on their own time, without impacting plans.

I get the internal stuff, and in theory I buy all of the accountability arguments of the escaped items too. In practice though I have observed the interruptions to correct the defects is both velocity- and morale-killing and they make predictable project plans darn near impossible to build. Plus unhappy engineers are darn near impossible to keep.

It is not that we have bad engineers who are just generating bugs left and right; it is that we have highly complex systems with mixes of new and legacy technology and significant external, internal, and governmental timeline pressures resulting in an environment that does not result in pristine software. Bugs exist and that's a fact.

I am trying to make peace with this. I feel like I can be more open to our approach, but I also want to minimize the highly costly disruptions the process puts upon the teams and the human costs of the demands of off-hours work. I feel like we are trying to solve a quality problem backwards by fixing bugs instead of focusing on preventing their generation in the first place.

Are there any experienced project managers out there with some words of wisdom to share on this topic?

Answer 1

And not only are they suppose to correct them, they are suppose to correct them on their own time without impacting plans.

This is your problem. Why don't your plans include the time for fixing bugs that you know will be there?

We all know it's impossible to write perfect code. We all know that bugs inevitably creep in. Expecting engineers to fix their own bugs on their own time without planning for that work is not "hardcore": it's bad project management. It means the work required to fix the bugs is hidden work, invisible to upper management & other stakeholders.

There are a variety of ways to plan for that work, & there are a number of other questions here that discuss that. But you have to get it in the plan, and stop expecting your engineers to be perfect.

Which is not to say that you shouldn't also continue to work on bug prevention, of course. But your approach as a PM and as a team needs to accommodate the fact that bugs exist.

If I were going to put forward an argument against this practice, I think I'd do it with a risk management approach. Identify bugs as a known risk, and list a variety of mitigations, including all your bug reduction practices as well as planning for bugs that escape nevertheless. Maybe add poor morale and/or exhausted engineers as another known risk, and suggest as a mitigation ending or limiting the practice of having employees fix bugs on their own time. Point to some studies on the effects of overtime on employee performance.

If you need to have something else to put in that accountability slot -- which presently seems to be more punitive and individually-oriented than is generally encouraged in agile development, which emphasizes the team and the team's investment in a quality product -- hm, I don't know. Maybe focus on accountability for violating the bug-reduction practices, instead of on bug escapes which are a fact of life.

Answer 2

Just as a side note to the other good answers - developers tend to have minds that look at process and (un)consciously find ways to game it.

What you are training your developers to do here is to not raise tickets for defects they find when they are developing (as either they, or possibly worse for them, one of their colleagues) would then have to work late to fix it.

If it's a subtle bug, they may just hope nobody notices (particularly if it's not directly related to what they are doing at that point in time).

It's well known that the earlier you find a bug, the cheaper it is to fix - if this results in more bugs being found in live rather than during development cycles, then this can't be a good thing.

Answer 3

Over the last year we've become pretty hardcore in adopting the principal that an engineers must fix their own defects (those found internally and those that escape to the end users). Not only are they supposed to correct them, they are supposed to correct them on their own time without impacting plans.

Let me ask you a question. Whenever a plan changes, do you change the plan on your own time?

Nobody is a robot. Normal humans make mistakes. And the more valuable the work of someone is, the more time it will take to fix a bug in it. I work on core components that save the company tons of time. However, if there happens to be a bug, then lots of components are affected and it will take a lot of time to properly fix it. The second you let me fix a bug in that on my own time, you will get a letter from my lawyer. You want me to work, you better pay for it. You want me to do regular overtime? You better have a reason. "Normal business" is no such reason. Most likely it will take a few weeks to hand in my resignation for even trying to pull such a stupid stunt on me.

As a new developer, I would never start at your company. If I am inexperienced, I will make mistakes. That's why you pay me less than others. However, I will be punished for the same reason you already pay me less again when I have to fix it. Why would I work for you then?

People having to fix their own bugs is a good thing. How about asking how that bug slipped through your testing? You do have testing, right?

In my opinion, the error lies squarely with your planning. You are planning people to work 100% and then you need to fix a bug. As if a bug is something you could never have imagined to happen. Bugs are part of developing and improving software. If you want to fix them on time, only plan tasks for your people for 70%. Or 60%. Or 90%. Whatever works for your number of defects. But it's your job to get that right. Assuming there will be no bugs and then letting the developers pay for your failure to come up with a plan that matches reality is going to get you a very empty development department really soon.

If you want to have fewer bugs, you will need to educate your developers on techniques to prevent them. You will need to give them the time to actually use those techniques. And you can do metrics on the bug tickets. But be careful with that. I have rarely seen a developer rushing a feature to production because he or she was keen on delivering a buggy product. The most likely reason is a project manager breathing down the developers neck because he or she stupidly promised that feature to the customer without consulting the developer for a realistic timeline. So when you have metrics, make sure you include all factors. And don't be surprised when the decisive factor is outside the development department.

---

To sum it up: making your plans work including bugs is the project managers job. If the bugs conflict with your plan, it was a faulty plan. If the plan only works if things work perfectly, the world would not need project managers. I could do that with a two-line script. So train your project managers to work with reality and train your developers to make reality more predictable.

Answer 4

This is a troubling post. Your company is penalizing its workers for what is a normal and expected occurrence--performance variability. The whole reason to "punish" someone is for a behavior change, to replace a maladaptive behavior with an adaptive one. In this case your punishment will yield nothing because we do not have the capacity of reducing variability. In fact, neither do machines. Machines also produce variable results and, now and again, will produce a result out of tolerances. Would you make the machines fix such a result on "its own time"? That's not even possible.

EDIT: Much of performance variability are random, i.e., you cannot control them. If you measure the results of a process, it is assumed that observations between +3 and -3 sigma are random results. However, those results could fall out of tolerances and thus be deemed a defect. Punishment to try to control them is EXACTLY like punishing you for rolling a 3 or a 6 on a fair dice toss, with the expectation that 3 and 6 will occur less after a few punishing electric shocks. If you are looking for an argument to change this policy, this is it. Many defects occur because of random drivers over which NO ONE has control. To try to hold an engineer accountable for them is chasing stochastic noise, a futile endeavor.

You are not scheduling time for rework and quality fixes? I don't even know how to respond to that. I guess this: start doing it.

Your developers are most likely exempt employees. However, if you adopted this across the board with other job roles, some of which are non exempt, then you would have a HUGE labor violation on your hands as there is no such thing as unpaid work for a non exempt role. And the reason is because companies make really bad employment decisions such as having its workers fix their errors on their time.

Don't try to get good with your company's policy here. Change it. It's simply wrong under every conceivable standard.

Answer 5

Aside from the main issue you are asking about, there's also something a little concerning about this part: "defects (those found internally and those that escape to the end users)" I don't see anything about QA being asked to create the missing tests on their own time. This (assuming this is correct) along with your main concern demonstrates to me that the project management group in general is antagonistic towards the developers. I've seen this a lot and it never ends well. I think usually it's rooted in the PMs not understanding how hard it is to precisely predict how long things are going to take in development. That's what I hear when PMs start talking about 'accountability'.

That aside, the policy you refer to is extremely wrong-headed. The other answers give good reasons, bad morale, mental exhaustion, attrition but it's also very self-defeating at a very basic level.

Any seasoned developer has some idea of the following curves in their head:

The key here is that if I am rushing stuff out, I will spend more time fixing stuff than I did writing it. If I spend a little more time on the dev phase, I can get that under control. But there's a diminishing return on spending more and more time on initial dev. Most of the bugs are easy to find but the really tricky ones could take ages to spot, if you spot them at all. QA time has a similar curve to the fix time.

What you care about as a PM is the total cost (purple.) That curve comes down initially as the developer takes more care but starts to go up again. You want to encourage developers to be careful enough to get close to the bottom of that dip but you don't want them to be more careful than that.

The policy you describe is going to push them towards the overly careful end. The reason is if they take lots of extra time to avoid creating bugs (the blue line), that falls under their normal working hours. The fixing cost (red line) comes out of their free time. They can minimize the time they spend off-hours on fixing but as a result that increases the total cost of the product.

Answer 6

The short answer: No, it isn't!

The not-as-short answer:

Your company has come up with the idea that the existence of bugs is a professional failure on the part of the developer. This is not true.

All code contains bugs. Quality code contains fewer bugs.

Your developers are doing quality work for you when they find and fix bugs. This is them doing their job. Would you try insisting your quality control department had to do all their work in their own time if you were in manufacturing?

"Accountability" might mean fixing your own bugs. (Actually, the most important thing is that somebody is fixing the biggest bugs.) Whatever. At your company, it has been twisted to mean your programmers should work for free.

Have any of your devs quit recently? Just asking...

NB: You can obviously see there's a problem with this, so good on you and well done for asking about it. Whoever is propagating this insane philosophy in your company is to blame for tying your hands and your dev team's hands behind your collective backs.

Answer 7

"Hardcore" indeed. I don't have much to add to the other good answers, but I'll relate an experience of my own as a developer. I worked for a company in which the culture was similar to what you are describing, in that there was heavy pressure to work long hours and weekends without pay fixing stuff that we'd been required to produce in unreasonably short amounts of time. I can still feel the stress reaction in my body as I write about it.

I think it is fair to say that I was the strongest developer in our small team. I came up with all the original work, setting the patterns which the others then tended to copy. I solved a lot of difficult problems in the challenging application we were developing. I was also the only team member who sometimes said "no" to demands for unpaid overtime - I had parenting responsibilities and a creative life outside of my main development gig. I believe the independence of mind which allowed me to be a creative developer went hand-in-hand with my not being a "yes man". However when time came to appoint a team leader, and my immediate boss put forward my name, his manager said that I "pushed back too much on overtime" and decided to advertise instead. I knew at that point that enough was enough and I quit, soon finding a job that offered 50% higher pay and work-life balance to boot. Feeling well nurtured, remunerated and rested, I produce better work and I feel loyal to my company. I feel invested in the product and I happily work long hours when necessary because I enjoy being part of the team.

Culture matters. A positive, appreciative culture will have flow-on effects in terms of code quality, employee loyalty and retention of talent. Ultimately, when you punish developers for bugs, you are telling them that you don't care about them as people but only as cogs in your imaginary well-oiled machine.

Bugs are normal. They aren't sins. No-one - not Gates, not Wozniack, not Torvald, no-one - writes pure, bug-free code straight up. Software development is an iterative process and the eradication of defects is just one normal part of that process. Honestly, not paying developers for fixing bugs is like not paying gardeners for weeding.

Answer 8

This practice is good way to drive out your best and brightest, leaving you with a skeleton crew of your bottom performers. I have developed software for generation 4&5 fighter jets and managed software-intensive programs for the USN: PMP Certification, multiple graduate engineering degrees, Eagle scout, yada yada, yada. The original posts leads me to wonder if the issue truly is integrating better SQA practice at the line-level to detect, remove and, most importantly, prevent defects.

Every half-decent engineer will gladly fix a minor 'happy to glad', however, it would not surprise me if the system requirements are not properly documented and any gaps are labeled as defects. Defects in my industry kill people, so engineers take them seriously.

In my experience (and documented across the industry) my ability as the project manager to ruthlessly remove scope creep and reduce function points drives down the number of defects. Reducing scope so that a team of three (or less) good programmers produces higher quality (minimizes software bugs).

Do not attempt to make peace with this practice: you have a serious problem you must address. If left to fester, it (to use a flight analogy) will ultimately reach a flat-spin and end up Class A. Instead, listen to your peers and focus on SQA procedures for defect prevention and you must plan for defect correction (these items should be prioritized into the backlog with customer) in schedules and software process development. Do you have an SQA plan?

Answer 9

In addition to others, I also find this post disturbing.

I've seen Project Managers want to place all sort of metrics on developers, but never on themselves, Product Managers, QA, management, etc. It takes more than software developers to release a project and get it right.

Do you keep track of what was the root cause of the bug? Who has to work on their own time if it was a missing requirement, or something else that is beyond the scope of the developers control? What sort of processes are you putting in place to make sure that type of bug doesn't happen again, or will get caught earlier in the process?

Bugs are part of the software development process and need to be accounted for as part of time/resource planning. Period. Even more so in an environment with 'highly complex systems with mixes of new and legacy technology and significant external, internal, and governmental timeline pressures'. It definitely sounds like your company is antagonistic towards developers and is trying to make scapegoats out of them.

I've worked on both sides of the fence as both a PM and a developer, and I can tell you that as a developer I would only put up with this 'policy' as long as it took me to update my resume. As a PM I would also be looking for another position where developers are treated with more respect.

Answer 10

Defects are part of software development cycle. It certainly cannot be blamed on just a single person. The whole chain is responsible.

Any found defect should be categorized and prioritized. Then it should be picked up by the responsible team, quality is a team effort not individual. Afterwards the original developer with the team should do a root-cause analyse to figure out how to prevent something similar of happening in the future. Just blindly fixing them as soon as possible will not improve the process.

Setup a good "zero defect policy" process to prevent just fixing everything instantly:

I would add a fifth classification "Closed" for defects that should never get priority, because they are to trivial. Make sure you communicate clearly and fair.

Answer 11

OK, let's skip all the moral and legal arguments by assuming you are working on a cure for the zombie plague and all the developers work 24/7 for their share of the last remaining pure water to meet the approaching (un)deadline.

So you find some bugs in v1 of the cure and look around to assign someone to fix them. You have been bitten too, so you need to get this finished and not upset The Plan as much as anyone else.

Do you choose:

• Developer A: He missed a semicolon due to tiredness and is the cause of the bug.
• Developer B: Superfast programmer finished his task early and got a good nights sleep.
• Developer C: Newly trained developer, super keen, but low experience means high risk of mistakes
• Developer D: Been here ages and almost fully a zombie and is disliked by the team. No moral penalty, but it takes ages to get things done.

The point is of course that you want to get the quickest/least risky solution to get the job done. Not play some weird morality 'hunger games' with your employees.

Answer 12

Kudos to Robert Grant for pointing out the difference between a contractor and a fixed price contract, that jumped out at me too. Even as a contractor billing by the hour I've done work for free a couple times (an hour at $120/hr on 2 occasions) but for reasons of my own for good clients.

As an employee, unpaid overtime leads to job searches on company time and workplace theft.

OP: Have you considered this thought experiment? Flip the process and pay the developers for bug fixes, but have them write code on their own time. Silly, right?

Answer 13

Try suggesting an alternative approach to the management:

Integrate code quality metrics, and test coverage into the CI server and have developers comply with that.

1. This checks the problem before it makes to production not after
2. Makes the developers happy as they are not fixing someone else's problems, they're testing their own code
3. Provides an objective measure of what's expected from the developers as opposed to "NO BUGS!" which is unattainable
4. Compliance with standard and test coverage is a better indicator of individual performance than number of bugs, which could be due to the complexity of work assigned and not the person’s skill

If the management aren't interested in going this route, that tells me they’re more interested in getting free work than anything about quality. Having developers write more tests would require allocating more time. Letting things blow up in production and guilt tripping them into late night debugging sessions costs nothing.

It’s a false economy anyway. You’ll end up with lower team morale and a bunch of disgruntled developers ready to jump the ship. And cost of fixing that could be more than what was gained with the free bug fixes.

Answer 14

You can make a remarkable number of approaches work. I generally agree with the rest of the answers here: the idea of trying to demand that developers fix their bugs off-the-clock is a bad way to do things. However, for sake of argument, let's say this is a reality. Can it still work? Of course. There's even a model for this:

Contractors.

What your company has done is turn all of their employees into contractors. If the employee has to finish the task on their own time (for bugs or any other reason), then they are being paid as a contractor. They get paid for the quoted number of hours, and if it takes more hours to do the job, they have to eat that cost.

This approach moves risk off of the business onto the contractor (or, in your case, the employee). Contractors are compensated for this risk: they charge higher $/hr rates because they are assuming that risk. They also typically have contracts to bound that risk (if you go on the graphic artists stack exchange, there's dozens of examples of how to write support contracts that don't burn you out).

How much are they paying their employees? Are they paying contractor rates? If so, expecting contractor grade performance is reasonable. If they are paying more typical employee rates, then they should not be expecting their employees to assume risk as though they were a contractor.

Answer 15

The bugs are symptoms. Not the programmers are bad but the management and technical system. Someone with are very mechanical, inhuman thinking must have invented or decided over this.

The developers will not learn how to improve but how to hide their bugs and not getting caught. This is simple psychology and even children, cats and dogs behave so. In the long run they will create a messy code hell. Find the reason why they make bugs and solve the underlying problems. As the PM work with the devs and not against them. Give them time, education, chances to learn and share their knowledge. Care for documentation as a source of knowledge. Such a complex system requires a profound understanding. Think of refactoring to make the system less error prone.

Answer 16

I'm not a PM. I'm a developer.

But we've partially solved that problem in an interesting way where I work. We have a team dedicated to new development. They're agile, and don't get distracted by bugs.

We have a separate team dedicated to bug fixing. That team isn't agile. We just take things in stride and are constantly being interrupted.

So, this doesn't solve the accountability problem, but it does prevent the main dev team from becoming dissatisfied and frustrated. Although, the side effect is you do have to have a small team that takes on the brunt of the stress. Those positions are hard to fill.

Answer 17

Bugs are your deliverables. You should not be afraid of them or make anyone afraid. You should be embracing them, encouraging everybody to find and report them, as much as possible. You should never punish your team or programmers for bugs! That would be a terrible mistake. Watch this video: https://www.youtube.com/watch?v=7DYr8GYzJ6Q

Answer 18

The problem you describe seems to be going on for some time in your work team. Are your engineers paid 1 1/2 their salary in off time or more? It is time to act right now not to enter a pattern in your work team that will be more difficult to correct in the next few years. Management studies show that it is not the salary that is the No 1 factor influencing the motivation of employees/workers. It is the quality of their work environment. People prefer to have a job where they can accomplish themselves before wanting an higher salary. You describe your tasks in the development of your products as complex causing more bugs that you would wish. Programming in 2018 is not the same thing as in the early 1980s when I took a course of Basic programming language before doing my studies in Business Administration till I got my MBA. Bugs in computer programs have less reasons to exist now than before because we have reached a degree of technology never seen before. Solving bugs is not the task the most interesting for an engineer to do. Maybe you could select or hire a few people that do only debugging; people that likes that. That move can frustrate your team of engineers. It would not be a bad thing. You have to send a clear message that you are not satisfied at all with what is going on before it's too late. Most people in North America spend more than what they earn. You can have a serious negative reaction from your engineers if they absolutely need the extra money. Also, sometimes, the members of a team sabotage their own work to get rid of their manager. Do not wait till you get there. Today, you have simulators to see if your programs work; what I understand is that certain members of your team leave unsolved the bugs they find or create. Project management, working as a team with a budget and tight deadlines need constant control of the procedures. Business administration is to plan, to organize, to direct and to evaluate/control. Since the past two decades some heresies entered management. Some people do not tolerate that their work is evaluated and controlled. My point is that you must act quickly before an exceptional procedure like working off hours become the norm in your enterprise.

Answer 19

Are there any experienced project managers out there with some words of wisdom to share on this topic?

As expected, all the other answers predictable yelled "absurd!" but not a lot of alternatives were proposed. After all, it is frustrating for management to deal with "the operation was successful, but the patient died", or, in our case "the code is complete, but it doesn't work".

What we have done is shrink the teams to 2-3 developers and a dedicated QA resource. This way a "unit of development" is incomplete as long as QA hasn't approved it. Since QA and the developers work hand-in-hand, it's not one against the other, it's teamwork aiming for perfection.

To be fully released, the item also have to survive the automated testing run, which is updated continuously to include all the new features.

This ensures that your "perfect" feature isn't breaking anything else.

As a result we have more teams, thus more management overhead. We have more QA resources. We have a slower cadence. But, when the "unit of development" is complete the quality is better.