Implementing new software development process

Question

Question:

Apologies for the long post, but I'm not sure how to handle this.

Background:

We have a development team of 3 situated in another city. The lead developer has been around for 6 years and the other developers have joined for < 6 months and both grads.

The last project left everyone frayed due to poor communication, constantly changing requirements and what looks like poor development practices/process. One part which I found contentious was where developers saw their job was to 'code' and it was the job of the business to 'test'. This meant unskilled helpdesk staff had huge test scripts and were testing by hand functionality that could have been automated and testing took twice as long as the 'development' part. They also insisted that it was blackbox testing so we were likely testing things they had already tested. The lead developer would break down work into tasks tracked in their development tool however refused to let anyone know what the tasks were or reveal how he estimated.

Scenario:

I've been brought on to oversee development of the next version. Management doesn't want to spend any money and have never come across any software development process other than picking up the phone and shouting down requirements. I want to move to an Agile methodology but no one, including developers, has come across anything like it before. As a result, I've aked for some small changes first:

1. A move to 2 week iterations to avoid changing requirements mid-way
2. Writing down headline requirements rather than verbal agreements
3. Developers do more testing before QA
4. Loose estimation of work and tracking by project manager in consultation with stakeholders

To date:

The lead developer has accepted: (2) writing down requirements, and now wants everything written down as a complete specification. Even with this, developers refuse to test according to this exhaustive spec because 'it's the job of the QA team'. I feel like I am being held hostage as management has seen the benefit of this, however is not interested in making further changes because getting involved in the argument is 'too much effort'. (1) was rejected as they refuse to estimate, lest they are held to that estimation. (3) was refused because their job is to 'develop' and the QA team's job is to find bugs. (4) was rejected because estimation 'takes too long' and they don't want to be held to it. As the lead developer has termed it: it's a matter of trust and we should trust the number they provide is the date they will have the software done.

We now have a process where time is broken down into: x (writing specs) + x ('developing') + 2x (testing) - development is sitting around for half of this time waiting.

Recap:

Apologies for long post, but I'm not too sure what to try. I feel as though I am being held hostage, and what I felt were small changes have all been 'too hard' for everyone involved?

Answer 1

My approach would be focused on persuading your lead developer to accept that-

1. Although writing code & detecting bugs are specialist jobs but "quality" is everyone's responsibility
2. Better quality does not always means catching bugs and fixing them but also includes writing unit tests,exerting due diligence while checking in risky code changes,working closely with the test team in ascertaining impact ,developing test automation to reduce test cycles
3. And I am looking upto my lead developer to lead this . This is not purely technical work , it is leadership

I think, it just boils down to maintaining this discipline and a person leading this change. Before bringing in any other winds of change through a process , I will start with these 'cultural' changes.

You need someone who believes in this and can persuade/enforce thee changes , lead developer ? an architect ? or you ?

Answer 2

welcome to PMSE!

First of all, I tend to prefer a clear and thoughtful topic than any two line questions from people who doesn't even know what want to know. You instead, offered a lot of details on your situation, which might lead to great and accurate answers that (hopefully) will fit into your scenario.

Reading all problems you presented, I'd focus my efforts on how to handle the 'developer only codes' behaviour. Thinking of Agile, the concept of people doing only one role is ridiculous. In Agile, everyone uses several hats, especially developers*.

So, the key would be to understand why developers have such culture.

I'd wonder some causes:

• The company always accepted them like this, and changing one's culture is a tuff job;
• They're simply lazy and don't want to work a minute more than the minimum required;
• They're boycotting the company / project for a reason you aren't aware of;
• They aren't comfortable with the current structure in place (maybe they'd like to have a dev lead, use another technology, etc.);
• They'd like to have / were expecting some kind of acknowledgement;

This list could be huge... but I'd say that a one on one meeting candid conversation with your developers to understand why exists the culture of 'developers are only (poor) coders'.

The other problems, from my point of view, are collateral. The root problem is the quality of the developers you have at hand.

Important to highlight that also the QA concept is wrong: QA, as the name says, is to assure the quality of the deliverable. A QA team is not a test team. Ideally, no bugs would be detected by QA, but usually QA finds only a few bugs. Not most of them. That's developer's task, for sure.

Important to also highlight that, when the time arrives, you may prefer to evaluate the team as a whole.

Remember: Trying to move to Agile having this developer's mentality will be a waste of efforts. One of basis of Agile is everyone doing everything. So, forget about 'sprints' for now. I'd say that a mix of Agile + Waterfall could lead to better results (and less waste of resources, like the devs playing card games while the 'QA' tests their codes).

'* Disclaimer: MY Agile knowledge is highly limited, so Agile-guys please correct me if I'm wrong!

Answer 3

I think your first step needs to be a culture change with your developers. I'm actually dealing with a similar problem, where many of my developers (I was until very recently a team lead) considered - like yours - it their job to write the code, QA's job to find the errors, and theirs to fix them.

We are - slowly - trying to change that culture so that it becomes the developers responsibility not to write code, but to produce a product that meets the requirements, which obviously includes that it meet whatever quality standards you have.

One thing we've started doing is to instruct QA that if they ever find (for example) 5 bugs in 5 minutes on a project - or one very obvious bug in that time - they stop testing, and send it back to development to have them fix it. This is a pretty strong motivator, because it's embarrassing to be told your work is so bad it's not even worth testing.

Answer 4

Wow, it sounds like you have your work cut out here Jeff!! I agree with Tiago, I don't think a wholesale move to agile will work well in this situation, as it really needs the entire team to function as a cross functional unit as much as possible, and you are not there yet. There are a lot of agile techniques and artifacts that you can bring in over time that may help improve the short term situation though. It sounds like the primary issue is the development team mindset rather than the process itself (although there are a few issues there too!).

You have already done the right initial steps by trying to get development to take more responsibility to the testing/quality role, but that was unsuccessful. I wonder whether you could try again by coming at it from the unit testing side? Perhaps the development team will be more open to improving the unit test coverage (since that really should be part of a developers role). You could then use this as a way to get them a little more onside with the concept of quality assurance being something that happens throughout the project. Once they are more actively creating good unit tests, you could suggest they assist with some form of automated UI testing. My team have just started creating Selenium keywords for all new functionality as part of the development Definition of Done criteria. They were hesitant at first but are now quite pro the idea as it has been the fastest and most productive way to create them. For your team, I would stick to automated testing that involves some element of writing code, as that will be an easier sell.

Regarding the documented requirements, this is an area that agile may be able to help. If you use the user story format and push the fact that the stories themselves are just aide memoirs for a proper conversation, that may help get the development team and the business communicating more closely. The user story format is an easy format for the business to create, and since they are short (because a lot of the detail is in the actual conversation) the activation energy needed to persuade the business to create them should be low. You can extend this by listing the Conditions of Satisfaction for each story and get the developers to demo these in a regular demo. This will push the developers to at least do some level of testing before it is 'ready' for QA. This is clearly not perfect but it is a baby step towards nirvana.

For the 2 week iterations and estimation, could you push towards using story points as an estimation mechanism? These are not time based but do give a way to track progress. I'm not 100% confident that this will work too well without a good working relationship between the QA and development teams, but it could be worth trying. I presume the reason the developers are so anti-estimating is because they have been burnt historically by being held to estimates. You need to work hard to ensure this does not happen. Story points are a good way to try something different as a kick starter, but once you have some estimates you have to make sure they are used carefully. You don't want to lose the teams trust, as it sounds like it will be difficult to get back.

So, in summary, I suggest the following steps:

1. Push unit testing with the development team. Once this is established, try to extend this role to include more testing, but stick to automation testing initially so that at least the testing involves cutting code.
2. Try using user stories as a way to document requirements. However, make sure everyone knows that the real requirements are captured in a conversation still, and the stories are just a reminder of the conversation.
3. Use Conditions of Satisfaction and a regular demo of progress to push developers to engage and perform some level of feature testing.
4. Try using story points for estimating the effort required, but make sure you are careful about how you use the estimated progress.

Good luck, it sounds like you have a fun few months ahead of you. Changing peoples' mindset is not a quick thing to do, so don't rush this process. If you can only improve one aspect, don't worry at least you've improved something.

Answer 5

I'm not sure where your authority lies in this, but it's not a methodology problem you have, it's a responsibility/accountability problem. And until/unless management gets involved, nothing will change.

The problem is that you have a group of people who are determining what their job is, and what they're willing to do, even if they're told different.

The only way to correct or change this is to demonstrate 'why' management must get involved. Outline the problems form the last project. Show how the suggested changes will improve things. Show them how much this attitude is costing them (re-work, delays, bug-fixes, etc.)

Explain to mgmt that you have a team that is unwilling to estimate how long something is going to take because they dont' want to be help to that. Mgmt is paying for that time, so how can they forecast anything or track unless they're given something at the outset? anything less means the dev team has an open budget to take as long as they want.

Bottom line, until you can find a way to force mgmt to see the problems and act, nothing will change.

Answer 6

Jeff, Implementing new process is a complex task. Too complex - I believe - to describe it in one PMSE answer. Moreover, implementing is not the goal, the goal should be sustaining the new process and making the change stable.

On the short term, you can make some improvements and changes, but in the long run you need to change the culture in both management and dev team. I never saw a change, forced by a middle-level management only, which succeeded. That means you need to sell the concept to the management and developers, which involves politics.

Situation you described, I would paraphrase like this:

You have a bunch of co-workers who had a good time sitting in their comfort zone, pushing their job on the shoulders of others or not doing it at all. For example management didn't manage the changing requirements and developers didn't care of quality. Now you can make them realise the situation and earn the support OR decide if you want to work in a mess...

My department moved from "no-process-at-all" to Agile so feel free to contact me in case you need some "offshore support". Also, you will find many short term solutions on this PMSE site.

Answer 7

As a software developer myself, I can tell you with 100% certainty that no matter what development process you are using (agile/waterfall/RUP/hacking) it is the software team's job to test that the software they deliver works. This not only includes unit tests but also (at a minimum) software integration testing. It very well may not be practical for the software team to do system level testing because of hardware resource and/or providing dedicated hardware resources to the software team but they very well should be able to perform software acceptance tests which meet ALL of the software requirements.

Unfortunately, if you have no management support then you are really hosed. However, if you do have some management support (even a little) the single thing I would push for is requiring the software team to pass software acceptance test procedures prior to passing the software on to QA. This means, all the requirements being implemented for the particular release have to be tested in some way. If the software team doesn't want to write the procedures then that is fine, but after their software continually fails the acceptance tests then they will eventually gladly take over that duty. Then it simply becomes a matter of somebody verifying that their procedures do indeed test the intent of the requirements.

Answer 8

A central point, I think, is that you shouldn't try to persuade the team to do anything.

If you've got a low-trust culture, a newly-formed team, and the developers see you as an extension of management trying to force them to change their ways, you'll have a hard time improving the process. However, if you are able to 1) get the team to suggest their own improvements, and 2) follow through on this to the best of your ability, then you're building the confidence needed for the team to grow and learn.