How do I improve the development process at our small startup?

Question

I work for a small, independent software development startup. We’ve been in business for nearly three years now and things are going pretty well. We’ve got a pretty steady stream of projects for an ever expanding client base, and as a result have carefully started hiring new staff and temporary help. We’re now at five permanent members of staff, of which two are full-time developers and one divides his time between development, design, and managing the business.

Because the workload has been getting higher, we’ve run into some problems: we’re making long hours, are missing deadlines, are unable to give accurate estimates regarding when a project will be completed, and the quality of our code is dropping. I’ve been asked to come up with a development process that better fits our company and projects.

We usually work on several projects at once – this is inevitable duet to the amount of projects we’ve taken on and the small number of developers we have. We currently more or less divide this up in two week intervals: work on project A for two weeks, work on project B for two weeks, and then inevitably some problem with project A or C will pop up and we’ll spend a day intended for project B on project A and C.

Our clients are from all over the country and from various backgrounds. Some are talkative, some are not. Some expect weekly updates, some have no problems with not hearing from us for two months. We prefer to work in small iterations: get a prototype out the door and into the client’s hands as fast as possible (usually about a month after development started) and then iterate on that every two to four weeks. We’ve found that this is essential for the kind of projects we do and the fact that our clients often have no real idea what they want until they’ve had something approaching what they want in their hands for a while.

Right now, our development process is pretty straight forward, but obviously lacking. Our designer comes up with an idea for an application and a rudimentary graphic design, and our developers (one, two, or all three) sort of divide the work on an ad hoc basis and start building. Every now and then, we check where we are, and adjust if necessary. There’s not much more to it than that.

I’ve been thinking of doing code reviews: every piece of code is checked at by at least one developer who didn’t write it. This seems like a good way to start improving the quality of our code, but it takes a lot of time that we just don’t have. The same goes for unit testing: we’ve spent weeks writing tests and often found ourselves constantly having to tweak the tests because we found out they weren’t testing the right things, or weren’t testing enough. That project took significantly longer than projects where we didn’t unit tests and just squashed bugs when we found them. Also we didn’t have anything to show the client for weeks. Possibly we were just doing it wrong, of course.

I’ve also been thinking of adding a dedicated tester to our team: someone who finds, categorizes and reports bugs to us, so the developers have to spend less time finding bugs and playing support for our clients, and have more time to spend on actually fixing bugs and developing the application. Obviously this is a significant investment for a small team like ours, but I have a suspicion that it’s a more effective way to spend money than just throwing more developers at a project.

As to providing better estimates, I have no real idea. We’ve dabbled in planning poker, but in my experience that amounted to little more than the wildly inaccurate guesswork we were already doing. Finally, I want to formalize the design process more, to a certain extent. Obviously due to the iterative development we can’t really set any design in stone, but it probably wouldn’t hurt if we spent some more time up front formalizing both the application architecture and the graphical design.

Basically, I have a reasonable idea of what we can improve and what we’re currently lacking, but I have no real concrete ideas about how we can achieve this goal. Hence, I turn to Stack Exchange. There are probably many of you in companies similar to ours who struggled with the same problems. How did you overcome them, or what traps did you fall into that you’d suggest we avoid?

Answer 1

Focus on quality and estimates

You identified four problems. Of those, I would view the following two as your main issues (long hours and missed deadlines are likely resulting from these):

1. Quality of your code is dropping: When the quality is bad, you end up doing a lot of unplanned work. This results in long hours and missed deadlines not to forget the hit to team morale.

2. Unable to give accurate estimates regarding when a project will be completed: This would also result in long hours and either poor quality of code or missed deadlines or both.

You seem to have tried or considered several options for improving the quality of your code:

1. Code reviews: In my previous assignment, we had code quality problems. By introducing code review, we were able to catch bugs prior to release and improve release quality substantially. At the very least, review high risk code.

2. Unit tests: We struggled a lot trying to introduce unit testing. If you have already introduced it, my suggestion is to keep continuing it at least in some projects. Some users have reported huge benefits from unit testing. Determine for yourself that it is not yielding the benefits before abandoning it.

3. Dedicated tester: I would strongly suggest a dedicated tester. Developers have blind spots. Also, developers do not have the patience to do repetitive tasks, which are often needed for meticulous testing.

For better estimations, I have three suggestions:

1. Continue planning poker: At a minimum, it breaks down the monolithic project into smaller chunks (epics/stories). By reviewing the estimates with the actuals, you can see what are the areas where your team tends to get it wrong. Also, it gets the team members talking about the scope of work, development approach and any risks/assumptions at an early stage. Capture these in the story/epic.

2. Schedule provisions: Make the following provisions while coming up with schedules for new projects:

   • 10% (one day out of every two week sprints) for maintenance work on existing projects.

   • 10% for vacations, training and helping with bidding on future projects.

3. Refine your estimates and schedule after prototype feedback: When you get the feedback from your client that is when the scope is clearer (to your client and to you). It is time to refine your estimate and indicate any change in schedule to your client.

Answer 2

• Do not take on any more extra work, there is a temptation but if your organisation is brought into line then the money will come. Just keep enough to maintain your cash flow. Greed is a very destructive force in business.
• Use something like a SWOT Analysis to ascertain your current position.
• Produce a Risk Matrix to find where the emergencies are so you can produce a plan of action.
• Use SMARTER principles to help produce goals or milestones.
• Get some good controls in place. Find from your risk analysis some indicators that will show deterioration in your change program. First stabilise, then look to improve - gradually.
• Take the initiation of work projects away from designers. Their jobs are to design what the customer needs ... your customers should be providing the pull. If you are producing products then having to find buyers for you are wasting resources in the worst way. Simple economics : supply and demand.
• Pull the whole team together, include all stakeholders (anyone who interacts with your company in anyway) start to find where you can standardise your procedures. Is there a way to standardise your reporting methods? The more you can bring common elements together the more time you will save.
• Start to look for areas of constraint or "bottlenecks" in your production process. This will help find where the quick wins can be made.
• Do not look to update your practices all at once, this can lead to a lot of instability. You should create a plan with input from everyone in your organisation.
• Look to staff skills : is there any crossover? Are there any who are multi-skilled? Your staff are your greatest resource, try to find a way to best utilise all of their abilities, they will feel better appreciated and your productivity will increase.
• Look at the mood of the staff ; long hours and missed deadlines can lead to decreases in motivation with a knock on effect on production. Keep an eye on management ; their job is to manage, it may sound patronising but all too often they can forget their roles ; rolling up one's sleeves to help the workers is sometimes not the best way to assist.
• For time management find a couple of good tools and indicators and stick with them. Too many times people go after the next big thing, stay with it for five minutes are then after the next tool. Ensure you understand how they work and if they are what you need. I would humbly always suggest the simplest things : Gantt charts, Product flow diagrams, Takt time to name a Few. Keep it simple ; everyone in your organisation must be able to understand them so you can all work to the same rhythm toward the same goals.
• Get in a consultant. You need to balance lost revenue against the cost of getting a professional to help you solve these issues. Change management should be handled by someone who does not get caught up in the business-as-usual affairs of the company.

There are many books on your situation and books on every point I have mentioned here. The most important point I will repeat though is to deal with your current situation. You must stabilise and create a firm foundation before anything else. I have seen too many companies in the situation where they have "built houses on sand".

Answer 3

I think you may be asking the wrong question here. You've mentioned that the problem is that you're missing deadlines and are unable to give accurate estimates of when work will be complete. The natural instinct of most technical people is to look for a delivery management solution - which is what you've done.

I would suggest, however, that what you're experiencing is a customer relationship issue. If customers are complaining about missed deadlines and inaccurate delivery estimates, you could approach those issues by managing the relationship with your customers better.

I would recommend that you assign someone, perhaps a different person per customer, as the contact point for each customer. They should establish a relationship with that customer which will allow them to negotiate on timelines and delivery expectations. This will release the pressure you're currently feeling to work long hours and deliver against tight timelines.

You may also find some process improvements which help you, but the reality is that the majority of research in this area relates to larger, more well-structured teams. As you've found, trying to apply "best practice" for the industry tends to be very problematic for small start-up groups.

I couldn't disagree more with the answer which suggested "don't take on any more work". Winning work is the lifeblood of small organisations and you should absolutely sign up every customer you can. But don't fall into the engineer's trap of thinking that the only solution to a customer perception issue is to improve delivery quality. Customers may be more easily satisfied with improved customer service and some personal attention.

Answer 4

I'd highly recommend James Shore's 'Art of Agile Development' for kickstarting the migration to an agile culture.

Baked within the practices suggested in the book are concepts such as 'less is more' along with a good definition of 'success' (intersection of technical, personal and organizational success) and how to achieve it.

Answer 5

It sounds like you are facing the classic "victim of your own success" situation. The process that has made you successful, is hindering your future growth.

I'd suggest that the first place to start is to understand all of the reasons why you are missing deadlines. It is unlikely to be just one. I'd suggest using a tool such as a Ishikawa diagram, http://en.m.wikipedia.org/wiki/Ishikawa_diagram. Make sure your team are involved in helping identify all of the reasons.

Secondly, prioritise the issues, which reasons will have the greatest impact, in the shortest time. This is the issue to attack first.

Once you decide to improve a single issue, be tenacious. For instance, if you determine that bugs are stopping you from missing deadlines, implement early code reviews. Keep at it, until you see improvement and the change becomes adopted into your culture.

Also, try to capture data on your process wherever possible, how many delays are you finding, what caused them. This is will help you demonstrate the value of your improvement efforts.

I'd also advise not to Ty to fix more then one issue at a time.