What is the most common problem you face regularly and how you can avoid it?
I wonder if we, as a community, will agree on the same thing?
I ask for some short answers (just in a few words) to share our experience.
Uncontrolled scope creep - the project starts with the best of intentions. It is well defined and understood, but somewhere along the way, someone asks for an extra bit of functionality, or a new interface, or some additional management information. Sure, it's not in the original specification, but it is only a small change... But unfortunately it is one of many, and none has been through proper governance - so the impact on cost, risk, time, and quality have not been considered. The decision to include some of these minor changes may even be left to a junior technician, rather than being a senior management responsibility. Designs may change. Integration begins to get more complex. Cost rise, deadlines slip, and the benefits don't get delivered. All because someone kept on saying "Yes, sure, we can just slip one more minor change into the project."
How to avoid this? - Proper controls. Whether you are using traditional methods or more agile processes, make sure that the right person is making the right decisions, and for the right reasons. Consider deferring new features until release 2 (or 3 or 4). Make sure that the focus remains on what you set out to deliver - or make the change deliberately, consciously, and with all due diligence.
Some good answers, but the most important one seems to be missing, the one that addresses al of the others - the main factor that causes projects to fail is weak project management.
Looking at all of the answers so far, all of them can and should have been addressed and controlled by the PM. Scope creep, change control, project kick-off, assumptions, requirements - ALL are the responsibility of the PM.
It's the PM's responsibility to be clear on all of these, and to make sure everyone's in agreement on them before the project starts, and to then monitor and adjust as the project proceeds.
If I should pick one it would be poor or/and infrequent communication in as many aspects as you can name.
Poor or/and infrequent communication
I'm sure you we (as a community) can name many more consequences of poor communication. Fill free to add more.
From experience turning around too many messy situations:
Weak vision - poor understanding of the end goal. (Symptom: focus on features over the business case)
Undocumented requirements - too much in peoples head. (Symptom: new requirements in testing)
Poor metrics - lack of rigorous quantifiable progress updates. (Symptom: late slips of large magnitude despite no red flags in status)
Narrow boundaries - not anticipating external interfaces. (Symptom: System works but can't pass acceptance testing or realize value due tonconnectivity issues)
In my experience the single biggest problem is communication. With communication you can get everybody on the same page, agree on simplifications, avoid unnecessary work, identify problems, address problems, resolve issues, mitigate risk, and all the other ingredients of a 'decent' project.
Therefore more recommendation would always be: Fix the communication problem and you have can improve the chances of success substantially. (This could mean that you may have to replace people in some leadership roles.)
Poor project kickoff, chartering: One of my agile mentors, Ainsley Nies, has been a very successful retrospective consultant for the last decade. In more than 80% of the project level retrospectives she's facilitated the cause was traced to a poor project start (poor requirements definition, unrealistic constraints, team issues, etc). All things that could have been identified if the time was spent up front.
The final validation? Ainsley and Diana Larson (Co-Author of a leading Agile Retrospectives) just published "Project Liftoff," a book on agile chartering.
"Eight hours of planning will save eight weeks of work."
Inaccurate or poorly communicated assumptions - At the start of the project, the project sponsors, management, an other key stakeholders have a vision about what the project will do and how it will likely progress. To do that, they have to make some logical guesses on how things stand now and what will happen soon. Often times, that "common understanding" or "obvious conditions" are not really true. Even if they are true, they may not be communicated.
The project team may launch down a particular path that does not match the pre-launch assumptions and a lot of time and energy can be spent doing thing that the stakeholders did not actually intend.
Attempts to create certainty where no certainty exists.
This usually results in over-detailed specifications around new, complex areas, with the same detail then being carried into the simpler parts of the project which were already well-understood anyway - so, scope creep.
However, it also results in managers trying to fit their employees to the model they already thought of, or architects trying to shoe-horn a project into some particular shape of stack /technology that they dreamt of ahead of time, or business experts specifying a project as if it will be successful when what they really want is a prototype to try out, or teams making assumptions about each others' interfaces without actually wiring them together to see.
It tends to hide risk under the carpet until late on in a project, when it's too late to do anything about it. This isn't unique to Waterfall projects, either - I've seen it happen in Agile projects, with over-specified story backlogs and arguments about what can or can't be delivered happening way ahead of any reasonable data to support them.
Human beings hate uncertainty. The only way to avoid pretending it doesn't exist is through education, by teaching them about this tendency, and exposure, by putting people into places where it's safe to fail and letting them see how experiment and fast feedback can help. Conversations are really great ways to run mental experiments and provide feedback, which is why they're so important.
There is no way to avoid it in companies that don't make it safe to fail.
I have read in different places, like in the introduction of "Applying UML and Patters (Craig Larman)", that the main reason for IT projects to fail (hmm, maybe your question is not IT-related) is having bad requirements (as in vague, insufficient, full of implicit assumptions, etc). That is, not knowing what you want to get built or why.
Without getting into a waterfall-ish approach, it is crucial to know what needs to be built, for whom, and why. Using techniques from User-Centered Design and Contextual Enquiries can help a lot both the "client" and the "production" sides understand what they really need and what expectations they have.
A consequence of "doing it wrong" is feature creep and other usual problems.
Enforcement/Commitment All projects adhere to certain rules of conduct. When these "rules" aren't being enforced, things go down the drain rapidly. This can range from CEOs that ask for last minute changes twice a week without going through the normal RFC process to coders that say "Lets do that unit test later" or "I'll document the stuff later". Once enforcement becomes lax, it quickly gets out of hand and creates a downward spiral. A big part of project management is accountability, once that gets eroded, it becomes very hard to get it back.
Sure there will be times when this is inevitable, but these should be the exception and not the norm, no one should think that its "ok" or "normal" to short cut the system or the team will be dysfunctional eventually.
LACK OF VISION AND FOCUS (often misinterpreted as scope creep)
More than anything else, using scrum/agile specifically always seems to uncover the lack of real business vision and focus and how often the team is left confused or misdirected. Often what is uncovered is how often we are asked to change direction (this could also be accounted towards "scope creep").
In my experience a common failure theme is "promise" failure
1.Makerting over promising to customers without bothering about technical feasibility or achieveability
2.(And often in turn) Project management falsely &/or meekly promising to deliver (such) unrealistic schedules or deliverables .
3.Delivery team in this situation are rendered messengers or manisfestation of the bad news .
How to avoid -
Do the hard research and learning early on and resist every new shiny thing
PMO and Delivery leadership in general needs to be fearless in saying No. Loud and persistent.
I have done 10 full life-cycle large IT (SAP CRM) implementation projects in the past fifteen years and the conclusion I get from both successful projects and projects ended miserably is always the same, it's the people.
And of course the higher you look on that ladder the more it matters.
The first thing is the people, point, if you have a great team of experienced professionals you have great chance of success. As Gary Klein greatly explained in a final chapter of his great book a good team read each other mind. Sure, you must have good communication in place, you must document what you are doing, you must adopt the right tools to support you, you need to know how to manage change, how to manage the customer, how to cope with adversities... But all these things are always there, what makes a difference? The team, the people. As Warren Buffet once said: "Surround yourself with people better than you and you will drift in that direction..."
The second thing is PMO, which is not just that guy the project manager, it's a culture, it's an habit, and it's not a methodology or any methodology, it's keeping things simple but under control, all things, every thing, every little detail. It requires time, and usually is underestimated, it's often boring, and often seems there are more important things to do than update a gantt, a to-do list, or write the documentation. And often people do these things just because they thing they have to, or they are asked, or because it's a deliverable. In some other cases it's overdone, that's true.
Which drag me to the conclusion.
It's a matter of finding the right compromise; the perfect project doesn't exist, the perfect people does not exist, the perfect customer does not exist, the right time doesn't exist. You need to find the right compromise given the resources, time and money you have at your disposal, you need to do not take things too seriously, but seriously enough, you need to have the sense, and the sensibility. You need to have all these things, all these great people.
And of course chance are you will fail anyway... :-)
It's a damn perilous path...
old but still valid: why IT projects fail