How a scrum master does compare to a traditional project manager, and can either role coincide with the role of product manager or project sponsor / project owner?
Scrum makes the distinction between supporting the team in WHAT work they do and HOW they do it. Trying to have one person play both of these would create a great deal of conflict. How do you choose in a stressful situation between negotiating features or helping the team grow and reach consensus. A scrum master is a facilitator, how can he/she facilitate if she owns the project too?
"The Product Owner and Scrum Master roles complement each other; The Product Owner is primarily responsible for the “what” – creating the right product. The Scrum Master is primarily responsible for the “how” – using Scrum the right way. Only when the right product is created with the right process is enduring success achieved." –Roman Pichler, “Agile Product Management with Scrum”
Here is my visual view between the tradition vs. scrum roles...
In summary, to specifically answer "Why can't the scrum master and the project manager be the same person"? is because you can not (should not) ask one person to be responsible for HOW the does their work and WHAT work the team does because under business pressure, the person will cave or favor one side of the spectrum. Mostly likely the "What" part. With some there watching over the "high" you ensure that quality and good team practices and more are also getting the focus that is needed to keep a healthy performing team, while the other is focused on making sure that healthy performing team is getting good focused input and vision from one source.
It's simply the conflict of interests and focus. The project owner's main focus is on the success of the project, and make it happen, however the scrum master's main focus is on the team and its development. Two different things which cannot be done successfully by one person at the same time.
On the other hand there is the time issue. The project owner communicates with customers and stakeholders and reports to executive management and these activities are quite time consuming and I've never seen anybody who managed to take care of a team while doing it.
If you need any connection to the traditional project management then the project owner is the project manager flavoured with a bit of architectural skills.
According to the guidelines for scrum, the Scrum master is accountable for removing impediments to the ability of the group to deliver the sprint goal/deliverables.
The product owner has other responsibilities, such as conveying the vision, organizing and prioritizing the product backlog, interviewing stakeholders to determine what they need the product to do in order to solve their problem.
The one thing the product owner must not do is micro-manage the team. Not only would handling product owner responsibilities and removing roadblocks be an impossible combination, the person trying to wear too many hats would likely end up doing just the opposite of what the scrum master is supposed to do. Instead of eliminating impediments, this person would likely be the impediment.
The Scrum Master is responsible for creating an environment in which the team communicates honestly and transparently and delivers to the best of their abilities. This also provides data on which the Project Manager can act.
The Project Manager is responsible for the success of the project.
Some projects should not be successful. There is no recipe for enduring success. Some projects will never pay back for themselves, and despite the many claims, even Scrum won't make those projects successful.
These are often projects which are competing against existing products in the market, or against existing legacy products internally, and which need to provide minimum features from those products to be successful, so iterative delivery won't help in this respect.
Done well, Scrum will show up the death-march or escalating budget early. This is often in direct competition with the Project Manager's goals, and that's why the Scrum Master role is useful.
For a PM to be able to take on the role of an SM they would have to work in an organisation which allowed for transparent communication above the PM management layer, and for projects that ought to die to be killed off. This isn't the case in most organisations, so most PMs can't act as SMs.
The ScrumMaster and the project manager shouldn't be the same thing because the team needs to feel that they are accountable to the team (not the scrum master). Daily Standups are reports to the Team (not the ScrumMaster). The ScrumMaster is there to remove obstacles and impediments, not to manage the team - the team is self-managing.
The team should agree on a Product Owner and a Scrum Master and nothing should constrain the team in picking the best people - or person - for those roles. If the team trusts a single person to do both and the stakeholders are satisfied with the value delivered by the team, then we're done here.
Empowering the Development Team is the essential core of Scrum. Both the Product Owner and the Scrum Master need to understand that their most important job is to stay out of the team's way and allow, or even force, the team to learn to solve its own problems.
There's nothing incompatible about those responsibilities, and neither one is necessarily a full-time job (especially as the Scrum Team becomes experienced), so there's no reason they can't be performed by the same person. Since neither one of them gets to tell the team what to do, conflict of interest shouldn't enter into it.
Traditional/hierarchical job titles like "product manager" (or "dev lead") are irrelevant to Scrum and should be ignored by the team. Those individuals might be PO or SM or they might not. There's a specific risk in picking "managers" or "leads" for Scrum roles, which is that the team may be in the habit of deferring to those people, which undermines self-organization. In that case, significant effort is needed to ensure that the team manages itself and takes responsibility for its own work!
if you see the problem in terms of conflict, focus, different goals, WHAT, HOW and segregation, yes SM and PO can't be the same person unless it has dual personality.
but, if you see things from collaboratively teamwork, cross-competence and passion to deliver value to the customer (Agile) I see no point why SM and PO should not be the same person.
if you see at micro level in term of tasks and responsibilities, yes, it's a huge difference. but if you look from macro level perspective changes considerably. it's a matter of opinion where you set your perspective. I've learned this lesson from a great dude, that actually creates something for Agile community that it calls "dude's law".
I also must admit that the second perspective it's very hard to be achieved in high hierarchical, segregated, non-Agile companies.
A Project Manager is the know it all man/woman on a power trip telling you what to do and you hate them because of that.
The Scrum Master is a sage like Mr Miyagi who teaches the hero of the story how to win. We all love Mr Miyagi.
People can't love and hate you at the same time, so you can't be both.
Now for a serious answer.
If you are willing to accept the definition of "Project Manager" as someone who applies knowledge, skill and techniques to meet project requirements, a Scrum Master IS a PM. "Facilitation" and "authority over process" is management so that requirements are met. If you want to narrow the definition you are at odds with "industry" (I got the definition from the PMI).
Scrum is an Agile process framework. People forget that an Agile project is limited by time and cost. Rather than up front Scope, agile projects do what time and cost allow. You just prioritize scope (backlog) according to business value and pray that enough satisfaction is achieved for it to be considered "successful". Agile has come about about because some projects (software projects) have a lot of uncertainty and it's been identified that the best way to manage uncertainty is to remove it quickly. You may assume that a "Project Manager" never works in this kind of environment and you would be wrong.
Sometimes, a person with PM as their job title may just be a Scrum Master for a non Agile project. They don't make any decisions. They facilitate a process. They are their to "remove impediments". They don't define or do any of the actual work. The only authority they have is over project management process. Sometimes they may do more, just as a Scrum Master may have to step outside his role and do more.
One key difference between a PM and Scrum Master is that a PM will tune the process to the project. The Scrum framework seems to be a "colour by numbers" project management framework for small software teams and colouring outside the lines is considered "bad".
In most of the answers I read, many argue that the project manager does not have the time to micro manage the team. So we need a Scrum master to manage the team and the project manager takes the more major responsabilities like communicating with clients, etc. I never acted that way. I always managed completely the team I was responsable of. I would never implement a scrum, I would feel limited in what I want to do and how to do it.
So I prefer to talk about the:
That is the traditionnal way to manage a project, but I think it is still the best.
The problem in any project is to communicate effectively. I prefer direct human relations to emails but not all the time. I am convinced that talking directly to your team members, to solve problems and to innovate, builts a better team spirit than anything else.
I prefer a lighter, self desinged model, that anyone could agree on after a meeting.
Emails or planification sofwares should be used to do regular repetitive tasks to help the project manager do something else instead: manage his team and building a team spirit. It is a very personal view based on my own experience.
I have learned to trust my own abilities, as all of us, and those of my team members, especially on innovating and getting the job done in time but mostly on agreeing together on how to do it. I have so much fun working that way.
I witnessed often people spending most or their time trying to implement a model that, in fact, was not that useful. I find models to be sometimes very complex to do simple management tasks.
I learned in my studies the basic dimentions of management for any kind of situation:
1) Planning: What are you going to do and with what kind of ressources.
2) Organizing: Putting people together with material ressources to do the job.
3) Directing: Leading the team trough the process from the beginning till the end.
4) Evaluating: Constant feedback to evaluate if the goals set in step one are being reached.
Those are the four dimentions of management for any kind of situations. I always apply them, but finding a new and innovating way every time for any new projects. If the project is the same, we repeat one past succesful experience. I do not like being in a too rigid frame honestly.
My point of view is that any kind of project should have only a project manager, no other manager like a scrum master for example. In the Scrum model, I feel they divided in 2 or more the tasks of the project manager unnecessarily. It is a receipy for autority conflicts to my point of view based on my experience.