How can my team reconcile flow and pair-programming?

Question

For those of you who have read Peopleware, you were probably exposed to the concept of flow. Flow is a mental state attained by creativity workers (engineers, writers, programmers, etc.) which is often described as a state of immersion in which time seems to pass unknowingly and creative work flows from the mind. Throughout Peopleware, there is the recurring concept that to engage in flow, privacy and quiet are the main two ingredients.

On the other hand, I have read a substantial body of research in regards to pair programming (which should be familiar to the XP folks reading this). Pair programming, advocates a two person team which functions as an single organic, programming entity to accomplish a single goal. For those of you not acquainted, this paper should help: http://www.cs.utexas.edu/users/downing/papers/PairProgrammingKindergarten2000.pdf

So now we have two slightly contradicting ideas in the realm of software PM which proclaim to increase programmer productivity, code quality, and happiness.

My question to you is, are these ideas reconcilable? Is there any research on this topic, or how have you lead your software teams in regard to this?

EDIT:

The closest scheme that I have seen to bridge this particular kind of paradox is that of a combination. Some of the work is done in pair programming, while some of the work is done in private. How to balance this team/individual work seems to me to be the biggest question in regards to this compromise.

Answer 1

TL;DR

I'm not aware of much formal research in this area, but empirical results show that both models can be effective under the right circumstances. Flow and pair programming are somewhat orthogonal in the sense that they can't be performed simultaneously. They also represent practices that optimize for different things.

Different Optimizations

Personally, I can't talk and type at the same time. So, while I find pair-programming to be an extremely effective practice that I try to bake into my project processes, I also try to ensure that there's a balance within the team between increasing individual flow and encouraging teamwork.

In my experience, flow optimizes for individual productivity and individual comprehension, as many development tasks require a great deal of working memory and difficult tasks increase the cognitive load. I find it best to optimize for flow when the cone of uncertainty is small and the short-term goal is functional throughput.

On the other hand, pair-programming increases cross-functional skills within the team, provides opportunities for creative problem solving that leverage multiple perspectives, and tends to reduce knowledge silos and preventable defects when done well. I find it best to optimize for pairing when the cone of uncertainty is large, or when long-term sustainability is the primary objective.

The Team Management Perspective

Here are some additional team-oriented points to consider when contrasting the concepts:

1. Flow scales badly in mediocre teams. (Hint: average teams are mediocre by definition, which is why they are average rather than exceptional.)
2. Pair programming won't make mediocre teams great, but it often reduces the defect rate and overall effectiveness of mediocre teams.
3. Pair programming increases communication, which is a core element of success for any project.
4. Effective communication underlies most of the key principles of the Agile Manifesto.
5. Sitting together and pair-programming are practices rather than framework mandates, so use them where appropriate and avoid them when they don't benefit your team.

There are doubtless other considerations as well. In the end, what matters is what works for your team. As an argument from analogy, consider that various sources like Wikipedia and DairyReporter claim that vanilla is the most popular flavor of ice cream, but mint chocolate chip works best with my palate. I don't select my ice cream based on what tastes best to other people; I order what tastes best to me.

Pragmatic project management takes the same approach: do what works best for your team. As always, your mileage may vary.