Integrating UX into Scrum - how to do usability testing within a sprint?

Question

I am trying integrate a UX team of about 14 UX designers into a software engineering organization of about 100 developers. One of many challenges I'm trying to figure out is how to deal with things like usability testing. I'm working off lots of assumptions here (please correct me if these are bad assumptions):

1. The UX team's work should be in the backlog as user stories. Point of clarification - right now, there is a design backlog and then there is an engineering backlog. So we're not "full stack."

2. A user story should be able to be completed in one sprint.

So here's my question:

1. Does it make any sense to write a user story for a usability test? It doesn't seem user focused. For example, "As a user I want the application to be usability tested so that we can identify problems and fix them." What? That's nonsense. How should we integrate these kinds of tasks into our backlog?

2. It usually takes 4+ weeks to complete a usability test. We have to recruit the participants (very narrow demographic), plan the test, run the test, and then report on the test. Can't get all that done in a 2 week sprint. What now? How do I write a story for something smaller than the full usability test? A story just for writing the test plan, then another story for recruiting the participants? That really seems to stretch the definition of a user story.

Answer 1

Your question is titled about integrating into a Scrum team, but it sounds like you are not organised like a typical Scrum team would be

Typically, your UX designers would be team members, they develop software, the same as your programmers.

You should not create user stories for UX tasks. User stories should meet INVEST criteria. UX tasks would likely violate:

• Independent (I): the UX tasks depend on a corresponding 'engineering' story
• Negotiable (N): doing the UX work is not optional
• Valuable (V): the user testing adds no value to the end user
• Testable (T): your stories prescribe testing rather than being testable

Instead, you should embed a UX designer into your engineering teams and incorporate the UX tasks into your Definition of Done.

Your team (that means programmers and UX) takes responsibility for the completing all of the UX work. Self organising teams means that if there is too much work for the individual UX guy to do, then the other team members should help too. That means they will have to collaborate together to complete the tasks.

As for your second point, it sounds like you're applying a waterfall process to your UX testing. This is going to limit the rate at which you can release software. In Scrum, each sprint should deliver potentially shippable software. If you haven't done UX testing then you should not release. You should try to break down your UX work into small enough sections, working closely with developers to do UX on newly created features.

You haven't really given enough detail as to the type of work that the UX team do so it is difficult to give any more advice than to apply standard Scrum practices.

Answer 2

The best place to fulfill this requirement would be your teams Definition of Done. A process policy used to ensure quality.

Speak with you team about how to best test usabilility in the sprint cycle. I'm sure they will have lots of interesting ideas and a Definition of Done policy is best owned by the team. Some suggestions might be;

  Pair devs with UX
  Have UX reviews
  Get an end user to UAT 

You will find this approach means a full on usability test at the end of a set period will be less pain full.

It also sounds like your usability testing takes a long time. May I suggest you take a look at Steve Krugs new book, Rocket Science Made Easy http://www.amazon.com/gp/aw/d/0321657292/ref=redir_mdp_mobile/184-1692377-5775040

Answer 3

If you're releasing software every 2 to 4 weeks, you can do the usability test on an actually released product, outside of the sprint, but as quickly after the feature is finished. You'll get feedback and that goes back on the backlog. Until you get the feedback, you've been able to release a working product.

If you're able to all that in a sprint, by all means do it. If you're not, then do the testing outside of the sprint, but try not to let the scrum team do all the work to prepare the tests.

Some small testing, like having actual users use the features before they're even checked in is ideal. Having a couple of end-users on call who you can show some of your work can give you very quick feedback in a very short amount of time.

Just make sure that the time between releasing the product (even if it's a beta, or release candidate) and getting your feedback is as short as possible. Otherwise the feedback comes in too late and reworking it will be a lot more expensive.

Answer 4

On my own experience I would recommend to:

1. Include Usability into the Acceptance Criteria. If it fails then one have to redo the story. In the same sprint or the next sprint - whenever there is time and priority

2. You have to cut your effort. Always recruit participants on a 2 weeks timing (depending on sprint length). Do not care what to show them.

A day before, check what is ready and working and show it to them. Make a sort of standard test plan to save time.

Invite programmers or product managers to your testings as observers or film it. After the test discuss it briefly and put together a power point presentation to show it next day the dev team. Show the video if necessary.

Talk about your recommendations with the dev team and that's it! No 10 pager is needed anymore, it is transported via mouth, face to face.

It is more a testing of parts of the software. After some time it might be useful to test all together in a classical usability test.

Even here I can recommend the Ucd UX blog from Autodesk UX team. They have good tips for efficient recruiting, but I can't find it right now.