How to gather client feedback?

Question

I have just started as a product engineer for a web-based product that we have around 150 clients, spread out across a large country.
- In short, it is a mapping application that lets you mashup your business data onto maps.

I want to kick things off by putting a survey together to get a better grasp on what they want out of the product.
- This will be a 1st step, afterwards I will plan workshops and I am also looking at implementing IdeaTorrent to catalog what features they want in the product, but thats later.

I have two questions (but I think it makes sense to include them both into this one Q)

Q1 - What Questions should I be asking our clients?
Q2 - What is the best platform to ask these Questions
- Our organisation usually uses SurveyMonkey, and I see they have a template for product feedback that I could use as a starting point.

I was reading an old blog post from Jeff, about how over time, cramming loads of new features into a product leads to a degradation in the product.

For every new all-singing, all-dancing feature, WinAmp becomes progressively slower, even larger, and more complicated

This has definitely happened for our product, and it has actually been fueled from all our different clients asking for this, and we bolt it on, and now the product could do with a bit of a clean up.
Therefore, although I am interested in new features they might want, I would like to try and focus their attention more to other areas such as:
- which existing features do they struggle with?
- how can we improve the user experience for them?
- get an understanding on their workflows and where the product slows them down.
etc.

Answer 1

From a high level view, it sounds like the organization is aware of some type of problem for which they chose this mapping application solution. However, from the lower levels, I infer from your question that you are unaware of lower level problems and, therefore, have no solution design. This is the perfect scenario in which using Issue-Based Methods may work for you.

This method is a controlled way of developing theories, hypotheses, and tests so that, at the end of it, you will have defined lower level problems and a direction for a solution, i.e., you will know what your customers are worried about with this solution and what they will want to see from it.

General Outline

1. Problem Statement

   1.1 Questions

   1.1.1 Hypotheses
   
         1.1.1.1  Test Questions
   

1 and 1.1 are essentially what you theorize are things your customer would like to see in this mapping application, or said another way: problems you think this mapping application will solve for them.

1.1.1 are testable statements or the answers you think the customer will give you. and 1.1.1.1 are the questions you need to ask to either support or reject your hypotheses.

Set-up

You will need to conduct several brainstorming sessions with your team and maybe with a few customer representatives to arrive at at your initial problem statement(s) and set of hypotheses. Once done, you will clearly see the types of questions you will need to ask in your data collection methods. This serves as a structured way to get to your questions.

Data Collection

There is no best way to collect this data and do not let anyone tell you there is. You must use several methods and treat each of those methods equally. Surveys, focus groups, one on one interviews, town hall meetings. Use all of them that your budget and environmental constraints allow.

Understand that the Issue-Based Method is iterative. You may uncover additional theories and, thus, hypotheses as you move along and you may discover that an initial theory or two was almost immediately rejected. You do need to iterate your set of theories but do not get wishy washy with it.

Answer 2

Sounds like the project could benefit from having the sponsor clarify or define the goal of the project. That is, what is the business case, value or problem you are trying to solve for the organization that is paying for the project?

Adding features in itself is not a goal. It is the means to a goal.

The goal will provide context to your decisions and help you (and the team) make decisions on what to include and why.

A useful tool to consider is a requirements traceability matrix.

Answer 3

Q1 - What Questions should I be asking our clients?

I would recommend special focus on better listening to the already existing feedback rather than focus on what to ask. There is nothing more frustrating than giving feedback with no follow-up/ brainless refusal and later being asked what you think should be improved.

I do not maintain that consumers do not know what they want. They know it or at least they feel it quite well. What they do not do well is to articulate what they want and more often than not - they ask for the wrong solutions for what they want.

Someone might require the improvement X which in your expert opinion is a bad idea. That doesn't mean you should hurry onto the next task on your schedule.

You should ask yourself: - "What is the bigger need/ what problem is behind his suggestion"; - "In what other ways does that need reveal itself? How do other users experience it?"; - "What made him think X would be the proper solution for that need?"; - "How is that competition tacking that bigger problem behind his suggestion?"

In short: be available for feedback and focus on deeper analysis of available feedback first.

If you insist too much on asking questions of your own, though, I'd recommend you to have users compare your product with a competitor's. This makes it easy for them to be specific and accurate. They tend speak in too general otherwise and their words tend not to match their actual behaviour.

Answer 4

As you mentioned that you are just starting out, I would mention that you don't listen to "only" just your client feedback and build "only" those features.

• Are there features that you can add to the product that your clients don't know that they need?
• Do you have any competitors in the area of your product? Can you build any parity features? How about differentiators?

Also, over time reviewing customer care issues are another great way of eliciting feedback. I'm very fond of zendesk (zendesk.com).

Answer 5

There is a whole area called "Usability Testing" that attempts to answer some of these questions. A good primer to get you started is this book by Steve Krug:

http://www.amazon.com/Rocket-Surgery-Made-Easy--Yourself/dp/0321657292/ref=sr_1_1?ie=UTF8&qid=1313335497&sr=8-1

Answer 6

With my experience of clients from different regions I can conclude that you should be precise with the feedback form. Don not try to put everything in a document which eventually irritate the customer and will not gurantee you the reliable feedback.

I have done an experimental feedback for our product and our service to the product from our clients in USA, UK, Denmark and Germany and I was shocked that people from Denmark and Germany are not willing to fill the form until we shrink the number of questions. Where as, customers from UK has filled everything we asked, with a little exception. Lesson I learned to keep the feedback form/doc as simple as it could be to get the geniune response.

If it is an online feedback than make sure user need not to scroll more than once. make it objective rather than subjective, it becomes easier for user to answer the question. Always provide an optional text place for user to suggest anything regarding that question.

In brief, feedback should be

1. Precise
2. Short
3. More objective than subjective

Answer 7

I wrote an article answering that first question, which questions should be asked"? http://digitalartsuk.com/news/six-questions-you-should-be-asking-your-clients-design/