I work as a project manager, not a product manager. I have an understanding of what A/B testing is, but I'd like to know what do you do as a product manager to perform an A/B test.

Does it include writing some code? In which programming language? Do you write it yourself or assign this task to a developer?


There are two components to A/B Testing. The first is the "why". The second is the "how".

I'll start with the "how" first. This is the implementation. This is the technical details that developers should handle. Depending on what exactly you need to test they might need to configure something, make a small change in the code, or make a big change in the code. They decide how to implement the request for this test. For example, if your A/B Testing involves just changing the color of a button on a landing page, for developers this is just a conditional statement in the code, no big deal. If the visitor is an odd number visitor make the button red, otherwise make it green. If you have a more sophisticated A/B Testing scenario, maybe using two different landing pages, then the developers will have to write more code (two landing pages, different images for each, different styling, etc) so it will be a bigger deal. The developers are also responsible for deploying the functionality, and also the developers need to provide you with the data gathered after the test. That means they need to record how many users clicked on the green button, and how many on the red one. Or how they interacted with one landing page, versus another landing page and what actions they took afterwards. What code that involves and in what language it needs to be written is the developers' job, not that of the Product Manager.

The "why" is the part of the Product Manager. Why are you doing this? What's your mission statement? What do you want to check with this test? What theory are you testing? How do you want to test it (i.e. just red/green buttons on the same page? Different landing pages)? How do you select users to take part in this test (random or specific users)? What data do you want to obtain from this test? What data do you need to evaluate the results afterwards? How will you determine if the test is a success and if it's better to use a green button instead of a red one? How will you take business decisions from the information you get? etc.

So basically, this isn't any different than implementing other requirements,

  • the job of the Product Manager is to define the reasons why A/B Testing is needed, what you want to accomplish with it, and how you will interpret the results (i.e. what are the requirements for the A/B Testing).

  • and the job of the developers is to build the test implementation and provide the Product Manager with the data necessary for the Product Manager to extract knowledge out of the result of the test (i.e. build the functionality into the application as per the received requirements).

A web search for "A/B Testing for Product Managers" should give you more resources to read, like this one for example: The ultimate A/B testing guide for product managers.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.