I am new to product management and currently practice creating a PRD of a collaboration product for case study purposes. I will consider Figma as an example. In this product, the user can create a canvas, and then invite other people in teams to collaborate with.

Suppose the Dev Mode is a new feature, and the objective of the Dev Mode is to help the developer in the team minimize the time it takes to transfer UI to code.

When setting the success metrics for this feature, I thought it would be ideal to set the Time to transfer code from UI as a key metric to assess whether or not the Dev Mode helps developers save time. However, we cannot monitor the time for this process because there are activities outside of Figma. So, this metric cannot be measurable.

I decided to choose a more generic metric like Adoption Rate as a key metric. I don't know whether or not it is a good metric to set.

  • If yes,
    • I don't know if the Adoption Rate counts at the user level (each user counts as a separate adopted "one") or at the project level (just need one developer in the team used counted as an adopted "one").
    • Dev mode is just occasionally used, not on a regular basis. So does it affect how we count the Adoption rate?
  • If not, which one could be a good key metric in this case? What other support metrics should we use?

I hope someone can help me clarify this confusion.

Many thanks!

Dev Mode in Figma

2 Answers 2



Avoid proxy metrics like the plague. Figure out what the feature is supposed to accomplish for the product, why it matters, and how you can measure it directly rather than indirectly (if possible). Then slice up the feature into small, independently measurable work items so that you don't have to wait until the end of feature development to determine if the idea was a good one or just a boondoggle.

Proxy Metrics are Problematic

You are making a rather common but very fundamental logic error with your proxy metric. Specifically, when you say:

[T]he objective of the Dev Mode is to help the developer in the team minimize the time it takes to transfer UI to code.

you are making a rather basic assumption that all "units of UI" are basically the same size, and that all resulting code is of equal complexity. Neither is inherently true, so what really happens is that you end up measuring lead time to turn an arbitrary mock-up into a piece of working code in a way that isn't necessarily transferrable across multiple features.

As just one example, if the specification is to move all elements one pixel to the left that likely has a very different level-of-effort than a specification to add a new tab with multiple panes showing various elements. In other words, you're comparing apples to oranges with this metric, and it simply won't yield the results you want.

Measure What You Care About

In general, from a product management standpoint, what you really want to know is "Do people like this feature?" and presumably "Are they willing to pay for this feature?" Payment doesn't even have to be money; it can be payment in time spent learning or using a given feature, or switching from a product that doesn't have that feature to your product. Regardless, the point is to figure out:

  1. What you expect as an outcome.
  2. Why that outcome adds value to the product.
  3. How you can measure that outcome and/or value as directly as possible.

Many people use a variant of the Net Promoter Score (NPS) as a reasonable proxy for product affinity or loyalty, but there are certainly plenty of other ways to measure these sorts of things more directly. Direct metrics are always best. Consider the following rules of thumb for identifying appropriate metrics, but feel free to adapt what and how you measure to suite the desired business outcomes.

  1. Measure Specific Business or Usability Outcomes

    If developers are more likely to use your product if you add a Dev Mode feature, you could directly measure:

    • How many developers buy/use/download your product?
    • How many developers choose to use your product over some other product when given a choice? (NB: You may need to unpack the reasons, as correlation doesn't equal causality.)
  2. Measure Value

    "Value" is often subjective, but can usually be reduced to some quantitative or qualitative measurement. Examples include:

    • Surveys, A/B testing, or specialized application analytics report shorter average or mean lead times to completion of a product increment.
    • Value may also be measured as a perception rather than a concrete measurement. For example, consider the difference between measuring customer satisfaction or product return rates.
    • Value might be directly tied to cost measurements. For example, customers might be willing to pay more for a product with the feature, or more willing to purchase it as an add-on.
  3. Directness

    Measure the metric that is most closely aligned with your desired outcome rather than using proxies that deliver implied results. For example:

    • If the feature is intended to drive sales, then measuring closure rates with and without the feature would be an example of measuring the goal (e.g. more or faster sales) directly rather than indirectly.
    • If the feature is intended to drive productivity, measuring the lead time to completion of a fixed unit of work between a control group and a group properly trained on the feature would be a direct measurement.
    • Productivity in software is notoriously hard to measure, so subjective surveys about whether or not users and managers find it improves "productivity" (whatever that means to them) after a suitable learning curve is less direct, but possibly more valuable.
    • If tool or feature adoption is the goal, then measure tool churn (e.g. how many people switch to your tool, or away to some other tool) based on the presence of the feature.
    • If "adoption rate" (by which we'll assume you mean how many people actually use the tool when it's available) you could directly measure how often people open Dev Mode. How long they spend with the feature within a session is largely irrelevant since if it's boosting productivity it may result in less time inside the feature. Then again, maybe not! But the number of times the tool is accessed is a pretty direct indicator of how often a developer reaches for the tool because they find it valuable.

One Size Does Not Fit All

I wish there was an easier way to tell you what you should measure. Unfortunately, product management is often an exercise in validated learning from an empirical process that you test in the marketplace. Just ask any movie studio whose mega-budget film idea did really well in focus groups and then bombed at the box office. It's really only the empirical results that count in the end.

With that in mind, drive iteratively or incrementally towards validated learning to see if you have a good product/market fit. No matter how great the idea, it's worthless if nobody buys it or uses it. On the other hand, if you don't test the idea, you'll never know whether or not it has real (rather than just theoretical) value.

Small, testable units of value are generally easier and cheaper to learn from than big-bang, all-at-once features. If you can slice your feature using INVEST you're much more likely to get the validation without chasing sunk costs. That said, product management is as much an art as a science, so your mileage may certainly vary.

  • 1
    Thank you for your well-written comment! As this is just a proposal case study, I think I should change the objective of the feature to be more specific to easily set the metrics for it! Need more practice to feel the metric settings.
    – chrismake
    Commented Jan 3 at 6:24

you can consider to measure the stickiness of the feature e.g.

  • Once a team uses the dev mode once, is it used again?
  • Once a team uses the dev mode twice (so, it was not just "checking what new button does"), how many are coming back for a third time?

we are trying to see if the feature becomes part of the workflow of a team

Features that are targeting only a subset of your users (here: designers that hand over their work to devs, or teams that have devs with figma access etc) usually are harder to measure as absolute numbers, but if a subset of users are going back to that feature, you are delivering value for them, so you may try to check the return rate.

  • It's a sign of an adoption rate! I would add it to the support metrics. I want to upvote for you, but it needs more points to start the upvote! Thank you for your reply!
    – chrismake
    Commented Jan 3 at 6:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.