1

So my boss put forth a 'research and find out question' to me:

  • Are there ANY (absolutely any, no matter the dumbest, but any) metrics for requirements management? Eg. Requirements completeness, consistency and productivity (being the most difficult to nail down i.e. what is meant by productivity).

Similar question with respect to design and architecture level metrics. I.e., do any exist and what are they if they do.

It's kinda fuzzy right now but I wanted to know what are the metrics that you have commonly used at these stages (if at all) and what have you found to be worthwhile. The notion is that most companies can (and do) come up with proprietary metrics to measure stuff related to the requirements/design/architecture phases but are there any 'commonly applicable' ones?

Note: I'm not looking for code related metrics like cyclomatic complexity or bugs/100 LOC or function points

I'm aware of the product backlog in the agile community along with release burndown release burndown charts tend to give you a good idea of things but they don't work as well in a non-story-point estimation environment, IMO. They are good proxies to measure the aforementioned requirements consistency, completeness and productivity aspects but not sure about how well can they cater to that.

I'm well aware of it being a hard problem, I'm not looking for a solution but mere opinion/view points on what have you used or seen/heard being used in this regard. Research paper references more than welcome too :)

Improve this question
3
  • Not a duplicate, but similar to What are good metrics for an SRS?
    – Adam Wuerl
    Commented Jul 8, 2011 at 16:02
  • Yup! Had looked at that. Most texts say that you should "track" such metrics, my question is "what metrics" have been able to quantify those ideas concretely and what has been the experience of using them. Eg. It's good for requirements to not be ambiguous, but is there any (no matter how obscure, less used, dumb) metric that helps quantify ambiguity??
    – PhD
    Commented Jul 8, 2011 at 16:04
  • The best and only surefire way I know to ensure a requirement is unambiguous is to document the verification methodology. Not just the method (e.g. analysis, text, inspection, etc.) but a detailed description of exactly what you're going to do and objective pass/fail criteria. That way the meaning of the requirement is clear. Once the system has been put through the verification methodology and passed, that's what it means to satisfy the requirement.
    – Adam Wuerl
    Commented Jul 8, 2011 at 16:10

3 Answers 3

Reset to default
2

In terms of metrics for a set of requirements:

  • number of requirements is a good indication of system complexity (it may seem silly at first as not all requirements are the same "size", but in aggregate it seems to work no worse than counting lines of code does for software complexity)
  • number of TBXs both in absolute and percentage of requirements that are not yet complete. It may also help to track the different flavors, as a TBD(etermined) is less mature than a TBR(esolved/reviewed) or a TBS(supplied)
  • number of requirements with verification methods and methodologies, as requirements that no one has thought about how to verify probably aren't that good yet
  • number of requirements flowed-down to lower-level specifications or similarly the number of childless requirements in top-level specs or orphans in lower-level specs

If you're talking about measuring the efficiency of your requirements management processes:

  • hours charged per change request released
  • verification burn-down plan vs. actual progress
  • percentage of the specifications in the specification tree that are released and under configuration management
Improve this answer
3
  • Nice Set! I had suggested the number of original captured requirements to number made it in final deliverables and so on. But the question was "is there anything in literature or should we come up with our own" :) And that's the one making me scratch my head (and hopefully the community's too :)
    – PhD
    Commented Jul 8, 2011 at 16:33
  • 1
    If there is a body of literature on requirements metrics then I'm not aware if it, which is entirely possible. But most big companies I know of have captured and distilled their institutional knowledge into internal processes. That is, I would not be surprised if you failed to find anything in literature and decided your best bet was to think if your own and capture them for future use like several other companies have done.
    – Adam Wuerl
    Commented Jul 9, 2011 at 19:38
  • Absolutely! That's "exactly" what I told my boss :)
    – PhD
    Commented Jul 10, 2011 at 5:03
1

Although it's not exclusively related to requirements engineering or design and architecture phases, defect removal effectiveness can be used to find out how effective your requirements engineering and design activities, along with all any other activities throughout the lifecycle are.

The idea is that you create a table that shows where defects were found, where they were fixed, and where they came from. Using this information, you can find out how many requirements defects were found in requirements, found in design, or slipped all the way through to field deployment. You can track this per iteration (since every iteration will deal with requirements engineering through deployment) as well as per project. However, you need to complete a few iterations to have sufficient information as to the effectiveness of your practices at each phase and make corrections.

Other measurements and metrics would be related to rates of change of requirements, such as added, removed, and modified requirements. Also, relating requirements to implementation time or test coverage might also be suitable. I think that Adam Wuerl's answer discusses this pretty well, though.

Improve this answer
3
  • Yup! We do that very well currently! That is exactly (and probably only) types of metrics we track...I am more keen on finding out other more direct ones as I mentioned in the OP
    – PhD
    Commented Jul 10, 2011 at 17:50
  • 1
    @Nupul The only other thing that I've found that might be interesting would be correlating customer satisfaction to requirements. There is probably some kind of a link between "how good the requirements are" and "how happy the customers and users are with the software". Unfortunately, I couldn't find any strong research in this direction. Perhaps looking at research and studies on TQM might yield something in this direction, I'm actually a little surprised at the lack of easy to find and/or well-known work toward downstream activities and software quality.
    – Thomas Owens ♦
    Commented Jul 10, 2011 at 18:02
  • Totally agree on the difficulty of finding...
    – PhD
    Commented Jul 10, 2011 at 18:41
0

Revic 9.2 cost estimation model also estimates the requirements and review stages. Note, though, that it's based on lines of code, so it's a ballpark estimate. Plus it's a U.S. Air Force model, so it's aimed at military projects.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.