0

In our project, we have a testbook with some test scenarios to be performed by a QA team. Each scenario must be tested on multiple device/OS/browser (in this particular case, just 3 combinations), and for each scenario/target we have multiple statuses, for instance:

Scenario Status - iOS/Safari Status - Android/Chrome Status - Win/Chrome
Click on the "Bold" button Todo Todo Todo
Surfing in the USA Passed Passed Blocked
Hit the road Jack Failed Failed Todo
Test world peace feature Todo Passed Todo

Which are the possible way to report to mid-management and program manager the overall status, periodically? The data to show are:

  • count of test scenario in the testbook
  • count of executed test scenario
  • success rate for each target device/os/browser, based on the number of executed scenario to date

The only thing that comes to my mind (avoiding a simple table) is the stacked bar chart (one bar for each target device/os/browser). Is there anything clearer?

2
  • 1
    1) What does management want to know? Personally I'd ask management how they wanted this reported, because there is one and only one opinion that matters. 2) For my management chain, they do not want details; they want to know done or not done, and any obstacles that require management intervention. If you tell them that 27 of 38 tests have passed, their eyes will glaze over. If you tell them we're running 1 day late, but are confident testing will complete tomorrow COB, they will (probably) be happy. – MCW Mar 29 at 18:40
  • 1
    Schedule, qualitative information and items where the leadership attention is needed are already reported in a easier/fastest way to read. Executive/sponsor summary just contains these few information. Mid-management and program level want to have a little bit more details on this (the metric I listed in the question). As long as this is the only detailed and "boring" information, I would like to report it in a fashion that could grab the attention. – Antonio Mar 29 at 19:31
0

One challenge is that there is no "right" answer. But this is how I'd get to a "good" answer:

The real problem you're tackling is that middle managers have reason to want both the at-a-glance view that the people above them need and the specific view that team members need. You said you handle the exec summary view fine, so I'll leave that aside.

For the second, I would look at it broader. Many people need this view. I expect many QA team members need this at a glance. Those test cases may be separate, but the system they are testing isn't. Knowing that Hit-the-road jack failed on everything but the bold button only fails on Safari is important information. Similarly, your devs want this same context. That is to say nothing of planning time for testing and working bugs that come out of it. Now, once you've solved that problem, you can just give the middle manager access to that view and your problem is solved.

0

I think that this level of reporting would not really be meaningful outside of a team-operations level. Instead, I would try to attach each test or group of tests, both to a particular "aspect" of the product, and in this case to a particular "platform." And, I would try to attach to each test some indication of actual severity: green-light, yellow-light, red-light, thermonuclear explosion. Maybe also "flashing yellow" if something is blocking us from having results yet.

Now, tell me the status of each "aspect" of the product on each "platform." And, some assessment of the "severity" of each square in that grid.

Testbooks necessarily emphasize coverage. So, a testbook might contain many different almost-redundant tests – "wheel #1 is round" and so-forth. But this probably means that "the number of" tests is not particularly interesting, or management-useful.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.