Estimation and testing in project management

Question

I'm currently writing a paper regarding project management, and was wondering if anyone with experience could answer the following questions?

1. What are the steps involved in creating a testing process? (Are there a standard set of steps, as after searching the internet I could not find any)

2. How can one refine cost estimation when in the design stage of a project?

For the first question, would the following be a valid explanation, or is this not a test process?

• Design test cases

• Prepare test data

• Run program with test data

• Compare results to test data

Answer 1

1.) What are the steps involved in creating a testing process? (Are there a standard set of steps, as after searching the internet I could not find any)

For the first question, would the following be a valid explanation, or is this not a test process?

-Design test cases

-Prepare test data

-Run program with test data

-Compare results to test data

I have a feeling that you are actually asking on how to implement a proper Test Process in an organization. As for your initial question - to create a testing process, one needs to agree on a given workflow and requirements regarding the way the product should be tested in a given company. Be it a software workshop or for example a factory, which creates umbrellas. It's the middle of the night over here, so let me hyperbolize a little.

Stop here for a second and try to investigate what Quality Assurance would really mean for the company.

So here you are, managing the production process for the example Umbrellas Incorporated. You have a set of requirements describing how the final product should work and also, possibly need to agree on how good a product should be, before you will release it to final customers. Thus, you need some standards. Also, you would like to know what happens during production and what happens on site. How should your Quality Assurance department tackle any tasks? When should they start the testing procedure and how should they report? Also what if a final customer calls you saying that the umbrella is leaking water, not working correctly at all or needs some fixes right now for one angry customer standing in front of some luxury hotel in Paris, who demands immediate changes to the umbrella, cause his product is not working correctly given rain conditions on-site? Thus, you need a proper organizational workflow.

Also, think about what happens when the production line is not working correctly? What should the Quality Assurance department do? Are they allowed to influence the production process strongly? Are they allowed to stop the production process? Thus along with the workflow, you'll need guidelines for exceptional situations.

Now following the question - we now know what is needed to effectively implement a test process. The actual execution of the tests is just part of it. What we do now is to ensure that the test process can work effectively for our Umbrella Incorporated company? Let's create a document describing our agreements.

• Standards

  • Describe the areas of the product required to be verified.
  • Describe the artefacts used during a test process (documents, bugs in some tracking system, test data)
  • Agree on a possible list of priorities/severity values for issues found during tests; for example: small severity being issues with the UI (bad colour of the material used during the production of an example umbrella), medium ones being some nuisances in the way the product works (problems with the springs used in the umbrella), and highly severe issues being problems blocking users from accessing the product's functionalities (canopy cannot be opened at all)
  • Describe the list of reports to be provided after the execution of tests

• Workflow

  • Agree on a workflow, include proper description of responsibilities in a project or project portfolio. Incorporate the Test Process into the current Production Process. Determine the possible information exchange points between the production team and Quality Assurance. Let it be a list of issues, or some IT tool to handle the project information. Agree on a proper policies for introducing new functionality. Determine when the Test Cases are being written and whose responsibility is it to manage the Test Cases in accordance to user's requirements. Determine what different test scenarios are being used. Be it regression testing or tests for new functionalities. Agree on what fully tested and verified product means. Include the information on tests used during the product's lifetime - determine example processes, which you described (Example: After a new requirement is created, prepare a list of test cases, prepare the test data and add said test case to the lists of tests/test scenarios to be performed during the production cycle, after the internal releases and after final customer-ready releases)
  • Determine the roles for the team members - describe who is responsible for starting the test process, who is allowed to report problems and issues. Who is responsible for creation and acceptance of test scenarios.
  • Make sure that all the information is readily available and the team understands what is involved during the Test Process.
  • Describe possible situations, which may involve immediate reaction of the Quality Assurance. It must be clear, whether the test process can block the release or not.
  • Agree on the needed service level to be delivered. Describe how the issues are being selected to be fixed. When should the fixes be introduced to the product.

• Execution of tests - as you initially noted in your question, you need to also create a detailed description of steps needed in the process. The points you listed are just subtasks used in the whole process, but obviously, one needs them to eventually perform the tests.

Note: these is not a handbook or theoretical definition - just my opinion and my feeling on how this question can be answered. Also, I'm in no way associated with any company possibly involved in production of umbrellas, and myself am rarely using ones, just couldn't resist after reading the_reluctant_tester's proper description of what a Test Process is ;-)

---

2.) How can one refine cost estimation when in the design stage of a project?

Getting the cost estimation appropriately is highly connected to the schedule and tasks' estimation of the given project. When the project is in a design phase, the most effective way of overcoming any complications is to describe the desired output better. One can assess tasks more effectively if given more specific details about the product to be created. One way to achieve this is to extend the phase responsible for gathering of the requirements and then scheduling team's work to estimate lower level requirements. Such solution is not always possible to be introduced, given software development - it depends on the customer's strategy, how advanced the vision for the product is, etc. In my opinion there is no really easy way to improve the estimates.

One can also use some estimation techniques to improve the assessment - like using FPE (Function Points Estimation) or WBS based estimation (Work Breakdown Structure based estimation). If possible, the desired development team should also help during the estimation, but keep in mind that developers do tend to either underestimate or overestimate the tasks they will work on. IMHO, expertise and experience are the key factors during the estimation process.

Answer 2

Looking at your answer to question 1,I think you are confusing test process with test execution .

Test process is a bigger umbrella activity that usually serves the following two purposes -

1. Guides/explains the test team on how to - design,execute,report,advocate - tests,test results,risks

2. Provides project or business management with the outputs they need to make business decisions based on test information.

Note that software testing being such a complex activity which involves continuous exploring,learning and improvisation (according tp project dynamics) , it is hard ( and rather not advisable) for a testing process to be more than a bunch of easy to understand & concise documents , providing guidance rather than carved in stone instrcutions

Answer 3

I will try to answer to question #1
To me, you basically answered your question, but I would add (with a project manager perspective):

1/ Testing is checking the conformance to requirements (this is the definition of quality in ISO 9001 : conformance to requirements). As a results, before going to design test case, your need to plan the work starting from the list of requirements and organize testing to match this list (one way or another). This will be usefull for (eg) rating the quality of the product you test : % of requirements fullfilled.

2/ You need to make sure that product is testable. This has impacts on product design and development.
I will try to illustrate with a simple example: if you test a web application with Selenium, it will help if DOM elements are identified by specific #ids so that you can assert (or reuse) values.
Lesson : test design may have impacts on design/dev and must be tackled early in the project (at least your test strategy).
(one step further : you could consider that your test scripts must be available before dev, so that they are used as the "living specification". This goes back to TDD).

Answer 4

A precision concerning the cost estimation: a cost estimation is very difficult to get exactly right. You can consult the reports of similar already completed projects with the same tasks or so, to see their own estimations. That can give you a good idea where you are going. When your estimations are done, add an account not link with any task: an amount for the unexpected or the non planned events. I can be 5% (for example) of the total budjet of your project. So, if a problem surface you will have a way to adapt to it without your hole project going over budjet.