Documentation is an integral part of the formalization of the test process. The IEEE Standard for Software Test Documentation [829] provides a really good description of test documents and of their relationship with one another and with the testing process. As integral as this process is for testing purposes, it doesn’t get a lot of attention from modern generation testers. Hence, we enlist some top tips that might come in hand while testing.
Test documents include, inter alia, test plan, test design specification, test procedure specification, test case specification, test log and test incident or problem report. The programs under test, with specified version and identified hardware/software requirements before testing can begin, is documented as the test item. Test documentation should be produced and continually updated, at the same standards as other documentation in development.
1. Don’t use test documentation templates: A template won’t help unless you don’t need it
A template is usually a structure used for creating test documentation by filling in the sections/blanks. It is important to note that a test documentation template is no substitute for skill. The problem with templates is that they make it easier to write a content-free document that looks good superficially. But to use a template to write good test documentation, you have to own it. You must understand what each section of the template means, why it’s there at all and when it should be deleted. If you know all that, you don’t need a template. If you don’t know that, it’s better not to use templates at all. They’ll steer you in counterproductive directions because you don’t understand the requirements and tradeoffs that were balanced in the mind of the templates’ author
2. Use standard, predictable format for test documentation: They foster consistent communication
Whether you wish to present your work to a third party or to a customer as part of a custom-engineered product or if you are writing test documentation that another company will use to test a product or if you are writing test documentation that will be inspected by auditors or regulators or that might be used as courtroom evidence of good or bad testing after an accident, in all these cases- your material will communicate more effectively and people will find your material easier to use if you use a standard, predictable format, present things in standard places, cover a standard set of issues and use standard terminology.
3. Use the IEEE Standard 829 for test documentation
The IEEE Standard 829 project was led by David Gelperin, who has done a tremendous amount to foster diversity, creativity and the development and application of skill in the software testing field. If you want to create procedurally scripted test cases, Standard 829 tells you what to call them and where to put them in the document. The standard provides a framework, a structure, and a set of definitions, not a mandated set of sections. IEEE Standard 829 is widely used as the basis for many test documentation templates that circulate from company to company and has become the basis of a conversation about test planning and test documentation (for better or worse) in and across many companies.
4. Be aware of drawbacks while using Standard 829
While IEEE Standard 829 is one of the best use case scenarios, there are some issues that every tester must be aware of, some of them given below-
- The assumption underlying the Standard appears to be a waterfall-like approach in which tests are developed early, documented carefully, and then not changed. The cost of change (the maintenance cost of the documents) has a chilling effect on change. Ideally, you should account to minimize such costs beforehand.
- Massive test documents create a compliance mentality. Do what the plan says. This is fundamentally different from the mentality of the alert, critical tester who pays attention to any hints available and follows up all promising leads.
- The standard provides no structure for analysis of the test documentation requirements. It provides no suggestions or guidelines regarding when to provide what type of information.
- There is no apparent awareness and no discussion of the (enormous) cost of providing all of these types of information. Time spent on that documentation may well be time not spent on testing.
- The standard appears to emphasize on the breadth of documentation. More seems to be better. It appears to be a good thing to generate a Test Plan, Test Design Specification, Test Procedure Specification, Test Case Specification, and so on.
- There are no criteria for deciding whether specific instances of test documentation are good or bad. In practice, volume seems to be the accepted substitute for clarity and coverage.
5. Analyze your requirements before deciding what products to build
Your decision about what to include in a test documentation set and what to skip should be governed by your project’s needs. IEEE Standard 829’s format and categories might sometimes be useful while it may not be that helpful in other cases. Until you have completed your documentation requirements analysis, the choice of IEEE Standard 829 (or any other detailed specification) as your structure and user interface is, to say the least, premature.
6. To analyze your test documentation requirements, ask the following questions
Some questions that are specific to test documentation are-
- What is your group’s mission, and what are your objectives in testing this product?
- Is your test documentation a product or a tool?
- How quickly is the design changing?
- Is it that your testing style depends on defined tests or does it depend on exploration?
- Have you identified the primary readers of these test documents and how important do you think are they in the process?
- What is the extent to which you think test documentation should support testing progress and tracking and reporting of project status?
- Do you think the testing documentation should support the delegation of work to new testers?
Conclusion
Test documentation provides essential training material for new group members and creates archives suitable for regulatory or litigation use, apart from finding bugs, delegating work and tracking status. Review your test documents in order to ensure that the demands of every interested stakeholder on the project are met.
