Test Artifacts

In order to demonstrate the desirability of your product, you will need to do performance testing and validation testing. It is vital to document the methods used to do the testing, so others can repeat the testing in the future. Testing is one reason why the U.S. Space Program has been so successful.

Content

Test Results: Performance and validation test results will likely be presented during stage reviews as evidence of the quality of the design. Results are important evidence of the desirability of the design. They must be clearly and thoroughly presented, generally as support for the measured or predicted values in the requirement artifacts.

Custom Models: Where you have developed custom models, the tests artifacts should include a derivation of the custom models. The derivation can be neatly handwritten, but it will generally be more useful in the future if it is converted to some sort of digitally reproducible form, such as a MathCad file, a LaTeX document, or a word processing document with embedded equations. If the solution is implemented in a computer program, a listing as well as the program file location and name should be provided.

Models Using Off-the-shelf or Commercial Codes: Where models use off-the-shelf codes, such as a finite element model created in a commercial software package, the tests artifacts should include both a description of the model and a set of input files for the commercial package (names and location). The version number of the commercial packages should always be included in the input files as a comment and in the description of the model. This is vital, because input files are often version-specific. The description indicates how the model is created in the package. For a finite element stress code, the description would include the types of elements used, the material models, the boundary conditions applied, and the loading cases. The description serves two purposes: it explains the model and it provides the information necessary to implement the model in an alternative package. The input files for the model would generally be included in a listing (human readable form) as well as in digital files (machine readable form). If you use a SPICE circuit simulation to verify the performance of your circuits, include the overall schematics, conditions, and outputs in a form that will allow for the reproduction of your work by another engineer.

Experimental Procedures: When experimental tests are made (using either models or prototypes), the procedure used to perform the test should be recorded. Ideally, the procedure is planned before the test is performed. The procedure should be recorded in enough detail that someone unaffiliated with the team can repeat the test. If experimental checklists or data-recording forms are developed, they should become part of the experimental procedures. If Excel spreadsheets, Matlab files, or other computer codes are developed to analyze the data, they should be included as well. For software components, a complete description of the tests to be performed to ensure the software meets the intended performance is required. This includes inputs to and outputs from the software.

Market Studies: During most stages of product development, the team will have some contact with market representatives in attempts to discover the market requirements and validate the quality of the design. Procedures for selecting individuals as market representatives, as well as for interacting with the representatives to obtain the desired information should be recorded as part of the test artifacts.

Suggested Format

In general, tests may be provided in both human-readable and machine-readable format. Where only one of these formats apply, only one is necessary.

Procedure Descriptions: Procedure descriptions can be created as individual documents, subject to version control and providing a revision history.

Computer Files: Where computer files are important to performance determination, the name of the file and a description of its importance should be included in the test artifacts. A copy of the computer files should also be provided. The computer files need not be printed. Follow appropriate coding standards to ensure that your code can be understood by an engineer who was not involved with the project.