1Functional and Stress Tests:  Definition and Structure
2======================================================
3
4This document describes what functional and stress tests are, how they
5are created, and the structure that they follow in CVS.
6
7Functional Test Definition
8--------------------------
9
10Functional tests test the behavior of functional areas of the POSIX*
11specification.  They do not relate directly to lines in the POSIX
12specifications, but rather test that the overall behavior specified by
13the functional area is present.
14
15Stress Test Definition
16----------------------
17
18Stress tests are designed to monitor how the system behaves when it is
19taxed by excessively using the functional areas in the POSIX specification
20or monitoring how the functional areas in the POSIX specification behave
21when the system is taxed.
22
23Directory Structure
24-------------------
25Functional and stress tests follow the same structure.
26
27The functional/ and stress directories off of posixtestsuite are where the
28functional and stress tests, respectively, are stored.
29
30Each of these are further subdivided into POSIX area (Timers, Semaphores,
31etc.).  For example, functional/timers.
32
33Within the functional/<POSIX area> directory, you should create the
34following files.
35- Makefile - master Makefile which will make all tests within the area
36
37- run.sh - file which will run all functional tests.  The run.sh can run
38conformance tests, functional tests from a different area (i.e.,
39functional/threads/run.sh can call functional/timers/threadstests/test.c),
40as well as functional tests in the current area.  If there are tests that
41apply to more than one area, they can go in either area and be called from
42the other if needed.
43Assume that the run.sh is called from the directory it is currently in
44when writing path names to files.
45
46- assertions.xml - This file maps test descriptions to test cases as in
47conformance tests.  The grammar for functional/stress tests is:
48<?xml version "1.0"?>
49<!DOCTYPE assertions [
50  <!ELEMENT assertion>
51  <!ATTLIST assertion
52      files  CDATA    #REQUIRED
53      tag    CDATA    #REQUIRED
54  >
55]>
56Where the tag follows the same structure outlined in HOWTO_Assertions, and
57the files="..." field tags a comma delimited list of files which are
58used to test the assertion.  For example,
59  <assertion id="1" files="threadone,threadtwo,threadthree" tag="pt:THR">
60  Three threads can be run in parallel.
61  </assertion>
62
63- coverage.txt - This document contain information on the completion of
64test cases for assertions as in the conformance tests.
65Content is lines with content:
66<assertion ID>		<YES/NO - for completed/not completed>
67
68So long as the functional/stress tests follow these guidelines, they
69can be built and run by the framework.  How the directory is subdivided
70is up to the area creator to define, but some suggestions have been:
71 -  create directories corresponding to POSIX chapters (see .1-1990 or
72    .1-1996 specs)
73 - create a number corresponding to the assertion ID for each directory
74
75* POSIX (R) is a registered trademark of the IEEE
76
77Contributors:	julie.n.fleischer REMOVE-THIS AT intel DOT com
78                ajosey REMOVE-THIS AT rdg DOT opengroup DOT org
79                geoffrey.r.gustafson REMOVE-THIS AT intel DOT com
80