Guide to the LSB Test Suites

The aim of this document is to help people understand how the installation and setup scripts for the TET based LSB test suites work. Also described is how the test suites and the associated framework are stored in CVS and way in which the source based tarballs can be generated.

Parts of this document may be unclear, out of date or simply wrong. I encourage you to send me feedback on problems you may find or other suggestions. In many cases looking at the shell scripts themselves will give you the best detailed description, but hopefully the information here will act as a good outline to how things are meant to work at a higher level.

Building and running the source based LSB test suites

The source based LSB test suites are distributed as a single tarball (tet_vsxgen_<version>.tar.gz) containing TET and VSXgen and multiple optional tarballs one for each test suite (eg VSX-PCTS, LSB-FHS, VSTHlite).

Installation Process

The script handles the installation of the TET & VSXgen framework and the individual test suites. This script must be run as the root user. A summary of the process is:

Setup Process

The setup of the test suites must be done as the vsx0 user. The is the starting point for the setup and buidling and running of the installed test suites.

A summary of is: is responsible for performing the rest of the setup of the test suite and VSXgen framework and the building of the VSXgen framework, but notably not the building of the test programs.

The setup framework was designed to allow for the configuration of TET/VSXgen based test suites both which have already been written such as VSX-PCTS and LSB-FHS as those which are still being more actively developed and tested such LSB-OS and Users/Groups. The configuration demands are dependent on which specific test suites have been installed on the current system. Prior to this system, specific scripts for each test suite had to be written which contained a lot of common code, but required separate maintenance. To allow for test suite specific configurations the looks for and calls test set specific scripts and various stages of the setup of the test suites. The points at which this is done are:

Refer to the description of the generic structure of the test suite tarballs for more information on how the test suite specific setup scripts are stored and searched for.

A summary of the process of the script:

Build and Execution of the test suites optionally builds (or rebuilds) the test programs that comprise the test suites. It then exeecutes the test programs. A summary of the test results is displayed and the tests which failed are optionally re-executed in another run.

The script should not be run directly but instead the script rerun_tests should be called which sets up the environment correctly. This is especially true after a reboot of the machine as features such as loopback devices often need to be reconfigured again.

CVS Storage of the Test Suites

The source code for the test suites are stored in the test_sets module of the LSB project on Source Forge. The README file in the root directory of the module has a description of the directory layout.

In addition to the test suite code, the test_sets module also contains code for the installation scripts, and the VSXgen harness. The VSXgen harness has been modified from the original source. The directory hierarchy containing the VSXgen source also contains intermingled with it part of the installation script framework.

None of the TET code is checked in to CVS and it is used by the test suites unmodified with the single exception of a configured Makefile. The TET code was obtained from the 3.3f release. There are also a few necessary scripts which are not checked in to CVS (I thought they were, but looking at CVS now they aren't).

Creating the TET/VSXgen tarball

This tarball is logically a merge between TET and the contents of various directories in the harness directory of the module. The root directory of the tarball is the same as that of TET. As this process is fairly involved in practice when updating the tarball I haven't built it from the original TET source in quite a while, but instead I unpack the tarball and update the needed files from a checked out CVS tree and then rebuild the tarball. It is also necessary to set the execution permission bit of a few scripts as they were initially checked into CVS without it and it doesn't seem possible to change it now.

What we really should have here is a script which generates the tarball automatically entirely (perhaps with TET being an exception) from CVS. This would ensure we don't accidentally get out of sync and that CVS is always updated properly.

Creating the test suite tarballs

These are generated with the root directory of the tarball being the subdirectory of the test_sets module for each test suite. They should not contain the CVS directories. The cvs export command is very useful for checking out the latest version for creating the tarballs.

Structure of the individual test suites in CVS