3.5 Running Tests Under the Test Runner Executable

The test-runner is the bridge from generic Python code to ns-3 code. It is written in C++ and uses the automatic test discovery process in the ns-3 code to find and allow execution of all of the various tests.

Although it may not be used directly very often, it is good to understand how test.py actually runs the various tests.

In order to execute the test-runner, you run it like any other ns-3 executable – using waf. To get a list of available options, you can type:

  ./waf --run "test-runner --help"

You should see something like the following:

  Waf: Entering directory `/home/craigdo/repos/ns-3-allinone-test/ns-3-dev/build'
  Waf: Leaving directory `/home/craigdo/repos/ns-3-allinone-test/ns-3-dev/build'
  'build' finished successfully (0.353s)
  --basedir=dir:          Set the base directory (where to find src) to ``dir''
  --constrain=test-type:  Constrain checks to test suites of type ``test-type''
  --help:                 Print this message
  --kinds:                List all of the available kinds of tests
  --list:                 List all of the test suites (optionally constrained by test-type)
  --out=file-name:        Set the test status output file to ``file-name''
  --suite=suite-name:     Run the test suite named ``suite-name''
  --verbose:              Turn on messages in the run test suites

There are a number of things available to you which will be familiar to you if you have looked at test.py. This should be expected since the test- runner is just an interface between test.py and ns-3. You may notice that example-related commands are missing here. That is because the examples are really not ns-3 tests. test.py runs them as if they were to present a unified testing environment, but they are really completely different and not to be found here.

One new option that appears here is the --basedir option. It turns out that the tests may need to reference the source directory of the ns-3 distribution to find local data, so a base directory is always required to run a test. To run one of the tests directly from the test-runner, you will need to specify the test suite to run along with the base directory. So you could do,

  ./waf --run "test-runner --basedir=`pwd` --suite=pcap-file-object"

Note the “backward” quotation marks on the pwd command. This will run the pcap-file-object test quietly. The only indication that you will get that the test passed is the absence of a message from waf saying that the program returned something other than a zero exit code. To get some output from the test, you need to specify an output file to which the tests will write their XML status using the --out option. You need to be careful interpreting the results because the test suites will append results onto this file. Try,

  ./waf --run "test-runner --basedir=`pwd` --suite=pcap-file-object --out=myfile.xml''

If you look at the file myfile.xml you should see something like,

<TestSuite>
  <SuiteName>pcap-file-object</SuiteName>
  <TestCase>
    <CaseName>Check to see that PcapFile::Open with mode ``w'' works</CaseName>
    <CaseResult>PASS</CaseResult>
    <CaseTime>real 0.00 user 0.00 system 0.00</CaseTime>
  </TestCase>
  <TestCase>
    <CaseName>Check to see that PcapFile::Open with mode ``r'' works</CaseName>
    <CaseResult>PASS</CaseResult>
    <CaseTime>real 0.00 user 0.00 system 0.00</CaseTime>
  </TestCase>
  <TestCase>
    <CaseName>Check to see that PcapFile::Open with mode ``a'' works</CaseName>
    <CaseResult>PASS</CaseResult>
    <CaseTime>real 0.00 user 0.00 system 0.00</CaseTime>
  </TestCase>
  <TestCase>
    <CaseName>Check to see that PcapFileHeader is managed correctly</CaseName>
    <CaseResult>PASS</CaseResult>
    <CaseTime>real 0.00 user 0.00 system 0.00</CaseTime>
  </TestCase>
  <TestCase>
    <CaseName>Check to see that PcapRecordHeader is managed correctly</CaseName>
    <CaseResult>PASS</CaseResult>
    <CaseTime>real 0.00 user 0.00 system 0.00</CaseTime>
  </TestCase>
  <TestCase>
    <CaseName>Check to see that PcapFile can read out a known good pcap file</CaseName>
    <CaseResult>PASS</CaseResult>
    <CaseTime>real 0.00 user 0.00 system 0.00</CaseTime>
  </TestCase>
  <SuiteResult>PASS</SuiteResult>
  <SuiteTime>real 0.00 user 0.00 system 0.00</SuiteTime>
</TestSuite>

If you are familiar with XML this should be fairly self-explanatory. It is also not a complete XML file since test suites are designed to have their output appended to a master XML status file as described in the test.py section.

This document was generated on November 13, 2009 using texi2html 1.82.