A Discrete-Event Network Simulator
Home
Tutorials ▼
English
Documentation ▼
Installation
Manual
Models
Contributing
Wiki
Development ▼
API Docs
Issue Tracker
Merge Requests
API
Loading...
Searching...
No Matches
example-as-test.h
Go to the documentation of this file.
1
/*
2
* Copyright (c) 2020 Lawrence Livermore National Laboratory
3
*
4
* This program is free software; you can redistribute it and/or modify
5
* it under the terms of the GNU General Public License version 2 as
6
* published by the Free Software Foundation;
7
*
8
* This program is distributed in the hope that it will be useful,
9
* but WITHOUT ANY WARRANTY; without even the implied warranty of
10
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
11
* GNU General Public License for more details.
12
*
13
* You should have received a copy of the GNU General Public License
14
* along with this program; if not, write to the Free Software
15
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
16
*
17
* Author: Peter D. Barnes, Jr. <pdbarnes@llnl.gov>
18
*/
19
20
#ifndef NS3_EXAMPLE_AS_TEST_SUITE_H
21
#define NS3_EXAMPLE_AS_TEST_SUITE_H
22
23
#include "
test.h
"
24
25
#include <string>
26
27
/**
28
* \file
29
* \ingroup testing
30
* Enable examples to be run as meaningful tests.
31
* Declaration of classes ns3::ExampleAsTestSuite and ns3::ExampleAsTestCase.
32
*/
33
34
namespace
ns3
35
{
36
37
/**
38
* \ingroup testing
39
* Execute an example program as a test, by comparing the output
40
* to a reference file.
41
*
42
* User can subclass and override the GetCommandTemplate() and
43
* GetPostProcessingCommand() methods if more complex
44
* example invocation patterns are required.
45
*
46
* \see examples-as-tests-test-suite.cc
47
*/
48
class
ExampleAsTestCase
:
public
TestCase
49
{
50
public
:
51
/**
52
* Constructor.
53
* \param [in] name The test case name, typically the program name
54
* and summary of the arguments, such as `my-example-foo`
55
* \param [in] program The actual example program names, such as `my-example`
56
* \param [in] dataDir The location of the reference file.
57
* This is normally provided by the symbol
58
* `NS_TEST_SOURCEDIR` in the `module-examples-test-suite.cc`
59
* file.
60
* The reference file should be named after
61
* the test case name,
62
* for example `my-example-foo.log`. If you use
63
* the `--update` argument to `test.py` or
64
* `test-runner` the reference file will be created
65
* with the correct name.
66
* \param [in] args Any additional arguments to the program.
67
* \param [in] shouldNotErr Whether an error return status should be
68
* considered a test failure. This is useful when testing
69
* error detection which might return a non-zero status.
70
* The output (on `std::cout` and `std::cerr`) will
71
* be compared to the reference logs as normal.
72
*/
73
ExampleAsTestCase
(
const
std::string name,
74
const
std::string program,
75
const
std::string dataDir,
76
const
std::string args =
""
,
77
const
bool
shouldNotErr =
true
);
78
79
/** Destructor. */
80
~ExampleAsTestCase
()
override
;
81
82
/**
83
* Customization point for more complicated patterns
84
* to invoke the example program.
85
*
86
* \returns The string to be given to the `ns3 --command-template=` argument.
87
*/
88
virtual
std::string
GetCommandTemplate
()
const
;
89
90
/**
91
* Customization point for tests requiring post-processing of stdout.
92
*
93
* For example to sort return `"| sort"`
94
*
95
* One common case is to mask memory addresses, which can change
96
* when things are built on different platforms, recompiled locally,
97
* or even from run to run. A simple post-processing filter could be
98
*
99
* `"| sed -E 's/0x[0-9a-fA-F]{8,}/0x-address/g'"`
100
*
101
* Default is `""`, no additional processing.
102
*
103
* \returns The string of post-processing commands
104
*/
105
virtual
std::string
GetPostProcessingCommand
()
const
;
106
107
// Inherited
108
void
DoRun
()
override
;
109
110
protected
:
111
std::string
m_program
;
/**< The program to run. */
112
std::string
m_dataDir
;
/**< The source directory for the test. */
113
std::string
m_args
;
/**< Any additional arguments to the program. */
114
bool
m_shouldNotErr
;
/**< Whether error return status is a test failure. */
115
116
};
// class ExampleAsTestCase
117
118
/**
119
* \ingroup testing
120
* Execute an example program as a test suite.
121
*
122
* You can use this TestSuite to add an example to the test suite with
123
* checking of the example output (`std::out` and `std::err`). This
124
* is an alternative to adding an example using the
125
* `examples-to-run.py` file. The key difference between the two
126
* methods is what criteria is used to for success. Examples added to
127
* `examples-to-run.py` will be run and the exit status checked
128
* (non-zero indicates failure). ExampleAsTestSuite adds checking of
129
* output against a specified known "good" reference file.
130
*
131
* \warning If you are thinking about using this class, strongly
132
* consider using a standard test instead. The TestSuite class has
133
* better checking using the NS_TEST_* macros and in almost all cases
134
* is the better approach. If your test can be done with a TestSuite
135
* class you will be asked by the reviewers to rewrite the test when
136
* you do a pull request.
137
*
138
* \par Test Addition
139
*
140
* To use an example program as a test you need to create a test suite
141
* file and add it to the appropriate list in your module CMakeLists.txt
142
* file. The "good" output reference file needs to be generated for
143
* detecting regressions.
144
*
145
* Let's assume your module is called `mymodule`, and the example
146
* program is `mymodule/examples/mod-example.cc`. First you should
147
* create a test file `mymodule/test/mymodule-examples-test-suite.cc`
148
* which looks like this:
149
*
150
* \code{.cpp}
151
* #include "ns3/example-as-test.h"
152
* static ns3::ExampleAsTestSuite g_modExampleOne("mymodule-example-mod-example-one",
153
* "mod-example", NS_TEST_SOURCEDIR, "--arg-one");
154
* static ns3::ExampleAsTestSuite g_modExampleTwo("mymodule-example-mod-example-two",
155
* "mod-example", NS_TEST_SOURCEDIR, "--arg-two"); \endcode
156
*
157
* The arguments to the constructor are the name of the test suite, the
158
* example to run, the directory that contains the "good" reference file
159
* (the macro `NS_TEST_SOURCEDIR` is normally the correct directory),
160
* and command line arguments for the example. In the preceding code
161
* the same example is run twice with different arguments.
162
*
163
* You then need to add that newly created test suite file to the list
164
* of test sources in `mymodule/CMakeLists.txt`. Building of examples
165
* is an option so you need to guard the inclusion of the test suite:
166
*
167
* \code{.py}
168
* if (bld.env['ENABLE_EXAMPLES']):
169
* module.source.append('model/mymodule-examples-test-suite.cc')
170
* \endcode
171
*
172
* Since you modified a CMakeLists.txt file you need to reconfigure and
173
* rebuild everything.
174
*
175
* You just added new tests so you will need to generate the "good"
176
* output reference files that will be used to verify the example:
177
*
178
* `./test.py --suite="mymodule-example-*" --update`
179
*
180
* This will run all tests starting with "mymodule-example-" and save
181
* new "good" reference files. Updating the reference file should be
182
* done when you create the test and whenever output changes. When
183
* updating the reference output you should inspect it to ensure that
184
* it is valid. The reference files should be committed with the new
185
* test.
186
*
187
* \par Test Verification
188
*
189
* You can run the test with the standard `test.py` script. For
190
* example to run the suites you just added:
191
*
192
* `./test.py --suite="mymodule-example-*"`
193
*
194
* This will run all `mymodule-example-...` tests and report whether they
195
* produce output matching the reference files.
196
*
197
* \par Writing good examples for testing
198
*
199
* When setting up an example for use by this class you should be very
200
* careful about what output the example generates. For example,
201
* writing output which includes simulation time (especially high
202
* resolution time) makes the test sensitive to potentially minor
203
* changes in event times. This makes the reference output hard to
204
* verify and hard to keep up-to-date. Output as little as needed for
205
* the example and include only behavioral state that is important for
206
* determining if the example has run correctly.
207
*
208
*/
209
class
ExampleAsTestSuite
:
public
TestSuite
210
{
211
public
:
212
/**
213
* \copydoc ExampleAsTestCase::ExampleAsTestCase
214
* \param [in] duration Amount of time this test takes to execute
215
* (defaults to QUICK).
216
*/
217
ExampleAsTestSuite
(
const
std::string name,
218
const
std::string program,
219
const
std::string dataDir,
220
const
std::string args =
""
,
221
const
Duration
duration =
Duration::QUICK
,
222
const
bool
shouldNotErr =
true
);
223
};
// class ExampleAsTestSuite
224
225
}
// namespace ns3
226
227
#endif
/* NS3_EXAMPLE_TEST_SUITE_H */
ns3::ExampleAsTestCase
Execute an example program as a test, by comparing the output to a reference file.
Definition:
example-as-test.h:49
ns3::ExampleAsTestCase::m_shouldNotErr
bool m_shouldNotErr
Whether error return status is a test failure.
Definition:
example-as-test.h:114
ns3::ExampleAsTestCase::DoRun
void DoRun() override
Implementation to actually run this TestCase.
ns3::ExampleAsTestCase::GetPostProcessingCommand
virtual std::string GetPostProcessingCommand() const
Customization point for tests requiring post-processing of stdout.
ns3::ExampleAsTestCase::ExampleAsTestCase
ExampleAsTestCase(const std::string name, const std::string program, const std::string dataDir, const std::string args="", const bool shouldNotErr=true)
Constructor.
ns3::ExampleAsTestCase::m_args
std::string m_args
Any additional arguments to the program.
Definition:
example-as-test.h:113
ns3::ExampleAsTestCase::m_dataDir
std::string m_dataDir
The source directory for the test.
Definition:
example-as-test.h:112
ns3::ExampleAsTestCase::GetCommandTemplate
virtual std::string GetCommandTemplate() const
Customization point for more complicated patterns to invoke the example program.
ns3::ExampleAsTestCase::m_program
std::string m_program
The program to run.
Definition:
example-as-test.h:111
ns3::ExampleAsTestCase::~ExampleAsTestCase
~ExampleAsTestCase() override
Destructor.
ns3::ExampleAsTestSuite
Execute an example program as a test suite.
Definition:
example-as-test.h:210
ns3::ExampleAsTestSuite::ExampleAsTestSuite
ExampleAsTestSuite(const std::string name, const std::string program, const std::string dataDir, const std::string args="", const Duration duration=Duration::QUICK, const bool shouldNotErr=true)
Constructor.
ns3::TestCase
encapsulates test code
Definition:
test.h:1061
ns3::TestCase::Duration
Duration
How long the test takes to execute.
Definition:
test.h:1065
ns3::TestCase::Duration::QUICK
@ QUICK
Fast test.
ns3::TestSuite
A suite of tests to run.
Definition:
test.h:1273
ns3
Every class exported by the ns3 library is enclosed in the ns3 namespace.
test.h
ns3::TestCase, ns3::TestSuite, ns3::TestRunner declarations, and NS_TEST_ASSERT macro definitions.
src
core
model
example-as-test.h
Generated on Tue May 28 2024 23:34:30 for ns-3 by
1.9.6