Bug 1847 - [feature request] Summary of crashed / failed tests
[feature request] Summary of crashed / failed tests
Product: ns-3
Classification: Unclassified
Component: test framework
All All
: P5 enhancement
Assigned To: Tom Henderson
Depends on:
  Show dependency treegraph
Reported: 2014-01-26 12:36 EST by Tommaso Pecorella
Modified: 2014-04-30 16:10 EDT (History)
1 user (show)

See Also:

patch to provide feature (2.39 KB, text/plain)
2014-04-20 13:20 EDT, Tom Henderson

Note You need to log in before you can comment on or make changes to this bug.
Description Tommaso Pecorella 2014-01-26 12:36:07 EST
Is it possible to have test.py summarize the list of failed/crashed tests at the end of the run ?

I'm not sure about suppressing the PASS ones, as they're useful to see the tests progression.

Thanks, T.
Comment 1 Tom Henderson 2014-04-20 13:20:37 EDT
Created attachment 1820 [details]
patch to provide feature
Comment 2 Tom Henderson 2014-04-20 13:24:42 EDT
Peter also asked about this feature; would be useful for testing.  Attached is a proposed patch.  It does not change the output upon successful full completion (no skips, failed, crashed, valgrind) .  It does print out the list of names of each of those categories if one or more exist.  Example output below (all of the lines below the line starting with '28 of 30' would be new output in this example):

PASS: Example src/core/examples/sample-simulator
SKIP: Example src/core/examples/main-ptr
PASS: Example src/core/examples/sample-random-variable
SKIP: Example src/core/examples/sample-simulator.py
PASS: TestSuite random-number-generators
PASS: TestSuite threaded-simulator
PASS: TestSuite random-variable-stream-generators
28 of 30 tests passed (28 passed, 2 skipped, 1 failed, 1 crashed, 0 valgrind errors)
List of SKIPped tests: src/core/examples/sample-simulator.py src/core/examples/main-ptr
List of FAILed tests: watchdog
List of CRASHed tests: examples/tutorial/fourth
Comment 3 Tommaso Pecorella 2014-04-20 13:33:34 EDT
Lovely, it's what I had in mind.

Comment 4 Tom Henderson 2014-04-30 16:10:54 EDT
pushed in changeset 04ea88ac6d8d