Bug 1847

Summary:	[feature request] Summary of crashed / failed tests
Product:	ns-3	Reporter:	Tommaso Pecorella <tommaso.pecorella>
Component:	test framework	Assignee:	Tom Henderson <tomh>
Status:	RESOLVED FIXED
Severity:	enhancement	CC:	tomh
Priority:	P5
Version:	ns-3-dev
Hardware:	All
OS:	All
Attachments:	patch to provide feature

Description Tommaso Pecorella 2014-01-26 12:36:07 EST

Is it possible to have test.py summarize the list of failed/crashed tests at the end of the run ?

I'm not sure about suppressing the PASS ones, as they're useful to see the tests progression.

Thanks, T.

Comment 1 Tom Henderson 2014-04-20 13:20:37 EDT

Created attachment 1820 [details]
patch to provide feature

Comment 2 Tom Henderson 2014-04-20 13:24:42 EDT

Peter also asked about this feature; would be useful for testing.  Attached is a proposed patch.  It does not change the output upon successful full completion (no skips, failed, crashed, valgrind) .  It does print out the list of names of each of those categories if one or more exist.  Example output below (all of the lines below the line starting with '28 of 30' would be new output in this example):

...
PASS: Example src/core/examples/sample-simulator
SKIP: Example src/core/examples/main-ptr
PASS: Example src/core/examples/sample-random-variable
SKIP: Example src/core/examples/sample-simulator.py
PASS: TestSuite random-number-generators
PASS: TestSuite threaded-simulator
PASS: TestSuite random-variable-stream-generators
28 of 30 tests passed (28 passed, 2 skipped, 1 failed, 1 crashed, 0 valgrind errors)
List of SKIPped tests: src/core/examples/sample-simulator.py src/core/examples/main-ptr
List of FAILed tests: watchdog
List of CRASHed tests: examples/tutorial/fourth

Comment 3 Tommaso Pecorella 2014-04-20 13:33:34 EDT

Lovely, it's what I had in mind.

+1

Comment 4 Tom Henderson 2014-04-30 16:10:54 EDT

pushed in changeset 04ea88ac6d8d