Bug 1289

I have previously compiled repositories where tests suites ran correctly with ./test.py. But after I do "./waf distclean" on such repositories, the test suites are not executed any more. That's weird!

Maybe this bug is due to my recent upgrade to Ubuntu 11.10, possibly some package was installed that is not making the test suites run. Still, I consider this a bug (./waf configure does not give any hint about what could be missing that causes the test suites not to run).

Created attachment 1270 [details]
patch enabling logging in src/core/model/test.cc

after applying the proposed test logging patch, I get the following:

nicola@pcnbaldo:/locale/ns-3-dev$ NS_LOG="Test" ./waf --run test-runner
Waf: Entering directory `/locale/ns-3-dev/build'
Waf: Leaving directory `/locale/ns-3-dev/build'
'build' finished successfully (0.285s)
Test:Run()
Test:TestRunnerImpl(0x7f895c050820)
Test:Run(0x7f895c050820)
Test:FilterTests(0x7f895c050820)
Test:FilterTests(): m_suites.size () : 0
Test:~TestRunnerImpl(0x7f895c050820)


My understanding is that m_suites should not be empty. Any idea on what's going on?

(In reply to comment #0)
> using ns-3-dev changeset:   7574:6b792c9b85cc
> 
> even with tests enabled, test suites are not run (only examples are run).
> 
> Here is how to reproduce:
> 
> $ ./waf configure --enable-modules=wifi --enable-tests --enable-examples
> [snip]
> Build tests                   : enabled
> Build examples                : enabled
> GNU Scientific Library (GSL)  : not enabled (GSL not found)
> 'configure' finished successfully (2.575s)
> 
> nicola@pcnbaldo:/locale/ns-3-dev$ ./test.py 
> Waf: Entering directory `/locale/ns-3-dev/build'
> Waf: Leaving directory `/locale/ns-3-dev/build'
> 'build' finished successfully (0.296s)
> 
> Modules built:
> core                      mobility                  network                  
> propagation               wifi                     
> 
> Modules not built:
> click                     openflow                 
> 
> PASS: Example /locale/ns-3-dev/build/examples/tutorial/fourth
> PASS: Example /locale/ns-3-dev/build/examples/tutorial/hello-simulator
> 2 of 2 tests passed (2 passed, 0 skipped, 0 failed, 0 crashed, 0 valgrind
> errors)

I was not able to reproduce this on my Fedora 14 machine.

(In reply to comment #1)
> I have previously compiled repositories where tests suites ran correctly with
> ./test.py. But after I do "./waf distclean" on such repositories, the test
> suites are not executed any more. That's weird!
> 
> Maybe this bug is due to my recent upgrade to Ubuntu 11.10, possibly some
> package was installed that is not making the test suites run. Still, I consider
> this a bug (./waf configure does not give any hint about what could be missing
> that causes the test suites not to run).

I was not able to reproduce this on my Fedora 14 machine.

more testing:

$ ./waf configure --enable-modules=spectrum --enable-tests --enable-examples

$ NS_LOG="Test:SpectrumInterferenceTest" ./waf --run test-runner
Waf: Entering directory `/locale/ns-3-dev/build'
[ 570/1023] cxx: src/spectrum/test/spectrum-interference-test.cc -> build/src/spectrum/test/spectrum-interference-test.cc.3.o
[1016/1023] cxxshlib: build/src/spectrum/test/spectrum-interference-test.cc.3.o build/src/spectrum/test/spectrum-value-test.cc.3.o build/src/spectrum/test/spectrum-ideal-phy-test.cc.3.o -> build/libns3-spectrum-test.so
[1017/1023] cxxprogram: build/utils/test-runner.cc.1.o -> build/utils/test-runner
Waf: Leaving directory `/locale/ns-3-dev/build'
'build' finished successfully (1.432s)
Test:Run()
Test:TestRunnerImpl(0x7f5806bd9820)
Test:Run(0x7f5806bd9820)
Test:FilterTests(0x7f5806bd9820)
Test:FilterTests(): m_suites.size () : 0
Test:~TestRunnerImpl(0x7f5806bd9820)


note that the log line "creating SpectrumInterferenceTestSuite" (spectrum-interference-test.cc:144) is not printed. So I infer that the test library of the spectrum module is not loaded by test-runner, even though the spectrum module is enabled and the spectrum test library has been built, as shown by the following waf output:

[1016/1023] cxxshlib: build/src/spectrum/test/spectrum-interference-test.cc.3.o build/src/spectrum/test/spectrum-value-test.cc.3.o build/src/spectrum/test/spectrum-ideal-phy-test.cc.3.o -> build/libns3-spectrum-test.so

running "./waf -v" after touching src/core/model/test.cc :

[snip]

[1017/1023] cxxprogram: build/utils/test-runner.cc.1.o -> build/utils/test-runner
12:05:33 runner ['/usr/bin/g++', 'utils/test-runner.cc.1.o', '-o', '/locale/ns-3-dev/build/utils/test-runner', '-Wl,-Bstatic', '-Wl,-Bdynamic', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-lns3-spectrum-test', '-lns3-spectrum', '-lns3-applications-test', '-lns3-applications', '-lns3-internet-test', '-lns3-internet', '-lns3-bridge', '-lns3-mpi', '-lns3-propagation-test', '-lns3-propagation', '-lns3-mobility-test', '-lns3-mobility', '-lns3-config-store', '-lns3-tools-test', '-lns3-tools', '-lns3-network-test', '-lns3-network', '-lns3-core-test', '-lns3-core', '-lrt', '-lgsl', '-lgslcblas', '-lm', '-lgtk-x11-2.0', '-lgdk-x11-2.0', '-latk-1.0', '-lgio-2.0', '-lpangoft2-1.0', '-lpangocairo-1.0', '-lgdk_pixbuf-2.0', '-lcairo', '-lpango-1.0', '-lfreetype', '-lfontconfig', '-lgobject-2.0', '-lgmodule-2.0', '-lgthread-2.0', '-lrt', '-lglib-2.0', '-lxml2', '-pthread', '-pthread', '-pthread']

hence libns3-spectrum-test is linked with test-runner. But for some reason the static test suite instance is not created.

$ g++ --version
g++ (Ubuntu/Linaro 4.6.1-9ubuntu3) 4.6.1

problem persists with g++-4.5

works with g++-4.4 !!!

$ ./waf configure --enable-modules=spectrum --enable-tests --enable-examples
Setting top to                           : /locale/ns-3-dev 
Setting out to                           : /locale/ns-3-dev/build 
Checking for 'gcc' (c compiler)          : /usr/bin/gcc 
Checking for 'g++' (c++ compiler)        : g++-4.4 
[snip]

$ ./test.py 
[snip]
PASS: TestSuite attributes
PASS: TestSuite config
PASS: TestSuite command-line
PASS: TestSuite callback
PASS: TestSuite int64x64
PASS: TestSuite global-value
PASS: TestSuite object
PASS: TestSuite object-name-service
[snip]

to summarize, the bug can be reproduced with the following procedure (using changeset:   7574:6b792c9b85cc on Ubuntu 11.10)

$ ./waf distclean

$ export CXX=g++-4.6

$ g++-4.6 --version
g++-4.6 (Ubuntu/Linaro 4.6.1-9ubuntu3) 4.6.1

$ ./waf configure --enable-modules=spectrum --enable-tests --enable-examples

$ ./test.py
[only examples are run, no test suite is run]



And here is the workaround:

install ubuntu package g++-4.4

$ ./waf distclean

$ export CXX=g++-4.4

$ ./waf configure --enable-modules=spectrum --enable-tests --enable-examples

$ ./test.py
[test suites are executed as expected]

for the record, here are the g++ versions that I tested:

$ g++-4.4 --version
g++-4.4 (Ubuntu/Linaro 4.4.6-11ubuntu2) 4.4.6

$ g++-4.5 --version
g++-4.5 (Ubuntu/Linaro 4.5.3-9ubuntu1) 4.5.4

$ g++-4.6 --version
g++-4.6 (Ubuntu/Linaro 4.6.1-9ubuntu3) 4.6.1

seems to be fixed by the fix to bug 1292