|
Bugzilla – Full Text Bug Listing |
| Summary: | Run regression tests as WAF build tasks, for parallel execution | ||
|---|---|---|---|
| Product: | ns-3 | Reporter: | Gustavo J. A. M. Carneiro <gjcarneiro> |
| Component: | build system | Assignee: | ns-bugs <ns-bugs> |
| Status: | RESOLVED FIXED | ||
| Severity: | normal | CC: | mathieu.lacage, tomh |
| Priority: | P5 | ||
| Version: | ns-3-dev | ||
| Hardware: | All | ||
| OS: | All | ||
| Attachments: |
regression system refactoring
patch (fixes ns2 mobility test) patch (removes leftover debug code) patch (fixes clean build) |
||
Created attachment 358 [details]
patch (fixes ns2 mobility test)
Created attachment 359 [details]
patch (removes leftover debug code)
_uber_ cool ! Hi, I tried the most recent patch as follows cd ns-3-allinone (was already downloaded) mv ns-3-dev ns-3-dev.bak hg clone http://code.nsnam.org/ns-3-dev cd ns-3-dev patch -p1 < your patch ./waf configure --with-regression-traces=../ns-3-dev-ref-traces ./waf -j4 ./waf check ./waf --regression and here is what it output: Entering directory `/home/tomh/hg/ns-3-allinone/ns-3-dev/build' Traceback (most recent call last): File "./waf", line 127, in <module> Scripting.prepare(t, cwd, VERSION, wafdir) File "/home/tomh/hg/ns-3-allinone/ns-3-dev/.waf-1.5.2-93db9eb3e2d4af3786fb4bf4f2b52f0b/wafadmin/Scripting.py", line 122, in prepare prepare_impl(t,cwd,ver,wafdir) File "/home/tomh/hg/ns-3-allinone/ns-3-dev/.waf-1.5.2-93db9eb3e2d4af3786fb4bf4f2b52f0b/wafadmin/Scripting.py", line 115, in prepare_impl main() File "/home/tomh/hg/ns-3-allinone/ns-3-dev/.waf-1.5.2-93db9eb3e2d4af3786fb4bf4f2b52f0b/wafadmin/Scripting.py", line 168, in main f(bld) File "/home/tomh/hg/ns-3-allinone/ns-3-dev/wscript", line 426, in build regression.run_regression(bld, regression_traces) File "/home/tomh/hg/ns-3-allinone/ns-3-dev/regression.py", line 216, in run_regression build_traces_dir = bld.path.find_dir('regression/traces').abspath(bld.env) AttributeError: 'NoneType' object has no attribute 'abspath' this behavior doesn't manifest itself on a non-patched ns-3-dev Created attachment 361 [details]
patch (fixes clean build)
Yes, sorry about that. Should be fixed in this patch.
|
Created attachment 357 [details] regression system refactoring I have a patch that refactors the whole regression testing framework to make it run as WAF tasks. The main advantage is that those tasks will run in parallel on multicore systems. In the process I made some simplifications. Before, this happened: WAF: insert fake module in sys.modules mod = import test script mod.run(bunch of parameters) mod.run: import fake module call fake module with bunch of parameters So we have a lot of indirection, waf calls script, script calls back some code that runs in waf space. Extensibility is painful. I got rid of this system. Instead, now the modules are only used for consulting. WAF: import test script get test dir name, or infer from the test script name if not defined call may_run(env) to check if the test can run, or assume it can if not defined get program name, guess program name if not defined get program arguments, assume no arguments if not defined Bottom line is that now most test scripts are empty, as everything is guessed. Some scripts override the defaults, for instance test-csma-bridge.py: --------------- import os.path def may_run(env): """Returns 0 when it can run, return non-zero or string (reason) when it cannot run""" if env['ENABLE_PYTHON_BINDINGS']: return 0 else: return "Python bindings not available." pyscript = os.path.join('examples', 'csma-bridge.py') ------------------ The only problem I am having with this patch is that the test ns-2-mob is always failing for me. It works if I regenerate traces. I am not sure whether it is the fault of the refactoring or not. --- /home/gjc/projects/ns/ns-3-allinone/ns-3-dev-ref-traces/main-ns2-mob.ref/out.tr 2009-01-22 15:46:45.000000000 +0000 +++ /home/gjc/projects/ns/ns-3-allinone/ns-3-dev/build/debug/regression/traces/main-ns2-mob.ref/out.tr 2009-01-22 15:56:09.000000000 +0000 @@ -1,3 +0,0 @@ -3000000000ns POS: x=0, y=25, z=0; VEL:25, y=0, z=0 -4800000000ns POS: x=45, y=25, z=0; VEL:0, y=0, z=0 -5000000000ns POS: x=45, y=25, z=0; VEL:25, y=0, z=0