Bugzilla – Bug 449
waf regression exit code is zero even if a test fails
Last modified: 2009-01-24 10:51:29 UTC
Both "./waf --regression" and "./waf --valgrind --regression" return exit code zero even if one of the tests failed. This makes it harder for scripts such as the nightly regression to check whether there was a failure or not. If it is easy to set exit code to one upon a test failure, I would suggest to make that modification; otherwise, I'll have the nightly test look for strings in the log output that indicate errors.
Created attachment 338 [details] patch Sorry about that. I accidentally broke it recently when refactoring the regression stuff into a separate file (don't remember the bug number). This patch fixes it.
(In reply to comment #1) > Created an attachment (id=338) [details] > patch > > Sorry about that. I accidentally broke it recently when refactoring the > regression stuff into a separate file (don't remember the bug number). This > patch fixes it. > That fixed "./waf --regression" (patch was applied) but I still notice a problem with "./waf --valgrind --regression" which is due to valgrind not reporting an error code. The below option would fix it, I believe-- I suggest to check it in after the release. diff -r 2efae18e7379 wscript --- a/wscript Wed Dec 17 12:54:16 2008 -0800 +++ b/wscript Thu Dec 18 05:50:18 2008 -0800 @@ -461,7 +461,7 @@ def get_command_template(*arguments): if Params.g_options.valgrind: if Params.g_options.command_template: Params.fatal("Options --command-template and --valgrind are conflicting") - cmd = "valgrind --leak-check=full %s" + cmd = "valgrind --leak-check=full --error-exitcode=1 %s" else: cmd = Params.g_options.command_template or '%s' for arg in arguments:
I believe this was fixed for the 3.3 release? If so, Gustavo, please close it out.
(In reply to comment #2) > > That fixed "./waf --regression" (patch was applied) but I still notice a > problem with "./waf --valgrind --regression" which is due to valgrind not > reporting an error code. The below option would fix it, I believe-- I suggest > to check it in after the release. > > diff -r 2efae18e7379 wscript > --- a/wscript Wed Dec 17 12:54:16 2008 -0800 > +++ b/wscript Thu Dec 18 05:50:18 2008 -0800 > @@ -461,7 +461,7 @@ def get_command_template(*arguments): > if Params.g_options.valgrind: > if Params.g_options.command_template: > Params.fatal("Options --command-template and --valgrind are > conflicting") > - cmd = "valgrind --leak-check=full %s" > + cmd = "valgrind --leak-check=full --error-exitcode=1 %s" > else: > cmd = Params.g_options.command_template or '%s' > for arg in arguments: > The patch looks good. I actually was not aware that valgrind returned 0 even when errors were detected.