Bugzilla – Bug 1469
lte-phy-error-test fails in ns-3-dev
Last modified: 2012-10-17 11:40:35 UTC
I'm logging this issue to notify people that the regression error in this test due to the random variable checkins of 10 July is a known issue and is planned to be fixed later this week.
Assigning to Marco after this email exchange: > On 07/11/2012 08:13 AM, Tom Henderson wrote: >> Some work that I did on the random variables caused some divergence >> in how the random variables get assigned to streams. >> >> One test affected is lte-phy-error-model. However, it is failing in >> a way that can be reproduced prior to the checkin. >> >> Specifically for ns-3.14 release, using the default RngRun=1: >> >> NS_GLOBAL_VALUE="RngRun=1" ./test.py -s lte-phy-error-model -t >> out.txt (passes) >> >> but: >> >> NS_GLOBAL_VALUE="RngRun=2" ./test.py -s lte-phy-error-model -t >> out.txt (fails) >> >> The failure states (out.txt): >> >> Message: Unexpected BER distribution! >> Condition: lambda (actual) < np (limit) + m_bernQuantile (tol) >> && lambda (actual) > np (limit) - m_bernQuantile (tol) >> Actual: 632 >> Limit: 666.65 +- 29 >> File: ../src/lte/test/lte-test-phy-error-model.cc >> Line: 253 >> >> >> >> and other values of RngRun fail, up to RngRun=11. However, RngRun=11 >> succeeds. >> >> With the new random variable system just checked in, the default >> RngRun=1 fails, but RngRun=11 passes. >> >> I'm wondering whether the tolerance that you chose for fitting the >> simulated data to the expected cure is too tight? On 07/11/2012 11:54 AM, Marco Miozzo wrote: > I've already verified that is a problem of confidence interval, I > decided to move to 99% CI, which implies a quantile of 39 and therefore > the test will not fail. Tomorrow, I'll synch the lena-dev repo with > latest ns3-dev, I'll solve the bug and then I'll update the ns3-dev repo > with the lena one polished by the test error. >
Add further investigation I decided for the usage of the 99.9 % CI, since the quantiles were still too tight.
Solved with 99.9% of CI, the reason behind incrementing the CI is the quantization of the tables used by the error model
pushed in ns3-dev changeset: 8890:311667416cd2