Bug 1469

Summary:	lte-phy-error-test fails in ns-3-dev
Product:	ns-3	Reporter:	Tom Henderson <tomh>
Component:	lte	Assignee:	Marco Miozzo <mmiozzo>
Status:	RESOLVED FIXED
Severity:	normal	CC:	nicola, ns-bugs
Priority:	P5
Version:	pre-release
Hardware:	All
OS:	All

Description Tom Henderson 2012-07-11 09:37:22 UTC

I'm logging this issue to notify people that the regression error in this test due to the random variable checkins of 10 July is a known issue and is planned to be fixed later this week.

Comment 1 Nicola Baldo 2012-07-11 09:47:04 UTC

Assigning to Marco after this email exchange:

> On 07/11/2012 08:13 AM, Tom Henderson wrote:
>> Some work that I did on the random variables caused some divergence
>> in how the random variables get assigned to streams.
>>
>> One test affected is lte-phy-error-model.  However, it is failing in
>> a way that can be reproduced prior to the checkin.
>>
>> Specifically for ns-3.14 release, using the default RngRun=1:
>>
>> NS_GLOBAL_VALUE="RngRun=1" ./test.py -s lte-phy-error-model -t
>> out.txt (passes)
>>
>> but:
>>
>> NS_GLOBAL_VALUE="RngRun=2" ./test.py -s lte-phy-error-model -t
>> out.txt (fails)
>>
>> The failure states (out.txt):
>>
>>       Message:    Unexpected BER distribution!
>>       Condition: lambda (actual) < np (limit) + m_bernQuantile (tol)
>> && lambda (actual) > np (limit) - m_bernQuantile (tol)
>>       Actual:    632
>>       Limit:     666.65 +- 29
>>       File:      ../src/lte/test/lte-test-phy-error-model.cc
>>       Line:      253
>>
>>
>>
>> and other values of RngRun fail, up to RngRun=11.  However, RngRun=11
>> succeeds.
>>
>> With the new random variable system just checked in, the default
>> RngRun=1 fails, but RngRun=11 passes.
>>
>> I'm wondering whether the tolerance that you chose for fitting the
>> simulated data to the expected cure is too tight?



On 07/11/2012 11:54 AM, Marco Miozzo wrote:
> I've already verified that is a problem of confidence interval, I
> decided to move to 99% CI, which implies a quantile of 39 and therefore
> the test will not fail. Tomorrow, I'll synch the lena-dev repo with
> latest ns3-dev, I'll solve the bug and then I'll update the ns3-dev repo
> with the lena one polished by the test error.
>

Comment 2 Marco Miozzo 2012-07-11 09:56:48 UTC

Add further investigation I decided for the usage of the 99.9 % CI, since the quantiles were still too tight.

Comment 3 Marco Miozzo 2012-07-11 10:41:51 UTC

Solved with 99.9% of CI, the reason behind incrementing the CI is the quantization of the tables used by the error model

Comment 4 Marco Miozzo 2012-07-11 11:03:42 UTC

pushed in ns3-dev changeset:   8890:311667416cd2