Bug 2658 - TracedValue<T> silently performs unwanted type conversions
TracedValue<T> silently performs unwanted type conversions
Status: RESOLVED FIXED
Product: ns-3
Classification: Unclassified
Component: core
ns-3-dev
All All
: P3 normal
Assigned To: Peter Barnes
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2017-02-19 16:47 UTC by Christoph Döpmann
Modified: 2017-06-06 09:39 UTC (History)
3 users (show)

See Also:


Attachments
proposed patch (13.50 KB, patch)
2017-03-15 05:05 UTC, Christoph Döpmann
Details | Diff
proposed patch (with TracedValue<...> return types) (13.72 KB, patch)
2017-03-15 16:10 UTC, Christoph Döpmann
Details | Diff
proposed patch to tcp-hybla.cc (678 bytes, patch)
2017-03-15 16:11 UTC, Christoph Döpmann
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Christoph Döpmann 2017-02-19 16:47:56 UTC
A while ago, I modified some piece of code to operate on TracedValue<uint32_t> instead of uint32_t directly, because I obviously wanted to make the value tracable. After some time (and countless invalid simulations...), I now realised that this introduces a subtly different behavior which I was not aware of and which I think should be changed or documented more clearly.

In my case, I ran into unwanted double-to-uint truncation. Let's consider the following code:

  #include "ns3/core-module.h"
  #include "ns3/traced-value.h"
  
  using namespace ns3;
  
  int 
  main (int argc, char *argv[])
  {
    uint32_t i = 2;
    TracedValue<uint32_t> i_traced = i;
    double d = 0.3;
  
    NS_LOG_UNCOND ("Expected: " << i*d);
    NS_LOG_UNCOND ("Got:      " << i_traced*d);
  }

The output of this is:

  Expected: 0.6
  Got:      0

Unfortunately, TracedValue<T> behaves differently from T in that the return type of any operations on it is, again, TracedValue<T>. When, for example, multiplicated with a value of type U, the result is forcefully converted to T again instead of the type that multiplication would normally return. Especially in more complex, compound expressions, this creates a major source for possible bugs, just as in my case.

I would propose to change this behavior - I can't see why one might want to (silently) override the original operator's return type.

To stick with the above example of multiplication, I would propose a change like the following, changing

  template <typename T, typename U>
  TracedValue<T> operator * (const TracedValue<T> &lhs, const U &rhs) {
    TRACED_VALUE_DEBUG ("x*");
    return TracedValue<T> (lhs.Get () * rhs);
  }

to

  template <typename T, typename U>
  auto operator * (const TracedValue<T> &lhs, const U &rhs) -> TracedValue<decltype(lhs.Get () * rhs)> {
    TRACED_VALUE_DEBUG ("x*");
    return TracedValue<decltype(lhs.Get () * rhs)> (lhs.Get () * rhs);
  }

Of course, the changes to all the other operators would be analogous.

This makes use of C++11 features, which should be fine as recently discussed on the developers mailing list.
Comment 1 Tom Henderson 2017-03-02 22:13:18 UTC
I can confirm this is a serious bug as I unexpectedly encountered it in another project today.  Elevating the priority...
Comment 2 Christoph Döpmann 2017-03-08 11:41:16 UTC
Do you, in principle, like the kind of solution I suggested? Or should we maybe make the operators return T instead of TracedValue<T> all together? I have difficulties coming up with a valid use case in which the result of an operator should again be a TracedValue.
Comment 3 Tom Henderson 2017-03-08 12:27:36 UTC
(In reply to Christoph Döpmann from comment #2)
> Do you, in principle, like the kind of solution I suggested? Or should we
> maybe make the operators return T instead of TracedValue<T> all together? I
> have difficulties coming up with a valid use case in which the result of an
> operator should again be a TracedValue.

Perhaps return decltype(lhs.Get() * rhs) instead of a traced value?  I also don't see the use case for returning TracedValue types.
Comment 4 Christoph Döpmann 2017-03-15 04:49:05 UTC
I've put together a patch that tackles the issue. Please have a look at it.

> Perhaps return decltype(lhs.Get() * rhs) instead of a traced value?  I
> also don't see the use case for returning TracedValue types.

I did it this way, since I totally agree. However, we are running into some issues with user code using the results in std::max(...) or std::min(...). For example, consider this call from tcp-vegas.cc:

  std::max (tcb->m_ssThresh, 3 * tcb->m_cWnd / 4);

The change in return type effectively leads to calling std::max with parameters of different types, one being 'TracedValue<unsigned int>', the other being 'unsigned int'. While these are comparable via the comparison operators implemented in traced-value.h, std::max and std::min do not seem to work that way, requiring exact same types.

Of course, we could easily work around this issue in the ns-3 code base, but I'm not sure how much third-party code this will break. Therefore, we might consider sticking to TracedValue<...> return types, just for the sake of backwards compatibility. What do you think?
Comment 5 Christoph Döpmann 2017-03-15 05:05:37 UTC
Created attachment 2798 [details]
proposed patch
Comment 6 Tom Henderson 2017-03-15 10:11:13 UTC
> 
> Of course, we could easily work around this issue in the ns-3 code base, but
> I'm not sure how much third-party code this will break. Therefore, we might
> consider sticking to TracedValue<...> return types, just for the sake of
> backwards compatibility. What do you think?

Thanks for preparing the patch.

In my opinion, breakage of code using templated library functions will be caught by compilers and is easily fixed by calling "Get ()" on the traced value.

However, if we leave it as is, this kind of code will silently fail to produce the expected result.  As I mentioned, I traced a bug last week to this.

  TracedValue<uint32_t> a = 5;
  double b = 6; 
  //   a/b evaluates to integer zero

The doxygen does describe this behavior, so one could argue it is expected behavior:

 * Infix arithmetic operator for TracedValue.
 *
 * This returns the arithmetic result in a new TracedValue,
 * which has no Callback connected.

but since the class doxygen states "This instance will behave just like the original class ...", I think it will surprise those who do not read operator doxygen.

So I'm in favor of making this change.
Comment 7 Christoph Döpmann 2017-03-15 12:08:43 UTC
Oh, I guess I didn't make myself clear on what I wanted to express.

There's absolutely no doubt that we need to fix these silent conversions like the one you and I encountered. The only thing I was still wondering is whether the return type of the infix operators should be of type

  decltype(lhs.Get () * rhs)

or

  TracedValue<decltype(lhs.Get () * rhs)>.

While we both don't see the use case for the second one, it would avoid code breakage. Both get rid of the silent conversions.

If that - possibly breaking - change is something that is acceptable, I'd definitely go for the first variant. I've also adapted the doxygen accordingly in the patch, so we'd be ready to go, as far as I am concerned.
Comment 8 Tom Henderson 2017-03-15 12:28:32 UTC
> There's absolutely no doubt that we need to fix these silent conversions
> like the one you and I encountered. The only thing I was still wondering is
> whether the return type of the infix operators should be of type
> 
>   decltype(lhs.Get () * rhs)
> 
> or
> 
>   TracedValue<decltype(lhs.Get () * rhs)>.
> 
> While we both don't see the use case for the second one, it would avoid code
> breakage. Both get rid of the silent conversions.

I see... in that case, is there any harm in the second option then?  Infix operators are most likely going to produce rvalues and will then eventually return the underlying type upon assignment.
Comment 9 Christoph Döpmann 2017-03-15 12:35:13 UTC
> I see... in that case, is there any harm in the second option then?

I don't think so, apart from aesthetic reasons. Would you like me to update the patch accordingly?
Comment 10 Tom Henderson 2017-03-15 14:14:49 UTC
(In reply to Christoph Döpmann from comment #9)
> > I see... in that case, is there any harm in the second option then?
> 
> I don't think so, apart from aesthetic reasons. Would you like me to update
> the patch accordingly?

Yes, please, if you don't mind.
Comment 11 Peter Barnes 2017-03-15 14:29:09 UTC
With respect to:

"In my opinion, breakage of code using templated library functions will be caught by compilers and is easily fixed by calling "Get ()" on the traced value."

What about implementing casts from TracedValue<T> to T.  

  T operator T() const { return Get(): }

Isn't the compiler allowed to insert such a cast silently, in order to obtain the required type to match a signature?

Then we could

a.  return decltype(lhs.Get() * rhs), which we agree is the right thing

b.  not break existing code like 
  std::max (TracedValue<T>, 
            [arithmetic with TracedValue<T>, which evaluates to type T]);
Comment 12 Christoph Döpmann 2017-03-15 16:08:25 UTC
> What about implementing casts from TracedValue<T> to T.

That sounded great, so I tried it. Unfortunately, it does not work as expected. The reason is that implicit conversions are part of the overload resolution, no the template argument deduction that needs to take place first. E.g. the following works:

  std::max<int> (TracedValue<int>(42), 41)

But in the following, much more common case, implicit conversions are not considered, so compilation fails:

  std::max (TracedValue<int>(42), 41)

[source: http://en.cppreference.com/w/cpp/language/template_argument_deduction]

So I updated the patch to use TracedValue<...> return types. Everything seems to work as expected. One piece of code in the ns-3 code base fails to compile (tcp-hybla.cc), but it turns out to be exactly what we want because there was a floating-point calculation implicitly being used as unsigned int. I fixed that in another patch.

One more thing I added to the traced-value.h patch: The copy constructor from a TracedValue<U> with compatible type U seemed to be broken. Up until now, this actually wasn't covered by the test case because the previous "hard-coded" return types masked it, but after applying the patch, it became visible.

The problem is that in the following code...

  template <typename U>
  TracedValue (const TracedValue<U> &other)
    : m_v (other.m_v)
  {}

...accessing other.m_v is only valid if U is actually the exact same type as T, because m_v is a private member. I changed this to use other.Get () instead, so it only depends on whether T and U are compatible, which is what we want (instead of being equal).

With that, I think we should be set.
Comment 13 Christoph Döpmann 2017-03-15 16:10:06 UTC
Created attachment 2799 [details]
proposed patch (with TracedValue<...> return types)
Comment 14 Christoph Döpmann 2017-03-15 16:11:23 UTC
Created attachment 2800 [details]
proposed patch to tcp-hybla.cc
Comment 15 Peter Barnes 2017-03-15 19:04:37 UTC
Is the problem just with a handful of templates like std::max and std::min?

Could we implement specializations to transform

template<typename T, typename U>
std::max (TracedValue<T>, U) 
{
  return std::max< [numerical super type of T and U] >(TracedValue<T>, U);
}

So we can 

  return decltype(lhs.Get () * rhs)

and not break 

  std::max (TracedValue<int>(42), 41)
Comment 16 Christoph Döpmann 2017-03-19 08:26:23 UTC
In general, this can occur with any user-written template code. However, I find it extremely hard to estimate how much of an impact it would actually be.

Concerning your proposition, I am not sure if this really is what we want. After all, omitting TracedValue<...> as the operators' return types is mainly meant to make the code cleaner and more comprehensible. It feels like, by introducing special handling for some well-known helpers like std::max/min, we introduce more magic and opaque behavior than is the case with the current situation. People may wonder why everything works with std::max, but not with their custom, fancy version of max(...). And, personally, I'd rather be in favor of having a well-defined API that is less elegant (like the one we have right now) than having a cool API that is hard to understand and may fail (from the user's perspective) on some edge cases (e.g. some custom template code).
Comment 17 Peter Barnes 2017-03-20 19:32:14 UTC
> [O]mitting TracedValue<...> as the operators' return types is mainly meant to make 
> the code cleaner and more comprehensible. ...[B]y introducing special handling for 
> some well-known helpers like std::max/min, we introduce more magic 

Sigh.  Agreed.  

How about in the short term we deprecate the TracedValue<> return type, putting people on notice that it will change to decltype in 1-2 releases?

And in a last ditch effort to get us there sooner, how about implementing a cast to TracedValue, so we can return decltype, and the compiler can cast where the old return type is still needed?  This is the inverse of the cast I suggested earlier:

  template <typename T, typename U>
  TracedValue<T> operator TracedValue<T>() (U value) const 
  { return TracedValue<T> (value): }
Comment 18 Christoph Döpmann 2017-03-29 02:36:20 UTC
Sorry for being a bit unresponsive, lately...

I like the idea of deprecation. I'm not exactly sure how this would essentially work, though. See, if I as an end user get a deprecation warning, I'd normally like to adjust my code to the new variant, avoiding the deprecated piece of code and thus having a gently move over to the new API. I can't however see how this can be done in this specific case. We cannot offer "two variants" of the code under consideration because it's an operator. For the same reason, a deprecation warning wouldn't quite give the user any additional time to prepare for the upcoming breaking change because he cannot deliberately choose the "new" or the "old" API. Or am I mistaken on this?

> And in a last ditch effort to get us there sooner, how about implementing a
> cast to TracedValue, so we can return decltype, and the compiler can cast
> where the old return type is still needed?  This is the inverse of the cast
> I suggested earlier:

Have you tried this? I haven't, but I'm reasonably sure it will run into the same issue as the previous cast did.
Comment 19 Tom Henderson 2017-04-05 12:28:37 UTC
Peter and Christoph, are we settling on the currently proposed patch as the solution?
Comment 20 Christoph Döpmann 2017-04-05 15:13:52 UTC
As far as I am concerned, yes. We'd be sticking with the TracedValue<...> return types for now, which we agree is something we'd like to change in the future. But as it remains unsolved for now how to do this in the most backward-compatible way, this shouldn't be a blocker in my opinion.
Comment 21 Peter Barnes 2017-04-05 15:59:40 UTC
+1 for this patch.

Please leave the bug open so we remember to move forward to decltype.

Could we get that started with a simple doxygen comment?

/**
 * ...
 * \deprecated In a future release of ns-3 the return type will be changed from 
 * TracedValue<T> to \c decltype(auto)
 * ...
 */

(Of course, actually *using* auto for the return type requires C++14...)
Comment 22 Christoph Döpmann 2017-04-05 17:04:33 UTC
> (Of course, actually *using* auto for the return type requires C++14...)

What's the benefit of using decltype(auto) here instead of the previously mentioned decltype(lhs.Get() * rhs)-like approach that uses trailing return type syntax and works with C++11?
Comment 23 Peter Barnes 2017-04-06 15:47:17 UTC
> What's the benefit of using decltype(auto)

It was easier for me to type in the example. :)
Comment 24 Tom Henderson 2017-05-30 09:38:48 UTC
I would like to merge a solution to this by end of the week; moving to LAST CALL state.
Comment 25 Tom Henderson 2017-06-03 15:45:29 UTC
committed both patches in changesets 12912 and 12913; leaving bug open for eventually moving to decltype.
Comment 26 Stefano Avallone 2017-06-06 04:57:59 UTC
Changeset 12912:24d666283766 breaks the generation of python bindings for me:

----------- Log -------------------

[stefano@localhost ns-3-dev-git]$ python2 ./waf --apiscan=core
Waf: Entering directory `/home/stefano/Tools/ns-3-dev-git/build'
Modules to scan:  ['core']
[  58/2578] Compiling install-ns3-header: ns3/traced-value.h
api-scan-gcc_ILP32
api-scan-gcc_LP64
<ns3modulegen_core_customizations.SmartPointerTransformation object at 0x7f190cf643d0>
<ns3modulegen_core_customizations.SmartPointerTransformation object at 0x7f4b8cb773d0>
INFO Parsing source file "/home/stefano/Tools/ns-3-dev-git/src/core/bindings/scan-header.h" ... 
INFO gccxml cmd: /usr/bin/gccxml  --gccxml-cxxflags "-m32 -DPYTHON_SCAN"   -I"." -I"/home/stefano/Tools/ns-3-dev-git/build"   "/home/stefano/Tools/ns-3-dev-git/src/core/bindings/scan-header.h" -fxml="/tmp/tmpHP9y0o.xml"
INFO Parsing source file "/home/stefano/Tools/ns-3-dev-git/src/core/bindings/scan-header.h" ... 
INFO gccxml cmd: /usr/bin/gccxml  --gccxml-cxxflags "-m64 -DPYTHON_SCAN"   -I"." -I"/home/stefano/Tools/ns-3-dev-git/build"   "/home/stefano/Tools/ns-3-dev-git/src/core/bindings/scan-header.h" -fxml="/tmp/tmpJrhP_T.xml"
Traceback (most recent call last):
Traceback (most recent call last):
  File "/home/stefano/Tools/ns-3-dev-git/bindings/python/ns3modulescan-modular.py", line 280, in <module>
  File "/home/stefano/Tools/ns-3-dev-git/bindings/python/ns3modulescan-modular.py", line 280, in <module>
        ns3_module_scan(sys.argv[1], sys.argv[2], eval(sys.argv[3]), sys.argv[4], sys.argv[5])
ns3_module_scan(sys.argv[1], sys.argv[2], eval(sys.argv[3]), sys.argv[4], sys.argv[5])
  File "/home/stefano/Tools/ns-3-dev-git/bindings/python/ns3modulescan-modular.py", line 260, in ns3_module_scan
  File "/home/stefano/Tools/ns-3-dev-git/bindings/python/ns3modulescan-modular.py", line 260, in ns3_module_scan
    gccxml_options=gccxml_options)
  File "/home/stefano/Tools/pybindgen/pybindgen/gccxmlparser.py", line 696, in parse_init
    gccxml_options=gccxml_options)
  File "/home/stefano/Tools/pybindgen/pybindgen/gccxmlparser.py", line 696, in parse_init
    self.declarations = parser.parse(header_files, self.gccxml_config)
    self.declarations = parser.parse(header_files, self.gccxml_config)
  File "/usr/lib/python2.7/site-packages/pygccxml/parser/__init__.py", line 50, in parse
  File "/usr/lib/python2.7/site-packages/pygccxml/parser/__init__.py", line 50, in parse
    answer = parser.read_files(files, compilation_mode)
      File "/usr/lib/python2.7/site-packages/pygccxml/parser/project_reader.py", line 225, in read_files
answer = parser.read_files(files, compilation_mode)
  File "/usr/lib/python2.7/site-packages/pygccxml/parser/project_reader.py", line 225, in read_files
    return self.__parse_file_by_file(files)
  File "/usr/lib/python2.7/site-packages/pygccxml/parser/project_reader.py", line 250, in __parse_file_by_file
    return self.__parse_file_by_file(files)
  File "/usr/lib/python2.7/site-packages/pygccxml/parser/project_reader.py", line 250, in __parse_file_by_file
    decls = reader.read_file( header )
  File "/usr/lib/python2.7/site-packages/pygccxml/parser/source_reader.py", line 197, in read_file
    decls = reader.read_file( header )
  File "/usr/lib/python2.7/site-packages/pygccxml/parser/source_reader.py", line 197, in read_file
    return self.read_gccxml_file( source_file )
    return self.read_gccxml_file( source_file )
  File "/usr/lib/python2.7/site-packages/pygccxml/parser/source_reader.py", line 224, in read_gccxml_file
  File "/usr/lib/python2.7/site-packages/pygccxml/parser/source_reader.py", line 224, in read_gccxml_file
    raise error
    raise error
pygccxml.parser.source_reader.gccxml_runtime_error_t: Error occured while running GCC-XML: In file included from /home/stefano/Tools/ns-3-dev-git/build/ns3/core-module.h:88,
                 from /home/stefano/Tools/ns-3-dev-git/src/core/bindings/scan-header.h:3:
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:444: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:450: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:456: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:463: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:469: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:475: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:482: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:488: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:494: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:501: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:507: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:513: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:520: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:526: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:532: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:539: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:545: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:551: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:558: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:564: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:570: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:577: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:583: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:589: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:596: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:602: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:608: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:615: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:621: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:627: error: expected initializer before '->' token
pygccxml.parser.source_reader
.gccxml_runtime_error_t: Error occured while running GCC-XML: In file included from /home/stefano/Tools/ns-3-dev-git/build/ns3/core-module.h:88,
                 from /home/stefano/Tools/ns-3-dev-git/src/core/bindings/scan-header.h:3:
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:444: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:450: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:456: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:463: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:469: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:475: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:482: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:488: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:494: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:501: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:507: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:513: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:520: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:526: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:532: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:539: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:545: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:551: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:558: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:564: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:570: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:577: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:583: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:589: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:596: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:602: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:608: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:615: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:621: error: expected initializer before '->' token
/home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:627: error: expected initializer before '->' token

Waf: Leaving directory `/home/stefano/Tools/ns-3-dev-git/build'
Build failed
 -> task in '' failed (exit status 1): 
        {task 140186332951504: apiscan_task core-module.h -> modulegen__gcc_ILP32.py}
''
 -> task in '' failed (exit status 1): 
        {task 140186332952656: apiscan_task core-module.h -> modulegen__gcc_LP64.py}
''

------------------------------


I verified that python bindings generation works after reverting the above mentioned commit ("core: (fixes #2658) Avoid unexpected TracedValue type conversions")
Comment 27 Tom Henderson 2017-06-06 09:39:47 UTC
(In reply to Stefano Avallone from comment #26)
> Changeset 12912:24d666283766 breaks the generation of python bindings for me:
> 
> ----------- Log -------------------
> 
> [stefano@localhost ns-3-dev-git]$ python2 ./waf --apiscan=core
> Waf: Entering directory `/home/stefano/Tools/ns-3-dev-git/build'
> Modules to scan:  ['core']
> [  58/2578] Compiling install-ns3-header: ns3/traced-value.h
> api-scan-gcc_ILP32
> api-scan-gcc_LP64
> <ns3modulegen_core_customizations.SmartPointerTransformation object at
> 0x7f190cf643d0>
> <ns3modulegen_core_customizations.SmartPointerTransformation object at
> 0x7f4b8cb773d0>
> INFO Parsing source file
> "/home/stefano/Tools/ns-3-dev-git/src/core/bindings/scan-header.h" ... 
> INFO gccxml cmd: /usr/bin/gccxml  --gccxml-cxxflags "-m32 -DPYTHON_SCAN"  
> -I"." -I"/home/stefano/Tools/ns-3-dev-git/build"  
> "/home/stefano/Tools/ns-3-dev-git/src/core/bindings/scan-header.h"
> -fxml="/tmp/tmpHP9y0o.xml"
> INFO Parsing source file
> "/home/stefano/Tools/ns-3-dev-git/src/core/bindings/scan-header.h" ... 
> INFO gccxml cmd: /usr/bin/gccxml  --gccxml-cxxflags "-m64 -DPYTHON_SCAN"  
> -I"." -I"/home/stefano/Tools/ns-3-dev-git/build"  
> "/home/stefano/Tools/ns-3-dev-git/src/core/bindings/scan-header.h"
> -fxml="/tmp/tmpJrhP_T.xml"
> Traceback (most recent call last):
> Traceback (most recent call last):
>   File
> "/home/stefano/Tools/ns-3-dev-git/bindings/python/ns3modulescan-modular.py",
> line 280, in <module>
>   File
> "/home/stefano/Tools/ns-3-dev-git/bindings/python/ns3modulescan-modular.py",
> line 280, in <module>
>         ns3_module_scan(sys.argv[1], sys.argv[2], eval(sys.argv[3]),
> sys.argv[4], sys.argv[5])
> ns3_module_scan(sys.argv[1], sys.argv[2], eval(sys.argv[3]), sys.argv[4],
> sys.argv[5])
>   File
> "/home/stefano/Tools/ns-3-dev-git/bindings/python/ns3modulescan-modular.py",
> line 260, in ns3_module_scan
>   File
> "/home/stefano/Tools/ns-3-dev-git/bindings/python/ns3modulescan-modular.py",
> line 260, in ns3_module_scan
>     gccxml_options=gccxml_options)
>   File "/home/stefano/Tools/pybindgen/pybindgen/gccxmlparser.py", line 696,
> in parse_init
>     gccxml_options=gccxml_options)
>   File "/home/stefano/Tools/pybindgen/pybindgen/gccxmlparser.py", line 696,
> in parse_init
>     self.declarations = parser.parse(header_files, self.gccxml_config)
>     self.declarations = parser.parse(header_files, self.gccxml_config)
>   File "/usr/lib/python2.7/site-packages/pygccxml/parser/__init__.py", line
> 50, in parse
>   File "/usr/lib/python2.7/site-packages/pygccxml/parser/__init__.py", line
> 50, in parse
>     answer = parser.read_files(files, compilation_mode)
>       File
> "/usr/lib/python2.7/site-packages/pygccxml/parser/project_reader.py", line
> 225, in read_files
> answer = parser.read_files(files, compilation_mode)
>   File "/usr/lib/python2.7/site-packages/pygccxml/parser/project_reader.py",
> line 225, in read_files
>     return self.__parse_file_by_file(files)
>   File "/usr/lib/python2.7/site-packages/pygccxml/parser/project_reader.py",
> line 250, in __parse_file_by_file
>     return self.__parse_file_by_file(files)
>   File "/usr/lib/python2.7/site-packages/pygccxml/parser/project_reader.py",
> line 250, in __parse_file_by_file
>     decls = reader.read_file( header )
>   File "/usr/lib/python2.7/site-packages/pygccxml/parser/source_reader.py",
> line 197, in read_file
>     decls = reader.read_file( header )
>   File "/usr/lib/python2.7/site-packages/pygccxml/parser/source_reader.py",
> line 197, in read_file
>     return self.read_gccxml_file( source_file )
>     return self.read_gccxml_file( source_file )
>   File "/usr/lib/python2.7/site-packages/pygccxml/parser/source_reader.py",
> line 224, in read_gccxml_file
>   File "/usr/lib/python2.7/site-packages/pygccxml/parser/source_reader.py",
> line 224, in read_gccxml_file
>     raise error
>     raise error
> pygccxml.parser.source_reader.gccxml_runtime_error_t: Error occured while
> running GCC-XML: In file included from
> /home/stefano/Tools/ns-3-dev-git/build/ns3/core-module.h:88,
>                  from
> /home/stefano/Tools/ns-3-dev-git/src/core/bindings/scan-header.h:3:
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:444: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:450: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:456: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:463: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:469: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:475: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:482: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:488: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:494: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:501: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:507: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:513: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:520: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:526: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:532: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:539: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:545: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:551: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:558: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:564: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:570: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:577: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:583: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:589: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:596: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:602: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:608: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:615: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:621: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:627: error:
> expected initializer before '->' token
> pygccxml.parser.source_reader
> .gccxml_runtime_error_t: Error occured while running GCC-XML: In file
> included from /home/stefano/Tools/ns-3-dev-git/build/ns3/core-module.h:88,
>                  from
> /home/stefano/Tools/ns-3-dev-git/src/core/bindings/scan-header.h:3:
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:444: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:450: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:456: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:463: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:469: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:475: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:482: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:488: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:494: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:501: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:507: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:513: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:520: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:526: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:532: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:539: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:545: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:551: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:558: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:564: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:570: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:577: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:583: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:589: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:596: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:602: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:608: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:615: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:621: error:
> expected initializer before '->' token
> /home/stefano/Tools/ns-3-dev-git/build/ns3/traced-value.h:627: error:
> expected initializer before '->' token
> 
> Waf: Leaving directory `/home/stefano/Tools/ns-3-dev-git/build'
> Build failed
>  -> task in '' failed (exit status 1): 
>         {task 140186332951504: apiscan_task core-module.h ->
> modulegen__gcc_ILP32.py}
> ''
>  -> task in '' failed (exit status 1): 
>         {task 140186332952656: apiscan_task core-module.h ->
> modulegen__gcc_LP64.py}
> ''
> 
> ------------------------------
> 
> 
> I verified that python bindings generation works after reverting the above
> mentioned commit ("core: (fixes #2658) Avoid unexpected TracedValue type
> conversions")

We have other C++11 issues elsewhere in rescanning the bindings.  I'd rather not reopen this bug because we have another issue (e.g 2451) to deal with gccxml limitation.  Even if we revert this patch, the recent patch to fix 2668 (and others) cause problems.