Bug 1657

(In reply to comment #0)
> Here is the current bakeconf.xml for pybindgen; pybindgen-dev is being used for
> ns-3-allinone; pybindgen-0.16.0.825 is being used for ns-3.17:
> 
>     <module name="pybindgen-dev">
>       <source type="bazaar">
>         <attribute name="url" value="https://launchpad.net/pybindgen"/>
> <!--    <attribute name="revision" value="revno:825"/>
> -->
>       </source>
>       <build type="waf" objdir="yes">
>         <attribute name="configure_arguments" value="configure
> --prefix=$INSTALLDIR"/>
>         <attribute name="build_arguments" value="--generate-version"/>
>       </build>
>       <depends_on name="python-dev" optional="True"/>
>       <depends_on name="pygraphviz" optional="True"/>
>       <depends_on name="pygoocanvas" optional="True"/>
>       <depends_on name="pygccxml" optional="True"/>
>       <depends_on name="gccxml-ns3" optional="False"/>
>     </module>
> 
>     <module name="pybindgen-0.16.0.825">
>       <source type="bazaar">
>         <attribute name="url" value="https://launchpad.net/pybindgen"/>
>         <attribute name="revision" value="revno:825"/>
>       </source>
>       <depends_on name="python-dev" optional="True"/>
>       <depends_on name="pygraphviz" optional="True"/>
>       <depends_on name="pygoocanvas" optional="True"/>
>       <build type="waf" objdir="yes">
>         <attribute name="configure_arguments" value="configure
> --prefix=$INSTALLDIR"/>
>         <attribute name="build_arguments" value="--generate-version"/>
>       </build>
>     </module>
> 
> 
> I have some questions about these choices.
> 
> 1) under what conditions do we need to actually build pybindgen in advance?  I
> noticed that the old build.py script in ns-3-allinone wasn't building pybindgen

You do not build pybindgen.  But you may need to generate a version.py file, the same way that the download.py script was doing.

> 
> 2) are these correct dependencies?  I don't see all of these in the README for
> pybindgen.  Is python-dev optional?  Are pygraphviz and pygoocanvas needed? 
> Why is gcc-xml required by pygccxml optional?

Well, python-dev is needed if you want to actually build the extension modules that pybindgen generates.  I would say it is pointless to install pybindgen if python-dev is not installed.

gccxml is a runtime dependency of pygccxml but, again, it is pointless to install pygccxml without gccxml.

pygoocanvas and pygraphviz are dependencies of the pyviz visualizer; they have nothing to do with pybindgen.

Let me see if I understand clearly; can you please confirm?

To use existing bindings, we need to download pybindgen and either
1) generate fake version.py as download.py script does
2) build pybindgen

The latter requires python-dev, which is why the former is done by download.py (to lessen the system requirement from python-dev to python).

To generate bindings, we also do not need to build pybindgen, and we still do not need python-dev, but we need to build and install gccxml and pygccxml.

If gccxml and pygccxml are not installed to the system, and are instead installed to bake's build/ directory, then gccxml needs to be in PATH and pygccxml needs to be in PYTHONPATH for ns-3's waf to find them.

upgrading to blocker; this needs to be straightened out before release

(In reply to comment #2)
> Let me see if I understand clearly; can you please confirm?
> 
> To use existing bindings, we need to download pybindgen and either
> 1) generate fake version.py as download.py script does
> 2) build pybindgen
> 
> The latter requires python-dev, which is why the former is done by download.py
> (to lessen the system requirement from python-dev to python).

1 is correct.

2 is not so simple.  pybindgen official releases come with a version.py already pre-generated.  In the official release, there is nothing important to "build" except examples and run unit tests (for that it uses WAF).  

However, we are using pybindgen from bazaar, which has no version.py.  To generate a version.py file in pybindgen, you would do ./waf --generate-version, which inspects the bazaar commit history and tags to derive what to put in version.py.  But this functionality requires bazaar python bindings, which not everyone has, and that's why I made download.py generate a fake version.py.

To summarise, "building pybindgen" does not require python-dev at all, or pygccxml, it just needs python.  But building the python modules generated by pybindgen does need python-dev , obviously (but still not pygccxml).

> 
> To generate bindings, we also do not need to build pybindgen, and we still do
> not need python-dev, but we need to build and install gccxml and pygccxml.

Correct.

> 
> If gccxml and pygccxml are not installed to the system, and are instead
> installed to bake's build/ directory, then gccxml needs to be in PATH and
> pygccxml needs to be in PYTHONPATH for ns-3's waf to find them.

Correct.

(In reply to comment #0)
> Here is the current bakeconf.xml for pybindgen; pybindgen-dev is being used for
> ns-3-allinone; pybindgen-0.16.0.825 is being used for ns-3.17:
(snip)
>       <depends_on name="python-dev" optional="True"/>
>       <depends_on name="pygraphviz" optional="True"/>
>       <depends_on name="pygoocanvas" optional="True"/>

these 3 lines were added by me.
I intended to build pyviz with these bakeconf.xml.

So, these dependencies might be moved to ns-3-dev (ns-3.17) as a optional dependencies to build pyviz if available ?

(In reply to comment #5)
> (In reply to comment #0)
> > Here is the current bakeconf.xml for pybindgen; pybindgen-dev is being used for
> > ns-3-allinone; pybindgen-0.16.0.825 is being used for ns-3.17:
> (snip)
> >       <depends_on name="python-dev" optional="True"/>
> >       <depends_on name="pygraphviz" optional="True"/>
> >       <depends_on name="pygoocanvas" optional="True"/>
> 
> these 3 lines were added by me.
> I intended to build pyviz with these bakeconf.xml.
> 
> So, these dependencies might be moved to ns-3-dev (ns-3.17) as a optional
> dependencies to build pyviz if available ?

Another alternative might be to create a new module called "pyviz-prerequisites" and make ns-3 optionally depend on that.  

It seems to me that for ns-3.17, the critical issue is to make sure that pybindgen gets the version.py needed, and having bake support installation of packages, for which we have relied on package management tools in the past, can be deferred.

> 
> To summarise, "building pybindgen" does not require python-dev at all, or
> pygccxml, it just needs python.  But building the python modules generated by
> pybindgen does need python-dev , obviously (but still not pygccxml).

I was able to remove python-dev from my system and have the python modules build, which is why I was questioning the requirement yesterday, but I just tried to do this on another fresh buildslave and it doesn't work.

Anyway, it seems like the things needed now are:

- change build type of pybindgen to 'none' (or 'inline'-- see below)
- remove all bake prerequisites on it
- support the generation of version.py (the 'patch' feature could be used for a static version.py, or perhaps the 'inline' feature for dynamically generating version.py, as in the NSC module)

(In reply to comment #7)

> - remove all bake prerequisites on it

I've fixed with the above part.

bake: changeset 4dc76e5f0d

The pybindgen dependencies where reorganized following the orientations of Gustavo and the waf --generate-version is used to generate the version.py file, also as suggested by Gustavo. 

changeset 250