Bug 2526 - The version of pybindgen that is downloaded with the Dev build is not compatible with the NS-3 build environment.
The version of pybindgen that is downloaded with the Dev build is not compati...
Status: RESOLVED FIXED
Product: ns-3
Classification: Unclassified
Component: python bindings
ns-3-dev
PC Linux
: P5 blocker
Assigned To: Gustavo J. A. M. Carneiro
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2016-10-19 15:34 UTC by Robert Ammon
Modified: 2016-10-19 23:20 UTC (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Robert Ammon 2016-10-19 15:34:11 UTC
When setting up a new environment from the development release, the version of pybindgen that is downloaded using ./download.py from ns-3-allinone downloads and builds a version of pybindgen that is not compatible with the build script.

The following is a partial output of the execution of ./build.py that shows the version number incompatibility of pybindgen:


robertammon@rammon-Inspiron-5537:~/NS3/ns-3-allinone$ ./build.py
.
.
.
# Build NS-3
Entering directory `./ns-3-dev'
 =>  /usr/bin/python waf configure --with-pybindgen ../pybindgen
Setting top to                           : /home/robertammon/NS3/ns-3-allinone/ns-3-dev 
Setting out to                           : /home/robertammon/NS3/ns-3-allinone/ns-3-dev/build 
Checking for 'gcc' (C compiler)          : /usr/bin/gcc 
Checking for cc version                  : 5.4.0 
Checking for 'g++' (C++ compiler)        : /usr/bin/g++ 
.
.
.
Checking for pybindgen location                                    : ../pybindgen (given) 
Checking for python module 'pybindgen'                             : 0.17.0.post57+nga6376f2.d20161017 
Checking for pybindgen version                                     : 0.17.0.post57+nga6376f2.d20161017 
pybindgen (found '0.17.0.post57+nga6376f2.d20161017'), (need '0.17.0.post57+nga6376f2')
.
.
.
'configure' finished successfully (2.251s)
 =>  /usr/bin/python waf build
Waf: Entering directory `/home/robertammon/NS3/ns-3-allinone/ns-3-dev/build'
Waf: Leaving directory `/home/robertammon/NS3/ns-3-allinone/ns-3-dev/build'
Build commands will be stored in build/compile_commands.json
'build' finished successfully (0.749s)
Comment 1 Tom Henderson 2016-10-19 16:20:50 UTC
The problem here is that, in pybindgen, the command:

python setup.py clean

generates a pybindgen/version.py file with the string:

# coding: utf-8
# file generated by setuptools_scm
# don't change, don't track in version control
version = '0.17.0.post57+nga6376f2.d20161019'

The problem is the date suffix .d20161019.  I don't know where that is coming from; it didn't used to be appended.  I can reproduce it today, but this seems to be newly occurring.

We could improve the ns-3 wscript to detect and ignore this, or try to understand why version.py now gets this suffix.

The workaround for the time being is to manually edit version.py and remove the .d20161019 date suffix.
Comment 2 Robert Ammon 2016-10-19 17:26:34 UTC
I have edited pybindgen/version.py to change the version string to:

# coding: utf-8
# file generated by setuptools_scm
# don't change, don't track in version control
version = '0.17.0.post57+nga6376f2'

However, this doesn't solve the issue, ./build.py (and ./waf) still report the version of the application as:

Checking for pybindgen location                                    : ../pybindgen (given) 
Checking for python module 'pybindgen'                             : 0.17.0.post57+nga6376f2.d20161017 
Checking for pybindgen version                                     : 0.17.0.post57+nga6376f2.d20161017 
pybindgen (found '0.17.0.post57+nga6376f2.d20161017'), (need '0.17.0.post57+nga6376f2')

Do I need to rebuild something after changing version.py?
Comment 3 Gustavo J. A. M. Carneiro 2016-10-19 18:01:34 UTC
There's a this in pybindgen's setup.py:
      use_scm_version={"version_scheme": "post-release",
                       "write_to": "pybindgen/version.py"},
      setup_requires=['setuptools_scm'],

This will download the python package setuptools_scm, if not available already.  It causes the file pybindgen/version.py to be generated every time setup.py runs.

I think I managed to figure it out.  It seems that setuptools_scm generates a version string with dYYYMMMDD at the end whenever it finds the git repo "not clean".  As it turns out, simply running setup.py would cause the required setuptools_scm package to be downloaded into a .eggs directory at the project root, and in turn this .eggs directory would make the git repo "not clean", thereby causing the date to be added to the version.

The solution I found then was to add .eggs/ to .gitignore, and now the version string is generated without the date.

For now, can someone please make ns-3 update to pybindgen 0.17.0.post58+ngcf00cc0 ? This version has the correct .gitignore and should generate the version correctly, as long as you don't have untracked files lying around.  If in doubt, check the output of `git status`, if it reports untracked files then report them to me and I'll add to .gitignore.
Comment 4 Tom Henderson 2016-10-19 19:21:17 UTC
Gustavo, it works for me, so I pushed the update in changeset 12378:02935defe772.  Thanks!

Robert, can you confirm that rerunning ./download.py clears your problem?  If so, we can close this bug.
Comment 5 Robert Ammon 2016-10-19 21:35:55 UTC
The following is the results that I got from doing a brand new install:

Checking for pybindgen location                                    : ../pybindgen (given) 
Checking for python module 'pybindgen'                             : 0.17.0.post57+nga6376f2 
Checking for pybindgen version                                     : 0.17.0.post57+nga6376f2 
pybindgen (found '0.17.0.post57+nga6376f2'), (need '0.17.0.post58+ngcf00cc0')

The issue appears to have been caused by a previous version being created while sudo.  Not sure if that was something I did earlier

robertammon@rammon-Inspiron-5537:~/NS3/ns-3-allinone/pybindgen$ python setup.py install
running install
error: can't create or remove files in install directory

The following error occurred while trying to add or remove files in the
installation directory:

    [Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/test-easy-install-3420.write-test'

The installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:

    /usr/local/lib/python2.7/dist-packages/

Perhaps your account does not have write access to this directory?  If the
installation directory is a system-owned directory, you may need to sign in
as the administrator or "root" account.  If you do not have administrative
access to this machine, you may wish to choose a different installation
directory, preferably one that is listed in your PYTHONPATH environment
variable.

For information on other options, you may wish to consult the
documentation at:

  https://pythonhosted.org/setuptools/easy_install.html

Please make the appropriate changes for your system and try again.

robertammon@rammon-Inspiron-5537:~/NS3/ns-3-allinone/pybindgen$ sudo python setup.py install
running install
.
.
.
Processing PyBindGen-0.17.0.post58+ngcf00cc0-py2.7.egg
Copying PyBindGen-0.17.0.post58+ngcf00cc0-py2.7.egg to /usr/local/lib/python2.7/dist-packages
Removing PyBindGen 0.17.0.post57+nga6376f2.d20161019 from easy-install.pth file
Adding PyBindGen 0.17.0.post58+ngcf00cc0 to easy-install.pth file

Installed /usr/local/lib/python2.7/dist-packages/PyBindGen-0.17.0.post58+ngcf00cc0-py2.7.egg
Processing dependencies for PyBindGen==0.17.0.post58+ngcf00cc0
Finished processing dependencies for PyBindGen==0.17.0.post58+ngcf00cc0

Rebuilding while sudo did resolve the issue so you should consider the issue closed for general application.