Bug 1317 - python programs not working for OS X Lion
python programs not working for OS X Lion
Status: RESOLVED FIXED
Product: ns-3
Classification: Unclassified
Component: python bindings
pre-release
All All
: P1 blocker
Assigned To: Gustavo J. A. M. Carneiro
:
Depends on: 1327
Blocks:
  Show dependency treegraph
 
Reported: 2011-12-18 17:25 UTC by Tom Henderson
Modified: 2012-01-15 12:57 UTC (History)
2 users (show)

See Also:


Attachments
config.log of a bugged Lion (46.18 KB, text/plain)
2011-12-21 11:40 UTC, Tommaso Pecorella
Details
output of "./waf -v" (55.95 KB, application/octet-stream)
2011-12-21 12:52 UTC, Tommaso Pecorella
Details
patch to fix (614 bytes, patch)
2012-01-11 12:59 UTC, Tom Henderson
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Tom Henderson 2011-12-18 17:25:20 UTC
Not sure whether this is local to the ns-buildmaster machine, or more general problem for OS X Lion.  This error is not found on our Snow Leopard (darwin) build machine.

To reproduce on ns-buildmaster:

./waf configure --enable-examples 
./waf build
./waf --pyrun examples/tutorial/first.py

yields
Fatal Python error: PyThreadState_Get: no current thread

alternatively, test.py will show all python examples crashing

python --version:  Python 2.7.1
Comment 1 Tommaso Pecorella 2011-12-18 18:48:44 UTC
I can confirm the behavior in my Lion machine. I'm not a Python fan and my code is pure-C++, so I haven't noticed it before.

Some googling pointed me to this:
- when building python from sources you need to run ./configure --enable-shared
  otherwise you might experience crash with message:
  "Fatal Python error: PyThreadState_Get: no current thread"

Dunno if it's relevant to not.

After the crash, however, my machine wanna send the crash report to Apple, so it might as well be a bug in their Python libraries.

Could also be that .pyc files are compiled with python2.6 instead of the 2.7 ? Self-answer: not really, ns-3 correctly finds python 2.7.1.

Really, don't know anything about python, and the last time I tried to fix a python error I ended up reinstalling the whole OS. If one of you need to test on another Lion machine I can setup an ipv6 address (tunneled) to my machine and an appropriate user, but about finding the bug... meh, sorry, I'm not good at this.

Tommaso
Comment 2 Tom Henderson 2011-12-20 00:54:13 UTC
elevate to P1 blocker
Comment 3 Tom Henderson 2011-12-20 13:45:41 UTC
I was able to narrow this down to changeset ac0569e8cb7d
http://code.nsnam.org/ns-3-dev/rev/ac0569e8cb7d
Comment 4 Tommaso Pecorella 2011-12-20 16:01:51 UTC
(In reply to comment #3)
> I was able to narrow this down to changeset ac0569e8cb7d
> http://code.nsnam.org/ns-3-dev/rev/ac0569e8cb7d

I don't know if it can help, but this tool has been patched recently.

The latest one is here:
http://code.google.com/p/waf/source/browse/waflib/Tools/python.py

I tried to change the one in my directory with this, but it doesn't seems to help a lot.

I know it's definitely a bad idea, but waf 1.6.8 is out, and maybe sooner or later we could upgrade it. I don't know if it would fix the problem tho, and changing it right now with a release pending... it's like upgrading your server distro on Friday at 14:00. You could get stuck in the office during the weekend.

T.
Comment 5 Tom Henderson 2011-12-21 00:48:42 UTC
I was able to narrow it down further to having something to do with the visualizer module (which pulls in a bunch of other modules, but it seems to be the presence of visualizer in the build that raises the error).  

If ns-3 is built with .ns3rc as:

modules_enabled = ['core']
examples_enabled = True
tests_enabled = False

then ./waf --pyrun src/core/examples/sample-simulator.py will run successfully.

If ns-3 is built with .ns3rc as:

modules_enabled = ['core', 'visualizer']
examples_enabled = True
tests_enabled = False

then ./waf --pyrun src/core/examples/sample-simulator.py will fail to import ns.core with the error:  Fatal Python error: PyThreadState_Get: no current thread
Comment 6 Gustavo J. A. M. Carneiro 2011-12-21 05:52:15 UTC
Again, sorry I am not very responsive lately, I have to deliver my phd thesis this year.  Anyway, you guys are doing a great work narrowing down the bug.

I think it is very plausible that the visualizer module causes this sort of error, since it requests the "pyembed" feature to waf, to link to the python library for embedding the python interpreter (visual-simulator-impl.cc needs it).
However, the same module also has Python bindings (we need to access the C++ API provided by pyviz.cc/h).  It may be happening that in OSX Lion sometimes the "library for embedding" and "library for module extension" are different libraries that cannot be used at the same time.  I will try to confirm this.

If confirmed, then best the solution is not very clear:

1- Try to fix the WAF python detection tool somehow, to make the python library for embedding be the shared python library instead of the static one;
2- Split the visualizer module in two modules (ugh), one with visualer-simulator-impl.cc but not python bindingins, one with pyviz.cc and python bindings;
3- Blacklist the visualizer module in this system configuration;
4- ...?
Comment 7 Gustavo J. A. M. Carneiro 2011-12-21 06:28:34 UTC
Hi again, I am not having luck to reproduce the problem on the ns regression host.  I wonder, how is this reproduced exactly?  Can someone with this problem attach the file build/config.log, maybe that will give some clue?


ns-buildmaster:ns-3.13-RC1 nsnam$ ./waf --pyrun examples/tutorial/first.py 
Waf: Entering directory `/Users/nsnam/Downloads/ns-allinone-3.13-RC1/ns-3.13-RC1/build'
Waf: Leaving directory `/Users/nsnam/Downloads/ns-allinone-3.13-RC1/ns-3.13-RC1/build'
'build' finished successfully (1.846s)

Modules built:
aodv                      applications              bridge                   
config-store              core                      csma                     
csma-layout               dsdv                      energy                   
flow-monitor              internet                  lte                      
mesh                      mobility                  mpi                      
netanim                   network                   nix-vector-routing       
olsr                      point-to-point            point-to-point-layout    
propagation               spectrum                  stats                    
test                      tools                     topology-read            
uan                       virtual-net-device        visualizer               
wifi                      wimax                    

Modules not built:
click                     emu                       openflow                 
tap-bridge               

Sent 1024 bytes to 10.1.1.2
Received 1024 bytes from 10.1.1.1
Received 1024 bytes from 10.1.1.2
ns-buildmaster:ns-3.13-RC1 nsnam$
Comment 8 Gustavo J. A. M. Carneiro 2011-12-21 07:11:37 UTC
Confirmation why everything works in ns-buildmaster: the linked-to python library is a dynamic library:



[1210/1309] cxxshlib: build/src/visualizer/model/dummy-file-for-static-builds.cc.2.o build/src/visualizer/model/pyviz.cc.2.o build
/src/visualizer/model/visual-simulator-impl.cc.2.o -> build/libns3-visualizer.dylib
03:12:30 runner ['/usr/bin/g++', 'src/visualizer/model/dummy-file-for-static-builds.cc.2.o', 'src/visualizer/model/pyviz.cc.2.o', 'src/visualizer/model/visual-simulator-impl.cc.2.o', '-o', '/Users/nsnam/Downloads/ns-allinone-3.13-RC1/ns-3.13-RC1/build/libns3-visualizer.dylib', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L/System/Library/Frameworks/Python.framework/Versions/2.7/lib', '-lns3-internet', '-lns3-bridge', '-lns3-point-to-point', '-lns3-mpi', '-lns3-wifi', '-lns3-propagation', '-lns3-mobility', '-lns3-network', '-lns3-core', '-lpython2.7', '-dynamiclib', '-Wl,-F.']

and:

[1246/1309] cxxshlib: build/src/visualizer/bindings/ns3module.cc.5.o -> build/bindings/python/ns/visualizer.so
03:12:33 runner ['/usr/bin/g++', 'src/visualizer/bindings/ns3module.cc.5.o', '-o', '/Users/nsnam/Downloads/ns-allinone-3.13-RC1/ns
-3.13-RC1/build/bindings/python/ns/visualizer.so', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-
L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.',
 '-L.', '-L.', '-L/System/Library/Frameworks/Python.framework/Versions/2.7/lib', '-lns3-aodv', '-lns3-dsdv', '-lns3-mesh', '-lns3-
test', '-lns3-csma-layout', '-lns3-point-to-point-layout', '-lns3-netanim', '-lns3-olsr', '-lns3-csma', '-lns3-lte', '-lns3-wimax'
, '-lns3-spectrum', '-lns3-applications', '-lns3-virtual-net-device', '-lns3-uan', '-lns3-energy', '-lns3-flow-monitor', '-lns3-ni
x-vector-routing', '-lns3-visualizer', '-lns3-internet', '-lns3-bridge', '-lns3-point-to-point', '-lns3-mpi', '-lns3-wifi', '-lns3
-propagation', '-lns3-mobility', '-lns3-config-store', '-lns3-tools', '-lns3-stats', '-lns3-topology-read', '-lns3-network', '-lns
3-core', '-lsqlite3', '-lpython2.7', '-Wl,-F.', '-Wl,-F.', '-Wl,-F.', '-bundle', '-undefined', 'dynamic_lookup', '-Wl,-F.']

and:

ns-buildmaster:ns-3.13-RC1 nsnam$ ls /System/Library/Frameworks/Python.framework/Versions/2.7/lib
libpython2.7.dylib	pkgconfig		python2.7
Comment 9 Tommaso Pecorella 2011-12-21 11:40:38 UTC
Created attachment 1287 [details]
config.log of a bugged Lion

17:38:30:~/Development/workspace/ns-3-allinone/ns-3-dev pecos$ ./waf --pyrun examples/tutorial/first.py
Waf: Entering directory `/Users/pecos/Development/workspace/ns-3-allinone/ns-3-dev/build'
Waf: Leaving directory `/Users/pecos/Development/workspace/ns-3-allinone/ns-3-dev/build'
'build' finished successfully (2.353s)

Modules built:
aodv                      applications              bridge                   
config-store              core                      csma                     
csma-layout               dsdv                      energy                   
flow-monitor              internet                  lte                      
mesh                      mobility                  mpi                      
netanim                   network                   nix-vector-routing       
olsr                      point-to-point            point-to-point-layout    
propagation               spectrum                  stats                    
test                      tools                     topology-read            
uan                       virtual-net-device        visualizer               
wifi                      wimax                    

Modules not built:
click                     emu                       openflow                 
tap-bridge               

Fatal Python error: PyThreadState_Get: no current thread
Command ['/usr/bin/python', 'examples/tutorial/first.py'] terminated with signal SIGIOT. Run it under a debugger to get more information (./waf --run <program> --command-template="gdb --args %s <args>").
Comment 10 Gustavo J. A. M. Carneiro 2011-12-21 11:44:46 UTC
It's the same /System/Library/Frameworks/Python.framework/Versions/2.7/lib library path.  What is the contents of this directory?  To see if the library is shared or static...
Comment 11 Tom Henderson 2011-12-21 12:06:35 UTC
(In reply to comment #7)
> Hi again, I am not having luck to reproduce the problem on the ns regression
> host.  I wonder, how is this reproduced exactly?  Can someone with this problem
> attach the file build/config.log, maybe that will give some clue?
> 
>

Hmm, I just downloaded a fresh copy from the server, as user nsnam on the ns-buildmaster machine, and was able to reproduce:

Fatal Python error: PyThreadState_Get: no current thread
Command ['/usr/bin/python', 'examples/tutorial/first.py'] terminated with signal SIGIOT. Run it under a debugger to get more information (./waf --run <program> --command-template="gdb --args %s <args>").
ns-buildmaster:ns-3.13-RC1 nsnam$ pwd
/Users/nsnam/temp/ns-allinone-3.13-RC1/ns-3.13-RC1

Might it be do to the way it was built?
./build.py --enable-examples --enable-tests
cd ns-3.13-RC1
./waf --pyrun examples/tutorial/first.py
Comment 12 Tommaso Pecorella 2011-12-21 12:19:27 UTC
*** I FOUND IT ***

The problem is related to the python installed by macports.

There is a libpython2.7.dylib in /opt/local/lib, and it's version 2.7.2, while
the MacOS framework one is 2.7.1

I still don't know how to fix it for good, but manually removing that library
in /opt/local/lib and reconfiguring/rebuilding all works.

Solutions can be:
1) have was find the Python installed in /opt/local/lib, or
2) have waf IGNORE the one installed there.

In any case it can't do mixed things.

Waf configure say:
Checking for python version                        : (2, 7, 1, 'final', 0) 
Checking for library python2.7 in LIBDIR           : yes 
Checking for program python2.7-config,python-config-2.7,python2.7m-config :
/opt/local/bin/python2.7-config 
Checking for header Python.h                                              : yes 

but...


[1372/1542] cxxshlib:
build/src/visualizer/model/dummy-file-for-static-builds.cc.2.o
build/src/visualizer/model/pyviz.cc.2.o
build/src/visualizer/model/visual-simulator-impl.cc.2.o ->
build/libns3-visualizer.dylib
17:47:49 runner ['/usr/bin/g++',
'src/visualizer/model/dummy-file-for-static-builds.cc.2.o',
'src/visualizer/model/pyviz.cc.2.o',
'src/visualizer/model/visual-simulator-impl.cc.2.o', '-o',
'/Users/pecos/Development/workspace/ns-3-allinone/ns-3-dev/build/libns3-visualizer.dylib',
'-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.',
'-L/opt/local/lib',
'-L/System/Library/Frameworks/Python.framework/Versions/2.7/lib',
'-lns3-internet', '-lns3-bridge', '-lns3-point-to-point', '-lns3-mpi',
'-lns3-wifi', '-lns3-propagation', '-lns3-mobility', '-lns3-network',
'-lns3-core', '-lgsl', '-lgslcblas', '-lm', '-lpython2.7', '-dynamiclib',
'-Wl,-F.']

"/opt/local/lib" is preferred path over
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib".

BOOM.

Said that, I have no clue about how to fix it, as removing macport's python is
kinda a harsh solution, as it's used by a lot of other extremely useful
macports modules.

Cheers,

Tommaso
Comment 13 Gustavo J. A. M. Carneiro 2011-12-21 12:23:23 UTC
(In reply to comment #12)
[...]
> "/opt/local/lib" is preferred path over
> "/System/Library/Frameworks/Python.framework/Versions/2.7/lib".
> 
> BOOM.
> 
> Said that, I have no clue about how to fix it, as removing macport's python is
> kinda a harsh solution, as it's used by a lot of other extremely useful
> macports modules.
> 

Well, if in OSX the /opt/local/lib takes precedence over the command-line -L path, then I really do not see what can be done.  I can add code to try to detect this problem, but other than that... *shrugs*
Comment 14 Tom Henderson 2011-12-21 12:30:54 UTC
(In reply to comment #13)
> (In reply to comment #12)
> [...]
> > "/opt/local/lib" is preferred path over
> > "/System/Library/Frameworks/Python.framework/Versions/2.7/lib".
> > 
> > BOOM.
> > 
> > Said that, I have no clue about how to fix it, as removing macport's python is
> > kinda a harsh solution, as it's used by a lot of other extremely useful
> > macports modules.
> > 
> 
> Well, if in OSX the /opt/local/lib takes precedence over the command-line -L
> path, then I really do not see what can be done.  I can add code to try to
> detect this problem, but other than that... *shrugs*

This explanation is consistent with the following logs.

This is a working version, before changeset 7545

09:17:33 runner ['/usr/bin/g++', 'src/core/bindings/ns3module.cc.8.o', 'src/core/bindings/module_helpers.cc.8.o', '-o', '/Users/tomh/ns-3-allinone/ns-3-post/build/bindings/python/ns/_core.so', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L/System/Library/Frameworks/Python.framework/Versions/2.7/lib', '-L/opt/local/lib', '-L/System/Library/Frameworks/Python.framework/Versions/2.7/lib', '-lns3-visualizer', '-lns3-internet', '-lns3-bridge', '-lns3-point-to-point', '-lns3-mpi', '-lns3-wifi', '-lns3-propagation', '-lns3-mobility', '-lns3-network', '-lns3-core', '-lpython2.7', '-lgsl', '-lgslcblas', '-lm', '-lpython2.7', '-dynamiclib', '-Wl,-F.', '-Wl,-F.', '-Wl,-F.', '-Wl,-F.', '-Wl,-F.', '-Wl,-F.']

This is as of changeset 7545, which breaks it presumably due to /opt/local/lib taking precedence:

09:10:02 runner ['/usr/bin/g++', 'src/core/bindings/ns3module.cc.8.o', 'src/core/bindings/module_helpers.cc.8.o', '-o', '/Users/tomh/ns-3-allinone/ns-3-post/build/bindings/python/ns/_core.so', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L.', '-L/opt/local/lib', '-L/System/Library/Frameworks/Python.framework/Versions/2.7/lib', '-lns3-visualizer', '-lns3-internet', '-lns3-bridge', '-lns3-point-to-point', '-lns3-mpi', '-lns3-wifi', '-lns3-propagation', '-lns3-mobility', '-lns3-network', '-lns3-core', '-lgsl', '-lgslcblas', '-lm', '-lpython2.7', '-Wl,-F.', '-Wl,-F.', '-Wl,-F.', '-bundle', '-undefined', 'dynamic_lookup', '-Wl,-F.']


is there some rearranging of the linkflags due to changeset 7545 that could be tweaked to make the OS X frameworks version be picked up here?
Comment 15 Tommaso Pecorella 2011-12-21 12:52:58 UTC
Created attachment 1289 [details]
output of "./waf -v"
Comment 16 Tommaso Pecorella 2011-12-21 13:06:20 UTC
(In reply to comment #13)
> (In reply to comment #12)
> [...]
> > "/opt/local/lib" is preferred path over
> > "/System/Library/Frameworks/Python.framework/Versions/2.7/lib".
> > 
> > BOOM.
> > 
> > Said that, I have no clue about how to fix it, as removing macport's python is
> > kinda a harsh solution, as it's used by a lot of other extremely useful
> > macports modules.
> > 
> 
> Well, if in OSX the /opt/local/lib takes precedence over the command-line -L
> path, then I really do not see what can be done.  I can add code to try to
> detect this problem, but other than that... *shrugs*

/opt/local/lib is added by LIBXML2 for what I can understand. When the configure is working, it's adding -L according to LIBXML2 needs. And that is done before checking for Python.
GSL is in /opt/local/lib as well.

Basically, -L /opt/local/lib is needed, but it should be after the "normal" python path. Best guessing tho, I don't know how to force it after the configure to prove it.

T.
Comment 17 Gustavo J. A. M. Carneiro 2011-12-21 13:07:16 UTC
     -Ldir       Add dir to the list of directories in which to search for libraries.  Directories specified with -L are searched in the order they appear on the command line and before the default search path. In Xcode4 and later, there can be a space between the -L and directory.

So, it really appears the order of -L options is wrong.  It probably happens that ns-3 is linking to some other library in /opt/local/lib (gsl, looks like), so we can't remove the other -L.  To be honest, I feel like changing the order of the -L options is a hack, the user shouldn't have the same python version library in multiple places, it just asks for trouble...
Comment 18 Tommaso Pecorella 2011-12-21 13:34:04 UTC
(In reply to comment #17)
>      -Ldir       Add dir to the list of directories in which to search for
> libraries.  Directories specified with -L are searched in the order they appear
> on the command line and before the default search path. In Xcode4 and later,
> there can be a space between the -L and directory.
> 
> So, it really appears the order of -L options is wrong.  It probably happens
> that ns-3 is linking to some other library in /opt/local/lib (gsl, looks like),
> so we can't remove the other -L.  To be honest, I feel like changing the order
> of the -L options is a hack, the user shouldn't have the same python version
> library in multiple places, it just asks for trouble...

I do totally agree, it's asking for troubles.

Still, MacPorts is installing python and there's nothing we can do to avoid it. beside trashing it and installing all those nice ports manually... and that would be a real pain in the [].

Hence, I'd go for the hack :)

T.
Comment 19 Tom Henderson 2011-12-21 13:35:14 UTC
(In reply to comment #18)
> (In reply to comment #17)
> >      -Ldir       Add dir to the list of directories in which to search for
> > libraries.  Directories specified with -L are searched in the order they appear
> > on the command line and before the default search path. In Xcode4 and later,
> > there can be a space between the -L and directory.
> > 
> > So, it really appears the order of -L options is wrong.  It probably happens
> > that ns-3 is linking to some other library in /opt/local/lib (gsl, looks like),
> > so we can't remove the other -L.  To be honest, I feel like changing the order
> > of the -L options is a hack, the user shouldn't have the same python version
> > library in multiple places, it just asks for trouble...
> 
> I do totally agree, it's asking for troubles.
> 
> Still, MacPorts is installing python and there's nothing we can do to avoid it.
> beside trashing it and installing all those nice ports manually... and that
> would be a real pain in the [].
> 
> Hence, I'd go for the hack :)
> 
> T.

(aside) ns-2 has had this very problem with multiple Tcl/Tk versions, leading to a mess of autoconf-related files to try to deal with it.

Since Python.org recommends to install a different version of Python on the mac (http://www.python.org/getit/mac/), this will likely occur in the wild fairly frequently.  If we choose not to fix this directly by hacking the search order, I think we should at least provide guidance or configuration option on how to resolve this if it occurs.  

What about a "--with-python=/path/to/python" option in configure when checking for program python?  It seems like these alternate versions of python take care not to name themselves 'python' so as to not take precedence over the framework version, but --with-python=/opt/local/bin/python2.7 might align the python program with the library found in /opt/local/lib.

(I tested this by editing build/c4che/_cache.py PYTHON, PYTHONARCHDIR, and PYTHONDIR, and it seemed to work).
Comment 20 Tom Henderson 2011-12-21 13:40:20 UTC
(In reply to comment #19)
> (In reply to comment #18)
> > (In reply to comment #17)
> > >      -Ldir       Add dir to the list of directories in which to search for
> > > libraries.  Directories specified with -L are searched in the order they appear
> > > on the command line and before the default search path. In Xcode4 and later,
> > > there can be a space between the -L and directory.
> > > 
> > > So, it really appears the order of -L options is wrong.  It probably happens
> > > that ns-3 is linking to some other library in /opt/local/lib (gsl, looks like),
> > > so we can't remove the other -L.  To be honest, I feel like changing the order
> > > of the -L options is a hack, the user shouldn't have the same python version
> > > library in multiple places, it just asks for trouble...
> > 
> > I do totally agree, it's asking for troubles.
> > 
> > Still, MacPorts is installing python and there's nothing we can do to avoid it.
> > beside trashing it and installing all those nice ports manually... and that
> > would be a real pain in the [].
> > 
> > Hence, I'd go for the hack :)
> > 
> > T.
> 
> (aside) ns-2 has had this very problem with multiple Tcl/Tk versions, leading
> to a mess of autoconf-related files to try to deal with it.
> 
> Since Python.org recommends to install a different version of Python on the mac
> (http://www.python.org/getit/mac/), this will likely occur in the wild fairly
> frequently.  If we choose not to fix this directly by hacking the search order,
> I think we should at least provide guidance or configuration option on how to
> resolve this if it occurs.  
> 
> What about a "--with-python=/path/to/python" option in configure when checking
> for program python?  It seems like these alternate versions of python take care
> not to name themselves 'python' so as to not take precedence over the framework
> version, but --with-python=/opt/local/bin/python2.7 might align the python
> program with the library found in /opt/local/lib.
> 
> (I tested this by editing build/c4che/_cache.py PYTHON, PYTHONARCHDIR, and
> PYTHONDIR, and it seemed to work).

erm, looks like we already have that option --with-python.  but it doesn't touch PYTHONARCHDIR or PYTHONDIR.

However, it seems to work for me.  Problem solved?
Comment 21 Tommaso Pecorella 2011-12-21 13:48:36 UTC
(In reply to comment #20)
> (In reply to comment #19)
> > (In reply to comment #18)
> > > (In reply to comment #17)
> > > >      -Ldir       Add dir to the list of directories in which to search for
> > > > libraries.  Directories specified with -L are searched in the order they appear
> > > > on the command line and before the default search path. In Xcode4 and later,
> > > > there can be a space between the -L and directory.
> > > > 
> > > > So, it really appears the order of -L options is wrong.  It probably happens
> > > > that ns-3 is linking to some other library in /opt/local/lib (gsl, looks like),
> > > > so we can't remove the other -L.  To be honest, I feel like changing the order
> > > > of the -L options is a hack, the user shouldn't have the same python version
> > > > library in multiple places, it just asks for trouble...
> > > 
> > > I do totally agree, it's asking for troubles.
> > > 
> > > Still, MacPorts is installing python and there's nothing we can do to avoid it.
> > > beside trashing it and installing all those nice ports manually... and that
> > > would be a real pain in the [].
> > > 
> > > Hence, I'd go for the hack :)
> > > 
> > > T.
> > 
> > (aside) ns-2 has had this very problem with multiple Tcl/Tk versions, leading
> > to a mess of autoconf-related files to try to deal with it.
> > 
> > Since Python.org recommends to install a different version of Python on the mac
> > (http://www.python.org/getit/mac/), this will likely occur in the wild fairly
> > frequently.  If we choose not to fix this directly by hacking the search order,
> > I think we should at least provide guidance or configuration option on how to
> > resolve this if it occurs.  
> > 
> > What about a "--with-python=/path/to/python" option in configure when checking
> > for program python?  It seems like these alternate versions of python take care
> > not to name themselves 'python' so as to not take precedence over the framework
> > version, but --with-python=/opt/local/bin/python2.7 might align the python
> > program with the library found in /opt/local/lib.
> > 
> > (I tested this by editing build/c4che/_cache.py PYTHON, PYTHONARCHDIR, and
> > PYTHONDIR, and it seemed to work).
> 
> erm, looks like we already have that option --with-python.  but it doesn't
> touch PYTHONARCHDIR or PYTHONDIR.
> 
> However, it seems to work for me.  Problem solved?

./waf --enable-examples --with-python=/opt/local/bin/python2.7  configure

works.

+1

To add it programmatically to the test, it's "enough" to ask a bit more nicely
what's the python version.

If (there is a damn libpython2.7.dylib in /opt/local/lib) 
then
  look for python in /opt/local/bin/python2.7
else
  do the normal things.

Possible ?

T.
Comment 22 Tom Henderson 2011-12-21 13:57:42 UTC
I updated the wiki about this in any case:
https://www.nsnam.org/wiki/index.php/NS-3_Python_Bindings#Troubleshooting
Comment 23 Tom Henderson 2011-12-22 00:16:12 UTC
marking resolved (documentation fix).
Comment 24 Tom Henderson 2012-01-10 00:28:03 UTC
despite this working for such OS X systems:

./waf configure --with-python=/opt/local/bin/python2.7 ...

The python programs CRASH under test.py, even when running test.py such as 
/opt/local/bin/python2.7 test.py

so reopening to get a fix for test.py.
Comment 25 Tom Henderson 2012-01-11 00:36:56 UTC
(In reply to comment #24)
> despite this working for such OS X systems:
> 
> ./waf configure --with-python=/opt/local/bin/python2.7 ...
> 
> The python programs CRASH under test.py, even when running test.py such as 
> /opt/local/bin/python2.7 test.py
> 
> so reopening to get a fix for test.py.

This hack seems to resolve it for me:

--- a/test.py	Mon Jan 09 18:48:14 2012 +0100
+++ b/test.py	Tue Jan 10 21:33:08 2012 -0800
@@ -666,7 +666,7 @@
     suppressions_path = os.path.join (base, VALGRIND_SUPPRESSIONS_FILE)
 
     if is_python:
-        path_cmd = "python " + os.path.join (base, shell_command)
+        path_cmd = "/opt/local/bin/python2.7 " + os.path.join (base, shell_command)
     else:
         if len(build_path):
             path_cmd = os.path.join (build_path, shell_command)


probably the right solution for test.py is to find _cache.py and use the python specified in the PYTHON variable.  There is already some similar code in read_waf_config().
Comment 26 Tom Henderson 2012-01-11 12:59:47 UTC
Created attachment 1304 [details]
patch to fix

Attached patch should fix this once the problems introduced by the patch to bug 1317 are resolved.
Comment 27 Tom Henderson 2012-01-11 13:00:15 UTC
(In reply to comment #26)

Attached patch should fix this once the problems introduced by the patch to bug
1327 are resolved.
Comment 28 Tom Henderson 2012-01-15 12:57:50 UTC
latest fix: 9918e57b9f8d