Bug 2734 - visualizer calls PySys_SetArgv(int, wchar_t**) with char**
visualizer calls PySys_SetArgv(int, wchar_t**) with char**
Status: PATCH PENDING
Product: ns-3
Classification: Unclassified
Component: visualizer
ns-3-dev
All All
: P3 minor
Assigned To: Gustavo J. A. M. Carneiro
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2017-05-03 19:04 EDT by Tommaso Pecorella
Modified: 2019-04-15 17:45 EDT (History)
4 users (show)

See Also:


Attachments
Patch to fix python 3 compatibility in visualizer (6.64 KB, patch)
2018-11-13 04:31 EST, Mikołaj Chwalisz
Details | Diff
Extended patch (13.63 KB, patch)
2019-04-15 13:22 EDT, Alex Afanasyev
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Tommaso Pecorella 2017-05-03 19:04:31 EDT
The problem has been reported in this thread:
https://groups.google.com/forum/#!topic/ns-3-users/50lKSa6T58I

It seems that the Manjaro Linux distribution is not keen to convert a char** to a wchar_t**
Perhaps other Linux distributions are more relaxed.
Comment 1 Mikołaj Chwalisz 2018-11-13 04:31:56 EST
Created attachment 3214 [details]
Patch to fix python 3 compatibility in visualizer

The bug is caused by changes in Python3. In which the strings are in UTF-8 and it is required to parse locale information in command line arguments. The attached patch should fix the problem.

However, it is not end of the story. As it discovers a new bug (should I report it as new bug?) with `import ns.mesh`. This bug can be reproduced with adding this import to any python example.

```
Waf: Entering directory `/home/Code/ns3/ns-3-dev-git/build/debug'
[2602/2652] Linking build/debug/scratch/scratch-simulator
[2604/2652] Linking build/debug/scratch/subdir/subdir
[2606/2652] Linking build/debug/utils/ns3-dev-test-runner-debug
[2612/2652] Linking build/debug/utils/ns3-dev-print-introspected-doxygen-debug
Waf: Leaving directory `/home/chwalisz/Code/ns3/ns-3-dev-git/build/debug'
Build commands will be stored in build/debug/compile_commands.json
'build' finished successfully (11.530s)
TypeError: PyModule_AddObject() needs non-NULL value

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "examples/wireless/wifi-ap.py", line 32, in <module>
    import ns.mesh
SystemError: <built-in function __import__> returned a result with an error set
Command ['/home/.local/share/virtualenvs/ns3-QbuZIpeN/bin/python', 'examples/wireless/wifi-ap.py'] exited with code 1
```
Comment 2 Tom Henderson 2018-11-13 14:05:27 EST
Thank you for this patch; I will test and apply if it works for me.

Regarding this error:
> TypeError: PyModule_AddObject() needs non-NULL value

I believe that this is just due to stale Python bindings and I will fix this separately.
Comment 3 Mikołaj Chwalisz 2018-11-15 08:35:37 EST
I pulled updated bindings but discovered further incompatibilities in the Python code for the visualizer.

Is actually Python 3 intended to be supported by ns-3? Imho it should be as the Py2 support is going to end in near future.
Comment 4 Tom Henderson 2018-11-15 09:42:18 EST
(In reply to Mikołaj Chwalisz from comment #3)
> I pulled updated bindings but discovered further incompatibilities in the
> Python code for the visualizer.
> 
> Is actually Python 3 intended to be supported by ns-3? Imho it should be as
> the Py2 support is going to end in near future.

Yes, Python 3 is intended to be fully supported, but I believe that most maintainers use distros still defaulting to Python 2, and the visualizer testing is not automated so it may fall through the cracks.  Patches would be welcome in this regard because the visualizer is not being actively maintained by anyone.
Comment 5 Tom Henderson 2018-11-17 01:16:19 EST
After applying the patch, the C++ code successfully builds but when I try:

./waf --pyrun src/flow-monitor/examples/wifi-olsr-flowmon.py -vis

I get:

", line 700, in Visualizer
    if _import_error is None:
NameError: name '_import_error' is not defined

Test machine is a venv on Linux Mint 19 Python 3.6.6.
Comment 6 Tom Henderson 2018-12-10 00:46:16 EST
I have some further information.  This patch introduces some regressions on Python 2, so more work is needed.

First, base.py needs:
from __future__ import print_function

More importantly, when I try to run this command on a patched ns-3-dev with Python 2:

$ ./waf --pyrun src/flow-monitor/examples/wifi-olsr-flowmon.py --vis

I get many instances of the following:

No visualization support (cannot import name core).
Traceback (most recent call last):
  File "<string>", line 2, in <module>
  File "/ns-3-allinone/ns-3-dev/src/visualizer/visualizer/core.py", line 1847, in start
    import sys
RuntimeError: maximum recursion depth exceeded while calling a Python object
Comment 7 Alex Afanasyev 2019-04-15 13:22:40 EDT
Created attachment 3245 [details]
Extended patch

A little bit extended patch to make Visualizer work with Python3. Works for me, but didn't test with Python2.
Comment 8 Tom Henderson 2019-04-15 17:45:42 EDT
Thanks, I tried the latest and it didn't work for me on Python 2.7.  However, I have a patch now (this patch plus some others) that seems to work for both Python 2.7 and Python 3.6; can you please test it?

https://gitlab.com/nsnam/ns-3-dev/merge_requests/57

Or if you prefer, it is on the 'visualizer-upgrade' branch of
https://gitlab.com/tomhenderson/ns-3-dev.git