Difference between revisions of "Ns-3 on Visual Studio"

From Nsnam
Jump to: navigation, search
m (Alternative Proposals: add missing include line)
Line 112: Line 112:
 
(2) With GNU ld, using the --export-all linker flag
 
(2) With GNU ld, using the --export-all linker flag
  
* ''This approach was not evaluated for two reasons, first the goal of the project was to set up a Visual Studio development environment that did not require any other tools or software beyond Visual Studio.  Second, using an approach like this would eliminate developer control of what was exported.  While this may not be an issue for core ns-3 modules, with a contributor model, contributor may not want to make all of the implementation visible or external.
+
* ''This approach will not work because gcc and Visual Studio do not use identical rules for name mangling of C++ objectsThe --export-all output would be specfic to gcc and would not work for Visual Studio.
** ''Exporting all symbols is the current behavior (with two possible exceptions, see below)What's wrong with keeping this as the default?'' [[User:Pdbarnes|Pdbarnes]] ([[User talk:Pdbarnes|talk]])
+
** ''Is there room to modify the overall goals for the project? For example, instead of "pure VS environment" we could set the lower bar of "can be built and developed from VS", which leaves room for hybrid solutions, such as continuing to use waf, or other non-VS build tools/steps.  In particular, this might enable us to do something like --export-all and leverage the list of exported headers already present in wscript to indicate what should be exported.  This would provide the same granularity as presently available.'' [[User:Pdbarnes|Pdbarnes]] ([[User talk:Pdbarnes|talk]])
+
  
 
==== Digression on Current Symbol Export ====
 
==== Digression on Current Symbol Export ====
Line 140: Line 138:
 
#* ''Any name can be used for the export / import symbol name.  The names chosen and list in ns3dll.h were created by me to be short (since they have to be added to the source code) yet provide enough detail on what they represent.
 
#* ''Any name can be used for the export / import symbol name.  The names chosen and list in ns3dll.h were created by me to be short (since they have to be added to the source code) yet provide enough detail on what they represent.
 
# Why does the value of the symbol need to be dynamic?
 
# Why does the value of the symbol need to be dynamic?
 +
#* ''As noted in the alternative proposal what the symbol resolves to depends on if the header file is being included in the module where the implementation resides or if it is used by another module to reference the module where the implementation resides.
 
#Why don't all headers include ns3dll.h?
 
#Why don't all headers include ns3dll.h?
 
#* ''All headers need the declarations provided in ns3dll.h.  If ns3dll.h is included by another header file that is included in the header file (either directly or indirectly), it has already been included and adding it to the header file is not necessary.  NOTE:  There is no reason that this header file could not be included in every other header file, the patches submitted for review only included this file where it was necessary to reduce the size of the changes.
 
#* ''All headers need the declarations provided in ns3dll.h.  If ns3dll.h is included by another header file that is included in the header file (either directly or indirectly), it has already been included and adding it to the header file is not necessary.  NOTE:  There is no reason that this header file could not be included in every other header file, the patches submitted for review only included this file where it was necessary to reduce the size of the changes.
Line 276: Line 275:
 
* Works equally well with contrib modules, since <code>ns3-modules.h</code> is auto-generated at build time, when the modules are known.
 
* Works equally well with contrib modules, since <code>ns3-modules.h</code> is auto-generated at build time, when the modules are known.
 
* Works transparently through macros, including the <code>ATTRIBUTE_</code> macros.
 
* Works transparently through macros, including the <code>ATTRIBUTE_</code> macros.
 +
**''This approach will work and the patch has been updated to use this approach with one exception.  Autogenerating the file at build time wont work because that would cause every source file in the whole project to be recompiled every time any modification is made to any file because Visual Studio uses the file modification time to determine which files need to be compiled. Visual Studio doesn't provide an analog to ./waf configure to configure the project before build time.
 +
  
 
=== Phase 4 – NS-3 Implementation Changes for Visual Studio ===
 
=== Phase 4 – NS-3 Implementation Changes for Visual Studio ===

Revision as of 23:29, 3 May 2018

Main Page - Current Development - Developer FAQ - Tools - Related Projects - Project Ideas - Summer Projects

Installation - Troubleshooting - User FAQ - HOWTOs - Samples - Models - Education - Contributed Code - Papers

This is a page to summarize the Visual Studio compatibility work that Robert Ammon initiated during the ns-3.28 release cycle, and continues towards ns-3.29.

Some previous ports for older versions of Visual Studio (2010, 2012) can be found by searching the wiki, but are no longer maintained.

The ns-3 tracker issue is https://www.nsnam.org/bugzilla/show_bug.cgi?id=2726. A per-module status table can be found at https://www.nsnam.org/bugzilla/attachment.cgi?id=3088. As of ns-3.28 release (March 2018), the project is still working through the Phase 1 patches.

Summary of changes

The following provides a summary of the changes for the Visual Studio Development environment. It highlights the changes necessary to the existing NS-3 files to support development for Windows using Visual Studio.

To support integration of the changes in a more manageable fashion, the changes necessary for the VS development environment are segregated into four phases. The four phases are defined as

  • Phase 1 – Visual Studio Compiler Warning Resolution
  • Phase 2 – Visual Studio Windows Conditional Compilation
  • Phase 3 – Visual Studio Class Declaration Changes
  • Phase 4 – NS-3 Implementation Changes for Visual Studio

Phase 1 – Visual Studio Compiler Warning Resolution

The changes in this phase are changes to the existing NS-3 source code files to eliminate compiler warnings under Visual Studio. Under the Visual Studio development environment, all compiler warnings are treated s errors. The changes for this phase eliminate the compiler warnings under Visual Studio. The changes mostly fall into the following categories:

  • Elimination of automatic type casts to lower resolution data types (for example uint16_t to uint8_t)
These changes either add static cast declarations or modify the code (if possible) to eliminate the need for a type case
  • Elimination of unused function parameters
These changes either eliminate the unused parameters (if possible) or add NS_UNUSED references to the parameter.
  • Elimination of hidden variables (for example, a for loop using index i inside of a for loop using index i)
These changes rename the inner variable that is hiding the outer variable.

The Phase 1 changes will be applied in first before the Phase 2, Phase 3 and Phase 4 changes. The Phase 1 changes apply to the majority of the NS-3 modules but not all modules.

Phase 2 – Visual Studio Windows Conditional Compilation

The changes in this phase are changes to the existing NS-3 source code files to add conditional code for the Visual Studio development environment. These changes add code that is executed under the Windows configuration only.

For example, the following code is an example of a header file that is not used and does not exist on Windows systems:

#ifndef _WIN32
#include <unistd.h>
#endif

The modifications in this phase affect a minority of the NS-3 source code files.

Phase 3 – Visual Studio Class Declaration Changes

While the Windows DLLs are similar to the Unix shared libraries, there are a couple of significant differences between them. One of the differences which cause changes in the ns-3 source code is items in a Windows DLL are hidden from consumers of the DLL unless specifically identified as exported items. This means that the classes in the ns-3 modules that are accessible by other ns-3 modules or user programs.

Here is an example provided by Peter Barnes that illustrates the issue. Assuming you have the following code:

src/foo/model/foo.h:

 class foo 
 {
    ...
 }; // class foo  

src/bar/model/bar.h:

 #include “ns3/foo.h”
 class bar
 {
    ...
 }; // class bar 

When compiling module foo, the symbols in foo.h should be labeled dllexport (because the implementation lives in this executable unit foo which is a separate DLL). When compiling bar.cc, those same symbols in foo.h should be labeled dllimport (because the implementation is located in a different execution unit, the DLL containing bar needs to import them from the DLL containing foo). In the specific case of ns-3, each module is a separate DLL so references to a class that is exposed for consumption by other executable units (DLLs or programs) need to change this declaration depending on what's being compiled.

Proposed Method

This change adds a new header file to the core module and an additional conditional declaration to each class declaration.

src/core/ns3dll.h code fragment

 #ifndef NS3DLL_H
 #define NS3DLL_H
 
 #ifdef _WIN32
 #  ifdef _ANT
 #    define NS3ANTLIB __declspec(dllexport)
 #  else
 #    define NS3ANTLIB __declspec(dllimport)
 #  endif
 ...
 #else  /* _WIN32 */
 #  define NS3ANTLIB
 ...
 #endif  /* _WIN32 */
 
 #endif /* NS3DLL_H */
  • Clarification: NS3ANTLIB refers to the antenna module. In the current proposal a stanza like this exists for every module. The build system toggles the _MODULE flag for the module DLL currently being built, so its symbols are marked dllexport, while symbols in all other headers (which this module depends on) are marked dllimport. Pdbarnes (talk)
  • Here's a decent discussion: https://gcc.gnu.org/wiki/Visibility Pdbarnes (talk)

Each exported symbol needs the corresponding module export/import token as a label. Here is an example class declaration:

class NS3ANTLIB AntennaModel : public Object

This approach has these drawbacks:

  • Every module needs a mention in a core file, ns3dll.h
  • Not clear what to do with contrib modules. Does bake have to know how to insert contrib modules in this core file?
  • How do we prevent modified ns3dll.h files from being pushed accidentally?

These questions need to be addressed.

Alternative Solutions

A question that has been received is "are there are other ways to indicate export/import"?

(1) For Microsoft Windows targets there are alternative methods for including the symbol in the DLL’s export table such as using a .def file with an EXPORTS section

  • This will work, but the names in the EXPORTS section must be the "object code" names of the items. Remember that in C++, the actual names of functions, global variables, classes, etc. are NOT the names assigned in the source code, they are decorated names created by the compiler. In order to use a .def file for this, a developer would have to know the compiler decorated name of the item. Also, Microsoft strongly recommends that the decorated names NOT be used by developers since the scheme used to create the decorated name by the compiler can change with a new compiler version
    • Agreed, manually updating mangled names is a bad idea. Can this be automated?`` Pdbarnes (talk)

(2) With GNU ld, using the --export-all linker flag

  • This approach will not work because gcc and Visual Studio do not use identical rules for name mangling of C++ objects. The --export-all output would be specfic to gcc and would not work for Visual Studio.

Digression on Current Symbol Export

The current behavior is to export all symbols in headers listed in wscript:

 headers = bld(features='ns3header')
 headers.module = 'internet'
 headers.source = [
    ''model/udp-header.h',
     ...

Private headers are simply not listed, so they are not copied to build/ns3/ and so aren't visible outside the declaring module.

An alternative method is to include a "private header" stanza in wscript (see a remnant from src/internet/wscript):

 privateheaders = bld(features='ns3privateheader')
 privateheaders.module = 'internet'
 privateheaders.source = [
    ] 

This feature is not used at the moment. It was used at one point to expose a private (non-exported) header to test code.

Questions/Discussion

  1. Can this use the existing NS3_MODULE_x symbol for the declaration?
    • Any name can be used for the export / import symbol name. The names chosen and list in ns3dll.h were created by me to be short (since they have to be added to the source code) yet provide enough detail on what they represent.
  2. Why does the value of the symbol need to be dynamic?
    • As noted in the alternative proposal what the symbol resolves to depends on if the header file is being included in the module where the implementation resides or if it is used by another module to reference the module where the implementation resides.
  3. Why don't all headers include ns3dll.h?
    • All headers need the declarations provided in ns3dll.h. If ns3dll.h is included by another header file that is included in the header file (either directly or indirectly), it has already been included and adding it to the header file is not necessary. NOTE: There is no reason that this header file could not be included in every other header file, the patches submitted for review only included this file where it was necessary to reduce the size of the changes.

ATTRIBUTE_x Macro Modifications

Another example is all of theATTRIBUTE_HELPER_* macros have been extended. The ATTRIBUTE_* macros declare a three classes for each underlying type X: XValue, XAccessor and XChecker. These are impacted by the import/export tag discussed above. As a result, the tag for the module invoking the macro has to be passed through to the macro, so they now have an additional parameter (same one as previous section) that defines the DLL the "implementation" is located in. The following is an example from the CORE module. Use of any of the ATTRIBUTE_HELPER_* macros within a base ns-3 module require the new macro with the additional parameter.

 ATTRIBUTE_CHECKER_DEFINE_LIB (Boolean, NS3CORELIB);
 ATTRIBUTE_ACCESSOR_DEFINE_LIB (Boolean, NS3CORELIB);

Note: Use of the ATTRIBUTE_HELPER_* macros locally within a program continue to use the previous macro and parameter sequence. An example is the following code from attribute-test-suite.c

 ATTRIBUTE_HELPER_HEADER (ValueClassTest);
 ATTRIBUTE_HELPER_CPP (ValueClassTest);

The modifications in this phase affect almost every header file in the ns-3 source code.

Alternative Proposals

Desirable attributes of any solution are:

  • All (exported) headers would have a common format
  • No need to mention every module in a core file (ns3dll.h)
  • Scales to contrib modules
  • No need to extend the ATTRIBUTE_x macros

Could a refinement of the primary proposal be made to work?

  • Auto-generate ns3-modules.h file (see below for a sketch)
    • Which defines a unique numeric symbol for each module present in the build.

This file would be added to the hgignore file, so it never gets added to the repo. By dynamically (at build time) assigning module numbers, we avoid numbering conflict from statically assigned module symbols, yet still facilitate simple (integer) comparisons between module tags and the current compilation unit tag.

This would enable a variation of an earlier (unworkable) proposal:

build/ns3/ns3-modules.h:

 /* This file autogenerated-do not edit. */
 
 /* Make sure NS3_MODULE is undefined.
   It will be defined by the file including this file. */
 #ifdef NS3_MODULE
 #  undef NS3_MODULE
 #endif
 
 /* Define the module ids */
 #ifndef NS3_MODULES_DEFINED
 #define NS3_MODULES_DEFINED
 
 #define NS3_ANTENNA_MODULE   1
 #define NS3_AODV_MODULE      2
 /* ... */
 
 #endif /* NS3_MODULES_DEFINED */

src/core/model/ns3-export.h:

 /* No include guard.  This is meant to appear many times
    in a single compilation unit
 */
 
 /* Usage:  
    Start each header with this three line block, after the last header inclusion (assuming this is the core module):
      #include "ns3-modules.h"
      #define NS3_MODULE NS3_CORE_MODULE
      #include "ns3/ns3-export.h"
  
    Label each class with
  
      class NS3_EXPORT ...
 
    The build system defines NS3_COMPILATION_MODULE to be
    the current module being built.
 */
 	
 #ifdef NS3_EXPORT
 #  undef NS3_EXPORT
 #endif
 
 #if NS3_MODULE == NS3_COMPILATION_MODULE
 #  define NS3_EXPORT __declspec(dllexport)
 #else
 #  define NS3_EXPORT __declspec(dllimport)
 #endif 

and an example source file:

src/.../example.cc:

 #include <iostream>
 #include <iomanip>
 
 // The build system:
 #ifndef NS3_COMPILATION_MODULE
 #  define NS3_COMPILATION_MODULE NS3_AODV_MODULE
 #endif
 
 // A module header file:
 #include "ns3-modules.h"
 #define NS3_MODULE NS3_ANTENNA_MODULE
 #include "ns3-export.h"
 
 // Helpers for this demo
 #define stringify_it( name ) # name
 #define stringify(name) stringify_it(name)
 #define doc(name) std::left << std::setw (20) << # name << "\t" << stringify (name) << "\n"
 
 int
 main (int argc, char** argv)
 {
 
   std::cout << doc (NS3_MODULE)
 	     << doc (NS3_COMPILATION_MODULE)
 	     << doc (NS3_EXPORT);
 
   return 0;
 } 

Two example builds and runs:

 g++ example.cc -o example  && ./example
 NS3_MODULE          	1
 NS3_COMPILATION_MODULE	2
 NS3_EXPORT          	__declspec(dllimport)
 
 g++ example.cc -o example -DNS3_COMPILATION_MODULE=NS3_ANTENNA_MODULE && ./example
 NS3_MODULE          	1
 NS3_COMPILATION_MODULE	1
 NS3_EXPORT          	__declspec(dllexport) 

This appears to address the requirements above, in a more direct way than the original proposal:

  • All exported headers have the same three line block defining their module identity, between two ns-3 includes
  • No core file listing all modules; it's auto-generated and only exists in the build tree.
  • Works equally well with contrib modules, since ns3-modules.h is auto-generated at build time, when the modules are known.
  • Works transparently through macros, including the ATTRIBUTE_ macros.
    • This approach will work and the patch has been updated to use this approach with one exception. Autogenerating the file at build time wont work because that would cause every source file in the whole project to be recompiled every time any modification is made to any file because Visual Studio uses the file modification time to determine which files need to be compiled. Visual Studio doesn't provide an analog to ./waf configure to configure the project before build time.


Phase 4 – NS-3 Implementation Changes for Visual Studio

In a few of the NS-3 implementations, there are differences in the implementation between Linux and Windows. These are cases where the same source code does not produce the exact same results, either to differences in the OS, third party libraries of compiler code generation. For these changes, the source code has been modified to produce equivalent behavior on all platforms that matches the original implementation.

The number of changes in this phase are limited to a few examples.