Difference between revisions of "Ns-3 on Visual Studio"

From Nsnam
Jump to: navigation, search
(Alternative Solutions: comment on goals)
(ATTRIBUTE_x Macro Modifications: clarification)
Line 167: Line 167:
  
 
==== ATTRIBUTE_x Macro Modifications ====
 
==== ATTRIBUTE_x Macro Modifications ====
Another example is all of the ATTRIBUTE_HELPER_* macros have been extended.  All ATTRIBUTE_HELPER_* macros now have an additional parameter (same one as previous section) that defines the DLL the "implementation" is located in.  The following is an example from the CORE module.  Use of any of the ATTRIBUTE_HELPER_* macros within a base ns-3 module require the new macro with the additional parameter.
+
Another example is all of the<code>ATTRIBUTE_HELPER_*</code> macros have been extended.  The <code>ATTRIBUTE_*</code> macros declare a three classes for each underlying type <code>X</code>:  <code>XValue</code>, <code>XAccessor</code> and <code>XChecker</code>.  These are impacted by the import/export tag discussed above. As a result, the tag for the module ''invoking'' the macro has to be passed through to the macro, so they now have an additional parameter (same one as previous section) that defines the DLL the "implementation" is located in.  The following is an example from the CORE module.  Use of any of the <code>ATTRIBUTE_HELPER_*</code> macros within a base ns-3 module require the new macro with the additional parameter.
 
  <nowiki>
 
  <nowiki>
 
  ATTRIBUTE_CHECKER_DEFINE_LIB (Boolean, NS3CORELIB);
 
  ATTRIBUTE_CHECKER_DEFINE_LIB (Boolean, NS3CORELIB);
 
  ATTRIBUTE_ACCESSOR_DEFINE_LIB (Boolean, NS3CORELIB);</nowiki>
 
  ATTRIBUTE_ACCESSOR_DEFINE_LIB (Boolean, NS3CORELIB);</nowiki>
  
Note:  Use of the ATTRIBUTE_HELPER_* macros locally with a program continue to use the previous macro and parameter sequence.  An example is the following code from attribute-test-suite.c
+
Note:  Use of the <code>ATTRIBUTE_HELPER_*</code> macros locally within a program continue to use the previous macro and parameter sequence.  An example is the following code from attribute-test-suite.c
  
 
  <nowiki>
 
  <nowiki>

Revision as of 20:34, 30 April 2018

Main Page - Current Development - Developer FAQ - Tools - Related Projects - Project Ideas - Summer Projects

Installation - Troubleshooting - User FAQ - HOWTOs - Samples - Models - Education - Contributed Code - Papers

This is a page to summarize the Visual Studio compatibility work that Robert Ammon initiated during the ns-3.28 release cycle, and continues towards ns-3.29.

Some previous ports for older versions of Visual Studio (2010, 2012) can be found by searching the wiki, but are no longer maintained.

The ns-3 tracker issue is https://www.nsnam.org/bugzilla/show_bug.cgi?id=2726. A per-module status table can be found at https://www.nsnam.org/bugzilla/attachment.cgi?id=3088. As of ns-3.28 release (March 2018), the project is still working through the Phase 1 patches.

Summary of changes

The following provides a summary of the changes for the Visual Studio Development environment. It highlights the changes necessary to the existing NS-3 files to support development for Windows using Visual Studio.

To support integration of the changes in a more manageable fashion, the changes necessary for the VS development environment are segregated into four phases. The four phases are defined as

  • Phase 1 – Visual Studio Compiler Warning Resolution
  • Phase 2 – Visual Studio Windows Conditional Compilation
  • Phase 3 – Visual Studio Class Declaration Changes
  • Phase 4 – NS-3 Implementation Changes for Visual Studio

Phase 1 – Visual Studio Compiler Warning Resolution

The changes in this phase are changes to the existing NS-3 source code files to eliminate compiler warnings under Visual Studio. Under the Visual Studio development environment, all compiler warnings are treated s errors. The changes for this phase eliminate the compiler warnings under Visual Studio. The changes mostly fall into the following categories:

  • Elimination of automatic type casts to lower resolution data types (for example uint16_t to uint8_t)
These changes either add static cast declarations or modify the code (if possible) to eliminate the need for a type case
  • Elimination of unused function parameters
These changes either eliminate the unused parameters (if possible) or add NS_UNUSED references to the parameter.
  • Elimination of hidden variables (for example, a for loop using index i inside of a for loop using index i)
These changes rename the inner variable that is hiding the outer variable.

The Phase 1 changes will be applied in first before the Phase 2, Phase 3 and Phase 4 changes. The Phase 1 changes apply to the majority of the NS-3 modules but not all modules.

Phase 2 – Visual Studio Windows Conditional Compilation

The changes in this phase are changes to the existing NS-3 source code files to add conditional code for the Visual Studio development environment. These changes add code that is executed under the Windows configuration only.

For example, the following code from CORE is an example of code changed for Windows to produce the same log output:

#ifndef _WIN32
#define NS_LOG_APPEND_FUNC_PREFIX                               \
  if (g_log.IsEnabled (ns3::LOG_PREFIX_FUNC))                   \
    {                                                           \
      std::clog << g_log.Name () << ":" <<                      \
      __FUNCTION__ << "(): ";                                   \
    }                                                           \

#else
#define NS_LOG_APPEND_FUNC_PREFIX                               \
  if (g_log.IsEnabled (ns3::LOG_PREFIX_FUNC))                   \
    {                                                           \
      std::clog << g_log.Name () << ":" <<                      \
      __func__ << "(): ";                                       \
    }                                                           \

#endif

The modifications in this phase affect a minority of the NS-3 source code files.

  • In this particular case, GCC also accepts __FUNCTION__ '(see GCC Function Names) so we don't need to make this change. Pdbarnes (talk)
    • This changes is modifying the use of __FUNCTION__ under GCC to __func__ under Visual Studio. Under Visual Studio, __FUNCTION__ includes the complete path name, __func__ matches what __FUNCTION__ under GCC does. Robert Ammon
    • I'm suggesting we can avoid conditional code completely in this case. How important is it to show full path names in log messages? Finding the matching function is what IDE's, grep, Doxygen are for. Pdbarnes (talk)
    • Now that I stare at the current code, I see we use __FUNCTION__ already, so even to get full paths why is any change needed? Pdbarnes (talk)
    • The purpose of the change is to make the Visual Studio implementation match the GCC implementation (and not include full path names under Visual Studio). If you want them to be different, we can do that but I don't think the user community is going to like the difference because it increases the size of the log files generated by a factor of 2 or 3 when the full path name of the EXE is included in each log message when function names are enabled. Robert Ammon
    • Yet this patch shows __FUNCTION__ for WIN32, and __func__ for all others. It sounds like WIN32 should use __func__, in which case they can both use __func__, and no need for conditional compilation. Pdbarnes (talk)
    • You have the cases reversed, the conditional at the start of the block is #ifndef _WIN32 not #ifdef _WIN32. Robert Ammon
    • Doh! In any case can't they both use the same symbol __func__? Pdbarnes (talk)
    • I don't have any way to verify the behavior on the non Windows platforms other than gcc on Ubuntu 16.04 Robert Ammon

Phase 3 – Visual Studio Class Declaration Changes

While the Windows DLLs are similar to the Unix shared libraries, there are a couple of significant differences between them. One of the differences which cause changes in the ns-3 source code is items in a Windows DLL are hidden from consumers of the DLL unless specifically identified as exported items. This means that the classes in the ns-3 modules that are accessible by other ns-3 modules or user programs.

Here is an example provided by Peter Barnes that illustrates the issue. Assuming you have the following code:

src/foo/model/foo.h:

 class foo 
 {
    ...
 }; // class foo  

src/bar/model/bar.h:

 #include “ns3/foo.h”
 class bar
 {
    ...
 }; // class bar 

When compiling module foo, the symbols in foo.h should be labeled dllexport (because the implementation lives in this executable unit foo which is a separate DLL). When compiling bar.cc, those same symbols in foo.h should be labeled dllimport (because the implementation is located in a different execution unit, the DLL containing bar needs to import them from the DLL containing foo). In the specific case of ns-3, each module is a separate DLL so references to a class that is exposed for consumption by other executable units (DLLs or programs) need to change this declaration depending on what's being compiled.

Proposed Method

This change adds a new header file to the core module and an additional conditional declaration to each class declaration.

src/core/ns3dll.h code fragment

 #ifndef NS3DLL_H
 #define NS3DLL_H
 
 #ifdef _WIN32
 #  ifdef _ANT
 #    define NS3ANTLIB __declspec(dllexport)
 #  else
 #    define NS3ANTLIB __declspec(dllimport)
 #  endif
 ...
 #else  /* _WIN32 */
 #  define NS3ANTLIB
 ...
 #endif  /* _WIN32 */
 
 #endif /* NS3DLL_H */
  • Clarification: NS3ANTLIB refers to the antenna module. In the current proposal a stanza like this exists for every module. The build system toggles the _MODULE flag for the module DLL currently being built, so its symbols are marked dllexport, while symbols in all other headers (which this module depends on) are marked dllimport. Pdbarnes (talk)
  • Here's a decent discussion: https://gcc.gnu.org/wiki/Visibility Pdbarnes (talk)

Each exported symbol needs the corresponding module export/import token as a label. Here is an example class declaration:

class NS3ANTLIB AntennaModel : public Object

This approach has these drawbacks:

  • Every module needs a mention in a core file, ns3dll.h
  • Not clear what to do with contrib modules. Does bake have to know how to insert contrib modules in this core file?
  • How do we prevent modified ns3dll.h files from being pushed accidentally?

Alternative Solutions

A question that has been received is "are there are other ways to indicate export/import"?

(1) For Microsoft Windows targets there are alternative methods for including the symbol in the DLL’s export table such as using a .def file with an EXPORTS section

  • This will work, but the names in the EXPORTS section must be the "object code" names of the items. Remember that in C++, the actual names of functions, global variables, classes, etc. are NOT the names assigned in the source code, they are decorated names created by the compiler. In order to use a .def file for this, a developer would have to know the compiler decorated name of the item. Also, Microsoft strongly recommends that the decorated names NOT be used by developers since the scheme used to create the decorated name by the compiler can change with a new compiler version
    • Agreed, manually updating mangled names is a bad idea. Can this be automated?`` Pdbarnes (talk)

(2) With GNU ld, using the --export-all linker flag

  • This approach was not evaluated for two reasons, first the goal of the project was to set up a Visual Studio development environment that did not require any other tools or software beyond Visual Studio. Second, using an approach like this would eliminate developer control of what was exported. While this may not be an issue for core ns-3 modules, with a contributor model, contributor may not want to make all of the implementation visible or external.
    • Exporting all symbols is the current behavior (with two possible exceptions, see below). What's wrong with keeping this as the default? Pdbarnes (talk)
    • Is there room to modify the overall goals for the project? For example, instead of "pure VS environment" we could set the lower bar of "can be built and developed from VS", which leaves room for hybrid solutions, such as continuing to use waf, or other non-VS build tools/steps. In particular, this might enable us to do something like --export-all and leverage the list of exported headers already present in wscript to indicate what should be exported. This would provide the same granularity as presently available. Pdbarnes (talk)

Digression on Current Symbol Export

The current behavior is to export all symbols in headers listed in wscript:

 headers = bld(features='ns3header')
 headers.module = 'internet'
 headers.source = [
    ''model/udp-header.h',
     ...

Private headers are simply not listed, so they are not copied to build/ns3/ and so aren't visible outside the declaring module.

An alternative method is to include a "private header" stanza in wscript (see a remnant from src/internet/wscript):

 privateheaders = bld(features='ns3privateheader')
 privateheaders.module = 'internet'
 privateheaders.source = [
    ] 

This feature is not used at the moment. It was used at one point to expose a private (non-exported) header to test code.

Questions/Discussion

  1. Can this use the existing NS3_MODULE_x symbol for the declaration?
    • Any name can be used for the export / import symbol name. The names chosen and list in ns3dll.h were created by me to be short (since they have to be added to the source code) yet provide enough detail on what they represent.
  2. Why does the value of the symbol need to be dynamic?
  3. Why don't all headers include ns3dll.h?
    • All headers need the declarations provided in ns3dll.h. If ns3dll.h is included by another header file that is included in the header file (either directly or indirectly), it has already been included and adding it to the header file is not necessary. NOTE: There is no reason that this header file could not be included in every other header file, the patches submitted for review only included this file where it was necessary to reduce the size of the changes.

ATTRIBUTE_x Macro Modifications

Another example is all of theATTRIBUTE_HELPER_* macros have been extended. The ATTRIBUTE_* macros declare a three classes for each underlying type X: XValue, XAccessor and XChecker. These are impacted by the import/export tag discussed above. As a result, the tag for the module invoking the macro has to be passed through to the macro, so they now have an additional parameter (same one as previous section) that defines the DLL the "implementation" is located in. The following is an example from the CORE module. Use of any of the ATTRIBUTE_HELPER_* macros within a base ns-3 module require the new macro with the additional parameter.

 ATTRIBUTE_CHECKER_DEFINE_LIB (Boolean, NS3CORELIB);
 ATTRIBUTE_ACCESSOR_DEFINE_LIB (Boolean, NS3CORELIB);

Note: Use of the ATTRIBUTE_HELPER_* macros locally within a program continue to use the previous macro and parameter sequence. An example is the following code from attribute-test-suite.c

 ATTRIBUTE_HELPER_HEADER (ValueClassTest);
 ATTRIBUTE_HELPER_CPP (ValueClassTest);

The modifications in this phase affect almost every header file in the ns-3 source code.

Alternative Proposal

This alternative has been proposed, but not implemented or tested.

src/core/model/ns3-export.h:

 //  No include guard.  This is meant to appear many times in a single compilation unit
 
 /*Usage:  start each header with:
     #define NS3_MODULE myModule
     #include "ns3/ns-export.h"
 
     Label each class with
 
     class NS3_EXPORT ...
 
     Have waf define NS3_COMPILATION_MODULE as the current module.
     (Or repurpose what it currently defines, NS3_MODULE_COMPILATION=1
     to hold the module name.)
 */
 	
 #ifdef NS3_EXPORT
 #  undef NS3_EXPORT
 #endif
 
 # From
 # https://stackoverflow.com/questions/2335888/how-to-compare-strings-in-c-conditional-preprocessor-directives
 constexpr int
 NS3_MODULE_STRCMP (char const* lhs, char const* rhs)
 {
   return (('\0' == lhs[0]) && ('\0' == rhs[0])) ? 0
     : (lhs[0] != rhs[0]) ? ((int)lhs[0] - (int)rhs[0])
     : NS3_MODULE_STRCMP ( lhs + 1, rhs + 1 );
 }
 
 #if 0 == NS3_MODULE_STRCMP (NS3_COMPILATION_MODUL, NS3_MODULE)
 #  define NS3_EXPORT __declspec(dllexport)
 #else
 #  define NS3_EXPORT __declspec(dllimport)
 #endif 

This proposal would simplify the implementation somewhat:

  • All (exported) headers would have a common format
  • No need to mention every module in a core file (ns3dll.h)
  • Scales to contrib modules
  • No need to extend the ATTRIBUTE_x macros

Phase 4 – NS-3 Implementation Changes for Visual Studio

In a few of the NS-3 implementations, there are differences in the implementation between Linux and Windows. These are cases where the same source code does not produce the exact same results, either to differences in the OS, third party libraries of compiler code generation. For these changes, the source code has been modified to produce equivalent behavior on all platforms that matches the original implementation.

The number of changes in this phase are limited to a few examples.