Difference between revisions of "GSoC2018 DCE Upgrade"

From Nsnam
Jump to: navigation, search
(Project Timeline)
(Project Timeline)
Line 104: Line 104:
 
* DCE is able to load the LKL shared Library and successfully calls the init function.
 
* DCE is able to load the LKL shared Library and successfully calls the init function.
 
* Tested a function from Import structure, LKL is able to make calls to import functions.
 
* Tested a function from Import structure, LKL is able to make calls to import functions.
 +
 +
== Week 6 (June 18 - June 24) ==

Revision as of 18:49, 17 June 2018

Main Page - Current Development - Developer FAQ - Tools - Related Projects - Project Ideas - Summer Projects

Installation - Troubleshooting - User FAQ - HOWTOs - Samples - Models - Education - Contributed Code - Papers

Return to GSoC 2018 Accepted Projects page.

Project overview

  • Project name: Direct Code Execution upgrade
  • Abstract: Direct Code Execution is a framework for ns-3. It facilitates to execute Linux kernel and Linux userspace networking applications inside ns-3 without changing their code. In order to provide such functionality, DCE integrates network simulator with custom Linux kernels like net-next-sim or net-next-nuse (LibOs). But net-next-nuse has not kept pace with new versions of Linux kernel. This has been the major hindrance for DCE to use latest Linux kernel. An active alternate to LibOs is Linux Kernel Library (LKL) which provides similar features of LibOs and supports latest Linux kernel versions. In this project, DCE will extend to support the networking stack of LKL and embed real libc in LKL.
  • About me: I am a master student at National Institute of Technology, Karnataka India. Currently, I am building a framework over ns-3-dce that automates the testing of TCP and AQM by using standard tools like netperf and iperf.


Technical Approach

In general, LKL offers userspace applications to use different kernel subsystems like the virtual file system, network stack, and scheduling. The plan is to adapt network stack functionality using LKL system call API. At the time of binary deployment over a node, DCE will load the LKL compiled binary. It then searches for sim_init function and tells to LKL the required set of function for a node. In return, LKL will give a structure of pointer to those functions so that DCE can use it later. In order to make DCE compatible with LKL, we need to adapt the host related function from LKL. There are approximately 35-40 host-related functions which are declared as structure lkl_host_operations in LKL. The objective is to redefine lkl_host_operation in the DCE environment. In addition to redefining those functions for DCE, we need to add glue to code to map LKL host functions to redefined DCE functions.

Steps to create DCE Upgrade Working Environment on Ubuntu

Note: Tools needed to download the source code of DCE and LKL are mercurial and git.

Install Direct Code Execution

  • Required packages for DCE: This is a set of packages needed to install and run ns-3-dce.
 sudo apt install cmake cvs git bzr unrar p7zip-full autoconf build-essential bison flex g++ libc6 
 sudo apt install libc6-amd64 libdb-dev libexpat1-dev libpcap-dev libssl-dev python-dev python-pygraphviz 
 sudo apt install python-pygoocanvas  python-setuptools qt4-dev-tools
  • Clone the Bake: Software utility used to download and build ns-3-dce.
 hg clone http://code.nsnam.org/bake bake
  • Environment Variables: Following are the environment variables required to build the ns-3-dce
 export BAKE_HOME=`pwd`/bake
 export PATH=$PATH:$BAKE_HOME
 export PYTHONPATH=$PYTHONPATH:$BAKE_HOME
  • Create a folder dce: This folder will contain ns-3-dce source code
 mkdir dce
 cd dce
  • Configure DCE: This command will configure the ns-3-dce. Here -e indicates DCE version.
 bake.py configure -e dce-linux-dev
  • Download and build ns-3-dce: Commands below will download and build the ns-3-dce and other dependent source code. Option -v is optional for debugging.
 bake.py download -v
 bake.py build -v

The above steps are taken from https://www.nsnam.org/docs/bake/tutorial/html/bake-tutorial.html


Install Linux Kernel library

  • Clone the LKL source code
 git clone https://github.com/sourabhjains/linux
  • Switch to dce-support branch
 cd linux
 git checkout dce-support  
  • Build the LKL
 make -C tools/lkl
  • Verify LKL shared library
 ls tools/lkl/lib/

Note: There should be a file called liblkl.so inside tools/lkl/lib

  • More about LKL library
 readelf -h liblkl.so  : reads the header
 readeld -d liblkl.so  : read the dynamic section
 ldd liblkl.so         : print the shared object dependencies

Importing LKL library in DCE example

  • Set Network stack with LKL library
 Use DceManagerHelper::SetNetworkStack and pass StringValue ("liblkl.so") as third arrgument.
 Set the DCE_PATH to liblkl.so accordingly.

Project Timeline

Week 1 (May 14 - May 20)

  • Initialized the project repositories ns-3-dce and LKL and attached it with circleci.
  • Added a DCE host for LKL here.
  • Defined couple of host functions related to the thread, memory allocation and time.

Week 2 (May 21 - May 27)

  • Understood the existing execution flow of net-next-nuse and the call forwarder mechanism to SimExported functions.
  • Defined the DceImport and DceExport structures in LKL to handle the Linux system call and the callbacks to DCE.
  • Defined the initialization function in LKL, this functions will initiate the Linux kernel and fills the global DceImport and DceExport structure.

Week 3 (May 28 - June 3)

  • Enable the LKL to build the DCE host by adding a separate function in Makefile.
  • Fixed the issues with DCE host to build LKL.

Week 4 (June 3 - June 10)

  • Added new callbacks in DceImport structure related to pthread functions of DCE.
  • Successfully build the LKL with DCE host [1] and generated the LKL shared library.
  • Attempted loading shared library of LKL in DCE with dce-iperf.cc example. It failed due to external dependencies like libc.so.

Week 5 (June 11 - June 17)

  • Removed all dependency over standard libraries.
  • DCE is able to load the LKL shared Library and successfully calls the init function.
  • Tested a function from Import structure, LKL is able to make calls to import functions.

Week 6 (June 18 - June 24)