Back to GSoC 2021 projects
- Project Name: Direct Code Execution Modernization
- Student: Parth Pratim Chatterjee
- Mentors: Tom Henderson, Apoorva Bhargava, Vivek Jain
- Project Goals: DCE currently makes use of net-next-nuse to extend the Linux kernel internals like the networking stack to host applications but over the years the project hasn't been updated with the latest releases of the Linux kernel. As Linux progressed with newer releases, a major part of the source code changed, making previous glue code incompatible with the newer implementations of the network stack as some of the init calls and function usage changed significantly making migration to newer releases non-trivial. This project aims at enabling support for latest Linux kernel features and toolchains in the DCE environment with support for the socket networking stack, sysctl interfaces, system call access, etc. without any changes to the user APIs currently being used by host applications. The project aims at incorporating an upgraded net-next-nuse port for Linux +v5.10 or the LKL(Linux Kernel Library) into the DCE environment for host applications to effortlessly make use of Linux kernel stacks with minimum to no change in existing simulation scripts.
- About Me: I'm a freshman Computer Science undergraduate student at Kalinga Institute of Industrial Technology, Bhubaneshwar, India. I have a keen interest in Linux internals and computer networking. I was a grand prize winner at Google Code-In, 2018 for ns-3 organization, which helped me initially get introduced to DCE. I have an aptitude for Competitive Programming and heavily make use of C/C++, STL and other OOP concepts in solving algorithmic puzzles. I have an experience with C/C++ and Python of more than 3 years, working on projects for numerous Hackathons.
Milestones and Deliverables
The overall project goal is to update DCE such that the latest Linux systems are supported and the latest Linux kernel code could be used.
- Detailed Project Plan (will be continuously updated throughout the GSoC program duration)
- Phase 1
- Ubuntu 20.04 support
- Goal is that most capabilities presently available for Ubuntu 16 DCE will be available for Ubuntu 20.04 (native)
- Also produce Docker image and documentation to ease installation process
- Also contact glibc maintainers about a non-patched solution
- Upgrade net-next-nuse Linux kernel support to recent kernel
- Investigate SMP architecture for LKL by querying the LKL developers list
- More about this here
- Ubuntu 20.04 support
- Phase 2
- To be determined based on Phase 1 results
- Phase 3
- To be determined
- Community Bonding Period (May 17 - June 7)
- Figured out possible Scheduling bottlenecks in LKL in blocking network calls. Section 4.5
- Developed a beta docker port for ns-3-dce. Section 1.4
- Discussed the bright side of porting the latest Linux kernel using the net-next-nuse architecture.
- Discussed possible regression tests for verifying both performance and results.
- Week 1 (June 7 - June 14)
- Week 2 (June 14 - June 21)
- Opened PR to integrate Github Actions Workflow for DCE #118
- Opened PR to the DCE repo for adding support for custom Glibc build #117
- Opened PR to Bake to support pulling and building required dependencies !9
- Got patches on pyViz dependency checks and configure_arguments attribute for depends_on field, merged into upstream bake : !8 and !7
- Initiated discussions on issues with current Bake environment and fixes to problems like regexp based file lookups
- Debugging Linux kernel timekeeping inconsistencies, net-device xmit packet loss and untimely socket connection request timeout over custom(P2P,Csma,Wifi etc.) registered net-device
- Week 3 (June 21 - June 28)
- Integrated ns-3-dev way of generating Python Bindings for DCE. (PR yet to be made)
- Identified possible positions where packets are being dropped in the IP Layer for the UDP protocol, in the Linux Kernel.
- Week 4 (June 28 - July 5)
- Figured out specific setuptools(50.3.2) and setuptools_scm(5.0.0) versions which supports building python bindings on Ubuntu-16.04
- Identified Linux kernel commit which valides packets with CHECKSUM_PARTIAL (possibly linked to hardware offloading)
- Linux-4.9.273 (LTS release with BBR support) : 35 tests passed
- Linux-5.10.47 (LTS release) : Fixed timeout issues, spotted inconsistent results and one-way traffic due to probable packet drops.
- Repos :
- Linux-4.9.273 : https://github.com/ParthPratim/net-next-nuse-4.9.273
- Linux-5.10.47 : https://github.com/ParthPratim/net-next-nuse-5.10.47
- Week 5 (July 5 - July 12)
- DCE-Linux-5.10.47 : Fixed NULL current task pointer : commit
- Linux-5.10.47 : Dropped down to net-next-nuse's iterative sysctl interface, Fixed failure on encountering link in table
- Linux-5.10.47 : Setup ethtool , Tested offloading and checksum operations and fetched kernel device properties
- Linux-5.10.47 : Patch for HW checksum BUG_ON and xmit skb_checksum_help : commit
- Python --apiscan argument for DCE : Repo for pybind-apiscan
- Python Bindings for DCE : Pull Request #120