Nsnam - User contributions [en]

GSOC2014LTP

2015-04-10T13:13:17Z

RubenMartinez: /* Weekly Reports */

{{TOC}}

Return to [[GSOC2014AcceptedProjects | GSoC 2014 Accepted Projects]] page.

== Project overview ==

* '''Project name:''' Licklider Transmission Protocol (LTP).

* '''Student:''' [mailto:rubenmartinez.vidal@gmail.com Rubén Martínez]

* '''Mentors:''' [mailto:tomh@tomh.org Tom Henderson], [mailto:tommaso.pecorella@unifi.it Tommaso Pecorella]

* '''Abstract:''' Licklider Transmission Protocol (LTP) is a point to point protocol for environments with long round-trip delays (such as deep space communications). This project will provide a RFC compliant implementation, it will consist of a ns-3 module with its corresponding testsuits, examples, helpers and documentation.

* '''Code Repository ''': http://code.nsnam.org/rmartinez/ns-3-dev-ltp

= Approach =

This protocol can be deployed at two different leves: on top of either the Data-Link-Layer or the Transport Layer (UDP). The best approach should be to start with an UDP implementation, this will ensure compliance with already
existing implementations. A data-link-layer implementation can also be done depending on how the project advances, and will be be taken into consideration after the results of the mid-term review (see schedule section).

== Development methodology ==

I will use an iterative-incremental development methodology, each iteration will include implementation, documentation (doxygen style comments, thorough the source code), and testing. Development will be divided
on 3 main tasks:

* Task 1: This task consist in developing the ltp-headers and a SDNV codec (Self-Delimiting Numeric Values) a type of data encoding used in DTN protocols. This task consist on defining several data-structures and their serialization/deserialization methods.

* Task 2: This task will focus on developing the backbone of the protocol by defining the classes: ltp-packet-queues, ltp-session-records and the basic structure of the ltp-protocol. This will define the main data structures required in control logic used to handle: packets, sessions, protocol state, timers, etc.
* Task 3: This task will focus on the Send() and Receive() methods of the ltp-protocol class, these methods implement the control logic of the protocol.

== Testing approach ==

This project will have testing and cross-validation against existing implementations.

First, testing will we performed as part of the development cycle. It will use a test case based approach using the standard methods provided by the ns-3 API in the form of the TestSuite and TestCase classes. This project will have two kind of tests:

* Unit Tests: each task will have a group of test vectors, those tests will ensure that the internal workings of each developed class works in isolation.
* System tests: these tests will be provided mainly with examples, it will consist on using the developed module together with already tested ns-3 modules.

Lastly, the implementation will be cross-validated against existing implementations of the LTP, such as ION-DTN (JPL) or LTPlib (Trinity College Dublin). This cross-validation will be performed using a shared file descriptor to which both implementations can read or write traffic. The FdNetDevice class provided by the ns-3 API will be used to achieve this task.

== Design ==
=== Block Diagram ===
[[File:Ltp-block-diagram.png]]
=== LTP Protocol Send Logic===
[[File:send-diagram.png|1000px]]
=== LTP Protocol Receive Logic===
[[File:Ltp-recieve-diagram.png|1000px]]

= Deliverables =

The final product of the project will be a ns-3 module, so the main deliverables will be the several sections of this module:

* Deliverable 1 - model and helper: will contain the main functionalities of the protocol and the helper classes to simplify its use.

::Deliverable 1.1 - Packet Headers and SDNV codec.
::Deliverable 1.2 - LTP-Protocol auxiliary structures (Data Queues and Session State Records).
::Deliverable 1.3 - LTP-Protocol Core.
::Deliverable 1.4 - LtpNetDevice.
::Deliverable 1.5 - LtpStackHelper.
* Deliverable 2 - test: this deliverable will contain the several testsuits used.

::Deliverable 2.1 - Unit Tests. Packet Headers and SDNV codec: check correctness.
::Deliverable 2.2 - Unit Tests, LTP-Protocol Auxiliary, check methods, timeouts and data handling correctness.
::Deliverable 2.3 - Unit Tests, NetDevice implementation, check methods.
::Deliverable 2.4 - Unit Tests: protocol implementation, communication between nodes, retransmissions, cancellation of communication.

*Deliverable 3 - examples : this deliverable will contain several examples on how to install and use the protocol.
*Deliverable 4 - Cross-Validation: This deliverable will contain several tests using FdNetDevices to perform cross-validation with existing implementations. (4.1 early tests basic protocol, 4.2 final validation).

= Schedule =

*Week 1 - May 19 - May 25. '''D-1.1. D-2.1.'''
*Week 2 - May 26 - June 1.
*Week 3 - June 2 - June 8. '''D-1.2. D-2.2.'''
*week 4 - June 9 - June 15.
*Week 5 - June 16 - June 22.
*Week 6 - June 23 - June 29: Mid-Term Evaluation.
*Week 7 - June 30 - July 6. '''D-1.3. D-2.3. D-4.1.'''
*Week 8 - June 7 - July 13.
*Week 9 - July 14 - July 20. '''D-1.4.'''
*Week 10 - July 21 - July 27. '''D-1.5'''
*Week 11 - July 28 - August 3. ''' D-3, D-2.4'''
*Week 12 - August 4 - August 10. '''D-4.2'''

== Mid Term Review ==

The code is composed of 2 patches in Rietveld issues 97540043 [1] and
105340046 [2].

The first corresponds to the SDNV [3] encoding supportused in DTN protocols, from which I contributed the test-suite.

The second is a partial implementation of the LTP protocol containing:
packet data structures, protocol management classes and basic API
functions for communication with higher layer protocols.

To run test suites, the command is ./test.py -s ltp-protocol. Additionally the interested
reader can look at the ltp-protocol.rst document, or the wiki page [4] for details on the purpose of each class and how they interact with each other.

*[1] https://codereview.appspot.com/97540043/
*[2] https://codereview.appspot.com/105340046/
*[3] http://www.dtnrg.org/wiki/SDNV
*[4] http://www.nsnam.org/wiki/GSOC2014LTP

== Final Code Review ==

The final code review for the project can be found in [1], this review contains the final version of the model, including a helper, examples, tests and documentation.

The provided test-suites can be run as follows:
./test.py -s ltp-protocol
./test.py -s ltp-channel-loss

[1] https://codereview.appspot.com/123190043/

= List of Features =

== Convergence Layer Adapter ==
*Link state cues (callbacks in convergence layer adapter) : '''Link Up''', '''Link Down''', '''CP sent''', '''RS sent''', '''EOB dequeued''', '''CX dequeued'''.

== Ltp Core==
*Drop packet procedures: simple discard or Cancellation segment transmission.
*Start Transmission procedure: Event triggered on reception of '''Link Up''' signal. Response: dequeue and tx all packets.
*Start CP Timer: Event triggered on reception of '''CP received''' signal, Response: start CP timer.
*Stop CP timer : Event triggered on reception of RS segments, response: stop CP timer.
*Start RS Timer: Event triggered on reception of '''RS Timer''' signal, Response: start RS timer.
*Stop RS Timer: Event triggered by reception of RA, response: stop RS timer.
*Start Cancel Timer: Event triggered on reception of '''CX dequeued''', response: start cancel timer.
*Stop Cancel Timer: Event triggered on reception of CAX segment, response: stop cancel timer.
*Stop Transmission procedure: Event triggered on reception of '''Link Down''', Response: start dequeing of segments.
*Suspend Timers: Event triggered on reception of '''Remote peer link Down''', Response: suspend timers.
*Resume Timers: Event triggered on reception of '''Remote peer link Up''', Response: resume timers.
*Retrans CP: Event triggered by expiration of CP timer, Response: Check Retransmission limit, enqueue new CP segment.
*Retrans RS: Event triggered by expiration of RS timer, Response: Check Retransmission limit, enqueue new RS segment.
*Signify Red Part: Reception of EORP and CP, response: Send notification to Client Service instance.
*Signify Green Data Segment: Reception of GD segment, response: Send notification to Client Service instance.
*Signify Transmission Completed: All data transmited (signal '''EOB dequeued''') and RP ackd, response: send notification to client service instance and start Close session.
*Send RP: Original reception of regular CP or asynchronous CP. Response: perform checks then: 1. drop and send CR OR 2. compute bounds of reception claim and send RP.
*Retrans Data: triggered on reception of RS, response: Send RA segment, stop CP timer, if reception claim indicates incomplete data reception: check retransmission limit then send CS if exceeded or retransmit missed Data segment if not.
*Retransmit Cancellation segment: triggered on expiration of Cancel timer. response: send copy of CX.
*ACK cancellation: triggered on reception of CX, response: if received CS send CAS, if received CR sned CAR.
*Cancel Session: triggered on reception of CAX, reponse: stop cancel timer and close session.
*Handle Miscolored segments: reception of red-part with offset higher than green-part. response: discard.

== Other==
*Management Information Base: Database with times of operation of LTP engines, provides link state cues: '''Remote peer link Up, Remote peer link Down'''.
*Handle System Error conditions.
*Notifications to Client Service instance: session start, green-part segment arrival, red-part reception, transmission session completion, transmission-session cancellation, reception-session cancellation, initial-transmission completion.
*Security Extensions.
*Integrate SOCIS 2013 bundle protocol implementation with LTP in a separate repo (Tom)
*Set up virtual machine environment with CORE network emulator to test interoperability against ns-3 implementation (Tom)

== Features ordered by priority ==
=== 1. Basic data transmission ===
# Link state cues. '''DONE'''
# Start Transmission procedure. '''DONE'''
# Drop packet procedures. '''DONE''' (No Cancellation sent for now)
# Handle Miscolored segments. '''DONE'''
# Notifications to Client Service instance. '''DONE'''
# Signify Red Part. '''DONE'''
# Signify Green Data Segment. '''DONE'''
# Signify Transmission Completed. '''DONE'''
# Stop Transmission procedure. '''DONE (Partially taking RPs into account)'''

=== 2. Model validation ===
# Integrate SOCIS 2013 bundle protocol implementation with LTP in a separate repo (Tom)
# Set up virtual machine environment with CORE network emulator to test interoperability against ns-3 implementation (Tom)

=== 3. Retransmission based reliability ===
# Send CP. '''DONE''' (Only syncronous checkpoints for now).
# Send RS. '''DONE'''
# Send RAS. '''DONE'''
# Start CP Timer. '''DONE'''
# Start RS Timer.'''DONE'''
# Retrams Data + Retrams CP. ''' DONE (merged both cases together)'''
# Stop CP Timer. '''DONE'''
# Retrans RS: '''DONE'''
# Stop RS timer.'''DONE'''

=== 4. Cancellation ===
# Cancel Session (Send CX)
# Start Cancel Timer
# Cancel ACK (Send CAX)
# Stop Cancel Timer
# Retransmit CX

=== 5. Advanced kwnoledge about environment and other considerations ===
# Management Information Base
# Suspend Timers
# Resume Timers
# Handle System Error conditions.
# Authentication Extensions.
# Security Extensions.

= Protocol Validation =
The validation is performed using The Common Open Research Emulator (CORE).
Ns-3 implementation is validated for interoperability against LTPLib [https://down.dsg.cs.tcd.ie/ltplib/]

== Setup ==

Start core daemon.

sudo /etc/init.d/core-daemon start

Run core-gui.

core-gui

Topology setup: two nodes connected with a point to point link

10.0.0.1/24 <-------------> 10.0.0.2/24

[[File:Core-tcp-scenario.png]]

Notes on installation of LTPlib:

autoreconf --install
./configure
sudo apt-get install libssl-dev (required library not being checked by configure)
make
sudo make install

Default build is intended for (http://artes.esa.int/projects/mudsat-dtn-m2m-satellite-applications). This includes an implementation specific behaviour: MUDSAT blcok re-tx control (mutiple re-transmission of green data segments).

This is not specified in the RFC. In order to avoid it, manually remove this line:

File: include/ltp_int.h
Line 49:/// This is for the MUDSAT build, TODO: make this a configure param later
Line 50:#define MUDSAT

== Validation ==

* LTP is run over UDP. Layer 2 MTU is set to 1500 bytes.
* Node 2 acts as a server and runs LTPlib.
* Node 1 acts as client and uses both LTPlib and NS-3 emulation (for comparison purposes).

==== Server running LTPlib - Node 2 (10.0.0.2) ====
Run LTPlib in server mode on 10.0.0.2 and specify 10.0.0.1 as the source (use ltp-deepspace IATA defined port in both sides).

ltpd -v -m S -L 10.0.0.2:1113 -S 10.0.0.1:1113

==== Client using LTPlib - Node 1 (10.0.0.1) ====

* Using LTPLib as client.

ltpd -v -m C -D 10.0.0.2:1113 -S 10.0.0.1:1113

* Block data is specified in file: ltpd.in.

* In order to send green data the '-R' (length of red data) parameter must be used with a value smaller than the size of the block.

ltpd -v -m C -R 100 -D 10.0.0.2:1113 -S 10.0.0.1:1113

==== Client using NS-3 emu - Node 1 (10.0.0.1) ====

Set eth0 to prosmiscuous mode:

ifconfig eth0 promisc

cd into ns-3 directory and run:
./waf --run "scratch/ltp-emu"

Block data is specified in script as std::vector<uint8_t>

To perform tests use parameters -testType and -testNum:

./waf --run "scratch/ltp-emu -testType=nominal -testNum=0"
./waf --run "scratch/ltp-emu -testType=rx -testNum=0"

Test numbers are assigned according to the following section.

=== Nominal operation tests ===

Behaviour is analogous for both implementations (only minor changes in sizes and serial numbers).

====Test 1: 500 byte block ( < MTU ) full red data. (SUCCESS)====
* 10.0.0.1 ---> 10.0.0.2. Data block is sent in a single segment - Type (0x03 - Red Data, Checkpoint , EORP, EOB).
* 10.0.0.1 <--- 10.0.0.2. Report Segment - Type (0x08).
* 10.0.0.1 ---> 10.0.0.2. Report ACK Segment - Type (0x09).
* PCAP Files: [[File:node1-ltplib-blockSize-500.pcap| LTPlib pcap file]] - [[File:node1-ns3-blockSize-500.pcap| NS3-emu pcap file]]

./waf --run "scratch/ltp-emu -testType=nominal -testNum=1"
ltpd -v -m C -w 0 -D 10.0.0.2:1113 -S 10.0.0.1:1113

====Test 2: 5000 byte block( > MTU ) full red data. (SUCCESS)====

* 10.0.0.1 ---> 10.0.0.2. Data Block is split into 4 red segments.
* 1st to 3th Segment Type (0x00 - Red Data). 4th Segment Type (0x03 - Red Data, Checkpoint , EORP, EOB)
* 10.0.0.1 <--- 10.0.0.2. Report Segment - Type (0x08).
* 10.0.0.1 ---> 10.0.0.2. Report ACK Segment - Type (0x09).
* PCAP Files: [[File:node1-ltplib-blockSize-5000.pcap| LTPlib pcap file]] - [[File:node1-ns3-blockSize-5000.pcap| NS3-emu pcap file]]

./waf --run "scratch/ltp-emu -testType=nominal -testNum=2"
ltpd -v -m C -w 0 -D 10.0.0.2:1113 -S 10.0.0.1:1113

==== Test 3: 500 byte block ( < MTU) , 200 bytes red and 300 bytes green data (SUCCESS) ====

LTPlib follows this optional approach: RFC 5326 - 4.1. Transmission Request

For bandwidth efficiency reasons, implementations MAY choose to instead mark the
entire segment (within which the red-part boundary falls) as red-
part, causing green-part data falling within the segment to also be
treated as red-part.

* 10.0.0.1 ---> 10.0.0.2. Both red and green data is sent together in a single segment - Type (0x03 - Red Data, Checkpoint , EORP, EOB).
* 10.0.0.1 <--- 10.0.0.2. Report Segment - Type (0x08).
* 10.0.0.1 ---> 10.0.0.2. Report ACK Segment - Type (0x09).

NS-3 emu

* 10.0.0.1 ---> 10.0.0.2. Red Data - Type (0x03 - Red Data, Checkpoint , EORP).
* 10.0.0.1 ---> 10.0.0.2. Green Data - Type (0x07 - Green Data, Checkpoint , EOB).
* 10.0.0.1 <--- 10.0.0.2. Report Segment - Type (0x08).
* 10.0.0.1 ---> 10.0.0.2. Report ACK Segment - Type (0x09).

Despite of that, interoperability is still possible and the full block is successfully transmitted.

./waf --run "scratch/ltp-emu -testType=nominal -testNum=3"
ltpd -v -m C -w 0 -R 200 -D 10.0.0.2:1113 -S 10.0.0.1:1113

==== Test 4: 5000 byte block ( > MTU), red data and green data (SUCCESS) ====

LTPLib:

* 10.0.0.1 ---> 10.0.0.2. Data Block is split into 2 red segments and 2 Green Segments.
* 10.0.0.1 ---> 10.0.0.2. Type (0x00 - Red Data)
* 10.0.0.1 ---> 10.0.0.2. Type (0x02 - Red Data, EORP) ---> A chunk of Green data is merged with red data in this segment
* 10.0.0.1 ---> 10.0.0.2. Type (0x04 - Green Data)
* 10.0.0.1 <--- 10.0.0.2. Report Segment - Type (0x08).
* 10.0.0.1 ---> 10.0.0.2. Type (0x07 - Green Data, EOB)
* 10.0.0.1 ---> 10.0.0.2. Report ACK Segment - Type (0x09).

NS3 emu:

* 10.0.0.1 ---> 10.0.0.2. Data Block is split into 2 red segments and 3 Green Segments.
* 10.0.0.1 ---> 10.0.0.2. Type (0x00 - Red Data)
* 10.0.0.1 ---> 10.0.0.2. Type (0x02 - Red Data, EORP)
* 10.0.0.1 ---> 10.0.0.2. Type (0x04 - Green Data)
* 10.0.0.1 ---> 10.0.0.2. Type (0x04 - Green Data)
* 10.0.0.1 ---> 10.0.0.2. Type (0x07 - Green Data, EOB)
* 10.0.0.1 <--- 10.0.0.2. Report Segment - Type (0x08).
* 10.0.0.1 ---> 10.0.0.2. Report ACK Segment - Type (0x09).

./waf --run "scratch/ltp-emu -testType=nominal -testNum=4"
ltpd -v -m C -w 0 -R 2000 -D 10.0.0.2:1113 -S 10.0.0.1:1113

==== Test 5: 500 byte block ( < MTU) , all green data (SUCCESS) ====

*10.0.0.1 --> 10.0.0.2. Type 0x07 (Green Data, EOB).
*LTPlib only: 10.0.0.1 --> 10.0.0.2. Type 0xC ( Cancel from Sender).

LTPlib specific behaviour: closing an open socket associated to an unfinished block generates a CS segment.

IMO the all-green case is not being properly handled.

./waf --run "scratch/ltp-emu -testType=nominal -testNum=5"
ltpd -v -m C -w 0 -R 0 -D 10.0.0.2:1113 -S 10.0.0.1:1113

* PCAP Files: [[File:ltplib-500-all-green.pcap| LTPlib pcap file]] - [[File:ns3-500-all-green.pcap| NS3-emu pcap file]]

==== Test 6: 5000 byte block ( > MTU) , all green data (SUCCESS) ====

*10.0.0.1 --> 10.0.0.2. Type 0x04 (Green Data).
*10.0.0.1 --> 10.0.0.2. Type 0x04 (Green Data).
*10.0.0.1 --> 10.0.0.2. Type 0x04 (Green Data).
*10.0.0.1 --> 10.0.0.2. Type 0x07 (Green Data, EOB).
*LTPlib only: 10.0.0.1 --> 10.0.0.2. Type 0xC ( Cancel from Sender).
* PCAP Files: [[File:ltplib-5000-all-green.pcap| LTPlib pcap file]] - [[File:ns3-5000-all-green.pcap| NS3-emu pcap file]]

./waf --run "scratch/ltp-emu -testType=nominal -testNum=6"
ltpd -v -m C -w 0 -R 0 -D 10.0.0.2:1113 -S 10.0.0.1:1113

=== Retransmission tests ===

- Use byte block of 100 KB.
Edit the point-to-point link using Packet Error Rates of: 1%,5%,10%,15%,20%,25%,30%.

./waf --run "scratch/ltp-emu -testType=retrans"

== Mobility ==

[[File:core-mobility-ltp.png]]

*Set-up two MBR nodes and a wireless network.
*Perform disconnections by manually drag and dropping the nodes out of range.

==== Test 1. Nodes are disconnected at the start of the transmission, then connectivity is resumed after 30 seconds.====
*The transmission was successful in this case.

====Test 2. Nodes are connected at the start, they lose connectivity in the middle of the transmission, and reconnect after 30 seconds.====

*All the data is successfully transmitted and acknowledged, but there is redundancy.
*The ns-3 client repeats the transmission several times before closing the transmission session.

= Weekly Reports =

== Week 1 ==

'''Finished - Deliverables 1.1 and 2.1'''

*Providing SDNV [http://www.dtnrg.org/wiki/SDNV] encoding support (this a numeric format commonly used on packet fields of DTN protocols). This encoding format was already implemented in Dizhi Zhou's BP [http://www.nsnam.org wiki/SOCIS2013BundleProtocolProject] project. I have expanded it with the corresponding test-suite and moved it to src/network/utils/, for use in both Bundle and LTP protocols. Currently under review in https://codereview.appspot.com/97540043/]
*Implementation of all LTP packet structures, this includes: Segment Header, Segment Trailer, and Content Header (for all segment variants: data, reports, report acknowledgments, etc.). The implementation including the corresponding tests and documentation.

== Week 2 ==

'''Working on Deliverables 1.2 and 2.2 '''

*Implementation of the Ltp queue-set, this is a class containing the dual queue structure (internal operations and application data) used for LTP outbound traffic. The implementation and the corresponding test case can be found on the project repository.

*Implementation of the session state record, this is a class to keep track of the state of an LTP transmission session, this class is basically a wrapper that handles several flags, counters, timers and lists. An early version can be found in the project repository.

== Week 3 ==

'''Working on Deliverables 1.2 and 2.2 '''

*Fixed several issues in the session state record.
*Provided a test-case for the session state record.
*Provided an early concept design for the LTP API.

== Week 4 ==
'''Working on Deliverables 1.3 and 2.3 '''

*Finished the design of the LTP API functions used for interaction with the upper layer.
*Included all the required auxiliary structures inside the LTP core class.
*Performed minor fixes to auxiliary structures.
*Provided a test-case for the LTP API functions.

== Week 5 ==

'''Working on Deliverables 1.3 and 2.3 '''

*Documentation: ltp-protocol.rst
*Preparing the mid-term review.
*Fixing issues in the LTP API class and small tweaks to auxiliary classes.
*Improving test-case class.

== Week 6 ==
-Provided an UDP Convergence Layer Adapter; it offers an API to LTP, performs socket management, and address bindings (mapping between LTP Engine Ids and IP addresses).
- Provided a LtpProtocolHelper and a basic (not reliable) transmission example between two nodes.
== Week 7 ==

- Moved LTP Engine Ids to IP addresses resolution to a LtpRoutingProtocol Class.
- Defined a base ConvergenceLayerAdapter class from which other types of CLAs can inherit.
- Improved UDP CLA.
- Implemented method UdpSocketImpl::GetTxAvailable() for use in UDP CLA.
- Fixed problems with the provided example and expanded it to transmit red and green data segments.
- Improved main protocol receiver logic to handle additional situations: start of reception, reception of red data, reception of green data, miss-colored segments, reception of checkpoint.
- Reviewed another GSOC's student mid-term code.
== Week 8 ==
- Fixed a problem which caused segments with the wrong type to be delivered.
- Provided several fixes and improved functionality of convergence layer, udp convergence layer and ltp-helper
- Provided a basic link state cue implementation.
- Offered a full list of features ordered by priority and re-scheduled the project timeline accordingly.
== Week 9 ==

I have finished all the features required for basic data transmission:

Protocol events are based on notifications from lower layer link state cues.
Start Transmission procedure.
Drop packet procedures.
Handle Miscolored segments.
Notifications to Client Service instance (application using the protocol).
Signify Red Part (data sent reliably).
Signify Green Data Segment (data that does not require reliability).
Signify Transmission Completed.
Stop Transmission procedure.

I have also been working on providing retransmission based reliability, and implemented the basic "handshake" mechanism:

1. Sender sends Data Checkpoint.
2. Receiver confirms reception of data up to this point with a Report Segment.
3. Sender answers with a Report Acknowledgement Segment.

== Week 10 ==

The protocol currently supports basic data transmission, and offers
a simplified protocol for retransmission based reliability.

== Week 11 ==

*Submitting fixes and providing a stable version of the protocol capable of retransmission based reliability.
*Writing tests.
*Writing documentation.

File:Core-mobility-ltp.png

2015-04-10T13:00:34Z

RubenMartinez: