Bug 1039

Summary: TCP Nagle algorithm and RTO calculation
Product: ns-3 Reporter: Tom Henderson <tomh>
Component: tcpAssignee: Adrian S.-W. Tam <adrian.sw.tam>
Status: RESOLVED FIXED    
Severity: normal CC: adrian.sw.tam, jpelkey, ns-bugs
Priority: P5    
Version: pre-release   
Hardware: All   
OS: All   
Attachments: Increase min RTO and use Nagle's algorithm
Patch with attribute for enabling/disabling nagle
Patch with attribute for enabling/disabling nagle
Patch with attribute for enabling/disabling nagle
Change attribute to TcpNoDelay and add test suite

Description Tom Henderson 2011-01-03 19:35:14 EST
I'm filing a reminder to look at two changes to TCP behavior when the new TCP socket architecture was defined.  These can be seen in the packet traces for routing-aodv-regression (tcp-chain) but need some separate tests once ns-3.10 is released.

In looking at the tcp-chain traces in routing-aodv-regression, there are a few substantive differences in the model.

  a) this application is writing data periodically into the buffer (1200 bytes every 150 ms).  The old TCP would not send a partial segment but the new TCP will try to empty its buffer even if it is not a full segment.  In other words, it does not use Nagle's algorithm by default.

 b) in this experiment, there comes a time (after 1.9 seconds) where acks are lost.  Both the old and new TCP will take a coarse timeout, but the retransmission timeout is quite different.  In the old TCP, the retransmission of the segment sent  at 1.905 seconds comes at 2.791 seconds; about 800 ms.  In the new TCP, it comes at time 2.116 seconds; about 200 ms.  I think that this should also be looked at; 200ms seems too small for an RTO.  According to RFC 2988, 1 seconds is the recommended min RTO.
Comment 1 Josh Pelkey 2011-03-16 11:42:43 EDT
(In reply to comment #0)
> I'm filing a reminder to look at two changes to TCP behavior when the new TCP
> socket architecture was defined.  These can be seen in the packet traces for
> routing-aodv-regression (tcp-chain) but need some separate tests once ns-3.10
> is released.
> 
> In looking at the tcp-chain traces in routing-aodv-regression, there are a few
> substantive differences in the model.
> 
>   a) this application is writing data periodically into the buffer (1200 bytes
> every 150 ms).  The old TCP would not send a partial segment but the new TCP
> will try to empty its buffer even if it is not a full segment.  In other words,
> it does not use Nagle's algorithm by default.

I can't find the logic for this in the old TCP implementation either, though things are a bit harder to find in that one. I think this logic would fit in TcpSocketBase::Send fairly easily.


>  b) in this experiment, there comes a time (after 1.9 seconds) where acks are
> lost.  Both the old and new TCP will take a coarse timeout, but the
> retransmission timeout is quite different.  In the old TCP, the retransmission
> of the segment sent  at 1.905 seconds comes at 2.791 seconds; about 800 ms.  In
> the new TCP, it comes at time 2.116 seconds; about 200 ms.  I think that this
> should also be looked at; 200ms seems too small for an RTO.  According to RFC
> 2988, 1 seconds is the recommended min RTO.

The minimum RTO (200ms) is set in rtt-estimator.cc. I'm betting this was set in ns-3 to follow what Linux does, but I'm guessing Linux also does some special rtt estimation to support a lower min RTO.  +1 to bump it up to 1 second.

I hope to get a patch up here soon.
Comment 2 Josh Pelkey 2011-03-16 14:33:41 EDT
Created attachment 1043 [details]
Increase min RTO and use Nagle's algorithm

Ok, this is a pretty simple patch. For the RTO stuff, I just modified the min RTO value in rtt-estimator. After looking at the logs in routing-aodv-regression, it seems like this takes care of it.

For Nagle's algorithm, I decided to put the logic in SendPendingData. Two thoughts: 1) Is it ok to have DelAck default to 2 while using Nagle's algorithm? I have some memory of delack and Nagle's algorithm not working well together. Finally, I could add a BooleanValue attribute to tcp-socket to enable/disable Nagle's algorithm. Would we like to be able to do that?

Finally, with these changes, we will have to regenerate some test vectors.
Comment 3 Josh Pelkey 2011-03-16 14:36:40 EDT
(In reply to comment #2)
> Created attachment 1043 [details]
> Increase min RTO and use Nagle's algorithm
> 
> Ok, this is a pretty simple patch. For the RTO stuff, I just modified the min
> RTO value in rtt-estimator. After looking at the logs in
> routing-aodv-regression, it seems like this takes care of it.
> 
> For Nagle's algorithm, I decided to put the logic in SendPendingData. Two
> thoughts: 1) Is it ok to have DelAck default to 2 while using Nagle's
> algorithm? I have some memory of delack and Nagle's algorithm not working well
> together. Finally, I could add a BooleanValue attribute to tcp-socket to
> enable/disable Nagle's algorithm. Would we like to be able to do that?
> 
> Finally, with these changes, we will have to regenerate some test vectors.

Please disregard the  change to the aodv test for regenerating the vectors. I meant to pull that out before posting.
Comment 4 Josh Pelkey 2011-03-16 15:18:15 EDT
Created attachment 1044 [details]
Patch with attribute for enabling/disabling nagle
Comment 5 Josh Pelkey 2011-03-16 15:20:24 EDT
Created attachment 1045 [details]
Patch with attribute for enabling/disabling nagle

fixed typo
Comment 6 Josh Pelkey 2011-03-16 16:07:22 EDT
Created attachment 1046 [details]
Patch with attribute for enabling/disabling nagle

actually check the attribute in tcp-socket-base to enable/disable nagle :)
Comment 7 Tom Henderson 2011-03-17 00:52:28 EDT
(In reply to comment #2)
> Created attachment 1043 [details]
> Increase min RTO and use Nagle's algorithm
> 
> Ok, this is a pretty simple patch. For the RTO stuff, I just modified the min
> RTO value in rtt-estimator. After looking at the logs in
> routing-aodv-regression, it seems like this takes care of it.
> 
> For Nagle's algorithm, I decided to put the logic in SendPendingData. Two
> thoughts: 1) Is it ok to have DelAck default to 2 while using Nagle's
> algorithm? I have some memory of delack and Nagle's algorithm not working well
> together. 

Wikipedia describes this interaction:
http://en.wikipedia.org/wiki/Nagle%27s_algorithm
So does Stuart Cheshire:
http://www.stuartcheshire.org/papers/NagleDelayedAck/

I think setting DelAck to 2 would be the right default, however.

> Finally, I could add a BooleanValue attribute to tcp-socket to
> enable/disable Nagle's algorithm. Would we like to be able to do that?

Yes, but I would slightly prefer naming this TcpNoDelay to match the implementations (TcpNoDelay defaults to false).

The logic seems correct in the patch. Any chance for a test case or example (on/off)?
Comment 8 Josh Pelkey 2011-03-17 09:53:59 EDT
(In reply to comment #7)
> Yes, but I would slightly prefer naming this TcpNoDelay to match the
> implementations (TcpNoDelay defaults to false).
> 
> The logic seems correct in the patch. Any chance for a test case or example
> (on/off)?

I'll rename to TcpNoDelay and try to put together a simple test case for both on and off.
Comment 9 Josh Pelkey 2011-03-17 12:23:08 EDT
Created attachment 1047 [details]
Change attribute to TcpNoDelay and add test suite

The test suite is pretty simple. I initially send 5 packets to get some data and acks flowing. I then write 1 byte followed by 535 bytes to the socket. With TcpNoDelay on, both of these will be wrapped up and sent individually. Without TcpNoDelay, i.e. Nagle on, the 1 byte and 535 byte will be sent together in one packet. This leads to a different number of Rx events at the packet sink (a difference of one packet). For the test I check the total number of Rx events at the packet sink, plus the sizes of each packet received.

This patch does not fix the two failing test cases, ns3-tcp-cwnd and routing-aodv-regression, by regenerating the test vectors. That can be done separately.