Bugzilla – Bug 1039
TCP Nagle algorithm and RTO calculation
Last modified: 2011-12-10 21:32:53 EST
I'm filing a reminder to look at two changes to TCP behavior when the new TCP socket architecture was defined. These can be seen in the packet traces for routing-aodv-regression (tcp-chain) but need some separate tests once ns-3.10 is released. In looking at the tcp-chain traces in routing-aodv-regression, there are a few substantive differences in the model. a) this application is writing data periodically into the buffer (1200 bytes every 150 ms). The old TCP would not send a partial segment but the new TCP will try to empty its buffer even if it is not a full segment. In other words, it does not use Nagle's algorithm by default. b) in this experiment, there comes a time (after 1.9 seconds) where acks are lost. Both the old and new TCP will take a coarse timeout, but the retransmission timeout is quite different. In the old TCP, the retransmission of the segment sent at 1.905 seconds comes at 2.791 seconds; about 800 ms. In the new TCP, it comes at time 2.116 seconds; about 200 ms. I think that this should also be looked at; 200ms seems too small for an RTO. According to RFC 2988, 1 seconds is the recommended min RTO.
(In reply to comment #0) > I'm filing a reminder to look at two changes to TCP behavior when the new TCP > socket architecture was defined. These can be seen in the packet traces for > routing-aodv-regression (tcp-chain) but need some separate tests once ns-3.10 > is released. > > In looking at the tcp-chain traces in routing-aodv-regression, there are a few > substantive differences in the model. > > a) this application is writing data periodically into the buffer (1200 bytes > every 150 ms). The old TCP would not send a partial segment but the new TCP > will try to empty its buffer even if it is not a full segment. In other words, > it does not use Nagle's algorithm by default. I can't find the logic for this in the old TCP implementation either, though things are a bit harder to find in that one. I think this logic would fit in TcpSocketBase::Send fairly easily. > b) in this experiment, there comes a time (after 1.9 seconds) where acks are > lost. Both the old and new TCP will take a coarse timeout, but the > retransmission timeout is quite different. In the old TCP, the retransmission > of the segment sent at 1.905 seconds comes at 2.791 seconds; about 800 ms. In > the new TCP, it comes at time 2.116 seconds; about 200 ms. I think that this > should also be looked at; 200ms seems too small for an RTO. According to RFC > 2988, 1 seconds is the recommended min RTO. The minimum RTO (200ms) is set in rtt-estimator.cc. I'm betting this was set in ns-3 to follow what Linux does, but I'm guessing Linux also does some special rtt estimation to support a lower min RTO. +1 to bump it up to 1 second. I hope to get a patch up here soon.
Created attachment 1043 [details] Increase min RTO and use Nagle's algorithm Ok, this is a pretty simple patch. For the RTO stuff, I just modified the min RTO value in rtt-estimator. After looking at the logs in routing-aodv-regression, it seems like this takes care of it. For Nagle's algorithm, I decided to put the logic in SendPendingData. Two thoughts: 1) Is it ok to have DelAck default to 2 while using Nagle's algorithm? I have some memory of delack and Nagle's algorithm not working well together. Finally, I could add a BooleanValue attribute to tcp-socket to enable/disable Nagle's algorithm. Would we like to be able to do that? Finally, with these changes, we will have to regenerate some test vectors.
(In reply to comment #2) > Created attachment 1043 [details] > Increase min RTO and use Nagle's algorithm > > Ok, this is a pretty simple patch. For the RTO stuff, I just modified the min > RTO value in rtt-estimator. After looking at the logs in > routing-aodv-regression, it seems like this takes care of it. > > For Nagle's algorithm, I decided to put the logic in SendPendingData. Two > thoughts: 1) Is it ok to have DelAck default to 2 while using Nagle's > algorithm? I have some memory of delack and Nagle's algorithm not working well > together. Finally, I could add a BooleanValue attribute to tcp-socket to > enable/disable Nagle's algorithm. Would we like to be able to do that? > > Finally, with these changes, we will have to regenerate some test vectors. Please disregard the change to the aodv test for regenerating the vectors. I meant to pull that out before posting.
Created attachment 1044 [details] Patch with attribute for enabling/disabling nagle
Created attachment 1045 [details] Patch with attribute for enabling/disabling nagle fixed typo
Created attachment 1046 [details] Patch with attribute for enabling/disabling nagle actually check the attribute in tcp-socket-base to enable/disable nagle :)
(In reply to comment #2) > Created attachment 1043 [details] > Increase min RTO and use Nagle's algorithm > > Ok, this is a pretty simple patch. For the RTO stuff, I just modified the min > RTO value in rtt-estimator. After looking at the logs in > routing-aodv-regression, it seems like this takes care of it. > > For Nagle's algorithm, I decided to put the logic in SendPendingData. Two > thoughts: 1) Is it ok to have DelAck default to 2 while using Nagle's > algorithm? I have some memory of delack and Nagle's algorithm not working well > together. Wikipedia describes this interaction: http://en.wikipedia.org/wiki/Nagle%27s_algorithm So does Stuart Cheshire: http://www.stuartcheshire.org/papers/NagleDelayedAck/ I think setting DelAck to 2 would be the right default, however. > Finally, I could add a BooleanValue attribute to tcp-socket to > enable/disable Nagle's algorithm. Would we like to be able to do that? Yes, but I would slightly prefer naming this TcpNoDelay to match the implementations (TcpNoDelay defaults to false). The logic seems correct in the patch. Any chance for a test case or example (on/off)?
(In reply to comment #7) > Yes, but I would slightly prefer naming this TcpNoDelay to match the > implementations (TcpNoDelay defaults to false). > > The logic seems correct in the patch. Any chance for a test case or example > (on/off)? I'll rename to TcpNoDelay and try to put together a simple test case for both on and off.
Created attachment 1047 [details] Change attribute to TcpNoDelay and add test suite The test suite is pretty simple. I initially send 5 packets to get some data and acks flowing. I then write 1 byte followed by 535 bytes to the socket. With TcpNoDelay on, both of these will be wrapped up and sent individually. Without TcpNoDelay, i.e. Nagle on, the 1 byte and 535 byte will be sent together in one packet. This leads to a different number of Rx events at the packet sink (a difference of one packet). For the test I check the total number of Rx events at the packet sink, plus the sizes of each packet received. This patch does not fix the two failing test cases, ns3-tcp-cwnd and routing-aodv-regression, by regenerating the test vectors. That can be done separately.