Bugzilla – Bug 2954
BulkSendApplication not keeping TCP buffer full
Last modified: 2018-07-28 22:01:17 EDT
Created attachment 3143 [details] TCP logging for one flow Overview: I have a remote host transferring packets to many UEs over LTE-EPC. The remote host is running the BulkSendApplication and the UE is running the PacketSinkApplication. The UE should always receive 536 bytes because it is the TCP maximum segment size. But it sometimes receives 488 bytes because the buffer empties. The buffer shouldn't empty because BulkSendApplication will send more packets once buffer space frees up. Steps to Reproduce: Will add after minimizing test case. Actual Results: TCP buffer empties during simulation Expected Results: TCP buffer is full during simulation Attached tcp_log.txt containing TCP logging for one TCP flow. Node 232 is the remote host and 7.0.0.97 is the UE. The remaining data eventually hits zero. However, the remaining data is not monotonically decreasing. In one case, it went from 9912 to 10400 bytes. But eventually it hits 0. The end of the log shows UE alternating between sending 536 and 488 bytes.
Created attachment 3144 [details] test case reproduction This test case reproduces the problem. - 1 remote host - 210 UEs It creates a 2-3-2 hex grid with three-sector sites for a total of 7 sites. Each site has 10 UEs distributed uniformly. A remote host is connected via LTE-EPC to each UE. The remote host runs BulkSenderApplication and the UEs run PacketSinkApplication. The data transfer is downlink only and over TCP.
Adding repro steps. Steps to Reproduce: 1. Move simple_lte.cc to scratch directory 2. ./waf configure -d debug 3. ./waf build 4. ./waf --run "simple_lte 1" > debug_log.txt 2>&1 5. Let simulation proceed for some time (~35 minutes) 6. cat debug_log.txt | grep "via TcpL4Protocol to 7.0.0.97" Result: near beginning of file: [node 232] Send segment of size 536 with remaining data 130536 via TcpL4Protocol to 7.0.0.97. [node 232] Send segment of size 536 with remaining data 130512 via TcpL4Protocol to 7.0.0.97. [node 232] Send segment of size 536 with remaining data 129976 via TcpL4Protocol to 7.0.0.97. [node 232] Send segment of size 536 with remaining data 130464 via TcpL4Protocol to 7.0.0.97. near end of file: [node 232] Send segment of size 488 with remaining data 0 via TcpL4Protocol to 7.0.0.97. [node 232] Send segment of size 536 with remaining data 488 via TcpL4Protocol to 7.0.0.97. [node 232] Send segment of size 488 with remaining data 0 via TcpL4Protocol to 7.0.0.97. [node 232] Send segment of size 536 with remaining data 488 via TcpL4Protocol to 7.0.0.97. [node 232] Send segment of size 488 with remaining data 0 via TcpL4Protocol to 7.0.0.97.
Reproduced with examples/tcp/tcp-bulk-send.cc. Result: [node 0] Send segment of size 488 with remaining data 0 via TcpL4Protocol to 10.1.1.2. Header 49153 > 9 [ACK] Seq=691225 Ack=1 Win=32768 ns3::TcpOptionTS(9933;9927) [node 0] Send segment of size 536 with remaining data 488 via TcpL4Protocol to 10.1.1.2. Header 49153 > 9 [ACK] Seq=691713 Ack=1 Win=32768 ns3::TcpOptionTS(9951;9945) [node 0] Send segment of size 488 with remaining data 0 via TcpL4Protocol to 10.1.1.2. Header 49153 > 9 [ACK] Seq=692249 Ack=1 Win=32768 ns3::TcpOptionTS(9951;9945) [node 0] Send segment of size 536 with remaining data 488 via TcpL4Protocol to Unable to reproduce with src/lte/examples/lena-dual-stripe.cc. Result: [node 37] Send segment of size 536 with remaining data 123992 via TcpL4Protocol to 7.0.0.20. Header 49173 > 10019 [ACK] Seq=946577 Ack=1 Win=32768 ns3::TcpOptionTS(12175;12152) [node 37] Send segment of size 536 with remaining data 124480 via TcpL4Protocol to 7.0.0.20. Header 49173 > 10019 [ACK] Seq=947113 Ack=1 Win=32768 ns3::TcpOptionTS(12175;12155) [node 37] Send segment of size 536 with remaining data 123944 via TcpL4Protocol to 7.0.0.20. Header 49173 > 10019 [ACK] Seq=947649 Ack=1 Win=32768 ns3::TcpOptionTS(12175;12155)
Actually in src/lte/examples/lena-dual-stripe.cc, the problem is there too but not as severe as hitting zero data in the TCP buffer. Result: [node 37] Send segment of size 536 with remaining data 46640 via TcpL4Protocol to 7.0.0.20. Header 49173 > 10019 [ACK] Seq=890297 Ack=1 Win=32768 ns3::TcpOptionTS(11678;11649) [node 37] Send segment of size 536 with remaining data 46104 via TcpL4Protocol to 7.0.0.20. Header 49173 > 10019 [ACK] Seq=890833 Ack=1 Win=32768 ns3::TcpOptionTS(11682;11660) [node 37] Send segment of size 536 with remaining data 45568 via TcpL4Protocol to 7.0.0.20. Header 49173 > 10019 [ACK] Seq=891369 Ack=1 Win=32768 ns3::TcpOptionTS(11682;11660) [node 37] Send segment of size 536 with remaining data 130256 via TcpL4Protocol to 7.0.0.20. Header 49173 > 10019 [ACK] Seq=806681 Ack=1 Win=32768 ns3::TcpOptionTS(11691;11660) [node 37] Send segment of size 536 with remaining data 130536 via TcpL4Protocol to 7.0.0.20. Header 49173 > 10019 [ACK] Seq=891905 Ack=1 Win=32768 ns3::TcpOptionTS(11740;11718) [node 37] Send segment of size 536 with remaining data 130512 via TcpL4Protocol to 7.0.0.20. Header 49173 > 10019 [ACK] Seq=892441 Ack=1 Win=32768 ns3::TcpOptionTS(11982;11960)
Narrowed down the bug. Confirmed caused by enabling RLC_AM. When using RLC_AM, BulkSendHelper does not keep the transmit buffer full. [node 232] Send segment of size 488 with remaining data 0 via TcpL4Protocol to 7.0.0.97. Header 49245 > 10096 [ACK] Seq=682009 Ack=1 Win=32768 ns3::TcpOptionTS(10425;10119) [node 232] Send segment of size 536 with remaining data 488 via TcpL4Protocol to 7.0.0.97. Header 49245 > 10096 [ACK] Seq=682497 Ack=1 Win=32768 ns3::TcpOptionTS(10425;10130) [node 232] Send segment of size 488 with remaining data 0 via TcpL4Protocol to 7.0.0.97. Header 49245 > 10096 [ACK] Seq=683033 Ack=1 Win=32768 ns3::TcpOptionTS(10425;10130) When using RLC_UM, BulkSendHelper keeps the transmit buffer full. [node 232] Send segment of size 536 with remaining data 122584 via TcpL4Protocol to 7.0.0.97. Header 49245 > 10096 [ACK] Seq=680721 Ack=1 Win=32768 ns3::TcpOptionTS(9713;9691) [node 232] Send segment of size 536 with remaining data 123072 via TcpL4Protocol to 7.0.0.97. Header 49245 > 10096 [ACK] Seq=681257 Ack=1 Win=32768 ns3::TcpOptionTS(9730;9708) [node 232] Send segment of size 536 with remaining data 122536 via TcpL4Protocol to 7.0.0.97. Header 49245 > 10096 [ACK] Seq=681793 Ack=1 Win=32768 ns3::TcpOptionTS(9730;9708) Uploading lena-dual-stripe.cc that reproduces this. This is reproducible by changing RLC_UM to RLC_AM and vice-versa on Line 386. Will there be a fix for this? Is there a workaround for now?
Created attachment 3154 [details] lena-dual-stripe.cc