Created attachment 2096 [details]
Old flow monitor results

Created attachment 2097 [details]
New flow monitor results

(In reply to ywbyjiqo from comment #0)
> TCP should advertise RCV.WND, that is m_rxBuffer->MaxBufferSize (),
> but actually advertises m_rxBuffer->MaxBufferSize () - m_rxBuffer->Size ().
> Receiver expects data from RCV.NXT up to RCV.NXT + RCV.WND - 1. Sender may
> send data from SND.UNA up to SND.UNA + AWND - 1, where AWND is the
> advertised window. Therefore, receiver should advertise window equal to
> RCV.WND. Check RFC 793 for more details.


Without substracting the actual size of the buffer, with your proposal there is the possibility (using small buffer or very powerful link) to saturate the buffer itself.

Example:

Buffer size = 99
RCV.NXT = 0

--> segment 0 is lost
--> segment 1-99 are fully received

if I substract m_rxBuffer->Size () in the advertised window, for each ACK it becomes: 99, 98, 97... when segment 99 arrives, I have only 1 available slot on the buffer, so my window is 1, for accepting segment 100. Then, at 99 segment in the buffer, the sender is stopped.

With your proposal:

for each ACK, the adv window stays constant at 99. When the segment 99 arrives, the ACK reports that I have space for 99 more segments, which is wrong (we have seen that the space available after the segment 99 is 1, not 99). The sender sends segment 100, 101, and goes on. From segment 101, I have no more space to save them, and they are lost.

There's something I'm missing?

Nat

> Buffer size = 99
> RCV.NXT = 0
> --> segment 0 is lost
> --> segment 1-99 are fully received

If buffer size equals 99 MSS, then sender sends 99 segments, and you say it sends 100 segments. I will assume that you wanted to say "Buffer size = 100 MSS" and "segment i" means segment with bytes in range [i * MSS, i * MSS + MSS - 1], including both ends.

> With your proposal:
>
> for each ACK, the adv window stays constant at 99. When the segment 99 arrives, the ACK reports that I have space for 99 more segments, which is wrong (we have seen that the space available after the segment 99 is 1, not 99).

The sender sends 100 segments. First segment is lost. Receiver gets 99 segments and sends 99 ACKs, each with ACK field equal to 0 (initial sequence is 0 for simplicity) and AWND = 100.
As ACK field is always equal to 0, the sender window does not advance, i.e., SND.UNA keeps equal to 0 for each ACK received. 

> The sender sends segment 100, 101, and goes on.

The sender may only send bytes from SND.UNA up to SND.UNA + min(CWND, AWND) ≤ SND.UNA + AWND. So it still can only send bytes from the range [0, 100 MSS - 1].

> if I substract m_rxBuffer->Size () in the advertised window, for each ACK it becomes: 99, 98, 97... when segment 99 arrives, I have only 1 available slot on the buffer, so my window is 1, for accepting segment 100. Then, at 99 segment in the buffer, the sender is stopped.

It is too late to stop it when it receives the last ACK. By this time it has already entered fast recovery, reduced its CWND by half and inflated it back for each ACK received, each time trying to send more segments. And the only thing that prevents it from sending more segments is AWND as we assume that it is not CWND-limited and AWND actually matters.

The sender should be stopped when it receives the first ACK. That is why you don't inform the sender about the space available, but about the range of bytes you want to receive. You indicate that you want to receive bytes from ACK = 0 to ACK + AWND = 100 * MSS, but you are not ready for bytes > 100 MSS yet. The sender knows that it should not send bytes above 100 MSS all the time, not when it receives the last ACK.

If you indicate ACK = 0 and AWND = 1 MSS, you effectively say that you only want to receive bytes from the range [0, MSS - 1], even though you have already received and stored bytes [MSS, 100 * MSS - 1]. It is what RFC 793 calls "shrinking the window". From RFC 793, page 42:

  The mechanisms provided allow a TCP to advertise a large window and to
  subsequently advertise a much smaller window without having accepted
  that much data.  This, so called "shrinking the window," is strongly
  discouraged.  The robustness principle dictates that TCPs will not
  shrink the window themselves, but will be prepared for such behavior
  on the part of other TCPs.

If receiver shrinks the window, it means that something is really wrong on its side, like it is short on memory and tries to gather it by reducing receive buffers of all its TCP connections, killing processes by OOM killer and so on. It is emergency behaviour, we probably don't want to model it in NS-3.

I am satisfied with this description. I have already included the patch in my repo.

+1

(In reply to ywbyjiqo from comment #4)
> If receiver shrinks the window, it means that something is really wrong on
> its side, like it is short on memory and tries to gather it by reducing
> receive buffers of all its TCP connections, killing processes by OOM killer
> and so on. It is emergency behaviour, we probably don't want to model it in
> NS-3.

I have another question.

Under this description, and applying your patch, an ns-3 receiver does not advertise a 0-window, pratically. But what if the application is reading really slowly from the socket? This means that data remains in the buffer, but we are not advertising that, leading to a data loss (because there is no space in the buffer). What is your suggestion?

Nat

Fixed in 11711:902e19ae0511