Created an attachment (id=253) [details]
sends data via PacketSocket from one node to another

This doesn't have the cleanest output, but I wanted to get it out. If its too
hard to read, I can work on that. What it does is take a user specified size
for a data stream and break it into FragSize (50000 bytes for ease of use)
packets and sends that to the receiver. The receiver prints out the packets it
gets and decrements a byte count that main set. When the count hits zero it
prints out that the packet has been delivered. This should be the last thing
printed.

If USE_HANDLESEND is defined, then we set a SendCallback in the sender. I try
to only send whole packets to make life a little easier.

This test program generates the same output if USE_HANDLESEND is not defined.

Created an attachment (id=255) [details]
updated version, now prints out the size of the tx Queue before the Send.

To clarify, what I expect to happen is that PacketSocket::Send will not return
-1, and to the best of my knowledge, the number of items in the DropTail queue,
contained in av, returned via side effect in
node::GetDevice(0)GetAttribute("TxQueue",av) will not increase. NOTE: that the
current version has the code that uses SetSendCallback #ifdef'ed out, so this
is just using PacketQueue::Send.

What I am seeing is each call to SrcRtSwitch::SendPacket seems to add another
item to the Queue, but nothing ever seems to take things off of it, so that
after 100 Sends, the DropTail queue fills up and the Send in the above
SendPacket method fails.

Created an attachment (id=256) [details]
example of a failed run.

To generate the error attached I did a:
./p2p_test --size=5050001 > failed.output

The uploaded file, failed.output, has the following text inserted in it to show
where the error occurs. What I expected for correct operation was that this
last byte would be written out and the 'Send returned -1' that is printed after
the NOTE, would have written 'Send returned 0' as it had for previous Sends. 

NOTE: The line below is the first failed packet. Before we try to do a send
around line 310 in SrcRtSwitch::SendPacket(), the queue has filled up. I don't
think it should do that. When we do try the Send in SendPacket, it returns -1
indicating a failure.

the following untested patch should fix your problem:
diff -r a60aeddab0fe src/node/packet-socket.cc
--- a/src/node/packet-socket.cc    Sun Sep 14 17:55:30 2008 -0700
+++ b/src/node/packet-socket.cc    Mon Sep 15 22:16:54 2008 -0700
@@ -26,6 +26,7 @@
 #include "ns3/packet.h"
 #include "ns3/uinteger.h"
 #include "ns3/trace-source-accessor.h"
+#include "ns3/simulator.h"

 #include <algorithm>

@@ -328,7 +329,8 @@ PacketSocket::SendTo (Ptr<Packet> p, uin
     }
   if (!error)
     {
-      NotifyDataSent (p->GetSize ());
+      Simulator::ScheduleNow (&PacketSocket::NotifyDataSent, this, p->GetSize
());
+      //NotifyDataSent (p->GetSize ());
     }

   if (error)

Tried Mathieu's untested patch, but it didn't work for me.

yes, it occured to me that it would not work. This is a pretty deep/tricky bug.

Created an attachment (id=262) [details]
Patch to add NetDevice::SetSendReadyCallback

As a starting point for a eventual soln at the Socket level. This patch adds a
callback members NetDevice::m_sendReadyCallback, of type
NetDevice::SendReadyCallback, and a method to set it,
NetDevice::SetSendReadyCallback. If the callback is defined, it is invoked when
a Packet is removed from the Queue. net-device.h has a stub definition, and
PointToPointNetDevice has a defn that does something.

(In reply to comment #8)
> Created an attachment (id=262) [details] [details]
> Patch to add NetDevice::SetSendReadyCallback
> 
> As a starting point for a eventual soln at the Socket level. This patch adds a
> callback members NetDevice::m_sendReadyCallback, of type
> NetDevice::SendReadyCallback, and a method to set it,
> NetDevice::SetSendReadyCallback. If the callback is defined, it is invoked when
> a Packet is removed from the Queue. net-device.h has a stub definition, and
> PointToPointNetDevice has a defn that does something.
> 
Hi David,
Did you test this patch? does it work?
Best regards
Faker Moatamri

(In reply to comment #9)
> (In reply to comment #8)
> > Created an attachment (id=262) [details] [details] [details]
> > Patch to add NetDevice::SetSendReadyCallback
> > 
> > As a starting point for a eventual soln at the Socket level. This patch adds a
> > callback members NetDevice::m_sendReadyCallback, of type
> > NetDevice::SendReadyCallback, and a method to set it,
> > NetDevice::SetSendReadyCallback. If the callback is defined, it is invoked when
> > a Packet is removed from the Queue. net-device.h has a stub definition, and
> > PointToPointNetDevice has a defn that does something.
> > 
> Hi David,
> Did you test this patch? does it work?
> Best regards
> Faker Moatamri

Faker,

This was a while ago, and I just don't recall. We're not using ns3 anymore (ns3
is great, but our problem turned out not to be a good match for ns3), so I
haven't been active in ns3 for a while. I am certain that If I had posted it to
the world I would have tested it. So it probably does what it says, but I
recall that it wasn't the eventual solution we were shooting for.

\dae