Also:
[lally@sol ns-3.2.1]$ uname -a
SunOS sol 5.11 snv_101b i86pc i386 i86pc

Your bug report is against ns-3.2.  Since then, there have been a couple of patches to fix the ArpCache behavior in this regard; can you give your code a try with ns-3-dev (at least, you might be able to drop in the arp-cache.{cc,h} and arp-l4-protocol.{cc,h} files that are in the current ns-3-dev without otherwise touching your code)?

Or, try applying these patches to see if it clears up?
http://www.nsnam.org/bugzilla/show_bug.cgi?id=362
http://www.nsnam.org/bugzilla/show_bug.cgi?id=451

(In reply to comment #2)
> Your bug report is against ns-3.2.  Since then, there have been a couple of
> patches to fix the ArpCache behavior in this regard; can you give your code a
> try with ns-3-dev (at least, you might be able to drop in the arp-cache.{cc,h}
> and arp-l4-protocol.{cc,h} files that are in the current ns-3-dev without
> otherwise touching your code)?
> 
> Or, try applying these patches to see if it clears up?
> http://www.nsnam.org/bugzilla/show_bug.cgi?id=362
> http://www.nsnam.org/bugzilla/show_bug.cgi?id=451
> 

Copying the files seems to have worked.  I tried first to build ns-3-dev, but that didn't work out.  Something about not liking '-c -o'.

*** This bug has been marked as a duplicate of bug 451 ***

(In reply to comment #3)
> (In reply to comment #2)
> > Your bug report is against ns-3.2.  Since then, there have been a couple of
> > patches to fix the ArpCache behavior in this regard; can you give your code a
> > try with ns-3-dev (at least, you might be able to drop in the arp-cache.{cc,h}
> > and arp-l4-protocol.{cc,h} files that are in the current ns-3-dev without
> > otherwise touching your code)?
> > 
> > Or, try applying these patches to see if it clears up?
> > http://www.nsnam.org/bugzilla/show_bug.cgi?id=362
> > http://www.nsnam.org/bugzilla/show_bug.cgi?id=451
> > 
> 
> Copying the files seems to have worked.  I tried first to build ns-3-dev, but
> that didn't work out.  Something about not liking '-c -o'.
> 

Please file a separate bug if you think there is a problem with building ns-3-dev.

Nevermind.  The bug reappears, only requiring more simulated clients to do so.  

Now it works at 36 clients, and fails at 37.  Before copying in the files suggested, the numbers were 23 and 24, respectively.

Should I post this under 451's entry?

(In reply to comment #6)
> Nevermind.  The bug reappears, only requiring more simulated clients to do so.  
> 
> Now it works at 36 clients, and fails at 37.  Before copying in the files
> suggested, the numbers were 23 and 24, respectively.
> 
> Should I post this under 451's entry?
> 
No, this is fine, it may be a separate problem.

The below is half-baked, but can you see whether it fixes your problem?

diff -r 8e5d0354c509 src/internet-stack/arp-cache.cc
--- a/src/internet-stack/arp-cache.cc   Wed Jan 14 15:36:19 2009 +0000
+++ b/src/internet-stack/arp-cache.cc   Thu Jan 15 22:34:02 2009 -0800
@@ -213,7 +213,12 @@ ArpCache::HandleWaitReplyTimeout (void)
                 }
             }
        }
-
+      else
+        {
+          // This will cause the entry expiration time to line up
+          // with the WaitReply timer for future timeouts
+          entry->UpdateSeen ();
+        }
     }
   if (restartWaitReplyTimer)
     {
diff -r 8e5d0354c509 src/internet-stack/arp-cache.h
--- a/src/internet-stack/arp-cache.h    Wed Jan 14 15:36:19 2009 +0000
+++ b/src/internet-stack/arp-cache.h    Thu Jan 15 22:34:02 2009 -0800
@@ -199,6 +199,7 @@ public:
      */
     void ClearRetries (void);
 
+    void UpdateSeen (void);
   private:
     enum ArpCacheEntryState_e {
       ALIVE,
@@ -206,7 +207,6 @@ public:
       DEAD
     };
 
-    void UpdateSeen (void);
     Time GetTimeout (void) const;
     ArpCache *m_arp;
     ArpCacheEntryState_e m_state;

(In reply to comment #7)
> (In reply to comment #6)
> > Nevermind.  The bug reappears, only requiring more simulated clients to do so.  
> > 
> > Now it works at 36 clients, and fails at 37.  Before copying in the files
> > suggested, the numbers were 23 and 24, respectively.
> > 
> > Should I post this under 451's entry?
> > 
> No, this is fine, it may be a separate problem.
> 
> The below is half-baked, but can you see whether it fixes your problem?
> 
> diff -r 8e5d0354c509 src/internet-stack/arp-cache.cc
> --- a/src/internet-stack/arp-cache.cc   Wed Jan 14 15:36:19 2009 +0000
> +++ b/src/internet-stack/arp-cache.cc   Thu Jan 15 22:34:02 2009 -0800
> @@ -213,7 +213,12 @@ ArpCache::HandleWaitReplyTimeout (void)
>                  }
>              }
>         }
> -
> +      else
> +        {
> +          // This will cause the entry expiration time to line up
> +          // with the WaitReply timer for future timeouts
> +          entry->UpdateSeen ();
> +        }
>      }
>    if (restartWaitReplyTimer)
>      {
> diff -r 8e5d0354c509 src/internet-stack/arp-cache.h
> --- a/src/internet-stack/arp-cache.h    Wed Jan 14 15:36:19 2009 +0000
> +++ b/src/internet-stack/arp-cache.h    Thu Jan 15 22:34:02 2009 -0800
> @@ -199,6 +199,7 @@ public:
>       */
>      void ClearRetries (void);
> 
> +    void UpdateSeen (void);
>    private:
>      enum ArpCacheEntryState_e {
>        ALIVE,
> @@ -206,7 +207,6 @@ public:
>        DEAD
>      };
> 
> -    void UpdateSeen (void);
>      Time GetTimeout (void) const;
>      ArpCache *m_arp;
>      ArpCacheEntryState_e m_state;
> 

Thanks.  I tried it.  38 works. 39 doesn't.

I'm on opensolaris, I can get you dtrace logs of what's going on, if there's something you want me to check out.  It's no problem, I'm happy to help.

Also, at larger numbers, say 100-ish, it dies out with this:
No Packet on queue during CsmaNetDevice::TransmitAbort()

Donno if that helps.

*** Bug 495 has been marked as a duplicate of this bug. ***

Created attachment 381 [details]
patch to fix

This is actually a carry-over of bug 451, which seems to have been the wrong way to fix this problem.

A single WaitReply timer is maintained in an ArpCache, and it fires every WaitReplyTimeout seconds.  The attribute documentation says it should behave as follows:
    .AddAttribute ("WaitReplyTimeout",                   "When this timeout expires, the cache entries will be scanned and entries in WaitReply state will resend ArpRequest unless MaxRetries has been exceeded, in which case the entry is marked dead",

However, the current code with the IsExpiring() check will suppress the first retransmission in cases in which there are multiple entries in WaitReply state in the cache.

If the IsExpiring() check is deleted, then the behavior will align with the attribute documentation.  What this means is that if there is already an entry in WaitReply state in the cache for another destination, the first retransmission of a new arp request (for a new destination) will occur sometime between 0 and WaitReplyTimeout seconds, depending on when the original request occurred.  Subsequent retransmissions will occur on the WaitReplyTimeout intervals.  

Regression traces do not change for this patch.

I would like to push this tomorrow if there are no further comments.

changeset 724c9442a9f0