RE: [Nolug] Cox Connection Dropping - Packet Loss Topic

From: John Souvestre <johns_at_sstar.com>
Date: Tue, 12 Oct 2004 17:12:16 -0500
Message-ID: <00fa01c4b0a8$88e6d500$6401a8c0@JohnS>

Hi Scott.

I have no experience with the CMTS's. With Cisco routers they too will not
respond to pings for a second or so every minute while they are redoing the
route calculations. That's about the only thing that I've seen stop a Cisco
from responding, unless it is overloaded. Indeed, if you fast ping (say 0.2
sec) a Cisco router running BGP you can generally make out the 61 or 62 sec
"footprint".

Of course you are right - this doesn't stop them from passing traffic
(including ICMP) through, so it's OK.

I did run across a case about a year ago where it was a problem, however.
Every 62 sec's or so packets would get blocked from going through a particular
router to everything behind it. It took about 2 weeks to find the cause.

It turns out that a buffer on an interface was sized 10 times too small and
would drop characters when the router got busy (typically during routing calc
time). This was because the default setting was off by a factor of 10. So it
was really a Cisco IOS bug, not a configuration error.

Charles - this might sound familiar. It was on the Cox backbone link between
here and Atlanta. :-) PingPlotter showed it clear as a bell. All the loss
indicators lined up in a row every 62 sec after that router. But my link
(between me and Cox backbone) at the time wouldn't believe it. I had to do
FTP tests and show them that a FTP would stop for about 1 sec every 62 seconds
before they would believe me. The FTP hash marks looked somewhat steady,
because they are somewhat buffered. But NetPerf showed the data on the line
coming to a complete stop.

Now they somewhat believe in PingPlotter too. :-)

It was actually a problem to find files big enough to FTP for 1 or 2 minutes
at a time on a 40M link. If memory serves, it only takes about 1.5 minutes to
grab a 600M Linux ISO file, which is what we ended up testing with.

There! I somehow managed to work Linux back into the discussion. <g>

John

    John Souvestre - Southern Star - (504) 888-3348 - www.sstar.com

-----Original Message-----
From: owner-nolug@joeykelly.net [mailto:owner-nolug@joeykelly.net] On Behalf
Of Scott Harney
Sent: Tuesday, October 12, 2004 3:34 PM
To: nolug@joeykelly.net
Subject: Re: [Nolug] Cox Connection Dropping - Packet Loss Topic

John Souvestre wrote:
> Making ICMP low priority doesn't create packet loss unless the router is
> topped out or close to it. But I agree with you that as long as it passes
> ICMP, even though it doesn't respond itself, this is good enough.

It's not uncommon for Cisco CMTS's to drop ICMP pings -- even when not
overloaded (ie processor, or network). Of course transit ICMP to
destinations other than the CMTS go through fine so as I say, you'll
traces that show loss at the first hop, but no loss on subsequent hops
after that.

> One of the nice features of some of the various combo ping/traceroute
programs
> is that they attempt to ping all the hops in a route in very quick
succession.
> This lets you see the effect of a busy router blocking pings to hops behind
> it, even if it is sporadic. Note that it isn't the number of pings per hop
> which is fast, but that when 1 hop is pinged all are (at nearly the same
> time).

that's what mtr does for you. Similar to ping plotter but for *nix :)

so if our user pings his first hop and sees loss, it may be meaningless.
  But if he pings say, yahoo.com, and sees significant loss over several
pings, he needs to look closer using tools like mtr or ping plotter to
see where that loss starts. Maybe it starts right at the first hop or
at the ISP's edge and naturally gets worse. Or maybe it starts further
out at an apparent peering point....

> While the Cox article you mentioned is generally good, they are wrong in one
> respect. No core router should ever be configured to block all ICMP. It's
> bad enough tat many firewalls do this, but for a core router to do so would
be
> unforgivable!

Agreed. I think the author meant to say edge routers. Lots of people
filter ICMP at or near their edge. So it's not uncommon to see latter
hops in a traceroute fail.

Not sure how this is relevant to linux anymore except the discussion of
network analysis tools common to our environment of choice. I encourage
folks to explore traceroute, fping, hping, and mtr.

-- 
Scott Harney <scotth@scottharney.com>
"Asking the wrong questions is the leading cause of wrong answers"
gpg key fingerprint=7125 0BD3 8EC4 08D7 321D CEE9 F024 7DA6 0BC7 94E5
___________________
Nolug mailing list
nolug@nolug.org
___________________
Nolug mailing list
nolug@nolug.org
Received on 10/12/04

This archive was generated by hypermail 2.2.0 : 12/19/08 EST