Your 1st traceroute shows loss happening within the Los Angeles region possibly within Level3 (hop #8 from the look of it, and it looks like multipathing is involved). Because traceroutes going only one direction are provided (due to the destination you chose and the nature of asymmetric routing), only half of the picture is shown, which is why I say "possibly". Your 2nd traceroute shows two things; - Loss showing up starting at hop #3 (possibly within Zayo/Abovenet space, again "possibly", re: asymmetric routing) - Loss showing up at hop #8 (could be an effect of something happening earlier in the path) For reference, here are your traceroutes: https://puck.nether.net/pipermail/outages/2014-March/006787.html So when you say "I think it's a problem with Tustin > LA", the 2nd traceroute you provided doesn't seem to support that theory, nor the statement "I am unable to reproduce it to any other network". Do you have any other tools available to you, like mtr? mtr won't narrow down where the issue is happening, but would make it more clear as to where the issue is happening, especially if you let it run for a while (say 10 minutes or so) from both places (you didn't provide source IPs for either traceroute, so that doesn't help either). -- | Jeremy Chadwick jdc@koitsu.org | | UNIX Systems Administrator http://jdc.koitsu.org/ | | Making life hard for others since 1977. PGP 4BD6C0CB | On Tue, Mar 11, 2014 at 10:00:59AM -0700, jd wrote:
I had done this earlier, forgot to paste it, current results from Tustin, CA to 4.2.2.2
I also have presence in One Wilshire and 800 S Hope, I don't think we see the level3 issue there.
I think the route has to do with Tustin > LA, every trace that has replied to this thread has been through an alternate route.
Thanks.
---- statistics ---- 50 packets transmitted, 43 packets received, 14% packet loss rtt min/avg/median/max/mdev/stddev = 16/18.698/20/28/1.503/2.69 ms
50 packets transmitted, 44 packets received, 12% packet loss rtt min/avg/median/max/mdev/stddev = 16/20.727/20/44/1.933/6.283 ms
On 3/11/2014 9:41 AM, Jeremy Chadwick wrote:
Does L3's LG come in handy for reproducing the issue? It might not given how packets flow (e.g. the loss is across a specific interface), but you might spend 15 minutes or so poking at it: