It looks like the situation has resolved itself in the last minute - I'm still waiting for an update from Level 3, but from 2 off-list replies it looks like there were a few people who've experienced routing issues with Level 3 in Houston today. Regards, Chris Boulton Bigcommerce Email: chris@bigcommerce.com Web: http://www.bigcommerce.com/ On Fri, Jan 25, 2013 at 6:45 AM, Chris Boulton <chris@bigcommerce.com>wrote:
Ah, my apologies on the confusion. They're two distinct paths out, not traceroutes from each end.
I chose not to include the destination IP because the issues we're seeing a more wide spread than to a single IP.
Level 3 managed to get back to us - they were originally reporting back to us that they are experiencing connectivity issues for some of their clients in Austin, which they're now denying and still continuing to investigate.
Regards,
Chris Boulton Bigcommerce
Email: chris@bigcommerce.com
Web: http://www.bigcommerce.com/
On Fri, Jan 25, 2013 at 6:39 AM, Jeremy Chadwick <jdc@koitsu.org> wrote:
Chris, is there some reason you're not providing actual source/destination IPs for each of your mtrs?
The reason I'm asking is because I was left to believe what you were providing was both ends of an asymmetric connection (i.e. mtr #1 was for x.x.x.x to y.y.y.y, mtr #2 was for y.y.y.y to x.x.x.x).
However, your 2nd paragraph below ("we're seeing issues over one connections but not another") seems to indicate what you provided are two mtrs across completely different circuits.
So which is it?
-- | Jeremy Chadwick jdc@koitsu.org | | UNIX Systems Administrator http://jdc.koitsu.org/ | | Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB |
On Fri, Jan 25, 2013 at 06:14:05AM +1100, Chris Boulton wrote:
Thanks, I am aware of what the traceroute shows and how a traceroute works. The intent of the traceroute was primarily to show where I see our paths between Level 3 and TW Telecom converge.
Plain and simple - we are experiencing connectivity issues over our Level 3 connection, which we're not seeing over our TW Telecom connection for traffic destined to the same location.
Regards,
Chris Boulton Bigcommerce
Email: chris@bigcommerce.com
Web: http://www.bigcommerce.com/
On Fri, Jan 25, 2013 at 6:11 AM, Jeremy Chadwick <jdc@koitsu.org> wrote:
Please see my below post on this matter, which I posted only 2 days ago:
https://puck.nether.net/pipermail/outages/2013-January/005111.html
What you've shown in your mtr does not show any issue of any kind with Level 3. The packet loss shown at hops 4/5/6 does not "trickle down" each subsequent hop (look at hop 7 onward for proof), nor is there any sign of latency increase either.
On the other hand, the 2nd mtr you provided shows some degree of packet loss starting at hop 2 (TW Telecom) which *does* "trickle down" through all subsequent hops (egress). If you want to know how many packets were lost, look at the --order flag to mtr (I recommend --order=LSRNABW).
Plain and simple: talk to TW Telecom about the loss in their network.
-- | Jeremy Chadwick jdc@koitsu.org | | UNIX Systems Administrator http://jdc.koitsu.org/ | | Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB |
On Fri, Jan 25, 2013 at 05:46:27AM +1100, Chris Boulton wrote:
Seeing connectivity issues as traffic passes through Level 3 in Dallas - not seeing the same issues for where routes converge (above.net in DFW) over our TW Telecom link.
Anybody else experiencing this as well?
Level 3:
Host Loss% Snt Last Avg Best Wrst StDev 1. vlan358.car1.Houston1.Level3.net 0.0% 167 105.9 80.5 31.7 276.4 59.8 2. ae-2-5.bar1.Houston1.Level3.net 0.0% 167 32.5 34.6 32.2 93.8 7.3 3. ae-13-13.ebr1.Dallas1.Level3.net 0.0% 167 39.5 37.4 36.7 41.2 0.9 4. ae-91-91.csw4.Dallas1.Level3.net 65.1% 167 44.5 43.5 39.7 53.8 4.0 5. ae-4-90.edge2.Dallas3.Level3.net 5.4% 166 37.1 37.9 36.1 75.6 5.1 6. ex-3-1-0.er1.dfw2.us.above.net 66.1% 166 39.5 43.5 38.9 104.8 12.5 7. xe-3-1-0.cr1.dfw2.us.above.net 0.0% 166 37.3 40.4 37.0 114.1 10.6 8. xe-0-0-0.er3.dfw2.us.above.net 0.0% 166 36.2 39.8 36.1 99.3 10.2
Host Loss% Snt Last Avg Best Wrst StDev 1. 50.58.18.1 0.0% 829 4.3 23.9 4.1 121.2 14.5 2. dal2-pr2-xe-0-3-0-0.us.twtelecom.net 0.1% 829 55.4 31.2 9.4 121.0 17.2 dal2-pr2-xe-2-3-0-0.us.twtelecom.net 3. xe-3-0-0.er2.dfw2.us.above.net 0.1% 829 44.8 31.6 9.5 143.7 17.9 4. xe-0-1-0.cr2.dfw2.us.above.net 0.2% 829 40.5 31.2 10.3 116.2 15.9 xe-3-0-0.cr2.dfw2.us.above.net 5. xe-0-2-0.er3.dfw2.us.above.net 0.2% 829 42.7 32.4 9.6 109.2 16.5
Regards,
Chris Boulton Bigcommerce
Email: chris@bigcommerce.com
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages