In the Orlando, FL area, via Level3, I'm unable to reach anything of google's (google.com, its NS's 216.239.32.10, 216.239.34.10, 216.239.36.10, 216.239.38.10, their 8.8.8.8 DNS service), puck.nether.net, bing.com, and probably lots of other things I haven't checked. Traces die at 146.137.69.4.in-addr.arpa name = ae-0-11.bar2.Orlando1.Level3.net. I'm seeing this both from work, where we have Level3 and TWTelecom, and from home on my BHN cable (which relies heavily on Level3 for transit). At work, shutting our BGP session to Level3 has resolved things...but I hate being singlehomed. Our NOC claimed they couldn't get through (busy signal) to Level3's NOC, so I just called L3 from my cell, got through, opened a ticket, and the guy I spoke with was unaware of any issues. Anyone else seeing this? ---------------------------------------------------------------------- Jon Lewis, MCP :) | I route Senior Network Engineer | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
I confirm this outage on time warner's network as well for over 2.5 hours, although it looks like it just came back up. MATT--- On Thu, Oct 13, 2011 at 4:36 AM, Jon Lewis <jlewis@lewis.org> wrote:
In the Orlando, FL area, via Level3, I'm unable to reach anything of google's (google.com, its NS's 216.239.32.10, 216.239.34.10, 216.239.36.10, 216.239.38.10, their 8.8.8.8 DNS service), puck.nether.net, bing.com, and probably lots of other things I haven't checked.
Traces die at 146.137.69.4.in-addr.arpa name = ae-0-11.bar2.Orlando1.Level3.net.
I'm seeing this both from work, where we have Level3 and TWTelecom, and from home on my BHN cable (which relies heavily on Level3 for transit). At work, shutting our BGP session to Level3 has resolved things...but I hate being singlehomed. Our NOC claimed they couldn't get through (busy signal) to Level3's NOC, so I just called L3 from my cell, got through, opened a ticket, and the guy I spoke with was unaware of any issues.
Anyone else seeing this?
---------------------------------------------------------------------- Jon Lewis, MCP :) | I route Senior Network Engineer | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________ _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
-- MATT--- ViciDial Open Source Call Center Suite http://www.vicidial.com
I didn't get the call from our NOC (who heard from Level3's) that things were supposed to be fixed until 0630. I turned our Level3 BGP back on, and verified that things were back to normal. Based on traces to what was broken, it looks like there was something wrong between Level3's Orlando and Miami POPs. On Thu, 13 Oct 2011, VICIDIAL wrote:
I confirm this outage on time warner's network as well for over 2.5 hours, although it looks like it just came back up.
MATT---
On Thu, Oct 13, 2011 at 4:36 AM, Jon Lewis <jlewis@lewis.org> wrote:
In the Orlando, FL area, via Level3, I'm unable to reach anything of google's (google.com, its NS's 216.239.32.10, 216.239.34.10, 216.239.36.10, 216.239.38.10, their 8.8.8.8 DNS service), puck.nether.net, bing.com, and probably lots of other things I haven't checked.
Traces die at 146.137.69.4.in-addr.arpa name = ae-0-11.bar2.Orlando1.Level3.net.
I'm seeing this both from work, where we have Level3 and TWTelecom, and from home on my BHN cable (which relies heavily on Level3 for transit). At work, shutting our BGP session to Level3 has resolved things...but I hate being singlehomed. Our NOC claimed they couldn't get through (busy signal) to Level3's NOC, so I just called L3 from my cell, got through, opened a ticket, and the guy I spoke with was unaware of any issues.
Anyone else seeing this?
---------------------------------------------------------------------- Jon Lewis, MCP :) | I route Senior Network Engineer | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________ _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
-- MATT---
ViciDial Open Source Call Center Suite http://www.vicidial.com
---------------------------------------------------------------------- Jon Lewis, MCP :) | I route Senior Network Engineer | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
----- Original Message -----
From: "Jon Lewis" <jlewis@lewis.org>
In the Orlando, FL area, via Level3, I'm unable to reach anything of google's (google.com, its NS's 216.239.32.10, 216.239.34.10, 216.239.36.10, 216.239.38.10, their 8.8.8.8 DNS service), puck.nether.net, bing.com, and probably lots of other things I haven't checked.
Traces die at 146.137.69.4.in-addr.arpa name = ae-0-11.bar2.Orlando1.Level3.net.
Tracing to the first IP from tamqflta, 400 N Tampa: manta:/home/jra # mtr -c 10 -r 216.239.32.10 HOST: manta Loss% Snt Last Avg Best Wrst StDev 1.|-- 173.241.205.130 0.0% 10 0.7 1.8 0.4 13.4 4.1 2.|-- gi2-4.border4.esnet.com 0.0% 10 0.5 0.5 0.3 0.8 0.1 3.|-- ge-6-20.car4.Tampa1.Level 0.0% 10 120.6 46.0 0.3 144.2 63.6 4.|-- ae-24-24.car2.Tampa1.Leve 0.0% 10 1.1 0.7 0.7 1.1 0.1 5.|-- ae-2-7.bar2.Tampa1.Level3 0.0% 10 64.6 10.1 0.6 64.6 21.4 6.|-- ae-12-12.ebr1.Dallas1.Lev 0.0% 10 24.2 24.5 24.2 26.4 0.7 7.|-- ae-91-91.csw4.Dallas1.Lev 0.0% 10 24.6 24.4 24.2 26.0 0.6 8.|-- ae-4-90.edge2.Dallas3.Lev 0.0% 10 24.5 26.3 24.5 41.8 5.4 9.|-- GOOGLE-INC.edge2.Dallas3. 0.0% 10 25.5 25.7 25.3 26.4 0.4 10.|-- 72.14.233.67 0.0% 10 25.4 33.0 25.3 97.4 22.6 11.|-- 72.14.237.217 0.0% 10 25.4 25.7 25.3 26.9 0.5 | `|-- 72.14.237.221 12.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 13.|-- 66.249.94.7 0.0% 10 27.8 28.3 27.7 31.3 1.1 | `|-- 66.249.94.23 14.|-- 72.14.239.127 0.0% 10 28.0 28.0 27.7 28.1 0.1 | `|-- 209.85.254.247 15.|-- 209.85.253.209 80.0% 10 33.2 37.0 33.2 40.7 5.4 16.|-- ns1.google.com 0.0% 10 28.6 28.5 28.4 28.8 0.1 IE: WFM. Cheers, -- jra -- Jay R. Ashworth Baylink jra@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA http://photo.imageinc.us +1 727 647 1274
participants (3)
-
Jay Ashworth -
Jon Lewis -
VICIDIAL