February 2010 - Outages - Outages Lists

Re: [outages] RESOLVED Re: AT&T issues?
by Scott Weeks 22 Feb '10

22 Feb '10

----- Original Message ---- Seeing major issues with AT&T in Hawaii. Anyone out there seeing it too? 42 routes instead of the full table. voice, etc... -------------------------------------- It turns out that is within the State of Hawaii only. ----------------------------------------------- One last post as it turns out that there're some folks here asking. It's just a couple of Gbps down: "AT&T fiber cut between Makaha & Pearl City, 43 DS3s down. Splicer's are onsite working on 23 of the DS3s according to priority. No ETR at this time." scott -------------- --------------- ----------------

1 0

RESOLVED Re: AT&T issues?
by Scott Weeks 22 Feb '10

22 Feb '10

----- Original Message ---- From: Scott Weeks <surfer(a)mauigateway.com> Seeing major issues with AT&T in Hawaii. Anyone out there seeing it too? 42 routes instead of the full table. voice, etc... -------------------------------------- It turns out that is within the State of Hawaii only. scott -------------- --------------- ----------------

1 0

AT&T issues?
by Scott Weeks 22 Feb '10

22 Feb '10

Seeing major issues with AT&T in Hawaii. Anyone out there seeing it too? 42 routes instead of the full table. voice, etc... scott

2 1

Fiber cut in NYC metro area affecting AboveNet dark fiber
by Jason Shoemaker 21 Feb '10

21 Feb '10

We have a dark fiber line from AboveNet between downtown Manhattan and Newark, NJ that went down as of 9:37 PM EST. According to AboveNet's NMC, they are aware of a fiber cut affecting multiple customers in the NYC metro area. Tickets have already been generated for affected customers. A third party is currently working in the location of the cut, and AboveNet is currently waiting on the third party for access to the site in order to diagnose and repair the issue. -Jason

2 1

Time Warner Cable Dayton Ohio outage
by Josh Luthman 20 Feb '10

20 Feb '10

Customers with Time Warner Cable on dedicated access (fiber) in the Dayton area are all down. No ETR. Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373 “Success is not final, failure is not fatal: it is the courage to continue that counts.” --- Winston Churchill

1 1

facebook issues
by Wes Young 20 Feb '10

20 Feb '10

http://twitter.com/#search?q=facebook -- 200 tweets / min describing a face-book outage. [it's kinda phunny] -- Wes http://claimid.com/wesyoung

8 7

ns12/ns13.verio.net fallen off the net?
by Gert Doering 19 Feb '10

19 Feb '10

Hi, one of my collagues complained about selfhtml.org unreachability, so I checked, and what I found is disturbing: ;; ANSWER SECTION: selfhtml.org. 82408 IN NS ns12.verio.de. selfhtml.org. 82408 IN NS ns13.verio.de. ;; ADDITIONAL SECTION: ns12.verio.de. 86259 IN A 213.198.55.91 ns13.verio.de. 86259 IN A 213.198.65.162 Cisco-M-XXX>sh ip b 213.198.55.91 % Network not in table Cisco-M-XXX>sh ip b 213.198.65.192 % Network not in table Verified at route-views.oregon-ix.net, so it's not a local problem in .DE Does anybody know more? gert -- USENET is *not* the non-clickable part of WWW! //www.muc.de/~gert/ Gert Doering - Munich, Germany gert(a)greenie.muc.de fax: +49-89-35655025 gert(a)net.informatik.tu-muenchen.de

3 6

Connectivity problems between US and India
by Carl Perry 19 Feb '10

19 Feb '10

We've been seeing a lot of customers call in with reports of slow access to their servers in the US. Traceroutes back to them all yield round-trip times of 300ms once the connection gets across the ocean. Anyone have any leads on a root cause? 4. 12.87.122.245 0% 9 9 9 9 10 11 5. cr2.hs1tx.ip.att.net 0% 9 9 46 46 46 46 6. cr1.dlstx.ip.att.net 0% 9 9 47 46 47 47 7. cr2.dlstx.ip.att.net 0% 9 9 47 46 47 47 8. cr2.la2ca.ip.att.net 0% 9 9 46 45 46 46 9. gar4.lsrca.ip.att.net 0% 9 9 47 47 47 48 10. 12.86.240.10 0% 9 9 46 45 46 49 11. segment-124-7.sify.net 0% 8 9 269 268 269 269 12. segment-119-227.sify.net 0% 8 9 268 267 268 271 13. segment-119-227.sify.net 0% 8 8 269 267 268 271 14. segment-119-227.sify.net 0% 8 8 270 269 271 274 15. segment-119-227.sify.net 0% 8 8 270 268 270 276 16. segment-119-227.sify.net 0% 8 8 271 269 270 271 17. 210.18.13.233.sify.net 0% 8 8 268 268 268 269 18. segment-124-30.sify.net 0% 8 8 294 284 292 299 4. 12.87.122.245 0% 11 11 10 9 10 11 5. cr2.hs1tx.ip.att.net 0% 11 11 16 16 16 17 6. cr1.dlstx.ip.att.net 0% 11 11 16 16 16 17 7. ggr7.dlstx.ip.att.net 0% 11 11 170 15 56 170 8. 192.205.36.42 0% 11 11 15 15 22 75 9. customer-side-bharti-4-pal6.pal.seabone.net 0% 10 11 286 286 287 287 10. 125.21.167.114 0% 10 10 285 285 285 285 11. 61.95.240.130 0% 10 10 283 283 283 286 12. abts-tn-static-248.253.246.61.airtelbroadband.in 0% 10 10 290 287 290 294

5 4

Re: [outages] ns12/ns13.verio.net fallen off the net?
by Jason L. Sparks 19 Feb '10

19 Feb '10

Long week? ------Original Message------ From: Larry Sheldon Sender: outages-bounces(a)outages.org To: outages(a)outages.org Subject: Re: [outages] ns12/ns13.verio.net fallen off the net? Sent: Feb 19, 2010 10:54 AM On 2/19/2010 2:45 AM, Gert Doering wrote: > Hi, > > one of my collagues complained about selfhtml.org unreachability, so I > checked, and what I found is disturbing: > > ;; ANSWER SECTION: > selfhtml.org. 82408 IN NS ns12.verio.de. > selfhtml.org. 82408 IN NS ns13.verio.de. > > ;; ADDITIONAL SECTION: > ns12.verio.de. 86259 IN A 213.198.55.91 > ns13.verio.de. 86259 IN A 213.198.65.162 > > Cisco-M-XXX>sh ip b 213.198.55.91 > % Network not in table > Cisco-M-XXX>sh ip b 213.198.65.192 > % Network not in table > > Verified at route-views.oregon-ix.net, so it's not a local problem in .DE > > Does anybody know more? I'm a little confused, not that that matters much. In some places you say "verio.net", in others "verio.de" Similarly, you are looking, in one place, at "213.198.65.162", another, "213.198.65.192". I wonder if part of the problem is a general sloppiness in the trouble-shooting. -- "Government big enough to supply everything you need is big enough to take everything you have." Remember: The Ark was built by amateurs, the Titanic by professionals. Requiescas in pace o email Ex turpi causa non oritur actio Eppure si rinfresca ICBM Targeting Information: http://tinyurl.com/4sqczs http://tinyurl.com/7tp8ml _______________________________________________ outages mailing list outages(a)outages.org https://puck.nether.net/mailman/listinfo/outages Sent from my Verizon Wireless BlackBerry

1 0

Re: [outages] Trans-Atlantic connectivity outages?
by Carl Perry 19 Feb '10

19 Feb '10

It doesn't look isolated to India any more either. Our internal links between our US Data Centers and out London Data Centers are showing a 120ms increase in latency as well. Mind you, the latency to India is as high as 300ms so I'm wondering if there are two outages... On 2/19/10 8:27 AM, Bradley Bopp wrote: > Re: [outages] Trans-Atlantic connectivity outages? Hello Carl, > > I take a slightly different path than you but we have had tickets > since yesterday regarding this. We have a major customer with > operations in India that have been unable to RDP or VPN to there > systems here for about 24 hours. > > This is were it seems to break; > > 13. xe-11-2-0-xcr1.fra.cw.net 0.0% 10 116.6 127.2 116.6 > 174.4 18.3 > 14. ge-2-0-0-dcr2.fra.cw.net 0.0% 10 115.9 151.2 115.9 > 314.2 63.7 > 15. bharti-gw2.fra.cw.net 10.0% 10 292.3 291.6 287.8 > 301.8 4.5 > 16. 59.145.7.134 10.0% 10 289.0 293.7 289.0 > 303.8 4.5 > 17. 61.95.240.130 10.0% 10 288.0 294.5 288.0 > 298.2 3.3 > 18. ABTS-TN-Static-248.253.246.6 10.0% 10 306.9 295.8 290.8 > 306.9 4.7 > M > > I am not sure who the Fiber Carrier is out to this point I checked > with HA and it's not there fiber. I doubt CW built this leg. > > If we can determine the carrier perhaps we can find out if there is > a Trans-Atlantic issue. > > > > > On 2/19/10 9:04 AM, "Carl Perry" <caperry(a)edolnx.net> wrote: > > We've been seeing a lot of customers call in with reports of > slow access > to their servers in the US. Traceroutes back to them all yield > round-trip times of 300ms once the connection gets across the ocean. > Anyone have any leads on a root cause? > > 4. > 12.87.122.245 > > > > 0% 9 9 9 9 10 11 > 5. > cr2.hs1tx.ip.att.net > > > > 0% 9 9 46 46 46 46 > 6. > cr1.dlstx.ip.att.net > > > > 0% 9 9 47 46 47 47 > 7. > cr2.dlstx.ip.att.net > > > > 0% 9 9 47 46 47 47 > 8. > cr2.la2ca.ip.att.net > > > > 0% 9 9 46 45 46 46 > 9. > gar4.lsrca.ip.att.net > > > > 0% 9 9 47 47 47 48 > 10. > 12.86.240.10 > > > > 0% 9 9 46 45 46 49 > 11. > segment-124-7.sify.net > > > > 0% 8 9 269 268 269 269 > 12. > segment-119-227.sify.net > > > > 0% 8 9 268 267 268 271 > 13. > segment-119-227.sify.net > > > > 0% 8 8 269 267 268 271 > 14. > segment-119-227.sify.net > > > > 0% 8 8 270 269 271 274 > 15. > segment-119-227.sify.net > > > > 0% 8 8 270 268 270 276 > 16. > segment-119-227.sify.net > > > > 0% 8 8 271 269 270 271 > 17. > 210.18.13.233.sify.net > > > > 0% 8 8 268 268 268 269 > 18. > segment-124-30.sify.net > > > > 0% 8 8 294 284 292 299 > > > 4. > 12.87.122.245 > > > > 0% 11 11 10 9 10 11 > 5. > cr2.hs1tx.ip.att.net > > > > 0% 11 11 16 16 16 17 > 6. > cr1.dlstx.ip.att.net > > > > 0% 11 11 16 16 16 17 > 7. > ggr7.dlstx.ip.att.net > > > > 0% 11 11 170 15 56 170 > 8. > 192.205.36.42 > > > > 0% 11 11 15 15 22 75 > 9. > customer-side-bharti-4-pal6.pal.seabone.net > > > > 0% 10 11 286 286 287 287 > 10. > 125.21.167.114 > > > > 0% 10 10 285 285 285 285 > 11. > 61.95.240.130 > > > > 0% 10 10 283 283 283 286 > 12. > abts-tn-static-248.253.246.61.airtelbroadband.in > > > > 0% 10 10 290 287 290 294 > > ---------------------------------------------------------------------- _______________________________________________ outages mailing list outages(a)outages.org https://puck.nether.net/mailman/listinfo/outages > Bradley D. Bopp > Director Of Engineering > http://www.nationalnet.com > http://as22384.peeringdb.net > NationalNet is committed to the highest level of Customer Service > available in the Web Hosting business. If you have any questions, > comments or concerns feel free to contact us at 678-247-7000 ext 1 > (or toll-free, 888-4-NATNET).

2 1