Just got an RFO from Level 3... A Core router in Atlanta crashed and failed over. But the redundant router didn't pass any traffic so they had to reload the router that crashed. They have opened a ticket with their hardware vendor. My tech that called wasn't as nosey as I wanted him to be so he didn't get any device and node information. But this sounds like Cisco Cisco and more Cisco issues. The GSR's are EOL so I bet it was one of them. Sean From: outages-bounces@outages.org [mailto:outages-bounces@outages.org] On Behalf Of Josh Luthman Sent: Tuesday, October 20, 2009 1:13 PM To: Jeremy Chadwick Cc: outages@outages.org Subject: Re: [outages] Level 3 down in Atlanta There are four routes into Atlanta, I believe. Getting to Atlanta wasn't a problem. Between the Atlanta hub and the customers was the problem. Can't give every customer two connections for the price of one =/ Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373 "When you have eliminated the impossible, that which remains, however improbable, must be the truth." --- Sir Arthur Conan Doyle On Tue, Oct 20, 2009 at 1:00 PM, Jeremy Chadwick <outages@jdc.parodius.com<mailto:outages@jdc.parodius.com>> wrote: On Tue, Oct 20, 2009 at 09:28:21AM -0700, Scott Howard wrote:
Looks like it's all back up as of about 30 mins ago.
Apparently either a core switch or router failed, which took down much of their network in Atlanta, as well as Memphis and Nashville. Level 3 has a single router or switch handling packets at a major POP? I doubt this, but the outage is confirmation something bad happened.
That said: where's the redundancy, and why didn't it kick in? -- | Jeremy Chadwick jdc@parodius.com<mailto:jdc@parodius.com> | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | _______________________________________________ outages mailing list outages@outages.org<mailto:outages@outages.org> https://puck.nether.net/mailman/listinfo/outages