Thanks for sharing that info. That lines up with us seeing HTTPv6 alarms clearing at 10:31 am Central, too. -----Original Message----- From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Andree Toonk Sent: Tuesday, May 07, 2013 12:19 PM To: outages@outages.org Subject: Re: [outages] Centurylink nationwide outage While going over some of our BGP data I noticed significant instability for AS5668 CenturyTel Internet Holdings, Inc. See for example: http://portal.bgpmon.net/data/AS5668-outages-may7-2013.png Clearly visible are the routes flapping (few minute outages) between 08:06 UTC and 15:34. Below an example for 98.125.0.0/16 (CenturyTel Service Group, LLC) down for 3 minutes 2013-05-07 08:06 - 2013-05-07 08:09 down for 3 minutes 2013-05-07 08:25 - 2013-05-07 08:28 down for 3 minutes 2013-05-07 08:53 - 2013-05-07 08:56 down for 4 minutes 2013-05-07 09:04 - 2013-05-07 09:08 down for 4 minutes 2013-05-07 09:20 - 2013-05-07 09:24 down for 3 minutes 2013-05-07 10:05 - 2013-05-07 10:08 down for 5 minutes 2013-05-07 10:11 - 2013-05-07 10:16 down for 3 minutes 2013-05-07 10:56 - 2013-05-07 10:59 down for 3 minutes 2013-05-07 12:14 - 2013-05-07 12:17 down for 3 minutes 2013-05-07 12:34 - 2013-05-07 12:37 down for 4 minutes 2013-05-07 13:02 - 2013-05-07 13:06 down for 3 minutes 2013-05-07 13:12 - 2013-05-07 13:15 down for 3 minutes 2013-05-07 14:11 - 2013-05-07 14:14 down for 9 minutes 2013-05-07 14:24 - 2013-05-07 14:33 down for 3 minutes 2013-05-07 14:45 - 2013-05-07 14:48 down for 3 minutes 2013-05-07 14:55 - 2013-05-07 14:58 down for 3 minutes 2013-05-07 15:24 - 2013-05-07 15:27 down for 3 minutes 2013-05-07 15:31 - 2013-05-07 15:34 Cheers, Andree .-- My secret spy satellite informs me that at 2013-05-07 9:33 AM Frank Bulk wrote:
Sounds like it was scheduled maintenance gone bad: "Perrine said he spoke with CenturyLink at 6 a.m. where they advised him the company was doing scheduled maintenance. At 7:35 a.m. they told him something had gone wrong during the maintenance and it was affecting customers in 13 states."
http://www.tallahassee.com/article/20130507/NEWS/130507010/Centurylink-outag
e-affecting-local-internet-provider?nclick_check=1
I bite my tongue in regards to Network Tallahasee's singlehomedness and Perrine's comment on his questioning of the timing of the update.
I encourage all my competitors to stay single homed and to do their maintenance Sunday morning, while their vendors are at their lowest staffing levels.
Frank
-----Original Message----- From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Frank Bulk Sent: Tuesday, May 07, 2013 8:29 AM To: 'Marco Prechel'; outages@outages.org Subject: Re: [outages] Centurylink nationwide outage
I first saw timeouts with www.qwest.com over IPv6 at 3:08 am Central -- I was wondering why, now I know. =)
Frank
-----Original Message----- From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Marco Prechel Sent: Tuesday, May 07, 2013 7:33 AM To: outages@outages.org Subject: [outages] Centurylink nationwide outage
Saw our Centurylink Ethernet circuit lose L3 connectivity in SWFL around 0400 EDT. ANS/ABS states it's a nationwide routing issue. No ETR. _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages