
I know customers on Time Warner Cable's fiber and cable (coax) service had problems from about 9:05 to 9:20 EST this morning. Had anyone else seen issues? I wonder if other ISP's were effected, such as CenturyLink (a customer that contracted me has this as a backup, another primarily). Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373

TWC updated me, core router from Chicago to New York crashed. Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373 On Mon, Nov 7, 2011 at 9:33 AM, Josh Luthman <josh@imaginenetworksllc.com> wrote:
I know customers on Time Warner Cable's fiber and cable (coax) service had problems from about 9:05 to 9:20 EST this morning. Had anyone else seen issues?
I wonder if other ISP's were effected, such as CenturyLink (a customer that contracted me has this as a backup, another primarily).
Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373

Level 3 outage per our NOC. Zeek Bower -----Original Message----- From: outages-bounces@outages.org [mailto:outages-bounces@outages.org] On Behalf Of Josh Luthman Sent: Monday, November 07, 2011 9:33 AM To: outages@outages.org Subject: [outages] Time Warner Cable outage I know customers on Time Warner Cable's fiber and cable (coax) service had problems from about 9:05 to 9:20 EST this morning. Had anyone else seen issues? I wonder if other ISP's were effected, such as CenturyLink (a customer that contracted me has this as a backup, another primarily). Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373 _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages ********************************************************************** This e-mail message and any attachments contain confidential information from Medco. If you are not the intended recipient, you are hereby notified that disclosure, printing, copying, distribution, or the taking of any action in reliance on the contents of this electronic information is strictly prohibited. If you have received this e-mail message in error, please immediately notify the sender by reply message and then delete the electronic message and any attachments.

On that note we also saw issues this morning related to Level 3 "issues" this morning. Specifically DNS resolution for US Blackberry SRP servers an some VPN tunnels. On Mon, Nov 7, 2011 at 1:25 PM, Bower, Zeek <Zeek_Bower@medco.com> wrote:
Level 3 outage per our NOC.
Zeek Bower
-----Original Message----- From: outages-bounces@outages.org [mailto:outages-bounces@outages.org] On Behalf Of Josh Luthman Sent: Monday, November 07, 2011 9:33 AM To: outages@outages.org Subject: [outages] Time Warner Cable outage
I know customers on Time Warner Cable's fiber and cable (coax) service had problems from about 9:05 to 9:20 EST this morning. Had anyone else seen issues?
I wonder if other ISP's were effected, such as CenturyLink (a customer that contracted me has this as a backup, another primarily).
Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373 _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
********************************************************************** This e-mail message and any attachments contain confidential information from Medco. If you are not the intended recipient, you are hereby notified that disclosure, printing, copying, distribution, or the taking of any action in reliance on the contents of this electronic information is strictly prohibited. If you have received this e-mail message in error, please immediately notify the sender by reply message and then delete the electronic message and any attachments.
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages

We also saw a level3 outage this morning around 7:05 MST. -----Original Message----- From: outages-bounces@outages.org [mailto:outages-bounces@outages.org] On Behalf Of Bower, Zeek Sent: Monday, November 07, 2011 11:26 AM To: Josh Luthman; outages@outages.org Subject: Re: [outages] Time Warner Cable outage Level 3 outage per our NOC. Zeek Bower -----Original Message----- From: outages-bounces@outages.org [mailto:outages-bounces@outages.org] On Behalf Of Josh Luthman Sent: Monday, November 07, 2011 9:33 AM To: outages@outages.org Subject: [outages] Time Warner Cable outage I know customers on Time Warner Cable's fiber and cable (coax) service had problems from about 9:05 to 9:20 EST this morning. Had anyone else seen issues? I wonder if other ISP's were effected, such as CenturyLink (a customer that contracted me has this as a backup, another primarily). Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373 _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages ********************************************************************** This e-mail message and any attachments contain confidential information from Medco. If you are not the intended recipient, you are hereby notified that disclosure, printing, copying, distribution, or the taking of any action in reliance on the contents of this electronic information is strictly prohibited. If you have received this e-mail message in error, please immediately notify the sender by reply message and then delete the electronic message and any attachments. _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages

Witnessed an outage at 8 CST from upstream. Sounded like someone was broadcasting a BGP route with an invalid attribute and caused BGP session restarts or Core Dumps/crashes on any Juniper router running version 10.4 -Dustin Rhodes -----Original Message----- From: outages-bounces@outages.org [mailto:outages-bounces@outages.org] On Behalf Of Bower, Zeek Sent: Monday, November 07, 2011 12:26 PM To: Josh Luthman; outages@outages.org Subject: Re: [outages] Time Warner Cable outage Level 3 outage per our NOC. Zeek Bower -----Original Message----- From: outages-bounces@outages.org [mailto:outages-bounces@outages.org] On Behalf Of Josh Luthman Sent: Monday, November 07, 2011 9:33 AM To: outages@outages.org Subject: [outages] Time Warner Cable outage I know customers on Time Warner Cable's fiber and cable (coax) service had problems from about 9:05 to 9:20 EST this morning. Had anyone else seen issues? I wonder if other ISP's were effected, such as CenturyLink (a customer that contracted me has this as a backup, another primarily). Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373 _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages ********************************************************************** This e-mail message and any attachments contain confidential information from Medco. If you are not the intended recipient, you are hereby notified that disclosure, printing, copying, distribution, or the taking of any action in reliance on the contents of this electronic information is strictly prohibited. If you have received this e-mail message in error, please immediately notify the sender by reply message and then delete the electronic message and any attachments. _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages

Most likely related to this: http://www.theregister.co.uk/2011/11/07/global_net_outage/ - ferg On Mon, Nov 7, 2011 at 10:25 AM, Bower, Zeek <Zeek_Bower@medco.com> wrote:
Level 3 outage per our NOC.
Zeek Bower
-----Original Message----- From: outages-bounces@outages.org [mailto:outages-bounces@outages.org] On Behalf Of Josh Luthman Sent: Monday, November 07, 2011 9:33 AM To: outages@outages.org Subject: [outages] Time Warner Cable outage
I know customers on Time Warner Cable's fiber and cable (coax) service had problems from about 9:05 to 9:20 EST this morning. Had anyone else seen issues?
I wonder if other ISP's were effected, such as CenturyLink (a customer that contracted me has this as a backup, another primarily).
Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373 _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
********************************************************************** This e-mail message and any attachments contain confidential information from Medco. If you are not the intended recipient, you are hereby notified that disclosure, printing, copying, distribution, or the taking of any action in reliance on the contents of this electronic information is strictly prohibited. If you have received this e-mail message in error, please immediately notify the sender by reply message and then delete the electronic message and any attachments.
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
-- "Fergie", a.k.a. Paul Ferguson fergdawgster(at)gmail.com

Level 3 had a Big problem this morning at that time (sorry no TKT#). Our Internet access was affected - They confirmed they had a problem, but did not know exactly what it was. -----Original Message----- From: outages-bounces@outages.org [mailto:outages-bounces@outages.org] On Behalf Of Josh Luthman Sent: Monday, November 07, 2011 9:33 AM To: outages@outages.org Subject: [outages] Time Warner Cable outage I know customers on Time Warner Cable's fiber and cable (coax) service had problems from about 9:05 to 9:20 EST this morning. Had anyone else seen issues? I wonder if other ISP's were effected, such as CenturyLink (a customer that contracted me has this as a backup, another primarily). Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373 _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages ********************************************************************** This e-mail message and any attachments contain confidential information from Medco. If you are not the intended recipient, you are hereby notified that disclosure, printing, copying, distribution, or the taking of any action in reliance on the contents of this electronic information is strictly prohibited. If you have received this e-mail message in error, please immediately notify the sender by reply message and then delete the electronic message and any attachments.

Much talked about in the news this morning -- just Google it. Frank -----Original Message----- From: outages-bounces@outages.org [mailto:outages-bounces@outages.org] On Behalf Of Josh Luthman Sent: Monday, November 07, 2011 8:33 AM To: outages@outages.org Subject: [outages] Time Warner Cable outage I know customers on Time Warner Cable's fiber and cable (coax) service had problems from about 9:05 to 9:20 EST this morning. Had anyone else seen issues? I wonder if other ISP's were effected, such as CenturyLink (a customer that contracted me has this as a backup, another primarily). Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373 _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages

Hi Josh, I saw an outage on TWC this morning in the Raleigh NC area, same customer has a century link T that was fine. Have seen no other complaints.... Bryan Riley -----Original Message----- From: outages-bounces@outages.org [mailto:outages-bounces@outages.org] On Behalf Of Josh Luthman Sent: Monday, November 07, 2011 9:33 AM To: outages@outages.org Subject: [outages] Time Warner Cable outage I know customers on Time Warner Cable's fiber and cable (coax) service had problems from about 9:05 to 9:20 EST this morning. Had anyone else seen issues? I wonder if other ISP's were effected, such as CenturyLink (a customer that contracted me has this as a backup, another primarily). Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373 _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages

On Mon, Nov 07, 2011 at 06:33:29AM -0800, Josh Luthman wrote:
I know customers on Time Warner Cable's fiber and cable (coax) service had problems from about 9:05 to 9:20 EST this morning. Had anyone else seen issues?
I wonder if other ISP's were effected, such as CenturyLink (a customer that contracted me has this as a backup, another primarily).
My home internet connection (TWC Coax) was impacted this morning at about the same time you indicate. There is a thread on NANOG discussing[1]. As someone else already replied, looks like a Level3 issue. Ray [1] http://markmail.org/message/zetqzguq2vudvh2e

We had an interesting issue with AS209. When the Level(3) problem occurred, our BGP session with them started hold time expiring every 2 - 10 minutes, and horrendous packet loss to our advertised prefix. Finally got a human around 1830 UTC with access to their router, he claimed he did nothing yet it conveniently fixed itself after we received cease from their end. Now they're blaming local loop even though we saw 0% loss when sourcing pings from the interface IP assigned by them. On Nov 7, 2011, at 12:39, Josh Luthman <josh@imaginenetworksllc.com> wrote:
I know customers on Time Warner Cable's fiber and cable (coax) service had problems from about 9:05 to 9:20 EST this morning. Had anyone else seen issues?
I wonder if other ISP's were effected, such as CenturyLink (a customer that contracted me has this as a backup, another primarily).
Josh Luthman Office: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373 _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
participants (11)
-
Bower, Zeek
-
Bryan Riley
-
Chris Burwell
-
Christopher J. Pilkington
-
Dustin Rhodes
-
Frank Bulk
-
Geoffrey Mina
-
Johnson, Robert P.
-
Josh Luthman
-
Paul Ferguson
-
Ray Van Dolson