October 2018 - Outages - Outages Lists

Large Scale ATT Outage
by Tim Beers 30 Oct '18

30 Oct '18

We just saw a bunch of sites in the Midwest drop off the face of the Earth. Seemed large scale, no bgp from our peers on 7018

2 1

Vonage Enterprise (Telesphere) experiencing outage
by David Rubin 30 Oct '18

30 Oct '18

Multiple customers of mine using Vonage Enterprise are currently experiencing an outage. Phones are showing Offline. -- David Rubin Cubex Group

1 0

Verizon outage
by Sebastian J. Orsini II 29 Oct '18

29 Oct '18

Multiple reports of Verizon issues across the country. I lost something like 30 LTE modems. Circuit provider working with Verizon says they have over 1000 sites down, with reports of cell phones out as well. looks pretty bad for VZW

7 8

Cogent IP outage in Europe
by Chris Haycox (etherFAX) 29 Oct '18

29 Oct '18

Cogent is having a fairly broad outage in Europe at the moment, last 40 mins. Master ticket HD10421872, top item on http://status.cogentco.com/ We are seeing spotty routing to Deutsche Telekom from source IPs in the same advertised block - looks like a source/dest map and their techs say failed hardware. Chris

1 0

Re: [outages] GoDaddy account auth and some authoritative DNS servers down
by Cary Wiedemann 29 Oct '18

29 Oct '18

Even their WHOIS server whois.godaddy.com is down. This is awful. My TTLs are too short. - Cary On Sun, Oct 28, 2018 at 7:14 PM Cary Wiedemann <carywiedemann(a)gmail.com> wrote: > Nobody can log into their GoDaddy accounts right now. I also have two > authoritative DNS servers for a few of my domains not responding: > ns29.domaincontrol.com > ns30.domaincontrol.com > > Just spoke to GoDaddy support and they're aware of the problem and say it > started "45 minutes ago" as of 7:10pm EDT. They're giving all customers > that call in a 2 hour ETR, but that doesn't sound rooted in fact. The > phone staff can't even check accounts, it's a big outage. > > Fingers crossed it resolves soon, this could make for a terrible Monday > morning. > > - Cary >

3 2

GoDaddy account auth and some authoritative DNS servers down
by Cary Wiedemann 28 Oct '18

28 Oct '18

Nobody can log into their GoDaddy accounts right now. I also have two authoritative DNS servers for a few of my domains not responding: ns29.domaincontrol.com ns30.domaincontrol.com Just spoke to GoDaddy support and they're aware of the problem and say it started "45 minutes ago" as of 7:10pm EDT. They're giving all customers that call in a 2 hour ETR, but that doesn't sound rooted in fact. The phone staff can't even check accounts, it's a big outage. Fingers crossed it resolves soon, this could make for a terrible Monday morning. - Cary

1 0

Lightower down NY
by Sulaman, Muhammad 25 Oct '18

25 Oct '18

--------------------------- This Message was Encrypted. --------------------------- Anyone have issues with Lightower/Crowncastle services in NY? We are seeing massive outages affecting 20 facilities. Cheers, Muhammad Sulaman IT Director Institute for Family Health 2006 Madison Ave | New York NY | 10035 D: 917.999.4910 C: 917.566.6746 Helpdesk: 212.675.3558 msulaman(a)institute.org www.institute.org

3 3

Level 3 fiber cut San Bernardino, CA
by Brian Knight 25 Oct '18

25 Oct '18

Got a Level 3 fiber cut in San Bernardino, CA going on for about 17 hours now. Our first link alarm was at 3:47pm CT yesterday. $DAYJOB has two 1G circuits affected. Looks like they had a hell of a time getting the replacement cable and splice conduit installed. Splicing is estimated to start in another 2.5-3.5 hours on the east side of the cut. Here's the latest we received from Level 3. 10/24/2018 12:29:56 GMT - *** CASCADED EXTERNAL NOTES 24-Oct-2018 12:22:53 GMT From CASE: 15371425 - Event The Outside Plant Engineer has reported that cable preparations on both the west side and the east side of the cable is ongoing. It is estimated splicing on the west side will begin at 14:00 GMT and splicing on the east side will begin between 15:00 GMT and 16:00 GMT. 10/24/2018 11:10:34 GMT - *** CASCADED EXTERNAL NOTES 24-Oct-2018 11:04:06 GMT From CASE: 15371425 - Event The Outside Plant Engineer has confirmed the trench is dug and the replacement cable is being ran between manholes at the west side of the damage. Preparations are complete at the east side and splicing expected to begin at 11:30 GMT with the 264 count cable, then the 84 count cable followed by a 60 count lateral cable. Restoration efforts are ongoing with an ETTR that remains unchanged, 17:30 GMT. 10/24/2018 10:05:30 GMT - *** CASCADED EXTERNAL NOTES 24-Oct-2018 09:58:52 GMT From CASE: 15371425 - Event The Outside Plant Engineer reported the replacement section of cable got stuck when pulling it into position. A second splice enclosure will be installed in the next manhole. Field Operations will dig a trench between the two enclosures to complete the installation of the 2,400 foot section of 264 count replacement cable. The trench is expected to be complete at approximately 10:30 GMT with an estimated time splicing will begin of 12:30 GMT. 10/24/2018 08:08:37 GMT - *** CASCADED EXTERNAL NOTES 24-Oct-2018 08:01:50 GMT From CASE: 15371425 - Event The Outside Plant Engineer reported the replacement section of fiber cable has arrived and is currently being pulled into position. Along with the section of fiber cable, a new splice enclosure is being prepped and will be replaced to complete repairs of the 264 count cable. The 84 count cable has been prepped and ready for splicing. Field Operations have moved to an adjacent manhole looking for available slack to repair to the 168 count cable. The Outside Plant Engineer reported splicing is expected to commence at 10:00 GMT. The next update will be sent at that time or if additional information is received. -Brian

2 3

Mediacom Outages
by Chris Cherry 23 Oct '18

23 Oct '18

Anyone have any info on Mediacom, service is out in Southern Illinois. Thanks, [image: Logo Small.png]*Chris Cherry* The Abstract & Title Companies *Systems Administrator* *(317) 745-4300* *ccherry(a)abstract-title.com <ccherry(a)abstract-title.com>* *71 West Marion Street* *Danville, IN 46122* *www.abstract-title.com <http://www.abstract-title.com>* [image: signature-1] -- **WARNING – FRAUDULENT FUNDING INSTRUCTIONS** Email hacking and fraud are on the rise to fraudulently misdirect funds. Please call your escrow officer immediately using contact information found from an independent source, such as the sales contract or internet, to verify any funding instructions received. We are not responsible for any wires sent by you to an incorrect bank account. *CONFIDENTIALITY NOTICE:* This e-mail transmission and any materials attached hereto are private and confidential and are the property of the sender. The information contained herein is privileged and is intended only for the use of the individual(s) or entity(ies) named above. If you are not the intended recipient, be advised that any unauthorized disclosure, copying, distribution, use or the taking of any action in reliance on this transmission is strictly prohibited. If you have received this transmission in error, please notify the sender immediately to arrange for its return.

3 4

Re: [outages] Correction -- Century Link and ATT was: Cyberlink and ATT
by Larry Sheldon 20 Oct '18

20 Oct '18

On 10/19/2018 21:10, Larry Sheldon via Outages wrote: > I have one report of an outage or outages involving Century Link and ATT in > Omaha. [edited] -- "Eppur si muove" From Larry's Cox account.

1 0