September 2016 - Outages

Big fiber cut due to manhole(s) explosion(s) in NYC/Union Square
by Eric Dugas 30 Sep '16

30 Sep '16

We've had a transport link down since 09-26 02:39 EDT leaving Montreal going to Manhattan. The only source of news from this fiber cut is the carrier via its partner. I've found this piece of news<http://pix11.com/2016/09/26/manhole-explosion-causes-hundreds-to-lose-power…> that matches with the even but that's it. Updates from the master ticket are basically saying "we don't have access to the manholes because they need to be rebuilt", "can't re-route because other nearby conduits can't be jointed togheter" and "no ETR yet". I was wondering if someone nearby had any idea of what's really going. Eric Dugas edugas(a)zerofail.com Coordonnateur d'interconnexion | Peering coordinator ZEROFAIL<zerofail.com> | Openface<openface.com>

5 5

AT&T/XO issues - SFO area
by Sokhan, Yuriy 29 Sep '16

29 Sep '16

Anyone seeing high latency/drops around San Francisco area? We have an L3VPN circuit dropping for past 48 hours for up to 60 seconds at a time multiple times per day

3 2

Verizon Wireless Calls to NY/CT/NJ fail
by Peter Beckman 27 Sep '16

27 Sep '16

Verizon Wireless: I called a phone in NJ this morning, got a strange message: "Welcome to Verizon Wireless. Long Distance or International Dialing is not included with your service. To inquire about Long Distance or International Dialing, hang up and dial *611 from your wireless phone. Announcement 7 Location 117137" I called Verizon Wireless support and they are saying that due to some upgrades relating to LTE Advanced/Enhanced, since 9pm last night customers in those areas may not be able to make calls, and calls on Verizon Wireless TO these areas will get this misleading message. I was able to make calls to other non-local numbers without issue. Has anyone heard more about this? Can't find much on the web about it. The outage map shows a lot of complaints in and through the east coast corridor between DC and NYC, even toward Boston. http://downdetector.com/status/verizon/map/ Beckman --------------------------------------------------------------------------- Peter Beckman Internet Guy beckman(a)angryox.com http://www.angryox.com/ ---------------------------------------------------------------------------

1 1

CSpire Call Routing Issue
by Ahmed Jalal 26 Sep '16

26 Sep '16

We are not getting calls to our DID's on Level-3's network in Mississippi from CSpire's network at this time. Calls are being treated with a 404 not found probably due to a bad route. Is anyone from CSpire on this list who could look into this issue. I can share call examples and also contact whichever ULC calls are failing on at this time. Best Regards, Ahmed Jalal

1 0

Routing Loops in Singapore via Telstra/Level3
by Ryan Gelobter 26 Sep '16

26 Sep '16

Anyone else affected by these routing loops? According to Telstra they turned up a new link about 7 hours ago between them and Level3 and since then there has been routing loops. They turned the link down for an hour and that fixed it but now its back up and they are continuing to troubleshoot these issues. Example: traceroute to 103.254.44.93 (103.254.44.93), 30 hops max, 60 byte packets 1 104.131.0.254 (104.131.0.254) 12.342 ms 104.131.0.253 (104.131.0.253) 0.291 ms 104.131.0.254 (104.131.0.254) 12.193 ms 2 138.197.248.36 (138.197.248.36) 0.199 ms 0.228 ms 138.197.248.48 (138.197.248.48) 0.194 ms 3 nyk-b3-link.telia.net (62.115.45.9) 1.550 ms nyk-b3-link.telia.net (62.115.45.5) 1.457 ms nyk-b3-link.telia.net (62.115.45.9) 1.444 ms 4 nyk-bb1-link.telia.net (62.115.112.248) 0.999 ms nyk-bb1-link.telia.net (80.91.247.18) 14.412 ms nyk-bb1-link.telia.net (62.115.112.248) 1.020 ms 5 sjo-b21-link.telia.net (213.155.135.129) 75.181 ms sjo-b21-link.telia.net (213.248.80.1) 74.733 ms 74.789 ms 6 pacnet-ic-155719-sjo-b21.c.telia.net (80.239.195.30) 74.573 ms 74.267 ms 74.267 ms 7 te0-0-0-7.gw3.sjc1.10026.telstraglobal.net (202.147.50.189) 74.636 ms te0-0-0-7.gw4.sjc1.10026.telstraglobal.net (202.147.50.193) 74.828 ms 74.994 ms 8 te0-1-0-1.gw2.lax3.10026.telstraglobal.net (202.147.61.206) 87.601 ms te0-2-0-3.gw1.lax3.10026.telstraglobal.net (202.147.50.145) 84.794 ms te0-1-0-1.gw2.lax3.10026.telstraglobal.net (202.147.61.206) 87.652 ms 9 te0-1-0-1.gw2.lax3.10026.telstraglobal.net (202.147.61.206) 87.956 ms gi4-1-0.gw3.lax1.10026.telstraglobal.net (202.147.61.153) 84.492 ms te0-1-0-1.gw2.lax3.10026.telstraglobal.net (202.147.61.206) 88.172 ms 10 TenGigabitEthernet1-2.ar5.LAX2.gblx.net (64.215.81.145) 74.691 ms gi4-0-0.gw2.sin3.10026.telstraglobal.net (61.14.157.134) 247.127 ms TenGigabitEthernet1-2.ar5.LAX2.gblx.net (64.215.81.145) 74.571 ms 11 ae-2-7.edge1.SanJose3.Level3.net (4.69.209.169) 79.520 ms 79.179 ms te0-0-0-4.gw3.sin3.10026.telstraglobal.net (202.147.32.101) 247.118 ms 12 i-0-5-0-13-peer.eqnx03.pr.telstraglobal.net (134.159.61.105) 76.599 ms 77.862 ms be3.wr2.sin0.10026.telstraglobal.net (61.14.157.164) 255.062 ms 13 ip-202-147-50-116.asianetcom.net (202.147.50.116) 83.185 ms 82.131 ms be3.wr2.sin0.10026.telstraglobal.net (61.14.157.164) 255.052 ms 14 te0-0-0-7.gw4.sjc1.10026.telstraglobal.net (202.147.50.193) 82.705 ms gi4-1-0.gw3.lax1.10026.telstraglobal.net (202.147.61.153) 247.262 ms te0-0-0-7.gw3.sjc1.10026.telstraglobal.net (202.147.50.189) 83.970 ms 15 TenGigabitEthernet1-2.ar5.LAX2.gblx.net (64.215.81.145) 237.238 ms te0-1-0-1.gw1.lax3.10026.telstraglobal.net (202.147.50.174) 94.014 ms te0-2-0-3.gw1.lax3.10026.telstraglobal.net (202.147.50.145) 94.092 ms 16 gi4-1-0.gw3.lax1.10026.telstraglobal.net (202.147.61.153) 93.543 ms 93.831 ms 93.724 ms 17 TenGigabitEthernet1-2.ar5.LAX2.gblx.net (64.215.81.145) 83.557 ms 83.733 ms te0-2-0-3.gw1.lax3.10026.telstraglobal.net (202.147.50.145) 92.795 ms 18 ae-1-8.edge1.SanJose3.Level3.net (4.69.209.165) 90.036 ms ae-2-7.edge1.SanJose3.Level3.net (4.69.209.169) 87.366 ms ae-1-8.edge1.SanJose3.Level3.net (4.69.209.165) 89.989 ms 19 ip-202-147-50-116.asianetcom.net (202.147.50.116) 246.050 ms i-0-5-0-13-peer.eqnx03.pr.telstraglobal.net (134.159.61.105) 84.520 ms 87.447 ms 20 ip-202-147-50-116.asianetcom.net (202.147.50.116) 107.645 ms 107.024 ms ae-2-7.edge1.SanJose3.Level3.net (4.69.209.169) 87.170 ms 21 te0-0-0-7.gw4.sjc1.10026.telstraglobal.net (202.147.50.193) 90.804 ms te0-1-2-0.wr1.sin0.10026.telstraglobal.net (61.14.157.125) 421.885 ms te0-0-0-7.gw3.sjc1.10026.telstraglobal.net (202.147.50.189) 92.939 ms 22 te0-2-0-3.gw1.lax3.10026.telstraglobal.net (202.147.50.145) 101.118 ms te0-1-0-1.gw1.lax3.10026.telstraglobal.net (202.147.50.174) 103.348 ms te0-2-0-3.gw1.lax3.10026.telstraglobal.net (202.147.50.145) 100.807 ms 23 gi4-1-0.gw3.lax1.10026.telstraglobal.net (202.147.61.153) 102.958 ms 102.824 ms 100.196 ms 24 be3.wr2.sin0.10026.telstraglobal.net (61.14.157.164) 427.112 ms gi4-0-0.gw2.sin3.10026.telstraglobal.net (61.14.157.134) 419.180 ms TenGigabitEthernet1-2.ar5.LAX2.gblx.net (64.215.81.145) 90.393 ms 25 ae-2-7.edge1.SanJose3.Level3.net (4.69.209.169) 95.480 ms te0-1-0-0.gw1.lax3.10026.telstraglobal.net (61.14.158.59) 418.651 ms ae-2-7.edge1.SanJose3.Level3.net (4.69.209.169) 95.090 ms 26 i-0-5-0-13-peer.eqnx03.pr.telstraglobal.net (134.159.61.105) 92.455 ms gi4-1-0.gw3.lax1.10026.telstraglobal.net (202.147.61.153) 100.818 ms 419.534 ms 27 ip-202-147-50-116.asianetcom.net (202.147.50.116) 97.987 ms TenGigabitEthernet1-2.ar5.LAX2.gblx.net (64.215.81.145) 409.187 ms ip-202-147-50-116.asianetcom.net (202.147.50.116) 97.960 ms 28 te0-0-0-7.gw4.sjc1.10026.telstraglobal.net (202.147.50.193) 102.242 ms te0-0-0-7.gw3.sjc1.10026.telstraglobal.net (202.147.50.189) 102.585 ms gi4-1-0.gw3.lax1.10026.telstraglobal.net (202.147.61.153) 419.491 ms 29 i-0-5-0-13-peer.eqnx03.pr.telstraglobal.net (134.159.61.105) 412.651 ms 96.244 ms te0-1-0-1.gw1.lax3.10026.telstraglobal.net (202.147.50.174) 108.387 ms 30 gi4-1-0.gw3.lax1.10026.telstraglobal.net (202.147.61.153) 108.342 ms ip-202-147-50-116.asianetcom.net (202.147.50.116) 417.999 ms gi4-1-0.gw3.lax1.10026.telstraglobal.net (202.147.61.153) 112.199 ms

1 0

Bright House Network Issues
by Ryan Collins 24 Sep '16

24 Sep '16

Any confirmed issues with Bright House Networks? Having multiple issues reported from Bright House numbers not reaching my telephony network for fax purposes. Regards, Ryan

2 1

Zayo fiber cut ATL
by Slade Edmonds 22 Sep '16

22 Sep '16

Apparently there's a Zayo fiber cut impacting folks in Atlanta. We're at QTS (metro) and the connection between QTS and Telx at 56 Marietta has been down for hours due to this cut. Supposedly Zayo is repairing as I type this e-mail.

1 0

webair data center in NY is under DDOS?
by Izzy Goldstein - TeleGo 22 Sep '16

22 Sep '16

anyone else experiencing issues with webair ? i spoke to them and they report that they are having a under a DDOS -- Izzy Goldstein Main: (212) 477.1000 Fax: (212) 477.8900 igoldstein(a)telego.com Website <http://www.telego.com/> | LinkedIn <https://www.linkedin.com/company/telego> | Blog <http://www.telego.com/blog> | Confidentially Notice: This e-mail may contain confidential and/or privileged information. If you are not the intended recipient or have received this e-mail in error please notify us immediately by email reply and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. Any views or opinions presented in this email are solely those of the author and do not necessarily represent those of TeleGo (T). Employees of TeleGo are expressly required not to make defamatory statements and not to infringe or authorize any infringement of copyright or any other legal right by email communications. Any such communication is contrary to TeleGo policy and outside the scope of the employment of the individual concerned. TeleGo will not accept any liability in respect of such communication, and the employee responsible will be personally liable for any damages or other liability arising. Think before you print.

1 0

Cogent - NYC
by Sokhan, Yuriy 21 Sep '16

21 Sep '16

Anyone seeing issues with Cogent in New York City - Manhattan (midtown). I have a point to point line down to western NY. "At this time some off-net customers in the NY Metro area who receive service through Verizon may be experiencing an outage due to vendor equipment failure. The Cogent Noc is working with the Vendor to have service restored. The master case is HD6658376."

4 3

FaceBook Outage in Midwest
by Eric Rogers 21 Sep '16

21 Sep '16

I see "Service Unavailable" and keep refreshing, and it is extremely slow. Midwest (Indianapolis, IN) Eric Rogers www.pdsconnect.me (317) 831-3000 x200

2 1