June 2015 - Outages - Outages Lists

Re: [outages] large latency increase on zayo
by Roderique, Nick 30 Jun '15

30 Jun '15

Could the Century Link fiber cut be effecting your issue? -----Original Message----- From: Jeff Nishi via Outages [outages(a)outages.org] Sent: Tuesday, June 30, 2015 06:34 PM Central Standard Time To: outages(a)outages.org Subject: [outages] large latency increase on zayo Anyone else seeing abnormal latency between San Jose and Los Angeles on routes using zayo? 10 25 ms 25 ms 26 ms v22.ae29.cr2.sjc2.us.zip.zayo.com [64.125.31.37] 11 142 ms 143 ms 142 ms v21.ae29.cr2.lax112.us.zip.zayo.com [64.125.31.9] 12 148 ms 143 ms 147 ms ae10.er4.lax112.us.zip.zayo.com [64.125.21.114] 13 143 ms 142 ms 142 ms ae9.er3.lax112.us.zip.zayo.com [64.125.25.169] Jeff

3 2

large latency increase on zayo
by Jeff Nishi 30 Jun '15

30 Jun '15

Anyone else seeing abnormal latency between San Jose and Los Angeles on routes using zayo? 10 25 ms 25 ms 26 ms v22.ae29.cr2.sjc2.us.zip.zayo.com [64.125.31.37] 11 142 ms 143 ms 142 ms v21.ae29.cr2.lax112.us.zip.zayo.com [64.125.31.9] 12 148 ms 143 ms 147 ms ae10.er4.lax112.us.zip.zayo.com [64.125.21.114] 13 143 ms 142 ms 142 ms ae9.er3.lax112.us.zip.zayo.com [64.125.25.169] Jeff

1 0

Re: [outages] CenturyLink fiber cut between Modesto, CA and San Jose, CA this AM.. Start time 4:26AM PST
by Peter Kranz 30 Jun '15

30 Jun '15

This appears to be part of a larger act of intentional sabotage: "We have just been advised that crews have been removed from the area while law enforcement investigates the damage, which was caused by vandalism. Three enclosures were damaged impacting three different carriers. There is no time given for when they will be allowed to begin work again, subsequently, there is still no ETR. "

3 2

Microsoft Azure outage in US
by Izzy Goldstein - TeleGo 30 Jun '15

30 Jun '15

http://azure.microsoft.com/en-us/status/ Network Infrastructure - West US, East US 2, South Central US, and East US - Partial Performance Degradation 11 mins ago Starting at approximately 30 Jun, 2015 14:45 UTC an intermittent connectivity issue is being investigated in West US. Preliminary root cause is attributed to a fiber cut in Oregon resulting in 3-5% degradation in network connectivity. Our engineers are working to mitigate and continue to work with a 3rd party partner to resolve this issue. Customers may experience intermittent connectivity issues for services deployed in West US, East US, East US 2, and South Central US. An update will be provided in 60 minutes or as events warrant. -- Izzy Goldstein Main: (212) 477.1000 Fax: (212) 477.8900 igoldstein(a)telego.net Website <http://www.telego.com/> | LinkedIn <https://www.linkedin.com/company/telego> | Blog <http://www.telego.com/blog> | Confidentially Notice: This e-mail may contain confidential and/or privileged information. If you are not the intended recipient or have received this e-mail in error please notify us immediately by email reply and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. Any views or opinions presented in this email are solely those of the author and do not necessarily represent those of TeleGo (T). Employees of TeleGo are expressly required not to make defamatory statements and not to infringe or authorize any infringement of copyright or any other legal right by email communications. Any such communication is contrary to TeleGo policy and outside the scope of the employment of the individual concerned. TeleGo will not accept any liability in respect of such communication, and the employee responsible will be personally liable for any damages or other liability arising. Think before you print.

4 4

CenturyLink fiber cut between Modesto, CA and San Jose, CA this AM.. Start time 4:26AM PST
by Peter Kranz 30 Jun '15

30 Jun '15

7AM PST update - "UPDATE TO CUSTOMER OFFNET FIBER CUT BETWEEN MODESTO, CA AND SAN JOSE, CA. ON THE OFFNET NETWORK. ALL CREWS AND MANAGEMENT HAVE BEEN ENGAGED. MORE UPDATES TO FOLLOW THANK YOU CENTURYLINK ETAC" Peter Kranz www.UnwiredLtd.com <http://www.unwiredltd.com/> Desk: 510-868-1614 x100 Mobile: 510-207-0000 pkranz(a)unwiredltd.com <mailto:pkranz@unwiredltd.com>

1 0

whois.enom.com
by Jim Popovitch 28 Jun '15

28 Jun '15

Hello, For several days now it seems that ENOM's whois is broken. I constantly get "Connection reset by peer" despite waiting hours, even days, as well as trying from different networks. ------------------------------- ~$ whois speedyiguana.com -h whois.enom.com Your request cannot be completed at this time due to query limit controls. Please try your request again after 30 minutes. fgets: Connection reset by peer ------------------------------ Are they rate limiting everyone? -Jim P.

4 5

Denver CO C-Link issues
by Joe O'Meara 25 Jun '15

25 Jun '15

Got word from Nitel that a large number of customers in DNVRCO are down due to a cut on Century Link plant. Affecting the east side of Denver. Said line is in a high traffic area so repairs won't start until 6 PM MDT tomorrow night. -- Blackfoot Tech support support(a)blackfoot.net 877-881-1155

1 0

Nation wide t-mobile inbound and outbound
by Karl Putland 25 Jun '15

25 Jun '15

T-Mobile confirms nation wide inbound and outbound call completion issues. May also affect emergency services as call attempts to 611 received immediate disconnect as well as calls to t-mobile numbers or calls originating from t-mobile devices. --Karl

10 12

Anybody having any issues with ADP Payroll?
by Luke Rockwell 22 Jun '15

22 Jun '15

Having users reporting they cant login, no data, and cant access certain functions Level 1 support told ADP it must be on our side however I can get in just fine, Traceroute looks a little funny Traceroute has started… traceroute to portal.gslb.adp.com (170.146.234.48), 64 hops max, 72 byte packets 1 0.614 ms 0.326 ms 0.319 ms 2 0.904 ms 0.809 ms 0.628 ms 3 1.217 ms 1.068 ms 1.045 ms 4 g 50.637 ms 1.850 ms 1.270 ms 5 ae7.cr1.sjc2.us.zip.zayo.com (64.125.32.233) 2.762 ms 5.685 ms 2.812 ms 6 ae9.mpr3.sjc7.us.zip.zayo.com (64.125.31.202) 3.114 ms 4.732 ms 4.511 ms 7 zayo-level3.sjc7.us.zip.zayo.com (64.125.13.242) 4.500 ms 2.934 ms 3.445 ms 8 * * * 9 adp-brokera.ear1.atlanta2.level3.net (4.28.26.134) 64.724 ms 64.914 ms 64.693 ms 10 * * * 11 * * * 12 170.146.240.49 (170.146.240.49) 65.974 ms 65.080 ms 65.117 ms 13 * * * 14 170.146.234.48 (170.146.234.48) 65.272 ms 67.504 ms 65.887 ms

1 0

Telia Outage - East Coast US
by Justin Head 17 Jun '15

17 Jun '15

We're seeing zero connectivity from New York to Ashburn, VA and Amsterdam, as well as intermittent connectivity across the rest of the US, all through Telia. Traffic has been rerouted numerous times. Anybody else seeing anything similar? - Justin

3 3