December 2014 - Outages - Outages Lists

Barracuda ESS and Cloud Protection Layer global outage
by Scott Lambert 24 Dec '14

24 Dec '14

08:55:31 Tue Dec 23 $ telnet d61692a.ess.barracudanetworks.com 25 Trying 64.235.150.197... telnet: connect to address 64.235.150.197: Connection refused Trying 64.235.154.66... telnet: connect to address 64.235.154.66: Connection refused telnet: Unable to connect to remote host >From web site sales chat: " There is an outage with ESS and cloud protection layer affecting our customers globally at the moment our engineering team are working to fix this issue right now we do not have any ETA's at the moment " Since "early this morning" -- Scott Lambert KC5MLE Unix SysAdmin lambert(a)lambertfam.org

3 2

ipv6.google.com
by Frank Bulk 17 Dec '14

17 Dec '14

Anyone else see ipv6.google.com [2607:f8b0:4009:808::2000] down from 7:21 to 7:25 pm (U.S. Central) this evening? Frank root@nagios:/# tcptraceroute6 2607:f8b0:4009:808::2000 traceroute to 2607:f8b0:4009:808::2000 (2607:f8b0:4009:808::2000) from 2607:fe28:0:1000::5, port 80, from port 38504, 30 hops max, 60 bytes packets 1 router-core.mtcnet.net (2607:fe28:0:1000::1) 0.847 ms 0.257 ms 0.203 ms 2 sxct.spnc.mtcnet.net (2607:fe28:11:1002::194) 0.196 ms 0.135 ms 0.155 ms 3 v6-premier.movl-mlx.fbnt.netins.net (2001:5f8:7f0a:1::1) 4.951 ms 11.178 ms 4.900 ms 4 v6-ins-kb1-te-12-2-3031.kmrr.netins.net (2001:5f8:2:2::1) 8.831 ms 8.756 ms 8.851 ms 5 v6-ins-kc1-et-9-3.kmrr.netins.net (2001:5f8::28:1) 8.862 ms 9.098 ms 8.912 ms 6 2001:438:fffe::e99 (2001:438:fffe::e99) 14.566 ms 13.961 ms 13.968 ms 7 ae5.cr1.ord2.us.zip.zayo.com (2001:438:ffff::407d:1e59) 23.338 ms 22.321 ms 22.347 ms 8 ae4.er1.ord7.us.zip.zayo.com (2001:438:ffff::407d:1c32) 23.726 ms 22.467 ms 22.442 ms 9 2001:4860:1:1:0:193d:0:1d (2001:4860:1:1:0:193d:0:1d) 22.449 ms 22.453 ms 22.383 ms 10 2001:4860::1:0:3f7 (2001:4860::1:0:3f7) 22.770 ms 26.090 ms 22.576 ms 11 2001:4860:0:1::1247 (2001:4860:0:1::1247) 23.265 ms 23.246 ms 23.279 ms 12 2607:f8b0:4009:808::2000 (2607:f8b0:4009:808::2000) 22.808 ms [open] 22.719 ms 22.708 ms root@nagios:/#

1 0

Cloudflare - sporadic ipv
by Brandon Applegate 17 Dec '14

17 Dec '14

Anyone else seeing anything ? Their status page is all green at the moment. I’m getting failures on certain Cloudflare IPv6 address. Traceroutes look okay - i.e. I’m not super concerned that they don’t complete, they make it to GTT from where I’m having the issues. www.cloudflare.com is an example, as is (some of the records for) 0.t.4cdn.org. Traceroutes and curl loops below. root@phosphor:/home/bapplegate# traceroute -I -6 0.t.4cdn.org traceroute to 0.t.4cdn.org (2400:cb00:2048:1::8d65:721a), 30 hops max, 80 byte packets 1 2607:ff70:11::1 (2607:ff70:11::1) 0.485 ms 0.529 ms 0.608 ms 2 pc44.core0-5.cncndc.net (2607:ff70:2::2d) 0.276 ms 0.308 ms 0.361 ms 3 tenG-0-2-0.transit0-5.cncndc.net (2607:ff70:2::be) 0.644 ms 0.641 ms 0.638 ms 4 xe-5-0-0.bar1.Cincinnati1.Level3.net (2001:1900:2100::2b5) 0.360 ms 0.363 ms 0.363 ms 5 vl-4066.car2.Chicago1.Level3.net (2001:1900:4:1::27d) 24.373 ms 24.454 ms 24.549 ms 6 vl-4062.edge2.Chicago2.Level3.net (2001:1900:4:1::2a) 23.884 ms 23.800 ms 23.811 ms 7 2001:1900:4:1::be (2001:1900:4:1::be) 24.321 ms 24.276 ms 24.277 ms 8 vl-4061.car1.NewYork2.Level3.net (2001:1900:4:1::106) 29.840 ms 29.811 ms 29.875 ms 9 vl-4041.edge1.NewYork1.Level3.net (2001:1900:4:1::101) 29.135 ms 29.250 ms 29.229 ms 10 * * * 11 NTT-level3-40G.NewYork1.Level3.net (2001:1900:4:3::2b6) 29.440 ms 29.353 ms 29.380 ms 12 ae-3.r05.nycmny01.us.bb.gin.ntt.net (2001:418:0:2000::2bd) 29.957 ms 30.181 ms 30.231 ms 13 2001:590:2:1b::1 (2001:590:2:1b::1) 44.675 ms 39.849 ms 39.800 ms 14 ae8-60g.cr1.nyc3.us.as4436.gtt.net (2001:590::451f:5f5c) 29.564 ms 29.540 ms 29.526 ms 15 ip6.gtt.net (2001:668:0:3::8000:30a1) 29.472 ms 29.585 ms 29.915 ms 16 xe-3-0-0.nyc41.ip6.gtt.net (2001:668:0:2::1:2672) 104.521 ms 37.085 ms 29.862 ms 17 lo0.cr1.ewr1.us.as4436.gtt.net (2001:590::451f:5ff8) 30.635 ms 30.138 ms 30.032 ms 18 * * * 19 * * * 20 * * * 21 * * * 22 *^C root@phosphor:/home/bapplegate# curl --connect-timeout 5 -v -I -6 www.cloudflare.com * About to connect() to www.cloudflare.com port 80 (#0) * Trying 2400:cb00:2048:1::c629:d6a3... Timeout * Trying 2400:cb00:2048:1::c629:d7a3... Timeout * connect() timed out! * Closing connection #0 curl: (28) connect() timed out! Note: had to use HEAD for this - my curl is brain dead on IPv6 literals: root@phosphor:/home/bapplegate# dig +short 0.t.4cdn.org aaaa | while read ipv6; do echo $ipv6; HEAD -s -d -t 5 https://[$ipv6]/; echo; done 2400:cb00:2048:1::be5d:f51a 500 Can't connect to 2400:cb00:2048:1::be5d:f51a:443 (timeout) 2400:cb00:2048:1::be5d:f719 500 Can't connect to 2400:cb00:2048:1::be5d:f719:443 (timeout) 2400:cb00:2048:1::8d65:721a 500 Can't connect to 2400:cb00:2048:1::8d65:721a:443 (certificate verify failed) 2400:cb00:2048:1::8d65:731a 500 Can't connect to 2400:cb00:2048:1::8d65:731a:443 (timeout) 2400:cb00:2048:1::be5d:f41a 500 Can't connect to 2400:cb00:2048:1::be5d:f41a:443 (certificate verify failed) I haven’t submitted anything on their site / support / contact yet but will soon. Wanted to see if anyone from Cloudflare was on here or anyone else who could otherwise corroborate. Thanks. -- Brandon Applegate - CCIE 10273 PGP Key fingerprint: 830B 4802 1DD4 F4F9 63FE B966 C0A7 189E 9EC0 3A74 "SH1-0151. This is the serial number, of our orbital gun."

2 1

Anyone else seeing problems with 8x8 today?
by Mike Lyon 15 Dec '14

15 Dec '14

Have a bunch of phones that cant register. Their tech support simply said there are large problems on the internet today and alot of customers are having problems. No details though. Some details would be nice... Anyone else having issues? -Mike

1 0

Verizon FiOS issues in Philadelphia/Delaware
by eric-list＠truenet.com 15 Dec '14

15 Dec '14

We've received numerous calls from customers about issues, so just wanted to let people know. It seems like it's limited to FiOS Voice and Internet services... Sincerely, Eric Tykwinski TrueNet, Inc. P: 610-429-8300 F: 610-429-3222

3 2

Level3 Looping Traffic in LAX
by Ca By 13 Dec '14

13 Dec '14

Seems hop 7 and 8 are the same box.... This traceroute works fine from AT&T and Sprint, they deliver it correctly to Korea instead of black-holing it in LAX traceroute to 1.201.0.39 (1.201.0.39), 30 hops max, 60 byte packets x 3 te-0-4-0-21-sur03.seattle.wa.seattle.comcast.net (68.86.113.61) 25.810 ms 25.771 ms 25.733 ms 4 be-20-ar01.seattle.wa.seattle.comcast.net (69.139.164.129) 25.670 ms 25.630 ms 25.577 ms 5 he-1-3-0-0-10-cr01.seattle.wa.ibone.comcast.net (68.86.93.165) 25.508 ms 25.473 ms 25.405 ms 6 4.68.63.65 (4.68.63.65) 22.755 ms 13.858 ms 16.425 ms 7 * * ae-3-80.edge5.LosAngeles1.Level3.net (4.69.144.139) 61.317 ms 8 * ae-3-80.edge5.LosAngeles1.Level3.net (4.69.144.139) 58.978 ms 58.956 ms 9 * * * 10 * * * 11 * * * 12 * * *

2 1

Cogent - Latency/Packet Loss out of SFO
by Robert Glover 13 Dec '14

13 Dec '14

Good morning, Anyone else seeing gnarly (50%+) packet loss through Cogent in the Bay Area?? Looks like it's been coming & going for the last hour. Looks like in the last 5 minutes things have stabilized. -Robert

2 2

Experiencing major packet loss and Latency out of Latisys Chicago
by Alexander Vanino 10 Dec '14

10 Dec '14

Hi Outages.org List, I'm seeing very intermittent packet loss and latency spikes inbound and outbound from Latisys Chicago. Is anyone else experiencing issues? Our host who collocates their equipment there has told us that Latisys is having a network event. Any more information on this? IP: 192.184.114.232 TCPING from Verizon-NJ to Latisys Chicago: http://pastebin.com/CYvxP1qi TRACERT from Verizon-NJ to Latisys Chicago: http://pastebin.com/bskiJk4r TCPING From EarthlinkBusiness-NYC to Latisys Chicago: http://pastebin.com/KyNLVR0n TRACERT From EarthlinkBusiness-NYC to Latisys Chicago: http://pastebin.com/m94nXyd0 PING from Latisys to GoogleDNS: http://pastebin.com/p8w3HC7m Traceroute from Latisys to GoogleDNS: http://pastebin.com/TXsJjypf According to the tracert's the issue seems to be within Latisys. -- -- Alex Vanino LinkedIn: http://lnkd.in/v63TbM

1 0

Charter non-PRI phones - Central/Northern Wisconsin
by Anthony Hook 10 Dec '14

10 Dec '14

Non-PRI phones by Charter Communications are down in Central and Northern Wisconsin, included the ability to call their support/ticketing entirely. Update: as I type this, it looks like things have been resolved. Further updates as events warrant. -- - Anthony Hook

1 0

NameCheap DNS failure
by Jim Popovitch 08 Dec '14

08 Dec '14

"We regret to let you know that we are experiencing a DDoS attack against our default DNS system v2." http://status.namecheap.com/archives/17727

2 1