June 2014 - Outages - Outages Lists

AT&T and HE issue
by Ca By 30 Jun '14

30 Jun '14

anyone else seeing issues between Hurrican Electric and AT&T? rviews(a)route-server.ip.att.net's password: --- JUNOS 12.1R3-S4 built 2013-03-24 11:22:31 UTC rviews(a)route-server.ip.att.net> show route 2001:470:0:76::2 inet6.0: 16339 destinations, 195992 routes (16339 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both ::/0 *[Static/5] 56w2d 21:26:30 > to 2001:1890:111d:1::1 via em0.0 rviews(a)route-server.ip.att.net>

2 2

Forwarded from UKNOF, a British Telecom outage
by Bill Woodcock 28 Jun '14

28 Jun '14

Begin forwarded message: > From: Simon Jones <Simon(a)saq.net> > Subject: [uknof] DNS global issue? > Date: June 28, 2014 at 2:15:07 AM PDT > To: "uknof(a)lists.uknof.org.uk" <uknof(a)lists.uknof.org.uk> > > Hi folks, I’m noticing reports of multiple DNS issues – is there a problem with root dns servers? My network seems ok but I’m hearing ebay, twitter, some other biggies are unavailable for some people.. > > Simon Jones > SEMTEC Limited T/A SAQ Begin forwarded message: > From: William Salt <williamejsalt(a)gmail.com> > Subject: Re: [uknof] DNS global issue? > Date: June 28, 2014 at 2:24:06 AM PDT > To: Simon Jones <Simon(a)saq.net> > Cc: uknof(a)lists.uknof.org.uk > > I have just had a similar report from our support Centre. Anyone else? Begin forwarded message: > From: Mark Blackman <mark(a)exonetric.com> > Subject: Re: [uknof] DNS global issue? > Date: June 28, 2014 at 2:27:11 AM PDT > To: William Salt <williamejsalt(a)gmail.com> > Cc: uknof(a)lists.uknof.org.uk, Simon Jones <Simon(a)saq.net> > > Sounds like BT from the twitter reports. > > https://twitter.com/search?q=DNS&src=typd Begin forwarded message: > From: Rob Greenwood <bilco105(a)gmail.com> > Subject: Re: [uknof] DNS global issue? > Date: June 28, 2014 at 2:27:22 AM PDT > To: William Salt <williamejsalt(a)gmail.com> > Cc: uknof(a)lists.uknof.org.uk, Simon Jones <Simon(a)saq.net> > > Issue with BT DNS only I believe. Google DNS is working fine. > > -Rob Begin forwarded message: > From: Job Snijders <job(a)instituut.net> > Subject: Re: [uknof] DNS global issue? > Date: June 28, 2014 at 2:29:33 AM PDT > To: Mark Blackman <mark(a)exonetric.com> > Cc: uknof(a)lists.uknof.org.uk, William Salt <williamejsalt(a)gmail.com>, Simon Jones <Simon(a)saq.net> > > On Sat, Jun 28, 2014 at 10:27:11AM +0100, Mark Blackman wrote: >> Sounds like BT from the twitter reports. > > I heard through backchannels that AS 5400 withdrew half of their > prefixes on peering sessions, so would be good bet something funky is > going on. > > Kind regards, > > Job Begin forwarded message: > From: Ben King <ben(a)warwicknet.com> > Subject: Re: [uknof] DNS global issue? > Date: June 28, 2014 at 2:50:13 AM PDT > To: Rob Greenwood <bilco105(a)gmail.com> > Cc: "uknof(a)lists.uknof.org.uk" <uknof(a)lists.uknof.org.uk>, William Salt <williamejsalt(a)gmail.com>, Simon Jones <Simon(a)saq.net> > > I think BT are having big issues we have seen a massive drop of traffic with AS2856 since 9.25 this morning. In fact nothing coming in from them at all. Begin forwarded message: > From: Henry Merrett <Henry.Merrett(a)serverchoice.com> > Subject: Re: [uknof] DNS global issue? > Date: June 28, 2014 at 2:53:28 AM PDT > To: "uknof(a)lists.uknof.org.uk" <uknof(a)lists.uknof.org.uk> > Cc: William Salt <williamejsalt(a)gmail.com>, Simon Jones <Simon(a)saq.net> > > We’re having BT issues too: > > $ host bt.com > bt.com has address 193.113.9.164 > > .the.as50056.net#sh ip route 193.113.9.164 > Can't find matching entry > > Kind regards, > > Henry Merrett > Senior Network Engineer > ServerChoice Ltd Begin forwarded message: > From: Gavin Henry <ghenry(a)suretec.co.uk> > Subject: Re: [uknof] DNS global issue? > Date: June 28, 2014 at 3:13:40 AM PDT > To: Job Snijders <job(a)instituut.net> > Cc: uknof(a)lists.uknof.org.uk, Paul Mansfield <paul+uknof(a)mansfield.co.uk> > > BT IP Exchange is down due to this too. > > Can't get through to IPX help desk since 09:20. They do have a message on there saying something is up. Begin forwarded message: > From: Ben King <ben(a)warwicknet.com> > Subject: Re: [uknof] DNS global issue? > Date: June 28, 2014 at 3:19:55 AM PDT > To: Rob Greenwood <bilco105(a)gmail.com> > Cc: "uknof(a)lists.uknof.org.uk" <uknof(a)lists.uknof.org.uk>, William Salt <williamejsalt(a)gmail.com>, Simon Jones <Simon(a)saq.net> > > BT have acknowledged the issue to us, and I am seeing routes again, and traffic is climbing, so hopefully its all sorted now. Begin forwarded message: > From: Gavin Henry <ghenry(a)suretec.co.uk> > Subject: Re: [uknof] DNS global issue? > Date: June 28, 2014 at 3:25:42 AM PDT > To: Job Snijders <job(a)instituut.net> > Cc: uknof(a)lists.uknof.org.uk, Paul Mansfield <paul+uknof(a)mansfield.co.uk> > > Alerting has detected it's back.

1 0

Verizon
by Mark Kierzkowski 28 Jun '14

28 Jun '14

Is any one see any issues on verizon east cost network?

7 6

AAG outage?
by Matthew Petach 24 Jun '14

24 Jun '14

I'm hearing whispered reports of AAG being down since last Friday, but I'm not seeing anything in the current press at the moment. If folks have circuits on AAG, would love a quiet indication if you're seeing link down on your AAG paths or not. Thanks! Matt

1 0

Feedly again?
by Paul Ferguson 22 Jun '14

22 Jun '14

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 I know Feedly just experienced a DDoS attack a couple of days ago, but now I am seeing errors trying to login this morning that informs me that "Feedly is unavailable right now, we are working to..." blah blah. Any ideas? - - ferg - -- Paul Ferguson VP Threat Intelligence, IID PGP Public Key ID: 0x54DC85B2 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iF4EAREIAAYFAlOkU7oACgkQKJasdVTchbIQdAD/RlOHyUSf2QPQnZAsKCDFtDZ8 PYGpNwSqSjsoWJpmsnsA/1of/PllXm4iKqn4wZf3i2Ba6BdEpXmLA/M7QFesVEFn =7K/C -----END PGP SIGNATURE-----

2 2

Century Link Fiber Cut in NJ
by Bunde, Andrew 20 Jun '14

20 Jun '14

Just a heads up CL has advised of a "massive" fiber-cut in NJ. If you are not the intended recipient of this email, please notify the sender, delete the email, and do not read, act upon, print, disclose, retain or distribute this email. To view the full terms that apply to this email, go to Agio Disclaimer<http://agio.com/email-disclaimer/>

2 1

Experiencing reports of connectivity disruption at static.oak01.latisys.net in Chicago
by Alex Vanino 20 Jun '14

20 Jun '14

Hi Outages, We have been receiving reports from users all over the country that they have been failing to connect to our application. For other users everything is working fine. Our application is hosted at Latisys(static.oak01.latisys.net) in Chicago. I myself am experiencing mixed connectivity issue's. I can connect to one of my IP's at Latisys, but not the other. Tracert's indicate that packet is getting to their network. Does not work for me: Tracing route to cleardata03.abilto.com [192.65.157.58] over a maximum of 30 hops: 1 2 ms <1 ms 1 ms 192.168.25.1 2 8 ms 8 ms 8 ms L100.NWRKNJ-VFTTP-95.verizon-gni.net [98.109.10. 1] 3 11 ms 12 ms 13 ms G1-2-4-6.NWRKNJ-LCR-22.verizon-gni.net [100.41.2 02.206] 4 11 ms 11 ms 11 ms 130.81.162.84 5 10 ms 13 ms 14 ms 0.xe-11-0-0.GW13.NYC1.ALTER.NET [152.63.18.114] 6 9 ms 11 ms 10 ms comcast.com.customer.alter.net [152.179.220.126] 7 18 ms 11 ms 25 ms he-0-15-0-0-cr01.sacramento.ca.ibone.comcast.net [68.86.85.85] 8 33 ms 35 ms 35 ms he-0-10-0-0-cr01.350ecermak.il.ibone.comcast.net [68.86.86.222] 9 30 ms 31 ms 29 ms 68.86.87.242 10 43 ms 62 ms 39 ms 50.242.148.210 11 46 ms 29 ms 35 ms 69.8.117.130 12 32 ms 35 ms 31 ms 69.8.117.150 13 36 ms 31 ms 32 ms 207-223-34-12.static.oak01.latisys.net [207.223. 34.12] 14 * * * Request timed out. 15 * * * Request timed out. 16 * * * Request timed out. 17 * ^C Works fine for me. Tracing route to cleardata01.abilto.com [192.65.157.49] over a maximum of 30 hops: 1 2 ms 2 ms 1 ms 192.168.25.1 2 9 ms 10 ms 8 ms L100.NWRKNJ-VFTTP-95.verizon-gni.net [98.109.10. 1] 3 12 ms 13 ms 13 ms G1-3-3-1.NWRKNJ-LCR-21.verizon-gni.net [130.81.2 18.238] 4 15 ms 14 ms 15 ms 130.81.162.64 5 11 ms 14 ms 15 ms 0.xe-10-0-1.GW13.NYC1.ALTER.NET [152.63.20.169] 6 13 ms 11 ms 14 ms comcast.com.customer.alter.net [152.179.220.126] 7 16 ms 17 ms 15 ms he-0-15-0-0-cr01.sacramento.ca.ibone.comcast.net [68.86.85.85] 8 35 ms 36 ms 36 ms he-0-10-0-0-cr01.350ecermak.il.ibone.comcast.net [68.86.86.222] 9 30 ms 32 ms 32 ms he-0-10-0-1-pe04.350ecermak.il.ibone.comcast.net [68.86.83.54] 10 34 ms 69 ms 38 ms 50.242.148.210 11 36 ms 32 ms 29 ms 69.8.117.130 12 29 ms 33 ms 34 ms 69.8.117.150 13 30 ms 30 ms 31 ms 207-223-34-12.static.oak01.latisys.net [207.223. 34.12] 14 * * * Request timed out. 15 * * * Request timed out. 16 * * ^C (ICMP Is off at my host's routers, hense timeout) I'm on the phone now with the Network Architect at my hosting provider, but it looks to me to be a problem in the Latisys datacenter, or with one of their providers. Anyone have any information? Regards, Alex Vanino | AbilTo Principal Network Architect Phone: (646) 795-4411 Website: http://www.abilto.com<http://www.abilto.com/> LinkedIn: http://lnkd.in/v63TbM

3 4

Level3 outage in California ongoing
by Owen Roth 19 Jun '14

19 Jun '14

FYI: Our symptom noticed is slow page loads and web traffic affected. Small packets and voice unimpacted. Last updated from Level3 is a core router is down in San Jose. Other information, confirmation? ------------ Owen Roth NOC Manager Impulse Advanced Communications owen(a)impulse.net 805-884-6332

1 0

Short facebook outage overnight
by Frank Bulk 19 Jun '14

19 Jun '14

Our monitoring system saw HTTPv6 access to www.facebook.com down from 2:53 to 3:24 am Central. Seems to match up with this news article: http://www.channelnewsasia.com/news/singapore/facebook-service-outage/117872 4.html Regards, Frank Bulk

1 0

Fiber Outage at 111 8th Ave, NY?
by Tim Huffman 18 Jun '14

18 Jun '14

Hey everybody, I just had two fiber links between 350 Cermak in Chicago and 111 8th in NY fail within minutes of each other. Anybody know of any fiber cuts or seeing the same thing? Tim Huffman Director of Engineering Business Only Broadband 999 Oak Creek Dr, Lombard, IL 60148 Direct: 630.590.6012 | Main: 630.590.6000 | Fax: 630.986.2496 thuffman(a)bobbroadband.com<mailto:thuffman@bobbroadband.com> | http://www.bobbroadband.com/ Cell: 630.340.1925 | Toll-Free Customer Support: 877.262.4553 [https://staticapp.icpsc.com/icp/loadimage.php/mogile/933825/747f0f3e66a4e0c…] Follow Us on LinkedIn<http://www.linkedin.com/company/business-only-broadband> | [https://files.icontact.com/templates/v2/CleanAndSimple/images/twitter.gif] Follow Us on Twitter<https://twitter.com/#%21/BOBbroadband> P please consider the environment prior to printing

3 2