August 2016 - Outages - Outages Lists

Allreciepes
by Kyle Farmer 03 Sep '16

03 Sep '16

Im showing theyre down?

4 3

Comcast Miami
by Marty Strong 30 Aug '16

30 Aug '16

At around 12:20 UTC I noticed a loss of around 50% of the BGP table on our Comcast connection in Miami at NOTA. Anybody else notice anything? Regards, Marty Strong -------------------------------------- CloudFlare - AS13335 Network Engineer marty(a)cloudflare.com +44 7584 906 055 smartflare (Skype) https://www.peeringdb.com/asn/13335

2 4

Packet loss to 8.8.8.8 and other google IPs (Dallas)
by Dan White 30 Aug '16

30 Aug '16

We are seeing packet loss to google hosts, hosted in Dallas. Any anyone seeing this? MTR output: Host Loss% Snt Last Avg Best Wrst StDev 1. 10.0.2.1 0.0% 51 2.4 1.9 0.5 5.2 0.7 2. olp-67-217-152-5.olp.net 0.0% 51 0.4 3.7 0.3 27.8 4.9 3. olp-67-217-158-29.olp.net 0.0% 50 0.3 4.5 0.3 22.0 5.3 4. olp-67-217-151-125.olp.net 0.0% 50 0.6 4.9 0.6 33.3 6.8 5. olp-67-217-158-30.olp.net 0.0% 50 0.6 6.4 0.5 64.0 10.5 6. olp-67-217-158-21.olp.net 0.0% 50 11.4 4.4 1.1 13.6 4.1 7. xe-8-3-0.bar2.tulsa2.level3.net 0.0% 50 1.2 2.7 1.2 27.5 5.5 8. ae-0-11.bar1.tulsa2.level3.net 0.0% 50 2.0 4.1 1.9 44.5 7.8 9. ae-3-3.ebr4.dallas1.level3.net 91.8% 50 20.5 20.5 20.5 20.5 0.0 10. ae-8-0.ear3.dallas1.level3.net 93.9% 50 20.6 20.6 20.6 20.7 0.0 11. google-level3-100g.dallas1.leve28.0% 50 20.5 20.7 20.4 29.6 1.5 12. 108.170.240.193 22.0% 50 20.8 20.7 20.7 20.9 0.0 13. 64.233.175.9 26.0% 50 20.7 20.7 20.7 20.8 0.0 14. google-public-dns-a.google.com 24.0% 50 20.5 20.6 20.5 25.9 0.9 The problematic hop appears to be either 10 or 11 (level3-100g). Compare to: Host Loss% Snt Last Avg Best Wrst StDev 1. 10.0.2.1 0.0% 124 0.8 1.6 0.5 4.6 0.5 2. olp-67-217-152-5.olp.net 0.0% 124 3.8 4.3 0.3 29.9 5.6 3. olp-67-217-158-29.olp.net 0.0% 124 0.3 3.5 0.3 26.1 4.9 4. olp-67-217-151-125.olp.net 0.0% 124 6.6 3.3 0.6 20.0 3.7 5. olp-67-217-158-30.olp.net 0.0% 124 0.6 7.1 0.6 44.1 9.8 6. olp-67-217-158-21.olp.net 0.8% 124 11.9 4.8 1.1 14.9 4.3 7. xe-8-3-0.bar2.tulsa2.level3.net 0.0% 123 1.3 4.0 1.1 68.3 11.3 8. ae-0-11.bar1.tulsa2.level3.net 0.8% 123 2.1 3.9 1.9 62.3 8.1 9. ae-3-3.ebr4.dallas1.level3.net 94.3% 123 20.5 20.6 20.5 20.6 0.0 10. ae-2-3512.edge2.dallas1.level3. 0.0% 123 20.4 22.1 20.3 68.0 7.1 11. b.resolvers.level3.net 0.0% 123 20.2 20.3 20.2 22.8 0.3 -- Dan White BTC Broadband Network Admin Lead Ph 918.366.0248 (direct) main: (918)366-8000 Fax 918.366.6610 email: dwhite(a)olp.net http://www.btcbroadband.com

12 13

Heavy Packet Loss on Level 3
by Jim Lohiser 30 Aug '16

30 Aug '16

We are seeing heavy packet loss on our Level 3 connection in Cleveland, OH. Looks like it started around 5am Eastern time. Is anyone else seeing an issue? HOST Loss% Snt Last Avg Best Wrst StDev 1. gi1-2-v100.core1.n2net.net 0.0% 100 0.4 5.6 0.3 147.9 22.5 2. ge-6-10.car2.Cleveland1.Leve 0.0% 100 9.5 26.9 1.3 216.8 49.6 3. 0.ae12.pr0.dca20.tbone.rr.co 15.0% 100 10.5 13.6 10.3 108.3 14.6 4. bu-ether15.vinnva0510w-bcr00 24.0% 100 10.5 12.1 10.2 14.4 1.2 5. be1.clevohek01r.midwest.rr.c 13.0% 100 18.6 20.2 18.4 22.6 1.1 6. be1.clevohek02r.midwest.rr.c 11.0% 100 21.1 22.8 18.8 26.6 2.3 7. be1.clhgohcc01r.midwest.rr.c 16.0% 100 19.1 19.1 18.8 19.9 0.2 8. 24.33.103.87 16.0% 100 27.2 35.5 23.6 113.3 9.6 9. teng18-10.clhgohcc04m.neo.rr 13.0% 100 23.0 40.4 22.9 417.7 42.0 10. cpe-24-165-186-90.neo.res.rr 15.0% 100 31.6 33.3 30.0 77.5 6.2 HOST Loss% Snt Last Avg Best Wrst StDev 1. gi1-2-v100.core1.n2net.net 0.0% 20 0.4 1.3 0.4 15.1 3.2 2. ge-6-10.car2.Cleveland1.Leve 0.0% 20 1.7 5.3 1.3 48.2 11.6 3. Verizon-level3-100G.Washingt 15.0% 20 15.3 13.7 9.4 48.4 9.5 4. ae203-0.RCMDVA-VFTTP-305.ver 10.0% 20 15.8 16.0 15.6 18.8 0.7 5. ??? 100.0 20 0.0 0.0 0.0 0.0 0.0 Host Loss% Last Avg Best Wrst StDev 1. 192.168.99.1 0.0% 0.2 0.2 0.2 0.3 0.0 2. gi1-2-v100.core1.n2net.net 0.0% 0.7 3.5 0.5 105.1 14.9 3. ge-6-10.car2.cleveland1.level3.net 0.0% 3.4 25.8 1.6 215.3 50.5 4. att-level3-te.washingtondc12.level3.net 17.7% 11.3 13.7 11.0 28.1 2.8 5. cr1.wswdc.ip.att.net 15.2% 25.1 23.3 21.2 25.1 1.2 6. 12.122.2.202 17.7% 22.4 22.8 20.9 24.6 0.7 7. ??? 8. 12.123.151.149 16.5% 39.4 41.6 20.9 107.2 23.3 9. 206.121.183.14 17.9% 23.5 24.9 23.4 56.8 5.8 10. 12.28.240.162 11.4% 23.8 26.4 23.8 94.4 11.3 11. ??? Host Loss% Last Avg Best Wrst StDev 1. 192.168.99.1 0.0% 0.2 0.2 0.2 0.3 0.0 2. gi1-2-v100.core1.n2net.net 0.0% 0.6 1.2 0.5 21.6 3.5 3. ge-6-10.car2.cleveland1.level3.net 0.0% 1.6 40.8 1.5 201.6 68.2 4. ae-1-60.edge3.washington4.level3.net 5.9% 10.0 10.5 9.9 16.4 1.5 5. cogent-comm.edge3.washington4.level3.net 20.0% 10.5 10.6 10.5 11.9 0.3 6. be2657.ccr42.dca01.atlas.cogentco.com 11.4% 11.5 11.4 11.2 11.6 0.1 7. be2892.ccr22.cle04.atlas.cogentco.com 14.3% 29.2 29.0 28.8 29.2 0.1 8. be2733.rcr21.cmh02.atlas.cogentco.com 17.6% 31.9 31.9 31.6 32.2 0.1 9. te0-0-2-0.nr11.b029193-0.cmh02.atlas.cogentco.com 17.6% 32.6 32.6 32.4 32.9 0.1 10. 38.104.22.62 20.6% 32.3 32.3 32.1 32.4 0.1 11. 38.111.183.169 14.7% 37.1 37.9 36.7 47.6 2.5 12. 38.111.183.170 20.6% 48.4 59.0 43.9 205.7 32.7 Jim Lohiser | Systems Engineer N2Net | 815 Superior Ave. E | Suite 425 | Cleveland | Ohio | 44114-2704

3 2

Gandi widespread outage
by Jeremy Chadwick 30 Aug '16

30 Aug '16

It appears gandi.net (provider of DNS, Email, SSL certs, VPSes, etc.) is currently experiencing an extremely widespread DDoS affecting quite literally *all* of their services. It started at 17:07 PDT (UTC-7). The only confirmation I can provide people right now is via their Twitter account. Their status page is intermittently unavailable: https://twitter.com/gandinoc/status/770413543292342272 https://status.gandi.net/timeline/events/562 I happened to notice because I was in the middle of doing some DNS updates and an SSL certificate renewal; great timing. -- | Jeremy Chadwick jdc(a)koitsu.org | | UNIX Systems Administrator http://jdc.koitsu.org/ | | Making life hard for others since 1977. PGP 4BD6C0CB |

1 0

Internet Outage @ 14:57 EST
by Irfan, Asif 29 Aug '16

29 Aug '16

We faced heavy packet loss for internet destinations over Cox circuit in Hampton at 2:57 pm EST. We run voice on WAN links and this caused us an outage today. Anyone else faced the same issue today? Regards Asif Global Network Operations

1 0

Frontier North East Outage
by Matt Hoppes 27 Aug '16

27 Aug '16

Is anyone aware of any outages affecting Frontier Internet in the north east? (PA, OH, NY, etc)? I'm seeing a bunch of customers of mine offline currently.

3 4

Re: [outages] Facebook is down
by Sneath, Dustin 27 Aug '16

27 Aug '16

No problems in Elk Grove Village, IL (Chicagoland area) Dustin Sneath Systems Administrator Elk Grove Park District (847)228-3846 (desk) (224)800-2146 (cell) Sent from my Verizon Wireless 4G LTE DROID Scott Sadlocha via Outages <outages(a)outages.org> wrote: Metro Detroit area as well. Both web and mobile. [cid:7b1fdcd39acb43e8a82bbc1c1ce47a97] [cid:c1411761-cec3-41ad-8667-c246015ef73d] Scott Sadlocha Security Engineer III p 800.981.8898 ext. 4514 unitedshore.com<www.unitedshore.com> YOUNITED TO MAKE LENDING EASY Click here<mailto:ushare@unitedshore.com> to share your experience The contents of this message may be confidential. If you are not the intended recipient, do not disseminate, disclose, copy or use this message without the permission of the author. Nothing contained in this message or in any attachment shall satisfy the requirements for a writing, and nothing contained herein shall constitute a contract or electronic signature. If this message has been received in error, please delete it immediately. From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Jay Farrell via Outages Sent: Friday, August 26, 2016 7:16 PM To: outages(a)outages.org Subject: Re: [outages] Facebook is down Appears to be very widespread, judging by the rapidly incrementing discus comments on downdetector, as well as their live outage map. In Philadelphia it died for me at ~7pm EDT. http://downdetector.com/status/facebook/map/

2 1

Re: [outages] Facebook is down
by Jay Farrell 27 Aug '16

27 Aug '16

Appears to be very widespread, judging by the rapidly incrementing discus comments on downdetector, as well as their live outage map. In Philadelphia it died for me at ~7pm EDT. http://downdetector.com/status/facebook/map/

3 2

Re: [outages] Facebook is down
by Jay Farrell 27 Aug '16

27 Aug '16

Facebook is back up in Philadelphia as well. On Fri, Aug 26, 2016 at 7:17 PM, Randall (Randy) Raitz via Outages < outages(a)outages.org> wrote: > Seeing recoveries in Denver, CO at this time. > > -----Original Message----- > From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Seth > Mattinen via Outages > Sent: Friday, August 26, 2016 5:15 PM > To: outages(a)outages.org > Subject: Re: [outages] Facebook is down > > On 8/26/16 16:02, Josh Luthman via Outages wrote: > > > > Coming from Troy, OH via Frontier > > > Also seeing broken from Nevada. > > ~Seth > _______________________________________________ > Outages mailing list > Outages(a)outages.org > https://puck.nether.net/mailman/listinfo/outages > > > *Disclaimer* > > The information contained in this communication from the sender is > confidential. It is intended solely for use by the recipient and others > authorized to receive it. If you are not the recipient, you are hereby > notified that any disclosure, copying, distribution or taking action in > relation of the contents of this information is strictly prohibited and may > be unlawful. > > This email has been scanned for viruses and malware, and may have been > automatically archived. > > _______________________________________________ > Outages mailing list > Outages(a)outages.org > https://puck.nether.net/mailman/listinfo/outages > >

1 0