September 2021 - Outages

SF Bay Outage - Internet Archive down
by Jay R. Ashworth 19 Sep '21

19 Sep '21

Among, presumably, lots of other stuff; datacenter outage where they're hosted: https://twitter.com/brewster_kahle/status/1439622667120754689 https://twitter.com/tvzuke/status/1439607856811020296 PG&E is, as you might expect, taking it in the nuts over how fog and mist can knock out power on this scale. Let's restrict followups on this one to reposts of notices from your datacenter that they're out, and/or back on line. Cheers, -- jra -- Jay R. Ashworth Baylink jra(a)baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://www.bcp38.info 2000 Land Rover DII St Petersburg FL USA BCP38: Ask For It By Name! +1 727 647 1274

1 0

voip.ms outage
by Sadiq Saif 17 Sep '21

17 Sep '21

VOIP services provider voip.ms has been down for multiple hours now due to a DDoS attack. Their website and their SIP POPs are currently inaccessible. Updates are being provided via their Twitter account. Latest update: https://twitter.com/voipms/status/1439009468520583169 -- Sadiq Saif https://sadiqsaif.com

1 0

AEM Cloud Service Issue
by Chris Boyd 17 Sep '21

17 Sep '21

Adobe is having problems with their AEM service which is apparently the platform that Trader Joe’s uses. —Chris

1 0

Vocus routing US Salesforce.com traffic through Australia?
by Eric Lester 08 Sep '21

08 Sep '21

Received latency complaints from one location in Nashville with Centurylink Internet. Traceroutes showed traffic routing through Australia. Looks like 1 of 8 Salesforce.com publicly resolved IP's is currently being routed through San Jose, California and then to Australia before coming back to US. We sent the info to Salesforce.com through a ticket but I'm doubtful of it making it to someone that can figure out what to do with the info. Just curious if anyone has insight into what is going on. I am not familiar with Australia ISP's etc. Public IP's I'm receiving NSLOOKUP salesforce.com Name: salesforce.com Addresses: 23.1.35.132 23.1.99.130 23.1.106.133 104.109.10.129 104.109.11.129 184.25.179.132 184.31.3.130 184.31.10.133 Traceroutes from a couple of public hosts Lumen Traceroute NASHVILLE TN USA Traceroute results for: 23.1.35.132 (a23-1-35-132.deploy.static.akamaitechnologies.com) Tracing route to 23.1.35.132 1 ae-2-3605.edge9.SanJose1.Level3.net (4.69.219.65) 61ms 70ms 60ms 2 0.0.0.0 (0.0.0.0) * * * 3 be100.bdr03.sjc01.ca.us.vocus.network (114.31.199.32) 205ms 205ms be203.cor01.syd11.nsw.vocus.network (114.31.199.40) 208ms 4 be101.bdr04.syd03.nsw.vocus.network (114.31.192.47) 208ms be202.cor02.syd04.nsw.vocus.network (114.31.199.42) 205ms 205ms 5 static-74.173.255.49.in-addr.VOCUS.net.au (49.255.173.74) 210ms be100.bdr04.syd03.nsw.vocus.network (114.31.192.45) 205ms static-74.173.255.49.in-addr.VOCUS.net.au (49.255.173.74) 223ms 6 a23-1-240-228.deploy.static.akamaitechnologies.com (23.1.240.228) 208ms static-74.173.255.49.in-addr.VOCUS.net.au (49.255.173.74) 212ms 212ms 7 0.0.0.0 (0.0.0.0) * * a23-1-240-143.deploy.static.akamaitechnologies.com (23.1.240.143) 205ms 8 static-73.173.255.49.in-addr.VOCUS.net.au (49.255.173.73) 208ms 0.0.0.0 (0.0.0.0) * * 9 be114.cor02.syd04.nsw.vocus.network (114.31.192.44) 356ms 356ms static-73.173.255.49.in-addr.VOCUS.net.au (49.255.173.73) 205ms 10 be114.cor02.syd04.nsw.vocus.network (114.31.192.44) 356ms be202.bdr03.sjc01.ca.us.vocus.network (114.31.199.43) 359ms be114.cor02.syd04.nsw.vocus.network (114.31.192.44) 357ms 11 173.205.35.1 (173.205.35.1) 359ms 356ms be202.bdr03.sjc01.ca.us.vocus.network (114.31.199.43) 356ms 12 173.205.35.1 (173.205.35.1) 356ms ae18-3416.cr11-fra2.ip4.gtt.net (89.149.180.134) 356ms ae6.cr6-sjc1.ip4.gtt.net (89.149.180.78) 356ms 13 ae18-3416.cr11-fra2.ip4.gtt.net (89.149.180.134) 357ms 354ms ip4.gtt.net (98.124.180.42) 360ms 14 ae12.r02.sjc01.icn.netarch.akamai.com (23.207.232.40) 360ms ip4.gtt.net (98.124.180.42) 356ms 356ms 15 ae4.r02.ord01.icn.netarch.akamai.com (23.32.63.26) 405ms 425ms ae12.r02.sjc01.icn.netarch.akamai.com (23.207.232.40) 357ms 16 ae3.r01.lga01.icn.netarch.akamai.com (23.32.63.4) 425ms ae4.r02.ord01.icn.netarch.akamai.com (23.32.63.26) 405ms 400ms 17 ae1.r02.lga01.ien.netarch.akamai.com (23.203.156.37) 426ms 426ms ae3.r01.lga01.icn.netarch.akamai.com (23.32.63.4) 422ms 18 ae1.r02.lga01.ien.netarch.akamai.com (23.203.156.37) 423ms 0.0.0.0 (0.0.0.0) * ae1.r02.lga01.ien.netarch.akamai.com (23.203.156.37) 423ms 19 a23-1-35-132.deploy.static.akamaitechnologies.com (23.1.35.132) 226ms 228ms 0.0.0.0 (0.0.0.0) * Hurricane Electric Traceroute core2.bna1.he.net> traceroute 23.1.35.132 source 216.218.252.20 Target 23.1.35.132 Hop Start 1 Hop End 30 Hop Packet 1 Packet 2 Packet 3 Hostname 1 5.469 ms 5.579 ms 5.586 ms 100ge6-2.core1.ind1.he.net (184.105.65.149) 2 11.324 ms 11.347 ms 11.391 ms 100ge0-33.core2.stl1.he.net (184.104.196.14) 3 16.556 ms 16.854 ms 16.982 ms 100ge0-68.core2.mci3.he.net (72.52.92.237) 4 16.800 ms 16.811 ms 16.930 ms 100ge4-1.core1.mci3.he.net (184.104.197.69) 5 27.761 ms 27.783 ms 27.788 ms 100ge7-2.core1.den1.he.net (184.104.195.213) 6 51.918 ms 50.625 ms 50.712 ms 100ge0-29.core2.sjc2.he.net (184.105.64.69) 7 50.842 ms 50.672 ms 50.647 ms 100ge16-1.core1.sjc2.he.net (184.104.192.213) 8 51.078 ms 80.874 ms 81.187 ms 100ge13-2.core1.sjc1.he.net (184.105.65.113) 9 52.429 ms 52.441 ms 52.506 ms vocus.gigabitethernet2-13.core1.sjc1.he.net (64.71.184.46) 10 202.821 ms 202.354 ms * be202.cor02.syd04.nsw.vocus.network (114.31.199.42) 21 463.994 ms 463.937 ms 464.130 ms 100ge0-33.core2.nyc4.he.net (184.105.81.62) 22 420.303 ms 420.423 ms 420.768 ms 100ge3-2.core1.bos1.he.net (184.105.64.54) 23 224.026 ms 224.042 ms 224.045 ms a23-1-35-132.deploy.static.akamaitechnologies.com (23.1.35.132) Eric Lester | Vice President of Technology Acadia Healthcare Company, Inc. Email: eric.lester(a)acadiahealthcare.com<mailto:eric.lester@acadiahealthcare.com> Address: 6100 Tower Circle - Suite 1000 - Franklin, TN 37067 [cid:image001.png@01D7A4CD.13E1E020]

4 3

Zayo AZ/TX
by Scott Harvanek 08 Sep '21

08 Sep '21

Looks like Zayo has a fiber cut somewhere between Tucson and Dallas (again). Our waves are down along that path as of 0702MST, we have a ticket open with them.

1 0

Spectrum/LTWC Ohio/KY
by Neader, Brent 07 Sep '21

07 Sep '21

Seeing enterprise DIA sites drop off and come back repeatedly in these areas, Dayton, Cinci, Cleveland, Middletown, Louisville.

9 10

Re: [outages] Outages Digest, Vol 160, Issue 6
by Michael Peterman 07 Sep '21

07 Sep '21

Is anything else seeing an issue with Spectrum in the Kenosha/Racine/Milwaukee, WI area, we seeing both Fiber DIA and Coax based connection flat both up and down for the past hour. It seems like the issue is only lasting 5-10 minutes at a time. Michael Peterman Small Business Technology Solutions Inc. -----Original Message----- From: Outages <outages-bounces(a)outages.org> On Behalf Of outages-request(a)outages.org Sent: Tuesday, September 7, 2021 12:12 PM To: outages(a)outages.org Subject: Outages Digest, Vol 160, Issue 6 Send Outages mailing list submissions to outages(a)outages.org To subscribe or unsubscribe via the World Wide Web, visit https://puck.nether.net/mailman/listinfo/outages or, via email, send a message with subject or body 'help' to outages-request(a)outages.org You can reach the person managing the list at outages-owner(a)outages.org When replying, please edit your Subject line so it is more specific than "Re: Contents of Outages digest..." Today's Topics: 1. Re: Spectrum/LTWC Ohio/KY (Josh Luthman) ---------------------------------------------------------------------- Message: 1 Date: Tue, 7 Sep 2021 13:11:09 -0400 From: Josh Luthman <josh(a)imaginenetworksllc.com> To: Savarrah Goss <Savarrah.Goss(a)hamiltonrelay.com> Cc: Mike Markwood <mmarkwood(a)geostarllc.net>, "Neader, Brent" <Brent.Neader(a)druryhotels.com>, "outages(a)outages.org" <outages(a)outages.org> Subject: Re: [outages] Spectrum/LTWC Ohio/KY Message-ID: <CAN9qwJ_h4G40W5=wVz_1SknEYjEa2akD_2-Dp6nGqJxQHyg80w(a)mail.gmail.com> Content-Type: text/plain; charset="utf-8" Looks like there is an issue with the coax network. > /tool traceroute 1.1.1.1 # ADDRESS LOSS SENT LAST AVG BEST WORST 1 142.254.145.13 9.8% 143 timeout 13.5 7.9 100.7 2 24.29.6.105 9.1% 143 24.3ms 42.4 13.4 226.8 3 65.29.36.238 10.. 143 14.1ms 19.7 9.8 179.4 4 65.29.1.46 14% 143 25ms 27.2 16.7 95.6 5 66.109.6.66 10.. 143 34.1ms 30.7 23.8 141.6 6 66.109.3.24 13.. 143 26.6ms 35.4 24.5 202 7 209.18.43.59 8.4% 143 28.3ms 30.7 24.3 95.1 8 173.245.63.56 12.. 143 42.2ms 39.3 27.5 135.5 9 172.70.44.4 11.. 142 43.5ms 35.1 24.2 161.6 10 1.1.1.1 6.3% 142 44.1ms 33.4 24.1 149.8 11 1.1.1.1 0% 1 26.1ms 26.1 26.1 26.1 Fiber DIA seems just fine to me. > /tool traceroute 1.1.1.1 # ADDRESS LOSS SENT LAST AVG BEST WORST 1 10.10.10.1 0% 171 0.3ms 0.3 0.3 3.6 2 192.149.254.1 0% 171 0.4ms 0.4 0.3 1.3 3 74.218.88.129 0% 171 2ms 1.4 0.6 31.9 4 65.189.190.189 0% 171 8.7ms 10.4 0.8 143.4 5 65.29.38.112 0% 171 14.2ms 12.5 8.3 16.9 6 65.29.1.46 0% 170 11.4ms 9.6 8.2 14.9 7 66.109.6.12 0% 170 17.5ms 16.5 16.1 20 8 66.109.3.24 0% 170 16.9ms 17.1 16.5 42.2 9 209.18.43.59 0% 170 26.2ms 20.2 16.2 64.7 10 24.30.200.73 0% 170 20.4ms 31.1 16.7 89.4 11 172.70.44.2 0% 170 17.2ms 19.5 16.6 72.3 12 1.1.1.1 0% 170 16.2ms 16.4 16.2 31.5 Josh Luthman 24/7 Help Desk: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373 On Tue, Sep 7, 2021 at 1:09 PM Savarrah Goss via Outages < outages(a)outages.org> wrote: > > https://app.fing.com/internet/outage/DROP:US-Ohio--Spectrum%20Business > @2021-09-07-1650-00000 > > > > *Savarrah Goss |* *Hamilton Relay * > > *O: 229.338.7086 | C: 229.854.8872 | Savarrah.Goss(a)hamiltonrelay.com > <Savarrah.Goss(a)hamiltonrelay.com> **| HamiltonRelay.com > <http://www.hamiltonrelay.com> * > > *[image: 1575557226300]* <https://hamiltonrelay.com/> > > [image: 1575557235476] *Hamilton Relay > <http://facebook.com/HamiltonRelay> ** | *[image: 1575557246386]* > @HamiltonRelay <http://twitter.com/HamiltonRelay> * > > > > *From:* Outages <outages-bounces(a)outages.org> *On Behalf Of *Mike > Markwood via Outages > *Sent:* Tuesday, September 7, 2021 1:00 PM > *To:* 'outages(a)outages.org' <outages(a)outages.org>; Neader, Brent < > Brent.Neader(a)druryhotels.com> > *Subject:* Re: [outages] Spectrum/LTWC Ohio/KY > > > > **This email originated from outside of the Hamilton network. Please > use extra caution with links and attachments. ** > > We're seeing our Spectrum cable modem customers across Northeast Ohio > bouncing as well as Columbus, OH and a few customers in TN. > > > > Mike Markwood | Director of Network Operations > P: 866.986.1066 E: mmarkwood(a)geostarllc.net > > 1000 S. Cleveland Massillon Road, Suite 3 Fairlawn, OH 44333 > > > ------------------------------ > > *From:* Outages <outages-bounces(a)outages.org> on behalf of Neader, Brent > via Outages <outages(a)outages.org> > *Sent:* Tuesday, September 7, 2021 12:52 PM > *To:* 'outages(a)outages.org' <outages(a)outages.org> > *Subject:* [outages] Spectrum/LTWC Ohio/KY > > > > Seeing enterprise DIA sites drop off and come back repeatedly in these > areas, Dayton, Cinci, Cleveland, Middletown, Louisville. > > > > > CONFIDENTIALITY NOTE: This email and any attachments are confidential and > may be protected by legal privilege. If you have received this email in > error, please immediately notify the sender by return email and delete all > copies of this email. > _______________________________________________ > Outages mailing list > Outages(a)outages.org > https://puck.nether.net/mailman/listinfo/outages >

3 3

Comcast / xfinity USA MA
by Jason Kuehl 07 Sep '21

07 Sep '21

Anyone else seeing issues with Comcast / Xfinity. Over the past hour, I have gotten several tickets (17) around people not being access to VPN today. Other people in the same locations on RCN/ATT work fine it seems to be only Comcast / Xfinity users. Anyone else seeing these odd reports? -- Sincerely, Jason W Kuehl Cell 920-419-8983 jason.w.kuehl(a)gmail.com

3 2

Mikrotik dynamic DNS is down
by Ross Tajvar 06 Sep '21

06 Sep '21

Looks like the domain was suspended # whois mynetname.net | grep -i sus > Name Server: ns1.suspended-domain.com > Name Server: ns2.suspended-domain.com # whois mynetname.net | grep -i date > Updated Date: 2021-09-06T10:48:09Z > Creation Date: 2014-04-29T08:21:38Z > Registry Expiry Date: 2022-04-29T08:21:38Z > Domain Status: clientUpdateProhibited > https://icann.org/epp#clientUpdateProhibited The domain was updated around 12 hours ago, which is when it seems to have stopped working.

1 0

Packet Loss Cogent: Nuremberg -> Munich
by Lukas Tribus 01 Sep '21

01 Sep '21

There's a lot of packet loss on the Cogent Network between Nuremberg and Munich, affecting Hetzner: https://rich-traceroute.io/t/f0ccbfed5bd562b149c564a2f584d3f2263b28f1 Lukas

1 0