March 2021 - Outages - Outages Lists

Apparently Equifax is down for consumers and businesses.
by Chapman, Brad (NBCUniversal) 25 Mar '21

25 Mar '21

I was ordering fiber optic internet from Frontier Communications—my doctor said I should consume more fiber—but their system said my credit check failed and was demanding an equipment deposit. I pushed back. They ran it three times. Then they got word in a group chat that Equifax’s consumer and business portals are down, so they can’t process any credit checks. Woe be unto you if you work in finance or support a business that depends on this service. I hope they didn’t have another breach. Maybe they will need a tiny excavator to dig them out of this mess like that freighter stuck in the Suez canal<https://www.reddit.com/r/CatastrophicFailure/comments/mcw8og/new_pictures_f…> for the last two days... Cheers, Brad

6 5

Cloudflare issues via Equinix Chicago IX
by Faisal Guerra 19 Mar '21

19 Mar '21

Hello, Anybody else experiencing issues with traffic to/from Cloudflare via Equinix Chicago? We have been seeing heavy packet loss for two/three days, and haven't received any word from Cloudflare when we reached out to them. Sample test: $ mtr 8.6.112.90 20c01.ring.nlnog.net (0.0.0.0) Fri Mar 19 18:00:11 2021 Keys: Help Display mode Restart statistics Order of fields quit Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. 204.130.133.130 0.0% 100 0.3 0.8 0.1 39.3 4.2 2. 208.200.137.246 0.0% 100 1.2 2.9 1.1 48.9 7.0 3. 64.127.129.13 0.0% 100 2.3 3.4 1.9 73.4 8.1 4. 208.115.136.180 16.0% 100 55.3 10.5 2.3 70.6 15.3 5. 198.41.140.65 18.0% 100 30.7 33.8 30.2 107.1 11.5 6. ??? 7. 8.6.112.90 24.0% 100 24.5 26.2 24.3 83.4 9.5 -- *Faisal Guerra* *Network Engineer* fguerra(a)365datacenters.com Phone: 212-792-9950 x 5 Support: 866-365-6246 [image: https://www.365datacenters.com] <https://www.365datacenters.com>

4 4

Facebook/Whatsapp/Instagram - intermittent issues
by Josh Luthman 19 Mar '21

19 Mar '21

Messages/links seem to be having issues. Downdetector is showing a spike in all 3 services. Prepare for the onslaught of calls saying my internet doesn't work. Instagram specifically doesn't load...possibly related? https://downdetector.com/status/instagram/ Josh Luthman 24/7 Help Desk: 937-552-2340 Direct: 937-552-2343 1100 Wayne St Suite 1337 Troy, OH 45373

1 0

DNS resolution in Europe
by Jason 17 Mar '21

17 Mar '21

We are getting alerted that URLs are not reachable from London, UK. I see in /r/sysadmin a thread re: Google's DNS, but it might be larger than Google to include Cloudflare and others. Any confirmation here? -- #FarmerPower farmobile.com <https://www.farmobile.com> Jason Forst | Farmobile LLC <https://www.farmobile.com> Director of Infrastructure Direct: 913-766-4625 <tel:913-766-4625> Office: 844-337-2255 <tel:844-337-2255> x 725 Mobile: 913-481-7325 <tel:913-481-7325> This email, including any attachments, may contain confidential information. If it has been sent to you in error, please reply to advise the sender of the error and then immediately delete the email.

7 6

Reports of Recaptcha failures
by Rich Lafferty 17 Mar '21

17 Mar '21

Lots of noise on Twitter and DownDetector that Google’s captcha-as-a-service, Recaptcha, is failing (and it sounds like it’s failing closed, refusing access to all). If you use it on your signup/signin pages you might want to turn that off until things clear up. https://twitter.com/search?q=recaptcha&src=typed_query&f=live https://downdetector.com/status/recaptcha/ Today, we are all robots. -Rich Rich Lafferty rich(a)lafferty.ca / 647-377-1165 Toronto, Canada http://www.lafferty.ca

1 0

Re: [outages] NetSol issues via Cloudflare in Toronto (ns61.worldnic.com) ?
by Rusty Dekema 17 Mar '21

17 Mar '21

[Learning to pay attention to my to: lines] This cleared up from here about 5 minutes ago. Both of the previously unreachable worldnic IPs are now reachable for me. -Rusty On Wed, Mar 17, 2021 at 11:25 AM Rusty Dekema <rdekema(a)gmail.com> wrote: > > I am having similar problems from AS 7922 near Detroit. I am unable to > reach ns61.worldnic.com/207.204.40.131 as well as > ns75.worldnic.com/207.204.40.138. > > Tracing route to ns75.worldnic.com [207.204.40.138] > over a maximum of 30 hops: > > 7 19 ms 18 ms 19 ms > be-2403-pe03.350ecermak.il.ibone.comcast.net [96.110.37.30] > 8 19 ms 18 ms 23 ms 173.167.56.42 > 9 21 ms 20 ms 19 ms 108.162.217.23 > 10 21 ms 20 ms 19 ms 108.162.217.140 > 11 * * * Request timed out. > > Cheers, > Rusty > > > > On Wed, Mar 17, 2021 at 11:06 AM mike tancsa via Outages > <outages(a)outages.org> wrote: > > > > I noticed a couple of domains that have netsol as authoritative timing > > out, specifically ns61.worldnic.COM resolving to 207.204.40.131 which I > > pick it up via Cloudflare in Toronto Canada via our Torix peer > > > > % traceroute -I -q1 -n 207.204.40.131 > > traceroute to 207.204.40.131 (207.204.40.131), 64 hops max, 48 byte packets > > 1 199.212.134.10 0.139 ms > > 2 205.211.165.118 4.263 ms > > 3 206.108.34.208 31.456 ms > > 4 108.162.240.24 4.331 ms > > 5 108.162.240.14 4.404 ms > > 6 * > > > > > > e.g > > > > % time host -tns washconnect.com > > washconnect.com name server ns62.worldnic.com. > > washconnect.com name server ns61.worldnic.com. > > > > % time host host1.washconnect.com ns62.worldnic.com. > > Using domain server: > > Name: ns62.worldnic.com. > > Address: 207.204.21.131#53 > > Aliases: > > > > host1.washconnect.com has address 204.186.31.41 > > 0.000u 0.002s 0:00.10 0.0% 0+0k 3+0io 0pf+0w > > > > % time host host1.washconnect.com ns61.worldnic.com. > > ;; connection timed out; no servers could be reached > > 0.000u 0.002s 0:10.05 0.0% 0+0k 3+0io 0pf+0w > > > > I am having problems with their whois server as well > > > > > > % time whois -h whois.networksolutions.com washconnect.com > > whois: connect(): Operation timed out > > 0.000u 0.001s 1:15.18 0.0% 0+0k 1+0io 0pf+0w > > > > route-views.routeviews.org doesnt see their prefix either :( > > > > route-views>sh ip bgp 64.69.216.61 > > BGP routing table entry for 0.0.0.0/0, version 33108733 > > Paths: (1 available, best #1, table default, RIB-failure(17)) > > Not advertised to any peer > > Refresh Epoch 1 > > 53767 3257 > > 162.251.163.2 from 162.251.163.2 (162.251.162.3) > > Origin IGP, localpref 100, valid, external, best > > Community: 53767:5000 > > path 7FE0F6ED4BE8 RPKI State not found > > rx pathid: 0, tx pathid: 0x0 > > route-views> > > > > > > % host whois.networksolutions.com > > whois.networksolutions.com has address 64.69.216.61 > > whois.networksolutions.com has IPv6 address 2607:f408:1010:a:64:69:216:61 > > > > % whois -h 2607:f408:1010:a:64:69:216:61 washconnect.com > > whois: connect(): Operation timed out > > > > > > > > > > > > > > _______________________________________________ > > Outages mailing list > > Outages(a)outages.org > > https://puck.nether.net/mailman/listinfo/outages

2 2

NetSol issues via Cloudflare in Toronto (ns61.worldnic.com) ?
by mike tancsa 17 Mar '21

17 Mar '21

I noticed a couple of domains that have netsol as authoritative timing out, specifically ns61.worldnic.COM resolving to 207.204.40.131 which I pick it up via Cloudflare in Toronto Canada via our Torix peer % traceroute -I -q1 -n 207.204.40.131 traceroute to 207.204.40.131 (207.204.40.131), 64 hops max, 48 byte packets 1 199.212.134.10 0.139 ms 2 205.211.165.118 4.263 ms 3 206.108.34.208 31.456 ms 4 108.162.240.24 4.331 ms 5 108.162.240.14 4.404 ms 6 * e.g % time host -tns washconnect.com washconnect.com name server ns62.worldnic.com. washconnect.com name server ns61.worldnic.com. % time host host1.washconnect.com ns62.worldnic.com. Using domain server: Name: ns62.worldnic.com. Address: 207.204.21.131#53 Aliases: host1.washconnect.com has address 204.186.31.41 0.000u 0.002s 0:00.10 0.0% 0+0k 3+0io 0pf+0w % time host host1.washconnect.com ns61.worldnic.com. ;; connection timed out; no servers could be reached 0.000u 0.002s 0:10.05 0.0% 0+0k 3+0io 0pf+0w I am having problems with their whois server as well % time whois -h whois.networksolutions.com washconnect.com whois: connect(): Operation timed out 0.000u 0.001s 1:15.18 0.0% 0+0k 1+0io 0pf+0w route-views.routeviews.org doesnt see their prefix either :( route-views>sh ip bgp 64.69.216.61 BGP routing table entry for 0.0.0.0/0, version 33108733 Paths: (1 available, best #1, table default, RIB-failure(17)) Not advertised to any peer Refresh Epoch 1 53767 3257 162.251.163.2 from 162.251.163.2 (162.251.162.3) Origin IGP, localpref 100, valid, external, best Community: 53767:5000 path 7FE0F6ED4BE8 RPKI State not found rx pathid: 0, tx pathid: 0x0 route-views> % host whois.networksolutions.com whois.networksolutions.com has address 64.69.216.61 whois.networksolutions.com has IPv6 address 2607:f408:1010:a:64:69:216:61 % whois -h 2607:f408:1010:a:64:69:216:61 washconnect.com whois: connect(): Operation timed out

2 1

srsplus.com and it's associated DNS seems down.
by Lyle Giese 17 Mar '21

17 Mar '21

I checked from my home Comcast and from my virtual server at IONOS and could not reach SRSPlus or resolve dns queries associated with domains associated with SRSPlus(dns1.srsplus.com or dns2.srsplus.com) Also checked Down for Everyone or Just me and it reports SRSPlus website is down. Lyle Giese LCR Computer Services, Inc.

1 0

After Action: Azure
by Jay Ashworth 16 Mar '21

16 Mar '21

Key rotation failure. https://www.zdnet.com/article/microsofts-latest-cloud-authentication-outage… Comments to -discuss. Cheers, -- jra -- Sent from my Android device with K-9 Mail. Please excuse my brevity.

2 1

Microsoft
by Garcia, Leo 15 Mar '21

15 Mar '21

Looks like there's an ongoing Microsoft service outage sine 3:30 PM Downdetector<https://downdetector.com/> Microsoft 365 Service health status (office365.com)<https://status.office365.com/> Teams messaging and calling seems to be down but emails are working OK for now. Regards, [cid:image001.png@01D719BC.9AB72980] MARRIOTT VACATIONS WORLDWIDE Leo Garcia Senior Technical Analyst INFORMATION TECHNOLOGY 24/7 Tech Ops 305-925-6379 T 305-925-7335 M 305-967-0833 9995 N Kendall Dr Miami, FL 33176 marriottvacationsworldwide.com NOTICE: This MVW communication may be confidential. Unauthorized copying, distribution, disclosure, or use is prohibited. If sent to you in error, please advise me and then delete. Not intended to be an electronic signature under applicable law.

1 0