April 2023 - Outages - Outages Lists

Allstream / Zayo SIP outage Ontario, Canada ?
by mike tancsa 26 Apr '23

26 Apr '23

All of a sudden cant make outbound SIP calls, but inbound seems to be OK. Looks like others as well according to https://downdetector.ca/status/allstream/ Anyone know any details ? ---Mike

1 1

SoCal T-Mobile IPv4 Outage?
by Samuel Jackson 24 Apr '23

24 Apr '23

I am checking in to see if this is isolated to just us. On T-Mobile, we do not seem to be able to get to ipv4 sites. ipv4.google.com does not load, ipv6.google.com does. Even www.t-mobile.com does not seem to be working. Tried this with LTE (phone) and their home-internet service. Anyone else seeing this? Thanks in advance.

1 0

Spectrum/Charter IPv6 connectivity to CloudFlare down in Texas
by Carl Perry 21 Apr '23

21 Apr '23

Things have been rather slow the last couple of days, just figured out why; No IPv6 connectivity to any CloudFlare services. > mtr -6 --report-wide cloudflare.com Start: 2023-04-19T23:20:42+0000 HOST: defiant Loss% Snt Last Avg Best Wrst StDev 1.|-- 2603-8081-2340-2868-0215-17ff-fe94-6457.res6.spectrum.com 0.0% 10 3.6 19.1 1.5 93.2 36.1 2.|-- 2603-90c5-0003-0180-0000-0000-0000-0001.inf6.spectrum.com 0.0% 10 14.5 36.6 11.3 171.2 50.0 3.|-- lag-63.hcr02rdrktxaz.netops.charter.com 0.0% 10 18.8 37.3 12.0 143.9 45.3 4.|-- lag-19.ausxtxir02r.netops.charter.com 60.0% 10 65.8 30.2 11.7 65.8 24.9 5.|-- lag-22.rcr01hstqtx02.netops.charter.com 80.0% 10 26.5 25.5 24.5 26.5 1.4 6.|-- lag-16.hstqtx0209w-bcr00.netops.charter.com 90.0% 10 25.7 25.7 25.7 25.7 0.0 7.|-- lag-12.dllstx976iw-bcr00.netops.charter.com 40.0% 10 25.4 26.3 18.2 44.4 9.5 8.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 IPv4 works however: > mtr -4 --report-wide cloudflare.com Start: 2023-04-19T23:20:21+0000 HOST: defiant Loss% Snt Last Avg Best Wrst StDev 1.|-- _gateway 0.0% 10 2.1 1.8 1.4 2.1 0.2 2.|-- cpe-70-114-160-1.austin.res.rr.com 0.0% 10 51.9 34.4 14.3 149.7 42.1 3.|-- lag-63.hcr01rdrktxaz.netops.charter.com 0.0% 10 13.1 38.5 13.1 159.1 44.6 4.|-- lag-19.ausutxla01r.netops.charter.com 0.0% 10 18.1 21.0 14.3 34.8 6.0 5.|-- lag-22.rcr01dllatx37.netops.charter.com 0.0% 10 23.4 23.3 18.3 33.3 4.3 6.|-- lag-14.dllstx976iw-bcr00.netops.charter.com 0.0% 10 19.8 27.0 16.1 51.2 11.9 7.|-- lag-0.pr3.dfw10.netops.charter.com 0.0% 10 39.3 28.5 14.1 62.3 16.7 8.|-- 141.101.74.104 40.0% 10 53.2 44.3 19.4 120.2 39.2 9.|-- 172.71.168.2 0.0% 10 28.3 28.8 10.8 79.4 19.5 10.|-- 104.16.133.229 0.0% 10 122.0 42.8 16.9 122.0 34.4 It's kind of wild how many things don't fall back to IPv4 quickly (lots of Android TV apps) or at all (.NET). I tried reaching out to Spectrum support, but being a lowly consumer I was only told that "The modem doesn't have any filters" and that "the problem may be on the CloudFlare side", which was sadly what I predicted. So, I figured I would raise the issue here in the hopes that actual network people would see it. -Carl

3 4

M365 application degradation workaround
by Chris Wallace 20 Apr '23

20 Apr '23

The service is degraded in some tenants and Microsoft is investigating - it's an active incident in the service health page for my tenant; your mileage may vary. You/your users can use this direct links for applications (it worked on multiple tenants I tried and showed all applications, not just custom): https://myapplications.microsoft.com/ Additionally, directly to the admin center: https://admin.microsoft.com/Adminportal/Home#/homepage Chris -----Original Message----- From: Outages <outages-bounces(a)outages.org> On Behalf Of outages-request(a)outages.org Sent: Thursday, April 20, 2023 9:54 AM To: outages(a)outages.org Subject: Outages Digest, Vol 174, Issue 8 [EXTERNAL] Send Outages mailing list submissions to outages(a)outages.org To subscribe or unsubscribe via the World Wide Web, visit https://puck.nether.net/mailman/listinfo/outages or, via email, send a message with subject or body 'help' to outages-request(a)outages.org You can reach the person managing the list at outages-owner(a)outages.org When replying, please edit your Subject line so it is more specific than "Re: Contents of Outages digest..." Today's Topics: 1. Re: Office365 (Keith Stokes) ---------------------------------------------------------------------- Message: 1 Date: Thu, 20 Apr 2023 13:54:11 +0000 From: Keith Stokes <keiths(a)salonbiz.com> To: "outages(a)outages.org" <outages(a)outages.org>, Bruce Freshwater <BFreshwater(a)SierraExperts.com> Subject: Re: [outages] Office365 Message-ID: <PH0PR22MB32007B4CF6269468CD531C37B3639(a)PH0PR22MB3200.namprd22.prod.outlook.com> Content-Type: text/plain; charset="us-ascii" One of my company users in Tampa reported the same. ________________________________ From: Outages <outages-bounces(a)outages.org> on behalf of Bruce Freshwater via Outages <outages(a)outages.org> Sent: Thursday, April 20, 2023 8:49 AM To: outages(a)outages.org <outages(a)outages.org> Subject: [outages] Office365 Receiving multiple reports of customers missing apps in Office365, anyone else seeing this? Note: this is not a new user [cid:image001.png@01D9736D.68CFF930] Regards, Bruce Freshwater Direct: 412.722.1701 Mobile: 412.292.0282 Main: 866.707.5869 Fax: 412.774.2469 Email: BFreshwater(a)SierraExperts.com<mailto:BFreshwater@SierraExperts.com> Request Help: Support Portal (SIM)<sim.sierraincidentmanager.com> Sierra Experts [cid:image002.jpg@01D1C645.CF870330] 2 Robinson Plaza, Suite 300, Pittsburgh, PA 15205-1017 Our Services: Managed Services<http://www.sierraexperts.com/managed-services/> | Development & Design<http://www.sierraexperts.com/development-and-design/> | Telephony & VoIP<http://www.sierraexperts.com/telephony-and-voip/> | Hardware & Software<http://www.sierraexperts.com/hardware-and-software/> | Data Center<http://sierradatacenters.com/> Stay Connected: Website<http://www.sierraexperts.com/> | Awards<http://www.sierraexperts.com/about-us/awards/> | Facebook<https://www.facebook.com/SierraExperts> | Twitter<https://twitter.com/sierraexperts> | LinkedIn<https://www.linkedin.com/company/sierraexperts> | Blog<http://www.sierraexperts.com/about-us/blog/> [cid:image001.png@01D573C1.8B84DB80]

1 0

Re: [outages] Office365
by Joseph Wilker 20 Apr '23

20 Apr '23

>From Microsoft Status Portal: Some users may be intermittently unable to view or access web apps in Microsoft 365 OO544150, Last updated: April 20, 2023 8:29 AM Estimated start time: April 20, 2023 8:19 AM Affected services: Microsoft 365 for the web Issue type: Incident Issue origin: Microsoft Status: Service degradation User impact Users may be intermittently unable to view or access web apps in Microsoft 365. Title: Some users may be intermittently unable to view or access web apps in Microsoft 365 User impact: Users may be intermittently unable to view or access web apps in Microsoft 365. Current status: We're reviewing service monitoring telemetry to isolate the root cause and develop a remediation plan. Scope of impact: Impact is specific to some users who are served through the affected infrastructure. Next update by: Thursday, April 20, 2023, 10:30 AM (3:30 PM UTC) __________________________ Joseph Wilker Infrastructure Manager ITSO - SysNet Operations (608)663-6719 (608)663-6788 fax Edgewood College 1000 Edgewood College Dr. Madison, WI 53711 From: Outages <outages-bounces(a)outages.org<mailto:outages-bounces@outages.org>> On Behalf Of Bruce Freshwater via Outages Sent: Thursday, April 20, 2023 8:50 AM To: outages(a)outages.org<mailto:outages@outages.org> Subject: [outages] Office365 CAUTION: This email did not originate from Edgewood College. Do not click links or open attachments unless you've verified the content is safe. Receiving multiple reports of customers missing apps in Office365, anyone else seeing this? Note: this is not a new user [cid:image001.png@01D97365.EFA920B0] Regards, Bruce Freshwater Direct: 412.722.1701 Mobile: 412.292.0282 Main: 866.707.5869 Fax: 412.774.2469 Email: BFreshwater(a)SierraExperts.com<mailto:BFreshwater@SierraExperts.com> Request Help: Support Portal (SIM)<http://sim.sierraincidentmanager.com/> Sierra Experts [cid:image002.jpg@01D97365.EFA920B0] 2 Robinson Plaza, Suite 300, Pittsburgh, PA 15205-1017 Our Services: Managed Services<http://www.sierraexperts.com/managed-services/> | Development & Design<http://www.sierraexperts.com/development-and-design/> | Telephony & VoIP<http://www.sierraexperts.com/telephony-and-voip/> | Hardware & Software<http://www.sierraexperts.com/hardware-and-software/> | Data Center<http://sierradatacenters.com/> Stay Connected: Website<http://www.sierraexperts.com/> | Awards<http://www.sierraexperts.com/about-us/awards/> | Facebook<https://www.facebook.com/SierraExperts> | Twitter<https://twitter.com/sierraexperts> | LinkedIn<https://www.linkedin.com/company/sierraexperts> | Blog<http://www.sierraexperts.com/about-us/blog/> [cid:image003.png@01D97365.EFA920B0]

1 0

Office365
by Bruce Freshwater 20 Apr '23

20 Apr '23

Receiving multiple reports of customers missing apps in Office365, anyone else seeing this? Note: this is not a new user [cid:image001.png@01D9736D.68CFF930] Regards, Bruce Freshwater Direct: 412.722.1701 Mobile: 412.292.0282 Main: 866.707.5869 Fax: 412.774.2469 Email: BFreshwater(a)SierraExperts.com<mailto:BFreshwater@SierraExperts.com> Request Help: Support Portal (SIM)<sim.sierraincidentmanager.com> Sierra Experts [cid:image002.jpg@01D1C645.CF870330] 2 Robinson Plaza, Suite 300, Pittsburgh, PA 15205-1017 Our Services: Managed Services<http://www.sierraexperts.com/managed-services/> | Development & Design<http://www.sierraexperts.com/development-and-design/> | Telephony & VoIP<http://www.sierraexperts.com/telephony-and-voip/> | Hardware & Software<http://www.sierraexperts.com/hardware-and-software/> | Data Center<http://sierradatacenters.com/> Stay Connected: Website<http://www.sierraexperts.com/> | Awards<http://www.sierraexperts.com/about-us/awards/> | Facebook<https://www.facebook.com/SierraExperts> | Twitter<https://twitter.com/sierraexperts> | LinkedIn<https://www.linkedin.com/company/sierraexperts> | Blog<http://www.sierraexperts.com/about-us/blog/> [cid:image001.png@01D573C1.8B84DB80]

2 1

freedns.afraid.org
by John Kristoff 19 Apr '23

19 Apr '23

Appears to be an uncharacteristically prolonged outage with this well known free and dynamic DNS provider. Also see: <https://www.reddit.com/r/sysadmin/comments/12rksq5/looks_like_freednsafraid…> John

1 0

Lots of USPS stuff appears broken
by Aaron de Bruyn 18 Apr '23

18 Apr '23

A lot of stuff under https://usps.com <https://usps.com> appears broken. PO Box renewal, their support link, "informed delivery", etc... Pages are returning 400 Bad Request Your browser sent a request that this server could not understand. Size of a request header field exceeds server limit. Tried it from several locations throughout Washington and Oregon and a mix of Charter, Comcast and two lesser-known providers. -A

3 2

ipv6 packet loss in DEN for comcast?
by Paul Ebersman 18 Apr '23

18 Apr '23

I'm seeing heavy packet loss within DEN metro for comcast on ipv6 (but not ipv4). mtr to my mail host follows. packet loss the same hop for any site on the internet i try to reach via ipv6. anyone else seeing issues? HOST: localhost. Loss% Snt Last Avg Best Wrst StDev 1.|-- xxxxxxxxxxxxxxxxx 0.0% 10 4.1 4.8 2.5 12.9 2.9 2.|-- xxxxxxxxxxxxxxxxx 0.0% 10 14.2 15.4 13.1 20.2 2.2 3.|-- ae-251-1203-rur01.monument.co.denver.comcast.net 0.0% 10 15.9 20.7 13.2 36.0 8.2 4.|-- ae-12-ar01.denver.co.denver.comcast.net 0.0% 10 23.5 20.6 14.9 32.9 5.4 5.|-- 2001:558:1c0:1058::1 40.0% 10 15.4 18.1 14.8 27.3 4.8 6.|-- be-36031-cs03.1601milehigh.co.ibone.comcast.net 90.0% 10 15.8 15.8 15.8 15.8 0.0 7.|-- be-1314-cr14.1601milehigh.co.ibone.comcast.net 40.0% 10 17.8 17.2 16.1 17.8 0.6 8.|-- be-303-cr14.champa.co.ibone.comcast.net 70.0% 10 18.9 21.1 18.4 26.0 4.2 9.|-- be-1214-cs02.champa.co.ibone.comcast.net 60.0% 10 16.8 17.3 16.7 17.8 0.6 10.|-- be-1212-cr12.champa.co.ibone.comcast.net 30.0% 10 17.2 19.4 15.7 25.5 3.3 11.|-- be-301-cr11.sunnyvale.ca.ibone.comcast.net 0.0% 10 42.2 42.1 39.6 43.8 1.3 12.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 13.|-- be-3202-pe02.529bryant.ca.ibone.comcast.net 0.0% 10 42.6 43.4 40.7 48.3 2.3 14.|-- 2001:559::14e 0.0% 10 42.7 43.5 41.3 48.8 2.1 15.|-- mail.dragon.net 0.0% 10 42.1 42.7 40.7 47.5 1.9

2 2

AWS us-west-2 packet loss
by Denys Korotkyi 18 Apr '23

18 Apr '23

Is there anyone from AWS us-west networking team who can ping me off-list regarding the following issue? MTR shows packet loss within Amazon backbone when reaching both AWS VPN endpoints. Thank you in advance. Cheers, Den Korotkyi <denys.korotkyi(a)dreamhost.com> | Network Engineer | Dreamhost <http://www.dreamhost.com> M: +380509910515

2 1