August 2024 - Outages - Outages Lists

Major loss/latency on Twelve99 around NY/SJO
by Phil Lavin via Outages 15 Aug '24

15 Aug '24

It’s a good week for the Internet. Third outages list post this week :( Seeing major loss on Twelve99 around SJO: UK to AWS USW2: My traceroute [v0.95] VONMVPCVXYVFD (172.20.0.14) -> 216.147.3.209 (216.147.3.209) 2024-08-15T09:29:13+0100 Keys: Help Display mode Restart statistics Order of fields quit Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. 172.20.0.1 0.0% 9 1.1 1.2 1.0 1.7 0.2 2. lo0-0.bng2.ixn-lon.zen.net.uk 0.0% 9 5.5 6.8 5.4 11.2 1.9 3. lag-5.p2.ixn-lon.zen.net.uk 0.0% 8 5.5 6.0 5.2 6.5 0.5 4. ldn-b13-link.ip.twelve99.net 25.0% 8 5.8 5.5 5.0 6.0 0.4 5. ldn-bb2-link.ip.twelve99.net 0.0% 8 6.6 6.8 5.9 7.5 0.7 6. nyk-bb1-link.ip.twelve99.net 0.0% 8 75.2 75.2 74.5 75.9 0.5 7. sjo-b23-link.ip.twelve99.net 57.1% 8 296.2 312.5 296.2 343.7 27.0 8. (waiting for reply) 9. 15.230.28.67 62.5% 8 294.4 295.7 294.4 296.8 1.2 10. 54.240.242.119 25.0% 8 337.1 305.7 294.6 337.1 16.2 11. (waiting for reply) 12. 52.93.15.36 57.1% 8 398.2 400.8 395.4 408.8 7.1 13. 52.93.15.31 42.9% 8 388.7 395.5 388.7 403.3 6.2 14. 52.93.14.34 28.6% 8 431.8 432.2 393.2 488.2 35.8 15. 52.93.246.177 42.9% 8 397.7 399.3 397.7 401.1 1.4 16. 52.93.246.81 25.0% 8 393.4 396.9 393.2 400.6 3.0 17. (waiting for reply) 18. (waiting for reply)19. (waiting for re 28.6% 8 394.3 397.5 394.3 400.2 3.0 AWS USW2 to UK: Host Loss% Snt Last Avg Best Wrst StDev 1. ip-10-96-50-153.us-west-2.compute.internal 0.0% 87 0.2 0.1 0.1 0.2 0.0 2. ec2-44-233-118-55.us-west-2.compute.amazonaws.com 0.0% 87 7.3 11.9 1.0 66.9 16.4 ec2-34-221-151-47.us-west-2.compute.amazonaws.com ec2-44-233-117-187.us-west-2.compute.amazonaws.com 3. 240.0.212.4 0.0% 87 0.5 0.4 0.3 1.7 0.2 240.0.72.7 4. 242.2.110.95 0.0% 87 0.8 1.8 0.4 20.6 3.4 5. 240.2.144.13 0.0% 87 5.8 5.8 5.7 5.9 0.1 6. amazon-ic-375418.ip.twelve99-cust.net 0.0% 87 5.7 5.6 5.6 6.1 0.1 7. port-b4-link.ip.twelve99.net 0.0% 87 6.2 6.1 5.9 6.5 0.1 8. port-b3-link.ip.twelve99.net 0.0% 86 5.6 6.3 5.6 18.5 1.8 9. palo-b24-link.ip.twelve99.net 0.0% 86 21.4 21.3 21.1 21.6 0.1 10. nyk-bb2-link.ip.twelve99.net 19.8% 86 165.3 172.2 165.3 206.1 4.7 11. ldn-bb1-link.ip.twelve99.net 68.2% 86 391.5 401.2 387.3 411.8 6.3 12. ldn-b3-link.ip.twelve99.net 57.0% 86 397.1 401.4 395.5 409.0 2.7 13. zeninternet-ic-363994.ip.twelve99-cust.net 40.7% 86 397.0 412.3 397.0 473.3 18.0 14. lag-1.bng2.ixn-lon.zen.net.uk 63.5% 86 387.2 402.5 382.5 422.8 9.0 15. 62-3-100-30.dsl.in-addr.zen.co.uk 55.8% 86 391.6 404.5 391.6 412.5 6.1 16. (waiting for reply) _______________________________________________ Outages mailing list Outages(a)outages.org https://puck.nether.net/mailman/listinfo/outages

2 1

GitHub Service Outage
by Keith I Myers via Outages 14 Aug '24

14 Aug '24

ust a heads up, GitHub.com seems to be suffering a major outage at the moment. The Website is showing a error and the API is not responding. SSH access to git repos are also down. @All developers who rely on GitHub, enjoy your early end of day -- [image: --] Keith I Myers [image: http://]http://KMyers.me Mobile : (305)-929-3475 _______________________________________________ Outages mailing list Outages(a)outages.org https://puck.nether.net/mailman/listinfo/outages

2 1

GitHub ist kaputt
by Glenn Stone via Outages 14 Aug '24

14 Aug '24

GitHub is throwing the unicorn page. Status page sez they're working on it... whee. -- *Glenn Stone* / Sr. SRE - TechOps P: They/Them *M: *(internal) *TZ: *America/Los_Angeles _______________________________________________ Outages mailing list Outages(a)outages.org https://puck.nether.net/mailman/listinfo/outages

1 0

AS6453 / TATA - Issues around Singapore
by Phil Lavin via Outages 14 Aug '24

14 Aug '24

Hello, We are seeing significant packet loss on TATA on traffic leaving AWS in Singapore. The route changed to Cogent, briefly, which was OK. It’s now back on TATA and still suffering loss. My traceroute [v0.94] jcasc-rtc01 (10.36.21.252) -> hc-ping.com 2024-08-14T08:28:04+0000 Keys: Help Display mode Restart statistics Order of fields quit Packets Pings Host Loss% Snt Last Avg Best Wrst StDev ……. 7. 240.1.172.10 0.0% 53 0.8 0.9 0.8 1.1 0.1 8. 15.230.29.158 0.0% 52 11.0 8.9 0.7 53.8 11.3 9. ix-be-2.ecore1.svw-singapore.as6453.net 3.8% 52 1.0 1.0 0.9 4.2 0.5 10. if-bundle-41-2.qcore2.svw-singapore.as6453.net 82.4% 52 257.4 251.6 231.5 258.0 8.2 if-bundle-42-2.qcore1.svw-singapore.as6453.net 11. if-ae-33-2.tcore2.svw-singapore.as6453.net 68.6% 52 250.8 253.3 248.7 260.8 4.2 12. if-ae-44-2.tcore2.tv2-tokyo.as6453.net 61.5% 52 229.8 231.0 229.7 237.5 1.8 13. if-et-24-2.hcore2.kv8-chiba.as6453.net 88.2% 52 89.0 113.3 87.7 239.5 61.8 if-ae-28-5.tcore2.sv1-santaclara.as6453.net 14. if-bundle-20-2.qcore1.sqn-sanjose.as6453.net 62.7% 52 229.7 111.0 87.8 229.7 52.8 if-et-1-2.hcore1.kv8-chiba.as6453.net 15. if-ae-5-2.tcore2.sv1-santaclara.as6453.net 34.6% 52 247.6 247.7 241.7 250.8 1.6 63.243.218.16 16. 63.243.218.9 23.5% 52 238.5 246.0 230.5 251.2 5.2 if-bundle-20-2.qcore1.sqn-sanjose.as6453.net 17. 63.243.218.16 51.0% 52 248.7 252.4 247.6 258.6 3.9 18. 63.243.218.9 59.6% 52 256.9 251.8 247.1 259.0 3.9 19. (waiting for reply) _______________________________________________ Outages mailing list Outages(a)outages.org https://puck.nether.net/mailman/listinfo/outages

1 0

Twelve99 packet loss around London
by Phil Lavin via Outages 13 Aug '24

13 Aug '24

Seeing significant loss and latency to 8.8.8.8 over Twelve99. Looks to be around London/Dublin: My traceroute [v0.95] VONMVPCVXYVFD (172.20.0.76) -> 8.8.8.8 (8.8.8.8) 2024-08-12T14:32:13+0100 Keys: Help Display mode Restart statistics Order of fields quit Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. 172.20.0.1 0.0% 79 1.8 1.2 0.6 3.7 0.5 2. lo0-0.bng8.thn-lon.zen.net.uk 0.0% 79 6.8 7.3 5.1 30.1 3.3 3. 51-148-244-20.dsl.zen.co.uk 0.0% 79 5.9 6.5 5.2 9.8 1.0 4. ldn-b3-link.ip.twelve99.net 1.3% 78 5.6 6.7 5.3 14.1 1.3 5. ldn-bb2-link.ip.twelve99.net 1.3% 78 7.1 7.0 5.4 17.3 1.4 6. dln-b3-link.ip.twelve99.net 42.3% 78 15.6 15.9 14.9 18.2 0.7 7. 72.14.243.178 55.8% 78 32.4 38.4 16.6 54.5 11.4 8. 108.170.227.21 17.9% 78 32.3 34.7 16.7 54.4 11.2 9. 209.85.244.231 32.5% 78 28.3 35.1 18.6 53.6 10.1 10. dns.google 25.6% 78 38.2 34.0 16.2 54.4 10.3 _______________________________________________ Outages mailing list Outages(a)outages.org https://puck.nether.net/mailman/listinfo/outages

3 4

Akamai taking a hit?
by Casey Johnson via Outages 12 Aug '24

12 Aug '24

Anyone else noticing Akamai taking a hit tonight? Ping times and traceroutes from Comcast and Centurylink showing high when getting to their network. _______________________________________________ Outages mailing list Outages(a)outages.org https://puck.nether.net/mailman/listinfo/outages

1 0

Re: [outages] Outages Digest, Vol 190, Issue 1
by John-Paul Reed via Outages 05 Aug '24

05 Aug '24

It looks like Microsoft services may be having issues again. One Drive not loading properly, Microsoft Store issues and other MS issues. Regards, John-Paul Reed IT Consultant On Mon, Aug 5, 2024, 9:18 AM <outages-request(a)outages.org> wrote: > Send Outages mailing list submissions to > outages(a)outages.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://puck.nether.net/mailman/listinfo/outages > or, via email, send a message with subject or body 'help' to > outages-request(a)outages.org > > You can reach the person managing the list at > outages-owner(a)outages.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Outages digest..." > > > Today's Topics: > > 1. Re: Microsoft Azure, 365 and status page all apparently down. > (Shaun Potts) > 2. Re: Microsoft Azure, 365 and status page all apparently down. > (Josh Luthman) > 3. Centurylink down in Colorado (Casey Johnson) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 30 Jul 2024 13:09:11 -0400 > From: Shaun Potts <shaun.m.potts(a)gmail.com> > To: Bill Woodcock <woody(a)pch.net> > Cc: Jeff Martinez <jeff(a)bbdp.com>, Chris Adams via Outages > <outages(a)outages.org> > Subject: Re: [outages] Microsoft Azure, 365 and status page all > apparently down. > Message-ID: > < > CAMQvQDfa5Sry7Ofbkkb4gQnRo992hEn89tnTojVGZc88h8hiQQ(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > i find it *really **really *hard to believe that you or your org doesn't > use AD, or GitHub, or any external service that interfaces with > Entra/AAD/whatever microsoft has named it this month, or <you name it here> > but if that's the case, good on you. other cloud providers have outages > too, just as bad, and in the grand scheme of things just as frequently, > just not recently. > > also worth saying, crowdstrike still wasn't a microsoft issue, aside from > being the affected platform. crowdstrike has caused kernel panics on linux > machines too *shrug* > > anyways, not looking to get into a nerd fight with someone who has > realistically done a lot for the world's internet infrastructure, > > it always just really saddens me when i see people in prestigious positions > in tech taking positions like this, which are really below people in their > station. i guess i would just say try to put on a better face as someone > people in tech should be looking up to instead of gaining internet points > by batting at low-hanging fruit when a major provider has a major outage. > > next time layer3 or akamai or <insert backbone provider here> has a major > outage, would you quickly jump to go "haha, not affected here either!" or > are you just letting personal bias get in the way of rational discourse? > > my $0.02 > > (p.s. neither of the companies i work for or make technical decisions for > have a huge foot in microsoft tech either, this is just me being purely > objective) > > On Tue, Jul 30, 2024 at 12:49?PM Bill Woodcock <woody(a)pch.net> wrote: > > > > > > > > On Jul 30, 2024, at 18:37, Shaun Potts <shaun.m.potts(a)gmail.com> > wrote: > > > right because no vendor any of us use has interdependencies with > > microsoft services > > > > That was exactly my point. Some people make choices which prove to have > > Microsoft dependencies, some don?t. It?s a choice. Some people make it > > each way. Then Microsoft has yet another outage, as they VERY FREQUENTLY > > do, and somehow, some people are surprised, all over again. But it was > > their choice. Some people try to convince themselves that Microsoft is > in > > the ?death and taxes? category. This is just your reminder that it?s > not, > > it?s in the ?self harm/own-goal? category. If you were affected, AGAIN, > > you could make other choices, and join the camp that doesn?t have > anything > > to complain about when Microsoft has an outage AGAIN. > > > > Admittedly, I was a bit worried at first when I saw airlines complaining > > about the Crowdstrike/Microsoft outage, but my travel was all on United, > > and it turned out that they hadn?t made the mistake of using either > > anywhere critical, just on kiosk display stuff. So, good for them, > though > > it would be better if they didn?t flirt with these kinds of risks at all, > > since they seemingly propagate when people aren?t paying attention. > > > > -Bill > > > > > _______________________________________________ Outages mailing list Outages(a)outages.org https://puck.nether.net/mailman/listinfo/outages

1 0

Centurylink down in Colorado
by Casey Johnson via Outages 05 Aug '24

05 Aug '24

Anyone with Lumen/Centurylink fiber or MPLS circuits in Colorado down? I have 5 customers that are down hard right now. _______________________________________________ Outages mailing list Outages(a)outages.org https://puck.nether.net/mailman/listinfo/outages

5 5