Major loss/latency on Twelve99 around NY/SJO
It’s a good week for the Internet. Third outages list post this week :( Seeing major loss on Twelve99 around SJO: UK to AWS USW2: My traceroute [v0.95] VONMVPCVXYVFD (172.20.0.14) -> 216.147.3.209 (216.147.3.209) 2024-08-15T09:29:13+0100 Keys: Help Display mode Restart statistics Order of fields quit Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. 172.20.0.1 0.0% 9 1.1 1.2 1.0 1.7 0.2 2. lo0-0.bng2.ixn-lon.zen.net.uk 0.0% 9 5.5 6.8 5.4 11.2 1.9 3. lag-5.p2.ixn-lon.zen.net.uk 0.0% 8 5.5 6.0 5.2 6.5 0.5 4. ldn-b13-link.ip.twelve99.net 25.0% 8 5.8 5.5 5.0 6.0 0.4 5. ldn-bb2-link.ip.twelve99.net 0.0% 8 6.6 6.8 5.9 7.5 0.7 6. nyk-bb1-link.ip.twelve99.net 0.0% 8 75.2 75.2 74.5 75.9 0.5 7. sjo-b23-link.ip.twelve99.net 57.1% 8 296.2 312.5 296.2 343.7 27.0 8. (waiting for reply) 9. 15.230.28.67 62.5% 8 294.4 295.7 294.4 296.8 1.2 10. 54.240.242.119 25.0% 8 337.1 305.7 294.6 337.1 16.2 11. (waiting for reply) 12. 52.93.15.36 57.1% 8 398.2 400.8 395.4 408.8 7.1 13. 52.93.15.31 42.9% 8 388.7 395.5 388.7 403.3 6.2 14. 52.93.14.34 28.6% 8 431.8 432.2 393.2 488.2 35.8 15. 52.93.246.177 42.9% 8 397.7 399.3 397.7 401.1 1.4 16. 52.93.246.81 25.0% 8 393.4 396.9 393.2 400.6 3.0 17. (waiting for reply) 18. (waiting for reply)19. (waiting for re 28.6% 8 394.3 397.5 394.3 400.2 3.0 AWS USW2 to UK: Host Loss% Snt Last Avg Best Wrst StDev 1. ip-10-96-50-153.us-west-2.compute.internal 0.0% 87 0.2 0.1 0.1 0.2 0.0 2. ec2-44-233-118-55.us-west-2.compute.amazonaws.com 0.0% 87 7.3 11.9 1.0 66.9 16.4 ec2-34-221-151-47.us-west-2.compute.amazonaws.com ec2-44-233-117-187.us-west-2.compute.amazonaws.com 3. 240.0.212.4 0.0% 87 0.5 0.4 0.3 1.7 0.2 240.0.72.7 4. 242.2.110.95 0.0% 87 0.8 1.8 0.4 20.6 3.4 5. 240.2.144.13 0.0% 87 5.8 5.8 5.7 5.9 0.1 6. amazon-ic-375418.ip.twelve99-cust.net 0.0% 87 5.7 5.6 5.6 6.1 0.1 7. port-b4-link.ip.twelve99.net 0.0% 87 6.2 6.1 5.9 6.5 0.1 8. port-b3-link.ip.twelve99.net 0.0% 86 5.6 6.3 5.6 18.5 1.8 9. palo-b24-link.ip.twelve99.net 0.0% 86 21.4 21.3 21.1 21.6 0.1 10. nyk-bb2-link.ip.twelve99.net 19.8% 86 165.3 172.2 165.3 206.1 4.7 11. ldn-bb1-link.ip.twelve99.net 68.2% 86 391.5 401.2 387.3 411.8 6.3 12. ldn-b3-link.ip.twelve99.net 57.0% 86 397.1 401.4 395.5 409.0 2.7 13. zeninternet-ic-363994.ip.twelve99-cust.net 40.7% 86 397.0 412.3 397.0 473.3 18.0 14. lag-1.bng2.ixn-lon.zen.net.uk 63.5% 86 387.2 402.5 382.5 422.8 9.0 15. 62-3-100-30.dsl.in-addr.zen.co.uk 55.8% 86 391.6 404.5 391.6 412.5 6.1 16. (waiting for reply) _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
Can confirm this from AT&T Uverse (fibre) out of California as well. mtr is below. Please note that loss shown at hops #5 and #6 tends to be normal (i.e. ICMP prioritisation). Hard to say where the issue actually is, and I obviously can't see the return path. Due to the varying providers discussed, I suspect the problem is with Arelion/Telia and not with AT&T / AWS / etc.. I HAVE NOT tried using Arelion/Telia's Looking Glass yet, as it's too late in the evening for me to dig into this: https://lg.twelve99.net/ XXXXXXXXXXXXXXX (XXXXXXXXXXXX) -> www.oo-software.com 2024-08-15T01:48:59-0700 Host Loss% Snt Rcv Last Avg Best Wrst 1. XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 0.0% 16 16 0.3 0.4 0.3 0.5 2. 172-10-232-1.lightspeed.sntcca.sbcglobal.net (172.10.232.1) 0.0% 16 16 1.9 2.9 1.2 5.3 3. 71.148.149.42 (71.148.149.42) 0.0% 16 16 2.7 3.1 2.0 5.4 4. (waiting for reply) 5. 32.130.91.81 (32.130.91.81) 93.3% 16 1 7.5 7.5 7.5 7.5 6. 192.205.32.182 (192.205.32.182) 25.0% 16 12 6.2 7.3 5.9 9.4 7. nyk-bb1-link.ip.twelve99.net (62.115.118.110) 60.0% 16 6 279.3 280.4 278.9 281.9 8. ldn-bb2-link.ip.twelve99.net (62.115.112.245) 68.8% 16 5 351.7 352.1 349.7 354.3 9. prs-bb2-link.ip.twelve99.net (62.115.133.239) 40.0% 16 9 367.6 365.1 361.1 369.8 10. ffm-bb2-link.ip.twelve99.net (62.115.122.139) 93.3% 16 1 371.0 371.0 371.0 371.0 11. ffm-b1-link.ip.twelve99.net (62.115.121.7) 56.2% 16 7 371.8 373.4 368.8 377.8 12. ffm-bb1-link.ip.twelve99.net (62.115.121.0) 40.0% 16 9 375.4 371.6 368.3 375.4 13. (waiting for reply) 14. ionos-ic-350360.ip.twelve99-cust.net (62.115.181.11) 33.3% 16 10 366.0 370.1 366.0 374.8 15. 11-INTERNET.ear6.Frankfurt1.Level3.net (62.67.23.38) 66.7% 16 5 366.9 367.7 363.7 370.4 16. lo-0-0.rc-a.bap.rhr.de.net.ionos.com (212.227.117.208) 33.3% 16 10 373.7 374.7 371.6 377.4 17. port-channel-7.de-rhr-bangdsr01.net.ionos.com (212.227.120.227) 40.0% 16 9 372.6 374.0 369.8 378.7 18. 212.227.36.18 (212.227.36.18) 40.0% 16 9 371.3 373.4 370.0 378.3 19. (waiting for reply) -- | Jeremy Chadwick jdc@koitsu.org | | UNIX Systems Administrator PGP 0x2A389531 | | Making life hard for others since 1977. | On Thu, Aug 15, 2024 at 09:34:14AM +0100, Phil Lavin via Outages wrote:
It’s a good week for the Internet. Third outages list post this week :(
Seeing major loss on Twelve99 around SJO:
UK to AWS USW2:
My traceroute [v0.95] VONMVPCVXYVFD (172.20.0.14) -> 216.147.3.209 (216.147.3.209) 2024-08-15T09:29:13+0100 Keys: Help Display mode Restart statistics Order of fields quit Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. 172.20.0.1 0.0% 9 1.1 1.2 1.0 1.7 0.2 2. lo0-0.bng2.ixn-lon.zen.net.uk 0.0% 9 5.5 6.8 5.4 11.2 1.9 3. lag-5.p2.ixn-lon.zen.net.uk 0.0% 8 5.5 6.0 5.2 6.5 0.5 4. ldn-b13-link.ip.twelve99.net 25.0% 8 5.8 5.5 5.0 6.0 0.4 5. ldn-bb2-link.ip.twelve99.net 0.0% 8 6.6 6.8 5.9 7.5 0.7 6. nyk-bb1-link.ip.twelve99.net 0.0% 8 75.2 75.2 74.5 75.9 0.5 7. sjo-b23-link.ip.twelve99.net 57.1% 8 296.2 312.5 296.2 343.7 27.0 8. (waiting for reply) 9. 15.230.28.67 62.5% 8 294.4 295.7 294.4 296.8 1.2 10. 54.240.242.119 25.0% 8 337.1 305.7 294.6 337.1 16.2 11. (waiting for reply) 12. 52.93.15.36 57.1% 8 398.2 400.8 395.4 408.8 7.1 13. 52.93.15.31 42.9% 8 388.7 395.5 388.7 403.3 6.2 14. 52.93.14.34 28.6% 8 431.8 432.2 393.2 488.2 35.8 15. 52.93.246.177 42.9% 8 397.7 399.3 397.7 401.1 1.4 16. 52.93.246.81 25.0% 8 393.4 396.9 393.2 400.6 3.0 17. (waiting for reply) 18. (waiting for reply)19. (waiting for re 28.6% 8 394.3 397.5 394.3 400.2 3.0
AWS USW2 to UK:
Host Loss% Snt Last Avg Best Wrst StDev 1. ip-10-96-50-153.us-west-2.compute.internal 0.0% 87 0.2 0.1 0.1 0.2 0.0 2. ec2-44-233-118-55.us-west-2.compute.amazonaws.com 0.0% 87 7.3 11.9 1.0 66.9 16.4 ec2-34-221-151-47.us-west-2.compute.amazonaws.com ec2-44-233-117-187.us-west-2.compute.amazonaws.com 3. 240.0.212.4 0.0% 87 0.5 0.4 0.3 1.7 0.2 240.0.72.7 4. 242.2.110.95 0.0% 87 0.8 1.8 0.4 20.6 3.4 5. 240.2.144.13 0.0% 87 5.8 5.8 5.7 5.9 0.1 6. amazon-ic-375418.ip.twelve99-cust.net 0.0% 87 5.7 5.6 5.6 6.1 0.1 7. port-b4-link.ip.twelve99.net 0.0% 87 6.2 6.1 5.9 6.5 0.1 8. port-b3-link.ip.twelve99.net 0.0% 86 5.6 6.3 5.6 18.5 1.8 9. palo-b24-link.ip.twelve99.net 0.0% 86 21.4 21.3 21.1 21.6 0.1 10. nyk-bb2-link.ip.twelve99.net 19.8% 86 165.3 172.2 165.3 206.1 4.7 11. ldn-bb1-link.ip.twelve99.net 68.2% 86 391.5 401.2 387.3 411.8 6.3 12. ldn-b3-link.ip.twelve99.net 57.0% 86 397.1 401.4 395.5 409.0 2.7 13. zeninternet-ic-363994.ip.twelve99-cust.net 40.7% 86 397.0 412.3 397.0 473.3 18.0 14. lag-1.bng2.ixn-lon.zen.net.uk 63.5% 86 387.2 402.5 382.5 422.8 9.0 15. 62-3-100-30.dsl.in-addr.zen.co.uk 55.8% 86 391.6 404.5 391.6 412.5 6.1 16. (waiting for reply) _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
participants (2)
-
Jeremy Chadwick via Outages -
Phil Lavin via Outages