A few minutes ago I was notified that one of my servers in Denver, CO was down (ip: 72.18.145.210). Anyone seeing a similar loss on he.net in Chicago? nog2.angryox.com (0.0.0.0) Tue Apr 26 15:54:54 2016 Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. xl0.tic.nova.org 0.0% 36 0.1 0.1 0.1 1.0 0.0 2. wsip-70-168-111-17.dc.dc.cox.net 2.8% 36 1.8 13.3 1.1 183.2 40.1 3. ip68-100-3-109.dc.dc.cox.net 0.0% 36 1.1 3.0 1.1 25.6 4.9 4. ashbbprj01-ae5.0.rd.as.cox.net 0.0% 36 1.7 3.6 1.7 20.7 4.3 5. 68.105.30.114 2.8% 36 7.8 15.1 1.8 120.4 21.0 6. 10ge3-6.core2.chi1.he.net 2.8% 36 33.2 35.2 28.1 51.4 6.9 7. ?? But from Verizon it is up: max.angryox.com (0.0.0.0) Tue Apr 26 15:56:08 2016 Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. tomato.angryox.com 0.0% 158 0.3 0.4 0.3 2.4 0.2 2. lo0-100.WASHDC-VFTTP-312.verizon-gni 0.0% 158 3.7 7.0 3.0 153.9 13.1 3. T1-3-0-4.WASHDC-LCR-21.verizon-gni.n 0.0% 158 7.9 8.1 3.1 156.9 12.0 4. ??? 5. ??? 6. 0.ae3.GW11.IAD8.ALTER.NET 0.0% 157 6.5 8.5 5.4 118.2 12.1 7. teliasonera-gw.customer.alter.net 0.0% 157 7.2 9.0 5.5 164.1 15.7 8. ash-bb3-link.telia.net 0.0% 157 8.0 8.9 5.5 148.9 14.4 9. sjo-b21-link.telia.net 0.0% 157 71.1 72.6 68.8 150.8 7.2 10. den-b1-link.telia.net 0.0% 157 99.3 99.6 98.0 122.4 2.4 11. handynet-ic-145041-den-b1.c.telia.ne 0.0% 157 130.4 100.1 98.0 130.4 3.3 12. ae2-14.core1.denver2.handynetworks.c 0.0% 157 97.1 97.1 95.5 113.8 1.8 13. ae0-10.dist2.denver2.handynetworks.c 0.0% 157 99.5 103.9 95.6 828.6 58.2 14. vlan954.sw2-g4.denver2.handynetworks 0.0% 157 98.5 104.7 98.1 761.2 52.8 15. vlan11.sw1-h3.softsyshosting.com 0.0% 157 99.8 103.8 97.1 430.8 33.8 16. 72.18.145.210 0.0% 157 97.3 97.1 95.5 128.1 2.7 --------------------------------------------------------------------------- Peter Beckman Internet Guy beckman@angryox.com http://www.angryox.com/ ---------------------------------------------------------------------------
We noticed a large dip in traffic from HE in Minnesota to Chicago as well but I didn't have time to run any traces before it returned to normal now. On Tue, Apr 26, 2016 at 2:59 PM, Peter Beckman via Outages < outages@outages.org> wrote:
A few minutes ago I was notified that one of my servers in Denver, CO was down (ip: 72.18.145.210).
Anyone seeing a similar loss on he.net in Chicago?
nog2.angryox.com (0.0.0.0) Tue Apr 26 15:54:54 2016 Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. xl0.tic.nova.org 0.0% 36 0.1 0.1 0.1 1.0 0.0 2. wsip-70-168-111-17.dc.dc.cox.net 2.8% 36 1.8 13.3 1.1 183.2 40.1 3. ip68-100-3-109.dc.dc.cox.net 0.0% 36 1.1 3.0 1.1 25.6 4.9 4. ashbbprj01-ae5.0.rd.as.cox.net 0.0% 36 1.7 3.6 1.7 20.7 4.3 5. 68.105.30.114 2.8% 36 7.8 15.1 1.8 120.4 21.0 6. 10ge3-6.core2.chi1.he.net 2.8% 36 33.2 35.2 28.1 51.4 6.9 7. ??
But from Verizon it is up:
max.angryox.com (0.0.0.0) Tue Apr 26 15:56:08 2016 Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. tomato.angryox.com 0.0% 158 0.3 0.4 0.3 2.4 0.2 2. lo0-100.WASHDC-VFTTP-312.verizon-gni 0.0% 158 3.7 7.0 3.0 153.9 13.1 3. T1-3-0-4.WASHDC-LCR-21.verizon-gni.n 0.0% 158 7.9 8.1 3.1 156.9 12.0 4. ??? 5. ??? 6. 0.ae3.GW11.IAD8.ALTER.NET 0.0% 157 6.5 8.5 5.4 118.2 12.1 7. teliasonera-gw.customer.alter.net 0.0% 157 7.2 9.0 5.5 164.1 15.7 8. ash-bb3-link.telia.net 0.0% 157 8.0 8.9 5.5 148.9 14.4 9. sjo-b21-link.telia.net 0.0% 157 71.1 72.6 68.8 150.8 7.2 10. den-b1-link.telia.net 0.0% 157 99.3 99.6 98.0 122.4 2.4 11. handynet-ic-145041-den-b1.c.telia.ne 0.0% 157 130.4 100.1 98.0 130.4 3.3 12. ae2-14.core1.denver2.handynetworks.c 0.0% 157 97.1 97.1 95.5 113.8 1.8 13. ae0-10.dist2.denver2.handynetworks.c 0.0% 157 99.5 103.9 95.6 828.6 58.2 14. vlan954.sw2-g4.denver2.handynetworks 0.0% 157 98.5 104.7 98.1 761.2 52.8 15. vlan11.sw1-h3.softsyshosting.com 0.0% 157 99.8 103.8 97.1 430.8 33.8 16. 72.18.145.210 0.0% 157 97.3 97.1 95.5 128.1 2.7
--------------------------------------------------------------------------- Peter Beckman Internet Guy beckman@angryox.com http://www.angryox.com/ --------------------------------------------------------------------------- _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
-- Darin Steffl Minnesota WiFi www.mnwifi.com 507-634-WiFi <http://www.facebook.com/minnesotawifi> Like us on Facebook <http://www.facebook.com/minnesotawifi>
We are still having issues. Traces to the west coast die in NY. On Tue, Apr 26, 2016 at 4:01 PM, Darin Steffl via Outages < outages@outages.org> wrote:
We noticed a large dip in traffic from HE in Minnesota to Chicago as well but I didn't have time to run any traces before it returned to normal now.
On Tue, Apr 26, 2016 at 2:59 PM, Peter Beckman via Outages < outages@outages.org> wrote:
A few minutes ago I was notified that one of my servers in Denver, CO was down (ip: 72.18.145.210).
Anyone seeing a similar loss on he.net in Chicago?
nog2.angryox.com (0.0.0.0) Tue Apr 26 15:54:54 2016 Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. xl0.tic.nova.org 0.0% 36 0.1 0.1 0.1 1.0 0.0 2. wsip-70-168-111-17.dc.dc.cox.net 2.8% 36 1.8 13.3 1.1 183.2 40.1 3. ip68-100-3-109.dc.dc.cox.net 0.0% 36 1.1 3.0 1.1 25.6 4.9 4. ashbbprj01-ae5.0.rd.as.cox.net 0.0% 36 1.7 3.6 1.7 20.7 4.3 5. 68.105.30.114 2.8% 36 7.8 15.1 1.8 120.4 21.0 6. 10ge3-6.core2.chi1.he.net 2.8% 36 33.2 35.2 28.1 51.4 6.9 7. ??
But from Verizon it is up:
max.angryox.com (0.0.0.0) Tue Apr 26 15:56:08 2016 Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. tomato.angryox.com 0.0% 158 0.3 0.4 0.3 2.4 0.2 2. lo0-100.WASHDC-VFTTP-312.verizon-gni 0.0% 158 3.7 7.0 3.0 153.9 13.1 3. T1-3-0-4.WASHDC-LCR-21.verizon-gni.n 0.0% 158 7.9 8.1 3.1 156.9 12.0 4. ??? 5. ??? 6. 0.ae3.GW11.IAD8.ALTER.NET 0.0% 157 6.5 8.5 5.4 118.2 12.1 7. teliasonera-gw.customer.alter.net 0.0% 157 7.2 9.0 5.5 164.1 15.7 8. ash-bb3-link.telia.net 0.0% 157 8.0 8.9 5.5 148.9 14.4 9. sjo-b21-link.telia.net 0.0% 157 71.1 72.6 68.8 150.8 7.2 10. den-b1-link.telia.net 0.0% 157 99.3 99.6 98.0 122.4 2.4 11. handynet-ic-145041-den-b1.c.telia.ne 0.0% 157 130.4 100.1 98.0 130.4 3.3 12. ae2-14.core1.denver2.handynetworks.c 0.0% 157 97.1 97.1 95.5 113.8 1.8 13. ae0-10.dist2.denver2.handynetworks.c 0.0% 157 99.5 103.9 95.6 828.6 58.2 14. vlan954.sw2-g4.denver2.handynetworks 0.0% 157 98.5 104.7 98.1 761.2 52.8 15. vlan11.sw1-h3.softsyshosting.com 0.0% 157 99.8 103.8 97.1 430.8 33.8 16. 72.18.145.210 0.0% 157 97.3 97.1 95.5 128.1 2.7
--------------------------------------------------------------------------- Peter Beckman Internet Guy beckman@angryox.com http://www.angryox.com/
--------------------------------------------------------------------------- _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
-- Darin Steffl Minnesota WiFi www.mnwifi.com 507-634-WiFi <http://www.facebook.com/minnesotawifi> Like us on Facebook <http://www.facebook.com/minnesotawifi>
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
Issue seems to have been resolved as of 1 minute ago. On Tue, Apr 26, 2016 at 4:06 PM, Dovid Bender <dovid@telecurve.com> wrote:
We are still having issues. Traces to the west coast die in NY.
On Tue, Apr 26, 2016 at 4:01 PM, Darin Steffl via Outages < outages@outages.org> wrote:
We noticed a large dip in traffic from HE in Minnesota to Chicago as well but I didn't have time to run any traces before it returned to normal now.
On Tue, Apr 26, 2016 at 2:59 PM, Peter Beckman via Outages < outages@outages.org> wrote:
A few minutes ago I was notified that one of my servers in Denver, CO was down (ip: 72.18.145.210).
Anyone seeing a similar loss on he.net in Chicago?
nog2.angryox.com (0.0.0.0) Tue Apr 26 15:54:54 2016 Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. xl0.tic.nova.org 0.0% 36 0.1 0.1 0.1 1.0 0.0 2. wsip-70-168-111-17.dc.dc.cox.net 2.8% 36 1.8 13.3 1.1 183.2 40.1 3. ip68-100-3-109.dc.dc.cox.net 0.0% 36 1.1 3.0 1.1 25.6 4.9 4. ashbbprj01-ae5.0.rd.as.cox.net 0.0% 36 1.7 3.6 1.7 20.7 4.3 5. 68.105.30.114 2.8% 36 7.8 15.1 1.8 120.4 21.0 6. 10ge3-6.core2.chi1.he.net 2.8% 36 33.2 35.2 28.1 51.4 6.9 7. ??
But from Verizon it is up:
max.angryox.com (0.0.0.0) Tue Apr 26 15:56:08 2016 Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. tomato.angryox.com 0.0% 158 0.3 0.4 0.3 2.4 0.2 2. lo0-100.WASHDC-VFTTP-312.verizon-gni 0.0% 158 3.7 7.0 3.0 153.9 13.1 3. T1-3-0-4.WASHDC-LCR-21.verizon-gni.n 0.0% 158 7.9 8.1 3.1 156.9 12.0 4. ??? 5. ??? 6. 0.ae3.GW11.IAD8.ALTER.NET 0.0% 157 6.5 8.5 5.4 118.2 12.1 7. teliasonera-gw.customer.alter.net 0.0% 157 7.2 9.0 5.5 164.1 15.7 8. ash-bb3-link.telia.net 0.0% 157 8.0 8.9 5.5 148.9 14.4 9. sjo-b21-link.telia.net 0.0% 157 71.1 72.6 68.8 150.8 7.2 10. den-b1-link.telia.net 0.0% 157 99.3 99.6 98.0 122.4 2.4 11. handynet-ic-145041-den-b1.c.telia.ne 0.0% 157 130.4 100.1 98.0 130.4 3.3 12. ae2-14.core1.denver2.handynetworks.c 0.0% 157 97.1 97.1 95.5 113.8 1.8 13. ae0-10.dist2.denver2.handynetworks.c 0.0% 157 99.5 103.9 95.6 828.6 58.2 14. vlan954.sw2-g4.denver2.handynetworks 0.0% 157 98.5 104.7 98.1 761.2 52.8 15. vlan11.sw1-h3.softsyshosting.com 0.0% 157 99.8 103.8 97.1 430.8 33.8 16. 72.18.145.210 0.0% 157 97.3 97.1 95.5 128.1 2.7
--------------------------------------------------------------------------- Peter Beckman Internet Guy beckman@angryox.com http://www.angryox.com/
--------------------------------------------------------------------------- _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
-- Darin Steffl Minnesota WiFi www.mnwifi.com 507-634-WiFi <http://www.facebook.com/minnesotawifi> Like us on Facebook <http://www.facebook.com/minnesotawifi>
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
Interesting -- we lost DNS resolvability to nosig.not-signed.net (hosted in the Netherlands) at just around that same time, as well as HTTPv6 access to www.ipv6.bt.com and www.t-home.hu (resolved to two AAAA). I don't know if they're related to this HE hiccup, but I'm chasing this down. Frank -----Original Message----- From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Peter Beckman via Outages Sent: Tuesday, April 26, 2016 3:00 PM To: outages@outages.org Subject: [outages] HE.net Chicago router link issue? A few minutes ago I was notified that one of my servers in Denver, CO was down (ip: 72.18.145.210). Anyone seeing a similar loss on he.net in Chicago? nog2.angryox.com (0.0.0.0) Tue Apr 26 15:54:54 2016 Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. xl0.tic.nova.org 0.0% 36 0.1 0.1 0.1 1.0 0.0 2. wsip-70-168-111-17.dc.dc.cox.net 2.8% 36 1.8 13.3 1.1 183.2 40.1 3. ip68-100-3-109.dc.dc.cox.net 0.0% 36 1.1 3.0 1.1 25.6 4.9 4. ashbbprj01-ae5.0.rd.as.cox.net 0.0% 36 1.7 3.6 1.7 20.7 4.3 5. 68.105.30.114 2.8% 36 7.8 15.1 1.8 120.4 21.0 6. 10ge3-6.core2.chi1.he.net 2.8% 36 33.2 35.2 28.1 51.4 6.9 7. ?? But from Verizon it is up: max.angryox.com (0.0.0.0) Tue Apr 26 15:56:08 2016 Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. tomato.angryox.com 0.0% 158 0.3 0.4 0.3 2.4 0.2 2. lo0-100.WASHDC-VFTTP-312.verizon-gni 0.0% 158 3.7 7.0 3.0 153.9 13.1 3. T1-3-0-4.WASHDC-LCR-21.verizon-gni.n 0.0% 158 7.9 8.1 3.1 156.9 12.0 4. ??? 5. ??? 6. 0.ae3.GW11.IAD8.ALTER.NET 0.0% 157 6.5 8.5 5.4 118.2 12.1 7. teliasonera-gw.customer.alter.net 0.0% 157 7.2 9.0 5.5 164.1 15.7 8. ash-bb3-link.telia.net 0.0% 157 8.0 8.9 5.5 148.9 14.4 9. sjo-b21-link.telia.net 0.0% 157 71.1 72.6 68.8 150.8 7.2 10. den-b1-link.telia.net 0.0% 157 99.3 99.6 98.0 122.4 2.4 11. handynet-ic-145041-den-b1.c.telia.ne 0.0% 157 130.4 100.1 98.0 130.4 3.3 12. ae2-14.core1.denver2.handynetworks.c 0.0% 157 97.1 97.1 95.5 113.8 1.8 13. ae0-10.dist2.denver2.handynetworks.c 0.0% 157 99.5 103.9 95.6 828.6 58.2 14. vlan954.sw2-g4.denver2.handynetworks 0.0% 157 98.5 104.7 98.1 761.2 52.8 15. vlan11.sw1-h3.softsyshosting.com 0.0% 157 99.8 103.8 97.1 430.8 33.8 16. 72.18.145.210 0.0% 157 97.3 97.1 95.5 128.1 2.7 --------------------------------------------------------------------------- Peter Beckman Internet Guy beckman@angryox.com http://www.angryox.com/ --------------------------------------------------------------------------- _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
nosig.not-signed.net normally resolves to 134.91.78.139 and is hosted by Universitaet Duisburg-Essen (http://bgp.he.net/ip/134.91.78.139), but traceroutes show it stopping Chicago with HE. root@nagios:/# tcptraceroute 134.91.78.139 53 Selected device eth0.3, address 96.31.0.5, port 44198 for outgoing packets Tracing the path to 134.91.78.139 on TCP port 53 (domain), 30 hops max 1 router-core-inside.mtcnet.net (96.31.0.254) 0.268 ms 0.197 ms 0.200 ms 2 sxct.sxcy.mtcnet.net (167.142.156.197) 0.280 ms 0.182 ms 0.142 ms 3 premier.sxcy-mlx.fbnt.netins.net (173.215.60.5) 1.502 ms 1.450 ms 1.455 ms 4 ins-db4-te-0-6-0-4.219.desm.netins.net (167.142.60.157) 7.897 ms 7.640 ms 7.521 ms 5 ins-dc1-po200.desm.netins.net (167.142.67.193) 9.178 ms 9.215 ms 9.162 ms 6 ins-kc2-po20.kmrr.netins.net (167.142.67.30) 9.299 ms 9.239 ms 9.210 ms 7 v504.core1.oma1.he.net (184.105.18.169) 26.753 ms 26.724 ms 26.729 ms 8 10ge11-15.core2.chi1.he.net (184.105.81.229) 26.490 ms 29.558 ms 36.265 ms 9 * * * 10 * * * 11 * * * 12 * * * 13 * * * 14 * * * 15 * * * 16 * * * 17 * * * 18 * * * 19 * * * 20 * * * 21 * * * 22 * * * 23 *^C root@nagios:/pnsi# www.ipv6.bt.com resolves to 2a00:1a48:10a1::538a:9648 (there are some openresolvers like OpenDNS and Google DNS that cache these things a little longer) and that does traceroute just fine and port 80 is open, but I can't even get the nameservers for bt.com right now: root@nagios:/# dig NS bt.com ; <<>> DiG 9.7.3 <<>> NS bt.com ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 38895 ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;bt.com. IN NS ;; Query time: 3000 msec ;; SERVER: 199.120.69.24#53(199.120.69.24) ;; WHEN: Tue Apr 26 15:20:18 2016 ;; MSG SIZE rcvd: 24 root@nagios:/# Tracing each of their nameservers, I see they are all reachable through....HE. root@nagios:/# dig NS bt.com @8.8.8.8 +short | while read LINE
do dig $LINE @8.8.8.8 +short done 193.113.32.156 193.113.32.157 193.113.57.243 193.113.57.242 root@nagios:/# tcptraceroute 193.113.32.156 Selected device eth0.3, address 96.31.0.5, port 35839 for outgoing packets Tracing the path to 193.113.32.156 on TCP port 80 (www), 30 hops max 1 router-core-inside.mtcnet.net (96.31.0.254) 0.262 ms 0.231 ms 0.193 ms 2 sxct.spnc.mtcnet.net (167.142.156.194) 0.261 ms 0.171 ms 0.136 ms 3 premier.spnc-mlx.fbnt.netins.net (173.215.60.1) 6.426 ms 4.758 ms 4.793 ms 4 ins-kb1-te-12-2-3031.kmrr.netins.net (167.142.64.253) 8.548 ms 8.508 ms 8.534 ms 5 ins-kc2-et-po101.kmrr.netins.net (167.142.67.49) 8.506 ms 9.653 ms 9.781 ms 6 v504.core1.oma1.he.net (184.105.18.169) 25.929 ms 26.008 ms 25.960 ms 7 10ge11-15.core2.chi1.he.net (184.105.81.229) 32.073 ms 25.762 ms 25.760 ms 8 * * * 9 * *^C root@nagios:/#
So something is not healthy in Chicago, yet. Our monitoring showed that access to www.t-home.hu recovered just after 3:07 pm (Central). Frank -----Original Message----- From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Frank Bulk via Outages Sent: Tuesday, April 26, 2016 3:07 PM To: 'Peter Beckman' <beckman@angryox.com>; outages@outages.org Subject: Re: [outages] HE.net Chicago router link issue? Interesting -- we lost DNS resolvability to nosig.not-signed.net (hosted in the Netherlands) at just around that same time, as well as HTTPv6 access to www.ipv6.bt.com and www.t-home.hu (resolved to two AAAA). I don't know if they're related to this HE hiccup, but I'm chasing this down. Frank -----Original Message----- From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Peter Beckman via Outages Sent: Tuesday, April 26, 2016 3:00 PM To: outages@outages.org Subject: [outages] HE.net Chicago router link issue? A few minutes ago I was notified that one of my servers in Denver, CO was down (ip: 72.18.145.210). Anyone seeing a similar loss on he.net in Chicago? nog2.angryox.com (0.0.0.0) Tue Apr 26 15:54:54 2016 Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. xl0.tic.nova.org 0.0% 36 0.1 0.1 0.1 1.0 0.0 2. wsip-70-168-111-17.dc.dc.cox.net 2.8% 36 1.8 13.3 1.1 183.2 40.1 3. ip68-100-3-109.dc.dc.cox.net 0.0% 36 1.1 3.0 1.1 25.6 4.9 4. ashbbprj01-ae5.0.rd.as.cox.net 0.0% 36 1.7 3.6 1.7 20.7 4.3 5. 68.105.30.114 2.8% 36 7.8 15.1 1.8 120.4 21.0 6. 10ge3-6.core2.chi1.he.net 2.8% 36 33.2 35.2 28.1 51.4 6.9 7. ?? But from Verizon it is up: max.angryox.com (0.0.0.0) Tue Apr 26 15:56:08 2016 Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. tomato.angryox.com 0.0% 158 0.3 0.4 0.3 2.4 0.2 2. lo0-100.WASHDC-VFTTP-312.verizon-gni 0.0% 158 3.7 7.0 3.0 153.9 13.1 3. T1-3-0-4.WASHDC-LCR-21.verizon-gni.n 0.0% 158 7.9 8.1 3.1 156.9 12.0 4. ??? 5. ??? 6. 0.ae3.GW11.IAD8.ALTER.NET 0.0% 157 6.5 8.5 5.4 118.2 12.1 7. teliasonera-gw.customer.alter.net 0.0% 157 7.2 9.0 5.5 164.1 15.7 8. ash-bb3-link.telia.net 0.0% 157 8.0 8.9 5.5 148.9 14.4 9. sjo-b21-link.telia.net 0.0% 157 71.1 72.6 68.8 150.8 7.2 10. den-b1-link.telia.net 0.0% 157 99.3 99.6 98.0 122.4 2.4 11. handynet-ic-145041-den-b1.c.telia.ne 0.0% 157 130.4 100.1 98.0 130.4 3.3 12. ae2-14.core1.denver2.handynetworks.c 0.0% 157 97.1 97.1 95.5 113.8 1.8 13. ae0-10.dist2.denver2.handynetworks.c 0.0% 157 99.5 103.9 95.6 828.6 58.2 14. vlan954.sw2-g4.denver2.handynetworks 0.0% 157 98.5 104.7 98.1 761.2 52.8 15. vlan11.sw1-h3.softsyshosting.com 0.0% 157 99.8 103.8 97.1 430.8 33.8 16. 72.18.145.210 0.0% 157 97.3 97.1 95.5 128.1 2.7 --------------------------------------------------------------------------- Peter Beckman Internet Guy beckman@angryox.com http://www.angryox.com/ --------------------------------------------------------------------------- _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
On Apr 26, 2016, at 4:06 PM, Frank Bulk via Outages <outages@outages.org> wrote:
Interesting -- we lost DNS resolvability to nosig.not-signed.net (hosted in the Netherlands) at just around that same time, as well as HTTPv6 access to www.ipv6.bt.com and www.t-home.hu (resolved to two AAAA).
I spotted this via a DNS issue as well. I got a nagios alert from a client (dual-homed to HE) that my home FiOS was down. A closer look showed my duckdns.org hostname was not able to be resolved. All three duckdns.org servers are hosted on AWS. Here’s a traceroute to ns1.duckdns.org: traceroute 54.187.92.222 traceroute to 54.187.92.222 (54.187.92.222), 64 hops max, 52 byte packets 1 xxxx (216.220.xx.xx) 0.138 ms 0.138 ms 0.140 ms 2 v200.core1.nyc4.he.net (209.51.171.25) 3.187 ms 0.205 ms 0.187 ms 3 * * * 4 * * * 5 * * * 6 * * * 7 * * * 8 * * * 9 * * * 10 * * * ^C Turned down HE transit until I hear back from their NOC. Charles
I don't know if they're related to this HE hiccup, but I'm chasing this down.
Frank
-----Original Message----- From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Peter Beckman via Outages Sent: Tuesday, April 26, 2016 3:00 PM To: outages@outages.org Subject: [outages] HE.net Chicago router link issue?
A few minutes ago I was notified that one of my servers in Denver, CO was down (ip: 72.18.145.210).
Anyone seeing a similar loss on he.net in Chicago?
nog2.angryox.com (0.0.0.0) Tue Apr 26 15:54:54 2016 Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. xl0.tic.nova.org 0.0% 36 0.1 0.1 0.1 1.0 0.0 2. wsip-70-168-111-17.dc.dc.cox.net 2.8% 36 1.8 13.3 1.1 183.2 40.1 3. ip68-100-3-109.dc.dc.cox.net 0.0% 36 1.1 3.0 1.1 25.6 4.9 4. ashbbprj01-ae5.0.rd.as.cox.net 0.0% 36 1.7 3.6 1.7 20.7 4.3 5. 68.105.30.114 2.8% 36 7.8 15.1 1.8 120.4 21.0 6. 10ge3-6.core2.chi1.he.net 2.8% 36 33.2 35.2 28.1 51.4 6.9 7. ??
But from Verizon it is up:
max.angryox.com (0.0.0.0) Tue Apr 26 15:56:08 2016 Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. tomato.angryox.com 0.0% 158 0.3 0.4 0.3 2.4 0.2 2. lo0-100.WASHDC-VFTTP-312.verizon-gni 0.0% 158 3.7 7.0 3.0 153.9 13.1 3. T1-3-0-4.WASHDC-LCR-21.verizon-gni.n 0.0% 158 7.9 8.1 3.1 156.9 12.0 4. ??? 5. ??? 6. 0.ae3.GW11.IAD8.ALTER.NET 0.0% 157 6.5 8.5 5.4 118.2 12.1 7. teliasonera-gw.customer.alter.net 0.0% 157 7.2 9.0 5.5 164.1 15.7 8. ash-bb3-link.telia.net 0.0% 157 8.0 8.9 5.5 148.9 14.4 9. sjo-b21-link.telia.net 0.0% 157 71.1 72.6 68.8 150.8 7.2 10. den-b1-link.telia.net 0.0% 157 99.3 99.6 98.0 122.4 2.4 11. handynet-ic-145041-den-b1.c.telia.ne 0.0% 157 130.4 100.1 98.0 130.4 3.3 12. ae2-14.core1.denver2.handynetworks.c 0.0% 157 97.1 97.1 95.5 113.8 1.8 13. ae0-10.dist2.denver2.handynetworks.c 0.0% 157 99.5 103.9 95.6 828.6 58.2 14. vlan954.sw2-g4.denver2.handynetworks 0.0% 157 98.5 104.7 98.1 761.2 52.8 15. vlan11.sw1-h3.softsyshosting.com 0.0% 157 99.8 103.8 97.1 430.8 33.8 16. 72.18.145.210 0.0% 157 97.3 97.1 95.5 128.1 2.7
--------------------------------------------------------------------------- Peter Beckman Internet Guy beckman@angryox.com http://www.angryox.com/ --------------------------------------------------------------------------- _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
Our NLNOG ring alerted us to a brief issue which lasted about 5 minutes: rackcentral02.ring.nlnog.net AS58443 (AU) mtr -i0.5 -c5 -r -w -n 163.53.233.70 HOST: beanfield01.ring.nlnog.net Loss% Snt Last Avg Best Wrst StDev 1.|-- 66.207.196.129 0.0% 5 0.4 1.1 0.3 3.9 1.3 2.|-- 72.15.51.2 0.0% 5 0.8 0.8 0.7 0.9 0.0 3.|-- 72.15.49.83 0.0% 5 1.0 0.9 0.9 1.1 0.0 4.|-- 206.108.34.112 0.0% 5 0.4 1.9 0.4 8.1 3.3 5.|-- ??? 100.0 5 0.0 0.0 0.0 0.0 0.0 vocus01.ring.nlnog.net AS4826 (AU) mtr -i0.5 -c5 -r -w -n 175.45.73.66 HOST: beanfield01.ring.nlnog.net Loss% Snt Last Avg Best Wrst StDev 1.|-- 66.207.196.129 0.0% 5 0.3 0.3 0.3 0.4 0.0 2.|-- 72.15.51.2 0.0% 5 0.6 0.8 0.6 1.0 0.0 3.|-- 72.15.49.83 0.0% 5 1.0 1.1 1.0 1.2 0.0 4.|-- 206.108.34.112 0.0% 5 8.8 10.4 8.8 12.8 1.6 5.|-- ??? 100.0 5 0.0 0.0 0.0 0.0 0.0 Once it came back: BlackBox-3:Beanfield jlixfeld$ mtr -c 5 -rw rackcentral02.ring.nlnog.net Start: Tue Apr 26 16:04:49 2016 HOST: BlackBox-3.local Loss% Snt Last Avg Best Wrst StDev 1.|-- vl4091.trs01.002.77mowatav01.yyz.beanfield.com 0.0% 5 3.1 4.9 1.9 15.5 5.9 2.|-- te0-2-0-2.bfr01.905kingstw01.yyz.beanfield.com 0.0% 5 3.1 2.6 2.3 3.1 0.0 3.|-- te-0-2-0-14.bfr01.151frontstw01.yyz.beanfield.com 0.0% 5 3.1 3.1 2.9 3.4 0.0 4.|-- he.ip4.torontointernetxchange.net 0.0% 5 2.0 4.2 2.0 9.2 3.1 5.|-- 100ge13-1.core1.chi1.he.net 0.0% 5 17.8 26.2 11.9 68.6 24.0 6.|-- 10ge11-4.core1.pao1.he.net 0.0% 5 70.2 73.0 70.1 79.2 3.6 7.|-- 10ge4-1.core1.sjc1.he.net 0.0% 5 76.7 81.0 76.7 92.4 6.5 8.|-- tpg-internet-pty-ltd.10gigabitethernet3-8.core1.lax2.he.net 0.0% 5 250.9 251.2 249.5 252.2 0.9 9.|-- 203-219-35-193.static.tpgi.com.au 0.0% 5 233.9 233.4 231.2 235.2 1.5 10.|-- nme-apt-bur-wgw1-be-10.tpgi.com.au 0.0% 5 264.4 264.9 264.1 266.7 0.9 11.|-- 203-219-107-206.static.tpgi.com.au 0.0% 5 260.6 260.6 260.2 261.5 0.0 12.|-- po8.mburndist01.aapt.net.au 0.0% 5 443.5 296.5 259.4 443.5 82.2 13.|-- ??? 100.0 5 0.0 0.0 0.0 0.0 0.0 14.|-- 59-100-253-114.mel.static-ipl.aapt.com.au 0.0% 5 249.3 251.9 249.1 256.8 3.1 15.|-- 163.53.233.254 0.0% 5 285.7 256.3 248.9 285.7 16.4 16.|-- 163.53.233.70 0.0% 5 250.4 250.3 250.0 250.5 0.0 BlackBox-3:Beanfield jlixfeld$ mtr -c 5 -rw vocus01.ring.nlnog.net Start: Tue Apr 26 16:05:15 2016 HOST: BlackBox-3.local Loss% Snt Last Avg Best Wrst StDev 1.|-- vl4091.trs01.002.77mowatav01.yyz.beanfield.com 0.0% 5 2.2 2.4 2.0 2.8 0.0 2.|-- te0-2-0-2.bfr01.905kingstw01.yyz.beanfield.com 0.0% 5 2.1 2.6 2.1 3.7 0.5 3.|-- te-0-2-0-14.bfr01.151frontstw01.yyz.beanfield.com 0.0% 5 2.7 2.8 2.7 3.0 0.0 4.|-- he.ip4.torontointernetxchange.net 0.0% 5 2.6 6.4 2.0 13.5 4.5 5.|-- 100ge13-1.core1.chi1.he.net 0.0% 5 11.8 14.8 11.8 16.9 1.9 6.|-- eqix-ch1.vocusconnect.com 0.0% 5 104.1 117.0 103.1 148.2 20.1 7.|-- bundle153.cor02.lax01.ca.vocus.net 0.0% 5 266.8 266.5 266.4 266.8 0.0 8.|-- 100g-0-1-0-0.cor01.lax01.ca.vocus.net 0.0% 5 266.2 266.4 266.2 266.6 0.0 9.|-- 100g-0-1-0-0.cor01.sjc01.ca.vocus.net 0.0% 5 266.4 266.4 266.3 266.5 0.0 10.|-- ten-0-2-0-3.cor01.syd04.nsw.vocus.net.au 0.0% 5 266.6 266.8 266.4 267.3 0.0 11.|-- 175.45.72.95 0.0% 5 266.0 266.1 265.8 266.4 0.0 12.|-- ring.vocus.net.au 0.0% 5 269.8 269.8 269.7 270.0 0.0 BlackBox-3:Beanfield jlixfeld$
On Apr 26, 2016, at 3:59 PM, Peter Beckman via Outages <outages@outages.org> wrote:
A few minutes ago I was notified that one of my servers in Denver, CO was down (ip: 72.18.145.210).
Anyone seeing a similar loss on he.net in Chicago?
nog2.angryox.com (0.0.0.0) Tue Apr 26 15:54:54 2016 Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. xl0.tic.nova.org 0.0% 36 0.1 0.1 0.1 1.0 0.0 2. wsip-70-168-111-17.dc.dc.cox.net 2.8% 36 1.8 13.3 1.1 183.2 40.1 3. ip68-100-3-109.dc.dc.cox.net 0.0% 36 1.1 3.0 1.1 25.6 4.9 4. ashbbprj01-ae5.0.rd.as.cox.net 0.0% 36 1.7 3.6 1.7 20.7 4.3 5. 68.105.30.114 2.8% 36 7.8 15.1 1.8 120.4 21.0 6. 10ge3-6.core2.chi1.he.net 2.8% 36 33.2 35.2 28.1 51.4 6.9 7. ??
But from Verizon it is up:
max.angryox.com (0.0.0.0) Tue Apr 26 15:56:08 2016 Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. tomato.angryox.com 0.0% 158 0.3 0.4 0.3 2.4 0.2 2. lo0-100.WASHDC-VFTTP-312.verizon-gni 0.0% 158 3.7 7.0 3.0 153.9 13.1 3. T1-3-0-4.WASHDC-LCR-21.verizon-gni.n 0.0% 158 7.9 8.1 3.1 156.9 12.0 4. ??? 5. ??? 6. 0.ae3.GW11.IAD8.ALTER.NET 0.0% 157 6.5 8.5 5.4 118.2 12.1 7. teliasonera-gw.customer.alter.net 0.0% 157 7.2 9.0 5.5 164.1 15.7 8. ash-bb3-link.telia.net 0.0% 157 8.0 8.9 5.5 148.9 14.4 9. sjo-b21-link.telia.net 0.0% 157 71.1 72.6 68.8 150.8 7.2 10. den-b1-link.telia.net 0.0% 157 99.3 99.6 98.0 122.4 2.4 11. handynet-ic-145041-den-b1.c.telia.ne 0.0% 157 130.4 100.1 98.0 130.4 3.3 12. ae2-14.core1.denver2.handynetworks.c 0.0% 157 97.1 97.1 95.5 113.8 1.8 13. ae0-10.dist2.denver2.handynetworks.c 0.0% 157 99.5 103.9 95.6 828.6 58.2 14. vlan954.sw2-g4.denver2.handynetworks 0.0% 157 98.5 104.7 98.1 761.2 52.8 15. vlan11.sw1-h3.softsyshosting.com 0.0% 157 99.8 103.8 97.1 430.8 33.8 16. 72.18.145.210 0.0% 157 97.3 97.1 95.5 128.1 2.7
--------------------------------------------------------------------------- Peter Beckman Internet Guy beckman@angryox.com http://www.angryox.com/ --------------------------------------------------------------------------- _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
Thanks All. Looks like someone routed around HE for me. Came back at 16:00 EDT: nog2.angryox.com (0.0.0.0) Tue Apr 26 16:13:08 2016 Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. xl0.tic.nova.org 0.0% 5 0.1 0.1 0.1 0.2 0.0 2. wsip-70-168-111-17.dc.dc.cox.net 0.0% 5 1.4 1.4 1.2 1.6 0.0 3. ip68-100-3-109.dc.dc.cox.net 0.0% 5 1.1 1.5 1.1 2.3 0.0 4. ashbbprj01-ae5.0.rd.as.cox.net 0.0% 4 1.7 1.8 1.7 2.1 0.0 5. xe-5-0-0.edge2.Washington4.Level3.net 0.0% 4 2.3 2.0 1.8 2.3 0.0 6. ae-3-80.edge1.Washington4.Level3.net 0.0% 4 3.3 3.1 2.6 3.8 0.0 7. ash-b1-link.telia.net 0.0% 4 3.3 18.3 3.3 59.8 27.7 8. ash-bb4-link.telia.net 0.0% 4 2.8 3.0 2.8 3.3 0.0 9. sjo-b21-link.telia.net 0.0% 4 76.7 76.2 75.5 76.7 0.0 10. den-b1-link.telia.net 0.0% 4 73.0 73.0 72.7 73.5 0.0 11. handynet-ic-145041-den-b1.c.telia.net 0.0% 4 72.7 73.1 72.7 73.4 0.0 12. ae2-14.core1.denver2.handynetworks.com 0.0% 4 73.0 72.9 72.7 73.1 0.0 13. ae0-10.dist2.denver2.handynetworks.com 0.0% 4 73.5 73.8 73.5 74.3 0.0 14. vlan954.sw2-g4.denver2.handynetworks.com 0.0% 4 74.1 74.6 73.7 76.7 1.2 15. vlan11.sw1-h3.softsyshosting.com 0.0% 4 72.8 73.0 72.8 73.4 0.0 16. 72.18.145.210 0.0% 4 73.4 73.0 72.7 73.4 0.0 On Tue, 26 Apr 2016, Jason Lixfeld wrote:
Our NLNOG ring alerted us to a brief issue which lasted about 5 minutes:
rackcentral02.ring.nlnog.net AS58443 (AU) mtr -i0.5 -c5 -r -w -n 163.53.233.70 HOST: beanfield01.ring.nlnog.net Loss% Snt Last Avg Best Wrst StDev 1.|-- 66.207.196.129 0.0% 5 0.4 1.1 0.3 3.9 1.3 2.|-- 72.15.51.2 0.0% 5 0.8 0.8 0.7 0.9 0.0 3.|-- 72.15.49.83 0.0% 5 1.0 0.9 0.9 1.1 0.0 4.|-- 206.108.34.112 0.0% 5 0.4 1.9 0.4 8.1 3.3 5.|-- ??? 100.0 5 0.0 0.0 0.0 0.0 0.0
vocus01.ring.nlnog.net AS4826 (AU) mtr -i0.5 -c5 -r -w -n 175.45.73.66 HOST: beanfield01.ring.nlnog.net Loss% Snt Last Avg Best Wrst StDev 1.|-- 66.207.196.129 0.0% 5 0.3 0.3 0.3 0.4 0.0 2.|-- 72.15.51.2 0.0% 5 0.6 0.8 0.6 1.0 0.0 3.|-- 72.15.49.83 0.0% 5 1.0 1.1 1.0 1.2 0.0 4.|-- 206.108.34.112 0.0% 5 8.8 10.4 8.8 12.8 1.6 5.|-- ??? 100.0 5 0.0 0.0 0.0 0.0 0.0
Once it came back:
BlackBox-3:Beanfield jlixfeld$ mtr -c 5 -rw rackcentral02.ring.nlnog.net Start: Tue Apr 26 16:04:49 2016 HOST: BlackBox-3.local Loss% Snt Last Avg Best Wrst StDev 1.|-- vl4091.trs01.002.77mowatav01.yyz.beanfield.com 0.0% 5 3.1 4.9 1.9 15.5 5.9 2.|-- te0-2-0-2.bfr01.905kingstw01.yyz.beanfield.com 0.0% 5 3.1 2.6 2.3 3.1 0.0 3.|-- te-0-2-0-14.bfr01.151frontstw01.yyz.beanfield.com 0.0% 5 3.1 3.1 2.9 3.4 0.0 4.|-- he.ip4.torontointernetxchange.net 0.0% 5 2.0 4.2 2.0 9.2 3.1 5.|-- 100ge13-1.core1.chi1.he.net 0.0% 5 17.8 26.2 11.9 68.6 24.0 6.|-- 10ge11-4.core1.pao1.he.net 0.0% 5 70.2 73.0 70.1 79.2 3.6 7.|-- 10ge4-1.core1.sjc1.he.net 0.0% 5 76.7 81.0 76.7 92.4 6.5 8.|-- tpg-internet-pty-ltd.10gigabitethernet3-8.core1.lax2.he.net 0.0% 5 250.9 251.2 249.5 252.2 0.9 9.|-- 203-219-35-193.static.tpgi.com.au 0.0% 5 233.9 233.4 231.2 235.2 1.5 10.|-- nme-apt-bur-wgw1-be-10.tpgi.com.au 0.0% 5 264.4 264.9 264.1 266.7 0.9 11.|-- 203-219-107-206.static.tpgi.com.au 0.0% 5 260.6 260.6 260.2 261.5 0.0 12.|-- po8.mburndist01.aapt.net.au 0.0% 5 443.5 296.5 259.4 443.5 82.2 13.|-- ??? 100.0 5 0.0 0.0 0.0 0.0 0.0 14.|-- 59-100-253-114.mel.static-ipl.aapt.com.au 0.0% 5 249.3 251.9 249.1 256.8 3.1 15.|-- 163.53.233.254 0.0% 5 285.7 256.3 248.9 285.7 16.4 16.|-- 163.53.233.70 0.0% 5 250.4 250.3 250.0 250.5 0.0 BlackBox-3:Beanfield jlixfeld$ mtr -c 5 -rw vocus01.ring.nlnog.net Start: Tue Apr 26 16:05:15 2016 HOST: BlackBox-3.local Loss% Snt Last Avg Best Wrst StDev 1.|-- vl4091.trs01.002.77mowatav01.yyz.beanfield.com 0.0% 5 2.2 2.4 2.0 2.8 0.0 2.|-- te0-2-0-2.bfr01.905kingstw01.yyz.beanfield.com 0.0% 5 2.1 2.6 2.1 3.7 0.5 3.|-- te-0-2-0-14.bfr01.151frontstw01.yyz.beanfield.com 0.0% 5 2.7 2.8 2.7 3.0 0.0 4.|-- he.ip4.torontointernetxchange.net 0.0% 5 2.6 6.4 2.0 13.5 4.5 5.|-- 100ge13-1.core1.chi1.he.net 0.0% 5 11.8 14.8 11.8 16.9 1.9 6.|-- eqix-ch1.vocusconnect.com 0.0% 5 104.1 117.0 103.1 148.2 20.1 7.|-- bundle153.cor02.lax01.ca.vocus.net 0.0% 5 266.8 266.5 266.4 266.8 0.0 8.|-- 100g-0-1-0-0.cor01.lax01.ca.vocus.net 0.0% 5 266.2 266.4 266.2 266.6 0.0 9.|-- 100g-0-1-0-0.cor01.sjc01.ca.vocus.net 0.0% 5 266.4 266.4 266.3 266.5 0.0 10.|-- ten-0-2-0-3.cor01.syd04.nsw.vocus.net.au 0.0% 5 266.6 266.8 266.4 267.3 0.0 11.|-- 175.45.72.95 0.0% 5 266.0 266.1 265.8 266.4 0.0 12.|-- ring.vocus.net.au 0.0% 5 269.8 269.8 269.7 270.0 0.0 BlackBox-3:Beanfield jlixfeld$
On Apr 26, 2016, at 3:59 PM, Peter Beckman via Outages <outages@outages.org> wrote:
A few minutes ago I was notified that one of my servers in Denver, CO was down (ip: 72.18.145.210).
Anyone seeing a similar loss on he.net in Chicago?
nog2.angryox.com (0.0.0.0) Tue Apr 26 15:54:54 2016 Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. xl0.tic.nova.org 0.0% 36 0.1 0.1 0.1 1.0 0.0 2. wsip-70-168-111-17.dc.dc.cox.net 2.8% 36 1.8 13.3 1.1 183.2 40.1 3. ip68-100-3-109.dc.dc.cox.net 0.0% 36 1.1 3.0 1.1 25.6 4.9 4. ashbbprj01-ae5.0.rd.as.cox.net 0.0% 36 1.7 3.6 1.7 20.7 4.3 5. 68.105.30.114 2.8% 36 7.8 15.1 1.8 120.4 21.0 6. 10ge3-6.core2.chi1.he.net 2.8% 36 33.2 35.2 28.1 51.4 6.9 7. ??
But from Verizon it is up:
max.angryox.com (0.0.0.0) Tue Apr 26 15:56:08 2016 Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. tomato.angryox.com 0.0% 158 0.3 0.4 0.3 2.4 0.2 2. lo0-100.WASHDC-VFTTP-312.verizon-gni 0.0% 158 3.7 7.0 3.0 153.9 13.1 3. T1-3-0-4.WASHDC-LCR-21.verizon-gni.n 0.0% 158 7.9 8.1 3.1 156.9 12.0 4. ??? 5. ??? 6. 0.ae3.GW11.IAD8.ALTER.NET 0.0% 157 6.5 8.5 5.4 118.2 12.1 7. teliasonera-gw.customer.alter.net 0.0% 157 7.2 9.0 5.5 164.1 15.7 8. ash-bb3-link.telia.net 0.0% 157 8.0 8.9 5.5 148.9 14.4 9. sjo-b21-link.telia.net 0.0% 157 71.1 72.6 68.8 150.8 7.2 10. den-b1-link.telia.net 0.0% 157 99.3 99.6 98.0 122.4 2.4 11. handynet-ic-145041-den-b1.c.telia.ne 0.0% 157 130.4 100.1 98.0 130.4 3.3 12. ae2-14.core1.denver2.handynetworks.c 0.0% 157 97.1 97.1 95.5 113.8 1.8 13. ae0-10.dist2.denver2.handynetworks.c 0.0% 157 99.5 103.9 95.6 828.6 58.2 14. vlan954.sw2-g4.denver2.handynetworks 0.0% 157 98.5 104.7 98.1 761.2 52.8 15. vlan11.sw1-h3.softsyshosting.com 0.0% 157 99.8 103.8 97.1 430.8 33.8 16. 72.18.145.210 0.0% 157 97.3 97.1 95.5 128.1 2.7
--------------------------------------------------------------------------- Peter Beckman Internet Guy beckman@angryox.com http://www.angryox.com/ --------------------------------------------------------------------------- _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
--------------------------------------------------------------------------- Peter Beckman Internet Guy beckman@angryox.com http://www.angryox.com/ ---------------------------------------------------------------------------
participants (6)
-
Charles Sprickman -
Darin Steffl -
Dovid Bender -
Frank Bulk -
Jason Lixfeld -
Peter Beckman