Heavy Packet Loss on Level 3
We are seeing heavy packet loss on our Level 3 connection in Cleveland, OH. Looks like it started around 5am Eastern time. Is anyone else seeing an issue? HOST Loss% Snt Last Avg Best Wrst StDev 1. gi1-2-v100.core1.n2net.net 0.0% 100 0.4 5.6 0.3 147.9 22.5 2. ge-6-10.car2.Cleveland1.Leve 0.0% 100 9.5 26.9 1.3 216.8 49.6 3. 0.ae12.pr0.dca20.tbone.rr.co 15.0% 100 10.5 13.6 10.3 108.3 14.6 4. bu-ether15.vinnva0510w-bcr00 24.0% 100 10.5 12.1 10.2 14.4 1.2 5. be1.clevohek01r.midwest.rr.c 13.0% 100 18.6 20.2 18.4 22.6 1.1 6. be1.clevohek02r.midwest.rr.c 11.0% 100 21.1 22.8 18.8 26.6 2.3 7. be1.clhgohcc01r.midwest.rr.c 16.0% 100 19.1 19.1 18.8 19.9 0.2 8. 24.33.103.87 16.0% 100 27.2 35.5 23.6 113.3 9.6 9. teng18-10.clhgohcc04m.neo.rr 13.0% 100 23.0 40.4 22.9 417.7 42.0 10. cpe-24-165-186-90.neo.res.rr 15.0% 100 31.6 33.3 30.0 77.5 6.2 HOST Loss% Snt Last Avg Best Wrst StDev 1. gi1-2-v100.core1.n2net.net 0.0% 20 0.4 1.3 0.4 15.1 3.2 2. ge-6-10.car2.Cleveland1.Leve 0.0% 20 1.7 5.3 1.3 48.2 11.6 3. Verizon-level3-100G.Washingt 15.0% 20 15.3 13.7 9.4 48.4 9.5 4. ae203-0.RCMDVA-VFTTP-305.ver 10.0% 20 15.8 16.0 15.6 18.8 0.7 5. ??? 100.0 20 0.0 0.0 0.0 0.0 0.0 Host Loss% Last Avg Best Wrst StDev 1. 192.168.99.1 0.0% 0.2 0.2 0.2 0.3 0.0 2. gi1-2-v100.core1.n2net.net 0.0% 0.7 3.5 0.5 105.1 14.9 3. ge-6-10.car2.cleveland1.level3.net 0.0% 3.4 25.8 1.6 215.3 50.5 4. att-level3-te.washingtondc12.level3.net 17.7% 11.3 13.7 11.0 28.1 2.8 5. cr1.wswdc.ip.att.net 15.2% 25.1 23.3 21.2 25.1 1.2 6. 12.122.2.202 17.7% 22.4 22.8 20.9 24.6 0.7 7. ??? 8. 12.123.151.149 16.5% 39.4 41.6 20.9 107.2 23.3 9. 206.121.183.14 17.9% 23.5 24.9 23.4 56.8 5.8 10. 12.28.240.162 11.4% 23.8 26.4 23.8 94.4 11.3 11. ??? Host Loss% Last Avg Best Wrst StDev 1. 192.168.99.1 0.0% 0.2 0.2 0.2 0.3 0.0 2. gi1-2-v100.core1.n2net.net 0.0% 0.6 1.2 0.5 21.6 3.5 3. ge-6-10.car2.cleveland1.level3.net 0.0% 1.6 40.8 1.5 201.6 68.2 4. ae-1-60.edge3.washington4.level3.net 5.9% 10.0 10.5 9.9 16.4 1.5 5. cogent-comm.edge3.washington4.level3.net 20.0% 10.5 10.6 10.5 11.9 0.3 6. be2657.ccr42.dca01.atlas.cogentco.com 11.4% 11.5 11.4 11.2 11.6 0.1 7. be2892.ccr22.cle04.atlas.cogentco.com 14.3% 29.2 29.0 28.8 29.2 0.1 8. be2733.rcr21.cmh02.atlas.cogentco.com 17.6% 31.9 31.9 31.6 32.2 0.1 9. te0-0-2-0.nr11.b029193-0.cmh02.atlas.cogentco.com 17.6% 32.6 32.6 32.4 32.9 0.1 10. 38.104.22.62 20.6% 32.3 32.3 32.1 32.4 0.1 11. 38.111.183.169 14.7% 37.1 37.9 36.7 47.6 2.5 12. 38.111.183.170 20.6% 48.4 59.0 43.9 205.7 32.7 Jim Lohiser | Systems Engineer N2Net | 815 Superior Ave. E | Suite 425 | Cleveland | Ohio | 44114-2704
On Tue, Aug 30, 2016 at 9:09 AM, Jim Lohiser via Outages <outages@outages.org> wrote:
We are seeing heavy packet loss on our Level 3 connection in Cleveland, OH. Looks like it started around 5am Eastern time. Is anyone else seeing an issue?
Yes; this is toward ebay.com [66.211.160.86] from Comcast in FL: 1. 192.168.0.1 0.0% 122 0.5 0.4 0.2 0.5 0.0 2. 96.120.37.85 0.0% 122 7.9 8.1 4.1 23.7 2.6 3. xe-10-2-2-sur01.lakeworth.fl.pompano.comcast.net 0.0% 122 6.2 8.1 4.0 28.0 2.4 4. 162.151.123.33 0.0% 122 8.9 9.5 4.3 50.5 5.4 5. he-0-17-0-0-ar01.pompanobeach.fl.pompano.comcast.ne 0.0% 122 8.4 9.4 6.7 19.8 1.3 6. ae14.edge4.Miami1.Level3.net 11.5% 122 14.3 15.4 9.7 382.2 35.7 7. ae-2-30.bear2.Phoenix1.Level3.net 86.8% 122 62.4 63.4 61.4 73.3 2.9 8. ???
Jim Lohiser | Systems Engineer N2Net | 815 Superior Ave. E | Suite 425 | Cleveland | Ohio | 44114-2704
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
We saw similar issues this morning. They started around 4:45a EST and seem to be resolved at about 9:15a. Initially noticed issues with our customers that have IPSec tunnels back to us. Level3 is our primary carrier and the customers were also on Level3. MTR tests was showing heavy packet loss in Chicago for us. Matt Elliott | Network Engineer | SIS | www.thinksis.com<http://www.thinksis.com/> From: Andrew Wentzell via Outages <outages@outages.org><mailto:outages@outages.org> Reply: Andrew Wentzell <awentzell@gmail.com><mailto:awentzell@gmail.com> Date: August 30, 2016 at 9:40:59 AM To: outages@outages.org <outages@outages.org><mailto:outages@outages.org> Subject: Re: [outages] Heavy Packet Loss on Level 3 On Tue, Aug 30, 2016 at 9:09 AM, Jim Lohiser via Outages <outages@outages.org> wrote:
We are seeing heavy packet loss on our Level 3 connection in Cleveland, OH. Looks like it started around 5am Eastern time. Is anyone else seeing an issue?
Yes; this is toward ebay.com [66.211.160.86] from Comcast in FL: 1. 192.168.0.1 0.0% 122 0.5 0.4 0.2 0.5 0.0 2. 96.120.37.85 0.0% 122 7.9 8.1 4.1 23.7 2.6 3. xe-10-2-2-sur01.lakeworth.fl.pompano.comcast.net 0.0% 122 6.2 8.1 4.0 28.0 2.4 4. 162.151.123.33 0.0% 122 8.9 9.5 4.3 50.5 5.4 5. he-0-17-0-0-ar01.pompanobeach.fl.pompano.comcast.ne 0.0% 122 8.4 9.4 6.7 19.8 1.3 6. ae14.edge4.Miami1.Level3.net 11.5% 122 14.3 15.4 9.7 382.2 35.7 7. ae-2-30.bear2.Phoenix1.Level3.net 86.8% 122 62.4 63.4 61.4 73.3 2.9 8. ???
Jim Lohiser | Systems Engineer N2Net | 815 Superior Ave. E | Suite 425 | Cleveland | Ohio | 44114-2704
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages This message and attachment(s), if any, is intended for the sole use of the individual and/or entity for which it is addressed, and may contain information that is privileged, confidential and prohibited from disclosure under applicable law. If you are not the addressee, or authorized to receive this on behalf of the addressee, you are hereby notified that you may not use, copy, disclose or distribute to anyone this message or any part thereof. If you have received this in error, please immediately advise the sender by e-mail and delete this information and all attachments from your computer and network. Thank you.
participants (3)
-
Andrew Wentzell -
Jim Lohiser -
Matt Elliott