Level 3 DNS 4.2.2.1/4.2.2.2 Atlanta Issues
Level 3 appears to be having issues with 4.2.2.1 and 4.2.2.2 (The Atlanta Anycasted ones). Appears to have Packet Loss (Seen in Below MTR's). As well as having timeoutes resolving DNS against them. From me (AS36295) Host Loss% Snt Last Avg Best Wrst StDev 1. eth-vl0504-far1.coco-vr.flhsi.com 0.0% 468 0.3 0.4 0.2 3.7 0.3 2. eth-ge0007-car1.coco-vr.flhsi.com 0.0% 468 0.5 0.4 0.2 2.0 0.2 3. eth-ge0003-bar1.coco-vr.flhsi.com 0.0% 468 0.4 0.5 0.3 2.5 0.3 4. 66-194-200-17.static.twtelecom.net 0.0% 467 4.3 4.5 4.1 40.2 2.4 5. mia2-pr1-xe-1-3-0-0.us.twtelecom.net 0.0% 467 11.1 14.9 10.9 103.2 12.5 6. 4.68.71.97 3.4% 467 12.4 13.1 12.1 62.4 4.8 7. ae-2-3015.ear1.Atlanta2.Level3.net 97.4% 467 22153 22228 21002 23399 797.3 8. b.resolvers.Level3.net 53.3% 467 38.5 36.4 31.0 42.4 2.0 From another NLNOG Ring Participant (AS31939) Host Loss% Snt Last Avg Best Wrst StDev 1. c-65-50-1-69.ips.direcpath.com 0.0% 179 0.5 0.7 0.4 16.9 1.3 2. 172.24.0.13 0.0% 179 0.5 0.7 0.4 3.9 0.6 3. 172.24.0.242 0.0% 179 15.3 9.1 0.5 188.7 28.7 4. te0-3-0-5.ccr21.atl04.atlas.cogentco.com 0.0% 179 2.5 1.6 0.9 5.4 0.7 5. level3.atl04.atlas.cogentco.com 2.2% 179 0.6 1.7 0.5 48.0 5.3 6. ae-2-3015.ear1.Atlanta2.Level3.net 98.3% 178 22958 22009 20605 22958 1240. 7. a.resolvers.level3.net 65.7% 178 20.8 18.4 15.2 20.9 1.2 Nick Olsen Network Operations (855) FLSPEED x106
Oh. This wasn't my client's uplink. I saw this yesterday after 1600EST, but didn't have time then to run it down. And that was over a 6 hop path from VDSL in Clearwater FL. This minute, traces to 4.2.2.1 aren't dropping any packets. 4.2.2.2 is, about 35%. Those appear to be the ATL machines for me as well: HOST: stanley Loss% Snt Last Avg Best Wrst StDev 1.|-- 192.168.141.1 0.0% 10 1.0 1.0 0.9 1.1 0.0 2.|-- L100.TAMPFL-DSL-02.verizo 0.0% 10 27.4 27.8 27.1 29.1 0.6 3.|-- P0-1-0-0.TAMPFL-LCR-21.ve 0.0% 10 32.0 31.0 28.1 32.7 1.8 4.|-- 130.81.199.26 0.0% 10 28.7 35.5 28.7 86.4 17.9 5.|-- 0.xe-5-0-0.BR3.ATL4.ALTER 0.0% 10 44.8 44.7 43.7 45.5 0.5 6.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 7.|-- b.resolvers.Level3.net 70.0% 10 62.6 60.8 58.9 62.6 1.9 ----- Original Message -----
From: "Nick Olsen via Outages" <outages@outages.org> To: outages@outages.org Sent: Monday, January 12, 2015 5:00:10 PM Subject: [outages] Level 3 DNS 4.2.2.1/4.2.2.2 Atlanta Issues Level 3 appears to be having issues with 4.2.2.1 and 4.2.2.2 (The Atlanta Anycasted ones). Appears to have Packet Loss (Seen in Below MTR's). As well as having timeoutes resolving DNS against them.
From me (AS36295) Host Loss% Snt Last Avg Best Wrst StDev 1. eth-vl0504-far1.coco-vr.flhsi.com 0.0% 468 0.3 0.4 0.2 3.7 0.3 2. eth-ge0007-car1.coco-vr.flhsi.com 0.0% 468 0.5 0.4 0.2 2.0 0.2 3. eth-ge0003-bar1.coco-vr.flhsi.com 0.0% 468 0.4 0.5 0.3 2.5 0.3 4. 66-194-200-17.static.twtelecom.net 0.0% 467 4.3 4.5 4.1 40.2 2.4 5. mia2-pr1-xe-1-3-0-0.us.twtelecom.net 0.0% 467 11.1 14.9 10.9 103.2 12.5 6. 4.68.71.97 3.4% 467 12.4 13.1 12.1 62.4 4.8 7. ae-2-3015.ear1.Atlanta2.Level3.net 97.4% 467 22153 22228 21002 23399 797.3 8. b.resolvers.Level3.net 53.3% 467 38.5 36.4 31.0 42.4 2.0
From another NLNOG Ring Participant (AS31939) Host Loss% Snt Last Avg Best Wrst StDev 1. c-65-50-1-69.ips.direcpath.com 0.0% 179 0.5 0.7 0.4 16.9 1.3 2. 172.24.0.13 0.0% 179 0.5 0.7 0.4 3.9 0.6 3. 172.24.0.242 0.0% 179 15.3 9.1 0.5 188.7 28.7 4. te0-3-0-5.ccr21.atl04.atlas.cogentco.com 0.0% 179 2.5 1.6 0.9 5.4 0.7 5. level3.atl04.atlas.cogentco.com 2.2% 179 0.6 1.7 0.5 48.0 5.3 6. ae-2-3015.ear1.Atlanta2.Level3.net 98.3% 178 22958 22009 20605 22958 1240. 7. a.resolvers.level3.net 65.7% 178 20.8 18.4 15.2 20.9 1.2
Nick Olsen Network Operations (855) FLSPEED x106
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
-- Jay R. Ashworth Baylink jra@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://www.bcp38.info 2000 Land Rover DII St Petersburg FL USA BCP38: Ask For It By Name! +1 727 647 1274
Seeing the same from Atlanta as well as others, this is a 12 hour view: http://i.imgur.com/wYbxUNy.png On Tue, Jan 13, 2015 at 10:52 AM, Jay Ashworth via Outages < outages@outages.org> wrote:
Oh. This wasn't my client's uplink.
I saw this yesterday after 1600EST, but didn't have time then to run it down. And that was over a 6 hop path from VDSL in Clearwater FL.
This minute, traces to 4.2.2.1 aren't dropping any packets. 4.2.2.2 is, about 35%.
Those appear to be the ATL machines for me as well:
HOST: stanley Loss% Snt Last Avg Best Wrst StDev 1.|-- 192.168.141.1 0.0% 10 1.0 1.0 0.9 1.1 0.0 2.|-- L100.TAMPFL-DSL-02.verizo 0.0% 10 27.4 27.8 27.1 29.1 0.6 3.|-- P0-1-0-0.TAMPFL-LCR-21.ve 0.0% 10 32.0 31.0 28.1 32.7 1.8 4.|-- 130.81.199.26 0.0% 10 28.7 35.5 28.7 86.4 17.9 5.|-- 0.xe-5-0-0.BR3.ATL4.ALTER 0.0% 10 44.8 44.7 43.7 45.5 0.5 6.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 7.|-- b.resolvers.Level3.net 70.0% 10 62.6 60.8 58.9 62.6 1.9
----- Original Message -----
From: "Nick Olsen via Outages" <outages@outages.org> To: outages@outages.org Sent: Monday, January 12, 2015 5:00:10 PM Subject: [outages] Level 3 DNS 4.2.2.1/4.2.2.2 Atlanta Issues Level 3 appears to be having issues with 4.2.2.1 and 4.2.2.2 (The Atlanta Anycasted ones). Appears to have Packet Loss (Seen in Below MTR's). As well as having timeoutes resolving DNS against them.
From me (AS36295) Host Loss% Snt Last Avg Best Wrst StDev 1. eth-vl0504-far1.coco-vr.flhsi.com 0.0% 468 0.3 0.4 0.2 3.7 0.3 2. eth-ge0007-car1.coco-vr.flhsi.com 0.0% 468 0.5 0.4 0.2 2.0 0.2 3. eth-ge0003-bar1.coco-vr.flhsi.com 0.0% 468 0.4 0.5 0.3 2.5 0.3 4. 66-194-200-17.static.twtelecom.net 0.0% 467 4.3 4.5 4.1 40.2 2.4 5. mia2-pr1-xe-1-3-0-0.us.twtelecom.net 0.0% 467 11.1 14.9 10.9 103.2 12.5 6. 4.68.71.97 3.4% 467 12.4 13.1 12.1 62.4 4.8 7. ae-2-3015.ear1.Atlanta2.Level3.net 97.4% 467 22153 22228 21002 23399 797.3 8. b.resolvers.Level3.net 53.3% 467 38.5 36.4 31.0 42.4 2.0
From another NLNOG Ring Participant (AS31939) Host Loss% Snt Last Avg Best Wrst StDev 1. c-65-50-1-69.ips.direcpath.com 0.0% 179 0.5 0.7 0.4 16.9 1.3 2. 172.24.0.13 0.0% 179 0.5 0.7 0.4 3.9 0.6 3. 172.24.0.242 0.0% 179 15.3 9.1 0.5 188.7 28.7 4. te0-3-0-5.ccr21.atl04.atlas.cogentco.com 0.0% 179 2.5 1.6 0.9 5.4 0.7 5. level3.atl04.atlas.cogentco.com 2.2% 179 0.6 1.7 0.5 48.0 5.3 6. ae-2-3015.ear1.Atlanta2.Level3.net 98.3% 178 22958 22009 20605 22958 1240. 7. a.resolvers.level3.net 65.7% 178 20.8 18.4 15.2 20.9 1.2
Nick Olsen Network Operations (855) FLSPEED x106
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
-- Jay R. Ashworth Baylink jra@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://www.bcp38.info 2000 Land Rover DII St Petersburg FL USA BCP38: Ask For It By Name! +1 727 647 1274 _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
If you're a L3 customer then I'd suggest reporting it to them via standard channels. If you're not a L3 customer, then why are you attempting to use their revolvers? If you want a public resolver, use Google. 4.2.2.x are NOT intended for public use, and never have been. Scott On Tue, Jan 13, 2015 at 10:28 PM, Justin via Outages <outages@outages.org> wrote:
Seeing the same from Atlanta as well as others, this is a 12 hour view: http://i.imgur.com/wYbxUNy.png
On Tue, Jan 13, 2015 at 10:52 AM, Jay Ashworth via Outages < outages@outages.org> wrote:
Oh. This wasn't my client's uplink.
I saw this yesterday after 1600EST, but didn't have time then to run it down. And that was over a 6 hop path from VDSL in Clearwater FL.
This minute, traces to 4.2.2.1 aren't dropping any packets. 4.2.2.2 is, about 35%.
Those appear to be the ATL machines for me as well:
HOST: stanley Loss% Snt Last Avg Best Wrst StDev 1.|-- 192.168.141.1 0.0% 10 1.0 1.0 0.9 1.1 0.0 2.|-- L100.TAMPFL-DSL-02.verizo 0.0% 10 27.4 27.8 27.1 29.1 0.6 3.|-- P0-1-0-0.TAMPFL-LCR-21.ve 0.0% 10 32.0 31.0 28.1 32.7 1.8 4.|-- 130.81.199.26 0.0% 10 28.7 35.5 28.7 86.4 17.9 5.|-- 0.xe-5-0-0.BR3.ATL4.ALTER 0.0% 10 44.8 44.7 43.7 45.5 0.5 6.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 7.|-- b.resolvers.Level3.net 70.0% 10 62.6 60.8 58.9 62.6 1.9
----- Original Message -----
From: "Nick Olsen via Outages" <outages@outages.org> To: outages@outages.org Sent: Monday, January 12, 2015 5:00:10 PM Subject: [outages] Level 3 DNS 4.2.2.1/4.2.2.2 Atlanta Issues Level 3 appears to be having issues with 4.2.2.1 and 4.2.2.2 (The Atlanta Anycasted ones). Appears to have Packet Loss (Seen in Below MTR's). As well as having timeoutes resolving DNS against them.
From me (AS36295) Host Loss% Snt Last Avg Best Wrst StDev 1. eth-vl0504-far1.coco-vr.flhsi.com 0.0% 468 0.3 0.4 0.2 3.7 0.3 2. eth-ge0007-car1.coco-vr.flhsi.com 0.0% 468 0.5 0.4 0.2 2.0 0.2 3. eth-ge0003-bar1.coco-vr.flhsi.com 0.0% 468 0.4 0.5 0.3 2.5 0.3 4. 66-194-200-17.static.twtelecom.net 0.0% 467 4.3 4.5 4.1 40.2 2.4 5. mia2-pr1-xe-1-3-0-0.us.twtelecom.net 0.0% 467 11.1 14.9 10.9 103.2 12.5 6. 4.68.71.97 3.4% 467 12.4 13.1 12.1 62.4 4.8 7. ae-2-3015.ear1.Atlanta2.Level3.net 97.4% 467 22153 22228 21002 23399 797.3 8. b.resolvers.Level3.net 53.3% 467 38.5 36.4 31.0 42.4 2.0
From another NLNOG Ring Participant (AS31939) Host Loss% Snt Last Avg Best Wrst StDev 1. c-65-50-1-69.ips.direcpath.com 0.0% 179 0.5 0.7 0.4 16.9 1.3 2. 172.24.0.13 0.0% 179 0.5 0.7 0.4 3.9 0.6 3. 172.24.0.242 0.0% 179 15.3 9.1 0.5 188.7 28.7 4. te0-3-0-5.ccr21.atl04.atlas.cogentco.com 0.0% 179 2.5 1.6 0.9 5.4 0.7 5. level3.atl04.atlas.cogentco.com 2.2% 179 0.6 1.7 0.5 48.0 5.3 6. ae-2-3015.ear1.Atlanta2.Level3.net 98.3% 178 22958 22009 20605 22958 1240. 7. a.resolvers.level3.net 65.7% 178 20.8 18.4 15.2 20.9 1.2
Nick Olsen Network Operations (855) FLSPEED x106
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
-- Jay R. Ashworth Baylink jra@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://www.bcp38.info 2000 Land Rover DII St Petersburg FL USA BCP38: Ask For It By Name! +1 727 647 1274 _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
On 1/14/15 6:27, Scott Howard via Outages wrote:
If you're a L3 customer then I'd suggest reporting it to them via standard channels.
If you're not a L3 customer, then why are you attempting to use their revolvers? If you want a public resolver, use Google. 4.2.2.x are NOT intended for public use, and never have been.
+1 And they have never, even for L3 customers, been advocated for use as an automated ping target to test if "the Internet is working" or to trigger a backup or failover scenario. -- Jay Hennigan - CCIE #7880 - Network Engineering - jay@impulse.net Impulse Internet Service - http://www.impulse.net/ Your local telephone and internet company - 805 884-6323 - WB6RDV
Still seeing it here as well. But also to both 4.2.2.1 and 4.2.2.2 Host Loss% Snt Last Avg Best Wrst StDev 1. eth-vl0504-far1.coco-vr.flhsi.com 0.0% 432 0.3 0.2 0.1 3.0 0.2 2. eth-ge0007-car1.coco-vr.flhsi.com 0.0% 432 0.3 0.3 0.2 9.6 0.6 3. eth-ge0003-bar1.coco-vr.flhsi.com 0.0% 431 0.4 0.3 0.2 3.9 0.2 4. 66-194-200-17.static.twtelecom.net 0.0% 431 3.2 3.3 2.9 54.8 2.7 5. mia2-pr1-xe-1-3-0-0.us.twtelecom.net 0.2% 431 9.8 13.5 9.7 103.1 12.2 6. 4.68.71.97 0.5% 431 11.1 11.8 10.9 64.6 4.6 7. ae-2-3015.ear1.Atlanta2.Level3.net 97.4% 431 22139 22248 21053 23307 652.5 8. a.resolvers.level3.net 38.6% 431 37.5 42.3 31.1 71.3 7.4 Nick Olsen Network Operations (855) FLSPEED x106 ---------------------------------------- From: "Jay Ashworth via Outages" <outages@outages.org> Sent: Tuesday, January 13, 2015 12:51 PM To: outages@outages.org Subject: Re: [outages] Level 3 DNS 4.2.2.1/4.2.2.2 Atlanta Issues Oh. This wasn't my client's uplink. I saw this yesterday after 1600EST, but didn't have time then to run it down. And that was over a 6 hop path from VDSL in Clearwater FL. This minute, traces to 4.2.2.1 aren't dropping any packets. 4.2.2.2 is, about 35%. Those appear to be the ATL machines for me as well: HOST: stanley Loss% Snt Last Avg Best Wrst StDev 1.|-- 192.168.141.1 0.0% 10 1.0 1.0 0.9 1.1 0.0 2.|-- L100.TAMPFL-DSL-02.verizo 0.0% 10 27.4 27.8 27.1 29.1 0.6 3.|-- P0-1-0-0.TAMPFL-LCR-21.ve 0.0% 10 32.0 31.0 28.1 32.7 1.8 4.|-- 130.81.199.26 0.0% 10 28.7 35.5 28.7 86.4 17.9 5.|-- 0.xe-5-0-0.BR3.ATL4.ALTER 0.0% 10 44.8 44.7 43.7 45.5 0.5 6.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 7.|-- b.resolvers.Level3.net 70.0% 10 62.6 60.8 58.9 62.6 1.9 ----- Original Message -----
From: "Nick Olsen via Outages" <outages@outages.org> To: outages@outages.org Sent: Monday, January 12, 2015 5:00:10 PM Subject: [outages] Level 3 DNS 4.2.2.1/4.2.2.2 Atlanta Issues Level 3 appears to be having issues with 4.2.2.1 and 4.2.2.2 (The Atlanta Anycasted ones). Appears to have Packet Loss (Seen in Below MTR's). As well as having timeoutes resolving DNS against them.
From me (AS36295) Host Loss% Snt Last Avg Best Wrst StDev 1. eth-vl0504-far1.coco-vr.flhsi.com 0.0% 468 0.3 0.4 0.2 3.7 0.3 2. eth-ge0007-car1.coco-vr.flhsi.com 0.0% 468 0.5 0.4 0.2 2.0 0.2 3. eth-ge0003-bar1.coco-vr.flhsi.com 0.0% 468 0.4 0.5 0.3 2.5 0.3 4. 66-194-200-17.static.twtelecom.net 0.0% 467 4.3 4.5 4.1 40.2 2.4 5. mia2-pr1-xe-1-3-0-0.us.twtelecom.net 0.0% 467 11.1 14.9 10.9 103.2 12.5 6. 4.68.71.97 3.4% 467 12.4 13.1 12.1 62.4 4.8 7. ae-2-3015.ear1.Atlanta2.Level3.net 97.4% 467 22153 22228 21002 23399 797.3 8. b.resolvers.Level3.net 53.3% 467 38.5 36.4 31.0 42.4 2.0
From another NLNOG Ring Participant (AS31939) Host Loss% Snt Last Avg Best Wrst StDev 1. c-65-50-1-69.ips.direcpath.com 0.0% 179 0.5 0.7 0.4 16.9 1.3 2. 172.24.0.13 0.0% 179 0.5 0.7 0.4 3.9 0.6 3. 172.24.0.242 0.0% 179 15.3 9.1 0.5 188.7 28.7 4. te0-3-0-5.ccr21.atl04.atlas.cogentco.com 0.0% 179 2.5 1.6 0.9 5.4 0.7 5. level3.atl04.atlas.cogentco.com 2.2% 179 0.6 1.7 0.5 48.0 5.3 6. ae-2-3015.ear1.Atlanta2.Level3.net 98.3% 178 22958 22009 20605 22958 1240. 7. a.resolvers.level3.net 65.7% 178 20.8 18.4 15.2 20.9 1.2
Nick Olsen Network Operations (855) FLSPEED x106
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
-- Jay R. Ashworth Baylink jra@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://www.bcp38.info 2000 Land Rover DII St Petersburg FL USA BCP38: Ask For It By Name! +1 727 647 1274 _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
participants (5)
-
Jay Ashworth -
Jay Hennigan -
Justin -
Nick Olsen -
Scott Howard