Odd name resolution Errors OpenDNS In NC & SC
Seeing some odd name resolutions errors in the NC and SC regions getting random name resolutions that resolve to mail*.newsgeni.us. This is happening across multiple domain names even seeing it with reverse IP resolutions. Our monitoring alerts detected this so far at about 10+ sites in those states. The fix has been to move the sites off 208.67.222.222 and 208.67.220.220 and using a different DNS servers/providers. Anyone else seeing similar?
potentially related, we are seeing problems with rr/charter/spectrum/brighthouse/tw getting traffic back to level3/centurylink this morning. Seems anything routing via 0.ae0.pr0.dca20.tbone.rr.com just dies if its destined for as3356. If it takes a path via Atlanta, works great. Interestingly, both of the following destinations are in the same subnet, just a different fourth octet. roampile (0.0.0.0) Wed Oct 9 10:03:51 2019 Keys: Help Display mode Restart statistics Order of fields quit Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. Linksys20360 0.0% 4 0.8 1.1 0.8 1.9 0.0 2. cpe-69-132-132-1.carolina.res.rr.com 0.0% 4 11.7 13.3 11.7 16.1 1.9 3. cpe-174-111-099-000.triad.res.rr.com 0.0% 4 20.3 18.7 15.1 22.8 3.4 4. 24.93.70.64 0.0% 4 9.2 13.8 9.2 18.7 4.1 5. ge-5-1-0.rlghncpop-rtr1.southeast.rr.com 0.0% 4 34.4 22.7 16.8 34.4 8.2 6. 66.109.10.176 0.0% 4 32.9 32.9 27.2 40.6 5.7 7. ae-2-0.c1.chi75.tbone.rr.com 0.0% 4 35.9 35.2 29.8 41.3 4.7 8. 0.ae0.pr0.dca20.tbone.rr.com 0.0% 4 30.9 33.7 26.7 48.0 9.7 9. ??? dead..goes no further.. roampile (0.0.0.0) Wed Oct 9 10:04:20 2019 Keys: Help Display mode Restart statistics Order of fields quit Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. Linksys20360 0.0% 7 0.9 1.1 0.8 2.3 0.4 2. cpe-69-132-132-1.carolina.res.rr.com 0.0% 7 12.8 13.5 6.4 19.3 3.8 3. cpe-174-111-099-002.triad.res.rr.com 0.0% 7 12.3 18.4 11.0 34.4 9.3 4. chrlncnor-vis1.carolina.rr.com 0.0% 7 13.9 13.9 8.8 21.2 4.2 5. 24.93.67.204 0.0% 7 16.9 18.0 13.8 26.1 4.1 6. bu-ether11.atlngamq46w-bcr00.tbone.rr.com 0.0% 7 19.3 23.5 19.3 29.6 3.5 7. 66.109.5.125 0.0% 7 14.7 16.7 13.5 22.3 3.3 8. ae-10.edge4.Atlanta2.Level3.net 0.0% 7 19.1 22.2 18.6 32.7 4.8 9. ??? 10. hidden in dallas 0.0% 7 38.8 37.5 32.1 41.0 3.5 11. hidden in dallas 0.0% 6 32.7 34.5 32.7 36.9 1.5 On Wed, Oct 9, 2019 at 10:27 AM Jordan Morris via Outages <outages@outages.org> wrote:
Seeing some odd name resolutions errors in the NC and SC regions getting random name resolutions that resolve to mail*.newsgeni.us. This is happening across multiple domain names even seeing it with reverse IP resolutions. Our monitoring alerts detected this so far at about 10+ sites in those states. The fix has been to move the sites off 208.67.222.222 and 208.67.220.220 and using a different DNS servers/providers. Anyone else seeing similar?
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
I've noticed the exact same thing since yesterday on my home Spectrum network. Traceroute to 4.2.2.2 dies on the same hop. I spent the last hour on the phone arguing with them and they're finally opening a ticket to look into it. C:\Users\Eric>tracert 4.2.2.2 Tracing route to b.resolvers.Level3.net [4.2.2.2] over a maximum of 30 hops: 1 <1 ms <1 ms <1 ms 192.168.1.1 2 8 ms 8 ms 8 ms 142.254.150.77 3 16 ms 20 ms 21 ms ae59.nolmoh2201h.midwest.rr.com [24.164.119.89] 4 9 ms 10 ms 9 ms be23.sgvlohaj01r.midwest.rr.com [24.33.103.92] 5 15 ms 15 ms 15 ms be11.clevohek02r.midwest.rr.com [65.29.1.97] 6 15 ms 15 ms 15 ms be25.clevohek01r.midwest.rr.com [65.29.1.32] 7 36 ms 39 ms 39 ms bu-ether17.vinnva0510w-bcr00.tbone.rr.com [66.109.6.70] 8 32 ms 31 ms 32 ms 0.ae3.pr0.dca20.tbone.rr.com [107.14.19.160] 9 * * * Request timed out. 10 * * * Request timed out. 11 * * * Request timed out. 12 * * * Request timed out. 13 * * * Request timed out. 14 * * * Request timed out. Eric Rechnitzer eric.rechnitzer@gmail.com On Wed, Oct 9, 2019 at 11:05 AM Shawn Solomon via Outages < outages@outages.org> wrote:
potentially related, we are seeing problems with rr/charter/spectrum/brighthouse/tw getting traffic back to level3/centurylink this morning. Seems anything routing via 0.ae0.pr0.dca20.tbone.rr.com just dies if its destined for as3356. If it takes a path via Atlanta, works great. Interestingly, both of the following destinations are in the same subnet, just a different fourth octet.
roampile (0.0.0.0) Wed Oct 9 10:03:51 2019 Keys: Help Display mode Restart statistics Order of fields quit
Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. Linksys20360 0.0% 4 0.8 1.1 0.8 1.9 0.0 2. cpe-69-132-132-1.carolina.res.rr.com 0.0% 4 11.7 13.3 11.7 16.1 1.9 3. cpe-174-111-099-000.triad.res.rr.com 0.0% 4 20.3 18.7 15.1 22.8 3.4 4. 24.93.70.64 0.0% 4 9.2 13.8 9.2 18.7 4.1 5. ge-5-1-0.rlghncpop-rtr1.southeast.rr.com 0.0% 4 34.4 22.7 16.8 34.4 8.2 6. 66.109.10.176 0.0% 4 32.9 32.9 27.2 40.6 5.7 7. ae-2-0.c1.chi75.tbone.rr.com 0.0% 4 35.9 35.2 29.8 41.3 4.7 8. 0.ae0.pr0.dca20.tbone.rr.com 0.0% 4 30.9 33.7 26.7 48.0 9.7 9. ??? dead..goes no further..
roampile (0.0.0.0) Wed Oct 9 10:04:20 2019 Keys: Help Display mode Restart statistics Order of fields quit
Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. Linksys20360 0.0% 7 0.9 1.1 0.8 2.3 0.4 2. cpe-69-132-132-1.carolina.res.rr.com 0.0% 7 12.8 13.5 6.4 19.3 3.8 3. cpe-174-111-099-002.triad.res.rr.com 0.0% 7 12.3 18.4 11.0 34.4 9.3 4. chrlncnor-vis1.carolina.rr.com 0.0% 7 13.9 13.9 8.8 21.2 4.2 5. 24.93.67.204 0.0% 7 16.9 18.0 13.8 26.1 4.1 6. bu-ether11.atlngamq46w-bcr00.tbone.rr.com 0.0% 7 19.3 23.5 19.3 29.6 3.5 7. 66.109.5.125 0.0% 7 14.7 16.7 13.5 22.3 3.3 8. ae-10.edge4.Atlanta2.Level3.net 0.0% 7 19.1 22.2 18.6 32.7 4.8 9. ??? 10. hidden in dallas 0.0% 7 38.8 37.5 32.1 41.0 3.5 11. hidden in dallas 0.0% 6 32.7 34.5 32.7 36.9 1.5
On Wed, Oct 9, 2019 at 10:27 AM Jordan Morris via Outages <outages@outages.org> wrote:
Seeing some odd name resolutions errors in the NC and SC regions getting
random name resolutions that resolve to mail*.newsgeni.us. This is happening across multiple domain names even seeing it with reverse IP resolutions. Our monitoring alerts detected this so far at about 10+ sites in those states. The fix has been to move the sites off 208.67.222.222 and 208.67.220.220 and using a different DNS servers/providers. Anyone else seeing similar?
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
I think you are correct that this is the root cause a few of the sites even the alternative DNS servers routed out 0.ae1.pr0.dca20.tbone.rr.com and went dead in the water after that. Jordan Morris SALES: 800-249-1348 SUPPORT: 888-920-5685 www.crosswindvoip.com 2201 N. Camino Principal Ste. 125 Tucson, AZ 85715 -----Original Message----- From: Shawn Solomon <sdsolomo@gmail.com> Sent: Wednesday, October 9, 2019 8:04 AM To: Jordan Morris <jmorris@crosswindvoip.com> Cc: outages@outages.org Subject: Re: [outages] Odd name resolution Errors OpenDNS In NC & SC potentially related, we are seeing problems with rr/charter/spectrum/brighthouse/tw getting traffic back to level3/centurylink this morning. Seems anything routing via 0.ae0.pr0.dca20.tbone.rr.com just dies if its destined for as3356. If it takes a path via Atlanta, works great. Interestingly, both of the following destinations are in the same subnet, just a different fourth octet. roampile (0.0.0.0) Wed Oct 9 10:03:51 2019 Keys: Help Display mode Restart statistics Order of fields quit Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. Linksys20360 0.0% 4 0.8 1.1 0.8 1.9 0.0 2. cpe-69-132-132-1.carolina.res.rr.com 0.0% 4 11.7 13.3 11.7 16.1 1.9 3. cpe-174-111-099-000.triad.res.rr.com 0.0% 4 20.3 18.7 15.1 22.8 3.4 4. 24.93.70.64 0.0% 4 9.2 13.8 9.2 18.7 4.1 5. ge-5-1-0.rlghncpop-rtr1.southeast.rr.com 0.0% 4 34.4 22.7 16.8 34.4 8.2 6. 66.109.10.176 0.0% 4 32.9 32.9 27.2 40.6 5.7 7. ae-2-0.c1.chi75.tbone.rr.com 0.0% 4 35.9 35.2 29.8 41.3 4.7 8. 0.ae0.pr0.dca20.tbone.rr.com 0.0% 4 30.9 33.7 26.7 48.0 9.7 9. ??? dead..goes no further.. roampile (0.0.0.0) Wed Oct 9 10:04:20 2019 Keys: Help Display mode Restart statistics Order of fields quit Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. Linksys20360 0.0% 7 0.9 1.1 0.8 2.3 0.4 2. cpe-69-132-132-1.carolina.res.rr.com 0.0% 7 12.8 13.5 6.4 19.3 3.8 3. cpe-174-111-099-002.triad.res.rr.com 0.0% 7 12.3 18.4 11.0 34.4 9.3 4. chrlncnor-vis1.carolina.rr.com 0.0% 7 13.9 13.9 8.8 21.2 4.2 5. 24.93.67.204 0.0% 7 16.9 18.0 13.8 26.1 4.1 6. bu-ether11.atlngamq46w-bcr00.tbone.rr.com 0.0% 7 19.3 23.5 19.3 29.6 3.5 7. 66.109.5.125 0.0% 7 14.7 16.7 13.5 22.3 3.3 8. ae-10.edge4.Atlanta2.Level3.net 0.0% 7 19.1 22.2 18.6 32.7 4.8 9. ??? 10. hidden in dallas 0.0% 7 38.8 37.5 32.1 41.0 3.5 11. hidden in dallas 0.0% 6 32.7 34.5 32.7 36.9 1.5 On Wed, Oct 9, 2019 at 10:27 AM Jordan Morris via Outages <outages@outages.org> wrote:
Seeing some odd name resolutions errors in the NC and SC regions getting random name resolutions that resolve to mail*.newsgeni.us. This is happening across multiple domain names even seeing it with reverse IP resolutions. Our monitoring alerts detected this so far at about 10+ sites in those states. The fix has been to move the sites off 208.67.222.222 and 208.67.220.220 and using a different DNS servers/providers. Anyone else seeing similar?
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
My ticket # is sci-251834 and I sent them Shawn's traceroutes as well. Thanks, Eric Rechnitzer eric.rechnitzer@gmail.com On Wed, Oct 9, 2019 at 11:16 AM Jordan Morris via Outages < outages@outages.org> wrote:
I think you are correct that this is the root cause a few of the sites even the alternative DNS servers routed out 0.ae1.pr0.dca20.tbone.rr.com and went dead in the water after that.
Jordan Morris SALES: 800-249-1348 SUPPORT: 888-920-5685 www.crosswindvoip.com 2201 N. Camino Principal Ste. 125 Tucson, AZ 85715
-----Original Message----- From: Shawn Solomon <sdsolomo@gmail.com> Sent: Wednesday, October 9, 2019 8:04 AM To: Jordan Morris <jmorris@crosswindvoip.com> Cc: outages@outages.org Subject: Re: [outages] Odd name resolution Errors OpenDNS In NC & SC
potentially related, we are seeing problems with rr/charter/spectrum/brighthouse/tw getting traffic back to level3/centurylink this morning. Seems anything routing via 0.ae0.pr0.dca20.tbone.rr.com just dies if its destined for as3356. If it takes a path via Atlanta, works great. Interestingly, both of the following destinations are in the same subnet, just a different fourth octet.
roampile (0.0.0.0) Wed Oct 9 10:03:51 2019 Keys: Help Display mode Restart statistics Order of fields quit
Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. Linksys20360 0.0% 4 0.8 1.1 0.8 1.9 0.0 2. cpe-69-132-132-1.carolina.res.rr.com 0.0% 4 11.7 13.3 11.7 16.1 1.9 3. cpe-174-111-099-000.triad.res.rr.com 0.0% 4 20.3 18.7 15.1 22.8 3.4 4. 24.93.70.64 0.0% 4 9.2 13.8 9.2 18.7 4.1 5. ge-5-1-0.rlghncpop-rtr1.southeast.rr.com 0.0% 4 34.4 22.7 16.8 34.4 8.2 6. 66.109.10.176 0.0% 4 32.9 32.9 27.2 40.6 5.7 7. ae-2-0.c1.chi75.tbone.rr.com 0.0% 4 35.9 35.2 29.8 41.3 4.7 8. 0.ae0.pr0.dca20.tbone.rr.com 0.0% 4 30.9 33.7 26.7 48.0 9.7 9. ??? dead..goes no further..
roampile (0.0.0.0) Wed Oct 9 10:04:20 2019 Keys: Help Display mode Restart statistics Order of fields quit
Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. Linksys20360 0.0% 7 0.9 1.1 0.8 2.3 0.4 2. cpe-69-132-132-1.carolina.res.rr.com 0.0% 7 12.8 13.5 6.4 19.3 3.8 3. cpe-174-111-099-002.triad.res.rr.com 0.0% 7 12.3 18.4 11.0 34.4 9.3 4. chrlncnor-vis1.carolina.rr.com 0.0% 7 13.9 13.9 8.8 21.2 4.2 5. 24.93.67.204 0.0% 7 16.9 18.0 13.8 26.1 4.1 6. bu-ether11.atlngamq46w-bcr00.tbone.rr.com 0.0% 7 19.3 23.5 19.3 29.6 3.5 7. 66.109.5.125 0.0% 7 14.7 16.7 13.5 22.3 3.3 8. ae-10.edge4.Atlanta2.Level3.net 0.0% 7 19.1 22.2 18.6 32.7 4.8 9. ??? 10. hidden in dallas 0.0% 7 38.8 37.5 32.1 41.0 3.5 11. hidden in dallas 0.0% 6 32.7 34.5 32.7 36.9 1.5
On Wed, Oct 9, 2019 at 10:27 AM Jordan Morris via Outages < outages@outages.org> wrote:
Seeing some odd name resolutions errors in the NC and SC regions getting
random name resolutions that resolve to mail*.newsgeni.us. This is happening across multiple domain names even seeing it with reverse IP resolutions. Our monitoring alerts detected this so far at about 10+ sites in those states. The fix has been to move the sites off 208.67.222.222 and 208.67.220.220 and using a different DNS servers/providers. Anyone else seeing similar?
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
I'm seeing the same problem from RoadRunner/Spectrum 96.10.190.170 $ mtr --report-wide 4.2.2.2 Start: 2019-10-09T14:29:48-0400 HOST: asdf Loss% Snt Last Avg Best Wrst StDev 1.|-- _gateway 0.0% 10 0.3 0.3 0.2 0.3 0.0 2.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 3.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 4.|-- cpe-174-111-105-048.triad.res.rr.com 0.0% 10 16.9 19.5 14.7 29.3 4.9 5.|-- cpe-024-025-041-106.ec.res.rr.com 0.0% 10 14.9 15.7 8.4 22.4 4.6 6.|-- be31.drhmncev01r.southeast.rr.com 0.0% 10 26.2 24.8 14.1 48.4 11.8 7.|-- 66.109.10.176 0.0% 10 24.2 25.7 19.9 30.8 3.3 8.|-- ae-2-0.c1.chi75.tbone.rr.com 0.0% 10 34.6 28.2 22.0 34.6 3.8 9.|-- 0.ae1.pr0.dca20.tbone.rr.com 0.0% 10 36.1 29.0 21.5 38.3 6.1 10.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 On Wed, Oct 9, 2019 at 11:34 AM Eric Rechnitzer via Outages < outages@outages.org> wrote:
My ticket # is sci-251834 and I sent them Shawn's traceroutes as well.
Thanks, Eric Rechnitzer eric.rechnitzer@gmail.com
On Wed, Oct 9, 2019 at 11:16 AM Jordan Morris via Outages < outages@outages.org> wrote:
I think you are correct that this is the root cause a few of the sites even the alternative DNS servers routed out 0.ae1.pr0.dca20.tbone.rr.com and went dead in the water after that.
Jordan Morris SALES: 800-249-1348 SUPPORT: 888-920-5685 www.crosswindvoip.com 2201 N. Camino Principal Ste. 125 Tucson, AZ 85715
-----Original Message----- From: Shawn Solomon <sdsolomo@gmail.com> Sent: Wednesday, October 9, 2019 8:04 AM To: Jordan Morris <jmorris@crosswindvoip.com> Cc: outages@outages.org Subject: Re: [outages] Odd name resolution Errors OpenDNS In NC & SC
potentially related, we are seeing problems with rr/charter/spectrum/brighthouse/tw getting traffic back to level3/centurylink this morning. Seems anything routing via 0.ae0.pr0.dca20.tbone.rr.com just dies if its destined for as3356. If it takes a path via Atlanta, works great. Interestingly, both of the following destinations are in the same subnet, just a different fourth octet.
roampile (0.0.0.0) Wed Oct 9 10:03:51 2019 Keys: Help Display mode Restart statistics Order of fields quit
Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. Linksys20360 0.0% 4 0.8 1.1 0.8 1.9 0.0 2. cpe-69-132-132-1.carolina.res.rr.com 0.0% 4 11.7 13.3 11.7 16.1 1.9 3. cpe-174-111-099-000.triad.res.rr.com 0.0% 4 20.3 18.7 15.1 22.8 3.4 4. 24.93.70.64 0.0% 4 9.2 13.8 9.2 18.7 4.1 5. ge-5-1-0.rlghncpop-rtr1.southeast.rr.com 0.0% 4 34.4 22.7 16.8 34.4 8.2 6. 66.109.10.176 0.0% 4 32.9 32.9 27.2 40.6 5.7 7. ae-2-0.c1.chi75.tbone.rr.com 0.0% 4 35.9 35.2 29.8 41.3 4.7 8. 0.ae0.pr0.dca20.tbone.rr.com 0.0% 4 30.9 33.7 26.7 48.0 9.7 9. ??? dead..goes no further..
roampile (0.0.0.0) Wed Oct 9 10:04:20 2019 Keys: Help Display mode Restart statistics Order of fields quit
Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. Linksys20360 0.0% 7 0.9 1.1 0.8 2.3 0.4 2. cpe-69-132-132-1.carolina.res.rr.com 0.0% 7 12.8 13.5 6.4 19.3 3.8 3. cpe-174-111-099-002.triad.res.rr.com 0.0% 7 12.3 18.4 11.0 34.4 9.3 4. chrlncnor-vis1.carolina.rr.com 0.0% 7 13.9 13.9 8.8 21.2 4.2 5. 24.93.67.204 0.0% 7 16.9 18.0 13.8 26.1 4.1 6. bu-ether11.atlngamq46w-bcr00.tbone.rr.com 0.0% 7 19.3 23.5 19.3 29.6 3.5 7. 66.109.5.125 0.0% 7 14.7 16.7 13.5 22.3 3.3 8. ae-10.edge4.Atlanta2.Level3.net 0.0% 7 19.1 22.2 18.6 32.7 4.8 9. ??? 10. hidden in dallas 0.0% 7 38.8 37.5 32.1 41.0 3.5 11. hidden in dallas 0.0% 6 32.7 34.5 32.7 36.9 1.5
On Wed, Oct 9, 2019 at 10:27 AM Jordan Morris via Outages < outages@outages.org> wrote:
Seeing some odd name resolutions errors in the NC and SC regions
getting random name resolutions that resolve to mail*.newsgeni.us. This is happening across multiple domain names even seeing it with reverse IP resolutions. Our monitoring alerts detected this so far at about 10+ sites in those states. The fix has been to move the sites off 208.67.222.222 and 208.67.220.220 and using a different DNS servers/providers. Anyone else seeing similar?
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
It looks like they just rerouted traffic through another device and I'm able to hit 4.2.2.2 now. Tracing route to b.resolvers.Level3.net [4.2.2.2] over a maximum of 30 hops: 1 <1 ms <1 ms <1 ms gw 2 8 ms 8 ms 7 ms 142.254.150.77 3 24 ms 23 ms 21 ms ae59.nolmoh2201h.midwest.rr.com [24.164.119.89] 4 9 ms 9 ms 11 ms be23.sgvlohaj01r.midwest.rr.com [24.33.103.92] 5 14 ms 16 ms 14 ms be11.clevohek02r.midwest.rr.com [65.29.1.97] 6 16 ms 15 ms 15 ms be25.clevohek01r.midwest.rr.com [65.29.1.32] 7 38 ms 39 ms 39 ms bu-ether17.vinnva0510w-bcr00.tbone.rr.com [66.109.6.70] 8 38 ms 39 ms 39 ms bu-ether12.nwrknjmd67w-bcr00.tbone.rr.com [66.109.6.29] 9 * * * Request timed out. 10 * * * Request timed out. 11 39 ms 38 ms 38 ms b.resolvers.Level3.net [4.2.2.2] Eric Rechnitzer eric.rechnitzer@gmail.com On Wed, Oct 9, 2019 at 2:37 PM Mark Price <mprice@tqhosting.com> wrote:
I'm seeing the same problem from RoadRunner/Spectrum 96.10.190.170
$ mtr --report-wide 4.2.2.2 Start: 2019-10-09T14:29:48-0400 HOST: asdf Loss% Snt Last Avg Best Wrst StDev 1.|-- _gateway 0.0% 10 0.3 0.3 0.2 0.3 0.0 2.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 3.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 4.|-- cpe-174-111-105-048.triad.res.rr.com 0.0% 10 16.9 19.5 14.7 29.3 4.9 5.|-- cpe-024-025-041-106.ec.res.rr.com 0.0% 10 14.9 15.7 8.4 22.4 4.6 6.|-- be31.drhmncev01r.southeast.rr.com 0.0% 10 26.2 24.8 14.1 48.4 11.8 7.|-- 66.109.10.176 0.0% 10 24.2 25.7 19.9 30.8 3.3 8.|-- ae-2-0.c1.chi75.tbone.rr.com 0.0% 10 34.6 28.2 22.0 34.6 3.8 9.|-- 0.ae1.pr0.dca20.tbone.rr.com 0.0% 10 36.1 29.0 21.5 38.3 6.1 10.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
On Wed, Oct 9, 2019 at 11:34 AM Eric Rechnitzer via Outages < outages@outages.org> wrote:
My ticket # is sci-251834 and I sent them Shawn's traceroutes as well.
Thanks, Eric Rechnitzer eric.rechnitzer@gmail.com
On Wed, Oct 9, 2019 at 11:16 AM Jordan Morris via Outages < outages@outages.org> wrote:
I think you are correct that this is the root cause a few of the sites even the alternative DNS servers routed out 0.ae1.pr0.dca20.tbone.rr.com and went dead in the water after that.
Jordan Morris SALES: 800-249-1348 SUPPORT: 888-920-5685 www.crosswindvoip.com 2201 N. Camino Principal Ste. 125 Tucson, AZ 85715
-----Original Message----- From: Shawn Solomon <sdsolomo@gmail.com> Sent: Wednesday, October 9, 2019 8:04 AM To: Jordan Morris <jmorris@crosswindvoip.com> Cc: outages@outages.org Subject: Re: [outages] Odd name resolution Errors OpenDNS In NC & SC
potentially related, we are seeing problems with rr/charter/spectrum/brighthouse/tw getting traffic back to level3/centurylink this morning. Seems anything routing via 0.ae0.pr0.dca20.tbone.rr.com just dies if its destined for as3356. If it takes a path via Atlanta, works great. Interestingly, both of the following destinations are in the same subnet, just a different fourth octet.
roampile (0.0.0.0) Wed Oct 9 10:03:51 2019 Keys: Help Display mode Restart statistics Order of fields quit
Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. Linksys20360 0.0% 4 0.8 1.1 0.8 1.9 0.0 2. cpe-69-132-132-1.carolina.res.rr.com 0.0% 4 11.7 13.3 11.7 16.1 1.9 3. cpe-174-111-099-000.triad.res.rr.com 0.0% 4 20.3 18.7 15.1 22.8 3.4 4. 24.93.70.64 0.0% 4 9.2 13.8 9.2 18.7 4.1 5. ge-5-1-0.rlghncpop-rtr1.southeast.rr.com 0.0% 4 34.4 22.7 16.8 34.4 8.2 6. 66.109.10.176 0.0% 4 32.9 32.9 27.2 40.6 5.7 7. ae-2-0.c1.chi75.tbone.rr.com 0.0% 4 35.9 35.2 29.8 41.3 4.7 8. 0.ae0.pr0.dca20.tbone.rr.com 0.0% 4 30.9 33.7 26.7 48.0 9.7 9. ??? dead..goes no further..
roampile (0.0.0.0) Wed Oct 9 10:04:20 2019 Keys: Help Display mode Restart statistics Order of fields quit
Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. Linksys20360 0.0% 7 0.9 1.1 0.8 2.3 0.4 2. cpe-69-132-132-1.carolina.res.rr.com 0.0% 7 12.8 13.5 6.4 19.3 3.8 3. cpe-174-111-099-002.triad.res.rr.com 0.0% 7 12.3 18.4 11.0 34.4 9.3 4. chrlncnor-vis1.carolina.rr.com 0.0% 7 13.9 13.9 8.8 21.2 4.2 5. 24.93.67.204 0.0% 7 16.9 18.0 13.8 26.1 4.1 6. bu-ether11.atlngamq46w-bcr00.tbone.rr.com 0.0% 7 19.3 23.5 19.3 29.6 3.5 7. 66.109.5.125 0.0% 7 14.7 16.7 13.5 22.3 3.3 8. ae-10.edge4.Atlanta2.Level3.net 0.0% 7 19.1 22.2 18.6 32.7 4.8 9. ??? 10. hidden in dallas 0.0% 7 38.8 37.5 32.1 41.0 3.5 11. hidden in dallas 0.0% 6 32.7 34.5 32.7 36.9 1.5
On Wed, Oct 9, 2019 at 10:27 AM Jordan Morris via Outages < outages@outages.org> wrote:
Seeing some odd name resolutions errors in the NC and SC regions
getting random name resolutions that resolve to mail*.newsgeni.us. This is happening across multiple domain names even seeing it with reverse IP resolutions. Our monitoring alerts detected this so far at about 10+ sites in those states. The fix has been to move the sites off 208.67.222.222 and 208.67.220.220 and using a different DNS servers/providers. Anyone else seeing similar?
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
participants (4)
-
Eric Rechnitzer -
Jordan Morris -
Mark Price -
Shawn Solomon