Google's DNS 8.8.8.8 & 8.8.4.4
In los angeles, experiencing packet loss, slow resolution, and high ping times. it appears google's any cast dns is not working properly. --- 8.8.8.8 ping statistics --- 21 packets transmitted, 11 received, +1 errors, 47% packet loss, time 20229ms rtt min/avg/max/mdev = 44.635/151.432/258.331/76.230 ms from multiple providers. i would check over any2, but it appears my link to any2 is down (hopefully a separate issue) any confirm that they are seeing this issue? Thanks
On Wed, Aug 23, 2017 at 02:30:53AM -0700, Mr. NPP via Outages <outages@outages.org> wrote a message of 49 lines which said:
--- 8.8.8.8 ping statistics ---
ping is a bad test for Google Public DNS, which has aggressive rate limiters for ICMP echo.
I am seeing 6ms on L3, Qwest, TWTC, and TWC out of Louisville, Atlanta, Charlotte, Richmond, and Tampa. Thanks, Riley Childs -- Riley Childs phone: (704) 497-2086 web: www.rileychilds.me On Wed, Aug 23, 2017 at 5:30 AM, Mr. NPP via Outages <outages@outages.org> wrote:
In los angeles, experiencing packet loss, slow resolution, and high ping times. it appears google's any cast dns is not working properly.
--- 8.8.8.8 ping statistics --- 21 packets transmitted, 11 received, +1 errors, 47% packet loss, time 20229ms rtt min/avg/max/mdev = 44.635/151.432/258.331/76.230 ms
from multiple providers. i would check over any2, but it appears my link to any2 is down (hopefully a separate issue)
any confirm that they are seeing this issue?
Thanks
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
Hi, Yes, is having random timeouts for us . TR to 8.8.8.8: 4. Google-level3-100G.LosAngeles1.Level3.net 0.0% 2 9.9 9.9 9.9 9.9 0.0 5. 108.170.247.208 0.0% 2 10.0 10.0 10.0 10.1 0.0 6. 108.170.230.123 0.0% 2 17.1 17.1 17.1 17.1 0.0 7. 72.14.239.162 0.0% 2 39.0 38.8 38.7 39.0 0.2 8. 209.85.142.20 0.0% 2 38.6 38.8 38.6 38.9 0.2 9. ??? On Wed, Aug 23, 2017 at 12:30 PM, Mr. NPP via Outages <outages@outages.org> wrote:
In los angeles, experiencing packet loss, slow resolution, and high ping times. it appears google's any cast dns is not working properly.
--- 8.8.8.8 ping statistics --- 21 packets transmitted, 11 received, +1 errors, 47% packet loss, time 20229ms rtt min/avg/max/mdev = 44.635/151.432/258.331/76.230 ms
from multiple providers. i would check over any2, but it appears my link to any2 is down (hopefully a separate issue)
any confirm that they are seeing this issue?
Thanks
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
I forgot to say that DNS requests timeout randomly. I posted TR in case to help locate which Google DC gets this requests On Wed, Aug 23, 2017 at 12:36 PM, cgm <999cgm@gmail.com> wrote:
Hi,
Yes, is having random timeouts for us . TR to 8.8.8.8:
4. Google-level3-100G.LosAngeles1.Level3.net 0.0% 2 9.9 9.9 9.9 9.9 0.0 5. 108.170.247.208 0.0% 2 10.0 10.0 10.0 10.1 0.0 6. 108.170.230.123 0.0% 2 17.1 17.1 17.1 17.1 0.0 7. 72.14.239.162 0.0% 2 39.0 38.8 38.7 39.0 0.2 8. 209.85.142.20 0.0% 2 38.6 38.8 38.6 38.9 0.2 9. ???
On Wed, Aug 23, 2017 at 12:30 PM, Mr. NPP via Outages <outages@outages.org
wrote:
In los angeles, experiencing packet loss, slow resolution, and high ping times. it appears google's any cast dns is not working properly.
--- 8.8.8.8 ping statistics --- 21 packets transmitted, 11 received, +1 errors, 47% packet loss, time 20229ms rtt min/avg/max/mdev = 44.635/151.432/258.331/76.230 ms
from multiple providers. i would check over any2, but it appears my link to any2 is down (hopefully a separate issue)
any confirm that they are seeing this issue?
Thanks
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
Hello, cannot confirm Paketlosses here in Berlin, Germany. Best regards Marcel --- www.mrgeneration.de - The home of Anime Marcel Herrguth Tel: 0176/65 92 44 85 Postfach 90 02 05 12402 Berlin, Germany The home of Anime @ Facebook:https://www.facebook.com/mrgeneration.de Twitter: https://twitter.com/Mr_Generation Am 23.8.2017 11:30, schrieb Mr. NPP via Outages:
In los angeles, experiencing packet loss, slow resolution, and high ping times. it appears google's any cast dns is not working properly.
--- 8.8.8.8 ping statistics --- 21 packets transmitted, 11 received, +1 errors, 47% packet loss, time 20229ms rtt min/avg/max/mdev = 44.635/151.432/258.331/76.230 ms
from multiple providers. i would check over any2, but it appears my link to any2 is down (hopefully a separate issue)
any confirm that they are seeing this issue?
Thanks _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
Same here from Choopa in Los Angeles, MTR to 8.8.8.8: Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. ??? 2. 173.199.80.193 0.0% 54 16.0 17.7 6.9 72.3 8.3 3. vl823-br1-mlx.lax3.choopa.net 0.0% 54 30.6 4.3 0.3 40.1 7.6 4. 72.14.219.158 0.0% 54 0.7 0.7 0.4 5.1 0.6 5. 108.170.247.208 0.0% 54 0.7 0.7 0.4 1.8 0.0 108.170.247.193 6. 108.170.234.25 0.0% 54 0.8 7.3 0.5 65.4 14.3 108.170.238.31 7. 72.14.239.160 22.6% 53 130.1 96.2 24.6 287.4 80.3 google-public-dns-a.google.com 8. 64.233.174.97 24.5% 53 239.5 97.8 24.6 299.4 89.7 google-public-dns-a.google.com 9. google-public-dns-a.google.com 52.8% 53 274.8 152.2 41.1 315.8 82.8 Pings from several locations worldwide: http://xor.meo.ws/MaZlLiwGq3a8pDCQZWbJzaGILoeNiU6Z.txt MTRs: http://xor.meo.ws/j3EOvCwf78JC0xTlzhB56rRNHOIfWjwm.txt On 2017-08-23 11:30, Mr. NPP via Outages wrote:
In los angeles, experiencing packet loss, slow resolution, and high ping times. it appears google's any cast dns is not working properly.
--- 8.8.8.8 ping statistics --- 21 packets transmitted, 11 received, +1 errors, 47% packet loss, time 20229ms rtt min/avg/max/mdev = 44.635/151.432/258.331/76.230 ms
from multiple providers. i would check over any2, but it appears my link to any2 is down (hopefully a separate issue)
any confirm that they are seeing this issue?
Thanks
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
It seems they have solved their issue by now, at least there is not much ICMP loss anymore from my probes: http://xor.meo.ws/NSEuWr9EZnjyT447rEC9DSP0mSjenR4u/8.8.8.8_loss.png On 2017-08-23 11:44, Katie Holly via Outages wrote:
Same here from Choopa in Los Angeles, MTR to 8.8.8.8: Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. ??? 2. 173.199.80.193 0.0% 54 16.0 17.7 6.9 72.3 8.3 3. vl823-br1-mlx.lax3.choopa.net 0.0% 54 30.6 4.3 0.3 40.1 7.6 4. 72.14.219.158 0.0% 54 0.7 0.7 0.4 5.1 0.6 5. 108.170.247.208 0.0% 54 0.7 0.7 0.4 1.8 0.0 108.170.247.193 6. 108.170.234.25 0.0% 54 0.8 7.3 0.5 65.4 14.3 108.170.238.31 7. 72.14.239.160 22.6% 53 130.1 96.2 24.6 287.4 80.3 google-public-dns-a.google.com 8. 64.233.174.97 24.5% 53 239.5 97.8 24.6 299.4 89.7 google-public-dns-a.google.com 9. google-public-dns-a.google.com 52.8% 53 274.8 152.2 41.1 315.8 82.8
Pings from several locations worldwide: http://xor.meo.ws/MaZlLiwGq3a8pDCQZWbJzaGILoeNiU6Z.txt MTRs: http://xor.meo.ws/j3EOvCwf78JC0xTlzhB56rRNHOIfWjwm.txt
On 2017-08-23 11:30, Mr. NPP via Outages wrote:
In los angeles, experiencing packet loss, slow resolution, and high ping times. it appears google's any cast dns is not working properly.
--- 8.8.8.8 ping statistics --- 21 packets transmitted, 11 received, +1 errors, 47% packet loss, time 20229ms rtt min/avg/max/mdev = 44.635/151.432/258.331/76.230 ms
from multiple providers. i would check over any2, but it appears my link to any2 is down (hopefully a separate issue)
any confirm that they are seeing this issue?
Thanks
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
as someone said, icmp is an unreliable test because ping requests get dropped by busy servers or network devices. consider using something like this as a test of a resolver: /usr/bin/time -f '%e' dig +short +noanswer +time=5 cnn.com @8.8.8.8
just an fyi, i did have smokeping dig probes running, it also noticed some very bad latency. It does appear google had an issue last night. On Wed, Aug 23, 2017 at 1:01 PM, Paul M - outages via Outages < outages@outages.org> wrote:
as someone said, icmp is an unreliable test because ping requests get dropped by busy servers or network devices.
consider using something like this as a test of a resolver:
/usr/bin/time -f '%e' dig +short +noanswer +time=5 cnn.com @8.8.8.8 _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
We've been noticing A LOT of DNS amplification targeting Google IP space over the last few days. Seems to be mostly in the evening hours in the US. Any other network operators with access to flows seeing the same thing? We've been chasing down the offending customers with open resolvers, but its been very consistent. -M On Aug 23, 2017 15:05, "Mr. NPP via Outages" <outages@outages.org> wrote: just an fyi, i did have smokeping dig probes running, it also noticed some very bad latency. It does appear google had an issue last night. On Wed, Aug 23, 2017 at 1:01 PM, Paul M - outages via Outages < outages@outages.org> wrote:
as someone said, icmp is an unreliable test because ping requests get dropped by busy servers or network devices.
consider using something like this as a test of a resolver:
/usr/bin/time -f '%e' dig +short +noanswer +time=5 cnn.com @8.8.8.8 _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
On Wed, Aug 23, 2017 at 02:30:53AM -0700, Mr. NPP via Outages <outages@outages.org> wrote a message of 49 lines which said:
In los angeles,
It's more important to know the AS than the city, there are more problems ISP-related than location-related. According to 100 RIPE Atlas probes in the US, no obvious problems: % atlas-resolve -t AAAA -e 8.8.4.4 -c US -r 100 -i -m 9250172 www.bortzmeyer.org Nameserver 8.8.4.4 [2001:4b98:dc0:41:216:3eff:fece:1902 2605:4500:2:245b::42] : 99 occurrences Average RTT 52 ms [TIMEOUT(S)] : 1 occurrences Average RTT 0 ms [TIMEOUT] : 1 occurrences Average RTT 0 ms Test #9250172 done at 2017-08-23T09:35:51Z % atlas-resolve -t AAAA -e 8.8.8.8 -c US -r 100 -i -m 9250176 www.bortzmeyer.org Nameserver 8.8.8.8 [2001:4b98:dc0:41:216:3eff:fece:1902 2605:4500:2:245b::42] : 98 occurrences Average RTT 32 ms [TIMEOUT(S)] : 2 occurrences Average RTT 0 ms [TIMEOUT] : 2 occurrences Average RTT 0 ms Test #9250176 done at 2017-08-23T09:38:46Z
I monitor* the time for 8.8.8.8 to respond to DNS resolution requests and I've not seen any problems from the UK with the latency increasing. BTW, can I point people to the RIPE Atlas internet measurement project: https://atlas.ripe.net/ its very useful in situations like this, as well as monitoring/measuring the availability of your own services. * I use a munin plugin to test performance of various DNS services, even though I run caching resolvers. If anyone wants a copy, contact me off list.
Consistently at 1ms or less with no packet loss here in Seattle with over 50 packets sent. Ylan Muller IT TEAM MEMBER • GRAVITY PAYMENTS t: 206-388-5900 x:621 <206-388-5900,621> <https://www.linkedin.com/company/gravity-payments> On Wed, Aug 23, 2017 at 3:41 AM, Paul M - outages via Outages < outages@outages.org> wrote:
I monitor* the time for 8.8.8.8 to respond to DNS resolution requests and I've not seen any problems from the UK with the latency increasing.
BTW, can I point people to the RIPE Atlas internet measurement project: https://atlas.ripe.net/ its very useful in situations like this, as well as monitoring/measuring the availability of your own services.
* I use a munin plugin to test performance of various DNS services, even though I run caching resolvers. If anyone wants a copy, contact me off list. _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
participants (10)
-
cgm -
Geeknik Labs -
Katie Holly -
Marcel Herrguth -
Matt Erculiani -
Mr. NPP -
Paul M - outages -
Riley Childs -
Stephane Bortzmeyer -
Ylan Muller