Google and Level 3 public DNS

8.8.8.8 and 4.2.2.2 are returning SERVFAIL from our downtown Los Angeles datacenter about 80% of the time at the moment. Other public DNS servers seem to be happily returning what they need to. Anyone else seeing this?

Do you have a traceroute you can share? Where are you coming from ? etc. --John On Wed, May 30, 2018 at 11:11 AM, James Sharp via Outages < outages@outages.org> wrote:
8.8.8.8 and 4.2.2.2 are returning SERVFAIL from our downtown Los Angeles datacenter about 80% of the time at the moment. Other public DNS servers seem to be happily returning what they need to. Anyone else seeing this?
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages

I’ve been seeing issues the last hour or so from San Francisco (up at the moment). Connectivity to 8.8.8.8 noticeably different to 4.4.4.4: To 4.4.4.4: My traceroute [vUNKNOWN] https.local (10.33.106.41) 2018-05-30T10:39:41-0700 Keys: Help Display mode Restart statistics Order of fields quit Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. 10.33.104.1 0.0% 8 1.4 172.9 1.2 1190. 413.1 2. 157.130.196.213 0.0% 8 99.2 25.2 2.0 99.2 36.4 3. 0.ae5.br1.lax15.alter.net 0.0% 8 180.2 34.7 11.7 180.2 58.9 4. 89.221.35.190 0.0% 8 132.4 57.9 12.0 170.6 65.7 5. 195.22.221.201 0.0% 8 252.5 226.6 172.2 365.6 80.4 6. 195.22.221.231 0.0% 8 202.7 206.9 168.7 316.1 58.9 7. ??? 8. ??? 9. xe-5-2-0-102.bar2.minneapolis2.level3.net 0.0% 8 298.1 264.2 207.5 496.6 107.6 10. ??? To 8.8.8.8 (8.8.4.4 is similar): My traceroute [vUNKNOWN] https.local (10.33.106.41) 2018-05-30T10:41:26-0700 Keys: Help Display mode Restart statistics Order of fields quit Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. 10.33.104.1 0.0% 26 66.3 237.0 1.0 5793. 1133. 2. 157.130.196.213 0.0% 26 12.1 11.2 2.0 84.6 20.3 3. bundle-ether4.gw6.sjc7.alter.net 0.0% 26 44.3 15.6 4.6 130.5 25.1 4. google-gw.customer.alter.net 0.0% 26 154.0 16.0 4.6 154.0 32.4 5. 108.170.242.237 0.0% 26 101.1 20.6 5.8 146.9 35.9 6. 74.125.253.151 0.0% 26 48.8 17.8 5.3 108.3 27.5 7. 209.85.252.206 0.0% 26 26.4 44.3 25.5 147.4 25.7 8. 209.85.250.172 0.0% 26 98.3 33.4 24.6 98.3 19.2 9. 209.85.142.115 0.0% 26 68.5 40.0 26.0 174.5 36.3 10. ??? 11. ??? 12. ??? 13. ??? 14. ??? 15. ??? 16. ??? 17. ??? 18. ??? 19. ??? 20. google-public-dns-a.google.com 0.0% 20 140.0 50.5 25.1 162.9 49.4
On May 30, 2018, at 10:11 AM, James Sharp via Outages <outages@outages.org> wrote:
8.8.8.8 and 4.2.2.2 are returning SERVFAIL from our downtown Los Angeles datacenter about 80% of the time at the moment. Other public DNS servers seem to be happily returning what they need to. Anyone else seeing this?
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages

The traceroute to 4.4.4.4 is NOT a DNS server. You might be confusing it with 8.8.4.4 and 4.2.2.1 - 4.2.2.6.

I've checked my measurements of Google dns responsiveness on 8.8.8.8 and not seen anything unusual of late. Those measurements made in uk from AS13037 Btw, I find that 8.8.4.4 gives more consistent performance, less likely to have occasional big spikes in latency. On Wed, 30 May 2018, 18:11 James Sharp via Outages, <outages@outages.org> wrote:
8.8.8.8 and 4.2.2.2 are returning SERVFAIL

On Wed, May 30, 2018 at 1:12 PM James Sharp via Outages <outages@outages.org> wrote:
8.8.8.8 and 4.2.2.2 are returning SERVFAIL from our downtown Los Angeles datacenter about 80% of the time at the moment. Other public DNS servers seem to be happily returning what they need to. Anyone else seeing this?
When you say "are returning SERVFAIL", for what queries? W
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
-- I don't think the execution is relevant when it was obviously a bad idea in the first place. This is like putting rabid weasels in your pants, and later expressing regret at having chosen those particular rabid weasels and that pair of pants. ---maf

On May 30, 2018, at 11:10 AM, Warren Kumari <warren@kumari.net> wrote:
On Wed, May 30, 2018 at 1:12 PM James Sharp via Outages <outages@outages.org> wrote:
8.8.8.8 and 4.2.2.2 are returning SERVFAIL from our downtown Los Angeles datacenter about 80% of the time at the moment. Other public DNS servers seem to be happily returning what they need to. Anyone else seeing this?
When you say "are returning SERVFAIL", for what queries? W
The majority of the queries we were sending across many different domains.

On Wed, May 30, 2018 at 11:25 AM James Sharp via Outages < outages@outages.org> wrote:
On May 30, 2018, at 11:10 AM, Warren Kumari <warren@kumari.net> wrote: On Wed, May 30, 2018 at 1:12 PM James Sharp via Outages < outages@outages.org> wrote:
8.8.8.8 and 4.2.2.2 are returning SERVFAIL from our downtown Los Angeles datacenter about 80% of the time at the moment. Other public DNS servers seem to be happily returning what they need to. Anyone else seeing this?
When you say "are returning SERVFAIL", for what queries?
The majority of the queries we were sending across many different domains.
Is this an ongoing issue for you, still affecting either 8.8.8.8 or 4.2.2.2? Damian

On May 30, 2018, at 12:26 PM, Damian Menscher <damian@google.com> wrote:
On Wed, May 30, 2018 at 11:25 AM James Sharp via Outages <outages@outages.org <mailto:outages@outages.org>> wrote:
On May 30, 2018, at 11:10 AM, Warren Kumari <warren@kumari.net <mailto:warren@kumari.net>> wrote: On Wed, May 30, 2018 at 1:12 PM James Sharp via Outages <outages@outages.org <mailto:outages@outages.org>> wrote:
8.8.8.8 and 4.2.2.2 are returning SERVFAIL from our downtown Los Angeles datacenter about 80% of the time at the moment. Other public DNS servers seem to be happily returning what they need to. Anyone else seeing this?
When you say "are returning SERVFAIL", for what queries?
The majority of the queries we were sending across many different domains.
Is this an ongoing issue for you, still affecting either 8.8.8.8 or 4.2.2.2?
Damian
no. They both seem to have settled down and started behaving again. Looks like I’m going to have to add a couple more completely diverse servers to the pool. Hurricane Electric’s public server seemed to be doing really well this morning.
participants (7)
-
Damian Menscher
-
James Sharp
-
Jeff Neilson - Micro-Ram Computer Systems, Inc.
-
John Kinsella
-
John Schiel
-
Paul M - outages
-
Warren Kumari