Google Public DNS: major outage in Southern Europe (TLD: .us .biz)

Dear list, Google's public DNS does not resolve .us and .biz TLDs (not even the NS records of it) in many european countries. Reports are from at least Switzerland, Austria, Italy and Slovenia, likely Germany as well. This is from the Public DNS instance in Milan, Italy, .us and .biz TLD are dead: C:\Users\lukas>nslookup -type=NS us. 8.8.8.8 Server: dns.google Address: 8.8.8.8 DNS request timed out. timeout was 2 seconds. *** Zeitüberschreitung bei Anforderung an dns.google. C:\Users\lukas> https://dns.google/ shows (screenshot attached also): { "AD": false, "CD": true, "Comment": "Name servers did not respond [2001:500:3682::11, 156.154.126.70, 156.154.128.70, 2001:503:e239::3:1, 156.154.125.70, 156.154.124.70, 209.173.58.70, 2001:502:ad09::29, 156.154.127.70].", "Question": [ { "name": "us.", "type": 2 } ], "RA": true, "RD": true, "Status": 2, "TC": false } Lukas

Works fine from Germany right now. AS3209 Mit freundlichen Grüßen Julian Panke \-------- Original-Nachricht -------- Am 29. Mai 2021, 12:12, Lukas Tribus via Outages schrieb:
Dear list,
Google's public DNS does not resolve .us and .biz TLDs (not even the NS records of it) in many european countries. Reports are from at least Switzerland, Austria, Italy and Slovenia, likely Germany as well.
This is from the Public DNS instance in Milan, Italy, .us and .biz TLD are dead:
C:\\Users\\lukas>nslookup -type=NS us. [8.8.8.8][] Server: [dns.google][] Address: [8.8.8.8][]
DNS request timed out. timeout was 2 seconds. \*\*\* Zeitüberschreitung bei Anforderung an [dns.google][].
C:\\Users\\lukas>
https://dns.google/ shows (screenshot attached also):
\{ "AD": false, "CD": true, "Comment": "Name servers did not respond \[2001:500:3682::11, [156.154.126.70][], [156.154.128.70][], 2001:503:e239::3:1, [156.154.125.70][], [156.154.124.70][], [209.173.58.70][], 2001:502:ad09::29, [156.154.127.70][]\].", "Question": \[ \{ "name": "us.", "type": 2 \} \], "RA": true, "RD": true, "Status": 2, "TC": false \}
Lukas \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
[8.8.8.8]: http://8.8.8.8 [dns.google]: http://dns.google [156.154.126.70]: http://156.154.126.70 [156.154.128.70]: http://156.154.128.70 [156.154.125.70]: http://156.154.125.70 [156.154.124.70]: http://156.154.124.70 [209.173.58.70]: http://209.173.58.70 [156.154.127.70]: http://156.154.127.70

Hi everyone, Google DNS doesn't even resolve it's own .us domain: { "AD": false, "CD": false, "Comment": "Name servers did not respond [2001:500:3682::11, 156.154.124.70, 2001:503:e239::3:1, 156.154.127.70, 156.154.125.70, 156.154.126.70, 209.173.58.70, 156.154.128.70, 2001:502:ad09::29].", "Question": [ { "name": "google.us.", "type": 1 } ], "RA": true, "RD": true, "Status": 2, "TC": false } tested from Switzerland AS1836 Am 29/05/2021 um 12:30 schrieb Julian Panke via Outages:
Works fine from Germany right now. AS3209
Mit freundlichen Grüßen
Julian Panke
-------- Original-Nachricht -------- Am 29. Mai 2021, 12:12, Lukas Tribus via Outages schrieb:
Dear list,
Google's public DNS does not resolve .us and .biz TLDs (not even the NS records of it) in many european countries. Reports are from at least Switzerland, Austria, Italy and Slovenia, likely Germany as well.
This is from the Public DNS instance in Milan, Italy, .us and .biz TLD are dead:
C:\Users\lukas>nslookup -type=NS us. 8.8.8.8 <http://8.8.8.8> Server: dns.google <http://dns.google> Address: 8.8.8.8 <http://8.8.8.8>
DNS request timed out. timeout was 2 seconds. *** Zeitüberschreitung bei Anforderung an dns.google <http://dns.google>.
C:\Users\lukas>
https://dns.google/ shows (screenshot attached also):
{ "AD": false, "CD": true, "Comment": "Name servers did not respond [2001:500:3682::11, 156.154.126.70 <http://156.154.126.70>, 156.154.128.70 <http://156.154.128.70>, 2001:503:e239::3:1, 156.154.125.70 <http://156.154.125.70>, 156.154.124.70 <http://156.154.124.70>, 209.173.58.70 <http://209.173.58.70>, 2001:502:ad09::29, 156.154.127.70 <http://156.154.127.70>].", "Question": [ { "name": "us.", "type": 2 } ], "RA": true, "RD": true, "Status": 2, "TC": false }
Lukas _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages

SERVFAIL resolving google.us at 8.8.4.4 from different vantage points in Italy, Austria and Switzerland, (900 seconds intervall): https://atlas.ripe.net/measurements/30456676/#probes

Probes in India, New Zealand, US... receive SERVFAIL as well. https://atlas.ripe.net/measurements/30457254/#probes Mit freundlichen Grüßen Julian Panke ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Saturday, May 29th, 2021 at 1:18 PM, Lukas Tribus via Outages <outages@outages.org> wrote:
SERVFAIL resolving google.us at 8.8.4.4 from different vantage points
in Italy, Austria and Switzerland, (900 seconds intervall):
Outages mailing list
Outages@outages.org

Possibly related, I’ve been noticing some DNS resolution issues on Google Cloud beginning couple of days back. I can easily replicate launching an “apt update” from my Cloud Shell: Err:1 https://apt.releases.hashicorp.com <https://apt.releases.hashicorp.com> buster InRelease Temporary failure resolving 'apt.releases.hashicorp.com <http://apt.releases.hashicorp.com> ' [...] Err:8 http://repo.mysql.com/apt/debian <http://repo.mysql.com/apt/debian> buster InRelease Temporary failure resolving ‘repo.mysql.com <http://repo.mysql.com> ' This is inconsistent though, dig for the two domains above mostly works. >From Cloud Shell I can also resolve ‘google.us <http://google.us> ’ on 156.154.128.70 which is one of the authoritative NS failing in Lukas’ query on dns.google <http://dns.google> . There is indeed something weird going on in Google’s network/DNS but I don’t think we can fully track it down from the outside. On 29 May 2021, at 13:18, Lukas Tribus via Outages <outages@outages.org <mailto:outages@outages.org> > wrote: SERVFAIL resolving google.us <http://google.us> at 8.8.4.4 from different vantage points in Italy, Austria and Switzerland, (900 seconds intervall): https://atlas.ripe.net/measurements/30456676/#probes <https://atlas.ripe.net/measurements/30456676/#probes> _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages

On 5/29/21 12:12 PM, Lukas Tribus via Outages wrote:
Google's public DNS does not resolve .us and .biz TLDs (not even the NS records of it) in many european countries. Reports are from at least Switzerland, Austria, Italy and Slovenia, likely Germany as well.
hi, just made a comprehensive probe on the entire TLD list: it turns out that SERVFAIL happens on .bank .biz .insurance .us when 8.8.8.8 is queried for RR NS -- antonio

On Sat, May 29, 2021 at 9:26 AM Antonio Prado via Outages <outages@outages.org> wrote:
On 5/29/21 12:12 PM, Lukas Tribus via Outages wrote:
Google's public DNS does not resolve .us and .biz TLDs (not even the NS records of it) in many european countries. Reports are from at least Switzerland, Austria, Italy and Slovenia, likely Germany as well.
hi,
just made a comprehensive probe on the entire TLD list: it turns out that SERVFAIL happens on
.bank .biz .insurance .us
All of these are on Neustar's UltraDNS I believe, but I don't think they are the only TLDs there. Rubens

On Sat, May 29, 2021 at 09:29:22AM -0300, Rubens Kuhl via Outages <outages@outages.org> wrote a message of 28 lines which said:
All of these are on Neustar's UltraDNS I believe,
So, may be a routing issue between Google and UltraDNS, making all the authoritative name servers unreachable?

All of these are on Neustar's UltraDNS I believe, So, may be a routing issue between Google and UltraDNS, making all the authoritative name servers unreachable?
that is not supposed to happen; 2182 randy

Easily reproducible on a Google Cloud VM in the location Zurich, Switzerland: user@instance-1:~$ host google.us Host google.us not found: 2(SERVFAIL) user@instance-1:~$ host google.us 8.8.8.8 Using domain server: Name: 8.8.8.8 Address: 8.8.8.8#53 Aliases: Host google.us not found: 2(SERVFAIL) user@instance-1:~$ host google.us 8.8.4.4 Using domain server: Name: 8.8.4.4 Address: 8.8.4.4#53 Aliases: Host google.us not found: 2(SERVFAIL) user@instance-1:~$ user@instance-1:~$ host google.us 1.1.1.1 Using domain server: Name: 1.1.1.1 Address: 1.1.1.1#53 Aliases: google.us has address 142.250.184.36 google.us has IPv6 address 2a00:1450:4002:404::2004 google.us mail is handled by 0 . user@instance-1:~$ mtr 8.8.8.8 My traceroute [v0.93] instance-1 (10.172.0.2) 2021-05-29T15:34:37+0000 Keys: Help Display mode Restart statistics Order of fields quit Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. (waiting for reply) 2. 142.250.63.49 0.0% 5 0.5 0.8 0.5 1.6 0.5 3. 142.250.37.132 0.0% 5 1.5 1.5 1.5 1.6 0.1 4. 74.125.243.129 0.0% 5 3.1 2.8 2.5 3.1 0.2 5. 172.253.50.3 0.0% 5 2.1 2.3 1.9 2.5 0.3 6. dns.google 0.0% 5 1.3 1.4 1.2 1.6 0.2 This is now a multi-day outage affecting multiple TLD's in multiple countries. I'm unable to open a serious support case without spending money, and I filed a ticket with Google NOC about 5 hours ago. Lukas

This was resolved sometime between 18:10 and 18:30 UTC yesterday. I'll add that the RIPE Atlas probe still shows spurious SERVFAILs, so this needs a separate investigation (ripe-atlas mailing-list). Lukas
participants (8)
-
Antonio Prado
-
Giorgio Bonfiglio
-
Julian Panke
-
Lukas Tribus
-
Randy Bush
-
Rubens Kuhl
-
Seraphim Strub
-
Stephane Bortzmeyer