GoDaddy DNS - Here We Go Again...

A few of our client domains just suddenly started failing to resolve. As it turns out, those domains are ones whose name service is handled via GoDaddy. There's a thread on Reddit that suggests that this is pretty widespread, but the exact scope isn't known yet, and GD has not announced the issue. http://www.reddit.com/r/sysadmin/comments/2fczdc/godaddy_premium_dns_down/ Dig queries to domains that are on PDNS01 or PDNS02 fail. There are probably others.

We have several sites with DNS at the GD Premium DNS service, all services appeared to have returned to normal about ten minutes ago. *Sean E. Smith* Director of Network Operations at Simpleview T 520.575.1151 x131 www.simpleviewinc.com On Wed, Sep 3, 2014 at 8:12 AM, Mike Phipps via Outages <outages@outages.org
wrote:
A few of our client domains just suddenly started failing to resolve. As it turns out, those domains are ones whose name service is handled via GoDaddy. There's a thread on Reddit that suggests that this is pretty widespread, but the exact scope isn't known yet, and GD has not announced the issue.
http://www.reddit.com/r/sysadmin/comments/2fczdc/godaddy_premium_dns_down/
Dig queries to domains that are on PDNS01 or PDNS02 fail. There are probably others.
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages

On Wed, Sep 3, 2014 at 12:12 PM, Mike Phipps via Outages < outages@outages.org> wrote:
A few of our client domains just suddenly started failing to resolve. As it turns out, those domains are ones whose name service is handled via GoDaddy. There's a thread on Reddit that suggests that this is pretty widespread, but the exact scope isn't known yet, and GD has not announced the issue.
http://www.reddit.com/r/sysadmin/comments/2fczdc/godaddy_premium_dns_down/
Dig queries to domains that are on PDNS01 or PDNS02 fail. There are probably others.
We are seeing queries to PDNS02 being refused depending on anycast location, but we can't pinpoint them exactly as hostname.bind queries are not answered. Some locations answer fine, some not, depending not only on country but also on transit provider. Rubens

More than just their 'pdns' premium DNS servers are down today. NS31.DOMAINCONTROL.COM and NS32.DOMAINCONTROL.COM ping but don't respond to A or MX requests regardless of where I source from: (From Savvis) cary@brn-web-ftp:~$ dig +trace a sunriseseniorliving.com +short NS m.root-servers.net. from server 204.70.127.127 in 2 ms. NS c.root-servers.net. from server 204.70.127.127 in 2 ms. NS g.root-servers.net. from server 204.70.127.127 in 2 ms. NS f.root-servers.net. from server 204.70.127.127 in 2 ms. NS j.root-servers.net. from server 204.70.127.127 in 2 ms. NS e.root-servers.net. from server 204.70.127.127 in 2 ms. NS b.root-servers.net. from server 204.70.127.127 in 2 ms. NS k.root-servers.net. from server 204.70.127.127 in 2 ms. NS d.root-servers.net. from server 204.70.127.127 in 2 ms. NS a.root-servers.net. from server 204.70.127.127 in 2 ms. NS l.root-servers.net. from server 204.70.127.127 in 2 ms. NS i.root-servers.net. from server 204.70.127.127 in 2 ms. NS h.root-servers.net. from server 204.70.127.127 in 2 ms. ;; connection timed out; no servers could be reached (AT&T) root@CaryBox:~# host -t mx sunriseseniorliving.com NS32.DOMAINCONTROL.COM ;; connection timed out; no servers could be reached As I typed this email 'NS31.DOMAINCONTROL.COM' came back up, but NS32 is still down. Their phone tech support says they can't find a problem. And their status page says "No Significant Issues": http://support.godaddy.com/system-alerts/ Totally inexcusable. - Cary On Wed, Sep 3, 2014 at 11:53 AM, Rubens Kuhl via Outages < outages@outages.org> wrote:
On Wed, Sep 3, 2014 at 12:12 PM, Mike Phipps via Outages < outages@outages.org> wrote:
A few of our client domains just suddenly started failing to resolve. As it turns out, those domains are ones whose name service is handled via GoDaddy. There's a thread on Reddit that suggests that this is pretty widespread, but the exact scope isn't known yet, and GD has not announced the issue.
http://www.reddit.com/r/sysadmin/comments/2fczdc/godaddy_premium_dns_down/
Dig queries to domains that are on PDNS01 or PDNS02 fail. There are probably others.
We are seeing queries to PDNS02 being refused depending on anycast location, but we can't pinpoint them exactly as hostname.bind queries are not answered. Some locations answer fine, some not, depending not only on country but also on transit provider.
Rubens
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages

I have been using dnsmadeeasy.com since their last outage 2 years ago and have not looked back. On Thu, Sep 4, 2014 at 12:18 PM, Cary Wiedemann via Outages < outages@outages.org> wrote:
More than just their 'pdns' premium DNS servers are down today. NS31.DOMAINCONTROL.COM and NS32.DOMAINCONTROL.COM ping but don't respond to A or MX requests regardless of where I source from:
(From Savvis) cary@brn-web-ftp:~$ dig +trace a sunriseseniorliving.com +short NS m.root-servers.net. from server 204.70.127.127 in 2 ms. NS c.root-servers.net. from server 204.70.127.127 in 2 ms. NS g.root-servers.net. from server 204.70.127.127 in 2 ms. NS f.root-servers.net. from server 204.70.127.127 in 2 ms. NS j.root-servers.net. from server 204.70.127.127 in 2 ms. NS e.root-servers.net. from server 204.70.127.127 in 2 ms. NS b.root-servers.net. from server 204.70.127.127 in 2 ms. NS k.root-servers.net. from server 204.70.127.127 in 2 ms. NS d.root-servers.net. from server 204.70.127.127 in 2 ms. NS a.root-servers.net. from server 204.70.127.127 in 2 ms. NS l.root-servers.net. from server 204.70.127.127 in 2 ms. NS i.root-servers.net. from server 204.70.127.127 in 2 ms. NS h.root-servers.net. from server 204.70.127.127 in 2 ms. ;; connection timed out; no servers could be reached
(AT&T) root@CaryBox:~# host -t mx sunriseseniorliving.com NS32.DOMAINCONTROL.COM ;; connection timed out; no servers could be reached
As I typed this email 'NS31.DOMAINCONTROL.COM' came back up, but NS32 is still down.
Their phone tech support says they can't find a problem. And their status page says "No Significant Issues": http://support.godaddy.com/system-alerts/
Totally inexcusable.
- Cary
On Wed, Sep 3, 2014 at 11:53 AM, Rubens Kuhl via Outages < outages@outages.org> wrote:
On Wed, Sep 3, 2014 at 12:12 PM, Mike Phipps via Outages < outages@outages.org> wrote:
A few of our client domains just suddenly started failing to resolve. As it turns out, those domains are ones whose name service is handled via GoDaddy. There's a thread on Reddit that suggests that this is pretty widespread, but the exact scope isn't known yet, and GD has not announced the issue.
http://www.reddit.com/r/sysadmin/comments/2fczdc/godaddy_premium_dns_down/
Dig queries to domains that are on PDNS01 or PDNS02 fail. There are probably others.
We are seeing queries to PDNS02 being refused depending on anycast location, but we can't pinpoint them exactly as hostname.bind queries are not answered. Some locations answer fine, some not, depending not only on country but also on transit provider.
Rubens
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages

tl;dr: Seeing the same with GoDaddy's ns51 and ns52 servers not working. On bottom is a successful lookup from the 8.8.8.8s (must be cached). anthony@theintersect:~$ dig +trace a rmmsolutions.com +short NS f.root-servers.net. from server 127.0.1.1 in 33 ms. NS k.root-servers.net. from server 127.0.1.1 in 33 ms. NS h.root-servers.net. from server 127.0.1.1 in 33 ms. NS l.root-servers.net. from server 127.0.1.1 in 33 ms. NS m.root-servers.net. from server 127.0.1.1 in 33 ms. NS b.root-servers.net. from server 127.0.1.1 in 33 ms. NS i.root-servers.net. from server 127.0.1.1 in 33 ms. NS a.root-servers.net. from server 127.0.1.1 in 33 ms. NS d.root-servers.net. from server 127.0.1.1 in 33 ms. NS e.root-servers.net. from server 127.0.1.1 in 33 ms. NS j.root-servers.net. from server 127.0.1.1 in 33 ms. NS g.root-servers.net. from server 127.0.1.1 in 33 ms. NS c.root-servers.net. from server 127.0.1.1 in 33 ms. RRSIG NS 8 0 518400 20140911000000 20140903230000 8230 . NTA7IrhowJqzdHiepsggdC18DJDlx1Ikw4+UNsP9Y4gRqfPWdjq7lxmd f38tRj0DdYjPThWpWRVVi6x0tVAifz3Zb23guHPq4qTn3Vrckr/N7+5N LNK5GsPX3yK2jWUf/pftqmP6zRPToZBuVFmX4Wno2Qp+a07iHvYOAaqP c1E= from server 127.0.1.1 in 33 ms. ;; connection timed out; no servers could be reached anthony@theintersect:~$ dig rmmsolutions.com ns ; <<>> DiG 9.9.5-3-Ubuntu <<>> rmmsolutions.com ns ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 47040 ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 4 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4000 ;; QUESTION SECTION: ;rmmsolutions.com. IN NS ;; ANSWER SECTION: rmmsolutions.com. 3600 IN NS ns52.domaincontrol.com. rmmsolutions.com. 3600 IN NS ns51.domaincontrol.com. ;; ADDITIONAL SECTION: ns51.domaincontrol.com. 7200 IN A 216.69.185.26 ns51.domaincontrol.com. 7200 IN AAAA 2607:f208:206::1a ns52.domaincontrol.com. 7200 IN A 208.109.255.26 ;; Query time: 97 msec ;; SERVER: 127.0.1.1#53(127.0.1.1) ;; WHEN: Thu Sep 04 12:14:13 CDT 2014 ;; MSG SIZE rcvd: 157 anthony@theintersect:~$ dig @ns51.domaincontrol.com rmmsolutions.com a ; <<>> DiG 9.9.5-3-Ubuntu <<>> @ns51.domaincontrol.com rmmsolutions.com a ; (2 servers found) ;; global options: +cmd ;; connection timed out; no servers could be reached anthony@theintersect:~$ dig @ns52.domaincontrol.com rmmsolutions.com a ; <<>> DiG 9.9.5-3-Ubuntu <<>> @ns52.domaincontrol.com rmmsolutions.com a ; (2 servers found) ;; global options: +cmd ;; connection timed out; no servers could be reached **************And now to test some other DNS provider (the 8s) anthony@theintersect:~$ dig @8.8.8.8 rmmsolutions.com a ; <<>> DiG 9.9.5-3-Ubuntu <<>> @8.8.8.8 rmmsolutions.com a ; (1 server found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 9980 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 512 ;; QUESTION SECTION: ;rmmsolutions.com. IN A ;; ANSWER SECTION: rmmsolutions.com. 3599 IN A 67.227.189.79 ;; Query time: 116 msec ;; SERVER: 8.8.8.8#53(8.8.8.8) ;; WHEN: Thu Sep 04 12:19:59 CDT 2014 ;; MSG SIZE rcvd: 61 On Thu, Sep 4, 2014 at 11:36 AM, Dovid B via Outages <outages@outages.org> wrote:
I have been using dnsmadeeasy.com since their last outage 2 years ago and have not looked back.
On Thu, Sep 4, 2014 at 12:18 PM, Cary Wiedemann via Outages < outages@outages.org> wrote:
More than just their 'pdns' premium DNS servers are down today. NS31.DOMAINCONTROL.COM and NS32.DOMAINCONTROL.COM ping but don't respond to A or MX requests regardless of where I source from:
(From Savvis) cary@brn-web-ftp:~$ dig +trace a sunriseseniorliving.com +short NS m.root-servers.net. from server 204.70.127.127 in 2 ms. NS c.root-servers.net. from server 204.70.127.127 in 2 ms. NS g.root-servers.net. from server 204.70.127.127 in 2 ms. NS f.root-servers.net. from server 204.70.127.127 in 2 ms. NS j.root-servers.net. from server 204.70.127.127 in 2 ms. NS e.root-servers.net. from server 204.70.127.127 in 2 ms. NS b.root-servers.net. from server 204.70.127.127 in 2 ms. NS k.root-servers.net. from server 204.70.127.127 in 2 ms. NS d.root-servers.net. from server 204.70.127.127 in 2 ms. NS a.root-servers.net. from server 204.70.127.127 in 2 ms. NS l.root-servers.net. from server 204.70.127.127 in 2 ms. NS i.root-servers.net. from server 204.70.127.127 in 2 ms. NS h.root-servers.net. from server 204.70.127.127 in 2 ms. ;; connection timed out; no servers could be reached
(AT&T) root@CaryBox:~# host -t mx sunriseseniorliving.com NS32.DOMAINCONTROL.COM ;; connection timed out; no servers could be reached
As I typed this email 'NS31.DOMAINCONTROL.COM' came back up, but NS32 is still down.
Their phone tech support says they can't find a problem. And their status page says "No Significant Issues": http://support.godaddy.com/system-alerts/
Totally inexcusable.
- Cary
On Wed, Sep 3, 2014 at 11:53 AM, Rubens Kuhl via Outages < outages@outages.org> wrote:
On Wed, Sep 3, 2014 at 12:12 PM, Mike Phipps via Outages < outages@outages.org> wrote:
A few of our client domains just suddenly started failing to resolve. As it turns out, those domains are ones whose name service is handled via GoDaddy. There's a thread on Reddit that suggests that this is pretty widespread, but the exact scope isn't known yet, and GD has not announced the issue.
http://www.reddit.com/r/sysadmin/comments/2fczdc/godaddy_premium_dns_down/
Dig queries to domains that are on PDNS01 or PDNS02 fail. There are probably others.
We are seeing queries to PDNS02 being refused depending on anycast location, but we can't pinpoint them exactly as hostname.bind queries are not answered. Some locations answer fine, some not, depending not only on country but also on transit provider.
Rubens
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
-- - Anthony Hook
participants (6)
-
Anthony Hook
-
Cary Wiedemann
-
Dovid B
-
Mike Phipps
-
Rubens Kuhl
-
Sean Smith