ultradns issues? unable to resolve amazon.com

Seeing problems trying to reach amazon.com. Looks like the ultradns name servers aren't responding? at least from here, and other external dns servers I query can't seem to resolve amazon either. Friend of mine still has some of their DNS in cache, and hardcoding dns in my hosts file allows me to hit Amazon without any issue. I was surfing fine until my DNS flushed and couldn't re-resolve. Anyone else seeing anything fun? Looks like they might be coming back online - my trace is actually making it to them now, sort of. - DL Host Loss% Snt Last Avg Best Wrst StDev 1. 192.168.0.1 0.0% 40 0.5 0.5 0.4 0.6 0.0 2. *censored* 0.0% 40 7.4 8.7 6.6 26.0 3.7 3. *censored* 0.0% 40 5.3 5.8 4.6 9.8 1.0 4. ge-6-11-440.car2.Seattle1.Level3.net 0.0% 39 20.0 31.0 5.6 209.0 57.9 5. ae-32-52.ebr2.Seattle1.Level3.net 0.0% 39 7.3 12.0 5.1 35.3 6.1 6. ae-1-100.ebr1.Seattle1.Level3.net 2.6% 39 6.4 12.7 5.8 27.6 5.4 7. ae-7.ebr2.SanJose1.Level3.net 5.1% 39 27.3 29.0 22.4 54.4 5.9 8. ae-72-72.csw2.SanJose1.Level3.net 0.0% 39 33.1 30.1 23.0 80.4 9.2 9. ae-2-79.edge1.SanJose3.Level3.net 12.8% 39 24.1 29.9 22.8 92.1 17.0 10. 4.68.111.162 0.0% 39 24.5 47.8 23.5 431.2 80.4 11. te3-1-10G.ar1.PAO2.gblx.net 0.0% 39 26.3 30.2 24.9 148.0 20.8 12. 64.209.106.2 84.2% 39 40.5 31.5 27.6 40.5 4.5 13. ??? 14. 204.69.234.1 97.4% 39 52.2 52.2 52.2 52.2 0.0

On Wed, Dec 23, 2009 at 5:08 PM, dl <kokushibyou@gmail.com> wrote:
Seeing problems trying to reach amazon.com. Looks like the ultradns name servers aren't responding? at least from here, and other external dns servers I query can't seem to resolve amazon either.
Friend of mine still has some of their DNS in cache, and hardcoding dns in my hosts file allows me to hit Amazon without any issue. I was surfing fine until my DNS flushed and couldn't re-resolve.
Anyone else seeing anything fun?
Looks like they might be coming back online - my trace is actually making it to them now, sort of.
I can confirm there's some sort of outage going on - my current client site is not publicly resolvable at the moment. -- -george william herbert george.herbert@gmail.com

It looks like this is a West coast issue - resolving via DNS servers on the East coast does in fact return DNS for Amazon. Sounds like theres some murmur on nanog about this too. ; <<>> DiG 9.6.0-APPLE-P2 <<>> amazon.com @ns1.nyc.pnap.net ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 20214 ;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 6, ADDITIONAL: 8 ;; QUESTION SECTION: ;amazon.com. IN A ;; ANSWER SECTION: amazon.com. 60 IN A 72.21.210.250 amazon.com. 60 IN A 72.21.207.65 amazon.com. 60 IN A 207.171.166.252 - DL On Wed, Dec 23, 2009 at 17:08, dl <kokushibyou@gmail.com> wrote:
Seeing problems trying to reach amazon.com. Looks like the ultradns name servers aren't responding? at least from here, and other external dns servers I query can't seem to resolve amazon either.
Friend of mine still has some of their DNS in cache, and hardcoding dns in my hosts file allows me to hit Amazon without any issue. I was surfing fine until my DNS flushed and couldn't re-resolve.
Anyone else seeing anything fun?
Looks like they might be coming back online - my trace is actually making it to them now, sort of.
- DL
Host Loss% Snt Last Avg Best Wrst StDev 1. 192.168.0.1 0.0% 40 0.5 0.5 0.4 0.6 0.0 2. *censored* 0.0% 40 7.4 8.7 6.6 26.0 3.7 3. *censored* 0.0% 40 5.3 5.8 4.6 9.8 1.0 4. ge-6-11-440.car2.Seattle1.Level3.net 0.0% 39 20.0 31.0 5.6 209.0 57.9 5. ae-32-52.ebr2.Seattle1.Level3.net 0.0% 39 7.3 12.0 5.1 35.3 6.1 6. ae-1-100.ebr1.Seattle1.Level3.net 2.6% 39 6.4 12.7 5.8 27.6 5.4 7. ae-7.ebr2.SanJose1.Level3.net 5.1% 39 27.3 29.0 22.4 54.4 5.9 8. ae-72-72.csw2.SanJose1.Level3.net 0.0% 39 33.1 30.1 23.0 80.4 9.2 9. ae-2-79.edge1.SanJose3.Level3.net 12.8% 39 24.1 29.9 22.8 92.1 17.0 10. 4.68.111.162 0.0% 39 24.5 47.8 23.5 431.2 80.4 11. te3-1-10G.ar1.PAO2.gblx.net 0.0% 39 26.3 30.2 24.9 148.0 20.8 12. 64.209.106.2 84.2% 39 40.5 31.5 27.6 40.5 4.5 13. ??? 14. 204.69.234.1 97.4% 39 52.2 52.2 52.2 52.2 0.0

dl wrote:
It looks like this is a West coast issue - resolving via DNS servers on the East coast does in fact return DNS for Amazon.
See no problem from here in Denver, CO.... [cstone@csmdv ~]$ dig a amazon.com +trace +short NS K.ROOT-SERVERS.NET. from server 127.0.0.1 in 21 ms. NS I.ROOT-SERVERS.NET. from server 127.0.0.1 in 21 ms. NS A.ROOT-SERVERS.NET. from server 127.0.0.1 in 21 ms. NS L.ROOT-SERVERS.NET. from server 127.0.0.1 in 21 ms. NS F.ROOT-SERVERS.NET. from server 127.0.0.1 in 21 ms. NS D.ROOT-SERVERS.NET. from server 127.0.0.1 in 21 ms. NS H.ROOT-SERVERS.NET. from server 127.0.0.1 in 21 ms. NS E.ROOT-SERVERS.NET. from server 127.0.0.1 in 21 ms. NS C.ROOT-SERVERS.NET. from server 127.0.0.1 in 21 ms. NS J.ROOT-SERVERS.NET. from server 127.0.0.1 in 21 ms. NS B.ROOT-SERVERS.NET. from server 127.0.0.1 in 21 ms. NS M.ROOT-SERVERS.NET. from server 127.0.0.1 in 21 ms. NS G.ROOT-SERVERS.NET. from server 127.0.0.1 in 21 ms. A 72.21.210.250 from server udns2.ultradns.net in 115 ms. A 72.21.207.65 from server udns2.ultradns.net in 115 ms. A 207.171.166.252 from server udns2.ultradns.net in 115 ms. Chris

UltraDNS is having a DDOS attack at its San Jose and Palo Alto locations. No ETA yet ________________________________ From: dl <kokushibyou@gmail.com> To: outages@outages.org Sent: Wed, December 23, 2009 5:24:13 PM Subject: Re: [outages] ultradns issues? unable to resolve amazon.com It looks like this is a West coast issue - resolving via DNS servers on the East coast does in fact return DNS for Amazon. Sounds like theres some murmur on nanog about this too. ; <<>> DiG 9.6.0-APPLE-P2 <<>> amazon.com @ns1.nyc.pnap.net ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 20214 ;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 6, ADDITIONAL: 8 ;; QUESTION SECTION: ;amazon.com. IN A ;; ANSWER SECTION: amazon.com. 60 IN A 72.21.210.250 amazon.com. 60 IN A 72.21.207.65 amazon.com. 60 IN A 207.171.166.252 - DL On Wed, Dec 23, 2009 at 17:08, dl <kokushibyou@gmail.com> wrote:
Seeing problems trying to reach amazon.com. Looks like the ultradns name servers aren't responding? at least from here, and other external dns servers I query can't seem to resolve amazon either.
Friend of mine still has some of their DNS in cache, and hardcoding dns in my hosts file allows me to hit Amazon without any issue. I was surfing fine until my DNS flushed and couldn't re-resolve.
Anyone else seeing anything fun?
Looks like they might be coming back online - my trace is actually making it to them now, sort of.
- DL
Host Loss% Snt Last Avg Best Wrst StDev 1. 192.168.0.1 0.0% 40 0.5 0.5 0.4 0.6 0.0 2. *censored* 0.0% 40 7.4 8.7 6.6 26.0 3.7 3. *censored* 0.0% 40 5.3 5.8 4.6 9.8 1.0 4. ge-6-11-440.car2.Seattle1.Level3.net 0.0% 39 20.0 31.0 5.6 209.0 57.9 5. ae-32-52.ebr2.Seattle1.Level3.net 0.0% 39 7.3 12.0 5.1 35.3 6.1 6. ae-1-100.ebr1.Seattle1.Level3.net 2.6% 39 6.4 12.7 5.8 27.6 5.4 7. ae-7.ebr2.SanJose1.Level3.net 5.1% 39 27.3 29.0 22.4 54.4 5.9 8. ae-72-72.csw2.SanJose1.Level3.net 0.0% 39 33.1 30.1 23.0 80.4 9.2 9. ae-2-79.edge1.SanJose3.Level3.net 12.8% 39 24.1 29.9 22.8 92.1 17.0 10. 4.68.111.162 0.0% 39 24.5 47.8 23.5 431.2 80.4 11. te3-1-10G.ar1.PAO2.gblx.net 0.0% 39 26.3 30.2 24.9 148.0 20.8 12. 64.209.106.2 84.2% 39 40.5 31.5 27.6 40.5 4.5 13. ??? 14. 204.69.234.1 97.4% 39 52.2 52.2 52.2 52.2 0.0
_______________________________________________ outages mailing list outages@outages.org https://puck.nether.net/mailman/listinfo/outages

So I guess this means they can't claim they don't have outages? -----Original Message----- From: Monika Bajwa <mbajwa75@yahoo.com> Date: Wed, 23 Dec 2009 17:33:32 To: <outages@outages.org> Subject: Re: [outages] ultradns issues? unable to resolve amazon.com _______________________________________________ outages mailing list outages@outages.org https://puck.nether.net/mailman/listinfo/outages

On Dec 23, 2009, at 5:41 PM, mhernand1@comcast.net wrote:
So I guess this means they can't claim they don't have outages?
Actually, no. Given some folks indicated they weren't having issues, I suspect their use of anycast did what it was supposed to do in the face of a massive D(D)oS -- limit the damage to the network topologically nearest transmitting nodes. Of course, I have no real details... Regards, -drc

David Conrad wrote:
On Dec 23, 2009, at 5:41 PM, mhernand1@comcast.net wrote:
So I guess this means they can't claim they don't have outages?
Actually, no. Given some folks indicated they weren't having issues, I suspect their use of anycast did what it was supposed to do in the face of a massive D(D)oS -- limit the damage to the network topologically nearest transmitting nodes. Of course, I have no real details...
Meh, technicality. It acts like an outage to the geographic area affected, and it should be considered as such. ~Seth

On Dec 23, 2009, at 7:52 PM, Seth Mattinen wrote:
David Conrad wrote:
On Dec 23, 2009, at 5:41 PM, mhernand1@comcast.net wrote:
So I guess this means they can't claim they don't have outages? Actually, no. Given some folks indicated they weren't having issues, I suspect their use of anycast did what it was supposed to do in the face of a massive D(D)oS -- limit the damage to the network topologically nearest transmitting nodes. Of course, I have no real details...
Meh, technicality. It acts like an outage to the geographic area affected, and it should be considered as such.
Perhaps, however I'm sure their marketing folks will be happy to use the technicality to justify their 100% uptime claims (like every other anycast DNS provider)... Regards, -drc

On Wed, Dec 23, 2009 at 8:14 PM, David Conrad <drc@virtualized.org> wrote:
On Dec 23, 2009, at 7:52 PM, Seth Mattinen wrote:
Meh, technicality. It acts like an outage to the geographic area affected, and it should be considered as such.
Perhaps, however I'm sure their marketing folks will be happy to use the technicality to justify their 100% uptime claims (like every other anycast DNS provider)...
If I have to explain the event in person to a client CEO whose site was down - and I did - then it's an outage. If we're wonking about what happened here - which we are - it's a failure. Not all failures are outages. But this was an outage. One cannot ignore the reality of marketing-speak to come. But we don't have to indulge it on operational lists. -- -george william herbert george.herbert@gmail.com

When I tried to go to amazon.com about 30 mins ago (probably a few minutes before your mail) I got an error. A (possibly undeserved) curse at Comcast and a few reloads later and it started working again. However I'm now seeing : Non-authoritative answer: www.amazon.com nameserver = ns-911.amazon.com ns-911.amazon.com internet address = 207.171.178.13 Which may imply something unusual is going on... Scott On Wed, Dec 23, 2009 at 5:08 PM, dl <kokushibyou@gmail.com> wrote:
Seeing problems trying to reach amazon.com. Looks like the ultradns name servers aren't responding? at least from here, and other external dns servers I query can't seem to resolve amazon either.
Friend of mine still has some of their DNS in cache, and hardcoding dns in my hosts file allows me to hit Amazon without any issue. I was surfing fine until my DNS flushed and couldn't re-resolve.
Anyone else seeing anything fun?
Looks like they might be coming back online - my trace is actually making it to them now, sort of.
- DL
Host Loss% Snt Last Avg Best Wrst StDev 1. 192.168.0.1 0.0% 40 0.5 0.5 0.4 0.6 0.0 2. *censored* 0.0% 40 7.4 8.7 6.6 26.0 3.7 3. *censored* 0.0% 40 5.3 5.8 4.6 9.8 1.0 4. ge-6-11-440.car2.Seattle1.Level3.net 0.0% 39 20.0 31.0 5.6 209.0 57.9 5. ae-32-52.ebr2.Seattle1.Level3.net 0.0% 39 7.3 12.0 5.1 35.3 6.1 6. ae-1-100.ebr1.Seattle1.Level3.net 2.6% 39 6.4 12.7 5.8 27.6 5.4 7. ae-7.ebr2.SanJose1.Level3.net 5.1% 39 27.3 29.0 22.4 54.4 5.9 8. ae-72-72.csw2.SanJose1.Level3.net 0.0% 39 33.1 30.1 23.0 80.4 9.2 9. ae-2-79.edge1.SanJose3.Level3.net 12.8% 39 24.1 29.9 22.8 92.1 17.0 10. 4.68.111.162 0.0% 39 24.5 47.8 23.5 431.2 80.4 11. te3-1-10G.ar1.PAO2.gblx.net 0.0% 39 26.3 30.2 24.9 148.0 20.8 12. 64.209.106.2 84.2% 39 40.5 31.5 27.6 40.5 4.5 13. ??? 14. 204.69.234.1 97.4% 39 52.2 52.2 52.2 52.2 0.0 _______________________________________________ outages mailing list outages@outages.org https://puck.nether.net/mailman/listinfo/outages
participants (8)
-
Chris Stone
-
David Conrad
-
dl
-
George Herbert
-
mhernand1@comcast.net
-
Monika Bajwa
-
Scott Howard
-
Seth Mattinen