Here is a more detailed analysis of what happened: https://arstechnica.com/information-technology/2018/04/suspicious-event-hija... On Apr 24, 2018, at 14:19 , Ryan McGinnis via Outages wrote: I suspect this was related to this issue (via ycombinator hacker news): https://doublepulsar.com/hijack-of-amazons-internet-domain-service-used-to-r... On Tue, Apr 24, 2018 at 8:51 AM, Zach Hanna via Outages wrote: Resolved here too.. On Tue, Apr 24, 2018 at 7:30 AM Phil Lavin via Outages wrote: Those prefixes had been withdrawn now – traffic is flowing correctly again for us. If that was the cause, I suspect things are back to rights for everyone now? One wonders why HE doesn’t apply filters on a peer with 20 legit prefixes… From: Outages On Behalf Of Joseph B via Outages Sent: 24 April 2018 13:56 To: outages@outages.org Subject: Re: [outages] Google 8.8.8.8 Resolution of Route53 domains Tue Apr 24 11:05:41 UTC onwards one of Hurricane Electric's peers AS10297 started advertising the following subnets via HE. 205.251.192.0 205.251.193.0 205.251.195.0 205.251.197.0 205.251.199.0 These are all Amazon subnets, usually originated as part of /23s and seemingly host a fair bit of AWS Route53. If you (or your DNS resolver) are a HE transit customer you will be impacted the most. Cheers, Joseph On Tue, Apr 24, 2018, at 9:50 PM, Phil Lavin via Outages wrote: This doesn’t feel right, though I’ll admit I’ve never checked before. Our only route to ns-163.awsdns-20.com (205.251.192.163) is through HE: inet.0: 757581 destinations, 2107440 routes (757301 active, 0 holddown, 522 hidden) + = Active Route, - = Last Active, * = Both 205.251.192.0/24 *[BGP/170] 01:12:08, localpref 70 AS path: 6939 10297 I, validation-state: unverified > to 216.66.90.21 via ge-1/0/5.0 AS10297 is eNET inc. Is this expected? From: Outages On Behalf Of Phil Lavin via Outages Sent: 24 April 2018 13:04 To: outages@outages.org Subject: Re: [outages] Google 8.8.8.8 Resolution of Route53 domains Looks more specific to AWS than it does to Google+AWS. Can’t resolve against some of AWS’s NS directly: phil@phil-debian:~$ dig cloudcall.com IN A @ns-163.awsdns-20.com ; DiG 9.10.3-P4-Debian cloudcall.com IN A @ns-163.awsdns-20.com ;; global options: +cmd ;; connection timed out; no servers could be reached From: Outages On Behalf Of Phil Lavin via Outages Sent: 24 April 2018 12:56 To: outages@outages.org Subject: Re: [outages] Google 8.8.8.8 Resolution of Route53 domains Yeh. Still digging into it. From: Outages On Behalf Of Zach Hanna via Outages Sent: 24 April 2018 12:54 To: outages@outages.org Subject: [outages] Google 8.8.8.8 Resolution of Route53 domains Anyone else seeing SERVFAIL for route53-hosted domains trying to resolve with Google DNS? _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages -- -Ryan McGinnis Platte Valley Communications 308-237-9512 PGP: 62E39BC1 _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages