
Just got confirmation from Internap NOC that they are being attacked again. Causing quite a bit of chaos for my network in SoCal. I'm having to route over to Level3 to minimize the issue.

On Wed, Feb 12, 2014 at 9:45 AM, Bryan Inks <Binks@keyinfo.com> wrote:
Just got confirmation from Internap NOC that they are being attacked again.
Causing quite a bit of chaos for my network in SoCal.
I'm having to route over to Level3 to minimize the issue.
*grumble* I wonder if that's what's causing the NANOG video stream to be choppy? I know Internap is providing the connectivity for the conference. If so, I sure wish the attackers would have waited one more day. :( Matt

Close your NTP amplifiers and prevent the spoofing.. Will solve this one. Openntpproject.org can help you. Jared Mauch
On Feb 12, 2014, at 12:45 PM, "Bryan Inks" <Binks@keyinfo.com> wrote:
Just got confirmation from Internap NOC that they are being attacked again.
Causing quite a bit of chaos for my network in SoCal.
I’m having to route over to Level3 to minimize the issue. _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages

To second Jared on this one, we’ve seen a HUGE increase in NTP-based attacks over the past several weeks with our colo customers. It’s very efficient too – even a pretty low end machine can saturate a 100M link. It reminds me of SQL slammer… If you haven’t yet checked that you’re safe from this you should. See: https://www.us-cert.gov/ncas/alerts/TA14-013A and https://www.us-cert.gov/ncas/alerts/TA14-017A for more info… -Bill From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Jared Mauch Sent: Wednesday, February 12, 2014 1:21 PM To: Bryan Inks Cc: outages@outages.org Subject: Re: [outages] Internap Being DDoS'd Close your NTP amplifiers and prevent the spoofing.. Will solve this one. Openntpproject.org<http://Openntpproject.org> can help you. Jared Mauch On Feb 12, 2014, at 12:45 PM, "Bryan Inks" <Binks@keyinfo.com<mailto:Binks@keyinfo.com>> wrote: Just got confirmation from Internap NOC that they are being attacked again. Causing quite a bit of chaos for my network in SoCal. I’m having to route over to Level3 to minimize the issue. _______________________________________________ Outages mailing list Outages@outages.org<mailto:Outages@outages.org> https://puck.nether.net/mailman/listinfo/outages

Good info, I’ll definitely be looking into this. But, I’m not being directly attacked. Internap is one of my upstreams, and they are the one that reported that they were being attacked when we called to let them know about the problem. From: Bill Wichers [mailto:billw@waveform.net] Sent: Wednesday, February 12, 2014 10:27 AM To: Jared Mauch; Bryan Inks Cc: outages@outages.org Subject: RE: [outages] Internap Being DDoS'd To second Jared on this one, we’ve seen a HUGE increase in NTP-based attacks over the past several weeks with our colo customers. It’s very efficient too – even a pretty low end machine can saturate a 100M link. It reminds me of SQL slammer… If you haven’t yet checked that you’re safe from this you should. See: https://www.us-cert.gov/ncas/alerts/TA14-013A and https://www.us-cert.gov/ncas/alerts/TA14-017A for more info… -Bill From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Jared Mauch Sent: Wednesday, February 12, 2014 1:21 PM To: Bryan Inks Cc: outages@outages.org<mailto:outages@outages.org> Subject: Re: [outages] Internap Being DDoS'd Close your NTP amplifiers and prevent the spoofing.. Will solve this one. Openntpproject.org<http://Openntpproject.org> can help you. Jared Mauch On Feb 12, 2014, at 12:45 PM, "Bryan Inks" <Binks@keyinfo.com<mailto:Binks@keyinfo.com>> wrote: Just got confirmation from Internap NOC that they are being attacked again. Causing quite a bit of chaos for my network in SoCal. I’m having to route over to Level3 to minimize the issue. _______________________________________________ Outages mailing list Outages@outages.org<mailto:Outages@outages.org> https://puck.nether.net/mailman/listinfo/outages

It’s the other way around though isn’t it? If your NTP servers are vulnerable then you’re assisting in the attack as your NTP servers are being used to attack the victim. Though I imagine that if your NTP servers were being used you’d bee seeing a large spike in your outbound UDP traffic. On Feb 12, 2014, at 10:33 AM, Bryan Inks <Binks@keyinfo.com> wrote:
Good info, I’ll definitely be looking into this.
But, I’m not being directly attacked. Internap is one of my upstreams, and they are the one that reported that they were being attacked when we called to let them know about the problem.
From: Bill Wichers [mailto:billw@waveform.net] Sent: Wednesday, February 12, 2014 10:27 AM To: Jared Mauch; Bryan Inks Cc: outages@outages.org Subject: RE: [outages] Internap Being DDoS'd
To second Jared on this one, we’ve seen a HUGE increase in NTP-based attacks over the past several weeks with our colo customers. It’s very efficient too – even a pretty low end machine can saturate a 100M link. It reminds me of SQL slammer…
If you haven’t yet checked that you’re safe from this you should. See:
https://www.us-cert.gov/ncas/alerts/TA14-013A and https://www.us-cert.gov/ncas/alerts/TA14-017A
for more info…
-Bill
From: Outages [mailto:outages-bounces@outages.org] On Behalf Of Jared Mauch Sent: Wednesday, February 12, 2014 1:21 PM To: Bryan Inks Cc: outages@outages.org Subject: Re: [outages] Internap Being DDoS'd
Close your NTP amplifiers and prevent the spoofing.. Will solve this one.
Openntpproject.org can help you.
Jared Mauch
On Feb 12, 2014, at 12:45 PM, "Bryan Inks" <Binks@keyinfo.com> wrote:
Just got confirmation from Internap NOC that they are being attacked again.
Causing quite a bit of chaos for my network in SoCal.
I’m having to route over to Level3 to minimize the issue. _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages _______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages

On 02/12/2014 11:33 AM, Bryan Inks wrote:
Good info, I'll definitely be looking into this.
But, I'm not being directly attacked. Internap is one of my upstreams, and they are the one that reported that they were being attacked when we called to let them know about the problem.
*From:*Bill Wichers [mailto:billw@waveform.net] *Sent:* Wednesday, February 12, 2014 10:27 AM *To:* Jared Mauch; Bryan Inks *Cc:* outages@outages.org *Subject:* RE: [outages] Internap Being DDoS'd
To second Jared on this one, we've seen a HUGE increase in NTP-based attacks over the past several weeks with our colo customers. It's very efficient too -- even a pretty low end machine can saturate a 100M link. It reminds me of SQL slammer...
If you haven't yet checked that you're safe from this you should. See:
https://www.us-cert.gov/ncas/alerts/TA14-013A
and
https://www.us-cert.gov/ncas/alerts/TA14-017A
for more info...
And some info on how to mitigate it so you are not a reflector. http://www.team-cymru.org/ReadingRoom/Templates/secure-ntp-template.html --John
-Bill
*From:*Outages [mailto:outages-bounces@outages.org] *On Behalf Of *Jared Mauch *Sent:* Wednesday, February 12, 2014 1:21 PM *To:* Bryan Inks *Cc:* outages@outages.org <mailto:outages@outages.org> *Subject:* Re: [outages] Internap Being DDoS'd
Close your NTP amplifiers and prevent the spoofing.. Will solve this one.
Openntpproject.org <http://Openntpproject.org> can help you.
Jared Mauch
On Feb 12, 2014, at 12:45 PM, "Bryan Inks" <Binks@keyinfo.com <mailto:Binks@keyinfo.com>> wrote:
Just got confirmation from Internap NOC that they are being attacked again.
Causing quite a bit of chaos for my network in SoCal.
I'm having to route over to Level3 to minimize the issue.
_______________________________________________ Outages mailing list Outages@outages.org <mailto:Outages@outages.org> https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages

I see some attributes in the "UNIX ntpd" example there which are missing. I would suggest people follow the defaults provided by some of the OSS distros (ex. FreeBSD 9): http://svnweb.freebsd.org/base/stable/9/etc/ntp.conf?revision=259974&view=ma... Specifically these lines for starters: restrict default kod nomodify notrap nopeer noquery restrict -6 default kod nomodify notrap nopeer noquery restrict 127.0.0.1 restrict -6 ::1 restrict 127.127.1.0 The last 3 lines are effectively "allow" statements. You'll need to modify your ntp.conf accordingly; e.g. if the system in question is used as an NTP server for other machines on 192.168.1.0/24, you'd need something like: restrict 192.168.1.0 mask 255.255.255.0 But I recommend folks read (not skim -- it actually reads quite easily, just the formatting isn't easily skimmable) the following page, as it goes over the difference between "restrict default {bunch of modifiers}" vs. "restrict default ignore": http://support.ntp.org/bin/view/Support/AccessRestrictions It's remarkable how neglected NTP is as a service. :/ -- | Jeremy Chadwick jdc@koitsu.org | | UNIX Systems Administrator http://jdc.koitsu.org/ | | Making life hard for others since 1977. PGP 4BD6C0CB | On Wed, Feb 12, 2014 at 11:37:00AM -0700, John wrote:
On 02/12/2014 11:33 AM, Bryan Inks wrote:
Good info, I'll definitely be looking into this.
But, I'm not being directly attacked. Internap is one of my upstreams, and they are the one that reported that they were being attacked when we called to let them know about the problem.
*From:*Bill Wichers [mailto:billw@waveform.net] *Sent:* Wednesday, February 12, 2014 10:27 AM *To:* Jared Mauch; Bryan Inks *Cc:* outages@outages.org *Subject:* RE: [outages] Internap Being DDoS'd
To second Jared on this one, we've seen a HUGE increase in NTP-based attacks over the past several weeks with our colo customers. It's very efficient too -- even a pretty low end machine can saturate a 100M link. It reminds me of SQL slammer...
If you haven't yet checked that you're safe from this you should. See:
https://www.us-cert.gov/ncas/alerts/TA14-013A
and
https://www.us-cert.gov/ncas/alerts/TA14-017A
for more info...
And some info on how to mitigate it so you are not a reflector.
http://www.team-cymru.org/ReadingRoom/Templates/secure-ntp-template.html
--John
-Bill
*From:*Outages [mailto:outages-bounces@outages.org] *On Behalf Of *Jared Mauch *Sent:* Wednesday, February 12, 2014 1:21 PM *To:* Bryan Inks *Cc:* outages@outages.org <mailto:outages@outages.org> *Subject:* Re: [outages] Internap Being DDoS'd
Close your NTP amplifiers and prevent the spoofing.. Will solve this one.
Openntpproject.org <http://Openntpproject.org> can help you.
Jared Mauch
On Feb 12, 2014, at 12:45 PM, "Bryan Inks" <Binks@keyinfo.com <mailto:Binks@keyinfo.com>> wrote:
Just got confirmation from Internap NOC that they are being attacked again.
Causing quite a bit of chaos for my network in SoCal.
I'm having to route over to Level3 to minimize the issue.
_______________________________________________ Outages mailing list Outages@outages.org <mailto:Outages@outages.org> https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages
_______________________________________________ Outages mailing list Outages@outages.org https://puck.nether.net/mailman/listinfo/outages

On Wed, Feb 12, 2014 at 10:53:35AM -0800, Jeremy Chadwick wrote:
I see some attributes in the "UNIX ntpd" example there which are missing. I would suggest people follow the defaults provided by some of the OSS distros (ex. FreeBSD 9):
http://svnweb.freebsd.org/base/stable/9/etc/ntp.conf?revision=259974&view=ma...
Specifically these lines for starters:
restrict default kod nomodify notrap nopeer noquery restrict -6 default kod nomodify notrap nopeer noquery
Only use "kod" if you also use "limited": https://bugzilla.redhat.com/show_bug.cgi?id=1048196 "The current default restrict line in ntp.conf is: restrict default kod nomodify notrap nopeer noquery This can be confusing as the kod option is active only when the limited options is also present. This is documented in ntp_acc(5) man page. The upcoming ntp-4.2.8 will warn about this and we probably want to avoid getting that warning in the future. http://bugs.ntp.org/show_bug.cgi?id=2060 The fix is to remove kod from the default restrict line."

On Wed, 12 Feb 2014 10:53:35 -0800 Jeremy Chadwick <jdc@koitsu.org> wrote:
I see some attributes in the "UNIX ntpd" example there which are missing.
Hi Jeremy, Being the original author of the template with some contributions from others, I'll simply note that we are aware of some of these other options and are considering changes. We just received some additional suggestions about another part of the template today. Earlier we had received somewhat conflicting advice from two people both associated with the ntp.org project, though easily reconcilable. :-) I think what we're going to do is highlight some of these seemingly different suggestions by offering more detailed discussion about the options and make it easier for people to tune it to local conditions and preference. Thanks for your comments here, this is a helpful discussion for us as well. John
participants (9)
-
Bill Wichers
-
Bryan Inks
-
Chris Vervais
-
Chuck Anderson
-
Jared Mauch
-
Jeremy Chadwick
-
John
-
John Kristoff
-
Matthew Petach