Attacks from the cloud.

Update #1: 12 APR 2010. Amazon NOC’s response.
Update #2: 12 APR 2010. Amazon Statement.
Update #3: 13 APR 2010. Amazon Response.

Complaints of rampant SIP Brute Force Attacks coming from servers with Amazon EC2 IP Addresses cause many admins to simply drop all Amazon EC2 traffic. Generally, SIP brute force attacks attempt to register various peer names to a system and/or attempt to guess passwords of known/guesses peers or endpoints.

The complaints mentioned this weekend show an excessive amount of traffic; with some providers claiming 6GB of traffic dedicated to such attacks. Since we ourselves received an attack from an Amazon hosted server, we also reported and complained to the Amazon NOC/Abuse depts. As of this posting, no response or acknowledgement has been received from Amazon. The response from Amazon is below.

There are various techniques to assist with minimizing DDoS and Brute Force attacks, such as limiting access via the public internet, using strong passwords, not mapping extension name to peer/endpoint name, limiting simultaneous calls, and aggressively monitoring usage. Automatic blocking of abusive IP’s (fail2ban, blockhosts, etc.) can also assist with minimizing damage.

Update #1: 12 APR 2010. “Response” from Amazon’s NOC

So when this happened, I submitted a report to Amazon complaining of the attack. The report was sent to their abuse and noc mails and contained the standard abuse report, including their host, my host, the protocol, ports, and description of activity; as well as a sample log.

About 48 hours later, they sent this as a response:

From: Amazon.com <ec2-abuse@amazon.com>
Subject: Your Amazon EC2 Inquiry
Date: April 12, 2010 7:31:59 AM EDT
To: Fred Posner

Hello.

Thank you for contacting Amazon Web Services. We take reports of unauthorized network activity from our environment very seriously. It is specifically forbidden in our terms of use.

Because Amazon EC2 Public IP addresses may change ownership frequently, without additional information we will be unable to identify the correct owner of the IP address for the period of time in question.

So that we can process your report and identify the actual customer in question, we require the following information. Please note that we will not open attachments under any circumstance

• Source IP
• Destination IP (your IP)
• Destination Port and Protocol
• Accurate Date, Time and *Time Zone* of activity
• Intensity and frequency of activity in short log extracts, no larger than 4KB
• Your contact details (phone and email)

For a faster response, please file your report using the AWS Abuse form at the link below:

https://www.amazon.com/gp/html-forms-controller/AWSAbuse/

We appreciate your help in providing the necessary information requested.

Best regards,

-EC2 Abuse Team

So, their laziness aside (since the report to them included ALL the information requested), I filled out the form– which of course failed with an unknown error. (below)

For people interested in moving to Amazon’s cloud, this is a good example of the quality of people conducting your network administration. I’ve sent another response by email. Others have blocked the entire EC2 IP range and have requested their upstream providers do the same.

So my question to you… What would your next step be?

Update #2: 12 APR 2010. Amazon Statement.

Following a request for interview/statement, VoIP Tech Chat received the following communication from Amazon’s Public Relations Manager:

Hello Fred and thank you for contacting us.  Over the weekend, we received a report of a suspicious account and began an investigation.  Our normal process is to connect the two involved parties to give them an opportunity to talk in case the abuse is not malicious but is simply heavy traffic from a legitimate customer.  If that is not successful, we then move to isolate the traffic from the abusing party.  Normally this process works quite well for situations our customers have encountered, however this incident has highlighted the need for an escalation process to address potentially malicious attacks more quickly.  Additionally, we are working on quickly putting better protections and processes in place to better guard against unwanted SIP traffic.  We take the security of our customers and our quality of service very seriously, and will  continue to work to improve our processes and services for customers.

Thanks

Kay Kinton
Public Relations Manager
Amazon Web Services

I’ve replied to again ask for an interview and will update if a response is received. The statement states this was over a weekend, however doesn’t address that the attacks continued today. It also states a “report” was received, but there were many reports submitted. That being said, at least they responded.

Update #3: 13 APR 2010. Amazon Response. Decline of Interview.

After Kay Kinton’s statement, I asked her for a phone interview.

Sent: Monday, April 12, 2010 2:00 PM
To: Kinton, Kay
Subject: Re: Amazon Web Services

Kay,

Thank you for your statement. I would like to interview you about this for VoIP Tech Chat… it would be an over-the-phone interview and would be for 5 minutes or however much longer you would like.

—fred

Kay’s response was quick, and to the point:

What else can I tell you Fred?

I truly dislike email for interviews for no other reason that not getting the tone of that response. Did Kay mean that as “Sure, great! What else can I help you with?” Or, was it more along the lines of, “I answered you. What now?”

Giving her the benefit of the doubt, I replied:

Date: April 12, 2010 5:24:14 PM EDT
To: “Kinton, Kay”
Subject: Re: Amazon Web Services

We would like to interview on this. I thank you for the statement, however I have additional questions:

I know of 12 complaints since Saturday (from different reporters) that were submitted regarding SIP attacks from EC2 to outside systems. How many complaints did you receive since Saturday?

I know attacks continued today and may even be ongoing. There were attacks as of 1pm EST hitting systems with over 640K of data. Are you still seeing attacks? How many hosts were identified?

Were the attacks submitted from one customer/client of yours or many?

Those are my initial questions, however I do request a phone interview rather than email. I find them much easier to exchange information as well as generally a better expressive forum for an interview.

—fred

Good thing I didn’t hold my breath. The next day, after not receiving a response, I called Kay several times and emailed her for an update. Her response via email:

Hello Fred. We believe that we’ve identified and shut down the illegal activity and are closing the loop with customers. We’d certainly be interested in hearing of the cases you refer to below so we can follow up.

I tried reaching out to her but have not had responses. Which leaves me with this…

Her response did not answer my question and I certainly have no basis to believe that Amazon is currently taking any interest in this matter. They’ve told us prior that they cannot pinpoint IP to timeframe as well as that during an attack, they’d try to mediate between parties rather than actually stopping the attack in progress (to give them an opportunity to talk). Sadly… when I’m being flooded, I want the flood to stop. Afterwards, I’ll be glad to talk. But I digress…

Since Kay did not answer any of the additional questions we asked, but did state that she’d be interested in hearing about the other cases, we will encourage anyone with information or feelings about this issue to contact Kay Kinton directly:

Kay Kinton
kinton@amazon.com
Public Relations Manager
Amazon Web Services
Phone: 206-266-8387

For More Information:

• Asterisk User’s Mailing List Archives
• 7 Steps to Better SIP Security (Digium)
• Blockhosts
• Voice over IP Security Alliance (VOIPSA)
• SIP Brute Force Attack Originating From Amazon EC2 Hosts (Building The Net)
• Properly stopping a SIP flood (joshua stein)
• Automatically Block Failed SIP Peer Registrations (Team Forrest)

Report your error with our form. Fail.