It’s not uncommon to want to make requests to AWS from your servers. For example you might be using S3 to store user uploads, pushing custom metrics into CloudWatch or accessing one of the datastores such as DynamoDB or SimpleDB. All of these require authentication, and getting your access key / secret key onto your instances has historically been a little messy.

For a little while, AWS has provided IAM which, among other things, allows you to create users with fine grained sets of permissions. For example you could create a user that only had the ability to store files in the S3 bucket your app uses for uploads but with no access to any other of the AWS apis (annoyingly one thing you can’t do is restrict which instances EC2 apis can affect). Clearly this is much nicer than having to embed your master api key in the instance.

You still need to get the api keys onto your instances. You probably don’t want them checked into source control, so you need to put them onto instances as part of the deploy process. This still requires them to be stored somewhere or for you to create credentials on the fly when you do a deploy. The resulting complication means that it’s hard to follow best practices such as regular key rotation.

Roles to the rescue

If you run on EC2, you’re in luck because Amazon recently introduced the concept of an IAM Role. A role is effectively a container for a set of IAM policies. Roles in turn can be attached to instance profiles. You can then launch an EC2 instance with an instance profile (or configure an autoscaling group to launch instances with a profile).

When an instance is launched in this manner, the instance metadata service will provide credentials. Instead of reading credentials from a file you provide, your application can get them from the metadata service. If you’re using the standard aws sdks for ruby, java etc. this all happens behind your back - you don’t need to do anything. As of 1.4.0, the awesome cloud library Fog also supports IAM roles, although rather than the feature silently being used, you need to ask for it to be enabled. The credentials are temporary ones, both in that amazon will rotate the credentials every few hours and in that in addition to an access key and a secret key, you get a session token.

While the design of the api seems to allow attaching multiple roles to a profile, currently you’re limited to 1 role per instance profile, which sort of makes a role basically the same thing as a profile (if you use the IAM web console you’ll notice that policies aren’t mentioned at all).

Code code code

Fog is the library I tend to use for this sort of stuff, here’s how you’d use a profile that allowed access to a specific S3 bucket:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

iam = Fog::AWS::IAM.new
iam.create_instance_profile('app_server')
iam.create_role('app_server', Fog::AWS::IAM::EC2_ASSUME_ROLE_POLICY)


iam.put_role_policy('app_server', 's3_access', {
  "Version" => "2008-10-17",
  "Statement" => [
    {
      "Effect"=>"Allow",
      "Action"=> ["s3:*"],
      "Resource"=> ["arn:aws:s3:::somebucket/*"]
    }
  ]
})

This you’d run somewhere where you have existing credentials (I’ve assumed they’re in your .fog file). Obviously you could restrict that policy further, for example code that makes backups to s3 might have the right to put files in an s3 bucket but not to delete existing files.

Once you’ve created a profile, you can start instances that use it. With fog you’d do

1
2
3

compute = Fog::Compute::AWS.new
compute.servers.create :flavor_id => 't1.micro', :image_id => 'ami-ed65ba84',
                       :iam_instance_profile_name => 'app_server'

To launch an amazon linux t1.micro instance using that iam profile. You can’t currently add a profile to an instance once it has been started although you can change the policies associated with the profile as much as you want. In your app code, rather than providing a full set of credentials just do

1

Fog::Storage::AWS.new :use_iam_profile => true

and fog will fetch the credentials for you. It will also keep track of when they expire, so if you make a request when they’re about to expire or after they’ve expired, fog will fetch fresh credentials (amazon makes new credentials available 5 minutes before the previous ones expire).

If you’re only interacting with fog using something like paperclip or carrierwave, you should be able to change the hash of credentials you pass to just :use_iam_profile => true. If you use dragonfly, then you’ll need to wait for this pull request.

The only thing you can’t do at all with instance profiles do is grant access to IAM itself, but that’s not a restriction I see myself ever running into. One limitation that I suppose could be troublesome is if you’re generating long lived signed S3 download urls. These will (I believe) only last as long as the credentials themselves, i.e. a couple of hours at most.