AWS credentials are very powerful – in the wrong hands you could lose data or incur large costs – so you need to manage them carefully. I don’t think there’s a one size fits all here and as usual security and convenience tend to pull in opposite directions, but I think there are at least some general guideslines:

• You should very rarely be using the master account, instead use IAM to create users with specific permissions
• Try to attach policies to groups rather than users – this makes them easier to manage
• Make groups task focussed rather than service focussed. For example I’d have a backup group that had relevant access to S3 and glacier rather than having an S3 group and a glacier group
• Only give people access to what they need. Your accountant for example only needs access to the billing data, not the api or console

The downside of this is that you now have multiple sets of credentials to manage for all these IAM users. For me, credentials fall into 3 categories:

• Credentials that the app uses itself
• Credentials for using the web console
• Credentials for management scripts (provisioning new instances etc.)

Or how to waste half an hour

This weekend I lost my season ticket. Normally it’s not the end of the world – as long it’s the first time that year you can order a duplicate. If you bought the ticket in person you can do this at the station, but if you ordered the ticket from the first capital connect website (as I had) then you have to apply for a duplicate through the website.

I’d already tried and failed to find this on the website a few days ago (my ticket had recently stopped working at the barriers) but this time I had no choice but to persist. The website certainly used to have this feature – I’m not on my first lost ticket – but the website had recently received some sort of overhaul.

The short version is that you can probably call the trainline on 0871 244 1545 or possibly 0870 024 0463, but since it took me quite a few rounds of phone tag to get to the bottom of it I feel I have to inflict the full version on you.

I recently wanted to start recording various business level metrics for our Rails app. We already use New Relic for performance monitoring and while you can inject custom metrics you can only surface them if you have the Pro version (unless they happen to surface naturally as one of the top n slowest/highest throughput/highest total time metrics). I’d been hearing a lot of good things about statsd and graphite so decided to try my hand at setting those up. Graphite is written in python, so this meant venturing out of my cosy little ruby world.

Amazon EC2 instances can have two forms of storage attached: EBS, which is basically a SAN and instance local storage. The instance local storage is free (the amount you get depends on the instance type), but doesn’t persist once the instance is terminated. EBS storage is pricier (you pay per GB), but is persistent and you can do neat things like snapshot volumes or move volumes from one instance to another.

Glacier is Amazon’s latest storage service, designed for long term storage of data. It’s not a direct replacement for S3: retrieving data is slow and potentially expensive, so stay away if you expect to retrieve data often or if waiting a few hours for retrieval is too slow.

One good use case would be if you have to keep data for a very long time for regulatory reasons. Storage is a lot cheaper than S3: storage costs $0.01/GB/month. By comparison if you have less than 50TB S3 charges $0.11, and its 1-5PB rate is $0.08/GB (more on pricing later). In case the long-termness needs hammering in, Amazon actually charges a fee for data deleted within 90 days of upload.

As of early September 2012 the AWS ruby sdk doesn’t include glacier support. If like me you want to use glacier from your ruby apps, one of your options is the fog gem that supports glacier from version 1.6 onwards.