Space Vatican

Ramblings of a curious coder

Starting EC2 Instances With Ephemeral Storage

Amazon EC2 instances can have two forms of storage attached: EBS, which is basically a SAN and instance local storage. The instance local storage is free (the amount you get depends on the instance type), but doesn’t persist once the instance is terminated. EBS storage is pricier (you pay per GB), but is persistent and you can do neat things like snapshot volumes or move volumes from one instance to another.

Although every instance type (except the micro instances) comes with instance storage, if you have an EBS backed instance the default is to not attach any instance storage. You can’t attach ephemeral storage at a later date, which is a bit annoying – it’s often useful to have a decent size chunk of space for temporary operations (even an m1.small has 150GB).

You ask for instance storage by specifying a Block Device Mapping when you create the instance (or setup the spot request, autoscaling group etc.). You can do this from the web console, but most of the instances I work with are created and managed through the api. You add an entry for each ephemeral device you want (on the larger instances the storage is split across 2 or more devices), setting DeviceName to where you want to attach the storage and VirtualName to ephemeral0 (ephemeral1 for the second and so on). On amazon linux it will be automatically mounted at /media/ephemeral0.

I forget the details every time I need to do this, so for posterity’s sake here’s how to do it with fog

compute =
compute.servers.create(:image_id => 'ami-61555115',
                       :flavor_id => 'm1.small',
                       :key_name => 'default',
                       :groups => 'ssh',
                       :block_device_mapping => [
                          {:DeviceName => '/dev/sdg', :VirtualName => 'ephemeral0'},

This creates an EBS backed m1.small instance running amazon linux which will have 150G of ephemeral space mounted at /media/ephemeral0 on /dev/sdg and setup so that I’ll be able to ssh into it.