I had a pretty good response to the AMIs posted a couple of days ago. Here’s how I made them.

Firstly, a word on security. I’ve been careful to ensure that there’s no trace of my machines or public keys on these images by not adding them in the first place. Many of the CentOS AMIs I tried out had entries in the last log from previous logins, and some even had root passwords set. Using the kickstart file I created, the root account on the machine is locked and no password based logins are allowed over SSH.

Anyway, here’s what I did:

Create suitable packages

An AMI isn’t so useful without cloud-init, so I rebuilt it for el6. I also rebuilt ec2-ami-tools, modifying the udev script so that devices appear where they’re supposed to. Without it, a drive specified as sda will appear as sde, because the EL6 Xen kernel reserves the first four drive letters. If you’re interested, take a look at the SRPMs.

Create the instance-store image

Install the image

I did this on a remote machine I had handy running Xen, although there’s no reason you couldn’t do this on a local machine with KVM – there’s no separate -xen kernel with EL6, Xen is fully merged in. I used a remote machine because it was in a data centre with a very speedy Internet connection, something that’s very useful when you’re going to be uploading large images.

I created the image as 5GB. You can use any size you like from 1GB-10GB for an instance store image, but remember that the large it is, the longer it’ll take to start the machine. Plus you’ll be using ephemeral or EBS storage for data on the machine anwyay.
I created a kickstart file which installs the image and carries out the necessary modifications. Specifically it sorts out networking by removing mac addresses from the ifcfg-eth0 file and udev, labels the partition and fixes the fstab, and modifies the grub config. Anyway, enough of the chat, on with the install:

$ sudo virt-install -n "centos-ami" -r 1024 --nographics -l http://mirrors.melbourne.co.uk/centos/6/os/x86_64/ -x "ks=http://bashton.com/downloads/centos-ami/ami-kickstart.cfg" -f /dev/dsk/centosami --noreboot

Select a suitable kernel

We’re going to be using the EL6 kernel within the image, so we use the ‘PV Grub’ loader within Amazon to boot this.
You can find a suitable kernel by running:

$ ec2-describe-images -a -F image-type=kernel -F manifest-location=*pv-grub*

For instance store images, use an image tagged hd0. For EBS backed images, use an image tagged hd00. Note that the Amazon Kernel Images (AKIs) are region specific.

Bundle the image and upload the image to S3

We need to expose the first (and only) partition of the image:

$ kpartx -a /dev/dsk/centosami

Now we bundle this into an image using the kernel we selected above. This will write the image files into ~bundle

$ ec2-bundle-image -r x86_64 -d ~/bundle -p myimagename -u 561795456677 -k ~/amazonkey.pem --kernel aki-825ea7eb -c amazoncert.pem --image /dev/mapper/centosamip1

Now we upload it to a S3 bucket:

$ ec2-upload-bundle -b mybucketname -m ~/bundle/myimagename.manifest.xml -a S3_ACCESS_KEY -s S3_SECRET

And register it as an AMI:

$ ec2-register mybucketname/myimagename.manifest.xml

Create the EBS backed image

We need an EC2 instance to write to the EBS volume, so spin one up. This can use the instance store AMI you created in step 1, but it doesn’t have to.

Create a volume and attach

Again, I’m creating an EBS image of 5GB. You can create EBS root images of up to 100GB, so do whatever suits you.
Make sure you create the volume in the same availability zone as the EC2 instance you just started up.

$ ec2-create-volume -s 5 -z eu-west-1a
$ ec2-attach-volume vol-abcdwxyz -i i-1234abcd -d sdg

Partition and format the volume

We’re going to create two partitions, a root and a small swap. The small swap is useful on t1.micro instances, where there is no ephemeral storage.

$ fdisk /dev/sdg
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0xd94fa889.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Calling ioctl() to re-read partition table.
Syncing disks.

Format the partition and label, then set fsck never to be run on boot

$ sudo mkfs.ext4 -L '/' /dev/sdg1
$ sudo tune2fs -c 0 -i 0 /dev/sdg1

Format and label swap

$ sudo mkswap -L 'ebs-swap' /dev/sdg2


Copy disk image across

We need to copy the disk image across. You can either use instance storage to temporarily store this image, or create an EBS volume. Either way, make a directory that’s writeable by the ec2-user.
On the machine you did the bare install on for the instance store:

$ dd if=/dev/mapper/centosamip1 of=- bs=1m | ssh -C ec2-user@newawsinstance.amazonaws.com "cat - > /media/ephemeral0/img/theimage.img"

Copy image files

Mount the image file as loopback
$ sudo mkdir /loop
$ sudo mount -o loop /media/ephemeral0/theimage.img /loop


Mount the destination EBS partition and copy the files across

$ sudo mount /dev/sdg1 /mnt
$ sudo rsync -avHx /loop/ /mnt

Modify grub config and add EBS swap to fstab


$ sudo sed -i -e 's/hd0/hd0,0/' /mnt/boot/grub/menu.lst
$ echo "LABEL=ebs-swap none swap sw 0 0" | sudo tee -a /mnt/etc/fstab

We’re done! Unmount the image.


$ umount /mnt


Create a snapshot

Detach the volume and create a snapshot:


$ ec2-detach-volume vol-abcdwxyz
$ ec2-create-snapshot -d "My really great AMI snapshot" vol-abcdwxyz


Find a suitable kernel to use, selecting an ‘hd00′ one this time:

$ ec2-describe-images -a -F image-type=kernel -F manifest-location=*pv-grub*

Last step – register your image!

$ ec2-register -b "/dev/sda=snap-12345678::false" -b "/dev/sdb=ephemeral0" -n "My Great AMI" -a x86_64 --kernel aki-12345678

amazon ec2, aws

I think Amazon Web Services is superb. We’re doing a lot of new customer deployments to it, and for creating a scalable, highly available infrastructure there’s really nothing that can match it.
But for a lot of our use cases Amazon Linux doesn’t really fit. We don’t want new versions of packages every six months, we want to set something up and be confident that it’ll continue working with backported security fixes for several years.
To fill that need, I’ve created Amazon Machine Images for CentOS 6. There were already some public images out there, but they all seemed to be pretty hacky in nature – random repositories enabled, random extra packages installed, no instance based swap etc etc.
These images are as close to a minimal CentOS 6 install as I think makes sense on AWS – CentOS6, with cloud-init and ec2-utils. Ephemeral storage will be mounted under /media/ephemeral0, swap will be mounted too. The EBS backed images include a small amount (~256MB) of swap to make them more useful on t1.micro – to disable this just run ‘swapoff LABEL=ebs-swap’.
I’ll be writing a separate blog post detailing how these images were created, and how you can create your own in the next few days.
Log in with the username ‘ec2-user’, using your SSH keypair. You’ll have full sudo access without a password.

amazon, ami, aws, centos, el6