Stretching Out with Elastic Compute Cloud
This chapter discusses EC2 — the Elastic Compute Cloud, which is the most widely used AWS service. Even the term “cloud computing” emphasizes computing — and its computing that EC2 delivers, at scale, in wide varieties of types, and at ridiculously low prices.
Introducing EC2
EC2 is the most revolutionary of the AWS services because it has transformeda fundamental part of IT: the use of provisioning servers. EC2 provides virtual servers in a matter of minutes, all via self-service. It’s difficult to overstate the shift that this strategy represents compared to how things used to be done.In earlier days when you needed a server, you had to scare up enough money to buy one, complete the purchase process, and then have the server delivered, installed, and connected to the network. Finally (finally!), you gained access to your server. It wasn’t uncommon for this process to take from three to six months!
S3-backed instances don’t store changes made to the file system when theinstance is shut down (terminated). The next time the image is launched, the running instance reflects the layout of the image as originally created. It’s similar to a gold image or a LiveCD (in case you’ve used a CD-based Linux system). Understanding the transient nature of the file system for S3-backed instances is critical. No changes made to an instance are persistent post-termination —unlike in any operating system you’ve ever used (except for LiveCD). If your instance will process and save data, you must find a way to save the data outside of the instance. Simply put, S3-backed images don’t make data persistent.
Understanding images
An image is the collection of bits needed to create a running instance. This collection includes the elements described in this list:
✓ At minimum, the operating system that will run on the instance: That means it can be Windows or Linux.
✓ Any software packages you’ve chosen to install: The package can be software that you’ve written or a package from a third-party provider (assuming, of course, that the software license supports this type of use). For example, you may include the Apache web server along with the load balancer HAProxy — both are open source products that can be freely included in your image.
✓ Any configuration information needed for the instance to operateproperly: For example, in an image containing Linux, Apache, and HAProxy, you may include configuration information for HAProxy to communicate with the Apache server located on the same instance. Adding this information to the image prevents having to configure the packages every time you launch the image. An image carries access rights: Someone owns it, and the owner can control who may launch (or even see) the image. The following list describes the image-ownership categories, which are listed on the drop-down menu
✓ Owned by me: Images created by your account, whether you are its sole user or you share it with others; may include both public and privateimages
✓ Amazon images: Images created by Amazon and made available to anyone who wants to use them
✓ Public images: Images owned by other accounts but made available to anyone who wants to use them
✓ Private images: Images owned by you and made available only to you or to other accounts you specify
✓ EBS images: Images that use Elastic Block Storage (EBS) as the storage for the AMI
✓ Instance-store images: Images that are stored in Simple Storage Service (S3)
✓ 32-bit: Images built on 32-bit operating systems (can be either instanceorEBS-backed)
✓ 64-bit: Images built on 64-bit operating systems (can be either instanceorEBS-backed)
✓ AWS Marketplace: Images, created by third parties, that are available for a fee
Other AMI information:
✓ AMI ID: Peeking out from the AMI Type drop-down menu to identify every AMI, this AWS-assigned number is unique for every AMI.
✓ Source: The description of the AMI typically includes information about the AMI creator’s name, the operating system, and the software componentsinstalled on the AMI.
✓ Owner: This long number is the image owner’s AWS account number.
✓ Visibility: Tells you who can see the AMI
✓ Platform: Points out which operating system is installed on the AMI
✓ Root device: Indicates whether the image is an S3-backed instance or an EBS-backed instance
✓ Virtualization: Specifies how the instance interacts with the virtualization hypervisor in EC2 (information that you generally don’t need to beconcerned about or, indeed, have control of)
S3-backed images
S3 images are stored as multiple 10MB files, along with a special XML file called a manifest. The manifest file is similar to the assembly instructions in an Ikea flat-pack piece of furniture — it gives AWS the information it needs to construct a running instance from the collection of 10MB S3 objects.
S3-backed images come with limitations, and you should fully understand what they are. The following list spells them out for you:
✓ Root device limitations: It’s crucial to understand precisely what’s stored in the image proper. With S3-backed images, all you have is the root device — the part of the system containing system files (includingt he operating system) — plus, any other software that was installed when the image was created.An S3-backed image is limited to 10GB in the root device. All other parts of the file system are constructed at the time of launch; for example, in a small instance with 170GB of disk space, 160GB of the instance storage is created at launch-time, and only 10GB is persistent. If you want to include a lot of software packages or data in the root device, you may exceed this 10GB limit.
✓ Long launch times: Because the instance has to be created from the various 10MB files making up a collection, it takes a while to assemble them, which extends the launch times. Removing an S3-backed instance from production requires terminating it — no ifs, ands, or buts: Doing so discards all data written to its file system since the launch.
✓ The AWS management console doesn’t support the creation of a new image from an S3-backed instance. If you want to create a new image from an S3-backed instance, first install AWS AMI tools on the instance,and then run scripts to create the image. Though this task is perfectly possible, it’s not a trivial matter, so it’s a definite limitation.
EBS-backed images
EBS-backed images have been available since late 2009. The primary difference between an EBS-backed image and an S3-backed image is that the former uses a persistent EBS volume for instance storage — instances can now be stopped and started after launch, in addition to being terminated.
In addition to the luxury of instance stops and starts and data persistence across the stop/start cycle, EBS-backed images offer a number of advantages versus S3-backed images:
✓ EBS-backed images allow for much larger root volumes thanS3-backed images. Instead of the mere 10GB root volumes found inS3-backed images, EBS-backed images can have root volumes as large as 1TB, which allows more room for additional software packages or datastorage on the image.
✓ If an EBS-backed instance is stopped, you incur no EC2 charges: Concern yourself only with the image storage costs on EBS, similar tothe storage cost you pay to store an S3 image in S3.
✓ If something goes wrong with an EBS-backed image, its root volume can be mounted on another instance. You can then examine or even repair it.
✓ If you need to adjust the instance size of an EBS-backed instance, stop it and then restart it at the new size. This method is a lot more flexible and faster than S3-backed instances. (Don’t forget that the data in the EBS-backed instance is persistent across that stop/start cycle.)
✓ Only an EBS-backed type supports the EC2 Micro instance type. TheMicro type is useful for testing and for certain use cases of low-volume processing, and, crucially, it’s part of the Free Usage Tier that Amazon makes available. If you want to work with EC2 at no charge, you must use EBS-backed images.
Though EBS-backed images let you stop and start instances, with data persistence across the stop/start cycle, they do not — I repeat, do not — persist data in the event of a termination. Any data that isn’t present on the image is discarded when an EBS-backed instance terminates. For that reason, you should follow the same practice with EBS-backed instances that you follow with S3-backed instances: Place all data that requires persistence on separate EBS volumes that persist even if the instance they’re attached to terminates.