APIs: Understanding the Basics
With respect to Amazon, the API is the sole external interface to computing resources and services. Without API calls being made, nothing gets done. API is short for application programming interface, as I mention elsewhere.
A good way to describe an API is to say that it represents a way for one program to interact with another via a defined interface — in other words, a mechanism by which any other program that communicates with the program can be assured that it will fulfill its role. The idea is that if a calling program provides the right information within the correct syntax, the program with the API will respond in the requested manner.
Benefiting from Web Services
This web services revolution that I describe makes possible a number of interesting benefits:
✓ Innovation: Just as musical mash-ups let people combine musical resources into new creations, so, too, do web services foster innovation.
✓ Niche market support: In a non-web-services world, the only people who can develop applications are those working for organizations. Only they have access to the computing resources or data — so the only applications that are developed are ones that the company deems useful. However, once those resources and data are made available via web services, anyone can create an application, which allows the development of applications targeted at niche markets. For example, someone can combine Google Maps with a municipal bus schedule in a mobile app to allow users to see when and where the next bus will be available nearby.
✓ New sources of revenue: Companies can provide a web services interface into their business transaction systems and allow outside entities to sell their goods. Netflix has made its web services interface to its video offerings available, and many device and
game manufacturers have used the interface to incorporate Netflix into their products. Netflix can gain new revenues every time someone buys a Wii or an Xbox and decides that it would be cool to use his new toy to access online movies and television.
An Overview of the AWS API
I’ve said it before and I’ll say it again: The only way you can interact with AWS is via its API. Every service you can ever use is called (and returns data) via its API, so using the API is critical to working with AWS. However, don’t worry about having to know the details of a low-level programming interface — you’ll likely never have to interact directly with the API. Nevertheless, you must understand at least the broad outline of how the AWS API functions, so that’s what I describe in this section.
Web services: SOAP or REST?
You can choose from a couple of different schools of thought for how web services should be delivered. The older approach, SOAP (short for Simple Object Access Protocol), had widespread industry support, complete with a comprehensive set of standards. Those standards were too comprehensive, unfortunately. The people designing SOAP set it up to be extremely flexible — it can communicate across the web, e-mail, and private networks. To ensure security and manageability, a number of supporting standards that integrate with SOAP were also defined.
SOAP is based on a document encoding standard known as Extensible Markup Language (XML, for short), and the SOAP service is defined in such a way that users can then leverage XML no matter what the underlying communication network is. For this system to work, though, the data transferred by SOAP (commonly referred to as the payload) also needs to be in XML format. Notice a pattern here? The push to be comprehensive and flexible (or, to be all things to all people) plus the XML payload requirement meant that SOAP ended up being quite complex, making it a lot of work to use properly. As you might guess, many IT people found SOAP daunting and, consequently, resisted using it.
REST, or Representational State Transfer.
REST, which is far less comprehensive than SOAP, aspires to solve fewer problems. It doesn’t address some aspects of SOAP that seemed important but that, in retrospect, made it more complex to use — security, for example. The most important aspect of REST is that it’s designed to integrate with standard web protocols so that REST services can be called with standard web verbs and URLs.
That’s all it takes to make a query to the REST service of examplecompany to see my personnel information. The HTTP verb that accompanies this request is GET, asking for information to be returned. To delete information, you use the verb DELETE. To insert my information, you use the verb POST. To update my information, you use the verb PUT. For the POST and PUT actions, additional information would accompany the empname and be separated by an ampersand (&) to indicate another argumentto be used by the service.
As you might expect, REST’s simpler use model, its alignment with standard web protocols and verbs, and its less restrictive payload formatting made it catch on with developers like a house on fire. There’s no sense in tempting fatewith AWS — if you want to interact with the AWS API, use REST, which isAmazon’s long-term direction.
The AWS API
As you might imagine, given the comprehensiveness of AWS services and the way Amazon has been improving and extending them, the AWS API is one large puppy — the AWS S3 API reference manual is 269 pages. (Think that’s a lot of pages? The AWS EC2 API reference manual is 561 pages.) However, if you take a quick look at the following example of an API call, you’ll quickly see that it closely resembles the (quite simple) REST example
The AWS API in real-world use
Never fear. Though the down-and-dirty details of using the AWS API are quite challenging, they’re unlikely to become stumbling blocks to achieving success with AWS. That’s because many clever people have recognized that the API is difficult to use and have created tools to make AWS simpler to use. In Figure 2-1, you can see the four major categories of AWS interaction mechanisms that spare you from the burden of interacting with the AWS API directly.
✓ AWS management console: Amazon offers a graphical web interface that allows you to interact with service (and your own) computing resources. For many people, the AWS management console is the primary mechanism they use to operate AWS. Even people who use the other two mechanisms to interact with AWS also make heavy use of the management console.
✓ CLI/SDK: Many software engineers write applications that need to interact with AWS services directly. To help them, Amazon and other companies have created language libraries (commonly called SDKs, standing for Software Development Kits) and a command line interface (commonly called a CLI), which allows commands to be entered in a terminal connected to AWS.
The idea here is to offer a simpler programmatic interface to the set of functions that do the heavy lifting of interacting with the AWS API. A software engineer can more easily incorporate library routines into an application, making it easier and faster to build AWS-based applications.
✓ Third-party tools: Many companies build tools that incorporate AWS. Some of these tools extend or simplify AWS itself, similar to what the language libraries do for software engineers. Other tools are products that offer separate functionality or even entire applications.
For example, my company provides cloud management software that offers additional functionality not offered by the AWS management console. Other examples include programming environments from companies like Heroku and Engine Yard, data warehousing technology from Informatica and JasperSoft, and load-testing services from SOASTA. What these tools have in common is that they provide functionality to shield users from interacting with the AWS API, making AWS easier and faster to use.
All four of these AWS API interaction mechanisms act as proxies on your behalf — under the covers they make the necessary calls to the AWS API to use the AWS functionality for actions you want to perform.
AWS API Security
Essentially, for every service call you want carried out, you (or a tool operating on your behalf) do the following:
1. Create the service call payload.
This is the data you need to send to AWS. It may be an object you want to store in S3 or the image identifier of an image you want to launch. (You’ll also attach other pieces of information to the payload, but because they vary according to the specifics of the service call, I don’t list them here. One piece of data is the current time.)
2. Encrypt the payload using the secret access key. Doing so ensures that no one can examine the payload and discover what’s in it.
3. Digitally sign the encrypted payload by adding the secret access key to the encrypted payload and performing a digital signature process using the secret access key. Secret access keys are longer and more random than typical user passwords; the lengthy secret access key makes the encryption performed with it more secure than it would be if it were performed with a typical user password.
4. Send the total encrypted payload, along with your access key, to AWS via a service call. Amazon uses the access key to look up your secret access key, which it uses to decrypt the payload. If the decrypted payload represents readable text that can be executed, AWS executes the service call. Otherwise, it concludes that something is wrong with the service call (perhaps that it was called by a malevolent actor) and doesn’t execute the service call.
If, like most AWS users, you use a proxy method to interact with AWS — the AWS management console, a language library, or a third-party tool — you need to provide your access key and secret access key to the proxy. When the proxy executes AWS service calls on your behalf, it includes the access key in the call and use the secret access key to perform payload encryption.
AWS offers a service to make managing keys/secret access keys easier. It’s called IAM. IAM allows you to assign keys and secret access keys to individuals or to applications, making it much easier to avoid wholesale changes when one person leaves an organization; it’s also a great help when you need to give each application access to the AWS services and resources that it needs.