The S3 REST interface is truly RESTful—you think in terms of resources, such as services (to get a list of all your buckets), buckets, and objects—and they have standard methods. See the following for a list of resources and methods:
http://docs.amazonwebservices.com/AmazonS3/2006-03-01/RESTAPI.html
Using the REST interface is a bit tricky because of the following:
In this section, I will show one specific, relatively simple GET
example to demonstrate how to use the REST interface. Let’s first use the query string request authentication alternative, which doesn’t require the use of HTTP Authorization headers. As the documentation indicates, “The practice of signing a request and giving it to a third-party for execution is suitable only for simple object GET
requests.”
The REST endpoint is as follows:
You need three query parameters:
AWSAccessKeyId
: Your access key
Expires
: When the signature expires, specified as the number of seconds since the epoch (00:00:00 UTC on January 1, 1970)
Signature
: The URL encoding of the Base64 encoding of the HMAC-SHA1 of StringToSign
(defined in a moment)
I’ll use the example data given in the documents and generate some Python and PHP code to demonstrate how to calculate the Signature
. That is, I’ll show you how to reproduce the results in the documentation. I’ll use parameters (listed in Table 16-1) that draw from examples at the following location:
http://docs.amazonwebservices.com/AmazonS3/2006-03-01/RESTAuthentication.html
The parameters shown here would be used to access the object whose key is photos/puppy.jpg
in the bucket named johnsmith
. (Note that AWSAccessKeyId
and AWSSecretAccessKey
are not actually valid keys but are presented to illustrate the calculations.)
Setting | Value |
AWSAccessKeyId 0PN5J17HBGZHT7JJ3X82 | |
AWSSecretAccessKey uV3F3YluFJax1cknvbcGwgjvx4QpvB+leU8dUj2o | |
Expires 1175139620 | |
Host johnsmith.s3.amazonaws.com | |
Key photos/puppy.jpg | |
HTTP-Verb GET | |
Content-MD5 | |
Content-Type | |
CanonicalizedAmzHeaders | |
CanonicalizedResource /johnsmith/photos/puppy.jpg |
The pseudo-code for calculating the signature is (quoting from the documentation) as follows:
StringToSign = ?HTTP-?VERB + "\n" + ?Content-?MD5 + "\n" + ?Content-?Type + "\n" + Expires + "\n" + CanonicalizedAmzHeaders + CanonicalizedResource; Signature = ?URL-?Encode( Base64( ?HMAC-?SHA1( UTF-8-Encoding-Of( StringToSign ) ) ));
We’re told that the Signature
based on the parameters in Table 16-1 should be rucSbH0yNEcP9oM2XNlouVI3BH4%3D
. Let’s figure out how we can reproduce this signature in Python and PHP.
First, here is the Python code to calculate the Signature
:
import sha, hmac, base64, urllib AWSAccessKeyId = "0PN5J17HBGZHT7JJ3X82" AWSSecretAccessKey = "uV3F3YluFJax1cknvbcGwgjvx4QpvB+leU8dUj2o" Expires = 1175139620 HTTPVerb = "GET" ContentMD5 = "" ContentType = "" CanonicalizedAmzHeaders = "" CanonicalizedResource = "/johnsmith/photos/puppy.jpg" string_to_sign = HTTPVerb + "\n" + ContentMD5 + "\n" + ContentType + "\n" + Â str(Expires) + "\n" + CanonicalizedAmzHeaders + CanonicalizedResource sig = base64.b64encode(Â hmac.new(AWSSecretAccessKey, string_to_sign, sha).digest()) print urllib.urlencode({'Signature':sig})
This produces the following:
Signature=rucSbH0yNEcP9oM2XNlouVI3BH4%3D
Here’s some corresponding PHP code to calculate the Signature
:
<?php # base64.encodestring # The hex2b64 function is excerpted from the Amazon S3 PHP example library. # http://developer.amazonwebservices.com/connect/entry.jspa?externalID=126 function hex2b64($str) { $raw = ''; for ($i=0; $i < strlen($str); $i+=2) { $raw .= chr(hexdec(substr($str, $i, 2))); } return base64_encode($raw); } require_once 'Crypt/HMAC.php'; require_once 'HTTP/Request.php'; $AWSAccessKeyId = "0PN5J17HBGZHT7JJ3X82"; $AWSSecretAccessKey = "uV3F3YluFJax1cknvbcGwgjvx4QpvB+leU8dUj2o"; $Expires = 1175139620; $HTTPVerb = "GET"; $ContentMD5 = ""; $ContentType = ""; $CanonicalizedAmzHeaders = ""; $CanonicalizedResource = "/johnsmith/photos/puppy.jpg"; $string_to_sign = $HTTPVerb . "\n" . $ContentMD5 . "\n" . $ContentType . "\n" . $Expires . "\n" . $CanonicalizedAmzHeaders . $CanonicalizedResource; $hasher =& new Crypt_HMAC($AWSSecretAccessKey, "sha1"); $sig = hex2b64($hasher->hash($string_to_sign)); echo 'Signature=',urlencode($sig); ?>
Note that this PHP code depends on two PEAR libraries that need to be installed:
Unfortunately, after you get those libraries installed, I can’t recommend using the S3 sample code on a remote host, because it requires sending the secret over the wire. Run it on your own secure machine.
Once you have calculated the Signature
, you can package the corresponding HTTP GET
request:
http://johnsmith.s3.amazonaws.com/photos/puppy.jpg?AWSAccessKeyId=0PN5J17HBGZHT7JJ3XÂ?82&Signature=rucSbH0yNEcP9oM2XNlouVI3BH4%3D&Expires=1175139620
Now that you understand the basics behind signing an Amazon S3 request, I’ll show you how to get a list of your S3 buckets. First I’ll show the code, and then I’ll offer an explanation:
def listBuckets(AWSAccessKeyId,AWSSecretAccessKey): """ use the REST interface to get the list of buckets -- without the use the Authorization HTTP header """ import sha, hmac, base64, urllib import time # give an hour for the request to expire (3600s) expires = int(time.time()) + 3600 string_to_sign = "GET\n\n\n%s\n/" % (expires) sig = base64.b64encode(Â hmac.new(AWSSecretAccessKey, string_to_sign, sha).digest()) request = "http://s3.amazonaws.com?AWSAccessKeyId=%s&Expires=%s&%s" % \ (AWSAccessKeyId, expires, urllib.urlencode({'Signature':sig})) return request if __name__ == "__main__": AWSAccessKeyId='[AWSAccessKeyID]' AWSSecretAccessKey = '[SecretAccessKey]' print listBuckets(AWSAccessKeyId,AWSSecretAccessKey)
This code generates a URL of the following form that returns a list of buckets:
http://s3.amazonaws.com?AWSAccessKeyId={AWSAccessKeyId}&Expires=1196114919Â &Signature={Signature}
Note how we use some of the same parameters as in the previous example (AWSAccessKeyId
, AWSSecretAccessKey
, and Expires
) and calculate the Signature
with the same combination of SHA-1 hashing and Base64 encoding.