Using the S3 REST Interface

The S3 REST interface is truly RESTful—you think in terms of resources, such as services (to get a list of all your buckets), buckets, and objects—and they have standard methods. See the following for a list of resources and methods:

Using the REST interface is a bit tricky because of the following:

In this section, I will show one specific, relatively simple GET example to demonstrate how to use the REST interface. Let’s first use the query string request authentication alternative, which doesn’t require the use of HTTP Authorization headers. As the documentation indicates, “The practice of signing a request and giving it to a ­third-­party for execution is suitable only for simple object GET requests.”

The REST endpoint is as follows:

You need three query parameters:

I’ll use the example data given in the documents and generate some Python and PHP code to demonstrate how to calculate the Signature. That is, I’ll show you how to reproduce the results in the documentation. I’ll use parameters (listed in Table 16-1) that draw from examples at the following location:

The parameters shown here would be used to access the object whose key is photos/puppy.jpg in the bucket named johnsmith. (Note that AWSAccessKeyId and AWSSecretAccessKey are not actually valid keys but are presented to illustrate the calculations.)

Table 16.1. Table 16-1. The Values Used in This Example Calculation for S3 Parameters
Setting Value
AWSSecretAccessKey uV3F3YluFJax1cknvbcGwgjvx4QpvB+leU8dUj2o
Expires 1175139620
Key photos/puppy.jpg
CanonicalizedResource /johnsmith/photos/puppy.jpg

The ­pseudo-­code for calculating the signature is (quoting from the documentation) as follows:

         StringToSign = ?HTTP-?VERB + "\n" +  ?Content-?MD5 + "\n" +  ?Content-?Type + "\n" + 
           Expires + "\n" + CanonicalizedAmzHeaders + CanonicalizedResource;
         Signature = ?URL-?Encode( Base64( ?HMAC-?SHA1( UTF-8-Encoding-Of( StringToSign ) ) ));

We’re told that the Signature based on the parameters in Table 16-1 should be rucSbH0yNEcP9oM2XNlouVI3BH4%3D. Let’s figure out how we can reproduce this signature in Python and PHP.

First, here is the Python code to calculate the Signature:

             import sha, hmac, base64, urllib
             AWSAccessKeyId = "0PN5J17HBGZHT7JJ3X82"
             AWSSecretAccessKey = "uV3F3YluFJax1cknvbcGwgjvx4QpvB+leU8dUj2o"
             Expires = 1175139620
             HTTPVerb = "GET"
             ContentMD5 = ""
             ContentType = ""
             CanonicalizedAmzHeaders = ""
             CanonicalizedResource = "/johnsmith/photos/puppy.jpg"
             string_to_sign = HTTPVerb + "\n" +  ContentMD5 + "\n" +  ContentType + "\n" + Â
               str(Expires) + "\n" + CanonicalizedAmzHeaders + CanonicalizedResource
             sig = base64.b64encode(Â
           , string_to_sign, sha).digest())
             print urllib.urlencode({'Signature':sig})

This produces the following:


Here’s some corresponding PHP code to calculate the Signature:

         # base64.encodestring
         # The hex2b64 function is excerpted from the Amazon S3 PHP example library.
             function hex2b64($str) {
               $raw = '';
               for ($i=0; $i < strlen($str); $i+=2) {
                 $raw .= chr(hexdec(substr($str, $i, 2)));
               return base64_encode($raw);
             require_once 'Crypt/HMAC.php';
             require_once 'HTTP/Request.php';
             $AWSAccessKeyId = "0PN5J17HBGZHT7JJ3X82";
             $AWSSecretAccessKey = "uV3F3YluFJax1cknvbcGwgjvx4QpvB+leU8dUj2o";
             $Expires = 1175139620;
             $HTTPVerb = "GET";
             $ContentMD5 = "";
             $ContentType = "";
             $CanonicalizedAmzHeaders = "";
             $CanonicalizedResource = "/johnsmith/photos/puppy.jpg";
             $string_to_sign = $HTTPVerb . "\n" . $ContentMD5 . "\n" . $ContentType . "\n" . 
               $Expires . "\n" . $CanonicalizedAmzHeaders . $CanonicalizedResource;
             $hasher =& new Crypt_HMAC($AWSSecretAccessKey, "sha1");
             $sig = hex2b64($hasher->hash($string_to_sign));
             echo 'Signature=',urlencode($sig);

Note that this PHP code depends on two PEAR libraries that need to be installed:

Unfortunately, after you get those libraries installed, I can’t recommend using the S3 sample code on a remote host, because it requires sending the secret over the wire. Run it on your own secure machine.

Once you have calculated the Signature, you can package the corresponding HTTP GET request:

Listing Buckets Using the REST Interface

Now that you understand the basics behind signing an Amazon S3 request, I’ll show you how to get a list of your S3 buckets. First I’ll show the code, and then I’ll offer an explanation:

            def listBuckets(AWSAccessKeyId,AWSSecretAccessKey):
                use the REST interface to get the list of buckets -- 
                without the use the Authorization HTTP header
                import sha, hmac, base64, urllib
                import time
                # give an hour for the request to expire (3600s)
                expires = int(time.time()) + 3600
                string_to_sign = "GET\n\n\n%s\n/" % (expires)
                sig = base64.b64encode(Â
        , string_to_sign, sha).digest())
                request = "" % \
                          (AWSAccessKeyId, expires, urllib.urlencode({'Signature':sig}))
                return request
            if __name__ == "__main__":
                AWSSecretAccessKey = '[SecretAccessKey]'
                print listBuckets(AWSAccessKeyId,AWSSecretAccessKey)

This code generates a URL of the following form that returns a list of buckets:


Note how we use some of the same parameters as in the previous example (AWSAccessKeyId, AWSSecretAccessKey, and Expires) and calculate the Signature with the same combination of SHA-1 hashing and Base64 encoding.