In the previous section, I showed you how to call web APIs that use XML-RPC and SOAP. That still leaves many APIs that fall under the name of REST—ones that look a lot like the Flickr REST interface. These APIs take some things we are familiar with from web browsers, such as going to a specific URL to get back some results and submitting HTML forms to make a query, but they have one important difference: instead of sending mostly HTML (which is directed at human consumption), you send primarily XML, a lingua franca of computer data exchange. For that reason, you should remind yourself of what you’ve learned from the previous chapters as you embark on a study of other REST APIs.
In the following sections, I’ll make sense of the world of web APIs, covering how to find out what APIs are available and then how to use a particular API. I’ll start my discussion by introducing perhaps the single most useful website about web APIs: Programmableweb.com. There’s a lot of information that is both readily apparent and waiting to be discovered in this treasure trove of data.
![]() | Note |
---|---|
Other directories of web services that are worth exploring are |
Programmableweb.com, started and maintained by Jon Musser, is an excellent resource for learning about what APIs are available, the basic parameters for the APIs, and the mashups that use any given API. Some noteworthy sections are as follows:
http://www.programmableweb.com/apis
is the “API dashboard” that lists the latest APIs to be registered and the most popular APIs being used in mashups.
ht
tp://www.programmableweb.com/apilist/bycat
lists APIs by categories. Understanding the various categories that have emerged is helpful for understanding for which fields of endeavor people are making APIs.
http://www.programmableweb.com/apilist/bymashups
lists APIs by how many times they are used in the mashups registered at Programmableweb.com.
I highly recommend a close and periodic study of Programmableweb.com for anybody wanting to learn about web APIs and mashups. Let me show some of the things you can learn from the website, based both on what is directly presented on the site and on data that John Musser has sent me. Although web APIs and corresponding mashups are rapidly changing, the data (and derived results), accurate for August 11, 2007, demonstrates some trends that I think will hold for a while yet.
The first thing to know is that of the 494 web APIs in the database, we get the distribution of number of APIs by protocol supported shown in Table 7-2. Note that some APIs are multiply represented.
Protocol | Number of APIs with Support |
REST 255 | |
SOAP 131 | |
XML-RPC 19 | |
JavaScript 30 | |
Other 16 |
Some other observations drawn from the database are as follows:
Ninety-three APIs have WSDL files associated with them.
Of the 131 APIs that support SOAP, 42 also support REST—leaving 89 that support SOAP but not REST. Eighty-eight APIs support only SOAP.
XML-RPC is the only choice for nine APIs.
JavaScript is listed as the exclusive protocol for 25 APIs.
The following are my conclusions based on this data:
REST is the dominant mode of presenting web APIs, but a significant number of APIs exist where your only choice is SOAP.
There are a relatively small number of APIs listing XML-RPC as the only choice of protocol.
It’s therefore useful to know how to use SOAP and XML-RPC, even if they are not your first choice.
![]() | Note |
---|---|
A large number of APIs list JavaScript as a protocol. I’ll cover such APIs in the next chapter. |
Table 7-3 lists the top 20 APIs on Programmableweb.com by mashup count and also lists the type of protocols supported by the API.
API Name | Number of Mashups | Protocols Support |
Google Maps 1110 JavaScript | ||
Flickr 243 REST, SOAP, XML-RPC | ||
Amazon E-Commerce Service 174 REST, SOAP | ||
YouTube 149 REST, XML-RPC | ||
Microsoft Virtual Earth 97 JavaScript | ||
Yahoo! Maps 95 REST, JavaScript, Flash | ||
411Sync 89 RSS input over HTTP, SOAP | ||
eBay 89 SOAP, REST | ||
del.icio.us 83 REST | ||
Google Search 79 SOAP | ||
Yahoo! Search 78 REST | ||
Yahoo! Geocoding 66 REST | ||
Technorati 40 REST | ||
Yahoo! Image Search 31 REST | ||
Yahoo! Local Search 30 REST | ||
Last.fm 28 REST | ||
Google home page 27 JavaScript | ||
Google Ajax Search 24 JavaScript | ||
Upcoming.org 21 REST | ||
Windows Live Search 21 SOAP | ||
Feedburner 21 REST |
What can you do with this information? To learn about popular APIs, one approach would be to go down the list systematically to figure out how each works. Indeed, through the rest of the book, I’ll cover many of the APIs in the table. The Flickr API is the second most used API in mashups and is a main subject throughout this book. I’ll cover the JavaScript-based maps (first and foremost Google Maps but also Yahoo! Maps and Virtual Earth) first in Chapter 8 and then in depth in Chapter 13. I’ll cover the Yahoo! Geocoding API extensively also in Chapter 13. I’ll cover various search APIs (Google Search, Yahoo! Search, Yahoo! Image Search, and Windows Live Search) in Chapter 19. Finally, I’ll cover the del.icio.us API in Chapter 14 on social bookmarking. Indeed, the fact that I cover many APIs clustered by subject matter indicates that it is a natural way to think about APIs.
YouTube is probably the most famous video-sharing site on the Web—and it also uses tagging as one way of organizing content. The YouTube API is documented at http://www.youtube.com/dev
.
The YouTube API supports both a REST interface and an XML-RPC interface. The examples I give in this section use the REST interface. You can find a list of methods at http://?www.youtube.com/dev_docs
.
To use the API, you need to set up your own development profile; see http://www.youtube.com/
?my_profile_dev
.
An interesting feature of the registration process is that you enter your own secret (instead of having one set by YouTube). When you submit your profile information, you then get a “developer ID.” The following are some sample calls. To get the user profile for a user (for example, rdhyee
), you do an HTTP GET
on the following:
http://www.youtube.com/api2_rest?method=youtube.users.get_profile&dev_id={youtube-key}&user=rdhyee
YouTube will send you a response something like this:
<?xml version="1.0" encoding="utf-8"?> <ut_response status="ok"> <user_profile> <first_name>Raymond</first_name> <last_name/> <about_me/> <age>40</age> <video_upload_count>2</video_upload_count> <video_watch_count>102</video_watch_count> [....] </user_profile> </ut_response>
To get the list of rdhyee
’s favorite videos, use this:
http://www.youtube.com/api2_rest?method=youtube.users.list_favorite_videos&dev_id={youtube-key}&user=rdhyee
To get details of a video with an ID of XHnE4umovw4
, use this:
http://www.youtube.com/api2_rest?method=youtube.videos.get_details&dev_id={youtube-key}&video_id=XHnE4umovw4
To get videos for the tag HolidayWeekend
, use this:
http://www.youtube.com/api2_rest?method=youtube.videos.list_by_tag&dev_id={youtube-key}&tag=HolidayWeekend&page=1&per_page=100
There’s more to the API, but you can get a feel for how it works through these examples.
![]() | Caution |
---|---|
Expect the YouTube API to evolve into something more like the rest of Google’s APIs: |
The Atom Publishing Protocol (APP), a companion to the Atom Syndication Format (Atom 1.0) described in Chapter 2, represents the next generation of the blogging APIs. APP is currently a draft IETF proposal:
http://tools.ietf.org/wg/atompub/draft-ietf-atompub-protocol/
which is linked from here:
http://tools.ietf.org/wg/atompub/
One of APP’s biggest supporters thus far has been Google, which has implemented GData, which is based on Atom 1.0 and RSS 2.0 feeds, combined with APP. GData, which incorporates Google-specific extensions, is the foundation of the APIs for many of its services, including Google Base, Blogger, Google Calendar, Google Code Search, and Google Notebook:
http://code.google.com/apis/gdata/index.html
The API for Blogger is documented here:
http://code.google.com/apis/blogger/developers_guide_protocol.html
In the following sections, you’ll learn the basics of the API for Blogger as a way of understanding GData and APP in general.
The first thing you need to have is a Google account to use Blogger. If you don’t have one, go to the following location to create one:
https://www.google.com/accounts/NewAccount
Next, with a Google account, you obtain an authorization token. One way to do so is to follow the procedure for ClientLogin (one of two Google authorization mechanisms) detailed here:
http://code.google.com/apis/blogger/developers_guide_protocol.html#client_login
Specifically, you make an HTTP POST
request to the following location:
https://www.google.com/accounts/ClientLogin
The body must contain the following parameters (using the
application/?
x-www-form-urlencoded
content type):
Email
: Your Google email (for example,
raymond.yee@gmail.com
)
Password
: Your Google password
source
: A string of the form
companyName-applicationName-versionID
to identify your program (for example,
mashupguide.net-Chap7-v1
)
service
: The name of the Google service, which in this case is blogger
Using the example parameters listed here, you can package up the authorization request as the following curl
invocation:
curl -v -X POST -d "Passwd={passwd}&source=mashupguide.net-Chap7-v1&Email=raymond.yee%40gmail.com&service=blogger" https://www.google.com/accounts/ClientLogin
If this call succeeds, you will get in the body of the response an Auth
token (of the form Auth=[AUTH-TOKEN])
. Retain the AUTH-TOKEN
for your next calls.
If you don’t have a blog on Blogger.com, create one here:
http://www.blogger.com/create-blog.g
Now figure out your Blogger user ID by going to and noting the URL associated with the View link (beside the Edit Profile link):
Your View link will be of the following form:
http://www.blogger.com/profile/{userID}
For example, my blog profile is as follows:
http://www.blogger.com/profile/13847941708302188690
So, my user ID is 13847941708302188690
.
Note that Blogger lists user blogs in a user’s profile:
http://www.blogger.com/profile/{userID}
From an API point of view, you can retrieve an Atom feed of a user’s blog here:
http://www.blogger.com/feeds/{userID}/blogs
That the list of blogs is an Atom feed and not some custom-purpose XML (such as that coming out of the Flickr API) is useful. You can look at the feed of your blogs to pull out the blog ID for one of your blogs. For instance, one of my blogs is entitled “Hypotyposis Redux” and is listed in the feed with the following tag:
<id>tag:blogger.com,1999:user-354598769533.blog-5586336</id>
From this you can determine its blog ID of 5586336
. With this blogID
, you can now send HTTP GET
requests to retrieve an Atom feed of posts here:
http://www.blogger.com/feeds/{blogID}/posts/default
For example:
Let’s now look at how to create a new post. A central design idea behind the Atom
Publishing Protocol and hence its derivatives—GData generally and the Blogger API
specifically—is the notion of a uniform interface based on the
standard HTTP methods. At this point, it’s useful to refer to the “Methods
Definition” of the HTTP 1.1 specification (http://www.w3.org/Protocols/
?rfc2616/rfc2616-sec9.html
), specifically the definition for
POST
:
The POST method is used to request that the origin server accept the entity enclosed in the request as a new subordinate of the resource identified by the Request-URI in the Request-Line.
You may be surprised to read this definition of the POST
method, considering, for instance, how POST is used for every single SOAP 1.1 call bound to HTTP—whether the call is for retrieving a simple piece of information, creating a new resource, or deleting it.
Let’s see how to create an HTTP POST
request to create a new blog entry and confirm how the process follows the earlier definition:
Create a blog entry formatted as an <entry>
Atom element, something like this:
<entry xmlns='http://www.w3.org/2005/Atom'> <title type='text'>Using Blogger to demo APP</title> <content type='xhtml'> <div xmlns='http://www.w3.org/1999/xhtml'> <p>This message is being created from invoking the blogger APP-based API.</p> <p>This process is documented at <a href="http://code.google.com/ apis/blogger/developers_guide_protocol.html#CreatingPosts">Blogger Data API -- Creating Posts</a></p> </div> </content> <author> <name>Raymond Yee</name> <email>raymond.yee@gmail.com</email> </author> </entry>
Save the file, say with the filename blogger.message.1.xml
.
Issue the following curl
invocation to POST
the contents of your file to http://?
www.blogger.com/feeds/{blogID}/posts/default
—which is the feed of all your entries for the blog—to create a new entry for the blog (a “new subordinate of the resource identified by the Request-URI”).
curl -X POST -v --data-binary "@blogger.message.1.xml" -H "Content-Type:application/atom+xml " -H "Authorization: GoogleLogin auth=[AUTH-TOKEN]" http://www.blogger.com/feeds/{blogID}/posts/default
If things go fine, you’ll get an HTTP 201 Created code and an <entry>
holding the new post. This <entry>
tells you things such as the post ID of your new entry. The response will look like the following:
HTTP/1.1 201 Created Content-Type: application/atom+xml; charset=UTF-8 Cache-Control: ?max-?age=0, ?must-?revalidate, private Location: http://www.blogger.com/feeds/5586336/posts/default/409227349217351 7704 Content-Location: http://www.blogger.com/feeds/5586336/posts/default/4092273 492173517704 Transfer-Encoding: chunked Date: Sat, 25 Aug 2007 14:38:49 GMT Server: GFE/1.3 <?xml version='1.0' encoding='UTF-8'?><?xml-stylesheet href="http://www.b logger.com/styles/atom.css" type="text/css"?><entryxmlns='http://www.w3.o rg/2005/Atom'><id>tag:blogger.com,1999:blog-5586336.post-4092273492173517 704</id><published>2007-08-25T07:38:00.001-07:00</published><updated>2007 -08-25T07:38:49.607-07:00</updated><title type='text'>Using Blogger to de mo APP</title><content type='html'><br /> <div xmlns='http:// www.w3.org/1999/xhtml'><br /> <p>This message is being created from invoking the blogger APP- based API.</p><br /> <p>This process is documented at <a href='http://code.google.com/apis/blogger/developers_guide_proto col.html#CreatingPosts'>Blogger Data API -- Creating Posts</a>&l t;/p><br /> </div><br /> </content><link rel='al ternate' type='text/html' href='http://hypotyposis.blogspot.com/2007_08_0 1_archive.html#4092273492173517704' title='Using Blogger to demo APP'/><l ink rel='replies' type='applicati* Connection #0to host www.blogger.com l eft intact* Closing connection #0
In this example, the POST request created a new blog entry with a post ID of 40922734921735177
.
You can update your blog entry using an HTTP PUT
request, in accordance to the HTTP 1.1 specification that states the following:
The PUT method requests that the enclosed entity be stored under the supplied Request-URI. If the Request-URI refers to an already existing resource, the enclosed entity SHOULD be considered as a modified version of the one residing on the origin server.
Let’s package this request for curl
after first creating an updated message in the blogger.message.2.xml
file:
curl -X PUT -v --data-binary "@blogger.message.2.xml" -H "Content-Type:application/atom+xml " -H "Authorization: GoogleLogin auth=[AUTH-TOKEN]" http://www.blogger.com/feeds/{blogID}/posts/default/{postID}
If you are unfamiliar with using the HTTP PUT
method, you’re hardly alone. As mentioned in Chapter 6, there is little support for it. (Remember, for instance, that the HTML forms define the GET
and POST
methods.) Recognizing that PUT
might not be supported by the client doing the entry update (or that firewalls might block PUT
requests), you can tunnel the PUT
request through a POST
request like so:
curl -X POST -v --data-binary "@blogger.message.2.xml" -H "X-HTTP-Method-Override:PUT" -H "Content-Type: application/atom+xml " -H "Authorization: GoogleLogin auth=[AUTH-TOKEN]" http://www.blogger.com/feeds/{blogID}/posts/default/{postID}
You can use the HTTP DELETE
method to delete an entry but send that request to the URL of the entry itself. As a curl
invocation, the request looks like this:
curl -X DELETE -v -H "Content-Type: application/atom+xml " -H "Authorization: GoogleLogin auth=[AUTH-TOKEN]" http://www.blogger.com/feeds/{blogID}/posts/default/{postID}
As with updating a blog entry, you can tunnel a DELETE
request through an HTTP POST
request using an “X-HTTP-Method-Override: DELETE” request header.
Now that you have seen how to use the Blogger API to retrieve feeds of blogs and blog entries, create new blog entries, update an entry, and delete an entry, you should notice how all these actions are performed while hewing closely to HTTP methods as they are actually defined in the HTTP specification. This pattern of using the HTTP methods as the fundamental methods of the API, in fact, repeats itself in all the APIs that are based on the Atom Publishing Protocol and therefore GData. Thus, the uniform interface of GData is the full collection of standard HTTP methods.