Through a number of scenarios in which I describe some problems that are particularly suited to be solved through mashups, I’ll show how you can use ProgrammableWeb to figure out what mashups might already exist to solve these problems. Often, there won’t be a perfect—or even a good—solution, but the existing ones show you what is possible, what is easy to do, and what might be difficult to do. Moreover, by using ProgrammableWeb, you can immediately see what APIs are being used, as well as what mashups have gotten a following in the community of ProgrammableWeb readers.
One scenario is to develop a system to handle book-related information in all the ways you might deal with books. Such a system would track books that
you own,
you have out from the various libraries and when they are due,
you’ve lent to others or borrowed from others,
you would like to read one day,
you would buy if they dropped below a certain price,
you’d buy from a used bookstore,
you have just been published,
you have just shown up your local libraries, or
you cite in your writing.
Moreover, you probably want to keep some of this information private, some available only to friends, and some pieces of information completely public.
For my own books, I currently use a mishmash of web sites, desktop applications, and web applications to track books—all of which I would like to mash together:
Amazon.com to look up and buy new books
Amazon.com wishlists to store books that I would like to buy, borrow, or just ponder
http://worldcatlibraries.org/
to locate the book in my local library
The online card catalogs of my local libraries (that of Berkeley Public Library and UC Berkeley)
LibraryThing, a web site where I often enter books I’m reading and follow what others are reading
Zotero (http://zotero.org
), a Firefox extension that I use to track references
Bn.com and other online bookstores
http://www.half.ebay.com/
to buy used books
What would I like to a complete book mashup to do? Lots of things—but some scenarios are as follows:
If I place a book in my Amazon.com wishlist, I want to be informed whenever that book becomes available at any of the bookstores for which I have borrowing privileges.
I want to synchronize books that I have listed in Zotero and LibraryThing.
I want the due dates of all my library books to show up on my Google Calendar.
I want to be able to format any subset of books from anywhere into a citation for the bibliography I’m compiling.
In some ways, the problem I’d like to solve is an elaboration of the problem I first discussed in Chapter 1. There I talked about the LibraryLookup bookmarklet that shows you how to find a library book in your local library catalog based on an ISBN. Here, I’d like my book information to flow easily among all the places I am referring to books.
I don’t actually expect any existing mashup to bring them altogether—partly because mashups generally aren’t that all-encompassing yet and partly because the mix of elements I want mashed up is rather idiosyncratic. But let’s look at what mashups are out there, what they bring together, and whether we can mash up the mashups themselves.
Let’s use ProgrammableWeb to help us to find possible solutions. One approach is to do a full-text search for book among the mashup profiles, sorting the results by popularity:
http://www.programmableweb.com/mashups/directory/1?q=book&sort=popular
Another approach is to use the tags. You can look through the tag cloud here to get a sense of popular book-related tags:
http://www.programmableweb.com/search
You’ll see a link listed to the tag books
. I recommend sorting by popularity first:
http://www.programmableweb.com/tag/books/1?sort=popular
and then by date to see the latest developments among mashups tagged with books
:
http://www.programmableweb.com/tag/books/1?sort=date
I recommend looking through the results and reading the descriptions of each mashup. Try some. You’ll get a feel for the range of possibilities among book-related mashups.
Here, I’ll highlight ones that stand out in my viewing. One question to ask while looking through the mashup profiles is, what are the important APIs involved in these mashups? It would be nice to have ProgrammableWeb return the list of APIs involved in a given set of mashups sorted by the number of times it is used. In this case, such a feature would make it easy to see what the most commonly used APIs for mashups tagged with books
are. However, even with a casual glance through this:
http://www.programmableweb.com/tag/books/1?sort=popular
you’ll see several mentions of the Amazon.com E-Commerce Service API among the various APIs:
http://www.programmableweb.com/api/amazon-ecommerce
I’m sure you won’t be surprised to see the Amazon.com web services show
up in this context, given Amazon.com’s prominence in two areas: online book
retailing and web APIs. It’s still helpful, though, to see how people have used
the Amazon.com APIs in book-related mashups. You can use the advanced search on
ProgrammableWeb (see the “Using the Advanced Search for Mashups and APIs”
sidebar) to narrow down the list of mashups tagged with books
to ones that
use the Amazon.com E-Commerce Service API:
http://www.programmableweb.com/tag/books/1?apis=Amazon+eCommerce&sort=popular
This ability to focus a search on a specific API (or a combination of APIs) in conjunction with a specific mashup tag can often demonstrate the capabilities of an API for a specific context more vividly than reading the documentation for the API!
Let’s now look at a few of the books
-tagged mashups and how they
can help in my quest to bring together all aspects of my book-related
activities.
BlueOrganizer is a Firefox extension that attempts to recognize when a web page is referring to items of certain categories such as wine, music, stocks, and—most important in this context—books:
http://www.programmableweb.com/mashup/blueorganizer
This ProgrammableWeb profile links to the URL for BlueOrganizer:
specifically:
http://www.adaptiveblue.com/smartlinks_books.html
From reading the documentation for the plug-in, or actually installing it and trying it, you’ll see that it recognizes books and gives you a button to take a variety of actions, including the following:
Adding the book to the Amazon.com wishlist
Adding the book to LibraryThing or Shelfari (a LibraryThing competitor)
From studying BlueOrganizer, you can learn about web sites that might provide useful sources of book information such as AbeBooks, which advertises an API in its affiliate program (but you need to contact them: http://www.abebooks.com/docs/AffiliateProgram/
).
Although I am a fan of LibraryThing, I follow the development of other web sites that allow readers to track books that they read and share that information with friends or the world. The New York Times covered this genre here:
http://www.nytimes.com/2007/03/04/business/yourmoney/04novel.html
Although I knew about Shelfari (http://www.shelfari.com
) and Goodreads (http://www.goodreads.com
) before consulting ProgrammableWeb, I learned about GuruLib and BookBump from ProgrammableWeb:
One thing that keeps me from investing too heavily in these sites is the struggle of how to move my book data in and out of any of these sites. For any given site, I look for APIs that help in that regard as well as any feeds that might allow users to easily import and export data.
Here are some things that we learned by thinking through how to create a full-featured book mashup with the help of ProgrammableWeb:
When it comes to book-related information, Amazon.com is a good API to start with in regard to book searching. The API gives you access to the Amazon.com wishlist (as you will see in Chapter 17).
Don’t expect all APIs of interest to be listed on ProgrammableWeb. As of writing, there is no mention of Zotero, WorldCat, and LibraryThing—even though they are all programmable to one degree or another.
Lots of web sites I use don’t have APIs at all, such as Bn.com and my local library catalogs. We are left with the question of how to deal with those sites. Should we screen-scrape the sites?
There are other angles that won’t be covered by looking only at
ProgrammableWeb. For example, for the latest books at my library, I have to
look at http://www.berkeley-public.org/ftlist
for recent arrivals.
There is no API.
Let’s move from tracking books to tracking plane tickets. Suppose I want to buy a round-trip ticket between San Francisco and New York City. I know roughly when I want to travel but have some flexibility in terms of exactly when I can leave and return (within a day or two) and which airlines I can take. I’m planning far ahead of time to try to get the best price. However, I really don’t want to have to leave before 8 a.m. or arrive in New York City after 9 p.m.
For a long time, it would be difficult for me as a typical consumer to be able to monitor over periods of weeks or months airfares for trips that meet such criteria so that I could wait for a good time to buy. However, some of the newer travel sites are giving users the ability to perform increasingly sophisticated searches, filter results by such criteria as the time of day of departure, and receive e-mail alerts for canned searches. Now, given that there are even travel web sites with APIs, I’m wondering whether I could use these APIs to get closer to finding the plane tickets at the prices I want.
We can use ProgrammableWeb to look through a collection of travel-tagged mashups:
http://www.programmableweb.com/tag/travel
John Musser wrote a recent analysis of the travel APIs:
http://blog.programmableweb.com/2007/10/29/5-travel-apis-from-comparison-to-booking/
You can search for travel-tagged APIs here:
http://www.programmableweb.com/apis/directory/1?q=travel
but if you limit the display to APIs tied to any mashup profiles (and sorting by popularity), like so:
http://www.programmableweb.com/apis/directory/1?q=travel&sort=mashups
you quickly find only two APIs at the time of writing:
Since the Yahoo! Travel API is focused on travel plans made by users on the Yahoo! Travel web site, and not on the purchase of airplane tickets, we’ll focus then on the Kayak Search API:
http://www.kayak.com/labs/api/search/
Kayak (http://www.kayak.com/
) is a web site like Expedia and Travelocity that allows users to search for flights. Given that APIs for travel sites are a new concept, I wasn’t surprised that there were few mashups listed as using the Kayak Search API (http://www.programmableweb.com/api/kayak/mashups
). Kayak’s deals from cell phones profiled here made me think of alternate interfaces to Kayak’s travel information:
http://www.programmableweb.com/mashup/kayak.com-deals-from-cell-phone
One mashup that I have yet to see is one of Kayak with Google Calendar. When I schedule flights, it’s useful to see what else I have going on in my personal schedule. A Kayak/Google Calendar mashup could present possible flights as a Google calendar that I could juxtapose with my personal calendar. The mashup might even read my personal schedule to filter out prospective flights to begin with. (See Chapter 15 for how to use the Google Calendar API.)
Apart from connecting Kayak to alternative interfaces such as cell phones and calendars, a mashup of Kayak could allow you to conduct a more thorough search through the complicated combinations of parameters possible when flying. I found that manually varying parameters such as departure dates and return dates and keeping the best deals in my head got rather tiring after five to ten tries. I suspect that a mashup of the Kayak Search API and a smart search algorithm could possibly find better flights than I could find manually.
Real estate–oriented APIs and mashups promise to make home buying a bit easier and maybe more fun. Specifically, let’s look at how we might use a variety of web feeds and APIs to track houses that come on the market in a given area.
You can start on ProgrammableWeb by searching for mashups tagged with realestate
and sorting the results by popularity:
http://www.programmableweb.com/tag/realestate/1?view=desc
More to the point, by searching for APIs tagged with realestate
and listing them by popularity by going here:
http://www.programmableweb.com/apitag/realestate/1?sort=mashups
you find two relevant APIs for American real estate:
Zillow (http://www.zillow.com/
) focuses on providing estimates of home valuations and details of properties, while also listing homes for sale and that have been recently sold. Trulia (http://www.trulia.com/
) aggregates listings of homes for sale. Note that although the Trulia API (http://developer.trulia.com/
) doesn’t currently return any individual listings, you can use Trulia feeds to access some of the listings. For example, the following is an RSS 2.0 feed of some of the current properties available in Berkeley, California:
http://www.trulia.com/rss2/CA/Berkeley/
Currently, I do not know of what seems to be an obvious combination—a mashup of the Zillow and Trulia APIs, one that, for instance, would compare the sale price of houses listed for sale on Trulia with what Zillow estimates to be the value of the house. ProgrammableWeb doesn’t list any such mashup:
http://www.programmableweb.com/mashups/directory/1?apis=trulia%2Czillow
Something I learned by looking through the realestate
mashups is that Google Base is an interesting source of real estate data. Take a look at mashups tagged by realestate
using the Google Base API:
http://www.programmableweb.com/tag/realestate?apis=Google+Base
In Chapter 4, you learned how to use Yahoo! Pipes to pull together various news feeds into a single feed. In this section, I’ll cover how to plot those current events on a map.
I often read about places in the world for which I have only the vaguest idea where they are located. Online maps certainly make it easy to look places up now. But much like how Housingmaps.com helps with visualizing real estate on a map, perhaps displaying news on a map of the world could have the similar benefits.
Let’s see what ProgrammableWeb has to say about mashups of news and maps. The news
tag is a popular tag for mashups on ProgrammableWeb.
http://www.programmableweb.com/tag/news/1?view=desc
When you look through this section, you’ll see several mashups of interest:
http://www.programmableweb.com/mashup/bbc-news-map
points to
a now-defunct mashup that mapped BBC News items about the United Kingdom on
a map.
http://www.programmableweb.com/mashup/ap-national-news-google-maps
points to http://www.81nassau.com/apnews/
, which displays items from a choice of Associated Press feeds (including national news, sports, and business) on a Google map.
http://www.programmableweb.com/mashup/mapified-rss
points to
http://o.gosselin.free.fr/Projects/MapifiedRss/
, which maps
to Google Maps entries from one of the preconfigured RSS feeds (for example,
Reuters, Associated Press top headlines, or Google News) or from the URL of a feed
entered the user.
Seeing these mashups reminded me how easy it is now to display feeds that contain geographic locations on a map. Let’s use Yahoo! Pipes, which you have already learned how to use in Chapter 4. The key to georeferencing a feed so that it can be displayed on a map is the Location Extractor Operator in Yahoo! Pipes:
http://pipes.yahoo.com/pipes/docs?doc=operators#LocationExtractor
Another thing you need to know is that Yahoo! Pipes is able to output KML that can then be displayed on Google Earth and Google Maps. (Chapter 13 contains more details about KML.)
I constructed a Yahoo! pipe that takes as input a URL to a feed to be georeferenced:
http://pipes.yahoo.com/raymondyee/locationextractor
The default value for this URL is that for the New York Times International News feed:
http://www.nytimes.com/services/xml/rss/nyt/International.xml
The KML output for this default feed is as follows:
or as follows:
You can display this KML feed on a Google map, like so:
http://maps.google.com/maps?f=q&hl=en&geocode=&time=&date=&ttype=&q=http:%2F%2F pipes.yahoo.com%2Fpipes%2Fpipe.run%3FInputURL%3Dhttp%253A%252F%252Fwww.nytimes.com%252F services%252Fxml%252Frss%252Fnyt%252FInternational.xml%26_id%3DcInT4D7B3BGMoxPNiXrL0A% 26_render%3Dkml&ie=UTF8&ll=28.921631,53.4375&spn=150.976999,360&z=2&om=1
or like so:
Since in this section we’re looking at mapping, it’s helpful to look at the mapping vertical market coverage on ProgrammableWeb: