Pro Web 2.0 Mashups
   Next

Pro Web 2.0 Mashups

Remixing Data and Web Services

Dedication

For Laura, the love of my life

Table of Contents

Copyright
Pro Web 2.0 Mashups
About the Author
About the Technical Reviewer
Acknowledgments
Introduction
The Web 2.0 Movement
Overall Flow of the Book
Intended Audience
Updates
I. Remixing Information Without Programming
1. Learning from Specific Mashups
Looking for Patterns in Mashups
Housingmaps.com
What Is Being Combined?
Why Are the Constituent Elements Being Combined? What’s the Problem Being Solved?
Where Is the Remixing Happening?
How Are These Elements Being Combined?
Comparable Mashups
Google Maps in Flickr
What Is Being Combined?
Why Are the Constituent Elements Being Combined? What’s the Problem Being Solved?
How Are These Elements Being Combined?
Comparable Mashups
LibraryLookup Bookmarklet
Configuring a LibraryLookup Bookmarklet
Invoking the LibraryLookup Bookmarklet
How Does This Mashup Work?
How Can This Mashup Be Extended?
Comparable Mashups
Tracking Other Mashups
Summary
2. Uncovering the Mashup Potential of Web Sites
What Makes Web Sites and Applications Mashable
Ascertaining the Fundamental Entities of the Web Site
Public APIs and Existing Mashups
Use of Ajax
Embedded Scriptability
Browser Plug-­Ins
Getting Data In and Out of the Web Site
The Community of Users and Developers
Mobile and Alternative Interfaces and the Skinnability of the Web Site
Documentation
Is the Web Site Run on Open Source?
Intellectual Property, Reusability, and Creative Commons
Tagging, Feeds, and Weblogging
URL Languages of Web Sites
Some Mashups Briefly Revisited
Flickr: The Fundamentally Mashup-­Friendly Site
Resources in Flickr
Users and Photos
Data Associated with an Individual Photo
Tags
User’s Archive: Browsing Photos by Date
Sets
Collections
Favorites
A User’s Popular Photos
Contacts
Groups
Account Management
Browsing Through Flickr
Search
Geotagged Photos in Flickr
The Flickr Organizer
Recent Activities
Mailing Interfaces
Interfacing to Weblogs
Syndication Feeds: RSS and Atom
Mobile Access
Third-Party Flickr Apps
Creative Commons Licensing
Cameras
The Mashup-­by-URL-Templating-and-Embedding Pattern
Google Maps
URL Language of Google Maps
Viewing KML Files in Google Maps
Connecting Yahoo! Pipes and Google Maps
Other Simple Applications of the Google Maps URL Language
Amazon
Amazon Items
Lists
Tags
Subject Headings
del.icio.us
Screen-Scraping and Bots
Summary
3. Understanding Tagging and Folksonomies
Tagging in Flickr
Tags in Flickr
How Tags Are Used in Practice
Creating Your Own Tags
Syntax of Tags in Flickr
Potential Weaknesses of Tags
Singular and Plural Forms of Tags in Flickr
Hacking the Tagging System: Geotagging and Machine Tags
Interesting Apps Using Flickr Tags
Tagging in del.icio.us
Mechanics of Adding Tags in del.icio.us
Dealing with Case and Multiword Phrases
Getting More Information
Gathering Content Through Tags in Technorati
Searching Technorati with Tags
How Technorati Finds Tags on the Web
Word Inflections and Syntactic Constraints in Technorati Tags
Using Tags to Mash Up Flickr and del.icio.us
Other Systems That Use Tagging
Relationship of Tags to Formal Classification Schemes
Summary
4. Working with Feeds, RSS, and Atom
What Are Feeds, and Why Are They Important?
RSS 2.0
RSS 1.0
Atom 1.0
Extensions to RSS 2.0 and Atom 1.0
Feeds from Flickr
Flickr Feed Parameters
Examining the Flickr Feeds
Exchange Formats Other Than RSS and Atom
Feeds from Other Web Sites
Finding Feeds and Feed Autodiscovery
Feeds from Weblogs
Wikipedia Feeds
Google and Yahoo! News
News Aggregators: Showing Flickr Feeds Elsewhere
Validating Feeds
Scraping Feeds Using GUI Tools
Remixing Feeds with Feedburner
Remixing Feeds with Yahoo! Pipes
A Simple First Pipe with Yahoo! News
Google News and Refactoring Pipes
Wikinews and NY Times: Filtering Feeds
Pulling the Feeds Together
Summary
5. Integrating with Blogs
Integration Scenarios for Blogs
Sending Flickr Pictures to Blogs
Configuring Flickr for Integration with Blogs
Blogging a Flickr Picture
How Does the Flickr Blog Integration Work?
Desktop Blogging Tools
Combining Feeds and Blogging to Generate Feedback Flows
Flock: Bringing Together Blogs and Flickr
RSD: Discoverability of Blog APIs
Linkbacks
Wiki Integration at an Early Stage
Summary
II. Remixing a Single Web Application Using Its API
6. Learning Web Services APIs Through Flickr
An Introduction to the Flickr API
What Does This XML Response Mean?
What Can You Do with the XML Response?
API Documentation, Community, and Policy
Terms of Use for the API
Using the Flickr API Explorer and Documentation
Calling a Basic Flickr API Method from PHP
HTTP Clients
A Refresher on HTTP
XML Processing
Pulling It All Together: Generating Simple HTML Representations of the Photos
Where Does This Leave Us?
The Flickr API in General
Using flickr.reflection Methods
Querying the Flickr Reflection Methods with PHP
Request and Response Formats
Flickr Authorization
Why Passing Passwords Around Doesn’t Work Too Well
Authorization for Web Apps
Using Flickr API Kits
PEAR::Flickr_API
phpFlickr
Phlickr
Limitations of the Flickr API
Summary
7. Exploring Other Web APIs
XML-RPC
What’s Happening on the Wire?
Using Wireshark and curl to Analyze and Formulate HTTP Messages
Parsing XML-­RPC Traffic
SOAP
The Dream: Plug-­and-Go Functionality Through WSDL and SOAP
geocoder.us
Amazon ECS
The Flickr API via SOAP
Learning About Specific Web APIs
Programmableweb.com
YouTube
GData and the Blogger API
Using the Blogger API As a Uniform Interface Based on HTTP Methods
Summary
8. Learning Ajax/JavaScript Widgets and Their APIs
What You Need to Know
What Difference Does Ajax Make?
Learning Firebug, DOM Inspector, and JavaScript Shell
Using the DOM Inspector
Using the Firebug Extension for Firefox
Using the JavaScript Shell
Working with JavaScript Libraries
YUI Widgets
Using the YUI Calendar
Installing YUI on Your Host
Learning Google Maps
Accessing Flickr via JavaScript
Using Greasemonkey to Access New York Times Permalinks
Learning More About JavaScript and Ajax
Summary
III. Making Mashups
9. Moving from APIs and Remixable Elements to Mashups
Getting Oriented to ProgrammableWeb
User-Generated Data in ProgrammableWeb
Can Any Directory of Mashups Keep Up?
Learning About the Overall Mashup Scene
Directory of Mashups
Using Feeds to Track Mashups
Using Tags to Describe Mashups
API and Mashup Verticals
Looking at a Specific Mashup Profile
Going from a Specific API to Mashups
Sample Problems to Solve Using Mashups
Tracking Interesting Books
Knowing When to Buy Airplane Tickets
Finding That Dream House
Mapping Breaking News
Summary
10. Creating Mashups of Several Services
The Design
Background: Geotagging in Flickr
Background: XMLHttpRequest and Containing Libraries
Using XMLHttpRequest Directly
Using the YUI Connection Manager
Building a Server-­Side Proxy
What Happens with XHR and Direct API Calls?
Building a Server-­Side Script for Geolocated Photos
Building a Simple Client-­Side Frame
Reading and Writing Elements
Handling Simple Events to Connect Form Input and Display Calculations
Hooking the Client-­Side Framework to Flickr
Writing a URL for Querying flickrgeo.php
Using XHR via the YUI Connection Manager to Read the JSON
Converting the JSON to HTML
Mashing Up Google Maps API with Flickr
Setting Up a Basic Google Map
Making the Map Respond to Changes in the Viewport of the Map
Bringing Together the Flickr and GMap Code
Wiring Up the Bounding Box of the Google Map
Making the Pictures Show Up in the Map
Google Mapplet That Shows Flickr Photos
Summary
11. Using Tools to Create Mashups
The Problem Mashup Tools Solve
What You Are Making in This Chapter
Making the Mashup: A ­Step-­by-Step Example
Familiarizing Yourself with the Google Mashup Editor
Reading and Displaying a Feed (Simple Template)
Introducing a Custom Template
Using Yahoo! Pipes to Access Flickr
Displaying Flickr Photos Using <gm:map>
Adding JavaScript to the Mashup
How to Persist Feeds and Use Tabs
The Final Product: Showing the Saved Entries on a Map
Analysis of ­Trade-­Offs in Using GME and Yahoo! Pipes
Other Mashup Tools
Summary
12. Making Your Web Site Mashable
Why Make Your Web Site Mashable?
Using Techniques That Do Not Depend on APIs
Use a Consistent and Rich URL Language
Use W3C Standards to Develop Your Web Site
Pay Attention to Web Accessibility
Consider Allowing Users to Tag Your Content
Make Feeds Available
Make It Easy to Post Your Content to Blogs and Other Web Sites
Encourage the Sharing of Content with Explicit Licenses
Develop Extensive Import and Export Options for User Content
Study How Users Remix Your Content and Make It Easier to Do So
Creating a ­Mashup-­Friendly API
Learn From and Emulate Other APIs
Keep in Mind Your Audiences for the API
Make Your API Easy to Learn
Test the Usability of Your API
Build a Granular, Loosely Coupled Architecture So That Creating an API Serves You As Much As It Does Others
Embrace REST But Also Support SOAP and ­XML-­RPC If You Can
Consider Using the Atom Publishing Protocol As a Specific Instantiation of REST
Encourage the Development of API Kits: Third Party or ­In-­House
Support Extensive Error Reporting in Your APIs
Accept Multiple Formats for Output and Input
Support UI Functionality in the API
Include a Search API for Your Own Site
Version Your API
Foster a Community of Developers
Don’t Try to Be Too Controlling in Your API
Consider Producing a ­Service-­Level Agreement (SLA)
Help API Users Consume Your Resources Wisely
Consider Open Sourcing Your Application
Easy-to-Understand Data Standards
Summary
IV. Exploring Other Mashup Topics
13. Remixing Online Maps and 3D Digital Globes
The Number of Online Maps
Examples of Map-­Based Mashups
Making Maps Without Programming
Mapbuilder.net
Google My Maps
A Mashup Opportunity: Mapping Yahoo! Local Collections
Transforming the Yahoo! Local XML into CSV for Mapbuilder.net
Collection Building in Microsoft’s Live Search Maps
Summary of Making Maps Without Programming
Data Exchange Formats
CSV
Microformats and Metatags for HTML
GeoRSS
Yahoo!’s Use of GeoRSS and Yahoo! YMaps Extensions
KML
Interoperability Among Formats: GeoRSS vs. KML
Creating Maps by API Programming
Google Maps API
Yahoo! Maps API
Microsoft’s Live Search Maps/Virtual Earth
Geocoding
Yahoo! Maps
Geocoder.us
Google Geocoder
Virtual Earth
Geocoding Non-­U.S. Addresses
Google Earth and KML
Displaying and Handling KML As End Users
KML
Programming Google Earth via COM and AppleScript
Mapstraction and OpenLayers
An Integrative Example: Showing Flickr Pictures in Google Earth
KML NetworkLink
Generating the KML for the Photos
The flickrgeo.php Code
Summary
14. Exploring Social Bookmarking and Bibliographic Systems
The Social Bookmarking Scene
Using Programmableweb.com to Examine the Popularity of APIs
del.icio.us
Using the del.icio.us API
Third-Party Tools for del.icio.us
Third-Party API Kits
Yahoo! Bookmarks and MyWeb
Connotea
A Flickr and del.icio.us Mashup
Summary
15. Accessing Online Calendars and?Event Aggregators
Google Calendar
Setting Up Google Calendar As an End User
Exploring the Feed Formats from Google Calendar
Using the ­GData-­Based Calendar API Directly
Using the PHP API Kit for Google Calendar
Using the Python API Kit for Google Calendar
30boxes.com
An End User Tutorial
30boxes.com API
Event Aggregators
Upcoming.yahoo.com
Eventful.com
Programming with iCalendar
Python and iCalendar
PHP and iCalendar
Exporting an Events Calendar to iCalendar and Google Calendar
The Source: UC Berkeley Event Calendars
Creating an iCalendar Feed of Critic’s Choice Using Python
Writing the Events to Google Calendar
Summary
16. Using Online Storage Services
Introducing Amazon S3
Rationale for S3
Conceptual Structure of Amazon S3
The Firefox S3 Extension Gets You Started with S3
Using the S3 REST Interface
Listing Buckets Using the REST Interface
Using the SOAP Interface to S3
Amazon S3 API Kits
PHP
Python
Summary
17. Mashing Up Desktop and ­Web-Based Office Suites
Mashup Scenarios for Office Suites
The World of Document Markup
The OpenDocument Format
Learning Basic ODF Tags
Create an ODF Text Document Without Any Styling of ODF Elements
Setting the Paragraph Text to ­text-­body
Formatting Lists to Distinguish Between Ordered and Unordered Lists
Getting Bold, Italics, Font Changes, and Color Changes into Text Spans
API Kits for Working with ODF
Odfpy
OpenDocumentPHP
Leveraging OO.o to Generate ODF
ECMA Office Open XML (OOXML)
Viewers/Validators for OOXML
Comparing ODF and OOXML
Online Office Suites
Usage Scenarios for Programmable Online Spreadsheets
Google Spreadsheets API
Python API Kit
Mashup: Amazon Wishlist and Google Spreadsheets Mashup
Zend PHP API Kit for Google Spreadsheets
A Final Variation: Amazon Wishlist to Microsoft Excel via COM
Zoho APIs
Summary
18. Using Microformats and RDFa As Embeddable Data Formats
Using Operator to Learn About Microformats
adr (Addresses)
hCard (Contacts)
hCalendar (Events)
geo (Locations)
tag (Tagspaces)
Definitions and Design Goals of Microformats
Microformats Design Patterns
rel-design-pattern
class-design-pattern
abbr-design-pattern
include-pattern
Examples of Microformats
rel-license
rel-tag
xfn
xFolk
geo
hCard and adr
hCalendar
Other Microformats
Microformats in Practice
Programming with Microformats
Language-Specific Libraries
Writing an Operator Script
Studying the Tutorial Script
Writing a Geocoding Script
Resources (RDFa): A Promising Complement to Microformats
Reference for Further Study
Summary
19. Integrating Search
Google Ajax Search
Manipulating Search Results
Yahoo! Search
Yahoo! Images
Microsoft Live.com Search
OpenSearch
Google Desktop HTTP/XML Gateway
Summary
A. Creative Commons ?Legal Code
Attribution-NonCommercial-ShareAlike 2.5
License

List of Figures

1.1. Figure 1-1. Housingmaps.com
1.2. Figure 1-2. The Flickr photo “Campanile in fog” (http://flickr.com/photos/raymondyee/18389540/) with associated geocoding embedded in the tags. (Reproduced with permission of Yahoo! Inc. ® 2007 by Yahoo! Inc. YAHOO! and the YAHOO! logo are trademarks of Yahoo! Inc.)
1.3. Figure 1-3. Clicking the GMap button opens a Google map in the browser. (Reproduced with permission of Yahoo! Inc. ® 2007 by Yahoo! Inc. YAHOO! and the YAHOO! logo are trademarks of Yahoo! Inc.)
1.4. Figure 1-4. Clicking the red pin opens a balloon containing the photo and further geotagging functionality offered by GMiF. (Reproduced with permission of Yahoo! Inc. ® 2007 by Yahoo! Inc. YAHOO! and the YAHOO! logo are trademarks of Yahoo! Inc.)
1.5. Figure 1-5. Clicking the GMiF Fly To button launches Google Earth, which then displays the photo at the latitude and longitude associated with the photo.
1.6. Figure 1-6. The LibraryLookup Bookmarklet Generator with parameters for the BPL
1.7. Figure 1-7. Invoking the LibraryLookup bookmarklet to look up Foundations of Ajax at the BPL. (Software copyright Innovative Interfaces, Inc. All rights reserved.)
4.1. Figure 4-1.Icon for subscribing to a feed in Flickr. (Reproduced with permission of Yahoo! Inc. ® 2007 by Yahoo! Inc. YAHOO! and the YAHOO! logo are trademarks of Yahoo! Inc.)
4.2. Figure 4-2.Choosing a news aggregator with which to subscribe to a feed in Firefox
4.3. Figure 4-3.Pipe for “Yahoo! News by Search Term (First Version)”. (Reproduced with permission of Yahoo! Inc. ® 2007 by Yahoo! Inc. YAHOO! and the YAHOO! logo are trademarks of Yahoo! Inc.)
5.1. Figure 5-1. A prompt from Flickr explaining the authentication process required to enable you to send Flickr photos to new Blogger blogs. (Reproduced with permission of Yahoo! Inc. ® 2007 by Yahoo! Inc. YAHOO! and the YAHOO! logo are trademarks of Yahoo! Inc.)
5.2. Figure 5-2. A prompt from Google requesting authorization to post to your Blogger blog from Flickr
5.3. Figure 5-3. Writing to a WordPress blog from the Windows w.bloggar client. Note that the post already exists on the blog and that w.bloggar is being used to post it for editing.
5.4. Figure 5-4. On a Windows desktop, SharpReader is looking at one of Udell’s posts, along with a right-­click invocation of w.bloggar to send this entry to a blog.
5.5. Figure 5-5. Blogging Flickr photos from Flock by dragging and dropping multiple Flickr photos into an editing window and then posting that entry into a configured blog. (Reproduced with permission of Yahoo! Inc. ® 2007 by Yahoo! Inc. YAHOO! and the YAHOO! logo are trademarks of Yahoo! Inc.)
6.1. Figure 6-1.The Flickr API Explorer for flickr.photos.getInfo. (Reproduced with permission of Yahoo! Inc. ® 2007 by Yahoo! Inc. YAHOO! and the YAHOO! logo are trademarks of Yahoo! Inc.)
6.2. Figure 6-2.Flickr authorization screen. (Reproduced with permission of Yahoo! Inc. ® 2007 by Yahoo! Inc. YAHOO! and the YAHOO! logo are trademarks of Yahoo! Inc.)
7.1. Figure 7-1. The WSDL SOAP Analyser panel loaded with the geocoder.us WSDL
8.1. Figure 8-1.Firebug applied to a Flickr photo page. (Reproduced with permission of Yahoo! Inc. ® 2007 by Yahoo! Inc. YAHOO! and the YAHOO! logo are trademarks of Yahoo! Inc.)
8.2. Figure 8-2.Interacting with the Yahoo! Calendar using the JavaScript Shell. (Reproduced with permission of Yahoo! Inc. ® 2007 by Yahoo! Inc. YAHOO! and the YAHOO! logo are trademarks of Yahoo! Inc.)
9.1. ProgrammableWeb Mashup Dashboard
10.1. Results of flickrgeo.3.html. Geotagged photos are displayed as HTML in response to the XHR request.
10.2. The Flickr Google Maps mashup
10.3. The Flickr Google Maps mapplet mashup
11.1. Figure 11-1. The Search tab of the Flickr/Google Maps mashup
11.2. Figure 11-2. The Saved Results tab of the Flickr/Google Maps mashup
13.1. Invoking the JavaScript Google Geocoder with the JavaScript Shell
13.2. A Google Maps collection displayed in Google Earth
13.3. KML for the Campanile and Evans Hall in Google Maps
13.4. View of the Campanile in Google Earth defined by the <LookAt> element
13.5. View of the Campanile in Google Earth defined by the <Camera> element
13.6. View of the Campanile in Google Earth defined by the <Camera> element with a 45-degree roll
13.7. Flickr photos displayed in Google Maps via a KML feed
14.1. Figure 14-1. Mashup of Flickr favorites and deli.cio.us. (Reproduced with permission of Yahoo! Inc. ® 2007 by Yahoo! Inc. YAHOO! and the YAHOO! logo are trademarks of Yahoo! Inc.)
16.1. Figure 16-1. S3 Firefox Explorer ­add-­on
17.1. Figure 17-1. Unzipping helloworld.zip. An OpenDocument file produced by OpenOffice.org is actually in the ZIP format.
17.2. Figure 17-2. The culminating ODF text document generated in this chapter
18.1. Figure 18-1. Operator toolbar showing microformats embedded in a page from Upcoming.yahoo.com. Actions available for the location microformat are shown boxed. (Reproduced with permission of Yahoo! Inc. ® 2007 by Yahoo! Inc. YAHOO! and the YAHOO! logo are trademarks of Yahoo! Inc.)

List of Tables

2.1. Representations of a Flickr Photo
2.2. URLs for the Various Sizes of Flickr Photo 508341822
2.3. Parameters Associated with Photo 508341822
2.4. Dissecting Parameters for a Link to Google Maps
3.1. Stemming of Terms Related to mouse in Flickr
4.1. Values for the format Parameter in Flickr Feeds
4.2. Values for the lang Parameter in Flickr Feeds
4.3. Possible Values for topic in Google News Feeds
4.4. Table 4-4.Possible Values for region in Google News Feeds
4.5. Parameters for Pipe Called “Feed from a URL Constructed from One Variable Parameter”
6.1. Parameters Associated with Photo 1153699093
6.2. The HTTP Request Parameters in a Flickr Call
6.3. The HTTP Response Parameters in a Flickr Call
7.1. Table 7-1. Panels and Parameters from the WSDL Soap Analyser in oXygen
7.2. Table 7-2. Number of APIs vs. Protocol in Programmableweb.com
7.3. Table 7-3. Top 21 APIs by Mashup Count
9.1. The Percentage of Mashups in ProgrammableWeb Grouped by Tags (January 13, 2008)
11.1. Table 11-1. Other Mashup Tools
13.1. getCollection Parameters
14.1. Table 14-1. A Summary of Bookmarking APIs*
16.1. Table 16-1. The Values Used in This Example Calculation for S3 Parameters
17.1. Table 17-1. The Sample Spreadsheet

List of Examples

13.1. Listing 13-1.XML for a Collection from Yahoo! Local
13.2. Listing 13-2.PHP Script to Convert Yahoo! Local XML to CSV
13.3. Listing 13-3.GeoRSS of a Map from Microsoft’s Live Search Maps
   Next
   Copyright