The first step I took was to build a pipe to handle the first source, Yahoo! News. I exploited the fact that you can generate an RSS Yahoo! News for a search term with this:

http://news.search.yahoo.com/news/rss?p={search-term}

I built two versions of a pipe to return a feed for a given search term. The first version—called “Yahoo! News by Search Term (First Version)”—is here:

http://pipes.yahoo.com/pipes/pipe.info?_id=Rg_rh3NA3BGdECIel7okhQ

You can run it and view the source if you are logged in. You can run a pipe to get an RSS 2.0 feed for a given search term here:

http://pipes.yahoo.com/pipes/pipe.run?_id={pipe-id}&_render=rss&search_term={search-term}

For example, to search for Hurricane Katrina, go here:

http://pipes.yahoo.com/pipes/pipe.run?_id=Rg_rh3NA3BGdECIel7okhQ&_render=rss&search_term=Hurricane+Katrina

The pipe uses three widgets to enable a user to pass in a search term and return a feed of Yahoo! News (see Figure 4-3):

Figure 4.3. Figure 4-3.Pipe for “Yahoo! News by Search Term (First Version)”. (Reproduced with permission of Yahoo! Inc. ® 2007 by Yahoo! Inc. YAHOO! and the YAHOO! logo are trademarks of Yahoo! Inc.)

The second news source that I need to pull in is Google News, which returns an Atom feed for a given search term here:

http://news.google.com/news?q={search-term}&output=atom

I use output=atom instead of output=rss to show that Yahoo! Pipes can handle Atom feeds.

One way to build a module to handle Google News is to clone the one for Yahoo! News and change the parameters in the URL Builder module. Instead, because I figured that there are plenty of feeds with URLs that consist of a single parameter and search term, I decided to build a utility pipe that would return feeds at URLs in the following form:

{base-URL}?{parameter_name}={parameter_value}{URL_suffix}

Constructing such a pipe is equivalent to writing a reusable function. By contrast, cloning a pipe is analogous to copying and pasting code. When you use pipes seriously, you begin to see patterns that can be captured and reused in a pipe.

The pipe I constructed for that purpose (called “Feed from a URL Constructed from One Variable Parameter”) is located here:

http://pipes.yahoo.com/pipes/pipe.info?_id=VoLceXZA3BGkqcJZJxOy0Q

Note the intermixing of the URL Builder and String Builder modules to concatenate parameters and build a URL that can actually be fed to Fetch Feed module.

I used that pipe and the fact you can write the URLs to retrieve feeds from Yahoo! News and Google News in terms of these four parameters (see Table 4-5).

The pipe that accesses Yahoo! News using the utility pipe (entitled “Yahoo! News by Search Term”) is as follows:

http://pipes.yahoo.com/pipes/pipe.info?_id=5NhmMndA3BGg5zQ5n0artA

The “Google News by Search Term” pipe is here:

http://pipes.yahoo.com/pipes/pipe.info?_id=OKWv6nNA3BGkPtA8qWIyXQ

Now we come to the third and fourth sources: Wikinews, which has a single feed:

http://feeds.feedburner.com/WikinewsLatestNews

and the New York Times National News (I select the National News feed because of the focus on Hurricane Katrina):

http://www.nytimes.com/services/xml/rss/nyt/National.xml

In contrast to the Yahoo! News and Google News for which I can generate a feed for a given search term by constructing the appropriate URL, I use search-term to filter the feed. To that end, I use the Filter module in the Filter for Given Term in Feed Description and Title pipe here:

http://pipes.yahoo.com/pipes/pipe.info?_id=KIYSv3pA3BGgloVbCB2yXQ

to pass along only those items in the feed whose description or title contains the search term.

With that pipe as a foundation, I construct the “Wikinews Filtered by Search Term” pipe here:

http://pipes.yahoo.com/pipes/pipe.info?_id=PA7iqHpA3BGb0AiVX0sBXw

and the NY Times National News Filtered by a Search Term pipe here:

http://pipes.yahoo.com/pipes/pipe.info?_id=yhBh7HxA3BGu_YRj1vC6Jw

With a pipe each for my four news sources, each of which takes a search term as input, I then create a pipe here:

http://pipes.yahoo.com/pipes/pipe.info?_id=qlUkcn1A3BGeWlNQjknRlg

that does the following:

You’re done. You can search for Hurricane Katrina in the four news sources here:

http://pipes.yahoo.com/pipes/pipe.info?Search_term=Hurricane+Katrina&_cmd=Run+Pipe& _id=qlUkcn1A3BGeWlNQjknRlg&_run=1

The corresponding feed is available here:

http://pipes.yahoo.com/pipes/pipe.run?Search_term=Hurricane+Katrina&_id=qlUkcn1A3BGeWlNQjknRlg&_render=rss

Obviously, the pipes I created could be refined. Indeed, you can do so right now by going to the list of pipes I created for this chapter and cloning and modifying them for your own use:

http://pipes.yahoo.com/pipes/tag.info?namespace=user&tag=mashupguide

This section used feeds that are relatively easy to access. If you start to use tools such as Openkapow.com to screen-­scrape new feeds, you have even more combinatorial possibilities for your mashups. Note that other tools that I discuss later in the book (such as the Google Mashup Editor) are able to consume feeds, such as those generated by Yahoo! Pipes and Feedburner.