MP4 - My Yahoo Pipe

Yahoo pipes is a cool implementation of the pipes and filter pattern. Basically, you get to take your choice of data of many shapes and forms, apply some logic using operators and filters and create lists as well as visual implementations for output. Creating mash up's for converging data of different types or homogenous themes is also a popular use for Yahoo Pipes.

I viewed the quick tutorial presented at pipes.yahoo.com in order to get started; and you do get to start right after.

So, here is how it works ...

- Yahoo pipes provides a drag-and-drop type of online IDE to be your sandbox.
- It appears that Yahoo Pipes IDE is in a cloud; so without access to the internet you can't develop.
- Once you figure out what pipe output you want to create, using the inputs, operators, filters, data feeds etc. you work in your IDE to bolt together a result.
- All output should be connected to a single output pipe.

Here's how my Yahoo Pipe works and what are some things I used while building mine:

- The topic for my mash up is to combine two rss or site feeds from two websites which list items, mostly technology related and gadgets, on a daily basis. The list that the two sites post are usually great deals for a bargain; my mash up uses the two sites; hotdealsclub.com and dealcatcher.com as source, asks the user what keyword do they want to search for deals and how old the deal can be. Based on these filters, I then combine the results of the search and return a final merged list based on the search criteria and the date range.

- I acquired the RSS feed url from the site and used the 'Fetch Feeds' Sources as my starting point.

- I then dropped on the date input and text input from user inputs for capturing the search criteria.

- Wired together the feed output to filters along with the data and text input.

- Tweaked the filters by adding more rules to apply the date rule and search criteria so that the filter will filter through only items which contained text of the search criteria and are not older than the entered date.

- I found that I could not assign the same filter on two sources, I had to create to filters ...

- Then I used the union operator to merge the two filters; keeping duplicates.

- I found that I couldn't pass the output from two filters to output pipe which is why I used the union operator ...

- Finally, display the merged results in the form of a list.

Very intuitive interface is developed for the IDE; almost like a work flow control app. It was very easy to use the IDE to build my mash up and even debug it on any stage using the debug feature. Even easier was publishing it.

I was pleasantly surprised and it was refreshing to see such functionality appear on Yahoo when things as such are usually expected from Google ... while I blog on Google Blogger :) Wonder if Google has such an online app to create mash ups.

Well, here is my first, of what I'm sure is one of many, mash up's using yahoo pipes:



Sadfeen.