Pinterest Stumbleupon Whatsapp
Ads by Google

feed aggregator softwareLet’s admit it, not all RSS feeds are perfect. Often you’re stuck with a lot of redundant, or poorly formatted content. RSS feeds are great to keep tabs on a number of sites, but for content generation, they aren’t always ideal. There’s usually not a lot you can do about it, except sucking it up.

Yahoo! Pipes is a website aggregator’s wet dream. It is feed aggregator software that allows you to fetch, combine and alter all kinds of data sources, not least RSS. The problem with Pipes is its steep learning curve. In a nutshell, Pipes can be compared to visual programming, combining nodes and adjusting parameters.

This article is intended, not only to introduce you to the Yahoo! Pipes feed aggregator software, but to show you a few simple RSS tricks that everyone should be able to pull off.

1. Using Pipes: The Basics

Pipes works by building a linear string out of the different components; from data input, via modifier modules, to the Pipe’s output. Although Pipes supports a number of different data sources, we’ll focus on RSS feeds in this particular article.

All of these modules can be found on the left hand side of the screen. You can find ‘Fetch Feed‘ under Sources and most of the other things we’ll be working with under Operators. To use them, simply drag and drop them on to your canvas, and connect the dots.

Ads by Google

Some of these Operators allow you to add parameters. This will tell them what to do, and where to do it. Don’t worry, we’ll help you out with the specifics in the examples below.

feed aggregator software

Once you’ve managed to build a Pipe, you need to save it before you can start using it. The Save button is located in the top-right corner of the screen, and helps you preserve the Pipe via your Yahoo! account.

feed aggregator software

A saved Pipe has its own page. Note that this is not the address of your newfound RSS feed – yet. You can find an RSS or JSON address on the page, as marked in red below.

yahoo pipes tutorial

Alternatively, you can also publish your Pipe, making it possible for all to see and use. Check out other, popular user-made pipes here. You can also share your Pipe via one of the many supported social networks.

Note that we’re scratching the surface of Yahoo! Pipes in this article. I would be lying if I said I understood even half of the possibilities. With the use of (e.g.) regex and SQL, you can make formidable Pipes, albeit a bit too complicated to discuss here.

2. Hiding The Original Feed

One of the most simple uses of Pipes is hiding the URL of the original RSS feed. There are a couple of reasons why you might want to do this. Some feeds require a username and password embedded in the URL to work, as can be seen in Jessica’s article on using Twitter feeds for email and SMS notifications A Quick Way to Get Free SMS & Email Alerts of Your Twitter Mentions A Quick Way to Get Free SMS & Email Alerts of Your Twitter Mentions Read More .

yahoo pipes tutorial

This Pipe uses only a single (input) module. You can find ‘Fetch Feed‘ in Sources. Enter the RSS feed, and connect it to the Pipe Output. That’s it!

Mind you, do not publish this Pipe or its feed outright, as people can still find the original by playing with the URL – and thus find out your password! This is only useful to hide the original feed URL from a direct encounter, and from someone who doesn’t start digging any deeper.

3. Filtering Specific (Author) Posts From An RSS Feed

Not all multi-author websites support individual RSS feeds for all of their writers. Or, as I found out the hard way, they are sometimes simply corrupt. If that’s the case, you can also use Pipes to filter the regular feed, and only take from it what you need.

Although this example shows you how to filter by author, you can also use it to filter on other subjects. For example, leaving out all posts that talk about Windows Mobile, or only showing those that mention Android.

yahoo pipes tutorial

This Pipe uses the Fetch Feed module from Sources, and the Filter module from Operators. Put the Filter between input and output, and connect the dots.

Next, set the filter to Permit all items that answer to the rule(s) below. To filter an author, Item.dc:creator must contain ‘his/her name’. Check the website to make sure how the author’s name appears.

The Item.dc:creator attribute is universally used to indicate the post’s author. In the unlikely case another attribute is used, you can browse the debugger field at the bottom of the page where you can find the different attributes and their corresponding values for every RSS post in the feed.

4. Translating The RSS Feed

Are you interested in a foreign blog or website? Perhaps you don’t need to fight your way through those articles, but can use Pipes to automatically translate the feed!

This Pipe uses the Fetch Feed module from Sources and the BabelFish module from Deprecated. After connecting the nodes, you can simply pick your source and destination languages from the drop down menu. Simple as that!

5. Leaving Out Part Of The RSS Feed

This one is especially handy if you use an RSS feed to generate content on your website. Instead of having it riddled with irrelevant content, you can simply select the items your interested in. Or rather, discard the rest.

This Pipe uses the Fetch Feed module from Sources and the Create RSS module from Operators. After connecting the nodes, you can generate a new RSS feed using items of the original. Use the above screenshot as a guide to select the right items from the drop down lists. It could very well be you’ve already dropped the redundant content by creating this standard RSS feed. Otherwise, leave the superfluous fields empty.

Do you know any other simple Pipes? Or perhaps an alternative feed aggregator software? Speak your mind in the comments section below!

  1. mirobi
    February 20, 2015 at 4:35 am

    HI,
    Thank you for sharing all these information :)
    I've two questions:
    1- is it possible to delete all the links from an rss feen and put an other link instead?
    2- I created an rss feed with 3 entriens and I would like to make it regenerating over and over with the same content is it possible?
    Thanks

  2. Tyson
    May 12, 2010 at 5:24 pm

    Pipes are awesome. Thank you for this example to spread the word.

  3. Hamza Ahmed
    May 12, 2010 at 7:07 am

    Thats a great trick...........!!!!

  4. Simon Slangen
    May 11, 2010 at 9:36 pm

    Ok, here's a Pipes alternative to make your original feed ENTIRELY invisible from prying eyes. Using this technique, you can also pass the feed to third parties without concern.

    This Pipes uses the module from , the from and the module from

    1. In the Private String module, enter ONLY the private part of your feed, and everything that comes after.
    2. In the URL Builder, enter the first part of your feed i.e. everything before the private part. Then connect the Private String node to the little grey bubble next to the Path elements field.
    ( ! ) make sure to include a slash either at the end of the Base URL or at the start of your private string ( ! )
    3. Connect the URL Builder node to the little grey bubble next to the Fetch Feed field.
    4. Connect the Fetch Feed node to the Output node.

    Screenshot Overview

  5. Ram
    May 11, 2010 at 11:01 pm

    Thank you!
    I had tried Pipes in the past and found it a little daunting and couldn't really think of something useful to do with it. But your example is great!

  6. Ram
    May 11, 2010 at 9:01 pm

    Thank you!
    I had tried Pipes in the past and found it a little daunting and couldn't really think of something useful to do with it. But your example is great!

  7. Simon Slangen
    May 11, 2010 at 7:36 pm

    Ok, here's a Pipes alternative to make your original feed ENTIRELY invisible from prying eyes. Using this technique, you can also pass the feed to third parties without concern.

    This Pipes uses the module from , the from and the module from

    1. In the Private String module, enter ONLY the private part of your feed, and everything that comes after.
    2. In the URL Builder, enter the first part of your feed i.e. everything before the private part. Then connect the Private String node to the little grey bubble next to the Path elements field.
    ( ! ) make sure to include a slash either at the end of the Base URL or at the start of your private string ( ! )
    3. Connect the URL Builder node to the little grey bubble next to the Fetch Feed field.
    4. Connect the Fetch Feed node to the Output node.

    Screenshot Overview

  8. sudoku oyna
    May 11, 2010 at 7:33 pm

    that was what i am looking for. thanks for information. i really need such thing

Leave a Reply

Your email address will not be published. Required fields are marked *