Aggregating and displaying content from hundreds of RSS feeds

0
votes

I'd like to build a website that aggregates and displays content from hundreds of RSS feeds. The feeds will be from different sites: Twitter, Flickr, Tumblr, etc, so the content will be very heterogenous.

In a perfect world — and this is more of a side issue — I would like to allow other people to help manage the list of feeds and assign tags to the content from each individual feed so that you can filter the items that are displayed.

What I've tried so far:

  1. Google Feeds API – I thought this would be the answer, but unless I'm missing something, the FeedController will only output the collected feed content as separate lists. Is there any way to ask the Google Feeds API to aggregate and sort the content from many RSS feeds before displaying?

  2. Yahoo! Pipes – This also seemed like a good solution at first. I setup a Pipe that accesses a list of RSS feeds stored in a Google Doc spreadsheet and then aggregates the content. However, the output leaves a lot to be desired; Tumblr video posts, for example, only show a title and a permalink to the post, the embedded Youtube video is lost.

  3. PHP – I've seen this question, which looks like a good approach. I'm less proficient in PHP, so although I'm willing to learn, I'd ideally like to find a different approach.

Any thoughts? Thanks.

5
javascriptrssaggregateyahoo-pipesgoogle-feed-api

5 Answers

1
votes

My favourite is still Yahoo Pipes. With a little bit of tinkering, you might be able to get the video tags preserved. Do you have a link to the pipe in question?

0
votes

Actually, i'm doing this for one of my projects.

I'm using ASP.NET & C#.

I have functionality that goes out regularly and retrieves the data in each feed. I store new items in SQL Server. This way I can group, sort, display the data in the feeds anyway I like on my website. Depending on the site and the keywords in the item titles, I assign tags to each item automatically when it's added to the database.

0
votes

put your CONTENT into a database, and use the best tools in the world to sort / filter.

0
votes

Google Feeds API is good for collecting RSS feeds, take the arduino forum feed it only contains the latest 5 entries. The historic entries feature of google feed api fixes that issue.

I would use use google feed api to feed into yahoo pipes as that gives you the controls your looking for, tagging, filtering...

0
votes

Two suggestions:

  1. The total package: Dave Winer's River2 http://quick.newsriver.org/
  2. The unofficial Google Reader API http://code.google.com/p/google-reader-api/