Wanted: RSS Reader which eliminates duplicate content



I’ve used RSS readers in the past – they are great for aggregating information from sites which I view. However – adding all of my daily sites, as well as all of my ‘sometimes’ sites I quickly have 50 or 100 RSS Feeds and it is impossible to keep up to date on feeds because getting through all content takes forever.

Part of this problem, I’ve noticed, is that many articles around the web mirror each other – for example when the MacBook Air was announced, every site came out with a news item about it. That means I have to click through so many items in my newsreader to just mark it as read and make sure I haven’t missed anything.

So – what I am looking for is an RSS client which will not segregate each individual RSS feed, but create a mesh of all of them and highlight the important items, and somehow also promote smaller, less popular items that may have been missed by other sites. This would involve somehow analyzing the text and date of the post and correlating the items together. For less popular posts but ones that are nonetheless interesting, maybe create a ranking system based on social bookmarking sites like reddit or stumbleupon.

In a way this is done by some sites – Google News is one example of news sites; and TechMeme is another one for technology related items.

Does anyone know of a program, either online or offline, which will do what I am asking? If not, would you be willing to program it?