I discovered Digg and Reddit a few months ago and was immediately impressed by the quality and relevance of content contained therein. So I subscribed to their RSS feeds so that I could keep up to date with the latest internet news in my RSS reader.

For those unfamiliar with Digg and Reddit, they are web sites that allow users to rate other web sites and articles that are found on the internet. Users essentially vote for web sites and articles that they like and popular sites show up in the top 100 list.

This presents a problem for RSS feeds, however, since every time a ranking of an article is changed (e.g. it is ranked 21 instead of 22) the timestamp is updated, so it appears at the top of the RSS feed again. Needless to say it is quite annoying to finding 30 or 40 Digg and Reddit articles at the top of my news list every time I refresh my subscriptions.

How should it work?

Preferably when an article breaks into the top 25 or top 50, it will show up in the RSS feed – and will never show up again. Even if it rises to the top. I want my news feed to contain news – not “olds”.

Solution

I created a feed filter that takes RSS feeds and filters out duplicates. Even if the timestamp has changed, an item that has been loaded once, will never be loaded again.
You can find the tool at http://feedfilter.weblite.ca.

Now I can subscribe to Digg and Reddit without being bombarded by old news every time I refresh.