We’ve added a second list of bots (ones that might look legit are not!) to block for you to choose from which in addition to badly behaving bots (the first list choice) includes download agents and useless web crawlers.

Further, the code is explained a bit better with some helpful comments, and there’s a README with license, instructions, and ‘stuff’. It’s also packaged a bit better.

Again, this is already in production on a damn busy website without problems.

Antidouche Lite for Bad Bots: download .zip.

This is a very small bit of php code that checks against a very basic list of known bad bots’ user agents which are typically scrappers or other crappy bots that are no use to you, eat up bandwidth and thus might be denying your users/readers/google a quick response.

Future updates will include additions to the list as they come available (feel free to just download the list to your own server too if you want). Additionally, we’re looking at a seperate whitelist list as well as adding other good bots (non google/yahoo/MSN) which you may want to block anyway.

This code has been tested against a site with ~800,000 pageviews/day for low overhead.

Give it a spin before you bother downloading! From your terminal wget –user-agent autoemailspider osdir.com/ml/php.cvs.zend/2008-01/msg00016.html

A lot of you out there don’t block annoying/useless bots with identifiable agent-users. Those who do don’t keep it up to date with all the new ‘social media’ bots that are multiplying every day.

Our first product is going to dynamically check for these & block ‘em for you. This certainly isn’t going to block spammers/emailharvesters/bandwidth-eaters with truly evil intent, but it will save you a ton of bandwidth and user angst.

NO, it will not block good bots like google, adsense, yahoo, msn, baidu, etc. who generally behave themselves and are on a leash by masters who monitor their actions.

Anyhoo, it will be really simple. Stand by.

We’re still working on defining it to a niche, but there are a lot more douchebags on the internet than just spammers.  We’re looking at helping you get rid of them.

One particular set of douche we’re examining is scrapers.  People who rip off the content you work so hard at creating and call it their own. Plagarism. What’s worse is that the douche who do this are much better at SEO than you are and they end up stealing not just your content, but your google traffic and adsense earnings.

We envision this being a simple php script call and/or firewall rule update.

More later as we get some prototypes up & running.