« Hard training ride | Main | Cool RSS trick from Feedster »

August 16, 2003

Upgrades and scumbags...

I spent a couple of hours tonight dealing with something I've been putting off for awhile: dealing with porn referrers in my Apache logs.

I took a slightly different tack than Paul, who set an Apache environment variable to "dontlog", then created a custom access_log without those entries.

Initially, I was going to just duplicate Mark Pilgrim's approach. I couldn't get the redirects to work, which seemed strange, until I realized I was running a fairly minimal Apache build that didn't have mod_rewrite. That would explain it.

I decided this was as good an opportunity as any to download Apache 2.0, and see what it's about. It's fairly straightforward, but I installed it in a new location so I could back out more easily, and then had to reconfigure all its defaults when it installed without a hitch. I quickly discovered that mod_rewrite only reduces the bandwidth being wasted on the porn spammers et al, so I turned to my Webalizer config file, where I added a number of Ignore lines, encompassing all the porn words I could think of (Oops - gotta add "lemurs"). Update: For the sake of accuracy, I actually used the "HideSite" directive in webalizer, so sites with porn words in their names won't show up on the stats page, but I will see the traffic they cause.

This sort of thing is one of the reasons I find TypePad attractive. Like a Krikkit warship at Lord's Cricket Ground, this sort of thing becomes Somebody Else's Problem.

Update: And, what the hell, I'll password my usage data. If you have any interest in my stats, e-mail me, and I'll send you the current username and ID.

August 16, 2003 in Weblogs | Permalink


I never got my regex stuff to work on the porn referral crap, so I just let it ride. It doesn't seem to be a consistent problem, so I'm not worrying too much about it.

I knew it would just take them out of the log and that was fine. I wish there was some way to send back a useful file (like the patch that seems to work with mixed results?!) or some kind of popup alert ("Hey, moron, your PC is infected with LovSan a/k/a MS Blaster. Fix it. Now.").

Perhaps I'll take another look at Apache 2.0 now that you've done it: I tried it, found I couldn't get some things to work, and just dropped back to 1.3.27.

Posted by: paul at Aug 18, 2003 4:31:34 PM

I thought about redirecting them to a nice, fat page whose bandwidth is billed to somebody else, but I've reduced their average traffic from 20k+ to 286 bytes, and eliminated whatever payoff there was to being in my stats, since they're no longer public.

Posted by: Frank at Aug 18, 2003 6:08:12 PM