While using Attribyte to do some research for my next post, I found an article on Real Tech News that looked relevant. Unfortunately, when I clicked through I was presented with this "precondition failed" error page. What an arrogant error response. Because my HTTP request didn’t meet certain vague preconditions I should…

• Scan my computer for viruses, Trojan horses and spyware?
• Mess around with the settings of my personal firewall?
• Turn-off or bypass my proxy server?
• Turn off any download accelerators? (“they actually run slower!”)
• Switch to Firefox?

I should try these things so that I can read a blog post? How much time might someone less tech-savvy waste after seeing this error message - thinking they had been infected with a virus?

I’m sure I’m getting these errors because Attribyte’s bot is running behind the same router as my home network. Sure enough. Three of the hourly checks in the past day have failed with the HTTP 412 (Precondition Failed) error. Not all. Three. Why? After a bit of research, I’m pretty sure this message is generated by a PHP plug-in, BadBehavior. It seems to think Attribyte is, well, behaving badly.

For the record:

• Attribyte’s bots send an appropriate User-Agent string with all requests.
• The bots do run behind a caching proxy server – one that is very common. One of the reasons they are behind this proxy is to try to reduce remote server bandwidth.
• The robots.txt file is checked before requests, but it may be cached for up to 24 hours. Because Attribyte also functions as a personal feed-reader, directives excluding feeds are ignored unless the agent is explicitly excluded. However, entries are not displayed on public pages when the feed is excluded. I think this is fair. Anyway, the RTN robots.txt does not exclude any of the resources that are being requested.