I've never set up the robot detection stuff, but yes, provided that that works, it is trivial to set htDig's user agent, and so the combination should win.
It's pretty easy to forge the user agent, so I guess it depends on the nature of the content that is being protected and indexed. Joel wants it indexed, so I guess it's not that confidential and he just wants users to register to see it. But it is protected, so maybe not.
