I don't think relying on the referer header is a good idea, since it might not always be sent by the browser. Looking in my access logs I see from time to time users whose referer is always blank, even when they are apparently navigating within the site. It's propably caused by some kind of (paranoid) web filter, firewall or proxy software. I see that too often to feel comfortable locking these users out of adminstrative functions on my sites.