Is it possibly a DNS issue? If a script runs which needs a dns lookup, sometimes timeouts will last for quite a while.