you should also check and make sure you increased your shared buffers on postgresql and that the db cache is populated.
It might be that in practice it is much faster once it has
run a couple times.
Whats the result of "explain analyze select ..." on that query?