I think you might need to tune your database.
Or maybe the query only is slow with the large amount of data you have.
How many shared buffers do you have? Also make sure in generaly your server has enough RAM for the operating system to cache the disk reads (on Linux anyway, I am not sure how it works on other platforms).
Genreally around 128mb shared buffers seems to help in many cases, I haven't found a site that needed more, but your results may vary. I would experiment but increasing shared buffers to at least 128mb if is not already.