Well, it's easy to see why it's slow ... it's that three-deep nested loop that does an index scan on group_element_index. That scan's being executed a shitpile of times if it accumulates over a second execution time over the life of the query ...
I'll dig deeper tomorrow, we're dealing with a complex set of nested selects on views.
The second half of the query doesn't suck quite so bad, only a two-deep nested loop ("only?") that in this case isn't executed at all so it runs fast.