I just realized how badly "not in" statements behave in PostgreSQL especially if you compare two big tables.
What you can do instead is a left join where you limit the result to the objects where the joined table has no entry. For contacts we have this:
select person_id from persons left join (select item_id from cr_items where content_type = 'contact_party_revision') items on item_id = person_id where item_id is null;
Did I make a magnificent mistake or is there a reason why google search did not bring this up on a prominent spot when searching for "postgres not in"