Right. My gutfeel is that something like a portal_page or a apm_parameter_value can hardly have a finely-grained permissioning record attached to it, so these rows may just be deleted from acs_object_context_index.
We probably also have to look at each individual package to determine which object an have finely grained permissioning. E.g. on one project I worked on they were using a group calender: in such a case a single cal_item wouldn't need a permission/context record.
