You do need the composition_rels between the groups. But I think you might be going about it a litle wrong with the relational segments.
You still want to add users in a role. I don't think relational segments work between composition_rels, only membership_rels. So you would create users as a certain type of member in each company group.
Now you can access the relational segment of that company group with a certain role/relationship_type.
