How many usergroups do you have set up on your forum? I have tried to keep our number of usergroups really low because when we make changes such as adding new forums, it can be a real pain with permissions. I hate it on Xenforo that there is the "inherit" feature because I haven't figured out how to see where it's inherited from to know what the perms are!