Normal registered users don't have the ability to add or change group membership, as they cannot access the dashboard.

Sorry. Wasn't quite clear I don't guess. My apologies. I have created a group called "Users" and provided them access to the Dashboard, File Manager, Sitemap and Global Settings, as well.

I simply don't want them to be able to add themselves or anyone else as an admin.

I hope that's clearer. Is this possible?

To to your sitemap. Check "Show System Page". Click "Users and Groups." Click "Set Permissions." Uncheck the box for the group that you don't want to be able to change user groups.

I want this group to be able to add users, but I don't want this group to be able to add administrators. I only want the SuperUser and Admins to be able to do that.

I've got the same issue here.
I got usergroup X with less rights than the administrator group. The users in group X are allowed to create other users. But it seems they are allowed to select the group where the user belongs to: group X / Administrators. So a user with no rights can upgrade itself by selecting a higher group. Is this a security flaw or are we doing something wrong?

Replying to myself after reading more forum items about this issue:

Administrators are a default group which is available at concrete install. Just example data. The superuser (we) isnt part of the group Administrators. There isnt a security flaw.

It seems (for me) impossible to know which groups have more rights then the other. Using the advanced permission settings its hard to find out if group X has more rights than group Y. But it would be great if the superuser can set the group selection list per usergroup. So if a member of usergroup X is logged in he cant select group Y.