When adding members in a specific project, I think it would be more convenient to display only users on the selection box when “Show users” is selected and display only groups when “Show groups” is selected.
In that case, the selection box would be segregated to groups and user depending on what option you choose.