Once you have added new users to your event, you can easily manage them from the same place. To do this, make sure that you're in the Event Workspace of the event that you'd like to collaborate with others on. Click on Accounts & Permissions from your Event Menu.
Here, you'll see the list of users that you have already added to this event. If you want to change their user types, simply click on the edit icon on the row of the user in question. You can also delete this user entirely so that they no longer have any access to your event.
Please note that deleting or changing the status of this user will not have any bearing on the past actions that they have taken.