If you perform an upgrade through the dashboard, the latest version of Concrete5 is downloaded and placed in the updates folder. A constant is added or modified in config/site.php that indicates the path to the current core files.

The idea is that if an upgrade goes bad, it is a small step to edit config/site.php and change the constant to point to the previous version. Everything would be back to "normal" pretty quickly. (You would have to reinstall the database, as well, if database changes were part of the upgrade).

If you don't ever plan to use a previous version, those folders could be deleted, but you can't delete the folders for the current version, or you will blow the current installation away (not a pretty sight).