This is a good start.
After that sometimes I have found that going to extensions -> db -> fix had some work to do.
This will get Joomla up to date (although you will have some leftover files, not a big deal). But it will not install the new features such as Joomla update icon in the control panel.
In order to do that you need to repackage the extensions, remove the files from the installation, and reinstall. It may take a few tries (you need to remove the language files and any off-folder files as well, either from the disk - and putting them in the installation zip - or from the .xml manifest.
The three extensions I found I wanted most were com_joomlaupdate, and the extensionupdate and joomlaupdate plugins. You can download them from this page:
it also contains the list of files which you should delete:
rm -rf administrator/components/com_joomlaupdate
rm -rf plugins/quickicon/extensionupdate
rm -rf plugins/quickicon/joomlaupdate
rm administrator/language/en-GB/en-GB.com_joomlaupdate.ini
rm administrator/language/en-GB/en-GB.com_joomlaupdate.sys.ini
rm administrator/language/en-GB/en-GB.com_joomlaupdate.ini
rm administrator/language/en-GB/en-GB.com_joomlaupdate.sys.ini
rm administrator/language/en-GB/en-GB.plg_quickicon_extensionupdate.ini
rm administrator/language/en-GB/en-GB.plg_quickicon_extensionupdate.sys.ini
rm administrator/language/en-GB/en-GB.plg_quickicon_joomlaupdate.ini
rm administrator/language/en-GB/en-GB.plg_quickicon_joomlaupdate.sys.ini
