Google Tag Manager and Google Analytics are completely different products, they just work well together.
Google Analytics stores, aggregates and manipulates data for you, which you can output upon request.
Google Analytics also provides a handful of recommended code snippets that allow you to 'install' it on your site. However, these are just recommendations. Ultimately, Google Analytics will process any data that it receives in the correct form, and doesn't care about how it is implemented.
Google Tag Manager allows you to insert custom HTML into your site. You can edit that HTML any time you like, and push changes instantly. It also provides listeners for a number of things such as clicks and form submits, and has a structure for the deployment of tags.
Properly used, Google Tag Manager deploys your tags for you. It dictates which data is collected, and when your tags fire. It can fire almost any analytics tag, and much much more. In fact, you could use it to insert pretty much any code onto your website at the push of a button.
So you see, Google Tag Manager inserts custom HTML into your site, including analytics tags. These take the data from your site, and send it to Google Analytics. Google Analytics then analyses this data.