All the things you mentioned like load balancing, monitoring and auto-scaling are definitely advantages.
However, you have to kind of think about it this way: In a true Platform as a Service (PAAS), the goal is to separate the application from the platform. As a developer, you only worry about your application. The platform is "rented" to you. The platform "instances" are automatically updated, administered, scaled, balanced, etc. for you. You just upload your WAR file and it just works (at least theoretically).
EC2 by itself is not PAAS. It is more like IAAS (Infrastructure as a Service). You still have to take care of the server instances, install software on them, keep them updated, etc.
Elastic Beanstalk is a PAAS system. So are App Engine and Azure among many others.
In a true PAAS system, the DBMS is a separate component from the web application server(s). The reason is obvious: The DBMS cannot be possibly installed on the instances that are being used for the application server because, as instances are created and destroyed based on your traffic, the DBMS would be lost! Having the DBMS and application server on the same machine/instance is not generally a good idea anyway.
In a PAAS system, the DBMS is a separate service. For Amazon, it would be Amazon RDS. Just like with Elastic Beanstalk, where you don't have to worry about the application server and you just upload your WAR file, with RDS, you don't have to worry about the DBMS and you just deploy your database(s).
Elastic Beanstalk and RDS work very well together, especially when deployed in the same availability zone, where the latency would be very low.
Finally, using Elastic Beanstalk doesn't cost anything more than the deployed resources (EC2 instances and the load balancer). However, RDS is not cheap and would definitely be more expensive than using a single EC2 instance for both the application server and the DBMS.