Handling AWS RDS Connection Pooling (with POSTGRES 13) and Node Lambdas

Question

We are using an RDS instance in AWS, it is POSTGRES 13. We have Node-based Lambdas that return and post data to the database. Currently they are opening and closing a single connection per transaction and we want to optimise this by implementing connection pooling.

There is an AWS service specifically designed for handling DB connections called AWS RDS Proxy BUT it isn't compatible with POSTGRES 13. We are looking for alternative approaches and would really appreciate any insight in this regard.

We were thinking about pulling the DB connection outside of the Lambda handler so that it maintains state (until the Lambda gets removed) BUT this wouldn't be able to close its connection to the DB and we were concerned about filling up all the available connections. This may or may not be the case.

Much thanks,

Sam

Answer 1

Here's some random thoughts that might help.

• Is there any special reason that you need to use PostgreSQL 13? If you could go back to 10/11 then you would be able to use RDS Proxy and then your problems would be solved
• Similarly, if you wanted to use RDS Aurora then you would be able to combine this with RDS Proxy
• You can use PgBouncer (or another third-party proxy - there's loads out there) but to get this to work with RDS is going to be quite painful. Most of them will require you to spin up an ECS Instance, run the proxy on this, and then connect to the proxy - this is deliberately vague, as each proxy works slightly differently
• You could increase the number of database connections allowed, but this is generally a bad idea as it's just kicking the can down the road
• You could look into horizontal or vertical scaling solutions, but again these won't solve the underlying problem, just make it not so apparent and take more concurrent hits before it rears up again
• You could stick a queue in front of your database, SNS or SQS, and only have one connection. This is fine if you're only ever writing data to the database and you're not too bothered about how long it actually takes to complete. However, it introduces an entirely different architecture, and one that defeats many of the good points of using RDS in the first place
• If you need to read data then you can use read replicas, but this is just another form of horizontal scaling/ sharding
• Similarly, you could ditch using Lambdas, as these are the "angry swarm of bees" that cause the problem in the first place, but again, it's a big architectural change

In my experience getting a connection pooling solution should be good enough to solve most use cases, but there are still limits when you're combining two technologies together that aren't 100% compatible.