Is there an asyncio-friendly version for presto

0
votes

I know that there exists a package called aiopg for working with Postgresql in asyncio-friendly connections. Is there a similar version/package for Presto? I'm trying to implement concurrency for querying in Presto connections in Python.

Thanks in advance!

1
pythonprestotrinoaiopg
How many concurrent queries to you expect to send to Presto from your Python app? - Piotr Findeisen
Hi @PiotrFindeisen! Thanks for taking the time to respond. I'm looking at ~5 Presto-connection memory intensive queries. I was hoping to gather these along with my Postgres-connection queries (~15 count) and run them in one asyncio loop (using their respective connections). The goal would be that if each query takes at least 5 mins or greater to complete, then at least at one point, early in the loop, all the queries would have been sent (to save time). But let me know if you have other suggestions. Thanks! - Samuel S Cortez

1 Answers

1
votes

Presto Python API does not provide asyncio integration, so you need to treat it as any other blocking IO in asyncio world.

I am not familiar with asyncio, but run_in_executor looks like the way to do this.

Since you're not running thousands of concurrent queries (which wouldn't make sense in Presto anyway), this isn't going to hurt performance of your application.

I recommend that you also create a feature request in https://github.com/prestosql/python-client/issues/new (this is a new repo, so you can get very low, cool issue number).