django async ORM sync_to_async executor swap out concurrent.futures.ThreadPoolExecutor when using gevent

0
votes

This is just a long way of asking: "How does sync_to_async work with blocking IO, and gevent/psycogreen"?

For example:

from myapp.models import SomeModel
from asgiref.sync import sync_to_async
from gevent.threadpool import ThreadPoolExecutor as GThreadPoolExecutor

conf = {
    "thread_sensitive": False, 
    "executor": GThreadPoolExecutor(max_workers=1)
}

await sync_to_async(SomeModel.objects.all, **conf)()

The third kwarg that can be passed to asgiref's sync_to_async is an executor the executor is a type of concurrent.futures.ThreadPoolExecutor

According to the documentation gevent.threadpool.ThreadPoolExecutor more or less inherits and wraps concurrent.futures.ThreadPoolExecutor

Say for example I want to use a werkzeug DispatcherMiddleware, and wrap an ASGI app.

Think FastAPI mounted to the inside of an older monolithic django WSGI app ( using eventlet / gevent / psycogreen / monkey patching )

Here's my attempt at doing it.

Basically, how to get django async-ish ORM?

try:
    from gevent.threadpool import ThreadPoolExecutor as GThreadPoolExecutor
    from django.conf import settings
    if settings.GEVENT_DJANGO_ASYNC_ORM:
        from gevent import monkey
        monkey.patch_all()
        def monkey_patch_the_monkey_patchers(ex):
            from .patch_gevent import _FutureProxy
            def submit(ex, fn, *args, **kwargs): # pylint:disable=arguments-differ
                print(fn, *args, **kwargs)
                with ex._shutdown_lock: # pylint:disable=not-context-manager
                    if ex._shutdown:
                        raise RuntimeError('cannot schedule new futures after shutdown')
                    future = ex._threadpool.spawn(fn, *args, **kwargs)
                    proxy_future = _FutureProxy(future)
                    proxy_future.__class__ = concurrent.futures.Future
                    return proxy_future
            ex.submit = submit
            return ex
        MonkeyPoolExecutor = monkey_patch_the_monkey_patchers(GThreadPoolExecutor)
        conf = {"thread_sensitive": False, "executor": MonkeyPoolExecutor(max_workers=1)}
        executor_ = MonkeyPoolExecutor
except Exception as e:
    print(e)
    print('defaulting django_async_orm')
    pass

related:

pythondjangopsycopg2geventeventlet