Does asyncio supports asynchronous I/O for file operations? If yes, how I can use this in Python 3.5 with async/await syntax code?
47
votes
I mean, i need non-blocking file I/O functionality. But in docs i found only descriptors monitoring functions.
- CthUlhUzzz
If you just want non-blocking IO you should just be able to use python's threads. Do you want async IO or non-blocking IO? stackoverflow.com/questions/319132/… might have some useful info.
- Tom Dalton
Let's explain. I have several Tasks in my IOLoop, that are working with sockets. And I want to add one more Task, that will read data to send from a file. Synchronization will be performed by asyncio.Queue.
- CthUlhUzzz
3 Answers
36
votes
Most operating systems don't support asynchronous file operations.
That's why asyncio doesn't support them either.
See the asyncio wiki for further explanation.
16
votes
That depends on what library you use.
curio offers this functionality, see https://curio.readthedocs.io/en/latest/reference.html#module-curio.file
meanwhile, plain asyncio doesn't, although there are 3rd party libraries, e.g. https://github.com/Tinche/aiofiles (where synchronous file access isolated in threads)
Modern operating systems do provide asynchronous file primitives, but these are varied, thus each would need own implementation. Please compare:
- http://man7.org/linux/man-pages/man7/aio.7.html
- https://msdn.microsoft.com/en-us/library/windows/desktop/aa365683(v=vs.85).aspx
- https://developer.apple.com/library/content/documentation/FileManagement/Conceptual/FileSystemProgrammingGuide/TechniquesforReadingandWritingCustomFiles/TechniquesforReadingandWritingCustomFiles.html
I suspect someone will soon rip out underlying async io from node.js and make a decent Python library, or perhaps someone already has.
Specifically for Linux, there are low-level bindings in https://pypi.org/project/liburing/
For a solid overview of asynchronous IO APIs in Linux, circa 2020, see https://www.scylladb.com/2020/05/05/how-io_uring-and-ebpf-will-revolutionize-programming-in-linux/
14
votes
