Python GC memory compaction

0
votes

According to my knowledge cpython has GC but it does not support memory compaction, which means that python long running processes can suffer high memory fragmentation/consumption.

I've read few threads on this topic:

https://lists.gt.net/python/python/1163027#1163027

https://lists.gt.net/python/python/1162114

https://dzone.com/articles/python-memory-issues-tips-and-tricks

If I understood first two threads correctly the problem lays in how the C memory allocator in Python works.

How this can be a problem if Java hotspot is written in C as well ? what am I missing?

Why cpython GC does not support memory compaction and as far as I know there are no plans to introduce it?

1
pythonmemorygarbage-collectioncpython

1 Answers

0
votes

The problem isn't so much CPython's memory allocator but CPython's C APIs. Java's native interface (the JNI) is carefully designed to isolate users from virtual machine implementation details such as garbage collection. By contrast, CPython's C API exposes much more of its implementation including memory layout and reference counting. Reference counting is baked into every C extension ever written. It is possible to emulate the CPython API on a VM with a different GC design—PyPy's cpyext support does this—but that emulation isn't perfect and adds overhead.