I recently came across this article about python memory allocation.
In this page it describes the memory usage of python and in there there is an example showing deepcopy of list of integers. I did the benchmark myself on Python 2.7
Line # Mem usage Increment Line Contents
================================================
4 28.051 MiB 0.000 MiB @profile
5 def function():
6 59.098 MiB 31.047 MiB x = list(range(1000000)) # allocate a big list
7 107.273 MiB 48.176 MiB y = copy.deepcopy(x)
8 99.641 MiB -7.633 MiB del x
9 99.641 MiB 0.000 MiB return y
so delete x directly only removes x and all the references to integer to x right?
Doing this could not help either (So what is the difference del x and del x[:]?):
Line # Mem usage Increment Line Contents
================================================
4 28.047 MiB 0.000 MiB @profile
5 def function():
6 59.094 MiB 31.047 MiB x = list(range(1000000)) # allocate a big list
7 107.270 MiB 48.176 MiB y = copy.deepcopy(x)
8 99.637 MiB -7.633 MiB del x[:]
9 99.637 MiB 0.000 MiB return y
And in contrast to deepcopy, if I use copy, after deletion seems the memory restores to previous state when x is newly created
Line # Mem usage Increment Line Contents
================================================
4 28.039 MiB 0.000 MiB @profile
5 def function():
6 59.090 MiB 31.051 MiB x = list(range(1000000)) # allocate a big list
7 66.895 MiB 7.805 MiB y = copy.copy(x)
8 59.262 MiB -7.633 MiB del x[:]
9 59.262 MiB 0.000 MiB return y
For dict:
Line # Mem usage Increment Line Contents
================================================
4 28.051 MiB 0.000 MiB @profile
5 def function():
6 100.523 MiB 72.473 MiB x = dict((e, e) for e in xrange(1000000))
7 183.398 MiB 82.875 MiB y = copy.deepcopy(x)
8 135.395 MiB -48.004 MiB del x
9 135.395 MiB 0.000 MiB return y
And for list of lists (compare to list of integers, I assume that del x or del x[:] only removes that huge array list on heap?):
Line # Mem usage Increment Line Contents
================================================
4 28.043 MiB 0.000 MiB @profile
5 def function():
6 107.691 MiB 79.648 MiB x = [[] for _ in xrange(1000000)]
7 222.312 MiB 114.621 MiB y = copy.deepcopy(x)
8 214.680 MiB -7.633 MiB del x[:]
9 214.680 MiB 0.000 MiB return y
So I want to ask:
- So if there just no way to claim back those memory occupied by integers? Integer is an object as well right? Why memory does not get released at all? Just integer cannot be claimed? Or float and string as well? Object references as well?
- Why there is -7 MB for memory? Is it because that the list, implemented as array list, is freed from heap?
- whether it is a list or a dict, del x can only free the data structure itself (what i mean is that the array list structure, or dict structure), but integers, objects references can be marked as free, but not returned to system?
And how do I or if there is a way to free all the underlining lists in x in this example?
Line # Mem usage Increment Line Contents
================================================
4 28.047 MiB 0.000 MiB @profile
5 def function():
6 248.008 MiB 219.961 MiB x = [list(range(10)) for _ in xrange(1000000)]
7 502.195 MiB 254.188 MiB y = copy.deepcopy(x)
8 494.562 MiB -7.633 MiB del x[:]
9 494.562 MiB 0.000 MiB return y