I found some some articles and the source code of Scala's futures and promises:
- https://viktorklang.com/blog/Futures-in-Scala-2.12-part-9.html
- https://github.com/scala/scala/blob/2.13.x/src/library/scala/concurrent/impl/Promise.scala
- Current implementation with a big comment explaining the promise linking: https://github.com/scala/scala/blob/2.12.x/src/library/scala/concurrent/impl/Promise.scala
- Stress test which leads to memory exhaustion without promise linking: https://github.com/scala/scala/blob/2.12.x/test/files/run/t7336.scala
but I still do not really understand how the promise linking really works. First of all, the memory exhaustion stress test references big arrays in the callbacks. These should cause the big arrays to stay in memory until the callbacks have been executed?
I have tried to write a scenario of the stress test in pseudo code.
Example by showing a call with i = 3:
x4 = call3
call3 = {
val array3 = new BigArray
val f3 = Future { 3 }
x3 = f3.flatMap( 3 => array3; call2 )
return x3
}
call2 = {
val array2 = new BigArray
val f2 = Future { 2 }
x2 = f2.flatMap( 2 => array2; call1 )
return x2
}
call1 = {
val array1 = new BigArray
val f1 = Future { 1 }
x1 = f1.flatMap( 1 => array1; call0 )
return x1
}
call0 = {
val array0 = new BigArray
val f0 = Future { 0 }
x0 = f0.flatMap( 0 => Future.succesful() )
return x0
}
Usually, x0, x1, x2 and x3 would be triggered when f0, f1, f2 and f3 are completed and then would call their function, for example:
1 => array1; call0
so it would call x1.completeWith(call0) which basically is x1.completeWith(x0).
This would lead to the following chain:
x4.completeWith(x3)
x3.completeWith(x2)
x2.completeWith(x1)
x1.completeWith(x0)
x0.completeWith(Future.successful())
From my understanding, since all calls lead to the same result, they can be linked like:
x4.completeWith(Future.successful())
As long as all callbacks from x0, x1, x2 and x3 are moved to x4. All futures/promises become identical?
Now, how does the promise linking behave exactly in Scala 2.13.xx?
Is x4 the root which waits for the other futures to be completed?
Are the other futures/promises now converted into Link[T]?
The method linkRootOf in the Scala 2.13.xx implementation seems to create a new link to the target and store it in the state of the future/promise.
It replaces its callbacks by it.
The callbacks are moved to the root future/promise target to be executed when the root future/promise is completed?
This happens when f0 is completed.
Even if there is a chain of links to the root future/promise, I don't get why it does not leak anymore?
The references of the big arrays are accessed when f0, f1, f2 and f3 are completed since then the callbacks are executed.
But isn't this the time when the links are created, so the references are already gone now? completeWith should be none blocking, so the arrays could be released, even without creating a link?
