Erlang Message Passing for Large Binaries, Optimizing Performance Recommendations

2
votes

Ok, here's the question I've been thinking about today (which I'll test next week if I don't get an answer before hand).

I have a number of Erlang processes that I need to pass large binaries between. What I had been doing is using gproc:send to locate and send the binary to the appropriate process. However, I'm not sending only a binary, I'm typically sending a tuple such as "{self(),atom, Msg\binary}". At first I was thinking that this would be sufficient to meet the requirements for message processing of large binaries to avoid memory copy operations.

8.2 Process messages

All data in messages between Erlang processes is copied, with the exception >of refc binaries on the same Erlang node.

From:Erlang Efficiency Guide

Now however, I'm starting to wonder, if the Message processor/compiler/etc going to understand there's a >64k binary in there? Or will it see the tuple and just think, let's copy the whole thing to the other process? Like I indicated I do plan on testing this next week to see and I'll gladly update the post with my findings.

I am using R16B03-1.

Thanks!

2
erlang

2 Answers

9
votes

Binaries greater than or equal to 64 bytes in size are stored in a shared area and are reference-counted, and so they are not copied between processes; only references to them are copied.

0
votes

Tuples don't actually contain their elements but rather pointers to them. So every value inside a tuple is also copied when the tuple is. Big (>64) and small binaries have different types underneath even though for a user they seem alike. Therfore VM can use different copy operations.