Disconnected from Phoenix channel when sending multiple large messages in a small time frame

2
votes

First of all I'd like to say that at my company we are huge fans of the Elixir ecosystem and particularly of the Phoenix framework.

Context

Environment

Broker app : Elixir 1.5.1 Phoenix v1.3 Freebsd jail v10.3 amd64

Client app : Elixir 1.5.1 Phoenix v1.3 Mac os X

Problem

When sending multiple messages over the channel, using for example a loop or Enum.each, only few messages manage to get to the broker channel handle_in function. Then the client crash with the following error:

21:45:58.027 [error] ** State machine #PID<0.471.0> terminating ** Last message in was {:send_frame, {:binary, <<31, 139, 8, 0, 0, 0, 0, 0, 0, 3, 188, 187, 231, 206, 235, 74, 146, 45, 248, 42, 131, 51, 63, 217, 183, 232, 93, 253, 163, 104, 69, 47, 209, 115, 106, 80, 160, 247, 222, 179, 113, 223, 125, 184, 247, 62, 231, 84, ...>>}} ** When State == :connected ** Data == {:context,
{:websocket_req, :wss, 'broker.dexyde.com', 443, '/socket/websocket?user_token=SFMyNTY.g3QAAAACZAAEZGF0YW4HAIClCS9hXAVkAAZzaWduZWRuBgC9YkVUXwE.ZRbhkBm4BxzGEXLynPpfXDuGTcuZQMdhgcJszmZd3zo', 30000, :undefined, 1, {:sslsocket, {:gen_tcp, #Port<0.20422>, :tls_connection, :undefined}, #PID<0.476.0>}, {:transport, :ssl, :ssl, :ssl_closed, :ssl_error, [mode: :binary, active: true, verify: :verify_none, packet: 0]}, "12krm8DaxFlr/ZuaS8p6Uw==", 139669, 1, 2, :undefined, :undefined}, {:transport, :ssl, :ssl, :ssl_closed, :ssl_error, [mode: :binary, active: true, verify: :verify_none, packet: 0]}, [], {:wss, 'broker.dexyde.com', 443, '/socket/websocket?user_token=SFMyNTY.g3QAAAACZAAEZGF0YW4HAIClCS9hXAVkAAZzaWduZWRuBgC9YkVUXwE.ZRbhkBm4BxzGEXLynPpfXDuGTcuZQMdhgcJszmZd3zo'}, {Phoenix.Channels.GenSocketClient.Transport.WebSocketClient, %{keepalive: 30000, socket: #PID<0.469.0>}}, <<31, 139, 8, 0, 0, 0, 0, 0, 0, 3, 188, 187, 231, 206, 235, 90, 118, 45, 248, 42, 141, 211, 63, 105, 155, 57, 213, 63, 138, 81, 204, 18, 51, 187, 26, 5, 230, 156, 51, 141, 251, 238, 151, ...>>, false, :undefined, 0} ** Reason for termination = ** {:error, :closed}

We are using binary messages with custom serializer on the broker side. The observed size of the payload is, before zipping it, between 500K and 1M character.

Tests done

  • Add a :timer.sleep(3000) between each push from the client so that the messages arrive in the handle_in function of the broker and it can reply to the handle_reply of the client.
  • Reducing the size of the message payload seems to work too.

Our conclusion...so far

It seems that when a client app is sending multiple large messages over the channel in a small time frame, it got disconnected from the broker channel.

Questions

  • Do anybody has already experience something similar when sending many large message in a small time frame ?
  • Is there a kind of channel or socket limitations to be known ?

Any though or remarks are most welcome on this subject.

Many thanks for your help.

Regards,

Julien

1
elixirphoenix-frameworklarge-datachannel
Just a wild guess: haven’t you tried spawning a Task with a payload to be able to immediately answer with {:noreply, socket}? – Aleksei Matiushkin
Yes tried it. Same problem, the payload won't enter in the handle_in function of the server channel. – J2U
Yes, I also experienced that. It happens very frequently if you type like a bot. – PatNowak

1 Answers