Erlang gen_server with long-running tasks

7
votes

Good day,

I have a gen_server process which does some long-running state-updating tasks periodically in handle_info:

handle_info(trigger, State) ->
    NewState = some_long_running_task(),
    erlang:send_after(?LOOP_TIME, self(), trigger),
    {noreply, NewState}.

But when such task runs, then whole server gets unresponsive and any call to it leads to whole server crash:

my_gen_server:status().
** exception exit: {timeout,{gen_server,call,[my_gen_server,status]}}
     in function  gen_server:call/2

How it is possible to avoid blocking of gen_server ? And when one call my_gen_server:status() at any time, the result should be something like: {ok, task_active}

2
erlangerlang-otpgen-server

2 Answers

13
votes

execute the long running task in a separate process. Let this process inform the gen_server of its progress with the task (that is if the task's progress can be tracked) OR let the process complete the task or fail but at least inform the gen_server of the results of the task.

Let the gen_server be linked with the process doing this long running task, and let the gen_server know the PID or registered name so that in case of exit signals, it can isolate the death of that important process from the Rest. 

handle_info(trigger, State) ->
    Pid = spawn_link(?MODULE,some_long_running_task,[State]),
    NewState = save_pid(Pid,State),
    {noreply, NewState};
handle_info({'EXIT',SomePid,_},State)->
    case lookup_pid(State) == SomePid of
        false -> %% some other process
            {noreply,State};
        true ->
            %% our process has died
            %% what do we do now ?
            %% spawn another one ?
            %% thats your decision to take
            ....
            ....
            {noreply,State}
    end;
handle_info({finished,TaskResult},State)->
    .....%% update state e.t.c.
    erlang:send_after(?LOOP_TIME, self(), trigger),
    {noreply,NewState}.

some_long_running_task(ServerState)->
    ....do work
    ....return results
5
votes

This call does not lead to a crash, but simply to an exception which can be caught:

status() ->
  try gen_server:call(my_gen_server, status)
  catch
    exit:{timeout,_} -> {ok, task_active}
  end.

However, the call will remain in the server's queue, and after it finishes handling the current message, it will send a reply message: {ServerRef, Reply}, which should be discarded by the calling process.

The only way to avoid blocking of any process in Erlang (whether gen_server or not) is not to run blocking tasks on it. So another alternative could be to run your long tasks on a different process which only talks to your server, so nobody cares that it's blocked.