Persist job results with Resque / Rails

Question

In my Rails app, I'm trying to take my working API calls and have them handled by background workers.

I have the following in app/jobs/api_request_job.rb:

class ApiRequestJob
  def self.perform(params)
    Query.new(params).start   
  end
end

The Query class is where the HTTParty requests are being executed (there are lots of methods for different query types with the same basic format as the parks method:

require 'ostruct'

class Query
  include FourSquare

  attr_reader   :results,
                :first_address,
                :second_address,
                :queries, 
                :radius

  def initialize(params)
    @results        = OpenStruct.new
    @queries        = params["query"]
    @first_address  = params["first_address"]
    @second_address = params["second_address"] 
    @radius         = params["radius"].to_f 
  end

  def start
    queries.keys.each do |query|
      results[query] = self.send(query)
    end
    results
  end

def parks
  category_id = "4bf58dd8d48988d163941735"
  first_address_results = FourSquare.send_request(@first_address, radius_to_meters, category_id)["response"]["venues"]
  second_address_results = FourSquare.send_request(@second_address, radius_to_meters, category_id)["response"]["venues"]
  response = [first_address_results, second_address_results]
end

And, finally, the controller. Before trying to farm this action out to background workers, this line was working fine: @results = Query.new(params).start

class ComparisonsController < ApplicationController
  attr_reader :first_address, :second_address

  def new
  end

  def show
    @first_address = Address.new(params["first_address"])
    @second_address = Address.new(params["second_address"])
    if @first_address.invalid?
      flash[:notice] = @first_address.errors.full_messages
      redirect_to :back    
    elsif Query.new(params).queries.nil? 
      flash[:notice] = "You must choose at least one criteria for your comparison."
      redirect_to comparisons_new_path(request.params)
    else
      @queries = params["query"].keys
      @results = Resque.enqueue(ApiRequestJob, params)  # <-- this is where I'm stuck
    end
  end

end

I'm running redis, have resque installed, and am running the task/starting the workers. The current value being returned for @results is true instead of the hash of results I was need to get back. Is there a way to have the results of the Resque job persist and return data instead of true? What am I missing about how to get background workers to return the same type of data my regular api calls were returning?

Many thanks in advance!

rafb3 rafb3 · Accepted Answer · 2014-12-03T21:52:35

The true you are receiving means the job was scheduled enqueued successfully. The worker will pick it up and run it on the background asynchronously, which means, not at same time as the thread that enqueued it. So there's no way to retrieve the returned value from the job.

If you need the value from that process, you have to run it from the controller without the worker. Also, you wouldn't gain anything from just pushing the work to be done by another process as the web process would have to wait for the response to then keep going anyway.

If you need that returned value right away and are doing this for performance reasons, then you could look into other forms of concurrency, like having another thread doing the request and then only grabbing the result when you need it on the view like:

class AsyncValue
  def initialize(&block)
    @thr = Thread.new(&block)
  end

  def value
    @thr.join
  end
end

on the controller

@results = AsyncValue.new { Query.new(params).start  }

and on the view

<%= @results.value.each .... %>

but you'd still have to work around error handling which can get pretty complicated, but is doable.

Personally, I'd just make the request in place, but you know your domain better than me.

Persist job results with Resque / Rails

1 Answers