Rails 3 Gemfile containing pg gem causes server crash

Question

My application makes https posts to PayPal. With the pg gem specified in the Gemfile, the post operation causes WEBBrick to crash. If I take the pg gem out of the Gemfile, then the crash doesn't happen and my post goes OK. (It took forever to figure out that this is what was causing it, but it is repeatable 100%.)

Even more interesting, if I take the pg gem out of the Gemfile but leave it installed, everything works OK. The minute I put it back into the Gemfile, the crash happens again.

Two questioins, then: a) has anyone seen this? b) what is the Gemfile doing that causes the crash even though the mere presence of the pg gem doesn't cause it?

The code that does the post looks like this (right out of the PayPal SDK):

#   def pay
      path = "/nvp/"
      data = "...."
      server = "api-3t.sandbox.paypal.com"
      port = 443
      header = {"Content-Type" => "html/text"}

      http = Net::HTTP.new(server, port)
      http.use_ssl = true
      http.verify_mode = OpenSSL::SSL::VERIFY_NONE

      resp = http.post2(path, data, header)

      @code = resp.code
   end

Here is the output of the crash when it happens:

    -- Ruby level backtrace information ----------------------------------------
/Users/aehven/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/webrick/server.rb:183:in `block in start_thread'
/Users/aehven/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/webrick/httpserver.rb:70:in `run'
/Users/aehven/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/webrick/httpserver.rb:111:in `service'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/rack-1.2.1/lib/rack/handler/webrick.rb:52:in `service'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/rack-1.2.1/lib/rack/content_length.rb:13:in `call'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/railties-3.0.3/lib/rails/rack/log_tailer.rb:14:in `call'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/railties-3.0.3/lib/rails/application.rb:77:in `method_missing'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/railties-3.0.3/lib/rails/application.rb:168:in `call'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/actionpack-3.0.3/lib/action_dispatch/middleware/static.rb:30:in `call'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/rack-1.2.1/lib/rack/lock.rb:11:in `call'
:10:in `synchronize'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/rack-1.2.1/lib/rack/lock.rb:11:in `block in call'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/activesupport-3.0.3/lib/active_support/cache/strategy/local_cache.rb:72:in `call'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/rack-1.2.1/lib/rack/runtime.rb:17:in `call'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/railties-3.0.3/lib/rails/rack/logger.rb:13:in `call'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/actionpack-3.0.3/lib/action_dispatch/middleware/show_exceptions.rb:46:in `call'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/actionpack-3.0.3/lib/action_dispatch/middleware/remote_ip.rb:48:in `call'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/rack-1.2.1/lib/rack/sendfile.rb:107:in `call'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/actionpack-3.0.3/lib/action_dispatch/middleware/callbacks.rb:44:in `call'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/activesupport-3.0.3/lib/active_support/callbacks.rb:415:in `_run_call_callbacks'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/actionpack-3.0.3/lib/action_dispatch/middleware/callbacks.rb:46:in `block in call'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/activerecord-3.0.3/lib/active_record/connection_adapters/abstract/connection_pool.rb:353:in `call'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/activerecord-3.0.3/lib/active_record/query_cache.rb:31:in `call'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/activerecord-3.0.3/lib/active_record/query_cache.rb:12:in `cache'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/activerecord-3.0.3/lib/active_record/connection_adapters/abstract/query_cache.rb:28:in `cache'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/activerecord-3.0.3/lib/active_record/query_cache.rb:32:in `block in call'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/actionpack-3.0.3/lib/action_dispatch/middleware/cookies.rb:295:in `call'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/actionpack-3.0.3/lib/action_dispatch/middleware/session/abstract_store.rb:149:in `call'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/actionpack-3.0.3/lib/action_dispatch/middleware/flash.rb:182:in `call'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/actionpack-3.0.3/lib/action_dispatch/middleware/params_parser.rb:21:in `call'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/rack-1.2.1/lib/rack/methodoverride.rb:24:in `call'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/actionpack-3.0.3/lib/action_dispatch/middleware/head.rb:14:in `call'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/actionpack-3.0.3/lib/action_dispatch/middleware/best_standards_support.rb:17:in `call'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/actionpack-3.0.3/lib/action_dispatch/routing/route_set.rb:492:in `call'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/rack-mount-0.6.13/lib/rack/mount/route_set.rb:139:in `call'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/rack-mount-0.6.13/lib/rack/mount/code_generation.rb:92:in `recognize'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/rack-mount-0.6.13/lib/rack/mount/code_generation.rb:68:in `optimized_each'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/rack-mount-0.6.13/lib/rack/mount/code_generation.rb:93:in `block in recognize'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/rack-mount-0.6.13/lib/rack/mount/route_set.rb:148:in `block in call'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/actionpack-3.0.3/lib/action_dispatch/routing/route_set.rb:27:in `call'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/actionpack-3.0.3/lib/action_dispatch/routing/route_set.rb:62:in `dispatch'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/actionpack-3.0.3/lib/action_dispatch/routing/route_set.rb:62:in `call'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/actionpack-3.0.3/lib/action_controller/metal.rb:178:in `block in action'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/actionpack-3.0.3/lib/action_controller/metal/rack_delegation.rb:14:in `dispatch'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/actionpack-3.0.3/lib/action_controller/metal.rb:138:in `dispatch'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/actionpack-3.0.3/lib/abstract_controller/rendering.rb:40:in `process'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/actionpack-3.0.3/lib/abstract_controller/base.rb:120:in `process'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/actionpack-3.0.3/lib/action_controller/metal/rescue.rb:17:in `process_action'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/actionpack-3.0.3/lib/action_controller/metal/instrumentation.rb:29:in `process_action'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/activesupport-3.0.3/lib/active_support/notifications.rb:52:in `instrument'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/activesupport-3.0.3/lib/active_support/notifications/instrumenter.rb:21:in `instrument'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/activesupport-3.0.3/lib/active_support/notifications.rb:52:in `block in instrument'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/actionpack-3.0.3/lib/action_controller/metal/instrumentation.rb:30:in `block in process_action'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/actionpack-3.0.3/lib/abstract_controller/callbacks.rb:17:in `process_action'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/activesupport-3.0.3/lib/active_support/callbacks.rb:93:in `run_callbacks'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/activesupport-3.0.3/lib/active_support/callbacks.rb:409:in `_run_process_action_callbacks'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/activesupport-3.0.3/lib/active_support/callbacks.rb:435:in `_run__1503509404736522407__process_action__285881469255263509__callbacks'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/actionpack-3.0.3/lib/abstract_controller/callbacks.rb:18:in `block in process_action'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/actionpack-3.0.3/lib/action_controller/metal/rendering.rb:11:in `process_action'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/actionpack-3.0.3/lib/abstract_controller/base.rb:151:in `process_action'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/actionpack-3.0.3/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
/Users/aehven/Work/Projects/Pico-Tesla/Workspace/trunk/app/controllers/payments_controller.rb:54:in `do_DCC'
/Users/aehven/Work/Projects/Pico-Tesla/Workspace/trunk/vendor/plugins/PayPalSDK/lib/caller.rb:70:in `call'
/Users/aehven/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/net/http.rb:978:in `post'
/Users/aehven/.rvm/gems/ruby-1.9.2-p180@littlenik/gems/rest-client-1.6.1/lib/restclient/net_http_ext.rb:17:in `request'
/Users/aehven/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/net/http.rb:1168:in `request'
/Users/aehven/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/net/http.rb:626:in `start'
/Users/aehven/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/net/http.rb:637:in `do_start'
/Users/aehven/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/net/http.rb:678:in `connect'
/Users/aehven/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/timeout.rb:87:in `timeout'
/Users/aehven/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/timeout.rb:44:in `timeout'
/Users/aehven/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/net/http.rb:678:in `block in connect'
/Users/aehven/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/net/http.rb:678:in `connect'
-

Answer 1

You almost certainly have your PostgreSQL client library (libpq) compiled against a different version of OpenSSL than your Ruby library's OpenSSL extension. I think the reason removing 'pg' from the gemfile works is that gives Ruby's libssl a chance to load first.

To fix this, compile everything you're using that requires SSL against the same version of OpenSSL.