environment variables with Redis

Question

I have a Rails app that sends email notifications using Gmail (it's currently only in development mode). I set the username and password with environment variables like this

GMAIL_USERNAME='[e-mail]' GMAIL_PASSWORD='mygreatpassword' rails s

and everything works fine. However, after I added sidekiq/redis in order to send emails as a background job, I'm getting an authentication error when it tries to send the email.

2013-04-24T19:26:45Z 887 TID-ovdxl6qxs WARN: {"retry"=>true, "queue"=>"default", "timeout"=>30, "class"=>"Sidekiq::Extensions::DelayedMailer", "args"=>["---\n- !ruby/class 'Notifier'\n- :answer_updated\n- - !ruby/object:Answer\n    attributes:\n      id: 32\n      content: Billy\n      accepted: \n      user_id: 37\n      question_id: 38\n      created_at: 2013-04-24 19:26:44.948753000 Z\n      updated_at: 2013-04-24 19:26:44.948753000 Z\n  - !ruby/object:User\n    attributes:\n      id: 35\n      email: [e-mail]\n      encrypted_password: $2a$10$5XLIJ6F1KNPOjbTUX3fqROE0oDHgf/WnSGE4OhCm7g.pzN1bzLS0q\n      reset_password_token: \n      reset_password_sent_at: \n      remember_created_at: 2013-04-19 19:28:52.926838000 Z\n      sign_in_count: 9\n      current_sign_in_at: 2013-04-24 19:17:31.694245000 Z\n      last_sign_in_at: 2013-04-22 18:24:29.946303000 Z\n      current_sign_in_ip: 127.0.0.1\n      last_sign_in_ip: 127.0.0.1\n      created_at: 2013-04-18 19:28:09.569895000 Z\n      updated_at: 2013-04-24 19:17:31.695692000 Z\n      name: [e-mail]\n      lawyer: \n      student: \n"], "jid"=>"4c9cb74b27080c4df581715c", "error_message"=>"530-5.5.1 Authentication Required. Learn more at\n", "error_class"=>"Net::SMTPAuthenticationError", "failed_at"=>2013-04-24 19:26:45 UTC, "retry_count"=>0}
2013-04-24T19:26:45Z 887 TID-ovdxl6qxs WARN: 530-5.5.1 Authentication Required. Learn more at

After adding sidekiq, I started the rails server the same way, with the GMAIL_USERNAME and GMAIL_PASSWORD environment variables preceding the 'rails s' command. I also started redis this way

redis-server /usr/local/etc/redis.conf

and sidekiq like this

bundle exec sidekiq

Then, after adding redis, I used sidekiq's 'delay' method instead of 'deliver' to send the message. However, as noted above, I'm getting the authentication error message.

    def after_create(answer)
      answer.question.watchers.each do |user|
      Notifier.delay.answer_updated(answer, user)
      # Notifier.answer_updated(answer, user).deliver  
    end 

Update:

These are my config action mailer settings

config.action_mailer.smtp_settings = {
  address: "smtp.gmail.com",
  port: 587,
  domain: "localhost:3000",
  authentication: "plain",
  enable_starttls_auto: true,
  user_name: ENV["GMAIL_USERNAME"],
  password: ENV["GMAIL_PASSWORD"]
}

This is what it says after "learn more at"

2013-04-25T00:11:30Z 6904 TID-owuofcbno WARN: /Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/1.9.1/net/smtp.rb:948:in `check_response'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/1.9.1/net/smtp.rb:917:in `getok'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/1.9.1/net/smtp.rb:832:in `mailfrom'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/1.9.1/net/smtp.rb:659:in `send_message'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/mail-2.4.4/lib/mail/network/delivery_methods/smtp.rb:145:in `block in deliver!'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/1.9.1/net/smtp.rb:520:in `start'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/mail-2.4.4/lib/mail/network/delivery_methods/smtp.rb:144:in `deliver!'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/mail-2.4.4/lib/mail/message.rb:2034:in `do_delivery'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/mail-2.4.4/lib/mail/message.rb:229:in `block in deliver'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/actionmailer-3.2.12/lib/action_mailer/base.rb:415:in `block in deliver_mail'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/activesupport-3.2.12/lib/active_support/notifications.rb:123:in `block in instrument'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/activesupport-3.2.12/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/activesupport-3.2.12/lib/active_support/notifications.rb:123:in `instrument'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/actionmailer-3.2.12/lib/action_mailer/base.rb:413:in `deliver_mail'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/mail-2.4.4/lib/mail/message.rb:229:in `deliver'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/sidekiq-2.10.1/lib/sidekiq/extensions/action_mailer.rb:23:in `perform'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/sidekiq-2.10.1/lib/sidekiq/processor.rb:49:in `block (3 levels) in process'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/sidekiq-2.10.1/lib/sidekiq/middleware/chain.rb:109:in `call'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/sidekiq-2.10.1/lib/sidekiq/middleware/chain.rb:109:in `block in invoke'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/sidekiq-2.10.1/lib/sidekiq/middleware/server/timeout.rb:11:in `block in call'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/1.9.1/timeout.rb:68:in `timeout'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/sidekiq-2.10.1/lib/sidekiq/middleware/server/timeout.rb:10:in `call'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/sidekiq-2.10.1/lib/sidekiq/middleware/chain.rb:111:in `block in invoke'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/sidekiq-2.10.1/lib/sidekiq/middleware/server/active_record.rb:6:in `call'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/sidekiq-2.10.1/lib/sidekiq/middleware/chain.rb:111:in `block in invoke'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/sidekiq-2.10.1/lib/sidekiq/middleware/server/retry_jobs.rb:50:in `call'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/sidekiq-2.10.1/lib/sidekiq/middleware/chain.rb:111:in `block in invoke'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/sidekiq-2.10.1/lib/sidekiq/middleware/server/logging.rb:11:in `block in call'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/sidekiq-2.10.1/lib/sidekiq/logging.rb:22:in `with_context'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/sidekiq-2.10.1/lib/sidekiq/middleware/server/logging.rb:7:in `call'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/sidekiq-2.10.1/lib/sidekiq/middleware/chain.rb:111:in `block in invoke'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/sidekiq-2.10.1/lib/sidekiq/middleware/chain.rb:114:in `call'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/sidekiq-2.10.1/lib/sidekiq/middleware/chain.rb:114:in `invoke'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/sidekiq-2.10.1/lib/sidekiq/processor.rb:48:in `block (2 levels) in process'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/sidekiq-2.10.1/lib/sidekiq/processor.rb:87:in `stats'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/sidekiq-2.10.1/lib/sidekiq/processor.rb:47:in `block in process'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/celluloid-0.12.4/lib/celluloid/calls.rb:23:in `call'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/celluloid-0.12.4/lib/celluloid/calls.rb:23:in `public_send'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/celluloid-0.12.4/lib/celluloid/calls.rb:23:in `dispatch'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/celluloid-0.12.4/lib/celluloid/future.rb:18:in `block in initialize'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/celluloid-0.12.4/lib/celluloid/internal_pool.rb:48:in `call'
/Users/me/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/celluloid-0.12.4/lib/celluloid/internal_pool.rb:48:in `block in create'

Answer 1

Sidekiq starts it's own rails environment and is missing the environment variables for configuring ActionMailer, this is why you get the exception when using sidekiq, but not when sending within the rails server process itself.

Basically you just have to start the sidekiq worker using the same variables like the server:

GMAIL_USERNAME='[e-mail]' GMAIL_PASSWORD='mygreatpwd' bundle exec sidekiq

Though this might make work okay, consider to use something like foreman to handle this (especially in development).

Answer 2

My guess is that the environment variables you are setting in your console are only being applied to your web server's process (which is started by rails s). So when sidekiq looks at the mailer config it gets back nil from ENV["GMAIL_USERNAME"] and ENV["GMAIL_PASSWORD"]. You need to put those environment variables where the sidekiq process can access them.

I'll lay out a few options below (#2 looks easiest), but there are already a few resources out there that have explained this in more detail (and credit goes to them for compiling these options). Check out:

• http://railsapps.github.io/rails-environment-variables.html
• https://devcenter.heroku.com/articles/config-vars#local-setup
  • If you are deploying on heroku and want to test locally
  • Just remember to put .env in .gitignore

Options

Setting environment vars in bash

I would not recommend this method, but wanted to include it as a counterpoint to the two options below.

export GMAIL_USERNAME="[e-mail]"

Running this in your bash console will create a value in ~/.bashrc accessible in ENV.

This solution, however, is not application-specific (it is available anywhere on your machine), and it is buried so far away from your application you may find it difficult to debug issues or forget to delete/change this private information.

Figaro gem

The figaro gem provides you with a .gitignored YAML file that you can put your sensitive config information.

It also provides helpers for uploading these config vars to heroku, while still keeping the config out of your git repos.

Reading from a yml config file

This is pretty much what the Figaro gem is doing, but it's worth understanding.

# 1. create a config file
touch config/gmail_config.yml
# 2. add the new file to .gitignore
# 3. add code to application.rb to load yml file
config.before_configuration do
  env_file = File.join(Rails.root, 'config', 'local_env.yml')
  YAML.load(File.open(env_file)).each do |key, value|
    ENV[key.to_s] = value
  end if File.exists?(env_file)
end

If the file exits (ie you are on your local machine) it will load up the file and set the variable. If the file doesn't exist, it will not load the file and you will need to set the variable in your production environment.