Binding Chef-server 12 to specific IP address: Connection refused connecting to https://127.0.0.1/users/, retry 1/5

1
votes

We have multiple applications running on the same server but on a different IPs (192.168.50.130:443 applicationA, 192.168.50.131:443 applicationB and so on)

We wanted to install chef server on the same machine. By default chef server's nginx listens on all IP address (0.0.0.0:443 and 0.0.0.0:80). As we had other application running on the same port 443, Chef couldn't bind its default ip address (0.0.0.0) to the port 443. So we changed the chef's nginx cookbooks to bind to a specific ip address. Now that we have chef server's nginx running on specific ip, 192.168.56.137:443. Just so you know we were also successfully able bring up chef.example.ca on a browser after changing it to a specific ip address.

The issue really is when we tried to create users using chef-server-ctl create-user command. We get the following error.

chef-server-ctl user-create user_name first_name last_name email password --filename FILE_NAME

Getting the error 5 times and message saying it gives up.

ERROR: Connection refused connecting to https://127.0.0.1/users/, retry 1/5

Here is my /etc/opscode/chef-server.rb

server_name = "chef.example.ca"
api_fqdn = server_name
notification_email = "NO-REPLY-CHEF@example.ca"

#######################
# nginx configuration #
#######################
nginx['url'] = "https://#{server_name}"
nginx['server_name'] = server_name
#nginx['ssl_certificate'] = "/etc/nginx-certs/chef/chef.example.ca.crt"
#nginx['ssl_certificate_key'] = "/etc/nginx-certs/chef/chef.example.ca.key"
#nginx['ssl_protocols'] = "TLSv1 TLSv1.1 TLSv1.2"

/etc/hosts file:

  192.168.56.137 chef.example.ca

Could you please help me understand why chef is still looking to connect via localhost (127.0.0.1) while the configuration for server_name and api_fqdn is "chef.example.ca". Also can this ip address be changed to 192.168.56.137? Thank you!

After changing the chef's nginx cookbook. I was able to enforce chef server to listen to on a specific ip address like I wanted.

 # default behavior to listen only on v4
      "listen  "+  node['private_chef']['nginx']['server_name'] +":#{listen_port};"

But when I tried to login into the web gui, I get the following error enter image description here

Chef-manage-ctl tail

2016-04-14_14:07:36.98112 I, [2016-04-14T11:07:36.974593 #51397]  INFO -- : Started GET "/login" for 127.0.0.1 at 2016-04-14 11:07:36 -0300
2016-04-14_14:07:37.07912 I, [2016-04-14T11:07:37.077410 #51397]  INFO -- : Processing by SessionsController#new as HTML
2016-04-14_14:07:37.40107 I, [2016-04-14T11:07:37.397369 #51397]  INFO -- :   Rendered sessions/new.html.erb within layouts/outside (171.2ms)
2016-04-14_14:07:37.43189 I, [2016-04-14T11:07:37.428147 #51397]  INFO -- :   Rendered shared/_header.html.erb (2.4ms)
2016-04-14_14:07:37.44882 I, [2016-04-14T11:07:37.447813 #51397]  INFO -- :   Rendered shared/_footer.html.haml (4.6ms)
2016-04-14_14:07:37.47773 I, [2016-04-14T11:07:37.476979 #51397]  INFO -- :   Rendered layouts/_client_config.html.erb (2.1ms)
2016-04-14_14:07:37.47985 I, [2016-04-14T11:07:37.479065 #51397]  INFO -- :   Rendered shared/_js.html.erb (0.9ms)
2016-04-14_14:07:37.48226 I, [2016-04-14T11:07:37.480695 #51397]  INFO -- :   Rendered shared/_marketo.html.erb (0.5ms)
2016-04-14_14:07:37.48230 I, [2016-04-14T11:07:37.481110 #51397]  INFO -- : Completed 200 OK in 404ms (Views: 397.9ms)
2016-04-14_14:07:43.69537 I, [2016-04-14T11:07:43.691233 #51397]  INFO -- : Started POST "/login" for 127.0.0.1 at 2016-04-14 11:07:43 -0300
2016-04-14_14:07:43.69540 I, [2016-04-14T11:07:43.692915 #51397]  INFO -- : Processing by SessionsController#create as HTML
2016-04-14_14:07:43.69541 I, [2016-04-14T11:07:43.692999 #51397]  INFO -- :   Parameters: {"utf8"=>"✓", "authenticity_token"=>"ZF7NM8/Rtr+ObtRxdUZcIQfSohbaKYtGi751bHbm/HkHfGejnQ1peCG+lFXuHYsFS5e/OS6HtxZnU+rC5d+qgw==", "to"=>"", "username"=>"srinivd1", "password"=>"[FILTERED]", "commit"=>"Sign In"}
2016-04-14_14:07:43.95797 E, [2016-04-14T11:07:43.956971 #51397] ERROR -- : Connection refused - Connection refused connecting to https://localhost:444/authenticate_user, giving up
2016-04-14_14:07:43.95799 E, [2016-04-14T11:07:43.957083 #51397] ERROR -- : /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/chef-11.16.2/lib/chef/http.rb:305:in `rescue in retrying_http_errors'
2016-04-14_14:07:43.95799 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/chef-11.16.2/lib/chef/http.rb:291:in `retrying_http_errors'
2016-04-14_14:07:43.95800 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/chef-11.16.2/lib/chef/http.rb:256:in `send_http_request'
2016-04-14_14:07:43.95800 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/chef-11.16.2/lib/chef/http.rb:143:in `request'
2016-04-14_14:07:43.95800 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/chef-11.16.2/lib/chef/http.rb:126:in `post'
2016-04-14_14:07:43.95800 /opt/chef-manage/embedded/service/chef-manage/app/controllers/sessions_controller.rb:86:in `authenticate'
2016-04-14_14:07:43.95800 /opt/chef-manage/embedded/service/chef-manage/app/controllers/sessions_controller.rb:27:in `create'
2016-04-14_14:07:43.95800 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/actionpack-4.2.5.2/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
2016-04-14_14:07:43.95801 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/actionpack-4.2.5.2/lib/abstract_controller/base.rb:198:in `process_action'
2016-04-14_14:07:43.95801 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/actionpack-4.2.5.2/lib/action_controller/metal/rendering.rb:10:in `process_action'
2016-04-14_14:07:43.95801 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/actionpack-4.2.5.2/lib/abstract_controller/callbacks.rb:20:in `block in process_action'
2016-04-14_14:07:43.95802 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/activesupport-4.2.5.2/lib/active_support/callbacks.rb:117:in `call'
2016-04-14_14:07:43.95802 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/activesupport-4.2.5.2/lib/active_support/callbacks.rb:117:in `call'
2016-04-14_14:07:43.95802 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/activesupport-4.2.5.2/lib/active_support/callbacks.rb:555:in `block (2 levels) in compile'
2016-04-14_14:07:43.95802 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/activesupport-4.2.5.2/lib/active_support/callbacks.rb:505:in `call'
2016-04-14_14:07:43.95802 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/activesupport-4.2.5.2/lib/active_support/callbacks.rb:505:in `call'
2016-04-14_14:07:43.95803 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/activesupport-4.2.5.2/lib/active_support/callbacks.rb:92:in `__run_callbacks__'
2016-04-14_14:07:43.95804 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/activesupport-4.2.5.2/lib/active_support/callbacks.rb:778:in `_run_process_action_callbacks'
2016-04-14_14:07:43.95804 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/activesupport-4.2.5.2/lib/active_support/callbacks.rb:81:in `run_callbacks'
2016-04-14_14:07:43.95804 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/actionpack-4.2.5.2/lib/abstract_controller/callbacks.rb:19:in `process_action'
2016-04-14_14:07:43.95804 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/actionpack-4.2.5.2/lib/action_controller/metal/rescue.rb:29:in `process_action'
2016-04-14_14:07:43.95804 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/actionpack-4.2.5.2/lib/action_controller/metal/instrumentation.rb:32:in `block in process_action'
2016-04-14_14:07:43.95805 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/activesupport-4.2.5.2/lib/active_support/notifications.rb:164:in `block in instrument'
2016-04-14_14:07:43.95805 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/activesupport-4.2.5.2/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
2016-04-14_14:07:43.95805 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/activesupport-4.2.5.2/lib/active_support/notifications.rb:164:in `instrument'
2016-04-14_14:07:43.95805 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/actionpack-4.2.5.2/lib/action_controller/metal/instrumentation.rb:30:in `process_action'
2016-04-14_14:07:43.95806 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/actionpack-4.2.5.2/lib/action_controller/metal/params_wrapper.rb:250:in `process_action'
2016-04-14_14:07:43.95806 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/actionpack-4.2.5.2/lib/abstract_controller/base.rb:137:in `process'
2016-04-14_14:07:43.95806 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/actionview-4.2.5.2/lib/action_view/rendering.rb:30:in `process'
2016-04-14_14:07:43.95806 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/actionpack-4.2.5.2/lib/action_controller/metal.rb:196:in `dispatch'
2016-04-14_14:07:43.95806 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/actionpack-4.2.5.2/lib/action_controller/metal/rack_delegation.rb:13:in `dispatch'
2016-04-14_14:07:43.95807 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/actionpack-4.2.5.2/lib/action_controller/metal.rb:237:in `block in action'
2016-04-14_14:07:43.95807 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/actionpack-4.2.5.2/lib/action_dispatch/routing/route_set.rb:74:in `call'
2016-04-14_14:07:43.95807 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/actionpack-4.2.5.2/lib/action_dispatch/routing/route_set.rb:74:in `dispatch'
2016-04-14_14:07:43.95807 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/actionpack-4.2.5.2/lib/action_dispatch/routing/route_set.rb:43:in `serve'
2016-04-14_14:07:43.95807 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/actionpack-4.2.5.2/lib/action_dispatch/journey/router.rb:43:in `block in serve'
2016-04-14_14:07:43.95807 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/actionpack-4.2.5.2/lib/action_dispatch/journey/router.rb:30:in `each'
2016-04-14_14:07:43.95808 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/actionpack-4.2.5.2/lib/action_dispatch/journey/router.rb:30:in `serve'
2016-04-14_14:07:43.95808 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/actionpack-4.2.5.2/lib/action_dispatch/routing/route_set.rb:815:in `call'
2016-04-14_14:07:43.95808 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/rack-1.6.4/lib/rack/etag.rb:24:in `call'
2016-04-14_14:07:43.95808 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/rack-1.6.4/lib/rack/conditionalget.rb:38:in `call'
2016-04-14_14:07:43.95809 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/rack-1.6.4/lib/rack/head.rb:13:in `call'
2016-04-14_14:07:43.95809 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/actionpack-4.2.5.2/lib/action_dispatch/middleware/params_parser.rb:27:in `call'
2016-04-14_14:07:43.95809 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/actionpack-4.2.5.2/lib/action_dispatch/middleware/flash.rb:260:in `call'
2016-04-14_14:07:43.95809 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/rack-1.6.4/lib/rack/session/abstract/id.rb:225:in `context'
2016-04-14_14:07:43.95809 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/rack-1.6.4/lib/rack/session/abstract/id.rb:220:in `call'
2016-04-14_14:07:43.95810 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/actionpack-4.2.5.2/lib/action_dispatch/middleware/cookies.rb:560:in `call'
2016-04-14_14:07:43.95810 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/actionpack-4.2.5.2/lib/action_dispatch/middleware/callbacks.rb:29:in `block in call'
2016-04-14_14:07:43.95810 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/activesupport-4.2.5.2/lib/active_support/callbacks.rb:88:in `__run_callbacks__'
2016-04-14_14:07:43.95810 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/activesupport-4.2.5.2/lib/active_support/callbacks.rb:778:in `_run_call_callbacks'
2016-04-14_14:07:43.95811 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/activesupport-4.2.5.2/lib/active_support/callbacks.rb:81:in `run_callbacks'
2016-04-14_14:07:43.95811 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/actionpack-4.2.5.2/lib/action_dispatch/middleware/callbacks.rb:27:in `call'
2016-04-14_14:07:43.95811 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/actionpack-4.2.5.2/lib/action_dispatch/middleware/remote_ip.rb:78:in `call'
2016-04-14_14:07:43.95811 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/actionpack-4.2.5.2/lib/action_dispatch/middleware/debug_exceptions.rb:17:in `call'
2016-04-14_14:07:43.95811 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/actionpack-4.2.5.2/lib/action_dispatch/middleware/show_exceptions.rb:30:in `call'
2016-04-14_14:07:43.95812 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/railties-4.2.5.2/lib/rails/rack/logger.rb:38:in `call_app'
2016-04-14_14:07:43.95812 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/railties-4.2.5.2/lib/rails/rack/logger.rb:22:in `call'
2016-04-14_14:07:43.95812 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/actionpack-4.2.5.2/lib/action_dispatch/middleware/request_id.rb:21:in `call'
2016-04-14_14:07:43.95812 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/rack-1.6.4/lib/rack/methodoverride.rb:22:in `call'
2016-04-14_14:07:43.95812 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/rack-1.6.4/lib/rack/runtime.rb:18:in `call'
2016-04-14_14:07:43.95812 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/activesupport-4.2.5.2/lib/active_support/cache/strategy/local_cache_middleware.rb:28:in `call'
2016-04-14_14:07:43.95813 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/actionpack-4.2.5.2/lib/action_dispatch/middleware/static.rb:116:in `call'
2016-04-14_14:07:43.95813 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/rack-1.6.4/lib/rack/sendfile.rb:113:in `call'
2016-04-14_14:07:43.95813 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/actionpack-4.2.5.2/lib/action_dispatch/middleware/ssl.rb:24:in `call'
2016-04-14_14:07:43.95813 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/sentry-raven-0.15.1/lib/raven/integrations/rack.rb:54:in `call'
2016-04-14_14:07:43.95814 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/railties-4.2.5.2/lib/rails/engine.rb:518:in `call'
2016-04-14_14:07:43.95814 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/railties-4.2.5.2/lib/rails/application.rb:165:in `call'
2016-04-14_14:07:43.95814 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/railties-4.2.5.2/lib/rails/railtie.rb:194:in `public_send'
2016-04-14_14:07:43.95814 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/railties-4.2.5.2/lib/rails/railtie.rb:194:in `method_missing'
2016-04-14_14:07:43.95814 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/unicorn-4.9.0/lib/unicorn/http_server.rb:580:in `process_client'
2016-04-14_14:07:43.95814 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/unicorn-worker-killer-0.4.3/lib/unicorn/worker_killer.rb:52:in `process_client'
2016-04-14_14:07:43.95815 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/unicorn-worker-killer-0.4.3/lib/unicorn/worker_killer.rb:92:in `process_client'
2016-04-14_14:07:43.95815 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/unicorn-4.9.0/lib/unicorn/http_server.rb:674:in `worker_loop'
2016-04-14_14:07:43.95815 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/unicorn-4.9.0/lib/unicorn/http_server.rb:529:in `spawn_missing_workers'
2016-04-14_14:07:43.95816 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/unicorn-4.9.0/lib/unicorn/http_server.rb:140:in `start'
2016-04-14_14:07:43.95816 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/gems/unicorn-4.9.0/bin/unicorn:126:in `<top (required)>'
2016-04-14_14:07:43.95816 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/bin/unicorn:23:in `load'
2016-04-14_14:07:43.95816 /opt/chef-manage/embedded/service/gem/ruby/2.2.0/bin/unicorn:23:in `<main>'
2016-04-14_14:07:44.02026 I, [2016-04-14T11:07:44.016528 #51397]  INFO -- :   Rendered errors/error_500.haml within layouts/outside (2.0ms)
2016-04-14_14:07:44.02028 I, [2016-04-14T11:07:44.018118 #51397]  INFO -- :   Rendered shared/_header.html.erb (0.4ms)
2016-04-14_14:07:44.02029 I, [2016-04-14T11:07:44.018601 #51397]  INFO -- :   Rendered shared/_footer.html.haml (0.3ms)
2016-04-14_14:07:44.02029 I, [2016-04-14T11:07:44.019163 #51397]  INFO -- :   Rendered layouts/_client_config.html.erb (0.4ms)
2016-04-14_14:07:44.02029 I, [2016-04-14T11:07:44.019458 #51397]  INFO -- :   Rendered shared/_js.html.erb (0.1ms)
2016-04-14_14:07:44.02029 I, [2016-04-14T11:07:44.019674 #51397]  INFO -- :   Rendered shared/_marketo.html.erb (0.1ms)
2016-04-14_14:07:44.02029 I, [2016-04-14T11:07:44.019874 #51397]  INFO -- : Completed 500 Internal Server Error in 327ms (Views: 5.9ms)
2
chef-infra
You should not change the internal chef cookbook, there's a configuration file name chef-server.rb to fine tune chef-server install and reconfigure tools accordingly. Documentation here - Tensibai
Yeah I have seen that documentation. Unfortunately there is no configuration to tune nginx's 443 and 80 listening port! - Dinesh
check the chef-server-running.json file in /etc/opscode you'll find the proper values. You didn't say if you had run chef-server-ctl reconfigure after changing the. Rb file (sorry for the formatting, typing on phone) - Tensibai
All settings here: docs.chef.io/config_rb_server_optional_settings.html - Tensibai
@Tensibai - I've seen this documentation. Could you be more specific on what I should changes? :) - Dinesh

2 Answers

1
votes

Ok, so after some digging and checks, there's no native way to restrict the IP on which nginx is listening, so the best way would be to use another port.

As your original problem is the SSL port, a supported way to have chef next another web server using 443 is to change nginx ssl port (from The documentation with this kind of chef-server.rb configuration:

nginx['ssl_port']=444

Then you have to configure your clients accordingly to use the correct port when you give them the chef-server url.

Reminder: after any change in chef-server.rb you have to run chef-server-ctl reconfigure to update services configuration and command line tools.

If you're wondering why restrincting on a specific IP is not supported, here's the results of my digging in the code with an amended chef-server.rb which should work at end I highly discourage this use, it will broke at every update unless a pull request is made to change this behavior:

The root cause is this line in the nginx erb helper used by the nginx_chef_api_lb.conf.erb template.

  # default behavior to listen only on v4
  "listen #{listen_port};"

So you have to modify the private-chef cookbook to make it listen on a specific IP, it may worth a PR to include the lb['vip'] as part of the template. (Side note: I had swear it was already used this way)

Then for everything to work properly from the outside, nginx should now what the server is, using api_fqdn should be enough as all other attributes takes this value if they're not set (excerpt from here:

def gen_api_fqdn_default
  PrivateChef["lb"]["api_fqdn"] ||= PrivateChef['api_fqdn']
  PrivateChef["lb"]["web_ui_fqdn"] ||= PrivateChef['api_fqdn']
  PrivateChef["nginx"]["server_name"] ||= PrivateChef['api_fqdn']
  PrivateChef["nginx"]["url"] ||= "https://#{PrivateChef['api_fqdn']}"
end

but then there's the chef-server-ctl configuration (namely pivotal.rb) which takes the lb['vip'] value and not the api_fqdn value.

So a working chef-server.rb file may be:

api_fqdn = "chef.example.ca"
lb['vip'] = server_name
notification_email = "NO-REPLY-CHEF@example.ca"
1
votes

I think setting you want is lb['vip'] = 'whatever'. That said, Chef Server is an appliance and expects to control the box entirely. You shouldn't run it on the same machine as other services. Either put it in its own VM or use something like LXC/D to lock it in a container.