(13: Permission denied) while connecting to upstream:[nginx]

Question

I am working with configuring Django project with Nginx and Gunicorn.

While I am accessing my port gunicorn mysite.wsgi:application --bind=127.0.0.1:8001 in Nginx server, I am getting the following error in my error log file;

2014/05/30 11:59:42 [crit] 4075#0: *6 connect() to 127.0.0.1:8001 failed (13: Permission denied) while connecting to upstream, client: 127.0.0.1, server: localhost, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8001/", host: "localhost:8080"

Below is the content of my nginx.conf file;

server {
    listen 8080;
    server_name localhost;
    access_log  /var/log/nginx/example.log;
    error_log /var/log/nginx/example.error.log;

    location / {
        proxy_pass http://127.0.0.1:8001;
        proxy_set_header X-Forwarded-For $remote_addr;
        proxy_set_header Host $http_host;
    }
}

In the HTML page I am getting 502 Bad Gateway.

What mistake am I doing?

Answer 1

Disclaimer

Make sure there are no security implications for your use-case before running this.

Answer

I had a similar issue getting Fedora 20, Nginx, Node.js, and Ghost (blog) to work. It turns out my issue was due to SELinux.

This should solve the problem:

setsebool -P httpd_can_network_connect 1

Details

I checked for errors in the SELinux logs:

sudo cat /var/log/audit/audit.log | grep nginx | grep denied

And found that running the following commands fixed my issue:

sudo cat /var/log/audit/audit.log | grep nginx | grep denied | audit2allow -M mynginx
sudo semodule -i mynginx.pp

Option #2 (untested, but probably more secure)

setsebool -P httpd_can_network_relay 1

https://security.stackexchange.com/questions/152358/difference-between-selinux-booleans-httpd-can-network-relay-and-httpd-can-net

References

http://blog.frag-gustav.de/2013/07/21/nginx-selinux-me-mad/
https://wiki.gentoo.org/wiki/SELinux/Tutorials/Where_to_find_SELinux_permission_denial_details
http://wiki.gentoo.org/wiki/SELinux/Tutorials/Managing_network_port_labels

Answer 2

I’ve run into this problem too. Another solution is to toggle the SELinux boolean value for httpd network connect to on (Nginx uses the httpd label).

setsebool httpd_can_network_connect on

To make the change persist use the -P flag.

setsebool httpd_can_network_connect on -P

You can see a list of all available SELinux booleans for httpd using

getsebool -a | grep httpd

Answer 3

I have solved my problem by running my Nginx as the user I'm currently logged in with, mulagala.

By default the user as nginx is defined at the very top section of the nginx.conf file as seen below;

user nginx; # Default Nginx user

Change nginx to the name of your current user - here, mulagala.

user mulagala; # Custom Nginx user (as username of the current logged in user)

However, this may not address the actual problem and may actually have casual side effect(s).

For an effective solution, please refer to Joseph Barbere's solution.

Answer 4

Had a similar problem on Centos 7. When I tried to apply the solution prescribed by Sorin, I started moving in cycles. First I had a permission {write} denied. Then when I solved that I had a permission { connectto } denied. Then back again to permission {write } denied.

Following @Sid answer above of checking the flags using getsebool -a | grep httpd and toggling them I found that in addition to the httpd_can_network_connect being off. http_anon_write was also off resulting in permission denied write and permission denied {connectto}

type=AVC msg=audit(1501830505.174:799183): avc:  
denied  { write } for  pid=12144 comm="nginx" name="myroject.sock" 
dev="dm-2" ino=134718735 scontext=system_u:system_r:httpd_t:s0 
tcontext=system_u:object_r:default_t:s0 tclass=sock_file

Obtained using sudo cat /var/log/audit/audit.log | grep nginx | grep denied as explained above.

So I solved them one at a time, toggling the flags on one at a time.

setsebool httpd_can_network_connect on -P

Then running the commands specified by @sorin and @Joseph above

sudo cat /var/log/audit/audit.log | grep nginx | grep denied | 
audit2allow -M mynginx
sudo semodule -i mynginx.pp

Basically you can check the permissions set on setsebool and correlate that with the error obtained from grepp'ing' audit.log nginx, denied

Answer 5

if "502 Bad Gateway" error throws on centos api url for api gateway proxy pass on nginx , run following command to solve the issue

sudo setsebool -P httpd_can_network_connect 1

Answer 6

1. Check the user in /etc/nginx/nginx.conf
2. Change ownership to user.

sudo chown -R nginx:nginx /var/lib/nginx

Now see the magic.

Answer 7

13-permission-denied-while-connecting-to-upstreamnginx on centos server -

setsebool -P httpd_can_network_connect 1

Answer 8

I’ve run into this problem too. I'm using Nginx with HHVM, below solution fixed my issue:

sudo semanage fcontext -a -t httpd_sys_rw_content_t "/etc/nginx/fastcgi_temp(/.*)?"

sudo restorecon -R -v /etc/nginx/fastcgi_temp

Answer 9

1. For first see what is denied:

sudo cat /var/log/audit/audit.log | grep nginx | grep denied

type=AVC msg=audit(1618940614.934:38415): avc:  denied  { connectto } for
pid=18016 comm="nginx" path="/home/deployer/project/tmp/sockets/puma.sock" scontext=system_u:system_r:httpd_t:s0
tcontext=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
tclass=unix_stream_socket permissive=1

2. In my case it helps on CentOS7:

sudo setenforce 0

setsebool httpd_can_network_connect on -P
setsebool httpd_can_network_relay on -P

After you can see what is enable:

getsebool -a | grep httpd

httpd_anon_write --> off
httpd_builtin_scripting --> on
httpd_can_check_spam --> off
httpd_can_connect_ftp --> off
httpd_can_connect_ldap --> off
httpd_can_connect_mythtv --> off
httpd_can_connect_zabbix --> off
httpd_can_network_connect --> on
httpd_can_network_connect_cobbler --> off
httpd_can_network_connect_db --> on
httpd_can_network_memcache --> off
httpd_can_network_relay --> on
httpd_can_sendmail --> off
httpd_dbus_avahi --> off
httpd_dbus_sssd --> off
httpd_dontaudit_search_dirs --> off
httpd_enable_cgi --> off
httpd_enable_ftp_server --> off
httpd_enable_homedirs --> off
httpd_execmem --> off
httpd_graceful_shutdown --> on
httpd_manage_ipa --> off
httpd_mod_auth_ntlm_winbind --> off
httpd_mod_auth_pam --> off
httpd_read_user_content --> off
httpd_run_ipa --> off
httpd_run_preupgrade --> off
httpd_run_stickshift --> off
httpd_serve_cobbler_files --> off
httpd_setrlimit --> off
httpd_ssi_exec --> off
httpd_sys_script_anon_write --> off
httpd_tmp_exec --> off
httpd_tty_comm --> off
httpd_unified --> off
httpd_use_cifs --> off
httpd_use_fusefs --> off
httpd_use_gpg --> off
httpd_use_nfs --> off
httpd_use_openstack --> off
httpd_use_sasl --> off
httpd_verify_dns --> off

Answer 10

Another reason could be; you are accessing your application through nginx using proxy but you did not add gunicorn.sock file for proxy with gunicorn.

You need to add a proxy file path in nginx configuration.

location / {
        include proxy_params;
        proxy_pass http://unix:/home/username/myproject/gunicorn.sock;
    }

Here is a nice tutorial with step by step implementation of this

https://www.digitalocean.com/community/tutorials/how-to-set-up-django-with-postgres-nginx-and-gunicorn-on-ubuntu-16-04#configure-nginx-to-proxy-pass-to-gunicorn

Note: if you did not created anyname.sock file you have to create if first, either use above or any other method or tutorial to create it.