Bokeh server with https via nginx, gunicorn, and flask

Question

I'm having trouble getting bokeh server to render a visualization when using https. I have a similar stack where bokeh server works as expected with http. Additionally, I can get an https connection for a static bokeh.html file. However, passing a script tag to my template.html doesn't work with https.

I have nginx as a reverse proxy in front of gunicorn running a flask app. The flask app pulls a session from a running bokeh server at a particular port. The bokeh server process and the gunicorn/flask process are both monitored by supervisor.

I've looked through the existing bokeh documentation at:

https://docs.bokeh.org/en/latest/docs/user_guide/server.html#reverse-proxying-with-nginx-and-ssl

I've also looked through some similar questions here on SO:

How to configure Nginx with gunicorn and bokeh serve

Bokeh Serve HTTPS instead of HTTP

How to enable SSL/HTTPS on bokeh 0.12.5?

Making script content 'safe' for HTTPS display (Bokeh)

I have tried combinations of options for relative_urls = True/False in the flask app and --allow-websocket-origin = "myIP"/["*"] when spinning up the bokeh server.

At this point I think I'm just missing something obvious and I hope someone else will see it. I think the key point is that https works up and down the stack when serving up a static bokeh.html file or other content, but fails when trying to embed bokeh server. I suspect I'm missing something about how to have flask pull a session.

Thanks in advance!

Below are what I think are all of the relevant sections of configuration and flask.app files:

Contents of: /etc/nginx/sites-available/flask_settings

upstream gunicorn_flask {
    server 127.0.0.1:6000 fail_timeout=0;
}

server {
    listen 443 ssl http2 default_server;
    server_name ###.###.###.###;
    include snippets/self-signed.conf;
    include snippets/ssl-params.conf;
    access_log /var/log/nginx/https_access.log;
    error_log /var/log/nginx/https_error.log;
    ssl on;
    location / {
                proxy_pass http://gunicorn_flask;
                proxy_set_header Upgrade $http_upgrade;
                proxy_set_header Connection "upgrade";
                proxy_http_version 1.1;
                proxy_set_header X-Forwarded_Proto $scheme;
                proxy_set_header X-Forwarded_For $proxy_add_x_forwarded_for;
                proxy_buffering off;
#                proxy_set_header X-Real-IP $remote_addr;
#                proxy_set_header Host $host:$server_port;

               }
}

server {
    listen 80;
    server_name ###.###.###.###;
    return 301 https://$server_name$request_uri;
}

Contents of: /etc/nginx/snippets/self-signed.conf

ssl_certificate /path/to/my/file.crt;
ssl_certificate_key /path/to/my/file.key;

Contents of: /etc/nginx/snippets/ssl-params.conf

ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_ciphers "ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4";
ssl_ecdh_curve secp384r1;
ssl_session_cache shared:SSL:10m;
ssl_session_tickets off;
ssl_stapling on;
ssl_stapling_verify on;
resolver 8.8.8.8 8.8.4.4 valid=300s;
resolver_timeout 5s;
add_header Strict-Transport-Security "max-age=63072000";
ssl_dhparam /etc/ssl/certs/dhparam.pem;

Contents of: /etc/supervisor/conf.d/bokeh.conf

[program:my_bokeh_viz]
directory=/my/path/to/bokeh/viz
command=/my/path/to/bokeh/bin/bokeh serve --allow-websocket-origin=###.###.###.### --address=127.0.0.1 --port 6001 --use-xheaders /my/path/to/bokeh/viz.py
autostart=true
autorestart=true
stderr_logfile=/var/log/my_bokeh_viz/Bokeh_Flask.err.log
stdout_logfile=/var/log/my_bokeh_viz/Bokeh_Flask.out.log

Contents of: /etc/supervisor/conf.d/gunicorn.conf

[program:gunicorn_bokeh]
directory=/my/path/to/flask/app
command=/my/path/to/anaconda3/bin/gunicorn my_flask_app:app --bind 127.0.0.1:6000 --pythonpath /my/path/to/flask/app
autostart=true
autorestart=true
stderr_logfile=/var/log/gunicorn_bokeh/Bokeh_Flask.err.log
stdout_logfile=/var/log/gunicorn_bokeh/Bokeh_Flask.out.log

Contents of: /path/to/my/flask/app.py

--- Relevant section of flask app.py
@app.route('/web/path/to/bokeh/viz', methods = ['GET'])
@login_required
def show_bokeh_viz():

  host_url = 'http://127.0.0.1:6001/my_bokeh_viz'
  my_session = pull_session(url = host_url)
  client_url = 'http://###.###.###.###/web/path/to/bokeh/viz'
  my_script = server_session(model = None,
                             session_id = my_session.id,
                             url = client_url,
                             relative_urls = False)
  return render_template("embed.html", script = my_script, template = "Flask")

@app.route('/testing', methods = ['GET'])
@login_required
def show_test():
  return render_template("stocks.html")

Contents of: /path/to/my/flask/app/templates/embed.html

<!doctype html>
<html>
  <body>
    {{ script|safe }}
  </body>
</html>

Page source from rendered result:

<!doctype html>
<html>
  <body>

<script
    src="https://###.###.###.###/web/path/to/my/bokeh/viz/autoload.js?bokeh-autoload-element=a4c3a28c-8bab-4e54-8ba8-8423914078aa&bokeh-app-path=/web/path/to/my/bokeh/viz&bokeh-absolute-url=https://###.###.###.###/web/path/to/my/bokeh/viz&bokeh-session-id=2GYRA1MPiBBX0qzlIB5ZEMyWQkV9AD1Br1KdgtZoew5P"
id="a4c3a28c-8bab-4e54-8ba8-8423914078aa"
    data-bokeh-model-id=""
    data-bokeh-doc-id=""
></script>
  </body>
</html> 

Answer 1

There are a couple of bits I think. By creating a bokeh session you actually just create the script that the browser calls to do bokeh things. In browser it will try to call to whatever you give it, and we need it to call to https://example.com/..... and for nginx to send the bokeh traffic to the bokeh server rather than the gunicorn server.

Best solution I found was to prefix all the bokeh server_session/server_document calls with something so that nginx can send them to the bokeh server rather than to flask and then strip out the prefix with nginx. I tried using the bokeh serve --prefix tag but had some problems with loading static assets.

@app.route('/web/path/to/bokeh/viz', methods = ['GET'])
@login_required
def show_bokeh_viz():
  my_session = pull_session(url = http://localhost:6001/my_bokeh_viz)

  # use actual url here
  host_url = 'https://example.com/bokeh_stuff/my_bokeh_viz'
  my_script = server_session(model = None,
                             session_id = my_session.id,
                             url = url,
                             relative_urls = False)
  return render_template("embed.html", script = my_script, template = "Flask")

relevant nginx bit:

# All bokeh plots are prefixed with some unique route, in this case 'bokeh_stuff'
# Here we will collect these, strip off the prefix, and send the result to the bokeh server
# Sends https://example.com/bokeh_stuff/sliders ⇒ http://localhost:6001/sliders
location /bokeh_stuff  {
    rewrite  ^/bokeh_stuff/(.*)  /$1 break;
    proxy_pass http://127.0.0.1:6001;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
    proxy_http_version 1.1;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $host:$server_port;
    proxy_buffering off;
}

# All traffic other than static and bokeh_stuff goes to the gunicorn server
location / {
    # forward application requests to the gunicorn server
    proxy_pass http://localhost:6000;
    proxy_redirect off;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}

Then in bokeh serve you need to add --allow-websocket-origin=example.com

To test what's going on have a look at the console and network requests as well as the bokeh server output (supervisor logs) and see what it gives you. You can also head to example.com/bokeh_stuff and should get the index page even if something is still wrong with your route functions.