When I access Kibana, which is setup behind nginx, the dashboard will not load due to an error in the kibana.bundle.js file. If I access Kibana directly (not via the proxy) this error doesn't happen.
The error in the kibana.bundle.js file is different for different browsers (Chrome, FF). This is due to the fact that the file in question seems not to be decompressed properly.
All my browsers are set to accept gzip encoding and this is what is served by Kibana. kibana.bundle.js is a 7mb file that ends up as 1.5 when compressed. As stated above, when I access Kibana directly, that file is decompressed just fine on the client.
Details on my setup:
- ELK (elastichsearch, logstash, kibana) stack setup in one VM, all version 5.4.0
- Kibana is setup for https under port 443 (no server.basePath not configured)
- Both Kibana and elasticsearch are using x-pack, not sure that matters much but it might increase the size of the optimized bundles (such as kibana.bundle.js)
- nginx setup in an other VM running under Ubuntu 14 LTS
- nginx forwards the requests to another proxy. I will spare the details but this setup is what I have to deal with. It is however worth nothing that when removing nginx and simply using the second proxy in front of Kibana it also works nicely.
Details on the nginx configuration:
Simplified version of the server block I have for the Kibana instance. Basically this will forward to the other proxy which in turn will access Kibana at the normal location under port 443.
server {
server_name blah;
listen 443 ssl;
ssl on;
ssl_certificate /etc/nginx/ssl/blah.pem;
ssl_certificate_key /etc/nginx/ssl/blah.pem;
access_log /var/log/nginx/access-blah.log;
error_log /var/log/nginx/error-blah.log warn;
location / {
proxy_pass http://other_proxy;
proxy_http_version 1.1;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
proxy_set_header Upgrade $http_upgrade;
proxy_cache_bypass $http_upgrade;
}
}
Other than that I use default settings for /etc/nginx/nginx.conf. I have tried many options (sendfile off; proxy_buffering off; but none of them worked).
Testing details:
When testing with Chrome or FF I get a normal response with the expected headers, all requests take below 1s to process.
Now when I curl the bundles directly I still get a sub 1s response for the direct call to Kibana, however when I curl via the proxy I get this type of response:
curl -v https://blah:443/bundles/kibana.bundle.js?v=15063
...
99 7111k 99 7087k 0 0 346k 0 0:00:20 0:00:20 --:--:-- 0* TLSv1.2 (IN), TLS alert, Client hello (1):
{ [2 bytes data]
* transfer closed with 8192 bytes remaining to read
* Curl_http_done: called premature == 1
* stopped the pause stream!
99 7111k 99 7103k 0 0 347k 0 0:00:20 0:00:20 --:--:-- 3971
* Closing connection 0
} [5 bytes data]
* TLSv1.2 (OUT), TLS alert, Client hello (1):
} [2 bytes data]
curl: (18) transfer closed with 8192 bytes remaining to read
It goes to 99% almost instantaneously but then it times out after 20 seconds on the remaining 8k bytes. I get errors from nginx saying that upstream closed connection while reading upstream.
I feel like there is a buffer size configuration I am missing somewhere. Hopefully some nginx/kibana expert can help me! :) Thanks!
