Haskell installation in docker container using stack failing: too many open files

Question

I have a simple Dockerfile

FROM haskell:8
WORKDIR "/root"
CMD ["/bin/bash"]

which I run mounting pwd folder to "/root". In my current folder I have a Haskell project that uses stack (funblog). I configured in stack.yml to use "lts-7.20" resolver, which aims to install ghc-8.0.1.

Inside the container, after running "stack update", I ran "stack setup" but I am getting "Too many open files in system" during GHC compilation.

This is my stack.yaml

flags: {}
packages:
- '.'
- location:
      git: https://github.com/agrafix/Spock.git
      commit: 2c60a48b2c0be0768071cc1b3c7f14590ffcc7d6
  subdirs:
    - Spock
    - Spock-core
    - reroute
- location:
      git: https://github.com/agrafix/Spock-digestive.git
      commit: 4c85647427e21bbaefbf04c4bc315d4bdfabba0e
extra-deps:
- digestive-bootstrap-0.1.0.1
- blaze-bootstrap-0.1.0.1
- digestive-functors-blaze-0.6.0.6
resolver: lts-7.20

One import note: I don't want to use Docker to deploy the app, just to compile it, i.e. as part of my dev process.

Any ideas?

Should I use another image without ghc pre-installed to use with docker? Which one?

update

Yes, I could use the built-in GHC in the container and it is a good idea, but wondered if there is any issue building GHC within Docker.

update 2

For anyone wishing to reproduce (on MAC OSX by the way), you can clone repo https://github.com/carlosayam/funblog and grab this commit 9446bc0e52574cc574a9eb5f2733f69e07b874ef

(I will probably move on using container's GHC)

please show us the contents of your stack.yaml, the docker version you run, what os are you running, did you read the section about docker in the haskellstack.org documentation? having checked the haskell:8 image on dockerhub one thing I can tell you for sure - it would be better to use FROM haskell:8.0.1 then you don't need to install ghc-8.0.1 as it should already be inside the docker image. — epsilonhalbe
thanks @epsilonhalbe, just added stack.yaml. You are right, I could use the GHC already in the image, what would be the point of building GHC if one uses an image with it, but wonder if there is an issue compiling GHC within docker. — carlosayam
Could you answer the questions I left in my previous comment, I then can try to reproduce the issue — epsilonhalbe
@epsilonhalbe, please clone this repo (github.com/carlosayam/funblog) and follow instructions in readme - hope you can reproduce it. — carlosayam
I tried to reproduce this on a linux mint box with docker 1.12.6 build 78d1802, and a stack 1.3.2 inside the haskell:8.0.1, I had no troubles whatsoever installing ghc 8.0.2 by creating a test cabal package + stack init with lts-8.6 and then running stack setup — epsilonhalbe

kenorb kenorb · Accepted Answer · 2017-10-05T12:24:21

By default, Docker for macOS limits number of file descriptors to avoid hitting macOS system-wide limits (default limit is 900). To increase the limit, follow these commands:

$ cd ~/Library/Containers/com.docker.docker/Data/database/
$ git reset --hard
HEAD is now at 9410b78 last-start-time changed at 1480947038
$ cat com.docker.driver.amd64-linux/slirp/max-connections 
900
$ echo 1200 > com.docker.driver.amd64-linux/slirp/max-connections 
$ git add com.docker.driver.amd64-linux/slirp/max-connections 
$ git commit -s -m 'Update the maximum number of connections'
[master 227a248] Update the maximum number of connections
 1 file changed, 1 insertion(+), 1 deletion(-)

Then check the notice messages by:

$ syslog -k Sender Docker
<Notice>: updating connection limit to 1200

To check how many files you got open, run: sysctl kern.num_files.

To check what's your current limit, run: sysctl kern.maxfiles.

To increase it system-wide, run: sysctl -w kern.maxfiles=20480.

Source: Containers become unresponsive due to "too many connections".

See also: Docker: How to increase number of open files limit.

On Linux, you can also try to run Docker with --ulimit, e.g.

docker run --ulimit nofile=5000:5000 <image-tag>

Source: Docker error: too many open files

Haskell installation in docker container using stack failing: too many open files

1 Answers