How to correctly initialize git submodules in Dockerfile for Docker Cloud

Question

We are building a Docker Container on Docker Cloud. The build process requires git submodules.

To initialize the submodules for a local build we added the following line to the Dockerfile:

RUN git submodule update --init --recursive

See: https://github.com/open62541/open62541/blob/master/Dockerfile#L9

Corresponding commit: https://github.com/open62541/open62541/commit/ee9c18a6a05722edfe7c0d8d8e140d802fa2e5f2 and Pull Request:
https://github.com/open62541/open62541/pull/3191

Note: In contrast to similar questions, all the submodules are public repos on github without authentication.

Situation without submodule init line:

Build on Docker Cloud: Success
https://cloud.docker.com/u/open62541/repository/registry-1.docker.io/open62541/open62541/builds/dcd3283c-76af-4f6b-bb00-cecbaaffc82f
Build locally: Fails

git clone https://github.com/open62541/open62541.git
cd open62541
# Parent commit without git submodule update
git checkout e97abd591a159ce894488d93796b858d9f0d00b9
# This will fail because the submodules are obviously not initialized
docker build .

Error:

CMake Error at CMakeLists.txt:830 (message):
  File /opt/open62541/deps/ua-nodeset/Schema/Opc.Ua.NodeSet2.xml not found.
  You probably need to initialize the git submodule for deps/ua-nodeset.

Situation with submodule init in Dockerfile:

Build on Docker Cloud: Fails
https://cloud.docker.com/u/open62541/repository/registry-1.docker.io/open62541/open62541/builds/28663705-c324-4c7f-a084-f94eb80057bc

Step 7/18 : RUN git submodule update --init --recursive
---> Running in b358c21c4d53
fatal: not a git repository: /src/b6tohshrfzzntavvhek3zna/.git/modules/deps/mdnsd
Unable to find current revision in submodule path 'deps/mdnsd'

Build locally: Success

git clone https://github.com/open62541/open62541.git
cd open62541
# Commit which added git submodule init in Dockerfile
git checkout ee9c18a6a05722edfe7c0d8d8e140d802fa2e5f2
# This will succeed
docker build .

How are submodules in dockerfiles correctly initialized so that it works on Docker cloud, and at the same time one can just pull the main repo and build the docker container?

Related questions:

Docker cloud submodule authentication
Private git submodules. No answers.
Google container builder DockerFile with git submodules
Private git submodules. Related to Google Container Builder
Docker Hub and git submodules
Docker Hub, but related to private submodules
docker autobuild with git private submodule
Docker Hub, but related to private submodules

Geod24 Geod24 · Accepted Answer · 2020-01-09T17:01:21

See the answer I gave here: https://stackoverflow.com/a/59640438/1021344

Reproduced here for simplicity: You need to use hooks: https://docs.docker.com/docker-hub/builds/advanced/#custom-build-phase-hooks

TL;DR: Place this in hooks/post_checkout:

#!/bin/bash
# Docker hub does a recursive clone, then checks the branch out,
# so when a PR adds a submodule (or updates it), it fails.
git submodule update --init

How to correctly initialize git submodules in Dockerfile for Docker Cloud

1 Answers