Multiple settings gradle files for multiple projects building

Question

I have the following project structure

-->Starnderd Location
        -->Project1 
           -->settings.gradle 
           -->build.gradle
           -->Subproject11
              -->build.gradle
           -->Subproject12
              -->build.gradle
        -->Project2 
           -->settings.gradle 
           -->build.gradle
           -->Subproject21
              -->build.gradle
           -->Subproject22
              -->build.gradle
        -->build.gradle
        -->settings.gradle

Idea of above project structure is that we have multiple projects which contains subprojects, each project can have dependencies to other projects. Also subprojects inside the project can have dependencies to other subprojects within the same project. Projects will be specified in the settings.gradle in the root. Also settings.gradle inside each project will say what are the subprojects of that particular project.

My settings.gradle in the root would looks like

include 'Project1',
         'Project2'

and Project1 settings.gradle will looks like

include 'SubProject11'
         'SubProject12'

other dependency orders are defined in the respective build.gradle files If I rand gradle clean build install inside the root location(Standar location) it doesn't seems to use configurations in the Project level settings.gradle file.

What I'm doing here wrong?

Answer 1

I was able to solve this problem in a relatively clean way. Improvements are certainly welcome!

Although Gradle does not support multiple settings.gradle scripts out of the box, it is possible to create individual sub-projects each with their own settings.gradle file. Let's say you have multi-project A that depends on multi-project B, each with their own sub-projects. Your directory structure might look like:

A
- settings.gradle
- foo
- faz
\ B
  - settings.gradle
  - bar
  - bap

Out of the box, Gradle expects A/settings.gradle to look something like this:

include ':foo', ':faz', 'B:bar', 'B:bap'

The problem with this is that every time B adds a new project, A/settings.gradle must also change even if the new project is only used by B. To avoid this situation, you could try to apply B/settings.gradle in A/settings.gradle instead of adding redundant declarations:

apply from: 'B/settings.gradle'
include ':foo', ':faz'

If you try this, you'll find that Gradle fails because it generates the wrong projectDir for :bar and :bap. It mistakenly assumes that B's includes are relative to settingsDir which happens to be A/ when Gradle is invoked from that project root. To fix this you can add another script such as B/settings-parent.gradle (the exact name is not significant):

apply from: 'settings.gradle'

def updateProjectPaths(Set<ProjectDescriptor> projects) {
    projects.each { ProjectDescriptor project ->
        String relativeProjectPath = project.projectDir.path.replace(settingsDir.path, "")
        project.projectDir = new File("B/$relativeProjectPath")
        // Recursively update paths for all children
        updateProjectPaths(project.children)
    }
}

updateProjectPaths(rootProject.children)

This strips settingsDir.path and prefixes the path with B/. This can be extended to multiple layers of settings[-parent].gradle files by having each layer add itself onto the path. Now you will apply this script to A/settings.gradle:

apply from: 'B/settings-parent.gradle'
include ':foo', ':faz'

With this scheme, new B projects do not unnecessarily break A/settings.gradle and all projects can be used without explicitly referencing the B sub-project. For example, if ':foo' wanted to use 'B:bap', it may simply declare:

compile project(':bap')

Answer 2

If you are using Gradle 3.x, try includeBuild(): https://docs.gradle.org/current/userguide/composite_builds.html

// settings.gradle
includeBuild './Project1'
includeBuild './Project2'

If you are using Gradle 2.x, I have wrote a demo for this function. Hope it will help you: https://github.com/xujiaao/gradle-composite-build-demo

// settings.gradle
def includeProject(String path, Closure closure) {
    ...

    apply {
        from ...
        to new SettingsProxy(...)
    }
}

class SettingsProxy {
    ...

    public getRootProject() { ... }

    public void include(String... paths) {
        for (String path : paths) {
            if (!mProjectSpec.accept(path)) {
                continue
            }

            def descendantPath = generateDescendantPath(path)
            mSettings.include(descendantPath)

            def descendantProjectDir = new File(mProject.projectDir, path.replace(':', '/'))
            mSettings.project(descendantPath).projectDir = descendantProjectDir
        }
    }
}

Answer 3

Currently, Gradle only supports a single settings.gradle files per build. This may change in the future.

Answer 4

as this topic crossed my daily work quite often now and the improvement (GRADLE-803) on it is still open, I would like to share my approach as well:

At first sight, it looks similar to James Wald's answer but has one difference. You do not need to split your settings files somehow in the subproject. There is a clean way to do everything in the super project if that is acceptable for you. Normally your small sub projects should not have to care about the surrounding super projects. They include their sub dependencies and that's it. It should also be irrelevant how the module directory is named, in Wald's approach the module itself needs to know its directory name ('B') here:

project.projectDir = new File("B/$relativeProjectPath")

In contrast, usually, a super project knows its subprojects and directories well, because they could be git submodules for instance, which have well defined fix names which can be, from the super project perspective, safely referenced.

That's my setup (using Gradle 2.4):

Super Project
├── build.gradle
├── settings.gradle (include 'A' include 'B')
├── <Subproject A>
    ├── build.gradle
    ├── settings.gradle (include 'subA')
    ├── <Subsubproject subA>
        ├── build.gradle
├── <Subproject B>
    ├── build.gradle
    ├── settings.gradle (include 'subB')
    ├── <Subsubproject subB>
        ├── build.gradle

In the super project settings.gradle you can now code the following:

include 'A'
include 'B'
apply from: 'A/settings.gradle'
apply from: 'B/settings.gradle'
project(':subA').projectDir = new File(rootDir, 'A/subA')
project(':subB').projectDir = new File(rootDir, 'B/subB')

It still looks rather verbose (and still does not add real hierarchical behavior), but keeps the extra effort in the super project where you normally need to know everything about your contained modules anyways.

The rest is pretty straight forward again.

If you want to see my approach in practice, read section 5.) of this blog post, where I explicitly require this independence between the modules or just check out my github project on cross-language benchmarks. But be aware, you need a running native compiler toolchain like gcc or Clang to execute it ;)

Hope this helps! Cheers Ben

Answer 5

I have also looked into this and you can kind of do it, but it is very ugly! The reason this works at all for us is that the vast majority of time, we just want to build from the top most level.

If it helps you at all, what you need to do is to have the top-most settings.gradle file properly reference every project-subproject directly. Get this working first.

Then if Project1 and Project2 (and so on) are capable of being independently built from one another you can make a local settings.gradle file for that project. Since, as I said above, this is not what we usually do, we call this file settings.project1. If we want to use this file, we copy it to settings.gradle. I know ugly.

But it actually gets worse :) Once you put this settings.gradle file in place, you build from Project1 will no longer see the top level build.gradle file where you probably have needed things defined. To invoke this, you would need something like this added to every project-level build.gradle file:

if (project.hasProperty('local')) {
    apply from: '../build.gradle'
}

Then you can run the build as: gradle -Plocal build

Ugly, but if you need it, it does at least work. And in the interest of full-disclosure, having put this into place a couple of weeks ago, none of the developers have needed and/or used it. Will probably remove it in another couple of weeks if it continues to not be used.

Remember, that if you build from subproject itself, only that subproject (and any dependent projects) will be built (although all the gradle scripts will be compiled/evaluated).

Answer 6

Building up on earlier answers this is what I came out with.

interface Includer {
  def include(String... args)
}

def applyFrom(String folder) {
  apply from: folder + '/settings.gradle'
  def includer = [
    include: { args ->
          args.each {
            project(it.startsWith(':') ? it : ':' + it).projectDir =
                new File(rootDir, folder + "/" + (it.startsWith(':') ? it.substring(1) : it).replace(':', '/'))
          }
    }] as Includer

  apply from: folder + '/settings.gradle', to: includer
}

applyFrom('A')
applyFrom('B')

The advantage is no duplication.