How does Scons compute the build signature?

Question

I keep different versions of one project in different directories. (This does make sense in this project. Sadly.) As there are only minor differences between the versions, I hope I can speed all builds after the first one by using a common cache directory for all builds.

Unfortunately I had to realise that, when building an object file from the same sources in different directories, SCons 2.3.3 stores the result on different locations in the cache. (The location is equal to the build signature, I assume.) The same sources are recompiled for each and every directory. So why does SCons determine different build signatures although

• the compile commands are identical and
• the sources and the include files are the same (identical output of of the preprocessor phase, gcc -E ...)
• I'm using the decider "MD5-timestamp"

Even the resulting object files are identical!

For a trivial example (helloworld from the SCons documentation) re-using the cache works. Though in the big project I'm working on, it does not. Maybe the "SCons build environment" influences the build signature, even if it does not have any effect on the compile command?

Are there any debug options that could help besides --cache-debug=-? Which method of SCons determines the build signature?

The folders look somewhat like this:

<basedir1>/
       SConstruct
       src/something.cpp …
       include/header.hpp …
<basedir2>/
       SConstruct
       src/something.cpp …
       include/header.hpp …
/SharedCache/
       0/ 1/ 2/ … F/

I check out the project in both basedir1 and basedir2 and call scons --build-cache-dir=/SharedCache in both of them. (EDIT: --build-cache-dir is a custom option, implemented in the SConstruct file of this project. It maps to env.CacheDir('/SharedCache').

EDIT2: Before I realized this problem, I did some tests to evaluate the effects of using --cache-implicit or SCons 2.4.0.

Answer 1

This is the code of the method get_cachedir_bsig() from the file src/engine/SCons/Node/FS.py:

def get_cachedir_bsig(self):
    """
    Return the signature for a cached file, including
    its children.

    It adds the path of the cached file to the cache signature,
    because multiple targets built by the same action will all
    have the same build signature, and we have to differentiate
    them somehow.
    """
    try:
        return self.cachesig
    except AttributeError:
        pass

    # Collect signatures for all children
    children = self.children()
    sigs = [n.get_cachedir_csig() for n in children]
    # Append this node's signature...
    sigs.append(self.get_contents_sig())
    # ...and it's path
    sigs.append(self.get_internal_path())
    # Merge this all into a single signature
    result = self.cachesig = SCons.Util.MD5collect(sigs)
    return result

It shows how the path of the cached file is included into the "cache build signature", which explains the behaviour you see. For the sake of completeness, here is also the code of the get_cachedir_csig() method from the same FS.py file:

def get_cachedir_csig(self):
    """
    Fetch a Node's content signature for purposes of computing
    another Node's cachesig.

    This is a wrapper around the normal get_csig() method that handles
    the somewhat obscure case of using CacheDir with the -n option.
    Any files that don't exist would normally be "built" by fetching
    them from the cache, but the normal get_csig() method will try
    to open up the local file, which doesn't exist because the -n
    option meant we didn't actually pull the file from cachedir.
    But since the file *does* actually exist in the cachedir, we
    can use its contents for the csig.
    """
    try:
        return self.cachedir_csig
    except AttributeError:
        pass

    cachedir, cachefile = self.get_build_env().get_CacheDir().cachepath(self)
    if not self.exists() and cachefile and os.path.exists(cachefile):
        self.cachedir_csig = SCons.Util.MD5filesignature(cachefile, \
            SCons.Node.FS.File.md5_chunksize * 1024)
    else:
        self.cachedir_csig = self.get_csig()
    return self.cachedir_csig

where the cache paths of the children are hashed into the final build signature.

EDIT: The "cache build signature" as computed above, is then used to build the "cache path". Like this, all files/targets can get mapped to a unique "cache path" by which they can get referenced and found in (= retrieved from) the cache. As the comments above explain, the relative path of each file (starting from the top-level folder of your SConstruct) is a part of this "cache path". So, if you have the same source/target (foo.c->foo.obj) in different directories, they will have different "cache paths" and get built independent of each other.

If you truly want to share sources between different projects, note how the CacheDir functionality is more intended for sharing the same sources between different developers, you may want to have a look at the Repository() method. It let's you mount (blend in) another source tree to your current project...