Scons ignores the environment for dependency tracking when using python function builders

Question

I have an issue in scons 2.5.1 related to passing parameters through the environment to python based builders. When an ordinary builder is called it seems like the result is flagged as dirty if any of the source files or the environment variables passed in have changed. When using python function builders (described here http://scons.org/doc/1.2.0/HTML/scons-user/x3524.html) it seems like scons only care about the source files.

Here is a minimal artificial example of where it fails. It's two implementations of passing a parameter through the environment and writing it to the target file using the shell. One implementation is just a command string, the other uses python subprocess to invoke it in a python function. I use an argument to scons to select what builder to use.

#SConstruct
import subprocess

def echo_fun(env, source, target):
    subprocess.check_call('echo %s > %s' % (env['MESSAGE'], str(target[0])), shell= True)
    return None

env = Environment(BUILDERS = {'echo' : Builder(action='echo $MESSAGE > $TARGET'),
                              'echo_py': Builder(action=echo_fun),
                              })
build_fn = env.echo_py if ARGUMENTS.get('USE_PYTHON', False) else env.echo
build_fn(['test.file'], [], MESSAGE = ARGUMENTS.get('MSG', 'None'))

Here is the result of running the scons script with different parameters:

PS C:\work\code\sconsissue> scons -Q MSG=Hello
echo Hello > test.file
PS C:\work\code\sconsissue> scons -Q MSG=Hello
scons: `.' is up to date.
PS C:\work\code\sconsissue> scons -Q MSG=HelloAgain
echo HelloAgain > test.file
PS C:\work\code\sconsissue> del .\test.file
PS C:\work\code\sconsissue> scons -Q MSG=Hello -Q USE_PYTHON=True
echo_fun(["test.file"], [])
PS C:\work\code\sconsissue> scons -Q MSG=Hello -Q USE_PYTHON=True
scons: `.' is up to date.
PS C:\work\code\sconsissue> scons -Q MSG=HelloAgain -Q USE_PYTHON=True
scons: `.' is up to date.

In the case of using an ordinary builder it detects that the result is dirty when MSG changes (and clean when MSG stays the same) but in the python command version it considered it up to date even if MSG is changed.

A workaround for this would be to put my builder scripts in a separate python script and invoke that python script with the environment dependencies as command line parameters but it seems convoluted.

Is this the expected behavior or a bug?

Is there an easier workaround than the one I described above where I can keep my build functions in the SConstruct file?

bdbaddog bdbaddog · Accepted Answer · 2017-12-10T19:22:38

This is expected behavior because there is no way for SCons to know that the function (as written) depends on MESSAGE.

However if you read the manpage http://scons.org/doc/production/HTML/scons-man.html

You'll see this (Under "Action Objects"):

The variables may also be specified by a varlist= keyword parameter; if both are present, they are combined. This is necessary whenever you want a target to be rebuilt when a specific construction variable changes. This is not often needed for a string action, as the expanded variables will normally be part of the command line, but may be needed if a Python function action uses the value of a construction variable when generating the command line.

...
# Alternatively, use a keyword argument.
a = Action(build_it, varlist=['XXX'])

So if you rewrite as:

#SConstruct
import subprocess

def echo_fun(env, source, target):
    subprocess.check_call('echo %s > %s' % (env['MESSAGE'], str(target[0])), shell= True)
    return None

env = Environment(BUILDERS = {'echo' : Builder(action='echo $MESSAGE > $TARGET'),
                              'echo_py': Builder(action=Action(echo_fun, varlist=['MESSAGE'])),
                              })
build_fn = env.echo_py if ARGUMENTS.get('USE_PYTHON', False) else env.echo
build_fn(['test.file'], [], MESSAGE = ARGUMENTS.get('MSG', 'None'))

It should behave as you desire.

Scons ignores the environment for dependency tracking when using python function builders

1 Answers