0
votes

In my CLI script I am using argparse to take in a few optional arguments and then a positional argument. The positional argument is used to determine a subparser to use which in turn runs a function that calls an external program that takes its own arguments. So, the command-line usage looks something like this:

myscript [OPTIONS] subcommand [SUBCOMMAND_OPTIONS]

Now my problem is that there are conflicts between my OPTIONS I've declared and the SUBCOMMAND_OPTIONS declared in the external program. The easy fix is to ensure I rename all conflicts in myscript but I can't do this for all options - most notably the "-h" option for help. Ideally I'd like argparse to stop parsing immediately after it encounters the subcommand and simply pass on the rest of the args to the external program.

So, the following invocation should show the help text for myscript:

myscript -h

While, in contrast the following should show the help text from the external program invoked by the "bar" subparser:

myscript --foo bar -h

Some more code to make the above clearer:

>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--foo', action='store_true')
>>> subparsers = parser.add_subparsers()
>>> subparsers.add_parser("bar")

>>> parser.parse_known_args("--foo bar --test".split())
(Namespace(foo=True), ['--test'])
# cool - this is what I want, I'll just pass --test on to the external program

>>> parser.parse_known_args("--foo bar -h".split())
usage:  bar [-h]

optional arguments:
  -h, --help  show this help message and exit

# unfortunately the above argparse help message is NOT what I wanted, instead I was looking for the result below:
(Namespace(foo=True), ['-h'])

>>> parser.parse_known_args("bar --test -- -h".split())
# this works, sort of, it requires educating the end-user to use the '--' parameter and I'd like to avoid that if possible.
(Namespace(foo=False), ['--test', '--', '-h'])
3
Have you tried the subparser mechanism in argparse? The fit might not be perfect, but it might be most productive for all of us if you read the docs, try it, and then come back with some more questions. - hpaulj
actually I explicitly mention the use of subparsers in my question above, so clearly I already know about it. The problem is not the ability to subparse the command it's the ability to stop parsing options after encountering the subcommand. And before you ask me - yes, I already am using argparse.parser_known_args() to stop argparse failing on the external commands arguments that it knows nothing about, the problem lies in the conflicting args - specifically. - ishaaq
-- tells it to treat everything that follows as arguments to a positional, whether they look like flags or not. nargs=argparse.PARSER and REMAINDEr can also be used to grab 'everything else' for a positional. The subparser mechanism uses PARSER to allocate arguments. Play with these and see if they suit your needs better than the full subparser mechanism. - hpaulj

3 Answers

1
votes

Your initial description was sufficiently close to subparsers that it takes some careful reading to identify what's wrong (for you).

From comments it sounds like the biggest fault is that the subparser captures the -h give you a help message, rather than passing it through to the extras. Subparsers, just like a main parser, takes a add_help=False parameter.

p=argparse.ArgumentParser()
p.add_argument('foo')
p.add_argument('--bar')
sp=p.add_subparsers(dest='cmd')
sp1=sp.add_parser('cmd1')   # with a subparser help
sp2=sp.add_parser('cmd2', add_help=False)  # will ignore -h

producing

p.parse_known_args('-h'.split())   # top level help

p.parse_known_args('--bar xxx foo cmd1 -h'.split())
# usage: ipython foo cmd1 [-h]
# exit msg

p.parse_known_args('--bar xxx foo cmd2 -h'.split())
# (Namespace(bar='xxx', cmd='cmd2', foo='foo'), ['-h'])

p.parse_known_args('foo cmd2 test -o --more --bar xxx'.split())
# (Namespace(bar=None, cmd='cmd2', foo='foo'),
# ['test', '-o', '--more', '--bar', 'xxx'])

In a comment I mentioned a couple of nargs values, argparse.PARSER and argparse.REMAINDER. To the main parser, subparsers are just a positional with a PARSER nargs (and choices). It's a specialaction` type, which goes on to invoke another parser based on the 1st value.

REMAINDER is like the * nargs, except that it takes everything, even strings that look like flags. PARSER is like +, requiring at least one string.

p=argparse.ArgumentParser()
p.add_argument('foo')
p.add_argument('--bar')
p.add_argument('rest', nargs=argparse.REMAINDER)

producing

In [32]: p.parse_args('--bar yyy foo'.split())
Out[32]: Namespace(bar='yyy', foo='foo', rest=[])

In [33]: p.parse_args('--bar yyy foo -h'.split())
Out[33]: Namespace(bar='yyy', foo='foo', rest=['-h'])

In [34]: p.parse_args('--bar yyy foo cmd2 test -o --more --bar xxx'.split())Out[34]: Namespace(bar='yyy', foo='foo', rest=['cmd2', 'test', '-o', '--more', '--bar', 'xxx'])

The REMAINDER note in the argparse docs is:

argparse.REMAINDER. All the remaining command-line arguments are gathered into a list. This is commonly useful for command line utilities that dispatch to other command line utilities:

and has an example similar to my last one.

>>> parser = argparse.ArgumentParser(prog='PROG')
>>> parser.add_argument('--foo')
>>> parser.add_argument('command')
>>> parser.add_argument('args', nargs=argparse.REMAINDER)
>>> print(parser.parse_args('--foo B cmd --arg1 XX ZZ'.split()))
Namespace(args=['--arg1', 'XX', 'ZZ'], command='cmd', foo='B')
0
votes

The normal sub-commands support in argparse does this just fine. Just note that when reusing argument names, you should specify a custom dest to make sure that your main commands’ values are not overwritten:

parser = argparse.ArgumentParser()
parser.add_argument('--foo', action='store_true')
subparsers = parser.add_subparsers()

parser_foo = subparsers.add_parser('foo')
parser_foo.add_argument('--foo', dest='foo_foo')
parser_foo.add_argument('--bar', dest='foo_bar')

parser_bar = subparsers.add_parser('bar')
parser_bar.add_argument('--foo', dest='bar_foo')

Examples:

>>> parser.parse_args('-h'.split())
usage: [-h] [--foo] {foo,bar} ...

positional arguments:
  {foo,bar}

optional arguments:
  -h, --help  show this help message and exit
  --foo

>>> parser.parse_args('foo -h'.split())
usage:  foo [-h] [--foo FOO_FOO] [--bar FOO_BAR]

optional arguments:
  -h, --help     show this help message and exit
  --foo FOO_FOO
  --bar FOO_BAR

>>> parser.parse_args('bar -h'.split())
usage:  bar [-h] [--foo BAR_FOO]

optional arguments:
  -h, --help     show this help message and exit
  --foo BAR_FOO

>>> parser.parse_args('--foo foo --foo test --bar baz'.split())
Namespace(foo=True, foo_bar='baz', foo_foo='test')
0
votes

The arpgarse module has a subparser feature, which you can combine with argparse.REMAINDER to capture any arguments without declaring those explicitly.

Update:

If you want to have more control than argparse provides, it might be worthwile to look into the click package instead. It has special support for ignoring unknown options and preventing options like --help to be handled. Details at

http://click.pocoo.org/4/api/#click.Context.ignore_unknown_options