python - How to insert newlines on argparse help text?

Question

I'm using argparse in Python 2.7 for parsing input options. One of my options is a multiple choice. I want to make a list in its help text, e.g.

from argparse import ArgumentParser

parser = ArgumentParser(description='test')

parser.add_argument('-g', choices=['a', 'b', 'g', 'd', 'e'], default='a',
    help="Some option, where\n"
         " a = alpha\n"
         " b = beta\n"
         " g = gamma\n"
         " d = delta\n"
         " e = epsilon")

parser.parse_args()

However, argparse strips all newlines and consecutive spaces. The result looks like

~/Downloads:52$ python2.7 x.py -h
usage: x.py [-h] [-g {a,b,g,d,e}]

test

optional arguments:
  -h, --help      show this help message and exit
  -g {a,b,g,d,e}  Some option, where a = alpha b = beta g = gamma d = delta e
                  = epsilon

How to insert newlines in the help text?

Answer 1

Try using RawTextHelpFormatter:

from argparse import RawTextHelpFormatter
parser = ArgumentParser(description='test', formatter_class=RawTextHelpFormatter)

Answer 2

If you just want to override the one option, you should not use RawTextHelpFormatter. Instead subclass the HelpFormatter and provide a special intro for the options that should be handled "raw" (I use "R|rest of help"):

import argparse

class SmartFormatter(argparse.HelpFormatter):

    def _split_lines(self, text, width):
        if text.startswith('R|'):
            return text[2:].splitlines()  
        # this is the RawTextHelpFormatter._split_lines
        return argparse.HelpFormatter._split_lines(self, text, width)

And use it:

from argparse import ArgumentParser

parser = ArgumentParser(description='test', formatter_class=SmartFormatter)

parser.add_argument('-g', choices=['a', 'b', 'g', 'd', 'e'], default='a',
    help="R|Some option, where\n"
         " a = alpha\n"
         " b = beta\n"
         " g = gamma\n"
         " d = delta\n"
         " e = epsilon")

parser.parse_args()

Any other calls to .add_argument() where the help does not start with R| will be wrapped as normal.

This is part of my improvements on argparse. The full SmartFormatter also supports adding the defaults to all options, and raw input of the utilities description. The full version has its own _split_lines method, so that any formatting done to e.g. version strings is preserved:

parser.add_argument('--version', '-v', action="version",
                    version="version...\n   42!")

Answer 3

Another easy way to do it is to include textwrap.

For example,

import argparse, textwrap
parser = argparse.ArgumentParser(description='some information',
        usage='use "python %(prog)s --help" for more information',
        formatter_class=argparse.RawTextHelpFormatter)

parser.add_argument('--argument', default=somedefault, type=sometype,
        help= textwrap.dedent('''\
        First line
        Second line
        More lines ... '''))

In this way, we can avoid the long empty space in front of each output line.

usage: use "python your_python_program.py --help" for more information

Prepare input file

optional arguments:
-h, --help            show this help message and exit
--argument ARGUMENT
                      First line
                      Second line
                      More lines ...

Answer 4

I've faced similar issue (Python 2.7.6). I've tried to break down description section into several lines using RawTextHelpFormatter:

parser = ArgumentParser(description="""First paragraph 

                                       Second paragraph

                                       Third paragraph""",  
                                       usage='%(prog)s [OPTIONS]', 
                                       formatter_class=RawTextHelpFormatter)

options = parser.parse_args()

And got:

usage: play-with-argparse.py [OPTIONS]

First paragraph 

                        Second paragraph

                        Third paragraph

optional arguments:
  -h, --help  show this help message and exit

So RawTextHelpFormatter is not a solution. Because it prints description as it appears in source code, preserving all whitespace characters (I want to keep extra tabs in my source code for readability but I don't want to print them all. Also raw formatter doesn't wrap line when it is too long, more than 80 characters for example).

Thanks to @Anton who inspired the right direction above. But that solution needs slight modification in order to format description section.

Anyway, custom formatter is needed. I extended existing HelpFormatter class and overrode _fill_text method like this:

import textwrap as _textwrap
class MultilineFormatter(argparse.HelpFormatter):
    def _fill_text(self, text, width, indent):
        text = self._whitespace_matcher.sub(' ', text).strip()
        paragraphs = text.split('|n ')
        multiline_text = ''
        for paragraph in paragraphs:
            formatted_paragraph = _textwrap.fill(paragraph, width, initial_indent=indent, subsequent_indent=indent) + '\n\n'
            multiline_text = multiline_text + formatted_paragraph
        return multiline_text

Compare with the original source code coming from argparse module:

def _fill_text(self, text, width, indent):
    text = self._whitespace_matcher.sub(' ', text).strip()
    return _textwrap.fill(text, width, initial_indent=indent,
                                       subsequent_indent=indent)

In the original code the whole description is being wrapped. In custom formatter above the whole text is split into several chunks, and each of them is formatted independently.

So with aid of custom formatter:

parser = ArgumentParser(description= """First paragraph 
                                        |n                              
                                        Second paragraph
                                        |n
                                        Third paragraph""",  
                usage='%(prog)s [OPTIONS]',
                formatter_class=MultilineFormatter)

options = parser.parse_args()

the output is:

usage: play-with-argparse.py [OPTIONS]

First paragraph

Second paragraph

Third paragraph

optional arguments:
  -h, --help  show this help message and exit

Answer 5

I wanted to have both manual line breaks in the description text, and auto wrapping of it; but none of the suggestions here worked for me - so I ended up modifying the SmartFormatter class given in the answers here; the issues with the argparse method names not being a public API notwithstanding, here is what I have (as a file called test.py):

import argparse
from argparse import RawDescriptionHelpFormatter

# call with: python test.py -h

class SmartDescriptionFormatter(argparse.RawDescriptionHelpFormatter):
  #def _split_lines(self, text, width): # RawTextHelpFormatter, although function name might change depending on Python
  def _fill_text(self, text, width, indent): # RawDescriptionHelpFormatter, although function name might change depending on Python
    #print("splot",text)
    if text.startswith('R|'):
      paragraphs = text[2:].splitlines()
      rebroken = [argparse._textwrap.wrap(tpar, width) for tpar in paragraphs]
      #print(rebroken)
      rebrokenstr = []
      for tlinearr in rebroken:
        if (len(tlinearr) == 0):
          rebrokenstr.append("")
        else:
          for tlinepiece in tlinearr:
            rebrokenstr.append(tlinepiece)
      #print(rebrokenstr)
      return '\n'.join(rebrokenstr) #(argparse._textwrap.wrap(text[2:], width))
    # this is the RawTextHelpFormatter._split_lines
    #return argparse.HelpFormatter._split_lines(self, text, width)
    return argparse.RawDescriptionHelpFormatter._fill_text(self, text, width, indent)

parser = argparse.ArgumentParser(formatter_class=SmartDescriptionFormatter, description="""R|Blahbla bla blah blahh/blahbla (bla blah-blabla) a blahblah bl a blaha-blah .blah blah

Blah blah bla blahblah, bla blahblah blah blah bl blblah bl blahb; blah bl blah bl bl a blah, bla blahb bl:

  blah blahblah blah bl blah blahblah""")

options = parser.parse_args()

This is how it works in 2.7 and 3.4:

$ python test.py -h
usage: test.py [-h]

Blahbla bla blah blahh/blahbla (bla blah-blabla) a blahblah bl a blaha-blah
.blah blah

Blah blah bla blahblah, bla blahblah blah blah bl blblah bl blahb; blah bl
blah bl bl a blah, bla blahb bl:

  blah blahblah blah bl blah blahblah

optional arguments:
  -h, --help  show this help message and exit

Answer 6

Starting from SmartFomatter described above, I ended to that solution:

class SmartFormatter(argparse.HelpFormatter):
    '''
         Custom Help Formatter used to split help text when '\n' was 
         inserted in it.
    '''

    def _split_lines(self, text, width):
        r = []
        for t in text.splitlines(): r.extend(argparse.HelpFormatter._split_lines(self, t, width))
        return r

Note that strangely the formatter_class argument passed to top level parser is not inheritated by sub_parsers, one must pass it again for each created sub_parser.

Answer 7

I admit I found this a very frustrating experience as it seems many others have, given the number of solutions I see posted and the number of times I see this asked across the web. But I find most of these solutions far too complicated for my likes and I'd like share the tersest simplest solution I have for it.

Here is the script to demonstrate:

#!/usr/bin/python3
import textwrap
from argparse import ArgumentParser, HelpFormatter

class RawFormatter(HelpFormatter):
    def _fill_text(self, text, width, indent):
        return "\n".join([textwrap.fill(line, width) for line in textwrap.indent(textwrap.dedent(text), indent).splitlines()])

program_descripton = f'''
    FunkyTool v1.0

    Created by the Funky Guy on January 1 2020
    Copyright 2020. All rights reserved.

    Licensed under The Hippocratic License 2.1
    https://firstdonoharm.dev/

    Distributed on an "AS IS" basis without warranties
    or conditions of any kind, either express or implied.

    USAGE:
    '''

parser = ArgumentParser(description=program_descripton, formatter_class=RawFormatter)
args = parser.parse_args()

And here is what it looks like in test.py:

$ ./test.py --help
usage: test.py [-h]

FunkyTool v1.0

Created by the Funky Guy on January 1 2020
Copyright 2020. All rights reserved.

Licensed under The Hippocratic License 2.1
https://firstdonoharm.dev/

Distributed on an "AS IS" basis without warranties
or conditions of any kind, either express or implied.

USAGE:

optional arguments:
  -h, --help  show this help message and exit

And so, all the basic formatting in the original description is preserved neatly and we've had, alas, to use a custom formatter, but it's a oneliner. It can be written more lucidly as:

class RawFormatter(HelpFormatter):
    def _fill_text(self, text, width, indent):
        text = textwrap.dedent(text)          # Strip the indent from the original python definition that plagues most of us.
        text = textwrap.indent(text, indent)  # Apply any requested indent.
        text = text.splitlines()              # Make a list of lines
        text = [textwrap.fill(line, width) for line in text] # Wrap each line 
        text = "\n".join(text)                # Join the lines again
        return text

But I prefer it one line.

Answer 8

Preface

For this question, argparse.RawTextHelpFormatter is helpful to me.

Now, I want to share how do I use the argparse.

I know it may not be related to question,

but these questions have been bothered me for a while.

So I want to share my experience, hope that will be helpful for someone.

Here we go.

3rd Party Modules

colorama: for change the text color: pip install colorama

Makes ANSI escape character sequences (for producing colored terminal text and cursor positioning) work under MS Windows

Example

import colorama
from colorama import Fore, Back
from pathlib import Path
from os import startfile, system

SCRIPT_DIR = Path(__file__).resolve().parent
TEMPLATE_DIR = SCRIPT_DIR.joinpath('.')


def main(args):
    ...


if __name__ == '__main__':
    colorama.init(autoreset=True)

    from argparse import ArgumentParser, RawTextHelpFormatter

    format_text = FormatText([(20, '<'), (60, '<')])
    yellow_dc = format_text.new_dc(fore_color=Fore.YELLOW)
    green_dc = format_text.new_dc(fore_color=Fore.GREEN)
    red_dc = format_text.new_dc(fore_color=Fore.RED, back_color=Back.LIGHTYELLOW_EX)

    script_description = \
        '\n'.join([desc for desc in
                   [f'\n{green_dc(f"python {Path(__file__).name} [REFERENCE TEMPLATE] [OUTPUT FILE NAME]")} to create template.',
                    f'{green_dc(f"python {Path(__file__).name} -l *")} to get all available template',
                    f'{green_dc(f"python {Path(__file__).name} -o open")} open template directory so that you can put your template file there.',
                    # <- add your own description
                    ]])
    arg_parser = ArgumentParser(description=yellow_dc('CREATE TEMPLATE TOOL'),
                                # conflict_handler='resolve',
                                usage=script_description, formatter_class=RawTextHelpFormatter)

    arg_parser.add_argument("ref", help="reference template", nargs='?')
    arg_parser.add_argument("outfile", help="output file name", nargs='?')
    arg_parser.add_argument("action_number", help="action number", nargs='?', type=int)
    arg_parser.add_argument('--list', "-l", dest='list',
                            help=f"example: {green_dc('-l *')} \n"
                                 "description: list current available template. (accept regex)")

    arg_parser.add_argument('--option', "-o", dest='option',
                            help='\n'.join([format_text(msg_data_list) for msg_data_list in [
                                ['example', 'description'],
                                [green_dc('-o open'), 'open template directory so that you can put your template file there.'],
                                [green_dc('-o run'), '...'],
                                [green_dc('-o ...'), '...'],
                                # <- add your own description
                            ]]))

    g_args = arg_parser.parse_args()
    task_run_list = [[False, lambda: startfile('.')] if g_args.option == 'open' else None,
                     [False, lambda: [print(template_file_path.stem) for template_file_path in TEMPLATE_DIR.glob(f'{g_args.list}.py')]] if g_args.list else None,
                     # <- add your own function
                     ]
    for leave_flag, func in [task_list for task_list in task_run_list if task_list]:
        func()
        if leave_flag:
            exit(0)

    # CHECK POSITIONAL ARGUMENTS
    for attr_name, value in vars(g_args).items():
        if attr_name.startswith('-') or value is not None:
            continue
        system('cls')
        print(f'error required values of {red_dc(attr_name)} is None')
        print(f"if you need help, please use help command to help you: {red_dc(f'python {__file__} -h')}")
        exit(-1)
    main(g_args)

Where the class of FormatText is the following

class FormatText:
    __slots__ = ['align_list']

    def __init__(self, align_list: list, autoreset=True):
        """
        USAGE::

            format_text = FormatText([(20, '<'), (60, '<')])
            red_dc = format_text.new_dc(fore_color=Fore.RED)
            print(red_dc(['column 1', 'column 2']))
            print(red_dc('good morning'))
        :param align_list:
        :param autoreset:
        """
        self.align_list = align_list
        colorama.init(autoreset=autoreset)

    def __call__(self, text_list: list):
        if len(text_list) != len(self.align_list):
            if isinstance(text_list, str):
                return text_list
            raise AttributeError
        return ' '.join(f'{txt:{flag}{int_align}}' for txt, (int_align, flag) in zip(text_list, self.align_list))

    def new_dc(self, fore_color: Fore = Fore.GREEN, back_color: Back = ""):  # DECORATOR
        """create a device context"""
        def wrap(msgs):
            return back_color + fore_color + self(msgs) + Fore.RESET
        return wrap

Answer 9

The following python 3 formatter appends the default value if one exists and preserves line lengths.

from argparse import ArgumentParser, ArgumentDefaultsHelpFormatter, \ 
                     RawTextHelpFormatter
import textwrap

class CustomArgumentFormatter(ArgumentDefaultsHelpFormatter, RawTextHelpFormatter):
    """Formats argument help which maintains line length restrictions as well as appends default value if present."""

    def _split_lines(self, text, width):
        text = super()._split_lines(text, width)
        new_text = []

        # loop through all the lines to create the correct wrapping for each line segment.
        for line in text:
            if not line:
                # this would be a new line.
                new_text.append(line)
                continue

            # wrap the line's help segment which preserves new lines but ensures line lengths are
            # honored
            new_text.extend(textwrap.wrap(line, width))

        return new_text

Then create your argument parser with your new formatter:

my_arg_parser = ArgumentParser(formatter_class=CustomArgumentFormatter)
# ... add your arguments ...
print(my_arg_parser.format_help())