argparse repeats itself when formatting help metavars

@rhettinger

BPO	45110
Nosy	@rhettinger, @jkloth, @LewisGaul

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-09-06.02:18:58.384>
labels = ['type-bug', 'library', '3.11']
title = 'argparse repeats itself when formatting help metavars'
updated_at = <Date 2021-09-11.08:55:11.744>
user = 'https://bugs.python.org/forest'

bugs.python.org fields:

activity = <Date 2021-09-11.08:55:11.744>
actor = 'LewisGaul'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2021-09-06.02:18:58.384>
creator = 'forest'
dependencies = []
files = []
hgrepos = []
issue_num = 45110
keywords = []
message_count = 13.0
messages = ['401110', '401111', '401112', '401113', '401114', '401115', '401116', '401118', '401119', '401120', '401121', '401122', '401635']
nosy_count = 5.0
nosy_names = ['rhettinger', 'forest', 'jkloth', 'paul.j3', 'LewisGaul']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue45110'
versions = ['Python 3.11']

When argparse actions have multiple option strings and at least one argument, the default formatter presents them like this:

-t ARGUMENT, --task ARGUMENT
Perform a task with the given argument.
-p STRING, --print STRING
Print the given string.

By repeating the metavars, the formatter wastes horizontal space, making the following side-effects more likely:

The easy-to-read tabular format is undermined by overlapping text columns.
An action and its description are split apart, onto separate lines.
Fewer actions can fit on the screen at once.
The user is presented with extra noise (repeat text) to read through.

I think the DRY principle is worth considering here. Help text would be cleaner, more compact, and easier to read if formatted like this:

-t, --task ARGUMENT Perform a task with the given argument.
-p, --print STRING Print the given string.

Obviously, actions with especially long option strings or metavars could still trigger line breaks, but they would be much less common and still easier to read without the repeat text.

I am aware of ArgumentParser's formatter_class option, but unfortunately, it is of little help here. Since the default formatter class reserves every stage of its work as a private implementation detail, I cannot safely subclass it to get the behavior I want. My choices are apparently to either re-implement an unreasonably large swath of its code in my own formatter class, or override the private _format_action_invocation() method in a subclass and risk future breakage (and still have to re-implement more code than is reasonable.)

Would it make sense to give HelpFormatter a "don't repeat yourself" option? (For example, a boolean class attribute could be overridden by a subclass and would be a small change to the existing code.)

Alternatively, if nobody is attached to the current behavior, would it make sense to simply change HelpFormatter such that it never repeats itself?

This is has been requested various times on StackOverflow, and possibly here (I'd have to do a search).

The closest thing to making a compact action_invocation is to set the metavar to '', and even thing we get a space, eg.

 -f , --foo   Help text

This repeat has been a part of argparse from the beginning, so I can't see changing the default behavior. But we could add a HelpFormatter subclass that changes one (or two methods) such as _format_action_invocation. Subclassing the formatter is the accepted way of adding help features. I, and possibly others, must have suggested such a change on SO.

Of course people can use such a subclass without it being part of the standard module.

https://bugs.python.org/issue42980 argparse: GNU-style help formatter

https://bugs.python.org/issue33389 argparse redundant help string

https://bugs.python.org/issue29626
Issue with spacing in argparse module while using help

https://bugs.python.org/issue27303
[argparse] Unify options in help output

https://stackoverflow.com/questions/23936145/python-argparse-help-message-disable-metavar-for-short-options

https://stackoverflow.com/questions/18275023/dont-show-long-options-twice-in-print-help-from-argparse

On Mon, 06 Sep 2021 04:58:38 +0000, paul j3 wrote:

This repeat has been a part of argparse from the beginning, so I can't
see changing the default behavior.

Yes, I guessed as much, which is why I first suggested making it optional.

But we could add a HelpFormatter subclass that changes one (or two methods)
such as _format_action_invocation. Subclassing the formatter is the accepted
way of adding help features.

If it was done in a subclass, how would people be expected to get the new
behavior *and* that of the other subclasses? For example, someone with
pre-formatted description and epilog text is currently directed to use the
RawDescriptionHelpFormatter subclass. If they also wanted to avoid repeat
metavars, and that behavior was implemented in another subclass, would they
be expected to write a third subclass inheriting from both module-defined
subclasses?

To me, multiclassing seems rather heavyweight for a simple behavior change
like this one, but yes, I recognize that argparse's current code uses that
approach. Pity, that.

Of course people can use such a subclass without it being part of the
standard module.

Technically: sure. But practically: not so much. An application would have
to subclass and override _format_action_invocation(), which (judging by the
leading underscore) appears to be intended as private to the module. Even
the module doc string says so:

"""
(Also note that HelpFormatter and RawDescriptionHelpFormatter are only
considered public as object names -- the API of the formatter objects is
still considered an implementation detail.)
"""

So, a subclass that isn't part of the standard module is implicity and
explicitly discouraged by the module itself.

If we're married to the module's current policy for formatter tweaks, I
guess that leaves a module-defined subclass as the only option. Here is
an example that works:

class OneMetavarHelpFormatter(argparse.HelpFormatter):
    """A formatter that avoids repeating action argument metavars.
    """
    def _format_action_invocation(self, action):
        "Format action help without repeating the argument metavar"
        if not action.option_strings or action.nargs == 0:
            return super()._format_action_invocation(action)

        default = self._get_default_metavar_for_optional(action)
        args_string = self._format_args(action, default)
        return ', '.join(action.option_strings) + ' ' + args_string

Here's another working example, allowing alternate separator strings (as
requested in bpo-33389) via subclassing:

class OneMetavarHelpFormatter(argparse.HelpFormatter):
    """A formatter that avoids repeating action metavars.
    """
    OPTION_SEPARATOR = ', '
    METAVAR_SEPARATOR = ' '

    def _format_action_invocation(self, action):
        """Format action help without repeating the argument metavar
        """
        if not action.option_strings or action.nargs == 0:
            return super()._format_action_invocation(action)

        default = self._get_default_metavar_for_optional(action)
        args_string = self._format_args(action, default)
        options_string = self.OPTION_SEPARATOR.join(action.option_strings)
        return options_string + self.METAVAR_SEPARATOR + args_string

To be clear, I wrote those examples to be non-invasive, not patch proposals.
A cleaner approach would be possible if patching argparse is an option. (I
believe the patch in bpo-42980 proposes such an approach.)

The idea of combining help features by defining a subclass that inherits from other subclasses was endorsed by the original developer (we could dig up an old bug/issue to prove that).

The provided subclasses all tweak a "private" method, often one that's buried deep in the calling stack.

I can't quote any official policy, but my sense is that Python developers are ok with users subclassing and modifying "private" methods. Methods, functions and classes (and variables) with leading '_' aren't documented, or imported via __all__, but otherwise the boundary between what is part of the user API and what's "hidden" is loose in Python.

Apparently some corporate policies prohibit use or modification of things that aren't in the public API, but I don't think that policy protects you from changes. 'argparse' changes at a glacial rate, with a lot of concern for backward compatibility. In fact it's that fear of unintended consequences that slows down the pace of change. Subclassing a help formatter is preferred because it minimizes the chance of hurting existing users.

Subclassing a help formatter is preferred because it minimizes the chance of
hurting existing users.

Fair enough.

Whatever the approach, I hope argparse can be made to support this through a
simple, documented interface. I had to grovel through standard library code
to figure out what to override and what to duplicate in order to get the
output I want. That seems like a needlessly high barrier for such a common
(and apparently oft-requested) format.

Big +1 from me for at least supporting a way to get the more concise output. I've never understood the verbosity of python's argparse where the metavar is repeated.

@rhettinger

I know that I'm just a beginner in Python and I'm trying to comment on Python contributors' message, but I would like to elaborate on @rhettinger point about:

repetition helps improve understanding

I'm currently writing simple cli task manager

parser.add_argument("-a", "--add", nargs=3, help="Add new task", metavar="'name' 'assigned to' 'description'")

And when I call my script I get this in zsh:

python3 src/cli.py -h 
usage: cli.py [-h] [-a "name" "assigned to" "description" "name" "assigned to" "description" "name" "assigned to" "description"]

Simple CLI program to track daily tasks.

options:
  -h, --help            show this help message and exit
  -a "name" "assigned to" "description" "name" "assigned to" "description" "name" "assigned to" "description", --add "name" "assigned to" "description" "name" "assigned to" "description" "name" "assigned to" "description"
                        Add new task

bash:

$ /bin/bash -c "python3 src/cli.py -h"
usage: cli.py [-h] [-a "name" "assigned to" "description" "name" "assigned to" "description" "name" "assigned to" "description"]

Simple CLI program to track daily tasks.

options:
  -h, --help            show this help message and exit
  -a "name" "assigned to" "description" "name" "assigned to" "description" "name" "assigned to" "description", --add "name" "assigned to" "description" "name" "assigned to" "description" "name" "assigned to" "description"
                        Add new task

So I think in some cases it could cause problems in understanding. It would be really great if there was an option to at least supress doubling METAVARS

This was fixed in #103372 and is the behaviour in 3.13 onward.

Navigation Menu

Repository navigation

argparse repeats itself when formatting help metavars #89273

12 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

argparse repeats itself when formatting help metavars #89273

Description

Activity

forest commented on Sep 6, 2021

12 remaining items

paulj3 commented on Sep 6, 2021

paulj3 commented on Sep 6, 2021

forest commented on Sep 6, 2021

forest commented on Sep 6, 2021

forest commented on Sep 6, 2021

paulj3 commented on Sep 6, 2021

forest commented on Sep 6, 2021

LewisGaul commented on Sep 11, 2021

0x6b61747a65 commented on May 26, 2023

savannahostrowski commented on Sep 24, 2024

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions