python - Type annotations for *args and **kwargs

Question

I'm trying out Python's type annotations with abstract base classes to write some interfaces. Is there a way to annotate the possible types of *args and **kwargs?

For example, how would one express that the sensible arguments to a function are either an int or two ints? type(args) gives Tuple so my guess was to annotate the type as Union[Tuple[int, int], Tuple[int]], but this doesn't work.

from typing import Union, Tuple

def foo(*args: Union[Tuple[int, int], Tuple[int]]):
    try:
        i, j = args
        return i + j
    except ValueError:
        assert len(args) == 1
        i = args[0]
        return i

# ok
print(foo((1,)))
print(foo((1, 2)))
# mypy does not like this
print(foo(1))
print(foo(1, 2))

Error messages from mypy:

t.py: note: In function "foo":
t.py:6: error: Unsupported operand types for + ("tuple" and "Union[Tuple[int, int], Tuple[int]]")
t.py: note: At top level:
t.py:12: error: Argument 1 to "foo" has incompatible type "int"; expected "Union[Tuple[int, int], Tuple[int]]"
t.py:14: error: Argument 1 to "foo" has incompatible type "int"; expected "Union[Tuple[int, int], Tuple[int]]"
t.py:15: error: Argument 1 to "foo" has incompatible type "int"; expected "Union[Tuple[int, int], Tuple[int]]"
t.py:15: error: Argument 2 to "foo" has incompatible type "int"; expected "Union[Tuple[int, int], Tuple[int]]"

It makes sense that mypy doesn't like this for the function call because it expects there to be a tuple in the call itself. The addition after unpacking also gives a typing error that I don't understand.

How does one annotate the sensible types for *args and **kwargs?

Answer 1

For variable positional arguments (*args) and variable keyword arguments (**kw) you only need to specify the expected value for one such argument.

From the Arbitrary argument lists and default argument values section of the Type Hints PEP:

Arbitrary argument lists can as well be type annotated, so that the definition:

def foo(*args: str, **kwds: int): ...

is acceptable and it means that, e.g., all of the following represent function calls with valid types of arguments:

foo('a', 'b', 'c')
foo(x=1, y=2)
foo('', z=0)

So you'd want to specify your method like this:

def foo(*args: int):

However, if your function can only accept either one or two integer values, you should not use *args at all, use one explicit positional argument and a second keyword argument:

def foo(first: int, second: Optional[int] = None):

Now your function is actually limited to one or two arguments, and both must be integers if specified. *args always means 0 or more, and can't be limited by type hints to a more specific range.

Answer 2

The proper way to do this is using @overload

from typing import overload

@overload
def foo(arg1: int, arg2: int) -> int:
    ...

@overload
def foo(arg: int) -> int:
    ...

def foo(*args):
    try:
        i, j = args
        return i + j
    except ValueError:
        assert len(args) == 1
        i = args[0]
        return i

print(foo(1))
print(foo(1, 2))

Note that you do not add @overload or type annotations to the actual implementation, which must come last.

You'll need a newish version of both typing and mypy to get support for @overload outside of stub files.

You can also use this to vary the returned result in a way that makes explicit which argument types correspond with which return type. e.g.:

from typing import Tuple, overload

@overload
def foo(arg1: int, arg2: int) -> Tuple[int, int]:
    ...

@overload
def foo(arg: int) -> int:
    ...

def foo(*args):
    try:
        i, j = args
        return j, i
    except ValueError:
        assert len(args) == 1
        i = args[0]
        return i

print(foo(1))
print(foo(1, 2))

Answer 3

As a short addition to the previous answer, if you're trying to use mypy on Python 2 files and need to use comments to add types instead of annotations, you need to prefix the types for args and kwargs with * and ** respectively:

def foo(param, *args, **kwargs):
    # type: (bool, *str, **int) -> None
    pass

This is treated by mypy as being the same as the below, Python 3.5 version of foo:

def foo(param: bool, *args: str, **kwargs: int) -> None:
    pass

Answer 4

Not really supported yet

While you can annotate variadic arguments with a type, I don't find it very useful because it assumes that all arguments are of the same type.

The proper type annotation of *args and **kwargs that allows specifying each variadic argument separately is not supported by mypy yet. There is a proposal for adding an Expand helper on mypy_extensions module, it would work like this:

class Options(TypedDict):
    timeout: int
    alternative: str
    on_error: Callable[[int], None]
    on_timeout: Callable[[], None]
    ...

def fun(x: int, *, **options: Expand[Options]) -> None:
    ...

The GitHub issue was opened on January 2018 but it's still not closed. Note that while the issue is about **kwargs, the Expand syntax will likely be used for *args as well.

Answer 5

In some cases the content of **kwargs can be a variety of types.

This seems to work for me:

from typing import Any

def testfunc(**kwargs: Any) -> None:
    print(kwargs)

or

from typing import Any, Optional

def testfunc(**kwargs: Optional[Any]) -> None:
    print(kwargs)

In the case where you feel the need to constrain the types in **kwargs I suggest creating a struct-like object and add the typing there. This can be done with dataclasses, or pydantic.

from dataclasses import dataclass

@dataclass
class MyTypedKwargs:
   expected_variable: str
   other_expected_variable: int


def testfunc(expectedargs: MyTypedKwargs) -> None:
    pass

Answer 6

If one wants to describe specific named arguments expected in kwargs, one can instead pass in a TypedDict which defines required and optional parameters. Optional parameters are what were the kwargs:

import typing

class RequiredProps(typing.TypedDict):
    # all of these must be present
    a: int
    b: str

class OptionalProps(typing.TypedDict, total=False):
    # these can be included or they can be omitted
    c: int
    d: int

class ReqAndOptional(RequiredProps, OptionalProps):
    pass

def hi(req_and_optional: ReqAndOptional):
    print(req_and_optional)