Tools for static type checking in Python

40
votes

I'm working with a large existing Python codebase and would like to start adding in type annotations so I can get some level of static checking. I'm imagining something like Erlang, Strongtalk, or Typed Scheme/Racket.

I've seen quick-and-dirty decorators that insert dynamic checks based on function parameter and return type annotations, but I'm looking for something that is more robust and that performs checks at compile-time.

What tools are available right now for this kind of thing? I'm familiar with compilers and type checking and am definitely willing to improve an incomplete tool if it has a good foundation.

(Note: I'm not interested in a discussion of the pros/cons of static typing.)

EDIT: An example:

def put(d, k, v):
   d[k] = v

I'd like to be able to annotate the put function as having type put<K,V>(dict<K,V>, K, V) -> None.

UPDATE: The new PEP 484 (Sep 2014) defines a standard for static typing and type annotations in Python 3.5+. There's a type-checking tool called mypy that is compatible with PEP 484.

7
pythonstatic-typing
You can't always do a perfect job, but there are tools that can infer a lot. For example: PySonar - Kannan Goundan
This is so close to impossible it's not worth it. It's impossible in general as you know. Also, you'd need a very sophisticated version of structural typing to get even close to it - d[k] = v word with every single object d that has __setitem__, for instance, not just with instance of dict. Depending on the actual type of d, there may be various incompatible requirements on k (e.g. hashable, has __index__) and even on v. And every single object has e.g. __hash__, some of them just "don't work" at runtime. Static typechecking is not possible in Python. - user395760
Doing a perfect job is not possible. I know this. But Erlang, Smalltalk, and Scheme have static checking tools that are still useful. The __setitem__ check seems like a standard structural type check. There are ways to specify a required relationship between the types of d and k. The selective implementation of __hash__ isn't ideal, but Java has the same issue with UnsupportedOperationException. And even if I can't solve every problem, it doesn't mean I can't have a static checker that is still useful for some things. - Kannan Goundan
All of the above reasons given for why Python can't have static type checking also apply to Scheme and it's descendant Racket. But optional static type checking has successfully been added to Racket while preserving the "dynamic" programming style of the language. Gradual Typing for Python (Jeremy Siek et al, mentioned in one answer, should be released soon) applies similar ideas, and more, to Python, so it can be done. I think some have misinterpreted the question as "Can all dynamic checks be replaced by static type checking?" - clearly no, even in Java static types don't do that. - RD1
I agree that static type checking is not possible for every Python program. However, for certain subsets of it, it is (i.e. those that don't use dynamic features). And for an even bigger subset, type can be statically checked 99.99% of the time. I'm having much trouble using Python in medium (not to say big) applications. Unless the community comes up with solutions, Python will be relegated to "small stuff"". - Mario Rossi

7 Answers

17
votes

Edit 2016-11-11: Just use mypy. Type hints can be added gradually. In Python 3 source code, it verifies standard PEP 484 type hints. Types can still be expressed in Python 2 using special comments. Guido likes it.

This post was originally written a long time ago before mypy was a thing. I've preserved the post's original content below, even though it isn't quite accurate.

Original post:

You might want to check out some of the projects mentioned in this related StackOverflow post on static analysis for Python.

In summary:

Since Python uses duck typing extensively, things that might be called "type errors" in other languages might end up being "object X doesn't support method Y" in Python.

Edit 2011-05-17:

I agree with delnan that static typing is not possible for Python [apparently wrong]. But since our skepticism doesn't seem to deter you, I can only give you more information on the subject. I present:

  • A discussion of type inference for Python. (Other links are from here.)
  • Guido van van Rossum's articles on adding optional static typing: part 1 and part 2.
  • RPython, a subset of Python that might stand a chance of being statically analyzed enough to do some form of type checking.
6
votes

Check out this post: PySonar: a Static Analyzer for Python. PySonar is a tool that infers types using abstract interpretation (partially executing) of code. It finds all possible execution paths of your program and finds all possible types of all variables.

There are basically three versions of PySonar:

  • Open-sourced Java (Jython indexer)
  • Closed-sourced Java (Hidden in Google)
  • Open-sourced Python (mini-pysonar)

None of them (except of closed source one) is fully implemented. But the basic idea is that you can use it as a basis for your work.

3
votes

I don't know if this helps but for what it's worth, Jeremy Siek at U of Colorado did some work on gradual typing, and I found this doing a quick search. http://www.wiki.jvmlangsummit.com/pdf/28_Siek_gradual.pdf

My guess (I might be wrong) is there isn't any promising open source tools you can find at the moment since his research looks relatively new.

Your best bet might be to contact the authors and ask if they can release their code to you.

2
votes

There is the 'gradual' package for Python 3; see PIP or the Bitbucket Repo

Apparently this is an implementation by the group around Jeremy Siek who seems to be quite an authority in the field of gradual typing.

Some annotations are apparently necessary, here is an example:

from gradual import *

@typed
def calculate_total(a:int, b:int) -> int:
    return a + b//100

As far as annotations go, this is not so bad. I have not used the package so I cannot speak to its quality, but the syntax (and the people behind it) certainly make it look promising.

2
votes

I like prospector, backend of landscape.io. It combines output of existing analyzers, such as pylint, pyflakes, pep8, frosted..., into one report. Neat.

1
votes

I had a similar need some time ago. All the existing solutions that I've found had some problem or does not have a feature that I'd like to have, so I've made my own.

Here's how you use it:

from requiretype import require

@require(name=str, age=(int, float, long))
def greet_person(name, age):
    print "Hello {0} ({1})".format(name, age)

>>> greet_person("John", 42)
Hello John (42)

>>> greet_person("John", "Doe")
# [...traceback...]
TypeError: Doe is not a valid type.
Valid types: <type 'int'>, <type 'float'>, <type 'long'>

>>> greet_person(42, 43)
# [...traceback...]
TypeError: 42 is not a <type 'str'> type

I hope this is useful for you.

For more details look at:

P.S.: (quoting myself from github repo)

For most cases I'd recommend using tests instead of type checking since it's more natural to do that in Python. But, there are some cases where you want/need to specify a specific type to use and since python does not have type checks for parameters here's where this is useful.