How does __future__ statements know the syntax of new python versions?

2
votes

I have gone through questions about how does the future module works.

What is __future__ in Python used for and how/when to use it, and how it works

and

How __future__ imports work under the hood .

But what I still don't understand is that how does a interpreter created today, knows the syntax of the code that is part of future releases.

What I'm thinking is that when they create a python interpreter for 2.7 version, they build the code and create an executable and don't touch it or modify it and the users will download it and use it. Let's say that python 2.7 executable is created in the year 2009.

And python 3.7 executable is created in 2017, how does python 2.7 knows what code to compile when it imports print_function? How does python 2.7 interpreter know what is the code for print_function is going to be which will be introduced after the interpreter is created.

Am I looking at it in the right way or completely wrong?

Can someone please demystify this.

2
pythonpython-3.xpython-2.7interpreter
The __future__ module contains methods that are 'backported' to python2, the interpreter doesn't need to know about any future developments. - rdas
The Python parser is configurable using runtime operations. - Barmar
The two questions you linked to mention how this is done, e.g. telling the Python interpreter Python that print is not a special keyword. This allows it to be defined as a function in the module. - Barmar
It is nothing more "mystical" than saying "I'm going to have a chicken sandwich for lunch this Friday". The plans for later versions language are laid out, implemented, and sometimes also made available in current versions as an "opt-in" future statement (to ease transitions between versions). - wim

2 Answers

5
votes

from __future__ import ... is just an overly cute way of saying a feature is available now, but you have to opt in. In the future, it will be the default or required.

Each defined import comes with two pieces of information: the version in which the feature is made optionally available, and the version in which the feature will become mandatory. No feature is ever removed from the __future__ module.

As of Python 3.8, there are nine features available in __future__. All but one, annotations, are mandatory. New code that will never need to run in older versions of Python don't need to import them. Older code that uses them don't need to be updated, even though the imports are effectively no-ops. annotations will remain an optional feature until Python 4.0 is released; there's no date for that, but it is guaranteed that annotations will be part of that release, and not a mandatory part of any earlier release.

4
votes

Python 2.7.0 was released after Python 3.0.0, i.e. after print has been turned into a function. Such changes in syntax can be made available to the compiler by passing options which are activated (recognized) through __future__ imports.

Such new syntax is implemented in the compiler indeed; only the activation is optional. That means you do need to download a more recent version of Python in order to opt-in for one of the __future__ features. Python 2.7.0 was released in 2010 while Python 3.0.0 was released in 2008. So from the 2.7 point of view it was actual an "old" feature. Even though Python 2.5.6 was released three years after 3.0.0 it doesn't contain the opt-in for the print function since it was not implemented in that minor version of Python (2.5.0 which was released in 2006). The term "future" here refers to future (minor) versions of Python where a feature will become mandatory. But the plans (the PEP) for that feature are actually created earlier (i.e. before the version that introduces that feature as optional is released). For example PEP 479: StopIteration handling inside generators:

  • PEP created in 2014
  • Optional in 3.5.0, released in 2015
  • Mandatory in 3.7, released in 2018

Even the feature became mandatory in 2018, the plans (its PEP) were created long before.

The __future__ docs give a good overview of when a feature became available. Not all features have been backported to all (minor) versions of Python even though they support the __future__ imports. This is because the features became available after these minor versions have been released.