PEP 614 – Relaxing Grammar Restrictions On Decorators

Author:: Brandt Bucher <brandt at python.org>
Sponsor:: Guido van Rossum <guido at python.org>
Status:: Final
Type:: Standards Track
Created:: 10-Feb-2020
Python-Version:: 3.9
Post-History:: 11-Feb-2020, 18-Feb-2020, 03-Mar-2020
Resolution:: Python-Dev thread

Table of Contents

Abstract
Motivation
Rationale
- Allowing Any Expression
- What Counts As An “Expression”
  - Tuple Displays Must Be Parenthesized
  - Named Expressions Need Not Be Parenthesized
Specification
Backwards Compatibility
How To Teach This
Reference Implementation
Copyright

Abstract

Python currently requires that all decorators consist of a dotted name, optionally followed by a single call. This PEP proposes removing these limitations and allowing decorators to be any valid expression.

Motivation

When decorators were first being introduced, Guido described the motivation to limit their syntax as a preference, not a technical requirement:

I have a gut feeling about this one. I’m not sure where it comes from, but I have it… So while it would be quite easy to change the syntax to @test in the future, I’d like to stick to with the more restricted form unless a real use case is presented where allowing @test would increase readability.

While these limitations were rarely encountered in practice, BPO issues and mailing list posts have consistently surfaced over the years requesting that they be removed. The most recent one (which prompted this proposal) contained a good example of code using the PyQt5 library that would become more readable, idiomatic, and maintainable if the existing restrictions were relaxed. Slightly modified:

buttons = [QPushButton(f'Button {i}') for i in range(10)]

# Do stuff with the list of buttons...

@buttons[0].clicked.connect
def spam():
    ...

@buttons[1].clicked.connect
def eggs():
    ...

# Do stuff with the list of buttons...

Currently, these decorations must be rewritten as something like:

button_0 = buttons[0]

@button_0.clicked.connect
def spam():
    ...

button_1 = buttons[1]

@button_1.clicked.connect
def eggs():
    ...

Further, the current grammar is already loose enough that it’s trivial to hack more complicated decorator expressions together. So rather than disallow arbitrarily complex expressions, as intended, the current restrictions only make them uglier and less efficient:

# Identity function hack:

def _(x):
    return x

@_(buttons[0].clicked.connect)
def spam():
    ...

# eval hack:

@eval("buttons[1].clicked.connect")
def eggs():
    ...

Rationale

Allowing Any Expression

The decision to allow any valid expression (and not just relaxing the current restrictions to allow, for example, subscripting) has been considered as the next logical step in the evolution of decorator grammar for quite some time. As Guido noted, during yet another mailing list thread:

I don’t think it’s reasonable to constrain it less than it currently is but more than a general expression.

Special-casing the grammar to allow some useful cases would only complicate the current situation, and all but guarantee that the process would repeat itself sometime in the future. Further, one purpose of this grammatical change is to discourage the temptation to use hacks like the eval and identity-function anti-patterns shown above.

In short: if we’re removing somewhat arbitrary restrictions, we should remove all of them.

What Counts As An “Expression”

Throughout this document, the word “expression” is used as defined in the Python Language Reference. This can be summarized as “anything that’s valid as a test in if, elif, and while blocks”. This differs subtly from a perhaps more popular definition, which can be summarized as “anything that’s valid as string input to eval”.

This definition of “expression” is convenient in that it fits our needs well, and reuses the allowed grammar of existing language constructs. It has two subtle differences from the other definition:

Tuple Displays Must Be Parenthesized

This is based on an observation Guido made in the same email. Continued immediately from above:

Though I wouldn’t allow commas– there’s no way that
@f, g
def pooh(): ...
can make sense.

Indeed, it may even lead inexperienced readers to conclude that several decorators are being applied, as if they were stacked. Requiring parentheses here makes the (admittedly nonsensical) intent clear without imposing further restrictions and grammar complications.

Named Expressions Need Not Be Parenthesized

Here, the choice of syntax is unambiguous. PEP 572 explains why it requires parentheses around top-level expression statements:

This rule is included to simplify the choice for the user between an assignment statement and an assignment expression – there is no syntactic position where both are valid.

Since an assignment statement is not valid here, assignment expressions should not be unnecessarily burdened with parentheses.

Specification

The grammar for decorators is currently:

decorator: '@' dotted_name [ '(' [arglist] ')' ] NEWLINE

This PEP proposes that it be simplified to:

decorator: '@' namedexpr_test NEWLINE

Last modified: 2021-02-11 22:32:18 GMT