PEP 306 – How to Change Python’s Grammar
- Author:
- Michael Hudson <mwh at python.net>, Jack Diederich <jackdied at gmail.com>, Nick Coghlan <ncoghlan at gmail.com>, Benjamin Peterson <benjamin at python.org>
- Status:
- Withdrawn
- Type:
- Informational
- Created:
- 29-Jan-2003
- Post-History:
- 30-Jan-2003
Note
This PEP has been moved to the Python dev guide [1].
Abstract
There’s more to changing Python’s grammar than editing
Grammar/Grammar
and Python/compile.c
. This PEP aims to be a
checklist of places that must also be fixed.
It is probably incomplete. If you see omissions, just add them if you can – you are not going to offend the author’s sense of ownership. Otherwise submit a bug or patch and assign it to mwh.
This PEP is not intended to be an instruction manual on Python grammar hacking, for several reasons.
Rationale
People are getting this wrong all the time; it took well over a
year before someone noticed [2] that adding the floor division
operator (//
) broke the parser
module.
Checklist
Grammar/Grammar
: OK, you’d probably worked this one out :)Parser/Python.asdl
may need changes to match theGrammar
. Runmake
to regenerateInclude/Python-ast.h
andPython/Python-ast.c
.Python/ast.c
will need changes to create the AST objects involved with theGrammar
change.Lib/compiler/ast.py
will need matching changes to the pure-python AST objects.Parser/pgen
needs to be rerun to regenerateInclude/graminit.h
andPython/graminit.c
. (make should handle this for you.)Python/symbtable.c
: This handles the symbol collection pass that happens immediately before the compilation pass.Python/compile.c
: You will need to create or modify thecompiler_*
functions to generate opcodes for your productions.- You may need to regenerate
Lib/symbol.py
and/orLib/token.py
and/orLib/keyword.py
. - The
parser
module. Add some of your new syntax totest_parser
, bang onModules/parsermodule.c
until it passes. - Add some usage of your new syntax to
test_grammar.py
. - The
compiler
package. A good test is to compile the standard library and test suite with thecompiler
package and then check it runs. Note that this only needs to be done in Python 2.x. - If you’ve gone so far as to change the token structure of
Python, then the
Lib/tokenizer.py
library module will need to be changed. - Certain changes may require tweaks to the library module
pyclbr
. - Documentation must be written!
- After everything’s been checked in, you’re likely to see a new
change to
Python/Python-ast.c
. This is because this (generated) file contains the SVN version of the source from which it was generated. There’s no way to avoid this; you just have to submit this file separately.
References
Copyright
This document has been placed in the public domain.
Source: https://github.com/python/peps/blob/main/pep-0306.txt
Last modified: 2018-07-21 23:07:02 GMT