Reducing the size of feature delivery batches

When multiple large changes are delivered together, a complex investigation may be required to determine the root cause of any new issues that arise. Large batch sizes also make it more likely that problems will be encountered, given that they include larger pieces of relatively untested code.

The easiest way to simplify those investigations and reduce the likelihood of users encountering problems is to reduce the size of the batches being shipped.

PEP 602 proposes to address this problem via the straightforward approach of reducing CPython’s typical batch size by 50%, shipping 12 months of changes each time, rather than accumulating 18+ months of changes.

PEP 605 proposes to address it by regularly delivering 2 months worth of changes to a subset of Python’s user base that opts in to running a rolling stream of beta releases (similar to running Windows Insider builds instead of the Windows retail release, or running Debian testing instead of Debian stable).

Reducing the latency of feature delivery

When only stable releases are seeing significant user adoption, and there’s a long period of time between stable releases, it creates an incredibly strong temptation for developers to push changes into stable releases before they’re really ready for general use.

PEP 602 proposes to address this problem by reducing the period of time between stable releases to 12 months rather than 18 months.

PEP 605 proposes to address it by actively creating a community of Python users that regularly install and use CPython beta releases, providing an incentive for core developers to start shipping changes earlier in the pre-release cycle, in order to obtain feedback before the feature gets locked down in a stable release.

Aligning the release cadence with the calendar year

While the current release cadence is nominally 18-24 months, in practice it has consistently been towards the 18 month end of that range. This means that the target dates for pre-releases and final releases move around from release to release, and the only way to remember them is to either look at the release PEP, or else to add those dates to your calendar. This is annoying for both individual volunteers and for corporate contributors, and also complicates alignment with events like PyCon US (typically April/May) and the now-annual core development sprints (typically in September).

PEP 602 proposes to address this problem by publishing a new release in October every year, and basing the pre-release calendar for each year off that.

PEP 605 proposes to address this problem by alternating between release years (where a new stable release is published in August), and non-release years (where only maintenance releases and new rolling beta releases are published).

Improving the pre-release design feedback cycle

One of the challenges of designing changes to the core interpreter and standard library APIs is that the user base in a position to provide feedback on nightly builds and the current pre-releases is relatively limited. This means that much user feedback isn’t received until after an API design has already shipped in a full X.Y.0 release.

If the API is a regular API, then deprecation cycles mean that it may take literally years to correct any design mistakes identified at that point. Marking APIs as provisional nominally offers a way to avoid that constraint, but actually taking advantage of that freedom causes other problems.

PEP 602 proposes to address this problem by starting the alpha period immediately after the previous stable release.

PEP 605 proposes to address this problem by actively promoting adoption of CPython pre-releases for running production workloads (not just for library and application compatibility testing), and adjusting the pre-release management process as necessary to make that a reasonable thing to do.

(Note: some standard library APIs are amenable to initially being shipped as part of separately versioned packages via PyPI, and only later incorporated into the standard library. This section is more about the lower level APIs and non-library features where that approach to obtaining early design feedback doesn’t apply)

Impact on users and redistributors that already skip some releases

It is already the case that not all users and redistributors update to every published CPython release series (for example, Debian stable and Ubuntu LTS sometimes skip releases due to the mismatch between their 24-month release cycles and CPython’s typically 18-month cycle).

The faster 12-month full release cadence in PEP 602 means that users in this category may end up skipping two releases where they would previously have only skipped one. However, the extended notice period for deprecations means that skipping a single release should no longer result in missed deprecation warnings.

The slower 24-month full release cadence in PEP 605 may move some of the users that have historically been in this category into the “update to every stable release” category.

Impact on users and redistributors that update to every release

Many of Python’s users never install a pre-release, but do update to every stable release series at some point after it is published.

PEP 602 aims to mitigate the potential negative impact on members of this group by keeping the minimum gap between releases to 12 months, and retaining the 18 month full support period for each release.

Keeping the 18-month full support period for each release branch means that the branches will spend roughly the same amount of time in full support and security-fix-only mode as they do now (~18 months and ~42 months, respectively).

PEP 605 aims to mitigate the potential negative impact on members of this group by increasing use during the pre-release period to achieve more stable final releases with wider ecosystem support at launch.

With a 24-month release cadence each release branch will spend proportionally more time in full support mode and less time in security-fix-only mode (~24 months and ~36 months, respectively).

Full discussion of the impact on this group is left to the individual PEPs.

Impact on users and redistributors of CPython nightly builds

Despite the difficulties of doing so, there are already some users and redistributors that take on the challenge of using or publishing the CPython master branch directly.

Neither PEP 602 nor PEP 605 should directly affect this group, but the rolling release stream proposal in PEP 605 aims to lower the barriers to more users adopting this style of usage, by allowing them to adopt the tested rolling beta stream, rather than needing to use the master branch directly.

Impact on maintainers of third party libraries

For maintainers of third party libraries, the key source of support complexity is the number of different Python versions in widespread use.

PEP 602 aims to mitigate the potential negative impact on members of this group by keeping the minimum gap between full releases to 12 months.

PEP 605 aims to mitigate the potential negative impact on members of this group by increasing the gap between full releases to 24 months, retaining the current policy of moving each release branch to security-fix-only mode not long after its successor is released, and retaining the “beta” naming scheme for the new rolling release stream (at least for the Python 3.9 release cycle).

Full discussion of the impact on this group is left to the individual PEPs.