Add License-Expression field

The License-Expression optional field is specified to contain a text string that is a valid SPDX license expression, as defined herein.

Publishing tools SHOULD issue an informational warning if this field is missing, and MAY raise an error. Build tools MAY issue a similar warning, but MUST NOT raise an error.

A license expression is a string using the SPDX license expression syntax as documented in the SPDX specification, either Version 2.2 or a later compatible version.

When used in the License-Expression field and as a specialization of the SPDX license expression definition, a license expression can use the following license identifiers:

• Any SPDX-listed license short-form identifiers that are published in the SPDX License List, version 3.17 or any later compatible version. Note that the SPDX working group never removes any license identifiers; instead, they may choose to mark an identifier as “deprecated”.
• The LicenseRef-Public-Domain and LicenseRef-Proprietary strings, to identify licenses that are not included in the SPDX license list.

When processing the License-Expression field to determine if it contains a valid license expression, build and publishing tools:

• SHOULD halt execution and raise an error if:
  • The field does not contain a valid license expression
  • One or more license identifiers are not valid (as defined above)
• SHOULD report an informational warning, and publishing tools MAY raise an error, if one or more license identifiers have been marked as deprecated in the SPDX License List.
• MUST store a case-normalized version of the License-Expression field using the reference case for each SPDX license identifier and uppercase for the AND, OR and WITH keywords.
• SHOULD report an informational warning, and MAY raise an error if the normalization process results in changes to the License-Expression field contents.

For all newly-upload distributions that include a License-Expression field, the Python Package Index (PyPI) MUST validate that it contains a valid, case-normalized license expression with valid identifiers (as defined here) and MUST reject uploads that do not. PyPI MAY reject an upload for using a deprecated license identifier, so long as it was deprecated as of the above-mentioned SPDX License List version.

Add License-File field

Each instance of the License-File optional field is specified to contain the string representation of the path in the project source tree, relative to the project root directory, of a license-related file. It is a multi-use field that may appear zero or more times, each instance listing the path to one such file. Files specified under this field could include license text, author/attribution information, or other legal notices that need to be distributed with the package.

As specified by this PEP, its value is also that file’s path relative to the root license directory in both installed projects and the standardized distribution package types. In other legacy, non-standard or new distribution package formats and mechanisms of accessing and storing core metadata, the value MAY correspond to the license file path relative to a format-defined root license directory. Alternatively, it MAY be treated as a unique abstract key to access the license file contents by another means, as specified by the format.

If a License-File is listed in a source or built distribution’s core metadata, that file MUST be included in the distribution at the specified path relative to the root license directory, and MUST be installed with the distribution at that same relative path.

The specified relative path MUST be consistent between project source trees, source distributions (sdists), built distributions (wheels) and installed projects. Therefore, inside the root license directory, packaging tools MUST reproduce the directory structure under which the source license files are located relative to the project root.

Path delimiters MUST be the forward slash character (/), and parent directory indicators (..) MUST NOT be used. License file content MUST be UTF-8 encoded text.

Build tools MAY and publishing tools SHOULD produce an informative warning if a built distribution’s metadata contains no License-File entries, and publishing tools MAY but build tools MUST NOT raise an error.

For all newly-uploaded distribution packages that include one or more License-File fields and declare a Metadata-Version of 2.4 or higher, PyPI SHOULD validate that the specified files are present in all uploaded distributions, and MUST reject uploads that do not validate.

Deprecate License field

The legacy unstructured-text License field is deprecated and replaced by the new License-Expression field. Build and publishing tools MUST raise an error if both these fields are present and their values are not identical, including capitalization and excluding leading and trailing whitespace.

If only the License field is present, such tools SHOULD issue a warning informing users it is deprecated and recommending License-Expression instead.

For all newly-uploaded distributions that include a License-Expression field, the Python Package Index (PyPI) MUST reject any that specify a License field and the text of which is not identical to that of License-Expression, as defined in this section.

Along with license classifiers, the License field may be removed from a new version of the specification in a future PEP.

Deprecate license classifiers

Using license classifiers in the Classifier field (described in the core metadata specification) is deprecated and replaced by the more precise License-Expression field.

If the License-Expression field is present, build tools SHOULD and publishing tools MUST raise an error if one or more license classifiers is included in a Classifier field, and MUST NOT add such classifiers themselves.

Otherwise, if this field contains a license classifier, build tools MAY and publishing tools SHOULD issue a warning informing users such classifiers are deprecated, and recommending License-Expression instead. For compatibility with existing publishing and installation processes, the presence of license classifiers SHOULD NOT raise an error unless License-Expression is also provided.

For all newly-uploaded distributions that include a License-Expression field, the Python Package Index (PyPI) MUST reject any that also specify any license classifiers.

New license classifiers MUST NOT be added to PyPI; users needing them SHOULD use the License-Expression field instead. Along with the License field, license classifiers may be removed from a new version of the specification in a future PEP.

Add string value to license key

A top-level string value is defined for the license key in the [project] table, which is specified to be a valid SPDX license expression, as defined previously. Its value maps to the License-Expression field in the core metadata.

Build tools SHOULD validate the expression as described in the Add License-Expression field section, outputting an error or warning as specified. When generating the core metadata, tools MUST perform case normalization.

If a top-level string value for the license key is present and valid, for purposes of backward compatibility tools MAY back-fill the License core metadata field with the normalized value of the license key.

Add license-files key

A new license-files key is added to the [project] table for specifying paths in the project source tree relative to pyproject.toml to file(s) containing licenses and other legal notices to be distributed with the package. It corresponds to the License-File fields in the core metadata.

Its value is a table, which if present MUST contain one of two optional, mutually exclusive subkeys, paths and globs; if both are specified, tools MUST raise an error. Both are arrays of strings; the paths subkey contains verbatim file paths, and the globs subkey valid glob patterns, which MUST be parsable by the glob module in the Python standard library.

Note: To avoid ambiguity, confusion and (per PEP 20, the Zen of Python) “more than one (obvious) way to do it”, allowing a flat array of strings as the value for the license-files key has been left out for now.

Path delimiters MUST be the forward slash character (/), and parent directory indicators (..) MUST NOT be used. Tools MUST assume that license file content is valid UTF-8 encoded text, and SHOULD validate this and raise an error if it is not.

If the paths subkey is a non-empty array, build tools:

• MUST treat each value as a verbatim, literal file path, and MUST NOT treat them as glob patterns.
• MUST include each listed file in all distribution archives.
• MUST NOT match any additional license files beyond those explicitly statically specified by the user under the paths subkey.
• MUST list each file path under a License-File field in the core metadata.
• MUST raise an error if one or more paths do not correspond to a valid file in the project source that can be copied into the distribution archive.

If the globs subkey is a non-empty array, build tools:

• MUST treat each value as a glob pattern, and MUST raise an error if the pattern contains invalid glob syntax.
• MUST include all files matched by at least one listed pattern in all distribution archives.
• MAY exclude files matched by glob patterns that can be unambiguously determined to be backup, temporary, hidden, OS-generated or VCS-ignored.
• MUST list each matched file path under a License-File field in the core metadata.
• SHOULD issue a warning and MAY raise an error if no files are matched.
• MAY issue a warning if any individual user-specified pattern does not match at least one file.

If the license-files key is present, and the paths or globs subkey is set to a value of an empty array, then tools MUST NOT include any license files and MUST NOT raise an error.

If the license-files key is not present and not explicitly marked as dynamic, tools MUST assume a default value of the following:

license-files.globs = ["LICEN[CS]E*", "COPYING*", "NOTICE*", "AUTHORS*"]

In this case, tools MAY issue a warning if no license files are matched, but MUST NOT raise an error.

If the license-files key is marked as dynamic (and not present), to preserve consistent behavior with current tools and help ensure the packages they create are legally distributable, build tools SHOULD default to including at least the license files matching the above patterns, unless the user has explicitly specified their own.

Deprecate license key table subkeys

Table values for the license key in the [project] table, including the text and file table subkeys, are now deprecated. If the new license-files key is present, build tools MUST raise an error if the license key is defined and has a value other than a single top-level string.

If the new license-files key is not present and the text subkey is present in a license table, tools SHOULD issue a warning informing users it is deprecated and recommending a license expression as a top-level string key instead.

Likewise, if the new license-files key is not present and the file subkey is present in the license table, tools SHOULD issue a warning informing users it is deprecated and recommending the license-files key instead.

If the specified license file is present in the source tree, build tools SHOULD use it to fill the License-File field in the core metadata, and MUST include the specified file as if it were specified in a license-file.paths field. If the file does not exist at the specified path, tools MUST raise an informative error as previously specified. However, tools MUST also still assume the specified default value for the license-files key and also include, in addition to a license file specified under the license.file subkey, any license files that match the specified list of patterns.

Table values for the license key MAY be removed from a new version of the specification in a future PEP.

Mapping license classifiers to SPDX identifiers

Most single license classifiers (namely, all those not mentioned below) map to a single valid SPDX license identifier, allowing tools to infer the SPDX license identifier they correspond to, both for use when analyzing and auditing packages, and providing a semi-automated mechanism of filling the license key or the License-Expression field following the specification above.

Some legacy license classifiers intend to specify a particular license, but do not specify the particular version or variant, leading to a critical ambiguity as to their terms, compatibility and acceptability. Tools MUST NOT attempt to automatically infer a License-Expression when one of these classifiers is used without affirmative user action:

• License :: OSI Approved :: Academic Free License (AFL)
• License :: OSI Approved :: Apache Software License
• License :: OSI Approved :: Apple Public Source License
• License :: OSI Approved :: Artistic License
• License :: OSI Approved :: BSD License
• License :: OSI Approved :: GNU Affero General Public License v3
• License :: OSI Approved :: GNU Free Documentation License (FDL)
• License :: OSI Approved :: GNU General Public License (GPL)
• License :: OSI Approved :: GNU General Public License v2 (GPLv2)
• License :: OSI Approved :: GNU General Public License v3 (GPLv3)
• License :: OSI Approved :: GNU Lesser General Public License v2 (LGPLv2)
• License :: OSI Approved :: GNU Lesser General Public License v2 or later (LGPLv2+)
• License :: OSI Approved :: GNU Lesser General Public License v3 (LGPLv3)
• License :: OSI Approved :: GNU Library or Lesser General Public License (LGPL)

A comprehensive mapping of these classifiers to their possible specific identifiers was assembled by Dustin Ingram, which tools MAY use as a reference for the identifier selection options to offer users when prompting the user to explicitly select the license identifier they intended for their project.

In addition, for the various special cases, the following mappings are considered canonical and normative for the purposes of this specification:

• Classifier License :: Public Domain MAY be mapped to the generic License-Expression: LicenseRef-Public-Domain. If tools do so, they SHOULD issue an informational warning encouraging the use of more explicit and legally portable license identifiers, such as those for the CC0 1.0 license (CC0-1.0), the Unlicense (Unlicense), or the MIT license (MIT), since the meaning associated with the term “public domain” is thoroughly dependent on the specific legal jurisdiction involved, some of which lack the concept entirely. Alternatively, tools MAY choose to treat these classifiers as ambiguous.
• The generic and sometimes ambiguous classifiers:
  • License :: Free For Educational Use
  • License :: Free For Home Use
  • License :: Free for non-commercial use
  • License :: Freely Distributable
  • License :: Free To Use But Restricted
  • License :: Freeware
  • License :: Other/Proprietary License

  MAY be mapped to the generic License-Expression: LicenseRef-Proprietary, but tools MUST issue a prominent, informative warning if they do so. Alternatively, tools MAY choose to treat these classifiers as ambiguous.

• The generic and ambiguous classifiers License :: OSI Approved and License :: DFSG approved do not map to any license expression, and thus tools SHOULD treat them as ambiguous, or if not MUST ignore them.
• The classifiers License :: GUST Font License 1.0 and License :: GUST Font License 2006-09-30 have no mapping to SPDX license identifiers, and no PyPI package uses them as of 2022-07-09.

When multiple license classifiers are used, their relationship is ambiguous, and it is typically not possible to determine if all the licenses apply or if there is a choice that is possible among the licenses, In this case, tools MUST NOT automatically infer a license expression, unless one license classifier is a parent of the other, i.e. the child contains all ::-delineated components of the parent, in which case tools MAY ignore the parent classifier but SHOULD issue an informative warning when doing so.

Re-use the License field

Following initial discussion, earlier versions of this PEP proposed re-using the existing License field, which tools would attempt to parse as a SPDX license expression with a fallback to free text. Initially, this would merely cause a warning (or even pass silently), but would eventually be treated as an error by modern tooling.

This offered the potential benefit of greater backwards-compatibility, easing the community into using SPDX license expressions while taking advantage of packages that already have them (either intentionally or coincidentally), and avoided adding yet another license-related field.

However, following substantial discussion, consensus was reached that a dedicated License-Expression field was the preferred overall approach. The presence of this field is an unambiguous signal that a package intends it to be interpreted as a valid SPDX identifier, without the need for complex and potentially erroneous heuristics, and allows tools to easily and unambiguously detect invalid content.

This avoids both false positive (License values that a package author didn’t explicitly intend as an explicit SPDX identifier, but that happen to validate as one), and false negatives (expressions the author intended to be valid SPDX, but due to a typo or mistake are not), which are otherwise not clearly distinguishable from true positives and negatives, an ambiguity at odds with the goals of this PEP.

Furthermore, it allows both the existing License field and the license classifiers to be more easily deprecated, with tools able to cleanly distinguish between packages intending to affirmatively conform to the updated specification in this PEP or not, and adapt their behavior (warnings, errors, etc) accordingly. Otherwise, tools would either have to allow duplicative and potentially conflicting License fields and classifiers, or warn/error on the substantial number of existing packages that have SPDX identifiers as the value for the License field, intentionally or otherwise (e.g. MIT).

Finally, it avoids changing the behavior of an existing metadata field, and avoids tools having to guess the Metadata-Version and field behavior based on its value rather than merely its presence.

While this would mean the subset of existing distributions containing License fields valid as SPDX license expressions wouldn’t automatically be recognized as such, this only requires appending a few characters to the key name in the project’s source metadata, and this PEP provides extensive guidance on how this can be done automatically by tooling.

Given all this, it was decided to proceed with defining a new, purpose-created field, License-Expression.

Re-Use the License field with a value prefix

As an alternative to the previous, prefixing SPDX license expressions with, e.g. spdx: was suggested to reduce the ambiguity inherent in re-using the License field. However, this effectively amounted to creating a field within a field, and doesn’t address all the downsides of keeping the License field. Namely, it still changes the behavior of an existing metadata field, requires tools to parse its value to determine how to handle its content, and makes the specification and deprecation process more complex and less clean.

Yet, it still shares a same main potential downside as just creating a new field: projects currently using valid SPDX identifiers in the License field, intentionally or not, won’t be automatically recognized, and requires about the same amount of effort to fix, namely changing a line in the project’s source metadata. Therefore, it was rejected in favor of a new field.

Don’t make License-Expression mutually exclusive

For backwards compatibility, the License field and/or the license classifiers could still be allowed together with the new License-Expression field, presumably with a warning. However, this could easily lead to inconsistent, and at the very least duplicative license metadata in no less than three different fields, which is squarely contrary to the goals of this PEP of making the licensing story simpler and unambiguous. Therefore, and in concert with clear community consensus otherwise, this idea was soundly rejected.

Don’t deprecate existing License field and classifiers

Several community members were initially concerned that deprecating the existing License field and classifiers would result in excessive churn for existing package authors and raise the barrier to entry for new ones, particularly everyday Python developers seeking to package and publish their personal projects without necessarily caring too much about the legal technicalities or being a “license lawyer”. Indeed, every deprecation comes with some non-zero short-term cost, and should be carefully considered relative to the overall long-term net benefit. And at the minimum, this change shouldn’t make it more difficult for the average Python developer to share their work under a license of their choice, and ideally improve the situation.

Following many rounds of proposals, discussion and refinement, the general consensus was clearly in favor of deprecating the legacy means of specifying a license, in favor of “one obvious way to do it”, to improve the currently complex and fragmented story around license documentation. Not doing so would leave three different un-deprecated ways of specifying a license for a package, two of them ambiguous, less than clear/obvious how to use, inconsistently documented and out of date. This is more complex for all tools in the ecosystem to support indefinitely (rather than simply installers supporting older packages implementing previous frozen metadata versions), resulting in a non-trivial and unbounded maintenance cost.

Furthermore, it leads to a more complex and confusing landscape for users with three similar but distinct options to choose from, particularly with older documentation, answers and articles floating around suggesting different ones. Of the three, License-Expression is the simplest and clearest to use correctly; users just paste in their desired license identifier, or select it via a tool, and they’re done; no need to learn about Trove classifiers and dig through the list to figure out which one(s) apply (and be confused by many ambiguous options), or figure out on their own what should go in the license key (anything from nothing, to the license text, to a free-form description, to the same SPDX identifier they would be entering in the license key anyway, assuming they can easily find documentation at all about it). In fact, this can be made even easier thanks to the new field. For example, GitHub’s popular ChooseALicense.com links to how to add SPDX license identifiers to the project source metadata of various languages that support them right in the sidebar of every license page; the SPDX support in this PEP enables adding Python to that list.

For current package maintainers who have specified a License or license classifiers, this PEP only recommends warnings and prohibits errors for all but publishing tools, which are allowed to error if their intended distribution platform(s) so requires. Once maintainers are ready to upgrade, for those already using SPDX license expressions (accidentally or not) this only requires appending a few characters to the key name in the project’s source metadata, and for those with license classifiers that map to a single unambiguous license, or another defined case (public domain, proprietary), they merely need to drop the classifier and paste in the corresponding license identifier. This PEP provides extensive guidance and examples, as will other resources, as well as explicit instructions for automated tooling to take care of this with no human changes needed. More complex cases where license metadata is currently specified may need a bit of human intervention, but in most cases tools will be able to provide a list of options following the mappings in this PEP, and these are typically the projects most likely to be constrained by the limitations of the existing license metadata, and thus most benefited by the new fields in this PEP.

Finally, for unmaintained packages, those using tools supporting older metadata versions, or those who choose not to provide license metadata, no changes are required regardless of the deprecation.

Don’t mandate validating new fields on PyPI

Previously, while this PEP did include normative guidelines for packaging publishing tools (such as Twine), it did not provide specific guidance for PyPI (or other package indices) as to whether and how they should validate the License-Expression or License-File fields, nor how they should handle using them in combination with the deprecated License field or license classifiers. This simplifies the specification and either defers implementation on PyPI to a later PEP, or gives discretion to PyPI to enforce the stated invariants, to minimize disruption to package authors.

However, this had been left unstated from before the License-Expression field was separate from the existing License, which would make validation much more challenging and backwards-incompatible, breaking existing packages. With that change, there was a clear consensus that the new field should be validated from the start, guaranteeing that all distributions uploaded to PyPI that declare core metadata version 2.4 or higher and have the License-Expression field will have a valid expression, such that PyPI and consumers of its packages and metadata can rely upon to follow the specification here.

The same can be extended to the new License-File field as well, to ensure that it is valid and the legally required license files are present, and thus it is lawful for PyPI, users and downstream consumers to distribute the package. (Of course, this makes no guarantee of such as it is ultimately reliant on authors to declare them, but it improves assurance of this and allows doing so in the future if the community so decides.) To be clear, this would not require that any uploaded distribution have such metadata, only that if they choose to declare it per the new specification in this PEP, it is assured to be valid.

Add expression and files subkeys to table

A previous working draft of this PEP added expression and files subkeys to the existing license table in the project source metadata, to parallel the existing file and text subkeys. While this seemed perhaps the most obvious approach at first glance, it had several serious drawbacks relative to that ultimately taken here.

Most saliently, this means two very different types of metadata are being specified under the same top-level key that require very different handling, and furthermore, unlike the previous arrangement, the subkeys were not mutually exclusive and can both be specified at once, and with some subkeys potentially being dynamic and others static, and mapping to different core metadata fields.

Furthermore, this leads to a conflict with marking the key as dynamic (assuming that is intended to specify the [project] table keys, as that PEP seems to imprecisely imply, rather than core metadata fields), as either or both would have to be treated as dynamic. Grouping both license expressions and license files under the same key forces an “all or nothing” approach, and creates ambiguity as to user intent.

There are further downsides to this as well. Both users and tools would need to keep track of which fields are mutually exclusive with which of the others, greatly increasing cognitive and code complexity, and in turn the probability of errors. Conceptually, juxtaposing so many different fields under the same key is rather jarring, and leads to a much more complex mapping between [project] keys and core metadata fields, not in keeping with PEP 621. This causes the [project] table naming and structure to diverge further from both the core metadata and native formats of the various popular packaging tools that use it. Finally, this results in the spec being significantly more complex and convoluted to understand and implement than the alternatives.

The approach this PEP now takes, using the reserved top-level string value of the license key, adding a new license-files key and deprecating the license table subkeys (text and file), avoids most of the issues identified above, and results in a much clearer and cleaner design overall. It allows license and license-files to be tagged dynamic independently, separates two independent types of metadata (syntactically and semantically), restores a closer to 1:1 mapping of [project] table keys to core metadata fields, and reduces nesting by a level for both. Other than adding one extra key to the file, there was no significant apparent downside to this latter approach, so it was adopted for this PEP.

Add an expression subkey instead of a string value

Adding just an expression subkey to the license table, instead of using the reserved top-level string value, would be more explicit for readers and writers, in line with this PEP’s goals. However, it still has the downsides listed above that are not specific to the inclusion of the files key.

Relative to a flat string value, it adds verbosity, complexity and an extra level of nesting, and requires users and tools to remember and handle the mutual exclusivity of the subkeys and remember which are deprecated and which are not, instead of cleanly deprecating the table subkeys as a whole. Furthermore, it is less clearly the “default” choice for modern use, given users tend to gravitate toward the simplest and most obvious option. Finally, it seems reasonable to follow the suggested guidance in PEP 621, given the top-level string value was specifically reserved for this purpose.

Define a new top-level license-expression key

An earlier version of this PEP defined a new, top-level license-expression under the [project] table, rather than using the reserved string value of the license key. This was seen as clearer and more explicit for readers and writers, in line with the goals of this PEP.

Additionally, while differences from existing tool formats (and core metadata field names) have precedent in PEP 621, using a key with an identical name as in most/all current tools to mean something different (and map to a different core metadata field), with distinct and incompatible syntax and semantics, does not, and could cause confusion and ambiguity for readers and authors.

Also, per the project source metadata spec, this would allow separately marking the [project] keys corresponding to the License and License-Expression metadata fields as dynamic, avoiding a potential concern with back-filling the License field from the License-Expression field as this PEP currently allows without it as license as dynamic (which would not be possible, since they both map to the same top-level key).

However, community consensus favored using the top-level string value of the existing license key, as reserved for this purpose by PEP 621:

A practical string value for the license key has been purposefully left out to allow for a future PEP to specify support for SPDX expressions (the same logic applies to any sort of “type” field specifying what license the file or text represents).

This is shorter and simpler for users to remember and type, avoids adding a new top-level key while taking advantage of an existing one, guides users toward using a license expression as the default, and follows what was envisioned in the original PEP 621.

Additionally, this allows cleanly deprecating the table values without deprecating the key itself, and makes them inherently mutually exclusive without users having to remember and tools having to enforce it.

Finally, consistency with other tool formats and the underlying core metadata was not considered a sufficient priority to override the advantages of using the existing key, and the dynamic concerns were mostly mitigated by not specifying legacy license to license expression conversion at build time, explicitly specifying backfilling the License field when not dynamic, and the fact that both fields are mutually exclusive, so there is little practical need to distinguish which is dynamic.

Therefore, a top-level string value for license was adopted for this PEP, as an earlier working draft had temporarily specified.

Add a type key to treat text as expression

Instead of using the reserved top-level string value of the license key in the [project] table, one could add a type subkey to the license table to control whether text (or a string value) is interpreted as free-text or a license expression. This could make backward compatibility a little more seamless, as older tools could ignore it and always treat text as license, while newer tools would know to treat it as a license expression, if type was set appropriately. Indeed, PEP 621 seems to suggest something of this sort as a possible alternative way that SPDX license expressions could be implemented.

However, all the same downsides as in the previous item apply here, including greater complexity, a more complex mapping between the project source metadata and core metadata and inconsistency between the presentation in tool config, project source metadata and core metadata, a much less clean deprecation, further bikeshedding over what to name it, and inability to mark one but not the other as dynamic, among others.

In addition, while theoretically potentially a little easier in the short term, in the long term it would mean users would always have to remember to specify the correct type to ensure their license expression is interpreted correctly, which adds work and potential for error; we could never safety change the default while being confident that users understand that what they are entering is unambiguously a license expression, with all the false positive and false negative issues as above.

Therefore, for these as well as the same reasons this approach was rejected for the core metadata in favor of a distinct License-Expression field, we similarly reject this here in favor of the reserved string value of the license key.

Must be marked dynamic to back-fill

The license key in the pyproject.toml could be required to be explicitly set to dynamic in order for the License core metadata field to be automatically back-filled from the top-level string value of the license key. This would be more explicit that the filling will be done, as strictly speaking the license key is not (and cannot be) specified in pyproject.toml, and satisfies a stricter interpretation of the letter of the previous PEP 621 specification that this PEP revises.

However, this doesn’t seem to be necessary, because it is simply using the static, verbatim literal value of the license key, as specified strictly in this PEP. Therefore, any conforming tool can trivially, deterministically and unambiguously derive this using only the static data in the pyproject.toml file itself.

Furthermore, this actually adds significant ambiguity, as it means the value could get filled arbitrarily by other tools, which would in turn compromise and conflict with the value of the new License-Expression field, which is why such is explicitly prohibited by this PEP. Therefore, not marking it as dynamic will ensure it is only handled in accordance with this PEP’s requirements.

Finally, users explicitly being told to mark it as dynamic, or not, to control filling behavior seems to be a bit of a mis-use of the dynamic field as apparently intended, and prevents tools from adapting to best practices (fill, don’t fill, etc) as they develop and evolve over time.

Add a type subkey to license-files

Instead of defining mutually exclusive paths and globs subkeys of the license-files [project] table key, we could achieve the same effect with a files subkey for the list and a type subkey for how to interpret it. However, the latter offers no real advantage over the former, in exchange for requiring more keystrokes, verbosity and complexity, as well as less flexibility in allowing both, or another additional subkey in the future, as well as the need to bikeshed over the subkey name. Therefore, it was summarily rejected.

Only accept verbatim paths

Globs could be disallowed completely as values to the license-files key in pyproject.toml and only verbatim literal paths allowed. This would ensure that all license files are explicitly specified, all specified license files are found and included, and the source metadata is completely static in the strictest sense of the term, without tools having to inspect the rest of the project source files to determine exactly what license files will be included and what the License-File values will be. This would also modestly simplify the spec and tool implementation.

However, practicality once again beats purity here. Globs are supported and used by many existing tools for finding license files, and explicitly specifying the full path to every license file would be unnecessarily tedious for more complex projects with vendored code and dependencies. More critically, it would make it much easier to accidentally miss a required legal file, silently rendering the package illegal to distribute.

Tools can still statically and consistently determine the files to be included, based only on those glob patterns the user explicitly specified and the filenames in the package, without installing it, executing its code or even examining its files. Furthermore, tools are still explicitly allowed to warn if specified glob patterns (including full paths) don’t match any files. And, of course, sdists, wheels and others will have the full static list of files specified in their distribution metadata.

Perhaps most importantly, this would also preclude the currently specified default value, as widely used by the current most popular tools, and thus be a major break to backward compatibility, tool consistency, and safe and sane default functionality to avoid unintentional license violations. And of course, authors are welcome and encouraged to specify their license files explicitly via the paths table subkey, once they are aware of it and if it is suitable for their project and workflow.

Only accept glob patterns

Conversely, all license-files strings could be treated as glob patterns. This would slightly simplify the spec and implementation, avoid an extra level of nesting, and more closely match the configuration format of existing tools.

However, for the cost of a few characters, it ensures users are aware whether they are entering globs or verbatim paths. Furthermore, allowing license files to be specified as literal paths avoids edge cases, such as those containing glob characters (or those confusingly or even maliciously similar to them, as described in PEP 672).

Including an explicit paths value ensures that the resulting License-File metadata is correct, complete and purely static in the strictest sense of the term, with all license paths explicitly specified in the pyproject.toml file, guaranteed to be included and with an early error should any be missing. This is not practical to do, at least without serious limitations for many workflows, if we must assume the items are glob patterns rather than literal paths.

This allows tools to locate them and know the exact values of the License-File core metadata fields without having to traverse the source tree of the project and match globs, potentially allowing easier, more efficient and reliable programmatic inspection and processing.

Therefore, given the relatively small cost and the significant benefits, this approach was not adopted.

Infer whether paths or globs

It was considered whether to simply allow specifying an array of strings directly for the license-files key, rather than making it a table with explicit paths and globs. This would be somewhat simpler and avoid an extra level of nesting, and more closely match the configuration format of existing tools. However, it was ultimately rejected in favor of separate, mutually exclusive paths and globs table subkeys.

In practice, it only saves six extra characters in the pyproject.toml (license-files = [...] vs license-files.globs = [...]), but allows the user to more explicitly declare their intent, ensures they understand how the values are going to be interpreted, and serves as an unambiguous indicator for tools to parse them as globs rather than verbatim path literals.

This, in turn, allows for more appropriate, clearly specified tool behaviors for each case, many of which would be unreliable or impossible without it, to avoid common traps, provide more helpful feedback and behave more sensibly and intuitively overall. These include, with paths, guaranteeing that each and every specified file is included and immediately raising an error if one is missing, and with globs, checking glob syntax, excluding unwanted backup, temporary, or other such files (as current tools already do), and optionally warning if a glob doesn’t match any files. This also avoids edge cases (e.g. paths that contain glob characters) and reliance on heuristics to determine interpretation—the very thing this PEP seeks to avoid.

Also allow a flat array value

Initially, after deciding to define license-files as a table of paths and globs, thought was given to making a top-level string array under the license-files key mean one or the other (probably globs, to match most current tools). This is slightly shorter and simpler, would allow gently nudging users toward a preferred one, and allow a slightly cleaner handling of the empty case (which, at present, is treated identically for either).

However, this again only saves six characters in the best case, and there isn’t an obvious choice; whether from a perspective of preference (both had clear use cases and benefits), nor as to which one users would naturally assume.

Flat may be better than nested, but in the face of ambiguity, users may not resist the temptation to guess. Requiring users to explicitly specify one or the other ensures they are aware of how their inputs will be handled, and is more readable for others, both human and machine alike. It also makes the spec and tool implementation slightly more complicated, and it can always be added in the future, but not removed without breaking backward compatibility. And finally, for the “preferred” option, it means there is more than one obvious way to do it.

Therefore, per PEP 20, the Zen of Python, this approach is hereby rejected.

Allow both paths and globs subkeys

Allowing both paths and globs subkeys to be specified under the license-files table was considered, as it could potentially allow more flexible handling for particularly complex projects, and specify on a per-pattern rather than overall basis whether license-files entries should be treated as paths or globs.

However, given the existing proposed approach already matches or exceeds the power and capabilities of those offered in tools’ config files, there isn’t clear demand for this and few likely cases that would benefit, it adds a large amount of complexity for relatively minimal gain, in terms of the specification, in tool implementations and in pyproject.toml itself.

There would be many more edge cases to deal with, such as how to handle files matched by both lists, and it conflicts in multiple places with the current specification for how tools should behave with one or the other, such as when no files match, guarantees of all files being included and of the file paths being explicitly, statically specified, and others.

Like the previous, if there is a clear need for it, it can be always allowed in the future in a backward-compatible manner (to the extent it is possible in the first place), while the same is not true of disallowing it. Therefore, it was decided to require the two subkeys to be mutually exclusive.

Rename paths subkey to files

Initially, it was considered whether to name the paths subkey of the license-files table files instead. However, paths was ultimately chosen, as calling the table subkey files resulted in duplication between the table name (license-files) and the subkey name (files), i.e. license-files.files = ["LICENSE.txt"], made it seem like the preferred/ default subkey when it was not, and lacked the same parallelism with globs in describing the format of the string entry rather than what was being pointed to.

Must be marked dynamic to use defaults

It may seem outwardly sensible, at least with a particularly restrictive interpretation of PEP 621’s description of the dynamic list, to consider requiring the license-files key to be explicitly marked as dynamic in order for the default glob patterns to be used, or alternatively for license files to be matched and included at all.

However, this is merely declaring a static, strictly-specified default value for this particular key, required to be used exactly by all conforming tools (so long as it is not marked dynamic, negating this argument entirely), and is no less static than any other set of glob patterns the user themself may specify. Furthermore, the resulting License-File core metadata values can still be determined with only a list of files in the source, without installing or executing any of the code, or even inspecting file contents.

Moreover, even if this were not so, practicality would trump purity, as this interpretation would be strictly backwards-incompatible with the existing format, and be inconsistent with the behavior with the existing tools. Further, this would create a very serious and likely risk of a large number of projects unknowingly no longer including legally mandatory license files, making their distribution technically illegal, and is thus not a sane, much less sensible default.

Finally, aside from adding an additional line of default-required boilerplate to the file, not defining the default as dynamic allows authors to clearly and unambiguously indicate when their build/packaging tools are going to be handling the inclusion of license files themselves rather than strictly conforming to the project source metadata portions of this PEP; to do otherwise would defeat the primary purpose of the dynamic list as a marker and escape hatch.

Flatten license files in subdirectories

Previous drafts of this PEP were silent on the issue of handling license files in subdirectories. Currently, the Wheel and (following its example) Setuptools projects flatten all license files into the .dist-info directory without preserving the source subdirectory hierarchy.

While this is the simplest approach and matches existing ad hoc practice, this can result in name conflicts and license files clobbering others, with no obvious defined behavior for how to resolve them, and leaving the package legally un-distributable without any clear indication to users that their specified license files have not been included.

Furthermore, this leads to inconsistent relative file paths for non-root license files between the source, sdist and wheel, and prevents the paths given in the “static” [project] table metadata from being truly static, as they need to be flattened, and may potentially overwrite one another. Finally, the source directory structure often implies valuable information about what the licenses apply to, and where to find them in the source, which is lost when flattening them and far from trivial to reconstruct.

To resolve this, the PEP now proposes, as did contributors on both of the above issues, reproducing the source directory structure of the original license files inside the .dist-info directory. This would fully resolve the concerns above, with the only downside being a more nested .dist-info directory. There is still a risk of collision with edge-case custom filenames (e.g. RECORD, METADATA), but that is also the case with the previous approach, and in fact with fewer files flattened into the root, this would actually reduce the risk. Furthermore, the following proposal rooting the license files under a licenses subdirectory eliminates both collisions and the clutter problem entirely.

Resolve name conflicts differently

Rather than preserving the source directory structure for license files inside the .dist-info directory, we could specify some other mechanism for conflict resolution, such as pre- or appending the parent directory name to the license filename, traversing up the tree until the name was unique, to avoid excessively nested directories.

However, this would not address the path consistency issues, would require much more discussion, coordination and bikeshedding, and further complicate the specification and the implementations. Therefore, it was rejected in favor of the simpler and more obvious solution of just preserving the source subdirectory layout, as many stakeholders have already advocated for.

Dump directly in .dist-info

Previously, the included license files were stored directly in the top-level .dist-info directory of built wheels and installed projects. This followed existing ad hoc practice, ensured most existing wheels currently using this feature will match new ones, and kept the specification simpler, with the license files always being stored in the same location relative to the core metadata regardless of distribution type.

However, this leads to a more cluttered .dist-info directory, littered with arbitrary license files and subdirectories, as opposed to separating licenses into their own namespace (which per the Zen of Python, PEP 20, are “one honking great idea”). While currently small, there is still a risk of collision with specific custom license filenames (e.g. RECORD, METADATA) in the .dist-info directory, which would only increase if and when additional files were specified here, and would require carefully limiting the potential filenames used to avoid likely conflicts with those of license-related files. Finally, putting licenses into their own specified subdirectory would allow humans and tools to quickly, easily and correctly list, copy and manipulate all of them at once (such as in distro packaging, legal checks, etc) without having to reference each of their paths from the core metadata.

Therefore, now is a prudent time to specify an alternate approach. The simplest and most obvious solution, as suggested by several on the Wheel and Setuptools implementation issues, is to simply root the license files relative to a licenses subdirectory of .dist-info. This is simple to implement and solves all the problems noted here, without clear significant drawbacks relative to other more complex options.

It does make the specification a bit more complex and less elegant, but implementation should remain equally simple. It does mean that wheels produced with following this change will have differently-located licenses than those prior, but as this was already true for those in subdirectories, and until this PEP there was no way of discovering these files or accessing them programmatically, this doesn’t seem likely to pose significant problems in practice. Given this will be much harder if not impossible to change later, once the status quo is standardized, tools are relying on the current behavior and there is much greater uptake of not only simply including license files but potentially accessing them as well using the core metadata, if we’re going to change it, now would be the time (particularly since we’re already introducing an edge-case change with how license files in subdirs are handled, along with other refinements).

Therefore, the latter has been incorporated into current drafts of this PEP.

Add new licenses category to wheel

Instead of defining a root license directory (licenses) inside the core metadata directory (.dist-info) for wheels, we could instead define a new category (and, presumably, a corresponding install scheme), similar to the others currently included under .data in the wheel archive, specifically for license files, called (e.g.) licenses. This was mentioned by the wheel creator, and would allow installing licenses somewhere more platform-appropriate and flexible than just the .dist-info directory in the site path, and potentially be conceptually cleaner than including them there.

However, at present, this PEP does not implement this idea, and it is deferred to a future one. It would add significant complexity and friction to this PEP, being primarily concerned with standardizing existing practice and updating the core metadata specification. Furthermore, doing so would likely require modifying sysconfig and the install schemes specified therein, alongside Wheel, Installer and other tools, which would be a non-trivial undertaking. While potentially slightly more complex for repackagers (such as those for Linux distributions), the current proposal still ensures all license files are included, and in a single dedicated directory (which can easily be copied or relocated downstream), and thus should still greatly improve the status quo in this regard without the attendant complexity.

In addition, this approach is not fully backwards compatible (since it isn’t transparent to tools that simply extract the wheel), is a greater departure from existing practice and would lead to more inconsistent license install locations from wheels of different versions. Finally, this would mean licenses would not be installed as proximately to their associated code, there would be more variability in the license root path across platforms and between built distributions and installed projects, accessing installed licenses programmatically would be more difficult, and a suitable install location and method would need to be created, discussed and decided that would avoid name clashes.

Therefore, to keep this PEP in scope, the current approach was retained.

Name the subdirectory license_files

Both licenses and license_files have been suggested as potential names for the root license directory inside .dist-info of wheels and installed projects. An initial draft of the PEP specified the former due to being slightly clearer and consistent with the name of the core metadata field (License-File) and the [project] table key (license-files). However, the current version of the PEP adopts the license name, due to a general preference by the community for its shorter length, greater simplicity and the lack of a separator character (_, -, etc.).

Map identifiers to license files

This would require using a mapping (as two parallel lists would be too prone to alignment errors), which would add extra complexity to how license are documented and add an additional nesting level.

A mapping would be needed, as it cannot be guaranteed that all expressions (keys) have a single license file associated with them (e.g. GPL with an exception may be in a single file) and that any expression does not have more than one. (e.g. an Apache license LICENSE and its NOTICE file, for instance, are two distinct files). For most common cases, a single license expression and one or more license files would be perfectly adequate. In the rarer and more complex cases where there are many licenses involved, authors can still safety use the fields specified here, just with a slight loss of clarity by not specifying which text file(s) map to which license identifier (though this should be clear in practice given each license identifier has corresponding SPDX-registered full license text), while not forcing the more complex data model (a mapping) on the large majority of users who do not need or want it.

We could of course have a data field with multiple possible value types (it’s a string, it’s a list, it’s a mapping!) but this could be a source of confusion. This is what has been done, for instance, in npm (historically) and in Rubygems (still today), and as result tools need to test the type of the metadata field before using it in code, while users are confused about when to use a list or a string. Therefore, this approach is rejected.

Map identifiers to source files

As discussed previously, file-level notices are out of scope for this PEP, and the existing SPDX-License-Identifier convention can already be used if this is needed without further specification here.

Don’t freeze compatibility with a specific SPDX version

This PEP could omit specifying a specific SPDX specification version, or one for the list of valid license identifiers, which would allow more flexible updates as the specification evolves without another PEP or equivalent.

However, serious concerns were expressed about a future SPDX update breaking compatibility with existing expressions and identifiers, leaving current packages with invalid metadata per the definition in this PEP. Requiring compatibility with a specific version of these specifications here and a PEP or similar process to update it avoids this contingency, and follows the practice of other packaging ecosystems.

Therefore, it was decided to specify a minimum version and requires tools to be compatible with it, while still allowing updates so long as they don’t break backward compatibility. This enables tools to immediate take advantage of improvements and accept new licenses, but also remain backwards compatible with the version specified here, balancing flexibility and compatibility.

Different licenses for source and binary distributions

As an additional use case, it was asked whether it was in scope for this PEP to handle cases where the license expression for a binary distribution (wheel) is different from that for a source distribution (sdist), such as in cases of non-pure-Python packages that compile and bundle binaries under different licenses than the project itself. An example cited was PyTorch, which contains CUDA from Nvidia, which is freely distributable but not open source. NumPy and SciPy also had similar issues, as reported by the original author of this PEP and now resolved for those cases.

However, given the inherent complexity here and a lack of an obvious mechanism to do so, the fact that each wheel would need its own license information, lack of support on PyPI for exposing license info on a per-distribution archive basis, and the relatively niche use case, it was determined to be out of scope for this PEP, and left to a future PEP to resolve if sufficient need and interest exists and an appropriate mechanism can be found.