http.cookiejar
—— HTTP 客户端的 Cookie 处理¶
http.cookiejar
模块定义了用于自动处理 HTTP cookie 的类。这对访问需要小段数据 —— cookies 的网站很有用,这些数据由 Web 服务器的 HTTP 响应在客户端计算机上设置,然后在以后的 HTTP 请求中返回给服务器。
常规的 Netscape cookie 协议和由 RFC 2965 定义的协议都可以被处理。 RFC 2965 的处理默认是关闭的。 RFC 2109 cookie 被解析为 Netscape cookie ,随后根据当前使用的 “策略”,被视为 Netscape 或 RFC 2965 cookie。http.cookiejar
试图遵循事实上的 Netscape cookie 协议(它与原始Netscape规范中的协议有很大不同),包括注意到 RFC 2965 中引入的``max-age``和``port``cookie 属性。
备注
在 Set-Cookie 和 Set-Cookie2 头中找到的各种命名参数通常指 attributes。为了不与 Python 属性相混淆,模块文档使用 cookie-attribute 代替。
此模块定义了以下异常:
- exception http.cookiejar.LoadError¶
FileCookieJar
实例在从文件加载 cookies 出错时抛出这个异常。LoadError
是OSError
的一个子类。
提供了以下类:
- class http.cookiejar.CookieJar(policy=None)¶
policy 是实现了
CookiePolicy
接口的一个对象。CookieJar
类储存 HTTP cookies。它从 HTTP 请求提取 cookies,并在 HTTP 响应中返回它们。CookieJar
实例在必要时自动处理包含 cookie 的到期情况。子类还负责储存和从文件或数据库中查找 cookies。
- class http.cookiejar.FileCookieJar(filename=None, delayload=None, policy=None)¶
policy 是实现了
CookiePolicy
接口的一个对象。对于其他参数,参考相应属性的文档。一个可以从硬盘中文件加载或保存 cookie 的
CookieJar
。 Cookies 不 会在load()
或revert()
方法调用前从命名的文件中加载。子类的文档位于段落 FileCookieJar subclasses and co-operation with web browsers。This should not be initialized directly – use its subclasses below instead.
在 3.8 版更改: 文件名形参支持 path-like object。
- class http.cookiejar.CookiePolicy¶
此类负责确定是否应从服务器接受每个 cookie 或将其返回给服务器。
- class http.cookiejar.DefaultCookiePolicy(blocked_domains=None, allowed_domains=None, netscape=True, rfc2965=False, rfc2109_as_netscape=None, hide_cookie2=False, strict_domain=False, strict_rfc2965_unverifiable=True, strict_ns_unverifiable=False, strict_ns_domain=DefaultCookiePolicy.DomainLiberal, strict_ns_set_initial_dollar=False, strict_ns_set_path=False, secure_protocols=('https', 'wss'))¶
构造参数只能以关键字参数传递,blocked_domains 是一个我们既不会接受也不会返回 cookie 的域名序列。allowed_domains 如果不是
None
,则是仅有的我们会接受或返回的域名序列。secure_protocols 是可以添加安全 cookies 的协议序列。默认将 https 和 wss (安全 WebSocket)考虑为安全协议。对于其他参数,参考CookiePolicy
和DefaultCookiePolicy
对象的文档。DefaultCookiePolicy
实现了 Netscape 和 RFC 2965 cookies 的标准接受 / 拒绝规则。 默认情况下,RFC 2109 cookies(即在 Set-Cookie 头中收到的 cookie-attribute 版本为 1 的 cookies )将按照 RFC 2965 规则处理。 然而,如果 RFC 2965 的处理被关闭,或者rfc2109_as_netscape
为True
,Cookie
实例的version
属性设置将被为 0, RFC 2109 cookiesCookieJar
实例将 “降级” 为 Netscape cookies。DefaultCookiePolicy
也提供一些参数以允许一些策略微调。
- class http.cookiejar.Cookie¶
这个类代表 Netscape、RFC 2109 和 RFC 2965 的 cookie 。 我们不希望
http.cookiejar
的用户构建他们自己的Cookie
实例。 如果有必要,请在一个CookieJar
实例上调用make_cookies()
。
参见
- 模块
urllib.request
URL 打开带有自动的 cookie 处理。
- 模块
http.cookies
HTTP cookie类,主要是对服务端代码有用。
http.cookiejar
和http.cookies
模块不相互依赖。- https://curl.se/rfc/cookie_spec.html
原始 Netscape cookie 协议的规范。 虽然这仍然是主流协议,但所有主要浏览器(以及
http.cookiejar
)实现的 “Netscape cookie协议” 与``cookie_spec.html``中描述的协议仅有几分相似之处。- RFC 2109 - HTTP状态管理机制
被 RFC 2965 所取代。使用 Set-Cookie version=1 。
- RFC 2965 - HTTP状态管理机制
修正了错误的 Netscape 协议。 使用 Set-Cookie2 来代替 Set-Cookie 。 没有广泛被使用。
- http://kristol.org/cookie/errata.html
未完成的:rfc:2965 勘误表。
RFC 2964 - HTTP状态管理使用方法
CookieJar 和 FileCookieJar 对象¶
CookieJar
对象支持 iterator 协议,用于迭代包含的 Cookie
对象。
CookieJar
有以下方法:
- CookieJar.add_cookie_header(request)¶
在 request 中添加正确的 Cookie 头。
如果策略允许(即
rfc2965
和hide_cookie2
属性在CookieJar
的CookiePolicy
实例中分别为 True 和 False ), Cookie2 标头也会在适当时候添加。The request object (usually a
urllib.request.Request
instance) must support the methodsget_full_url()
,has_header()
,get_header()
,header_items()
,add_unredirected_header()
and the attributeshost
,type
,unverifiable
andorigin_req_host
as documented byurllib.request
.在 3.3 版更改: request 对象需要
origin_req_host
属性。对已废弃的方法get_origin_req_host()
的依赖已被移除。
- CookieJar.extract_cookies(response, request)¶
从HTTP response 中提取 cookie,并在政策允许的情况下,将它们存储在
CookieJar
中。CookieJar
将 在*response* 参数中寻找允许的 Set-Cookie 和 Set-Cookie2 头信息,并适当地存储cookies(须经CookiePolicy.set_ok()
方法批准)。response 对象(通常是调用
urllib.request.urlopen()
或类似方法的结果)应该支持info()
方法,它返回email.message.Message
实例。The request object (usually a
urllib.request.Request
instance) must support the methodget_full_url()
and the attributeshost
,unverifiable
andorigin_req_host
, as documented byurllib.request
. The request is used to set default values for cookie-attributes as well as for checking that the cookie is allowed to be set.在 3.3 版更改: request 对象需要
origin_req_host
属性。对已废弃的方法get_origin_req_host()
的依赖已被移除。
- CookieJar.set_policy(policy)¶
设置要使用的
CookiePolicy
实例。
- CookieJar.make_cookies(response, request)¶
返回从 response 对象中提取的
Cookie
对象的序列。关于 response 和 request 参数所需的接口,请参见
extract_cookies()
的文档。
- CookieJar.clear([domain[, path[, name]]])¶
清除一些cookie。
如果调用时没有参数,则清除所有的cookie。 如果给定一个参数,只有属于该 domain 的cookies将被删除。如果给定两个参数,那么属于指定的 domain 和 URL path 的cookie将被删除。 如果给定三个参数,那么属于指定的 domain 、path 和 name 的cookie将被删除
如果不存在匹配的 cookie,则会引发
KeyError
。
- CookieJar.clear_session_cookies()¶
丢弃所有session cookie。
Discards all contained cookies that have a true
discard
attribute (usually because they had either nomax-age
orexpires
cookie-attribute, or an explicitdiscard
cookie-attribute). For interactive browsers, the end of a session usually corresponds to closing the browser window.Note that the
save()
method won’t save session cookies anyway, unless you ask otherwise by passing a true ignore_discard argument.
FileCookieJar
implements the following additional methods:
- FileCookieJar.save(filename=None, ignore_discard=False, ignore_expires=False)¶
Save cookies to a file.
This base class raises
NotImplementedError
. Subclasses may leave this method unimplemented.filename is the name of file in which to save cookies. If filename is not specified,
self.filename
is used (whose default is the value passed to the constructor, if any); ifself.filename
isNone
,ValueError
is raised.ignore_discard: save even cookies set to be discarded. ignore_expires: save even cookies that have expired
The file is overwritten if it already exists, thus wiping all the cookies it contains. Saved cookies can be restored later using the
load()
orrevert()
methods.
- FileCookieJar.load(filename=None, ignore_discard=False, ignore_expires=False)¶
Load cookies from a file.
Old cookies are kept unless overwritten by newly loaded ones.
Arguments are as for
save()
.The named file must be in the format understood by the class, or
LoadError
will be raised. Also,OSError
may be raised, for example if the file does not exist.
- FileCookieJar.revert(filename=None, ignore_discard=False, ignore_expires=False)¶
Clear all cookies and reload cookies from a saved file.
revert()
can raise the same exceptions asload()
. If there is a failure, the object’s state will not be altered.
FileCookieJar
instances have the following public attributes:
- FileCookieJar.filename¶
Filename of default file in which to keep cookies. This attribute may be assigned to.
- FileCookieJar.delayload¶
If true, load cookies lazily from disk. This attribute should not be assigned to. This is only a hint, since this only affects performance, not behaviour (unless the cookies on disk are changing). A
CookieJar
object may ignore it. None of theFileCookieJar
classes included in the standard library lazily loads cookies.
FileCookieJar subclasses and co-operation with web browsers¶
The following CookieJar
subclasses are provided for reading and
writing.
- class http.cookiejar.MozillaCookieJar(filename=None, delayload=None, policy=None)¶
A
FileCookieJar
that can load from and save cookies to disk in the Mozillacookies.txt
file format (which is also used by curl and the Lynx and Netscape browsers).备注
This loses information about RFC 2965 cookies, and also about newer or non-standard cookie-attributes such as
port
.警告
Back up your cookies before saving if you have cookies whose loss / corruption would be inconvenient (there are some subtleties which may lead to slight changes in the file over a load / save round-trip).
Also note that cookies saved while Mozilla is running will get clobbered by Mozilla.
- class http.cookiejar.LWPCookieJar(filename=None, delayload=None, policy=None)¶
A
FileCookieJar
that can load from and save cookies to disk in format compatible with the libwww-perl library’sSet-Cookie3
file format. This is convenient if you want to store cookies in a human-readable file.在 3.8 版更改: 文件名形参支持 path-like object。
CookiePolicy 对象¶
Objects implementing the CookiePolicy
interface have the following
methods:
- CookiePolicy.set_ok(cookie, request)¶
Return boolean value indicating whether cookie should be accepted from server.
cookie is a
Cookie
instance. request is an object implementing the interface defined by the documentation forCookieJar.extract_cookies()
.
- CookiePolicy.return_ok(cookie, request)¶
Return boolean value indicating whether cookie should be returned to server.
cookie is a
Cookie
instance. request is an object implementing the interface defined by the documentation forCookieJar.add_cookie_header()
.
- CookiePolicy.domain_return_ok(domain, request)¶
Return
False
if cookies should not be returned, given cookie domain.This method is an optimization. It removes the need for checking every cookie with a particular domain (which might involve reading many files). Returning true from
domain_return_ok()
andpath_return_ok()
leaves all the work toreturn_ok()
.If
domain_return_ok()
returns true for the cookie domain,path_return_ok()
is called for the cookie path. Otherwise,path_return_ok()
andreturn_ok()
are never called for that cookie domain. Ifpath_return_ok()
returns true,return_ok()
is called with theCookie
object itself for a full check. Otherwise,return_ok()
is never called for that cookie path.Note that
domain_return_ok()
is called for every cookie domain, not just for the request domain. For example, the function might be called with both".example.com"
and"www.example.com"
if the request domain is"www.example.com"
. The same goes forpath_return_ok()
.The request argument is as documented for
return_ok()
.
- CookiePolicy.path_return_ok(path, request)¶
Return
False
if cookies should not be returned, given cookie path.See the documentation for
domain_return_ok()
.
In addition to implementing the methods above, implementations of the
CookiePolicy
interface must also supply the following attributes,
indicating which protocols should be used, and how. All of these attributes may
be assigned to.
- CookiePolicy.netscape¶
Implement Netscape protocol.
- CookiePolicy.hide_cookie2¶
Don’t add Cookie2 header to requests (the presence of this header indicates to the server that we understand RFC 2965 cookies).
The most useful way to define a CookiePolicy
class is by subclassing
from DefaultCookiePolicy
and overriding some or all of the methods
above. CookiePolicy
itself may be used as a ‘null policy’ to allow
setting and receiving any and all cookies (this is unlikely to be useful).
DefaultCookiePolicy 对象¶
Implements the standard rules for accepting and returning cookies.
Both RFC 2965 and Netscape cookies are covered. RFC 2965 handling is switched off by default.
The easiest way to provide your own policy is to override this class and call its methods in your overridden implementations before adding your own additional checks:
import http.cookiejar
class MyCookiePolicy(http.cookiejar.DefaultCookiePolicy):
def set_ok(self, cookie, request):
if not http.cookiejar.DefaultCookiePolicy.set_ok(self, cookie, request):
return False
if i_dont_want_to_store_this_cookie(cookie):
return False
return True
In addition to the features required to implement the CookiePolicy
interface, this class allows you to block and allow domains from setting and
receiving cookies. There are also some strictness switches that allow you to
tighten up the rather loose Netscape protocol rules a little bit (at the cost of
blocking some benign cookies).
A domain blocklist and allowlist is provided (both off by default). Only domains
not in the blocklist and present in the allowlist (if the allowlist is active)
participate in cookie setting and returning. Use the blocked_domains
constructor argument, and blocked_domains()
and
set_blocked_domains()
methods (and the corresponding argument and methods
for allowed_domains). If you set an allowlist, you can turn it off again by
setting it to None
.
Domains in block or allow lists that do not start with a dot must equal the
cookie domain to be matched. For example, "example.com"
matches a blocklist
entry of "example.com"
, but "www.example.com"
does not. Domains that do
start with a dot are matched by more specific domains too. For example, both
"www.example.com"
and "www.coyote.example.com"
match ".example.com"
(but "example.com"
itself does not). IP addresses are an exception, and
must match exactly. For example, if blocked_domains contains "192.168.1.2"
and ".168.1.2"
, 192.168.1.2 is blocked, but 193.168.1.2 is not.
DefaultCookiePolicy
implements the following additional methods:
- DefaultCookiePolicy.blocked_domains()¶
Return the sequence of blocked domains (as a tuple).
- DefaultCookiePolicy.set_blocked_domains(blocked_domains)¶
Set the sequence of blocked domains.
- DefaultCookiePolicy.is_blocked(domain)¶
Return
True
if domain is on the blocklist for setting or receiving cookies.
- DefaultCookiePolicy.allowed_domains()¶
Return
None
, or the sequence of allowed domains (as a tuple).
- DefaultCookiePolicy.set_allowed_domains(allowed_domains)¶
Set the sequence of allowed domains, or
None
.
- DefaultCookiePolicy.is_not_allowed(domain)¶
Return
True
if domain is not on the allowlist for setting or receiving cookies.
DefaultCookiePolicy
instances have the following attributes, which are
all initialised from the constructor arguments of the same name, and which may
all be assigned to.
- DefaultCookiePolicy.rfc2109_as_netscape¶
If true, request that the
CookieJar
instance downgrade RFC 2109 cookies (ie. cookies received in a Set-Cookie header with a version cookie-attribute of 1) to Netscape cookies by setting the version attribute of theCookie
instance to 0. The default value isNone
, in which case RFC 2109 cookies are downgraded if and only if RFC 2965 handling is turned off. Therefore, RFC 2109 cookies are downgraded by default.
General strictness switches:
- DefaultCookiePolicy.strict_domain¶
Don’t allow sites to set two-component domains with country-code top-level domains like
.co.uk
,.gov.uk
,.co.nz
.etc. This is far from perfect and isn’t guaranteed to work!
RFC 2965 protocol strictness switches:
- DefaultCookiePolicy.strict_rfc2965_unverifiable¶
Follow RFC 2965 rules on unverifiable transactions (usually, an unverifiable transaction is one resulting from a redirect or a request for an image hosted on another site). If this is false, cookies are never blocked on the basis of verifiability
Netscape protocol strictness switches:
- DefaultCookiePolicy.strict_ns_unverifiable¶
Apply RFC 2965 rules on unverifiable transactions even to Netscape cookies.
- DefaultCookiePolicy.strict_ns_domain¶
Flags indicating how strict to be with domain-matching rules for Netscape cookies. See below for acceptable values.
- DefaultCookiePolicy.strict_ns_set_initial_dollar¶
Ignore cookies in Set-Cookie: headers that have names starting with
'$'
.
- DefaultCookiePolicy.strict_ns_set_path¶
Don’t allow setting cookies whose path doesn’t path-match request URI.
strict_ns_domain
is a collection of flags. Its value is constructed by
or-ing together (for example, DomainStrictNoDots|DomainStrictNonDomain
means
both flags are set).
- DefaultCookiePolicy.DomainStrictNoDots¶
When setting cookies, the ‘host prefix’ must not contain a dot (eg.
www.foo.bar.com
can’t set a cookie for.bar.com
, becausewww.foo
contains a dot).
- DefaultCookiePolicy.DomainStrictNonDomain¶
Cookies that did not explicitly specify a
domain
cookie-attribute can only be returned to a domain equal to the domain that set the cookie (eg.spam.example.com
won’t be returned cookies fromexample.com
that had nodomain
cookie-attribute).
The following attributes are provided for convenience, and are the most useful combinations of the above flags:
- DefaultCookiePolicy.DomainLiberal¶
Equivalent to 0 (ie. all of the above Netscape domain strictness flags switched off).
- DefaultCookiePolicy.DomainStrict¶
Equivalent to
DomainStrictNoDots|DomainStrictNonDomain
.
Cookie 对象¶
Cookie
instances have Python attributes roughly corresponding to the
standard cookie-attributes specified in the various cookie standards. The
correspondence is not one-to-one, because there are complicated rules for
assigning default values, because the max-age
and expires
cookie-attributes contain equivalent information, and because RFC 2109 cookies
may be ‘downgraded’ by http.cookiejar
from version 1 to version 0 (Netscape)
cookies.
Assignment to these attributes should not be necessary other than in rare
circumstances in a CookiePolicy
method. The class does not enforce
internal consistency, so you should know what you’re doing if you do that.
- Cookie.version¶
Integer or
None
. Netscape cookies haveversion
0. RFC 2965 and RFC 2109 cookies have aversion
cookie-attribute of 1. However, note thathttp.cookiejar
may ‘downgrade’ RFC 2109 cookies to Netscape cookies, in which caseversion
is 0.
- Cookie.name¶
Cookie name (a string).
- Cookie.path¶
Cookie path (a string, eg.
'/acme/rocket_launchers'
).
- Cookie.secure¶
True
if cookie should only be returned over a secure connection.
- Cookie.expires¶
Integer expiry date in seconds since epoch, or
None
. See also theis_expired()
method.
- Cookie.discard¶
True
if this is a session cookie.
- Cookie.comment_url¶
URL linking to a comment from the server explaining the function of this cookie, or
None
.
- Cookie.rfc2109¶
True
if this cookie was received as an RFC 2109 cookie (ie. the cookie arrived in a Set-Cookie header, and the value of the Version cookie-attribute in that header was 1). This attribute is provided becausehttp.cookiejar
may ‘downgrade’ RFC 2109 cookies to Netscape cookies, in which caseversion
is 0.
- Cookie.port_specified¶
True
if a port or set of ports was explicitly specified by the server (in the Set-Cookie / Set-Cookie2 header).
- Cookie.domain_specified¶
True
if a domain was explicitly specified by the server.
- Cookie.domain_initial_dot¶
True
if the domain explicitly specified by the server began with a dot ('.'
).
Cookies may have additional non-standard cookie-attributes. These may be accessed using the following methods:
- Cookie.has_nonstandard_attr(name)¶
Return
True
if cookie has the named cookie-attribute.
- Cookie.get_nonstandard_attr(name, default=None)¶
If cookie has the named cookie-attribute, return its value. Otherwise, return default.
- Cookie.set_nonstandard_attr(name, value)¶
Set the value of the named cookie-attribute.
The Cookie
class also defines the following method:
- Cookie.is_expired(now=None)¶
True
if cookie has passed the time at which the server requested it should expire. If now is given (in seconds since the epoch), return whether the cookie has expired at the specified time.
例子¶
The first example shows the most common usage of http.cookiejar
:
import http.cookiejar, urllib.request
cj = http.cookiejar.CookieJar()
opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj))
r = opener.open("http://example.com/")
This example illustrates how to open a URL using your Netscape, Mozilla, or Lynx cookies (assumes Unix/Netscape convention for location of the cookies file):
import os, http.cookiejar, urllib.request
cj = http.cookiejar.MozillaCookieJar()
cj.load(os.path.join(os.path.expanduser("~"), ".netscape", "cookies.txt"))
opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj))
r = opener.open("http://example.com/")
The next example illustrates the use of DefaultCookiePolicy
. Turn on
RFC 2965 cookies, be more strict about domains when setting and returning
Netscape cookies, and block some domains from setting cookies or having them
returned:
import urllib.request
from http.cookiejar import CookieJar, DefaultCookiePolicy
policy = DefaultCookiePolicy(
rfc2965=True, strict_ns_domain=Policy.DomainStrict,
blocked_domains=["ads.net", ".ads.net"])
cj = CookieJar(policy)
opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj))
r = opener.open("http://example.com/")