3
Z'íc)x ã @ s d dl mZ d dlZd dlZd dlZd dlZd dlZd dlmZ d dl m
Z d dl m Z
y.y d dlZW n ek
r d dlZY nX W n ek
r dZY nX ddlmZ ddlmZ dd lmZmZ dd
lmZmZmZm Z m Z m Z m Z m Z m!Z!m"Z" dd l#m$Z$ dd l%m&Z&m'Z' ej(e)Z*G d
d de+Z,G dd de+Z-G dd de+Z.edk rhG dd de+Z/G dd de+Z0dd Z1G dd dej2Z3dS )é )Úabsolute_importN)Úcontextmanager)Úerror)Útimeouté )Úutil)ÚHTTPHeaderDict)Ú BaseSSLErrorÚ
HTTPException)
ÚBodyNotHttplibCompatibleÚ DecodeErrorÚ HTTPErrorÚIncompleteReadÚInvalidChunkLengthÚ
InvalidHeaderÚ
ProtocolErrorÚReadTimeoutErrorÚResponseNotChunkedÚSSLError)Úsix)Ú is_fp_closedÚis_response_to_headc @ s$ e Zd Zdd Zdd Zdd ZdS )ÚDeflateDecoderc C s d| _ d| _tj | _d S )NTó )Ú
_first_tryÚ_dataÚzlibÚ
decompressobjÚ_obj)Úself© r ú3/tmp/pip-build-gk9425m9/urllib3/urllib3/response.pyÚ__init__* s zDeflateDecoder.__init__c C s t | j|S )N)Úgetattrr )r Únamer r r! Ú __getattr__/ s zDeflateDecoder.__getattr__c C s |s|S | j s| jj|S | j|7 _y | jj|}|rFd| _ d | _|S tjk
r d| _ tjtj | _z | j| jS d | _X Y nX d S )NF)r r Ú
decompressr r r r Ú MAX_WBITS)r ÚdataÚ decompressedr r r! r& 2 s" zDeflateDecoder.decompressN)Ú__name__Ú
__module__Ú __qualname__r" r% r& r r r r! r ) s r c @ s e Zd ZdZdZdZdS )ÚGzipDecoderStater r é N)r* r+ r, Ú FIRST_MEMBERÚ
OTHER_MEMBERSÚ SWALLOW_DATAr r r r! r- I s r- c @ s$ e Zd Zdd Zdd Zdd ZdS )Ú GzipDecoderc C s t jdt j | _tj| _d S )Né )r r r' r r- r/ Ú_state)r r r r! r" Q s zGzipDecoder.__init__c C s t | j|S )N)r# r )r r$ r r r! r% U s zGzipDecoder.__getattr__c C s¦ t }| jtjks| r t|S xy|| jj|7 }W n8 tjk
rn | j}tj| _|tj krht|S Y nX | jj
}|st|S tj | _tj dtj | _q"W d S )Nr3 )
Ú bytearrayr4 r- r1 Úbytesr r& r r r0 Ú unused_datar r' )r r( ÚretZprevious_stater r r! r& X s"
zGzipDecoder.decompressN)r* r+ r, r" r% r& r r r r! r2 P s r2 c @ s e Zd Zdd Zdd ZdS )Ú
BrotliDecoderc C s0 t j | _t| jdr"| jj| _n
| jj| _d S )Nr& )ÚbrotliZ Decompressorr Úhasattrr& Úprocess)r r r r! r" t s
zBrotliDecoder.__init__c C s t | jdr| jj S dS )NÚflushr )r; r r= )r r r r! r= { s
zBrotliDecoder.flushN)r* r+ r, r" r= r r r r! r9 p s r9 c @ s( e Zd ZdZdd Zdd Zdd ZdS ) Ú MultiDecodera
From RFC7231:
If one or more encodings have been applied to a representation, the
sender that applied the encodings MUST generate a Content-Encoding
header field that lists the content codings in the order in which
they were applied.
c C s dd |j dD | _d S )Nc S s g | ]}t |j qS r )Ú _get_decoderÚstrip)Ú.0Úmr r r! ú
<listcomp> s z)MultiDecoder.__init__.<locals>.<listcomp>ú,)ÚsplitÚ _decoders)r Úmodesr r r! r" s zMultiDecoder.__init__c C s | j d j S )Nr )rF r= )r r r r! r= s zMultiDecoder.flushc C s" x t | jD ]}|j|}q W |S )N)ÚreversedrF r& )r r( Údr r r! r& s zMultiDecoder.decompressN)r* r+ r, Ú__doc__r" r= r& r r r r! r> s r> c C s: d| krt | S | dkr t S td k r4| dkr4t S t S )NrD ÚgzipÚbr)r> r2 r: r9 r )Úmoder r r! r? s r? c @ st e Zd ZdZddgZedk r&edg7 Zdddd d
gZdQddZdd Zdd Z dd Z
e dd Z e dd Z
dd Zd d Zd d Zd!d" ZeejfZedk r¬eejf7 Zd#d$ Zd%d& Zed'd( Zd)d* ZdRd+d,ZdTd/d0Z e d1d2 Z d3d4 Z dUd5d6Z d7d8 Z!d9d: Z"e d;d< Z#d=d> Z$d?d@ Z%dAdB Z&dCdD Z'dEdF Z(dGdH Z)dIdJ Z*dVdKdLZ+dMdN Z,dOdP Z-dS )WÚ HTTPResponseaá
HTTP Response container.
Backwards-compatible with :class:`http.client.HTTPResponse` but the response ``body`` is
loaded and decoded on-demand when the ``data`` property is accessed. This
class is also compatible with the Python standard library's :mod:`io`
module, and can hence be treated as a readable object in the context of that
framework.
Extra parameters for behaviour not present in :class:`http.client.HTTPResponse`:
:param preload_content:
If True, the response's body will be preloaded during construction.
:param decode_content:
If True, will attempt to decode the body based on the
'content-encoding' header.
:param original_response:
When this HTTPResponse wrapper is generated from an :class:`http.client.HTTPResponse`
object, it's convenient to include the original for debug purposes. It's
otherwise unused.
:param retries:
The retries contains the last :class:`~urllib3.util.retry.Retry` that
was used during the request.
:param enforce_content_length:
Enforce content length checking. Body returned by server must match
value of Content-Length header, if present. Otherwise, raise error.
rK ÚdeflateNrL i- i. i/ i3 i4 Ú r TFc C s t |tr|| _n
t|| _|| _|| _|| _|| _|| _|
| _|| _ || _
d | _ d | _ d | _
| | _d| _| | _|| _|rt |tjtfr|| _ |
| _| | _t|dr¬|| _
d| _d | _| jjddj }dd |j dD }d |krìd
| _| j || _ |r| j r| j |d | _ d S ) Nr ÚreadFztransfer-encodingrP c s s | ]}|j V qd S )N)r@ )rA Úencr r r! ú <genexpr> s z(HTTPResponse.__init__.<locals>.<genexpr>rD ÚchunkedT)Údecode_content) Ú
isinstancer ÚheadersÚstatusÚversionÚreasonÚstrictrU ÚretriesÚenforce_content_lengthÚ
auto_closeÚ_decoderÚ_bodyÚ_fpÚ_original_responseÚ_fp_bytes_readÚmsgÚ _request_urlr Ú string_typesr6 Ú_poolÚ _connectionr; rT Ú
chunk_leftÚgetÚlowerrE Ú _init_lengthÚlength_remainingrQ )r ÚbodyrW rX rY rZ r[ Úpreload_contentrU Úoriginal_responseÚpoolÚ
connectionrd r\ r] Úrequest_methodÚ request_urlr^ Útr_encÚ encodingsr r r! r" É sB
zHTTPResponse.__init__c C s | j | jkr| jjdS dS )a
Should we redirect and where to?
:returns: Truthy redirect location string if we got a redirect status
code and valid location. ``None`` if redirect status and no
location. ``False`` if not a redirect status code.
ÚlocationF)rX ÚREDIRECT_STATUSESrW rj )r r r r! Úget_redirect_location s z"HTTPResponse.get_redirect_locationc C s, | j s| j rd S | j j| j d | _d S )N)rg rh Ú _put_conn)r r r r! Ú release_conn s zHTTPResponse.release_connc C s. y | j W n ttttfk
r( Y nX dS )zÏ
Read and discard any remaining HTTP response data in the response connection.
Unread data in the HTTPResponse connection blocks the connection from being released back to the pool.
N)rQ r
Ú SocketErrorr r
)r r r r! Ú
drain_conn s zHTTPResponse.drain_connc C s" | j r | j S | jr | jddS d S )NT)Ú
cache_content)r` ra rQ )r r r r! r( + s zHTTPResponse.datac C s | j S )N)rh )r r r r! rr 4 s zHTTPResponse.connectionc C s
t | jS )N)r ra )r r r r! Úisclosed8 s zHTTPResponse.isclosedc C s | j S )zì
Obtain the number of bytes pulled over the wire so far. May differ from
the amount of content returned by :meth:``urllib3.response.HTTPResponse.read``
if bytes are encoded on the wire (e.g, compressed).
)rc )r r r r! Útell; s zHTTPResponse.tellc C sâ | j jd}|dk r| jr(tjd dS y<tdd |jdD }t|dkrZtd| |j }W n t
k
r| d}Y nX |d k rd}yt | j }W n t
k
r° d }Y nX |dksÚd | koÌd
k n sÚ|dkrÞd }|S )zM
Set initial length value for Response content if available.
zcontent-lengthNzÒReceived response with both Content-Length and Transfer-Encoding set. This is expressly forbidden by RFC 7230 sec 3.3.2. Ignoring Content-Length and attempting to process response as Transfer-Encoding: chunked.c S s g | ] }t |qS r )Úint)rA Úvalr r r! rC ] s z-HTTPResponse._init_length.<locals>.<listcomp>rD r z8Content-Length contained multiple unmatching values (%s)r éÌ é0 éd éÈ ÚHEAD)r r )
rW rj rT ÚlogÚwarningÚsetrE Úlenr ÚpopÚ
ValueErrorr rX )r rs ÚlengthÚlengthsrX r r r! rl C s0
(zHTTPResponse._init_lengthc sh j jddj } jdkrd| jkr2t| _n2d|krd fdd|jdD }t|rdt| _dS )z=
Set-up the _decoder attribute if necessary.
zcontent-encodingrP NrD c s" g | ]}|j jkr|j qS r )r@ ÚCONTENT_DECODERS)rA Úe)r r r! rC s z.HTTPResponse._init_decoder.<locals>.<listcomp>)rW rj rk r_ r r? rE r )r Úcontent_encodingrv r )r r! Ú
_init_decoderw s
zHTTPResponse._init_decoderc C sx |s|S y| j r | j j|}W nD | jk
rb } z&| jjddj }td| |W Y dd}~X nX |rt|| j 7 }|S )zN
Decode the data passed in and potentially flush the decoder.
zcontent-encodingrP zEReceived response with content-encoding: %s, but failed to decode it.N)r_ r& ÚDECODER_ERROR_CLASSESrW rj rk r Ú_flush_decoder)r r( rU Ú
flush_decoderr r r r r! Ú_decode s zHTTPResponse._decodec C s$ | j r | j jd}|| j j S dS )zk
Flushes the decoder. Should only be called if the decoder is actually
being used.
r )r_ r& r= )r Úbufr r r! r ¤ s zHTTPResponse._flush_decoderc c sò d}zªy
dV W n t k
r2 t| jddY nv tk
rt } z(dt|krVt|t| jddW Y dd}~X n4 ttfk
r¦ } ztd| |W Y dd}~X nX d}W d|sÔ| j rÄ| j j
| j rÔ| j j
| j rì| j j rì| j
X dS )zç
Catch low-level python exceptions, instead re-raising urllib3
variants, so that low-level exceptions are not leaked in the
high-level api.
On exit, release the connection back to the pool.
FNzRead timed out.zread operation timed outzConnection broken: %rT)Ú
SocketTimeoutr rg r Ústrr r
r| r rb Úcloserh r r{ )r Ú
clean_exitr r r r! Ú_error_catcher¯ s(
zHTTPResponse._error_catcherc C sÌ | j s
td
}|r||ks*| jrª| j|krªtj rªtjsBtjd k rªtj }d }xR|dks`|dkr |dk r|t
||}||8 }n|}| j j |}|sP |j | ~qPW |j
S |dk r¾| j j |S | j j S dS )
a_
Read a response with the thought that reading the number of bytes
larger than can fit in a 32-bit int at a time via SSL in some
known cases leads to an overflow error that has to be prevented
if `amt` or `self.length_remaining` indicate that a problem may
happen.
The known cases:
* 3.8 <= CPython < 3.9.7 because of a bug
https://github.com/urllib3/urllib3/issues/2513#issuecomment-1152559900.
* urllib3 injected with pyOpenSSL-backed SSL-support.
* CPython < 3.10 only when `amt` does not fit 32-bit int.
r. é r é é
é Nr l iÿÿÿ)r r i )ra ÚAssertionErrorrm r ÚIS_SECURETRANSPORTÚ IS_PYOPENSSLÚsysÚ version_infoÚioÚBytesIOÚminrQ ÚwriteÚgetvalue)r ÚamtZ c_int_maxÚbufferZ
max_chunk_amtZ chunk_amtr( r r r! Ú_fp_readç s(
zHTTPResponse._fp_readc C sö | j |dkr| j}| jdkr$dS d}t| jdd}| j d |sN| j|nd}|dkr`d}n>d}|dkr| r| jj d}| jr| jdkrt | j
| jW dQ R X |rò| j
t |7 _
| jdk rÚ| jt |8 _| j |||}|rò|| _
|S )aT
Similar to :meth:`http.client.HTTPResponse.read`, but with two additional
parameters: ``decode_content`` and ``cache_content``.
:param amt:
How much of the content to read. If specified, caching is skipped
because it doesn't make sense to cache partial content as the full
response.
:param decode_content:
If True, will attempt to decode the body based on the
'content-encoding' header.
:param cache_content:
If True, will save the returned data such that the same result is
returned despite of the state of the underlying file object. This
is useful if you want the ``.data`` property to continue working
after having ``.read()`` the file object. (Overridden if ``amt`` is
set.)
NFÚclosedr Tr )r N)r rU ra r# r r® r r] rm r rc r r r` )r r¬ rU r~ r Z fp_closedr( r r r! rQ s6
zHTTPResponse.readr. r3 c c sZ | j r.| j r.xF| j||dD ]
}|V q W n(x&t| jsT| j||d}|r0|V q0W dS )a_
A generator wrapper for the read() method. A call will block until
``amt`` bytes have been read from the connection or until the
connection is closed.
:param amt:
How much of the content to read. The generator will return up to
much data per iteration, but may return less. This is particularly
likely when using compressed data. However, the empty string will
never be returned.
:param decode_content:
If True, will attempt to decode the body based on the
'content-encoding' header.
)rU )r¬ rU N)rT Úsupports_chunked_readsÚ read_chunkedr ra rQ )r r¬ rU Úliner( r r r! Ústream_ s zHTTPResponse.streamc
K sb |j }t|ts.tjr"tj|}n t|j }t|dd}| f |||j|j |j
||d|}|S )a
Given an :class:`http.client.HTTPResponse` instance ``r``, return a
corresponding :class:`urllib3.response.HTTPResponse` object.
Remaining parameters are passed to the HTTPResponse constructor, along
with ``original_response=r``.
r[ r )rn rW rX rY rZ r[ rp ) rd rV r r ÚPY2Ú from_httplibÚitemsr# rX rY rZ )Ú ResponseClsÚrÚ response_kwrW r[ Úrespr r r! rµ y s
zHTTPResponse.from_httplibc C s t jdtdd | jS )Nz|HTTPResponse.getheaders() is deprecated and will be removed in urllib3 v2.1.0. Instead access HTTPResponse.headers directly.r. )ÚcategoryÚ
stacklevel)ÚwarningsÚwarnÚDeprecationWarningrW )r r r r! Ú
getheaders s
zHTTPResponse.getheadersc C s t jdtdd | jj||S )NzHTTPResponse.getheader() is deprecated and will be removed in urllib3 v2.1.0. Instead use HTTPResponse.headers.get(name, default).r. )r» r¼ )r½ r¾ r¿ rW rj )r r$ Údefaultr r r! Ú getheader£ s
zHTTPResponse.getheaderc C s | j S )N)rW )r r r r! Úinfo s zHTTPResponse.infoc C s6 | j s| jj | jr | jj | js2tjj| d S )N)r¯ ra r rh r^ r§ ÚIOBase)r r r r! r ± s
zHTTPResponse.closec C sT | j stjjj| S | jd kr"dS t| jdr8| jj S t| jdrL| jjS dS d S )NTr r¯ )r^ r§ rÄ r¯ Ú__get__ra r; r )r r r r! r¯ » s
zHTTPResponse.closedc C s6 | j d krtdn t| j dr*| j j S tdd S )Nz-HTTPResponse has no file to get a fileno fromÚfilenozOThe file-like object this HTTPResponse is wrapped around has no file descriptor)ra ÚIOErrorr; rÆ )r r r r! rÆ È s
zHTTPResponse.filenoc C s4 | j d k r0t| j dr0t| j dd r0| j j S d S )Nr= r¯ F)ra r; r# r= )r r r r! r= Ó s
zHTTPResponse.flushc C s dS )NTr )r r r r! ÚreadableÛ s zHTTPResponse.readablec C s: | j t|}t|dkr dS ||d t|
< t|S d S )Nr )rQ r )r ÚbÚtempr r r! Úreadintoß s
zHTTPResponse.readintoc C s t | jdS )a
Checks if the underlying file-like object looks like a
:class:`http.client.HTTPResponse` object. We do this by testing for
the fp attribute. If it is present we assume it returns raw chunks as
processed by read_chunked().
Úfp)r; ra )r r r r! r° è s z#HTTPResponse.supports_chunked_readsc C sf | j d k rd S | jjj }|jddd }yt|d| _ W n& tk
r` | j t| |Y nX d S )Nó ;r r r3 ) ri ra rÌ ÚreadlinerE r r r r )r r² r r r! Ú_update_chunk_lengthñ s
z!HTTPResponse._update_chunk_lengthc C s¬ d }|d kr2| j j| j}|}| j jd d | _nv|| jk rZ| j j|}| j| | _|}nN|| jkr| j j|}| j jd d | _|}n | j j| j}| j jd d | _|S )Nr. )ra Ú
_safe_readri )r r¬ Úreturned_chunkÚchunkÚvaluer r r! Ú
_handle_chunkÿ s&
zHTTPResponse._handle_chunkc c sò | j | jstd| j s&td| j º | jrNt| jrN| jj dS | j j
dkr^dS x<| j | j dkrtP | j
|}| j||dd}|r`|V q`W |r²| j }|r²|V x | j j
j }|sÆP |dkr´P q´W | jrä| jj W dQ R X dS )aÆ
Similar to :meth:`HTTPResponse.read`, but with an additional
parameter: ``decode_content``.
:param amt:
How much of the content to read. If specified, caching is skipped
because it doesn't make sense to cache partial content as the full
response.
:param decode_content:
If True, will attempt to decode the body based on the
'content-encoding' header.
zHResponse is not chunked. Header 'transfer-encoding: chunked' is missing.zkBody should be http.client.HTTPResponse like. It should have have an fp attribute which returns raw chunks.Nr F)rU r s
)r rT r r° r r rb r r ra rÌ rÏ ri rÔ r r rÎ )r r¬ rU rÒ Údecodedr² r r r! r± sD
zHTTPResponse.read_chunkedc C s. | j dk r$t| j jr$| j jd jS | jS dS )zÄ
Returns the URL that was the source of this response.
If the request that generated this response redirected, this method
will return the final redirect location.
Nr éÿÿÿÿ)r\ r ÚhistoryÚredirect_locationre )r r r r! Úgeturl[ s zHTTPResponse.geturlc c s g }x| j ddD ]r}d|krz|jd}dj||d d V x |dd
D ]}|d V qNW |d rt|d g}qg }q|j| qW |rdj|V d S )
NT)rU ó
r r r rÖ rÖ rÖ )r³ rE ÚjoinÚappend)r r rÒ Úxr r r! Ú__iter__f s
zHTTPResponse.__iter__)rP Nr r Nr TTNNNNNFNNT)NNFé )rß N)N)NN).r* r+ r, rJ r r: rx r" ry r{ r} Úpropertyr( rr r r rl r rÇ r r r r r r r r® rQ r³ Ú classmethodrµ rÀ r rà r r¯ rÆ r= rÈ rË r° rÏ rÔ r± rÙ rÞ r r r r! rN £ sp
1
4
80
H
!
F rN )4Ú
__future__r r§ Úloggingr¥ r½ r Ú
contextlibr Úsocketr r| r r Z
brotlicffir: Ú ImportErrorrP r Ú _collectionsr rr r r
Ú
exceptionsr r r
r r r r r r r Úpackagesr Z
util.responser r Ú getLoggerr* r Úobjectr r- r2 r9 r> r? rÄ rN r r r r! Ú<module> s<
0