1:mod:`http.client` --- HTTP protocol client 2=========================================== 3 4.. module:: http.client 5 :synopsis: HTTP and HTTPS protocol client (requires sockets). 6 7**Source code:** :source:`Lib/http/client.py` 8 9.. index:: 10 pair: HTTP; protocol 11 single: HTTP; http.client (standard module) 12 13.. index:: module: urllib.request 14 15-------------- 16 17This module defines classes which implement the client side of the HTTP and 18HTTPS protocols. It is normally not used directly --- the module 19:mod:`urllib.request` uses it to handle URLs that use HTTP and HTTPS. 20 21.. seealso:: 22 23 The `Requests package <http://docs.python-requests.org/>`_ 24 is recommended for a higher-level HTTP client interface. 25 26.. note:: 27 28 HTTPS support is only available if Python was compiled with SSL support 29 (through the :mod:`ssl` module). 30 31The module provides the following classes: 32 33 34.. class:: HTTPConnection(host, port=None[, timeout], source_address=None) 35 36 An :class:`HTTPConnection` instance represents one transaction with an HTTP 37 server. It should be instantiated passing it a host and optional port 38 number. If no port number is passed, the port is extracted from the host 39 string if it has the form ``host:port``, else the default HTTP port (80) is 40 used. If the optional *timeout* parameter is given, blocking 41 operations (like connection attempts) will timeout after that many seconds 42 (if it is not given, the global default timeout setting is used). 43 The optional *source_address* parameter may be a tuple of a (host, port) 44 to use as the source address the HTTP connection is made from. 45 46 For example, the following calls all create instances that connect to the server 47 at the same host and port:: 48 49 >>> h1 = http.client.HTTPConnection('www.python.org') 50 >>> h2 = http.client.HTTPConnection('www.python.org:80') 51 >>> h3 = http.client.HTTPConnection('www.python.org', 80) 52 >>> h4 = http.client.HTTPConnection('www.python.org', 80, timeout=10) 53 54 .. versionchanged:: 3.2 55 *source_address* was added. 56 57 .. versionchanged:: 3.4 58 The *strict* parameter was removed. HTTP 0.9-style "Simple Responses" are 59 not longer supported. 60 61 62.. class:: HTTPSConnection(host, port=None, key_file=None, \ 63 cert_file=None[, timeout], \ 64 source_address=None, *, context=None, \ 65 check_hostname=None) 66 67 A subclass of :class:`HTTPConnection` that uses SSL for communication with 68 secure servers. Default port is ``443``. If *context* is specified, it 69 must be a :class:`ssl.SSLContext` instance describing the various SSL 70 options. 71 72 Please read :ref:`ssl-security` for more information on best practices. 73 74 .. versionchanged:: 3.2 75 *source_address*, *context* and *check_hostname* were added. 76 77 .. versionchanged:: 3.2 78 This class now supports HTTPS virtual hosts if possible (that is, 79 if :data:`ssl.HAS_SNI` is true). 80 81 .. versionchanged:: 3.4 82 The *strict* parameter was removed. HTTP 0.9-style "Simple Responses" are 83 no longer supported. 84 85 .. versionchanged:: 3.4.3 86 This class now performs all the necessary certificate and hostname checks 87 by default. To revert to the previous, unverified, behavior 88 :func:`ssl._create_unverified_context` can be passed to the *context* 89 parameter. 90 91 .. deprecated:: 3.6 92 93 *key_file* and *cert_file* are deprecated in favor of *context*. 94 Please use :meth:`ssl.SSLContext.load_cert_chain` instead, or let 95 :func:`ssl.create_default_context` select the system's trusted CA 96 certificates for you. 97 98 The *check_hostname* parameter is also deprecated; the 99 :attr:`ssl.SSLContext.check_hostname` attribute of *context* should 100 be used instead. 101 102 103.. class:: HTTPResponse(sock, debuglevel=0, method=None, url=None) 104 105 Class whose instances are returned upon successful connection. Not 106 instantiated directly by user. 107 108 .. versionchanged:: 3.4 109 The *strict* parameter was removed. HTTP 0.9 style "Simple Responses" are 110 no longer supported. 111 112 113The following exceptions are raised as appropriate: 114 115 116.. exception:: HTTPException 117 118 The base class of the other exceptions in this module. It is a subclass of 119 :exc:`Exception`. 120 121 122.. exception:: NotConnected 123 124 A subclass of :exc:`HTTPException`. 125 126 127.. exception:: InvalidURL 128 129 A subclass of :exc:`HTTPException`, raised if a port is given and is either 130 non-numeric or empty. 131 132 133.. exception:: UnknownProtocol 134 135 A subclass of :exc:`HTTPException`. 136 137 138.. exception:: UnknownTransferEncoding 139 140 A subclass of :exc:`HTTPException`. 141 142 143.. exception:: UnimplementedFileMode 144 145 A subclass of :exc:`HTTPException`. 146 147 148.. exception:: IncompleteRead 149 150 A subclass of :exc:`HTTPException`. 151 152 153.. exception:: ImproperConnectionState 154 155 A subclass of :exc:`HTTPException`. 156 157 158.. exception:: CannotSendRequest 159 160 A subclass of :exc:`ImproperConnectionState`. 161 162 163.. exception:: CannotSendHeader 164 165 A subclass of :exc:`ImproperConnectionState`. 166 167 168.. exception:: ResponseNotReady 169 170 A subclass of :exc:`ImproperConnectionState`. 171 172 173.. exception:: BadStatusLine 174 175 A subclass of :exc:`HTTPException`. Raised if a server responds with a HTTP 176 status code that we don't understand. 177 178 179.. exception:: LineTooLong 180 181 A subclass of :exc:`HTTPException`. Raised if an excessively long line 182 is received in the HTTP protocol from the server. 183 184 185.. exception:: RemoteDisconnected 186 187 A subclass of :exc:`ConnectionResetError` and :exc:`BadStatusLine`. Raised 188 by :meth:`HTTPConnection.getresponse` when the attempt to read the response 189 results in no data read from the connection, indicating that the remote end 190 has closed the connection. 191 192 .. versionadded:: 3.5 193 Previously, :exc:`BadStatusLine`\ ``('')`` was raised. 194 195 196The constants defined in this module are: 197 198.. data:: HTTP_PORT 199 200 The default port for the HTTP protocol (always ``80``). 201 202.. data:: HTTPS_PORT 203 204 The default port for the HTTPS protocol (always ``443``). 205 206.. data:: responses 207 208 This dictionary maps the HTTP 1.1 status codes to the W3C names. 209 210 Example: ``http.client.responses[http.client.NOT_FOUND]`` is ``'Not Found'``. 211 212See :ref:`http-status-codes` for a list of HTTP status codes that are 213available in this module as constants. 214 215 216.. _httpconnection-objects: 217 218HTTPConnection Objects 219---------------------- 220 221:class:`HTTPConnection` instances have the following methods: 222 223 224.. method:: HTTPConnection.request(method, url, body=None, headers={}, *, \ 225 encode_chunked=False) 226 227 This will send a request to the server using the HTTP request 228 method *method* and the selector *url*. 229 230 If *body* is specified, the specified data is sent after the headers are 231 finished. It may be a :class:`str`, a :term:`bytes-like object`, an 232 open :term:`file object`, or an iterable of :class:`bytes`. If *body* 233 is a string, it is encoded as ISO-8859-1, the default for HTTP. If it 234 is a bytes-like object, the bytes are sent as is. If it is a :term:`file 235 object`, the contents of the file is sent; this file object should 236 support at least the ``read()`` method. If the file object is an 237 instance of :class:`io.TextIOBase`, the data returned by the ``read()`` 238 method will be encoded as ISO-8859-1, otherwise the data returned by 239 ``read()`` is sent as is. If *body* is an iterable, the elements of the 240 iterable are sent as is until the iterable is exhausted. 241 242 The *headers* argument should be a mapping of extra HTTP headers to send 243 with the request. 244 245 If *headers* contains neither Content-Length nor Transfer-Encoding, 246 but there is a request body, one of those 247 header fields will be added automatically. If 248 *body* is ``None``, the Content-Length header is set to ``0`` for 249 methods that expect a body (``PUT``, ``POST``, and ``PATCH``). If 250 *body* is a string or a bytes-like object that is not also a 251 :term:`file <file object>`, the Content-Length header is 252 set to its length. Any other type of *body* (files 253 and iterables in general) will be chunk-encoded, and the 254 Transfer-Encoding header will automatically be set instead of 255 Content-Length. 256 257 The *encode_chunked* argument is only relevant if Transfer-Encoding is 258 specified in *headers*. If *encode_chunked* is ``False``, the 259 HTTPConnection object assumes that all encoding is handled by the 260 calling code. If it is ``True``, the body will be chunk-encoded. 261 262 .. note:: 263 Chunked transfer encoding has been added to the HTTP protocol 264 version 1.1. Unless the HTTP server is known to handle HTTP 1.1, 265 the caller must either specify the Content-Length, or must pass a 266 :class:`str` or bytes-like object that is not also a file as the 267 body representation. 268 269 .. versionadded:: 3.2 270 *body* can now be an iterable. 271 272 .. versionchanged:: 3.6 273 If neither Content-Length nor Transfer-Encoding are set in 274 *headers*, file and iterable *body* objects are now chunk-encoded. 275 The *encode_chunked* argument was added. 276 No attempt is made to determine the Content-Length for file 277 objects. 278 279.. method:: HTTPConnection.getresponse() 280 281 Should be called after a request is sent to get the response from the server. 282 Returns an :class:`HTTPResponse` instance. 283 284 .. note:: 285 286 Note that you must have read the whole response before you can send a new 287 request to the server. 288 289 .. versionchanged:: 3.5 290 If a :exc:`ConnectionError` or subclass is raised, the 291 :class:`HTTPConnection` object will be ready to reconnect when 292 a new request is sent. 293 294 295.. method:: HTTPConnection.set_debuglevel(level) 296 297 Set the debugging level. The default debug level is ``0``, meaning no 298 debugging output is printed. Any value greater than ``0`` will cause all 299 currently defined debug output to be printed to stdout. The ``debuglevel`` 300 is passed to any new :class:`HTTPResponse` objects that are created. 301 302 .. versionadded:: 3.1 303 304 305.. method:: HTTPConnection.set_tunnel(host, port=None, headers=None) 306 307 Set the host and the port for HTTP Connect Tunnelling. This allows running 308 the connection through a proxy server. 309 310 The host and port arguments specify the endpoint of the tunneled connection 311 (i.e. the address included in the CONNECT request, *not* the address of the 312 proxy server). 313 314 The headers argument should be a mapping of extra HTTP headers to send with 315 the CONNECT request. 316 317 For example, to tunnel through a HTTPS proxy server running locally on port 318 8080, we would pass the address of the proxy to the :class:`HTTPSConnection` 319 constructor, and the address of the host that we eventually want to reach to 320 the :meth:`~HTTPConnection.set_tunnel` method:: 321 322 >>> import http.client 323 >>> conn = http.client.HTTPSConnection("localhost", 8080) 324 >>> conn.set_tunnel("www.python.org") 325 >>> conn.request("HEAD","/index.html") 326 327 .. versionadded:: 3.2 328 329 330.. method:: HTTPConnection.connect() 331 332 Connect to the server specified when the object was created. By default, 333 this is called automatically when making a request if the client does not 334 already have a connection. 335 336 337.. method:: HTTPConnection.close() 338 339 Close the connection to the server. 340 341As an alternative to using the :meth:`request` method described above, you can 342also send your request step by step, by using the four functions below. 343 344 345.. method:: HTTPConnection.putrequest(method, url, skip_host=False, \ 346 skip_accept_encoding=False) 347 348 This should be the first call after the connection to the server has been 349 made. It sends a line to the server consisting of the *method* string, 350 the *url* string, and the HTTP version (``HTTP/1.1``). To disable automatic 351 sending of ``Host:`` or ``Accept-Encoding:`` headers (for example to accept 352 additional content encodings), specify *skip_host* or *skip_accept_encoding* 353 with non-False values. 354 355 356.. method:: HTTPConnection.putheader(header, argument[, ...]) 357 358 Send an :rfc:`822`\ -style header to the server. It sends a line to the server 359 consisting of the header, a colon and a space, and the first argument. If more 360 arguments are given, continuation lines are sent, each consisting of a tab and 361 an argument. 362 363 364.. method:: HTTPConnection.endheaders(message_body=None, *, encode_chunked=False) 365 366 Send a blank line to the server, signalling the end of the headers. The 367 optional *message_body* argument can be used to pass a message body 368 associated with the request. 369 370 If *encode_chunked* is ``True``, the result of each iteration of 371 *message_body* will be chunk-encoded as specified in :rfc:`7230`, 372 Section 3.3.1. How the data is encoded is dependent on the type of 373 *message_body*. If *message_body* implements the :ref:`buffer interface 374 <bufferobjects>` the encoding will result in a single chunk. 375 If *message_body* is a :class:`collections.Iterable`, each iteration 376 of *message_body* will result in a chunk. If *message_body* is a 377 :term:`file object`, each call to ``.read()`` will result in a chunk. 378 The method automatically signals the end of the chunk-encoded data 379 immediately after *message_body*. 380 381 .. note:: Due to the chunked encoding specification, empty chunks 382 yielded by an iterator body will be ignored by the chunk-encoder. 383 This is to avoid premature termination of the read of the request by 384 the target server due to malformed encoding. 385 386 .. versionadded:: 3.6 387 Chunked encoding support. The *encode_chunked* parameter was 388 added. 389 390 391.. method:: HTTPConnection.send(data) 392 393 Send data to the server. This should be used directly only after the 394 :meth:`endheaders` method has been called and before :meth:`getresponse` is 395 called. 396 397 398.. _httpresponse-objects: 399 400HTTPResponse Objects 401-------------------- 402 403An :class:`HTTPResponse` instance wraps the HTTP response from the 404server. It provides access to the request headers and the entity 405body. The response is an iterable object and can be used in a with 406statement. 407 408.. versionchanged:: 3.5 409 The :class:`io.BufferedIOBase` interface is now implemented and 410 all of its reader operations are supported. 411 412 413.. method:: HTTPResponse.read([amt]) 414 415 Reads and returns the response body, or up to the next *amt* bytes. 416 417.. method:: HTTPResponse.readinto(b) 418 419 Reads up to the next len(b) bytes of the response body into the buffer *b*. 420 Returns the number of bytes read. 421 422 .. versionadded:: 3.3 423 424.. method:: HTTPResponse.getheader(name, default=None) 425 426 Return the value of the header *name*, or *default* if there is no header 427 matching *name*. If there is more than one header with the name *name*, 428 return all of the values joined by ', '. If 'default' is any iterable other 429 than a single string, its elements are similarly returned joined by commas. 430 431.. method:: HTTPResponse.getheaders() 432 433 Return a list of (header, value) tuples. 434 435.. method:: HTTPResponse.fileno() 436 437 Return the ``fileno`` of the underlying socket. 438 439.. attribute:: HTTPResponse.msg 440 441 A :class:`http.client.HTTPMessage` instance containing the response 442 headers. :class:`http.client.HTTPMessage` is a subclass of 443 :class:`email.message.Message`. 444 445.. attribute:: HTTPResponse.version 446 447 HTTP protocol version used by server. 10 for HTTP/1.0, 11 for HTTP/1.1. 448 449.. attribute:: HTTPResponse.status 450 451 Status code returned by server. 452 453.. attribute:: HTTPResponse.reason 454 455 Reason phrase returned by server. 456 457.. attribute:: HTTPResponse.debuglevel 458 459 A debugging hook. If :attr:`debuglevel` is greater than zero, messages 460 will be printed to stdout as the response is read and parsed. 461 462.. attribute:: HTTPResponse.closed 463 464 Is ``True`` if the stream is closed. 465 466Examples 467-------- 468 469Here is an example session that uses the ``GET`` method:: 470 471 >>> import http.client 472 >>> conn = http.client.HTTPSConnection("www.python.org") 473 >>> conn.request("GET", "/") 474 >>> r1 = conn.getresponse() 475 >>> print(r1.status, r1.reason) 476 200 OK 477 >>> data1 = r1.read() # This will return entire content. 478 >>> # The following example demonstrates reading data in chunks. 479 >>> conn.request("GET", "/") 480 >>> r1 = conn.getresponse() 481 >>> while not r1.closed: 482 ... print(r1.read(200)) # 200 bytes 483 b'<!doctype html>\n<!--[if"... 484 ... 485 >>> # Example of an invalid request 486 >>> conn.request("GET", "/parrot.spam") 487 >>> r2 = conn.getresponse() 488 >>> print(r2.status, r2.reason) 489 404 Not Found 490 >>> data2 = r2.read() 491 >>> conn.close() 492 493Here is an example session that uses the ``HEAD`` method. Note that the 494``HEAD`` method never returns any data. :: 495 496 >>> import http.client 497 >>> conn = http.client.HTTPSConnection("www.python.org") 498 >>> conn.request("HEAD", "/") 499 >>> res = conn.getresponse() 500 >>> print(res.status, res.reason) 501 200 OK 502 >>> data = res.read() 503 >>> print(len(data)) 504 0 505 >>> data == b'' 506 True 507 508Here is an example session that shows how to ``POST`` requests:: 509 510 >>> import http.client, urllib.parse 511 >>> params = urllib.parse.urlencode({'@number': 12524, '@type': 'issue', '@action': 'show'}) 512 >>> headers = {"Content-type": "application/x-www-form-urlencoded", 513 ... "Accept": "text/plain"} 514 >>> conn = http.client.HTTPConnection("bugs.python.org") 515 >>> conn.request("POST", "", params, headers) 516 >>> response = conn.getresponse() 517 >>> print(response.status, response.reason) 518 302 Found 519 >>> data = response.read() 520 >>> data 521 b'Redirecting to <a href="http://bugs.python.org/issue12524">http://bugs.python.org/issue12524</a>' 522 >>> conn.close() 523 524Client side ``HTTP PUT`` requests are very similar to ``POST`` requests. The 525difference lies only the server side where HTTP server will allow resources to 526be created via ``PUT`` request. It should be noted that custom HTTP methods 527+are also handled in :class:`urllib.request.Request` by sending the appropriate 528+method attribute.Here is an example session that shows how to do ``PUT`` 529request using http.client:: 530 531 >>> # This creates an HTTP message 532 >>> # with the content of BODY as the enclosed representation 533 >>> # for the resource http://localhost:8080/file 534 ... 535 >>> import http.client 536 >>> BODY = "***filecontents***" 537 >>> conn = http.client.HTTPConnection("localhost", 8080) 538 >>> conn.request("PUT", "/file", BODY) 539 >>> response = conn.getresponse() 540 >>> print(response.status, response.reason) 541 200, OK 542 543.. _httpmessage-objects: 544 545HTTPMessage Objects 546------------------- 547 548An :class:`http.client.HTTPMessage` instance holds the headers from an HTTP 549response. It is implemented using the :class:`email.message.Message` class. 550 551.. XXX Define the methods that clients can depend upon between versions. 552