Thursday 31 December 2020

Python - urllib3 cannot parse response if the server answers before the POST is complete

I'm using the requests (which uses urllib3) library to upload a file from a Python script. My backend starts by inspecting the headers of the request and if it doesn't comply with the needed prerequisites, it stops the request right away and respond with a valid 400 response.

This behavior works fine in Postman, or with Curl; i.e. the client is able to parse the 400 response even though it hasn't completed the upload and the server answers prematurely. However, while doing so in Python with requests/urllib3, the library is unable to process the backend response :

response : None
request : <PreparedRequest [POST]>
2020-12-29 11:53:25,316 - ERROR - ('Connection aborted.', ConnectionResetError(10054, 'Une connexion existante a dû être fermée par l’hôte distant', None, 10054, None))

Because the server answers before the transfer is complete, it considers that the connection has been aborted, even though the server DOES return a valid response.

Is there a way to avoid this and parse the response nonetheless ?

The code for sending the POST request is pretty standard (I tried without the MultipartEncoder, using vanilla requests, the result is the same) :

mpEncoder = MultipartEncoder(fields={'file': (open(args.filepath, 'rb'))})
headers = { "Authorization": "Bearer " + jwt_token }

try:
    r = requests.post(endpoint, headers=headers, data=mpEncoder, verify=False)
except requests.exceptions.RequestException as e:
    for property, value in vars(e).items():
        print(property, ":", value)
    fatal(str(e))


from Python - urllib3 cannot parse response if the server answers before the POST is complete

No comments:

Post a Comment