Friday, 11 October 2019

Simple get/post request blocked in python 3 but not in python 2

I'm working on a simple web scraper in python 3 but when I send a get or a post request, the response is 403. In python 2 works fine though. I'm using the same version of requests libraries in both versions. I have also tried with Verify=False/True but the difference in both versions remains.

requests = 2.22.0

certifi = 2019.9.11

from requests import get
url = 'https://www.gamestop.com/'
header = {
    'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
    'Accept-Encoding': 'gzip, deflate, br',
    'Accept-Language': 'en-US,en;q=0.5',
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64; rv:56.0) Gecko/20100101 Firefox/56.0',
    'DNT': '1',
    'Upgrade-Insecure-Requests': '1',
    'Connection': 'keep-alive',
    'Host': 'www.gamestop.com'
}
res = get(url, headers=header, verify=False).status_code
print(res)
# 403 when using python 3
# 200 when using python 2

Edit by @blhsing:

The list below keeps track of which specific Python versions work and which versions fail according to the comments. So far successes and failures have been consistent for each specific Python version across platforms.

Feel free to edit this section of the question with your own results along with the specific Python versions used to produce the results.

2.7.14 works (blhsing)
2.7.16 works (repl.it)
3.6.5 works (blhsing)
3.6.8 fails (Reinderien and blhsing)
3.7.3 works (wim and blhsing)
3.7.4 fails (repl.it and blhsing)

Demo on repl.it: Python 2.7.16 and Python 3.7.4



from Simple get/post request blocked in python 3 but not in python 2

No comments:

Post a Comment