I previously used the code below to scrape the search result for a word search, for example book, on https://www.walmart.com/. They have currently changed their request and response parameters and this code does not get any response again.
params = {
'query': 'book',
'cat_id': 0,
'ps': 24,
'offset': 0,
'prg': 'desktop',
'stores': re.search(r'store/(\d+)', url).group(1)
}
try:
data1 = requests.get(api_url, params=params).json()
except Exception as e:
print("Sleeping for 10 seconds", e)
time.sleep(10)
try:
data1 = requests.get(api_url, params=params).json()
except Exception as e:
print("sleeping for 60 seconds", e)
time.sleep(60)
try:
data1 = requests.get(api_url, params=params).json()
except Exception as e:
print("sleeping for 360 seconds")
time.sleep(360)
data1 = requests.get(api_url, params=params).json()
I want to get the json response for a product page for example the product the in this url
https://www.walmart.com/ip/SKIPPY-Natural-Creamy-Peanut-Butter-Spread-15-oz/37447671
How could i rewrite the code with their current parameters to get the json response?
from how to scrape product data from website pages
No comments:
Post a Comment