I'm trying to grab different product names from this webpage. The product names, as in 0041-5053-005 generate dynamically. I can however scrape them using xhr with appropriate parameters.
It is necessary to use this following key and value within the headers to get the required data.
headers = {
'client_secret': '',
'client_id': '',
'correlation_id': '0196e1f2-fb29-0modod-6125-fcbb6c2c69c1',
}
This is how I scraped the titles:
import requests
link = "https://es-be-ux-search.cloudhub.io/api/ux/v2/search?"
payload = {
'queryText': '*:*',
'role': 'rockwell-search',
'spellingCorrect': 'true',
'spellcheckPremium': '10',
'segments': 'Productsv4',
'startIndex': 0,
'numResults': 10,
'facets': '',
'languages': 'en',
'locales': 'en_GLOBAL,en-US',
'sort': 'cat_a',
'collections': 'Literature,Web,Sample_Code',
'site': 'RA'
}
with requests.Session() as s:
r = s.get(link,params=payload,headers=headers)
for item in r.json()['response']['docs']:
print(item['catalogNumber'])
I've noticed that the value of client_secret and client_id are static but the value of correlation_id changes.
How can I use the value of
correlation_idwithin the headers without hardcoding?
from How not to hardcode the value of some correlation_id within headers to get required response?
No comments:
Post a Comment