Saturday, 19 December 2020

How not to hardcode the value of some correlation_id within headers to get required response?

I'm trying to grab different product names from this webpage. The product names, as in 0041-5053-005 generate dynamically. I can however scrape them using xhr with appropriate parameters.

It is necessary to use this following key and value within the headers to get the required data.

headers = {
    'client_secret': '',
    'client_id': '',
    'correlation_id': '0196e1f2-fb29-0modod-6125-fcbb6c2c69c1',
}

This is how I scraped the titles:

import requests

link = "https://es-be-ux-search.cloudhub.io/api/ux/v2/search?"

payload = {
    'queryText': '*:*',
    'role': 'rockwell-search',
    'spellingCorrect': 'true',
    'spellcheckPremium': '10',
    'segments': 'Productsv4',
    'startIndex': 0,
    'numResults': 10,
    'facets': '',
    'languages': 'en',
    'locales': 'en_GLOBAL,en-US',
    'sort': 'cat_a',
    'collections': 'Literature,Web,Sample_Code',
    'site': 'RA'
}

with requests.Session() as s:
    r = s.get(link,params=payload,headers=headers)
    for item in r.json()['response']['docs']:
        print(item['catalogNumber'])

I've noticed that the value of client_secret and client_id are static but the value of correlation_id changes.

How can I use the value of correlation_id within the headers without hardcoding?



from How not to hardcode the value of some correlation_id within headers to get required response?

No comments:

Post a Comment