I am trying to scrape this website https://segari.id/c/unggas . I tried using requests
url="https://api-v2.segari.id/v1.1/products/price?agentId=311&tagIds=838&size=999&page=0&paginationType=slice&totalPriceBefore=0&deliveryDate=2023-10-26&deliveryServiceType=NEXT_DAY_DELIVERY&pageOrigin=clp"
headers={
'Referer':'https://segari.id/',
'Sec-Ch-Ua':'"Chromium";v="118", "Google Chrome";v="118", "Not=A?Brand";v="99"',
'Sec-Ch-Ua-Mobile':'?0',
'Sec-Ch-Ua-Platform': '"Windows"',
'Sec-Fetch-Dest':'empty',
'Sec-Fetch-Mode':'cors',
'Sec-Fetch-Site':'same-site',
'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36',
'X-Segari-Pathname':'/c/unggas',
'X-Segari-Platform':'web',
'X-Segari-Sec': 'v1:MjAyMy0xMC0yNVQxMzoxODozNS40OTQwMDArMDc6MDA=:96a10a8bbb9f47b8cfb1db7bd3ad483632ede18b0631de80e9720e30d4928f07'
}
response=requests.get(url,headers=headers)
response returns <Response [400]>
response.content returns message":"04 - Signature Mismatch","code":"TODO","data":null,"errors":null
I tried opening the api url, and returns error. I checked that X-segari-sec is the only thing missing in the header. So I deduced that's the only missing thing. I also noticed a request to 'https://ift.tt/EFbgKkC'. I tried the X-segari-sec by emulating the javascript. It's the same.
here's the python ver for it
def get_X_segar_sec(hash_key, timestamp):
hash_key = "wCL8iD2xgQ5EdLrVBEYvAP3nRxX3q6TSNjIlISOx7PTphdhA78M2cjlN5RISAHwn"
base64_encoded_timestamp = base64.b64encode(timestamp.encode()).decode()
hash_string = f"{hash_key}{base64_encoded_timestamp}"
hash_value = hashlib.sha256(hash_string.encode()).hexdigest()
return f"v1:{base64_encoded_timestamp}:{hash_value}"
The response is 400 and returns a mismatch text from content. Can someone help me to get 200? Why is this happening? I tried using selenium too and no data is being loaded after waiting.
Thank you
from Why the response is 400 when using the same headers with python requests?
No comments:
Post a Comment