I'm trying to get the links of different companies from a webpage but the script I've tried with throws the error below. In chrome dev tools I could see that I can get the ids
of different companies using post
http requests. However, if I can get the ids
then I will be able to make use of this link 'https://angel.co/startups/{}'
adding id's in string format to make a full-fledged company link.
I've tried with:
import requests
link = 'https://angel.co/company_filters/search_data'
base = 'https://angel.co/startups/{}'
payload={'sort':'signal','page':'2'}
r = requests.post(link,data=payload,headers={
'x-requested-with':'XMLHttpRequest'
'User-Agent":"Mozilla/5.0'
})
print(r.json())
The above script throws the following error:
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
How can I get the links of different companies from the aforementioned site using requests?
from Can't scrape the links of different companies from a website using requests
No comments:
Post a Comment