I've created a script using requests module to scrape different addresses revealed upon clicking on a box like container on a map located in a webpage. To get the results, it is necessary to issue a get requests along with appropriate parameters.
In the parameters, there are three things that I wish not to hardcode. The value of x
,y
and wkid
within geometry
. Currrently they are hardcoded.
I've tried with (the script runs if you copy the updated token from that site):
import requests
url = 'https://www.bcassessment.ca//Property/Info/QTAwMDAwMDAyRg=='
link = 'https://arcgis.bcassessment.ca/ext_wa/rest/services/DCX/AFP_20210325/MapServer/0/query'
params = {
'token': 'NHkz-ovnHMz14v7omW68nK6cYpfv3NYcbxelXdxg6tlVj7sGSpGdR44TILwVDIqMK2hWKPv5keKrbs25dIYJGQ..',
'f': 'json',
'where': '',
'returnGeometry': 'true',
'geometry': "{'x':'-13715482.217555167','y':'6320767.279841547','spatialReference':{'wkid':102100}}",
'geometryType': 'esriGeometryPoint',
'inSR': '102100',
'outFields': 'AFP_OID,UNIT_NUMBER,TOTAL_ASSESSED,ROLL,AREA_EVBC,JUR,TOTAL_LAND,TOTAL_BUILDING,ADDRESS,DESCRIPTION,STREET_NUMBER,STREET_NAME,OID_EVBC,SHORT_ADDRESS,IS_STRATA,FARM_FLAG,UTILITY_FLAG,MAJ_INDUSTRY_FLAG,MANAGED_FOREST_FLAG',
'orderByFields': 'STREET_NAME,STREET_NUMBER,UNIT_NUMBER,SHORT_ADDRESS',
'outSR': '102100'
}
with requests.Session() as s:
s.headers['User-Agent'] = 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.150 Safari/537.36'
s.headers['Referer'] = 'https://www.bcassessment.ca/'
res = s.get(link,params=params)
for item in res.json()['features']:
print(item['attributes']['ADDRESS'])
Output:
1-1980 SASAMAT ST VANCOUVER
2-1980 SASAMAT ST VANCOUVER
3-1980 SASAMAT ST VANCOUVER
4-1980 SASAMAT ST VANCOUVER
5-1980 SASAMAT ST VANCOUVER
6-1980 SASAMAT ST VANCOUVER
7-1980 SASAMAT ST VANCOUVER
How can I not hardcode the value of x
,y
and wkid
within params
and still get the same results?
from Can't find the value of three keys required to be used within parameters in requests
No comments:
Post a Comment