Sunday, 4 July 2021

Can't find the value of three keys required to be used within parameters in requests

I've created a script using requests module to scrape different addresses revealed upon clicking on a box like container on a map located in a webpage. To get the results, it is necessary to issue a get requests along with appropriate parameters.

In the parameters, there are three things that I wish not to hardcode. The value of x,y and wkid within geometry. Currrently they are hardcoded.

I've tried with (the script runs if you copy the updated token from that site):

import requests

url = 'https://www.bcassessment.ca//Property/Info/QTAwMDAwMDAyRg=='
link = 'https://arcgis.bcassessment.ca/ext_wa/rest/services/DCX/AFP_20210325/MapServer/0/query'

params = {
    'token': 'NHkz-ovnHMz14v7omW68nK6cYpfv3NYcbxelXdxg6tlVj7sGSpGdR44TILwVDIqMK2hWKPv5keKrbs25dIYJGQ..',
    'f': 'json',
    'where': '',
    'returnGeometry': 'true',
    'geometry': "{'x':'-13715482.217555167','y':'6320767.279841547','spatialReference':{'wkid':102100}}",
    'geometryType': 'esriGeometryPoint',
    'inSR': '102100',
    'outFields': 'AFP_OID,UNIT_NUMBER,TOTAL_ASSESSED,ROLL,AREA_EVBC,JUR,TOTAL_LAND,TOTAL_BUILDING,ADDRESS,DESCRIPTION,STREET_NUMBER,STREET_NAME,OID_EVBC,SHORT_ADDRESS,IS_STRATA,FARM_FLAG,UTILITY_FLAG,MAJ_INDUSTRY_FLAG,MANAGED_FOREST_FLAG',
    'orderByFields': 'STREET_NAME,STREET_NUMBER,UNIT_NUMBER,SHORT_ADDRESS',
    'outSR': '102100'
}

with requests.Session() as s:
    s.headers['User-Agent'] = 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.150 Safari/537.36'
    s.headers['Referer'] = 'https://www.bcassessment.ca/'
    res = s.get(link,params=params)
    for item in res.json()['features']:
        print(item['attributes']['ADDRESS'])

Output:

1-1980 SASAMAT ST VANCOUVER
2-1980 SASAMAT ST VANCOUVER
3-1980 SASAMAT ST VANCOUVER
4-1980 SASAMAT ST VANCOUVER
5-1980 SASAMAT ST VANCOUVER
6-1980 SASAMAT ST VANCOUVER
7-1980 SASAMAT ST VANCOUVER

How can I not hardcode the value of x,y and wkid within params and still get the same results?



from Can't find the value of three keys required to be used within parameters in requests

No comments:

Post a Comment