I am trying to scrap the table with column office, cadre, designation, name, and asset_details:
Here P2 ranges from 1 to 38.
In the Network > Payload > Form Data has the page number (__EVENTARGUMENT: Page$x), which can't be input and has to be ascertained.
http://bpsm.bihar.gov.in/Assets2020/AssetDetails.aspx?P1=2&P2=7&P3=0&P4=0
Here is my attempt at code:
import json
import requests
import pandas as pd
api_url = (
"http://bpsm.bihar.gov.in/Assets2019/AssetDetails.aspx?P1=2&P2=33&P3=0&P4=0"
)
payload = {"P1": "2", "P2": "33", "P3": "0", "P4": "0"}
all_data = []
for P2 in range(1, 39): # <-- increase from 1 to 200
print(P2)
payload['P2'] = P2
data = requests.post(api_url, json=payload).json()
data = json.loads(data['d'])
if not data:
break
for name, count in data[0].items():
all_data.append({
})
from Web-scrapping ASPX page where loop over page number from Network>Payload
No comments:
Post a Comment