Friday, 8 April 2022

Scraping an asp.net website using python requests

I am using requests module to send requests to the site:

https://auditor.ashtabulacounty.us/ .

I have tried my best to get the desired output but I always fail. I am making requests right but not getting data I dont know about this as I have never encounterd it before I am sending request to get the view_state, view_state_gen and event_validation. After that I send a get request to get session and then send the post request to the first url to validate the data and get the valid view_state value. I dont know but It changes and the new one is the required which I can not get. I get different Value like previous one which needs to be validated. After that I send request to get data. There are two different requests based on condition based on query length. I dont get this data due to the unvalid view_state. The error I get on this request is

Invalid postback or callback argument.  Event validation is enabled using <pages enableEventValidation="true"/>
in configuration or <%@ Page EnableEventValidation="true" %> in a page.For security purposes,
this feature verifies that arguments to postback or callback events originate from the server control that originally rendered them.
If the data is valid and expected, use the ClientScriptManager.RegisterForEventValidation method in order to register the postback or callback data for validation.

Note:

you can not access site if you are outside the America. So I have added the code to github to public repository and hosted it using fastapi on heroku so every one can access it using simple request The request to heroku server is

import requests

r = requests.get('https://fast-ap.herokuapp.com/')
print(r.status_code)
j = r.json()

Repository link: https://github.com/faniAhmed/Upload/

Repository is public and you can make changes to it

You can clone the repo and make host it on your side too if you want but I have enabled autodeploy so any commit will update the server I will add code here too if required

The required output if input{query} is less than 12 digits

enter image description here

The required output if the digits are equal to 12

enter image description here

Please someone help me. I have tried alot but I cannot do it. Any help from experienced develpors will be great. Thanks in advance.

I will add bounty 50 minimum and max will be according to votes I get on this post



from Scraping an asp.net website using python requests

No comments:

Post a Comment