Problem description: I am having trouble figuring out what is missing from my HTTP request to get to the next page after this form: When I do this from my code, I land on a "welcome" page instead of landing on the expected one from Step 2 below:
-
Step 1: Filling it out with the following and clicking "Search" in a browser:
-
Step 2: I get to this next page:
I have looked up all form data being posted and did a simplified pre-filled form here, but clicking on Search in this case does not work:
What I have tried
- I have read this thread which advises against simplifying the form, but I believe I am doing this correctly: all the fields I am aware of are present, I simply change their HTML visualization (which has no bearing on what is POSTed).
- The jspSynchronizerToken field could be checked on the server side, therefore I already tried filling it out manually with a valid value retrieved in the same browser. This alone does not solve the problem.
- I have tried using the following cURL command, but this does not work either:
curl -A "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:87.0) Gecko/20100101 Firefox/87.0" \
-F 'pageName=com/sed/jsp/reports/ItdReportInput.jsp' \
-F 'menukey=15.03.00' \
-F 'locale=en_CA' \
-F 'jspSynchronizerToken=BzDQe5n2dIhV4LHKyqV_u341627003039920' \
-F 'PUBLIC_USERS=0' \
-F 'FORMAT=TXT' \
-F 'SORTED_BY=0' \
-F 'SELECT_TYPE=8' \
-F 'SELECT_TYPE_VALUE=Test' \
-F 'SELECT_TYPE_VALUE_SEARCH_TYPE=0' \
-F 'DATE_RANGE_TYPE=0' \
-F 'MONTH_FROM_PUBLIC=0' \
-F 'DAY_FROM_PUBLIC=1' \
-F 'YEAR_FROM_PUBLIC=2020' \
-F 'MONTH_TO_PUBLIC=11' \
-F 'DAY_TO_PUBLIC=31' \
-F 'YEAR_TO_PUBLIC=2020' \
-F 'Search=Search' \
https://www.sed.com/sed/SVTItdController?locale=en_CA \
> ~/Downloads/sed.html
- I have also coded a more in-depth python version in which I provide the referer as well as the retrieved token from the Step 1, but with no success. (This code is available but it is simply a less readable version of the above simplified code so I don't see the point in posting, let me know if I am wrong).
- I have considered cookies and javascript might be the problem, yet a browser with no javascript or cookies allowed correctly submits the form and lands on the correct result.
- I am currently looking at the requests / responses using Charles Proxy, but I am not getting any new insights so far...
I realize this question is a bit specific to the website I am showing, nevertheless it seems to me that there is a generic understanding of how websites process HTTP requests and how to validly mimic those which could be gained by a proper answer here. I have had other experience akin to this one in the past and am hoping to understand this properly once and for all :)
from Proper HTTP POST request to mimic submitting an online form (using HTML, cURL or python)
No comments:
Post a Comment