Wednesday, 18 October 2023

Unable to scrape two types of names from a website using the requests module

I've created a script in Python to scrape two types of names from a webpage using the requests module. I wish to grab the names under the GRANTOR and GRANTEE columns.

import requests
from bs4 import BeautifulSoup

link = 'https://dallas.tx.publicsearch.us/results?_docTypes=AMD%2CDT%2CMOD%2CMODT%2CDE%2CCORR%20DT&department=RP&limit=50&offset=0&recordedDateRange=20181101%2C20231010&searchOcrText=true&searchType=quickSearch&searchValue=%22adjustable%20rate%20rider%22'

headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36',
}

res = requests.get(link,headers=headers)
soup = BeautifulSoup(res.text,"lxml")
for item in soup.select("[data-tourid='searchResults'] table > tbody > tr"):
    grantor = item.select("td")[3].text
    grantee = item.select("td")[4].text
    print(grantor,grantee)

When I run the script, it yields nothing.



from Unable to scrape two types of names from a website using the requests module

No comments:

Post a Comment