我在通过 selenium python 抓取内容时遇到错误

Question

我正在通过 selenium 在https://www.indeed.ae/jobs-in-dubai上抓取工作结果的标题。我认为 .text 不起作用。我正在通过转到主网站的 selenium 运行代码，输入选择性关键字，然后从结果中删除所有标题。但是我收到错误，我该如何解决这个错误

这是我的代码

import time

from selenium import webdriver

from selenium.webdriver.common.by import By

from selenium.webdriver.support.ui import WebDriverWait

from selenium.webdriver.support import expected_conditions as EC

from selenium.webdriver.common.keys import Keys

Path = "C:\Program Files (x86)\chromedriver.exe"

driver = webdriver.Chrome(Path)

driver.get("https://indeed.ae/")

print(driver.title)

search = driver.find_element_by_name("l")

search.send_keys("Dubai")

search.send_keys(Keys.RETURN)

try:

    td = WebDriverWait(driver, 10).until(

        EC.presence_of_element_located((By.ID, "resultsCol"))

    )

    divs = td.find_elements_by_tag_name("div")

    for div in divs:

        header = div.find_element_by_class_name("title")

        print(header)

finally:

    driver.quit()

driver.quit()

我收到以下错误

Job Search | Indeed

Traceback (most recent call last):

  File "C:/Users/hp/Desktop/python projects/selenium-pycharm/selenium-bot.py", line 24, in <module>

    header = div.find_element_by_class_name("title")

  File "C:\Users\hp\Desktop\python projects\selenium-pycharm\venv\lib\site-packages\selenium\webdriver\remote\webelement.py", line 398, in find_element_by_class_name

    return self.find_element(by=By.CLASS_NAME, value=name)

  File "C:\Users\hp\Desktop\python projects\selenium-pycharm\venv\lib\site-packages\selenium\webdriver\remote\webelement.py", line 659, in find_element

    {"using": by, "value": value})['value']

  File "C:\Users\hp\Desktop\python projects\selenium-pycharm\venv\lib\site-packages\selenium\webdriver\remote\webelement.py", line 633, in _execute

    return self._parent.execute(command, params)

Process finished with exit code 1

提前致谢

Answer 1

您找不到标题，因为您从 resultsCol 中获取了所有的 div。这意味着有些 div 有标题，有些则没有。尝试这个 ：try:&nbsp; &nbsp; td = WebDriverWait(driver, 10).until(&nbsp; &nbsp; &nbsp; &nbsp; EC.presence_of_element_located((By.ID, "resultsCol"))&nbsp; &nbsp; )&nbsp; &nbsp; divs = td.find_elements_by_tag_name("div")&nbsp; &nbsp; #print(divs)&nbsp; &nbsp; for div in divs:&nbsp; &nbsp; &nbsp; &nbsp; try:&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; header = div.find_element_by_class_name("title")&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; print(header.text)&nbsp; &nbsp; &nbsp; &nbsp; except:&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; continuefinally:&nbsp; &nbsp; driver.quit()driver.quit()将标题作为输出：ReceptionistAdministrative Assistant/ Document ControllerRECEPTIONISTADMIN OFFICER IN UAEData Entry Assistant (Fresh Graduate)ReceptionistReplenishment Associate - Light Household - HypermarketDOCUMENT CONTROLLERSchool Administrative Assistant - DubaiACCOUNTANT