无法使用 Selenium 检索类值

Question

构建一个基本的网络爬虫，我无法检索包含产品价格的类值。谷歌搜索中列出的所有价格的类名称都是qptdjc。

这是价格的 HTML 标签 <div class="qptdjc">$179.99</div>

import selenium

from selenium import webdriver as wb

import pandas as pd

import time

browser = wb.Chrome(executable_path='C:/Users/ethan/Downloads/chromedriver_win32(1)/chromedriver')

browser.get('https://www.google.com/search?source=hp&ei=nnZ6X8SDO4WE9PwPj5KC4AQ&q=144hz+monitor&oq=144hz+monitor&gs_lcp=CgZwc3ktYWIQAzIFCAAQsQMyBQgAELEDMgIIADICCAAyAggAMgIIADICCAAyAggAMgIIADICCAA6DggAEOoCELQCEJoBEOUCOgsILhDHARCvARCTAjoFCC4QsQM6CAguEMcBEK8BOgsILhCxAxDHARCjAjoICAAQsQMQgwE6CAguELEDEIMBOg4ILhCxAxCDARDHARCvAVDyDligHmDsH2gBcAB4AIABiwGIAZMHkgEEMTIuMZgBAKABAaoBB2d3cy13aXqwAQY&sclient=psy-ab&ved=0ahUKEwjEyoykppzsAhUFAp0JHQ-JAEwQ4dUDCAk&uact=5')

productInfoList = browser.find_elements_by_class_name('qptdjc')

prices = browser.find_elements_by_xpath('//td[@class="qptdjc"]')

prices_list = []

for p in range(len(prices)):

    prices_list.append(prices[p].text)

print(len(productInfoList))

print(*prices_list, sep = ", ")

print(*prices, sep = ", ")

Answer 1

所以我等待所有的元素出现。抓取所有内容，循环并获取其内部 HTML。productInfoList=WebDriverWait(driver, 20).until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, "div.r4awE > span")))prices=WebDriverWait(driver, 20).until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, "div.qptdjc")))prices_list = []for price in prices:&nbsp; &nbsp; prices_list.append(price.get_attribute('innerHTML').split('<')[0].strip())pprint(prices_list)print(len(productInfoList))输出['$229.99',&nbsp;'$187.52',&nbsp;'$249.99']3进口from selenium.webdriver.common.by import Byfrom selenium.webdriver.support.ui import WebDriverWait&nbsp;from selenium.webdriver.support import expected_conditions as ECfrom pprint import pprint