如何使用 Beautifulsoup 从网站上抓取产品价格？

Question

我正在尝试从 StockX 中提取这款运动鞋的最新出价，但收到错误消息

IndexError: list index out of range

因为由于sneaker_price某种原因出现空白。有人可以帮忙吗？：

from bs4 import BeautifulSoup

from selenium import webdriver

driver = webdriver.Chrome()

driver.get("https://stockx.com/air-jordan-6-retro-travis-scott")

soup = BeautifulSoup(driver.page_source,"lxml")

driver.quit()

sneaker_price = soup.select("div.en-us stat-value stat-small")[0]

Answer 1

尝试 CSS 选择器div.en-us.stat-value.stat-small：sneaker_price = soup.select("div.en-us.stat-value.stat-small")[0]print(sneaker_price.text)印刷：€523注意：如果您获得验证码页面，请尝试指定更多 HTTP 标头和/或 Cookie。例如：import requestsfrom bs4 import BeautifulSoupheaders = {    'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:80.0) Gecko/20100101 Firefox/80.0',    'Accept-Language': 'en-US,en;q=0.5'}cookies = {    'stockx_homepage': "sneakers",}soup = BeautifulSoup(requests.get("https://stockx.com/air-jordan-6-retro-travis-scott", headers=headers, cookies=cookies).content,"lxml")sneaker_price = soup.select("div.en-us.stat-value.stat-small")[0]print(sneaker_price.text)

Answer 2

您可以使用.ask div.en-us.stat-value.stat-small选择器获取包含最新要价的 div。由于有多个元素，您可以选择最后一个元素，例如：ask_price = soup.select('.ask div.en-us.stat-value.stat-small')[-1]print(ask_price.text) # $655