使用美汤进行网页抓取（体育数据）

Question

当我尝试加载此代码时，出现两个错误。1：第一个是我无法正确抓取name_text的数据。

2：我收到 team = name_text.div.text 的缩进错误。我知道这可能很容易解决，但我尝试了不同的缩进，但似乎没有任何效果。

在网站上，我想抓取球队名称和赔率。

<div class="size14_f7opyze Endeavour_fhudrb0 medium_f1wf24vo participantText_fivg86r" data-automation-id="participant-one">Orlando Magic</div>

<div class="priceText_f71sibe"><span class="size14_f7opyze medium_f1wf24vo priceTextSize_frw9zm9" data-automation-id="price-text">5.85</span></div>

上面的html是从网站上复制的。

from bs4 import BeautifulSoup

from urllib.request import urlopen as uReq

my_url = 'https://www.sportsbet.com.au/betting/basketball-us'

uClient = uReq(my_url)

page_html = uClient.read()

uClient.close()

soup = BeautifulSoup(page_html, "html.parser")

price_text = soup.findAll("div",{"class":"priceText_f71sibe"})

name_text = soup.findAll("div",{"class":"size14_f7opyze Endeavour_fhudrb0 medium_f1wf24vo participantText_fivg86r"})

filename = "odds.csv"

f = open(filename,"w")

headers = "Team, odds_team\n"

print(name_text)

f.write(headers)

for price_text in price_texts:

team = name_text.div.text

odds = price_text.span.text

print(odds)

print(team + odds)

f.write(team + "," + odds + "\n")

f.close()

任何帮助都会很棒。干杯。

Answer 1

您的for loop缩进不正确。正确的缩进是：for price_text in price_texts:&nbsp; &nbsp; team = name_text.div.text&nbsp; &nbsp; odds = price_text.span.text&nbsp; &nbsp; team = name_text.div.text&nbsp; &nbsp; odds = price_text.span.text&nbsp; &nbsp; print(odds)&nbsp; &nbsp; print(team + odds)&nbsp; &nbsp; f.write(team + "," + odds + "\n")f.close()队前有 4 个空格和赔率。请阅读Python ForLoop 文档。此外，没有price_texts变量。当你做findAll时你需要分配它，你忘记了一个'S'：price_texts = soup.findAll("div",{"class":"priceText_f71sibe"})最后一件事，考虑使用with而不是open()和.close()写入您的文件。

Answer 2

你能试试这个吗？from bs4 import BeautifulSoupfrom urllib.request import urlopen as uReqmy_url = 'https://www.sportsbet.com.au/betting/basketball-us'uClient = uReq(my_url)page_html = uClient.read()uClient.close()soup = BeautifulSoup(page_html, "html.parser")price_texts = soup.findAll("div",{"class":"priceText_f71sibe"})name_texts = soup.findAll("div",{"class":"size14_f7opyze Endeavour_fhudrb0 medium_f1wf24voparticipantText_fivg86r"})filename = "odds.csv"f = open(filename,"w")headers = "Team, odds_team\n"print(name_text)f.write(headers)odds =''team=''for price_text in price_texts:&nbsp; &nbsp; odds = price_text.textfor name_text in name_texts:&nbsp; &nbsp; team = name_text.textprint(odds)print(team + odds)f.write(team + "," + odds + "\n")f.close()

Answer 3

我在想你可以做的只是迭代并将它们存储到列表中，然后写入文件。不幸的是，我无法在工作中访问该站点，因此无法测试代码，但我相信这应该会提供您正在寻找的输出：from bs4 import BeautifulSoupfrom urllib.request import urlopen as uReqimport csvfrom itertools import zip_longestmy_url = 'https://www.sportsbet.com.au/betting/basketball-us'uClient = uReq(my_url)page_html = uClient.read()uClient.close()soup = BeautifulSoup(page_html, "html.parser")price_text = soup.findAll("span",{"data-automation-id":"price-text"})name_text = soup.findAll("div",{"data-automation-id":"participant-one"})team_list = [ name.text.strip() for name in name_text ]odds_list = [ price.text.strip() for price in price_text ]d = [team_list, odds_list]export_data = zip_longest(*d, fillvalue = '')with open('odds.csv', 'w', encoding="ISO-8859-1", newline='') as myfile:&nbsp; &nbsp; &nbsp; wr = csv.writer(myfile)&nbsp; &nbsp; &nbsp; wr.writerow(("Team", "odds_team"))&nbsp; &nbsp; &nbsp; wr.writerows(export_data)myfile.close()