使用 python 进行网页抓取时 HTML 类的问题
我是 python 的新手，我正在尝试从购物网站上抓取有关计算机的信息。尽管代码看起来没问题（没有错误），但在过程的最后它不会给我 csv 文件。我已经更改了课程，但没有任何效果。我在另一个网站上使用了这段代码，效果很好。另一个小问题：如何抓取多页结果。提前致谢！
from bs4 import BeautifulSoup
import pandas as pd
from selenium import webdriver
import pandas as pd
products=[] #List to store name of the product
prices=[] #List to store price of the product
fretes=[] #List to store rating of the product
driver = webdriver.Chrome(executable_path = r'C:\Users\gomes\Desktop\chromedriver.exe')
driver.get("https://lista.mercadolivre.com.br/macbook")
content = driver.page_source
soup = BeautifulSoup(content, 'html.parser')
for a in soup.findAll('li',href=True, attrs={'class':'results-item'}):
name=a.find('span', attrs={'class':'main-title'})
price=a.find('span', attrs={'class':'price__fraction'})
frete=a.find('span', attrs={'class':'text-shipping'})
products.append(name.text)
prices.append(price.text)
fretes.append(frete.text)
data = dict({'Product Name': products,
'Price': prices,
'Frete': fretes
})
# create dataframe
products_df = pd.DataFrame(
dict([(k, pd.Series(v)) for k, v in data.items()])
)
products_df.to_csv("C:\\Users\\gomes\\Desktop\\mac.csv")
开心每一天1111浏览 124回答 1
1回答
-
偶然的你
import requestsimport pandas as pdfrom bs4 import BeautifulSoupurl = 'https://lista.mercadolivre.com.br/macbook'soup = BeautifulSoup(requests.get(url).content, 'html.parser')data = []for title, price in zip(soup.select('.item__title'), soup.select('.price__fraction')): frete = title.find_previous_sibling(attrs={'class':'item__shipping'}) data.append({'Product Name': title.get_text(strip=True), 'Price': price.get_text(strip=True), 'Frete': frete.get_text(strip=True) if frete else '-'})df = pd.DataFrame(data)print(df)df.to_csv('data.csv')印刷： Product Name Price Frete0 Macbook Pro 15 Core I7 8gb De Ram 2011 - Leia ... 2.699 -1 Macbook Pro I5 2.7ghz/8gb 1867mhz/ssd 256gb Mo... 5.100 Frete grátis2 Apple Macbook Pro Retina 13,3 I5 2.3/8gb/256gb... 9.830 Frete grátis3 Macbook Pro 2015 A1502 Retina I5 8gb 256ssd + ... 4.799 -4 Macbook A1502 Pro 2015 Retina I5 8gb 256ssd + ... 4.800 -5 Apple Macbook A1534 2017 Mnyk2bza Core M3 8gb ... 8.499 Frete grátis6 Macbook Pro (ver Detalhes Abaixo No Descritivo... 4.460 -7 Macbook Pro 2015 I5 8gb 2,7ghz 256ssd Impecáve... 5.993 Frete grátis8 Macbook Pro I5 2.7ghz/8gb 1867mhz/ssd 256gb Mo... 5.100 Frete grátis9 Macbook Pro 15 Mid 2009 2,66 Ghz Intel Core Du... 2.749 Frete grátis10 Macbook Pro Touch Bar 2016 I5 8gb 256ssd - Sem... 8.999 Frete grátis11 Macbook Pro 2017 - Retina - 2,3 Ghz - I5 - 8gb... 9.999 Frete grátis12 Macbook Pro 2019 16pol 2.4 I9 32gb 1tb 5500m E... 30.699 Frete grátis13 Apple Macbook Pro A1278 Core I5 8gb Hd Ssd 240... 4.480 Frete grátis14 Macbook Pro Md318ll/a Intel Core I7 2.2ghz 8gb... 4.490 -15 Macbook Air 13 128gb (versao Anterior) 7.499 Frete grátis16 Macbook Pro 2019 16pol 2.4 I9 64gb 1tb 5500m E... 33.299 Frete grátis17 Macbook Pro 2015 Retina I5 8gb 256ssd + Carreg... 5.494 Frete grátis18 Macbook Pro I5 2,3 Ghz 8gb Ram Tela 13,3 500gb... 3.790 Frete grátis19 Apple Macbook Pro (modelo Completo Na Descriçã... 4.490 Frete grátis20 Promoção: Macbook Pro 2015 Retina I5 8gb 256ss... 5.493 Frete grátis21 Macbook Pro 2015 I5 8gb 2,7ghz 256ssd Apple Se... 6.173 Frete grátis22 Macbook Pro 2015 Retina I5 8gb 256ssd C/ Carre... 5.493 Frete grátis23 Macbook Pro 2019 16pol 2.4 I9 32gb 1tb 5550m 30.399 Frete grátis24 Macbook Pro 2019 16 Pol 2.3 I9 16gb 1tb 5500m ... 25.999 Frete grátis25 Promoção: Macbook A1534 2015 8gb Ssd 256gb + C... 6.999 Frete grátis26 Macbook Pro Apple 2015 Retina I5 8gb 256ssd - ... 6.174 Frete grátis27 Macbook Pro A1398 I7 2014 15 Pol. 16gb 256ssd ... 7.680 Frete grátis28 Macbook Pro 2016 A1706 Touch Bar I5 8gb 256ssd... 8.759 Frete grátis29 Macbook Pro 2019 16pol 2.4 I9 64gb 1tb 5550x E... 29.499 Frete grátis30 Macbook Pro 13 - 2.3ghz 8gb 256ssd Cinza Espacial 9.500 Frete grátis31 Macbook Pro 2019 16pol 2.3 I9 16gb 1tb 5550x E... 22.999 Frete grátis32 Macbook A1466/2012 13.3 Polegadas 1.8ghz 8gb 1... 4.000 -33 Oferta! Macbook Pro 2015 I5 8gb 2,7ghz 256 C/ ... 6.173 Frete grátis34 Apple Macbook Pro 2015 I5 8gb 2,7ghz 256ssd - ... 6.173 Frete grátis35 Apple Macbook Pro 2015 I5 8gb 2,7ghz 256ssd - ... 6.305 Frete grátis36 Macbook Pro Touchbar 13 2019 2.4 8gb 512gb 119... 15.999 Frete grátis37 Macbook Pro 2019 13 I7 2.8 16gb 512gb Envio Ja 19.899 Frete grátis38 Macbook Pro A1278 Tela 13 Core I5 8gb Ssd 240 ... 4.280 -39 Macbook Air 13 256gb (versao Anterior) 10.211 Frete grátis40 Macbook Pro A1398 2014 I7 2,5ghz 16gb 256gb - ... 7.680 Frete grátis41 Macbook Pro 15 - Intel Core I7 - Ssd 500gb - A... 8.900 Frete grátis42 Macbook Pro Touchbar 13 2019 2.4 8gb 512gb 11999 17.849 Frete grátis43 Macbook Pro 2019 16 Pol I7 6 Core 2.6 16gb 512... 20.990 Frete grátis44 Macbook Pro 15, 16gb, I7, 13 Ciclos, Muito Nov... 7.500 Frete grátis45 Promoção Macbook Pro 2015 I5 8gb 2,7ghz 256ssd 6.173 Frete grátis并保存data.csv（来自 LibreOffice 的屏幕截图）：
随时随地看视频慕课网APP
相关分类
Python