如何使用 BeautifulSoup 在 Python 中接收网站链接

Question

我想从一个站点（https://www.vanglaini.org/）收集链接：/hmarchhak/102217 并将其打印为https://www.vanglaini.org/hmarchhak/102217。请帮忙 看图

import requests

import pandas as pd

from bs4 import BeautifulSoup

source = requests.get('https://www.vanglaini.org/').text

soup = BeautifulSoup(source, 'lxml')

for article in soup.find_all('article'):

    headline = article.a.text

    summary=article.p.text

    link = article.a.href

    print(headline)

    print(summary)

    print(link)

print()

这是我的代码。

Answer 1

除非我遗漏了一些标题和摘要似乎是相同的文本。您可以使用:hasbs4 4.7.1+ 来确保您article有一个孩子href；这似乎去掉了article不属于主体的标签元素，我怀疑这实际上是你的目标from bs4 import BeautifulSoup as bsimport requestsbase = 'https://www.vanglaini.org'r = requests.get(base)soup = bs(r.content, 'lxml')for article in soup.select('article:has([href])'):    headline = article.h5.text.strip()    summary = re.sub(r'\n+|\r+',' ',article.p.text.strip())    link = f"{base}{article.a['href']})"      print(headline)    print(summary)    print(link)