从侧边栏小部件 selenium python 中抓取评论

Question

我正在尝试练习从 booking.com 抓取评论。我随机选择一家酒店并让 selenium 获取元素，然后尝试定位以获取来自 BeautifulSoup 的评论，如下所示，但结果没有返回任何数据。我可以给我一些建议吗？出了什么问题以及如何解决？

from selenium import webdriver

from bs4 import BeautifulSoup

driver = webdriver.Chrome(executable_path="./driver/chromedriver.exe")

driver.get("https://booking.com")

driver.find_element_by_css_selector("input[class*='sb-searchbox__input']").send_keys("Eden The Residence at The Sea")

driver.find_element_by_css_selector("button[type='submit']").click()

driver.find_element_by_xpath("//span[contains(text(),'Eden The Residence at The Sea')]").click()

soup = BeautifulSoup(driver.page_source,"html.parser")

containers = soup.find_all("div", {"class": "review_list_container"})

reviews = containers[0].find_all("ul")

Answer 1

您必须将焦点切换到打开的新选项卡。你可以使用driver.switch_to.window(driver.window_handles[1])这是最终的代码：from selenium import webdriverfrom bs4 import BeautifulSoupimport timedriver = webdriver.Chrome()driver.get("https://booking.com")time.sleep(2)driver.find_element_by_css_selector("input[class*='sb-searchbox__input']").send_keys("Eden The Residence at The Sea")driver.find_element_by_css_selector("button[type='submit']").click()driver.find_element_by_xpath("//span[contains(text(),'Eden The Residence at The Sea')]").click()time.sleep(3)driver.switch_to.window(driver.window_handles[1])soup = BeautifulSoup(driver.page_source,"html.parser")containers = soup.find_all("span", {"class": "c-review__body"})for span in containers:    print(span.text)输出：“The location is great just few minutes walk to the beach”“The entire experience was flawless. This is the most beautiful villa I've ever stayed in, let alone, seen. The staff were outstanding and so friendly and professional. The private chefs that cooked breakfast in our Villa was amazing, and SO well pri…“The staff were great and it was very easy to order meals and have them prepared at a very reasonable price.”“The villa is spacious and airy with a large private pool.Bedrooms are large each with their own bathroom facilties.The staff go above and beyond to make your stay as comfortable as possible.”“Excellent , friendly helpful staffBeautiful patio, garden and poolVery spacious villa Fast laundry serviceDecorate the villa and bath with flowers for my wife’s birthday ”“We loved almost everything about Eden Residence from the location, the quality of the villa, the staff, the cost and the room service (cooks came and prepared meals at the cost to buy groceries plus 15%).The staff are super friendly and we felt ext…“Great accomodation. Lots of space. Great pool. Good service”“Everything! The spacious living room, big toilet, beautiful surroundings. Not to forget, fantastic service by all the staff! Sampai jumpa lagi semuanyaaa 🙏🏻”“Staff were exceptional. They went above and beyond to meet our needs at all hours of the day/night. Great, convenient location. Large spacious rooms all each with their own bathroom/ensuite. Pool was amazing. Staff cooked traditional suckling pig fo…“Luxury VillaExcellent friendly staffPrivate poolOcean View sunsets from rooftop Walking distance to beach”