即使使用 WebDriverWait 也是陈旧的元素

Question

我不明白为什么我会收到此错误：

raise exception_class(message, screen, stacktrace) selenium.common.exceptions.StaleElementReferenceException: Message: The element reference of <a id="u153-popover-trigger--3926" class="udlite-custom-focus-visible browse-course-card--link--3KIkQ" href="/course/kafka-streams-real-time-stream-processing-master-class/"> is stale; either the element is no longer attached to the DOM, it is not in the current frame context, or the document has been refreshed

我使用 WebDriverWait 两次来检查是否加载了新页面：

• 如果新页面的分页链接发生相应变化

• 如果新页面的课程列表div元素被加载

  from selenium import webdriver

    from selenium.webdriver.common.keys import Keys

    from selenium.webdriver.firefox.options import Options

    from selenium.webdriver.common.by import By

    from selenium.webdriver.support.ui import WebDriverWait

    from selenium.webdriver.support import expected_conditions as EC

    from selenium.common.exceptions import TimeoutException

    

    def waitForLoad(inputXPath): 

        Wait = WebDriverWait(driver, 10)

        Wait.until(EC.presence_of_element_located((By.XPATH, inputXPath)))

    

    options = Options()

    options.headless = True

    driver = webdriver.Firefox(options=options, service_log_path='NUL')

    

    driver.get("https://www.udemy.com/courses/development/?sort=highest-rated")

    

    courses = []

    f = open("0udemy.txt","a", encoding="utf-8")

    page = 1

    

    try:

        waitForLoad("//div[@class='filter-panel--paginated-course-list--2F0x1']")

    except TimeoutException as e:

        print("timeout!")

    

    while True:

        ## I also tried that : 

        #courses = driver.find_elements_by_xpath("//div[@class='course-list--container--3zXPS']//a[contains(@class, 'browse-course-card--link--3KIkQ')]")

        #for i in courses:

        #    f.write(f"{i.get_attribute('href')}\n")

我在第 20 页和第 80 页之间收到陈旧错误。我的世界纪录是第 78 页。

Answer 1

我一次只浏览了 1 页并打印出来（您可以使用 f.write）。您需要添加 time.sleep() 以便 Selenium 不会崩溃。这可以永远持续下去，直到页面用完为止。或者如果指定 if page==n: 则中断。它甚至可以经历硒的生命周期。while True:    try:        courses=WebDriverWait(driver, 30).until(EC.visibility_of_all_elements_located((By.XPATH, "//div[@class='course-list--container--3zXPS']//a[contains(@class, 'browse-course-card--link--3KIkQ')]")))        for course in courses:            print(course.get_attribute('href')+"\n")        driver.find_elements_by_xpath("//a[@class='udlite-btn udlite-btn-small udlite-btn-secondary udlite-heading-sm udlite-btn-icon udlite-btn-icon-small udlite-btn-icon-round pagination--next--5NrLo']")[0].click()        page=page+1        time.sleep(5)    except:        url=driver.current_url        driver.close()        driver = webdriver.Chrome(ChromeDriverManager().install(),options=options)        driver.get(url)