猿问

为什么部分标题为空?

我想在https://www.reddit.com/search?q=expiration&type=link&sort=new上抓取所有问题的链接、标题和时间戳。由于有很多问题,我们需要向下滚动以触发加载。这是代码:


from selenium import webdriver

import time


urlMethod = "https://www.reddit.com/search?q=expiration&type=link&sort=new"

driver = webdriver.Chrome()

driver.get(urlMethod)


SCROLL_PAUSE_TIME = 3

i = 0


last_height = driver.execute_script("return document.body.scrollHeight")


while (i <= 3):

    i = i + 1

    driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")

    time.sleep(SCROLL_PAUSE_TIME)    

    new_height = driver.execute_script("return document.body.scrollHeight")

    if new_height == last_height:

        break

    last_height = new_height


questions = driver.find_elements_by_xpath('//div[@data-click-id="body"]')

print(len(questions))


for q in questions:

    url = q.find_elements_by_xpath('.//a[@data-click-id="body"]')[0].get_attribute('href')

    title = q.find_element_by_xpath('.//a[@data-click-id="body"]').find_element_by_xpath('./h2/span').text

    print("title : " + title)

    datePostRel = q.find_elements_by_xpath('.//a[@data-click-id="timestamp"]')[0].get_attribute('innerHTML')

但是,我的测试表明部分标题是空的。有谁知道为什么?


PS:我尝试使用WebDriverWait,但它给了我一个TimeoutException错误。


潇湘沐
浏览 177回答 2
2回答

繁星点点滴滴

试试下面的代码。它应该可以工作。你必须使用文本而不是文本innerHTML。&nbsp; title = q.find_element_by_xpath('.//a[@data-click-id="body"]').find_element_by_xpath('./h2/span').get_attribute("innerHTML")输出:title : (Possibly Unpopular) Opinion: MTG Arena's Substandard Free-to-Play Economy Creates an Inferior Metatitle : Should I bring up the topic of a possible learning disability with my psychiatrist before I go to college or am I making a big deal out of nothing?title : MeasureUp vouchers for Microsoft technical practice examstitle : should i break up with him or not?title : Update on the blown up intervention. Things got worse. He went to jail. I am leaving to get treatment for trauma, PTSD, and codependency.title : AITA for taking two days to consider a employment offer?title : Hi, I am trying to find the most current application with which to renew DACA, but the only one I can find on the USCIS has 1/31/19 as the <em style="font-weight:700">expiration</em> date. Is this still a valid form, or will it be rejected? Current permit is to expire in September.title : Just like food, humans have <em style="font-weight:700">expiration</em> dates too..title : Drug <em style="font-weight:700">Expiration</em> Dates — Do They Mean Anything? - Harvard Health (This should be sticky'd btw)title : Here is a scientific study made on how long prescription drugs last after their expiry date. I thought it would be relevant here.title : If poison is past its <em style="font-weight:700">expiration</em> date, is it more poisonous or less poisonous?title : Is there any coming back from deep-seated resentment?title : 29 Domains for sale! | All Priced between $19-$79 | BIN via Efty/Paypaltitle : This Pringles can with a Leap Day <em style="font-weight:700">expiration</em> datetitle : Is it wrong for a relationship to have an <em style="font-weight:700">expiration</em> date?title : Buy Valtrex From a Usa Pharmacy Without a Prescription, How To Mail Order Valtrex Canadatitle : Fragment Bulletstitle : Calculating <em style="font-weight: 700;">Expiration</em> Dates - Previous Solution No Longer Workstitle : My current dilemma with excess backstory. TLDR at the bottomtitle : Worst (and not-so-bad) things for metric-born US residenttitle : Weird Question about sorting your papers?title : Hot Cash Mega Threadtitle : [40k] What would a modern earth's tithe consist of?title : TIL an FDA study requested by the military found 90% of more than 100 drugs, both prescription and over-the-counter, were still safe &amp; effective even 15 years after the <em style="font-weight: 700;">expiration</em> date. <em style="font-weight: 700;">Expiration</em> dates don’t really indicate a point at which the medication is no longer effective or unsafe to use.title : Do I still have stock options?title : CVS Coupon <em style="font-weight: 700;">Expiration</em> Policytitle : CVS Coupon <em style="font-weight: 700;">Expiration</em> Datetitle : Over 100 OTC and prescription drugs 90% effective 15 years after <em style="font-weight: 700;">expiration</em> datestitle : We need all hands on decktitle : My option grant contract up to scratch (concerned about repurchase/claw back)title : I'm really not happy anymoretitle : How to have a healthy non-committed relationship?title : TIL an FDA study requested by the military found 90% of more than 100 drugs, both prescription and over-the-counter, were still safe &amp; effective even 15 years after the <em style="font-weight: 700;">expiration</em> date. <em style="font-weight: 700;">Expiration</em> dates don’t really indicate a point at which the medication is no longer effective or unsafe to use.title : Marriage license questiontitle : Just plain ole happy....title : Need a favor! Swap Menopur in the DC areatitle : Understanding optionstitle : <em style="font-weight: 700;">Expiration</em>?title : Need answers about respiratory symptoms to calm my nervestitle : [HIRING] Looking for PHP developer for Pydiotitle : Top-DOCTORS.com make offertitle : PendingIntent.isTargetedToPackage() returns false for IntentService. Why, and how to fix it?title : Reconnecting with my [28f] ex [28m]?title : PSA (possibly): Colourpop is technically not allowed to ship to the EUtitle : Each time I start getting to know someone new, I can’t help think about the relationship having an <em style="font-weight: 700;">expiration</em> date.title : Can we get an <em style="font-weight: 700;">expiration</em> timer on all electronics like the sensor dart or a proximity killswitch?title : Do dates just live forever? No <em style="font-weight: 700;">expiration</em> date on the package. Am I going to die?title : WTW for opposite of a "limited time offer"title : S.102 - Committee Votetitle : But it's expired...title : this <em style="font-weight: 700;">expiration</em> stamp (2017 on the trademark though)title : /u/Nadieestaaqui on Do you think NAFTA is a horrible deal but USMCA is a great trade agreement?title : Classic teenager: forgets to check the pills <em style="font-weight: 700;">expiration</em> date and just chucks em down the chutetitle : Exit on vertical debit vs credit spreadstitle : TEFL Contrct Review China. Am I safe Or Could I Get In Trouble?title : Expired Betaine HCL w/ Pepsin still good?title : IKE phase 1 has expired, but PA doesn't start re-key for 45mins to an hour latertitle : ConnectWise workflow for Expiring Configurationstitle : Your portal (and all the data) on ITBoost is kept online even if you have cancelled months ago.title : Document Authentication Without Job Offertitle : The Deli Meat Sagatitle : Getting errors when trying to import a moduletitle : I am the dead bedroom.title : Does rank <em style="font-weight: 700;">expiration</em> affect Grand Champion rewards?title : How do I tell my (24f) boyfriend (25m) that his words hurt?title : Freaking out right nowtitle : Same doctor that said "no longer coronary risk" 2 wks prior to death now refuses to get us death certificates. Do we have legal recourse?title : A Footstep Echo is FREE on Kindle this week - Sci-fi / Time Travel - Mysterytitle : What makes milk/chocolate milk taste/smell bad if it's not expired?title : The <em style="font-weight: 700;">expiration</em> date on this bag of potato stickstitle : What are the major issues driving the potential lockout?title : Attention all epic conservatives!title : Anyone Else Considering Puts for the NIO Lock-up <em style="font-weight: 700;">Expiration</em>?title : Suggestion Chems/Food/Drinks <em style="font-weight: 700;">expiration</em> timetitle : Unknown <em style="font-weight: 700;">Expiration</em> Fuji 160S - Mamiya 645 65mm 2.8 Central Park, my lovely girlfriend with her Minolta SRT 101.title : What happens when someone sells a put a few days before <em style="font-weight: 700;">expiration</em> and $0.20 over the strike price.title : Are IUDs still effective near the end of their <em style="font-weight: 700;">expiration</em>?title : Topical ibuprofen gel out of date, should I be using it?title : Prestige Points wouldnt feel so awful of a system if we got them from spending RP on skins/emotes/etc tootitle : 70% off mens beach shortstitle : Begging for... Sushi?title : CAE vs IELTS Comparisontitle : The double strike <em style="font-weight: 700;">expiration</em> cancel bug is backtitle : ISO 13485:2016 Certification in Singaporetitle : questions about philly connections ticketstitle : [USA] [H][USA] [H] Persona 5, Nier, Spiderman, Yakuza 6, Monster Hunter World, Fallout 76, DBZ Fighterz list [PS4] Mario Odyssey, lets go evee list [SWITCH] Ever Oasis, List [3DS] [W] Ys VIII, list [PS4]Ys VIII: Lacrimosa of Dana, list [SWITCH]title : A few final questions for a new Meat DM.title : <em style="font-weight: 700;">Expiration</em> dating hand-made whip creamtitle : My chips have an <em style="font-weight: 700;">expiration</em> timetitle : Boss forgets 3 year old food containertitle : H.R.171: Military Federalism Act of 2018 AMENDMENT PERIODtitle : [Product Question] <em style="font-weight: 700;">Expiration</em> Date On Scar Creams, Do They Matter?title : DEA claim Chapter 35title : I am afraid of falling in love with a girl in my universitytitle : I'm Paige Feldman, a development and acquisitions exec and writer/director/producer - Ask Me Anything!title : Bungie Rewards <em style="font-weight: 700;">expiration</em> dates have been updatedtitle : ELI5: How are <em style="font-weight: 700;">expiration</em> dates on food/other products with <em style="font-weight: 700;">expiration</em> dates actually determined? Like how do they figure out when the product is going to expire?title : [H] VPN Accounts starting at just $1! [W] BTC/ETH/LTC/Paypaltitle : $MEET Earnings Playtitle : People should be able to activate 2 week free trial n word passestitle : Needle Teeth - <em style="font-weight: 700;">Expiration</em> Date [Cassette out next month on Solidarity Club Records]title : Issues with trying to add a new Mastercard to SPay. Anyone else having similar issues?title : Third Party Tool to Monitor AD Cert Authority?title : MSU ID Card <em style="font-weight: 700;">expiration</em>title : A law mandating that all best by/<em style="font-weight: 700;">expiration</em> dates be in the same spot and in the same format on all products for easy deciphering by consumers.title : met her after no contact for bit over 1mo and so confusingtitle : When u have the same <em style="font-weight: 700;">expiration</em> date as sour creamtitle : What character stayed past their <em style="font-weight: 700;">expiration</em> date and make the show/book/whatever worse because they were popular?title : Should I take the internship offer or go for the interview for a full-time position? Or both?title : Attention all Yasuo mains!!title : RMA without proof of Purchase?title : I was curious about whois of jw.(b)orgtitle : [Sell][US]Pick 4 freebies with your order! Selling Makeup &amp; Skincare: eSpoir, Cosrx, Innisfree, Acwell, Neogen, Peripera, Klairs, Etude House, Benton, Too cool for school, Glossier, The Ordinary, Annie's Way, DewyTree, Holika Holika, A'Pieu + more!title : Lucky day promotions: free wheel spin!! Enter code: RO8AKFZ Good Luck!!title : Virtually clueless on budget friendly but long lasting Acer Chromebook (uk)title : A Footstep Echo is Available FREE on Kindle this week!! Sci-fi / Mystery / Time Traveltitle : How long/often can I travel in the USA with an ESTA?title : How long can I travel in the USA with an ESTA?title : 3-15-19 (no new info so don't freak out)title : When should I receive SRB payout?title : [LPT] Everytime you close a contract, set yourself a reminder a couple of weeks/months (depending on your notice period) before <em style="font-weight: 700;">expiration</em> so that you get reminded about it before the contract extends itself.title : Dumb question of the day.title : Asp web api Access_token and Refresh tokentitle : Do you eat food that's past its <em style="font-weight: 700;">expiration</em> date? What is your reason?title : Facilitated Trading - EARN CREDIT THIS WEEK: T-Mobile Tuesdays Trading/Discussion thread March 05, 2019

慕妹3146593

有很多<a>s,所以也许您正在选择一些您不想要的。我会尽可能具体,例如title&nbsp;=&nbsp;q.find_elements_by_class_name('SQnoC3ObvgnGjWt90zD9Z').find_element_by_xpath('./h2/span').text使用find_elements_by_class_name和“标题链接”的类。
随时随地看视频慕课网APP

相关分类

Python
我要回答