class EastSpider(scrapy.Spider): name = 'East' allowed_domains = ['****.com'] start_urls = ['http://finance.***.com/news.html'] def parse(self, response): nextUrl = response.xpath('//*[contains(@class,"page-btn")]/@href') for url in nextUrl.extract(): time.sleep(1) yield Request(urljoin(response.url,url)) contentUrl = response.xpath('//p[@class="title"]/a/@href') for urls in contentUrl.extract(): time.sleep(1) yield Request(urls,callback = self.parse) pass
代码如上,但是在命令行运行scrapy crawl East -o East.csv
的结果,East.csv是个空文件,什么都没写进去。
我看人家说要yield,但是自己搞了搞也没行...
尝试了在for循环之外添加yield url
、yield urls
报错,说是在定义之前引用了,然后在for循环之内添加又没效果,还是空文件...
相关分类