Scrapy广泛使用信号来通知某些事件何时发生。你可以在你的垃圾项目中捕捉到这些信号（使用 extension 例如）执行其他任务或扩展scrapy以添加框外未提供的功能。 即使信号提供了几个参数，捕获它们的处理程序也不需要接受所有参数——信号调度机制只传递处理程序接收的参数。 您可以通过 信号API . 下面是一个简单的示例，演示如何捕捉信号并执行某些操作： from scrapy import signals
from scrapy import Spider
class DmozSpider(Spider):
name = "dmoz"
allowed_domains = ["dmoz.org"]
start_urls = [
"http://www.dmoz.org/Computers/Programming/Languages/Python/Books/",
"http://www.dmoz.org/Computers/Programming/Languages/Python/Resources/",
]
@classmethod
def from_crawler(cls, crawler, *args, **kwargs):
spider = super(DmozSpider, cls).from_crawler(crawler, *args, **kwargs)
crawler.signals.connect(spider.spider_closed, signal=signals.spider_closed)
return spider
def spider_closed(self, spider):
spider.logger.info('Spider closed: %s', spider.name)
def parse(self, response):
pass |
Archiver|手机版|笨鸟自学网 ( 粤ICP备20019910号 )
GMT+8, 2025-8-30 17:45 , Processed in 0.023506 second(s), 19 queries .
Powered by Discuz! X3.5
© 2001-2017 Discuz Team. Template By 【未来科技】【 www.wekei.cn 】
