Geek Answers Handbook

How can I get the spider name in the image pipeline

I have many image pipelines, but I want to use different conservation methods for different spiders.

I know that in other pipelines I can use spider.name , but how can I get this in the image scroll

 class MyImagesPipeline(ImagesPipeline): if spider.name in ['first']: def get_media_requests(self, item, info):
+4
source share
2 answers

The spider is passed as an argument to process_item :

https://scrapy.readthedocs.org/en/latest/topics/item-pipeline.html#item-pipeline-example

You can either set the variable during evaluation for use in the class, or implement the hook yourself if you need a spider before calling process_item.

 class MyImagesPipeline(ImagesPipeline): spider = None def process_item(self, item, spider): self.spider = spider if self.spider.name in ['first']: get_media_requests(item, info) return item def get_media_requests(self, item, info): # whatever

You can also get information directly from the base class, which has an internal SpiderInfo metaclass with the spider attribute.

see https://github.com/scrapy/scrapy/blob/master/scrapy/contrib/pipeline/media.py

+5
source

info.spider is what you want.

 def get_media_requests(self, item, info): info.spider.name
+1
source

More articles: