Geek Answers Handbook

Why is spider needed for treatment in spider.py?

I saw that in some spider files at the end of the class they use

class TestSpider(BaseSpider): pass SPIDER = TestSpider()

Why do we use SPIDER = TestSpider() ? I have not used it, and my spider is working fine.

+4
source share
2 answers

Scrapy originally used a twisted plug-in mechanism for managing spiders, and this requires an instance. This has been changed in favor of class checking for something that extends BaseSpider and has a name. You will still see instances created in the old code, although it is no longer required in any latest version of scrapy.

+4
source

In the case you should above, SPIDER = BaseSpider() will most likely work. It is also possible to leave the file empty if SPIDER is defined by default elsewhere. I'm not very good at therapy.

0
source

More articles:


All Articles