Getting crop error in scrapy python

Question

I have this code. When I use yyield to request a further link, I get this error message

Spider must return Request, BaseItem or None, got 'dict' 

I tried everything, but I can not get rid of the error

Code here

 def parse_items(self, response): hxs = HtmlXPathSelector(response) sites = hxs.select("//li[contains(concat(' ', @class, ' '), ' mod-searchresult-entry ')]") items = [] for site in sites[:2]: item = SeekItem() item['title'] = myfilter(site.select('dl/dd/h2/a').select("string()").extract()) item['link_url'] = myfilter(site.select('dl/dd/h2/em').select("string()").extract()) item['description'] = myfilter(site.select('dl/dd/p').select("string()").extract()) if item['link_url']: yield Request(urljoin('http://www.seek.com.au/', item['link_url']), meta = item, callback = self.parseItemDescription) yield item def parseItemDescription(self, response): item = response.meta hxs = HtmlXPathSelector(response) sites = hxs.select("//li[contains(concat(' ', @class, ' '), ' mod-searchresult-entry ')]") item['description'] = "mytest" return item 

Answer 1

What version of scrapy are you using. In the documentation for version 0.16.2, this method passes items to another callback .

 def parse_items(self, response): hxs = HtmlXPathSelector(response) sites = hxs.select("//li[contains(concat(' ', @class, ' '), ' mod-searchresult-entry ')]") items = [] for site in sites[:2]: item = SeekItem() item['title'] = myfilter(site.select('dl/dd/h2/a').select("string()").extract()) item['link_url'] = myfilter(site.select('dl/dd/h2/em').select("string()").extract()) item['description'] = myfilter(site.select('dl/dd/p').select("string()").extract()) if item['link_url']: request = Request("http://www.example.com/some_page.html", callback=self.parseItemDescription) request.meta['item'] = item return request def parseItemDescription(self, response): item = response.meta['item'] hxs = HtmlXPathSelector(response) sites = hxs.select("//li[contains(concat(' ', @class, ' '), ' mod-searchresult-entry ')]") item['description'] = "mytest" return item 

NB: this is not verified, as the rest of your code (spider, items.py, etc.) is missing and I'm not sure how this is done

Answer 2

Two exits

You give in twice - the first time - Request ; the second is dic . (request return (...) and return)

I assume that a second time is not needed and should be removed. Try this and comment below. (delete the line where the yield element is specified)