Can Scrapy work in PHP?

Question

Can I use Scrapy in PHP or are there similar tools that work with PHP?

I am not a technical person, I just study the available web search tools and their functions to support my technical colleagues.

Answer 1

Scrapy for python, and you cannot use this in PHP.

However, in PHP you can use Goutte to accomplish this task. It uses Guzzle HTTP and Symfony components like BrowserKit and DomCrawler backstage to do the job.

Check this:

use Goutte\Client; $client = new Client(); // Go to the symfony.com website $crawler = $client->request('GET', 'http://www.symfony.com/blog/'); // Get the latest post in this category and display the titles $crawler->filter('h2 > a')->each(function ($node) { echo $node->text().'\n'; }); 

Additional Usage Information

PS: Please note that JavaScript does not work.

Answer 2

You can check out PHP SimpleTest ScriptableBrowser ...

Answer 3

You cannot write Scrapy spiders using PHP.

However, it’s very simple to use Scrapy (write spiders in Python) and store the extracted data in a database or something available to your application. For example, it is quite easy to store the extracted elements directly in ElasticSearch and request your ES application to search / filter / aggregate data.

But, if your colleagues do not know Python, they will need to spend some time learning the language, and then the Scrapy framework.