Check if Google URL is indexed using PHP

Question

I would like to know if it is possible to check if Google URLs are indexed using PHP.

Is it against their ToS?

Answer 1

You can read here (corresponding link below) for an answer to the ToS part of this. Basically, without an API key and their permission, this is probably not a good idea. However, due to the amount that they process, you can get away from it if you do not make TONS requests.

Checking PageRank is something else that people often try to do, but they don’t attach so much weight to this merit (rumored), and old style API keys are really hard to find.

Do not use an unauthorized computer program to send pages, check ratings, etc. Such programs consume computing resources and violate our Terms of Service . Google does not recommend the use of products such as WebPosition Gold ™ that automatically send or programmatic requests to Google.

Answer 2

Well, not explicitly. But you can check every pageview using:

$agent = $_SERVER['HTTP_USER_AGENT']; if (strstr($agent, 'googlebot')){ // tell the database that google has crawled this page. } 

Answer 3

Do it without API against TOS. For small volume you can:

 // CHECK IF PAGE IS IN GOOGLE INDEX $domain = 'stackexchange.com'; if (strstr(file_get_contents("http://www.google.com/search?q=site:$domain"), 'did not match any documents')) { // Page is not in the index print 'No Go!'; } else { print 'All Good!'; } exit; 

Answer 4

For Polish, you should try checking between UTF-8 and ISO-8859-2 as follows:

 $encAry = array('ISO-8859-2', 'UTF-8'); $contentEncoding = mb_detect_encoding( $content, $encAry ); $googleSearchResult = mb_convert_encoding($content, 'UTF-8', $contentEncoding); 

It works for me.