Asterisk in robots.txt file

Surprisingly, the following will work on Google for robots.txt:

Deny: / *. action

I need to exclude all urls ending in .action.

Is it correct?

+5
source share
3 answers

To block files of a specific file type (e.g. .gif), use the following:

User-agent: Googlebot
Disallow: /*.gif$

So you are close. Use Disallow: / *. Action $ with trailing "$"

Of course, this is what Google offers: http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449

All bots are different.

+7
source

the robots.txt specification does not allow wildcards, but only the beginning of a URI.

Google , ( " robots.txt " " " ).

+6

I don’t think this will work, you will need to move all the .action files to a place that you then deny

0
source

All Articles