Is robot.txt used for subdomains?

Question

Is robot.txt used for subdomains?

Let's say I have a test folder (test.domain.com), and I don’t want search engines to crawl it, I need to have robot.txt in the test folder, or I can just place the robot. txt in the root, then just disable the test folder?

Thanks!

Answer 1

Each subdomain is usually considered as a separate site and requires its own robots.txt file.

Answer 2

When the crawler retrieves test.domain.com/robots.txt, that is, the robots.txt file that it sees. It will not see any other robots.txt file.

Answer 3

, robots.txt . ( ). - .htaccess, , robots.txt .

- , ( ) robots.txt , . , :

http://yourrootdomain.com/robots.txt
http://subdomain.yourrootdomain.com/robots.txt