Geek Answers Handbook

Does Visual Studio support editing Robots.txt?

Any robots.txt fixes used in VS2010 save the BOM file, which basically causes Google to reject it using ' Syntax is not clear error>. There is a question in this, but the option “Save with encoding” is not available for text files - even if that were the case, there should be a solution that simply works with CTRL + S instead of following an advanced route only to exclude BOM .

I can’t believe that I am the only person experiencing this problem, of course, is there a solution?

+4
source share
2 answers

It seems like an easy fix is ​​to have an empty line or comment as the first line of the file: http://www.12titans.net/p/robots-txt-bom.aspx

+1
source

VS adds the specification to the UTF-8 file, but why should your robots.txt file be encoded as UTF-8?

Since it contains URLs, it should be plain ASCII or ISO-8859-1. If your site’s URLs contain non-ASCII characters, you must encode the URLs appropriately. Creating a simple ANSI file will save the whole problem with the BOM.

Also see robots.txt; What is the encoding?

0
source

More articles:


All Articles