Geek Answers Handbook

Block some dynamic pages from search engines

I need to block some of my pages from search engines. How can i do this? The application was developed using ASP.net MVC and AngularJS. Thanks in advance.

These are the URLs I want to block from search engines.

http://localhost:12534/myurl123-event?participant=12957 http://localhost:12534/myurl123-event

Note: The last part of the URL is dynamic (i.e. myurl123-event?participant=12957 and myurl123-event ).

+1
source share
2 answers

You can use the robots.txt file with the prohibition setting:

 User-agent: * Disallow: /myurl123-event

If you want to block everything, then you can use

 Disallow: /*

or you can put all your dynamic pages in a route (or use the controller name if you route a simple controller / method / id)

 Disallow: /dynamic/
+2
source

Google's preferred way is to use canonical links:

<link rel="canonical" href="http://somedomain.com/myurl123-event" />

This tag function should tell the search engine which URL should be indexed on these pages containing dynamic query parameters.

Read more about them here: Google link

Using them, you don’t have to worry about accidentally blacklisting parts of your site, for example, using robots.txt files and getting into your traffic.

+1
source

More articles:


All Articles