How to configure a robots.txt file to block all but 2 directories

Question

I don’t want search engines to index most of my site.

However, I want search engines to index 2 folders (and their children). This is what I created, but I don’t think it works, I see the pages on Google that I wanted to hide:

Here is my robots.txt

User-agent: * Allow: /archive/ Allow: /lsic/ User-agent: * Disallow: / 

What is the correct way to ban all folders except 2?

Answer 1

I gave a tutorial about this in this forum here . And on Wikipedia here

Basically, the first matching robots.txt template always wins:

 User-agent: * Allow: /archive/ Allow: /lsic/ Disallow: / 

But I suspect it might be too late. Once a page has been indexed, it is quite difficult to delete it. The only way is to transfer it to another folder or simply password protect the folder. You should be able to do this on your cPanel hosts.