Optimize / configure shared scan

Question

With SharePoint Server 2007, there is a search function and a scanner. However, the crawler is somewhat limited in that it only supports Basic Auth when crawling external sites, and that there is no way to tell it to ignore the no-index, no-follow attributes.

Now there is a site that I would like to index, unfortunately, this site uses its own authentication system, and it uses no indexes, not tracking on pages. When I control this site, I can remove the attributes, but this is PITA for this. In addition, it does not solve the authentication problem.

So I'm just wondering, is it possible to somehow extend the Sharepoint Crawler?

Answer 1

The limitation of MOSS bypass sites with authentication of various forms should be addressed in MOSS Service Pack 1.

http://www.microsoft.com/downloads/details.aspx?FamilyID=ad59175c-ad6a-4027-8c2f-db25322f791b&displaylang=en

Here's a link to a post that describes how to get the hotfix for MOSS before SP1 to enable crawling sites with forms authentication:

http://blogs.microsoft.co.il/blogs/adir_ron/archive/2007/10/11/moss-search-for-sso-form-based-authentication-sites.aspx

Hope this helps!