“Access is Denied” when crawling content on MOSS 2007 hosted on Windows Server 2008

One of the SharePoint farms we’ve built recently runs on Windows Server 2008  and SQL Server 2008.  As usual, the installation is a least privileged account setup, with individual accounts running the various services and app pools. The farm is also patched to the latest level.

We’ve experienced one or two issues with this setup, but the most persistent one has been to do with crawling.  When crawls run, they would consistently fail with the following error in the Event viewer:

“The start address <https://site.domain.com> cannot be crawled.

Context: Application 'SharedServices1', Catalog 'Portal_Content'

Details:
    Access is denied. Check that the Default Content Access Account has access to this content, or add a crawl rule to crawl this content.   (0x80041205)”

In addition, the following error appeared in the SharePoint logs:

***** Couldn't retrieve server https://site.domain.com policy, hr = 80041205 - File:d:\office\source\search\search\gather\protocols\sts3\sts3util.cxx Line:548

And the crawl logs showed only errors, each having the following description:

“Access is denied. Verify that either the Default Content Access Account has access to this repository, or add a crawl rule to crawl this repository. If the repository being crawled is a SharePoint repository, verify the account you are using has “Full Read” permissions on the SharePoint Web Application being crawled.(The item was deleted because it was either not found or the crawler was denied access to it.)”

We’d checked all of the usual suspects including web application permissions for the account used by search, database permissions etc with no success.

The solution was to disable the loopback check on the servers hosting SharePoint. Adding the hostnames served to the BackConnectionHostNames list in the registry on the SharePoint servers wasn’t enough, the loopback check had to be completely disabled.

As an aside, another issue we’d experienced with an InfoPath form with code behind failing to load correctly on these servers was also solved disabling the loopback check on these servers.

For instructions on disabling the loopback check, see KB896861.

Technorati Tags: SharePoint,indexing,crawl,crawling,search,access denied,access is denied,error,loopback check,Windows Server 2008,SQL Server 2008