SPWakeUp (SPWakeUp3) v1.1.0 Released

I’ve implemented something that I’ve wanted to for a long time on SPWakeUp: The ability to wake additional URLs.

Version 1.1.0 allows the use of the ‘-Include’:’ command line parameter to specify additional URLs that will be woken once the detected site collections and subsites have been traversed.

SPWakeUp (SPWakeUp3)–Wake Up On-Premises SharePoint and WSS Instances

Since I started working with on-premises SharePoint instances, one of the solutions that I’ve used to wake up (pre-compile) the site collections and sub-sites contained within the web applications hosted by the farm is SPWakeUp.

This was originally a solution hosted on CodePlex and provided binaries for SharePoint 2007, then later SharePoint 2010 (the archive containing those can still be downloaded from the CodePlex Archive). I created compiled binaries for SharePoint 2013 and SharePoint 2016 and made those available as well.

I recently had need to use SPWakeUp on SharePoint 2019, so decided to produce a compiled for that version. As SPWakeUp doesn’t seem to have an active home anymore, I thought that it may be worthwhile putting the code and compiled versions on GitHub in case anyone else wants to use them! Note that if anyone objects to this happening, let me know and I’ll pull it down.

At the moment the repository hosts the original source code, the source code upgraded for use with Visual Studio 2019, compiled versions of SPWakeUp for SharePoint 2013, SharePoint 2016 and SharePoint 2019 and some instructions on how to compile the source yourself using Visual Studio Community 2019.

I hope it’s useful to someone!

SharePoint Crawl Rules Appears to Ignore Some URL Protocols

I recently came across an issue relating to crawling people information in SharePoint and the use of crawl rules to exclude certain content.

The issue revolved around a requirement to exclude content contained within peoples’ MySites, but include user profile information so that people searches could still be conducted. The following crawl rule had been configured and was successfully excluding MySite content, but was also excluding the user profile data (crawled using the sps3s:// protocol):

URL Exclude or Include
https://mysite.domain.com/* Exclude

Using the crawl rule test facility indicated that while SharePoint treats http:// and https:// differently, https:// and sps3s:// appear to be treated the same as far as crawling is concerned, so if the above crawl rule is in place, items in the MySite root site collection, both with an https:// and sps3s:// prefix, will not be crawled, and therefore user profile data and people search will not be available:

Crawl rule test

[Screen shot from lab SharePoint 2010 system. however the same tests have been performed against SharePoint 2013 and 2016 with the same results]

In fact what is happening is that the sps3s:// prefix tells SharePoint which connector to use, and in the case of people search, this is translated into a call to a web service at the host specified, i.e. https://mysite.domain.com/_vti_bin/spscrawl.asmx, so the final call that is made is in fact to an https:// prefix, hence the reason that the people data is not crawled.

Replacing the above crawl rule with the following rule corrects the issue allowing people data stored in the MySite root site collection to be indexed and therefore be available for users to search:

URL Exclude or Include
https://mysite.domain.com/personal/* Exclude

Hiding a Default SharePoint 2010 Content Type Field

Extending some of the default content types available in SharePoint 2010 can result in unwanted fields being shown. An example is extending the ‘Event’ default content type, which has three fields attached which you don’t seem to be able to edit:

Unwanted event fields

In particular, many users would like to remove the Workspace field from calendar entries:

Workspace field

Some users resort to editing the new/edit/view forms to remove the fields that are not wanted, however there is an easier way!

Using the following PowerShell, an unwanted field can be hidden so that it does not appear in the new/edit/view forms:

1 # Get a reference to the web we are using 2 $web = Get-SPWeb https://intranet.domain.com/site 3 4 # Get a reference to the list to which the content type is attached 5 $list = $web.Lists["Holiday Calendar"] 6 7 # Return a list of the fields 8 $fields = $list.fields 9 10 # Select the field we wish to hide 11 $field = $fields | where {$_.internalname -eq "WorkspaceLink"} 12 13 # Show the current 'hidden' status of the field 14 $field.Hidden 15 16 # Set the field to hidden (note that 'CanToggleHidden' must be true to allow this) 17 $field.Hidden = $true 18 19 # Update the field 20 $field.Update()

Creating a new holiday event on the calendar, the field allowing a user to create a workspace is now hidden:

New item form without workspace field

SharePoint 2010 Service Application Communication Scheme

The default communication scheme for many of the SharePoint 2010 Service Applications is ‘http’ (i.e. unsecured). This can be changed easily in the GUI by selecting the service application and clicking the ‘publish’ ribbon button:

Service Application publish ribbon toolbar button

It should however be noted that a number of Service Application communication schemes run by default over https and cannot be modified, these are:

  • Application Discovery and Load Balancer Service Application
  • Search Administration Web Service
  • Secure Store Service Application
  • Security Token Service Application

The communication scheme of a few Service Applications cannot be inspected using the GUI (and the publish ribbon button remains greyed out when they are selected):

  • SharePoint Server ASP.NET Session State Service
  • SharePoint Session State Service Application
  • WSS_UsageApplication

Modifying the communication scheme of all of the Service Applications can be time consuming and can be error prone, especially when using SharePoint 2010 Enterprise and the Office Web Applications with all of the Service Applications available configured for use. With this in mind, the following PowerShell will change the communication scheme of all of the Service Applications where it is possible to do so to https:

1 # This script sets the communication scheme of all Service Applications to be https instead of http 2 # Note that the communication scheme for a number of Service Applications cannot be changed 3 4 # Grab a list of the Farm's Service Applications 5 $ServiceApps = Get-SpServiceApplication | Sort-Object TypeName 6 7 # Iterate through the Service Applications 8 foreach ($ServiceApp in $ServiceApps) 9 { 10 if (($ServiceApp.TypeName -ne "Application Discovery and Load Balancer Service Application") ` 11 -and ($ServiceApp.TypeName -ne "Search Administration Web Service Application") ` 12 -and ($ServiceApp.TypeName -ne "Security Token Service Application") ` 13 -and ($ServiceApp.TypeName -ne "Secure Store Service Application") ` 14 -and ($ServiceApp.TypeName -ne "SharePoint Server ASP.NET Session State Service") ` 15 -and ($ServiceApp.TypeName -ne "State Service") ` 16 -and ($ServiceApp.TypeName -ne "Usage and Health Data Collection Service Application")) 17 { 18 # We can modify the communication scheme 19 Write-Host "Current communication shceme for" $ServiceApp.DisplayName ":" $ServiceApp.DefaultEndpoint.Name 20 if (($ServiceApp.DefaultEndpoint.Name -eq "https") -or ($ServiceApp.DefaultEndpoint.Name -eq "secure")) 21 { 22 Write-Host "Service Application already using https, skipping" -ForegroundColor Red 23 } else { 24 # Change the communication scheme to https 25 26 27 if ($ServiceApp.TypeName -eq "PowerPoint Service Application") { 28 # PowerPoint Service Application has "fast" instead of "http" and "secure" instead of "https" 29 $SAEhttps = $ServiceApp | Get-SPServiceApplicationEndpoint | where {$_.DisplayName -eq "secure"} 30 } else { 31 $SAEhttps = $ServiceApp | Get-SPServiceApplicationEndpoint | where {$_.DisplayName -eq "https"} 32 } 33 Write-Host "Setting Service Application communication scheme to https" -ForegroundColor Green 34 $ServiceApp.DefaultEndpoint = $SAEhttps 35 $ServiceApp.Update() 36 } 37 Write-Host "`n" 38 } 39 }

To reverse these changes, and set the communication scheme of all Service Applications for which it is possible to modify the communication scheme, the following PowerShell can be used:

1 # This script sets the communication scheme of all Service Applications to be http instead of https 2 # Note that the communication scheme for a number of Service Applications cannot be changed 3 4 # Grab a list of the Farm's Service Applications 5 $ServiceApps = Get-SpServiceApplication | Sort-Object TypeName 6 7 # Iterate through the Service Applications 8 foreach ($ServiceApp in $ServiceApps) 9 { 10 if (($ServiceApp.TypeName -ne "Application Discovery and Load Balancer Service Application") ` 11 -and ($ServiceApp.TypeName -ne "Search Administration Web Service Application") ` 12 -and ($ServiceApp.TypeName -ne "Security Token Service Application") ` 13 -and ($ServiceApp.TypeName -ne "Secure Store Service Application") ` 14 -and ($ServiceApp.TypeName -ne "SharePoint Server ASP.NET Session State Service") ` 15 -and ($ServiceApp.TypeName -ne "State Service") ` 16 -and ($ServiceApp.TypeName -ne "Usage and Health Data Collection Service Application")) 17 { 18 # We can modify the communication scheme 19 Write-Host "Current communication shceme for" $ServiceApp.DisplayName ":" $ServiceApp.DefaultEndpoint.Name 20 if (($ServiceApp.DefaultEndpoint.Name -eq "http") -or ($ServiceApp.DefaultEndpoint.Name -eq "") -or ($ServiceApp.DefaultEndpoint.Name -eq "fast")) 21 { 22 Write-Host "Service Application already using http, skipping" -ForegroundColor Red 23 } else { 24 # Change the communication scheme to https 25 if ($ServiceApp.TypeName -eq "Visio Graphics Service Application") 26 { 27 # Visio Graphics Service Application has "" instead of "http" (equivalent to "default" in the GUI) 28 $SAEhttp = $ServiceApp | Get-SPServiceApplicationEndpoint | where {$_.DisplayName -eq ""} 29 } elseif ($ServiceApp.TypeName -eq "PowerPoint Service Application") { 30 # PowerPoint Service Application has "fast" instead of "http" and "secure" instead of "https" 31 $SAEhttp = $ServiceApp | Get-SPServiceApplicationEndpoint | where {$_.DisplayName -eq "fast"} 32 } else { 33 $SAEhttp = $ServiceApp | Get-SPServiceApplicationEndpoint | where {$_.DisplayName -eq "http"} 34 } 35 Write-Host "Setting Service Application communication scheme to http" -ForegroundColor Green 36 $ServiceApp.DefaultEndpoint = $SAEhttp 37 $ServiceApp.Update() 38 } 39 Write-Host "`n" 40 } 41 }

Note that an IISRESET will be required on all servers in the farm once either of the above PowerShell scripts has been run to complete the modification of the communication scheme.

Incorrect Title Shown for Office 2007/2010 Documents in SharePoint 2010 Search Results

Office 2007/2010 format documents stored in SharePoint 2010 which have their title field populated show an incorrect title in the default search results. The screen shot below shows the search results for the following document:

Filename: Test document 1.docx
Title: Test Document 1 Title
First line of document: First line of test document 1

Search results Title 01a

The title that is displayed as the link to the document at the top of the individual search result is the first line of text from the document, not the title (metadata) field. It should be noted that if the title field for the document is not set, the filename is displayed as the link to the document at the top of the individual search result, not the first line of text from the document.

Luckily, correcting this particular ‘feature’ is simple:

  • Open registry editor
  • Navigate to HKLM\SOFTWARE\Microsoft\Office Server\14.0\Search\Global\Gathering Manager
  • Edit the ‘EnableOptimisticTitleOverride’ key and modify its value to 0:
    Search results Title 02a
  • Restart the SharePoint Server Search 14 service by starting an admin command prompt and issuing the following commands:
    net stop osearch14
    net start osearch14
    Search results Title 04a
  • Repeat the above steps for all SharePoint servers in the farm.
  • Perform a full crawl on the SharePoint content source(s)

Once the full crawl has completed, performing the search again gives the title field as the search result title for a document which has the title field populated (note that in the screen shot below, test document 1 has the title field set, test document 2 doesn’t):

Search results Title 07a

The following can be copied into a .reg file to automate setting the key, or the key could be set using PowerShell:

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office Server\14.0\Search\Global\Gathering Manager]
"EnableOptimisticTitleOverride"=dword:00000000

SharePoint 2007 with multiple domains

SharePoint can use multiple domains for user authentication, however I recently came across an issue when setting up an extranet using this scenario.

The steps for setting up SharePoint 2007 to use multiple domains for user authentication are relatively simple:

  • Add the second domain as a user profile source in the SSP
  • Issue the following stsadm command to encrypt the password for the account that is used to access the second forest or domain on each web front end server in the farm:
    stsadm –o setapppassword –password <password>
  • Issue the following stsadm command to add multiple domains to the people picker search list:
    stsadm –o set property –pn peoplepicker-searchadforests –pv domain:<original resource domain>;domain:<secondary domain>,<domain>\<username>,<password> –url <web application URL>

At this point I could successfully add users from the second domain to the security groups in a site collection or site, however when I attempted to log in as one of these users, I received a “500 – Internal server error”. Logging in as a user from the original resource domain worked fine however.

Modifying the web.config file for the web application to set CallStack to true and CustomErrors to Off didn’t give me any further information, at least in Internet Explorer 8, as I still saw the same “500 – Internal server error”, however viewing the web application in Firefox gave me a somewhat cryptic error:

Multi-domain login issue with FF - error

This error translates as STATUS_AUTHENTICATION_FIREWALL_FAILED, however the firewall wasn’t an issue in this scenario.

The solution was to grant the machine accounts for each of the web servers in the SharePoint farm an extra right in AD. The steps were:

  • On a domain controller, start “Active Directory Users and Computers”
  • On the view menu, ensure that the ‘Advanced Features’ option is checked
  • Locate the computer account for first of the web servers in the SharePoint farm
  • For the computer account, right-click and select properties and then click the security tab
  • Add the <external domain>\Domain Users group to the security list and grant the ‘Allowed to authenticate’ right. Click OK to close the dialog.
  • Repeat these steps for the computer accounts of all the other web servers in the farm

This resolved the issue and allowed users from the second domain to log into SharePoint.

Note that to achieve multiple domain authentication for SharePoint 2010, the same stsadm commands are used, and I therefore believe that the same issue may occur, however the above solution should also work for SharePoint 2010.

Publishing Access Services Database to SharePoint 2010 gives ‘<URL> did not respond…’

While building a portion of our demo SharePoint 2010 farm, I encountered an error when publishing an Access 2010 database to a SharePoint 2010 Access Services site.

The error which was shown was ‘<URL> did not respond. Either the server does not exist, Microsoft Access Services are not enabled on the server, or the server is using an older version of Microsoft Access Services which is not compatible with Access 2010’:

Access Services Publish Error

After checking that the Access Services farm and web application features were enabled, and that the enterprise features were enabled on the site collection to which I was attempting to publish the Access database (all were fine), I looked in the server application logs on the WFE servers in the demo farm, and on one of the farm servers saw the error ‘There is no default Access Services Application Proxy’:

Event Viewer Error

Checking the Service Application Associations (Central Administration –> Application Management –> Configure service application associations) showed that the Access Database Service proxy was not associated with the default proxy group:

Proxy Associations

After adding the Access Database Service proxy to the default group, publishing the Access database to SharePoint proceeded without a hitch.

In our case, the proxy not being associated with the default proxy group was due to us using PowerShell to configure the Access Service Application. If you do the same, check whether the proxy has been associated with the default group (or whatever proxy group you want it associated with).

Error during SharePoint 2010 Upgrade “Minimal master page failed to provision: could not find master page gallery”

During a recent SharePoint upgrade using the database attach method, I saw the following recorded in the upgrade error log:

[PerWebUpgradeAction (4.0.10.0)] [INFO] [13/04/2011 15:55:48]: SPSite Url=http://intranet.domain.com
[PerWebUpgradeAction (4.0.10.0)] [WARNING] [13/04/2011 15:55:48]: Minimal master page failed to provision: could not find master page gallery.
[PerWebUpgradeAction (4.0.10.0)] [INFO] [13/04/2011 15:55:48]: SPSite Url=http://intranet.domain.com
[PerWebUpgradeAction (4.0.10.0)] [WARNING] [13/04/2011 15:55:48]: V4 master page failed to provision: could not find master page gallery.
[PerWebUpgradeAction (4.0.10.0)] [INFO] [13/04/2011 15:55:48]: SPSite Url=http://intranet.domain.com

Further information was provided in the upgrade log that accompanies the error log:

[PerWebUpgradeAction (4.0.10.0)] [INFO] [13/04/2011 15:55:48]: Setting UIVersion metadata on existing default master page
[PerWebUpgradeAction (4.0.10.0)] [INFO] [13/04/2011 15:55:48]: Provisioning minimal master page in site: Site1
Site Url: http://intranet.domain.com/Site1
[Powershell] [PerWebUpgradeAction (4.0.10.0)] [WARNING] [13/04/2011 15:55:48]: Minimal master page failed to provision: could not find master page gallery.
[PerWebUpgradeAction (4.0.10.0)] [INFO] [13/04/2011 15:55:48]: Provisioning V4 master page in site: Site1
Site Url: http://intranet.domain.com/Site1
[PerWebUpgradeAction (4.0.10.0)] [WARNING] [13/04/2011 15:55:48]: V4 master page failed to provision: could not find master page gallery.
[PerWebUpgradeAction (4.0.10.0)] [INFO] [13/04/2011 15:55:48]: Setting UIVersion metadata on existing default master page
[PerWebUpgradeAction (4.0.10.0)] [WARNING] [13/04/2011 15:55:48]: Could not find master page gallery. Metadata on default.master not set.

The farm in question was originally a SharePoint 2003 farm that had been in-place upgraded to SharePoint 2007. The site recorded in the upgrade error log had been created in SharePoint 2003, but had not had its template selected at the time of original creation, allowing the first site user to pick the template to use. Critically however, this had never been done.

Once the database upgrade had been completed, navigating to the site gave me the expected screen to allow me to select the site template to use. Picking a template and clicking OK however produced a ‘file not found’ error.

The moral of the story seems to be ensure that any sites created without selecting a site template should either be completed, or deleted before the upgrade process starts. Note that you can still delete a site that has been upgraded to 2010 in the above state using PowerShell. Detection of sites such as these in SharePoint 2007 is straightforward using stsadm –o enumallwebs – any site that doesn’t have a template shown against it is in this state.