BM-Bloggers

The blogs of Black Marble staff

Test-SPContentDatabase False Positive

I was recently performing a SharePoint 2013 to 2016 farm upgrade and noticed an interesting issue when performing tests on content databases to be migrated to the new system.

As part of the migration of a content database, it’s usual to perform a ‘Test-SPContentDatabase’ operation against each database before attaching it to the web application. On the farm that I was migrating, I got mixed responses to the operation, with some databases passing the check successfully and others giving the following error:

PS C:\> Test-SPContentDatabase SharePoint_Content_Share_Site1

Category        : Configuration
Error           : False
UpgradeBlocking : False
Message         : The [Share WebSite] web application is configured with
                  claims authentication mode however the content database you
                  are trying to attach is intended to be used against a
                  windows classic authentication mode.
Remedy          : There is an inconsistency between the authentication mode of
                  target web application and the source web application.
                  Ensure that the authentication mode setting in upgraded web
                  application is the same as what you had in previous
                  SharePoint 2010 web application. Refer to the link
                  "
http://go.microsoft.com/fwlink/?LinkId=236865" for more
                  information.
Locations       :

This was interesting as all of the databases were attached to the same content web application, and had been created on the current system (I.e. not migrated to it from an earlier version of SharePoint) and therefore should all have been in claims authentication mode. Of note also is the reference to SharePoint 2010 in the error message, I guess the cmdlet hasn’t been updated in a while…

After a bit of digging, it turned out that the databases that threw the error when tested had all been created and some initial configuration applied, but nothing more. Looking into the configuration, there were no users granted permissions to the site (except for the default admin user accounts that had been added as the primary and secondary site collection administrators when the site collection had been created), but an Active Directory group had also been given site collection administrator permissions.

A quick peek at the UserInfo table for the database concerned revealed the following (the screen shot below is from a test system used to replicate the issue):

UserInfo Table

The tp_Login entry highlighted corresponds to the Active Directory group that had been added as a site collection administrator.

Looking at Trevor Seward’s blog post ‘Test-SPContentDatabase Classic to Claims Conversion’ blog post showed what was happening. When the Test-SPContentDatabase cmdlet runs, it’s looking for the first entry in the UserInfo table that matches the following rule:

  • tp_IsActive = 1 AND
  • tp_SiteAdmin = 1 AND
  • tp_Deleted = 0 AND
  • tp_Login not LIKE ‘I:%’

In our case, having an Active Directory Group assigned as a site collection administrator matched this set of rules exactly, therefore the query returned a result and hence the message was being displayed, even though the database was indeed configured for claims authentication rather than classic mode authentication.

For the organisation concerned, having an Active Directory domain configured as the site collection administrator for some of their site collections makes sense, so they’ll likely experience the same message next time they upgrade. Obviously in this case it was a false positive and could safely be ignored, and indeed attaching the databases that threw the error to a 2016 web application didn’t generate any issues.

Steps to reproduce:

  1. Create a new content database (to keep everything we’re going to test out of the way).
  2. Create a new site collection in the new database adding site collection administrators as normal.
  3. Add a domain group to the list of site collection administrators.
  4. Run the Test-SPContentDatabase cmdlet against the new database.

Steps Required to Configure WSUS to Distribute Windows 10 Version 1511 Upgrade

Microsoft recently made a hotfix available that patches WSUS on Windows Server 2012 and 2012 R2 to allow Windows 10 upgrade to version 1511. Installing the update is not, however, the only step that is required…

  1. Install the hotfix. This can be downloaded from https://support.microsoft.com/en-us/kb/3095113. Ensure that you pick the appropriate hotfix for the version of Windows Server on which you’re running WSUS. Note that if you’re running Windows Server 2012 R2, there’s also a pre-requisite install.
  2. Once the hotfix is installed and you’ve restarted your WSUS server, look in the ‘Products and Classifications’ option under the Classifications tab and ensure that the checkbox for upgrades is selected. This is not selected automatically for you:
    Upgrades Option
    Note that the upgrade files may take quite some time to download to your WSUS server at the next synchronisation.
  3. Add a MIME-Type for ‘.esd application/octet-stream’ in IIS on the WSUS server. To do this:
    Open IIS Manager
    Select the server name
    From the ‘IIS’ area in the centre of IIS Manager, open ‘MIME Types’
    Click ‘Add…’
    Enter the information above:
    Esd MIME Type
    Click OK to close the dialog.
    Note: Without this step, clients will fail to download the upgrade with the following error:
    Installation Failure: Windows failed to install the following update with error 0x8024200D: Upgrade to Windows 10 [SKU], version 1511, 10586.
  4. Approve the Upgrade for the classes of computer in your organisation that you want to be upgraded.

Once all of the above steps are in place, computers that are targeted for the upgrade should have this happen automatically at the next update cycle.

Deploying FileZilla with Configuration Manager 2012 R2

Some time ago I created an System Center Configuration Manager (SCCM) application to deploy FileZilla for users. Recently I created an updated application that upgraded FileZilla (using the usual ‘uninstall the old version, then install the new version’ method), which gave me an error when the application was upgraded:

Application Upgrade Filaed

Unable to make changes to your software

with the reported error being ‘The software change returned error code 0x87D00325 (-2016410843).

In addition, attempting to uninstall the application using the SCCM Software Center on the client machine gave a ‘Removal Failed’ with the same error details:

FileZilla Removal Failed

Digging through the AppEnforce log file on the client, an error was reported when attempting to uninstall the application as part of the change:

Prepared command line: "C:\Program Files (x86)\FileZilla FTP Client\uninstall.exe" /S
Post install behavior is BasedOnExitCode
Waiting for process 1616 to finish.  Timeout = 120 minutes.
Process 1616 terminated with exitcode: 0
Looking for exit code 0 in exit codes table...
Matched exit code 0 to a Success entry in exit codes table.
Performing detection of app deployment type FileZilla 3.13.0 x86
Discovered application
App enforcement completed (0 seconds) for App DT "FileZilla 3.13.0 x86"

This indicates that the uninstall process completed, but the application was detected as still being installed immediately afterwards. This is due to the way that uninstallation is performed, with the uninstall process triggered by SCCM spawning another process to actually perform the uninstallation, while the original process terminates. This means that the process that SCCM is waiting on to complete, does so almost immediately, while the actual work to perform the uninstallation is on-going, hence the reason that SCCM then detects the application as still present on the client system.

To get around the issue, we need to wait a period of time after the uninstallation is triggered to allow for completion before allowing SCCM to perform its detection. In order to do this, I’m using the ‘sleep.exe’ program that comes with the Windows 2003 Resource Kit Tools (available at http://www.microsoft.com/en-us/download/details.aspx?id=17657) to achieve this.

The items we’ll need to be copied to an appropriate location for deployment are:

  • The FileZilla installer
  • The sleep.exe program
  • A batch file to perform the uninstallation
  • An SCCM application that uses the batch file

The batch file should contain two lines:

"%ProgramFiles%\FileZilla FTP Client\uninstall.exe" /S
Sleep 20

This runs the uninstaller as before, then pauses for 20 seconds to allow the system to catch up. 20 seconds is long enough for our client machines, YMMV.

The SCCM application should then be created as normal:

Manually specify the application information:

Manually specify the application information

Provide a name for the application and optionally a publisher, version etc.:

Provide a name for the application and optionally a publisher, version etc.

Provide appropriate information for the Application Catalog:

Provide appropriate information for the Application Catalog

Click ‘Add…’ to add a Deployment Type to the application:

Click ‘Add…’ to add a Deployment Type to the application

Select ‘Manually specify the deployment type information’:

Select ‘Manually specify the deployment type information’

Provide a deployment type name and optionally administrator comments:

Provide a deployment type name and optionally administrator comments

For the content page of the wizard, specify the content location, for the installation program, specify ‘FileZilla_X.XX.X_win32-setup.exe /S’ and for the uninstall program, specify the batch file that was created previously:

For the content page of the wizard, specify the content location, installation program and uninsatll program

For the detection rules, we used the file version of the ‘filezilla.exe’ program file that gets installed, which seemed to work well:

For the detection rules, use the version of the 'filezilla.exe' file

In addition, if required, supersedence information can be added to the application.

The procedure above provided a robust SCCM application that can be installed and uninstalled without issue and can also be upgraded to a later version without issue. It should be noted that this method works for other applications with the same sort of uninstallation routine, e.g. Notepad++.

Windows Home Server 2011 Backup of UEFI/GPT Windows 8.1

Since upgrading to Windows 8.1 at home, I’ve had issues with backing up the computer using my Home Server (not that I helped by introducing a GPT disk and a UEFI rig at the same time…). The symptoms were that the client backup process appeared stuck at 1% progress for a long time before eventually failing.

I finally got a bit of time to look at the machines in question over the weekend and here are the issues that appeared to be causing problems for which I needed to find solutions:

  • The PC is a UEFI machine.
  • The PC uses a GPT hard disk.
  • A VSS error was appearing in the event log on the PC being backed up.
  • A CAPI2 error was appearing in the event log on the PC being backed up.

The first two issues were dealt with quickly by a hotfix for Home Server 2011: http://support.microsoft.com/kb/2781272. Note that the same issue also affects Windows Storage Server 2008 R2 Essentials and Windows Small Business Server 2011 Essentials. More information for these platforms can be found at http://support.microsoft.com/kb/2781278

The VSS error manifests as the event 8194 appearing in the event log of the PC that the backup attempt is run on:

VSS Error 8194

Volume Shadow Copy Service error: Unexpected error querying for the IVssWriterCallback interface. hr = 0x80070005, Access is denied.
. This is often caused by incorrect security settings in either the writer or requestor process.

Examination of the binary data for event 8194 indicates that ‘NT AUTHORITY\NETWORK SERVICE’ is account receiving the access denied error:

VSS Error Binary Data

Event 8194 is caused by the inability of one or more VSS system writers to communicate with the backup application VSS requesting process via the COM calls exposed in the IVssWriterCallback interface. The issue is not caused by a functional error in the backup application, but rather is a security issue caused by the selected VSS writers running as a service under the ‘Network Service’ (or ‘Local Service’) account, not the Local System or Administrator account. By default, in order for a Windows service to perform a COM activation it must be running as Local System or as a member of the Administrators group.

There are two ways to fix this issue; either change the account under which the erroring VSS writers are running from Network Service to Local System (at which point the service will be running with higher privileges than was originally designed), or add the Network Service account to the list of default COM activation permissions allowing this user account to activate the IVssWrtierCallback interface. This latter option is the preferred one to use and can be performed by completing the following steps:

  1. Run dcomcnfg to open the Component Services dialog.
  2. Expand Component Services, then Computers and then right-click on My Computer and select Properties:
    Component Services
  3. Select the COM Security tab and click the Edit Default… button in the Access Permissions area at the top of the dialog.
  4. Click Add and enter Network Service as the account to be added.
  5. Click OK and ensure that only the Local Access checkbox is selected.
  6. Click OK to close the Access Permission dialog, then clock OK to close the My Computer Properties dialog.
  7. Close the Component Services Dialog and restart the computer to apply the changes. Event 8194 should not longer appear in the event log for the Home Server backup.

The CAPI2 error manifests as the event 513 appearing in the event log of the PC that the backup attempt is run on:

CAPI2 Error 513

Cryptographic Services failed while processing the OnIdentity() call in the System Writer Object.
Details: AddLegacyDriverFiles: Unable to back up image of binary Microsoft Link-Layer Discovery Protocol.
System Error:
Access is denied.
.

The Microsoft Link-Layer Discovery Protocol binary is located at C:\Windows\System32\drivers\mslldp.sys. During the backup process, the VSS process running under the Network Service account calls cryptcatsvc!CSystemWriter::AddLegacyDriverFiles(), which enumerates all the driver records in Service Control Manager database and tries opening each one of them. The function fails on the MSLLDP record with an ‘Access Denied’ error.

The mslldp.sys configuration registry key is HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MsLldp and the binary security descriptor for the record is located at HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MsLldp\Security.

Examining the security descriptor for mslldp using AccessChk (part of the SysInternals suite, available at http://technet.microsoft.com/en-us/sysinternals/bb664922) gives the following result (note: your security descriptor may differ from the permissions below):

C:\>accesschk.exe -c mslldp

Accesschk v5.2 - Reports effective permissions for securable objects
Copyright (C) 2006-2014 Mark Russinovich
Sysinternals - www.sysinternals.com

mslldp
  RW NT AUTHORITY\SYSTEM
  RW BUILTIN\Administrators
  RW S-1-5-32-549
  R  NT SERVICE\NlaSvc

Checking the access rights of another driver in the same location gives the following result:

C:\>accesschk.exe -c mspclock

Accesschk v5.2 - Reports effective permissions for securable objects
Copyright (C) 2006-2014 Mark Russinovich
Sysinternals - www.sysinternals.com

mspclock
  RW NT AUTHORITY\SYSTEM
  RW BUILTIN\Administrators
  R  NT AUTHORITY\INTERACTIVE
  R  NT AUTHORITY\SERVICE

In the case of mslldp.sys, there is no entry for ‘NT AUTHORITY\SERVICE’, therefore no service account will have access to the mslldp driver, hence the error.

To correct this issue, complete the following steps:

  1. From an elevated command prompt, run
    sc sdshow mslldp
    You should receive the following output, or something similar:
    D:(D;;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;BG)(A;;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;SY)(A;;CCDCLCSWRPDTLOCRSDRCWDWO;;;BA)(A;;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;SO)(A;;LCRPWP;;;S-1-5-80-3141615172-2057878085-1754447212-2405740020-3916490453)S:(AU;FA;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;WD)
    Note: Details on Security Descriptor Definition Language can be found at http://msdn.microsoft.com/en-us/library/windows/desktop/aa379567(v=vs.85).aspx
  2. Add the ‘NT AUTHORITY\SERVICE’ entry immediately before the S::(AU;FA;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;WD) entry and use this with the sdset option, for example using the output from the sdshow option above, this would be:
    sc sdset MSLLDP D:(D;;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;BG)(A;;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;SY)(A;;CCDCLCSWRPDTLOCRSDRCWDWO;;;BA)(A;;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;SO)(A;;LCRPWP;;;S-1-5-80-3141615172-2057878085-1754447212-2405740020-3916490453)(A;;CCLCSWLOCRRC;;;SU)S:(AU;FA;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;WD)
    Note: The above should all be on a single line when entering/pasting it; do not include line breaks in the command. It’s also important to use the output you receive from the command rather than that which I got as yours may be different.
  3. Check the access permissions again with:
    accesschk.exe -c mslldp
    You should now see a list of permissions that includes ‘NT AUTHORITY\SERVICE’:
    C:\>accesschk.exe -c mslldp
  4. Accesschk v5.2 - Reports effective permissions for securable objects
    Copyright (C) 2006-2014 Mark Russinovich
    Sysinternals - www.sysinternals.com

    mslldp
      RW NT AUTHORITY\SYSTEM
      RW BUILTIN\Administrators
      RW S-1-5-32-549
      R  NT SERVICE\NlaSvc
      R  NT AUTHORITY\SERVICE

  5. Now that the ‘NT AUTHORIT\SERVICE’ permission has been added, Network Service should be able to access the mslldp.sys driver file.

Following the above fixes, my computer is now being successfully backed up using Home Server 2011.

Upgrading Data Protection Manager from 2012 SP1 to 2012 R2

Our recent upgrade of Data Protection Manager (DPM) from 2012 SP1 to 2012 R2 generated one issue which wasn’t mentioned by the documentation on Technet.

The documentation on Technet is good and the procedure for upgrading DPM was very quick and easy. Pay attention to the Upgrade Sequencing documentation as not following this can result in component failure for which no rollback procedure exists.

Once the upgrade is complete, the agents on each protected client need to be upgraded and a consistency check run against all protected sources. Depending upon the volume of data protected, this may take an extended period of time. Following this procedure I saw two errors, one on each of our DPM servers.

In each case, the DPM database that was being protected on the other DPM server would not show as consistent. Running a consistency check would change the status icon to green for a few seconds before the consistency check would again fail.

The error was occurring because during the DPM upgrade procedure, the DPM database names had been changed. Originally the DPM database names were of the form

DPMDB_ServerName

Following the upgrade, the DPM database names were of the form

DPMDB_ServerNameGUID

In each case it was a simple task to modify the protection group to include the new database name and once a backup had been taken, remove the protection on the original database name.

Edit: I've come across another issue that occurs during the upgrade - the notification settings had been reset to remove the e-mail addresses that I had entered. This means that we were no longer receiving e-mail notifications for DPM issues. Again this was quick and easy to resolve, but it would have been useful for the upgrade documentation to flag it.

Cross Flashing a Dell PERC H200 BIOS to Support Larger SATA Disks

The latest firmware for the Dell PERC H200 still doesn’t support SATA disks of greater than 2.2TB. In fact the card cannot even detect SATA drives that are larger than this.  As the PERC H200 is essentially a rebadged LSI 9211-8i card however, the firmware and BIOS from that card can be flashed onto the PERC H200 to provide support for larger SATA hard drives.

The procedure is as follows:

  1. Download the latest 9211-8i firmware package from LSI. I got my copy from http://www.lsi.com/products/storagecomponents/Pages/LSISAS9211-8i.aspx (click on the ‘Support & Downloads’ tab, then expand the firmware section). I downloaded the ‘9211_8i_Package_P16_IR_IT_Firmware_BIOS_for_MSDOS_Windows’ package and extracted the contents.
  2. Copy the required files from the extracted archive onto bootable media. I created a Windows 98SE boot USB stick and copied the files onto it. The required files are: 
    sas2flsh.exe – this is the  flash application. Copy the version from the sas2flash_dos_rel folder if using a dos boot disk.
    2118ir.bin – this is the firmware for the 9211-8i.
    mptsas2.rom – this is the card’s BIOS.
  3. Boot the server containing the PERC H200 from the bootable media. I’d recommend disconnecting any drives from the RAID card before flashing the firmware and BIOS.
  4. Once booted, change to the folder containing the files copied to the media, above, and issue the following commands
    sas2flsh –o –f 2118ir.bin
    sas2flsh –o –b mptsas2.rom
    sas2flsh –o –reset
    Each command should report success before you move onto the next one. If any indicate a failure, double check that you copied the correct files onto the bootable media.
  5. Reboot the server and test the card.

The above procedure allowed the H200 I was using to  detect and use two 3TB disks.

Error during SharePoint 2010 Upgrade “Minimal master page failed to provision: could not find master page gallery”

During a recent SharePoint upgrade using the database attach method, I saw the following recorded in the upgrade error log:

[powershell] [PerWebUpgradeAction (4.0.10.0)] [INFO] [13/04/2011 15:55:48]: SPSite Url=http://intranet.domain.com
[powershell] [PerWebUpgradeAction (4.0.10.0)] [WARNING] [13/04/2011 15:55:48]: Minimal master page failed to provision: could not find master page gallery.
[powershell] [PerWebUpgradeAction (4.0.10.0)] [INFO] [13/04/2011 15:55:48]: SPSite Url=http://intranet.domain.com
[powershell] [PerWebUpgradeAction (4.0.10.0)] [WARNING] [13/04/2011 15:55:48]: V4 master page failed to provision: could not find master page gallery.
[powershell] [PerWebUpgradeAction (4.0.10.0)] [INFO] [13/04/2011 15:55:48]: SPSite Url=http://intranet.domain.com

Further information was provided in the upgrade log that accompanies the error log:

[powershell] [PerWebUpgradeAction (4.0.10.0)] [INFO] [13/04/2011 15:55:48]: Setting UIVersion metadata on existing default master page
[powershell] [PerWebUpgradeAction (4.0.10.0)] [INFO] [13/04/2011 15:55:48]: Provisioning minimal master page in site: Site1
Site Url: http://intranet.domain.com/Site1
[Powershell] [PerWebUpgradeAction (4.0.10.0)] [WARNING] [13/04/2011 15:55:48]: Minimal master page failed to provision: could not find master page gallery.
[powershell] [PerWebUpgradeAction (4.0.10.0)] [INFO] [13/04/2011 15:55:48]: Provisioning V4 master page in site: Site1
Site Url: http://intranet.domain.com/Site1
[powershell] [PerWebUpgradeAction (4.0.10.0)] [WARNING] [13/04/2011 15:55:48]: V4 master page failed to provision: could not find master page gallery.
[powershell] [PerWebUpgradeAction (4.0.10.0)] [INFO] [13/04/2011 15:55:48]: Setting UIVersion metadata on existing default master page
[powershell] [PerWebUpgradeAction (4.0.10.0)] [WARNING] [13/04/2011 15:55:48]: Could not find master page gallery. Metadata on default.master not set.

The farm in question was originally a SharePoint 2003 farm that had been in-place upgraded to SharePoint 2007. The site recorded in the upgrade error log had been created in SharePoint 2003, but had not had its template selected at the time of original creation, allowing the first site user to pick the template to use. Critically however, this had never been done.

Once the database upgrade had been completed, navigating to the site gave me the expected screen to allow me to select the site template to use. Picking a template and clicking OK however produced a ‘file not found’ error.

The moral of the story seems to be ensure that any sites created without selecting a site template should either be completed, or deleted before the upgrade process starts. Note that you can still delete a site that has been upgraded to 2010 in the above state using PowerShell. Detection of sites such as these in SharePoint 2007 is straightforward using stsadm –o enumallwebs – any site that doesn’t have a template shown against it is in this state.