Windows Home Server 2011 Backup of UEFI/GPT Windows 8.1

Since upgrading to Windows 8.1 at home, I’ve had issues with backing up the computer using my Home Server (not that I helped by introducing a GPT disk and a UEFI rig at the same time…). The symptoms were that the client backup process appeared stuck at 1% progress for a long time before eventually failing.

I finally got a bit of time to look at the machines in question over the weekend and here are the issues that appeared to be causing problems for which I needed to find solutions:

  • The PC is a UEFI machine.
  • The PC uses a GPT hard disk.
  • A VSS error was appearing in the event log on the PC being backed up.
  • A CAPI2 error was appearing in the event log on the PC being backed up.

The first two issues were dealt with quickly by a hotfix for Home Server 2011: http://support.microsoft.com/kb/2781272. Note that the same issue also affects Windows Storage Server 2008 R2 Essentials and Windows Small Business Server 2011 Essentials. More information for these platforms can be found at http://support.microsoft.com/kb/2781278

The VSS error manifests as the event 8194 appearing in the event log of the PC that the backup attempt is run on:

VSS Error 8194

Volume Shadow Copy Service error: Unexpected error querying for the IVssWriterCallback interface. hr = 0x80070005, Access is denied.
. This is often caused by incorrect security settings in either the writer or requestor process.

Examination of the binary data for event 8194 indicates that ‘NT AUTHORITY\NETWORK SERVICE’ is account receiving the access denied error:

VSS Error Binary Data

Event 8194 is caused by the inability of one or more VSS system writers to communicate with the backup application VSS requesting process via the COM calls exposed in the IVssWriterCallback interface. The issue is not caused by a functional error in the backup application, but rather is a security issue caused by the selected VSS writers running as a service under the ‘Network Service’ (or ‘Local Service’) account, not the Local System or Administrator account. By default, in order for a Windows service to perform a COM activation it must be running as Local System or as a member of the Administrators group.

There are two ways to fix this issue; either change the account under which the erroring VSS writers are running from Network Service to Local System (at which point the service will be running with higher privileges than was originally designed), or add the Network Service account to the list of default COM activation permissions allowing this user account to activate the IVssWrtierCallback interface. This latter option is the preferred one to use and can be performed by completing the following steps:

  1. Run dcomcnfg to open the Component Services dialog.
  2. Expand Component Services, then Computers and then right-click on My Computer and select Properties:
    Component Services
  3. Select the COM Security tab and click the Edit Default… button in the Access Permissions area at the top of the dialog.
  4. Click Add and enter Network Service as the account to be added.
  5. Click OK and ensure that only the Local Access checkbox is selected.
  6. Click OK to close the Access Permission dialog, then clock OK to close the My Computer Properties dialog.
  7. Close the Component Services Dialog and restart the computer to apply the changes. Event 8194 should not longer appear in the event log for the Home Server backup.

The CAPI2 error manifests as the event 513 appearing in the event log of the PC that the backup attempt is run on:

CAPI2 Error 513

Cryptographic Services failed while processing the OnIdentity() call in the System Writer Object.
Details: AddLegacyDriverFiles: Unable to back up image of binary Microsoft Link-Layer Discovery Protocol.
System Error:
Access is denied.
.

The Microsoft Link-Layer Discovery Protocol binary is located at C:\Windows\System32\drivers\mslldp.sys. During the backup process, the VSS process running under the Network Service account calls cryptcatsvc!CSystemWriter::AddLegacyDriverFiles(), which enumerates all the driver records in Service Control Manager database and tries opening each one of them. The function fails on the MSLLDP record with an ‘Access Denied’ error.

The mslldp.sys configuration registry key is HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MsLldp and the binary security descriptor for the record is located at HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MsLldp\Security.

Examining the security descriptor for mslldp using AccessChk (part of the SysInternals suite, available at http://technet.microsoft.com/en-us/sysinternals/bb664922) gives the following result (note: your security descriptor may differ from the permissions below):

C:\>accesschk.exe -c mslldp

Accesschk v5.2 – Reports effective permissions for securable objects
Copyright (C) 2006-2014 Mark Russinovich
Sysinternals – www.sysinternals.com

mslldp
  RW NT AUTHORITY\SYSTEM
  RW BUILTIN\Administrators
  RW S-1-5-32-549
  R  NT SERVICE\NlaSvc

Checking the access rights of another driver in the same location gives the following result:

C:\>accesschk.exe -c mspclock

Accesschk v5.2 – Reports effective permissions for securable objects
Copyright (C) 2006-2014 Mark Russinovich
Sysinternals – www.sysinternals.com

mspclock
  RW NT AUTHORITY\SYSTEM
  RW BUILTIN\Administrators
  R  NT AUTHORITY\INTERACTIVE
  R  NT AUTHORITY\SERVICE

In the case of mslldp.sys, there is no entry for ‘NT AUTHORITY\SERVICE’, therefore no service account will have access to the mslldp driver, hence the error.

To correct this issue, complete the following steps:

  1. From an elevated command prompt, run
    sc sdshow mslldp
    You should receive the following output, or something similar:
    D:(D;;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;BG)(A;;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;SY)(A;;CCDCLCSWRPDTLOCRSDRCWDWO;;;BA)(A;;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;SO)(A;;LCRPWP;;;S-1-5-80-3141615172-2057878085-1754447212-2405740020-3916490453)S:(AU;FA;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;WD)
    Note: Details on Security Descriptor Definition Language can be found at http://msdn.microsoft.com/en-us/library/windows/desktop/aa379567(v=vs.85).aspx
  2. Add the ‘NT AUTHORITY\SERVICE’ entry immediately before the S::(AU;FA;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;WD) entry and use this with the sdset option, for example using the output from the sdshow option above, this would be:
    sc sdset MSLLDP D:(D;;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;BG)(A;;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;SY)(A;;CCDCLCSWRPDTLOCRSDRCWDWO;;;BA)(A;;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;SO)(A;;LCRPWP;;;S-1-5-80-3141615172-2057878085-1754447212-2405740020-3916490453)(A;;CCLCSWLOCRRC;;;SU)S:(AU;FA;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;WD)
    Note: The above should all be on a single line when entering/pasting it; do not include line breaks in the command. It’s also important to use the output you receive from the command rather than that which I got as yours may be different.
  3. Check the access permissions again with:
    accesschk.exe -c mslldp
    You should now see a list of permissions that includes ‘NT AUTHORITY\SERVICE’:
    C:\>accesschk.exe -c mslldp
  4. Accesschk v5.2 – Reports effective permissions for securable objects
    Copyright (C) 2006-2014 Mark Russinovich
    Sysinternals – www.sysinternals.com

    mslldp
      RW NT AUTHORITY\SYSTEM
      RW BUILTIN\Administrators
      RW S-1-5-32-549
      R  NT SERVICE\NlaSvc
      R  NT AUTHORITY\SERVICE

  5. Now that the ‘NT AUTHORIT\SERVICE’ permission has been added, Network Service should be able to access the mslldp.sys driver file.

Following the above fixes, my computer is now being successfully backed up using Home Server 2011.

Remote Mounting an ISO Image from Hyper-V

Building a Hyper-V virtual machine from scratch almost always seems to involve mounting an ISO image at some point during the installation process. I suspect that like us, many other organisations already have a network location in which we store ISO images. The ability to mount an ISO image from our usual network location saves us having to copy the ISO images to the local Hyper-V servers.

The ability to remote mount an ISO image requires that a couple of configuration changes are made. Attempting to remote mount an ISO image without making the configuration changes usually results in an err along the lines of

Inserting the disk failed

Failed to add device ‘Microsoft Virtual CD/DVD Disk’

The file ‘\\RemoteServer\Share\ISO_Image.iso’ does not have the required security settings. Error: ‘General access denied error’

There are two configuration changes that need to be made. The first is ensuring that the Hyper-V hosts can access the share itself that contains the ISO images. As a workaround, you can always grant ‘Everyone’ read access to the share. If you want to control access to individual servers, you need to specify the Active Directory computer object for each server you want to give access to.

The second configuration change that is required to to grant Constrained Delegation on the virtual host objects in Active Directory:

  • Log onto a domain controller and open Active Directory Users and Computers from the Administrative Tools menu, or open the remote administration Active Directory tools from another server or client
  • Locate the Hyper-V host computer object
  • Right-click the object and Select Properties from the context menu
  • Select the Delegation tab
  • Select the ‘Trust this computer for delegation to specified services only’ radio button and then the ‘Use any authentication protocol’ radio button
  • Click the ‘Add’ button:
    Active Directory computer object properties
  • The ‘Add Services’ dialog will open:
    Add Services dialog
  • Click the ‘Users or Computers’ button and add the remote server hosting the ISO images. Click OK
  • Select the ‘cifs’ service type from the list shown:
    Add Services dialog, cifs service selected
  • Click OK to close the dialog, the computer object properties should look like the following:
    Active Directory computer object properties, cifs service added
  • Click OK to apply the change
  • Repeat the above steps for any other Hyper-V hosts servers.

You should now be able to remote mount ISO images from the server specified.

SharePoint 2007 with multiple domains

SharePoint can use multiple domains for user authentication, however I recently came across an issue when setting up an extranet using this scenario.

The steps for setting up SharePoint 2007 to use multiple domains for user authentication are relatively simple:

  • Add the second domain as a user profile source in the SSP
  • Issue the following stsadm command to encrypt the password for the account that is used to access the second forest or domain on each web front end server in the farm:
    stsadm –o setapppassword –password <password>
  • Issue the following stsadm command to add multiple domains to the people picker search list:
    stsadm –o set property –pn peoplepicker-searchadforests –pv domain:<original resource domain>;domain:<secondary domain>,<domain>\<username>,<password> –url <web application URL>

At this point I could successfully add users from the second domain to the security groups in a site collection or site, however when I attempted to log in as one of these users, I received a “500 – Internal server error”. Logging in as a user from the original resource domain worked fine however.

Modifying the web.config file for the web application to set CallStack to true and CustomErrors to Off didn’t give me any further information, at least in Internet Explorer 8, as I still saw the same “500 – Internal server error”, however viewing the web application in Firefox gave me a somewhat cryptic error:

Multi-domain login issue with FF - error

This error translates as STATUS_AUTHENTICATION_FIREWALL_FAILED, however the firewall wasn’t an issue in this scenario.

The solution was to grant the machine accounts for each of the web servers in the SharePoint farm an extra right in AD. The steps were:

  • On a domain controller, start “Active Directory Users and Computers”
  • On the view menu, ensure that the ‘Advanced Features’ option is checked
  • Locate the computer account for first of the web servers in the SharePoint farm
  • For the computer account, right-click and select properties and then click the security tab
  • Add the <external domain>\Domain Users group to the security list and grant the ‘Allowed to authenticate’ right. Click OK to close the dialog.
  • Repeat these steps for the computer accounts of all the other web servers in the farm

This resolved the issue and allowed users from the second domain to log into SharePoint.

Note that to achieve multiple domain authentication for SharePoint 2010, the same stsadm commands are used, and I therefore believe that the same issue may occur, however the above solution should also work for SharePoint 2010.

Adding domain users to a local machine group using GPO

To add domain users to a local machine group using Group Policy, we need to use the Restricted Groups feature.  For the example shown below, I’ll be using a Windows Server 2003 domain functional level.

  1. Create a new global/universal security group in Active Directory to contain the users which you wish to add to the local group on the target machines.
  2. Make the domain users you wish to add to the local group on the target machines members of this new group.
  3. Open Group Policy Editor and navigate to the OU where the target machines reside.  For example, if we have a ‘Desktops’ OU which contains the machines to which we wish to add the domain users, that is the location of the group policy we need to edit or create.
  4. If a Group Policy already exists for the OU selected, edit the Group Policy.  If there is no Group Policy for the OU selected, create a new group policy and then edit it.
  5. Within the Group Policy, navigate to Computer Configuration –> Policies –> Windows Settings –> Security Settings –> Restricted Groups
  6. Right-click on either Restricted Groups in the left pane of the Group Policy Management Editor, or in the right pane, and select Add Group.
  7. The ‘Add Group’ window appears:
    RestrictedGroupsAddGroup
  8. Click the ‘Browse’ button to open the ‘Select Groups’ window and select the group created in step 1, above, then click OK.  Click OK on the Add group window.
  9. The Properties window for the Restricted Group appears:
    RestrictedGroupsProperties
  10. The Properties Window has two membership areas; ‘Members of this group’ and ‘This group is a member of’.  Adding users to the ‘Members of this group’ option would add domain users to the Active Directory group created in step 1, and would remove any members of that group already there. As we added the required users to the group created in step 2, we shouldn’t need to use this option. Adding group names to the ‘This group is a member of’ option adds the security group and its members to the group(s) specified.
  11. Click Add next to the ‘This group is a member of’ option and enter the names of the local groups you wish to have the domain users added to (e.g. Administrators, Users, Performance Monitor Users etc.) and click OK.
  12. To test that the above steps have worked, log onto one of the target machines, run ‘gpupdate’ from a command prompt and check the local groups specified above for the new members.

PerformancePoint message ‘unable to conenct to the specified server’ from the Dashboard Designer

I’ve recently been looking at PerformancePoint again. During the setup of my initial PerformancePoint test server, I hit an issue when looking at the Dashboard Designer:

Unable to connect to the specified server. Make sure the address is correct.

The error occurs after loading the Dashboard Designer and following the instructions onscreen to ‘refresh to load dashboards’. When I hit the refresh button, the message shown above was displayed.

I found a blog post by Nick Barclay which suggested a few possible solutions to the issue, but nothing I tried from the list of suggestions seemed to help me.

The error means that you cannot connect to the Monitoring Web Service which in configured in the server tab of the Dashboard Designer Options. In an attempt to work out why I was seeing the error, I tried connecting directly to the web service and saw the following:

Server Error in '/WebService' Application. Configuration Error.

The error indicated that a required DLL was not in the GAC.  The DLL in question is part of the ASP.NET AJAX 1.0 installation, reinstalling ASP.NET AJAX 1.0 fixed the issue straight away.

“Access is Denied” when crawling content on MOSS 2007 hosted on Windows Server 2008

One of the SharePoint farms we’ve built recently runs on Windows Server 2008  and SQL Server 2008.  As usual, the installation is a least privileged account setup, with individual accounts running the various services and app pools. The farm is also patched to the latest level.

We’ve experienced one or two issues with this setup, but the most persistent one has been to do with crawling.  When crawls run, they would consistently fail with the following error in the Event viewer:

“The start address <https://site.domain.com> cannot be crawled.

Context: Application ‘SharedServices1’, Catalog ‘Portal_Content’

Details:
    Access is denied. Check that the Default Content Access Account has access to this content, or add a crawl rule to crawl this content.   (0x80041205)”

In addition, the following error appeared in the SharePoint logs:

***** Couldn’t retrieve server https://site.domain.com policy, hr = 80041205 – File:d:\office\source\search\search\gather\protocols\sts3\sts3util.cxx Line:548

And the crawl logs showed only errors, each having the following description:

“Access is denied. Verify that either the Default Content Access Account has access to this repository, or add a crawl rule to crawl this repository. If the repository being crawled is a SharePoint repository, verify the account you are using has “Full Read” permissions on the SharePoint Web Application being crawled.(The item was deleted because it was either not found or the crawler was denied access to it.)”

We’d checked all of the usual suspects including web application permissions for the account used by search, database permissions etc with no success.

The solution was to disable the loopback check on the servers hosting SharePoint. Adding the hostnames served to the BackConnectionHostNames list in the registry on the SharePoint servers wasn’t enough, the loopback check had to be completely disabled.

As an aside, another issue we’d experienced with an InfoPath form with code behind failing to load correctly on these servers was also solved disabling the loopback check on these servers.

For instructions on disabling the loopback check, see KB896861.

 

Installation of SCVMM 2008 beta disables non-admin access to remote machines via Hyper-V manager

Yesterday I finally got around to installing SCVMM 2008 beta onto a virtual machine (mainly to help us with some virtual machine migrations we’ve got coming up).  I must say that I think SCVMM 2008 beta is very nice indeed!

On my Vista machine I use Tore Lervik’s Hyper-V Monitor Gadget for Windows Sidebar, and have done for some time.  With the number of virtual machines we run, I have found it an invaluable addition to my sidebar.

This morning however, when I tried to connect to one of the virtual machines listed by the gadget, I got an error message ‘An error occurred trying to find the virtual machine <GUID> on the server <servername>’.  In addition, when I tried to use Hyper-V manager, I received the error ‘The virtual machine management service is not available’.

We thought for a while that it was related to remote rights (WMI/DCOM) on the servers in question (well, technically it is…) and I spent a while trawling through John Howard’s articles relating to the required rights for remote management (well worth a read by the way).  Unfortunately even working through the articles didn’t solve my problem.

After a little more rummaging, it turns out that installation of the SCVMM agent onto the servers hosting the virtual machines I want to remotely manage is what is causing the problem.  Anyone who is a local admin on the servers in question can freely manage the remote virtual machines; if you’re not a local admin, you can’t.  There are two potential solutions to the problem:

  1. uninstall the SCVMM agent from the servers in question (which would no longer allow us to manage them from SCVMM)
  2. Make anyone who needs to remotely manage virtual machines a local administrator on the servers in question

Lets be honest, neither option is entirely appealing (it’s not that we don’t trust some of the people who need to remotely manage specific machines, I just always would prefer to work from a ‘minimum rights necessary’ point of view), but as we have some migrations coming soon for which SCVMM is going to really help, we’ve gone for the latter.

I hope that this is something that is corrected in the RTM version of SCVMM 2008!