Andy Dawson's Blog

The blog of Andy dawson

Offline Domain Join with Direct Access

I was recently in the position that I needed to rebuild a workstation at a remote location, but wanted to end up with it joined to the domain, and able to install software via the SCCM Software Center. Enter Offline Domain Join (djoin.exe)!

Offline Domain Join allows the creation of a machine account and the establishment of a trust relationship between a computer running Windows and a Domain. As part of the process, group policy information can also be transferred to the machine that will be joined to the domain.

Assuming Direct Access is available, the appropriate group policy information for Direct Access can be transferred as part of the process, and this should then allow the remote machine to establish a connection to the domain and from there all remaining group policy information can be transferred, the Configuration Manager client installed etc.

Information on ‘djoin.exe’ including examples for use can be found at https://technet.microsoft.com/en-us/library/offline-domain-join-djoin-step-by-step

My scenario was:

  • The machine account already existed in the correct OU and was a member of the appropriate groups for Direct Access (the machine name had already been used; this was a rebuild) and therefore I needed to use the ‘/reuse’ parameter.
  • The only group policy information I wanted to transfer to the remote machine was for Direct Access. I anticipated that all other group policy information would be transferred automatically once a Direct Access connection had been established.

In my case, the command I used on the provisioning server were:

djoin /provision /domain domain.com /machine MyWorkstation /savefile MyWorkstation-blob.txt /reuse /policynames “Direct Access Client”

The resultant blob should be transferred securely – take note of what the TechNet page says on the matter:

The base64-encoded metadata blob that is created by the provisioning command contains very sensitive data. It should be treated just as securely as a plaintext password. The blob contains the machine account password and other information about the domain, including the domain name, the name of a domain controller, the security ID (SID) of the domain, and so on. If the blob is being transported physically or over the network, care must be taken to transport it securely.

On the remote workstation, the command I used was:

djoin /requestODJ /loadfile MyWorkstation-blob.txt /windowspath %SystemRoot% /localos

At this point you’re prompted to reboot the workstation. Once the reboot was complete, I left the machine for a few minutes to allow it to establish a connection, then signed in. Everything worked as anticipated and I could log in as a domain user and a Direct Access connection was established. Following a group policy update, the Configuration Manager client was transferred and installed, and a short time later the Software Center became available and I could add software made available from SCCM.

DPM Protection for Windows 10 Anniversary Edition

Attempting to add protection to a Windows 10 Anniversary Edition workstation recently failed with the DPM server showing the workstation as ‘unavailable’ when looking at the ‘Production Servers’ list in the console.

It appears that the upgrade to Anniversary Edition removes a file that the DPM agent relies on, ‘sisbkup.dll’, and that as a consequence the services cannot start on the protected workstation.

The resolution is to copy the ‘sisbkup.dll’ file from c:\Windows\System32 on an older version of Windows 10 into C:\Windows\System32 on the Anniversary Update machine and then retry the connection from DPM.

DPM Protection for Windows 10 Anniversary Edition

Attempting to add protection to a Windows 10 Anniversary Edition workstation recently failed with the DPM server showing the workstation as ‘unavailable’ when looking at the ‘Production Servers’ list in the console.

It appears that the upgrade to Anniversary Edition removes a file that the DPM agent relies on, ‘sisbkup.dll’, and that as a consequence the services cannot start on the protected workstation.

The resolution is to copy the ‘sisbkup.dll’ file from c:\Windows\System32 on an older version of Windows 10 into C:\Windows\System32 on the Anniversary Update machine and then retry the connection from DPM.

Web Application Proxy Failure Following Outage

Following a ‘hiccup’, involving a Web Application Proxy (WAP) server, internal services were no longer being published to the outside world.

After some investigation, both the ADFS and WAP services showed as stopped on the server. Attempting to start the ADFS service from the services console produced the following error:

Windows could not start the Active Directory Federation Service service on Local Computer.
Error 1064: An exception occurred in the service when handling the control request.

Under the System section of the Windows Event Log, the following error was shown:

Event ID: 7023
The Active Directory Federation Services service terminated with the following error:
An exception occurred in the service when handling the control request.

Followed a few moments later by the following error:

Event ID: 7023
The Web Application Proxy Service terminated with the following error:
A certificate is required to complete client authentication

Looking in the ‘AD FS’ section of the Event Log (under ‘Applications and Services Logs’), the following errors were shown (note that the first error was generally shown multiple times, followed by a single instance of the second error):

Event ID: 383
The Web request failed because the web.config is malformed.
User Action:
Fix the malformed data in the web.config file.
Exception details:
Root element is missing (C:\Windows\ADFS\Config\microsoft.identityServer.proxyservice.exe.config)
Root element is missing.

Followed by:

Event ID: 199
The federation server proxy could not be started.
Reason: Error retrieving proxy configuration from the Federation Service.
Additional Data
Exception details:
An error occurred when attempting to load the proxy configuration.

Checking the file at C:\Windows\ADFS\Config\microsoft.identityServer.proxyservice.exe.config showed that while the file size was still indicated as 2k, the file was blank.

I’ve seen a number of reports online indicating that WAP seems happy to chew up the contents of this configuration file following an outage, although I can find no information on why this might happen. If you have a backup of the file in question, it should be a simple matter to restore this file and restart the ADFS and WAP services to restore service. If you don’t, and have no other example server from which you can pull a similar copy of the file then the following steps must be taken:

  1. Remove the Web Application Proxy role from the server. Once this is complete, a reboot will be required.
  2. Re-add the Web Application Proxy role to the server.
  3. Once this is complete, initiate the configuration wizard.
  4. Use the same configuration parameters as you used when configuring the service initially, namely federation service name (e.g. federation.domain.com), local admin details for the federation server and the federation certificate (unless you’ve replaced the certificate used, in which case obviously you should use the new certificate details); you noted those down during initial configuration, right?
  5. Once configuration is complete, the Remote Access Management Console should open automatically. All of your publishing rules should still be in place, and your published services should be available immediately.

For reference, here’s a sample config file, from which you should be able to reconstruct an appropriate file for your service:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <configSections>
    <section name="microsoft.identityServer.proxyservice" type="Microsoft.IdentityServer.Management.Proxy.Configuration.ProxyConfiguration, Microsoft.IdentityServer.Management.Proxy, Version=6.3.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL" />
  </configSections>

  <microsoft.identityServer.proxyservice>
    <congestionControl latencyThresholdInMSec="8000" minCongestionWindowSize="64"
      enabled="true" connectionTimeoutInSec="60" />
    <connectionPool connectionPoolSize="200" scavengeInterval="5" />
    <diagnostics eventLogLevel="15" />
    <host tlsClientPort="49443" httpPort="80" httpsPort="443" name="federation.domain.com" />
    <proxy address="" />
    <trust thumbprint="1234567890ABCDEF1234567890ABCDEF12345678"
      proxyTrustRenewPeriod="21600" />
  </microsoft.identityServer.proxyservice>
  <!-- <system.serviceModel>
    <diagnostics>
      <messageLogging logEntireMessage="true"
              logMessagesAtServiceLevel="true"
              logMessagesAtTransportLevel="true">
      </messageLogging>
    </diagnostics>
  </system.serviceModel> -->
</configuration>

 

Web Application Proxy Failure Following Outage

Following a ‘hiccup’, involving a Web Application Proxy (WAP) server, internal services were no longer being published to the outside world.

After some investigation, both the ADFS and WAP services showed as stopped on the server. Attempting to start the ADFS service from the services console produced the following error:

Windows could not start the Active Directory Federation Service service on Local Computer.
Error 1064: An exception occurred in the service when handling the control request.

Under the System section of the Windows Event Log, the following error was shown:

Event ID: 7023
The Active Directory Federation Services service terminated with the following error:
An exception occurred in the service when handling the control request.

Followed a few moments later by the following error:

Event ID: 7023
The Web Application Proxy Service terminated with the following error:
A certificate is required to complete client authentication

Looking in the ‘AD FS’ section of the Event Log (under ‘Applications and Services Logs’), the following errors were shown (note that the first error was generally shown multiple times, followed by a single instance of the second error):

Event ID: 383
The Web request failed because the web.config is malformed.
User Action:
Fix the malformed data in the web.config file.
Exception details:
Root element is missing (C:\Windows\ADFS\Config\microsoft.identityServer.proxyservice.exe.config)
Root element is missing.

Followed by:

Event ID: 199
The federation server proxy could not be started.
Reason: Error retrieving proxy configuration from the Federation Service.
Additional Data
Exception details:
An error occurred when attempting to load the proxy configuration.

Checking the file at C:\Windows\ADFS\Config\microsoft.identityServer.proxyservice.exe.config showed that while the file size was still indicated as 2k, the file was blank.

I’ve seen a number of reports online indicating that WAP seems happy to chew up the contents of this configuration file following an outage, although I can find no information on why this might happen. If you have a backup of the file in question, it should be a simple matter to restore this file and restart the ADFS and WAP services to restore service. If you don’t, and have no other example server from which you can pull a similar copy of the file then the following steps must be taken:

  1. Remove the Web Application Proxy role from the server. Once this is complete, a reboot will be required.
  2. Re-add the Web Application Proxy role to the server.
  3. Once this is complete, initiate the configuration wizard.
  4. Use the same configuration parameters as you used when configuring the service initially, namely federation service name (e.g. federation.domain.com), local admin details for the federation server and the federation certificate (unless you’ve replaced the certificate used, in which case obviously you should use the new certificate details); you noted those down during initial configuration, right?
  5. Once configuration is complete, the Remote Access Management Console should open automatically. All of your publishing rules should still be in place, and your published services should be available immediately.

For reference, here’s a sample config file, from which you should be able to reconstruct an appropriate file for your service:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <configSections>
    <section name="microsoft.identityServer.proxyservice" type="Microsoft.IdentityServer.Management.Proxy.Configuration.ProxyConfiguration, Microsoft.IdentityServer.Management.Proxy, Version=6.3.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL" />
  </configSections>

  <microsoft.identityServer.proxyservice>
    <congestionControl latencyThresholdInMSec="8000" minCongestionWindowSize="64"
      enabled="true" connectionTimeoutInSec="60" />
    <connectionPool connectionPoolSize="200" scavengeInterval="5" />
    <diagnostics eventLogLevel="15" />
    <host tlsClientPort="49443" httpPort="80" httpsPort="443" name="federation.domain.com" />
    <proxy address="" />
    <trust thumbprint="1234567890ABCDEF1234567890ABCDEF12345678"
      proxyTrustRenewPeriod="21600" />
  </microsoft.identityServer.proxyservice>
  <!-- <system.serviceModel>
    <diagnostics>
      <messageLogging logEntireMessage="true"
              logMessagesAtServiceLevel="true"
              logMessagesAtTransportLevel="true">
      </messageLogging>
    </diagnostics>
  </system.serviceModel> -->
</configuration>

SharePoint Crawl Rules Appears to Ignore Some URL Protocols

I recently came across an issue relating to crawling people information in SharePoint and the use of crawl rules to exclude certain content.

The issue revolved around a requirement to exclude content contained within peoples’ MySites, but include user profile information so that people searches could still be conducted. The following crawl rule had been configured and was successfully excluding MySite content, but was also excluding the user profile data (crawled using the sps3s:// protocol):

URL Exclude or Include
https://mysite.domain.com/* Exclude

Using the crawl rule test facility indicated that while SharePoint treats http:// and https:// differently, https:// and sps3s:// appear to be treated the same as far as crawling is concerned, so if the above crawl rule is in place, items in the MySite root site collection, both with an https:// and sps3s:// prefix, will not be crawled, and therefore user profile data and people search will not be available:

Crawl rule test

[Screen shot from lab SharePoint 2010 system. however the same tests have been performed against SharePoint 2013 and 2016 with the same results]

In fact what is happening is that the sps3s:// prefix tells SharePoint which connector to use, and in the case of people search, this is translated into a call to a web service at the host specified, i.e. https://mysite.domain.com/_vti_bin/spscrawl.asmx, so the final call that is made is in fact to an https:// prefix, hence the reason that the people data is not crawled.

Replacing the above crawl rule with the following rule corrects the issue allowing people data stored in the MySite root site collection to be indexed and therefore be available for users to search:

URL Exclude or Include
https://mysite.domain.com/personal/* Exclude

WSUS Non-Functional After KB3159706 Installed

Consider the following scenario:

  • You have WSUS installed on either Windows Server 2012 or 2012 R2
  • You install KB3159706

In this situation, WSUS fails to start correctly and thus fails to function.

There are additional steps that are required to configure this update once it is installed. The steps can be found in KB3159706.

Note: If using database mirroring or the SUSDB is part of an AlwaysOn Availability Group, this must be undone before performing the actions described in KB3159706 as a schema update is required for the database.

SPWakeUp for SharePoint 2016

If you use SharePoint, you’ll know that some mechanism to wake up the hosted sites after the application pools are recycled overnight is very helpful (essential even) for the end user experience.

I’ve compiled a version of SPWakeUp for SharePoint 2016, which can be downloaded from https://onedrive.live.com/redir?resid=439F1389F21A368F%21496648.

If you want to compile this for yourself, this is the method I followed to get the above version:

  1. Grab a copy of the source code for SPWakeUp from https://spwakeup.codeplex.com/downloads/get/152410 and unpack it.
  2. Open the solution in Visual Studio (I used Visual Studio 2015) and allow the automatic upgrade.
  3. Replace the reference to the Microsoft.SharePoint.dll in the solution with one pointing to the SharePoint 2016 version. You’ll want to grab a copy from C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI on a SharePoint 2016 server.
  4. Modify the target framework for the application. I used 4.6.1 for the build above.
  5. Build either the debug or release version.

Steps Required to Configure WSUS to Distribute Windows 10 Version 1511 Upgrade

Microsoft recently made a hotfix available that patches WSUS on Windows Server 2012 and 2012 R2 to allow Windows 10 upgrade to version 1511. Installing the update is not, however, the only step that is required…

  1. Install the hotfix. This can be downloaded from https://support.microsoft.com/en-us/kb/3095113. Ensure that you pick the appropriate hotfix for the version of Windows Server on which you’re running WSUS. Note that if you’re running Windows Server 2012 R2, there’s also a pre-requisite install.
  2. Once the hotfix is installed and you’ve restarted your WSUS server, look in the ‘Products and Classifications’ option under the Classifications tab and ensure that the checkbox for upgrades is selected. This is not selected automatically for you:
    Upgrades Option
    Note that the upgrade files may take quite some time to download to your WSUS server at the next synchronisation.
  3. Add a MIME-Type for ‘.esd application/octet-stream’ in IIS on the WSUS server. To do this:
    Open IIS Manager
    Select the server name
    From the ‘IIS’ area in the centre of IIS Manager, open ‘MIME Types’
    Click ‘Add…’
    Enter the information above:
    Esd MIME Type
    Click OK to close the dialog.
    Note: Without this step, clients will fail to download the upgrade with the following error:
    Installation Failure: Windows failed to install the following update with error 0x8024200D: Upgrade to Windows 10 [SKU], version 1511, 10586.
  4. Approve the Upgrade for the classes of computer in your organisation that you want to be upgraded.

Once all of the above steps are in place, computers that are targeted for the upgrade should have this happen automatically at the next update cycle.

Exchange 2013 Cert Change - Unable to Support the STARTTLS SMTP Verb

I saw a issue recently on an Exchange server after the certificate used to secure SMTP and IIS services was changed as the old certificate was about to expire.

The original default certificate that is self-generated had been replaced with one from a certificate authority. This had been used for several years without issue. The Exchange server is configured in hybrid mode, with incoming e-mail routed through Microsoft’s Exchange Online Protection (EOP) and TLS is configured to be required.

The actions taken were:

  1. Add the new certificate to the certificate store on the Exchange server. This made the new certificate available within the Exchange Admin Center on the server.
  2. Modify the services assigned to the new certificate to bind SMTP and IIS to the new certificate.
  3. Remove the original certificate from the server.
  4. Restart the Microsoft Exchange Transport Service on the server.

At this point, an error was thrown in the Application Event Log on the server and incoming mail from Exchange Online Protection stopped flowing. The error thrown was:

Log Name:      Application
Source:        MSExchangeFrontEndTransport
Date:          04/02/2016 12:17:20
Event ID:      12014
Task Category: TransportService
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      <Exchange Server FQDN>
Description:
Microsoft Exchange could not find a certificate that contains the domain name <I><Cert Issuer Details><S><Cert Subject Details> in the personal store on the local computer. Therefore, it is unable to support the STARTTLS SMTP verb for the connector <Receive Connector Name> with a FQDN parameter of <I><Cert Issuer Details><S><Cert Subject Details>. If the connector's FQDN is not specified, the computer's FQDN is used. Verify the connector configuration and the installed certificates to make sure that there is a certificate with a domain name for that FQDN. If this certificate exists, run Enable-ExchangeCertificate -Services SMTP to make sure that the Microsoft Exchange Transport service has access to the certificate key.

I ran the suggested command to ensure that the Exchange Transport Service had access to the certificate key, but this didn’t help.

Restoring the soon-to-expire certificate to the certificate store on the server and restarting the Microsoft Exchange Transport Service fixed the error, however the certificate in question was going to expire soon, and the use of expired certificates for TLS to EOP is no longer allowed, so this didn’t really help much.

While digging into the configuration for the receive connector specified in the error thrown, I noticed something interesting. Despite the new certificate being supplied by the same certificate authority as the old one, the issuer specified for the certificate had changed slightly. The subject information was still the same. Sure enough, the properties of the receive connector in question still showed the old certificate details even through Exchange had been configured with the new certificate for SMTP and IIS. The information on the receive connector can be found by issuing the following command:

Get-ReceiveConnector "<Receive Connector Name>" | fl

The property we’re interested in is TlsCertificateName.

To correct the error, the following steps were taken:

  1. Locate the issuer and subject information from the new certificate. This can be done by examining the certificate directly via the certificate store, or using PowerShell, e.g.
    $certs = Get-ExchangeCertificate
    Locate the certificate you want to use. The one we wanted was the first on the list.
  2. Assemble the new issuer and subject information in a suitable format for the Receive Connector configuration. Again this can be done by copying the required text from the certificate information, or using PowerShell, e.g.:
    $certinfo = “<I>” + $certs[0].issuer + “<S>” + $certs[0].subject
  3. Modify the Receive Connector configuration to include the new certificate information assembled above, e.g.:
    Set-ReceiveConnector “<Receive Connector Name>” –TlsCertificateName $certinfo
  4. Restart the Microsoft Exchange Transport Service.
  5. Remove the old certificate from the server.

A quick test of incoming and outgoing mail indicated that everything was flowing as expected.