Configure Server 2016 ADFS and WAP with custom ports using Powershell

A pull request for Chris Gardner’s WebApplicationProxyDSC is now inbound after a frustrating week of trying to automate the configuration of ADFS and WAP on a Server 2016 lab.

With Server 2016, the PowerShell commands to configure the ADFS and WAP servers include switches to specify a non-default port. I need to do this because the servers are behind a NetNat on a server hosting several labs, so port 443 is not available to me and I must use a different port.

This should be simple: Specify the SSLPort switch on the Install-ADFSFarm command and the HttpsPort on the Install-WebApplicationProxy command. However, when I do that, the WAP configuration fails with an error that it cannot read the FederationMetadata from the proxy.

I tried all manner of things to diagnose why this was failing and in the end, the fix is a crazy hack that should not work!

The proxy installation, despite accepting the custom port parameter, does not build the URLs correctly for the ADFS service, so is still trying to call port 443. You can set these URLs on a configured WAP service using the Set-WebApplicationProxyConfiguration command. However, when you run this command with no configured proxy, it fails.

Or so you think…

On the ADFS Server:

  1. Install-AdfsFarm specifiying the CertificateThumbprint, Credential, FederationServiceDisplayName, FederationServiceName and SSLPort params

On the WAP Server:

  1. Install-WebApplicationProxy specifiying the HttpsPort switch,  CertificateThumbprint, FederationServiceName and FederationServiceTrustCredential params.
  2. Set-WebApplicationProxyConfiguration specifying the ADFSUrl, OAuthAuthenticationURL and ADFSSignOutURL parameters with the correct URLs for your ADFS server (which include the port in the Url).
  3. Re-run the command in step 1.

Despite the fact that step 2 says it failed, it seems to set enough information for step 3 to succeed. My experience, however, is that only doing steps 2 and 3 does not work. Weird!

As a side note, testing this lot is a lot easier if you remember that the idpinitiatedsignon.aspx page we all normally use for testing ADFS is disabled by default in Server 2016. Turn it on with Set-AdfsProperties -EnableIdPInitiatedSignonPage $true

Setting Enroll Permissions on ADCS Certificate Template using DSC

As part of the work I have been doing around generating and managing lab environments using Lability and DSC, one of the things I needed to do was change the permissions on a certificate template within a DSC configuration. Previously, when deploying to Azure, I used the PSPKI PowerShell modules within code executed by the Custom Script extension. I was very focused on sticking with DSC this time, which ruled out PSPKI. Whilst there is a DSC module available to configure Certificate Services itself, this does not extend to managing Certificate Templates.

Nobody seemed to have done exactly this before. I used the following links as references in creating the code:

Get Effective template permissions with PowerShell by Vadims Podans

Duplicate AD Object Without Active Directory PS Tools

Add Object Specific ACEs using Active Directory PowerShell

Using Scripts to Manage Active Directory Security

The script finds the WebServer template and grants the Enroll extended permission to the Domain Computers AD group. This allows me to use xCertificate in the DSC configuration of domain member servers to request new certificates using the WebServer template.

Here is the code I include in my DSC configuration. $DomainCreds is a PSCredential object for the domain admin ( I create the AD domain in an earlier step using xActiveDirectory).

        #Enable Enroll on WebServer certificate template         Script EnableWebServerEnroll {             DependsOn = "[xAdcsCertificationAuthority]CertAuth"             PsDscRunAsCredential = $DomainCreds             GetScript = {                 return @{ 'Result' = $true}             }             TestScript = {                 #Find the webserver template in AD and grant the Enroll extended right to the Domain Computers                 $filter = "(cn=WebServer)"                 $ConfigContext = ([ADSI]"LDAP://RootDSE").configurationNamingContext                 $ConfigContext = "CN=Certificate Templates,CN=Public Key Services,CN=Services,$ConfigContext"                  $ds = New-object System.DirectoryServices.DirectorySearcher([ADSI]"LDAP://$ConfigContext",$filter)                 $Template = $ds.Findone().GetDirectoryEntry()                 if ($Template -ne $null) {                     $objUser = New-Object System.Security.Principal.NTAccount("Domain Computers")                     # The following object specific ACE is to grant Enroll                     $objectGuid = New-Object Guid 0e10c968-78fb-11d2-90d4-00c04f79dc55                      ForEach ($AccessRule in $Template.ObjectSecurity.Access) {                         If ($AccessRule.ObjectType.ToString() -eq $objectGuid) {                             If ($AccessRule.IdentityReference -like "*$($objUser.Value)") {                                 Write-Verbose "TestScript: WebServer Template Enroll permission for Domain Computers exists. Returning True"                                 return $true                             }                         }                     }                 }                 return $false             }             SetScript = {                 #Find the webserver template in AD and grant the Enroll extended right to the Domain Computers                 $filter = "(cn=WebServer)"                 $ConfigContext = ([ADSI]"LDAP://RootDSE").configurationNamingContext                 $ConfigContext = "CN=Certificate Templates,CN=Public Key Services,CN=Services,$ConfigContext"                  $ds = New-object System.DirectoryServices.DirectorySearcher([ADSI]"LDAP://$ConfigContext",$filter)                 $Template = $ds.Findone().GetDirectoryEntry()                  if ($Template -ne $null) {                     $objUser = New-Object System.Security.Principal.NTAccount("Domain Computers")                     # The following object specific ACE is to grant Enroll                     $objectGuid = New-Object Guid 0e10c968-78fb-11d2-90d4-00c04f79dc55                     $ADRight = [System.DirectoryServices.ActiveDirectoryRights]"ExtendedRight"                     $ACEType = [System.Security.AccessControl.AccessControlType]"Allow"                     $ACE = New-Object System.DirectoryServices.ActiveDirectoryAccessRule -ArgumentList $objUser,$ADRight,$ACEType,$objectGuid                     $Template.ObjectSecurity.AddAccessRule($ACE)                     $Template.commitchanges()                     Write-Verbose "SetScript: Completed WebServer additional permission"                 }             }         }

Creating Azure Virtual Networks using Powershell and XML Part 4: Local networks and site-site connectivity

This is part 4 of a series of posts building powershell functions to create and modify Azure Virtual Networks. Previous posts have covered functions to create virtual networks and then delete them. In this part, I’m going to show you functions that will define local networks and configure site-site VPN connectivity between a local and virtual network.

Next on my list is to create functions to delete the local networks and remove the site-site connections. Then I really must look at functions to edit the configuration.

Adding the functionality for local networks also meant that I had to modify the get-azureNetworkConfig function to create the LocalNetworkSites xml node if it does not already exist, ready to hold our local network definitions.

The Functions

get-azureNetworkConfig

This is an update to the function shown in part 2.

function get-azureNetworkXml {  $currentVNetConfig = get-AzureVNetConfig if ($currentVNetConfig -ne $null) { [xml]$workingVnetConfig = $currentVNetConfig.XMLConfiguration } else { $workingVnetConfig = new-object xml }  $networkConfiguration = $workingVnetConfig.GetElementsByTagName("NetworkConfiguration") if ($networkConfiguration.count -eq 0) { $newNetworkConfiguration = create-newXmlNode -nodeName "NetworkConfiguration" $newNetworkConfiguration.SetAttribute("xmlns:xsd","http://www.w3.org/2001/XMLSchema") $newNetworkConfiguration.SetAttribute("xmlns:xsi","http://www.w3.org/2001/XMLSchema-instance") $networkConfiguration = $workingVnetConfig.AppendChild($newNetworkConfiguration) }  $virtualNetworkConfiguration = $networkConfiguration.GetElementsByTagName("VirtualNetworkConfiguration") if ($virtualNetworkConfiguration.count -eq 0) { $newVirtualNetworkConfiguration = create-newXmlNode -nodeName "VirtualNetworkConfiguration" $virtualNetworkConfiguration = $networkConfiguration.AppendChild($newVirtualNetworkConfiguration) }  $dns = $virtualNetworkConfiguration.GetElementsByTagName("Dns") if ($dns.count -eq 0) { $newDns = create-newXmlNode -nodeName "Dns" $dns = $virtualNetworkConfiguration.AppendChild($newDns) }  $localNetworks = $virtualNetworkConfiguration.GetElementsByTagName("LocalNetworkSites") if ($localNetworks.count -eq 0) { $newlocalNetworks = create-newXmlNode -nodeName "LocalNetworkSites" $localNetworks = $virtualNetworkConfiguration.AppendChild($newLocalNetworks) }  $virtualNetworkSites = $virtualNetworkConfiguration.GetElementsByTagName("VirtualNetworkSites") if ($virtualNetworkSites.count -eq 0) { $newVirtualNetworkSites = create-newXmlNode -nodeName "VirtualNetworkSites" $virtualNetworkSites = $virtualNetworkConfiguration.AppendChild($newVirtualNetworkSites) }  return $workingVnetConfig } 

add-azureVnetLocalNetworkSite

Add-azureVnetLocalNetworkSite takes three parameters: networkName is the name for the new local network; addressPrefix is the network prefix for the local network and vpnGatewayAddress is the ip address of the local VPN gateway that will establish the vpn tunnel. The function checks that the local network does not already exist and then creates the appropriate XML.

function add-azureVnetLocalNetworkSite { param ( [string]$networkName, [string]$addressPrefix, [string]$vpnGatewayAddress )  #check if the network already exists $siteExists = $workingVnetConfig.GetElementsByTagName("LocalNetworkSite") | where {$_.name -eq $networkName} if ($siteExists.Count -ne 0) { 	write-Output "Local Network Site $networkName already exists" 	$newNetwork = $null 	return $newNetwork }   #get the parent node $workingNode = $workingVnetConfig.GetElementsByTagName("LocalNetworkSites") #add the new network node $newNetwork = create-newXmlNode -nodeName "LocalNetworkSite" $newNetwork.SetAttribute("name",$networkName) $network = $workingNode.appendchild($newNetwork)  #add new address space node $newAddressSpace = create-newXmlNode -nodeName "AddressSpace" $AddressSpace = $network.appendchild($newAddressSpace) $newAddressPrefix = create-newXmlNode -nodeName "AddressPrefix" $newAddressPrefix.InnerText = $addressPrefix  $AddressSpace.appendchild($newAddressPrefix)  #add the new vpn gateway address $newVpnGateway = create-newXmlNode -nodeName "VPNGatewayAddress" $newVpnGateway.InnerText = $vpnGatewayAddress $network.AppendChild($newVpnGateway)  #return our new network $newNetwork = $network return $newNetwork   }  

add-azureVnetSiteConnectivity

add-azureVnetSiteConnectivity takes two parameters: networkName is the name of the virtual network and localNetworkName is the name of the local network. It checks to make sure both are defined before creating the appropriate XML to define the connection. In order for the site-site VPN configuration to be applied, the virtual network must have a subnet named GatewaySubnet, so the function checks for that too. I already have a function to create subnets so I can use that to create the subnet. The function also currently specifies a type of IPSec for the connection as no other options are currently available for site-to-site vpn connections.

function add-azureVnetSiteConnectivity { param ( [string]$networkName, [string]$localNetworkName )  #get our target network $workingNode = $workingVnetConfig.GetElementsByTagName("VirtualNetworkSite") | where {$_.name -eq $networkName} if ($workingNode.Count -eq 0) { 	write-Output "Network $networkName does not exist" 	$newVnetSiteConnectivity = $null 	return $newVnetSiteConnectivity }  #check that the network has a GatewaySubnet $subNetExists = $workingNode.GetElementsByTagName("Subnet") | where {$_.name -eq "GatewaySubnet"} if ($subNetExists.count -eq 0) { 	write-Output "Virtual network $networkName has no Gateway subnet" 	$newVnetSiteConnectivity = $null 	return $newVnetSiteConnectivity }   #check that the local network site exists $localNetworkSite = $workingVnetConfig.GetElementsByTagName("LocalNetworkSite") | where {$_.name -eq $localNetworkName} if ($localNetworkSite.count -eq 0) { 	write-Output "Local Network Site $localNetworkSite does not exist" 	$newVnetSiteConnectivity = $null 	return $newVnetSiteConnectivity }  #check if the gateway node exists and if not, create $gateway = $workingNode.GetElementsByTagName("Gateway") if ($gateway.count -eq 0) { $newGateway = create-newXmlNode -nodeName "Gateway" $gateway = $workingNode.appendchild($newGateway) }  #check if the ConnectionsToLocalNetwork node exists and if not, create $connections = $workingNode.GetElementsByTagName("ConnectionsToLocalNetwork") if ($connections.count -eq 0) { $newConnections = create-newXmlNode -nodeName "ConnectionsToLocalNetwork" $connections = $gateway.appendchild($newConnections) }   #check to make sure our local site reference doesn't already exist $localSiteRefExists = $workingNode.GetElementsByTagName("LocalNetworkSiteRef") | where {$_.name -eq $localNetworkName} if ($localSiteRefExists.count -ne 0) { 	write-Output "Local Site Ref $localNetworkName already exists" 	$newVnetSiteConnectivity = $null 	return $newVnetSiteConnectivity  }  #add the local site ref $newVnetSiteConnectivity = create-newXmlNode -nodeName "LocalNetworkSiteRef" $newVnetSiteConnectivity.SetAttribute("name",$localNetworkName) $vNetSiteConnectivity = $connections.appendchild($newVnetSiteConnectivity) $newConnection = create-newXmlNode -nodeName "Connection" $newConnection.SetAttribute("type","IPsec")  $vNetSiteConnectivity.appendchild($newConnection)  #return our new subnet $newVnetSiteConnectivity = $vNetSiteConnectivity return $newVnetSiteConnectivity  } 

Using the functions

These functions modify an XML configuration that needs to be held in an object named $workingVnetConfig. Part 2 of this series showed how they can be loaded from a powershell file and called. Get-azureNetworkXml is required to get the XML configuration object. The functions here can then be used to remove items from that configuration, then save-azureNetworkXml will push the modified configuration back into Azure.

Gary Lapointe to the rescue: Using his Office 365 powershell tools to recover from a corrupted masterpage

I also need to give credit to the Office 365 support team over this. They were very quick in their response to my support incident, but I was quicker!

Whilst working on an Office 365 site for a customer today I had a moment of blind panic. The site is using custom branding and I was uploading a new version of the master page to the site when things went badly wrong. The upload appeared to finish OK but the dialog that was shown post upload was not the usual content type/fill in the fields form, but a plain white box. I left it for a few minutes but nothing changed. Unperturbed, I returned to the mater page gallery… Except I couldn’t. All I got was a white page. No errors, nothing. No pages worked at all – no settings pages, no content pages, nothing at all.

After some screaming, I tried SharePoint designer. Unfortunately, this was disabled (it is by default) and I couldn’t reach the settings page to enable it. I logged a support call and then suddenly remembered a recent post from Gary Lapointe about a release of some powershell tools for Office 365.

Those tools saved my life. I connected to the Office 365 system with :

Connect-SPOSite -Credential "<my O365 username>" -url "<my sharepoint online url>"

Success!

First of all I used set-spoweb to set the masterurl and custommasterurl properties of the failed site. That allow me back into the system (phew!):

Set-SPOWeb -Identity "/" -CustomMasterUrl "/_catalogs/masterpage/seattle.master"

Once in, I thought all was well, but I could only access content pages. Every time I tried to access the masterpages libary or one of the site settings pages I got an error, even using Seattle.master.

Fortunately, Gary also has a command that will upload a file to a library, so I attempted to overwrite my corrupted masterpage:

New-SPOFile -List "https://<my sharepoint online>.sharepoint.com/_catalogs/masterpage" -Web "/" -File "<local path to my master page file>" –Overwrite

Once I’d done that, everthing snapped back into life.

The moral of the story? Keep calm and always have PowerShell ISE open!

You can download Gary’s tools here and instructions on their use are here.

Big thanks, Gary!

Creating Azure Virtual Networks using Powershell and XML Part 3: Powershell functions for deletion

This is part three of a series of posts about using powershell to script the creation, deletion and (hopefully) modification of Azure Virtual Networks. In part 1 I went through the key steps with some rough code. Part 2 showed the much tidier functions I’ve now written to create virtual network elements. This is part 3, and I will present functions to remove elements. Hopefully I will manage to get the modification functions to work which be a fourth installment!

I’m not going to go through how to use the new functions in this part – I covered that before. I’m simply going to present the new functions that perform the following actions:

  1. Remove an entire virtual network definition.
  2. Remove a DNS definition.
  3. Remove a single subnet from a virtual network.
  4. Remove a DNS registration from a virtual network.

The big thing I learned when writing this code is that if I used the RemoveAll method on an xml node in my configuration xml object, it didn’t actually remove the node itself but only the attributes and child nodes. This left empty elements (such as <VirtualNetworkSite />) that confused Azure. The solution was to call the RemoveChild method on the parent node of the one I wanted rid of, specifying my target node.

The Functions

Delete-azureVnetNetwork

Delete-azureVnetNetwork takes one parameter: networkName. It makes sure the network exists, then removes the appropriate VirtualNetworkSite node and all it’s children.

function delete-azureVnetNetwork { param ( [string]$networkName )  #check that the network already exists $network = $workingVnetConfig.GetElementsByTagName("VirtualNetworkSite") | where {$_.name -eq $networkName} if ($network.Count -eq 0) { 	write-Output "Network $networkName does not exist" 	$removeNetwork = $null 	return $removeNetwork }  #remove the node and children $network.ParentNode.RemoveChild($network)   #return true as we deleted the node $removeNetwork = $true  return $removeNetwork } 

Delete-azureVnetSubnet

Delete-azureVnetSubnet takes two parameters: networkName and subnetName. It checks to make sure both exist, then removes the appropriate Subnet element from the specified network.

function delete-azureVnetSubnet { param ( [string]$networkName, [string]$subnetName )  #check that the network  exists $network = $workingVnetConfig.GetElementsByTagName("VirtualNetworkSite") | where {$_.name -eq $networkName} if ($network.Count -eq 0) { 	write-Output "Network $networkName does not exist" 	$removeSubnet = $null 	return $removeSubnet }  #check to make sure our subnet name exists $subNet = $network.GetElementsByTagName("Subnet") | where {$_.name -eq $subnetName} if ($subNet.count -eq 0) { 	write-Output "Subnet $subnetName does not exist in network" 	$removeSubnet = $null 	return $removeSubnet  }  #remove the node and children $subNet.ParentNode.RemoveChild($subNet)  #return true as we deleted the node $removeSubnet = $true return $removeSubnet } 

Delete-azureVnetDnsRef

Delete-azureVnetDnsRef takes two parameters: networkName and dnsName. It checks to make sure both the network and the DNS reference within it exist, then removes the appropriate DnsServerRef element from the specified network.

function delete-azureVnetDnsRef { param ( [string]$networkName, [string]$dnsName )  #check that the network  exists $network = $workingVnetConfig.GetElementsByTagName("VirtualNetworkSite") | where {$_.name -eq $networkName} if ($network.Count -eq 0) { 	write-Output "Network $networkName does not exist" 	$removeDnsRef = $null 	return $removeDnsRef }  #check that the dns reference is there $dnsRef = $network.GetElementsByTagName("DnsServerRef") | where {$_.name -eq $dnsName} if ($dnsRef.count -eq 0) { 	write-Output "DNS reference $dnsName does not exist" 	$removeDnsRef = $null 	return $removeDnsRef }  #remove the node and children $dnsRef.ParentNode.RemoveChild($dnsRef)   #return true as we deleted the node $removeDnsRef = $true  return $removeDnsRef } 

Delete-azureVnetDns

Delete-azureVnetDnsRef takes one parameter: dnsName. It checks to make sure that the DNS is not referenced by any virtual networks and that the DNS exists, then removes the appropriate DnsServer element.

function delete-azureVnetDns { param ( [string]$dnsName )  #check that the dns isn't referenced in any networks $dnsRef = $workingVnetConfig.GetElementsByTagName("DnsServerRef") | where {$_.name -eq $dnsName} if ($dnsRef.count -ne 0) { 	write-Output "DNS $dnsName is referenced in networks" 	$removeDns = $null 	return $removeDnsRef } #check that the DNS exists $dns = $workingVnetConfig.GetElementsByTagName("DnsServer") | where {$_.name -eq $dnsName} if ($dns.Count -eq 0) { 	write-Output "DNS Server $dnsName does not exists" 	$removeDns = $null 	return $removeDns }  #remove the node and children $dns.ParentNode.RemoveChild($dns)   #return true as we deleted the node $removeDns = $true  return $removeDns }

Using the functions

These functions modify an XML configuration that needs to be held in an object call $workingVnetConfig. My previous post showed how they can be loaded from a powershell file and called. Get-azureNetworkXml is required to get the XML configuration object. The functions here can then be used to remove items from that configuration, then save-azureNetworkXml will push the modified configuration back into Azure.

Creating Azure Virtual Networks using Powershell and XML Part 2: Powershell functions

In my previous post I talked about what was involved in creating an Azure network configuration using Powershell. In this post I’ll cover where I’ve got so so far, which is a series of functions that do the following:

  1. Contact Azure and get the current network configuration. Convert that to sensible XML and if it’s empty, create the basic structure.
  2. Create a new virtual network, checking to see if one with the same name already exists.
  3. Add a subnet to a virtual network, checking to see one with the same address prefix or name doesn’t already exist.
  4. Add a DNS reference to a virtual network, making sure the DNS is defined first.
  5. Create a DNS.
  6. Put the configuration back into Azure to be applied.

Still on my to-do list are removing networks and other elements, and modifying existing networks.

The Function Code

The end result so far is a powershell script that can be loaded to give a number of new functions:

Get-azureNetworkXml

get-azureNetworkXml runs the get-AzureVNetConfig command. It takes the XMLConfiguration from that command and puts it into a new XML object. If there is no configuration, it creates a new xml object. It then checks to see if the main XML elements are present and, if not, creates them.

Whilst this function returns an object, I need to make sure (right now) that the variable nme I use for that is $workingVnetConfig as other functions reference it. I’m not currently passing the XML object into each function. I probably should, but that tidying comes later.

function get-azureNetworkXml {  $currentVNetConfig = get-AzureVNetConfig if ($currentVNetConfig -ne $null) { [xml]$workingVnetConfig = $currentVNetConfig.XMLConfiguration } else { $workingVnetConfig = new-object xml }  $networkConfiguration = $workingVnetConfig.GetElementsByTagName("NetworkConfiguration") if ($networkConfiguration.count -eq 0) { $newNetworkConfiguration = create-newXmlNode -nodeName "NetworkConfiguration" $newNetworkConfiguration.SetAttribute("xmlns:xsd","http://www.w3.org/2001/XMLSchema") $newNetworkConfiguration.SetAttribute("xmlns:xsi","http://www.w3.org/2001/XMLSchema-instance") $networkConfiguration = $workingVnetConfig.AppendChild($newNetworkConfiguration) }  $virtualNetworkConfiguration = $networkConfiguration.GetElementsByTagName("VirtualNetworkConfiguration") if ($virtualNetworkConfiguration.count -eq 0) { $newVirtualNetworkConfiguration = create-newXmlNode -nodeName "VirtualNetworkConfiguration" $virtualNetworkConfiguration = $networkConfiguration.AppendChild($newVirtualNetworkConfiguration) }  $dns = $virtualNetworkConfiguration.GetElementsByTagName("Dns") if ($dns.count -eq 0) { $newDns = create-newXmlNode -nodeName "Dns" $dns = $virtualNetworkConfiguration.AppendChild($newDns) }  $virtualNetworkSites = $virtualNetworkConfiguration.GetElementsByTagName("VirtualNetworkSites") if ($virtualNetworkSites.count -eq 0) { $newVirtualNetworkSites = create-newXmlNode -nodeName "VirtualNetworkSites" $virtualNetworkSites = $virtualNetworkConfiguration.AppendChild($newVirtualNetworkSites) }  return $workingVnetConfig } 

Save-azureNetworkXml

Save-azureNetworkXml gets passed our XML object, writes it out to a file in the temp dir and then calls set-AzureVNetConfig to load the file and send it to Azure.

function save-azureNetworkXml($workingVnetConfig) { $tempFileName = $env:TEMP + "\azurevnetconfig.netcfg" $workingVnetConfig.save($tempFileName) notepad $tempFileName  set-AzureVNetConfig -configurationpath $tempFileName } 

Add-azureVnetNetwork

Add-azureVnetNetwork is called with three parameters: networkName, affinityGroup and addressPrefix. It will add a new VirtualNetworkSite element, with the name and affinity group as attributes. It checks to make sure the affinity group exists first. It then creates the address prefix within the network.

function add-azureVnetNetwork { param ( [string]$networkName, [string]$affinityGroup, [string]$addressPrefix )  #check if the network already exists $networkExists = $workingVnetConfig.GetElementsByTagName("VirtualNetworkSite") | where {$_.name -eq $networkName} if ($networkExists.Count -ne 0) { 	write-Output "Network $networkName already exists" 	$newNetwork = $null 	return $newNetwork }   #check that the target affinity group exists $affinityGroupExists = get-AzureAffinityGroup | where {$_.name -eq $affinityGroup} if ($affinityGroupExists -eq $null) { 	write-Output "Affinity group $affinityGroup does not exist" 	$newNetwork = $null 	return $newNetwork }  #get the parent node $workingNode = $workingVnetConfig.GetElementsByTagName("VirtualNetworkSites") #add the new network node $newNetwork = create-newXmlNode -nodeName "VirtualNetworkSite" $newNetwork.SetAttribute("name",$networkName) $newNetwork.SetAttribute("AffinityGroup",$affinityGroup ) $network = $workingNode.appendchild($newNetwork)  #add new address space node $newAddressSpace = create-newXmlNode -nodeName "AddressSpace" $AddressSpace = $Network.appendchild($newAddressSpace) $newAddressPrefix = create-newXmlNode -nodeName "AddressPrefix" $newAddressPrefix.InnerText=$addressPrefix  $AddressSpace.appendchild($newAddressPrefix)  #return our new network $newNetwork = $network return $newNetwork   } 

Add-azureVnetSubnet

Add-azureVnetSubnet takes three parameters: networkName, subnetName and addressPrefix. It makes sure the network exists, that the subnet doesn’t, and that the address prefix is not already used in the same network. It then adds the subnet to the network.

function add-azureVnetSubnet { param ( [string]$networkName, [string]$subnetName, [string]$addressPrefix )  #get our target network $workingNode = $workingVnetConfig.GetElementsByTagName("VirtualNetworkSite") | where {$_.name -eq $networkName} if ($workingNode.Count -eq 0) { 	write-Output "Network $networkName does not exist" 	$newSubnet = $null 	return $newSubnet }  #check if the subnets node exists and if not, create $subnets = $workingNode.GetElementsByTagName("Subnets") if ($subnets.count -eq 0) { $newSubnets = create-newXmlNode -nodeName "Subnets" $subnets = $workingNode.appendchild($newSubnets) }  #check to make sure our subnet name doesn't exist and/or prefix isn't already there $subNetExists = $workingNode.GetElementsByTagName("Subnet") | where {$_.name -eq $subnetName} if ($subNetExists.count -ne 0) { 	write-Output "Subnet $subnetName already exists" 	$newSubnet = $null 	return $newSubnet  } $subNetExists = $workingNode.GetElementsByTagName("Subnet") | where {$_.AddressPrefix -eq $subnetName} if ($subNetExists.count -ne 0) { 	write-Output "Address prefix $addressPrefix already exists in another network" 	$newSubnet = $null 	return $newSubnet  }  #add the subnet $newSubnet = create-newXmlNode -nodeName "Subnet" $newSubnet.SetAttribute("name",$subnetName) $subnet = $subnets.appendchild($newSubnet) $newAddressPrefix = create-newXmlNode -nodeName "AddressPrefix" $newAddressPrefix.InnerText = $addressPrefix  $subnet.appendchild($newAddressPrefix)  #return our new subnet $newSubnet = $subnet return $newSubnet  } 

Add-azureVnetDns

Add-azureVnetDns takes two parameters: dnsName and dnsAddress. It then creates a new DnsServer element for that DNS.

function add-azureVnetDns { param ( [string]$dnsName, [string]$dnsAddress )  #check that the DNS does not exist $dnsExists = $workingVnetConfig.GetElementsByTagName("DnsServer") | where {$_.name -eq $dnsName} if ($dnsExists.Count -ne 0) { 	write-Output "DNS Server $dnsName already exists" 	$newDns = $null 	return $newDns } # get our working node of Dns $workingNode = $workingVnetConfig.GetElementsByTagName("Dns")  #check if the DnsServersRef node exists and if not, create $dnsServers = $workingNode.GetElementsByTagName("DnsServers") if ($dnsServers.count -eq 0) { $newDnsServers = create-newXmlNode -nodeName "DnsServers" $dnsServers = $workingNode.appendchild($newDnsServers) }  #add new dns reference $newDnsServer = create-newXmlNode -nodeName "DnsServer" $newDnsServer.SetAttribute("name",$dnsName) $newDnsServer.SetAttribute("IPAddress",$dnsAddress) $newDns = $dnsServers.appendchild($newDnsServer)  #return our new dnsRef return $newDns   } 

Add-azureVnetDnsRef

Add-azureVnetDnsRef takes two parameters; networkName and dnsName. It makes sure the network exists and that the DNS exists before adding a DnsServerRef element for the DNS to the network.

function add-azureVnetDnsRef { param ( [string]$networkName, [string]$dnsName )  #get our target network $workingNode = $workingVnetConfig.GetElementsByTagName("VirtualNetworkSite") | where {$_.name -eq $networkName} if ($workingNode.count -eq 0) { 	write-Output "Network $networkName does not exist" 	$newSubnet = $null 	return $newSubnet }  #check if the DnsServersRef node exists and if not, create $dnsServersRef = $workingNode.GetElementsByTagName("DnsServersRef") if ($dnsServersRef.count -eq 0) { $newDnsServersRef = create-newXmlNode -nodeName "DnsServersRef" $dnsServersRef = $workingNode.appendchild($newDnsServersRef) }  #check that the DNS we want to reference is defined already $dnsExists = $workingVnetConfig.GetElementsByTagName("DnsServer") | where {$_.name -eq $dnsName} if ($dnsExists.Count -eq 0) { 	write-Output "DNS Server $dnsName does not exist so cannot be referenced" 	$newDnsRef = $null 	return $newDnsRef }  #check that the dns reference isn't already there $dnsRefExists = $workingNode.GetElementsByTagName("DnsServerRef") | where {$_.name -eq $dnsName} if ($dnsRefExists.count -ne 0) { 	write-Output "DNS reference $dnsName already exists" 	$newDnsRef = $null 	return $newDnsRef }  #add new dns reference $newDnsServerRef = create-newXmlNode -nodeName "DnsServerRef" $newDnsServerRef.SetAttribute("name",$dnsName) $newDnsRef = $dnsServersRef.appendchild($newDnsServerRef)  #return our new dnsRef return $newDnsRef   } 

Create-newXmlNode

Create-newXmlNode is called by all the other functions. It creates a new node in the XML object then hands it back to the calling function for modification and appending it to the relevant parent node.

function create-newXmlNode { param ( [string]$nodeName )  $newNode = $workingVnetConfig.CreateElement($nodeName,"http://schemas.microsoft.com/ServiceHosting/2011/07/NetworkConfiguration") return $newNode } 

Using the functions

Assuming all our functions are in a powershell file called Create-AzureNetwork.ps1, using the new functions is pretty straightforward. We load the functions from file (there is a space between the first . and the .\ in the first line. We can then call the functions.

Note that I use a variable called $workingVnetConfig here – that’s important, as we don’t pass the XML into each function, but rather use the way powershell handles variables which means that having defined it here, it’s available to all the functions when called.

. .\Create-AzureNetwork.ps1  $workingVnetConfig = get-azurenetworkxml  add-azureVnetNetwork -networkName "Mynetwork" -affinityGroup "MyAzureAffinity" -addressPrefix "10.0.0.0/8"  add-azureVnetSubnet -networkName "Mynetwork" -subnetName "subnet-1" -addressPrefix "10.0.0.0/11"  add-azureVNetDns -dnsName "test1" -dnsAddress "10.0.0.1"  add-azureVnetDnsRef -networkName "Mynetwork" -dnsName "test1"  save-azurenetworkxml($workingVnetConfig) 

It’s not the most elegant of code, I’ll admit, but it does what it says on the tin. All I have to do now is add functions to remove items from our network configuration, then add functions to modify existing items.

Creating Azure Virtual Networks using Powershell and XML

I’ll be honest, I expected this task to be easier than it is. What I’m working on is some powershell that we might use as part of automated build processes that will create a new Virtual Network in an Azure subscription. What I’m after is to add a new network to the existing configuration.

There aren’t many powershell commands for Azure virtual networks. The two we need to use are get-azureVnetConfig and set-azureVnetConfig.

Get-azureVnetConfig when run generates xml that details the configuration of all virtual networks within the current Azure subscriptions. Set-azureVnetConfig takes an xml configuration and modifies the entire virtual networking configuration to match that described in the file.

My original plan of simple powershell to add a new virtual network went quickly out of the window, then. My second thought was to grab the xml configuration, manipulate it using powershell, then stuff it back into Azure. That plan was hindered by the fact that the set-azureVnetConfig command insists on reading the configuration from a file on disk, so I can’t just hand it my XML object, created by manipulating the output of get-azureVnetConfig.

I’m still working on this – I now have a script with some tidy functions to do repetitive tasks. This post is simply going to outline the first bit of heavy lifting I’ve had to do in order to solve enough problems that I can get a config, add stuff to it and reload it into Azure.

The steps below don’t create all the configuration we will want, but it creates all the configuration we need to add a new network.

1. Get the current Azure Config

This bit is easy:

$currentVNetConfig = get-AzureVNetConfig

That gives us an object which contains the XML configuration. We need to get just the XML out, so:

[xml]$workingVnetConfig = $currentVNetConfig.XMLConfiguration

2. Find the VirtualNetworkSites element

The networks I want to create are all held in the VirtualNetworkSites element, each one in a VirtualNetworkSite element. I can create new VirtualNetworkSite elements, but I need to grab the element in which to create them first:

$virtNetCfg = $workingVnetConfig.GetElementsByTagName("VirtualNetworkSites")

3. Add a new Virtual Network

To add a new network we need to add a new VirtualNetworkSite element. I hit a snag with this, in that I kept getting a spurious xmlns attribute on the element that caused set-azureVnetConfig to spit out the file as invalid. It turns out that in order to avoid this, we have to specify the XML namespace URI when we create the new element. That’s the second parameter on the CreateElement method, below.

Creating the element itself is a two-stage process: First we create a new element inside our XML object, then we put that element in the right place by calling appendchild on the intended parent element. In addition, we need to add a couple of attributes to that element, specifying the name of the network and the affinity group it will sit in:

$newNetwork = $workingVnetConfig.CreateElement("VirtualNetworkSite","http://schemas.microsoft.com/ServiceHosting/2011/07/NetworkConfiguration")  $newNetwork.SetAttribute("name","myVirtualNetwork")  $newNetwork.SetAttribute("AffinityGroup","MyAffinityGroup")  $Network = $virtNetCfg.appendchild($newNetwork) 

4. Add an address space

This is a similar process. I need an AddressSpace element and within that sits an AddressPrefix element. That element needs text that tells Azure the IP address space to use, and that’s added by setting the innerText property.

$newAddressSpace = $workingVnetConfig.CreateElement("AddressSpace","http://schemas.microsoft.com/ServiceHosting/2011/07/NetworkConfiguration")  $AddressSpace = $Network.appendchild($newAddressSpace)  $newAddressPrefix = $workingVnetConfig.CreateElement("AddressPrefix","http://schemas.microsoft.com/ServiceHosting/2011/07/NetworkConfiguration")  $newAddressPrefix.InnerText="10.0.0.0/8"  $AddressSpace.appendchild($newAddressPrefix) 

5. Add a subnet

Virtual networks need subnets. There is a Subnets element that contains multiple Subnet elements, each of which has an AddressPrefix element.

$newSubnets = $workingVnetConfig.CreateElement("Subnets","http://schemas.microsoft.com/ServiceHosting/2011/07/NetworkConfiguration")  $Subnets = $Network.appendchild($newSubnets)  $newSubnet = $workingVnetConfig.CreateElement("Subnet","http://schemas.microsoft.com/ServiceHosting/2011/07/NetworkConfiguration")  $newSubnet.SetAttribute("name","Subnet-1")  $Subnet = $Subnets.appendchild($newSubnet)  $newAddressPrefix = $workingVnetConfig.CreateElement("AddressPrefix","http://schemas.microsoft.com/ServiceHosting/2011/07/NetworkConfiguration")  $newAddressPrefix.InnerText="10.0.0.0/11"  $Subnet.appendchild($newAddressPrefix) 

6. Write out to a file and then use that file

$tempFileName = $env:TEMP + "\azurevnetconfig.netcfg"  $workingVnetConfig.save($tempFileName)   set-AzureVNetConfig -configurationpath $tempFileName 

Next Steps

If you don’t have any Azure networks defined then get-azureVnetConfig will give you nothing. That means that more XML needs to be generated for a new network configuration. I’m working on a more expansive script right now and I’ll post that when I get something meaningful to show.

Content Types programmatically added to SharePoint libraries not appearing on New menu

This one caused some consternation, I can tell you. As usual, the solution could be found on the great wide web, but it took some digging, so as usual I am repeating it here.

As part of a SharePoint migration we did recently, we replaced a SharePoint 2007 feature that the client was using (which added content types to libraries from a central list) with a mix of content type replication and PowerShell to add the content types to the libraries.

The code below scans through any site collections whose url begins with our $hostheader variable, then adds the list of content types to the Shared Document library in every web in the site collection. It’s a simple modification of some code on Phil Childs’ Get-SPScripts.com site and I take no credit for it – it’s all Phil’s work.

# Specify name of the library to look for in each site
#
$hostheader = "http://mywebapp*"
$lookForList = "Shared Documents"
#find our site collections and run through each in turn
# get all the site collections, then step into each site in each site collection
# find the list with the name in the var lookforlist
# then change content types on the list specified
#
# note - this will only find site collections where the url starts with the host header var
#
get-spsite where {$_.url -like $hostheader}| Get-SPWeb -Limit all | ForEach-Object {
    write-host "Checking site:"$_.Title
    #Make sure content types are allowed on the list specified
    $docLibrary = $_.Lists[$lookForList]
    if ($docLibrary -ne $null)
    {
        $docLibrary.ContentTypesEnabled = $true
        $docLibrary.Update()
        # Add site content types to the list
                # change the name in the quotes to the name of your content type
                #
        $ctToAdd = $site.RootWeb.ContentTypes["Word Document"]
        $ct = $docLibrary.ContentTypes.Add($ctToAdd)
        write-host "Content type" $ct.Name "added to list" $docLibrary.Title
                # 
        # Add second site content types to the list
                # change the name in the quotes to the name of your content type
                #
        $ctToAdd = $site.RootWeb.ContentTypes["Excel Spreadsheet"]
        $ct = $docLibrary.ContentTypes.Add($ctToAdd)
        write-host "Content type" $ct.Name "added to list" $docLibrary.Title
                # 
        # Add third site content types to the list
                # change the name in the quotes to the name of your content type
                #
        $ctToAdd = $site.RootWeb.ContentTypes["PowerPoint Presentation"]
        $ct = $docLibrary.ContentTypes.Add($ctToAdd)
        write-host "Content type" $ct.Name "added to list" $docLibrary.Title
                # 
        # Update the library object to commit changes
                #
        $docLibrary.Update()
    }
    else
    {
        write-host "The list" $lookForList "does not exist in site" $_.Title
    }
}

When we ran this through, however, whilst the content types were added correctly to the libraries, the New menu failed to list them.

Much (and I mean much!) digging revealed the cause to be down to a property in the library (spList.RootFolder.UniqueContentTypeOrder) that isn’t automatically set when we add the content types using code (which makes sense when you think about it…). However, all our fiddling with PowerShell failed to work. Adding a content type to the property (which is an array of content types) steadfastly refused to work.

We then found a post on the TechNet forums which appeared to give the answer in the form of C# code. We spent a long time on this, so to cut it short: You can’t simply add an item to the UniqueContentTypeOrder property – you have to set it to Null and rebuild it. The trouble is that in order to do that you have to stuff in an object that has an iList interface. Try as we might, we couldn’t create a PowerShell object that would allow us to store an array of ContentTypes and present the iList interface to the UniqueContentTypeOrder property to set the New menu values. Many people said that ArrayList or SortedList should do it, but they didn’t.

In the end, then I got our devs to knock up a rough and ready command line tool based on the code in the TechNet post. It’s really rough and ready, so I won’t post it here. Follow the posting and get your own tame devs to do the same. I only hope that this article becomes easier to find on the web to save you guys some time.

Powershell to find missing features in SharePoint 2010

When migrating from SharePoint 2007 to 2010, no matter how hard you try there’s always the chance the the content database upgrade process will throw out errors about features being referenced that are not present in the farm. We have used Stefan Goßner’s WssAnalyzeFeatures and WSSRemoveFeatureFromSite (see his original article) to track down the references and exterminate them. It’s not the fastest thing on two legs though, and I have a fondness for having my SharePoint 2010 tooling in PowerShell because of the flexibility it gives me.

Here then, with a big hat-tip to Stefan, is the PowerShell to replicate his functionality. We differ slightly, in that I wanted one command to purge both site- and web-referenced features in one shot rather than calling a command with a scope switch. I have two functions – get-spmissingfeatures takes a Site Collection url as a parameter and scans through site and web feature collections, listing any features with a null definition property. The second function, remove-spmissingfeatures takes the same url and this time finds and removes the errant features in one pass.

Get-SpMissingFeatures:

function get-spmissingfeatures([string]$siteurl)
{
  $site = get-spsite $siteurl

  foreach ($feature in $site.features) {
    if ($feature.definition -eq $null) {
       write-host "Missing site feature:"
       write-host $feature.DefinitionId
       write-host $feature.parent
#      $site.features.remove($featureid)
    }
  }

  $webs = $site | get-spweb -limit all
  foreach ($web in $webs) {
  foreach ($feature in $web.features) {
    if ($feature.definition -eq $null) {
       write-host "Missing web feature:"
       write-host $web.url
       write-host $feature.DefinitionId
       write-host $feature.parent
#      $site.features.remove($featureid)
    }
  }

  }

}

Remove-SPMissingFeatures:

function remove-spmissingfeatures([string]$siteurl)
{
  $site = get-spsite $siteurl

  foreach ($feature in $site.features) {
    if ($feature.definition -eq $null) {
       write-host "Missing site feature:"
       write-host $feature.DefinitionId
       write-host $feature.parent
      $site.features.remove($feature.DefinitionId)
    }
  }

  $webs = $site | get-spweb -limit all
  foreach ($web in $webs) {
  foreach ($feature in $web.features) {
    if ($feature.definition -eq $null) {
       write-host "Missing web feature:"
       write-host $web.url
       write-host $feature.DefinitionId
       write-host $feature.parent
      $web.features.remove($feature.DefinitionId)
    }
  }

  }
}

Powershell script to rename files for use as SharePoint 2010 User Profile thumbnails

User profile photos have changed in SharePoint 2010 in that they are now stored in a single image library in the MySite Host root site collection. They have also changed in that when you change the profile photo, SharePoint takes the file and creates three new images at specific sizes, then discards the file you gave it. These files have specific names to link them to the user account and come in small, medium and large flavours.

We’ve just delivered a solution to a customer that involved heavy customisation of the Profile page for users. This also involved replacing the large thumbnail version of the profile picture with one which met our size requirements.

The customer had a large group of image files all named in the pattern <firstname surname>.<extension> which had been loaded in to SharePoint as profile pictures. We wanted a quick way to replace the large thumbnail with our own version.

Enter Powershell, stage left. The script below is a little rough and ready but works great. It gets a directory listing, splits the filename and then looks up in AD to see if there’s a user that matches the filename (sans extension). If it finds a match it renames the file to match the pattern <domain>_<SamAccountName>_LThumb.<extension> to match the profile picture naming convention.

As I said, it’s a little rough and ready but I place here for the greater good. You need the ActiveDirectory powershell module to use this. It’s available on Server 2008 and above, and Windows 7 if you install the remote management tools.

The Active Directory Powershell Blog is a great resource for this stuff!

#import-module ActiveDirectory
if (-not (get-module -name activedirectory)) {
    write-host "This script requires the ActiveDirectory powershell modules to run"
    exit
}
$domain="mydomain"
$filesuffix = "LThumb"
$files = get-childitem
foreach ($file in $files) {
    $filesplit = $file.Name.split(".")
    $fullname = $filesplit[0]
    $fileext = $filesplit[1]
    write-host "Searching for:" $fullname
    $user = get-aduser -Filter { Name -eq $fullname }
    if ($user.SamAccountName -eq $Null) {
        write-host "Not Found!"
    } else {
        $newfilename = $domain+"_"+$user.SamAccountName+"_"+$filesuffix+"."+$fileext
        write-host "Renaming:" $file.name "New name:" $newfilename
        rename-item $file.Name $newfilename
    }
}