Deploying the ASDK for effective development use

Microsoft Azure Stack is a truly unique beast in terms of the capabilities it can bring to an organisation, and the efficiencies it can bring to a project that spans Public Cloud and on-premises infrastructure through it’s consistency with public Azure.

We’ve been using Stack in anger for a customer project for a year now and have learned several things about development and testing, and how to configure the ASDK to be an effective tool to support the project. This post will summarise those learnings and how I deploy the ASDK so you may mirror our approach for your own projects.

Learning 1: Cloud-consistent, but not quite

Azure Stack is great in that you use the exact same technologies to develop and deploy as its big brother. However, there are some nuances that we’ve hit. Stack lags behind Azure in API versions and provides a subset of Resource Providers. Whilst it is possible to configure your Azure subscription to the same API versions as Stack using policies, that’s not a guarantee of compatibility. In our project a partner org has been using Azure with those policies applied, but they still managed to update the language-specific SDKs for Storage beyond those supported by Stack. You need to be very watchful about version support, and our experience is that there is no replacement for using Stack as part of your pre-production testing effort.

Learning 2: Performance may vary

This one is obvious, if you think about it. Stack is built on different infrastructure than Public Azure because we don’t get things like FPGA-accelerated networking, for example. That means you need to performance test your stuff on Stack or you may be upset that it doesn’t meet the stats you got from Azure. Don’t get me wrong – we’ve put some very big load on both single-box ASDK and multi-node MAS during our project, but stuff like VM provisioning times, network performance and storage read/write times are different than production Azure and if your solution relies on this stuff you need to load test on your target environment.

Learning 3: Identity, identity, identity

Stack can be configured in connected or disconnected modes and that is a big, fundamental different you need to be aware of. In connected mode you use Azure AD for a single source of identity across your Stack and Azure subscriptions. With disconnected mode you are using ADFS to connect Stack to an Active Directory. The key difference is if you are using Service Principals – in disconnected mode you can only do certificate authentication.  This is supported on Azure but not the default. Our approach has been (since our project is disconnected) to do all our dev in an ASDK for absolute confidence of compatibility but with careful governance you could use public Azure. The key difference is that with the ASDK you create a service principal using PowerShell and the certificate gets created at that point for you to extract. With public Azure you need to create the cert first and provide it when creating and service principal.

Learning 4: ‘D’ is for Demo, not Dev; at least out-of-the-box

The ASDK is a wonderful thing. It’s easy to download and deploy a single-node version of Stack for testing. However, the installer delivers the system in a bubble. To use it you have to RDP into the host to access the portal and mess around with VPNs. Connecting to your CI/CD tool of choice (we use VSTS) is hard verging on impossible and at the very least active use for development is impractical.

You also have to content with the fact that the ASDK is built on evaluation media, and that you can’t upgrade it from version to version. I have no argument as to why this is – you shouldn’t be able to use the dev kit as a cheap way of getting Stack into your prod environment, but it means that if you want to use the ASDK for development you need to roll your sleeves up a bit… More on that in a moment.

Learning 5: Matt McSpirit Rocks!

For a long time I kept thinking that I should collect the ragtag bunch of PowerShell I used to deploy our ASDK each month into something resembling a usable solution and publish it. Then Matt McSpirit released ConfigASDK and I no longer needed to! If you use ASDK at all, then you need Matt’s Script because it makes post-install configuration an absolute breeze!

Effective Deployment 1: Hacking the Installer

We now have more than one ASDK and I need to rebuild each one pretty much monthly. That means streamlining the process. To do that, I convert the downloaded VHD to a WIM and we import it into our Config Manager deployment so I can use PXE to easily reimage the servers. Once that’s done I can clear down the storage spaces left over from the previous install and rebuild. I need the ASDKs to be on our network, so I assign a /16 network range for each Stack as well as assigning an IP on our main subnet to the host. I also modify the deployment scripts so I can use a custom region and external DNS suffix which allows us to access the ASDKs over DirectAccess – handy for when we’re working on site with our customer.

To do all this I loosely follow the documentation to deploy using PowerShell.

Reconfigure Network Ranges

The networking for the ASDK is all configured from the OneNodeCustomerConfigTemplate.xml file that is in c:\CloudDeployment\Configuration. I simply open that file and replace all occurances of 192.168. with my own /16 IPv4 range (e.g. 10.1.). That will give you an ASDK with several address ranges and importantly, a full /24 block of IP addresses for services (storage endpoints, Public IP Addresses, etc) that can sit on your network.

You’ll need to configure your own network to publish the route into that /16 address range, with an IP address on your own network the gateway address – we’ll assign this to the VM in stack that does the networking during the installation.

Reconfigure Region and ExternalDomainSuffix

When you deploy AzureStack it will create an AD called Azurestack.local that the infrastructure VMs will join, including the host. You will also find that the portal, admin portal and services are published on a DNS domain of local.azurestack.external. In that domain, ‘local’ is the region and ‘azurestack.external’ is the external domain suffix. Understandably, for a dev/demo deployment these are not exposed to you as things you can change. However, changing them is actually pretty easy when you know where to look.

When you run the InstallAzureStackPOC.ps1 command, it actually calls the DeploySingleNode.ps1 script that is in c:\clouddeployment\setup and if you open that script you’ll find that there are parameters called RegionName and ExternalDomainSuffix and both these params have default values. Simply open that file in an editor and change them. Line 143 contains the $RegionName parameter with a default of ‘local’ so I change that to be the name of my ASDK host; line 130 contains the $ExternalDomainSuffix parameter with a default of ‘azurestack.local’ so I change that to stack.<my internal domain suffix>.

This approach means that I end up with all the ASDKs within stack.mydomain.local, with the name of the host being the region – that’s a good logical approach as far as I and my team are concerned!

Now, doing that isn’t quite all you need to do. Just like with the network routing requirement, you have to get the DNS requests to the Azure Stack DNS or nothing will work. If you follow my approach for the network address range, then the next step is to create some delegated subdomains in your internal DNS. We are using Windows Server and our internal DNS is linked to our AD, so the steps are as follows:

  1. Open the DNS console on your DC.
  2. Within your internal domain, create a HOST record for the stack DNS. I normally call mine <host>stackdns.<mydomain> because I have more than one. Assuming you’re done the neworking changes like me, the IP address for that is <your /16 subnet>.200.67 (e.g. 10.1.200.67).
  3. Within your internal domain create a subdomain of your internal domain called ‘stack’.
  4. Within that stack subdomain, right-click and select New Delegation. In the dialog that pops up enter the name you specified as the RegionName and then when asked for a DNS FQDN enter the hostname you just registered.
  5. If you are using App Services you also need to create a delegation for their domain – <region>.cloudapp.<external domain suffix>. Simply create another subdomain within stack called cloudapp and then repeat the previous step to register the <region> delegation.

Once you’ve configured your internal network and DNS, then modified the config file and script you can perform the installation. When doing so, make sure that in the parameters of the InstallAzureStackPOC.ps1 you specify the NatIPv4Address parameter as the IP address you set in your routing configuration to be the IP address you used in your network router configuration.

Effective Deployment 2: Removing the NAT

This bit hasn’t changed since the preview releases and I follow the notes on the AzureStack blog post. Once the ASDK install completes I simply run a short script that contains the powershell at the end of that post to remove the NAT, at which point that VM sits on my internal network with the IP address I set in the deployment and have configured in my router and traffic just flows.

Effective Deployment 3: ConfigASDK

Matt’s ConfigASDK script is excellent, an I use this to configure our ASDKs post-install. The script will deploy additional resource providers for you (SQL, MySQL and AppServices), install handy utils on the host and pull down a bunch of marketplace offerings you’ll find helpful. For our project we are not using the SQL and MySQL RPs so I skip those, along with the host customisation part.

Now, Matt’s script is built with the Stack team and as such doesn’t currently support the hacks for region and external DNS. I have a fork of the repo in my GitHub where I have added the RegionName and ExternalDomainSuffix params to Matt’s script, but that may not be up to date so you should look at what I’ve done and proceed with caution. I had to put a bunch of code in to replace what were understandably-fixed URLs. However, once I’d done that all Matt’s code worked as expected.

Usual disclaimers apply – use my version of Matt’s script at your own risk. make sure you understand the implications of what I’m describing here before you do this on your own network!

Post-Deployment Configuration Tips

Once you’ve got your ASDK up and running there are still a couple of things you should tweak for a happier time, particularly if you batter the thing as much as we do in terms of creation and deletion of resources and load testing.

Xrp VM RAM

Something you can do with ASDK but not with MAS is get at the VMs that form the fabric of Stack. In an ASDK these are not resilient and are smaller in config than their production counterparts. Our experience is that one of them does not have enough RAM configured by default. The AzS-Xrp01 VM is create with 8192MB of RAM. We see it’s demand climb well beyond this in use and as it does, you may see storage failure messages when trying to create accounts, access the storage APIS or blades in the portal. I give it another 4GB – increasing it to 12288MB in total and that seems to sort it. You don’t need to turn off the VM to do this – just go into Hyper-V Manager and increase the amount.

App Services ScaleSets

As of 1808 you can do this in the portal rather than through PowerShell. When App Services is installed there are a number of VM scalesets created that contain the VMs that host the App Hosting Plans. The thing is, the App Services team don’t know what you need, so the deploy with a single Shared worker tier VM and none in the other tiers. You need to go into the portal and change the instance count to an appropriate number for your project. I normally scale back Shared to zero and set the Small scaleset to a reasonable number based on the project. I also set the Management scaleset to to two instances. I’ve found that makes for a more reliable system, again, because we tend to batter our ASDK during development.

Parting thoughts

Please remember that the approach I describe here is not the way the Stack team expect you to deploy an ASDK. You need to have a reasonable understanding of networking and DNS to know the implications of what I change in my approach to deployment. Also bear in mind that Stack releases monthly updates so if you’re reading this in a year’s time, stuff might have changed so be careful.

Azure Stack is a very powerful solution for a very specific set of customer requirements – it’s not designed to be something everybody should use and you need to remember that. If you do have requirements that fit, hopefully this post will help you configure your ASDK to be a useful component in your development approach.

Configure Server 2016 ADFS and WAP with custom ports using Powershell

A pull request for Chris Gardner’s WebApplicationProxyDSC is now inbound after a frustrating week of trying to automate the configuration of ADFS and WAP on a Server 2016 lab.

With Server 2016, the PowerShell commands to configure the ADFS and WAP servers include switches to specify a non-default port. I need to do this because the servers are behind a NetNat on a server hosting several labs, so port 443 is not available to me and I must use a different port.

This should be simple: Specify the SSLPort switch on the Install-ADFSFarm command and the HttpsPort on the Install-WebApplicationProxy command. However, when I do that, the WAP configuration fails with an error that it cannot read the FederationMetadata from the proxy.

I tried all manner of things to diagnose why this was failing and in the end, the fix is a crazy hack that should not work!

The proxy installation, despite accepting the custom port parameter, does not build the URLs correctly for the ADFS service, so is still trying to call port 443. You can set these URLs on a configured WAP service using the Set-WebApplicationProxyConfiguration command. However, when you run this command with no configured proxy, it fails.

Or so you think…

On the ADFS Server:

  1. Install-AdfsFarm specifiying the CertificateThumbprint, Credential, FederationServiceDisplayName, FederationServiceName and SSLPort params

On the WAP Server:

  1. Install-WebApplicationProxy specifiying the HttpsPort switch,  CertificateThumbprint, FederationServiceName and FederationServiceTrustCredential params.
  2. Set-WebApplicationProxyConfiguration specifying the ADFSUrl, OAuthAuthenticationURL and ADFSSignOutURL parameters with the correct URLs for your ADFS server (which include the port in the Url).
  3. Re-run the command in step 1.

Despite the fact that step 2 says it failed, it seems to set enough information for step 3 to succeed. My experience, however, is that only doing steps 2 and 3 does not work. Weird!

As a side note, testing this lot is a lot easier if you remember that the idpinitiatedsignon.aspx page we all normally use for testing ADFS is disabled by default in Server 2016. Turn it on with Set-AdfsProperties -EnableIdPInitiatedSignonPage $true

Setting Enroll Permissions on ADCS Certificate Template using DSC

As part of the work I have been doing around generating and managing lab environments using Lability and DSC, one of the things I needed to do was change the permissions on a certificate template within a DSC configuration. Previously, when deploying to Azure, I used the PSPKI PowerShell modules within code executed by the Custom Script extension. I was very focused on sticking with DSC this time, which ruled out PSPKI. Whilst there is a DSC module available to configure Certificate Services itself, this does not extend to managing Certificate Templates.

Nobody seemed to have done exactly this before. I used the following links as references in creating the code:

Get Effective template permissions with PowerShell by Vadims Podans

Duplicate AD Object Without Active Directory PS Tools

Add Object Specific ACEs using Active Directory PowerShell

Using Scripts to Manage Active Directory Security

The script finds the WebServer template and grants the Enroll extended permission to the Domain Computers AD group. This allows me to use xCertificate in the DSC configuration of domain member servers to request new certificates using the WebServer template.

Here is the code I include in my DSC configuration. $DomainCreds is a PSCredential object for the domain admin ( I create the AD domain in an earlier step using xActiveDirectory).

        #Enable Enroll on WebServer certificate template         Script EnableWebServerEnroll {             DependsOn = "[xAdcsCertificationAuthority]CertAuth"             PsDscRunAsCredential = $DomainCreds             GetScript = {                 return @{ 'Result' = $true}             }             TestScript = {                 #Find the webserver template in AD and grant the Enroll extended right to the Domain Computers                 $filter = "(cn=WebServer)"                 $ConfigContext = ([ADSI]"LDAP://RootDSE").configurationNamingContext                 $ConfigContext = "CN=Certificate Templates,CN=Public Key Services,CN=Services,$ConfigContext"                  $ds = New-object System.DirectoryServices.DirectorySearcher([ADSI]"LDAP://$ConfigContext",$filter)                 $Template = $ds.Findone().GetDirectoryEntry()                 if ($Template -ne $null) {                     $objUser = New-Object System.Security.Principal.NTAccount("Domain Computers")                     # The following object specific ACE is to grant Enroll                     $objectGuid = New-Object Guid 0e10c968-78fb-11d2-90d4-00c04f79dc55                      ForEach ($AccessRule in $Template.ObjectSecurity.Access) {                         If ($AccessRule.ObjectType.ToString() -eq $objectGuid) {                             If ($AccessRule.IdentityReference -like "*$($objUser.Value)") {                                 Write-Verbose "TestScript: WebServer Template Enroll permission for Domain Computers exists. Returning True"                                 return $true                             }                         }                     }                 }                 return $false             }             SetScript = {                 #Find the webserver template in AD and grant the Enroll extended right to the Domain Computers                 $filter = "(cn=WebServer)"                 $ConfigContext = ([ADSI]"LDAP://RootDSE").configurationNamingContext                 $ConfigContext = "CN=Certificate Templates,CN=Public Key Services,CN=Services,$ConfigContext"                  $ds = New-object System.DirectoryServices.DirectorySearcher([ADSI]"LDAP://$ConfigContext",$filter)                 $Template = $ds.Findone().GetDirectoryEntry()                  if ($Template -ne $null) {                     $objUser = New-Object System.Security.Principal.NTAccount("Domain Computers")                     # The following object specific ACE is to grant Enroll                     $objectGuid = New-Object Guid 0e10c968-78fb-11d2-90d4-00c04f79dc55                     $ADRight = [System.DirectoryServices.ActiveDirectoryRights]"ExtendedRight"                     $ACEType = [System.Security.AccessControl.AccessControlType]"Allow"                     $ACE = New-Object System.DirectoryServices.ActiveDirectoryAccessRule -ArgumentList $objUser,$ADRight,$ACEType,$objectGuid                     $Template.ObjectSecurity.AddAccessRule($ACE)                     $Template.commitchanges()                     Write-Verbose "SetScript: Completed WebServer additional permission"                 }             }         }

Define Once, Deploy Everywhere (Sort of…)

Using Lability, DSC and ARM to define and deploy multi-VM environments

Configuration as code crops up a lot in conversation these days. We are searching for that DevOps Nirvana of a single definition of our environment that we can deploy anywhere.

The solution adopted at Black Marble by myself and my colleagues is not quite that, but it comes close enough to satisfy our needs. This document details the technologies and techniques we adopted to achieve our goal, which sounds simple, right?

I want to be able to deploy a collection of virtual machines to my own computer using Hyper-V, to Dev/Test Labs in Azure, and to Azure Stack, using the same description of those virtual machines and their configuration.

Defining Our Platforms

Right now, we use Lab Manager (part of Team Foundation Server) at Black Marble to manage multi-VM environments for testing, hosted on a number of servers managed by System Center Virtual Machine Manager. Those labs are composed of virtual machines that can also be deployed to a developer’s workstation.

The issue is that those environments are pre-built – the machines are configured and the environment saved as a whole. They must be patched when a new lab is created from the stored ‘template’ VMs and adding a new machine to the lab is a pain.

Lab Manager itself is now a end-of-life, so we are looking at alternatives (including Azure Stack – see below).

Microsoft Azure

We already use Azure to host virtual machines. However, even with the lower cost Dev/Test subscription type, running lots of machines in the public cloud can get very expensive.

Azure Dev/Test Labs helps to mitigate this cost issue somewhat by providing a governance wrapper. I can create a Lab and apply rules, such as what types of virtual machine can be created, and automatically shut down running VMs at a set time to limit costs.

Within Azure we use Azure Resource Templates, which are JSON declarations of the services we require, to deploy our virtual machines. Once running, we have extensions that can be injected into a VM and used to execute scripts to configure them. With Windows servers, that means using the Desired State Configuration (DSC) extension.

Dev/Test labs allows me to connect to a Git repository of artefacts. Those artefacts could be items I wish to install into a VM, but they can also be ARM templates to deploy complex environments of multiple VMs. Those ARM templates can then apply DSC configuration definitions to the VMs themselves.

Microsoft Azure Stack

Stack is coming soon. Right now, you can download a Technical Preview that runs on a single machine. Stack is aimed at organisations that have stuff they cannot put in the public cloud, for whatever reason, but want a consistent approach to their development that can span private and public cloud. The final form of Stack is expected to be similar to the current Cloud Platform Solution (CPS), which is way out of my budget. However, the POC runs on a server very close in specification and price point to my existing Lab Manager-controlled servers.

Stack aims to deliver parity with its public cloud older brother. That means that I can use the same ARM templates I use in Azure to deploy my IaaS services on Stack. I have the same DSC extension to inject my configuration, too.

What I don’t have right now on Stack (and it’s unclear what the final product will bring, so I won’t speculate) are the base operating system images that are provided by Microsoft in Azure. I can, however, create my own images and upload them to the internal Stack equivalent of the Azure Marketplace.

Hyper-V

On our desktops, laptops, and servers we use Hyper-V, Microsoft’s virtualisation technology. This offers some parity with Azure – it uses the same VHD disk file format, for example. I don’t get the same complex software-defined-networking but I still get virtual switches to which I can connect machines, and they can be private, internal, or external.

Private switches do what they say on the tin: They are a bubble within which my VMs can communicate with each other but not with the outside world. I can, therefore, have multiple identical bubbles all using the same IP address ranges without issue.

External switches are connected directly to a network adapter on the host. That’s really useful if I need to host servers that deliver services to my organisation, as I need to communicate with them directly. This is great on servers, and is useful on developer workstations with physical NICs. On laptops, however, it gets tricky if you’re using a WiFi network. Those were never designed with VMs in mind, and the way Windows connects an external switch to a wireless adapter is, quite frankly, a horrible kludge and I’ve always found it terribly unreliable.

Internal switches create a new virtual NIC on the host so it can communicate directly with VMs on the network. In Windows 10, we can use an internal switch alongside a NetNat, which allows Windows 10 to provide network address translation for the virtual network. This gives us a setup like your home internet – VMs can communicate out but there are no direct inbound connection allowed (yes, I know you can create NAT publishing rules too, but that’s not a topic for here).

One cool thing about a NetNat is that if you carefully define your IP address ranges, a single NetNat can pass traffic into the networks generated by multiple virtual switches. This allows me to have multiple environments that can coexist on separate subnets.

Lability

I’ve saved this until last because it’s sort of the secret sauce in what we’ve been working on. I stumbled on Lability totally by chance, and random internet searching. It’s an open source solution to defining and deploying VMs on Windows using DSC to declare both the configuration of the environment (the VMs and their settings) and the VMs themselves (the guest OS configuration).

Lability was created by a chap called Iain Brighton and he deserves a great deal of credit for what he’s built.

With Lability, I can use the same DSC configurations that I created for my Azure deployments. I can use the same base VHD images that I need for my Azure Stack Deployments. Lability uses a DSC PowerShell file (.ps1), which can include configurations for multiple nodes – each of the VMs in our environment. It then uses a PowerShell Data file (.psd1) to declare the configuration of the VMs themselves (CPU, RAM, virtual switch etc) as well as pass in configuration details to the DSC file.

If you look at the Lability repo on GitHub you will find links to some excellent articles by people who have used Lability and take you through setting up your Lability Host (your computer) and your first environment.

Identifying Differences

Applying DSC

Lability and the Azure DSC extension work in a subtly but importantly different manner. When you create a DSC configuration, you write a PowerShell configuration which imports DSC Resources that will do the actual configuration work and you call those resources with specified values that declare the state of the configuration you want. Within that PowerShell file you can put functions that figure out some of those values.

When you execute the PowerShell configuration, it runs through that script and generates a MOF file. That file is submitted to the DSC engine on the machine that you are configuring and used to pass parameters into the DSC Resources that are going to execute commands to apply your configuration.

When you use the DSC extension in Azure, it installs the necessary DSC resources on the VM and executes the PowerShell file on that machine, generating the MOF which is then applied.

When you use Lability, the PowerShell file is executed on the host machine and outputs the MOF files – you do this manually before executing a Lability command to create a new lab. Lability then takes care of injecting the MOF and the required DSC resources into the virtual machine, where the configuration is applied.

This is a critical difference! If you look at the examples in the Azure Quickstart Repo, all the DSC is written assuming that it is executed on the host, and uses PowerShell functions to do things like finding the network adapter, or the host IP address etc. If you look at the examples used in Lability labs, the data file provides many of those pieces of information. If you run the PowerShell from an Azure QuickStart template you’ll have some crazy failures, because all those functions execute on the host and therefore get totally incorrect information to pass to the configuration code.

Additionally, none of the Azure examples use a data file to provide configuration data. You might think this is because the data file is not supported. However, this is not true – you can pass a data file in using the DSC extension. Lability makes heavy use of that data file to define our environment.

Networking

In Azure, you cannot set a static IP address from within the VM itself. The networking fabric hands the machine its IP address via DHCP. You can set that IP to be static through the Azure fabric, but not through the VM. That might mean that we don’t know the IP address of a machine before we deploy it.

With Lability, we declare the IP address of the VM in the DSC data file. We could use a DHCP server running on the host, and I do just that myself, but it’s more stuff to install and manage, and for our approach to labs right now we’ve stuck to declaring address in the DSC data file.

We also have additional stuff to think about in Azure – public IP addresses, Network Security Groups and possibly User Defined Routing that controls how (and if) we allow inbound traffic from the internet onto our network, what can talk to what and on which ports within our network, and whether we want to push all traffic through appliances for security.

Azure API Versions

When you write an ARM template to define and deploy your services, each of the resources in that template is defined against a versioned API. You specify which API version you are using in the template, and different resource providers have different versions.

Azure Stack dos not have all the same versions of the various APIs that are in Azure. Ironically, whilst I have had to make few changes to existing ARM templates in terms of their content in order to successfully use them on Stack, I’ve had to change almost every API version referenced in them. Having said that, I am finding that the API versions I reference for Stack by and large work unchanged if I throw the template at Azure.

Declaring Specific Goals

We’ve discussed our target platforms and talked about how those differ in terms of our deployment configurations. Let’s talk about what our aims were as we embarked on our project to manage VM labs:

  1. All labs should deploy from greenfield. One of our biggest pain points with our old approach was that our labs were built as a collection of VMs. We couldn’t change the name of the AD domain; changing IP address was complex; adding new VMs was painful; patching a ‘new’ environment could take hours.
    We were very clear that we wanted to create all new labs from base media which we would try to keep current for patches (at least within a few months) and would allow us to create any number of machines and environments.
  2. There should be one configuration for each guest VM, which would be used everywhere. We were very clear that we would create one DSC configuration for each role that we needed (for example, a Domain Controller or an ADFS server) and that configuration would be used whether we were creating a lab on a local machine, in Azure or Azure Stack.
  3. Maintain a distinction between a virtual machine configuration and an environment configuration. We are building a collection of virtual Lego with our VM configurations. Our teams can combine those Lego bricks into environments that may be project specific. There should be a configuration for those environments. We should never alter an existing configuration for a new environment – we should create a new configuration using the existing one as a base (for example, we need additional roles on our DC for some reason).
  4. Take a common approach with Lability and Azure, whilst accepting we have to maintain two sets of resources.
    Our approach to Azure environments is already modular. We have templates for VMs that are combined into environments through Nested Deployments. This would not change. Our VM definitions would encompass a DSC configuration and an ARM template. Our environments would include both a DSC data file and an ARM template.
  5. Manage and automated the creation of base media. We would need a variety of base VHD files, analogous to the existing marketplace images in Azure: Windows Server (numerous versions), SQL Server, SharePoint, etc. Each of these must be created using scripts so they could be periodically rebuilt to achieve our goal of avoiding time consuming patching of new environments. In short, we would need an Image Factory.
  6. Setup and use should be straightforward. We need our developers to be able to install all the tooling and get a new lab up and running quickly. We need easy integration with Azure Dev/Test Labs, etc. This would need some process automation around the build and release of the VM configurations and anything else we would create as part of the project.

Things You Will Need

If you want to build the same Lab solution as we did you’re going to need a few things:

  1. Git Repository. All the code and configurations we create are ultimately stored in a central Git Repo. We are using Visual Studio Team Services, as it’s our chosen source control platform.
    Why Git? Two reasons: First of all, it allows us to easily deploy our solution to a developer workstation by simply cloning the repo. Second, Azure DevTest Labs needs a Git Repo to store Artifacts (our ARM templates) for deployment of environments.
  2. Build/Release automation. When we commit to our shared repo, our Build server executes some PowerShell to create deployment artifacts for Azure. It creates Zip archives from our configurations to be used with the DSC extension. It makes no sense to create these by hand and waste space in our repo. Our Release pipeline then automatically pushes our artifacts to an Azure storage account that can be accessed by our developers as a single, central store for VM configurations.
  3. Private PowerShell Repository. We use ProGet to provide a local Nuget/PowerShell/NPM etc repository. We had this in place before we started this project, but it has proved invaluable. The simple reason is that we want to publish DSC Resources or easy consumption and installation by our team. You be surprised at how many times we’ve hit a bug in a DSC resource which has been fixed in the source code repo but a new version has not yet been published. Maintaining our own repository allows us to publish our own versions of DSC resources (and in some case our own bespoke resources).
  4. A server to host your Image Factory. I’m not going to spend time documenting this part of our solution. Far cleverer people than I have written about this and we followed their guidance. You need somewhere to host your images and run the scripts on a schedule to build new ones. Our builds run overnight and we place images on a Windows fileshare.
  5. An Azure subscription. If you want to use the same configuration for on-prem and cloud, saying that you need and Azure sub seems a little obvious. However, we are using nested deployments. These use resources that must be accessible to the Azure fabric at deploy time, and the easiest way to do that is to use Azure Storage. You’ll also need a subscription to host your DevTest lab if that’s your preferred approach. Note that you could have multiple subscriptions – our devs can use their MSDN Azure Benefit to host environments within their own DevTest lab, whilst the artefact store is on a corporate subscription and the artefact repo is in our VSTS.
  6. A code editor that understands PowerShell, DSC and ARM. I prefer Visual Studio and the Azure SDK, but Visual Studio Code is an equally powerful tool for creating and managing the files we are going to use.

Managing our VMs and Environments

After much thought, we came up with a standard folder structure and approach to our VM and environment configurations and the supporting scripts needed to deploy them.

In our code repo we have a the following folder structure:

\Environments

This folder contains a series of folders, one per environment.

This folder is specified as that containing environment templates when the shared repo is connected to an Azure DevTest Lab

\Environment\MyEnv1

An environment folder contains three files:

\Environment\MyEnv1\MyEnv1.psd1

The psd1 data file must share the same name as the folder. This contains all the configuration settings for all VMs in our environment and is used by Lability and the VM DSC configs

\Environment\MyEnv1\azuredeploy.json

For DevTest labs, the environment template used in Azure must be named azuredeploy.json. This template calls a series of other templates to deploy the virtual network and VMs to Azure

\Environment\MyEnv1\metadata.json

This file is read by DevTest labs and provides a name and description for our environment

\VMs

This folder contains subfolders for each of our component Virtual Machines.

\VMs\MyVM1

A VM folder contains at least two files:

\VMs\MyVM1\MyVM1.ps1

The ps1 configuration file must share the same name as the folder. It contains the DSC PowerShell to apply the configuration to the guest VM

\VMs\MyVM1\MyVM1.json

The json file shares the folder name for consistency. It is called by the azuredeploy.json environment template to create the VM in Azure and Azure Stack

\Modules

The Modules folder contains shared code of various types

\Modules\Scripts

The scripts folder contains PowerShell scripts to install and configure our standard Lability deploy, wrapper the Lability create and remove commands and perform build and release tasks.

\Modules\Template

The template folder holds common ARM templates that create standard elements shared between environments and called by the azuredeploy.json

\Modules\DSC

This folder is used during the build process. All the DSC resources needed in an environment are downloaded to this folder. A script parses the VM DSC configurations called by an environment and creates Zip files to be uploaded into Azure storage that contain the correct DSC resources and DSC PowerShell for an environment

Wrapper Scripts for Lability

Lability is great but is built to work in a certain way. We have three scripts that perform key functions for our deployment.

Install Script

Our installation script performs the following function:

  1. Creates the C:\Virtualisation base folder we use to store VMs and the Lability working files.
  2. Sets the default Hyper-V locations for Virtual Machines and Virtual Hard disks to c:\Virtualisation
  3. Creates a new Internal Virtual Switch (named in accordance to our convention) and sets the IP address on the NIC created on the host to the required one. Our first switch creates a network of 192.168.254.0/24 and the host gets 192.168.254.1 as it’s IP address.
  4. Creates a new NetNat with an internal address prefix of 192.168.224.0/19. This will pass traffic into and out of up to thirty /24 subnets starting at 192.168.224.0/24, up to 192.168.254.0/24. We decided to work from the top down when creating new networks.
  5. Makes sure that the Nuget package provider is installed and registers our ProGet server as a new PowerShell repository. We then remove the default PowerShellGallery registration and make sure our repo is trusted.
  6. Check to see if Lability is installed and if not, we install it using Install-Module.
  7. Set the following Lability defaults using the Set-LabHostDefault command:
    ConfigurationPath: c:\Virtualisation\Configuration
    IsoPath: c:\Virtualisation\ISOs
    ParentVhdPath: c:\Virtualisation\MasterVirtualHardDisks
    DifferencingVhdPath: c:\Virtualisation\VMVirtualHardDisks
    ModuleCachePath: c:\Virtualisation\Modules
    ResourcePath: c:\Virtualisation\Resources
    HotfixPath: c:\Virtualisation\Hotfix
    RepositoryUri: <the URI of our ProGet Server, e.g. https://proget.mycorp.com/nuget/PowerShell/package>
  8. Set the default virtual switch for Lability environments to our newly created one using the Set-LabVMDefault command.
  9. Register our VHD base media by calling another script which loads a standard configuration data file. This is separate so we can perform this action independently.
  10. Set the Lability default media to our Windows Server 2012 R2 standard VDH using the Set-LabVMDefault command.
  11. Initialise Lability using our configuration with the Start-LabHostConfiguration command.

Once the install script has completed we have a fully configured host ready to deploy Lability labs.

Deploy-LocalLab script

Lability has a Start-LabConfiguration command which reads the psd1 configuration data file for an environment and creates the VMs. Before running that, however, you need to execute the PowerShell DSC scripts to generate the MOF files for each VM. Lability injects those, and the DSC resources, into the VMs. A second command, Start-Lab boot the VMs themselves, respecting boot order and delays that can be declared in the config file.

This is great unless you have a complex lab and need lots of DSC resources to make it work. Our wrapper script does the following, taking an environment name as a parameter:

  1. Reads the psd1 data file for our environment from the correct folder to identify the DSC resources we need (they are listed for Lability). It installs these resources so we can execute the PowerShell configuration scripts and generate the MOFs.
  2. Reads the psd1 data file to identify the VMs we are deploying. Based on the Role information in that file it will execute each of the configuration ps1 files from the VMs folder hierarchy, passing in the psd1 data file. The resultant MOFs get saved in the Lability configuration folder (c:\Virtualisation\Lability).
  3. Execute the Start-LabConfiguration command passing in the configuration data file.
  4. If we specify a -Start switch, the script starts the lab with the Start-Lab command.

Remove-LocalLab script

Our remove script takes the name of our environment as a parameter. It does the following:

  1. Identifies the VMs in the lab using the Get-LabVM command, passing in the psd1 data file. Check to see if any are running and if they are call the Stop-Lab command.
  2. Executes the Remove-LabConfiguration command, passing in the psd1 data file for the environment.

Virtual Machine Configuration

We’ve challenged ourselves to only use Desired State Configuration for our VMs. This has been a big change from our previous approach to Azure VMs, which mixed DSC with custom PowerShell scripts deployed with a separate Azure VM extension. This has raised four issues we had to solve:

  1. The list of DSC Resources is growing but not all-encompassing. There are many areas where no DSC modules exist. To overcome this, we have used a mix of SetScript code contained within a DSC configuration (which has some limitations) and bespoke DSC modules hosted in our ProGet repository.
  2. Existing Published DSC resources may contain bugs. In many cases code fixing those bugs has been supplied as pull requests but may be undergoing review, and sometimes no new release of the resource has been created. We now have our own separate code repository for DSC resources (including our own) where we keep these and we publish versions to our own repository. When a new official version including the fixes is released it will supersede our own.
  3. There are some good DSC resources out there on GitHub that aren’t published to the PowerShell gallery. We publish these into our own repository for access.
  4. Azure executes the DSC on the target VM to generate the MOF. Lability executes it on the host machine. That and other differences means that we have wrapper code to switch the config sections, mostly based on an input parameter named IsAzure. When called from the Azure DSC extension we specify that parameter and on a Lability host we don’t. I realise that purists will argue that this means we don’t really have a single configuration. I would counter that I have a single configuration file and therefore one thing to maintain. I don’t see any issue with logic inside that config deciding what happens.

Sample Configuration

Let’s illustrate our approach with an extract from a configuration. The code below is part of our DomainController config.

The config accepts some parameters. EnvPrefix is used to generate names within the environment. In Azure we use it to prefix our Azure resources. Within the environment it’s used to create things like the AD domain name. IsAzure tells the config whether it is being executed on the host or on the target VM inside Azure.

You’ll notice that we specify the DSC module versions. There are a few reasons why we do this – because some of the DSC resources are unofficial we want to make sure they come from our repository, and the way Lability downloads DSC resources from our ProGet Server means we need to specify a version number. Either way, we benefit from increased consistency – there have been some breaking changes between versions with the official DSC resources in the PowerShell Gallery!

If we’re in Azure we do things like find the network adapter through code and we don’t specify network addresses. We use the IsAzure parameter to wrapper this stuff in If blocks.

The configuration values come from the psd1 data file, regardless of whether we deploy to Azure or locally. We do this to enforce consistency. Even though we probably could have the Azure config self-contained in the script, we don’t.

 

Configuration DomainController {      param(         [ValidateNotNull()]         [System.Management.Automation.PSCredential]$Credential,          [string]$EnvPrefix,          [bool]$IsAzure = $false,          [Int]$RetryCount = 20,         [Int]$RetryIntervalSec = 30     )      Import-DscResource -ModuleName @{ModuleName="xNetworking";ModuleVersion="3.2.0.0"}     Import-DscResource -ModuleName @{ModuleName="xPSDesiredStateConfiguration";ModuleVersion="6.0.0.0"}     Import-DscResource -ModuleName @{ModuleName="xActiveDirectory";ModuleVersion="2.16.0.0"}     Import-DscResource -ModuleName @{ModuleName="xAdcsDeployment";ModuleVersion="1.1.0.0"}     Import-DscResource -ModuleName @{ModuleName="xComputerManagement";ModuleVersion="1.9.0.0"}      $DomainName = $EnvPrefix + ".local"      Write-Verbose "Processing Configuration DomainController"      Write-Verbose "Processing configuration: Node DomainController"     node $AllNodes.where({$_.Role -eq 'DomainController'}).NodeName {         Write-Verbose "Processing Node: $($node.NodeName)"          if ($IsAzure -eq $true) {             #Find the first network adapter             $Interface = Get-NetAdapter | Where-Object Name -Like "Ethernet*" | Select-Object -First 1             $InterfaceAlias = $($Interface.Name)         }                  LocalConfigurationManager {             RebootNodeIfNeeded = $true;             AllowModuleOverwrite = $true;             ConfigurationMode = 'ApplyOnly'             CertificateID = $node.Thumbprint;             DebugMode = 'All';         }          #ignore this is in Azure         if ($IsAzure -eq $false) {             # Set a fixed IP address if the config specifies one             if ($node.IPaddress) {                 xIPAddress PrimaryIPAddress {                     IPAddress = $node.IPAddress;                     InterfaceAlias = $node.InterfaceAlias;                     PrefixLength = $node.PrefixLength;                     AddressFamily = $node.AddressFamily;                 }             }         }           #ignore this is in Azure         if ($IsAzure -eq $false) {             # Set a default gateway if the config specifies one             if ($node.DefaultGateway){                 xDefaultGatewayAddress DefaultGateway {                     InterfaceAlias = $node.InterfaceAlias;                     Address = $node.DefaultGateway;                     AddressFamily = $node.AddressFamily;                 }             }         }          # Set the DNS server if the config specifies one         if ($IsAzure -eq $true) {             if ($node.DnsAddress){                 xDNSServerAddress DNSaddress {                     Address = $node.DnsAddress;                     InterfaceAlias = $InterfaceAlias;                     AddressFamily = $node.AddressFamily;                 }             }         }          else {             if ($node.DnsAddress){                 xDNSServerAddress DNSaddress {                     Address = $node.DnsAddress;                     InterfaceAlias = $node.InterfaceAlias;                     AddressFamily = $node.AddressFamily;                 }             }         }                  }  #End configuration DomainController }

Sample Data File

Below is a sample data file for an environment containing a Domain Controller and single domain-joined server. Note that the data file contains a mix of data to be processed by the DSC configuration and Lability-specific information that defines the environment, including VM settings and the required DSC resources. When we deploy the lab locally, Lability processes the file to create the Virtual Machines and their hard disks (and create new virtual switches if we declare them). When we deploy in Azure this information is ignored – we can safely use the same data file in both situations.

# Single Domain Controller Lab  @{     AllNodes = @(         @{             # DomainController             NodeName = "DC";             Role = 'DomainController';             DSdrive = 'C:';                          #Prevent credential error messages             PSDscAllowPlainTextPassword = $true;             PSDscAllowDomainUser = $true;               # Networking             IPAddress = '192.168.254.2';             DnsAddress = '127.0.0.1';             DefaultGateway = '192.168.254.1';             PrefixLength = 24;             AddressFamily = 'IPv4';             DnsConnectionSuffix = 'lab.local';             InterfaceAlias = 'Ethernet';               # Lability extras             Lability_Media = 'BM_Server_2012_R2_Standard_x64';             Lability_ProcessorCount = 2;             Lability_StartupMemory = 2GB;             Lability_MinimumMemory = 1GB;             Lability_MaximumMemory = 3GB;             Lability_BootOrder = 0;             Lability_BootDelay = 600;         };         @{             # MemberServer             NodeName = "SR01";             Role = 'MemberServer';             DSdrive = 'C:';                          #Prevent credential error messages             PSDscAllowPlainTextPassword = $true;             PSDscAllowDomainUser = $true;               # Networking             IPAddress = '192.168.254.3';             DnsAddress = '192.168.254.2';             DefaultGateway = '192.168.254.1';             PrefixLength = 24;             AddressFamily = 'IPv4';             DnsConnectionSuffix = 'lab.local';             InterfaceAlias = 'Ethernet';               # Lability extras             Lability_Media = 'BM_Server_2012_R2_Standard_x64';             Lability_ProcessorCount = 2;             Lability_StartupMemory = 2GB;             Lability_MinimumMemory = 1GB;             Lability_MaximumMemory = 3GB;             Lability_BootOrder = 1;         };      );      NonNodeData = @{         OrganisationName = 'Lab';          Lability = @{             EnvironmentPrefix = 'Lab-';              DSCResource = @(                 @{ Name = 'xNetworking'; RequiredVersion = '3.2.0.0';}                 @{ Name = 'xPSDesiredStateConfiguration'; RequiredVersion = '6.0.0.0';}                 @{ Name = 'xActiveDirectory'; RequiredVersion = '2.16.0.0';}                 @{ Name = 'xAdcsDeployment'; RequiredVersion = '1.1.0.0';}                 @{ Name = 'xComputerManagement'; RequiredVersion = '1.9.0.0';}             );         }      }; };  

Azure DSC Extension

Our Azure deployment uses the configuration and data file to configure the VM. The JSON for the DSC extension is shown below. Notice the following:

1. The modulesUrl setting specifies a Zip file that contains the DSC resources and configuration ps1 file. We create these zip files as part of our build process and upload them to an Azure storage account.

2. The configurationFunction setting specifies the name of the ps1 file to execute and the configuration within that we want to apply (a single file can contain more than one configuration, although ours don’t).

3. We pass in the EnvPrefix variable and set the IsAzure value to 1 so our configuration executes the right code.

4. The dataBlobUri within protectedSettings is our psd1 data file. The extension treats this as containing sensitive information – things held in this section are not displayed in any output from Azure Resource Manager.

In fairness, whilst at the moment we create JSON specific to each VM, I plan to refactor this to be common code that takes parameters rather than having an ARM template for each VM’s DSC.

      {         "name": "[concat(parameters('envPrefix'),parameters('vmName'),'/',parameters('envPrefix'),parameters('vmName'),'dsc')]",         "type": "Microsoft.Compute/virtualMachines/extensions",         "location": "[parameters('VirtualNetwork').Location]",         "apiVersion": "[parameters('ApiVersion').VirtualMachine]",         "dependsOn": [         ],         "tags": {           "displayName": "DomainController"         },         "properties": {           "publisher": "Microsoft.Powershell",           "type": "DSC",           "typeHandlerVersion": "2.1",           "autoUpgradeMinorVersion": true,           "settings": {             "modulesUrl": "[concat(parameters('artifactsLocation'), '/Environments/', parameters('envConfig'),'/',parameters('envConfig'),'.zip', parameters('artifactsSasToken'))]",             "configurationFunction": "DomainController.ps1\\DomainController",             "properties": {               "EnvPrefix": "[parameters('EnvPrefix')]",               "Credential": {                 "userName": "[parameters('adminUsername')]",                 "password": "PrivateSettingsRef:adminPassword"               },               "IsAzure": 1             }           },           "protectedSettings": {             "dataBlobUri": "[concat(parameters('artifactsLocation'), '/Environments/', parameters('envConfig'), '/', parameters('envConfig'),'.psd1', parameters('artifactsSasToken'))]",             "Items": {               "adminPassword": "[parameters('adminPassword')]"             }           }         }       }  

We don’t include the DSC extension within the ARM template that deploys the VM because by doing so we can sequence the deployment of configuration to deal with dependencies between servers.

Azure ARM Templates

The approach we take to deploying VMs in Azure has been consistent for some time now. My ResourceTemplates Repo in GitHub uses nested templates to deploy a three-server environment and we use exactly the same approach here. Our ‘master template’ is stored in the environment folder and it calls nested deploys for each VM, VM DSC extension and supporting stuff such as virtual networks. The VM and DSC templates are stored in the VM folder with the DSC config, and the supporting templates are in our Modules\Templates folder since they are shared.

Conclusion

This has been a very long article without a great deal of code in it. I hope this explains how we approach our environment definition and deployment. I plan to do more posts that document more specific elements of a configuration or an environment.

Ultimately, I’m not sure that the goal of a single definition that covers multiple platforms and both host and guest configurations exists. However, I think we’ve got pretty close with our solution and it has minimal rework involved, particularly once you have built up a good library of VM configs that you can combine into an environment.

I should also point out that we are not installing apps – we are deploying a platform onto which our developers and testers can then install the applications they develop. This means that we keep the environments quite generic. Deployment of apps is still scripted (and probably uses VSTS Release Management) but is not included in the configurations we build. Having said that, there is nothing stopping a team extending the DSC to deploy their applications and thus build a more bespoke definition.

I’ve spoken to quite a few people about what we’ve done over the past few weeks and, certainly within the Microsoft space many people want to do what we have done, but few were aware that tooling such as Lability and DSC were available to get it done. I hope this goes some way to plugging that gap.

Creating Website Slots and SQL Elastic Pools using Azure Resource Templates

Recently I have been helping a number of organisations automate the deployment of their applications to Azure and came across a couple of scenarios that were not documented: Deploying an App Services web site with slots and SQL connection string settings, and the creation of a SQL Elastic Pool. Of those, the SQL Elastic Pool I found to be written up already by Vincent-Philipe Lauzon and all credit to him – my template draws on his excellent article.

The Web slots and configuration, however, I didn’t find. There are templates that deploy a web site, and some that deploy configuration settings into that web site (indeed, creating a new Web+SQL template through Visual Studio does just that). However, I could find none that deployed slots and none that added the config to the slot.

You can find the full template in my GitHub Repo. The template code to deploy a slot and associated config is shown below. This sits in the nested resources bock within the website resource, for reference.

The trick with the config, as it turns out, is the resource type. If you examine the connectionStrings node within a slot through Resource Explorer you will see it reported as Microsoft.Web/sites/config. However, if you click the PowerShell tab for the same note you will see the type reported as Microsoft.Web/sites/slots/config. Make sure that the resource name matches the config section (i.e. connectionStrings – or appsettings, etc).

{           "apiVersion": "2015-08-01",           "name": "[concat(variables('website').websiteName, '/', variables('website').slotName)]",           "type": "Microsoft.Web/Sites/slots",           "location": "[resourceGroup().location]",           "dependsOn": [             "[concat('Microsoft.Web/Sites/', variables('website').websiteName)]"           ],           "tags": {             "displayName": "Slot"           },           "properties": {           },           "resources": [             {               "apiVersion": "2015-08-01",               "name": "[concat(variables('website').websiteName, '/', variables('website').slotName, '/connectionStrings')]",               "type": "Microsoft.Web/Sites/slots/config",               "location": "[resourceGroup().location]",               "dependsOn": [                 "[concat('Microsoft.Web/Sites/', variables('website').websiteName, '/slots/', variables('website').slotName)]"               ],               "tags": {                 "displayName": "SlotConnectionStrings"               },               "properties": {                 "DefaultConnection": {                   "value": "[concat('Data Source=tcp:', reference(concat('Microsoft.Sql/servers/', variables('sqlServer').name)).fullyQualifiedDomainName, ',1433;Initial Catalog=', variables('sqlServer').stagingDbname, ';User Id=', parameters('sqlAdminLogin'), '@', variables('sqlServer').name, ';Password=', parameters('sqlAdminPassword'), ';')]",                   "type": "SQLServer"                 }               }             }           ]         } 

Notes from the field: Using Hyper-V Nat Switch in Windows 10

The new NAT virtual switch that can be created on Windows 10 for Hyper-V virtual machines is a wonderful thing if you’re an on-the-go evangelist like myself. For more information on how to create one, see Thomas Maurer’s post on the subject.

This post is not about creating a new NAT switch. It is, however, about recreating one and the pitfalls that occur, and how I now run my virtual environment with some hack PowerShell and a useful DHCP server utility.

Problems Creating Nat Switch? Check Assigned IP Addresses

I spent a frustrating amount of time this week trying to recreate a NAT switch after deleting it. Try as I might, every time I executed the command to create the new switch it would die. After trial and error I found that the issue was down to the address range I was using. If I created a new switch with a new address range everything worked, but only that one time: If I deleted the switch and tried again, any address range that I’d used would fail.

This got me digging.

I created a new switch with a new address range. The first thing I noticed was that I had a very long routing table. Get-netroute showed routes for all the address ranges I had previously created. That let me to look at the network adapter created by the virtual switch. When you create a new nat switch the resulting adapter gets the first IP address in the range bound to it (so 192.168.1.0/24 will result in an IP of 192.168.0.1). My adapter had an IP address for every single address range I’d created and then deleted.

Obviously, when the switch is removed the IP configuration is being stored by windows somewhere. When a new switch is created all that old binding information is reapplied to the new switch. I’m not certain whether this is related to the interface index, name or what, since when I remove and re-add the switch on my machine it always seems to get the same interface index.

A quick bit of PowerShell allowed me to rip all the IP addresses from the adapter at once. The commands below are straightforward. The first allows me to find the adapter by name (shown in the Network Connections section of control panel) – replace the relevant text with the name of your adapter. From that I can find the interface index, and the second command gets all the IPv4 addresses (only IPv4 seems to have the problem here) and removes them from the interface – again, swap your interface index in here. I can then use PowerShell to remove the VMswitch and associated NetNat object.

Get-NetAdapter -Name "vEthernet (NATSwitch)" Get-NetIPAddress -InterfaceIndex 13 -AddressFamily IPv4 | Remove-NetIPAddress

Once that’s done I can happily create new virtual switches using NAT and an address range I’ve previously had.

Using DHCP on a NAT switch for ease

My next quest was for a solution to the IP addressing conundrum we all have when running VMs: IP addresses. I could assign each VM a static address, but then I have to keep track of them. I also have a number of VMs in different environments that I want to run and I need external DNS to work. DHCP is the answer, but Windows 10 doesn’t have a DHCP server and I don’t want to build a VM just to do that.

I was really pleased to find that somebody has already written what I need: DHCP Server for Windows. This is a great utility that can run as a service or as a try app. It uses an ini file for configuration and by editing the ink file you can manage things like address reservations. Importantly, you can choose which interface the service binds to which means it can be run only against the virtual network and not a use issues elsewhere.

There’s only one thing missing: DNS. Whilst the DHCP serer can run it’s own DNS if you like, it still has a static configuration for the forwarder address. In a perfect world I’d like to be able to tell it to had my PCs primary DNS address to clients requesting an IP.

Enter PowerShell, stage left…

Using my best Google-fu I tracked down a great post by Lee Homes from a long time ago about using PowerShell to edit ini files through the old faithful Windows API calls for PrivateProfileString. I much prefer letting Windows deal with my config file than write some complex PowerShell parser.

I took Lee’s code and created a single PowerShell module with three functions as per his post which I called Update-Inifiles.psm1. I then wrote another script that used those functions to edit the ini file for DHCPserver.

It’s dirty and not tested on anything but my machine, but here it is:

import-module C:\src\Update-IniFiles.psm1  $dnsaddr = (Get-DnsClientServerAddress -InterfaceIndex (get-netroute -DestinationPrefix 0.0.0.0/0)[0].ifIndex -AddressFamily IPv4).ServerAddresses[0]  if ($dnsaddr.Length -gt 0) { Set-PrivateProfileString "C:\Program Files\DHCPSrv\dhcpsrv.ini" GENERAL DNS_0 $dnsaddr } else { Set-PrivateProfileString "C:\Program Files\DHCPSrv\dhcpsrv.ini" GENERAL DNS_0 8.8.8.8 } 

The second line is the one that may catch you out. It gets the DNS server information for the interface that is linked to the default IPv4 route. On my machine there are multiple entries returned by the get-netroute command, so I grab the first one from the array. Similarly, there are multiple DNS servers returned and I only want the first one of those, too. I should really expand the code and check what’s returned, but this is only for my PC – edit as you need!

Just in case I get nothing back I have a failsafe which is to set the value to the Google public DNS server on 8.8.8.8.

Now I run that script first, then start my DHCP server and all my VMs get valid IP information and can talk on whatever network I am connected to, be it physical or wireless.

Unblocking a stuck Lab Manager Environment (the hard way)

This is a post so I don’t forget how I fixed access to one of our environments yesterday, and hopefully it will be useful to some of you.

We have a good many pretty complex environments deployed to our lab hyper-V servers, controlled by Lab manager. Operations such as starting, stopping or repairing those environments can take a long, long time, but this time we had one that was quite definitely stuck. The lab view showed the many servers in the lab with green progress bars about halfway across but after many hours we saw no progress. The trouble is, at this point you can’t issue any other commands to the environment from within the Lab Manager console – it’s impossible to cancel the operation and regain access to the environment.

Normally in these situations, stepping from Lab Manager to the SCVMM console can help. Stopping and restarting the VMs through SCVMM can often give lab manager the kick it needs to wake up. However, this time that had no effect. We then tried restarting the TFS servers to see if they’d got stuck, but that didn’t help either.

At this point we had no choice but to roll up our sleeves and look in the TFS database. You’d be surprised (or perhaps not) at how often we need to do that…

First of all we looked in the LabEnvironment table. That showed us our environment, and the State column contained a value of Repairing.

Next up, we looked in the LabOperation table. Searching for rows where the DataspaceId column value matched that of our environment in the LabEnvironment table showed a RepairVirtualEnvironment operation.

In the tbl_JobSchedule table we found an entry where the JobId column matched the JobGuid column from the LabOperation table. The interval on that was set to 15, from which we inferred that the repair job was being retried every fifteen minutes by the system. We found another entry for the same JobId in the tbl_JobDefinition table.

Starting to join the dots up, we finally looked in the LabObject database. Searching for all the rows with the same DataspaceId as earlier returned all the lab hosts, environments and machines that were associated with the Team Project containing the lab. In this table, our environment row had a PendingOperationId which matched that of the row in the LabOperation table we found earlier.

We took the decision to attempt to revive our stuck environment by removing the stuck job. That would mean carefully working through all the tables we’d explored and deleting the rows, hopefully in the correct order. As the first part of that, we decided to change the value of the State column in the LabEnvironment table to Started, hoping to avoid crashing TFS should it try to parse all the information about the repair job we were about to slowly remove.

Imagine our surprise, then, when having made that one change, TFS itself cleaned up the database, removed all the table entries referring to the repair environment job and we were immediately able to issue commands to the environment again!

Net Writer: A great UWP blog editor

I came across Net Writer some months ago, when it’s creator, Ed Anderson blogged about how he’d taken the newly-released Open Live Writer code and used it in his just-started Universal Windows Platform (UWP) app for Windows 10. In January it only supported blogger accounts, which meant that I was unable to use it. However, I checked again this weekend and discovered that it now supports a wide range of blog software including BlogEngine.net that powers blogs.blackmarble.co.uk.

I’m writing this post using the app. It’s great for quick posts (there’s no plugin support so posting code snippets is tricky) and most importantly, it works on my phone! That’s the big win as far as I’m concerned. I’ve been hankering for the ability to easily manage my blog form my phone for a long time and now I can.

You can find Net Writer in the Windows Store and learn more about it at Ed’s blog.

My Resource Templates from demos are now on GitHub

I’ve had a number of people ask me if I can share the templates I use in my Resource Template sessions at conferences. It’s taken me a while to find the time, but I have created a repo on GitHub and there is a new Visual Studio solution and deployment project with my code.

One very nice feature that this has enabled me to provide is the same ‘Deploy to Azure’ button as you’ll find in the Azure Quickstart Templates. This meant a few changes to the templates – it turns out that Github is case sensitive for file requests, for example, whilst Azure Storage isn’t. The end result is that you can try out my templates in your own subscription directly from Github!

Installing Windows 10 RSAT Tools on EN-GB Media-Installed Systems

This post is an aide memoir so I don’t have to suffer the same annoyance and frustration at what should be an easy task.

I’ve now switched to my Surface Pro 3 as my only system, thanks to the lovely new Pro 4 Type Cover and Surface Dock. That meant that I needed the Remote Server Administration Tools installing. Doing that turned out to be much more of an odyssey that it should have been and I’m writing this in the hope that it will allow others to quickly find the information I struggled to.

The RSAT tools download is, as before, a Windows Update that adds the necessary Windows Features to your installation. The trouble is, that download is EN-US only (really, Microsoft?!). If, like me, you used the EN-GB media to install you’re in a pickle.

Running the installed appears to work – it proceeds with no errors, albeit rather quickly – but the RSAT features were unavailable. I already had a US keyboard on my config (my pro keyboard is US), but that was obviously not enough. I added the US language, but still couldn’t get the installer to work.

I got more information on the problem by following the steps described in a TechNet article on using DISM to install Windows Updates. That led me to a pair of articles on the SysadminTips site about the installation problem, and how to fully add the US language pack to solve it.

It turns out that the EN-GB media doesn’t install the full US-EN language pack files, so when you add the US language it doesn’t add enough stuff into the OS to allow the RSAT tools. Frankly, that’s a mess and I hope Microsoft deal with the issue by releasing multi-language RSAT tools.