What I wish I had known when I started developing Lability DevTest Lab Environments

At Black Marble we have been migrating our DevTest labs to from on-premises TFS Lab Management to a mixture of on-premise and Azure hosted Lability defined Labs as discussed by Rik Hepworth on his blog. I have only been tangentially involved in this effort until recently, consuming the labs but not creating the definitions.

So this post is one of those I do where I don’t want to forget things I learnt the hard way, or to put it another way asking Rik or Chris after watching a 2 hour environment deploy fail for the Xth time.

  • You can’t log tool much. The log files are your friends, both the DSC ones and any generated by tools triggered by DSC. This is because most of the configuration process is done during boots so there is no UI to watch.
  • The DSC log is initially created in working folder the .MOF file is in on the target VM; but after a reboot (e.g. after joining a domain) the next and subsequent DSC log files are created in  C:\Windows\System32\Configuration\ConfigurationStatus
  • Make sure you specify the full path for any bespoke logging you do, relative paths make it too easy to lose the log file
  • Stupid typos get you every time, many will be spotted when the MOF file is generated, but too many such as ones in command lines or arguments are only spotted when you deploy an environment. Also too many of these don’t actually cause error messages, they just mean nothing happens. So if you expect a script/tool to be run and it doesn’t check the log and the definition for mismatches in names.
  • If you are using the Package DSC Resource to install an EXE or MSI couple of gotcha’s
    • For MSIs the ProductName parameter must exactly match the one in the MSI definition, and this must match the GUID ProductCode.  Both of these can be found using the Orca tool

      image

    • Package MongoDb {

      PsDscRunAsCredential = $DomainCredentialsAtDomain

      DependsOn = '[Package]VCRedist'

      Ensure = 'Present'

      Arguments = "/qn /l*v c:\bootstrap\MongoDBInstall.log INSTALLLOCATION=`"C:\Program Files\MongoDB\Server\3.6\`""

      Name = "MongoDB 3.6.2 2008R2Plus SSL (64 bit)"

      Path = "c:\bootstrap\mongodb-win32-x86_64-2008plus-ssl-3.6.2-signed.msi"

      ProductId = "88B5F0D8-0692-4D86-8FF4-FB3CDBC6B40F"

      ReturnCode = 0

      }

    • For EXEs the ProductName does not appear to be as critical, but you still need the Product ID. You can get this with PowerShell on a machine that already has the EXE installed
    • Get-WmiObject Win32_Product | Format-Table IdentifyingNumber, Name, Version
  • I had network issues, they could mostly be put does to incorrect Network Address Translation. In my case this should have been setup when Lability was initially configured, the commands ran OK creating a virtual switch and NetNat, but I ended up with a Windows failback network address of 169.x.x.x when I should have had an address of 192.168.x.x on my virtual switch. So if in doubt check the settings on your virtual switch, in the Windows ‘Networking and Share Center’ before you start doubting your environment definitions.

Hope these pointers help others, as well as myself, next time Lability definitions are written

Speaking at Leeds DevOps on the 21st of July

I will be speaking at Leeds DevOps on the 21st of July on the subject of Desired State Configuration (DSC).

‘In the Windows world, due to its API based architecture, deployment is too often not as simple as copying an EXE and updating a text configuration file. Desired State Configuration is an attempt to ease the pain we suffer in this space. Providing a set of tools that can be leveraged by any set of deployment tools whether in a Windows or heterogeneous environment. In this session we will look at what DSC is, what resource are available and how to write your own’.

The event is at the The Node in Leeds, tickets are free and are available over on Eventbrite or meetup.com

Using Release Management vNext templates when you don’t want to use DSC scripts

Update 21 Aug 2015 – This post contains all the basic information, but there is an improved PowerShell script discussed in Using Release Management vNext templates when you don’t want to use DSC scripts – A better script


Many web sites are basically forms over data, so you need to deploy some DB schema and something like a MVC website. Even for this ’bread and butter’ work it is important to have an automated process to avoid human error. Hence the rise in use of release tools to run your DACPAC and MSDeploy packages.

In the Microsoft space this might lead to the question of how Desired State Configuration (DSC) can help? I, and others, have posted in the past about how DSC can be used to achieve this type of deployment, but this can be complex and you have to ask is DSC the best way to manage DACPAC and MSDeploy packages? Or is DSC better suited to only the configuration of your infrastructure/OS features?

You might ask why would you not want to use DSC, well the most common reason I see is that you need to provide deployment script to end clients who don’t use DSC, or you have just decided want basic PowerShell. Only you will be able to judge which is the best for your systems, but I thought it worth outlining an alternative way to do deployment of these package using Release Management vNext pipelines that does not make use of DSC.

Background

Let us assume we have a system with a SQL server and a IIS web server that have been added to the Release Management vNext environment. These already have SQL and IIS enabled, maybe you used DSC for that?

The vNext release template allows you to run either DSC or PowerShell on the machines, we will ignore DSC, so what can you do if you want to use simple PowerShell scripts?

Where do I put my Scripts?

We will place the PowerShell scripts (and maybe any tools they call) under source control such that they end up in the build drops location, thus making it easy for Release Management to find them, and allowing the scripts (and tools) to be versioned.

Deploying a DACPAC

The script I have been using to deploy DACPACs is as follows

# find the script folder
$folder = Split-Path -parent $MyInvocation.MyCommand.Definition
Write-Verbose "Deploying DACPAC $SOURCEFILE using script in '$folder'"
& $folder\sqlpackage.exe /Action:Publish /SourceFile:$folder\..\$SOURCEFILE /TargetServerName:$TARGETSERVERNAME /TargetDatabaseName:$TARGETDATABASENAME | Write-Verbose -Verbose

Note that:

  1. First it finds the folder it is running in, this is the easiest way to find other resource I need
  2. The only way any logging will end up in the Release Management logs is if is logged at the verbose level i.e. write-verbose “your message” –verbose
  3. I have used a simple & my.exe to execute my command, but pass the output via the write-verbose cmdlet to make sure we see the results. The alternative would be to use invoke-process
  4. SQLPACKAGE.EXE (and its associated DLLs) are located in the same SCRIPTS folder as the PowerShell script and are under source control. Of course you could make sure any tools you need are already installed on the target machine.

I pass the three parameters need for the strips as custom configuration

image

Remember that you don’t have to be the SQL server to run SQLPACKAGE.EXE, it can be run remotely (that is why in the screen shot above the ServerName is ISS IIS8 not SQL as you might expect)

Deploying a MSDeploy Package

The script I use to deploy the WebDeploy package this is as follows

function Update-ParametersFile
{
    param
    (
        $paramFilePath,
        $paramsToReplace
    )

    write-verbose "Updating parameters file '$paramFilePath'" -verbose
    $content = get-content $paramFilePath
    $paramsToReplace.GetEnumerator() | % {
        Write-Verbose "Replacing value for key '$($_.Key)'" -Verbose
        $content = $content.Replace($_.Key, $_.Value)
    }
    set-content -Path $paramFilePath -Value $content

}


# the script folder
$folder = Split-Path -parent $MyInvocation.MyCommand.Definition
write-verbose "Deploying Website '$package' using script in '$folder'" -verbose

Update-ParametersFile -paramFilePath "$folder\..\_PublishedWebsites\$($package)_Package\$package.SetParameters.xml" -paramsToReplace @{
      "__DataContext__" = $datacontext
      "__SiteName__" = $siteName
      "__Domain__" = $Domain
      "__AdminGroups__" = $AdminGroups

}

write-verbose "Calling '$package.deploy.cmd'" -verbose
& "$folder\..\_PublishedWebsites\$($package)_Package\$package.deploy.cmd" /Y | Write-Verbose -verbose

Note that:

  1. First I declare a function that I use to replace the contents of the package.setparameters.xml file, a key step in using binary promotion and WebDeploy
  2. Again I finds the folder the script is running in so I can locate other resources
  3. I then declare the parameters I need to replace and call the replacement function 
  4. Finally I call the package.deploy.cmd command, and pass the output via the write-verbose to pass the output to the Release Management logs

This is called as follows

image

Summary

So I think these reusable scripts give a fairly  easy way to make use of  vNext Release Management pipelines. They can also easily be given to clients who just want to manually run something.