Steve Spencer's Blog

The blog of Steve Spencer

Setting Custom Domain for Traffic Manager and Azure Websites

Recently I’ve been looking at using traffic manager to front up websites hosted in Azure Websites. I needed to setup a custom domain name instead of using mydomain.trafficmanager.net.

In order to use Traffic Manager with an Azure website the website needs to be setup using a Standard Hosting Plan.

Each website you want to be included in the traffic manager routing will need to be added as an endpoint in the traffic manager portal.

Once you have this setup you will need to add the DNS CNAME record for your domain. This needs to be configured at your Domain provider. You set the CNAME to point to mydomain.trafficmanager.net

In order for the traffic to be routed to your Azure hosted website(s), each website setup as an endpoint in traffic manager will need to have your mapped domain e.g. www.mydomain.com  configured. This is done under settings->Custom Domains and SSL in the new portal and under the configure tab –> manage domains (or click the Manage Domains button)

If you don’t add this then you will see this 404 error page whenever you try to navigate to the site through the traffic manager custom domain name:

image

Azure Websites: Blocking access to the azurewebsites.net url

I’ve been setting up one of our services as the backend service for Azure API management. Part of this process we have mapped DNS to point to the service. As the service is hosted in Azure Websites there are now two urls that exist which can be used to access the service. I wanted to stop a user from accessing the site using the azurewebsites.net url and only access it via the mapped domain. This is easy to achieve and can be configured in the web.config file of the service.

In the <system.webServer> section add the following configuration

<rewrite>
    <rules>
        <rule name="Block traffic to the raw azurewebsites url"  patternSyntax="Wildcard" stopProcessing="true">
          <match url="*" />
          <conditions>
            <add input="{HTTP_HOST}" pattern="*azurewebsites.net*" />
          </conditions>
          <action type="CustomResponse" statusCode="403" statusReason="Forbidden"
          statusDescription="Site is not accessible" />
        </rule>
    </rules>
</rewrite>

Now if I try and access my site through the azurewebsites.net url, I get a 403 error, but accessing through the mapped domain is fine.

Azure Media Services Live Media Streaming General Availability

Yesterday Scott Guthrie announced a number of enhancements to Microsoft Azure. One of the enhancements is the General Availability of Azure Media Services Live Media Streaming. This gives us the ability to stream live events on a service that has already been used to deliver big events such as the 2014 Sochi Winter Olympics and the 2014 FIFA World Cup.

I’ve look at this for a couple of our projects and found it relatively fast and easy to set up a live media event even from my laptop using its built in camera. There’s a good blob post that walks you through the process of setting up the Live Streaming service. I used this post and was quickly streaming both audio and video from my laptop.

The main piece of software that you need to install is a Video/Audio Encode the supports Smooth Streaming or RTMP. I used the WireCast encoder as specified in the post. You can try out the encoder for 2 weeks as long as you don’t mind seeing the Wirecast Logo on your video (which is removed if you buy a license). Media services pricing can be found here

The Media Services team have provided a MPEG-DASH player to help you test your live streams.

It appears that once you have created a stream that is is still accessible on demand after the event has completed.Also there is around a 20s delay when you receive the stream on your player.

Azure Websites Slots and Configuration

One of the conundrums we have with deploying sites to test means that there is often a lot of configuration that is needed on a test site that is different to a live site. There is also the time and risk of deploying a new instance into the production sites once testing has completed.

Azure websites has introduced deployments slots which allows you to have multiple deployments and swap between them in a similar way you could do with the production and staging slots in cloud services. Websites has the added advantage that you can have more than two slots and you can call them whatever you want.

One approach we are looking at to ensure consistency with what is deployed is to configure up a number of slots on the website for a variety of uses e.g. Production, Staging, UAT. The issue with having multiple slots is that there are often sets of configurations that are required to ensure that each slot will work with the correct backend. By default all configuration stored in the appsettings in web.config will move with the slot. Details of the exact configuration settings that move with the deployment can be found here (http://azure.microsoft.com/en-gb/documentation/articles/web-sites-staged-publishing/)

For example, in my web.config file I have the following setting

<appSettings>

<add key="about" value="This is the web.config text" />

</appSettings>

This setting can be overridden in the Azure portal(s) and these by default will follow the deployment and not stay with the slot.

image

So in this example the "about" config will be set to "This is Now the Staging slot" and when the staging slot is swapped to be production, the new production configuration will also be "This is Now the Staging slot"

This is not necessarily what you want on production. Websites has a feature, that is currently unsupported by the management portal(s), which allows specific configuration items to become sticky i.e. they stay with the slot. There is a powershell cmdlet which allows individual appsettings to be marked as sticky and remain with the slot regardless of the deployment that is in the slot and they will also remain in the slot when the slots are swapped.

This can be set for both Appsettings and connection strings by running the following commands

Set-AzureWebsite -Name somesite -SlotStickyAppSettingNames @("about", "another_config_key")

Set-AzureWebsite -Name somesite -SlotStickyConnectionStringNames @("a_connection_string", "some_other_connectionstring")

After running the commands the example above will still have the configuration setting above, but once the deployment is swapped from the staging slot to production the configuration will remain on the staging slot.

This approach should now allow us to deploy to a UAT slot with UAT configuration and allow the customer to test, when they are happy we can move the same deployment that has just been tested to the staging slot with production configuration and be tested in isolation to live to ensure that it works. When you are happy that the staging slot is working this can then be swapped out to production.

For a more detailed introduction to slots and configuration see:

http://azure.microsoft.com/en-gb/documentation/articles/web-sites-staged-publishing/

http://blog.amitapple.com/post/2014/11/azure-websites-slots/#.VG22ik1yaAg

Moving an Azure Website to a separate set of Virtual Machines

When an Azure Website is created and is in production it will most likely be running in a Standard or Basic configuration. These are both sets of Virtual Machines and can be shared across your websites. In the old portal you could only scale the group of websites together but the new Azure Management portal now allows you to move your websites on to different virtual machines so that if one site is more heavily loaded than others it can be scaled out separately if required. The set of virtual machines is known as a Web Hosting Plan. If you want to move one or more of your websites to a different set of virtual machines then you will need to create a new web hosting plan for this.

In the new portal click on “Browse” in the left hand bar

image

This brings up the Browse Menu.

image

Select “Web hosting plans”

image

You can see that I only have 1 web hosting plan and it is currently hosting two websites. I would like to move them onto separate virtual machines so that I can scale them out independently.

To do this I need to navigate to the web site I wish to move.

image

The top menu needs to be expanded by clicking the 3 dots on the right of the menu bar. this then displays the web hosting plan button.

image

Clicking this displays the web hosting plan associated with this web site

image

Clicking on the new hosting plan option allows you to create a new plan

image

I’ve selected a standard small instance to host my website.

After clicking OK the new hosting plan will be created and the website moved to it. After a short while you should see that the hosting plan has changed in this website as well.

image

Note: you now have two hosting plans both of which will be a separate billing entity. I am also led to believe that if you move everything off of a hosting plan you will still be charged for it.Hosting plans can be deleted once all the websites have been moved off of it. This is done in the Web hosting plan page. right click on the plan you want to delete and select the Delete option

5 Tips for using Azure Web Jobs

1. Use public on the main program class.In order for web jobs to initialise correctly the main class that contains the web jobs needs to be made public. Once this has been added the individual jobs can then be read and should be visible in the output when running locally.

clip_image001

2. In order to store and view the invocation details for each web job you need to configure AzureWebJobsDashboard in the configure tab of the website you have deployed the web job to. Even if you have configured this in your app.config file.

clip_image003

If this is not configured in the website then you will receive the following error when you try and view the web jobs dashboard

clip_image002

3. Debug using Visual Studio. Once of the nice features of the web jobs SDK is the ability to run and debug the web job locally in Visual Studio. Following the Getting Started guide, you create a console application which you can debug in Visual Studio before deploying it to Azure

4. User TextWriter for debugging. The Azure Web Jobs SDK (see the logging section) provides a mechanism to log out information that can be viewed through the Azure Web jobs dashboard. By adding a TextWriter as an input parameter to your web job method, you can use WriteLine to then output information you wish to log.

5. Make your Blob Triggers more efficient by triggering them using BlobOutput. The mechanism that the BlobInput trigger uses has a 10-20 minute lag before the trigger can fire, but each time BlobOutput is used it triggers a rescan for Blob input.

“There is an optimization where any blob written via a [BlobOutput] (as opposed to being written by some external source) will optimistically check for any matching [BlobInputs],” See How does [BlobInput] work?. Storage Queues and Service Bus topics and Queues are generally processed within seconds so if you can use a queue to trigger a BlobOutput then use this to trigger any subsequent BlobInputs

Azure Service Bus Event Hub Firewall Port

I’m investigating the Azure Service Bus Event Hub using the getting started tutorial and I didn’t seem to be able to receive any data. It turns out that our firewall was blocking an outbound port. After some investigation I found a post which hinted at a port for the on premise service bus. Our IT guys kindly enabled the outbound port 5671 and I now can receive data from the event hub.

For completeness the following site has details of the other firewall ports required for service bus : http://msdn.microsoft.com/en-us/library/ee732535.aspx

Internet of Things (IoT): Gadgeteer and Service Bus

Internet of Things seems to bring together two of my favourite topics: Gadgeteer and Service Bus. Whilst researching IoT I came across an article in MSDN magazine written by Clemens Vasters (http://msdn.microsoft.com/en-us/magazine/jj190807.aspx). This article is from July 2012 and things have moved on a little since then, but the fact that he  has Gadgeteer talking to service bus meant that I had to give it a go myself. The first port of call was the previous article (http://msdn.microsoft.com/en-us/magazine/jj133819.aspx - note the link is wrong in the current article). This explains the architecture that the sample is based upon using service bus topics to send commands to the device and a different topic to allow the device to send data. There is also a provisioning service that allows the devices to be initialised with the correct configuration. this provisioning service also configures up the service bus access control service (ACS) to allow each device to have its own security key. This means you can turn off devices using ACS.

Before you start take a look at the Service Bus Explorer as this is a useful tool when you are trying to diagnose why things aren’t working.

As I’m using a GHI Electronics Fez Spider main board I am using the .Net Micro-Framework 4.2. Upgrading the project to 4.2 had a couple of errors which needed resolving. Firstly, you will need to change GetJoystickPosition to GetPosition; secondly, change ConvertBase.ToBase64String to Convert.ToBase64String. This allowed me to run the project on my Gadgeteer board. However, I kept getting an error whenever I tried to call the provisioning service. I kept getting Bad Request. I immediately assumed that my configuration was wrong but after a bit of searching and then turning WCF tracing on I found that the service could not load the service bus assembly so I removed and the re-added it to solve the problem. As I mention configuration its probably a good idea to say what each of the settings in the provisioning service is used for:

sharedSignature : Go to https://manage.windowsazure.com/ and login. Click on service bus and then select the service bus you are using. on the bottom menu click the Connecting Information button. This will popup a configuration window. there are two keys in here. The first is part of the connectionstring and is under the sas section. Copy the connection string and find the key. This is the sharedSignature for this configuration setting.

servicebusNamespace: This is the name of the service bus as it appears in the management portal.i.e. sb://<servicebusNamespace>.servicebus.windows.net

managementKey: In the same connection information popup where you found the shared signature there is a section at the bottom labelled ACS. the managementkey is the Default Key

Microsoft.ServiceBus.ConnectionString: I used the connection string that appears in the SAS section of the connection information popup.

The other configuration you need to do is to change the url for the provisioning service. This is hard coded on the Gadgeteer board and is located in Program.cs, serverAddress variable in the ServiceBusApp project.

The provisioning service should be ready to go. However, I had problems connecting to the service from the Gadgeteer board as I kept receiving NotSupportedException each time I called GetRequestStream. This was due to an issue with the Ethernet configuration when trying to connect over https. This can be solved by updating the ssl seed using the Fez Config tool (https://www.ghielectronics.com/community/forum/topic?id=13927). This is done by clicking the Deployment (Advanced) button and the clicking Update SSL Seed.

image

Once complete I could then connect to the provisioning service. The provisioning service should only be called once per device and it is up to the device to store its configuration in a persistent store. This did not appear to be working on my device. Some of the settings were being but the topic urls were not. I changed the type from a URI to a string and the persistence then seemed to work and I only went to provision once. Each time the provisioning service is called a new subscription is create and a new access control identity and rule are also created.

With all this fixed I could now send messages, but I could not see them. this was because I didn’t have a subscriber to the topic where the data was published. This is easily resolved by creating one, but it will only receive new messages. Any messages sent before the subscription is created will be lost.

The provisioning service also has a web page that allows you to send commands to each device. It will broadcast a message to all devices by putting a single message into the devices topic and it sets the Broadcast property to be true. During provisioning the subscription that is created has a SQL Filter applied which allows the subscription to only receive messages that are targeted specifically at the device or if they are broadcast. The web page puts a message into the topic to tell the device to set its temperature to a specific value.The device should be listening for messages to its subscription and will act on the command once it is received.

The device never seemed to receive the message even though the Service Bus Explorer showed that the message was waiting in the queue. Whenever we tried to connect to the subscription “Bad Request” was being returned. After investigation is turns out that the sample only ever sets the event topic uri and not the devices topic uri. When we try and retrieve the device commands we are trying to connect to the events topic which is not a subscription. The sample needs modifying in Microsoft.ServiceBus.Micro project in the MessagingClient class. I added an extra Uri to the constructor and modified the CreateReceiveRequest and CreateLockRequest methods to use this Uri.

The final thing I changed was the command that is sent from the web page and how it was received:

The sender code in Default.aspx.cs in the BackEndWebrole Project

deviceSender.Broadcast(new Dictionary<string, object> { { "Temperature", this.TextBox1.Text } }, "SetTemperature");

And the receiver code in Program.cs in the ServiceBusApp project:

switch (commandType)
{
      case "SetTemperature":
      if (cmd.Properties.Contains("Temperature"))
      {
               this.settings.TargetTemperature = double.Parse((string)cmd.Properties["Temperature"]);
               StoreSettings(this.settings);
     }
     break;
}

I now have a Gadgeteer device talking to the service bus with the ability to send data and receive commands. My next steps are to create a webjob to process the event data (see my previous post) and also look into event hubs.

Creating a Simple Workflow with Azure Webjobs and Service Bus

With the announcement of an upgrade to the webjobs service in Microsoft Azure service bus triggers to Topics and Queues were added.

This got me thinking about how they could be used and there are a lot of scenarios where events need to trigger simple actions as well as running things off of a timer. Utilising Service Bus Topics along with a number of filtered subscriptions led me to create a simple workflow using Webjobs and Topics.

Setting up web jobs is fairly easy and the two links at the top of this article will give you a good start to this along with the 3 articles written by Mike Stall (about Blobs, Queues and Tables) (although the syntax has changed with the latest update, but its easy enough to work out the changes)

The first thing to note which I didn’t pick up on straight away was that you need to  make sure that you set up connection strings for the diagnostics to work. I did this in the Configure section of the web site where I was deploying the webjob. I also added it to my app.config file so that I could debug locally in Visual Studio

image

They all point to my Azure Storage account.

The way I set this up was to create a message class that contained the data that I wanted to send between states. This message class would be wrapped in a BrokeredMessage and I would use the properties to determine the state of the message. The message is then added to a Service Bus Topic. By setting properties on the message, I could create a number of subscriptions that had an SqlFilter applied which would allow the subscription to only contain messages of a specific type.

When coding this I created a basic console application and added the following code to the Main method: 

// now create the subscriptions for the states
if (!_namespaceManager.SubscriptionExists(TopicName, TopicName + "start"))
{
    _namespaceManager.CreateSubscription(TopicName, TopicName + "start", 
                      new SqlFilter("State='WorkflowStart'"));
}
if (!_namespaceManager.SubscriptionExists(TopicName, TopicName + "state1"))
{
    _namespaceManager.CreateSubscription(TopicName, TopicName + "state1", 
                      new SqlFilter("State='WorkflowState1'"));
}
if (!_namespaceManager.SubscriptionExists(TopicName, TopicName + "state2"))
{
    _namespaceManager.CreateSubscription(TopicName, TopicName + "state2", 

                      new SqlFilter("State='WorkflowState2'"));
}

This allows me to create Topic triggers for the webjob for specific states as follows:

public static void SimpleWorkflowStart(
   [ServiceBusTrigger(TopicName, TopicName + "start")] BrokeredMessage message, TextWriter log)
{
      log.WriteLine("Workflow Started");
      .
      .
      .
 }

Two things to note with the method above. Firstly, the ServiceBusTrigger requires both the Topic name and the subscription name in order to receive the filtered list of messages. Secondly, by adding the TextWriter you can then send logging information to the dashboard, which is useful when trying to diagnose the webjobs when deployed to Azure.

I then setup a number of other methods for each subscription for each state. This allows me to perform whatever action I wanted to in each state. After carrying out the action I modified the message by changing the state property and then put it back onto the Topic. You can’t use the same brokered message so you either have to copy all the data out of the existing message and create a new one or you can use the Clone method on Brokered message (the easy option Smile).

// copied the old message
 BrokeredMessage newMessage = message.Clone();
// change the state
newMessage.Properties["State"] = "WorkflowState2";
TopicClient tc = TopicClient.CreateFromConnectionString(
               ConfigurationManager.ConnectionStrings["ServiceBus"].ConnectionString, TopicName);
// delay sending the message a little
newMessage.ScheduledEnqueueTimeUtc = DateTime.Now.AddSeconds(10);
// send it
tc.Send(newMessage);

In my example I also added a delay to the Enqueue of the message so that I could see things progressing in order. The enqueue time of the message is the delay before the message appears in the subscription. This could be also be used as a delayed trigger if you needed something done after a specific time interval. I also achieve this by adding the data to a Azure SQL db and then using a scheduled webjob to check the db for expired jobs. I created a separate console application for this which looked for messages that were expired due to a specific start date being older than a certain time, sending an email using SendGrid and then marking the record as complete in the db.

Webjobs made the actions easy to do and with the addition of the service bus triggers allowed me to have a fairly simple code structure to carry our a sequence of simple action (which is often what we need to do) without a heavy overhead of a workflow engine. It also utilised a hosting environment that I was already using.