Steve Spencer's Blog

The blog of Steve Spencer

Creating a Simple Workflow with Azure Webjobs and Service Bus

With the announcement of an upgrade to the webjobs service in Microsoft Azure service bus triggers to Topics and Queues were added.

This got me thinking about how they could be used and there are a lot of scenarios where events need to trigger simple actions as well as running things off of a timer. Utilising Service Bus Topics along with a number of filtered subscriptions led me to create a simple workflow using Webjobs and Topics.

Setting up web jobs is fairly easy and the two links at the top of this article will give you a good start to this along with the 3 articles written by Mike Stall (about Blobs, Queues and Tables) (although the syntax has changed with the latest update, but its easy enough to work out the changes)

The first thing to note which I didn’t pick up on straight away was that you need to  make sure that you set up connection strings for the diagnostics to work. I did this in the Configure section of the web site where I was deploying the webjob. I also added it to my app.config file so that I could debug locally in Visual Studio

image

They all point to my Azure Storage account.

The way I set this up was to create a message class that contained the data that I wanted to send between states. This message class would be wrapped in a BrokeredMessage and I would use the properties to determine the state of the message. The message is then added to a Service Bus Topic. By setting properties on the message, I could create a number of subscriptions that had an SqlFilter applied which would allow the subscription to only contain messages of a specific type.

When coding this I created a basic console application and added the following code to the Main method: 

// now create the subscriptions for the states
if (!_namespaceManager.SubscriptionExists(TopicName, TopicName + "start"))
{
    _namespaceManager.CreateSubscription(TopicName, TopicName + "start", 
                      new SqlFilter("State='WorkflowStart'"));
}
if (!_namespaceManager.SubscriptionExists(TopicName, TopicName + "state1"))
{
    _namespaceManager.CreateSubscription(TopicName, TopicName + "state1", 
                      new SqlFilter("State='WorkflowState1'"));
}
if (!_namespaceManager.SubscriptionExists(TopicName, TopicName + "state2"))
{
    _namespaceManager.CreateSubscription(TopicName, TopicName + "state2", 

                      new SqlFilter("State='WorkflowState2'"));
}

This allows me to create Topic triggers for the webjob for specific states as follows:

public static void SimpleWorkflowStart(
   [ServiceBusTrigger(TopicName, TopicName + "start")] BrokeredMessage message, TextWriter log)
{
      log.WriteLine("Workflow Started");
      .
      .
      .
 }

Two things to note with the method above. Firstly, the ServiceBusTrigger requires both the Topic name and the subscription name in order to receive the filtered list of messages. Secondly, by adding the TextWriter you can then send logging information to the dashboard, which is useful when trying to diagnose the webjobs when deployed to Azure.

I then setup a number of other methods for each subscription for each state. This allows me to perform whatever action I wanted to in each state. After carrying out the action I modified the message by changing the state property and then put it back onto the Topic. You can’t use the same brokered message so you either have to copy all the data out of the existing message and create a new one or you can use the Clone method on Brokered message (the easy option Smile).

// copied the old message
 BrokeredMessage newMessage = message.Clone();
// change the state
newMessage.Properties["State"] = "WorkflowState2";
TopicClient tc = TopicClient.CreateFromConnectionString(
               ConfigurationManager.ConnectionStrings["ServiceBus"].ConnectionString, TopicName);
// delay sending the message a little
newMessage.ScheduledEnqueueTimeUtc = DateTime.Now.AddSeconds(10);
// send it
tc.Send(newMessage);

In my example I also added a delay to the Enqueue of the message so that I could see things progressing in order. The enqueue time of the message is the delay before the message appears in the subscription. This could be also be used as a delayed trigger if you needed something done after a specific time interval. I also achieve this by adding the data to a Azure SQL db and then using a scheduled webjob to check the db for expired jobs. I created a separate console application for this which looked for messages that were expired due to a specific start date being older than a certain time, sending an email using SendGrid and then marking the record as complete in the db.

Webjobs made the actions easy to do and with the addition of the service bus triggers allowed me to have a fairly simple code structure to carry our a sequence of simple action (which is often what we need to do) without a heavy overhead of a workflow engine. It also utilised a hosting environment that I was already using.

Proximity Communication with Windows Phone

In my previous post I talked about using NFC to communicate between devices and hopefully gave you an introduction to the capabilities of NFC. This post will cover the Proximity API in more detail and show you how to set up a persistent connection between two Windows Phones which will allow you to communicate without needing to keep tapping phones together to transfer data. Windows Phone and Windows 8 apps can utilise the .NET Proximity APIs to connect together apps without needing to be connected to the same network. Proximity uses BlueTooth or WiFi Direct (Not on Windows Phone 8) to make a persistent connection between the two devices and utilising StreamSockets to communicate between the two devices. The proximity API uses a PeerFinder to browse for proximate devices or to sit and wait for a Tap to occur.

The first scenario is to set your app in a mode where it can accept incoming Tap requests, then set up a connection between the two devices.

The second scenario is to start your app and then browse for other devices that are running your app and which are within range of your device.

Both scenarios use the PeerFinder classes to initiate the communication.

Scenario 1: TAP to send

PeerFinder has a number of static methods and events that allow you to wire up your app to respond to peer requests. In the Tap to send scenario your app needs to wait for a tap to occur and then initiate opening up the communications channel. PeerFinder has an event called TriggeredConnectionStateChanged which is triggered upon a Tap initiation and is fired for each event state change that occurs. Upon successful connection an open socket is returned which can then be used for communication between the two devices. Once the event is wired up the PeerFinder needs to be started. This is required for both scenarios. You will also need to ensure that the Proximity and Network capabilities are added to you app manifest.

// Wire up the event and be notified when a device is tapped
PeerFinder.TriggeredConnectionStateChanged += PeerFinder_TriggeredConnectionStateChanged; 
// Start finding peer apps, while making this app discoverable by peers
PeerFinder.Start();

The event handler is as follows:

<Insert Code>

PeerFinder.TriggeredConnectionStateChanged += PeerFinder_TriggeredConnectionStateChanged;
.
.
.
private void PeerFinder_TriggeredConnectionStateChanged(object sender, 
                                            TriggeredConnectionStateChangedEventArgs args)
{
    switch (args.State)
    {
        case TriggeredConnectState.Listening:
            // Connecting as host
            WriteStatusMessage("Listening");

            break;
        case TriggeredConnectState.PeerFound:
            // Proximity gesture is complete and user can pull their devices away. Remaining work is to 
            // establish the connection using a different transport, like TCP/IP or Bluetooth
            WriteStatusMessage("Peer Found");

            break;
        case TriggeredConnectState.Connecting:
            // Connecting as a client
            WriteStatusMessage("Connecting");
            break;
        case TriggeredConnectState.Completed:
            // Connection completed, retrieve the socket over which to communicate
            if (!StreamSockets.ContainsKey(args.Socket.Information.RemoteHostName.CanonicalName))
            {
                StreamSockets.Add(args.Socket.Information.RemoteHostName.CanonicalName, args.Socket);
                _currentConnectionID = args.Socket.Information.RemoteHostName.CanonicalName; 
            }

            WriteStatusMessage("Connected");

            // start the listener off on a new thread
            _Receivethread = new System.Threading.Thread(new ThreadStart(ReceiveData));
            _Receivethread.Start();

            break;
        case TriggeredConnectState.Canceled:
            WriteStatusMessage("Canceled");

            break;
        case TriggeredConnectState.Failed:
            // Connection was unsuccessful
            WriteStatusMessage("Failed");
            break;
    }

}

The code above covers the whole connection process and the important part is the TriggeredConnectState.Completed state. It is in this state that we are presented with a valid socket and in the case of Windows phone 8 a BlueTooth connection between the two phones has been created with an open StreamSocket to enable the communication. We also setup a thread that listens on the socket for incoming data to allow two way communication to occur. Both apps need to go through this process and setup a mechanism for sending and receiving data. PeerFinder will find all apps with the same id but it is possible to find other apps that are compatible with your application by adding alternate application identities by adding the information to the AlternateIdentities collection. However, when I tried to add an alternate Windows Phone 8 identity, the key "WindowsPhone" was not accepted and it looks like only Windows can be used.

Using this method the connection is established when the Tap is initiated but this can get tedious if you want to connect to a number of devices. In this case Scenario 2 would be better.

Scenario 2: Browser for devices

It is also possible to connect to compatible applications directly without the need to Tap devices together. The act of starting a compatible app will allow the device to be discoverable to other devices running the same app. PeerFinder is again used but this time we need to wire up a different event, ConnectionRequested. This is fired when a device attempts to connect and allows the connection to be made. When ConnectionRequested fires, the connection information of the device trying to connect is passed through with the EventArgs and PeerFinder.ConnectAsync is called. On successful connection then the same methods for sending and listening are used as in scenario 1.

PeerFinder.ConnectionRequested += PeerFinder_ConnectionRequested;
// Start finding peer apps, while making this app discoverable by peers
PeerFinder.Start();
.
.
.
async void PeerFinder_ConnectionRequested(object sender, ConnectionRequestedEventArgs args)
{
    WriteStatusMessage("ConnectionRequested");

    StreamSocket socket = await PeerFinder.ConnectAsync(pi);
    PeerConnected(socket);
}

We've wired up the app so that it is capable of receiving a connection from a remote device, we now need to initiate the connection.

Again this is done using PeerFinder and the method FindAllPeersAsync. Will return a list of applications that are compatible which can then be connected to individually. Only one Peer connection can be made at any one time so you will need to connect, send data and then disconnect before moving on to the next peer. Iterating through the list of PeerInformation that is passed.

IReadOnlyList<PeerInformation> readOnlyList = await PeerFinder.FindAllPeersAsync();
PeersFound(readOnlyList);
.
.
.
private async void PeersFound( IReadOnlyList<PeerInformation> readOnlyList)
{
    foreach (PeerInformation pi in readOnlyList)
    {
        StreamSocket socket = await PeerFinder.ConnectAsync(pi);
        PeerConnected(socket);
        return;//for now lets just deal with the first connection
    }
            
}

private void PeerConnected(StreamSocket socket)
{
    OnConnectionStatusChanged("Connected");

    // Do send/receive stuff here

}

For a full worked example see here and for more details of messages

You will need to look into the capabilities of each device as some will only connect via BlueTooth and others only using WiFi Direct and you can use the properties of PeerFinder to control which mechanism you would like to use

PeerFinder.AllowBluetooth = true;
PeerFinder.AllowInfrastructure = false; 
PeerFinder.AllowWiFiDirect = false;

The two posts should now give you information about how to connect phone apps together and provide some insight into how you can connect other devices and provide a rich set of functionality to you apps that previously required backend services to communicate.

Introducing NFC and Proximity with Windows Phone

Near Field Communication has been seen demonstrated in two ways. Firstly scanning an NFC enabled card which can be programmed to perform a number of different actions such as take your device to a specific url, transfer contact details, start an app or take you to the store to install the app and a number of other options. Secondly you may have seen people touching devices together to do things. These actions are either done via NFC communications in the same way that a tag works but also communication could be done using the proximity apis to create a personal network between the two devices to allow a richer sharing of data between devices. This post will cover NFC communication and will show you how you can use NFC to communicate data to devices both in a static way or programmatically as part of an app.

The simplest way to use NFC to exchange data Is via a physical NFC tag. These tags can be programmed with a range of information such as redirecting your mobile browser to a web page; sharing contact details or starting or installing an app. Each of these can be programmed using an app like the Nokia NFC Writer app documented here.

An NFC enabled device will automatically try and resolve the NFC tag data but it is also possible to have custom messages within a tag. Each tag has different capabilities and data sizes which can be seen here

The data that can be written to a tag can also be seen here

The common tag protocols are : WindowsUri, WindowsMime & NDEF.

NDEF is the Nfc Data Exchange format and is the Raw message format for NFC. This can be used to create custom messages for exchanging other types of data. Information about NDEF can be found here

What is more interesting though is that your phone is not only a reader of NFC data it can also be a sender of data so that the tap and send scenarios can be easily enabled within apps, providing that your device supports NFC.

The code samples below utilise the .NET Proximity APIs in C# and will show you how to add basic send and receive capability to an application.

Firstly you will need to determine whether the device your app is running on supports proximity. This is done by get hold of the default proximity device using:

ProximityDevice device = ProximityDevice.GetDefault();

If the device returned is null then you do not have NFC enable or your device down not have NFC capabilities. You can check to see if you have NFC on your phone in settings. There should be a section on NFC with a slider to turn NFC on or off. Make sure that it is turned on. You will also need to ensure that the capabilities of your app allows proximity and this is done in the app manifest file (ID_CAP_PROXIMITY)

Once you have your proximity device there are a number of events you can wire up which will get fired when another NFC device arrives whether that’s a static tag or an NFC enabled phone or tablet. These are DeviceArrived and DeviceDeparted and they fire as you would expect when the NFC device is tapped and when it is removed. These two event give you a notification about the arrival or departure but not the actual data. For this you will need to subscribe for messages and set up call backs to receive and process the NFC data. Similarly you can use the DeviceArrived event to allow you to send data to the other device.

To receive data you do not need to handle the DeviceArrived or DeviceDeparted events but you will need to subscribe for the set of messages you wish to receive including custom messages. You can create different handlers for each type of message you wish to receive.

long subscriptionID = device.SubscribeForMessage("Windows.BlackMarbleMessage", messageReceived);

Here I am subscribing to a custom message of type Windows.BlackMarbleMessage which my messageReceived method knows how to process:

private void messageReceived(ProximityDevice sender, ProximityMessage message)
{

    Debug.WriteLine("Received from {0}:'{1}'", sender.DeviceId, message.DataAsString);
    WriteMessageText(message.DataAsString);
}

My messageReceived method just takes the content of the message and displays it on the screen, but in a real life scenario the data could be JSON or XML for example and you process them accordingly.

The standard set of tag protocols is documented here if you want to access the raw NDEF protocol then the codeplex project here can be used

An application cannot determine whether the device arrived is an active device or just a passive tag so if you want to send messages out you need to understand that there may not be a response so make sure that your app is not expecting a response in order to proceed.

Sending data can be achieved by sending a message to the proximity device when you know that a device has arrived, so for this you will need to wire up the DeviceArrived event.

device.DeviceArrived += device_DeviceArrived;

.
.
.


void device_DeviceArrived(ProximityDevice sender)
{
    if (_bSendingMessage)
    {
        if(string.IsNullOrEmpty(_messageToSend))
                _messageToSend = "Hello World";
        sender.PublishMessage("Windows.BlackMarbleMessage", _messageToSend);
                
    }
            
}

This example is part of a phone app which needs to check a check box and enter some data in a text box in order to send data. Sending data is done through the PublishMessage, PublishBinaryMessage or PublishUriMessage methods. If you use PublishMessage or PublishBinaryMessage then you will need to add a string identifier for the message type which needs to match the message type that the receiver has subscribed to. In my example this is "Windows.BlackMarbleMessage".

Let's assume that we've installed this app onto two Windows 8 phones, one of the phones has the send message check box checked and has some data in the data check box. The two phones are Tapped together. Both apps will receive the DeviceArrived event and the device_DeviceArrived method is called on both apps. The phone with the send message check box checked will then send a Windows.BlackMarble message containing the data in the data text box to the other phone. The other phone then receives the message in its messageReceived method and will display it on the screen.

This is a basic example but it shows how you can send through simple message data and to add Tap functionality to you apps. This should work across devices and operating systems as the communication is through NFC (although you will need an app that understands your messages if you are sending custom messages).

NFC and proximity can be used to set up a longer running connection between two devices with or without tapping to initiate. There will be a follow on post that covers this shortly.

Windows Store App Notifications, the Notification Hub and Background tasks

This article aims to talk about Windows Store Notifications and the Windows Azure Notifications Hub and it will attempt to collate the various articles in a single place to help you build notifications into your app.

In order for you to get an understanding of Windows notifications look at the following article

Introduction to Push Notifications - http://msdn.microsoft.com/en-us/library/windows/apps/hh913756.aspx. this provides a good overview of how push notifications work. To summarise the important bits.

1. Your store app needs to register with the Windows Notification Service to retrieve a unique URI for your instance of the app. Ideally you do this each time the app starts.

2. If the URI has changed then you need to notify your service that there is a new URI. Note: This URI expires every 30 days so your app needs to notify your service that this has been changed.

3. Your service sends notifications to this unique URI

You may have noticed above that I mentioned “Your service”. This is a critical piece of the notification mechanism and there are a number of ways to build this service. If you are not comfortable building backend services or you want something up and running quickly then mobile services might be the way to go for you. Here’s a tutorial that gets you started with mobile services http://www.windowsazure.com/en-us/develop/mobile/tutorials/get-started/

If, like me, you already have a source of data and a service then you will probably want to wire in notifications into your existing service. depending upon how many devices you have using your app may dictate the method that you get the notifications onto the users device. there are a number of options:

  1. Local updates
  2. Push Notifications
  3. Periodic Notifications

Local updates require the creation of a background task that Windows runs periodically that calls into your data service, retrieves the data to put on the tiles and sends out tile notifications using the Windows Store app SDK

Updating live tiles from a background task - http://msdn.microsoft.com/en-us/library/windows/apps/jj991805.aspx. Provides a tutorial on building a background task for your Windows Store App. this tutorial is for timer tasks but it can easily be used for push notification tasks. The bits that are likely to change are the details of the run method, the task registration and the package manifest.

Two more important links that you will require when you are dealing with notifications:

Tile template catalogue http://msdn.microsoft.com/en-us/library/windows/apps/hh761491.aspx

Toast template catalogue http://msdn.microsoft.com/en-us/library/windows/apps/hh761494.aspx

These two catalogues are important as they provide you with details of the xml you need for each type of notifications

Push notifications are sent through the Windows Notification Service to your device.

You can send notifications to your device from your service by creating a notification and sending it to each of the devices registered to your service via the Windows Notification Service.

If you have a large number of devices running your app then you will probably want to use the Windows Azure Notification Hub. This is the simplest way to manage notifications to your application as the notification hub handles scaling, managing of the device registration and also iterating around each device to send the notifications out. The Notification hub will also allow you to send notifications to Windows Phone, Apple and Android devices. To get started with the notification hubs follow this tutorial:http://www.windowsazure.com/en-us/manage/services/notification-hubs/getting-started-windows-dotnet/

The nice feature of the notification hub is that is makes the code needed to send notifications simple.

 

NotificationHubClient hub = NotificationHubClient.CreateClientFromConnectionString("<your notification hub connection string>", "<your hub name>");

 

var toast = @"<toast><visual><binding template=""ToastText01""><text id=""1"">Hello from a .NET App!</text></binding></visual></toast>";

 

await hub.SendWindowsNativeNotificationAsync(toast);

Compare this to the code to send the notification without the hub:

 

byte[] contentInBytes = Encoding.UTF8.GetBytes(xml);

 

 

HttpWebRequest request = HttpWebRequest.Create(uri) asHttpWebRequest;

request.Method =

"POST";

request.Headers.Add(

"X-WNS-Type", notificationType);

request.ContentType = contentType;

request.Headers.Add(

"Authorization", String.Format("Bearer {0}", accessToken.AccessToken));

 

 

using (Stream requestStream = request.GetRequestStream())

requestStream.Write(contentInBytes, 0, contentInBytes.Length);

 

 

In addition you will need to retrieve the list of devices that are registered for push notifications and iterate around the list to send this to each device. You will also require a service that receives the registrations and stores them in a data store. You need to manage the scalability of these services. On the down side the notification hub is charged per message which means the more often you send notifications the greater the costs where as hosting a service is load based and the notifications will be sent out slower as the number of devices increases but this would generally be a lower cost. If you also take into account that you will need to send out notifications for each tile size and that will increase the activity count on the notification hub for each tile size (currently 3).

[Update: You can send out a single notification for all tile sizes rather than 3 separate notifications by adding a binding for each tile size in your xml see http://msdn.microsoft.com/en-us/library/windows/apps/hh465439.aspx for more details]

It is possible to send custom notifications to your app which can be received directly in the app or by using a background task. These are called Raw notifications. In order to receive raw notifications in a background task your app needs to be configured to display on the start screen. However Raw Notifications can be received in your app whilst it is running when it is not configured to display on the start screen. A Raw Notification is a block of data up to 5KB in size and can be anything you want.

The following code will send a raw notifications using the notifications hub:

 

string rawNotification = prepareRAWPayload();

 

Notification notification = new Microsoft.ServiceBus.Notifications.WindowsNotification(rawNotification);

notification.Headers.Add(

"X-WNS-Cache-Policy", "cache");

notification.Headers.Add(

"X-WNS-Type", "wns/raw");

notification.ContentType =

"application/octet-stream";

 

 

var outcome = await hub.SendNotificationAsync(notification);

In order to receive Raw Notifications in your app you need to add an event to the channel you retrieve from the Windows Notification Service:

 

var channel = awaitPushNotificationChannelManager.CreatePushNotificationChannelForApplicationAsync();

 

channel.PushNotificationReceived += channel_PushNotificationReceived;

 

And then handle the notification received:

 

privatevoid channel_PushNotificationReceived(PushNotificationChannel sender, PushNotificationReceivedEventArgs args)

{

 

switch (args.NotificationType)

{

 

    casePushNotificationType.Raw:

 

        ReceiveNotification(args.RawNotification.Content);

 

    break;

}

}

 

Note: the content of the notification is the block of data that you sent out.

Sample background task for Raw Notifications is here: http://msdn.microsoft.com/en-us/library/windows/apps/jj709906.aspx

Guidelines for Raw Notifications can be found here: http://msdn.microsoft.com/en-us/library/windows/apps/hh761463.aspx

Periodic notifications also require a service but the application periodically calls into a service to retrieve the tile notifications without needing to process the source data and then create the notifications locally. details about how to use periodic notifications can be found here: http://msdn.microsoft.com/en-US/library/windows/apps/jj150587

In summary Windows Store application notifications can be send to the app in a variety of ways and the mechanism you choose will depend upon how quick and how many notifications are required. Push notifications allow notifications to be sent whenever they are ready to send. Periodic and Local updates are pull notifications and require a service to be available to pull the data from. All of these will require some sort of service and all have an associated costs. The notifications hub is a useful tool to assist with notifications and it can be useful to manage the device connections as well as sending out notifications to multiple device type. It does however come at a cost and you need to work out whether it is a cost effective mechanism for your solution.

Gadgeteer, Signal R, WebAPI & Windows Azure

After a good night in Hereford at the Smart Devs User Group and my presentation at DDDNorth

Here are the links from my presentation and some from questions asked:

Gadgeteer: http://www.netmf.com/gadgeteer/

Signal-R: http://www.asp.net/signalr/

Web API: http://www.asp.net/web-api

The Signal-R chat example can be found at: http://www.asp.net/signalr/overview/getting-started/tutorial-getting-started-with-signalr

Windows Azure Pricing Calculator : http://www.windowsazure.com/en-us/pricing/calculator/?scenario=full

Signal-R Scaleout using Service bus, SQL Server or Redis: http://www.asp.net/signalr/overview/performance-and-scaling/scaleout-in-signalr

The Windows Azure Training Kit: http://www.windowsazure.com/en-us/develop/net/other-resources/training-kit/

Gadgeteer Modules: http://proto-pic.co.uk/categories/development-boards/net.html

Fex Spider Starter Kit: http://proto-pic.co.uk/fez-spider-starter-kit/

 

In addition to these links I have more from my presentation at the DareDevs user group in Warrington

It is possible to drive a larger display from Gadgeteer using a VGA adapter. You use this the same way that the Display-T35 works using the SimpleGraphics interface for example.

VB eBook - Learn to Program with Visual Basic and Gadgeteer

Fez Cerberus Tinker Kit: https://www.ghielectronics.com/catalog/product/455 

Enabling Modern Apps

I’ve just finished presenting my talk on “Successfully Adopting the Cloud: TfGM Case Study”and there were a couple of questions that I said I would clarify.

1. What are the limits for the numbers of subscriptions per service bus topic. the answer is 2000. further details can be found at:http://msdn.microsoft.com/en-us/library/windowsazure/ee732538.aspx

2. what are the differences between Windows Azure SQL database and SQL Server 2012. The following pages provide the details:

Supported T-SQL: http://msdn.microsoft.com/en-us/library/ee336270.aspx

Partially supported T-SQL: http://msdn.microsoft.com/en-us/library/ee336267.aspx

Unsupported T-SQL: http://msdn.microsoft.com/en-us/library/ee336253.aspx

Guidelines and Limitations: http://msdn.microsoft.com/en-us/library/ff394102.aspx

3. Accessing the TfGM open data site requires you to register as a developer at: http://developer.tfgm.com

Thanks to everyone who attended I hope you found it useful.

Handling A Topic Dead Letter Queue in Windows Azure Service Bus

Whilst working on a project in which we we using the Topics on Windows Azure Service Bus, we noticed that our subscription queues (when viewed from the Windows Azure Management portal) didn’t seem to be empty even though our subscription queue processing code was working correctly. On closer inspection we found that our subscription queue was empty and the numbers in the management portal against the subscription were messages that had automatically faulted and had been moved into the Dead Letter queue.

The deadletter queue is a separate queue that allows messages that fail to be processed to be stored and analysed. The address of the deadletter queue is slightly different from your subscription queue and is the form:

YourTopic/Subscriptions/YourSubscription/ $DeadLetterQueue

for a subscription and

YourQueue/$DeadLetterQueue for a queue

Luckily you don’t have to remember this as there are helpful methods to retrieve the address for you:

SubscriptionClient.FormatDeadLetterPath(subscriptionClient.TopicPath, messagesSubscription.Name);

To create a subscription to the deadletter queue you need to append /$DeadLetterQueue to the subscription name when you create the subscription client

Once you have this address you can connect to the dead letter queue in the same way you would connect to the subscription queue. Once a deadletter brokered message is received the properties of the message should contain error information highlighting why it has failed. The message should also contain the message body from the original message. By default the subscription will move a faulty message to the dead letter queue after 10 attempts to deliver. You can also move the message yourself and put in sensible data in the properties if it fails to be processed by calling the DeadLetter method on the BrokeredMessage. The DeadLetter method allows you to pass in your own data to explain why the message has failed.

The DeadLetter can be deleted in the same was as a normal message by calling the Complete() method on the received dead letter message

Here is an example of retrieving a dead lettered message from a subscription queue

var baseAddress = Properties.Settings.Default.ServiceBusNamespace;
var issuerName = Properties.Settings.Default.ServiceBusUser;
var issuerKey = Properties.Settings.Default.ServiceBusKey;

Uri namespaceAddress = ServiceBusEnvironment.CreateServiceUri("sb", baseAddress, string.Empty);

this.namespaceManager = new NamespaceManager(namespaceAddress, 
                           TokenProvider.CreateSharedSecretTokenProvider(issuerName, issuerKey));
this.messagingFactory = MessagingFactory.Create(namespaceAddress, 
                           TokenProvider.CreateSharedSecretTokenProvider(issuerName, issuerKey));
var topic = this.namespaceManager.GetTopic(Properties.Settings.Default.TopicName);
if (topic != null)
{

    if (!namespaceManager.SubscriptionExists(topic.Path, 
                                  Properties.Settings.Default.SubscriptionName))
    {
        messagesSubscription = this.namespaceManager.CreateSubscription(topic.Path, 
                                            Properties.Settings.Default.SubscriptionName);
    }
    else
    {
        messagesSubscription = namespaceManager.GetSubscription(topic.Path, 
                                            Properties.Settings.Default.SubscriptionName);
    }
}
if (messagesSubscription != null)
{
    SubscriptionClient subscriptionClient = this.messagingFactory.CreateSubscriptionClient(
                                            messagesSubscription.TopicPath,
                                            messagesSubscription.Name, ReceiveMode.PeekLock);

   // Get the Dead Letter queue path for this subscription
    var dlQueueName = SubscriptionClient.FormatDeadLetterPath(subscriptionClient.TopicPath,
                                             messagesSubscription.Name);

   // Create a subscription client to the deadletter queue
    SubscriptionClient deadletterSubscriptionClient = messagingFactory.CreateSubscriptionClient(
                                           subscriptionClient.TopicPath, 
                                            messagesSubscription.Name + "/$DeadLetterQueue");

    // Get the dead letter message
    BrokeredMessage dl = deadletterSubscriptionClient.Receive(new TimeSpan(0, 0, 300));

   // get the properties
    StringBuilder sb = new StringBuilder();
    sb.AppendLine(string.Format("Enqueue Time {0}", dl.EnqueuedTimeUtc));
    foreach (var props in dl.Properties)
    {
        sb.AppendLine(string.Format("{0}:{1}", props.Key, props.Value));
    }
    dl.Complete();
}

MVP Cloud OS Community Week

Black Marble are participating in the Microsoft MVP Cloud OS Community Week. During the week commencing 9th September there will be daily events held at Cardinal Place, Victoria, London. Richard and I along with other MVPs will be participating in the event on Friday 13th Sept which is titled “Enables Modern Business Applications”. The sessions will include developing services for modern apps, case studies, ALM and much more. Further details and registration can be found at: http://mvpcloudosweek-eorg.eventbrite.co.uk/.

Registration for this specific day can be found at: http://enablemodernapps-es2.eventbrite.co.uk/?rank=1

Pricing Changes to Windows Azure

Scott Guthrie made an announcement on his blog about changes to the way Windows Azure is priced

http://weblogs.asp.net/scottgu/archive/2013/06/03/windows-azure-announcing-major-improvements-for-dev-test-in-the-cloud.aspx

The main two changes that will affect most people are

  1. Per minute Billing
  2. No charge for turned off VMs

Prior to this announcement you would be charge for Windows Azure usage in hour blocks of time which meant that if you used 5 minutes of compute time or 55 minutes of compute time you were charged for the full hour, similarly if you used 1 hour 5 minutes you were charged for 2 hours. This change now means that you will be charged for 5 minutes, 55 minutes and 1 hour 5 minutes, matching your actual usage, This doesn’t affect you if you have a fixed usage but for those who take advantage of the easy scalability of Windows Azure this could have a significant impact on your costs.

By far the biggest cost saving is the ability to turn off a VM and not be charged for its usage. Prior to this change you would need to undeploy the solution in order to not be charged and just turning it off would still incur charges, This allows for systems that are in staging, dev/test scenarios, training systems etc to be left deployed and configured but turned off without being charged. Saving both time and money. I have a number of customers where this change alone should half their monthly bills as they have systems running that are not needed all the time but can not be undeployed because of the time taken to deploy and configure them. The ability to turn off VMs will provide both customers and developers with flexibility whilst reducing costs. Always a good thing.

Windows Azure and SignalR with Gadgeteer

I’ve been playing with Gadgeteer (http://www.netmf.com/gadgeteer/) for a while now and I am a big fan of the simple way we can build embedded hardware applications with high functionality. We have a proof of concept device that includes a Colour touch screen, RFID reader and an Ethernet connections. This device is capable of connecting to a web api REST service which we have hosted in Windows Azure and we can use this service to retrieve data from our service depending upon the RFID code that is read. This works well but there are times when we would like to notify the device when something has changed. SignalR seems to be the right technology for this as it removes the need to write polling code in your application.

Gadgeteer uses the .Net Micro framework which is a cut down .Net framework and doesn’t support the ASP.NET SignalR libraries. As we can use web api using the micro framework using the WebRequest classes,  I wondered what was involved to get SignalR working on my Gadgeteer device.

The first problem was to work out the protocol used by SignalR and after a short while trawling the web for details of the protocol I gave up and got my old friend fiddler out to see what was really happening.

After creating a SignalR service I connected my working example to the signalR hub running on my local IIS..

The first thing that pleased me was that the protocol looked fairly simple. It starts with a negotiate which is used to return a token which is needed for the actual connection.

GET /signalr/negotiate?_=1369908593886

Which returns some JSON:

{"Url":"/signalr","ConnectionToken":"xyxljdMWO9CZbAfoGRLxNu54GLHm7YBaSe5Ctv6RseIJpQPRJIquHQKF4heV4B_C2PbVab7OA2_8KA-AoowOEeWCqKljKr4pNSxuyxI0tLIZXqTFpeO7OrZJ4KSx12a30","ConnectionId":"9dbc33c2-0d5e-458f-9ca6-68e3f8ff423e","KeepAliveTimeout":20.0,"DisconnectTimeout":30.0,"TryWebSockets":true,"WebSocketServerUrl":null,"ProtocolVersion":"1.2"}

I used this JSON to pull out the connection id and connection token. This was the first tricky part with the .Net Micro framework. There is not the same support for JSON serialisation you get with the full framework plus the string functions are limited as well. For this I used basic string functions using Substring and IndexOf as follows:

int index = negJson.IndexOf("\""+token+"\":\"");
if (index != -1)
{
    // Extracts the exact JSON value for then name represented by token
    int startindex = index + token.Length + 4;
    int endindex = negJson.IndexOf("\"", startindex);
    if (endindex != -1)
    {
        int length = endindex - startindex;
        stringToExtract = negJson.Substring(startindex, length);
    }
}

With the correct token received Fiddler led me to the actual connection of signalR:

GET /signalr/connect?transport=webSockets&connectionToken=yourtoken&connectionData=%5B%7B%22name%22%3A%22chathub%22%7D%5D&tid=2 HTTP/1.1

Looking at this I could determine that I needed to pass in the token I retrieved from negotiate, the transport type and the name of the hub I want to connect to. After a bit of investigating I used the transport of longPolling.

Now as I think I understood the protocol, I tried to implement it in SignalR. The first issue that arose was what to send with the negotiate call. I figured that this was some sort of id of the client that is trying to connect so I decided to use the current tick count. This seemed to work and I guess that as long as my devices don’t connect at exactly the same time then Signal R would work. I’ve had no problems so far with this.

Upon connecting to the hub I needed to create a separate thread to handle signalR so that the main device wouldn't stop running whilst the connection to the SignalR hub was waiting for a response. Once a response is received the response returns with a block of JSON data appropriate to the SignalR message being received. This needs to be decoded and passed onto the application. You then need to reconnect back to the SignalR hub. The period between receiving data and then reconnecting back to the hub needs to be small. Whilst the message is being processed it cannot receive any more message and may miss some data. I retrieve the response stream and then pass the processing of the stream to a separate thread so that I can reconnect to the hub as fast as possible.

This is not a full implementation of SignalR on the .Net Micro-framework but it is the implementation of a simple client and can be used fairly successfully on the Gadgeteer device. I still need to do a little more work to try to speed up the connections as it is possible to miss some data.

The SignalR hub is hosted on a Windows Azure website along side the web api service which allows both web, Windows 8 and Gadgeteer applications to work side by side.

Gadgeteer has opened up another avenue for development and helps us to provide more variety of devices in a solution