But it works on my PC!

The random thoughts of Richard Fennell on technology and software development

DevOps are testers best placed to fill this role?

DevOps seems to be the new buzz role in the industry at present. People who can bridge the gap between the worlds of development and IT pros. Given my career history this could be a description of  the path I took. I have done both, and now sit in the middle covering ALM consultancy where I work with both roles. You can’t avoid a bit of development and a bit of IT pro work when installing and configuring TFS with some automated build and deployment.

The growth of DevOps is an interesting move because of late I have seen the gap between IT Pros and developers grow. Many developers seem to have less and less understanding of operational issues as times go on. I fear this is a due to the greater levels of abstractions that new development tools cause. This is only going to get worse was we move into the cloud, why does a developer need to care about Ops issues, AppFabric does that for them – doesn’t it?

In my view this is dangerous, we all need at least a working knowledge of what underpins the technology we use. Maybe this should hint at good subjects for informal in-house training, why not get your developers to give intro training to the IT pros and vice versa? Or encourage people to listen to podcasts on the other roles subjects such as Dot Net Rocks (a dev podcast) and Run As Radio (an IT pro podcast). It was always a nice feature of the TechEd conference that it had a dev and IT pro track, so if the fancy took you could hear about technology from the view of the the other role.

However, these are longer term solutions, it is all well and good promoting these but in the short term who is best placed to bridge this gap now?

I think the answer could be testers, I wrote a post a while ago that it was great to be a tester as you got to work with a wide range of technologies, isn’t this just an extension of this role. DevOps needs a working understanding of development and operations, as well as a good knowledge of deployment and build technologies. All aspects of the tester role, assuming your organisation considers a tester not to be a person who just ticks boxes on a check list, but a software development engineer working in test.

This is not to say that DevOps and testers are the same, just that there is some commonality and so you may have more skills in house than you thought you did. DevOps is not new, someone was doing the work already, they just did not historically give it that name (or probably any name)

My experiences moving to BlogEngine.NET

Background

I have recently moved this blog server from using Community Server 2007 (CS2007) to BlogEngine.NET.

We started blogging in 2004 using .Text, moving through the free early versions of Community Server then purchased Community Server Small Business edition in 2007. This cost a few hundred pounds. We recently decided that we had to bring this service up to date, if for no other reason, to patch the underling ASP.NET system up to date. We checked how much it would cost to bring Community Server to the current version and were shocked by the cost, many thousands of dollars. Telligent, the developers, have moved to only servicing enterprise customers, they have no small business offering. So we needed to find a new platform.

Being a SharePoint house, we consider SharePoint as the blog host. However, we have always had the policy to have systems that have external content creation i.e. you can post a comment, not be on our primary business servers. As we did not want to install a dedicated SharePoint farm for just the blogs we decided to use another platform, remembering we needed on that could support multiple blogs that we could aggregate to provide a BM-Bloggers shared service.

We looked at what appears to be the market leader Wordpress, but to host this we needed a MySql Db, which we did not want to install, we don’t need another DB technology on our LAN to support. So we settled on BlogEngine.NET, the open source .NET4 blogging platform that can use many different storage technologies, we chose SQL2008 to use our existing SQL server investment.

Installation

So we did a default install of BlogEngine.NET. We did it manually as I knew we were going to use a custom build of the code, but we could have used the Web Platform Installer

We customised a blog as a template and the used this to create all the child blogs we needed. If we were not bring over old content we would have been finished here. It really would have been quick and simple.

Content Migration

To migrate our data we used BlogML. This allowed us to export CS2007 content as XML files which we then imported to BlogEngine.NET.

BlogEngine.NET provides support for BlogML our the box, but we had install a plug-in for CS2007

This was all fairly straight forward, we exported each blog and imported it to the new platform, but as you would expect we did find a few issues

Fixing Image Path (Do this prior to import)

The image within blog posts are hard coded as URLs in the export file. If you copied over the image files (that are stored on the blog platform) from the old platform to the new server, on matching urls, then there should be no problems.

However, I decided I wanted images in the location they are meant to be in i.e the [blog]\files folder using BlogEngine.NETs image.axd file to  load them. It was easiest to fix these in the BlogML XML file prior to importing it. The basic edited was to change

src=’http://blogs.blackmarble.co.uk/blogs/rfennell/image_file.png’

to

src=’http://blogs.blackmarble.co.uk/blogs/rfennell/image.axd?picture=image_file.png’

I did these edits with simple find and replace in a text editor, but you could use regular expressions.

Remember also the images need to be copied from the old server (…\blogs\rfennell\image_file.png)  to a the new server ( …\App_Data\blogs\rfennell\files\image_file.png)

We also had posts written with older versions of LiveWriter. This placed images in a folder structure (e.g.  ..\blogs\rfennell\livewriter\postsname\image_file.png). We also need to move these to the new platform and fix the paths appropriately.

Post Ownership

All the imported posts were shown to have an owner ID not the authors name e.g. 2103 as opposed to Richard.The simplest fix for this was a SQL update after import e.g.

update [BlogEngine].[dbo].[be_Posts] set [Author] = 'Richard' where [Author]='2103'

The name set should match the name of a user account created on the blog

Comment Ownership

Due to the issues over spam we had forced all users to register on CS2007 to post a comment. These external accounts were not pulled over in the export. However, BlogEngine.NET did not seem that bothered by this.

However no icons for these users was show.

image

These icons should be rendered using the websnapr.com as a image of the commenter's homepage, but this was failing. This it turned our due to their recent a API changes, you now need to pass a key. As an immediate solution to this I just removed the code that calls websnapr so the default noavatar.jpg image is shown. I intend to look at this when the next release of BlogEngine.NET appears as I am sure this will have a solution to the websnapr API change.

There was also a problem many of the comment author hyper links they all seemed to be http://. To fix the worst of this I ran a SQL query.

update be_PostComment set author = 'Anon' where Author = 'http://'

I am sure I could have done a better job with a bit more SQL, but our blog has few comments so I felt I could get away with this basic fix

Tags

CS2007 displays tag clouds that are based on categories. BlogEngine.Net does the more obvious and uses categories as categories and tags as tags.

To allow the BlogEngine.NET  to show tag clouds the following SQL can be used to duplicate categories to tags

insert into be_PostTag (BlogID,PostID, Tag)
select be_PostCategory.BlogID, postID, categoryname from be_PostCategory, be_Categories where be_PostCategory.CategoryID = be_Categories.CategoryID and be_PostCategory.BlogID ='[a guid from be_blogs table]'

A workaround for what could not be exported

Were we had a major problem was the posts that were made to the original .Text site that was upgraded to Community Server, these were posts from 2004 to 2007.

Unlike all the other blogs these posts would not export via the CS BlogML exporter. We just got a zero byte XML file. I suspect the issue was some flag/property was missing on these posts so the CS2007 internal API was having problems, throwing an internal exception and stopping.

To get around this I had to use the BlogML SDK and some raw SQL queries into CS2007 database. There was a good bit of trial and error here, but by looking at the source of BlogML CS2007 exporter and swapping API calls for my best guess at the SQL I got the posts and comments. It was a bit rough, but am I really that worried over 5 year old plus posts?

Blog Relationships

Parent/Child Relationship

When a child blog is created an existing blog is copied as a template. This includes all its page, posts and users. For this reason it is a really good idea to keep a ‘clean’ template that as as many of the setting correct as possible. So when a new child blog is create you basically only have to create new user accounts and set its name/template

Remember no user accounts are shared between blogs, so the admin on the parent is not the admin on the child, each blog has its own users.

Content Aggregation

A major problem for Black Marble was the lack of aggregation of child blogs. At present BlogEngine.NET allows child blogs, but no built in way to roll up the content to the parent. This is a feature that I understand the developers plan to add in a future release.

To get around this problem, I looked to see if it was easy to modify the FillPosts methods to return all post irrespective of the blog. This would, I my opinion,  have taken too much hacking/editing due to the reliance on the current context to refer to the current blog, so I decided on a more simplistic fix

  1. I create a custom template for the parent site that removes all the page/post lists and menu options
  2. Replaced the link to the existing syndication.axd with a hand crafted syndication.ashx
  3. Added the Rssdotnet.com open source project to the solution and used this to aggregate the Rss feeds of each child blog in the syndication.ashx page

This solution will be reviewed on each new release of BlogEngine.Net in case it is no longer required.

Summary

So how was the process? not as bad as I expected, frankly other than our pre-2007 content it all moved without any major issues.

It is a good feeling to now be on platform we can modify as we need, but has the backing of an active community.

We need to teach children computer science

The Today programme on BBC Radio 4 this morning had a section on children not being taught computer science. so says a variety of employers who cannot find the staff they need. It seems entry to Computer Sciences degrees has been dropping over the years and the current ICT course at school are to blame.

I was at school in the first generation to get access to computers. I did Computer Sciences O level (1982 I think, it is a while ago) and we learnt about flowcharts, CPU, memory and programmed in BASIC on a teletype and sent 5 hole punch paper tape off to a local Polytechnic for processing and a week or so later got back a printout saying error on line 10 (and my staff today complain about their slow PCs!).  During the course we did at least move onto a Tandy TRS80 so it did get a bit more immediate.

My sister is 4 years younger than me, and one of the first group to do GSCE, she also did Computer Science, but in those few short years the course was already moving towards using a computer as opposed to programming/how it works, though she did have access to a BBC Micro I don’t remember her ever writing code beyond making a Turtle robot draw a box. This trend to consumption instead of creation seems to have continued onwards. My son who is 9 now uses a computer for many lessons, but only it seems to look things up.

On listening to the radio article, I agreed it is good to teach the ‘how it works’ for computing, just as it is reasonable to know roughly how my car works, though I have no real intention of trying to fix it. A basic understanding of any tools allow you to use it to its best advantage. However, the biggest advantage of Computer Science in schools to me, which they failed to mention, is it teaches logical thinking and fault finding in an unforgiving world. When coding if you miss that semi colon off nothing is going to work.

This sort of skill that is vital in most modern jobs. At Black Marble we have been involved in this area for years, such as being the corporate sponsor/advisors of  the 2007 UK Winners in Microsoft Imaging Cup with ‘My First Programming Language’. This aimed to teach junior school children how to solve logical problems and program. It also actually addressed one of the issue mentioned in the radio article, the lack of teachers with programming skills (you can’t rely in a keen maths teacher who had built their own computers as my school did on the 80s). It incorporated some AI technology to do the first diagnostic to try to fix the code the student had written before interrupting the teacher, think a complier that can fix the code and explain why the student’s code has failed to the student. This is important when the teacher has 20+ children to help so can only give each one a few minutes.

During this project, and other research work we have done, it showed that many people can benefit from knowing just a little on how to program, not just school children. A good example is the office worker who if they can write an Excel macro can save themselves 20 minutes a day. This adds up so they can save themselves around 60 hours a year and probably reduces there number of errors in calculation too.

I agreed with a recent point on a Herding Code podcast that many people of my age got into computing because they were typing in games from the front of magazines in the 80s With a bit of application you could see how your favourite arcade games worked. This is less true of todays popular games, an XBox title is more like a 3D movie when all you can create is a holiday snap, it is just too big to comprehend. However, the games on phones and PDAs are more accessible, a kid in the garage can see how they work and create their own games and applications with relative ease, it is back like being the late 80s or early 90s.

So I do hope there is a move back to a real Computer Science in schools. It is not as if there are no jobs in this sector.

Black Marble at the Abbey Dash in Leeds

We got a good turn out of Black Marble staff at this years Age UK Abbey Dash 10K in Leeds. We were amongst over 8000 runners who turned out for a cool and foggy Yorkshire Sunday morning run.

image

We had a handicap race amongst ourselves, using a wisdom of crowds poll to estimate everyone target time. This was won by Steve Spencer our Development Director. He beat his target by 20 minutes. I am not sure whether this says more about his fitness or our staff’s opinion of him!

image

Congratulations to all who took part, I am sure plenty of good causes benefited from the efforts of everyone who ran

Tempted by the new Kindle?

I am back on the should I buy a Kindle train of thought. Todays announcements are certainly interesting, I am not talking so much about the new Kindle Fire, but the new entry level version and the Touch. For me the tempting feature is still the E-Ink and battery life.

The point is I have got used to reading on my phone, a Kindle might be easier on the eye, but it is more kit to carry, and I just don’t think I want to carry any more things.

Why can’t I see my custom work item types in Team Explorer?

If you are editing a TFS process template you have the choice editing XML files or using the Process Template Editor within the TFS 2010 PowerTools. Unfortunately neither is fool proof. You can make errors than means the revised template does not fully work.

The worst of these errors will be picked up when upload the process template to a Team project Collection as during this process the XML is validated.

However, this will not find everything. Today after uploading a new process template I found I could not see two of my revised work item type in the list when I tried to create a new work item.

The best way I found to work out the problem was to try to import the .WIT file (from your local copy of the process template) again using either the Visual Studio –> Tools –> Process Editor –> Work item Types –> Import from File, or the command line tool WITIMPORT

In my case I then got the far more useful error

--------------------------
Error
---------------------------
Error importing work item type definition:

TF237094: Field name 'Priority' is used by the field 'Microsoft.VSTS.Common.Priority', so it cannot be used by the field 'MyNewProcess.Priority'.
---------------------------
OK  
---------------------------

This gave me much more information to fix my problem

Bradford the centre of all scientific advance (at least this week)

Over the weekend I took my son to the excellent ‘Bang goes the Theory’ Live road show which is part of the Festival of Science in Bradford. If a similar event is near you I recommend you go along, fun for all the family even if science is not really your thing. The whole event makes the subject very accessible to all.

image 

Even if I had not known this was on I might have guessed something as afoot in Bradford as virtually every Radio 4 news program has had a short article on some new scientific advance from Bradford University. Today a lie detector using facial analysis and a new cancer treatment to name just the ones I remember. Now I went to Bradford and I know it is world leader in biomedical science, but I do wonder if they have been saving press released to get the biggest bang. Or is it the BBC and other media outlets are in Bradford at the festival so it is easy to get Bradford stories?

It is great to see more positive news articles on science and technology, but the danger is it is just a headline. We need to look behind the headlines to be better informed. Events like this road show a great step to get people interested to find out more, and always to keep ‘asking why is that……’

The community - a route to free training

I presented last night at the West Yorkshire BCS and received a number of comments that it was great to see free training being available. This for me is one of the most important areas of our community, and an area we are very lucky with. There are plenty of opportunities to access knowledge…..

Of late we have seen that the big vendor conferences (such as Microsoft’s TechEd, PDC and MIX) and the platform independent developer conferences (such as NDC) streaming their sessions live for free and also have their sessions available for free download within a few days. Microsoft’ Channel 9 is also an excellent resource of conference sessions and specially made material on a vast range of subjects from language design to scheduling of the buses on Microsoft campus. Or if you prefer to listen to something whist your drive why not try the .Net Rocks podcast. There are plenty more specialist resources out there on the web.

At the national level in the UK we also have vendor events such as Microsoft UK Techdays, and company run events like my company’s Black Marbles free events.

Outside company structures we have community events like DDD (general .NET interest) and SQLBits (SQL specific). These are free community run conferences where the members of the community submit session ideas, these are voted for by the community and a free Saturday conference is run using usually the  20 most popular sessions. Again, increasing some of these sessions are videoed and made available on the web.

At the even more local level there are always usergroup. In Leeds we have the BCS, Agile Yorkshire, Leeds SQL Usergroup, GeekUp and more I am sure I have not heard of. This pattern is reflected across the OK, it is just a matter of searching to see what is near you.

So there is no excuse for not trying to keep up to date, whether you want to learn via the web or in person. Why not have a look?

Lenovo W520 – one week on

I have had my Lenovo W520 a week now and must say I am very happy with it. I am still hitting the wrong keys a good deal of the time, the only problem being the Fn and Ctrl keys are reversed in position from my old Acer, it is a good job Fn-V and Fn-C does nothing dangerous!.

The keyboard is very nice to type on, a far more solid feel than any previous laptop I have had.

The performance is great, but you would expect it to be good with an Core i7 and 16Gb.

But the most impressive thing has been the battery life, a good five hours even running a TFS instance on Windows 7 and having VS2010 and Office open as I do most days. I had not expected that, the battery extender bits seems to do the job very well.

So all in all very happy, seems an excellent desktop replacement, I just need to find the time to get a lab management demo rig onto it to check it out in really bit load.

My first serious problem with my WP7 device

My WP7 LG-E900 decided to do a factory reset last night. It locked while playing the XBox Live GEODefense game on a particular busy level I had just lost (so lots of graphic processing). The phone paused for a while then restarted and ran the first use wizard, in German. This is because it is one of the PDC units that seem, given the packaging, to have been sourced from Vodafone German.

This is the first failure I have had on the phone, not a real pain as once the phone was back running in English it was quick to setup and sync, but 15 minutes I would rather have not spent.

[Update 17 Jun 20011] What I had fogotten to setup

  1. I had to set the APN for mobile data (just the name 'internet' with no UID or PWD for Vodafone UK, but I had forgotten that too and of course could not chekc them on the internet). When I first got this phone I am sure mobile data just worked. Anyway I only found it was not working when stuck in a huge traffic jam and I could not use the maps on my phone to get an alternative route. At home data had worked as I had setup WiFi
  2. Pair my bluetooth headset, again only found this out when tried to make a call from the huge traffic jam to say I would be late. Turns out it is hard to pair a headset in a traffic jam full of bluetooth devices.
  3. And I keep finding I am missing application and I can't remember what they are called......