With my current implementation of the GUITesting framework integrated with Test Projects within VS.Net, I end up with all the GUITests in a single [TestMethod]. This is a far from perfect solution, if nothing else it is hard to debug, you only see the first failing test.
So I thought what I need to do is automatically generate [TestMethod] flagged methods at runtime in something like the form below in the runtime UnitTest class within the Test Project.
public void TestMethod2()
// in the constructor put all the test in an array
// then call a helper method to do the test
So how to do this? Well I don’t have a working answer at present, but my initial attempts have raised a few questions:
- If I used Reflector to get the IL code for the above method, I could use the DynamicMethod system to generate a set of [TestMethod] methods. The best example of using this I have found is on Joel Pobar’s blog. However, I have not worked out how to put the actual test attribute into the IL code, and I am unclear if I can add the method to the current running containing class or have to create a new dynamic assembly. If I cannot do the former then this method is not going to work.
- Probably more importantly if I don't put the [TestMethod] methods in test project at design time the testing harness in VS.Net fails to run as it says there is no tests defined. I am not sure of a way around this if I generate them at runtime. It seems to me that the [TestMethod] framework only allows a single test result per method (not unsurprising really).
- It would be nice of the UnitTest framework had a way to return a number of tests from a single defined [TestMethod], it would remove complexity for me.
More I am sure to follow.
I have rebuilt the GUITesting framework using VS.Net 2005. No changes really, the built in converter from VS.Net 2003 to 2005 did the job with no errors. At this time I have not done any work to exploit the new features of .NET 2.0.
However, what I have had a look at is integrating the new testing features of VS.Net 2005 Team Developer edition. So in this V2.0 source release you will find a test project, sorry if you not have Team developer edition of VS.Net, just remove the test project when it fails to load!
So how did the integration with a Microsoft Test Project go? In the past I had my own test run EXE to do the testing, it looks for custom attributes that define the tests then runs them, this still work. However, as I wanted to use VS.NET to run the tests, I needed to run my declared tests via a [TestMethod] in a test project. I have written a wrapper to do this which can be seen on the test project. Using this wrapper code in the [TestMethod] the tests run fine, the problem is that all my defined tests appear as a single VS.Net test. Fine if they all pass, but not that helpful debugging. The only other option is to create a [TestMethod] for each test a have in my application under test, not really an options as these tests are user defined and discovered via reflection.
This is where I am up to at present, the new source release can be found at http://www.blackmarble.co.uk/guitester/GUITesterSourceV2.0.zip, the existing PDF manual is still correct for the implementation of tests found at http://www.blackmarble.co.uk/guitester/GUITester Usage Manual.pdf.