Further thoughts on mocking out a SMTP Server

I posted on the problems I had had mocking out an SMTP server, well I have moved on a bit. As I said in the update note in the last post, I had given up on nDumbster and moved over to LumiSoft's freeware mail server.

The Lumisoft server is a far better starting point as it is a 'real' server that supports all the major mail protocols. As all the source (and binaries if you can't be bothered to build it yourself) are shipped it is easy to create a wrapper class for unit testing purposes that can do whatever you need.

However even with this much improved server I still had a problem with System.Net.Mail calls. I had four tests

• Send a single email with System.Web.Mail
• Send a single email with System.Net.Mail
• Send a batch of three emails with System.Web.Mail
• Send a batch of three emails with System.Net.Mail

Before each test a new SMTP server was created, and when the test completed it was disposed of. Each test worked if run by itself.

The problem was that if the tests were run as a batch the final test failed. When the SMTP server was checked it was found to have no messages recorded, not the expected three. However, the logging said three messages had been sent OK. Swapping the order if the tests did not effect the basic issue that the second System.Net.Mail test reported no emails received, whilst the System.Web.Mail tests were fine.

By adding a unique identifier to each created SMTP server it could be seen that the fourth test was sending its mail to the second SMTP server (which should have been disposed of well before the fourth test was started) hence the final test failing.

The problem appears to be that the threading model inside System.Net.Mail holds the TcpClient object in memory for longer than you would expect, so somehow allows the fourth test to reuse the connection (and server) from the second test. Though it is unclear how you are able to have two servers both on port 25 at the same time. I guess this theory could also go some way to explaining the issues I had with the nDumbster implementation.

Though not perfect, the solution I used was to make the Smtp Server instance static for the test class so for all the tests I created just one instance of the server. Before each test I cleared down the received messages store. Thus far this is working reliably for both single tests and batches of tests.

Update 6 Oct 2008 - The use of a static Smtp server per class is not a full solution, but just moves the problem on i.e. you get the same problem is you have two classe in the batch each with their own server. I need to find a better way to rip the Smtp server of memory - Maybe an AppDomain?

Update 10 Oct 2008 - Well AppDomains don't help. The best solution I have found is to not use a static Smtp server but to instaniate a non static member one and then start it in the test systems TestInit method. In the test systems TestCleardown I stop the server and dispose of it. However this does not fix the problem. I therefore then send an email to the server I have just disposed using System.Net,Mail. This obviously fails but does cleardown the internal cached items inside the System.Net.Mail structure. I just catch the expected exception and carry on. I know it is not elegent but it works both within a test class and an assembly containing many classes