Significant Advances in Unit Testing Windows Workflow

This post describes a unit testing library for testing Windows Workflow Foundations.It is not a framework like HarnessIt, NUnit, or MsTest. Rather it’s a library that can be used in conjunction with any of these testing frameworks.

Download the library with sample test project here:
Kennedy.WorkflowTesting.zip (216 KB)

You can also just jump to the code.

First a Little History:

Last September I posted this teaser entitled Unit Testing Coming to a Workflow Near You. My intention was to post this article that you’re reading now shortly thereafter when I got some free time to polish things up. In that previous post, I highlighted what I could determine to be the current state-of-the-art with regard to unit testing workflows, circa September 2008.

Then I heard through some inside sources that this MSDN Magazine article was about to come out:

    Foundations: Unit Testing Workflows and Activities
    by Matt Milner.

So I decided to see what Matt’s article had to offer to the conversation. It’s a good article to be sure. It covers a lot of the things I thought were undiscussed and yet important to unit testing WF (e.g. using WF services as points of dependency injection for mocks and stubs). Thanks Matt! I don’t have to write about that now, but you’ll see it used in the sample with my library.

What I was really waiting to see was would that article make this post redundant? After reading it, I can say that there’s still a long way to go – and this library will get us most of the way there. Now let’s get to the good stuff!

Significant Advances:

That’s a pretty bold statement, significant advances: let’s see if I can back it up. Here’s what’s missing in one way or another from all the previous work on unit testing WF. (Please note that this discussion is in no way intended to belittle the work of anyone quoted above, just to build on their work and advance testing for us all).

Problems with unit testing WF today that are solved by this library:

  1. Testing single activities: Testing single activities in isolation is hard.
  2. References to the activity: Direct access to the activity under test for asserting on its properties is nearly impossible.
  3. Waiting on workflows: Workflows run on background threads which means waiting for the outcome inside the test method is more cumbersome than necessary (ManualWorkflowScheduler is unnecessarily cumbersome as well).
  4. Untyped name/value collections as input: Using untyped name/value collections as input and return values is error prone (for testing and general use).
  5. Expected exceptions: Testing “failure as success” cases for error handling is essentially broken for WF: Either the exception type is lost, or the call stack is lost, or both.

The Sample:

Let me set the stage first before we see the test code. I have a somewhat realistic workflow which will exchange two stocks and either debit or credit your bank account with the difference. So you might want to sell 5 shares of Google and buy 10 shares of Microsoft and pocket the difference.

Here’s the workflow which involves 4 different activities:

Testing On Single Activities:

The first thing to test is the individual activities (like BuyStock and DebitAccount). Here’s the code to test selling a stock (some details omitted for simplicity, exact code follows later). This method uses my library class WfRunner for executing the test.

[TestClass] public class WorkflowTests : IDisposable {     private WfRunner wfRunner = new WfRunner();     [TestMethod]     public void SellStockComputesCostCorrectlyTest()     {         StockDTO dto = new StockDTO( 7, "GOOG" );         SellStockActivity sellActivity =             wfRunner.RunSingleActivity( dto );         double price = testStockSvc.LookupPrice( dto.Ticker );         int quantity = dto.Quantity;         Assert.AreEqual( quantity * price, sellActivity.Cost );     }     // ... }

This test method (SellStockComputesCostCorrectlyTest) is remarkable for several reasons:

  1. We are taking a single Activity, not a workflow, and executing it.
  2. We are passing a strongly typed DTO (data transfer object) rather than name/value pairs in a Dictionary.
  3. Most Remarkably: We are getting the actual instance of the activity returned to us so that we can explore its properties. Notice that we assert on sellActivity.Cost directly:Assert.AreEqual( quantity * price, sellActivity.Cost );This is not easy to pull off – WF intentionally hides activities and workflows it creates behind workflow instances (proxies basically). This hack might be bad for production systems, but it *rocks* for unit testing.It means you get intellisense rather than programming against strings in name/value collections.
  4. We are not waiting for the workflow to complete or using the ManualWorkflowSchedule. WfRunner is doing that for us because the method RunSingleActivity is a blocking call.

Just so you don’t think I’m trying to pull a fast one: Here’s that same listing with all the gory details left in. Notice how we’re using WF Services for DI with our test stubs.

[TestClass] public class WorkflowTests : IDisposable {     //     // Define some objects that will be used across all tests.     //     private IAccountService testAccountSvc;     private IStockService testStockSvc;     private Account testAccont;     private WfRunner wfRunner = new WfRunner();     public WorkflowTests()     {         //         // Initialize common data for all tests.          // This is basically what the host of the wf-runtime would do         // but we're using test doubles / stubs for our services.         //         Dictionary<string, double> stocks = new Dictionary<string, double>();         stocks.Add( "goog", 350 );         stocks.Add( "msft", 25 );         testAccont = new Account( 1774, 5000 );         this.testStockSvc = new TestStockService( stocks );         this.testAccountSvc = new TestAccountLookup( testAccont );         // Install these services for use by our WF activities.         wfRunner.AddService( testStockSvc );         wfRunner.AddService( testAccountSvc );     }     [TestMethod]     public void SellStockComputesCostCorrectlyTest()     {         StockDTO dto = new StockDTO( 7, "GOOG" );         SellStockActivity sellActivity =             wfRunner.RunSingleActivity( dto );         double price = testStockSvc.LookupPrice( dto.Ticker );         int quantity = dto.Quantity;         Assert.AreEqual( quantity * price, sellActivity.Cost );     }     // ... }

Expected Exceptions

That was pretty awesome huh? We solve several of our problems I identified above (testing single activities, references to the activity, waiting on workflows, and untyped name/value collections as input). The last one to cover is exceptions as success.

When we try to buy a stock and we don’t have enough money, the workflow will throw an InsufficientFundsException. This type is a custom exception created as part of my wf application – it’s not part of .NET. We want to test for this exception:

[TestMethod] [ExpectedException(typeof (InsufficientFundsException))] public void CannotBuyWithInsufficentFundsTest() {     ExchangeStocksDTO dto =         new ExchangeStocksDTO             {                 AccountID = 1774,                 StockToBuy = "MSFT",                 StockToSell = "GOOG",                 SellQuantity = 5,                 BuyQuantity = 7000             };     wfRunner.RunWorkflow( dto ); }

Notice that we’re using the ExpectedException attribute. We just call RunWorkflow as a regular method and we get the exception back as a synchronous error. That is fantastic already. But look at the call stack:

SampleLibrary.InsufficientFundsException: Wrapped excpetion message: Insufficient funds for account 1774.
 ---> SampleLibrary.InsufficientFundsException: Insufficient funds for account 1774.
   at SampleLibrary.Account.Withdrawl(Double amount)
   at SampleLibrary.DebitAccount.Execute(ActivityExecutionContext ctx)
   at System.Workflow.ComponentModel.ActivityExecutor`1.Execute(T activity, ActivityExecutionContext executionContext)
   at System.Workflow.ComponentModel.ActivityExecutor`1.Execute(Activity activity, ActivityExecutionContext executionContext)
   at System.Workflow.ComponentModel.ActivityExecutorOperation.Run(IWorkflowCoreRuntime workflowCoreRuntime)
   at System.Workflow.Runtime.Scheduler.Run()
   --- End of inner exception stack trace ---
   at Kennedy.WorkflowTesting.WfRunner.TransformAndThrowIfRequired(Exception realException)
   at Kennedy.WorkflowTesting.WfRunner.RunWorkflow[T](Dictionary`2 namedArgumentValues)
   at Kennedy.WorkflowTesting.WfRunner.RunWorkflow[T](Object workflowDTO)
   at SampleLibrary.Tests.WorkflowTests.CannotBuyWithInsufficentFundsTest()

The WfRunner class has determined there was an exception. Rather than just rethrowing it and losing the callstack (notice it’s still intact), we do this operation called TransformAndThrowIfRequired. TransformAndThrowIfRequired takes the real exception, uses reflection to recreate another exception of that type and wraps the real exception as the inner exception.

This both preserves the exception type (critical for the ExpectedException behavior)
and the callstack (critical for debugging).

Running these tests inside Visual Studio we get all green!

I hope you find this library adds significant value to unit testing of your Windows Workflows. Personally, I think it makes unit testing of your Windows Workflows practical in the real world.

Enjoy!
Michael

Note: This only applies to WF 3.0/3.5. WF 4.0 which is part of .NET 4 which is shipping the end of 2009 may change this considerably.

Submit a comment