The folks at TypeMock have released a new UnitTesting tool aimed specifically at catching deadlocks in multithreaded code called TypeMock Racer, and what's more they are offering free licenses to folks willing to review it during the 21 day free trial period.  As anyone who knows me can testify to, I am a whore for free-bees, so I decided to take them up on this.

For the impatient, here is the executive summary:
This is a good tool.  Period.  It is, however, also a very specific tool that is intended to help find very specific problems.  If you are not doing any multithreaded code, there is no need to have it in your toolbox.  The cost ($890 US) is high enough that it doesn't really seem worth it to get it "just in case" (unless, of course, you right a nice review on your blog and get a free license).  If you do work with multithreaded code, and you are concerned about deadlocks, this tool will save you lots of time, which ultimately means money.

Now, for the details

First of all, this is a tool from TypeMock, so I expected some pretty incredible things, even at the 1.0 release.  After all, their flagship product TypeMock Isolator is phenomenally powerful, so much so that the free license I was given for posting an advertisement for the release of their ASP.Net product has gone largely unused.  I'm just scared of it.  It's like having access to a 50 horse-power tablesaw with no blade guard.  I may be careful enough to use it correctly most of the time, but the fact that it can take off my forearm without pausing to ask my permission makes me reluctant to get too close.  I do know that there are some very real problems that the tool can solve that just can't be done with any other tool, so I have every intent of getting up my courage, grabbing a first aid kit, and jumping in to see what I can do...eventually.

But Racer is different.  It is a very powerful tool with a very specific purpose.  It makes it easier to run tests in multiple threads, and detect deadlocks.  As far as I can tell, it just detects deadlocks, not race conditions, as it's name seems to suggest.  Not that this is bad, just that it only covers half of the rather shaky ground that is traveled while working with multithreaded code.  It is arguable, however, that dealing with deadlocks is the more difficult of the two problems, so getting this one tacked first is not a bad thing.

So how does it work?

Fairly straight forward really.  Start with a regular unit test, that is exercising some code that is utilizing locks in an attempt to be thread safe.  It looks like Racer supports just about every form of Synchronization supported by .Net, so you can feel free to use whichever mechanism you are more comfortable with to get the work done.  Now that you have a test, the simplest way to make use of Racer, is to add the Typemock.Racer library reference to your test project, and then add a ParrellelInspection attribute to the test.  This causes Racer to do it's thing, which by default means running the test once with a single thread, and then run it again using multiple threads (2 by default).  If there are no problems, nothing much new happens.  You see some additional information about the test being run first with one thread, and then multiple, and something about scenarios executed...nothing that exciting.  The coolness happens when you actually get a deadlock.  For starters you test output includes a bunch of new information, the most noticeable of which is a big "DEADLOCK discovered when running the following" message.  Also is a description of the scenario that caused the deadlock.  Something like "Thread 1 acquired lock A, Thread 1 released lock A, Thread 2 attempted to acquire lock B, etc".  Cooler yet is a message that says "To reproduce or debug this, copy the following attribute, paste it on top of your test and rerun the test:", followed by something that looks like

[SpecificScenario("2", 1, 2, 1, 1, 1, 1, 1, 2, 1, 1, 1, 2)]

Which, while being completely incomprehensible to humans, causes your test to run in the specific configuration which caused the deadlock to occur.  This means that you can accurately recreate the exact situation that lead to the problem.  If you have ever had to try to track down a deadlock in a live system, you will realize that this information just saved you countless hours of trying to recreate the production environment, and lots of trial and error getting things into the state that caused the problem.  One word: Brilliant!

So what is the down side?

I have to say that I've not yet been able to figure out how Racer determines what scenarios to run, or what the bits in the "SpecificScenario" attribute mean(well, the first string parameter is the number of threads, and the other numbers refer to the specific threads, and match the summary of the scenario, but beyond that, not a clue).  It would be interesting to know these things, but not really critical, as long as you are confident all appropriate scenarios are being executed.

There is also an interesting feature that I can't quite get my head around.  When you run a test with a deadlock, an image is generated, which is supposed to be a visual representation of the scenario that created the deadlock.  Here is an example:

 

Now, I see the three objects I was locking against (sync1, sync2, and sync3), and I guess the odd rhombus shaped objects represent the threads, but I'm not really sure what the diagram is trying to tell me.  This is, no doubt, something which is still fairly raw in this early version.  I think it could be very useful if it were clearer what the shapes and arrows represent, but at this point it is simply a cleaver bit of kit that you can show somebody so that they can be confused too.

The last issue I can see with it currently is the price.  At $890 US for a single user license, it isn't an impulse buy.  Granted, I think it can pay for itself easily after finding a few deadlocks in some production code (the earlier they are found the more that find is worth), but I don't see it being a terribly easy sell for management, at least if you are not actively trying to correct a threading issue.  I feel pretty fortunate that I work for a company that understands the value of good tools, and is willing to provide them if there is a need, but I think it would take some convincing to get management to agree to purchase Racer simply because it is a good tool, and could really pay off "one day".  If we were facing a threading issue, and I could demonstrate that racer would allow us to find it, and accurately reproduce it, it would be a fairly easy sell. 

So overall

I think this is an excellent tool.  Based on the fact that this is an early release, I can only see it getting better over time.  It is a rather pricey tool, however, so you may have to do some convincing to get the boss-man to get you a license.  There is a 21-day trial, however, so if you find yourself in a situation where you either have, or you could conceivably have, a risk of deadlock, then grab the trial, and use the first detected deadlock as justification to get a full license. 

Unit Testing ASP.NET? ASP.NET unit testing has never been this easy.

Typemock is launching a new product for ASP.NET developers – the ASP.NET Bundle - and for the launch will be giving out FREE licenses to bloggers and their readers.

The ASP.NET Bundle is the ultimate ASP.NET unit testing solution, and offers both Typemock Isolator, a unit test tool and Ivonna, the Isolator add-on for ASP.NET unit testing, for a bargain price.

Typemock Isolator is a leading .NET unit testing tool (C# and VB.NET) for many ‘hard to test’ technologies such as SharePoint, ASP.NET, MVC, WCF, WPF, Silverlight and more. Note that for unit testing Silverlight there is an open source Isolator add-on called SilverUnit.

The first 60 bloggers who will blog this text in their blog and tell us about it, will get a Free Isolator ASP.NET Bundle license (Typemock Isolator + Ivonna). If you post this in an ASP.NET dedicated blog, you'll get a license automatically (even if more than 60 submit) during the first week of this announcement.

Also 8 bloggers will get an additional 2 licenses (each) to give away to their readers

So, to take a slightly different turn from my usual meta discussions of process, theory, and architecture, there have been several people who have offered up some examples of extension methods that they have found useful, now that .Net 3.5 is roaring along nicely.  There are some collections of such utilities, like the Umbrella project, and some folks like Bill Wagner who have written books on the subject (okay, there are other things in there too), so I thought I might as well throw my hat into the ring as well.  Specifically, there was this tweet from @elijahmanor a few days ago.  It points to a positing which includes some extensions on string to convert from strings to value types (int, long, short, etc).  I pointed out that in our current project we have distilled this down to a single extension method: To().

He suggested I blog about it, so here it is:

So we actually have two classes of conversions, one converts from a string to a value type, and they other converts from a string to a Nullable value type.  In the case of our project, the nullable version came first, and so it became very easy to create the version that returned a non-nullable value.  Here are the methods of interest:

public static T To(this string input) where T : struct
{
    return input.To(default(T));
}

public static T To(this string input, T defaultValue) where T: struct
{
    return input.ToNullable() ?? defaultValue;
}

public static T? ToNullable(this string input) where T : struct
{
    if (string.IsNullOrEmpty(input))
        return null;
    var tryParse = GetTryParse();
    return (T?)tryParse(input);
}

Okay, so this is pretty straight forward.  The non-nullable version calls through to the nullable version, and if it is null, it returns default(T).  But, as I'm sure you are an astute reader, you will see that there is some magic going on; namely the GetTryParse() method.  This little guy goes off and looks for a TryParse method on whatever T happens to be, and then returns a Func<> delegate that will run the string input through the try parse and return either a null (in case the TryParse fails), or a boxed version of the result.  So, lets see what it looks like before we discuss pros and cons

private static Func GetTryParse()
{
    var tryParseEx = GetTryParseExpression();
    return (s) => tryParseEx.Compile()(s,default(T));
}

private static Expression> GetTryParseExpression()
{   
    if (_tryParseCache.ContainsKey(typeof(T)))
	return _tryParseCache[typeof(T)] as Expression>;

    MethodInfo tryParse = typeof(T).GetMethod("TryParse", new Type[] { typeof(string), typeof(T).MakeByRefType() });
    Ensure.IsNotNull(tryParse, string.Format("Cannot convert from type string to type {0} because {0} does not have a TryParse method", typeof(T).FullName));

    var stringArg = Expression.Parameter(typeof(string), "input");
    var tempArg = Expression.Parameter(typeof(T), "tmp");

    var tryParseEx = Expression.Lambda>(
        Expression.Condition(
            Expression.Call(tryParse, stringArg, tempArg)
                , Expression.Convert(tempArg, typeof(object))
                , Expression.Constant(null))
        , stringArg, tempArg);
    _tryParseCache.Add(typeof(T), tryParseEx);
    return tryParseEx;
}

So here we have some code looking for a TryParse method, and building an expression (using Linq Expression Trees) to execute it.  Now, I'll be honest, this is not the code I'm using in the project where this originally came from...mostly because I didn't think of it then.  In that case I'm actually doing a big case statement, checking the type of T and running the appropriate method.  This is much shorter, but potentially much slower at runtime.  So that is where the _tryParseCache comes in.  This is a simple static dictionary which contains the expressions created for each of the types, which means you only get the runtime performance hit once when you first ask to parse a specific type.  The declaration for this object looks like this:

private static Dictionary _tryParseCache = new Dictionary();

There you have it, my first (and possibly last??) contribution into the world of extension methods. Please commence criticisms

So previously I posed a question, which in it's simplest form is: Should you write code for the rest of your group (or at their proficency level), or should you write code as advanced as you need. and let it serve as an example for those on your team who are less advanced in their abilities.  The practical answer I have come up with is, like most answers of this type, "It depends".

I have decided to handle things this way: 
First, don't compromise.  If I feel something is a bad practice, or ultimately going to restrain future development, then do what needs to be done.
But Also: Try to avoid introducing advanced concepts and idioms until there is a compelling example of what their benefit is.  This is probably something that applies more to working with existing apps, then greenfield development, but it certainly has bearing in both cases.  The big example for this that I ran into was IoC.  I am a big fan of IoC, but it is hard to come up with a good, concise explanation of why you need this additional tool.  I've been wanting to introduce IoC since I started working at Envisage, but the explanation "This will make things much easier later on" is not good enough...particularly when you are trying to embrace YAGNI.  So the key is to wait until you can actually demonstrate the advantage, and provide a before and after example of how things are done.

Lead by example, but make sure you have the examples.  To bring the food analogy back into play, it isn't good enough to create a gourmet meal, and tell the people who say they don't like it that, no, it really is better, and their just not sophisticated enough to know it.  It is better to start smaller, and build their appreciation up in increments.

At Envisage, we have a 1 week iteration.  This means that we typically don't want a single story to go longer than one week.  We also change up the pair rotation once per week.  The estimation process is not perfect, though, and we have a lot of support from management to choose doing things right, rather than doing things now, which leads to stories going beyond a single iteration.  So when this happens, there is usually a small rock/paper/scissors session with the developers involved to determine who will be taking the story.  More often then not, the story goes with the machine (and the developer) that was used to do the work, since it is usually not practical to check in changes at the end of the week (also, our QA builds on Fri for a demo, so a green build is needed on Fri).

One approach to this, which occurred to myself, and one of our other developers at close to the same time, was to utilize the concept of a personal branch.  This is something that I believe is supported transparently in TFS, though I'm not 100% sure.  We are using Subversion, so the process is somewhat manual, but overall pretty easy to get started.  Once a branch was created, I found it extremely liberating.  I have apparently been making decisions about how much to change while I'm refactoring code at least partly based on how much work it will be to integrate directly back into the trunk.  Having a private, protected area for me to work made it much easier to change things in the way they needed to be changed.  It also meant that I could check in more often, including checking in changes which would leave the app in a non-functional state.  Having those commit points meant that I could more easily undo changes, and gave me a nice warm and fuzzy feeling about the changes I was making.  There was also another interesting advantage, and that was when another developer was asking me about how the API would look on some of the objects I was working on, and I was able to point him directly to my branch to see the changes.

There were a few issues, however.  The biggest was that, though we are running the Subversion 1.5 server, our repository has not been updated, so the automatic merge tracking was not working.  This meant that I had to keep track of the revision numbers myself whenever I needed to merge updates to the trunk into my branch.  And this also made the "Reintegrate Branch" merge function impossible when I was ready to check my changes in.  Despite these issues, I think in this particular case (the story lasted about 3 weeks) I was worth the extra effort, and made the overall process much easier.  We will be updating our repository this week, which may make having a personal branch a viable solution for normal day-to-day work, but as it is the effort involved was a bit more than what I would want to deal with on a regular basis.  I will certainly not hesitate to break out this tool whenever I've got either a long running, or a large scope (meaning either large numbers of files effected, or a large change to parts of the API) story.  I certainly recommend giving it a try.

Like so many others, I've been sucked into the alure of StackOverflow.

This is kinda like a porn addiction, only more destructive to personal relationships.

We had a friend visiting from out of town last night, and she is apparently fond of "Top Chef".  This was the first time I had encountered this particular "reality" based show, and I found in a lot of ways it was like the majority of the rest of the shows out there; trying hard to create drama where there really isn't any.  In this particular episode there was a challenge to cook food for a neighborhood block party.  The group that did the worst in this particular case cited the fact that they were preparing food for everyone including kids, as a reason for not getting jiggy with the menu (in a culinary sense).  The response from the judges was that good food would speak for itself, and they should have considered it their responsibility to raise the collective bar of quality for the neighborhood.

I find this quite interesting because it mirrors an internal dilemma I am facing with the code I am currently putting together.  The dilemma is this:  Should I create things which are more "advanced" and more difficult for those who are not familiar with the concepts, or should I take the audience into consideration, and forego some of the more advanced concepts in favor of simplicity and discoverability?

My gut tells me that I should do the latter.  I'm a big believer in the power of writing code that is easy to understand and maintain.  This is particularly true whenever you are writing code which will potentially be consumed by other developers working within the application.  If someone cannot figure out how to utilize the work you've done, then chances are they are going to go off and duplicate it.  The eventual result of this being half a dozen ways to accomplish what is basically the same thing.

The idea brought up on the show seems to go against this philosophy by suggesting that you have the ability to expand the pallets of those who are consuming your work (the food metaphor is staring to get pretty rich here), and so you would be doing them a disservice not to.  In my case, I've found places where my choice to keep things more like what everyone is used to has added complexity and ugliness elsewhere. 

So does this mean I should forge ahead and present concepts which are going to be new to others on the team, and potentially create code that will not be re-used as it should because it does not make sense to those needing to re-use it?  If I don't introduce these concepts am I missing an opportunity to bring up the overall quality of the code being by everyone?  Am I just struggling to fid answers to questions that I really shouldn't be asking, since the YAGNI police are keeping their eyes on things?

I think this falls pretty solidly in the DUH! category.

I'm currently finding myself in the midst of an evolutionary change.  And I'm not talking about my super-human mutant powers, I'm talking about the way I'm thinking about solving a specific set of problems.

Let's start with a sample....Lets take something like processing credit-cards as a benign and IP free place to start.  As a subject that I really have no practical experience with, it seems like an appropriate choice.  I'm going to assume that there are different rules for doing checksum validation on credit card numbers, depending on what the card is (Mastercard/Visa/Discover/etc). Now, here is evolutionary step 1: Use a basic case statement to process the various cards.  Here is what something like that would look like:

switch(CardType)
{
    case CardType.MasterCard:
        CardValidators.MasterCardValidator(card.CardNumber);
        break;
    case CardType.Visa:
        CardValidators.VisaValidator(card.CardNumber);
        break;
    case CardType.Discover:
        CardValidators.DiscoverValidator(card.CardNumber);
        break;
}

 

This looks pretty straight-forward, and as it stands it isn't too bad from a maintainability stand point.  But what happens when there are many different types of cards?  An then what happens when you find a large amount of duplication between the validation functions?

Well, any student of GoF should be able to tell you that a Chain Of Responsibility pattern looks like a perfect fit for this sort of scenario.  So, evolutionary step 2: Create separate classes to handle the different types of validation, and configure them in a Chain of Responsibility pattern where each instance decides for itself whether it can process the input.

Here is a quick and dirty look at what something like that would look like:

 

The two most interesting things here are the GetValidator() method in the AbstractCardValidator, and the individual CanValidate() methods in the concrete implementations.  What this does is it allows each class to decide for itself how it is going to determine whether or not it can be used as a validator for a specific card (thats the CanValidate() part), and also provides a single point which the consumer of the API can use to get the validator for the card instance they have.  You would probably want to build and Abstract Factory around this, which would instantiate all of the ICardValidator classes, and then run the GetValidator() method to get the correct one.

Now we are at a point where things are looking pretty good; we've got the ability to do some fairly complex logic to make the decision about which validator to use, and we have a way to simply ask for one, and the correct one appears.  Pretty cool.

This is actually the place where I have found myself in the not to distant past.  I have previously been perfectly content with this arrangement, and been fairly happy with the separation of concerns among the classes...I mean, after all, who better to decide whether or not a specific class should be used to validate a card than the class itself.  So what is the issue?  Well, recently I have become aware of two problems with this arrangement: Tight Coupling, and a violation of the Single Responsibility Principle.  Let's start with the first:

Tight Coupling
The credit card example may be a bit contrived when it comes to this issue, but bear with me.  Overall, the issue is that specific instances of ICardValidtor objects are being created and handed around.  The use of an interface and an Abstract Factory pattern would actually help the situation out some, but effectively all it does is move the coupling from the consuming class to the Factory (okay, it also consolidates coupling to a single class, which makes maintenance a lot easier).  As I said, contained, but still there.  It would be nice if the factory didn't need any knowledge of what concrete instances of ICardValidator were out there.  Before we tackle that, though, lets also look at the second issue:

Violation of the "Single Responsibility Principle"
The SRP states that a class should have one, and only one, thing it is responsible for.  Sounds pretty easy doesn't it?  The problem is that this can be difficult to obtain without a fair amount of discipline.  The violation of SRP which I'm seeing is that the ICardValidator is responsible for both validating a credit card and determining which validator is appropriate.  But wait!  Didn't I just say that moving this check into the ICardValidator instance was a "Good Thing"?  Well, lets go as far as saying it is better than the previous method, but still not perfect.  Applying the SRP would move the task of selecting a validator from the ICardValidator instance, and put it on it's own somewhere.  So, thusly we come to our:

Inversion Of Control Container.
That's right, we are now going to get crazy and move the responsibility of creating these instances to another component all together.  The nice thing about this is that it allows us to move all of the knowledge about dependencies off somewhere else.  How does this apply to this example?  Well, lets assume we have an object of type Card which requires as a dependency an instance of an ICardValidator.  We'll also assume that Card is subclassed based on the type of credit card.  It now becomes trivial to configure our IoC container to supply a specific implementation (read sub-type) of ICardValidator for each implementation (again, read sub-type) of Card.  Now, when you want a Card instance, you ask the IoC container for one, and depending on what type of card it is, you will get the appropriate ICardValidator as well.

What's the catch?  Well there is some additional complexity which will show up somewhere in the application due to the IoC, but typically IoC configuration can be delegated down to the configuration file level, so even then the ugliness is pushed away to it's own dark corner.

But wait!  Why should we have different instances of Card?  What if the Card class is just a container for the card data?  Well, our Ioc still gives us some advantages.  If we look back at our first example with the switch statement, we've got a nice CardType enum, which could be a property of our Card class.  Using an IoC container like the one provided by the Castle project, you have the ability to configure a key string for your instances.  This would make it trivial to map the enum choices to specific keys within the container, which the Card class would use to get an ICardValidator instance.  This would also make it possible to make the validators slightly more advanced by adding something like a Decorator pattern, in which specific aspects of the validation could be factored into separate classes, and then "stacked" to produce the final validation logic (This is the same concept used by the Stream classes in .Net and Java.  You can modify the behavior of a stream by passing it to the constructor of a stream with different behavior).

It is definatly worth mentioning that there is a sudden appearance of tight coupling to the IoC container itself from our consuming classes.  You probably want to try to abstract away the fact that the IoC container exists from the majority of the application.  Factory classes go a fair ways in making this happen, but another good idea is to introduce a single service to do type resolution.  The Factory classes can then ask this service for the object they want, and they never need to know the IoC container is there.  This approach also gives you the ability to create some objects using IoC and others in another (more traditional) way.

So is this it?  Have I finally found the answer I've been looking for?  It's hard to say right now.  For the time being this is a decent way to handle things, provided the complexity of the underlying system, and the need for loose-coupling are both high enough to justify the additional complexity of the IoC.  But who knows, in another couple months I may find something new, or even something old, which seems better, cleaner, simpler.  That, after all, is my final goal....And I need to remind myself of that regularly, lest I become complacent.

About two weeks ago I crafted my first Fluent Interface.  Since then I'm finding myself seeing more and more places where I think such an approach would be useful.  The part that I'm finding odd is that it is something that just recently became a possibility for me.  The big motivating factor behind that I believe was reading the post that Martin Fowler did on the subject, in which he basically describes it as a super-great idea (okay, I'm paraphrasing, but you get the idea).  The concept is fairly simple; write an API that reads like a sentence. 

This isn't a new concept, in fact I seem to recall reading quite a bit in the world of agile and TDD in which the authors encourage you to make method/property/variable names verbose and more like natural language in order to improve readability of code, and make the items more self-documenting.  I think the big difference, though, is that fluent interfaces tend to be more granular.  Instead of a single method that reads like a sentence, you are building a sentence using method and property names, with Intellisense there to help you determine what is possible at the end.

The big shift, I think, is in the realization that within this context method names like With, For, and And are perfectly okay, and as a matter of fact make things better in the end.  Its like a taboo has been lifted, and suddenly I have a whole new landscape of possibilities opened up.

Since the original implementation of a small fluent interface I created for a small part of my project (I'm using it to describe discrete elements of a document to be parsed), I've found myself creating a new fluent interface to play with .Net 3.5 extension methods (replicating the Ruby 10.Minutes.Ago semantics), and also adding a new interface to the same production project as the first which is being used to grab component services from my IoC container.

I'm not sure if this is a new paradigm, or just a new hammer looking for nails, but it is interesting none-the-less. It has also opened up new challenges around testing and intellisense documentation, which I've not quite figured out yet.