As of right about now, you should be able to mosey on over to the DxCore Community Plug-ins page, and grab a copy of CR_MoveFile.  This is a plug-in I created primarily as a tool to aid in working in a TDD environment, but which certainly has uses for non-TDD applications.  It does basically what the name suggests, it allows you to move a file from one directory in your solution/project structure to another, even one in a different project.  I implemented this as a code provider (since it could change the functionality if you move the file from one project to another), so it will appear in the Code menu when you have the cursor somewhere within the beginning blocks of a file (“using” sections, namespace declaration, or class/interface/struct declarations).  Once selected you are presented with a popup window which has a tree that represents your current solution structure, with your current directory highlighted.  You can use the arrow keys to navigate the directories and choose a new home for your file.

If you move files between projects, the plug-in will create project references for you, so you don’t need to worry about that.  When the file is moved the file contents remain unchanged, so all namespaces will be the same as they were originally.  I did this mostly to keep the plug-in simple, but also because I could see situations where this would be good, and situations where this would be bad, and it seemed like this was a bad choice to make for people.  I’ve been using this plug-in on a day-to-day basis for a while now, and things seem pretty clean, I did run into a small issue, however, using it within a solution that was under source control.  At this point you need to make sure the project files effected by the move are checked out, otherwise the plug-in goes through the motions, but doesn’t actually do anything, which is quite annoying.  There is also no checking going on to make sure the language is the same between the source and target project, so if you work on a solution that contains C# and VB.Net projects, you have to be careful not to move files around to projects that can’t understand what they are (oh, and the project icons used on the tree view are all the same, so there is no visual indication of what project contains what type of files).

That’s pretty much it.  Clean, simple, basic.  Used with other existing CodeRush/Refactor tools like “Move Type To File” and “Move to Namespace”, this provides for some pretty powerful code re-organization.  Just make sure you run all of your tests :).

Anyone who has been around me for more than a few hours while coding, or who pays any attention to me on Twitter will know that I am a huge fan of CodeRush and Refactor Pro! from DevExpress.  I consider these sorts of tools essential to getting the most out of your development environment, and I think CodeRush is one of the best tools available for a number of reasons, not the least of which is it’s extensibility.  CodeRush is built on top of DxCore, which is a freely available library for building Visual Studio plug-ins (incidentally, DevExpress also have a free version of CodeRush called CodeRush XPress, which is built on the same platform).  DxCore provides any developer who wants it access to the same tools that the folks at DevExpress have for building plug-ins and extensions on top of VisualStudio, and several developers (including yours truly) have done just that.

One of the more recent additions to the CodeRush arsenal are the CodeIssues.  As of the v9 release, CodeRush included an extensive collection of these mini code analyzers which will look at your code in real time and do everything from let you know when you have undisposed resources, to suggesting alternate language features you may not even be aware of.  A lot of these are also tied in to the refactoring and code generation tools that already exist within CodeRush and Refactor Pro! so that not only do you see that there is an issue or suggestion, but in a lot of cases you can tell the tool to correct it for you.  Pretty impressive stuff.

So what I would like to do is dig in to how the CodeIssue functionality works within CodeRush by creating a custom CodeIssue Provider.  Because I’m a TDD guy, one of the things I’ve been trying to do is build in some tooling around the TDD process to make it that much easier to write code TDD.  So based on that I’m going to show you how to implement a CodeRush CodeIssueProvider which will generate a warning whenever you have created a Unit Test method with no assertions (which would indicate that you are either dealing with an Integration Test, or your test is not correctly factored).  Note: Since the CodeIssue UI elements are part of the full CodeRush product, and not CodeRush XPress, this plug-in will note do anything unless you are running the full version of CodeRush.

Okay, so the first thing to do is to create a new Plug-In project.  This can either be done from the Visual Studio File –> New Project menu, or by selecting the New Plug-in option from the DevExpress menu in visual studio (if you are using CodeRush XPress and you don’t have the DevExpress menu, my man Rory Becker has a solution for you).  Regardless of which way you go, you will get a “New DxCore Plug-in Project” window, which will ask you what Language you want to write your plug-in in (C# or Visual Basic .Net), and what kind of plug-in you want, along with the standard stuff about what to name the solution and where to store the files.  For our purposes we’re going to go with C# as the Language, a Standard Plug-in, and we’ll call it CR_TestShouldAssert (the CR_ is a naming convention used by the CodeRush team to indicate it’s a CodeRush plug-in, as opposed to a Refactoring or DxCore plug-in).

Net up is the “DxCore Plug-in Project Settings” dialog.  This allows you to give your plug-in a title, and set some more advanced options which deal with how the plug-in gets loaded by the DxCore framework.  We’ll just leave everything as-is and move on to the good stuff.

Once your project loads you will be presented with a design surface, this is because a large number of the components that are available via DXCore can actually be found in the Visual Studio toolbox, and you can just drag them out onto your plug-in designer to get started.  The CodeIssueProvider is an exception, though, so we will have to crack open the designer file to add it to our plug-in.  So open up the PlugIn1.designer.cs file, and add the following line of code under the “Windows Form Designer Generated Code” section:

CodeIssueProvider cipTestsShouldAssert;

this.components = new System.ComponentModel.Container();
cipTestsShouldAssert = new CodeIssueProvider(this.components);
((System.ComponentModel.ISupportInitialize)(this)).BeginInit();
((System.ComponentModel.ISupportInitialize)(this)).EndInit();

Now if we switch back over to the designer, we will see our new provider on the design surface.  At this point we can use the Properties window to configure the provider.  The things we need to worry about filling out are the Description, DisplayName, and ProviderName properties.  The Description is the text that will be displayed in the Code Issue catalog, so it needs to clearly explain what the CodeIssueProvider is intended to do.  Let’s go with something like: “A Unit Test should have at least one explicit or implicit assertion.”  As for DisplayName, lets say something like “Unit Test Method Should Assert”, and make the ProviderName the same.

Ok, so now it’s time to actually do the work of finding a TestMethod that violates this condition.  So we need to switch over to the Events list for our provider, and Double-Click in the CheckCodeIssues drop-down so it generates an event handler for us.  You will now be taken to the code editor and presented with a empty handler that looks something like:

private void cipTestsShouldAssert_CheckCodeIssues(object sender, CheckCodeIssuesEventArgs ea)
{

}

This looks pretty much like your normal event handler, we’ve got the sender object (which would be our provider instance, and then we have a custom EventArgs object. Looking at this event args object, you can see quite a few methods, and a couple of properties.  The first few methods you see deal with actually adding your code issue, if it exists, to the list of issues reported by the UI.  You’ve got one method for each type of CodeIssue (AddDeadCode, AddError, AddHint, AddSmell,AddWarning), and then one method (AddIssue), which allows you to specify the CodeIssue Type.  Now this is where things start to get interesting because basically we’re at the point where the good folks who wrote DxCore have said “All right, go off and find your problem and report your finding back to me when your done”.  So from here we have to figure out whether or not there are any test methods without asserts floating around anywhere.  The good news is that there are a few tools in the CodeRush bag of tricks that can help us.

Perhaps the best tool for figuring out this sort of thing is the “Expression Lab” plug-in.  You can open this up by going to the DevExpress menu, opening the Tool Windows->Diagnostics->Expressions Lab.  This shows you in real time what the AST that CodeRush produces for your code looks like as you move about in a file.  You can also see all of the properties associated with the various syntax elements, and view how things are related.  This is a very handy tool to have.  Before we dig too deep into the Expressions Lab, lets get a start on finding our CodeIssue.  We know that we are going to be looking at methods here, since we are ultimately searching for test methods, so the first thing to do is to limit the scope of our search to just methods.  The CheckCodeIssues event is fired at a file level, so you are basically handed an entire file to search by the DxCore framework.  We need to filter that down a bit and only pay attention to the methods contained in the current file.  To do that we’re going to use the ResolveScope() method of the CheckCodeIssuesEventArgs object.  Calling the ResolveScope() method gives us a ScopreResolveResult object, which doesn’t sound very interesting, but this object has a wonderful little method on it called GetElementEnumerator().  This method will allow you to pass in a filter expression, and return all of the elements that match that filter expression as an enumerable collection. So to get to this, lets add the following to the body of our event handler:

var resolveScope = ea.ResolveScope();
foreach(IMethodElement method in resolveScope.GetElementEnumerator(ea.Scope,new ElementTypeFilter(LanguageElementType.Method)))
{
}

This looks pretty straightforward, but there are a couple of things I want to point out. First is the ea.Scope property that we are passing in to the GetElementEnumerable() method. This is the AST object that represents the top of the parse-tree that we are going to be searching for code issues in. Typically this is a file-level object, but I don't know that you can count on that always being the case (changing the parse settings could potentially effect how much of the code is considered invalid at a time, and so you could get larger or smaller segments of code).  The other interesting bit is the ElementTypeFilter().  This allows us to filter the list of AST elements given to us in our enumerable based on their LangueElementType (LanguageElement is the base class for syntax elements within the DxCore AST structure.  All nodes have an ElementType property which exposes a LanguageElementType enum value). In our case we’re only interested in methods, so we’re using LanguageElementType.Method.  The result is a collection of all of the methods within our Scope.

Now that we have all of our methods, we need to figure out if they are Test methods.  To do this we’ll have to look for the existence of an Attribute on the method.  Taking a look at Expressions Lab, we can see that a Method object has an Attributes collection associated with it. So we should be able to search the list of attributes for one with a Name property of “Test”.  Using Linq, we can do this pretty easily like this:

method.Attributes.OfType<IAttributeElement>().Count(a => a.Name == "Test")

if(method.Attributes.OfType<IAttributeElement>().Count(a => a.Name == "Test") > 0)
{
}

Ok, so now we have a list of methods, and we’re filtering them based on whether or not they are Test methods (defined by the existence of a Test attribute).  The next thing to do is look for an Assert statement within the text of our method.  This is another place where the Expressions Lab proves invaluable.  Looking at Expressions Lab we discover that our Assert statement is in fact an ElementReferenceExpression and is a child node of our Method object.  With this knowledge in hand we can use the FindElementByName method on our Method object to look for an Assert reference:

var assert = method.FindChildByName("Assert") as IElementReferenceExpression

if(assert == null)
{
    ea.AddIssue(CodeIssueType.CodeSmell,(SourceRange)method.NameRanges[0],"A Test Method should have at least one Assert");
}

With this in place we should now be able to run our project and try it out. Using F5 to debug a DxCore plug-in will launch a new instance of Visual Studio. From there if you create a new project, or open an existing project, and write a test method which does not have an Assert, you should see a red squiggle underneath the name of the method. Hovering over that with your mouse you'll see our Code Issue test presented. Adding an Assert will make the Code Issue disappear.

Well, things are looking good here, we’ve got code that is searching for an issue, and displaying the appropriate warning if our condition is met.  There is one other condition we should probably consider, however.  The one case I can think of when our rule does not apply is when we are expecting the code under test to throw an exception.  In that case there would be an ExpectedException attribute on the test class.  To make our users happy we should probably implement this functionality.

The good news is we already know how to accomplish this, since we are using the same technique to determine if the method we’re looking at is a test method.  All we need to do is change the test condition in our Count() method so it looks for “ExpectedException” instead of “Test”.  While we’re at it it seems like a reasonable thing to get an instance of the attribute and then check it for null, similar to how we’re handling the assert.  With all of this done the code should look like this:

var assert = method.FindChildByName("Assert") as IElementReferenceExpression;
var expectedException = method.Attributes.OfType<IAttributeElement>().FirstOrDefault(a => a.Name == "ExpectedException");
if (assert == null && expectedException == null)
{
    ea.AddIssue(CodeIssueType.CodeSmell, (SourceRange)method.NameRanges[0], "A Test Method should have at least one implicit or explicit Assertion");
}

private void cipTestShouldAssert_CheckCodeIssues(object sender, CheckCodeIssuesEventArgs ea)
{
    var resolveScope = ea.ResolveScope();
    foreach (IMethodElement method in resolveScope.GetElementEnumerator(ea.Scope, new ElementTypeFilter(LanguageElementType.Method)))
    {
        if (method.Attributes.OfType<IAttributeElement>().Count(a => a.Name == "Test") > 0)
        {
            var assert = method.FindChildByName("Assert") as IElementReferenceExpression;
            var expectedException = method.Attributes.OfType<IAttributeElement>().FirstOrDefault(a => a.Name == "ExpectedException");
            if (assert == null && expectedException == null)
            {
                ea.AddIssue(CodeIssueType.CodeSmell, (SourceRange)method.NameRanges[0], "A Test Method should have at least one implicit or explicit Assertion");
            }
        }
    }
}

And that's it. Granted there are some things here I would like to change before releasing this into the wild. We are specifically looking for NUnit/MbUnit style test method declarations for one, and we are also looking only for the short version of the attribute names, but this should give you a good idea of how things work.

If you are interested in seeing a more polished final version, you can either download the finished source for this post, or have a look at my CR_CreateTestMethod (admittedly poorly named) plug-in on the DxCore Community Plug-In's site.

The folks at TypeMock have released a new UnitTesting tool aimed specifically at catching deadlocks in multithreaded code called TypeMock Racer, and what's more they are offering free licenses to folks willing to review it during the 21 day free trial period.  As anyone who knows me can testify to, I am a whore for free-bees, so I decided to take them up on this.

For the impatient, here is the executive summary:
This is a good tool.  Period.  It is, however, also a very specific tool that is intended to help find very specific problems.  If you are not doing any multithreaded code, there is no need to have it in your toolbox.  The cost ($890 US) is high enough that it doesn't really seem worth it to get it "just in case" (unless, of course, you right a nice review on your blog and get a free license).  If you do work with multithreaded code, and you are concerned about deadlocks, this tool will save you lots of time, which ultimately means money.

Now, for the details

First of all, this is a tool from TypeMock, so I expected some pretty incredible things, even at the 1.0 release.  After all, their flagship product TypeMock Isolator is phenomenally powerful, so much so that the free license I was given for posting an advertisement for the release of their ASP.Net product has gone largely unused.  I'm just scared of it.  It's like having access to a 50 horse-power tablesaw with no blade guard.  I may be careful enough to use it correctly most of the time, but the fact that it can take off my forearm without pausing to ask my permission makes me reluctant to get too close.  I do know that there are some very real problems that the tool can solve that just can't be done with any other tool, so I have every intent of getting up my courage, grabbing a first aid kit, and jumping in to see what I can do...eventually.

But Racer is different.  It is a very powerful tool with a very specific purpose.  It makes it easier to run tests in multiple threads, and detect deadlocks.  As far as I can tell, it just detects deadlocks, not race conditions, as it's name seems to suggest.  Not that this is bad, just that it only covers half of the rather shaky ground that is traveled while working with multithreaded code.  It is arguable, however, that dealing with deadlocks is the more difficult of the two problems, so getting this one tacked first is not a bad thing.

So how does it work?

Fairly straight forward really.  Start with a regular unit test, that is exercising some code that is utilizing locks in an attempt to be thread safe.  It looks like Racer supports just about every form of Synchronization supported by .Net, so you can feel free to use whichever mechanism you are more comfortable with to get the work done.  Now that you have a test, the simplest way to make use of Racer, is to add the Typemock.Racer library reference to your test project, and then add a ParrellelInspection attribute to the test.  This causes Racer to do it's thing, which by default means running the test once with a single thread, and then run it again using multiple threads (2 by default).  If there are no problems, nothing much new happens.  You see some additional information about the test being run first with one thread, and then multiple, and something about scenarios executed...nothing that exciting.  The coolness happens when you actually get a deadlock.  For starters you test output includes a bunch of new information, the most noticeable of which is a big "DEADLOCK discovered when running the following" message.  Also is a description of the scenario that caused the deadlock.  Something like "Thread 1 acquired lock A, Thread 1 released lock A, Thread 2 attempted to acquire lock B, etc".  Cooler yet is a message that says "To reproduce or debug this, copy the following attribute, paste it on top of your test and rerun the test:", followed by something that looks like

[SpecificScenario("2", 1, 2, 1, 1, 1, 1, 1, 2, 1, 1, 1, 2)]

Which, while being completely incomprehensible to humans, causes your test to run in the specific configuration which caused the deadlock to occur.  This means that you can accurately recreate the exact situation that lead to the problem.  If you have ever had to try to track down a deadlock in a live system, you will realize that this information just saved you countless hours of trying to recreate the production environment, and lots of trial and error getting things into the state that caused the problem.  One word: Brilliant!

So what is the down side?

I have to say that I've not yet been able to figure out how Racer determines what scenarios to run, or what the bits in the "SpecificScenario" attribute mean(well, the first string parameter is the number of threads, and the other numbers refer to the specific threads, and match the summary of the scenario, but beyond that, not a clue).  It would be interesting to know these things, but not really critical, as long as you are confident all appropriate scenarios are being executed.

There is also an interesting feature that I can't quite get my head around.  When you run a test with a deadlock, an image is generated, which is supposed to be a visual representation of the scenario that created the deadlock.  Here is an example:

Now, I see the three objects I was locking against (sync1, sync2, and sync3), and I guess the odd rhombus shaped objects represent the threads, but I'm not really sure what the diagram is trying to tell me.  This is, no doubt, something which is still fairly raw in this early version.  I think it could be very useful if it were clearer what the shapes and arrows represent, but at this point it is simply a cleaver bit of kit that you can show somebody so that they can be confused too.

The last issue I can see with it currently is the price.  At $890 US for a single user license, it isn't an impulse buy.  Granted, I think it can pay for itself easily after finding a few deadlocks in some production code (the earlier they are found the more that find is worth), but I don't see it being a terribly easy sell for management, at least if you are not actively trying to correct a threading issue.  I feel pretty fortunate that I work for a company that understands the value of good tools, and is willing to provide them if there is a need, but I think it would take some convincing to get management to agree to purchase Racer simply because it is a good tool, and could really pay off "one day".  If we were facing a threading issue, and I could demonstrate that racer would allow us to find it, and accurately reproduce it, it would be a fairly easy sell. 

So overall

I think this is an excellent tool.  Based on the fact that this is an early release, I can only see it getting better over time.  It is a rather pricey tool, however, so you may have to do some convincing to get the boss-man to get you a license.  There is a 21-day trial, however, so if you find yourself in a situation where you either have, or you could conceivably have, a risk of deadlock, then grab the trial, and use the first detected deadlock as justification to get a full license. 

I recently decided to take the plunge and get CodeRush and Refactor! Pro (along with DxCore) loaded instead of Resharper.  Now, don't get me wrong, there is a lot I like about Resharper, but overall the performance was becoming an issue.  There were often problems with VS freezing for no particular reason, and then coming back as if nothing was wrong...I swear it was like my IDE had narcolepsy or something.

One of the things I noticed immediately about CodeRush was the fact that there was a single installer, and when I went to run it I was able to install it on all versions of Visual Studio, including the Orcas Beta.  This was nice when compared to Resharper's separate install for vs 2005 and vs 2003.  It also makes me feel good about improvements in the product being available for all versions of the IDE.  One thing that I noticed about R# was that there was some work being done in the VS 2005 version around performance, but that did not seem to trickle down to the VS 2003 version.  I think the big reason for this is the fact that CodeRush and Refactor! are implemented on top of DxCore, which provides a very clean abstraction from the scariness that is the Visual Studio integration layer.

Here are the things I really like about CodeRush/Refactor:

• The visualizations are stunning!  No, seriously this is some amazing stuff.  Circles, arrows, animations, eye candy yes, but useful eye candy.
• The Refactoring Live Preview is crazy brilliant.  Mark Miller mentioned this on a DNR episode, and I agree with his comment that the live preview allows you to discover new useful refactorings that may not be completely obvious from the name.
• Performance is great.  'Nuf said.
• It works with VB.Net.  Granted I don't use VB.Net, but some of my co-workers do, and occasionally I have to work on VB.Net projects.
• Dynamic Templates rock.  The fact that I can create a new mnemonic for my type, and then use predefined prefixes and do vee<space> to create a new Employee Entity for example is pure bliss.
• Template contexts are way cool.  By default they have NUnit templates defined, and with contexts, t<space> creates both a TestFixture class and a Test method
• Markers and navigation are dreamy.
• There are some crazy-cool template functions, like the ability to do a foreach within a template, so things like creating a switch/case statement for all items in an enum can be done easily.  This also powers a conditional to case refactoring that is pretty sweet.

Here are the things that I miss from Resharper:

• Automatically adding a using statement in VS 2003 was sweet.  VS 2005 can do it with the buit-in intellisense features, but I got very used to it.
• The VS 2003 test runner was very nice.  I use the Testdriven.Net plugin, which I cannot live without, but I like the graphical runner in the IDE.  The free test runner from JetBrains is for VS 2005 only, so it doesn't help those of us in VS 2003.  I do like the fact that they released it as a free tool, though.
• The "Extract Field" refactoring doesn't exist in Refactor!...This shocked me a lot.
• The Find Usages task.  This I think is part of the reason why R# was slow, but it did a brilliant job.  I think the rename in R# was more powerful as well.  I think there is a rename in Beta for Refactor! that is supposed to be able to work accross an entire project/solution, but I haven't had a chance to really test it yet.
• The pre-build error checking is nice.

The good news is that the DxCore extensibility model means that most if not all of these items could be recreated.  The bad news is that there isn't a lot of documentation around the extensibility model, particularly when it comes to creating new refactorings.  The test runner is one of the most painful points for me right now, so I've started exploring the process of creating one using the DxCore APIs.  It opens up the possibility of refining things too, which would be nice.  What I would really like would be the ability to detect and integrate with TestDriven.Net.

The folks over at Eleutian are evidently running both, which they claim is possible with some tweaking, but the performance issues for VS 2003 are the biggest downer on the R# side, so I will probably not go down that path.  I may load up the test runner for VS 2005, unless I get some time to try and build one using DxCore, in which case I'll share it with the rest of the world.