On November 27th, a beta release of the 9.3 version of the Developer Express components, including CodeRush and Refactor Pro! was made available to subscribers.  This release is pretty significant to me because it contains a major feature that I have been waiting for for a long time: A Unit Test Runner.  There were some teasers released by Mark Miller a while back, which only made me want to get my hands on the tool that much more.  My initial impressions are that it is very nice.  It is similar to TestDriven.Net in that it provides context menu options to run tests at various levels of granularity (single test, file, project, and solution level) and includes a debug option.  At this point it does not contain some of the additional coolness that TestDriven gives you like NCover/Team Coverage and TypeMock integration, but it does have the advantage of being extensible.  I know it was extensible because Mr. Miller told me it was extensible (the title “The Extensible Unit Test Runner You’ve Been Waiting For” was a clue).  I did not realize how extensible, however, until after I submitted a bug report to DevExpress.  The bug I was reporting (the NUnit TestCase attributes were not recognized), it turns out, was already brought to the attention of the DX team by way of a forum post, and they had already planned on correcting it with the next 9.3 release, but I could have saved myself (and Vito on DevExpress team) some time by taking a peek at the source samples bundled with the 9.3 release.  Yep, you guessed it, there with a shared source license were all of the test framework implementation projects.  So this meant I could whip together my own temporary fix while I was waiting for the next release.  It seemed like something that other folks might want to know about, so I thought I would share it here.

The biggest piece of the puzzle is a new TestExecuteTask class for handling the TestCaseAttribute.  Due to my complete lack of creativity, I called mine TestCaseExecuteTask, and it looks like this:

using System;
using System.Collections.Generic;
using System.Text;
using DevExpress.CodeRush.Core.Testing;
using System.Reflection;
using DevExpress.CodeRush.Core;

namespace CR_NUnitTesting
{
    public class TestCaseExecuteTask : TestExecuteTask
    {
        public override TaskExecuteResult CollectTestParameters()
        {
            TaskExecuteResult result = TaskExecuteResult.SkippedTaskResult;
            Attribute testCase = GetMethodAttribute("NUnit.Framework.TestCaseAttribute");
            if (testCase == null)
                return result;
            
            foreach(Attribute testCaseItem in TestMethod.GetCustomAttributes(true))
            {
                if(testCaseItem == null)
                    continue;
                var testCaseType = testCaseItem.GetType();
                if(testCaseType == null || testCaseType.FullName != "NUnit.Framework.TestCaseAttribute")
                    continue;
                PropertyInfo prop = testCaseType.GetProperty("Arguments");
                if(prop == null)
                    continue;
                foreach(MethodInfo getter in prop.GetAccessors())
                {
                    object[] parameters = getter.Invoke(testCaseItem, Type.EmptyTypes) as object[];
                    result.AddParameters(parameters);
                }
            }
        }
    }
}

This could be cleaned up some, and some of the magic strings extracted to constants, but overall it is pretty simple. Basically what is going on here is that we are looking for the TestCase attribute, and extracting the arguments for any attributes we find.  It just so happens that the TestExecuteTask base class has a CollectTestParameters() method we can override which allows for this sort of Row testing.  The parameters we extract get stashed in the execution result, which causes the test runner to execute the test once for each group of parameters (the result has a list of parameters, which gets populated with an array of objects for each TestCase attribute), and will correctly display which cases failed if there is a failure.

There are a couple other small changes that need to happen to get this to work.  There is an NUnitExtension.cs  class, which is the Plug-In class for the NUnit support, and it handles wiring everything up for us.  First off we need to initialize our new TestExecuteTask, and add it to the list of tasks that run for NUnit tests.  We do that in the InitializePlugin method of the NUnitExtension class:

public override void InitializePlugin()
{
    base.InitializePlugin();
    nUnitProvider.AvailableTasks.Add(new NUnitIgnoreTask());
    nUnitProvider.AvailableTasks.Add(new NUnitSetupTearDownTask());
    nUnitProvider.AvailableTasks.Add(new NUnitExpectedExceptionTask());
    nUnitProvider.AvailableTasks.Add(new NUnitValuesTask());
    nUnitProvider.AvailableTasks.Add(new NUnitRowTestTask());
    nUnitProvider.AvailableTasks.Add(new NUnitTimeoutTask());
    nUnitProvider.AvailableTasks.Add(new NUnitExplicitTask());
    nUnitProvider.AvailableTasks.Add(new NUnitTestCaseTask());
}

void nUnitProvider_CheckTestMethod(object sender, CheckTestMethodEventArgs ea)
{
    IMethodElement method = ea.Method;
    if(//method.Name != null && method.Name.StartsWith("Test")
       ea.GetAttribute("NUnit.Framework", "Test", method) != null
    || ea.GetAttribute("NUnit.Framework.Extensions", "RowTest", method) != null
    || ea.GetAttribute("NUnit.Framework", "TestCase", method) != null)
    {
        ea.IsTestMethod = true;
        ea.Description = ea.GetAttributeText("NUnit.Framework", "Description", method);
        ea.Category = ea.GetAttributeText("NUnit.Framework", "Category", method);
    }
}

The only change to the original code was the additional GetAttribute call at the end of the if statement (the comments were there when I got there, I swear).  Now the only thing left to do is to compile it and drop it in the plug-ins directory.  Now when you are looking at a test class, you should be able to run TestCase decorated test methods without problem.  Well, almost.  There is one thing I was not able to find a clean way to implement, and that is the Result property of the TestCase attribute.  This allows you to streamline tests which are doing equals assertions by having the test method return the actual result, and you specify the expected result by using the result property.  Unfortunately I could not find a way to hook into the actual execution of the test in such a way that I could have access to the specific test properties being used, and the result of the test method execution.  But considering the DevExpress folks will be fixing this issue, I’m sure when they release it there will be support for this feature.  After all, this is simply a stop-gap solution until the next CodeRush release is available, so I’m willing to live with this slight inconvenience.

Happy Testing!  

As of right about now, you should be able to mosey on over to the DxCore Community Plug-ins page, and grab a copy of CR_MoveFile.  This is a plug-in I created primarily as a tool to aid in working in a TDD environment, but which certainly has uses for non-TDD applications.  It does basically what the name suggests, it allows you to move a file from one directory in your solution/project structure to another, even one in a different project.  I implemented this as a code provider (since it could change the functionality if you move the file from one project to another), so it will appear in the Code menu when you have the cursor somewhere within the beginning blocks of a file (“using” sections, namespace declaration, or class/interface/struct declarations).  Once selected you are presented with a popup window which has a tree that represents your current solution structure, with your current directory highlighted.  You can use the arrow keys to navigate the directories and choose a new home for your file.

If you move files between projects, the plug-in will create project references for you, so you don’t need to worry about that.  When the file is moved the file contents remain unchanged, so all namespaces will be the same as they were originally.  I did this mostly to keep the plug-in simple, but also because I could see situations where this would be good, and situations where this would be bad, and it seemed like this was a bad choice to make for people.  I’ve been using this plug-in on a day-to-day basis for a while now, and things seem pretty clean, I did run into a small issue, however, using it within a solution that was under source control.  At this point you need to make sure the project files effected by the move are checked out, otherwise the plug-in goes through the motions, but doesn’t actually do anything, which is quite annoying.  There is also no checking going on to make sure the language is the same between the source and target project, so if you work on a solution that contains C# and VB.Net projects, you have to be careful not to move files around to projects that can’t understand what they are (oh, and the project icons used on the tree view are all the same, so there is no visual indication of what project contains what type of files).

That’s pretty much it.  Clean, simple, basic.  Used with other existing CodeRush/Refactor tools like “Move Type To File” and “Move to Namespace”, this provides for some pretty powerful code re-organization.  Just make sure you run all of your tests :).

I ran into this odd problem recently working with some Linq2SQL based persistence code.  There is some code someone put together to commit a list of changed entities to the database as part of a single transaction, which simply iterates through the list and performs the appropriate action.  The problem I was having was that I had an object referenced by another object that needed to be persisted first, otherwise there was a foreign key violation.  To add to the strangeness there seemed to be some magic going on (most likely utilizing the INotifyPropertyChanged goodness), so that even if I tried to persist just my dependent object first, both were still showing up in the list, and always in exactly the wrong order.  Now, I’m okay with magic.  Magic makes a lot of things a lot easier.  The problem arises whenever the magic is incomplete, and doesn’t follow through to take care of all of the operation.  Its like someone comming up to you and saying “Pick A Card”, at which point you do, and put the card back, and they say “I know what your card was” and walking away.  Not real convincing.  This is what was going on here.  There was the smarts to know that changes were being made to more than one entity, and there were even attributes to define what properties contained dependent objects, but no smarts to actually deal with a case when you would want to save more than one object in an object graph at a time.

So it occued to me I should be able to do some linqy magic and create some sort of iterator that would return dependent objects in the appropriate order, so the lest dependent of the objects get move to the beginning of the list.  My first step, since I wasn’t really sure how to do this, was to write a test.  And I made it more or less mirror the issue I was facing, a list of two items, one of which is a dependency of the other.  I don’t know if there is a lot of value in posting all of the test cases here, but the end result was rather nice.  Sure it took several iterations, and there was plenty of infinite looping and stack overflows (which does some fun things to studio when your running your tests with TestDriven.Net), but I think this is a reasonable solution to the problem:

public static IEnumerable<T> EnsureDependenciesFirst<T>(this IEnumerable<T> items, Func<T ,IEnumerable> selector)
{
    if(items.Count() < 2)
        return;
    var firstPass = items.SkipWhile(t => items.Intersect(selector(t)).Count() > 0);
    var remainingItems = items.Except(firstPass);
    if(items.Count() == remainingItems.Count())
        return remainingItems;
    return firstPass.Concat(remainingItems.EnsureDependenciesFirst(selector));
}

Ok, so what do we have here?  Well to start out I’m checking the item list to see if there are at least two items in it, if not I just return the list.  This provides a means to avoid an infinate loop due to the recursive call, and provides a shortcut for a scenario with only one item.  Next off I use the SkipWhile() method, combined with the user-supplied selector function to iterate through each item, retrieve it’s list of dependencies (which is what the selector function does), and checks to see if the current list contains any of the dependencies for the object.  The results of this first pass are the objects which have no dependencies at all, so therefore they need to be first in the list.  The next logical step is to run the operation again for a list that does not contains the items filtered out by the first pass.  This is done via a recursive call back to the EnsureDependenciesFirst extension.  You will notice we’re checking the count of the remaining items against the current list, and returning the list if they are the same.  This is another safety precaution for dealing with infinite loops.  If we have a circular dependency, this bit will just return the items that are interdependent.

You will note that this is a generic function that has really noting at all to do with the entities that I am dealing with.  This was largely due to the fact that this was built TDD, so I just used a simple class which had a property that could take another instance of itself.  To use this to overcome my entity committing problem, I would have to write a not too small function to retrieve the list of dependent objects from the entity (since there would need to be some reflection magic to look at attributes on the properties to determine which properties contain dependencies), but it pretty much will drop in to the foreach statement that is currently being used to persist the entities.

Incidently, I learned from my dev team what the “official” way of dealing with this is a “ReorderChanges” method, which takes two entities, in the order in which they should be persisted.  I think I like my solution better, mostly because it should mean I don’t have to worry about it again.

I was pleased to find recently that Roy Osherove’s Art of Unit Testing was available on Safari.  I have been following Roy’s blog for a while now, and was quite excited at the prospect of him writing a book on Unit Testing.  It was only my personal cheapness that kept me from shelling out the $25 to get the E-Book version from Manning ahead of time.  I have to say, now that I have read it, that it would have been well worth the money.  Before I get too deep I want to provide some context for what I am about to say.

I consider myself an experienced TDD practitioner and Unit Test Writer
So that means that I was reading this book hoping to gain some insight.  I wanted to find out how to write better, more readable, more maintainable tests.  I was also hoping for a little bit of “atta-boy” affirmation that the way I do things is the “right” way.  The astute reader may be able to tell that in order for my first hope to be true, the second may have to get some points taken away.  This was in fact the case, and to be honest, coming out of it I feel like I’ve gotten more value from the things I’ve learned than I received from whatever ego stroking may have occurred with what I am currently doing right.

So lets get started….
I was expecting the book to start out essentially as it did, some brief history about the author and an introduction to Unit Testing for those who may not be familiar with it.  I have to say I was expecting the book to be a little more TDD-centric than it was, but I think most of that was my own bias for TDD as “The Only Way To Write Software”.  Roy actually explained what TDD was, and also why he wasn’t going to harp too much on it throughout the book.  I have to say, I can see why he made the decision that he did.  I can also say that it seemed perfectly clear to me that TDD is a technique that he feels has a lot a value, which made me happy.  Since this is supposed to be a review from the perspective of an experienced practitioner of TDD and Unit Testing, I’m not going to go into anything that was touched on in the early chapters, apart from noting that they contained a general introduction to the tools, technique and philosophy of unit testing.  I can also say that, though I was already familiar with the material, I didn’t mind reading through it at all.  Overall, Roy’s writing style was light and quite pleasant, even for a technical book.

And now into the meat of the book…
For me, things started getting interesting in Part 3 of the book.  This is where issues of test design and organization are addressed.  This is one of those areas that I feel like I need some guidance on, mostly because I developed my testing idioms mostly through habit, and trial and error.  I look back on tests I have written in the past (which could be as little as two days ago) and I wonder how I could have come up with such a brittle, unmaintainable nightmare.  I feel like I need guidance from the experts on what I can do better when writing my tests.  Roy delivered on these items in chapter 7 “The pillars of good tests”.  One of the lessons I took away from this was the value in testing one concept per test.  I had heard this as “one assert per test” in the past, and scoffed at the idea.  But Roy presents a very compelling argument for why this is a good idea, if you are testing multiple concepts, you don’t know the extent of the problem when your test fails.  And lets face it, the failing test is the reason we’re doing this whole thing.  I’ve seen personally the failing test that just keeps failing.  You tackle the issue from one failed assert only to rebuild, and find one right after it which fails as well.  One of the issues I’ve had with this is the redundant setup and configuration that could be required for exercising this concept, but this issue is also addressed by the straight forward recommendation of creating clear and understandable configuration methods.  In the past I have generally not been really good about applying DRY to my test setup, which, I know, is another case of treating tests differently from regular code.  Having someone in a position of authority (like Roy) say, “put your setup in separate methods so you can re-use them and make your tests more readable” made it okay to do the thing that I knew I should be doing anyway.  The key concepts covered are making tests readable, maintainable, and an accurate portrayal of the authors intent.

Even more in depth….
Section 4 goes even further and talks about how to integrate unit testing into an organization which is not already doing it.  This is an interesting subject to me as I have recently moved to a company which has not been doing unit testing and TDD as part of their regular development process.  Roy draws on his experiences as a consultant to provide some really good advice for how to go about enacting this sort of change in an organization.  I particularly pleased with his candor when he describes his failed attempts at integrating unit testing.  It would have been quite easy to simply say “Based on my considerable expertise, these are the things you need to do”, but he chooses instead to share some real-world experience in a straight forward way that only adds to my respect for him as a professional.  In addition to this, he touches on techniques for integrating testing into “legacy” code (i.e. code which is not tested).  He does a good job at introducing some techniques for testing what is essentially untestable code, which a very large nod at Michael Feathers’ “Working Effectively with Legacy Code”.

The book ends with three appendices, one discussing the importance of testability in the design process, one listing vairous testing tools (both Java and .Net), and the last listing guidelines for conducting test reviews.  This last one is nice, because it presents a concise view of all of the guidelines presented throughout the book, and provides page references where you can get the “why” behind each. 

All in all…
This is a really good book, which should be part of any agile development library.  It doesn’t matter if you are writing your first unit tests, or you’re a seasoned pro, there is going to be something here for you.  I think it is great that Roy has chosen to share his experience with the developer community in this way.  I came into this book with some rather high expectations and I think they were met.

A note on TypeMock….
I remember seeing some criticism floating around on twitter suggesting the book was rather pro TypeMock.  There was also the comment that Roy’s affiliation with TypeMock was not made clear early on.  I can’t say I saw either of these things when I was reading it.  For starters, I already knew Roy worked for TypeMock, so perhaps that skewed my ability to objectively judge if the disclosure was done in a timely manner or not.  I can say that the places in the book which there seemed to be a preference for TypeMock were places where he stated things like “I feel TypeMock has a better syntax in this case”, or “TypeMock is the only tool with provides these capabilities”.  For starters, the first is a statement of preference.  Sure Roy helped design the API for TypeMock, so it seems only natural that he would prefer it to other frameworks, but having used it I would have to agree with the statement.  It is a great API, and example if a fluent interface done well.  The second comment is also plain fact.  Of the mocking libraries available in the .Net space, TypeMock is the only one that allows you to swap instances of objects in place, without making changes to the classes using them.  You can argue over whether or not this is a good or a bad thing, but the fact remains that it is a feature specific to TypeMock.  Maybe I was expecting something more blatant and obvious, but I just didn’t see it.

The folks at TypeMock have released a new UnitTesting tool aimed specifically at catching deadlocks in multithreaded code called TypeMock Racer, and what's more they are offering free licenses to folks willing to review it during the 21 day free trial period.  As anyone who knows me can testify to, I am a whore for free-bees, so I decided to take them up on this.

For the impatient, here is the executive summary:
This is a good tool.  Period.  It is, however, also a very specific tool that is intended to help find very specific problems.  If you are not doing any multithreaded code, there is no need to have it in your toolbox.  The cost ($890 US) is high enough that it doesn't really seem worth it to get it "just in case" (unless, of course, you right a nice review on your blog and get a free license).  If you do work with multithreaded code, and you are concerned about deadlocks, this tool will save you lots of time, which ultimately means money.

Now, for the details

First of all, this is a tool from TypeMock, so I expected some pretty incredible things, even at the 1.0 release.  After all, their flagship product TypeMock Isolator is phenomenally powerful, so much so that the free license I was given for posting an advertisement for the release of their ASP.Net product has gone largely unused.  I'm just scared of it.  It's like having access to a 50 horse-power tablesaw with no blade guard.  I may be careful enough to use it correctly most of the time, but the fact that it can take off my forearm without pausing to ask my permission makes me reluctant to get too close.  I do know that there are some very real problems that the tool can solve that just can't be done with any other tool, so I have every intent of getting up my courage, grabbing a first aid kit, and jumping in to see what I can do...eventually.

But Racer is different.  It is a very powerful tool with a very specific purpose.  It makes it easier to run tests in multiple threads, and detect deadlocks.  As far as I can tell, it just detects deadlocks, not race conditions, as it's name seems to suggest.  Not that this is bad, just that it only covers half of the rather shaky ground that is traveled while working with multithreaded code.  It is arguable, however, that dealing with deadlocks is the more difficult of the two problems, so getting this one tacked first is not a bad thing.

So how does it work?

Fairly straight forward really.  Start with a regular unit test, that is exercising some code that is utilizing locks in an attempt to be thread safe.  It looks like Racer supports just about every form of Synchronization supported by .Net, so you can feel free to use whichever mechanism you are more comfortable with to get the work done.  Now that you have a test, the simplest way to make use of Racer, is to add the Typemock.Racer library reference to your test project, and then add a ParrellelInspection attribute to the test.  This causes Racer to do it's thing, which by default means running the test once with a single thread, and then run it again using multiple threads (2 by default).  If there are no problems, nothing much new happens.  You see some additional information about the test being run first with one thread, and then multiple, and something about scenarios executed...nothing that exciting.  The coolness happens when you actually get a deadlock.  For starters you test output includes a bunch of new information, the most noticeable of which is a big "DEADLOCK discovered when running the following" message.  Also is a description of the scenario that caused the deadlock.  Something like "Thread 1 acquired lock A, Thread 1 released lock A, Thread 2 attempted to acquire lock B, etc".  Cooler yet is a message that says "To reproduce or debug this, copy the following attribute, paste it on top of your test and rerun the test:", followed by something that looks like

[SpecificScenario("2", 1, 2, 1, 1, 1, 1, 1, 2, 1, 1, 1, 2)]

Which, while being completely incomprehensible to humans, causes your test to run in the specific configuration which caused the deadlock to occur.  This means that you can accurately recreate the exact situation that lead to the problem.  If you have ever had to try to track down a deadlock in a live system, you will realize that this information just saved you countless hours of trying to recreate the production environment, and lots of trial and error getting things into the state that caused the problem.  One word: Brilliant!

So what is the down side?

I have to say that I've not yet been able to figure out how Racer determines what scenarios to run, or what the bits in the "SpecificScenario" attribute mean(well, the first string parameter is the number of threads, and the other numbers refer to the specific threads, and match the summary of the scenario, but beyond that, not a clue).  It would be interesting to know these things, but not really critical, as long as you are confident all appropriate scenarios are being executed.

There is also an interesting feature that I can't quite get my head around.  When you run a test with a deadlock, an image is generated, which is supposed to be a visual representation of the scenario that created the deadlock.  Here is an example:

Now, I see the three objects I was locking against (sync1, sync2, and sync3), and I guess the odd rhombus shaped objects represent the threads, but I'm not really sure what the diagram is trying to tell me.  This is, no doubt, something which is still fairly raw in this early version.  I think it could be very useful if it were clearer what the shapes and arrows represent, but at this point it is simply a cleaver bit of kit that you can show somebody so that they can be confused too.

The last issue I can see with it currently is the price.  At $890 US for a single user license, it isn't an impulse buy.  Granted, I think it can pay for itself easily after finding a few deadlocks in some production code (the earlier they are found the more that find is worth), but I don't see it being a terribly easy sell for management, at least if you are not actively trying to correct a threading issue.  I feel pretty fortunate that I work for a company that understands the value of good tools, and is willing to provide them if there is a need, but I think it would take some convincing to get management to agree to purchase Racer simply because it is a good tool, and could really pay off "one day".  If we were facing a threading issue, and I could demonstrate that racer would allow us to find it, and accurately reproduce it, it would be a fairly easy sell. 

So overall

I think this is an excellent tool.  Based on the fact that this is an early release, I can only see it getting better over time.  It is a rather pricey tool, however, so you may have to do some convincing to get the boss-man to get you a license.  There is a 21-day trial, however, so if you find yourself in a situation where you either have, or you could conceivably have, a risk of deadlock, then grab the trial, and use the first detected deadlock as justification to get a full license. 

So I had this analogy pop into my head a while back and I've been sitting on it because, quite frankly, I was almost embarrassed to have thought of it.  I finally decided that I might as well post it since no one is going to read this anyway, so here goes:

Writing software without TDD is like having unprotected sex.  Its extremely irresponsible in this day and age when everyone is supposed to know better, but it still happens a lot.  Sure, it feels better, and let's face it in the heat of the moment you really don't want to take the time to make sure you've taken all of the right precautions...you might loose your edge.  The problem is if we don't do it we have that lingering fear in the back of our minds, that "What if?" that keeps popping up at the worst moments.  Most of the time its okay, and everything is fine, but it only takes one time for things to go wrong to make you regret your decisions.

So come on folks, we all know what we should be doing, so no more making excuses.  After a while you get used to it, and you find that the peace of mind it gives you far outweighs the momentary delights of living on the wild side.

So if you recall from some of my earlier posts, I've talked about the concept of the "Friend" class in C++ and how it could apply to TDD within .Net.  Well, today, with the help of Roy Osherove, I just stumbled upon the InternalsVisibleToAttribute within .Net 2.0.  This allows you to specify within one assembly, another assembly that should have access to the internal members of your assembly.  This is genius, and goes a long way towards allowing you to keep your code encapsulated, while still being testable.  If we could just get them to go one step farther, and allow for access to private and protected members as well, life would be good, and there would be no more of this OOD vs TOOD junk.

The other interesting thing about this is that it could allow you to give a separate utility...say the Castle Microkernel, access to internal class constructors, and thus enforce creation of your objects through the Kernel from outside assemblies.  This is actually a feature I am desperately wanting in my current project,  but sadly, I am limited to .Net 1.1, so I can't quite get there.

Here is a quick look at how this works.  Here is a very unrealistic class in an assembly that I want to test:

class TestClass
{
    internal TestClass()
    {}

    private bool PrivateMethod()
    {
        return false;
    }

    internal bool SomeMethod()
    {
        return true;
    }

    public string PublicMethod()
    {
        return "You can see me";
    }
}

[assembly: InternalsVisibleTo("TestAssembly")]

And here is what Intellisense looks like in my test class

The project I'm working on now has a huge need for auto-update.  Strangely enough, there aren't a whole lot of documented solutions for an auto-update application for .Net 1.1.  In the 2.0 world you have ClickOnce, which handles those sorts of things for you (and in a way that isn't terribly difficult to manage as a developer...as long as you pay attention to what your doing), but the only real option you get from MS on this is the AutoUpdater Application Block from the Patterns and Practices guys.  I took a look at this when it was in it's 1.0 version a while back, and really didn't care for it much.  The big reason was that it required you set up an AppStart.exe file, which would take a look at your config file to determine which directory to start the app from.  When updates arrived, they were put in new directories based on versions.  This seemed like a lot of effort, and a lot of overhead.  The good news is that there is now a 2.0 version of the application block, and it looks like it is much more configurable, and has the ability to do inproc updates.

So here is my issue.  I want to change the default downloader (used to retrieve the updated files) from the BITSDownloader that ships with the block to one which will allow me to copy files from a UNC path.  Fortunately, a sample implementation of such a thing exists in the documentation, so there is a starting point.  It is pretty rough around the edges, and isn't testable, so I am working creating a nicer, testable version of the sample downloader.  Here is the problem, though; the dependencies are insane!  And it doesn't look like even mock objects will be able to help.  Initially I found this a little strange since I know that the Patterns and Practices group was headed up by some folks who were heavy into the Agile methodologies.  I also know for a fact that the Enterprise Library components have test included.  So I pulled up the AutoUpdater Application Block to see how they were testing things....and what do you think I found?  They were not testing anything!  I can only assume that the updater block came from another group, because there was no tests in sight.  So, since I had the source code, I'm reduced to making modifications to the block to support testing.  For the most part this involved marking public methods/properties as virtual so that Rhino.Mocks can mock them.  I also added some parameters to the constructor of the UpdateTask class so that I could supply mock versions of some of the dependencies.

I can't imagine trying to test this without the source code to modify...as it stands I don't feel real good about shipping around a non-standard version of the library.  Overall I'm quite disappointed in the P&P folks on this one.  I had high hopes whenever I saw the rest of the Enterprise Library that I would be able to test my extensions fairly easily.  I guess that goes to show that even in organizations (or specifically groups) where TDD is a best practice, it still is finding hurdles to acceptance.

Thinking about my earlier post discussing the OOP vs TOOP problem, I mentioned at the end that the best solution to this problem in my mind would be integrated language support for test classes.  Specifically, a way to let the Compiler/Runtime know that a specific class is a test class, and should therefore be able to access any and every property of a class.

It occurred to me that such blatant intrusion into the privacy of a class is not unknown in the programming world.  C++ has the notion of a "Friend" class.  This is a class that can access all members of another class regardless of their protection level.  To keep things civil, so that just any class can't declare itself to be a Friend of any class it wants, the class that the Friend class would be accessing would declare specifically that classes X, Y and Z are fiends, and so can have free reign.  Granted this is considered to be rather scary, and one of those features that makes C++ an ideal tool for shooting ones own foot off.

But, this concept has some merit within the context of Test classes.  In the .Net world we could potentially use attributes on a class to identify what the test class (or classes I suppose) for a class are.  The compiler and runtime could then use that information to provide unlimited access only to the classes listed in the test class list.  For more protection perhaps it would also validate that the classes in the list also have the appropriate attribute (TestFixture in NUnit, TestClass in TFS) before allowing access.  You would need some additional tooling in the development environment so that you would get Intellisense on all of the private/protected members, but that should be easy for the MS guys after building in the test support, right?

There is some additional danger in this approach since there is some potential to modify IL at runtime, but couldn't there be additional protections around such things?  As someone with no knowledge of the internal workings of the CLR, I can't say for sure, but its worth trying.

So, I know officially propose to Microsoft that this feature be added to the 4.0 version of the framework.

Do you think they heard me?

Here is something I've been kicking around in my head for a while, and thought I would put it down in more or less a "permanent" format so maybe I'll do something about it sometime...

Back when I was first trying to get my head around TDD, one of the things that I found most clarifying was an idea I first saw in Test Driven Development in Microsoft .Net (Microsoft Press).  The idea is that you write tests based on your requirements.  So as a developer, you should have hopefully been given a list of requirements by someone for the project you are working on.  When you go to build the software you start looking at the requirements list, and the pick on (usually a simple one) to start implementing.  Once you have your requirement you start brainstorming tests that can be implemented to fulfill that requirement.  Once you can no longer think of tests for a requirement, you move on to the next one.  After that once you have run out of requirements, then your done writing the software.

This struck me not only as a good way to approach the problem of how do I come up with tests, but also as a potentially powerful tool for tracking progress within a project.  I can imagine something not unlike the NUnit Test Runner that, instead of listing all of the tests, lists all of the requirements.  If there are tests written for a requirement, then you could potentially expand the requirement to see the list of tests.  When you execute the tests, you can then see which requirements are complete (more on this in a minute), and also which ones are functional.  The assumption being, if a test fails, then there is a functional problem with the requirements.

To support this a few modifications to the standard TDD toolset may be needed.  First a place to put the list of requirements in code would be useful.  This could be an XML document with the test descriptions and IDs, or maybe even a plain text document, or a piece of code.  I actually like the idea of the XML document because potentially you could build an InfoPath form or something that a BA could use to put together the requirement list, and then hand the XML doc over to the development team.  From there an additional attribute would be added to the NUnit tests, so that a specific test could be associated with a specific requirement.  If you wanted to be able to give a "percentage implemented" on a requirement, then you would probably want to do something like create all of the test methods you can come up with during a brainstorming session, and put an Ignore attribute on them.  At that point it should be feasible to see how many of the methods are implemented, and how many are still being Ignored, and then grab a percentage.  This number would change as time progressed and the development team came up with additional tests for specific requirements, but it would probably stabilize by the time serious work started into a specific requirement.

So that's it...my brilliant idea.  I actually think there wouldn't be a whole lot of work involved in getting it going either...A new attribute, an updated Unit Test Runner, and some XML work.  It would be nice to be able to define the requirements in a way that would let you use Intellisense to select the requirements while in VS.Net....maybe code-generate a struct or enum based on the XML doc?

I have officially crossed over....I am now using Mock Objects in my tests and loving it.  After much humming and hawing, and trying to figure out how to write truly effective tests, I decided to give it a go, and so grabbed a copy of Rhino Mocks and started the grueling task of converting some data access code so that I no longer needed a database to run the tests.  It took a little bit to get my mind around the new way of thinking, but I have to say it worked great.  I'm now able to test my data access routines with complete success, and about a 95% code coverage rate.  This is all on top of the fact that I'm using Enterprise Library (for 1.1) for data access and exception handling.

Now, there is a very interesting article from Martin Fowler which is called simply "Mocks aren't Stubs".  In this article he is discussing two approaches for dealing with the sort of problem that I had with my data access code.  One is to create object stubs within your tests that you can use to feed known values back to your tests.  This is not at all a bad approach, and I am using it in some cases in conjunction with my mock objects.  The other approach is to use Mock Objects for all of the tricky stuff.  He observed an interesting dichotomy between the sort of TDD folks who follow each approach:  The folks that use stubs tend to be very results oriented in their testing...That is they are verifying the values of various object after performing an action using the test frameworks Assert calls.  This, I think, is most in line with the way TDD is supposed to work.  For the Mock Object folks, though, another thing starts to happen.  These test gain the ability to verify that specific methods are called in a specific order, and therefore the tests that come from this approach start becoming less results-oriented, and rely more on the method verification from the Mocking framework.  He points out these two styles of testing without really saying one is better than the other (he does indicate that he tends to be in the Stub camp, but I think that is a personal choice and level of comfort thing), beyond pointing out that relying on the results of the test tends to be more in line with the pure TDD way of doing things.

Now, I had read this article before I started using mocks within my project.  I thought about using stubs throughout, but since I was testing data access code, I would have a lot of methods to implement in my stub objects that wouldn't really be part of what I was testing, so it seemed a large waste of my time.  As I started implementing mocks in the tests, I paid attention to how I was testing to see if I was becoming a "Mockist" in the Fowler sense.  Overall I would say that I am not, currently, a "Mockist".  I'm still overwhelmingly validating the results of calls based on the methods I'm testing, and just using the mocks to feed appropriate values into the classes I'm trying to test.  As a matter of fact, I could probably remove the call to _mocks.VerifyAll() at the end of my tests, since I don't really care.  I can say, though, that the ability to verify what is being called with what arguments has been extremely useful in my data access testing space.  I've basically got objects which are making calls to stored procs on Oracle to do my data work. The DAL is a real simple deal that doesn't have much in the way of smarts, but works well enough for a small project.  Because of the fact that I'm making stored proc calls to the database, I can now use the mocks to make sure that when I call the Save() method on my User object (for example), It's calling ExecuteNonQuery() with the proc name of SAVE_USER.  I can also verify that all of the input an output parameters are being set correctly, which is a big consideration when adding a property to the data objects.  In that way I can know that my Save() method is doing what it is supposed to even though it returns no value.  This checking could probably be done by stubbing the EnterpriseLibrary Database and DBCommand objects, and then setting values within my stub object, which I could verify using an Assert call, but that seems like an awful lot of work when I have the option to simply set expectations, and then do a VerifyAll().

I think the Fowler article may be making much of a practice which is not that common among people using mocks in TDD.  Or maybe there is a slow creep that happens to the developer after months and years of using mocks in their tests that turn them into Mockists.  Personally I find that there is a very broad and comfortable middle ground which allows me more flexibility to express my intent quickly and concisely.  And after all, expressing intent is one of the major boons of TDD.

powered by performancing firefox

powered by performancing firefox