Bugfree.dk – Ronnie Holm's blog

My notes from a talk on Windsor by Mogens Heller Grabe which I attended this week. Slides and code are available through Dropbox, but I don’t know for how long.

• Use an Inversion of Control container (IoC) like Windsor to create an architecture that responds well to change, i.e., an architecture that promotes looser coupling and higher cohesion
• The ability to write unit tests against a code base is a good measure of its degree of coupling
• By having components depend on interfaces, you’re free to switch the implementation at runtime, introducing flexibility into the software
• Avoid having concrete components talk to each other. It makes it hard for the two to vary independently, e.g., adding logging later can only be done by tearing components apart and putting them back together
• At runtime the IoC container recursively composes these smaller components into an object graph, e.g., by passing concrete implementations through the constructors to satisfy the dependencies
• One alternative to using an IoC container is using a service locator. Each component’s constructor would then ask the locator for its dependent components. While this approach works, it makes testing components in isolation difficult because all dependencies are now locked away inside the service locator. Instead, by having all dependencies supplied in the constructor, unit tests can directly supply fakes implementations
• When software is only used in one environment it tends to be fairly inflexible because it’s only suited for that one purpose. As soon as you design it to be used in two or more environments it becomes more flexible. Thing of different environments as unit test, staging, production, and so on
• The idea is for components to not have to be modified depending on the where they’re running. It’s the IoC container that dynamically tie components together based on the environment
• Suppose you didn’t use a IoC container. Then ultimately the top-level object would have to pass concrete instances down the chain. The top-level object may well be the UI layer. But having the UI layer know about data access components, service components, and logging components isn’t ideal. Instead use an IoC container, which is nothing more than a factory for components
• With the Windsor IoC container (and most others), the usage pattern typically involves three stages
  • Register: tie together interfaces with concrete implementations
  • Resolve: return concrete implementations of interfaces required to satisfy the dependencies of an object
  • Release: dispose of concrete implementations
• Most people only use a fraction of the functionality of an IoC container. Instead of taking dependency of a full-blown container, you could easily roll your own IoC container that implements the core functionality in a few dozen lines of code — in an associated .NET Rocks and DnrTV episode James Kovacs elaborates on his original article
• The goal of using a container isn’t to get rid of calls to the new operator. It’s to not use the new operator for the parts of an applications where flexibility is required
• Simple forms of Aspect Oriented Programming are possible using interceptors
• Windsor supports the decorator pattern so you can have one component wrap another at runtime. This lets you implement features such as logging without actually modifying the original component. It’s an example of adhering to the open/close principle by which classes should be open for extension, but closed for modification. In other words: don’t edit the original source code to introduce new behavior. Instead use methods of composition
• Avoid configuring Windsor through XML and instead use the fluent interface — perhaps in a separate assembly so only it needs to be redeployed when the configuration changes. When a value, such as a connection string needs to be configurable, create a section in app.config and add the value there. Windsor will know how to locate the value when instantiating objects

My notes from a talk on NServiceBus by Mogens Heller Grabe which I attended this week. Slides and code are available through Dropbox, but I don’t know for how long.

• Enterprise service bus
  • An architectural pattern rather than a specific technology or product
  • The ethernet of SOA
• Types of service bus
  • Centralized: a broker more than a bus, e.g., BizTalk
  • Decentralized: a real service bus, e.g., NServiceBus, Mass Transit, Rhino Service Bus, MS .NET Service bus
• The current version 2.0 of NServiceBus is free whereas the binary version 2.5 will contain limitations. The limitations can be disabled by downloading and compiling the source code yourself
• NServiceBus is designed to address the fallacies of distributed computing
  • It primarily addresses the fallacies through the use of reliable one-way messaging
  • Reliable means messages are stored in a queue, such a MSMQ, which is part of all recent Windows installations. The assumption being that the queue is always present and ready to receive messages
  • One-way means messages are asynchronous and fire-and-forget. The destination service doesn’t have to be online when a message goes into its queue, which makes services temporally independent. At some point in the future the service may post a reply to the client’s queue
  • Messaging means that any action is accomplished by posting a self-contained message, containing the type of operation and its parameters, to the queue of a service
  • With NServiceBus, a class is marked with an interface to describe that it’s a type of message. Instances of the class are then serialized using a standard XML or binary serializer and stored in a queue
• NServiceBus service != traditional web service
  • An NServiceBus service is a class that implements an interface
  • Using a message queue is more reliable when the service has to make calls to other services as part of its operation. With a traditional web service, calling other services from within makes the service slower and less reliable. If only one dependent service is down it’s like with old Christmas lights wired in series: the entire stack may be down. Queues on the other hand introduce a save place, a stabilization point, to temporarily store messages
  • A service is typically configured with an input queue that identifies it to others and an error queue that messages are routed to after some number of retries. In addition, each service may be configured with a number of threads to use to parallelize message processing

My notes from a talk on Windbg by Brian Rasmussen which I attended this week. The talk was recorded and parts one and two are available though Channel 9.

• Windbg isn’t a replacement for VS, but VS doesn’t handle some advanced cases
• Windbg is a free user mode/kernel mode debugger which is part of the Debugger Tools for Windows
• Customers may not be happy installing VS to debug code in production since it installs a lot of components and requires restarts
  • Windbg requires only a simple installation once you extract the redistributable from the Debugger Tools for Windows
• Loading SOS.dll from the Microsoft .NET folder into Windbg makes it understand .NET
  • With SOS.dll loaded, you can look into the CLR and its data structures
  • Make sure to load the right SOS.dll for your runtime
  • SOS.dll is also available for the Silverlight runtime
• Debugger Markup Language support is available for version 4 of SOS.dll
  • Provides hyperlinks in the command output of SOS
• Like with the VS debugger, you can insert Debug.Break() in your demo app and have Windbg halt on it
  • Release builds can be debugged with Windbg
  • Release builds also contain symbols. Like with debug builds, symbols are stored in a separate file
  • Release builds make debugging harder because the jitter kicks in and modifies the code
  • The 64 bit calling convention of passing arguments via registers makes it harder to locate information when debugging
• Windbg generally needs symbols loaded, although it’s less important when debugging managed code
  • Use MS’ public symbol server to load symbols on demand
  • Set the _NT_SYMBOL_PATH environment variable to point to your symbols (will affect VS as well)
  • Or use the .symfix command from within Windbg
• Popular extensions to Windbg: SOSEX and Psscor2 (replacement for SOS, useful for ASP.NET debugging)
• Create dump file to analyze: use task manager or ADPlus or ProcDump from Sysinternals, which can dump based on triggers
• ADPlus collects crash dumps or hang dumps
  • Hang dumps can be captured from the same process multiple times and may be useful when debugging deadlocks or resource leaks
  • When you capture a hang dump, the process is halted for the dump period and is then restarted
• A *32 process in task manager is actually a 64 bit process when you dump it
  • WoW64.dll is involved when dumping from the task manager
  • Not what you typically want because you don’t get full access to 32 bit process information
• A .NET application is hosted within the CLR which is itself hosted within a regular Windows process
  • Looking at memory usage with the task manager therefore doesn’t tell you much about the .NET part

Developer automation is a subject that I’ve always felt passionate about. Unit testing may be the most common example, but other tasks may include check-out of source code, build, deploy, setup, and warm-up of an application. I may even want one system rule them all and have the same automation drive continuous integration. Whatever the use, to fully reap the benefits of automation, a developer should master at least one automation tool the same way he masters other developer tools. This and later posts capture my experiences with a few such automation tools centered around Windows and .NET, starting with why the ubiquitous batch file is best avoided and how to characterize better solutions in terms of it.

Although Visual Studio, in tandem with the MSBuild engine, generally takes good care of compilation, I rarely want to rely on it solely. Instead, I’d prefer that any developer task be easily run from the command-line. This ensures that no magic is going on, that the task is kept simple and flexibility, and that it doesn’t rely on Visual Studio to work. The challenge, however, is which of the many tools available to pick. It must be one that’s flexible enough to meet most requirements with relative ease or the automation will likely not be a valid documentation medium in itself.

Why not to use Windows batch files

As a general example, consider the deployment of a SharePoint 2007 solution. With SharePoint a good deal more than compiling is needed to bring new functionality into a SharePoint installation. Whereas Visual Studio and MSBuild may still compile the code and WSPBuilder create the WSP installation package, both from within Visual Studio and from the command-line, getting everything setup cries for automation, even locally. A common approach (possibly inspired by popular SharePoint 2007 literature) is that of the batch file applying various operations to a WSP. For the sake of brevity, I’ve kept the example short, but imagine extending it with a check-out source code task, a compilation task, a WSPBuilder task, and a feature deactivation and activation task, and possibly a warm-up task. Add to this a master script that orchestrate the whole thing. In the end, I’d end up with scripts that become increasingly harder to maintain as they grow in number and size.

    @set STSADM="c:\program files\common files\microsoft shared\web server extensions\12\bin\stsadm"

    if "%1" == "uninstall" goto uninstall
    if "%1" == "install" goto install
    if "%1" == "" goto reinstall

    :uninstall
        %STSADM% -o retractsolution -name Foo.wsp -immediate
        %STSADM% -o execadmsvcjobs
        %STSADM% -o deletesolution -name Foo.wsp -override
        goto end

    :install
        %STSADM% -o addsolution -filename Foo.wsp
        %STSADM% -o deploysolution -name Foo.wsp -immediate -allowGacDeployment -force
        %STSADM% -o execadmsvcjobs
        goto end

    :reinstall
        call Foo uninstall
        call Foo install
        goto end

    :end

Still, because of the ubiquitous nature of command.com and cmd.exe, the batch file interpreters, batch files are everywhere. Regardless that the technology is a left-over from the days of MS DOS and haven’t evolved much since. Not only are the branching and looping constructs limited, so are the available commands. Suppose I want to find out if a command was indeed successful. This turns out to be really hard when all I have to work with is the errorlevel of the most recently executed command. Assuming the command adheres to the errorlevel convention, for the script to fail as early and as close to the real error as possible, I’d have to inspect the property after each command, causing the batch file to grow quite verbose. Sadly, batch files lack the equivalent of set -o errexit of Bash, where the interpreter checks for success after each command and aborts immediately on error. Relying solely on the errorlevel is oftentimes insufficient anyway. To determine success, I’d typically have to parse actual command output or inspect some system property by downloading additional commands or building my own.

Essential vs. incidental requirements

Unless I plan to sit idle and stare at the screen and be a human error detector while the batch file run, I think it’s safe to say that it’s mostly unsuitable for developer automation. Hence in late 2005 I rolled my own automation tool in Python. With Python or Ruby or a similar dynamic language acting as the glue that ties everything together, almost any task can be automated in a robust way. Of course, I could also automate with a static language like C# (it’s surprisingly common). But for script-like tasks, I don’t particularly fancy the long cycle of editing source code in Visual Studio, compiling it, deploying it, and having a hard time debugging it in environment without Visual Studio. A dynamic language, on the other hand, short-circuits the development cycle and allows for interactive programming through a REPL.

With a dynamic language that interacts with .NET, such as IronPython, IronRuby, or Powershell, possibly with supporting DSLs like Paver, Rake, or psake on top, the need for writing custom commands to interact with the operating system or the application almost vanishes. The question, then, is which of the dynamic languages to go with when at their technical core they’re so much alike. Besides sharing the concept of a REPL, the notion of a tuple, a list, a map, and operations on each are baked into their syntax, making code quite terse. It even makes it convenient to express any configuration in the language itself and not in XML where I’d first have to come up with a schema, and then create an instance of it before parsing it. On Windows, however, Powershell is gradually becoming the next ubiquitous interpreter, with applications shipping with cmdlets, whereas IronPython or IronRuby is a separate download.

Conclusion

No matter the tool, what I end up doing is defining tasks, form dependencies between tasks, and have the tool execute tasks in an order that satisfies their dependencies. As the tool traverses the dependency graph and executes tasks, it’s up to each task to detect success, and up to the tool to report on progress. A good tool is therefore characterized by the ease with which I can express these things, the available language constructs, the ease of debugging, and the tool’s ability to converse in foreign languages.

Recent months have brought about a proliferation of mocking frameworks that mocks what more traditional framework like Rhino Mocks cannot. Instead of creating and loading a mock implementation at runtime, the new breed of mocking frameworks hooks into the CLR to intercept and redirect calls. This opens up virtually every aspect of a class to mocking, which is useful for testing code not written with explicit testability in mind. Until recently, TypeMock was the only mocking framework around that took the latter approach, but it’s now being challenged by Moles from Microsoft Research and JustMock from Telerik.

Why traditional dependency-breaking techniques come short

After watching a screencast on how to use Moles to unit test LINQ to SQL without hitting the database, I thought it would be interesting to do the same with TypeMock. But first, let’s make sure we understand why traditional dependency-breaking techniques come short in testing LINQ to SQL. Assuming we want to put a repository under test, our goal is to mock how it accesses the database. Here’s a simple implementation of a repository that queries the Employee table of the AdventureWorks database:

    public class EmployeeRepository {
        public List<Employee> GetEmployeesByHireDate(DateTime start, DateTime end) {
            using (var ctx = new AdventureWorksDataContext())
                return (from e in ctx.Employees
                        where e.HireDate >= start && e.HireDate <= end
                        select e).ToList();
        }
    }

All calls to the database are routed through the AdventureWorksDataContext generated by Visual Studio. To mock access to the database, we therefore have to mock part of the data context. Easier said than done, though, for the context doesn’t expose an interface that a fake can implement. In addition, the tables are accessed through properties on the context that return a type of Table<TEntity>. Unfortunately, the constructor of Table<TEntity> is internal and the class itself is sealed, eliminating the hope of instantiating or subclassing the type by traditional means:

    public sealed class Table<TEntity> : IQueryProvider,
            ITable, IListSource, ITable<TEntity>, IQueryable<TEntity>,
            IEnumerable<TEntity>, IQueryable, IEnumerable
            where TEntity : class {
        internal Table(DataContext context, MetaTable metaTable) {
            ...
        }
    }

For an example of how the data context itself creates an instance of Table<TEntity>, take a look at the Employees property on the AdventureWorksDataContext. It relies on the GetTable<Employee> method on the DataContext class to create an instance of Table<Employee>. Despite its constructors being internal, the GetTable<TEntity> method has no trouble constructing an instance of the Table<TEntity> type, as they both reside in the System.Data.Linq assembly:

    public partial class AdventureWorksDataContext : DataContext {
        public Table<Employee> Employees {
            get {
                return GetTable<Employee>();
            }
        }
    }

How to break the unbreakable

The design of LINQ to SQL leaves us short of a traditional testing seam, as Michael Feathers would phrase it; a place at which we can alter the behavior of a program without editing in that place. This explains why, with LINQ to SQL, traditionally we’ve had to test against a real database with all its constraints, making our tests brittle, slow, and painful to write and maintain. With the new breed of mocking frameworks the issues of not being able to subclass or not being able to call an internal constructor go away (and new issues take their place). Regardless, here’s how to write a unit test for the CustomerRepository that doesn’t hit the database:

    [TestClass]
    public class CustomerRepositoryTest {
        private EmployeeRepository _repository;

        [TestInitialize]
        public void Initialize() {
            _repository = new EmployeeRepository();

            var fakeEmployees = new List<Employee> {
                new Employee {EmployeeID = 1, HireDate = new DateTime(2004, 12, 1)},
                new Employee {EmployeeID = 2, HireDate = new DateTime(2006, 7, 1)},
                new Employee {EmployeeID = 3, HireDate = new DateTime(2009, 3, 1)}
            }.AsQueryable();

            var fakeDataContext = Isolate.Fake.Instance<AdventureWorksDataContext>();
            Isolate.Swap.NextInstance<AdventureWorksDataContext>().With(fakeDataContext);

            // var fakeEmployeeTable = Isolate.Fake.Instance<Table<Employee>>();
            // Isolate.WhenCalled(() => fakeDataContext.Employees).WillReturn(fakeEmployeeTable);
            // Isolate.WhenCalled(() => fakeEmployeeTable).WillReturnCollectionValuesOf(fakeEmployees);
            // or by transitivity
            Isolate.WhenCalled(() => fakeDataContext.Employees).WillReturnCollectionValuesOf(fakeEmployees);
        }

        [TestMethod]
        public void GetEmployeesByHireDate_should_return_hires_from_2008_until_present() {
            var employees = _repository.GetEmployeesByHireDate(new DateTime(2008, 1, 1), DateTime.Now);
            Assert.AreEqual(1, employees.Count());
            Assert.AreEqual(3, employees[0].EmployeeID);
        }
    }

The test method itself looks exactly as if we’d been testing against a real database. The difference lies in the Initialize method, where we setup the fake data context and database contents. We instruct TypeMock to return the fake context in place of the real one inside EmployeeRepository. And whenever someone calls the Employees property on the fake context, we have TypeMock intercept the call and return a fake collection of type IQueryable<Employee>. We could’ve returned an instance of Table<Employee>, which implements IQueryable<Employee>, but in this case returning the collection is simpler and sufficient. Had we had more methods on our repository, we likely would’ve added additional rows to the Employee table and populated more of its columns.

It’s been four months and 350 clocked work-hours since I started using Org-mode for Emacs. I use Org-mode, mixed with some of the Pomodoro principles, to maintain my task lists at work and home.

In a nutshell Org-mode is a plug-in for Emacs that makes taking notes, maintaining lists, clocking time, and many other tasks more straightforward. The mode works by adding features on top of Emacs for editing plain text files. Features that enrich the interaction with the underlying text buffer by parsing and adding (meta-)information to it.

With Org-mode you group items into a hierarchical structure. A path to the item to work on is then formed by unfolding items in a way analogous to how you navigate the file system with Explorer.

To efficiently work with items, Org-mode supports moving items around the hierarchy. Items may also be tagged (buy, work, phone, etc.) or associated with a state (done, waiting, started, etc.). Additional meta-information may include a deadline, an estimate, or time spent working on a task. The clocking of time is accomplished by Org-mode adding a timestamp to the item. On completion, another timestamp and the elapsed time is calculated and added.

Below is an example of an expanded and a collapsed item containing clocked times and estimates.

With a few keystrokes, Org-mode is able to correlate estimates with time spent on an item-by-item basis. This truly short-circuits the estimation feedback cycle. Imagine doing this on a file containing a large hierarchy of items and the transparency it brings.

If you want to keep track of billable hours, it may make sense to clock all your time using Org-mode and generate timesheets from the clocking information. Especially since the billing context is automatically provided by the item storing the timestamp, and since you’re free to organize the file in any way you see fit. Few other tools manage to compete with the flexibility of a plain text buffer.

This post only scratches the surface of Org-mode. To get a sense of the possibilities, the manual amounts to 184 pages. However, for a quick introduction I encourage you to watch the Google Tech Talk on Org-mode.

Spending a great deal of time in front of my computer, I often get carried away by instant messaging, mail, RSS, Twitter, Facebook, etc. For the most part, these activities aren’t the reason I turned on the computer. Nevertheless they end up being where a lot of time is spent.

Like with any habit, every so often you should take the time to review it. I find TimeSnapper (the free edition) to be an invaluable tool for reviewing computer habits. It captures an image of the desktop every few seconds and is able to create a movie from the images. That way, a partial life log (mine is currently 393 days) is automatically created to help you remember what you did yesterday and the day before.

There’s nothing like reviewing a few of weeks of captured images to make habits stand out. I found that I’ve a tendency to re-visit the same maybe ten sites all too often. I also have difficulty single-tasking. Listening to a podcast or watching a movie, I’m often browsing the web instead of paying attention to the podcast or skipping it altogether. I may even have several media players launched at once, although only one playing.

Most of my distractions can be attributed to me always being online. I need Internet access for work, so turning it off altogether isn’t an alternative. So, not unlike Paul Graham, on several occasions, I’ve made myself adhere to a set of rules for using the Internet. And every time they’ve worked — for a short while.

In Disconnecting Distraction, Paul takes these rules one step further and argues that using two computers may be a better solution: one computer with Internet access and one without for doing real work. Ideally, the computers should be placed some distance apart to make you psychically get up to go on the Internet.

For a week I’ve given the two-computer approach a try. It was hard initially, but I’ve come to appreciate it. Forming permanent habits takes time. Thus, I’ll continue the experiment for a couple of weeks. Perhaps in time I’ll even learn to turn off instant messaging and only do mail once or twice a day. Or I may end up spending most of the time in front of the connected computer.

Then again, there’s more to life than work.

I’ve been doing some LINQ development lately. What I typically do is write a LINQ query within the production code and then attach a debugger to see what the query returns and perhaps what the underlying SQL statement looks like. For simple queries this technique is sufficient, but for more complex ones, several, cumbersome round-trips through the debugger are often necessary to get the query right.

Researching the issue I came across LINQPad (see LINQPad in action). The tool is a generic testbed for molding expressions or statements (LINQ being one such class) before embedding them into code. Also, because LINQPad is written by one of the authors of C# in a Nutshell, the tool comes with a lot of samples from the book.

On the implementation side, LINQPad works by emitting MSIL behind the scenes, conceptually similar to the DataContext classes generated by Visual Studio when using the database design surface.

As an interesting side note, LINQPad can also show the query after it has been transformed into lambdas, anonymous types, and the like:

   Users
      .Join (
         Macs,
         u => u.UserId,
         m => m.UserId,
         (u, m) =>
            new
            {
               u = u,
               m = m
            }
      )
      .Where (temp0 => (temp0.u.Created < DateTime.Now.AddHours (-24)))
      .Select (
         temp0 =>
            new
            {
               UserId = temp0.u.UserId,
               UserCreated = temp0.u.Created,
               Mac = temp0.m.Mac,
               Active = temp0.m.Active
            }
      )
      .Take (15)

Keep in mind how the .Net 3.5 framework and the .Net 3.5 compiler play together to ultimately transform the LINQ query into MSIL. What the framework does is transform the LINQ query into the above expression by applying a transform to the abstract syntax tree of the LINQ query. The .Net 3.5 compiler then transforms the lambdas, the anonymous types, and the like into .Net 2.0 MSIL (there’s no such thing as a .Net 3.5 CLR. For an overview of how the various versions of the framework, the compiler, and the CLR relate to each other, see Scott Hanselman’s blog entry).

But back to LINQPad. The only downside that I’ve encountered is that there’s no IntelliSense support in the current 1.24 release. However, even without IntelliSense support LINQPad is still a very valuable tool.

Download E107InactiveUserPurger-1.0.zip.

As part of my use of the e107 open source contents management system, I want to purge the user accounts that’ve not logged in recently. This is something not supported out of the box, but by going through MySQL and accessing the e107_user table directly, the date of last login is accessible to anyone.

I’ve therefore written a small command-line tool in C# that makes use of the e107_user information to sent inactive users a warning email that their accounts are about to be purged (or whatever you want to happen).

The tool may be run in one of two modes, determined by a command-line switch.

• GenerateWarningMails: emails users that have not logged in during the last WarningPeriod days.
• GeneratePurgeMailForAdministrator: emails site administrator a list of users that have not logged in during the last PurgePeriod days.

Before running E107InactiveUserPurger, make sure to go to the Debug directory and adjust the settings in E107InactiveUserPurger.exe.xml to match your environment. You might also want to adjust the WarningMailTemplate.txt, which serves as a template for generating the body of warning emails.

Note that within the WarningMailTemplate.txt file, you’re free to add placeholders for fields within the e107_user table. Thus, if you want to include the login name of a user within the warning email, you’d write $user_loginname$.

Now you might run E107InactiveUserPurger as a scheduled task. Keep in mind, though, that the tool doesn’t actually purge the user accounts. Therefore, running it too often with the GenerateWarningMails switch may result in users receiving more than one warning email.

I just learned about Google Alerts.

It’s a simple service where you enter keywords for Google to search. Then Google will monitor the search results and send you an email whenever new hits appear in the search result.

Time will tell if it’s truly useful to have your own personal intelligence service at your hands. So far I receive alerts about the company I work for, the region where I live, myself, and so on.

Update, May 29: In my experience there are too many ads at the top of each email I receive. Also, Google sends me a daily email for each alert even though there are no new search results.

Thus, bulking alerts would be a nice option. Or perhaps you could write something better by coding against the Google APIs.