Although Visual Studio, in tandem with the MSBuild engine, generally takes good care of compilation, I rarely want to rely on it solely. Instead, I’d prefer that any developer task be easily run from the command-line. This ensures that no magic is going on, that the task is kept simple and flexibility, and that it doesn’t rely on Visual Studio to work. The challenge, however, is which of the many tools available to pick. It must be one that’s flexible enough to meet most requirements with relative ease or the automation will likely not be a valid documentation medium in itself.

Why not to use Windows batch files

As a general example, consider the deployment of a SharePoint 2007 solution. With SharePoint a good deal more than compiling is needed to bring new functionality into a SharePoint installation. Whereas Visual Studio and MSBuild may still compile the code and WSPBuilder create the WSP installation package, both from within Visual Studio and from the command-line, getting everything setup cries for automation, even locally. A common approach (possibly inspired by popular SharePoint 2007 literature) is that of the batch file applying various operations to a WSP. For the sake of brevity, I’ve kept the example short, but imagine extending it with a check-out source code task, a compilation task, a WSPBuilder task, and a feature deactivation and activation task, and possibly a warm-up task. Add to this a master script that orchestrate the whole thing. In the end, I’d end up with scripts that become increasingly harder to maintain as they grow in number and size.

    @set STSADM="c:\program files\common files\microsoft shared\web server extensions\12\bin\stsadm"

    if "%1" == "uninstall" goto uninstall
    if "%1" == "install" goto install
    if "%1" == "" goto reinstall

    :uninstall
        %STSADM% -o retractsolution -name Foo.wsp -immediate
        %STSADM% -o execadmsvcjobs
        %STSADM% -o deletesolution -name Foo.wsp -override
        goto end

    :install
        %STSADM% -o addsolution -filename Foo.wsp
        %STSADM% -o deploysolution -name Foo.wsp -immediate -allowGacDeployment -force
        %STSADM% -o execadmsvcjobs
        goto end

    :reinstall
        call Foo uninstall
        call Foo install
        goto end

    :end

Still, because of the ubiquitous nature of command.com and cmd.exe, the batch file interpreters, batch files are everywhere. Regardless that the technology is a left-over from the days of MS DOS and haven’t evolved much since. Not only are the branching and looping constructs limited, so are the available commands. Suppose I want to find out if a command was indeed successful. This turns out to be really hard when all I have to work with is the errorlevel of the most recently executed command. Assuming the command adheres to the errorlevel convention, for the script to fail as early and as close to the real error as possible, I’d have to inspect the property after each command, causing the batch file to grow quite verbose. Sadly, batch files lack the equivalent of set -o errexit of Bash, where the interpreter checks for success after each command and aborts immediately on error. Relying solely on the errorlevel is oftentimes insufficient anyway. To determine success, I’d typically have to parse actual command output or inspect some system property by downloading additional commands or building my own.

Essential vs. incidental requirements

Unless I plan to sit idle and stare at the screen and be a human error detector while the batch file run, I think it’s safe to say that it’s mostly unsuitable for developer automation. Hence in late 2005 I rolled my own automation tool in Python. With Python or Ruby or a similar dynamic language acting as the glue that ties everything together, almost any task can be automated in a robust way. Of course, I could also automate with a static language like C# (it’s surprisingly common). But for script-like tasks, I don’t particularly fancy the long cycle of editing source code in Visual Studio, compiling it, deploying it, and having a hard time debugging it in environment without Visual Studio. A dynamic language, on the other hand, short-circuits the development cycle and allows for interactive programming through a REPL.

With a dynamic language that interacts with .NET, such as IronPython, IronRuby, or Powershell, possibly with supporting DSLs like Paver, Rake, or psake on top, the need for writing custom commands to interact with the operating system or the application almost vanishes. The question, then, is which of the dynamic languages to go with when at their technical core they’re so much alike. Besides sharing the concept of a REPL, the notion of a tuple, a list, a map, and operations on each are baked into their syntax, making code quite terse. It even makes it convenient to express any configuration in the language itself and not in XML where I’d first have to come up with a schema, and then create an instance of it before parsing it. On Windows, however, Powershell is gradually becoming the next ubiquitous interpreter, with applications shipping with cmdlets, whereas IronPython or IronRuby is a separate download.

Conclusion

No matter the tool, what I end up doing is defining tasks, form dependencies between tasks, and have the tool execute tasks in an order that satisfies their dependencies. As the tool traverses the dependency graph and executes tasks, it’s up to each task to detect success, and up to the tool to report on progress. A good tool is therefore characterized by the ease with which I can express these things, the available language constructs, the ease of debugging, and the tool’s ability to converse in foreign languages.

In a nutshell Org-mode is a plug-in for Emacs that makes taking notes, maintaining lists, clocking time, and many other tasks more straightforward. The mode works by adding features on top of Emacs for editing plain text files. Features that enrich the interaction with the underlying text buffer by parsing and adding (meta-)information to it.

With Org-mode you group items into a hierarchical structure. A path to the item to work on is then formed by unfolding items in a way analogous to how you navigate the file system with Explorer.

To efficiently work with items, Org-mode supports moving items around the hierarchy. Items may also be tagged (buy, work, phone, etc.) or associated with a state (done, waiting, started, etc.). Additional meta-information may include a deadline, an estimate, or time spent working on a task. The clocking of time is accomplished by Org-mode adding a timestamp to the item. On completion, another timestamp and the elapsed time is calculated and added.

Below is an example of an expanded and a collapsed item containing clocked times and estimates.

With a few keystrokes, Org-mode is able to correlate estimates with time spent on an item-by-item basis. This truly short-circuits the estimation feedback cycle. Imagine doing this on a file containing a large hierarchy of items and the transparency it brings.

If you want to keep track of billable hours, it may make sense to clock all your time using Org-mode and generate timesheets from the clocking information. Especially since the billing context is automatically provided by the item storing the timestamp, and since you’re free to organize the file in any way you see fit. Few other tools manage to compete with the flexibility of a plain text buffer.

This post only scratches the surface of Org-mode. To get a sense of the possibilities, the manual amounts to 184 pages. However, for a quick introduction I encourage you to watch the Google Tech Talk on Org-mode.

Like with any habit, every so often you should take the time to review it. I find TimeSnapper (the free edition) to be an invaluable tool for reviewing computer habits. It captures an image of the desktop every few seconds and is able to create a movie from the images. That way, a partial life log (mine is currently 393 days) is automatically created to help you remember what you did yesterday and the day before.

There’s nothing like reviewing a few of weeks of captured images to make habits stand out. I found that I’ve a tendency to re-visit the same maybe ten sites all too often. I also have difficulty single-tasking. Listening to a podcast or watching a movie, I’m often browsing the web instead of paying attention to the podcast or skipping it altogether. I may even have several media players launched at once, although only one playing.

Most of my distractions can be attributed to me always being online. I need Internet access for work, so turning it off altogether isn’t an alternative. So, not unlike Paul Graham, on several occasions, I’ve made myself adhere to a set of rules for using the Internet. And every time they’ve worked — for a short while.

In Disconnecting Distraction, Paul takes these rules one step further and argues that using two computers may be a better solution: one computer with Internet access and one without for doing real work. Ideally, the computers should be placed some distance apart to make you psychically get up to go on the Internet.

For a week I’ve given the two-computer approach a try. It was hard initially, but I’ve come to appreciate it. Forming permanent habits takes time. Thus, I’ll continue the experiment for a couple of weeks. Perhaps in time I’ll even learn to turn off instant messaging and only do mail once or twice a day. Or I may end up spending most of the time in front of the connected computer.

Then again, there’s more to life than work.

Among the good wiki engines out there (most notably MediaWiki, MoinMoin, and FlexWiki), I went with FlexWiki, because it being asp.net based means that all the software required for a basic configuration ships with Windows. In addition, FlexWiki’s provider-based storage model lets you store pages in an MS SQL server, if you want to.

My incentive for going the wiki way was that I expected my documents to become more structured when making use of the typical wiki-like features …

• Documents are automatically linked together.
  I found my pages to be fairly independent, so hyperlinks are almost never present.
• All modifications are recoded.
  Versioning isn’t really that big an issue, because I usually add contents to the end of a page rather than changing existing contents, and for pages such as a shopping list, I don’t really need versioning.
• More sophisticating layout capabilities.
  Although I occasionally created a table and used colors, generally I can do with simple formatting. For those few cases where I can’t, a Word document is in place, since it’s equally well indexed by MS or Google desktop search engines.

… but most of the time, I didn’t make use of the features FlexWiki provided over text files. So, if you also factor in the disadvantages …

• Extra infrastructure required.
  Web server, and possibly an SQL server, is a must-have-installed. Also, you probably need to figure out how to configure the web server so you don’t share your live with the world.
• Browser required for page edit.
  After launching the browser, you must navigate to the wiki page, which may add considerable overhead for quick edits as opposed to simply double clicking a text file from within Explorer.
• Lacking WYSIWYG support.
  With FlexWiki, editing a page is a two-step process. First you write content in a markup language. Then you click “Save” and the browser switches to display view, where your markup is converted to html. Now, if the result is not what you expected, you have to repeat the cycle.
• Build-in search less powerful than desktop search engines.
  If you use the file system for storing wiki pages then the pages are also searchable by desktop search engines. However, if you opt for MS SQL storage, you have to rely on the build-in search capabilities of the wiki to find pages.
• Backup is more involved.
  Backing up and restoring a database is more complicated than copying a directory.

… I’ve decided that the wiki doesn’t cut it for me: simply because there is less overhead associated with editing text or even Word documents and because with desktop search engines I retain most of the properties I need.

This doesn’t mean that a wikis aren’t suited for maintaining a corpus of personal information; just that the corpus should serve a different purpose to make better use of wiki features.