Bugfree.dk – Ronnie Holm's blog

It’s been four months and 350 clocked work-hours since I started using Org-mode for Emacs. I use Org-mode, mixed with some of the Pomodoro principles, to maintain my task lists at work and home.

In a nutshell Org-mode is a plug-in for Emacs that makes taking notes, maintaining lists, clocking time, and many other tasks more straightforward. The mode works by adding features on top of Emacs for editing plain text files. Features that enrich the interaction with the underlying text buffer by parsing and adding (meta-)information to it.

With Org-mode you group items into a hierarchical structure. A path to the item to work on is then formed by unfolding items in a way analogous to how you navigate the file system with Explorer.

To efficiently work with items, Org-mode supports moving items around the hierarchy. Items may also be tagged (buy, work, phone, etc.) or associated with a state (done, waiting, started, etc.). Additional meta-information may include a deadline, an estimate, or time spent working on a task. The clocking of time is accomplished by Org-mode adding a timestamp to the item. On completion, another timestamp and the elapsed time is calculated and added.

Below is an example of an expanded and a collapsed item containing clocked times and estimates.

With a few keystrokes, Org-mode is able to correlate estimates with time spent on an item-by-item basis. This truly short-circuits the estimation feedback cycle. Imagine doing this on a file containing a large hierarchy of items and the transparency it brings.

If you want to keep track of billable hours, it may make sense to clock all your time using Org-mode and generate timesheets from the clocking information. Especially since the billing context is automatically provided by the item storing the timestamp, and since you’re free to organize the file in any way you see fit. Few other tools manage to compete with the flexibility of a plain text buffer.

This post only scratches the surface of Org-mode. To get a sense of the possibilities, the manual amounts to 184 pages. However, for a quick introduction I encourage you to watch the Google Tech Talk on Org-mode.

Spending a great deal of time in front of my computer, I often get carried away by instant messaging, mail, RSS, Twitter, Facebook, etc. For the most part, these activities aren’t the reason I turned on the computer. Nevertheless they end up being where a lot of time is spent.

Like with any habit, every so often you should take the time to review it. I find TimeSnapper (the free edition) to be an invaluable tool for reviewing computer habits. It captures an image of the desktop every few seconds and is able to create a movie from the images. That way, a partial life log (mine is currently 393 days) is automatically created to help you remember what you did yesterday and the day before.

There’s nothing like reviewing a few of weeks of captured images to make habits stand out. I found that I’ve a tendency to re-visit the same maybe ten sites all too often. I also have difficulty single-tasking. Listening to a podcast or watching a movie, I’m often browsing the web instead of paying attention to the podcast or skipping it altogether. I may even have several media players launched at once, although only one playing.

Most of my distractions can be attributed to me always being online. I need Internet access for work, so turning it off altogether isn’t an alternative. So, not unlike Paul Graham, on several occasions, I’ve made myself adhere to a set of rules for using the Internet. And every time they’ve worked — for a short while.

In Disconnecting Distraction, Paul takes these rules one step further and argues that using two computers may be a better solution: one computer with Internet access and one without for doing real work. Ideally, the computers should be placed some distance apart to make you psychically get up to go on the Internet.

For a week I’ve given the two-computer approach a try. It was hard initially, but I’ve come to appreciate it. Forming permanent habits takes time. Thus, I’ll continue the experiment for a couple of weeks. Perhaps in time I’ll even learn to turn off instant messaging and only do mail once or twice a day. Or I may end up spending most of the time in front of the connected computer.

Then again, there’s more to life than work.

I’ve been doing some LINQ development lately. What I typically do is write a LINQ query within the production code and then attach a debugger to see what the query returns and perhaps what the underlying SQL statement looks like. For simple queries this technique is sufficient, but for more complex ones, several, cumbersome round-trips through the debugger are often necessary to get the query right.

Researching the issue I came across LINQPad (see LINQPad in action). The tool is a generic testbed for molding expressions or statements (LINQ being one such class) before embedding them into code. Also, because LINQPad is written by one of the authors of C# in a Nutshell, the tool comes with a lot of samples from the book.

On the implementation side, LINQPad works by emitting MSIL behind the scenes, conceptually similar to the DataContext classes generated by Visual Studio when using the database design surface.

As an interesting side note, LINQPad can also show the query after it has been transformed into lambdas, anonymous types, and the like:

   Users
      .Join (
         Macs,
         u => u.UserId,
         m => m.UserId,
         (u, m) =>
            new
            {
               u = u,
               m = m
            }
      )
      .Where (temp0 => (temp0.u.Created < DateTime.Now.AddHours (-24)))
      .Select (
         temp0 =>
            new
            {
               UserId = temp0.u.UserId,
               UserCreated = temp0.u.Created,
               Mac = temp0.m.Mac,
               Active = temp0.m.Active
            }
      )
      .Take (15)

Keep in mind how the .Net 3.5 framework and the .Net 3.5 compiler play together to ultimately transform the LINQ query into MSIL. What the framework does is transform the LINQ query into the above expression by applying a transform to the abstract syntax tree of the LINQ query. The .Net 3.5 compiler then transforms the lambdas, the anonymous types, and the like into .Net 2.0 MSIL (there’s no such thing as a .Net 3.5 CLR. For an overview of how the various versions of the framework, the compiler, and the CLR relate to each other, see Scott Hanselman’s blog entry).

But back to LINQPad. The only downside that I’ve encountered is that there’s no IntelliSense support in the current 1.24 release. However, even without IntelliSense support LINQPad is still a very valuable tool.

Download E107InactiveUserPurger-1.0.zip.

As part of my use of the e107 open source contents management system, I want to purge the user accounts that’ve not logged in recently. This is something not supported out of the box, but by going through MySQL and accessing the e107_user table directly, the date of last login is accessible to anyone.

I’ve therefore written a small command-line tool in C# that makes use of the e107_user information to sent inactive users a warning email that their accounts are about to be purged (or whatever you want to happen).

The tool may be run in one of two modes, determined by a command-line switch.

• GenerateWarningMails: emails users that have not logged in during the last WarningPeriod days.
• GeneratePurgeMailForAdministrator: emails site administrator a list of users that have not logged in during the last PurgePeriod days.

Before running E107InactiveUserPurger, make sure to go to the Debug directory and adjust the settings in E107InactiveUserPurger.exe.xml to match your environment. You might also want to adjust the WarningMailTemplate.txt, which serves as a template for generating the body of warning emails.

Note that within the WarningMailTemplate.txt file, you’re free to add placeholders for fields within the e107_user table. Thus, if you want to include the login name of a user within the warning email, you’d write $user_loginname$.

Now you might run E107InactiveUserPurger as a scheduled task. Keep in mind, though, that the tool doesn’t actually purge the user accounts. Therefore, running it too often with the GenerateWarningMails switch may result in users receiving more than one warning email.

I just learned about Google Alerts.

It’s a simple service where you enter keywords for Google to search. Then Google will monitor the search results and send you an email whenever new hits appear in the search result.

Time will tell if it’s truly useful to have your own personal intelligence service at your hands. So far I receive alerts about the company I work for, the region where I live, myself, and so on.

Update, May 29: In my experience there are too many ads at the top of each email I receive. Also, Google sends me a daily email for each alert even though there are no new search results.

Thus, bulking alerts would be a nice option. Or perhaps you could write something better by coding against the Google APIs.

On Linux based system, it’s a common idiom to impersonate the root user whenever you have to perform a privileged operation. This is typically accomplished using the su command, which, although used less frequently, has a Windows equivalent, called runat.exe.

runat.exe turned out to be quite useful when I had to do some work with MS SQL 2005 Server, Analysis Service. The thing with Analysis Service is that it’s Windows authentication only, whereas MS SQL 2005 Server supports mixed mode authentication.

My only option for getting in touch with Analysis Service was using SQL Server Management Studio by impersonating an existing Windows user, who is allowed access to Analysis Service (I didn’t have the option of creating a new account on the machine running Analysis Service and the machine wasn’t allowed to trust the domain my desktop computer was on (it’s a tough world out there)).

So, using runat.exe, I ran the command below (on one line, from a shortcut), in effect causing the display of a prompt asking for the password for my_impersonated_user. Upon entering the password sqlwb.exe starts up as if it was run from the local my_impersonated_user account.

C:\\Windows\\System32\\runas.exe /user:my_impersonated_user
"c:\\Program Files\\Microsoft SQL Server\\90\\Tools\\
Binn\\VSShell\\Common7\\IDE\\sqlwb.exe"

For the command to succeed my_impersonated_user have to be a user on the local machine with a username and password identical to the user that is allowed access to MS SQL 2005 Server/Analysis service.

Now Windows lets you logon to Analysis Service even if your local machine is on a different domain than the MS SQL 2005 Server.

I have quite a few small text files lying around, containing everything from to-do lists to shopping lists to code snippets to small howto’s. Now, instead of having all those text files floating around as regular files, nine months ago I decided to move them into a locally stored, for security reasons, personal wiki.

Among the good wiki engines out there (most notably MediaWiki, MoinMoin, and FlexWiki), I went with FlexWiki, because it being asp.net based means that all the software required for a basic configuration ships with Windows. In addition, FlexWiki’s provider-based storage model lets you store pages in an MS SQL server, if you want to.

My incentive for going the wiki way was that I expected my documents to become more structured when making use of the typical wiki-like features …

• Documents are automatically linked together.
  I found my pages to be fairly independent, so hyperlinks are almost never present.
• All modifications are recoded.
  Versioning isn’t really that big an issue, because I usually add contents to the end of a page rather than changing existing contents, and for pages such as a shopping list, I don’t really need versioning.
• More sophisticating layout capabilities.
  Although I occasionally created a table and used colors, generally I can do with simple formatting. For those few cases where I can’t, a Word document is in place, since it’s equally well indexed by MS or Google desktop search engines.

… but most of the time, I didn’t make use of the features FlexWiki provided over text files. So, if you also factor in the disadvantages …

• Extra infrastructure required.
  Web server, and possibly an SQL server, is a must-have-installed. Also, you probably need to figure out how to configure the web server so you don’t share your live with the world.
• Browser required for page edit.
  After launching the browser, you must navigate to the wiki page, which may add considerable overhead for quick edits as opposed to simply double clicking a text file from within Explorer.
• Lacking WYSIWYG support.
  With FlexWiki, editing a page is a two-step process. First you write content in a markup language. Then you click “Save” and the browser switches to display view, where your markup is converted to html. Now, if the result is not what you expected, you have to repeat the cycle.
• Build-in search less powerful than desktop search engines.
  If you use the file system for storing wiki pages then the pages are also searchable by desktop search engines. However, if you opt for MS SQL storage, you have to rely on the build-in search capabilities of the wiki to find pages.
• Backup is more involved.
  Backing up and restoring a database is more complicated than copying a directory.

… I’ve decided that the wiki doesn’t cut it for me: simply because there is less overhead associated with editing text or even Word documents and because with desktop search engines I retain most of the properties I need.

This doesn’t mean that a wikis aren’t suited for maintaining a corpus of personal information; just that the corpus should serve a different purpose to make better use of wiki features.

(I’ve created a Download page for the tool described in this post, with a short description of prerequisites and how to install it.)

For team development a way is needed to ensure that team members doesn’t inadvertently break the shared code, e.g., a developer forgets to commit all files related to a change, which is something not easily fixable by other developers.

Also, the team might want to run post-compile tasks, such as unit tests, on a dedicated machine to avoid the works-on-my-machine syndrome.

Ideally, the project’s code should always go from one stable state to another so that a developer doesn’t get broken code the next time he synchronizes with source control.

One way to achieve this is by means of continuous integration of changes into the code. Either by rolling your own build script that monitors source control for changes to trigger a build and inform developers of the result or by leveraging an existing continuous integration solution, such as CruiseControl.net (ccnet): a continuous integration server, an asp.net based front-end, and a desktop client.

Although CruiseControl (both the Java and .Net incarnation) appears to be a popular choice among developers, I find it somewhat immature: it violates the dry principle in its projects configuration file and, worst of all, it doesn’t support build queues to ensure that only one of several projects are actually building at once.

It’s not a problem for ccnet itself to perform parallel continuous integration. However, the build chain is no stronger than the weakest link, which in my case is devenv.exe, the Visual Studio executable used by ccnet to build the application (by running devenv.exe from the command line with a couple of arguments, you can leverage the build system within Visual Studio from a script).

The downside to using devenv.exe, however, is that all running instances share a temporary build folder for compiler output. Thus, when ccnet does continuous integration of branches in parallel, the instances of devenv.exe spawned by ccnet compete for write access to identically named files, namely the executables generated by the compiler.

Not surprisingly, this results in build failures for one or more ccnet projects, although there is nothing wrong with the code.

In my experience from the trenches this happens too often to be ignored for any serious use of ccnet: It’s just too common for a developer to cross-commit a bugfix to several branches or for independent developers to commit to their working branches in between when ccnet checks the branch for updates.

Having these false negatives make the team members not trust ccnet and not take proper action whenever they see a build failure, thus rendering continuous integration useless.

One solution to the above file locking problem is to serialize the builds manually by building branches at predefined intervals, thus interleaving the builds so that they shouldn’t be able to overlap. As a result the developer may have to wait a fair amount of time to be sure his latest commit doesn’t breaks anything. Also, if a build has not finished when the next is scheduled to start, it may cause a cascade of broken builds to occur until the interleaving is properly realigned.

Another solution is to head over to Jay Flowers. He has branched off the official 1.0 version of ccnet and created a ccnet that allows for a lot more constraints than just serializing builds. It happens at the expense of running the non-official, not-recent version of ccnet, though.

My solution to the parallel build problem was to write a front-end for ccnet that implements build queues without the knowledge of ccnet, i.e., by means of checking for changes on a configurable number of branches. If a change is detected, the branch is enqueued. Then when all branches are checked, the tool force builds each branch, monitoring the state of the ccnet server to ensure that only one branch is building at a time.

With this solution you integrate the nice features of ccnet, such as the nice webgui for displaying what went wrong with a build and the systray application that goes green, yellow, or red depending on the state of your builds, with rapid feedback in a way that is compatible with Visual Studio’s integrated build engine.

The biggest downside, however, is that you can no longer see the changes that triggered a build from the web gui as ccnet is no longer in charge of cvs operations.

The tool is roughly 100 lines of IronPython code and works by spawning cvs.exe, synchronizing a number of working copies with the cvs repository, and in the process monitors the output of cvs.exe for changes, additions, and removals of files to the working copies.

The state of CruiseControl is determined by screen scraping the webgui and builds are forced by simulating a click on the “build” button for the ccnet project in the webgui for which a change was detected.

The url of the ccnet webgui, the path to the cvs.exe, and mappings between the path on disk where each working copy resides and the corresponding name of the ccnet project are stored in the tools configuration file, which is itself Python code loaded into the tool on startup.

Using Python as a language for your configuration file, you don’t have to invent your own language or use verbose xml, which requires you to write code for parsing it.

PS: I recently stumbled across the Sequential Task Plug In that I want to try out sometime.

Update, April 16: It has come to my attention that there is a way to minimize duplication in the configuration file. It involves “… users to set up DTD entities within the CCNet configuration” as mentionen here. I haven’t tried it, though.

Update, August 10: CruiseControl.net 1.3 now comes with a build-in queue feature.

I’m currently managing the merge operations for a CVS repository with a branch layout looking partly as follows:

At some point in the past HEAD branched into B1, which subsequently branched into B1.1. Then development on B1 completed and it was merged back into HEAD so a release branch B2 could be created. B1.1, however, kept living its separate life for a while, because the features added aren’t to be released until with release branch B5. This also goes for the changes made to B3, so it makes sense for B1.1 to be merged into B3.

As a result, I just spend a few hours on tracking down a merge issue resulting from this merge, because a change made in B1 wasn’t present in B3 after B1.1 was merged into this branch. I finally figured out why: suppose a line is removed in B1 before the spin-off of B1.1. After that the very same line is added back in. Now when B1.1 is merged into B3, the line is removed again. There is no merge conflict at this point, because from CVS’ point of view it’s a clean merge.

It seems obvious what happened when looking at the diagram, but when you’re working with a large repository with lots of modified files and you’re merging code not written by yourself, this sort of indirect change propagation may inevitably bite you as well.

I use the SysInternals tools (now acquired by the Empire). The ones I use most often are:

• PageDefrag
  

  PageDefrag uses advanced techniques to provide you what commercial defragmenters cannot: the ability for you to see how fragmented your page file and Registry hives are, and to defragment them. In addition, it defragments event log files and Windows 2000/XP hibernation files [...].

  Has the ability to run at the next or every boot (that’s how it gets exclusive access to the system files). Running it every time I boot typically adds 10-20 seconds to my boot time, but I guess it depends on how often you boot, the amount of physical memory, and what kind of applications you run.

• Contig
  

  Contig is a single-file defragmenter that attempts to make files contiguous on disk. It’s perfect for quickly optimizing files that are continuously becoming fragmented, or that you want to ensure are in as few fragments as possible.

  Also has a recursive switch, so you can specify C:\ as the starting point and it’ll defragment its way through your entire drive.

• ShareEnum
  

  When you run ShareEnum it uses NetBIOS enumeration to scan all the computers within the domains accessible to it, showing file and print shares and their security settings.

  A worthy replacement for Network Neighborhood, although it sometimes is not able to find any shares, although I know there’s quite a few out there.

• Autoruns
  

  This utility, which has the most comprehensive knowledge of auto-starting locations of any startup monitor, shows you what programs are configured to run during system boot or login, and shows you the entries in the order Windows processes them. These programs include ones in your startup folder, Run, RunOnce, and other Registry keys.

  On my computer, it takes longer for Windows to become responsive after log on than it takes Windows to boot and display the log on screen, because Windows loads a ton of software, fills up the systray, and starts various services I really don’t need.

  Services such as Remote Registry for accessing the registry across the network, Windows Image Acquisition for popping up a dialog box when I scan an image on the scanner I don’t have, or Windows Zero Configuration for managing the WLAN access I never use.

  Those services are all on by default and can be disabled using AutoRuns (or using Control panel => Administrative tools => Services).

• Process Explorer
  

  The Process Explorer display consists of two sub-windows. The top window always shows a list of the currently active processes, including the names of their owning accounts, whereas the information displayed in the bottom window depends on the mode that Process Explorer is in: if it is in handle mode you’ll see the handles that the process selected in the top window has opened; if Process Explorer is in DLL mode you’ll see the DLLs and memory-mapped files that the process has loaded. Process Explorer also has a powerful search capability that will quickly show you which processes have particular handles opened or DLLs loaded.

  The quote speaks for itself.

There’s a lot more tools where these came from. Among the ones I use less frequently are FileMon and Regmon for real-time monitoring of file system and registry access, respectively. Also, I use TCPView as a more user friendly replacement for the netstat command.

Finally, take a look at dnrTV, where Scott Hanselman recently showed off some of these tools.