One of the new features of SharePoint 2010 is the ability to tag content using terms from a term store. The out of the box tooling for importing, exporting, and maintaining term stores is somewhat limited though. That’s why I created a PowerShell script to import term sets into the term store.

Term store overview

SharePoint organizes terms in either a taxonomy or a folksonomy. In a taxonomy, terms are organized hierarchically whereas in a folksonomy the structure is flat. A taxonomy or folksonomy is either global, making it available to all site collections within a web application that’s associated with a Managed Metadata Service, or local, making it available in one site collection only.

Multiple Managed Metadata Services may be associated with a web application or site collection, each service in turn responsible for a single term store. Within one term store, multiple groups may be created, each potentially storing multiple term sets. Finally, a term set may have terms nested up to seven levels deep:

Managed Metadata Service Application
  TermStore
    Group1
      TermSet1
        Term1
          Term1.2
            Term1.2.3
              Term1.2.3.4
                Term1.2.3.4.5
                  Term1.2.3.4.5.6
                    Term1.2.3.4.5.6.7

You manage the term store through the browser and its Term Store Management Tool (the _layouts/termstoremanager.aspx application page) or through the SharePoint API. The management tool supports importing term sets only from a special CSV file format  (_layouts/1033/ImportTermSet.csv holds a sample) with these columns:

Term Set Name
Term Set Description
LCID
Available for Tagging
Term Description
Level 1 Term
Level 2 Term
...
Level 7 Term

The CSV file is incomplete in terms of what the term store offers. It doesn’t offer columns for specifying translations and synonyms for terms. But users can use Wictor Wilén Excel sheet to maintain the basic information. It has the standard columns above and includes macros to simplify saving in the correct format and creating additional term sets.

The question remains: do you maintain compatibility with the existing CSV format so you can import term sets using the browser or a forms-based tool like the CSV Bulk Taxonomy Importer/Exporter? Or do you define your own, possibly XML based format since term sets are hierarchical by nature, with additional columns for translations and synonyms? A non-CSV based format may also be easier to process. If you want to pursue the latter approach, Ben Robb has created a PowerShell script to import his XML format.

Importing term sets using PowerShell

I decided to stick with the CSV file format and create a PowerShell script that would allow me to import a term set using the command-line:

function ImportTermSet([Microsoft.SharePoint.Taxonomy.TermStore]$store, [string]$groupName, [PSCustomObject]$termSet) {
  function ImportTerm([Microsoft.SharePoint.Taxonomy.Group]$group,
                      [Microsoft.SharePoint.Taxonomy.TermSet]$set,
                      [Microsoft.SharePoint.Taxonomy.Term]$parent,
                      [string[]]$path) {
    if ($path.Length -eq 0) {
      return
    } elseif ($group -eq $null) {
      $group = $store.Groups | where { $_.Name -eq $path[0] }
      if ($group -eq $null) {
        $group = $store.CreateGroup($path[0])
      }
    } elseif ($set -eq $null) {
      $set = $group.TermSets | where { $_.Name -eq $path[0] }
      if ($set -eq $null) {
        $set = $group.CreateTermSet($path[0])
      }
    } else {
      $node = if ($parent -eq $null) { $set } else { $parent }
      $parent = $node.Terms | where { $_.Name -eq $path[0] }
      if ($parent -eq $null) {
        $parent = $node.CreateTerm($path[0], 1033)
      }
    }

    ImportTerm $group $set $parent $path[1..($path.Length)]
  }

  function RemoveTermGroup([Microsoft.SharePoint.Taxonomy.TermStore]$store, [string]$groupName) {
    $group = $store.Groups | where { $_.Name -eq $groupName }
    if ($group -ne $null) {
      $group.TermSets | foreach { $_.Delete() }
      $group.Delete()
      $store.CommitAll()
    }
  }

  RemoveTermGroup $store $groupName
  $termSetName = $termSet[0]."Term Set Name"
  $termSet | where { $_."Level 1 Term" -ne "" } | foreach {
    $path = @($groupName, $termSetName) + @(for ($i = 1; $i -le 7; $i++) {
    $term = $_."Level $i Term"
      if ($term -eq "") {
        break
      } else {
        $term
      }
    })

    ImportTerm -path $path
  }
}

The way to use the ImportTermSet function is best illustrated with an example:

$session = Get-SPTaxonomySession -Site "http://localhost"
$store = $session.TermStores["Managed Metadata Service"]
$termSet = Import-Csv "C:\Users\Ronnie\Desktop\ImportTermSet.csv"
ImportTermSet $store "MyGroup" $termSet
$store.CommitAll()

In its current form the script ignores the Term Set Description, LCID, Available for Tagging, and Term Description columns from the CSV file. It would be possible to take these columns into account by passing them to ImportTerm as well. Instead of just removing the head of the list, you’d remove the number of items consumed by the current step before recursing.

One aspect of many functional languages that’s always fascinated me is the ease with which you can express a recursive structure, such as a language, and have it evaluated. Suppose you want to define the grammar of a DSL of propositions, in F# you could do so using the discriminated union type:

type Proposition =
    | True
    | And of Proposition * Proposition
    | Or of Proposition * Proposition
    | Not of Proposition

In an object-oriented language like C#, this definition would be analogous to a hierarchy of classes. Proposition would be the base class of True, And, Or, and Not. Each would then have a constructor that accepts some number of arguments, e.g., the And constructor would accept a two-tuple, two propositions, making the Proposition type recursive. In C# you might even model the recursive relationship using the Composite pattern.

In F#, an instance of the Proposition type may be created using a constructor syntax that resembles that of a function call:

let p = And(Or(True, Not(True)), Not(Not(True)))

Now, because of the recursive nature of the Proposition type, instances of it form a parse tree-like structure:

                        And
                        /\
                       /  \
                      Or  Not
                      /\    \
                     /  \    \
                  True  Not  Not
                          \    \
                           \    \
                          True  True

Having a parse tree is not very useful in itself. What you need is some way to execute or evaluate it. Given the recursive nature of the grammar, it makes sense for the evaluator to be recursively defined as well. The evaluator would do pattern matching on the node type, deconstructing it into the parts needed to evaluate it. Since our language of propositions maps directly to the boolean operations of F#, it’s very simple to evaluate the parse tree:

let rec eval (p: Proposition) =
    match p with
    | True -> true
    | And(p1, p2) -> eval p1 && eval p2
    | Or (p1, p2) -> eval p1 || eval p2
    | Not(p1) -> not (eval p1)

let e = eval p // e : bool = true

The simple language of propositions doesn’t take into account operator precedence and associativity, variables, and so on. But the features of F# used here allow for more complicated languages to be expressed and evaluated directly. With C#, you’d often fallback to describing the grammar of more complex languages in Backus-Naur form and feeding it to a parser generator that would map the node types of the parse tree to classes and be able to form a valid parse tree of some input.

Most enterprise projects have one or more console applications for utility tasks such as cleaning up or importing data into the database. These utilities tend to be project-specific and small in terms of code size, and instead of several smaller assemblies, it makes sense to combine these into a single assembly. The generic runner would read the utility, called the command, and arguments from the command-line and use the command pattern to create and execute it.

For the generic runner to work, each command has to fulfill the contract.

public enum ExitCode {
    Success = 0,
    Failure
};

public interface ICommand {
    string Usage { get; }
    string Description { get; }
    ExitCode Execute(string[] args);
}

I want the runner to adhere to the open/closed principle. For its behavior to be modified without altering its core delegation logic. This requires the use of reflection to retrieve and instantiate a command based on command-line arguments.

class Program {
    static IEnumerable<ICommand> GetCommands() {
        var iCommand = typeof (ICommand);
        return System.Reflection.Assembly.GetExecutingAssembly().GetTypes().ToList()
            .Where(t => iCommand.IsAssignableFrom(t) && t != iCommand)
            .Select(t => Activator.CreateInstance(t) as ICommand);
    }

    static void DisplayHelp() {
        Console.WriteLine("Console [Command] [Arg1] [Arg2] [ArgN]\n\n");
        GetCommands().ToList().ForEach(command =>
            Console.WriteLine(command.Usage + "\n" + command.Description + "\n\n"));
    }

    static int Main(string[] args) {
        if (args.Length == 0) {
            DisplayHelp();
            return (int)ExitCode.Failure;
        }

        var commandName = args[0];
        var command = GetCommands().Where(t => t.GetType().Name == commandName).SingleOrDefault();
        if (command == null)
            throw new ArgumentException(string.Format("Command '{0}' not found", commandName));

        var executeArguments = new List<string>(args);
        executeArguments.RemoveAt(0);

        var exitCode = command.Execute(executeArguments.ToArray());
        return (int)exitCode;
    }
}

A trivial example of a command that adds two numbers would be the following:

// $> GenericRunner.exe Calculator 2 3 => 2 + 3 = 5
public class Calculator : ICommand {
    public string Usage {
        get { return "Calculator [Op1] [Op2]"; }
    }

    public string Description {
        get { return "World's simplest calculator"; }
    }

    public ExitCode Execute(string[] args) {
        try {
            Console.WriteLine(
                string.Format(
                    "{0} + {1} = {2}",
                    args[0], args[1], int.Parse(args[0]) + int.Parse(args[1])));
            return ExitCode.Success;
        } catch (Exception e) {
            Console.WriteLine(e.ToString());
            return ExitCode.Failure;
        }
    }
}

Now multiple smaller assemblies can be grouped into one, with a description of all commands automatically being assembled, and without commands interfering (too much) with each other.

It’s been a while since I last took a serious look at F#. Back then I did a simple random fractal terrain generator which, even though the algorithm is simple, I found challenging to do. Nevertheless, functional programming is just one of those areas that I keep returning to. This time around I want to use the event receiver for versioning attachments in SharePoint lists to get familiar with object-oriented F#. Of course, translating a C# class to an F# class, the result will look like C# with different syntax and better type inference. The point here is to use classes in F# as a way to expose functionality to other .NET languages. In a real-world F# application the core logic would likely not be object-oriented.

namespace Dk.Bugfree

open System
open System.Globalization
open Microsoft.SharePoint

type public ListAttachmentVersioningEventReceiver() =
    inherit SPItemEventReceiver()

    member private r.CustomVersion = "CustomVersion"
    member private r.ShadowLibrary = "ShadowLibrary"

    // override ItemAdded : properties:SPItemEventProperties -> unit
    override r.ItemAdded properties =
        base.ItemAdded properties
        r.SetCustomVersionLabel properties.ListItem
        r.CreateSnapshot properties

    // override ItemUpdated : properties:SPItemEventProperties -> unit
    override r.ItemUpdated properties =
        base.ItemUpdated properties
        let item = properties.ListItem

        if r.RollbackHappened item then
            r.RestoreSnapshot properties
            r.SetCustomVersionLabel item
            r.CreateSnapshot properties
        else
            r.CreateSnapshot properties
            r.SetCustomVersionLabel item

    // member private CreateSnapshot : properties:SPItemEventProperties -> unit
    member private r.CreateSnapshot properties =
        use site = properties.OpenWeb()
        let item = properties.ListItem
        let shadowLibrary = site.Lists.[r.ShadowLibrary] :?> SPDocumentLibrary
        let path = String.Format("Versions/{0}/{1}", item.ID, r.GetOfficialVersionLabel(item))
        let shadowFolder = r.CreateFolderPath shadowLibrary path

        item.Attachments |> Seq.cast |> Seq.iter (fun fileName ->
            let existingFile = item.ParentList.ParentWeb.GetFile(item.Attachments.UrlPrefix + fileName)
            let newFile = shadowFolder.Files.Add(fileName, existingFile.OpenBinaryStream())
            newFile.Item.Update())

    // member private RollbackHappened : item:SPListItem -> bool
    member private r.RollbackHappened item =
        let culture = CultureInfo.InvariantCulture
        let currentVersion = Single.Parse(r.GetOfficialVersionLabel(item), culture)
        let lastVersion = Single.Parse(r.GetCustomVersionLabel(item), culture)
        currentVersion > lastVersion + 1.0f

    // member private RestoreSnapshot : properties:SPItemEventProperties -> unit
    member private r.RestoreSnapshot properties =
        let item = properties.ListItem
        let restoreVersion = r.GetCustomVersionLabel item
        r.EventFiringEnabled <- false    

        item.Attachments |> Seq.cast |> Seq.map (fun fileName -> unbox<string> fileName) |> Seq.toList
                         |> Seq.iter (fun fileName -> item.Attachments.Delete(fileName))

        use site = properties.OpenWeb()
        let path = String.Format("Versions/{0}/{1}", item.ID, restoreVersion)
        let shadowLibrary = site.Lists.[r.ShadowLibrary] :?> SPDocumentLibrary
        let source = r.CreateFolderPath shadowLibrary path

        source.Files |> Seq.cast |> Seq.iter (fun file ->
            let unboxedFile = unbox<SPFile> file
            item.Attachments.Add(unboxedFile.Name, unboxedFile.OpenBinary()))

        item.SystemUpdate false
        r.EventFiringEnabled <- true

    // member private CreateFolderPath : list:SPDocumentLibrary -> path:string -> SPFolder
    member private r.CreateFolderPath list path : SPFolder =
        r.CreateFolderPathRecursive list.RootFolder (path.Split [|'/'|] |> Array.toList)

    // member private CreateFolderPathRecursive : folder:SPFolder -> pathComponents:string list -> SPFolder
    member private r.CreateFolderPathRecursive folder pathComponents =
        match pathComponents with
        | [] -> folder
        | head :: tail ->
            try
                let existingFolder = folder.SubFolders.[head]
                r.CreateFolderPathRecursive existingFolder tail
            with
                :? ArgumentException ->
                    let newFolder = folder.SubFolders.Add head
                    r.CreateFolderPathRecursive newFolder tail

    // member private SetCustomVersionLabel : item:SPListItem -> unit
    member private r.SetCustomVersionLabel item =
        r.EventFiringEnabled <- false
        item.[r.CustomVersion] <- r.GetOfficialVersionLabel item
        item.SystemUpdate false
        r.EventFiringEnabled <- true  

    // member private GetCustomVersionLabel : item:SPListItem -> string
    member private r.GetCustomVersionLabel item =
        item.[r.CustomVersion] :?> string

    // member private GetOfficialVersionLabel : item:SPListItem -> string
    member private r.GetOfficialVersionLabel item =
        item.Versions.[0].VersionLabel

A couple of things to note about the F# implementation: first, it hardly specifies any types. They’re inferred by the compiler. Where type names do appear, it’s mainly because they’re required to unbox elements of an IEnumerable collection. Secondly, F# has flexible self identifiers. Methods must explicitly specify the name of the this reference in C# and use it when accessing members. Thirdly, arguments to general .NET methods are passed as a tuple value, i.e., as comma-delimited arguments surrounded by parenthesis.

In versioning attachments in a SharePoint list using snapshotting, an event receiver was responsible for the heavy lifting. To enable versioning of a list, I could therefore have associated the receiver with a list by adding the usual registration XML to a feature. But versioning is a truly reusable building block that shouldn’t be restricted to lists that are known when the feature is created. A better solution would be to extend the SharePoint list settings page for all lists on a site on which the versioning feature is enabled. The user may then activate or deactivate attachment versioning on the fly.

This would involve adding or removing event receivers from a list as the user enables or disables versioning. The following extension method is one way to accomplish the addition-part in a type-safe manner:

// definition
public static class SPListExtensions {
    public static void RegisterEventReceiver<TReceiver>(this SPList list,
            SPEventReceiverType receiverType,
            int sequenceNumber) where TReceiver : SPItemEventReceiver {
        var assemblyName = typeof(TReceiver).Assembly.FullName;
        var className = typeof(TReceiver).FullName;

        (from SPEventReceiverDefinition definition in list.EventReceivers
         where definition.Assembly == assemblyName &&
               definition.Class == className &&
               definition.Type == receiverType
         select list.EventReceivers[definition.Id])
        .ToList()
        .ForEach(receiverToDelete => receiverToDelete.Delete());

        var receiver = list.EventReceivers.Add();
        receiver.Type = receiverType;
        receiver.Assembly = assemblyName;
        receiver.Class = className;
        receiver.SequenceNumber = sequenceNumber;
        receiver.Update();
        list.Update();
    }
}

// use
list.RegisterEventReceiver<ListAttachmentVersioningEventReceiver>(
    SPEventReceiverType.ItemAdded, 10000);
list.RegisterEventReceiver<ListAttachmentVersioningEventReceiver>(
    SPEventReceiverType.ItemUpdated, 10001);

Under rare circumstances the (assembly, class, type) tuple may not be unique, i.e., the same receiver may be registered multiple times, albeit with different sequence numbers. In practice I never found any use for this functionality, though, which is why I didn’t include the sequence number in the where clause above, causing all registrations matching the tuple to be removed.

(See also the F# implementation and adding event receivers to a list on the fly.)

Both SharePoint 2007 and 2010 support versioning for list items but not their attachments. No matter which version of a list item I look at, its attachments will always be the most recent. The attachment support seems to have been bolded on as an afterthought, resulting in behavior that’s counter-intuitive for developers as well as end-users. With SharePoint 2007 (and 2010), Microsoft suggests using a document library for proper attachment versioning. But I can’t substitute one with the other, since a list item may hold any number of attachments and an item in a document library may hold just one.

Existing solution

I counted on someone else having experienced a similar pain and come up with a workable cure. But except for Tim Ebenezer the search come up empty. Tim on the other hand has done a great job of seamlessly integrating his attachment versioning feature into SharePoint. When I activate the feature on a site, it adds a versioning menu item to the list settings page for every list on the site. Unfortunately the core versioning logic, storing attachments in a shadow library using an event receiver, isn’t particularly robust. Among other use cases, it doesn’t properly deal with a user first deleting an attachment and then, some versions later, adding an attachment with the same name.

I therefore set out to implement my own solution based on Tim’s ideas, hooking into the synchronous ItemAdding, ItemUpdating, ItemAttachmentAdding, and ItemAttachmentDeleting events and maintaining a shadow library of versions. This approach, however, quickly turned into a painful one. When the synchronous events run, nothing has yet been written to the database – at this stage a new item doesn’t even have its Id set, and merely determining the number of attachments added and how far I’ve come with the processing is tricky.

The next challenge I encountered was that event handlers cannot easily share state across multiple calls because SharePoint creates a new instance of the receiver class for every event handled. Processing multiple attachments require a counter into the array of attachments to keep track of which ones I’d copied to the shadow list. I’d have to resort to some outside-object storage, keeping in mind that the receiver might execute concurrently. But which storage should I use? Session state may have been disabled, and polluting one of the property bags stored in the content database is messy and also not thread-safe.

Overall, with the synchronous approach too much work has to go into tracking the state of the versioning process.

New solution

A synchronous solution is very hard to get right because it’s forced to work at the level of individual attachments. SharePoint doesn’t have a synchronous event that fires after all attachments have been processed. After all, why provide such an event when everything has already happened? Thinking instead in terms of the asynchronous events of ItemUpdated and ItemAdded, I have exactly what’s needed to snapshot all attachments in one batch, making versioning a lot simpler. When these events fire the item and its attachments have already been written to the database and I can focus on how to generate the snapshots — copying attachments back and forth between lists — and not worry about what the user actual did to the attachments from one version to the next.

// Prerequisites:
// 1. Create a Document Library named ShadowLibrary on the same site as the list to version
// 2. Add a row named CustomVersion of type string to the list to version
public class ListAttachmentVersioningEventReceiver : SPItemEventReceiver {
    private const string CustomVersion = "CustomVersion";
    private const string ShadowLibrary = "ShadowLibrary";

    public override void ItemAdded(SPItemEventProperties properties) {
        base.ItemUpdated(properties);
        SetCustomVersionLabel(properties.ListItem);
        CreateSnapshot(properties);
    }

    public override void ItemUpdated(SPItemEventProperties properties) {
        base.ItemUpdated(properties);

        var item = properties.ListItem;
        if (RollbackHappened(item)) {
            RestoreSnapshot(properties);
            SetCustomVersionLabel(item);
            CreateSnapshot(properties);
        }
        else {
            CreateSnapshot(properties);
            SetCustomVersionLabel(item);
        }
    }

    private void CreateSnapshot(SPItemEventProperties properties) {
        using (var site = properties.OpenWeb()) {
            var item = properties.ListItem;
            var shadowLibrary = site.Lists[ShadowLibrary] as SPDocumentLibrary;
            var path = string.Format("Versions/{0}/{1}", item.ID, GetOfficialVersionLabel(item));
            var shadowFolder = CreateFolderPath(shadowLibrary, path);

            foreach (string fileName in item.Attachments) {
                SPFile existingFile = item.ParentList.ParentWeb.GetFile(item.Attachments.UrlPrefix + fileName);
                SPFile newFile = shadowFolder.Files.Add(fileName, existingFile.OpenBinaryStream());
                newFile.Item.Update();
            }
        }
    }

    private bool RollbackHappened(SPListItem item) {
        var culture = CultureInfo.InvariantCulture;
        var currentVersion = float.Parse(GetOfficialVersionLabel(item), culture);
        var lastVersion = float.Parse(GetCustomVersionLabel(item), culture);
        return currentVersion > lastVersion + 1;
    }

    private void RestoreSnapshot(SPItemEventProperties properties) {
        var item = properties.ListItem;
        var restoreVersion = GetCustomVersionLabel(item);
        EventFiringEnabled = false;

        item.Attachments.Cast<string>().ToList().ForEach(attachment => item.Attachments.Delete(attachment));
        using (var site = properties.OpenWeb()) {
            var path = string.Format("Versions/{0}/{1}", item.ID, restoreVersion);
            var shadowLibrary = site.Lists[ShadowLibrary] as SPDocumentLibrary;
            var source = CreateFolderPath(shadowLibrary, path);

            foreach (SPFile file in source.Files)
                item.Attachments.Add(file.Name, file.OpenBinary());
        }

        item.SystemUpdate(false);
        EventFiringEnabled = true;
    }

    // can only get folder creation to work with Document Libraries
    private SPFolder CreateFolderPath(SPDocumentLibrary list, string path) {
        return CreateFolderPathRecursive(list.RootFolder, path.Split('/').ToList());
    }

    private SPFolder CreateFolderPathRecursive(SPFolder folder, IList<string> pathComponents) {
        if (pathComponents.Count == 0)
            return folder;

        SPFolder newFolder;
        try {
            newFolder = folder.SubFolders[pathComponents.First()];
        }
        catch (ArgumentException) {
            newFolder = folder.SubFolders.Add(pathComponents.First());
        }

        pathComponents.RemoveAt(0);
        return CreateFolderPathRecursive(newFolder, pathComponents);
    }

    private void SetCustomVersionLabel(SPListItem item) {
        EventFiringEnabled = false;
        item[CustomVersion] = GetOfficialVersionLabel(item);
        item.SystemUpdate(false);
        EventFiringEnabled = true;
    }

    private string GetCustomVersionLabel(SPItem item) { return item[CustomVersion] as string; }
    private string GetOfficialVersionLabel(SPListItem item) { return item.Versions[0].VersionLabel; }
}

When a list item is saved, I take a snapshot of the attachments, storing them in a folder structure like {Id}/{VersionNumber}/{Attachments} in the shadow document library. When a list item is restored to a previous version, existing attachments are first deleted before the ones from the snapshot are added back in, creating a new version of the list item.

Restoring previous versions also has a counter-intuitive meaning in SharePoint. Suppose in one version of a list item, I store a key in the item’s property bag, then I’d expect the property bag values to be specific to this version. But behind the scenes restore seems to work by cloning the current version and then copying only the values of the fields from the restore version to the new one. In other words, I can’t use the item’s property bag to store version specific information, such as a version tag to detect when a restore has occurred. I also can’t use the Modified field because SharePoint sets it to the time of the restore. To carry over version information I have to create and maintain a field of my own. Hence the CustomVersion field on the list to version.

Remember that because the ItemUpdated and ItemAdded execute asynchronously, all the snapshotting logic executes on a background thread, after control has returned to the user. Should an error occur at this point, the user will never see it and the snapshot may be left in an incomplete state. On the other hand, this approach scales well and doesn’t have to be fast because no user is awaiting the result.

Lastly, there’s one place in SharePoint where the versioning abstraction leaks through. It’s in the list item version dialog which displays older versions and enables restore to any previous version. The dialog will always show the most recent attachments.

Improvements

I could use the ETag property of an SPFile object to implement a more efficient differential snapshotting algorithm that would conserve storage space. Compressing attachments before storing them in the shadow library might also be an option, although then I’d have to promote the ETag value to a shadow library field before compressing.

A quick word on organizing extension methods: I usually collect them in an Extensions folder, appending Extensions to the name of class being extended and keeping with the one class per file convention. For brevity I’ve left out the using and the namespace part below.

SPListCollection extensions

In SharePoint 2010 the TryGetList method has been added to the SPListCollection class. The method returns either an SPList instance matching the display name or null. Oftentimes, however, you want to do a lookup based on the internal name. Here’s an extension method that adheres to the semantics of TryGetList, but using the internal name. It relies on the fact that the RootFolder property of a list is actually its internal name:

// definition
public static class SPListCollectionExtensions {
    public static SPList TryGetListByInternalName(this SPListCollection lists, string internalName) {
        return (from SPList l in lists
            where l.RootFolder.Name == internalName
            select l).SingleOrDefault();
    }
}

// use
if (site.Lists.TryGetListByInternalName(internalListName) == null)
   // list not found

SPFieldCollection extensions

Using the CreateNewField method of the SPFieldCollection you can add new fields to a list. The particular annoying aspect of this method, however, is that when you want to continue working with its result, oftentimes you have to cast it to one of the SPField subclasses. But since the SPFieldType, provided as one of the arguments to CreateNewField, closely relates to the actual SPField return type, an extension method is able to do the casting. This’ll expose mismatches at compile time instead of at runtime.

All it takes is for us to map out the relation between SPField and SPFieldType:

// definition
public static class SPFieldCollectionExtensions {
    public static TSPField CreateField<TSPField>(this SPFieldCollection fields,
            string internalName, string displayName) where TSPField : SPField {
        var spFieldToFieldType = new Dictionary<Type, SPFieldType> {
            { typeof(SPFieldDateTime), SPFieldType.DateTime },
            { typeof(SPFieldNumber), SPFieldType.Number },
            { typeof(SPFieldUser), SPFieldType.User },
            { typeof(SPFieldBoolean), SPFieldType.Boolean },
            { typeof(SPFieldMultiLineText), SPFieldType.Note },
            { typeof(SPFieldText), SPFieldType.Text }
        };

        var fieldType = spFieldToFieldType[typeof(TSPField)];
        var list = fields.List;
        var field = list.Fields[list.Fields.Add(internalName, fieldType, false)];
        field.Title = displayName;
        field.Update();
        return field as TSPField;
    }
}

// use
l.Fields.CreateField<SPFieldBoolean>(internalName, "displayName");

Taking the CreateField extension method one step further, oftentimes you want to set properties besides internal name and display name. For that purpose I’ve defined a CreateField method that accepts an Action<TField>. This allows you to reuse common property settings across fields for brevity and consistency while at the same time maintaining strong typing.

// definition
public static TSPField CreateField<TSPField>(this SPFieldCollection fields,
        string internalName, string displayName,
        Action<TSPField> setAdditionalProperties) where TSPField : SPField {
    var newField = CreateField<TSPField>(fields, internalName, displayName);
    setAdditionalProperties(newField);
    newField.Update();
    return newField;
}

// use
public static Action<SPFieldMultiLineText> RichTextProperties = f => {
    f.RichText = true;
    f.RichTextMode = SPRichTextMode.FullHtml;
};

l.Fields.CreateField<SPFieldBoolean>(internalName, "displayName", f => f.Required = true);
l.Fields.CreateField(internalName, "displayName", RichTextProperties);

With the Comment field, you can leave out the type argument because the compiler infers it based on the type of the Action delegate.

Similar to CreateField, I’ve defined two additional extension methods for creating lookup fields:

// definition
public static TSPField CreateLookup<TSPField>(this SPFieldCollection fields,
        string lookupListName, string internalName,
        string displayName) where TSPField : SPFieldLookup {
    var currentList = fields.List;
    var lookupList = currentList.ParentWeb.Lists.TryGetListByInternalName(lookupListName);
    var newField = currentList.Fields[currentList.Fields.AddLookup(internalName, lookupList.ID, false)];
    newField.Title = displayName;
    newField.Update();
    return newField as TSPField;
}

public static TSPField CreateLookup<TSPField>(this SPFieldCollection fields,
        string lookupListName, string internalName, string displayName,
        Action<TSPField> setAdditionalProperties) where TSPField : SPFieldLookup {
    var newField = CreateLookup<TSPField>(fields, lookupListName, internalName, displayName);
    setAdditionalProperties(newField);
    newField.Update();
    return newField;
}

// use
l.Fields.CreateLookup<SPFieldLookup>(lookupListName, internalName, displayName, f => f.AllowMultipleValues = true);

These extension methods makes using the SharePoint API more type-safe and concise, and defining lists using these methods and the template approach saves me from writing a lot of repetitive code.

While cleaning up some PowerShell installations scripts that I wrote some time ago, I come across the following piece of code (modified slightly for anonymity) for creating a hierarchy of SharePoint sites and setting the locale for a subset of the sites. It works but it sure doesn’t adhere to the DRY principle.

New-SPWeb -url $base_url -addtoquicklaunch -template "STS#1" -name "Base"
New-SPWeb -url $base_url/Dk -addtoquicklaunch -template "STS#1" -name "Dk"
New-SPWeb -url $base_url/Uk -addtoquicklaunch -template "STS#1" -name "Uk"
New-SPWeb -url $base_url/Dk/BusinessUnit1 -addtoquicklaunch -template "STS#1" -name "BusinessUnit1"
New-SPWeb -url $base_url/Dk/BusinessUnit2 -addtoquicklaunch -template "STS#1" -name "BusinessUnit2"
New-SPWeb -url $base_url/Uk/BusinessUnit1 -addtoquicklaunch -template "STS#1" -name "BusinessUnit1"
New-SPWeb -url $base_url/Uk/BusinessUnit2 -addtoquicklaunch -template "STS#1" -name "BusinessUnit2"

$dk = Get-SPWeb $base_url/Dk
$dkBusinessUnit1 = Get-SPWeb $base_url/Dk/BusinessUnit1
$dkBusinessUnit2 = Get-SPWeb $base_url/Dk/BusinessUnit2

$locale = [System.Globalization.CultureInfo]::CreateSpecificCulture("da-DK")
$dk.Locale = $locale
$dk.Update()

$dkBusinessUnit1.Locale = $locale
$dkBusinessUnit1.Update()

$dkBusinessUnit2.Locale = $locale
$dkBusinessUnit2.Update()

How can we reduce this repetition? Well, it repeats because we’ve made the common mistake of mixing logic with data. One way to separate the two is using a table-driven approach, i.e., extract the varying parts, the data, into a table and rewrite the logic so it’s parameterized by the table. After some trial and error with PowerShell here’s the code I ended up with:

$sites = @( @("", "Base"),
            @("Dk", "Dk", "da-DK"),
            @("Uk", "Uk"),
            @("Dk/BusinessUnit1", "BusinessUnit1", "da-DK"),
            @("Dk/BusinessUnit2", "BusinessUnit2", "da-DK"),
            @("Uk/BusinessUnit1", "BusinessUnit1"),
            @("Uk/BusinessUnit2", "BusinessUnit2") )

$sites | foreach {
  $url, $title, $culture = $_
  $newSite = New-SPWeb -url "$base_url/$url" -addtoquicklaunch -template "STS#1" -name $title

  if ($culture -ne "") {
    $locale = [System.Globalization.CultureInfo]::CreateSpecificCulture($culture)
    $newSite.Locale = $locale
    $newSite.Update()
  }
}

So far so good. The un-installation part of the script, however, follows the same pattern of repetition:

Remove-SPWeb -identity $base_url/Uk/BusinessUnit2 -confirm:$false -ErrorAction SilentlyContinue
Remove-SPWeb -identity $base_url/Uk/BusinessUnit1 -confirm:$false -ErrorAction SilentlyContinue
Remove-SPWeb -identity $base_url/Dk/BusinessUnit2 -confirm:$false -ErrorAction SilentlyContinue
Remove-SPWeb -identity $base_url/Dk/BusinessUnit1 -confirm:$false -ErrorAction SilentlyContinue
Remove-SPWeb -identity $base_url/Dk -confirm:$false -ErrorAction SilentlyContinue
Remove-SPWeb -identity $base_url/Uk -confirm:$false -ErrorAction SilentlyContinue
Remove-SPWeb -identity $base_url -confirm:$false

But now that we have the table of sites at hand, we can easily make the site removal table-driven as well:

function Reverse($array) {
  $clone = $array.Clone()
  [Array]::Reverse($clone)
  $clone
}

Reverse($sites) | foreach {
  $url = $_[0]
  Remove-SPWeb -identity "$base_url/$url" -confirm:$false -ErrorAction SilentlyContinue
}

I’m sure the code could be written more elegantly, but given my working knowledge of PowerShell I’m satisfied with the result. I especially like the functional programming style.