Bugfree.dk – Ronnie Holm's blog

This post is part of a series (download code here)

1. SharePoint list definition using the Template pattern
2. SharePoint list access using the Repository pattern
3. Building and deploying SharePoint solutions with MSBuild (to come)

Maybe it’s that the SharePoint API is hard to use. Maybe it’s that coding against SharePoint is about making smaller additions here and there. Maybe it’s that the Patterns & Practices SharePoint Guidance isn’t widely known. Whatever the reason, developing for SharePoint requires equal attention to the separation of presentation, business, and data access code. Hence, starting with data access, we may want to create a repository and route queries through it (the SharePoint Guidance outlines a more sophisticated implementation than the one below):

    [TestClass]
    public class EmployeesRepositoryTest {
        private SPSite _siteCollection;
        private SPWeb _site;
        private EmployeesRepository _repository;

        private readonly Employee _duffyDuck = new Employee {
            Id = 1000, Name = "Duffy Duck", HireDate = new DateTime(2009, 12, 1),
            Remarks = "Looks like a duck, quacks like a duck, probably is a duck"
        };
        private readonly Employee _porkyPig = new Employee {
            Id = 1001, Name = "Porky Pig", HireDate = new DateTime(2010, 2, 1)
        };
        private readonly Employee _sylvesterTheCat = new Employee {
            Id = 1002, Name = "Sylvester the Cat", HireDate = new DateTime(2010, 3, 1)
        };
        private readonly Employee _bugsBunny = new Employee {
            Id = 1100, Name = "Bugs Bunny", HireDate = new DateTime(2010, 1, 1)
        };

        public void AddEmployees() {
            _repository.AddEmployee(_site, _duffyDuck);
            _repository.AddEmployee(_site, _porkyPig);
            _repository.AddEmployee(_site, _sylvesterTheCat);
        }

        public void ClearEmployees() {
            var definition = new EmployeesDefinition();
            var employees = _site.Lists[definition.ListName];
            while (employees.Items.Count > 0)
                employees.Items.Delete(0);
        }

        [TestInitialize]
        public void Initialize() {
            _siteCollection = new SPSite("http://localhost");
            _site = _siteCollection.OpenWeb("/");
            _repository = new EmployeesRepository();
            ClearEmployees();
            AddEmployees();
        }

        [TestCleanup]
        public void Cleanup() {
            _site.Dispose();
            _siteCollection.Dispose();
        }

        [TestMethod]
        public void AddEmployee_should_add_valid_employee() {
            _repository.AddEmployee(_site, _bugsBunny);
            var e = _repository.GetEmployeeById(_site, _bugsBunny.Id);
            Assert.AreEqual(_bugsBunny.Id, e.Id);
            Assert.AreEqual(_bugsBunny.Name, e.Name);
            Assert.AreEqual(_bugsBunny.HireDate, e.HireDate);
            Assert.AreEqual(_bugsBunny.Remarks, e.Remarks);
        }

        [TestMethod]
        public void GetEmployeesHiredBetween_should_return_2010_hires() {
            var from = new DateTime(2010, 1, 1);
            var to = new DateTime(2010, 12, 31);
            var employees = _repository.GetEmployeesHiredBetween(_site, from, to);
            Assert.AreEqual(2, employees.Count);
            Assert.AreEqual(_porkyPig.Id, employees[0].Id);
            Assert.AreEqual(_sylvesterTheCat.Id, employees[1].Id);
        }
    }

In the words of Martin Fowler, here’s the essence of the Repository pattern:

A Repository mediates between the domain and data mapping layers, acting like an in-memory domain object collection. Client objects construct query specifications declaratively and submit them to Repository for satisfaction. Objects can be added to and removed from the Repository, as they can from a simple collection of objects, and the mapping code encapsulated by the Repository will carry out the appropriate operations behind the scenes. Conceptually, a Repository encapsulates the set of objects persisted in a data store and the operations performed over them, providing a more object-oriented view of the persistence layer.

In SharePoint terms, business logic should query a repository which in turn queries a SharePoint list. Within the repository, the weakly typed items returned are then mapped to strongly typed data transfer objects, which are returned to the business layer.

With this approach to data access comes a number of advantages: (1) duplicate data access code is eliminated. Only within the repository do we setup the query and transform the weakly typed SPListItemCollection into strongly typed data transfer objects. (2) The list definition classes introduced in SharePoint list definition using the Template pattern may be used to construct CAML queries from strongly typed field names. Lastly, (3) accessing data through a repository makes it easier to mock the data access part of the application and to integration test that part.

The code below is a simple, yet usable, implementation of the Repository pattern. The idea is to have all repositories inherit from a common base class. Its purpose is to wrap the querying of a list and to log what’s going on. Because it’s a base class, it shouldn’t know how to transform the weakly typed result into strongly data transfer objects or which CRUD operations a particular repository supports:

    public abstract class ListRepository {
        protected ListDefinition Definition { get; set; }
        protected SPListItemCollection Result { get; set; }

        protected void Query(SPWeb site, string caml) {
            AssertValidSite(site);
            AssertValidCaml(caml);
            AssertListExistence(site);
            AssertListDefinitionSetBySubclass();

            var watch = new Stopwatch();
            var list = site.Lists[Definition.ListName];
            var query = new SPQuery {Query = caml};
            Debug.WriteLine(
                string.Format("About to run query against list '{0}': {1}", list, caml));
            watch.Start();
            Result = list.GetItems(query);
            watch.Stop();
            Debug.WriteLine(
                string.Format("Query against '{0}' returned {1} rows in {2} ms",
                              list, Result.Count, watch.ElapsedMilliseconds));
        }

        protected void AssertListExistence(SPWeb site) {
            if (!ListDefinition.ListExists(site, Definition.ListName))
                throw new ArgumentException(
                    string.Format("No '{0}' list on site '{1}'", Definition.ListName, site.Url));
        }

        protected void AssertListDefinitionSetBySubclass() {
            if (Definition == null)
                throw new NullReferenceException(
                    string.Format(
                        "Sublcass must set Definition property prior querying '{0}' list",
                        Definition.ListName));
        }

        protected void AssertValidCaml(string query) {
            if (string.IsNullOrEmpty(query))
                throw new NullReferenceException("Query must not be null or empty");
        }

        protected void AssertValidSite(SPWeb site) {
            if (site == null)
                throw new NullReferenceException("Site must not be null");
        }
    }

Each ListRepository connects to a corresponding ListDefinition, holding the name of the list to query and its strongly typed field names. It’s the responsibility of a concrete repository to set the Definition property prior to doing any querying. After running a query, the Result property holds the weakly typed result, which a concrete repository can then transform into data transfer objects to be passed to the business layer.

As an example, add to the concrete EmployeeRepository any CRUD method you see necessary to fulfill the business requirements:

    public class EmployeesRepository : ListRepository {
        public EmployeesRepository() {
            Definition = new EmployeesDefinition();
        }

        public void AddEmployee(SPWeb site, Employee e) {
            var list = site.Lists[Definition.ListName];
            var item = list.Items.Add();
            item[EmployeesDefinition.EmployeeId] = e.Id;
            item[EmployeesDefinition.Name] = e.Name;
            item[EmployeesDefinition.HireDate] = e.HireDate;
            item[EmployeesDefinition.Remarks] = e.Remarks;
            item.Update();
        }

        public Employee GetEmployeeById(SPWeb site, int id) {
            var caml =
                string.Format(@"
                      <Where>
                        <Eq>
                          <FieldRef Name=""{0}"" />
                          <Value Type=""Integer"">{1}</Value>
                        </Eq>
                      </Where>",
                    EmployeesDefinition.EmployeeId, id);
            Query(site, caml);

            IList<Employee> employees = Map(Result);
            if (employees.Count == 0)
                throw new ArgumentException(string.Format("No employee with id = {0} exists", id));
            return employees[0];
        }

        public ReadOnlyCollection<Employee> GetEmployeesHiredBetween(SPWeb site, DateTime from, DateTime to) {
            var caml =
                string.Format(@"
                      <Where>
                        <And>
                            <Geq>
                              <FieldRef Name=""{0}"" />
                              <Value IncludeTimeValue=""TRUE"" Type=""DateTime"">{1}</Value>
                            </Geq>
                            <Leq>
                              <FieldRef Name=""{0}"" />
                              <Value IncludeTimeValue=""TRUE"" Type=""DateTime"">{2}</Value>
                            </Leq>
                        </And>
                      </Where>",
                    EmployeesDefinition.HireDate,
                    SPUtility.CreateISO8601DateTimeFromSystemDateTime(from),
                    SPUtility.CreateISO8601DateTimeFromSystemDateTime(to));
            Query(site, caml);
            return new ReadOnlyCollection<Employee>(Map(Result));
        }

        protected IList<Employee> Map(SPListItemCollection items) {
            var employees = new List<Employee>();
            foreach (SPItem item in items) {
                var e = new Employee {
                                Id = (int)item[EmployeesDefinition.EmployeeId],
                                Name = (string)item[EmployeesDefinition.Name],
                                HireDate = (DateTime)item[EmployeesDefinition.HireDate],
                                Remarks = (string)item[EmployeesDefinition.Remarks]
                            };
                employees.Add(e);
            }
            return employees;
        }
    }

Calling the GetEmployeeById or GetEmployeesHiredBetween methods, the caller is required to pass in an SPWeb instance pointing to the the site holding the list to query. Outside the SharePoint context, you have to manually create this instance, like with the integration tests above. But within the SharePoint context, callers are likely to just pass in SPContext.Current.Web.

The above repository implementation deliberately ignores any issue of caching. If you find the need for it, however, you can replace SPList.GetItem with PortalSiteMapProvider.GetCachedListItemsByQuery. The advantage of using the PortalSiteMapProvider over Asp.Net caching of the result is that the provider takes care of invalidating cache entries when items are modified. The disadvantage is that the provider is part of the SharePoint publishing API, which isn’t part of WSS 3.0. In addition, it’s only available from code running within SharePoint, and likely requires cache settings to be tweaked.

Another, non-trivial, improvement would include the use of the Unit of Work pattern. A commonly used pattern in ORMs because it offers a way to “keep track of everything you do during a business transaction that can affect the database. When you’re done, it figures out everything that needs to be done to alter the database as a result of your work". Like with the DataContext class in LINQ to SQL or the DataSet class in ADO.NET, it could add transaction support to SharePoint lists.

This post is part of a series (download code here)

1. SharePoint list definition using the Template pattern
2. SharePoint list access using the Repository pattern
3. Building and deploying SharePoint solutions with MSBuild (to come)

Books on SharePoint often show how to create lists from code by calling the SharePoint API from directly within a feature receiver. This receiver would then contain in-place string literals for column names, create the columns, update the default view, and so on. While such an approach provides for an nice demonstration of the SharePoint API, it tends to carry over into production code.

My take on feature receivers, however, is that they should contain the least possible amount of code. Staying true to object-orientation and separation of concerns, list definition and creation should instead consist of a common set of classes and methods to be called from the receiver:

    public override void FeatureActivated(SPFeatureReceiverProperties p) {
        using (var site = p.Feature.Parent as SPWeb) {
            var employees = new EmployeesDefinition();
            if (!ListDefinition.ListExists(site, employees.ListName))
                employees.CreateOnSite(site);
        }
    }

The first area to improve on is that of string literal field names getting duplicated across the code base. String literals make it impossible for the compiler to enforce a consistent naming of columns. As column names are used in defining the list, in forming CAML queries against it, and in accessing the result, misspelled column names is a common source of runtime errors. The second area to improve on is that of duplication of code for doing common list operations, like checking for the existence of the list or removing its title column. Lastly, we want to get rid of duplicate control logic, i.e., verifying preconditions and calling the same series of methods with every list definition.

One approach to improving on these areas is to start by defining a base class for list definitions. The base class combines helper methods, working on lists, with the Template pattern for defining the common steps of list creation:

    public abstract class ListDefinition {
        public string ListName;
        public string ListDescription;
        protected SPList List;

        public void CreateOnSite(SPWeb site) {
            if (string.IsNullOrEmpty(ListName))
                throw new ArgumentException("Expected not null or not empty list name member");
            if (site == null) throw new NullReferenceException("Expected valid site");
            CreateList(site);
            if (List == null) throw new NullReferenceException("Expected list member not null");
            CreateFields(List);
            UpdateDefaultView(List);
            SetAdditionalProperties(List);
        }

        protected abstract void CreateList(SPWeb site);
        protected virtual void CreateFields(SPList l) {}
        protected virtual void UpdateDefaultView(SPList l) {}
        protected virtual void SetAdditionalProperties(SPList l) {}

        public static void HideAndMakeTitleFieldNotRequiredOnItemNewAndEditPage(SPList l) {
            var title = l.Fields.GetField("Title");
            title.Required = false;
            title.Hidden = true;
            title.Update();
        }

        public static bool ListExists(SPWeb site, string listName) {
            if (string.IsNullOrEmpty(listName))
                throw new ArgumentException("ListNameMustNotBeNullOrEmpty", listName);
            SPList list = null;
            try {
                list = site.Lists[listName];
            }
            catch (ArgumentException) {
                // list not found
            }
            return list != null ? true : false;
        }
    }

At the core of the Template pattern is the template method. It defines the steps of an algorithm through a series of method calls. Each either abstract or virtual depending on if the step is mandatory or voluntary in the algorithm. Subclasses then specify the behavior of individual steps by implementing or overriding methods. In case of the ListDefinition class, CreateOnSite is our template method. It ensures that context, in the form of an SPList instance, is passed along to each step in the algorithm.

Each subclass defines the specific characteristics of a list. It defines constants for column names and their data types, the views and which column to index, and so on. This makes for a concise, easy to read, and consistent list definition:

    public class EmployeesDefinition : ListDefinition {
        public const string EmployeeId = "EmployeeId";
        public const string Name = "Name";
        public const string HireDate = "HireDate";
        public const string Remarks = "Remarks";

        public EmployeesDefinition() {
            ListName = "AcmeEmployees";
            ListDescription = "Employees at Acme Corp.";
        }

        protected override void CreateList(SPWeb site) {
            var guid = site.Lists.Add(ListName, ListDescription, SPListTemplateType.GenericList);
            List = site.Lists[guid];
        }

        protected override void CreateFields(SPList l) {
            l.Fields.Add(EmployeeId, SPFieldType.Integer, true);
            l.Fields.Add(Name, SPFieldType.Text, true);
            l.Fields.Add(HireDate, SPFieldType.DateTime, true);
            var remarks = (SPFieldMultiLineText) l.Fields[l.Fields.Add(Remarks, SPFieldType.Note, false)];
            remarks.RichText = true;
            remarks.RichTextMode = SPRichTextMode.FullHtml;
            remarks.Update();
            l.Update();
        }

        protected override void UpdateDefaultView(SPList l) {
            var v = l.DefaultView;
            v.ViewFields.Delete("LinkTitle");
            v.ViewFields.Add(EmployeeId);
            v.ViewFields.Add(Name);
            v.ViewFields.Add(HireDate);
            v.ViewFields.Add(Remarks);
            v.Update();
        }

        protected override void SetAdditionalProperties(SPList l) {
            HideAndMakeTitleFieldNotRequiredOnItemNewAndEditPage(l);
            l.Fields[HireDate].Indexed = true;
            l.Update();
        }
    }

Since the string literals have now become public constants, we get IntelliSense and compile-time checking referencing them. In addition, by defining common operations on the base class and using the Template pattern, we reduce the amount of code that would otherwise have to go into each definition. Finally, the base class asserts, on behalf of all subclasses, that the list name is set before attempting to create the list and that the SPList field is set after calling the CreateList method.

Visual Studio has the ability to add a file as a link, making other parts of Visual Studio believe the file is actually located where it’s added. One useful application of the link feature is for signing assemblies by way of a key file. Signing an assembly using a digital signature may serve one or more purposes: It’s a precondition for deploying an assembly to the Global Assembly Cache (GAC), sharing an assembly among applications, and making side-by-side execution possible. Also, signing an assembly is useful for asserting its integrity, GAC deployed or otherwise.

The reason for bringing up the linked file feature in conjunction with assembly signing is that to me the feature seems particularly well-suited for linking one and only key file into multiple Visual Studio projects. Yet people seem to be unaware of the feature and so common approaches to signing is to either create a new key file for each assembly — most likely because Visual Studio makes it so easy to create a new key file from the Project Settings dialog — or signing assemblies by creating one key file and then copy/paste it to other projects in the solution.

Below is an example from the GAC showcasing how assemblies related to Pex naturally go together and are therefore signed using the same key (they share 76a274db078248c8, their public key token). The same goes for assemblies relating to other Microsoft products, such as the ones comprising a particular version of the .Net framework (because of the side-by-side execution, different versions of .Net framework assemblies are signed using different keys). So why not apply the same principle to what you deploy to the GAC?

While using the same key (copy or not) makes it possible to differentiate your assemblies from everyone else and for your four-part assembly name in web.config and spread around the code to be uniform — SharePoint developers often need to provide the four-part name for their assemblies to other parts of the solution — copying the key around is still a violation of the DRY principle and as such is indicative of a code smell. More importantly, using the link feature of Visual Studio, there’s no reason to keep the bad smell around.

Assuming you’ve already created a solution (in Visual Studio 2008) and added a project to it, the way to share a key file is to first create one by right clicking a project in Solution Explorer and selecting Properties. Now click the Signing tab and check the Sign the assembly check box. The dropdown lets you create a new key file by selecting New and filling out the information in the Create Name Key window. Alternatively, a key file is created using sn.exe, the strong name utility that comes with the Microsoft Windows SDK. On my machine, sn.exe is located in C:\Program Files\Microsoft SDKs\Windows\v6.0A\Bin and is invoked like so:

    > sn -k MySharedKeyPair.snk

How the key file is created isn’t particularly relevant to the rest of this post. Just make sure to place the key file in a common location, e.g., the top-level directory holding the Visual Studio solution file. Now go to Solution Explorer, right click on your project, and select Add existing Item. Then locate the key file within the solution directory and make sure to click the arrow on the Add button and select Add as Link. Note how the file is added to the project and marked by an arrow indicating it’s a link:

Returning to the Signing tab and selecting the dropdown, it now includes the linked key file. Don’t mind the absolute path; once you select MySharedKeyPair.snk, Visual Studio converts the path to a relative one (close and reopen the Signing tab and see for yourself). Also note that if you select Browse, the key file you select is copied to the project folder and added to the project as a regular file.

Another way to associate a key file with an assembly is through the AssemblyKeyFile attribute. The attribute instructs the compiler of where to look for the key file and, although not used by the runtime, the attribute is included in the assembly along with any other attribute for everyone to see.

Using AssemblyKeyFile is probably the simplest approach to sharing a key file between projects, but now compiling the assembly results in a compiler warning as displayed above. The use of the AssemblyKeyFile attribute has been deprecated by Microsoft because it may lead to you inadvertently disclosing sensitive information through the embedded path and because working with a relative path may confuse users and the build system (the relative path is retained in the assembly, though). However, it appears AssemblyKeyFile is still the preferred way for MS to sign their assemblies. Inspecting a couple of .Net 3.5 assemblies with Reflector, here’s what their AssemblyKeyFile looks like:

    [assembly: AssemblyKeyFile(@"f:\\dd\\tools\\devdiv\\EcmaPublicKey.snk")]
    [assembly: AssemblyKeyFile(@"f:\\dd\\tools\\devdiv\\35MSSharedLib1024.snk")]
    [assembly: AssemblyKeyFile(@"f:\\dd\\tools\\devdiv\\FinalPublicKey.snk")]
    …

To come full circle, the question of how Visual Studio, or rather MSBuild, actually signs an assembly using the information provided in the Properties dialog (real file or link) remains to be answered. Since a Visual Studio project file acts as input to MSBuild, we can see what’s going on under the hood of MSBuild by invoking a compile from the command line. On my machine msbuild.exe is located in C:\Windows\Microsoft.NET\Framework\v3.5 and assuming you have a console window open with its current directory set to the directory of your project file, msbuild is invoked like so:

    > msbuild /verbosity:diagnostic > log.txt

Bring up log.txt in your favorite editor and eventually you’ll come across where csc.exe, the C# compiler, is invoked:

As you might have guessed, MSBuild carries over the location of the key file by passing the /keyfile argument to csc.exe.

This post details my first use of TypeMock, a commercial mocking framework, for testing a web part that displays a dropdown bound to a LINQ query. Based on the item selected, the web part then displays a result bound to another LINQ query. For the purpose of becoming familiar with TypeMock, though, the focus of this post is on testing the code that initializes the LINQ data context. It does so by reading a connection string from a configuration file and passing it to the data context.

In order to test code whose branching behavior depends on a value read from a configuration file, the test should provide various inputs and assert that the code under test acts accordingly. In addition, the test should run in milliseconds to encourage frequent runs and it should be self-contained, meaning no setup should be required before running the test. These requirements are best met with the web part running in a controlled environment. Yet, as far as the code under test goes, it should behave as if part of an Asp.Net page.

For the sake of brevity, here’s my web part with the code to initialize the data context:

    public class MyWebPart : WebPart {
        MyDataContext _db;    /* LINQ database context */
        string _error;        /* Early stage error message */    	

        public MyWebPart() {
            Initialize();
        }

        private void Initialize() {
            string c = GetMyConnectionString();
            if (c == null)
                 _error = "Unable to do something";
            else
                _db = new MyDataContext(c);
        }

        private string GetMyConnectionString() {
            System.Configuration.ConnectionStringSettings c =
                System.Configuration.ConfigurationManager.
                    ConnectionStrings["MyConnection"];
            return c == null ? null : c.ConnectionString;
        }

        protected override void Render(HtmlTextWriter w) {
            // display error message if set
        }
    }

Notice how the constructor calls out to an Initialize method. Otherwise code within the constructor would fire before I’d have a chance to setup a controlled environment around the web part and I wouldn’t be able to test it. Turning the attention to Initialize itself, it’s made up of only a simple if statement. The idea is that if I can’t mock something as simple as reading a value from a configuration file, most likely I can’t mock something more involved either. On the other hand, for a developer already familiar with TypeMock and test first development, the code above may be born of the need to satisfy a failing test.

I want to test Initialize by controlling how GetMyConnectionString gets to return the connection string. While running in an Asp.Net context, GetMyConnectionString would attempt to read the string from web.config. Within my controlled environment, however, I want to inject into the web part an alternate implementation that returns a value of my choosing. That way I can control the path through Initialize and inspecting the state of the _error or _db fields, I’m able to determine if the code behaved as intended.

Using the method chaining DSL of TypeMock, the test takes on this form (of course, having more tests, I’d move shared code into helper methods decreasing the test to code ratio):

    [TestClass, ClearMocks]
    public class MyWebPartTest {
        [TestMethod, Isolated, VerifyMocks]
        public void Should_set_error_message_when_no_
                    connection_string_is_configured() {
            // arrange
            MyWebPart fake = Isolate.Fake.Instance<MyWebPart>();
            Isolate.NonPublic.WhenCalled(fake,
                "Initialize").CallOriginal();
            Isolate.NonPublic.WhenCalled(fake,
                "GetMyConnectionString").WillReturn(null);
            PrivateObject handle = new PrivateObject(fake);

            // act
            handle.Invoke("Initialize", null); 

            // assert
            Isolate.Verify.NonPublic.WasCalled(fake,
                "GetMyConnectionString");
            string error = (string)handle.GetField("_error");
            MyDataContext ctx = (MyDataContext)handle.GetField("_db");
            Assert.AreEqual("Unable to do something", error);
            Assert.AreEqual(null, ctx);
        }
    }

There’re three parts to tests using TypeMock: (1) the arrange part sets up expectations of what’s going to happen when a method, a property, or some other part of the code under test is invoked; (2) the act part carries out the actual interaction with the object under test, turning to PrivateObject to invoke the Initialize method; and finally (3) the assert part verifies that what was supposed to happen did happen. Worth noting is that this test asserts both state and behavior, i.e., as part of executing the code under test, _error or _db should’ve been set to an appropriate state and GetMyConnectionString should’ve been called once and only once.

This concludes my example, which I believe illustrates the basics of testing and mocking using MS Test and TypeMock. Comparing and contrasting TypeMock to other popular mocking frameworks, TypeMock has the distinct feature that it can mock almost anything, with or without using interfaces. Some argue that not enforcing the use of interfaces is also the disadvantage of TypeMock, because for code to be truly testable, it should be modularized using the Inversion of Control principle. However, with legacy code, such as the .Net framework or the SharePoint API, to me TypeMock appears to bridge the two worlds nicely, leaving the developer in charge.

To learn more about the basic concepts of mocking, Inversion of Control, and Dependency Injection (containers), I’d recommend listening to a couple of Alt.Net podcasts: Dependency Injection and Inversion of Control and More Dependency Injection and Inversion of Control.

Download EmailingExceptionModule-1.0.zip.

In any real-world application, unhalted exceptions are an unavoidable fact of life. Such exceptions should occur relatively infrequently, but when they do some kind of supporting infrastructure better be in place to capture the when and why of the exceptions. As a developer, you should have the ability to perform a postmortem analysis on what went wrong and learn from it. Paramount to such analysis is capturing and logging sufficient information at the point of failure.

What this post covers is the thoughts and development of an Asp.Net HttpModule that captures and emails such unhalted exception information to a designated email address.

To be clear, the goal of the EmailingExceptionModule isn’t to prevent unhalted exception from occurring per se. Within code, a developer may not know how to handle an exception, and so the exception should rightfully propagate the call stack. In case no caller steps in and handles the exception, the best choice is most likely to terminate the application. It’s almost always better to be upfront with the user than conceal the possibly inconsistent state of the application.

So when does the EmailingExceptionModule come in handy then? One scenario is that of C# unchecked exceptions, where the compiler doesn’t force the caller to catch all types of exceptions thrown by the callee. Unchecked exceptions may materialize as type cast or null reference exceptions when accessing variables. What unchecked exceptions boil down to is that, given the cyclomatic complexity of some methods, it’s impractical to manually work through every conceivable path of execution ahead of time. Another scenario is problems with the runtime environment, such as out of disk space or database server down. Nonetheless, some map of the path to failure is helpful in preventing others going down that same path.

(Summary part of example email. Click here for complete output.)

With web applications, we can take advantage of the application running in a centralized environment. In the spirit of Microsoft’s Doctor Watson technology, we can register our own web application error handler and hook it into the Http request pipeline, and have the application execute our code on unhalted exceptions. In practice, such an error handler is implemented either through the Application_Error method of an application’s Global.asax or by loading an HttpModule into the application.

Hinted by the title, I went for the HttpModule. The reason may best be understood by taking a peak behind the .Net curtains: Global.asax is a filename hardwired into the framework so that whenever an application is first hit, the framework asks HttpApplicationFactory, an internal factory class, for an HttpApplication representing the application (there’s a unique one for each virtual application). As part of manufacturing the HttpApplication, the factory locates the application’s Global.asax, compiles it into a dynamically generated DLL, loads the DLL, and reflects over the Global.asax class looking for methods adhering to the convention of modulename_eventname. Methods found, such as Application_Error, are then stored in a list of MethodInfos and passed along to HttpApplication. Then, during HttpApplication initialization, events are bound to the methods within Global.asax.

That’s how Application_Error of Global.asax gets called even though the method isn’t overriding a base class implementation as is typical for the template pattern. That’s also why code within global.asax isn’t binary reusable across applications. An HttpModule, on the other hand, can be loaded into any number of applications.

In practice, implementing an HttpModule it a matter of creating a class that implements the IHttpModule interface:

   public interface IHttpModule {
      void Init(HttpApplication context);
      void Dispose();
   }

The crux of error handling with Asp.Net is the Error event of HttpApplication. The easiest way to hook up the event to a method is within the Init method of the class. In addition, we assign the application object to a field so that later we can access the most recently thrown exception through it:

   public class EmailingExceptionModule : IHttpModule {
      private HttpApplication _application;

      public void Init(HttpApplication a) {
         _application = a;
         _application.Error += OnUnhaltedException;
      }

      public void Dispose() {}

      private void OnUnhaltedException(object sender, EventArgs e) {
         Exception ex = _application.Server.GetLastError().InnerException;
         // Implementation left out for brevity. Please download source
      }
   }

The OnUnhaltedException method is where to add code that collects information about the exception and the environment, and that composes and ships the email. Depending on the environment, you may find the need to include additional information in the email. Given the source code, adding to the email is a matter of adding key/value pairs in the form of labels and values to a dictionary. Each dictionary then gets rendered as a table of the key and value columns.

To have an Asp.Net application load the EmailingExceptionModule and to configure its email related settings, add the following nodes to the application’s web.config:

   <configuration>
      <appSettings>
         <add key="EmailingExceptionModule_SenderAddress" value="sender@domain.com"/>
         <add key="EmailingExceptionModule_ReceiverAddress" value="receiver@domain.com"/>
         <add key="EmailingExceptionModule_SmtpServer" value="mail.domain.com"/>
         <add key="EmailingExceptionModule_SubjectPrefix" value="MyApp: "/>
      </appSettings>
      <system.web>
         <httpModules>
            <add name="EmailingExceptionModule" type="Holm.AspNet.EmailingExceptionModule,
                       EmailingExceptionModule, Version=1.0.0.0, Culture=neutral,
                       PublicKeyToken=13806f613f05e959"/>
         </httpModules>
      </system.web>
   </configuration>

So what’s the user experience of unhalted exceptions? Outside the confines of the EmailingExceptionModule, there’s no indication that the application took note of what happened. Depending on the user’s profile, it might be helpful to prompt for additional information at the crash point. Or perhaps turn to the Windows Live Messenger IM Control & Presence API and have the user engage in an MSN conversation with a developer.

What’s the advantage of emailing developers unhalted exceptions over storing the information in the file system or a database? I believe in keeping things simple and practical, and handing off information to the file system or a database isn’t. Most likely the information will end up gathering virtual dust on some server. The email approach, on the other hand, is a proactive, visible, and efficient means to the end of rapidly course-correcting for wrong assumptions.

Update, Sep 15: While you don’t need a PDB file to debug code through Visual Studio (that’s what <compilation debug=”true”> in web.config is for), the PDB file is required for the stack trace to contain line numbers. Since the PDB file typically has to reside next to the corresponding DLL for the .Net runtime to pick it up, the PDB file may need to be deployed to the GAC along with the DLL. Refer to the first three paragraphs of the “Debugging assemblies that live in the GAC” section of this post for how to GAC deploy PDB files.

For a centrally deployed Asp.Net application, it may be acceptable to ship and deploy the PDB file with the application. Keep in mind, though, that because the PDB file is a mapping of locations in the IL to the source file, having access to the PDB file makes it easier for someone to reverse engineer the code. It’s also possible to get at line numbers without the PDB file being deployed, but the approach is somewhat involved. Based on the mapping nature of the PDB file, the idea is to write out IL offsets as part of the exception and to post-process the offsets using a PDB file at a separate location.

Yet another alternative to shipping the PDB file is to setup a symbol server. With this nifty piece of code, the StackTrace class loads the PDB file of the symbol server, adding line numbers to the stack trace.

In this post I’ll rework a piece of code that changes the enclosing tag of a web control from a span to a div. A substitution of particular value with SharePoint, where adding web controls is a common way to extend the out of box experience. With SharePoint, and Asp.Net in general, a page is typically composed of reusable controls, each responsible for rendering its part of the page.

To showcase anti-patterns of API usage, intermediate code is included, i.e., code that may render the correct tags, although not necessarily in the true spirit of the .Net framework.

To set the stage, imagine writing a web control that inherits from ASP.Net WebControl. The goal is then to modify the behavior of the control in such a way that it (1) encloses its content in a div tag and (2) adds a CSS class for styling, and (3) can be added to a page like so:

   <MyControls:MyControl CssClass="myCssClass" runat="server" />

After making its way through the page rendering pipeline, the control tag gets transformed to:

   <div class="myCssClass">
      Hello World
   </div>

Within the control, a straight-forward way of achieving the intended output may be to hand craft the html and render it in RenderContents of WebControl. It almost goes without saying that rendering html by hand is but only a starting point when working with .Net. The next progression from there may be to override RenderBeginTag and RenderEndTag of WebControl:

   // version 1
   public class MyControl : WebControl {
      public override void RenderBeginTag(HtmlTextWriter w) {
         w.RenderBeginTag(string.Format("div class=\"{0}\"", CssClass));
      }

      public override void RenderEndTag(HtmlTextWriter w) {
         w.RenderEndTag();
      }

      protected override void RenderContents(HtmlTextWriter w) {
         w.Write("Hello World");
      }
   }

Resulting in this piece of html:

   <div class="myCssClass">
      Hello World
   </div class="myCssClass">

Clearly, RenderBeginTag and RenderEndTag of HtmlTextWriter don’t know how to separate tag from attribute. Nonetheless, the code reveals the stack based approach taken by the Render-pair of methods.

Gleaning over the documentation of HtmlTextWriter, however, I encountered the AddAttribute method for setting attributes on a tag. Inside HtmlTextWriter, what AddAttribute does is add attributes and associated values to an array that gets rendered during the next call to RenderBeginTag. So besides introducing strongly typed attributes and tags, the next version contains a modified RenderBeginTag:

   // version 2
   public override void RenderBeginTag(HtmlTextWriter w) {
      w.AddAttribute(HtmlTextWriterAttribute.Class, CssClass);
      w.RenderBeginTag(HtmlTextWriterTag.Div);
   }

Now the control renders as expected, although the code still has a certain bad smell to it. WebControl ought to know how to properly render its begin and end tag. After all it’s WebControl that exposes, among others, the CssClass property. So taking another dive into the documentation, I found the virtual, read-only TagKey property of WebControl. Strangely named as it may be, overriding TagKey makes WebControl call our override as it renders the control, adding in optional attributes, such as CssClass. Implementing the property also turns RenderBeginTag and RenderEndTag obsolete:

   // version 3
   protected override HtmlTextWriterTag TagKey {
      get {
         return HtmlTextWriterTag.Div;
      }
   }

As an added bonus, changing the enclosing tag can also be done calling the WebControl’s constructor, passing in the desired tag:

   // version  4
   public class MyControl : WebControl {
      public MyControl() : base(HtmlTextWriterTag.Div) { }

      protected override void RenderContents(HtmlTextWriter w) {
         w.Write("Hello World");
      }
   }

As it turns out, calling the constructor is how WebControl derived classes get to use span in the first place. The default WebControl constructor passes control onto another constructor that sets the value of what gets returned by the base class TagKey implementation. In essence, overriding TagKey makes it a player in the template design pattern:

   // .Net framework code
   protected WebControl() : this(HtmlTextWriterTag.Span) { }

   public WebControl(HtmlTextWriterTag tag) {
      tagKey = tag;
   }

Also worth noting is that for controls deriving from WebControl derived classes, such as CompositeControl, the constructor cannot be used to set the enclosing tag. The constructor exposed by CompositeControl and friends doesn’t call up the constructor chain passing in the tag, making TagKey the choice to go.

Finally, deep down the inhertitance chain below WebControl lives WebPart, a type of control very popular with SharePoint. Interestingly enough, WebPart defaults to using div as its enclosing tag, because walking up the chain, we come across the constructor of Panel:

   // .Net framework code
   public Panel() : base(HtmlTextWriterTag.Div) { }

So, using Panel provides another way to implement a web control that owns a distinct part of a page.

Today I faced an interesting challenge while querying list items in SharePoint. On a page I have a control that displays list items based on the terms used to tag the page, i.e., the content type on which the page is based contains a multi-valued field that holds a subset of predefined tags. Similarly, for items in a SharePoint list, each item is tagged using a subset of the same predefined tags. The responsibility of the control is then to (1) read the tags associated with the page and (2) query the tags of the list items in an OR-wise fashion. The (3) outcome is then displayed by the control as a set of context specific items.

To decide on the items to display, an initial approach might be to go through the list one item at a time, looking for matching tags. But working with large lists in SharePoint this approach is not recommended. Alternatively, the predicate against which each item in the list is evaluated could take on the form of a CAML query. Looking for items matching a single tag is then easily expressed:

   <Where>
      <Eq>
         <FieldRef Name='MyField' />
         <Value Type='LookupMulti'>tag1</Value>
      </Eq>
   </Where>

Next, consider a query with two tags OR’ed together. Although more verbose, the query is still fairly straightforward to compose on the fly:

   <Where>
      <Or>
         <Eq>
            <FieldRef Name='MyField' />
            <Value Type='LookupMulti'>tag1</Value>
         </Eq>
         <Eq>
            <FieldRef Name='MyField' />
            <Value Type='LookupMulti'>tag2</Value>
         </Eq>
      </Or>
   </Where>

However, for CAML queries where more than two tags are OR’ed together, the binary nature of the OR operator works against us. Where real programming languages have their parsers automatically transform an expression like (a || b || c) into ((a || b) || c), the CAML query parser applies no such transform — In a sense CAML is more like an XML serialized, abstract syntax tree than a query language intended for direct use:

    <Where>
      <Or>
         <Or>
            <Eq>
               <FieldRef Name='MyField' />
               <Value Type='LookupMulti'>tag1</Value>
            </Eq>
            <Eq>
               <FieldRef Name='MyField' />
               <Value Type='LookupMulti'>tag2</Value>
            </Eq>
         </Or>
         <Eq>
            <FieldRef Name='MyField' />
            <Value Type='LookupMulti'>tag3</Value>
         </Eq>
      </Or>
   </Where>

So how do we go about composing a query that OR together n tags (not to mention a mix of ORs and ANDs)? One approach might be using the U2U CAML Query Builder. The tool delivers a UI for formulating queries, but also exposes its CAML query builder API as a .Net assembly. Unfortunately, documentation on how to use the builder is sparse. Nonetheless, I went for the case of composing a query that OR together three tags:

   string ComposeCamlQuery() {
      Builder b = new Builder(CamlTypes.GetListItems);
      b.AddViewField("Title");
      b.AddViewField("MyField");

      bool addCombinerNode;
      foreach (string tag in new string[] { "tag1", "tag2", "tag3" })
         b.AddWhereField("MyField", tag, "LookupMulti",
                         "Or", out addCombinerNode);
      return b.CamlDocument.InnerXml;
   }

The outcome is a rather strange looking CAML query with extra <And></And> and missing <Eq></Eq> tags. Funny thing is that this type of the query can be correctly composed through the UI, so most likely I’m not using the API correctly:

   <Where>
      <And>
         <And>
            <Or>
               <FieldRef Name="MyField" />
               <Value Type="LookupMulti">tag1</Value>
            </Or>
            <Or>
               <FieldRef Name="MyField" />
               <Value Type="LookupMulti">tag2</Value>
            </Or>
         </And>
         <Or>
            <FieldRef Name="MyField" />
            <Value Type="LookupMulti">tag3</Value>
         </Or>
      </And>
   </Where>

I eventually abandoned the idea of using the CAML Query Builder API. Instead, a colleague pointed me to Waldek Mastykarz’s post on generating dynamic CAML queries. As long as all tags are to be either AND’ed or OR’ed together, Mastykarz provides an elegant solution, which I modified to be recursive, more generic, and read like an induction proof (at a negligible performance cost):

   ComposeCamlQuery(new[] { "tag1", "tag2", "tag3" },
                    "Or",
                    "<Where>{0}</Where>",
                    @"<Eq>
                       <FieldRef Name='MyField' />
                       <Value Type='LookupMulti'>{0}</Value>
                      </Eq>");

   string ComposeCamlQuery(IList<string> ops, string relOp,
                           string query, string leaf) {
      return ops.Count == 1
         ? string.Format(query,
                         string.Format(leaf, ops[0]))
         : ComposeCamlQuery(ops.Skip(1).ToList(),
                            relOp,
                            string.Format(query,
                               string.Format("<{0}>{1}{{0}}</{0}>",
                               relOp,
                               string.Format(leaf, ops[0]))),
                            leaf);
   }

Obviously, more complex code could be written for a mix of AND’s and OR’s. Before doing so, however, you probably want to evaluate other approaches for filtering list items:

1. Formulate a simpler, too general query and post-process the result
2. Formulate smaller queries and optionally merge the results and do post-processing
3. Use the SharePoint Search API

I also considered such alternatives as Linq to SharePoint or Caml.net, but although the first holds a lot of promise, it isn’t ready for prime time. As for the latter, its focus is more on composing type-safe queries than dynamic ones, so it’s more of a supplement to the first two alternatives.

As for SharePoint Search, the downside is that search has to be configured and that the list must have been indexed for the outcome to be accurate. Depending on the frequency with which the list is modified, the time between incremental crawls, and the context in which the results are displayed, the search approach may or may not be the solution you’re looking for.

Update, July 19: Added recursive code solution.

As part of my preparation for the SharePoint configuration exams (70-630 and 70-731), I just finished working my way through Bill English’s SharePoint Administrator’s Companion.

This book obviously targets SharePoint administrators. But after finishing the book, I believe it’s a must read for SharePoint developers as well. As such, I’d argue that a developer should know what and how SharePoint is capable of, starting with the point and click approach of this book and then progressing towards code. And by capable I’m referring to all parts of SharePoint, including the Central Administration and the Shared Services Provider.

Knowing the capabilities of point and click comes in handy when discussing requirements. It often lays the groundwork for a prototype and helps you figure out where to extend SharePoint to fulfill the business needs. In addition, knowing what SharePoint can do on a general level makes estimates more reliable. Learning about administrative tasks also has the added benefit of establishing a common terminology between a developer and an administrator.

All in all, the SharePoint Administrator’s Companion does a solid job of providing an overview of the capabilities of the SharePoint platform, demonstrated mostly through point and click. I’d therefore advocate that any developer should know the content of this book by heart, and use it as a starting point before moving on to more code-centric books such as the Inside Microsoft Windows SharePoint Services 3.0 and the Inside Microsoft Office SharePoint Service 2007.

Today I passed my first Microsoft exam ever, the 70-541: Microsoft Windows SharePoint Services 3.0, Application Development.

In preparation for the exam I found the Inside Microsoft Windows SharePoint Services 3.0 book and the Windows SharePoint Services 3.0 SDK to be the best study materials. In addition, I looked to others for tips and tricks.

I found the SDK to be supprisingly thorough, compared to the book, which doesn’t cover the entire curriculum. The book alone does get one enough points to pass the exam, though.

Overall the exam questions could benefit from a review for verbosity and clear wording. Also, one question was about the RegisterHiddenField method and the RequestFromAdminPort property, which MSDN marks as obsolete. Another question was about adding a column to a content type and pushing the change to a list based on the content type. On that question I could’ve sworn that two of the possible answers were correct.

Nonetheless, learning SharePoint with a specific goal and deadline in mind serves as a good motivator.

Lately I’ve been working on creating a portal with Windows SharePoint Services 2007 (WSS 3.0) for a large customer. The easy and intuitive way to create such a portal is to configure it through the web browser and using SharePoint Designer (a renamed and spruced up FrontPage) for everything not code-related.

Unfortunately, using the web interface and SharePoint Designer doesn’t scale well. With the drag and drop tools there is just no real support for deploying your portal to another server, and hence to unify the individual contributions of team members.

Therefore, the current way to go for any real SharePoint development is to work with the 12 Hive directly. Named after Office 12 (Office 2007), the 12 Hive is a hierarchy of xml files residing in “Program Files\Common Files\Microsoft Shared\web server extensions\12″ that controls almost everything about how SharePoint and your portal behaves.

Using Visual Studio (VS), you’d create a 12 Hive parallel file hierarchy of new folders and files and work within VS as usual. Like with any other project, you’d use version control, continuous integration, and so on, on xml and .Net code. With VS, deployment (to your development box) would then involve checking out xml and code from source control and compiling the code.

Then, using a build script, you’d xcopy the xml and binary into the 12 Hive and GAC, respectively, and possibly run various stsadm commands to activate your features. Alternatively, you’d create a SharePoint solution package that pretty much does the same thing when invoked by SharePoint.