Bugfree.dk – Ronnie Holm's blog

Not anti-anything, just pro-quality

The given-expect testing pattern

Posted by Ronnie Holm on 25th April 2010

I was watching Brett Schuchert’s TDD screencast on implementing the shunting yard algorithm in C#. In it Brett builds up his tests in a style I hadn’t come across before. Each test is expressed as a given-expect statement. A pattern that is particularly useful in situations in which a class has a main method that accepts an open-ended number of dissimilar inputs.

I found the given-expect pattern useful in testing a piece of code that I was working on this week. I was refactoring and adding tests around an ASP.NET control adapter that makes SharePoint 2007 pages more XHTML compliant. I wanted to reuse the transformations outside the control adapter and hence ended up moving the transformation logic to a new class. It accepts possibly malformed HTML and relies on heuristics of the HTML Agility Pack to build a DOM off of it. I can then query the DOM, looking for known violations, and patch them before returning XHTML to the caller.

    public class HtmlToXHtmlTransformer {
        private readonly HtmlDocument _document;

        public HtmlToXHtmlTransformer(string html) {
            _document = new HtmlDocument();
            _document.DetectEncoding(new StringReader(html));
            _document.LoadHtml(html);
        }

        private void Transform(string xpath, Action<HtmlNode> nodeMatch) {
            var nodes = _document.DocumentNode.SelectNodes(xpath);
            if (nodes != null)
                foreach (var node in nodes)
                    nodeMatch.Invoke(node);
        }

        private void FixDuplicateBorderAttributeOnSPGridViewControl() {
            Transform("//table[count(@border)=2]", node => node.Attributes.Remove("border"));
        }

        public string Transform() {
            FixDuplicateBorderAttributeOnSPGridViewControl();
            _document.OptionWriteEmptyNodes = true;
            return _document.DocumentNode.WriteTo();
        }
    }

The complete HtmlToXHtmlTransformer collects a dozen transformations. Its Transform method is what we want to call with various HTML fragments to verify that they come out as XHTML. For this purpose, we might do the tests as Visual Studio data-driven tests that read their input and output from a text file. But in most cases I prefer traditional tests, so I can describe the purpose of a test with a descriptive method name and possibly a comment.

    [TestClass]
    public class HtmlToXHtmlTransformerTest {
        private string _result;

        [TestMethod]
        public void Must_selfclose_nodes_when_allowed() {
            Given("<br>");
            Expect("<br />");
        }

        [TestMethod]
        public void Must_remove_duplicate_border_on_SPGridView_control {
            Given(@"<table border=""0"" border=""0""></table>");
            Expect(@"<table border=""0""></table>");
        }

        private void Expect(string xhtml) {
            Assert.AreEqual(xhtml, _result);
        }

        private void Given(string html) {
            var transformer = new HtmlToXHtmlTransformer(html);
            _result = transformer.Transform();
        }
    }

I particularly like the clarity of the given-expect pattern and find that for a reasonable number of tests it’s a viable alternative to data-driven test. I do, however, recognize the value of data-driven tests in situations where a non-developer wants to test a class. Though at the unit test level I’ve never experienced this. It’s more characteristic of FitNesse for acceptance testing. However you unit test, just make sure your tests run with a minimum of effort on your part and that they run fast.

  • Share/Bookmark

Tags: , , ,
Posted in .Net, SharePoint | No Comments »

SharePoint Saturday EMEA virtual conference

Posted by Ronnie Holm on 4th April 2010

Back in January, I attended the one-day virtual SharePoint Saturday EMEA conference. It was my first virtual conference so I was excited to see how it would work out. Presenting through Office Live Meeting and interacting with the audience through text-only is definitely a challenge compared to live presentations. But I’d say it went well with 50-60 attendees at each of the three parallel tracks.

Here’re the talks that I attended, all recorded and available with slides.

SharePoint 2010 Planning and Best Practice Approaches to Upgrade

Upgrading to SharePoint 2010, you should first establish a copy of your existing environment on the latest patch level. Then run the stsadm command with the new PreUpgradeChecker option. Make sure your environment has no unused features, site templates, and so on, and identify everything that has changed from out-of-the-box by running a tool like WinMerge on the 12 hive. Also, assert that no pages has been ghosted, and don’t attempt to upgrade your SharePoint 2007 master page. Start with the 2010 ones and add in your 2007 modifications. SharePoint 2010 ships with both 2007 and 2010 master pages, but SharePoint 2007 and 2010 isn’t binary compatible.

Make sure all hardware requirements are satisfied: 8 GB RAM at the minimum on the Windows 2008 servers running SharePoint and MS SQL Server. On the client you need Internet Explorer 7 or 8 or Firefox 3.x to author content. Internet Explorer 6 isn’t supported. Not even with a degraded reader experience. The SharePoint 2010 development environment should be setup with Visual Studio 2010 on Windows Vista or Windows 7 in order to compile, debug, build, and deploy right out of Visual Studio 2010.

Understanding, using, and customizing PowerShell for a SharePoint 2010 Environment

This talk started with some PowerShell history and basics. PowerShell is made up of a shell that executes commands and an integrated scripting environment for development. .NET compiled code, called cmdlets, are the executable units combined using the PowerShell pipeline. The PowerShell language itself is dynamic and loosely typed, so being able to express yourself tops performance. Scripts may be compiled and deployed as a SnapIn, consisting of compiled DLLs and registry updates, or as a module using xcopy deployment.

SharePoint 2010 ships with 650 cmdlets, and its own base class for writing additional SharePoint cmdlets.

Design and Manage Site Collections in SharePoint 2010

By default every single site collection within a web application goes into one rapidly growing database (think about users migrating from file shares to site collections). Such a large database takes time to backup and restore. Hence, with SharePoint 2010, consider partitioning site collections into their own databases from the beginning. After a database is created, map it to one and only one site collection. Then, instead of putting a size limit on the web application, put it on the site collection itself.

In SharePoint 2007 you had to create multiple Shared Service Providers (SSP) if site collections had different needs or you didn’t want a site collection to access parts of an existing SSP. With SharePoint 2010, application services can now be shared between site collections.

With sandbox solutions, the solution is deployed to the database hosting the site, rather than to front-end servers. This makes it easier to deploy code, but the code is limited in what it can do, and you may have to set resource quotas across the site collection.

SharePoint 2010, Getting Ready

This talk provided an overview of SharePoint 2010. WSS 3.0 becomes SharePoint Foundation 2010 and MOSS 2007 becomes SharePoint Server 2010. Business Connectivity Services (BCS) are no longer part of the server edition, but has been moved to Foundation. A lot more services have been added, like state service, usage and health service, and Visio graphics service. Generally, with all the improvements to deployment and upgrade with PowerShell and the addition of multi-tenancy for the SSP, SharePoint 2010 is better in a hosting scenario.

The web user interface has been made more Office-like with the addition of the ribbon and the use of AJAX. In document libraries you can create document sets as a way to relate documents to each other and have versioning and workflow follow every document in the set. Through the BCS, you can expose external data as native SharePoint lists with full CRUD capabilities.

SharePoint 2010 Development Tips and Tricks

The focus of the first part this talk is on the SharePoint 2010 Foundation features. Instead of developing against the web services or the object model, you can use the new managed client object model. This way, you can work against SharePoint from .NET code, from a Silverlight application, or from JavaScript. Behind the scenes, the client object model works against a WCF service running on the SharePoint server, which talks to the server object model on the server, and returns JSON to the client.

The second part of this talk is about sandbox solutions. This feature provides site collection users with the ability to upload and deploy WSPs. Finally, the third part of the talk is about the fluid integration model, which solves the problem of accessing cross-domain resources, e.g., having a Silverlight application access information on a SharePoint server on a different domain.

Installing and Configuring SharePoint Server 2010 in a Virtual Environment

This talk recommends that everyone read the MS whitepaper on Virtualization of Microsoft SharePoint Products and Technologies. Configuring a development environment, the dilemma is weather to create one virtual machine hosting everything or one virtual machine for the MS SQL Server, the Active Directory, and the SharePoint server each. The presenter suggests a middle ground. Still, it’s important to create a number of service accounts and use them in your development environment rather than running everything as administrator.

The SharePoint 2010 installer comes with a prerequisite installer that goes onto the web and downloads required software. After that, you can run the SharePoint server installer. Here you can choose between Standalone and Server farm, but always pick server farm. Standalone should only be used for demo purposes.

  • Share/Bookmark

Tags: ,
Posted in SharePoint | No Comments »

Getting started with SharePoint presentation

Posted by Ronnie Holm on 31st March 2010

Download slides from the presentation.

Last week at work I did a 45 minutes presentation on how to get started with SharePoint development. Instead of the usual presentation on the technical merits of SharePoint, my focus was more on the change in mindset required coming from ASP.NET. For even though ASP.NET is a cornerstone of SharePoint, SharePoint is so much more. Not to mention different.

History makes a difference

SharePoint is a long-evolving product, dating back to the late nineties. From its inception and up till now new technologies have replaced old ones, APIs have emerged and died, product teams have merged, and Microsoft have bought and integrated other companies’ products into SharePoint. At the same time SharePoint has become the fastest growing server product in Microsoft history. All of this inevitably leaves its marks on the APIs, which aren’t as consistent and defect free as we’re used to with the .NET framework.

Provisioning is the magic sauce

What truly makes SharePoint development different from ASP.NET is the notion of provisioning. With ASP.NET you create a bunch of ASPX files, organize them in a static file structure, and deploy them to a server. With SharePoint, the structure emerges over time as SharePoint or the user creates instances of templates: sites from site templates, lists from list templates, pages from page templates, and so forth. From a developer’s perspective the challenge isn’t the instance creation in itself. It’s that SharePoint encourages users to not only dynamically evolve the structure, but also to modify the instances, severing their definition from the template. While it empowers users, it also makes it challenging to programmatically evolve the structure.

Developing with SharePoint is about creating templates and components for SharePoint or the user to compose a site of. To make this composition work, SharePoint imposes constraints on what can be done and how. While any constraint may at times be perceived as a hindrance, accepting it is what guarantees that a components will play nicely with others. These pieces of functionality are then packaged as features and added to a WSP installation file for deployment to SharePoint. Features may then be activated, making their functionality available to a site. Or deactivated and the WSP uninstalled.

Why learn SharePoint

By the end of the day, no doubt SharePoint is more challenging to work with than ASP.NET. Not acknowledging its history makes many ASP.NET developers so frustrated that they’ll never again want to work with SharePoint. Coming from ASP.NET I can understand why. At times SharePoint isn’t as smooth and evolved as ASP.NET. But knowing what to expect takes the worst of the frustration. And once you learn to leverage the power of SharePoint, you can build applications without writing a lot of plumping code. To a large extend the architecture is given, leaving you with the opportunity to fill in the blanks.

  • Share/Bookmark

Tags: , ,
Posted in .Net, SharePoint | No Comments »

SharePoint list access using the Repository pattern

Posted by Ronnie Holm on 18th January 2010

(Download code here. See related post on SharePoint list definition using the Template pattern)

Maybe it’s that the SharePoint API is hard to use. Maybe it’s that coding against SharePoint is about making smaller additions here and there. Maybe it’s that the Patterns & Practices SharePoint Guidance isn’t widely known. Whatever the reason, developing for SharePoint requires equal attention to the separation of presentation, business, and data access code. Hence, starting with data access, we may want to create a repository and route queries through it (the SharePoint Guidance outlines a more sophisticated implementation than the one below):

    [TestClass]
    public class EmployeesRepositoryTest {
        private SPSite _siteCollection;
        private SPWeb _site;
        private EmployeesRepository _repository;

        private readonly Employee _duffyDuck = new Employee {
            Id = 1000, Name = "Duffy Duck", HireDate = new DateTime(2009, 12, 1),
            Remarks = "Looks like a duck, quacks like a duck, probably is a duck"
        };
        private readonly Employee _porkyPig = new Employee {
            Id = 1001, Name = "Porky Pig", HireDate = new DateTime(2010, 2, 1)
        };
        private readonly Employee _sylvesterTheCat = new Employee {
            Id = 1002, Name = "Sylvester the Cat", HireDate = new DateTime(2010, 3, 1)
        };
        private readonly Employee _bugsBunny = new Employee {
            Id = 1100, Name = "Bugs Bunny", HireDate = new DateTime(2010, 1, 1)
        };

        public void AddEmployees() {
            _repository.AddEmployee(_site, _duffyDuck);
            _repository.AddEmployee(_site, _porkyPig);
            _repository.AddEmployee(_site, _sylvesterTheCat);
        }

        public void ClearEmployees() {
            var definition = new EmployeesDefinition();
            var employees = _site.Lists[definition.ListName];
            while (employees.Items.Count > 0)
                employees.Items.Delete(0);
        }

        [TestInitialize]
        public void Initialize() {
            _siteCollection = new SPSite("http://localhost");
            _site = _siteCollection.OpenWeb("/");
            _repository = new EmployeesRepository();
            ClearEmployees();
            AddEmployees();
        }

        [TestCleanup]
        public void Cleanup() {
            _site.Dispose();
            _siteCollection.Dispose();
        }

        [TestMethod]
        public void AddEmployee_should_add_valid_employee() {
            _repository.AddEmployee(_site, _bugsBunny);
            var e = _repository.GetEmployeeById(_site, _bugsBunny.Id);
            Assert.AreEqual(_bugsBunny.Id, e.Id);
            Assert.AreEqual(_bugsBunny.Name, e.Name);
            Assert.AreEqual(_bugsBunny.HireDate, e.HireDate);
            Assert.AreEqual(_bugsBunny.Remarks, e.Remarks);
        }

        [TestMethod]
        public void GetEmployeesHiredBetween_should_return_2010_hires() {
            var from = new DateTime(2010, 1, 1);
            var to = new DateTime(2010, 12, 31);
            var employees = _repository.GetEmployeesHiredBetween(_site, from, to);
            Assert.AreEqual(2, employees.Count);
            Assert.AreEqual(_porkyPig.Id, employees[0].Id);
            Assert.AreEqual(_sylvesterTheCat.Id, employees[1].Id);
        }
    }

In the words of Martin Fowler, here’s the essence of the Repository pattern:

A Repository mediates between the domain and data mapping layers, acting like an in-memory domain object collection. Client objects construct query specifications declaratively and submit them to Repository for satisfaction. Objects can be added to and removed from the Repository, as they can from a simple collection of objects, and the mapping code encapsulated by the Repository will carry out the appropriate operations behind the scenes. Conceptually, a Repository encapsulates the set of objects persisted in a data store and the operations performed over them, providing a more object-oriented view of the persistence layer.

In SharePoint terms, business logic should query a repository which in turn queries a SharePoint list. Within the repository, the weakly typed items returned are then mapped to strongly typed data transfer objects, which are returned to the business layer.

With this approach to data access comes a number of advantages: (1) duplicate data access code is eliminated. Only within the repository do we setup the query and transform the weakly typed SPListItemCollection into strongly typed data transfer objects. (2) The list definition classes introduced in SharePoint list definition using the Template pattern may be used to construct CAML queries from strongly typed field names. Lastly, (3) accessing data through a repository makes it easier to mock the data access part of the application and to integration test that part.

The code below is a simple, yet usable, implementation of the Repository pattern. The idea is to have all repositories inherit from a common base class. Its purpose is to wrap the querying of a list and to log what’s going on. Because it’s a base class, it shouldn’t know how to transform the weakly typed result into strongly data transfer objects or which CRUD operations a particular repository supports:

    public abstract class ListRepository {
        protected ListDefinition Definition { get; set; }
        protected SPListItemCollection Result { get; set; }

        protected void Query(SPWeb site, string caml) {
            AssertValidSite(site);
            AssertValidCaml(caml);
            AssertListExistence(site);
            AssertListDefinitionSetBySubclass();

            var watch = new Stopwatch();
            var list = site.Lists[Definition.ListName];
            var query = new SPQuery {Query = caml};
            Debug.WriteLine(
                string.Format("About to run query against list '{0}': {1}", list, caml));
            watch.Start();
            Result = list.GetItems(query);
            watch.Stop();
            Debug.WriteLine(
                string.Format("Query against '{0}' returned {1} rows in {2} ms",
                              list, Result.Count, watch.ElapsedMilliseconds));
        }

        protected void AssertListExistence(SPWeb site) {
            if (!ListDefinition.ListExists(site, Definition.ListName))
                throw new ArgumentException(
                    string.Format("No '{0}' list on site '{1}'", Definition.ListName, site.Url));
        }

        protected void AssertListDefinitionSetBySubclass() {
            if (Definition == null)
                throw new NullReferenceException(
                    string.Format(
                        "Sublcass must set Definition property prior querying '{0}' list",
                        Definition.ListName));
        }

        protected void AssertValidCaml(string query) {
            if (string.IsNullOrEmpty(query))
                throw new NullReferenceException("Query must not be null or empty");
        }

        protected void AssertValidSite(SPWeb site) {
            if (site == null)
                throw new NullReferenceException("Site must not be null");
        }
    }

Each ListRepository connects to a corresponding ListDefinition, holding the name of the list to query and its strongly typed field names. It’s the responsibility of a concrete repository to set the Definition property prior to doing any querying. After running a query, the Result property holds the weakly typed result, which a concrete repository can then transform into data transfer objects to be passed to the business layer.

As an example, add to the concrete EmployeeRepository any CRUD method you see necessary to fulfill the business requirements:

    public class EmployeesRepository : ListRepository {
        public EmployeesRepository() {
            Definition = new EmployeesDefinition();
        }

        public void AddEmployee(SPWeb site, Employee e) {
            var list = site.Lists[Definition.ListName];
            var item = list.Items.Add();
            item[EmployeesDefinition.EmployeeId] = e.Id;
            item[EmployeesDefinition.Name] = e.Name;
            item[EmployeesDefinition.HireDate] = e.HireDate;
            item[EmployeesDefinition.Remarks] = e.Remarks;
            item.Update();
        }

        public Employee GetEmployeeById(SPWeb site, int id) {
            var caml =
                string.Format(@"
                      <Where>
                        <Eq>
                          <FieldRef Name=""{0}"" />
                          <Value Type=""Integer"">{1}</Value>
                        </Eq>
                      </Where>",
                    EmployeesDefinition.EmployeeId, id);
            Query(site, caml);

            IList<Employee> employees = Map(Result);
            if (employees.Count == 0)
                throw new ArgumentException(string.Format("No employee with id = {0} exists", id));
            return employees[0];
        }

        public ReadOnlyCollection<Employee> GetEmployeesHiredBetween(SPWeb site, DateTime from, DateTime to) {
            var caml =
                string.Format(@"
                      <Where>
                        <And>
                            <Geq>
                              <FieldRef Name=""{0}"" />
                              <Value IncludeTimeValue=""TRUE"" Type=""DateTime"">{1}</Value>
                            </Geq>
                            <Leq>
                              <FieldRef Name=""{0}"" />
                              <Value IncludeTimeValue=""TRUE"" Type=""DateTime"">{2}</Value>
                            </Leq>
                        </And>
                      </Where>",
                    EmployeesDefinition.HireDate,
                    SPUtility.CreateISO8601DateTimeFromSystemDateTime(from),
                    SPUtility.CreateISO8601DateTimeFromSystemDateTime(to));
            Query(site, caml);
            return new ReadOnlyCollection<Employee>(Map(Result));
        }

        protected IList<Employee> Map(SPListItemCollection items) {
            var employees = new List<Employee>();
            foreach (SPItem item in items) {
                var e = new Employee {
                                Id = (int)item[EmployeesDefinition.EmployeeId],
                                Name = (string)item[EmployeesDefinition.Name],
                                HireDate = (DateTime)item[EmployeesDefinition.HireDate],
                                Remarks = (string)item[EmployeesDefinition.Remarks]
                            };
                employees.Add(e);
            }
            return employees;
        }
    }

Calling the GetEmployeeById or GetEmployeesHiredBetween methods, the caller is required to pass in an SPWeb instance pointing to the the site holding the list to query. Outside the SharePoint context, you have to manually create this instance, like with the integration tests above. But within the SharePoint context, callers are likely to just pass in SPContext.Current.Web.

The above repository implementation deliberately ignores any issue of caching. If you find the need for it, however, you can replace SPList.GetItem with PortalSiteMapProvider.GetCachedListItemsByQuery. The advantage of using the PortalSiteMapProvider over Asp.Net caching of the result is that the provider takes care of invalidating cache entries when items are modified. The disadvantage is that the provider is part of the SharePoint publishing API, which isn’t part of WSS 3.0. In addition, it’s only available from code running within SharePoint, and likely requires cache settings to be tweaked.

      Another, non-trivial, improvement would include the use of the Unit of Work pattern. A commonly used pattern in ORMs because it offers a way to “keep track of everything you do during a business transaction that can affect the database. When you’re done, it figures out everything that needs to be done to alter the database as a result of your work". Like with the DataContext class in LINQ to SQL or the DataSet class in ADO.NET, it could add transaction support to SharePoint lists.

      • Share/Bookmark

      Tags: , ,
      Posted in .Net, SharePoint | 1 Comment »

      SharePoint list definition using the Template pattern

      Posted by Ronnie Holm on 11th January 2010

      (Download code here. See related post on SharePoint list access using the Repository pattern)

      Books on SharePoint often show how to create lists from code by calling the SharePoint API from directly within a feature receiver. This receiver would then contain in-place string literals for column names, create the columns, update the default view, and so on. While such an approach provides for an nice demonstration of the SharePoint API, it tends to carry over into production code.

      My take on feature receivers, however, is that they should contain the least possible amount of code. Staying true to object-orientation and separation of concerns, list definition and creation should instead consist of a common set of classes and methods to be called from the receiver:

          public override void FeatureActivated(SPFeatureReceiverProperties p) {
              using (var site = p.Feature.Parent as SPWeb) {
                  var employees = new EmployeesDefinition();
                  if (!ListDefinition.ListExists(site, employees.ListName))
                      employees.CreateOnSite(site);
              }
          }

      The first area to improve on is that of string literal field names getting duplicated across the code base. String literals make it impossible for the compiler to enforce a consistent naming of columns. As column names are used in defining the list, in forming CAML queries against it, and in accessing the result, misspelled column names is a common source of runtime errors. The second area to improve on is that of duplication of code for doing common list operations, like checking for the existence of the list or removing its title column. Lastly, we want to get rid of duplicate control logic, i.e., verifying preconditions and calling the same series of methods with every list definition.

      One approach to improving on these areas is to start by defining a base class for list definitions. The base class combines helper methods, working on lists, with the Template pattern for defining the common steps of list creation:

          public abstract class ListDefinition {
              public string ListName;
              public string ListDescription;
              protected SPList List;
      
              public void CreateOnSite(SPWeb site) {
                  if (string.IsNullOrEmpty(ListName))
                      throw new ArgumentException("Expected not null or not empty list name member");
                  if (site == null) throw new NullReferenceException("Expected valid site");
                  CreateList(site);
                  if (List == null) throw new NullReferenceException("Expected list member not null");
                  CreateFields(List);
                  UpdateDefaultView(List);
                  SetAdditionalProperties(List);
              }
      
              protected abstract void CreateList(SPWeb site);
              protected virtual void CreateFields(SPList l) {}
              protected virtual void UpdateDefaultView(SPList l) {}
              protected virtual void SetAdditionalProperties(SPList l) {}
      
              public static void HideAndMakeTitleFieldNotRequiredOnItemNewAndEditPage(SPList l) {
                  var title = l.Fields.GetField("Title");
                  title.Required = false;
                  title.Hidden = true;
                  title.Update();
              }
      
              public static bool ListExists(SPWeb site, string listName) {
                  if (string.IsNullOrEmpty(listName))
                      throw new ArgumentException("ListNameMustNotBeNullOrEmpty", listName);
                  SPList list = null;
                  try {
                      list = site.Lists[listName];
                  }
                  catch (ArgumentException) {
                      // list not found
                  }
                  return list != null ? true : false;
              }
          }

      At the core of the Template pattern is the template method. It defines the steps of an algorithm through a series of method calls. Each either abstract or virtual depending on if the step is mandatory or voluntary in the algorithm. Subclasses then specify the behavior of individual steps by implementing or overriding methods. In case of the ListDefinition class, CreateOnSite is our template method. It ensures that context, in the form of an SPList instance, is passed along to each step in the algorithm.

      Each subclass defines the specific characteristics of a list. It defines constants for column names and their data types, the views and which column to index, and so on. This makes for a concise, easy to read, and consistent list definition:

          public class EmployeesDefinition : ListDefinition {
              public const string EmployeeId = "EmployeeId";
              public const string Name = "Name";
              public const string HireDate = "HireDate";
              public const string Remarks = "Remarks";
      
              public EmployeesDefinition() {
                  ListName = "AcmeEmployees";
                  ListDescription = "Employees at Acme Corp.";
              }
      
              protected override void CreateList(SPWeb site) {
                  var guid = site.Lists.Add(ListName, ListDescription, SPListTemplateType.GenericList);
                  List = site.Lists[guid];
              }
      
              protected override void CreateFields(SPList l) {
                  l.Fields.Add(EmployeeId, SPFieldType.Integer, true);
                  l.Fields.Add(Name, SPFieldType.Text, true);
                  l.Fields.Add(HireDate, SPFieldType.DateTime, true);
                  var remarks = (SPFieldMultiLineText) l.Fields[l.Fields.Add(Remarks, SPFieldType.Note, false)];
                  remarks.RichText = true;
                  remarks.RichTextMode = SPRichTextMode.FullHtml;
                  remarks.Update();
                  l.Update();
              }
      
              protected override void UpdateDefaultView(SPList l) {
                  var v = l.DefaultView;
                  v.ViewFields.Delete("LinkTitle");
                  v.ViewFields.Add(EmployeeId);
                  v.ViewFields.Add(Name);
                  v.ViewFields.Add(HireDate);
                  v.ViewFields.Add(Remarks);
                  v.Update();
              }
      
              protected override void SetAdditionalProperties(SPList l) {
                  HideAndMakeTitleFieldNotRequiredOnItemNewAndEditPage(l);
                  l.Fields[HireDate].Indexed = true;
                  l.Update();
              }
          }

      Since the string literals have now become public constants, we get IntelliSense and compile-time checking referencing them. In addition, by defining common operations on the base class and using the Template pattern, we reduce the amount of code that would otherwise have to go into each definition. Finally, the base class asserts, on behalf of all subclasses, that the list name is set before attempting to create the list and that the SPList field is set after calling the CreateList method.

      • Share/Bookmark

      Tags: , ,
      Posted in .Net, SharePoint | 1 Comment »

      Using VS linked files for strong naming assemblies

      Posted by Ronnie Holm on 17th February 2009

      Visual Studio has the ability to add a file as a link, making other parts of Visual Studio believe the file is actually located where it’s added. One useful application of the link feature is for signing assemblies by way of a key file. Signing an assembly using a digital signature may serve one or more purposes: It’s a precondition for deploying an assembly to the Global Assembly Cache (GAC), sharing an assembly among applications, and making side-by-side execution possible. Also, signing an assembly is useful for asserting its integrity, GAC deployed or otherwise.

      The reason for bringing up the linked file feature in conjunction with assembly signing is that to me the feature seems particularly well-suited for linking one and only key file into multiple Visual Studio projects. Yet people seem to be unaware of the feature and so common approaches to signing is to either create a new key file for each assembly — most likely because Visual Studio makes it so easy to create a new key file from the Project Settings dialog — or signing assemblies by creating one key file and then copy/paste it to other projects in the solution.

      Below is an example from the GAC showcasing how assemblies related to Pex naturally go together and are therefore signed using the same key (they share 76a274db078248c8, their public key token). The same goes for assemblies relating to other Microsoft products, such as the ones comprising a particular version of the .Net framework (because of the side-by-side execution, different versions of .Net framework assemblies are signed using different keys). So why not apply the same principle to what you deploy to the GAC?

      While using the same key (copy or not) makes it possible to differentiate your assemblies from everyone else and for your four-part assembly name in web.config and spread around the code to be uniform — SharePoint developers often need to provide the four-part name for their assemblies to other parts of the solution — copying the key around is still a violation of the DRY principle and as such is indicative of a code smell. More importantly, using the link feature of Visual Studio, there’s no reason to keep the bad smell around.

      Assuming you’ve already created a solution (in Visual Studio 2008) and added a project to it, the way to share a key file is to first create one by right clicking a project in Solution Explorer and selecting Properties. Now click the Signing tab and check the Sign the assembly check box. The dropdown lets you create a new key file by selecting New and filling out the information in the Create Name Key window. Alternatively, a key file is created using sn.exe, the strong name utility that comes with the Microsoft Windows SDK. On my machine, sn.exe is located in C:\Program Files\Microsoft SDKs\Windows\v6.0A\Bin and is invoked like so:

          > sn -k MySharedKeyPair.snk
      

      How the key file is created isn’t particularly relevant to the rest of this post. Just make sure to place the key file in a common location, e.g., the top-level directory holding the Visual Studio solution file. Now go to Solution Explorer, right click on your project, and select Add existing Item. Then locate the key file within the solution directory and make sure to click the arrow on the Add button and select Add as Link. Note how the file is added to the project and marked by an arrow indicating it’s a link:

         

      Returning to the Signing tab and selecting the dropdown, it now includes the linked key file. Don’t mind the absolute path; once you select MySharedKeyPair.snk, Visual Studio converts the path to a relative one (close and reopen the Signing tab and see for yourself). Also note that if you select Browse, the key file you select is copied to the project folder and added to the project as a regular file.

      Another way to associate a key file with an assembly is through the AssemblyKeyFile attribute. The attribute instructs the compiler of where to look for the key file and, although not used by the runtime, the attribute is included in the assembly along with any other attribute for everyone to see.

      Using AssemblyKeyFile is probably the simplest approach to sharing a key file between projects, but now compiling the assembly results in a compiler warning as displayed above. The use of the AssemblyKeyFile attribute has been deprecated by Microsoft because it may lead to you inadvertently disclosing sensitive information through the embedded path and because working with a relative path may confuse users and the build system (the relative path is retained in the assembly, though). However, it appears AssemblyKeyFile is still the preferred way for MS to sign their assemblies. Inspecting a couple of .Net 3.5 assemblies with Reflector, here’s what their AssemblyKeyFile looks like:

          [assembly: AssemblyKeyFile(@"f:\\dd\\tools\\devdiv\\EcmaPublicKey.snk")]
          [assembly: AssemblyKeyFile(@"f:\\dd\\tools\\devdiv\\35MSSharedLib1024.snk")]
          [assembly: AssemblyKeyFile(@"f:\\dd\\tools\\devdiv\\FinalPublicKey.snk")]
          …
      

      To come full circle, the question of how Visual Studio, or rather MSBuild, actually signs an assembly using the information provided in the Properties dialog (real file or link) remains to be answered. Since a Visual Studio project file acts as input to MSBuild, we can see what’s going on under the hood of MSBuild by invoking a compile from the command line. On my machine msbuild.exe is located in C:\Windows\Microsoft.NET\Framework\v3.5 and assuming you have a console window open with its current directory set to the directory of your project file, msbuild is invoked like so:

          > msbuild /verbosity:diagnostic > log.txt
      

      Bring up log.txt in your favorite editor and eventually you’ll come across where csc.exe, the C# compiler, is invoked:

      As you might have guessed, MSBuild carries over the location of the key file by passing the /keyfile argument to csc.exe.

      • Share/Bookmark

      Tags: , ,
      Posted in .Net, SharePoint | No Comments »

      An example of unit testing using TypeMock

      Posted by Ronnie Holm on 11th February 2009

      This post details my first use of TypeMock, a commercial mocking framework, for testing a web part that displays a dropdown bound to a LINQ query. Based on the item selected, the web part then displays a result bound to another LINQ query. For the purpose of becoming familiar with TypeMock, though, the focus of this post is on testing the code that initializes the LINQ data context. It does so by reading a connection string from a configuration file and passing it to the data context.

      In order to test code whose branching behavior depends on a value read from a configuration file, the test should provide various inputs and assert that the code under test acts accordingly. In addition, the test should run in milliseconds to encourage frequent runs and it should be self-contained, meaning no setup should be required before running the test. These requirements are best met with the web part running in a controlled environment. Yet, as far as the code under test goes, it should behave as if part of an Asp.Net page.

      For the sake of brevity, here’s my web part with the code to initialize the data context:

          public class MyWebPart : WebPart {
              MyDataContext _db;    /* LINQ database context */
              string _error;        /* Early stage error message */    	
      
              public MyWebPart() {
                  Initialize();
              }
      
              private void Initialize() {
                  string c = GetMyConnectionString();
                  if (c == null)
                       _error = "Unable to do something";
                  else
                      _db = new MyDataContext(c);
              }
      
              private string GetMyConnectionString() {
                  System.Configuration.ConnectionStringSettings c =
                      System.Configuration.ConfigurationManager.
                          ConnectionStrings["MyConnection"];
                  return c == null ? null : c.ConnectionString;
              }
      
              protected override void Render(HtmlTextWriter w) {
                  // display error message if set
              }
          }
      

      Notice how the constructor calls out to an Initialize method. Otherwise code within the constructor would fire before I’d have a chance to setup a controlled environment around the web part and I wouldn’t be able to test it. Turning the attention to Initialize itself, it’s made up of only a simple if statement. The idea is that if I can’t mock something as simple as reading a value from a configuration file, most likely I can’t mock something more involved either. On the other hand, for a developer already familiar with TypeMock and test first development, the code above may be born of the need to satisfy a failing test.

      I want to test Initialize by controlling how GetMyConnectionString gets to return the connection string. While running in an Asp.Net context, GetMyConnectionString would attempt to read the string from web.config. Within my controlled environment, however, I want to inject into the web part an alternate implementation that returns a value of my choosing. That way I can control the path through Initialize and inspecting the state of the _error or _db fields, I’m able to determine if the code behaved as intended.

      Using the method chaining DSL of TypeMock, the test takes on this form (of course, having more tests, I’d move shared code into helper methods decreasing the test to code ratio):

          [TestClass, ClearMocks]
          public class MyWebPartTest {
              [TestMethod, Isolated, VerifyMocks]
              public void Should_set_error_message_when_no_
                          connection_string_is_configured() {
                  // arrange
                  MyWebPart fake = Isolate.Fake.Instance<MyWebPart>();
                  Isolate.NonPublic.WhenCalled(fake,
                      "Initialize").CallOriginal();
                  Isolate.NonPublic.WhenCalled(fake,
                      "GetMyConnectionString").WillReturn(null);
                  PrivateObject handle = new PrivateObject(fake);
      
                  // act
                  handle.Invoke("Initialize", null); 
      
                  // assert
                  Isolate.Verify.NonPublic.WasCalled(fake,
                      "GetMyConnectionString");
                  string error = (string)handle.GetField("_error");
                  MyDataContext ctx = (MyDataContext)handle.GetField("_db");
                  Assert.AreEqual("Unable to do something", error);
                  Assert.AreEqual(null, ctx);
              }
          }
      

      There’re three parts to tests using TypeMock: (1) the arrange part sets up expectations of what’s going to happen when a method, a property, or some other part of the code under test is invoked; (2) the act part carries out the actual interaction with the object under test, turning to PrivateObject to invoke the Initialize method; and finally (3) the assert part verifies that what was supposed to happen did happen. Worth noting is that this test asserts both state and behavior, i.e., as part of executing the code under test, _error or _db should’ve been set to an appropriate state and GetMyConnectionString should’ve been called once and only once.

      This concludes my example, which I believe illustrates the basics of testing and mocking using MS Test and TypeMock. Comparing and contrasting TypeMock to other popular mocking frameworks, TypeMock has the distinct feature that it can mock almost anything, with or without using interfaces. Some argue that not enforcing the use of interfaces is also the disadvantage of TypeMock, because for code to be truly testable, it should be modularized using the Inversion of Control principle. However, with legacy code, such as the .Net framework or the SharePoint API, to me TypeMock appears to bridge the two worlds nicely, leaving the developer in charge.

      To learn more about the basic concepts of mocking, Inversion of Control, and Dependency Injection (containers), I’d recommend listening to a couple of Alt.Net podcasts: Dependency Injection and Inversion of Control and More Dependency Injection and Inversion of Control.

      • Share/Bookmark

      Tags: , , , ,
      Posted in .Net, SharePoint | 2 Comments »

      Asp.Net error handling by HttpModule

      Posted by Ronnie Holm on 30th August 2008

      Download EmailingExceptionModule-1.0.zip.

      In any real-world application, unhalted exceptions are an unavoidable fact of life. Such exceptions should occur relatively infrequently, but when they do some kind of supporting infrastructure better be in place to capture the when and why of the exceptions. As a developer, you should have the ability to perform a postmortem analysis on what went wrong and learn from it. Paramount to such analysis is capturing and logging sufficient information at the point of failure.

      What this post covers is the thoughts and development of an Asp.Net HttpModule that captures and emails such unhalted exception information to a designated email address.

      To be clear, the goal of the EmailingExceptionModule isn’t to prevent unhalted exception from occurring per se. Within code, a developer may not know how to handle an exception, and so the exception should rightfully propagate the call stack. In case no caller steps in and handles the exception, the best choice is most likely to terminate the application. It’s almost always better to be upfront with the user than conceal the possibly inconsistent state of the application.

      So when does the EmailingExceptionModule come in handy then? One scenario is that of C# unchecked exceptions, where the compiler doesn’t force the caller to catch all types of exceptions thrown by the callee. Unchecked exceptions may materialize as type cast or null reference exceptions when accessing variables. What unchecked exceptions boil down to is that, given the cyclomatic complexity of some methods, it’s impractical to manually work through every conceivable path of execution ahead of time. Another scenario is problems with the runtime environment, such as out of disk space or database server down. Nonetheless, some map of the path to failure is helpful in preventing others going down that same path.


      (Summary part of example email. Click here for complete output.)

      With web applications, we can take advantage of the application running in a centralized environment. In the spirit of Microsoft’s Doctor Watson technology, we can register our own web application error handler and hook it into the Http request pipeline, and have the application execute our code on unhalted exceptions. In practice, such an error handler is implemented either through the Application_Error method of an application’s Global.asax or by loading an HttpModule into the application.

      Hinted by the title, I went for the HttpModule. The reason may best be understood by taking a peak behind the .Net curtains: Global.asax is a filename hardwired into the framework so that whenever an application is first hit, the framework asks HttpApplicationFactory, an internal factory class, for an HttpApplication representing the application (there’s a unique one for each virtual application). As part of manufacturing the HttpApplication, the factory locates the application’s Global.asax, compiles it into a dynamically generated DLL, loads the DLL, and reflects over the Global.asax class looking for methods adhering to the convention of modulename_eventname. Methods found, such as Application_Error, are then stored in a list of MethodInfos and passed along to HttpApplication. Then, during HttpApplication initialization, events are bound to the methods within Global.asax.

      That’s how Application_Error of Global.asax gets called even though the method isn’t overriding a base class implementation as is typical for the template pattern. That’s also why code within global.asax isn’t binary reusable across applications. An HttpModule, on the other hand, can be loaded into any number of applications.

      In practice, implementing an HttpModule it a matter of creating a class that implements the IHttpModule interface:

         public interface IHttpModule {
            void Init(HttpApplication context);
            void Dispose();
         }
      

      The crux of error handling with Asp.Net is the Error event of HttpApplication. The easiest way to hook up the event to a method is within the Init method of the class. In addition, we assign the application object to a field so that later we can access the most recently thrown exception through it:

         public class EmailingExceptionModule : IHttpModule {
            private HttpApplication _application;
      
            public void Init(HttpApplication a) {
               _application = a;
               _application.Error += OnUnhaltedException;
            }
      
            public void Dispose() {}
      
            private void OnUnhaltedException(object sender, EventArgs e) {
               Exception ex = _application.Server.GetLastError().InnerException;
               // Implementation left out for brevity. Please download source
            }
         }
      

      The OnUnhaltedException method is where to add code that collects information about the exception and the environment, and that composes and ships the email. Depending on the environment, you may find the need to include additional information in the email. Given the source code, adding to the email is a matter of adding key/value pairs in the form of labels and values to a dictionary. Each dictionary then gets rendered as a table of the key and value columns.

      To have an Asp.Net application load the EmailingExceptionModule and to configure its email related settings, add the following nodes to the application’s web.config:

         <configuration>
            <appSettings>
               <add key="EmailingExceptionModule_SenderAddress" value="sender@domain.com"/>
               <add key="EmailingExceptionModule_ReceiverAddress" value="receiver@domain.com"/>
               <add key="EmailingExceptionModule_SmtpServer" value="mail.domain.com"/>
               <add key="EmailingExceptionModule_SubjectPrefix" value="MyApp: "/>
            </appSettings>
            <system.web>
               <httpModules>
                  <add name="EmailingExceptionModule" type="Holm.AspNet.EmailingExceptionModule,
                             EmailingExceptionModule, Version=1.0.0.0, Culture=neutral,
                             PublicKeyToken=13806f613f05e959"/>
               </httpModules>
            </system.web>
         </configuration>
      

      So what’s the user experience of unhalted exceptions? Outside the confines of the EmailingExceptionModule, there’s no indication that the application took note of what happened. Depending on the user’s profile, it might be helpful to prompt for additional information at the crash point. Or perhaps turn to the Windows Live Messenger IM Control & Presence API and have the user engage in an MSN conversation with a developer.

      What’s the advantage of emailing developers unhalted exceptions over storing the information in the file system or a database? I believe in keeping things simple and practical, and handing off information to the file system or a database isn’t. Most likely the information will end up gathering virtual dust on some server. The email approach, on the other hand, is a proactive, visible, and efficient means to the end of rapidly course-correcting for wrong assumptions.

      Update, Sep 15: While you don’t need a PDB file to debug code through Visual Studio (that’s what <compilation debug=”true”> in web.config is for), the PDB file is required for the stack trace to contain line numbers. Since the PDB file typically has to reside next to the corresponding DLL for the .Net runtime to pick it up, the PDB file may need to be deployed to the GAC along with the DLL. Refer to the first three paragraphs of the “Debugging assemblies that live in the GAC” section of this post for how to GAC deploy PDB files.

      For a centrally deployed Asp.Net application, it may be acceptable to ship and deploy the PDB file with the application. Keep in mind, though, that because the PDB file is a mapping of locations in the IL to the source file, having access to the PDB file makes it easier for someone to reverse engineer the code. It’s also possible to get at line numbers without the PDB file being deployed, but the approach is somewhat involved. Based on the mapping nature of the PDB file, the idea is to write out IL offsets as part of the exception and to post-process the offsets using a PDB file at a separate location.

      Yet another alternative to shipping the PDB file is to setup a symbol server. With this nifty piece of code, the StackTrace class loads the PDB file of the symbol server, adding line numbers to the stack trace.

      • Share/Bookmark

      Tags: , , ,
      Posted in .Net, SharePoint | 2 Comments »

      How to modify the enclosing tag of a web control

      Posted by Ronnie Holm on 30th July 2008

      In this post I’ll rework a piece of code that changes the enclosing tag of a web control from a span to a div. A substitution of particular value with SharePoint, where adding web controls is a common way to extend the out of box experience. With SharePoint, and Asp.Net in general, a page is typically composed of reusable controls, each responsible for rendering its part of the page.

      To showcase anti-patterns of API usage, intermediate code is included, i.e., code that may render the correct tags, although not necessarily in the true spirit of the .Net framework.

      To set the stage, imagine writing a web control that inherits from ASP.Net WebControl. The goal is then to modify the behavior of the control in such a way that it (1) encloses its content in a div tag and (2) adds a CSS class for styling, and (3) can be added to a page like so:

         <MyControls:MyControl CssClass="myCssClass" runat="server" />
      

      After making its way through the page rendering pipeline, the control tag gets transformed to:

         <div class="myCssClass">
            Hello World
         </div>
      

      Within the control, a straight-forward way of achieving the intended output may be to hand craft the html and render it in RenderContents of WebControl. It almost goes without saying that rendering html by hand is but only a starting point when working with .Net. The next progression from there may be to override RenderBeginTag and RenderEndTag of WebControl:

         // version 1
         public class MyControl : WebControl {
            public override void RenderBeginTag(HtmlTextWriter w) {
               w.RenderBeginTag(string.Format("div class=\"{0}\"", CssClass));
            }
      
            public override void RenderEndTag(HtmlTextWriter w) {
               w.RenderEndTag();
            }
      
            protected override void RenderContents(HtmlTextWriter w) {
               w.Write("Hello World");
            }
         }
      

      Resulting in this piece of html:

         <div class="myCssClass">
            Hello World
         </div class="myCssClass">
      

      Clearly, RenderBeginTag and RenderEndTag of HtmlTextWriter don’t know how to separate tag from attribute. Nonetheless, the code reveals the stack based approach taken by the Render-pair of methods.

      Gleaning over the documentation of HtmlTextWriter, however, I encountered the AddAttribute method for setting attributes on a tag. Inside HtmlTextWriter, what AddAttribute does is add attributes and associated values to an array that gets rendered during the next call to RenderBeginTag. So besides introducing strongly typed attributes and tags, the next version contains a modified RenderBeginTag:

         // version 2
         public override void RenderBeginTag(HtmlTextWriter w) {
            w.AddAttribute(HtmlTextWriterAttribute.Class, CssClass);
            w.RenderBeginTag(HtmlTextWriterTag.Div);
         }
      

      Now the control renders as expected, although the code still has a certain bad smell to it. WebControl ought to know how to properly render its begin and end tag. After all it’s WebControl that exposes, among others, the CssClass property. So taking another dive into the documentation, I found the virtual, read-only TagKey property of WebControl. Strangely named as it may be, overriding TagKey makes WebControl call our override as it renders the control, adding in optional attributes, such as CssClass. Implementing the property also turns RenderBeginTag and RenderEndTag obsolete:

         // version 3
         protected override HtmlTextWriterTag TagKey {
            get {
               return HtmlTextWriterTag.Div;
            }
         }
      

      As an added bonus, changing the enclosing tag can also be done calling the WebControl’s constructor, passing in the desired tag:

         // version  4
         public class MyControl : WebControl {
            public MyControl() : base(HtmlTextWriterTag.Div) { }
      
            protected override void RenderContents(HtmlTextWriter w) {
               w.Write("Hello World");
            }
         }
      

      As it turns out, calling the constructor is how WebControl derived classes get to use span in the first place. The default WebControl constructor passes control onto another constructor that sets the value of what gets returned by the base class TagKey implementation. In essence, overriding TagKey makes it a player in the template design pattern:

         // .Net framework code
         protected WebControl() : this(HtmlTextWriterTag.Span) { }
      
         public WebControl(HtmlTextWriterTag tag) {
            tagKey = tag;
         }
      

      Also worth noting is that for controls deriving from WebControl derived classes, such as CompositeControl, the constructor cannot be used to set the enclosing tag. The constructor exposed by CompositeControl and friends doesn’t call up the constructor chain passing in the tag, making TagKey the choice to go.

      Finally, deep down the inhertitance chain below WebControl lives WebPart, a type of control very popular with SharePoint. Interestingly enough, WebPart defaults to using div as its enclosing tag, because walking up the chain, we come across the constructor of Panel:

         // .Net framework code
         public Panel() : base(HtmlTextWriterTag.Div) { }
      

      So, using Panel provides another way to implement a web control that owns a distinct part of a page.

      • Share/Bookmark

      Tags: , ,
      Posted in .Net, SharePoint | No Comments »

      Code based, dynamic CAML query composition

      Posted by Ronnie Holm on 3rd July 2008

      Today I faced an interesting challenge while querying list items in SharePoint. On a page I have a control that displays list items based on the terms used to tag the page, i.e., the content type on which the page is based contains a multi-valued field that holds a subset of predefined tags. Similarly, for items in a SharePoint list, each item is tagged using a subset of the same predefined tags. The responsibility of the control is then to (1) read the tags associated with the page and (2) query the tags of the list items in an OR-wise fashion. The (3) outcome is then displayed by the control as a set of context specific items.

      To decide on the items to display, an initial approach might be to go through the list one item at a time, looking for matching tags. But working with large lists in SharePoint this approach is not recommended. Alternatively, the predicate against which each item in the list is evaluated could take on the form of a CAML query. Looking for items matching a single tag is then easily expressed:

         <Where>
            <Eq>
               <FieldRef Name='MyField' />
               <Value Type='LookupMulti'>tag1</Value>
            </Eq>
         </Where>
      

      Next, consider a query with two tags OR’ed together. Although more verbose, the query is still fairly straightforward to compose on the fly:

         <Where>
            <Or>
               <Eq>
                  <FieldRef Name='MyField' />
                  <Value Type='LookupMulti'>tag1</Value>
               </Eq>
               <Eq>
                  <FieldRef Name='MyField' />
                  <Value Type='LookupMulti'>tag2</Value>
               </Eq>
            </Or>
         </Where>
      

      However, for CAML queries where more than two tags are OR’ed together, the binary nature of the OR operator works against us. Where real programming languages have their parsers automatically transform an expression like (a || b || c) into ((a || b) || c), the CAML query parser applies no such transform — In a sense CAML is more like an XML serialized, abstract syntax tree than a query language intended for direct use:

          <Where>
            <Or>
               <Or>
                  <Eq>
                     <FieldRef Name='MyField' />
                     <Value Type='LookupMulti'>tag1</Value>
                  </Eq>
                  <Eq>
                     <FieldRef Name='MyField' />
                     <Value Type='LookupMulti'>tag2</Value>
                  </Eq>
               </Or>
               <Eq>
                  <FieldRef Name='MyField' />
                  <Value Type='LookupMulti'>tag3</Value>
               </Eq>
            </Or>
         </Where>
      

      So how do we go about composing a query that OR together n tags (not to mention a mix of ORs and ANDs)? One approach might be using the U2U CAML Query Builder. The tool delivers a UI for formulating queries, but also exposes its CAML query builder API as a .Net assembly. Unfortunately, documentation on how to use the builder is sparse. Nonetheless, I went for the case of composing a query that OR together three tags:

         string ComposeCamlQuery() {
            Builder b = new Builder(CamlTypes.GetListItems);
            b.AddViewField("Title");
            b.AddViewField("MyField");
      
            bool addCombinerNode;
            foreach (string tag in new string[] { "tag1", "tag2", "tag3" })
               b.AddWhereField("MyField", tag, "LookupMulti",
                               "Or", out addCombinerNode);
            return b.CamlDocument.InnerXml;
         }
      

      The outcome is a rather strange looking CAML query with extra <And></And> and missing <Eq></Eq> tags. Funny thing is that this type of the query can be correctly composed through the UI, so most likely I’m not using the API correctly:

         <Where>
            <And>
               <And>
                  <Or>
                     <FieldRef Name="MyField" />
                     <Value Type="LookupMulti">tag1</Value>
                  </Or>
                  <Or>
                     <FieldRef Name="MyField" />
                     <Value Type="LookupMulti">tag2</Value>
                  </Or>
               </And>
               <Or>
                  <FieldRef Name="MyField" />
                  <Value Type="LookupMulti">tag3</Value>
               </Or>
            </And>
         </Where>
      

      I eventually abandoned the idea of using the CAML Query Builder API. Instead, a colleague pointed me to Waldek Mastykarz’s post on generating dynamic CAML queries. As long as all tags are to be either AND’ed or OR’ed together, Mastykarz provides an elegant solution, which I modified to be recursive, more generic, and read like an induction proof (at a negligible performance cost):

         ComposeCamlQuery(new[] { "tag1", "tag2", "tag3" },
                          "Or",
                          "<Where>{0}</Where>",
                          @"<Eq>
                             <FieldRef Name='MyField' />
                             <Value Type='LookupMulti'>{0}</Value>
                            </Eq>");
      
         string ComposeCamlQuery(IList<string> ops, string relOp,
                                 string query, string leaf) {
            return ops.Count == 1
               ? string.Format(query,
                               string.Format(leaf, ops[0]))
               : ComposeCamlQuery(ops.Skip(1).ToList(),
                                  relOp,
                                  string.Format(query,
                                     string.Format("<{0}>{1}{{0}}</{0}>",
                                     relOp,
                                     string.Format(leaf, ops[0]))),
                                  leaf);
         }
      

      Obviously, more complex code could be written for a mix of AND’s and OR’s. Before doing so, however, you probably want to evaluate other approaches for filtering list items:

      1. Formulate a simpler, too general query and post-process the result
      2. Formulate smaller queries and optionally merge the results and do post-processing
      3. Use the SharePoint Search API

      I also considered such alternatives as Linq to SharePoint or Caml.net, but although the first holds a lot of promise, it isn’t ready for prime time. As for the latter, its focus is more on composing type-safe queries than dynamic ones, so it’s more of a supplement to the first two alternatives.

      As for SharePoint Search, the downside is that search has to be configured and that the list must have been indexed for the outcome to be accurate. Depending on the frequency with which the list is modified, the time between incremental crawls, and the context in which the results are displayed, the search approach may or may not be the solution you’re looking for.

      Update, July 19: Added recursive code solution.

      • Share/Bookmark

      Tags: ,
      Posted in SharePoint | 2 Comments »