Sunday, September 13, 2009

Sporm - Out of the Depths of SharePoint's XML Hell

In my last post I described how a new open source tool called sporm significantly simplifies unit testing SharePoint. Making SharePoint unit testable is my absolute favorite feature of sporm because SharePoint is notoriously hard to unit test. But sporm provides other benefits as well and its ability to pull us out of the depths of verbose loosely typed XML hell and into LINQ excellence is next on my list of favorite features. So in this post I’ll describe the pre-sporm technique of querying with CAML, how to query data using sporm, and finally how sporm supports SharePoint’s unique architecture of allowing multiple content types per list and what that means to you.

Caml’s Are Ugly

Warning: if you’re new to SharePoint then what you’re about to see may shock and upset you. If, like me, you hate both XML and loose typing then you will agree that CAML is awful, but bear with me I promise sporm will make it better. Much better.

CAML or Collaborative Application Markup Language is how one queries for data in SharePoint. A simple query might look like this:

<Query>
  <
Where>
    <
And>
      <
And>
        <
Eq>
          <
FieldRef Name='First_Name' />
          <
Value Type='Text'>Lee</Value>
        </
Eq>
        <
BeginsWith>
          <
FieldRef Name='Last_Name' />
          <
Value Type='Text'>Rich</Value>
        </
BeginsWith>
      </
And>
      <
Leq>
        <
FieldRef Name='Dob' />
        <
Value Type='DateTime'>2009-01-01T00:00:00Z</Value>
      </
Leq>
    </
And>
  </
Where>
</
Query>

Simple right? ;) In case you didn’t catch the meaning from the slightly, uh verbose, query this asks for records with a First_Name of “Lee”, a Last_Name starting with “Rich” and a “Dob” less than or equal January 1 2009.

There are a couple of things to note about this query:

  • Field names are loosly typed. If Dob were ever renamed to DateOfBirth the query would fail at runtime (probably during a demo to a customer, or if you’re lucky during integration tests), but certainly not at compile time.
  • And is a binary operator. This forces explicit precedence and removes the need for parenthesis, but at the cost of readability.
  • Types must be explicitly defined. I guess this is necessary since it’s XML, but somehow I just don’t feel like this should be necessary.
  • It’s XML. Ok obviously, but the point is you have to type everything twice. <BeginsWith> </BeginsWith>. Ouch, so verbose, so angley, I so hate XML.

Now, there are tools that make this better. U2U’s free CAML Query Builder tool significantly improves the experience of querying SharePoint data.

But it makes you wonder if something is wrong when you need a tool to retrieve data from your data store. Do you typically use tools to assist when you’re writing SQL? Probably not. But like I said hang in there, sporm make things better.

Unlinq my Caml

If you were to write the same query as above using sporm it would look like this:

IQueryable<Employee> employees = GetEmployees().Where(e =>
      e.FirstName == "Lee"
      && e.LastName.StartsWith("Rich")
      && e.Dob <= new DateTime(2009, 1, 1));

Just a little easier to read than CAML, right? A couple of things to note:

  • Fields are strongly typed. If you were to rename FirstName the compiler would catch every single instance at design time.
  • Operators are standard C# operators. &&, .StartsWith(), and <= are all familiar and concise and use a standard, known precedence.
  • Types are standard C# types. You never have to explicitly say that "Rich" is a string, it just is.
  • Your query is actual C# code. The lambda (closure) and the fact that it uses deferred execution might throw off a junior developer in some scenarios, but the query is readable and works exactly the same as if you were querying in memory objects or querying a database with LINQ to SQL or the Entity Framework.

Nice! What's beautiful about this is that sporm converts your C# directly into CAML using C# 3.0's expression trees feature. What sporm can't convert to CAML it executes in memory transparently to you. Sporm uses log4net and outputs its CAML queries to the console by default, so it is a good idea to watch the output if you're concerned about performance.

Now the following isn’t relevant to the comparison with CAML, but I would be negligent if I didn’t explain how the GetEmployees() function works.

SP != DB B/C of Content Types

First of all GetEmployees looks like this:

private IQueryable<Employee> GetEmployees() {
      return MySPDataContext
            .GetCurrent()
            .GetList<Employees>()
            .OfType<Employee>();
}

How it works is that the static GetCurrent() method retrieves sporm’s context object, which knows how to query a SharePoint site, from either the web context or thread local storage; next the GetList() method tells sporm which list you want to query; and the OfType() method tells sporm which content type within the list you want to query. This last part is important because sporm supports SharePoint’s ability to have multiple content types per list, which other LINQ providers like LINQ to SharePoint do not. But what are content types and why should you care?

SharePoint’s List/Content Type architecture seems odd at first, but it allows interesting scenarios not available in a traditional database. For instance you might have a calendar list that contains multiple types of records (list items) in the same list. Your calendar list might contain some combination of the following three record types: meetings with unique fields like Organizer (a person); iterations with unique fields like DeployedToProduction (a Boolean); and actions with unique fields like RelatedEmployee (a reference to another list). This architecture allows SharePoint to view all three types of records in a single view: like a per month calendar view. The data might look like this:

Title Start End ContentType Organizer Deployed To Production Related Employee
Iteration16 1/12/09 1/19/09 Iteration   false  
Stakeholder Demo 1/19/09 3 PM   Meeting Lee Richardson    
Tag Trunk 1/19/09   Action     Lee Richardson

Sporm’s unique architecture supports this scenario by allowing you to retrieve iterations like this:

return MySPDataContext
      .GetCurrent()
      .GetList<Calendar>()
      .OfType<Iteration>();

Or get meetings from the same list like this:

return MySPDataContext
      .GetCurrent()
      .GetList<Calendar>()
      .OfType<Meeting>();

While you may only use multiple content types per list occasionally you can feel comfortable knowing that sporm will support you when you need it. The rest of the time you can arrange your architecture to accommodate your 90% scenario of one content type per list. I’ll discuss the architecture I’m using on my current project in my next post.

For now I hope this has clarified some of the benefits of using sporm, and I hope that you’ll consider using it on your next SharePoint project.

Wednesday, September 9, 2009

Unit Testing SharePoint - Past, Present, and Sporm

As I described in SharePoint: The Wild West of Software Development there is a serious problem when you develop for SharePoint: ensuring quality through unit testing is really, really hard. And that's where a new open source tool just released today called sporm (SharePoint Object Relational Mapper) comes in. While sporm provides many benefits besides simplified unit testing I wanted to focus on this topic first, because sporm's approach, which models the entity framework in the way it supports POCO's, is a unique feature not available with other SharePoint tools like LINQ to SharePoint.

I'll start by describing the unit testing problem, then provide a conventional solution with mocking, then finish with the elegance of a sporm based solution.

Simple Unit Testing Turns Ugly

I would bet anyone that attempts to develop a tiered solution with SharePoint ends up with entities that look something like this:

public class Employee {
      public SPListItem ListItem { get; private set; }

      /// <summary>
      /// The constructor requires that you pass in a list item
      /// that is used by your strongly typed properties
      /// </summary>
      public Employee(SPListItem listItem) {
            ListItem = listItem;
      }

      /// <summary>
      /// This provides your entities strong typing to hide
      /// the native weak typing in SharePoint
      /// </summary>
      public virtual string LastName {
            get { return (string)ListItem["LastName"]; }
            set { ListItem["LastName"] = value; }
      }
}

And if you have some function that you'd like to unit test like this:

/// <summary>
///
Typically returns "[LastName], [FirstName]" but skips
/// the comma if either is missing
/// </summary>
public string GetNameFormatted() {
      bool noFirst = string.IsNullOrEmpty(FirstName);
      bool noLast = string.IsNullOrEmpty(LastName);
      if (noFirst && noLast) return "";
      if (noFirst) return LastName;
      if (noLast) return FirstName;
      return string.Format("{0}, {1}", LastName, FirstName);
}


Now you face a major problem: LastName is tightly coupled with SPListItem. And SPListItem only contains internal constructors making it unmockable. If you keep the tight coupling and pass an SPListItem into the constructor then you need a connection to SharePoint and an employee list (table) and you need to create an employee list item (record), then you test it, and finally you ought to clean up after yourself and delete the list item. Talk about a simple unit test turned fragile integration test. Yuck.

Mocking the Solution

So at this point you're undoubtedly thinking it's time to mock. To do this you'll have to make a public constructor that takes no arguments and mark your properties virtual. Then you can write a test like this:

[TestMethod]
public void TestGetNameFormatted_FirstAndLastExist_LastCommaFirst() {
      MockRepository mocks = new MockRepository();
      Employee mockEmployee = mocks.StrictMock<Employee>();
      Expect.Call(mockEmployee.FirstName).Return("Lee").Repeat.Any();
      Expect.Call(mockEmployee.LastName).Return("Richardson").Repeat.Any();
      mocks.ReplayAll();
      string actual = mockEmployee.GetNameFormatted();
      mocks.VerifyAll();
      Assert.AreEqual("Richardson, Lee", actual);
}

And then if you're like me you absolutely hate your unit tests. Because let alone that you had to modify your production code to accommodate unit testing, more importantly your unit tests are now unreadable. All of them. Even in simple methods without dependencies your unit tests contain almost as much plumbing code as actual test code. Enter sporm.

Unmocking with Sporm

Sporm is an open source tool released on codeplex that was developed by fellow Near Infinity employee Joe Ferner (who, incidentally, is one of the most brilliant developers I know). While this is a version 1.0 product Joe has been using it on his project and independently I've been using it on my project for several months now, so it is relatively mature.

So how it works is that at design time sporm will read from your SharePoint site and generate partial classes for each of your list items and each of your content types. The generated classes don't inherit from anything, giving you the ability architect your solution how you see fit. And the properties are auto-properties, making the classes just pure POCO. Here's an example if you're interested:

[SPOrmContentType(Name="Employee")]
public partial class Employee : ISPOrmContentType {
      public static class FieldTitles {
            public const string FirstName = "First Name";
      }

      public static class FieldStaticName {
            public const string FirstName = "FirstName";
      }

      [SPOrmIdField()]
      public virtual int Id { get; set; }

      [SPOrmField(Title = FieldTitles.FirstName, StaticName = FieldStaticName.FirstName)]
      public virtual string FirstName { get; set; }
}

Since generated entities look like that, your entities can now look like this:

public partial class Employee {
      public string GetNameFormatted() {
            bool noFirst = string.IsNullOrEmpty(FirstName);
            bool noLast = string.IsNullOrEmpty(LastName);
            ...
      }
}

Now if that isn't a picture of beauty to you then you haven't been working with SharePoint. But wait, I hear you asking, how does this have anything to do with SharePoint? How would an instance of an employee be able to retrieve the data in a SharePoint ListItem? The answer is in the DataContext. In your production code you write something like this:

public static Employee FindById(int id) {
      return MySPDataContext
            .GetCurrent()
            .GetList<TList>()
            .OfType<TContentType>()
            .FirstOrDefault(c => c.Id == id);
}

While it might not look so pretty at first you can hide these details in a base type and the nice thing is that it does handle SharePoint's ability to have multiple content types per list (which LINQ to SharePoint does not do, incidentally). But for now all you need to know is that when you return an Employee thorugh MySPDataContext it subtypes your Employee class and returns you a proxy at runtime. The proxy overrides each property to return you the appropriate value in the SPListItem. And if you instantiate an Employee directly (as you would in unit testing) then you can use it like a pure POCO. If you're familiar with the entity framework this should sound extremely similar. The result is that you can write your unit tests like this:

[TestMethod]
public void TestGetNameFormatted_FirstAndLastExist_CommaSeparate() {
      Employee employee = new Employee {
            FirstName = "Lee",
            LastName = "Richardson"
      };
      string actual = employee.GetNameFormatted();
      Assert.AreEqual("Richardson, Lee", actual);
}

No SharePoint dependencies. No mocking. Nothing but pure, unadulterated unit test. Now that a thing of beauty.