Derek Dysart

Indie Sitecore Developer

Menu Close

Troubleshooting LINQ to Sitecore performance in Coveo

I recently was trying to troubleshoot some performance issue in the search function of a Sitecore 7.5 site I was working on. The search results were served via an MVC controller action that returned JSON to the front end code. We were seeing very large request times, and one of the culprits was thought to be item access being performed after we received results back from Coveo. (there is a need to perform a lot of distance calculation on the results returned to Coveo, and we’ve yet to find an elegant way to offload this to the Coveo servers)

To verify this theory, I ran some tests with a performance profiler attached to the w3wp process. The results of the profiler pointed to the LINQ processing done by Coveo, and expanding the most expensive call stacks, the bulk of the time was spend in the XMLSerializer. Which I thought was odd.

After a bit more digging, the Coveo ContentSearch provider for Sitecore uses Coveo’s SOAP interface under the covers. As such, since our queries were returning large result sets, there was a lot of XML to shuttle through all of the SOAP proxy code. Turns out, Coveo addresses this in their documentation regarding optimizing LINQ performance.

Of the two options, optimizing the fields returned in a query via the CoveoQureyFieldPipeline is ultimately what we chose. This pipeline allows you to specify which fields are returned during LINQ to Sitecore calls to the index. This settings is global, so you’ll need to specify all of the fields that all of your LINQ queries need. What is sort of unfortunate is if calling the SOAP interface directly, you can specify the fields on a per query basis.

Ultimately, if we can move some more of our distance math to the Coveo servers, we can add Skip() and Take() calls to the LINQ query to further reduce the amount of data serialized in the SOAP call, but given the nature of the data in this particular situation, I don’t know that will be possible.

Field Name Collisions in Sitecore Powershell Extensions

I had a situation where I needed to do a bulk update to a whole bunch of dictionary items that were part of a translation dictionary. On the dictionary entry template, there is a field called ‘Key’ which is also the name of a property on the Sitecore.Data.Items.Item class. I turns out that any field names that collide with properties on this object get an underscore in front of their name. For example, where I needed to get at the ‘Key’ field, I simply needed to write my Powershell command as:

Get-ChildItem -Recurse | Where-Object {$_.TemplateName -e "Dictionary Entry"} | ForEach-Object {$_._Key = $_.Name}

This particular PSE command will reset the Key field of all the Dictionary Entry items to its name.

Get Rendered HTML From Sitecore Item

I thought I’d cross post this answer of mine from StackOverflow, as it is useful in a number of contexts. As I mentioned there, the code was cribbed from a similar situation where I needed a URL to feed to a PDF to HTML library, which behind the scenes fired up IE on the server and hit the site as an anonymous user. This needed to be launched from within the the Content Editor, but since the PDF library hit the site anonymously, I needed to silently login the user.

This way we could pass a limited time security token via the query string.

First we need an extension method on the User to create the token and a way to check if the token is valid:

public static class UserExtensions
    public const string TokenKey = "UserToken";
    public const string TokenDateKey = "UserTokenDate";

    public static ID CreateUserToken(this User user)
        if (user.IsAuthenticated)
            var token = ID.NewID;
            user.Profile.SetCustomProperty(TokenKey, token.ToString());
            user.Profile.SetCustomProperty(TokenDateKey, DateTime.Now.ToString());
            return token;
            return ID.Null;

    public static bool IsTokenValid(this User user, string token, TimeSpan maxAge)
        var tokenId = ID.Null;
        if (ID.TryParse(token, out tokenId))
            var minDate = DateTime.Now.Add(-maxAge);
            var tokenDateString = user.Profile.GetCustomProperty(TokenDateKey);
            var tokenDate = DateTime.MinValue;

            DateTime.TryParse(tokenDateString, out tokenDate);

            if (tokenDate < minDate)
                return false;

            var storedToken = user.Profile.GetCustomProperty(TokenKey);
            var storedTokenId = ID.NewID;
            if (ID.TryParse(storedToken, out storedTokenId))
                return storedTokenId == tokenId;

        return false;

Next we patch in a HttpRequestProcessor to look for the token:

public class SilentUserLogin : HttpRequestProcessor
    public TimeSpan MaximumAge

    public override void Process(HttpRequestArgs args)
        var userValue = args.Context.Request.QueryString["user"];
        var tokenValue = args.Context.Request.QueryString["token"];

        if (!string.IsNullOrEmpty(userValue) && !string.IsNullOrEmpty(tokenValue))
            // find user
            var user = User.FromName(userValue, AccountType.User);
            if (user != null)
                // Check token is valid
                if ((user as User).IsTokenValid(tokenValue, MaximumAge))
                    // log user in
                    AuthenticationManager.Login(user as User);
                    Log.Audit("User token has expired for user: '{0}'".FormatWith(user.Name), this);
                Log.Audit("Failed to locate auto login user " + userValue, this);

Patch this in with a config file:

<configuration xmlns:patch="">
                <processor type="Namespace.SilentUserLogin,Assembly" patch:after="*[@type='Sitecore.Pipelines.HttpRequest.StartMeasurements, Sitecore.Kernel']">

Finally, when we need to use this (either to pass a URL to the PDF library, or in the case of the question to WebClient or HtmlAgility), we build the URL as follows:

var token = Sitecore.Context.User.CreateUserToken();

var url = new UrlString();
url.HostName = HttpContext.Current.Request.Url.Host;
url.Protocol = HttpContext.Current.Request.IsSecureConnection ? "https" : "http";
url.Path = "/";

url["sc_itemid"] = myItem.ID.ToString();
url["sc_lang"] = myItem.Language.ToString();

// Add parameters to allow accessing the master DB
url["user"] = Sitecore.Context.User.Name;
url["token"] = token.ToString();

// Call the url here

Disabling the Sitecore Analytics Cookie

A while back my colleague Nick pointed out this message on Twitter:

We’d done this for a client of ours and I thought I’d help out. Basically for this, you need to tap the ASP.NET request process and hook the HttpContent::EndRequest event. For this we created an HttpModule we specified in the web.config:

      <add type="ClearCookieHttpModule, AssemblyName" name="ClearCookieHttpModule"/>

The class then looks like this:

public class ClearCookieHttpModule : IHttpModule
        public void Dispose() { }

        public void Init(HttpApplication context)
                //hook end request
                context.EndRequest += new EventHandler(OnEndRequest);

        void OnEndRequest(object sender, EventArgs e)
                if (clearCookies)
                        HttpApplication app = (HttpApplication)sender;
                        var request = app.Request;
                        foreach (var cookieName in response.Cookies.AllKeys)
                                if (cookieName == "SC_ANALYTICS_GLOBAL_COOKIE" ||

Basically, if the clearCookies boolean is set, we remove the Sitecore Analyitcs cookies from the cookie collection.

I originally shared this code via pastebin, but realized it probably deserved a better home (that didn’t expire.)

Styling ASP.NET Validation Controls

If you are using the intrinsic ASP.NET Validation controls on a Web Forms project, you’ll notice they always default to being red. While this is all fine and good since red for warnings makes sense. What if you (or your design team) want them to be a different color? If you look at the rendereded HTML, you’ll see it makes the text red by using inline CSS:

<span id="ctl09_uxValidatorPostalCode" title="Required Field" class="scfRequired" style="color:Red;">*</span>

Where this becomes an issue is if you try to style this via an external stylesheet (generally a good practice):

.scfRequired {
color: #005598;

Since there is a color specified in the inline CSS, it will take precedent. We could add !important to our external stylesheet, but that sort of feels like a hack. What we really want to do is get ASP.NET to stop adding the inline style. Turns out this isn’t that hard to do. Just add a blank ForeColor attribute to your validator:

<asp:RequiredFieldValidator ID="uxValidatorPostalCode"
ToolTip="Required Field"
ErrorMessage="Please Enter your ZIP/Postal Code"></asp:RequiredFieldValidator>

No inline CSS colors will be output and your external stylesheet rules will apply. I’m sure this isn’t news to many, but none of the first few pages of search results seemed to point this out. This applies for all of the validation controls including the ValidationSummary control which seem to have a predisposition to red.

© 2015 Derek Dysart. All rights reserved.