Tag Archives: Performance

SolR

Introduce a (SolR) Sitecore Search Abstraction

After my previous post on Supporting Integrations, I received a few comments asking why was SolR was in the integration’s module group, as it is part of the sitecore API.

In this blog i will explain why and in more detail how to isolate a SolR integration.

Yes Sitecore Search is part of the Sitecore API, but it relies on an 3rd party system! Please read my previous post about why you need to identify, separate and isolate modules with external dependencies, as Sitecore Search API faces exactly the same challenges.

With the bonus that there are 3 supported implementations (Lucene, SolR and Azure Search) which are almost the same, but not quite!

Sitecore Search Issues

In most of the helix-based solution I have seen indexing is implemented in the framework layer which provides some helper extensions. Then each feature uses indexing module with Sitecore Search API to implement their requirements. This typically leads to the following issues:

  • Duplicated code across features
  • No clear definition of the indexing/constraints/sorting requirements for the solution.
  • Non-consistent implementation across the solution i.e. Predicate builder vs LINQ.
  • Optimization is difficult.

With each feature implement their indexing requirements, it leads to duplicated code as it feature needs to build the query to add sitecore root item, base templates, language etc. for each request, before adding the feature specific part of the query.

Therefore when fixing a bug or performance issues you must track down all the places where Search is used and then determine if they require the same fix and or the optimization.

How to abstract away the SolR Search Implementation

  • Identify the indexing requirements
    • Introduce an abstraction in the foundation layer (Indexing).
  • Create the implementation (Solr Indexing) that implements the abstraction define by Indexing in the foundation layer.
    • Address the sorting issues (i.e. different items templates have different date fields)
  • Let the features use the indexing abstractions (i.e. Course, News, Calendar, etc.)

Identify the indexing requirements

There are 3 main components to define the indexing requirements constraints, pagination & sorting.

Constraints

Constraints define what the filters can be applied to reduce the number of items that are returned. In this example it will be possible to apply the following constraints:

  • Location in tree sitecore (i.e. site specific news folder, all content, etc.)
  • Language (i.e. return items with an English language version)
  • Template, i.e. does the item inherit from a specific template (i.e. news, calendar, etc.)
  • Taxonomy – return items based on their categorization (i.e. football, skiing, etc.)

Pagination

Defines the number of search result per page and which page you require.

Sorting

Is responsible for defining what is used to sort the result items and the direction (ascending or descending), for example using date to get the 10 latest news.

If you want to sort by date, one challenge is to determine how to sort he results, as different pages will have different fields. Some pages have no date apart from created/updated, news normally has a specific news date and calendar events have start/end dates.

The SolR implementation must NOT KNOW ABOUT PAGE TYPES, I will write a blog post soon with a solution.

The following code defines the indexing requirements.

public interface IConstraint
{
    Item RootItem { get; }
    Language Language { get; }
    ID BaseTemplate { get; }
    IEnumerable<Category> Categories { get; }
}
public interface IPagination
{
    int Number { get; }
    int Size { get; }
}
public enum SortDirection
{
    Ascending,
    Descending
}

Then we need to define the result of making a search and a repository to make the search

public interface IPagedSearchResult
{
    IEnumerable<Item> Results { get; }
    IPagination Pagination { get; }
    int TotalHits { get; }
    bool HasMoreResults { get; }
}
public interface IPagedSearchResultRepository
{
    IPagedSearchResult Get([NotNull] IConstraint searchConstraint, [NotNull] IPagination pagination, SortDirection sortDirection);
}

The definition of the search result could of been type safe, i.e. return a model of type T instead of the Item, but I wanted to keep the example simple and not use a specific binding framework.

Anyway I hope this post will help, Alan

Sitecore client and logon is very slow (properties table AGAIN)

The problem

I was at a customer to help identify why their Sitecore 7.2 client and logon was so slow!

Investigation

There are a number of things I look at when the Sitecore client starts to run slowly:

  1. Slow Event handlers (save, rename, create, etc)
  2. Slow pipeline processors (usually don’t check that it is a sitecore specific requests)
  3. History, Publish queue or Event queue have to many entries, see my blog on how to fix this.
  4. Property change events flooding the Event Queue in the Core database, see my blog on how to fix this

But after ensuring all the previous issues were not the problem, I found a new issue with the properties table.

Properties Table flooded with SC_Ticket entries

The properties table in the core database had over 500000 entries, it was filled with SC_TICKET_xxx entries.

tickets

Unfortunately the properties table does not have a created date column, so I could not write an SQL script to purge all the entries that where more than X days old.

I noticed that in the value column there was a time-stamp embedded in the Value field. My initial solution was to could create an sitecore agent to do the following:

  1. iterate over all the entries in the properties table
  2. Parse the value for SC_Ticket entries
  3. Remove all the entries that were older than X days.

I knew Sitecore must have a class, which had created all these entries. So using DotPeek I started my search and found the TicketManager class. The TicketManager even had a IsTicketExpired function.

is expired function

Solution

I found that there is already a Sitecore agent that checks for any tickets that are expired and removes them. It is called the CleanupAuthenticationTicketsAgent for some reason this was not in the web.config, but it is easy enough to add see below.

agent

But the important step is to reduce the number of days to keep the tickets as the default is 180. The Authentication.ClientPersistentLoginDuration setting is responsible for determining how long before the ticket should expire (see the IsTicketExpired function in the image above).

I set Authentication.ClientPersistentLoginDuration to 5 and it reduced the number of entries in the properties table to around 500, and then sitecore client and logon was much faster.

Prior to writing this post I wasn’t aware but the is a blog about how sitecore sessions can expire.

Sitecore – property change events flooding the Event Queue in the Core database

I had an issue that the EventQueue table in the Core database which was being flooded with events, which in turn was responsible for causing bad performance.

This was caused by the fact that each time the Lucene indexes are updated Sitecore raises the database:propertychanged event.

event queue core datbase

 

The default implementation subscribes to the event and adds it to the EventQueue table in the core database. I set the “Days To Keep” events to 1 day – but it still inserts over 100000 events a day 😦

Sitecore have registered this as a bug, but as yet there is no fix for 7.5 or 8.

Solution

This is not very nice at all, but  it was the best we could do as a workaround for this issue. The credit for this solution must go to my colleague Peter Wind (@peterwind) who is created the hack!

Use reflector to get the code for the default Sitecore.Eventing.Remote.RemoteEventMap class (defined in Sitecore.Kernel.dll), and create your own class, see below.

own class

Modify the SetupGlobalEventSubscribers function, so you subscribe to the database:propertychanged event – but do not add the event to the EventQueue, see below.

code change

The last step is to update the configuration in the web.config to use your new and improved class.

config

note: I have the support case number in the namespace & DLL name – so when the bug is fixed, I can remove this code and in addition any other developer at Pentia can see the history related to this support issue and why I have done such a nasty fix.

Anyway I hope this helps somebody out there 🙂