sc_itemid Query String – Having a detrimental effect on the SEO

Leave a reply

If the sc_itemid query string has a valid sitecore item ID (and or sitecore path), sitecore ignores the URL path and sets the current item to the specified item.

This is generally used by the sitecore client tools i.e., preview, experience editor, debugging, etc. to specify the current item.

Unfortunately, one customer identified that the use of sc_itemid, was having a detrimental effect on the SEO for their websites, but why was sc_itemid present on their public available websites?

I believed the correct solution was to identify why sc_itemid was present in the URL and correct the issue.

Unfortunately, after identifying several issues relating to editor’s content, legacy system and legacy code, the customer informed me that they could not be resolved/fixed and asked me to find an alternative solution.

Solution

If it is not a sitecore client request and sc_itemid is present make a permanent redirect, following the these rules:

  • If the sc_itemid has a valid item id or path, make a permanent redirect to canonical URL for the item.
  • If the sc_itemid does not have a valid item id or path, strip the sc_itemid query string and make a permanent redirect to that URL.

Step 1 – httpRequestBegin Pipeline Processor

Introduce a pipeline processor to check URL’s and add it to the httpRequestBegin pipeline just before the LayoutResolver processor.

Define which site’s should be ignored, for example the sitecore client typically uses the shell site context.

Define which URL paths should be ignored, for example URL’s starting with /sitecore are usually for the sitecore client, see the config below for more typical examples.

<?xml version="1.0"?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <pipelines>
      <httpRequestBegin>
        <processor type="Feature.ErrorHandling.Infrastructure.Pipelines.QueryStringPermanentlyRedirectHttpRequestProcessor, Feature.ErrorHandling"
				           patch:before="processor[@type='Sitecore.Pipelines.HttpRequest.LayoutResolver, Sitecore.Kernel']"
				           resolve="true">
          <IgnoreSites hint="list:AddIgnoreSite">
            <!--list of all the sites to ignore-->
            <site>shell</site>
            <site>admin</site>
            <site>login</site>
            <site>service</site>
            <site>modules_shell</site>
          </IgnoreSites>
          <IgnorePaths hint="list:AddIgnorePath">
            <!--list of all the sites to ignore-->
            <site>/sitecore</site>
            <site>/api</site>
          </IgnorePaths>
        </processor>
      </httpRequestBegin>
    </pipelines>
  </sitecore>
</configuration>

Step 2 – Which Requests/URL’s to ignore

The most important part of the any pipeline processors is to ensure that it identifies requests to ignore and exits as soon as possible, as all request go through the httpRequestBegin pipeline.

Therefore the QueryStringPermanentlyRedirectHttpRequestProcessor must check the following and exit if one of the conditions is true.

  • Local path is null
  • Http Context is null
  • The pipeline is aborted
  • The query string sc_itemid is not present
  • The Site context is in the Ignore Sites list (see configuration above)
  • The URL path starts with the path in the ignore paths list (see configuration above)
    • Context.PageMode.IsNormal is not true i.e. It is a sitecore client request – editing, preview, experience, debugging, etc.
  • Is a request for a physical file

Step 3 – Abort the Pipeline & Permanent Redirect

The last step is simple, make the permanent redirect depending on the following logic:

  • If sc_itemid identifies a valid item, get the canonical URL and redirect.
  • If sc_itemid does not identifies a valid item, remove the sc_itemid querystring and redirect to the URL.

Remember to abort the pipeline first before redirecting and catch the ThreadAbortException, see the code below

          try
            {
                args.AbortPipeline();
                args.HttpContext.Response.RedirectPermanent(redirectUrl);
            }
            catch (ThreadAbortException ex)
            {
                // do nothing, as this is caused by the redirect
            }

Hope this helps, Alan

Is the custom Sitecore field type being used?

Leave a reply

Took over a 18 year old sitecore solution, which had a number of custom field types and before upgrading to Sitecore 10 of course we wanted to remove any unused functionality.

Unfortunately the customer had no idea if the field types where being used as there are over 900+ sites.

Problem

How to determine if a custom field type is in use?

I thought it would be easy using the link database, the search box in the shell and or the search in content editor.

Unfortunately when I searched for the ID of the field type I found nothing.

I assumed it was because the field type is defined in the core database and used in the master database.

Solution

Therefore it was time to look into the SQL database, to try and determine if the custom fields where in use.

First step was to determine how sitecore knows the field type for a field? I hoped there was a field type column on the following tables that are responsible for storing field values.

  • SharedFields
  • VersionedFields
  • UnversionedFields

Unfortunately sitecore does not save the field type in the SQL database as a dedicated column, in fact they store the type like any other field, as a value on the field.

The field responsible for storing what type of field a field is, is the the Type field from the Template Field template. See the image below, notice the source field is set to get values from the core database.

The Type field id is “{AB162CC0-DC80-4ABF-8871-998EE5D7BA32}” and as you can see in the image below we can then restrict our search to fields values of that field.

This is when I discovered that the value stored for the field is not the Guid of the field type, but the name :-(.

So the following SQL can be used to search for any field type and determine if the field is used.

The ItemId is the Id of the field which uses the custom field type.

SELECT *
  FROM [dtu_master_upgrade].[dbo].[SharedFields] 
    where FieldId = 'AB162CC0-DC80-4ABF-8871-998EE5D7BA32' and 
          Value like '%[NAME OF CUSTOM FIELD TYPE]%'

SELECT *
  FROM [dtu_master_upgrade].[dbo].[UnversionedFields] 
    where FieldId = 'AB162CC0-DC80-4ABF-8871-998EE5D7BA32' and 
          Value like '%[NAME OF CUSTOM FIELD TYPE]%'

SELECT *
  FROM [dtu_master_upgrade].[dbo].[VersionedFields] 
    where FieldId = 'AB162CC0-DC80-4ABF-8871-998EE5D7BA32' and 
          Value like '%[NAME OF CUSTOM FIELD TYPE]%'

From the field you can get the template and you can then check if the template is being used, by using the Links button on the Navigate Menu, see below.

Hope this helps, Alan

Progress bar with progress messages for long running processes.

Leave a reply

Sitecore has made it very easy from within the sitecore client to click a button do the following:

  • Start a long running process as a background job.
  • Showing a progress bar dialog and define the title.
  • Provides the ability to update the message as the job progresses in the background.
  • Hide the progress bar once the background job completes.

It seems to be a feature that has not been used much, so I hope this blog post will draw attention to it. I will assume that you already know how to add a custom button and execute code when it is clicked, using a customer command class, if not see here and or google it.

Step 1 – Setup

In the overridden Execute function for the custom command get the details you need, in our case the item id, language and database. Then call Client page start, which calls the Run function.

public class ExportBookCommand : Command
{
        public override void Execute(CommandContext context)
        {
            Assert.ArgumentNotNull(context, "context");
            Assert.IsNotNull(context.Items, "context items are null");
            Assert.IsTrue(context.Items.Length > 0, "context items length is 0");
            var item = context.Items[0];
            var parameters = new NameValueCollection
            {
                [Constants.Parameters.Keys.Id] = item.ID.ToString(),
                [Constants.Parameters.Keys.Language] = item.Language.ToString(),
                [Constants.Parameters.Keys.Database] = item.Database.Name
            };
            Context.ClientPage.Start(this, nameof(Run), parameters);
        }
        protected void Run(ClientPipelineArgs args)

Step 2 – Show progress bar

The Run function (see code below) is responsible for the following:

  • Do any validation and show an alert if required.
  • Build any classes your background job is going to need
    • We are exporting a book to multiple word word documents.
  • Call ProgressBox.Execute which is responsisble for the following:
    • Show the progress bar
    • Defines the title for the progress bar
    • Defines the initial message for the progress bar
    • Starts the background job and calls the function StartProcess todo the work.
      • Adds the Job to the current client context, which we use later.
protected void Run(ClientPipelineArgs args)
{
	Assert.ArgumentNotNull(args, "args");
	// Validate args, show an alert if not valid
	if(notValid(args))
	{
		SheerResponse.Alert("explain what went wrong"));
		return;
	}
    // create the data, for the background job
	var book = _bookFactory.Create(args.Parameters);
	Assert.IsNotNull(book, "Could not create book model");
	var bookArray = new object[] {book};
		// show progress bar and start background job
	ProgressBox.Execute("Job name, used as the inital message shown on the prgress bar",
		Translate.Text("The title of the progress bar"), "~/Network/32x32/server_into.png",
		StartProcess, bookArray);
}

private void StartProcess(params object[] parameters)
{
	Assert.ArgumentNotNull(parameters, "parameters");
	var book = parameters[0] as Models.Book;
	Assert.IsNotNull(book, "The first element in the object array is not a book");
	var exportBookService = new ExportBookService();
	exportBookService.ExportBook(book);
}

Step 3 – Update progress message

To update the messages for the progress bar is simple as the current client context has a Job property which makes it simple to add messages, relating to the progress of the process.

Context.Job.Status.Messages.Add($"{Translate.Text("export:progress")}{folder.Name}");

In our solution we showed a new message for each top level folder (which was a letter) we exported, see the image below, which is currently processing the letter B

I hope this blog post has helps to give the user a meaningful indication of what is happing when a process takes a long time t complete.

Sitecore and Hangfire

Leave a reply

A while ago I wrote a blogpost on Sitecore & Azure Durable Functions, which looked at off loading CPU/Memory intensive scheduled tasks.

I recently encountered the same issue on another Sitecore solution that was pure on premise and we could not use Azure functions, but don’t worry Hangfire to the rescue.

Hangfire is an easy way to perform background processing in .NET and .NET Core applications, and it supports fire and forget jobs, delayed jobs, scheduled jobs, dependant jobs, batchs of background jobs that are created atomically and considered as a single entity. Hangfire is free for commercial usage.

Of course if a job fails, it is possible to define how long the job should wait to retry and how many times.

The sitecore solution had a few main performance issues to resolve, sending out lots of add hoc emails, and processing data from an external legacy system that was super slow.

Solution

We decided to setup and additional server with hangfire running as a.net core website. Then within the sitecore solution you need to install the Hangfire Nuget package and configure the connection string to SQL (can also chose redis if you prefer).

When we wanted to send an email, we just sent the job to Hangfire, using the code below. 

var jobId = BackgroundJob.Enqueue(
    <ISendEmailService>(uis => uis.SendEmail(eMailDetails));

We also needed some recurring jobs i.e., to replace the scheduled tasks running within sitecore. We had to introduce a rest API to implement the sitecore specific functionality. But in fact, that part of the code was minimal in comparison to the integration with their custom CRM. So on the hangfire server, we added recurring job sync users from their CRM to Sitecore.

RecurringJob.AddOrUpdate<IUserService>(uis => uis.SynchronizeUsers(), synchronizeUsersTab);

Well for once not so much code, but just more an idea how to off load you sitecore CM server.

Cheeer’s, Alan

Wrong Language

Leave a reply

We have a very big Sitecore solution with over 3000 editors, unfortunately they spend a great deal of time creating a content in the wrong language.
Easy enough to fix a single item, but when each page has a lot of renderings, which in turn have a lot of data source items (see image below), the task is not so easy and very time consuming.

Solution

Provide the ability to select a given item and move all the language versions from one language to another and then delete the source language version.

I decided to add 2 buttons, and I was lucky as there was already a Data Migration chunk I could add the buttons to.

See this blog for a detailed introduction to adding buttons to the sitecore client.

In order to know which language to move from and to, I added a source and target parameter to the move language command.

This is very easy todo, as on the “click” field for small button you can add parameters, then when sitecore calls the Execute function the CommandContext has the values in the Parameters list, see the code below, which can then identify the source and target language.

public override void Execute(CommandContext context)
{
  var currentItem = context?.Items[0];
  if (currentItem == null)
    return;

  string source = context?.Parameters["source"];
  Assert.IsNotNullOrEmpty(source, $"Command parameter:source can not be empty or null");
  string target = context?.Parameters["target"];
  Assert.IsNotNullOrEmpty(target, $"Command parameter:source can not be empty or null");
  context.Parameters.Add("item", currentItem.ID.ToString());
  context.ClientPage.Start(this, nameof(Move), context.Parameters);
}

We need some configuration and in the solution there was already a data migration settings item that could store the following:

  • Modules Folder Template id
    • So I can identify which sub item contains the data source items and don’t iterate over the entire tree.
  • List of fields to ignore
    • i.e. revision, updated, updated by, owner, security, etc.

The code to move the language versions is quite simple.

  1. Get the current item, using the source language.
  2. Get the settings item.
  3. Get the items in the modules folder, by looking for any sub items that derive from the Modules Folder Template, then adding their descendants.
  4. For each item
    1. Get all the source versions (see code below)
    2. For each version create the new target language version
      1. For each field
        1. Skip all shared fields
        2. Skip fields that are in the Fields to Exclude list
        3. Copy the field
    3. Remove all the source language version
private void Copy(
        [NotNull] Database database,
        [NotNull] Item source, 
        [NotNull] Language target,
        [NotNull] IEnumerable<Field> languageMigrationExcludedFields)
        {
            Assert.ArgumentNotNull(source, nameof(source));
            Assert.ArgumentNotNull(target, nameof(target));
            Assert.ArgumentNotNull(languageMigrationExcludedFields, nameof(languageMigrationExcludedFields));

            foreach (var sourceVersion in source.Versions.GetVersions())
            {
                if (sourceVersion == null)
                    continue;
                var targetItem = database.GetItem(source.ID, target);
                var newTargetVersion = targetItem.Versions.AddVersion();
                Copy(sourceVersion, newTargetVersion, languageMigrationExcludedFields);
                sourceVersion.Versions.RemoveVersion();
            }
        }

Hope this helps, cheers Alan

Sitecore config disappeared?

Leave a reply

We had a Sitecore 9 solution that required the include files to be rearranged to align with a new azure deployment/provisioning setup using Terraform and to fix some duplication and inconsistencies.

Problem – Unicorn Sync Failed

When we enabled unicorn sync in the CD pipeline (see here for more details about Unicorn) the deploy failed?

Unfortunately (see image above) the configuration relating to the unicorn’s sc.variable‘s were missing?

Don’t have 2 <sitecore> elements

If there are 2 <sitecore> elements in a sitecore include file, even if they have mutely exclusive require statements Sitecore’s config merge ignores the file, without reporting an error!

Solution

There are 2 solutions to the problem

  1. Split the configuration into 2 files each with their own <sitecore> element.
  2. Merge the 2 <sitecore> elements into one element and move the require constraints to each sub element.

We decided to solution 2, see the result below.

Use patch:before=”prototypes” that way the new variables are shown at the top when showconfig.aspx is used (see image below) instead of half way down the page.

I hope this will help and no one else has to waste hours figuring this out, cheers Alan

Azure Functions – Json: Self referencing loop detected for property

1 Reply

Problem

Whilst developing a Rest API using Azure functions I got the error below.

 Newtonsoft.Json: Self referencing loop detected for property 'task' with type 'System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[System.Collections.Generic.IEnumerable`1[Feature.Order.Domain.TravelCard.OrderedTravelCard],Feature.Order.Domain.TravelCard.OrderedTravelCardRepository+<Get>d__4]'. Path 'stateMachine.<>t__builder'

As we had created a lot of new models and called a number of external API’s, which had a bad habit of changing their models without notification. I assumed incorrectly that one of the swagger files for the API’s was not correct.

But after much investigation I could not detect any issue with the data returned from the external API’s and the models match correctly their definitions.

Eventually I found out what the issue was, can you see what is wrong with the following code?

Solution

See the image above that a Task<…> is returned from the OrderedTravelCardRepository.Get() function.

I had forgotten to await the asynchronous call of the function, see the fixed code below.

Simple when you know how, simply awful when you don’t!

I hope this blog post helps others avoid the time I wasted on this issue.

Where did my request body data go?

Leave a reply

Problem

The solution had several API controllers (see image above) that expected data to be posted in the body. Then for no reason the body data was null, and therefore the controller started throwing argument null exceptions, and the front end of course stopped working.

The issue was caused by a new custom HttpRequestProcessor (see image below), which was called in the httpRequestBegin pipeline.

private void ProcessApi(HttpRequestArgs args)
{
      using (var mem = new MemoryStream())
      {                
           HttpContext.Current.Request.InputStream.CopyTo(mem);
           HttpContext.Current.Request.InputStream.Seek(0, SeekOrigin.Begin);
                
           using (var reader = new StreamReader(mem))
           {
                mem.Seek(0, SeekOrigin.Begin);
                var body = reader.ReadToEnd();

                if (string.IsNullOrEmpty(body))
                    return;

                var requestBody = JsonConvert.DeserializeObject<RequestBody>(body);
                _setLanguageService.SetLanguage(requestBody);
           }
      }            
 }

It needed the data from the request body, for some reason the setting the position back to 0, did not work?

Therefore the controller got null instead of the data contained in the body.

Solution

Therefore it was necessary to copy the stream contents into a memory stream, read the data from that stream, then deserialize the class. See the solution below.

I hope this helps, and this issue was found Sitecore 8, with a lot of customization, patches, code and modifications over the past 16 years.

Site Context for ApiControllers

Leave a reply

Now almost every Sitecore project has rest API’s and I am always shocked when the database, language etc. is hard coded and or additional configuration is added to define default the language, database etc.

Wouldn’t it be nice if you can define the site context for the controller?

Then you can define a site declaration, and or use an existing site that each controller should use and then use the language, database etc. that is define for the site language, database etc.

Solution

The SiteContextAttribute provides the ability to define which site an ApiController should use, for example in the picture above it is setup to use the “Person” site.

    public class SiteContextAttribute : ActionFilterAttribute
    {
        protected readonly string SiteName;

        public SiteContextAttribute(string siteName)
        {
            this.SiteName = siteName;
        }

        private SiteContextSwitcher _siteContextSwitcher;
        private LanguageSwitcher _languageSwitcher;
        private DatabaseSwitcher _databaseSwitcher;

        public override void OnActionExecuting(HttpActionContext actionContext)
        {
            base.OnActionExecuting(actionContext);

            var siteContext = SiteContext.GetSite(this.SiteName);

            _siteContextSwitcher = new SiteContextSwitcher(siteContext);
            _databaseSwitcher = new DatabaseSwitcher(siteContext.Database);
            _languageSwitcher = new LanguageSwitcher(LanguageManager.GetLanguage(siteContext.Language));
        }

        public override void OnActionExecuted(HttpActionExecutedContext actionContext)
        {
            _languageSwitcher?.Dispose();
            _databaseSwitcher?.Dispose();
            _siteContextSwitcher?.Dispose();

            base.OnActionExecuted(actionContext);
        }
    }

The code gets the site name, then gets the site context and setups the language, database and site context for the controller.

For example, see below it is it possible to use the Context.Database and also the language of the item will also be correct.

I hope this helps, Alan

How SQL Index Fragmentation will kill Sitecore’s Performance

1 Reply

Thought I wrote a blog post about this years ago, but apparently I didn’t.

Problem

Poor index maintenance is a major cause of decreased SQL Server performance, which in turn will impact your Sitecore’s performance. The Sitecore databases contains tables with numerous entries, that get updated frequently, therefore high index fragmentation will occur.

Detecting SQL Server index fragmentation

The following script displays the average fragmentation, and as a help generates the SQL query to fix it.

SELECT OBJECT_NAME(ind.OBJECT_ID) AS TableName,
ind.name AS IndexName, indexstats.index_type_desc AS IndexType,
indexstats.avg_fragmentation_in_percent,
'ALTER INDEX ' + QUOTENAME(ind.name) + ' ON ' +QUOTENAME(object_name(ind.object_id)) +
CASE WHEN indexstats.avg_fragmentation_in_percent>30 THEN ' REBUILD '
WHEN indexstats.avg_fragmentation_in_percent>=5 THEN 'REORGANIZE'
ELSE NULL END as [SQLQuery] -- if <5 not required, so no query needed
FROM sys.dm_db_index_physical_stats(DB_ID(), NULL, NULL, NULL, NULL) indexstats
INNER JOIN sys.indexes ind ON ind.object_id = indexstats.object_id
AND ind.index_id = indexstats.index_id
WHERE
--indexstats.avg_fragmentation_in_percent , e.g. >10, you can specify any number in percent
ind.Name is not null
ORDER BY indexstats.avg_fragmentation_in_percent DESC

Below you can see the typical result of running the script above. I was shocked as the majority of indexes on my local SQL server where over 99%.

Solution

The script above generates the SQL statements needed to defragment the affected indexes, so you can automate the defragmentation process, using SQL Server Maintenance Plans.

Anyway I hope this helps keeping you sitecore solution running at its best, Alan