Pay attention to your index exclusions

I hit an interesting issue recently: Some code that worked fine on a QA instance of Sitecore had been deployed for UAT and was now failing with an odd error message. Whilst this issue was entirely our fault, there wasn’t much in Google about the error messages I was seeing, so I’m trying to correct that problem today…

The issue

The code in question was runing on a Sitecore 7 instance. It fell over on UAT with this error:

[ArgumentNullException: Value cannot be null.Parameter name: key]
  System.ThrowHelper.ThrowArgumentNullException(ExceptionArgument argument) +49
  System.Collections.Generic.Dictionary`2.FindEntry(TKey key) +14545882
  System.Collections.Generic.Dictionary`2.TryGetValue(TKey key, TValue& value) +20
  Sitecore.ContentSearch.ContentSearchManager.GetIndex(String name) +52
  Sitecore.ContentSearch.ContentSearchManager.CreateSearchContext(IIndexable indexable) +17

That’s not a very helpful error, so I spent some time tracking it down. From the stack trace you can see that the problem seems to be in translating an IIndexable into a search context, but it doesn’t say anything helpful about why…

And under the surface, the code which was crashing looked something like:

var root = Sitecore.Context.GetItem("{08D9E0A1-BB72-48FD-AAB2-EFCD6F3B3C92}");
using (var context = ContentSearchManager.CreateSearchContext(new SitecoreIndexableItem(root)))
{
   // run some search queries starting from the "root" item...
}

The cause

A bit of digging with my old friend ILSpy lead to me working out that the exception being thrown here is because CreateSearchContext() above does two things:

public static IProviderSearchContext CreateSearchContext(IIndexable indexable)
{ 
  return GetIndex(indexable).CreateSearchContext(SearchSecurityOptions.EnableSecurityCheck);
}

It translates the IIndexable into a valid index for that item. And then it creates a search context on that index.

The exception gets thrown because the translation returns null – that item doesn’t map to any index. Under the surface, there’s a pipeline that gets run to do this translation, and it iterates all the defined indexes. And asks them in turn if they contain the item in question.

In the scenario I was looking at, all indexes returned “no” when asked this, and the code in the pipeline returns null when that happens… So later on when Sitecore tries to fetch its index by looking up the index based on a null name, you get the exception above.

Some further digging found a key difference between the configurations of the “working” servers and the “broken” one:

• On the machines where it worked Sitecore’s search indexes were configured as “index all templates, except some specific ones”
• But the servers that were broken were configured as “exclude all templates, except some specific ones”…

So the real cause of this issue was that the Sitecore Item passed into ContentSearchManager.CreateSearchContext() was based on a template that was specifically excluded from search indexing. As soon as I fixed this, the code started working.

Conclusions

Well, the obvious conclusions are: 1) Don’t exclude items you’re going to call ContentSearchManager.CreateSearchContext() on from your indexes. And 2) Manage your configuration across deployments better than this…

Having done some further testing, the error message above seems specific to older versions of Sitecore. I’ve tested this on V9.1 as well, and it still crashes in this scenario, but it shows a different exception:

[NullReferenceException: Object reference not set to an instance of an object.] 

 Sitecore.ContentSearch.SitecoreItemCrawler.IsExcludedFromIndex(SitecoreIndexableItem indexable, Boolean checkLocation) +69
  Sitecore.ContentSearch.SitecoreItemCrawler.GetContextIndexRanking(IIndexable indexable) +113
  System.Linq.WhereSelectListIterator`2.MoveNext() +116
  System.Linq.Enumerable.Min(IEnumerable`1 source) +72

 Sitecore.ContentSearch.AbstractSearchIndex.Sitecore.ContentSearch.Pipelines.GetContextIndex.IContextIndexRankable.GetContextIndexRanking(IIndexable indexable) +119
  Sitecore.ContentSearch.Pipelines.GetContextIndex.<>c\_\_DisplayClass6\_0.<RankContextIndexes>b\_\_0(ISearchIndex i) +71
  System.Linq.WhereSelectEnumerableIterator`2.MoveNext() +237
  System.Linq.Buffer`1..ctor(IEnumerable`1 source) +280
  System.Linq.<GetEnumerator>d__1.MoveNext() +115
  System.Linq.Buffer`1..ctor(IEnumerable`1 source) +280
  System.Linq.Enumerable.ToArray(IEnumerable`1 source) +89

 Sitecore.ContentSearch.Pipelines.GetContextIndex.FetchIndex.GetContextIndex(IIndexable indexable, GetContextIndexArgs args) +699

 Sitecore.ContentSearch.Pipelines.GetContextIndex.FetchIndex.Process(GetContextIndexArgs args) +48 (Object , Object ) +13
  Sitecore.Pipelines.CorePipeline.Run(PipelineArgs args) +483
  Sitecore.Pipelines.DefaultCorePipelineManager.Run(String pipelineName, PipelineArgs args, String pipelineDomain, Boolean failIfNotExists) +235
  Sitecore.Pipelines.DefaultCorePipelineManager.Run(String pipelineName, PipelineArgs args, String pipelineDomain) +21
  Sitecore.Abstractions.CorePipelineWrapper.Run(String pipelineName, PipelineArgs args) +73

 Sitecore.ContentSearch.Pipelines.GetContextIndex.GetContextIndexPipeline.Run(ICorePipeline pipeline, GetContextIndexArgs args) +38

The trace is a bit more obvious – but for people new to Sitecore’s index configuration process that still might be confusing. So if you’re seeing either of the error messages shown above, check what you’re excluding from your indexes…