C# Parallel.ForEach() on SPListItemCollection causes exception (0x80010102)

Question

In my ASP.NET MVC application, I am trying to retrieve all items in a list with the version history, and then cast them to a custom object. To do this, I am using Microsoft.SharePoint.

I was initially doing this the following way:

Util.GetSPItemCollectionWithHistory method:

public static SPListItemCollection GetSPItemCollectionWithHistory(string listName, SPQuery filterQuery)
{
    using (SPSite spSite = new SPSite(sp_URL))
    {
        using (SPWeb spWeb = spSite.OpenWeb())
        {
            SPList itemsList = spWeb.GetList("/Lists/" + listName);
            SPListItemCollection listItems = itemsList.GetItems(filterQuery);

            return listItems;
        }
    }
}

GetSPObjectsWithHistory method:

protected static List<SPObjectWithHistory<T>> GetSPObjectsWithHistory(SPQuery query = null, List<string> filters = null)
{
    List<SPObjectWithHistory<T>> resultsList = new List<SPObjectWithHistory<T>>();

    Type objectType = typeof(T);
    string listName = "";

    query = query ?? Util.DEFAULT_SSOM_QUERY;

    if (objectType == typeof(SPProject)) listName = Util.PROJECTS_LIST_NAME;
    else if (objectType == typeof(SPTask)) listName = Util.TASKS_LIST_NAME;
    else throw new Exception(String.Format("Could not find the list name for {0} objects.", objectType.Name));

    SPListItemCollection results = Util.GetSPItemCollectionWithHistory(listName, query);
    foreach (SPListItem item in results)
    {
        resultsList.Add(new SPObjectWithHistory<T>(item, filters));
    }

    return resultsList;
}

SPObjectWithHistory Class constructor:

public SPObjectWithHistory(SPListItem spItem, List<string> filters = null)
{
    double.TryParse(spItem.Versions[0].VersionLabel, out _currentVersion);
    History = new Dictionary<double, T>();

    if (spItem.Versions.Count > 1)
    {
        for (int i = 1; i < spItem.Versions.Count; i++)
        {
            if (filters == null)
                History.Add(double.Parse(spItem.Versions[i].VersionLabel), SPObject<T>.ConvertSPItemVersionObjectToSPObject(spItem.Versions[i]));
            else
            {
                foreach (string filter in filters)
                {
                    if (i == spItem.Versions.Count - 1 || (string)spItem.Versions[i][filter] != (string)spItem.Versions[i + 1][filter])
                    {
                        History.Add(double.Parse(spItem.Versions[i].VersionLabel), SPObject<T>.ConvertSPItemVersionObjectToSPObject(spItem.Versions[i]));
                        break;
                    }
                }
            }
        }
    }
}

This way the code works, but it is extremely slow on large lists. One of the lists has over 80000 items in it, and creating one SPObjectWithHistory item takes about 0.3 seconds, due to the logic in the constructor.

To speed up the process, I wanted to use Parallel.ForEach instead of a regular foreach.

My GetSPObjectsWithHistory was then updated to this:

protected static List<SPObjectWithHistory<T>> GetSPObjectsWithHistory(SPQuery query = null, List<string> filters = null)
{
    ConcurrentBag<SPObjectWithHistory<T>> resultsList = new ConcurrentBag<SPObjectWithHistory<T>>();

    Type objectType = typeof(T);
    string listName = "";

    query = query ?? Util.DEFAULT_SSOM_QUERY;

    if (objectType == typeof(SPProject)) listName = Util.PROJECTS_LIST_NAME;
    else if (objectType == typeof(SPTask)) listName = Util.TASKS_LIST_NAME;
    else throw new Exception(String.Format("Could not find the list name for {0} objects.", objectType.Name));

    List<SPListItem> results = Util.GetSPItemCollectionWithHistory(listName, query).Cast<SPListItem>().ToList();
    Parallel.ForEach(results, item => resultsList.Add(new SPObjectWithHistory<T>(item, filters)));

    return resultsList.ToList();
}

When I now try to run the application, however, I receive the following exception at the Parallel.ForEach:

Message: One or more errors occurred.

Type: System.AggregateException

StackTrace:

at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)

at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)

at System.Threading.Tasks.Parallel.ForWorker[TLocal](Int32 fromInclusive, Int32 toExclusive, ParallelOptions parallelOptions, Action'1 body, Action'2 bodyWithState, Func'4 bodyWithLocal, Func'1 localInit, Action'1 localFinally)

at System.Threading.Tasks.Parallel.ForEachWorker[TSource,TLocal](IEnumerable'1 source, ParallelOptions parallelOptions, Action'1 body, Action'2 bodyWithState, Action'3 bodyWithStateAndIndex, Func'4 bodyWithStateAndLocal, Func'5 bodyWithEverything, Func'1 localInit, Action'1 localFinally)

at System.Threading.Tasks.Parallel.ForEach[TSource](IEnumerable'1 source, Action'1 body)

at GetSPObjectsWithHistory(SPQuery query, List`1 filters) in...

InnerException:

Message: Attempted to make calls on more than one thread in single threaded mode. (Exception from HRESULT: 0x80010102 (RPC_E_ATTEMPTED_MULTITHREAD))

Type: Microsoft.SharePoint.SPException

StackTrace:

at Microsoft.SharePoint.SPGlobal.HandleComException(COMException comEx)

at Microsoft.SharePoint.Library.SPRequest.SetVar(String bstrUrl, String bstrName, String bstrValue)

at Microsoft.SharePoint.SPListItemVersionCollection.EnsureVersionsData()

at Microsoft.SharePoint.SPListItemVersionCollection.get_Item(Int32 iIndex)

at line of double.TryParse(spItem.Versions[0].VersionLabel, out _currentVersion); in the SPObjectWithHistory constructor.

InnerException:

Message: Attempted to make calls on more than one thread in single threaded mode. (Exception from HRESULT: 0x80010102 (RPC_E_ATTEMPTED_MULTITHREAD))

Type: System.Runtime.InteropServices.COMException

StackTrace:

at Microsoft.SharePoint.Library.SPRequestInternalClass.SetVar(String bstrUrl, String bstrName, String bstrValue)

at Microsoft.SharePoint.Library.SPRequest.SetVar(String bstrUrl, String bstrName, String bstrValue)

Would there be anyone who knows how I could get my code to work?

Thanks in advance!

It sounds like the API is not thread safe as it goes out to COM. Simply don't do multiple threads on it. — Daniel A. White
you aren't really doing much to improve the performance anysway. — Daniel A. White
I think it would actually be a lot faster if I could use Parallel.ForEach, since the application would perform resultsList.Add(new SPObjectWithHistory<T>(item, filters)); (+-0.3 seconds) multiple times simultaniously. On a collection of 80000 items, that will make quite an impact. Even performing it on 2 parallel threads instead of one would make a huge difference. — DylanVB
COM is single threaded unless the code explicitly indicates otherwise, to avoid all the problems inherent with multithreading. This component does not indicate it's safe for threading, so it's not safe for threading, no matter how much you want it to be. Consider using lazy loading -- is it really necessary to retrieve all 80,000 items of that list item up front, for example? What user will browse that? Even if you want custom objects, you could store the necessary referral data in a custom collection and materialize/retrieve these on demand. — Jeroen Mostert
@JeroenMostert Thanks for the clarifying comment. Unfortunately, lazy loading is not an option in my case. It looks like I will have to make a different approach. — DylanVB

DylanVB DylanVB · Accepted Answer · 2018-01-16T10:16:52

Apparently, what I was trying to do is not possible. The Microsoft.SharePoint namespace's SP objects are not thread safe, like @JeroenMostert stated.

COM is single threaded unless the code explicitly indicates otherwise, to avoid all the problems inherent with multithreading. This component does not indicate it's safe for threading, so it's not safe for threading, no matter how much you want it to be. Consider using lazy loading -- is it really necessary to retrieve all 80,000 items of that list item up front, for example? What user will browse that? Even if you want custom objects, you could store the necessary referral data in a custom collection and materialize/retrieve these on demand.

Since lazy loading was not an option for me, I decided to split my logic into batches (using System.Threading.Task), which each execute the code from my original post (with the SPQuery.Query changing for every batch). After that, the results from my GetSPObjectsWithHistory are merged into a single list.

C# Parallel.ForEach() on SPListItemCollection causes exception (0x80010102)

1 Answers