Enumerating AppFabric cache throws NullReferenceException

Question

We're using Windows Server AppFabric 1.1 (rather than Azure) and I think we've managed to pollute the cache such that we get some odd results.

We're using a read-through provider so when if the key isn't in the cache, the read-through provider goes to the database, populates the cache and returns the value to the client. (As an aside, the team at Microsoft decided to require that the read-through provider should live in the GAC - this has caused no end of pain so I'd recommend against using read-through. I hope they reconsider this choice.)

The symptom of my problem is that when I try and get an item from the cache, it returns null when I would expect the read-through provider to fetch the value from the database. Repeated calls return null ruling out any transient problem with the database. If I remove the item from the cache and try to get it again, it succeeds.

To try and understand the extent of the problem, I thought I'd enumerate all of the items in the cache like this:

foreach (string regionName in cache.GetSystemRegions())
{
    var objectsInRegion = cache.GetObjectsInRegion(regionName);

    try
    {
        foreach (var keyValuePair in objectsInRegion)
        {
            var result = string.Format(
                "Key: {0}. Value: {1}", keyValuePair.Key, keyValuePair.Value);
            Console.WriteLine(result);
        }
    }
    catch (NullReferenceException)
    {
        Console.WriteLine("Unable to get key value pair");
    }
}

The exception handling is there because the cache was consistently throwing a null reference exception when enumerating objectsInRegion for particular regions.

From the exception:

Microsoft.ApplicationServer.Caching.Core

at Microsoft.ApplicationServer.Caching.ChunkStream..ctor(Byte[][] buffers, Boolean writable) at Microsoft.ApplicationServer.Caching.ChunkStream..ctor(Byte[][] buffers) at Microsoft.ApplicationServer.Caching.Utility.Deserialize(Byte[][] buffers, Boolean checkTypeToLoad) at Microsoft.ApplicationServer.Caching.CacheEnumerator.MoveNext()\r\n

I suspect that our read-through provider is doing something wrong under error conditions but I've yet to prove that. However, I wouldn't expect AppFabric to throw a null reference exception from it's core like this.

Is this a known problem? Has anyone any further information that might assist?

Robin Minto Robin Minto · Accepted Answer · 2012-12-03T15:43:26

Having done some testing, AppFabric Cache gracefully handles exceptions thrown by our read-through provider. It seems more likely that the cache was corrupted by the failure of one of the cache servers. We plan to use the code above to monitor the cache.

The code above can be changed to remove any problem regions by clearing the region where the null reference exception occurs:

catch (NullReferenceException)
{
    Console.WriteLine("Unable to get key value pair. Clearing region: {0}", regionName);
    cache.ClearRegion(regionName);
}

The cache can then be populated by the read-through provider again.

Enumerating AppFabric cache throws NullReferenceException

1 Answers