HttpRuntime.Cache corrupting data under high load on Windows Azure

Question

I have an application which caches data from the database in the HttpRuntime.Cache. When I load test this application with 1000 users per second some values in the cache become corrupted.

For example I have a page which simply queries the database for it's content. It first checks the cache and if available gets the data from there.

DataSet ds;

var cachedData = HttpRuntime.Cache["homepage"];

if (cachedData == null) {

    ds = getDataSet("SQL query...");

    addToCache("homepage", ds);

}
else {

    ds = (DataSet)cachedData;

}

This works fine up to about 100 users per second but when I start stress testing up to 1000 users some of the fields in the cached tables return DBNull.Value. After testing when I check what's in the cache I can see the fields are now DBNull.Value.

I have enabled logging and have checked that the DataSet is only added to the cache once but somehow it's getting corrupted during stress testing.

Has anyone seen this before or have some pointers on what's going wrong? It's being hosted under Windows Azure with dedicated worker roles for caching.

Mark Clancy Mark Clancy · Accepted Answer · 2013-12-21T17:35:13

The problem turned out to be where I have updated a column in a table after returning it from the cache it somehow corrupted the data in the cache but intermittently so was very hard to detect. Now after getting the DataSet from the cache I take a copy of it using DataSet.Copy to use instead of the original.

HttpRuntime.Cache corrupting data under high load on Windows Azure

1 Answers