nHibernate second-level cache with memcached and random invalid casts

Question

I have an MVC3 application using nHibernate and memcached as the second level cache provider. We've been intermittently (but much more frequently lately) getting weird casting issues. It happens randomly and invalidating the memcached cache will fix the problem for while.

It only happens in our Production environment because we don't run memcached in our other environments. However I run memcached locally and have tried to get this to occur locally with no luck.

We are using memcached 1.2.6 on Windows. Here's the stack trace. I know it's not going to be enough information to determine anything but if anyone has any ideas about how I could debug this that would be appreciated. I'm trying to get remote debugging on our Production machine but it's a busy time of year and risky.

Unable to cast object of type 'System.Int32' to type 'System.String'.

Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code. 

Exception Details: System.InvalidCastException: Unable to cast object of type 'System.Int32' to type 'System.String'.

Source Error: 

An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below.

Stack Trace: 


[InvalidCastException: Unable to cast object of type 'System.Int32' to type 'System.String'.]
   (Object , Object[] , SetterCallback ) +4270
   NHibernate.Tuple.Entity.PocoEntityTuplizer.SetPropertyValuesWithOptimizer(Object entity, Object[] values) +80

[PropertyAccessException: Invalid Cast (check your mapping for property type mismatches); setter of MyApplication.Business.Data.Program]
   NHibernate.Tuple.Entity.PocoEntityTuplizer.SetPropertyValuesWithOptimizer(Object entity, Object[] values) +207
   NHibernate.Tuple.Entity.PocoEntityTuplizer.SetPropertyValues(Object entity, Object[] values) +97
   NHibernate.Cache.Entry.CacheEntry.Assemble(Object[] values, Object result, Object id, IEntityPersister persister, IInterceptor interceptor, ISessionImplementor session) +306
   NHibernate.Cache.Entry.CacheEntry.Assemble(Object instance, Object id, IEntityPersister persister, IInterceptor interceptor, ISessionImplementor session) +147
   NHibernate.Event.Default.DefaultLoadEventListener.AssembleCacheEntry(CacheEntry entry, Object id, IEntityPersister persister, LoadEvent event) +434
   NHibernate.Event.Default.DefaultLoadEventListener.LoadFromSecondLevelCache(LoadEvent event, IEntityPersister persister, LoadType options) +800
   NHibernate.Event.Default.DefaultLoadEventListener.DoLoad(LoadEvent event, IEntityPersister persister, EntityKey keyToLoad, LoadType options) +560
   NHibernate.Event.Default.DefaultLoadEventListener.Load(LoadEvent event, IEntityPersister persister, EntityKey keyToLoad, LoadType options) +229
   NHibernate.Event.Default.DefaultLoadEventListener.ProxyOrLoad(LoadEvent event, IEntityPersister persister, EntityKey keyToLoad, LoadType options) +438
   NHibernate.Event.Default.DefaultLoadEventListener.OnLoad(LoadEvent event, LoadType loadType) +943
   NHibernate.Impl.SessionImpl.FireLoad(LoadEvent event, LoadType loadType) +99
   NHibernate.Impl.SessionImpl.Get(String entityName, Object id) +117
   NHibernate.Impl.SessionImpl.Get(Object id) +70
   MyApplication.Business.Repositories.Repository`1.Get(Object id) +148

What is MyApplication.Business.Data.Program ? Is it always the same Exception ? String=>Int32 makes me (randomly) think of Enum, or query.substitutions, implicit conversion... — jbl
It's always the same exception. Program is the nHibernate entity. Once the error starts it sticks around until the cache is invalidated. However the initial error happens seemingly at random. — Ryan Bosinger
Can you elaborate a little more about your setup? What version of nHibernate? Which cache provider? What distribution of memcached? etc. — DanP
Without further details, I think you're going to be very hard pressed to find a solution here. — DanP
All I can say is that we've been running the version 1.8.1 distribution from couchbase.com/download for some time now in a VERY busy production environment and have never run into issues. — DanP

Sir Rippov the Maple Sir Rippov the Maple · Accepted Answer · 2013-09-10T18:11:17

We've been encountering the same issue and we have pinpointed the problem to invalid cache entries in the second level cache. In our case the problem arises when we add or remove columns/properties from tables/entities that are cached in the second level cache and deploy the change. The code expects five columns (for argument's sake) whereas the cache stores six columns.

One strategy that we employed in our repository is to catch the exception and invalidate the cache. We then retry the get.

    public override T Get(int id)
    {
        try
        {
            return base.Get(id);
        }
        catch (InvalidCastException)
        {
            _sessionFactory.EvictEntity(typeof (T).Name);
            return Get(id);
        }
    }

Hope this helps!

nHibernate second-level cache with memcached and random invalid casts

2 Answers