Ignite Cache - Storing Externalizable objects as binary on server

Question

We are trying to integrate ignite cache to our solution in client server mode. We don't want all the classes to be replicated to all the ignite instances , also in some cases its impossible to have those classes predefined ( e.g customer custom classes ).

Some of classes stored in cache are Serializable , and some are Externalizable.

According to the documentation ( and from log ) we see that Serializable classes are stored as binary on server, while Externalizable are trying to get deserialized on server. Since those classes are not available on server we receive an error ( see below ).

We have tried to set PeerClassLoadingEnabled to true , and also tested all the deployment modes, however it appers that remote peer class loading is not working for cache. Also in documentation:

https://ignite.apache.org/docs/latest/code-deployment/peer-class-loading

Following comment exists:

The peer class loading functionality does not deploy the key and object classes of the entries stored in caches.

We have also tried to use JdkMarshaller ( configured it on both client and server ) , however it looks like the problem persist and the object is trying to get deserialized on server.

Therefore : is it possible to force ignite to store both Serializable and Externalizable classes on server as binary and not trying to deserialize them ( We can not implement Binarylizable on all Externalizable classes since some of them are not under our control) ? And in case no , is there an option to force peer class loading between nodes for cache?

BR

Yulian Oifa

10:42:34,690 WARNING [org.apache.ignite.internal.binary.BinaryContext] (RMI TCP Connection(4)-127.0.0.1) Class "org.restcomm.cluster.ClusteredUUID" cannot be serialized using BinaryMarshaller because it either implements Externalizable interface or have writeObject/readObject methods. OptimizedMarshaller will be used instead and class instances will be deserialized on the server. Please ensure that all nodes have this class in classpath. To enable binary serialization either implement Binarylizable interface or set explicit serializer using BinaryTypeConfiguration.setSerializer() method.
10:42:34,696 WARNING [org.apache.ignite.internal.binary.BinaryContext] (RMI TCP Connection(4)-127.0.0.1) Class "org.restcomm.cluster.ClusteredUUID" cannot be serialized using BinaryMarshaller because it either implements Externalizable interface or have writeObject/readObject methods. OptimizedMarshaller will be used instead and class instances will be deserialized on the server. Please ensure that all nodes have this class in classpath. To enable binary serialization either implement Binarylizable interface or set explicit serializer using BinaryTypeConfiguration.setSerializer() method.
10:42:34,713 INFO  [org.apache.ignite.internal.managers.deployment.GridDeploymentLocalStore] (sys-stripe-5-#6%08306b50-6ac3-11eb-8e3a-0a0000a633b8%) Class locally deployed: class org.apache.ignite.IgniteCheckedException
10:42:34,713 INFO  [org.apache.ignite.internal.managers.deployment.GridDeploymentLocalStore] (sys-stripe-5-#6%08306b50-6ac3-11eb-8e3a-0a0000a633b8%) Class locally deployed: class org.apache.ignite.binary.BinaryObjectException
10:42:34,714 INFO  [org.apache.ignite.internal.managers.deployment.GridDeploymentLocalStore] (sys-stripe-5-#6%08306b50-6ac3-11eb-8e3a-0a0000a633b8%) Class locally deployed: class org.apache.ignite.IgniteException
10:42:34,717 INFO  [org.apache.ignite.internal.managers.deployment.GridDeploymentLocalStore] (sys-stripe-5-#6%08306b50-6ac3-11eb-8e3a-0a0000a633b8%) Class locally deployed: class [Ljava.lang.StackTraceElement;
10:42:34,718 ERROR [stderr] (RMI TCP Connection(4)-127.0.0.1) class org.apache.ignite.IgniteException: Failed to unmarshal object with optimized marshaller
10:42:34,719 ERROR [stderr] (RMI TCP Connection(4)-127.0.0.1)   at org.apache.ignite.internal.processors.cache.GridCacheAdapter.containsKey(GridCacheAdapter.java:697)
10:42:34,719 ERROR [stderr] (RMI TCP Connection(4)-127.0.0.1)   at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.containsKey(IgniteCacheProxyImpl.java:1252)
10:42:34,719 ERROR [stderr] (RMI TCP Connection(4)-127.0.0.1)   at org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.containsKey(GatewayProtectedCacheProxy.java:796)

Answer 1

To enable binary serialization either implement Binarylizable interface or set explicit serializer using BinaryTypeConfiguration.setSerializer() method.

Have you tried the latter?

Answer 2

As for now i have workedaround the problem by adding wrapper for Externalizable classes.I am not sure about the perfomance and whether this is best option , but its working.

public class BinaryExternalizable implements Binarylizable
{
private Externalizable wrappedObject;

public BinaryExternalizable() {
    //empty class for reading
}

public BinaryExternalizable(Externalizable wrappedObject) {
    this.wrappedObject=wrappedObject;
}

@Override 
public void writeBinary(BinaryWriter writer) throws BinaryObjectException {
    ByteBuf buffer = Unpooled.buffer();
    ByteBufOutputStream bos = new ByteBufOutputStream(buffer);
    try {
        ObjectOutputStream out = new ObjectOutputStream(bos);
        out.writeUTF(wrappedObject.getClass().getCanonicalName());
        wrappedObject.writeExternal(out);
        out.close();
        bos.close();
    }
    catch(IOException ex) {
        throw new BinaryObjectException(ex.getMessage(), ex);
    }
    
    byte[] data=new byte[buffer.readableBytes()];
    buffer.readBytes(data);
    writer.rawWriter().writeByteArray(data);
}
 
@Override 
public void readBinary(BinaryReader reader) throws BinaryObjectException {
    byte[] data = reader.rawReader().readByteArray();
    ByteBufInputStream bis=new ByteBufInputStream(Unpooled.wrappedBuffer(data));
    try {
        ObjectInputStream ins=new ObjectInputStream(bis);       
        String className=ins.readUTF();         
        wrappedObject = (Externalizable)Class.forName(className).newInstance();
        wrappedObject.readExternal(ins);
        ins.close();
        bis.close();
    } catch(IOException | ClassNotFoundException | IllegalAccessException | InstantiationException ex) {
        throw new BinaryObjectException(ex.getMessage(),ex);
    }
}

public Externalizable getWrappedObject() {
    return this.wrappedObject;
}
}