Apache Ignite server crashes after incorporating Auditing events

Question

In start, it works fine, but after a certain time (1-2 hours) it crashes with the following exception in server logs.

ERROR 1 --- [-ignite-server%] : JVM will be halted immediately due to the failure: [failureCtx=FailureContext [type=CRITICAL_ERROR, err=class o.a.i.i.IgniteDeploymentCheckedException: Failed to obtain deployment for class: com.event.audit.AuditEventListener$$Lambda$1484/0x0000000800a7ec40]]

public static void remoteListener(Ignite ignite) {
    // This optional local callback is called for each event notification


    // that passed remote predicate listener.
    IgniteBiPredicate<UUID, CacheEvent> locLsnr = new IgniteBiPredicate<UUID, CacheEvent>() {
      @Override public boolean apply(UUID nodeId, CacheEvent evt) {
        System.out.println("Listener caught an event");

        //--- My custom code to persists the event in another cache
    };

    IgnitePredicate<CacheEvent> remoteListener = cacheEvent -> {
      return true;
    };

    // Register event listeners on all nodes to listen for task events.
    UUID lsnrId = ignite.events(ignite.cluster()).remoteListen(locLsnr, remoteListener, EVT_CACHE_OBJECT_PUT, EVT_CACHE_OBJECT_REMOVED);
  }
}

ilya ilya · Accepted Answer · 2021-06-03T13:08:22

As I understand you, you try to perform cache operations in event listener:

//--- My custom code to persists the event in another cache

Event listeners are called under the locks and this is bad idea to make any other cache operations in listeners. I suppose it could be the root cause of your issue.

Try to change you design, for example you can add your caught event in a queue and then read this queue in another thread and save the data in another cache.

Apache Ignite server crashes after incorporating Auditing events

1 Answers