I have very basic storm topology which contains 1 unreliable spout + 1 bolt. Spout is fed by Kafka broker via zookeeper. The Spout sends tuples to a bolt witch in its role do nothing ( has empty execute method). The topology run in local cluster mode and it crashes after a while from run. I tried to increase the app memory by -Xmx parameter many times, but the problem still appears. please help! Here is the code snippet that build the topology:
builder.setSpout("kafka_forwarding_spout", new KafkaForwardingSpout(),1);
builder.setBolt("fake_bolt", new FakeBolt(),1).localOrShuffleGrouping("kafka_forwarding_spout");
Here is a snippet from storm log:
1318482 [Thread-37-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (Expired)
1319574 [CuratorFramework-9] INFO com.netflix.curator.framework.state.ConnectionStateManager - State change: LOST
1312307 [Thread-39-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (SyncConnected)
1307441 [Thread-25-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (SyncConnected)
1227216 [Thread-29-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (Expired)
1191153 [Thread-31-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (Disconnected)
1771836 [Thread-39-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (Disconnected)
1688630 [ConnectionStateManager-0] WARN com.netflix.curator.framework.state.ConnectionStateManager - There are no ConnectionStateListeners registered.
1650422 [Thread-35-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (Disconnected)
1635500 [Thread-19-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (Expired)
1616369 [CuratorFramework-11] ERROR com.netflix.curator.framework.imps.CuratorFrameworkImpl - Background exception was not retry-able or retry gave up
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss
at com.netflix.curator.ConnectionState.getZooKeeper(ConnectionState.java:72) ~[curator-client-1.0.1.jar:na]
at com.netflix.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:74) ~[curator-client-1.0.1.jar:na]
at com.netflix.curator.framework.imps.CuratorFrameworkImpl.getZooKeeper(CuratorFrameworkImpl.java:353) [curator-framework-1.0.1.jar:na]
at com.netflix.curator.framework.imps.BackgroundSyncImpl.performBackgroundOperation(BackgroundSyncImpl.java:39) ~[curator-framework-1.0.1.jar:na]
at com.netflix.curator.framework.imps.OperationAndData.callPerformBackgroundOperation(OperationAndData.java:40) ~[curator-framework-1.0.1.jar:na]
at com.netflix.curator.framework.imps.CuratorFrameworkImpl.backgroundOperationsLoop(CuratorFrameworkImpl.java:547) [curator-framework-1.0.1.jar:na]
at com.netflix.curator.framework.imps.CuratorFrameworkImpl.access$200(CuratorFrameworkImpl.java:50) [curator-framework-1.0.1.jar:na]
at com.netflix.curator.framework.imps.CuratorFrameworkImpl$2.call(CuratorFrameworkImpl.java:177) [curator-framework-1.0.1.jar:na]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_05]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_05]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_05]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_05]
1617025 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2001] WARN org.apache.zookeeper.server.NIOServerCnxn - EndOfStreamException: Unable to read additional data from client sessionid 0x149dd7eb6a2000b, likely client has closed socket
1600212 [Thread-23-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (Disconnected)
1584086 [Thread-21-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (SyncConnected)
1570224 [Thread-9] INFO backtype.storm.daemon.worker - Shutting down receive thread
1569786 [ConnectionStateManager-0] WARN com.netflix.curator.framework.state.ConnectionStateManager - There are no ConnectionStateListeners registered.
2158302 [Thread-5] ERROR backtype.storm.event - Error when processing event
java.lang.RuntimeException: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss
at backtype.storm.util$wrap_in_runtime.invoke(util.clj:44) ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
at backtype.storm.zookeeper$get_children.invoke(zookeeper.clj:133) ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
at backtype.storm.cluster$mk_distributed_cluster_state$reify__2385.get_children(cluster.clj:101) ~[na:na]
at backtype.storm.cluster$mk_storm_cluster_state$reify__2814.assignments(cluster.clj:243) ~[na:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_05]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_05]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_05]
at java.lang.reflect.Method.invoke(Method.java:483) ~[na:1.8.0_05]
at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:93) ~[clojure-1.4.0.jar:na]
at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:28) ~[clojure-1.4.0.jar:na]
at backtype.storm.daemon.supervisor$assignments_snapshot.invoke(supervisor.clj:39) ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
at backtype.storm.daemon.supervisor$mk_synchronize_supervisor$this__6251.invoke(supervisor.clj:298) ~[na:na]
at backtype.storm.event$event_manager$fn__2921.invoke(event.clj:39) ~[na:na]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.4.0.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_05]
Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss
at com.netflix.curator.ConnectionState.getZooKeeper(ConnectionState.java:72) ~[curator-client-1.0.1.jar:na]
at com.netflix.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:74) ~[curator-client-1.0.1.jar:na]
at com.netflix.curator.framework.imps.CuratorFrameworkImpl.getZooKeeper(CuratorFrameworkImpl.java:353) ~[curator-framework-1.0.1.jar:na]
at com.netflix.curator.framework.imps.GetChildrenBuilderImpl$3.call(GetChildrenBuilderImpl.java:180) ~[curator-framework-1.0.1.jar:na]
at com.netflix.curator.framework.imps.GetChildrenBuilderImpl$3.call(GetChildrenBuilderImpl.java:173) ~[curator-framework-1.0.1.jar:na]
at com.netflix.curator.RetryLoop.callWithRetry(RetryLoop.java:85) ~[curator-client-1.0.1.jar:na]
at com.netflix.curator.framework.imps.GetChildrenBuilderImpl.pathInForeground(GetChildrenBuilderImpl.java:169) ~[curator-framework-1.0.1.jar:na]
at com.netflix.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:161) ~[curator-framework-1.0.1.jar:na]
at com.netflix.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:36) ~[curator-framework-1.0.1.jar:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_05]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_05]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_05]
at java.lang.reflect.Method.invoke(Method.java:483) ~[na:1.8.0_05]
at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:93) ~[clojure-1.4.0.jar:na]
at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:28) ~[clojure-1.4.0.jar:na]
at backtype.storm.zookeeper$get_children.invoke(zookeeper.clj:131) ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
... 13 common frames omitted
2117747 [Thread-4] ERROR com.netflix.curator.ConnectionState - Connection timed out
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss
at com.netflix.curator.ConnectionState.getZooKeeper(ConnectionState.java:72) ~[curator-client-1.0.1.jar:na]
at com.netflix.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:74) [curator-client-1.0.1.jar:na]
at com.netflix.curator.framework.imps.CuratorFrameworkImpl.getZooKeeper(CuratorFrameworkImpl.java:353) [curator-framework-1.0.1.jar:na]
at com.netflix.curator.framework.imps.ExistsBuilderImpl$2.call(ExistsBuilderImpl.java:149) [curator-framework-1.0.1.jar:na]
at com.netflix.curator.framework.imps.ExistsBuilderImpl$2.call(ExistsBuilderImpl.java:138) [curator-framework-1.0.1.jar:na]
at com.netflix.curator.RetryLoop.callWithRetry(RetryLoop.java:85) [curator-client-1.0.1.jar:na]
at com.netflix.curator.framework.imps.ExistsBuilderImpl.pathInForeground(ExistsBuilderImpl.java:134) [curator-framework-1.0.1.jar:na]
at com.netflix.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:125) [curator-framework-1.0.1.jar:na]
at com.netflix.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:34) [curator-framework-1.0.1.jar:na]
at backtype.storm.zookeeper$exists_node_QMARK_$fn__2170.invoke(zookeeper.clj:96) [na:0.9.1-incubating]
at backtype.storm.zookeeper$exists_node_QMARK_.invoke(zookeeper.clj:93) [na:0.9.1-incubating]
at backtype.storm.zookeeper$mkdirs.invoke(zookeeper.clj:107) [na:0.9.1-incubating]
at backtype.storm.cluster$mk_distributed_cluster_state$reify__2385.set_ephemeral_node(cluster.clj:69) [na:0.9.1-incubating]
at backtype.storm.cluster$mk_storm_cluster_state$reify__2814.supervisor_heartbeat_BANG_(cluster.clj:315) [na:0.9.1-incubating]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_05]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_05]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_05]
at java.lang.reflect.Method.invoke(Method.java:483) ~[na:1.8.0_05]
at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:93) [clojure-1.4.0.jar:na]
at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:28) [clojure-1.4.0.jar:na]
at backtype.storm.daemon.supervisor$eval6298$exec_fn__1459__auto____6299$heartbeat_fn__6301.invoke(supervisor.clj:378) [na:0.9.1-incubating]
at backtype.storm.timer$schedule_recurring$this__2141.invoke(timer.clj:92) [na:0.9.1-incubating]
at backtype.storm.timer$mk_timer$fn__2124$fn__2125.invoke(timer.clj:48) [na:0.9.1-incubating]
at backtype.storm.timer$mk_timer$fn__2124.invoke(timer.clj:41) [na:0.9.1-incubating]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.4.0.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_05]
2078727 [Thread-16] ERROR com.netflix.curator.ConnectionState - Connection timed out
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss
at com.netflix.curator.ConnectionState.getZooKeeper(ConnectionState.java:72) ~[curator-client-1.0.1.jar:na]
at com.netflix.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:74) [curator-client-1.0.1.jar:na]
at com.netflix.curator.framework.imps.CuratorFrameworkImpl.getZooKeeper(CuratorFrameworkImpl.java:353) [curator-framework-1.0.1.jar:na]
at com.netflix.curator.framework.imps.ExistsBuilderImpl$2.call(ExistsBuilderImpl.java:149) [curator-framework-1.0.1.jar:na]
at com.netflix.curator.framework.imps.ExistsBuilderImpl$2.call(ExistsBuilderImpl.java:138) [curator-framework-1.0.1.jar:na]
at com.netflix.curator.RetryLoop.callWithRetry(RetryLoop.java:85) [curator-client-1.0.1.jar:na]
at com.netflix.curator.framework.imps.ExistsBuilderImpl.pathInForeground(ExistsBuilderImpl.java:134) [curator-framework-1.0.1.jar:na]
at com.netflix.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:125) [curator-framework-1.0.1.jar:na]
at com.netflix.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:34) [curator-framework-1.0.1.jar:na]
at backtype.storm.zookeeper$exists_node_QMARK_$fn__2170.invoke(zookeeper.clj:96) [na:0.9.1-incubating]
at backtype.storm.zookeeper$exists_node_QMARK_.invoke(zookeeper.clj:93) [na:0.9.1-incubating]
at backtype.storm.zookeeper$exists.invoke(zookeeper.clj:141) [na:0.9.1-incubating]
at backtype.storm.cluster$mk_distributed_cluster_state$reify__2385.set_data(cluster.clj:85) [na:0.9.1-incubating]
at backtype.storm.cluster$mk_storm_cluster_state$reify__2814.worker_heartbeat_BANG_(cluster.clj:291) [na:0.9.1-incubating]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_05]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_05]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_05]
at java.lang.reflect.Method.invoke(Method.java:483) ~[na:1.8.0_05]
at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:93) [clojure-1.4.0.jar:na]
at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:28) [clojure-1.4.0.jar:na]
at backtype.storm.daemon.worker$do_executor_heartbeats.doInvoke(worker.clj:53) [na:0.9.1-incubating]
at clojure.lang.RestFn.invoke(RestFn.java:439) [clojure-1.4.0.jar:na]
at backtype.storm.daemon.worker$eval5665$exec_fn__1459__auto____5666$fn__5669.invoke(worker.clj:367) [na:0.9.1-incubating]
at backtype.storm.timer$schedule_recurring$this__2141.invoke(timer.clj:92) [na:0.9.1-incubating]
at backtype.storm.timer$mk_timer$fn__2124$fn__2125.invoke(timer.clj:48) [na:0.9.1-incubating]
at backtype.storm.timer$mk_timer$fn__2124.invoke(timer.clj:41) [na:0.9.1-incubating]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.4.0.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_05]
2079838 [CuratorFramework-11] INFO com.netflix.curator.framework.state.ConnectionStateManager - State change: LOST
2052508 [Thread-37-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (SyncConnected)
1948998 [yassin-long-d_localhost.localdomain-1416760311632-aa3d627e_watcher_executor] INFO kafka.consumer.ZookeeperConsumerConnector - [yassin-long-d_localhost.localdomain-1416760311632-aa3d627e], exception during rebalance
java.lang.NullPointerException: null
at org.I0Itec.zkclient.ZkConnection.readData(ZkConnection.java:103) ~[zkclient-0.3.jar:0.3]
at org.I0Itec.zkclient.ZkClient$9.call(ZkClient.java:770) ~[zkclient-0.3.jar:0.3]
at org.I0Itec.zkclient.ZkClient$9.call(ZkClient.java:766) ~[zkclient-0.3.jar:0.3]
at org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:675) ~[zkclient-0.3.jar:0.3]
at org.I0Itec.zkclient.ZkClient.readData(ZkClient.java:766) ~[zkclient-0.3.jar:0.3]
at org.I0Itec.zkclient.ZkClient.readData(ZkClient.java:761) ~[zkclient-0.3.jar:0.3]
at kafka.utils.ZkUtils$.readData(ZkUtils.scala:461) ~[kafka_2.10-0.8.1.jar:na]
at kafka.utils.ZkUtils$$anonfun$getCluster$1.apply(ZkUtils.scala:505) ~[kafka_2.10-0.8.1.jar:na]
at kafka.utils.ZkUtils$$anonfun$getCluster$1.apply(ZkUtils.scala:504) ~[kafka_2.10-0.8.1.jar:na]
at scala.collection.Iterator$class.foreach(Iterator.scala:727) ~[scala-library-2.10.1.jar:na]
at scala.collection.AbstractIterator.foreach(Iterator.scala:1157) ~[scala-library-2.10.1.jar:na]
at scala.collection.IterableLike$class.foreach(IterableLike.scala:72) ~[scala-library-2.10.1.jar:na]
at scala.collection.AbstractIterable.foreach(Iterable.scala:54) ~[scala-library-2.10.1.jar:na]
at kafka.utils.ZkUtils$.getCluster(ZkUtils.scala:504) ~[kafka_2.10-0.8.1.jar:na]
at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anonfun$syncedRebalance$1.apply$mcVI$sp(ZookeeperConsumerConnector.scala:407) ~[kafka_2.10-0.8.1.jar:na]
at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:141) [scala-library-2.10.1.jar:na]
at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:402) [kafka_2.10-0.8.1.jar:na]
at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anon$1.run(ZookeeperConsumerConnector.scala:355) [kafka_2.10-0.8.1.jar:na]
1872056 [Thread-39-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (Expired)
1853312 [Thread-35-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (Expired)
1840380 [Thread-31-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (Expired)
1808599 [Thread-41-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (SyncConnected)
1799622 [Thread-25-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (Disconnected)
2287292 [Thread-41-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (Disconnected)
2287075 [Thread-37-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (Disconnected)
2286862 [yassin-long-d_localhost.localdomain-1416760311632-aa3d627e_watcher_executor] INFO kafka.consumer.ZookeeperConsumerConnector - [yassin-long-d_localhost.localdomain-1416760311632-aa3d627e], end rebalancing consumer yassin-long-d_localhost.localdomain-1416760311632-aa3d627e try #0
2280400 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2001] WARN org.apache.zookeeper.server.NIOServerCnxn - EndOfStreamException: Unable to read additional data from client sessionid 0x149dd7eb6a20005, likely client has closed socket
2280180 [ConnectionStateManager-0] WARN com.netflix.curator.framework.state.ConnectionStateManager - There are no ConnectionStateListeners registered.
2267345 [Thread-9] INFO backtype.storm.messaging.loader - Shutting down receiving-thread: [events_topology-1-1416760309, 1027]
2242225 [Thread-19-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (SyncConnected)
2224569 [Thread-29-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (SyncConnected)
2226559 [Thread-21-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (Disconnected)
2209276 [Thread-23-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (Expired)
2296607 [Thread-41-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (Expired)
2293718 [Thread-37-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (Expired)
2292830 [Thread-25-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (SyncConnected)
2498572 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2001] WARN org.apache.zookeeper.server.NIOServerCnxn - EndOfStreamException: Unable to read additional data from client sessionid 0x149dd7eb6a20003, likely client has closed socket
2540517 [Thread-41-kafka_forwarding_spout-EventThread] ERROR org.apache.zookeeper.ClientCnxn - Error while calling watcher
java.lang.OutOfMemoryError: Java heap space
Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "main-SendThread(localhost:2001)"
2637439 [Thread-35-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (Disconnected)
2647739 [Thread-39-kafka_forwarding_spout-EventThread] INFO org.I0Itec.zkclient.ZkClient - zookeeper state changed (Disconnected)
2672605 [main-SendThread(localhost:2001)] ERROR org.apache.zookeeper.ClientCnxn - from main-SendThread(localhost:2001)
java.lang.OutOfMemoryError: Java heap space
2764243 [Thread-25-kafka_forwarding_spout-SendThread(zoo2:2182)] ERROR org.apache.zookeeper.ClientCnxn - from Thread-25-kafka_forwarding_spout-SendThread(zoo2:2182)
java.lang.OutOfMemoryError: Java heap space
2720032 [main-SendThread(localhost:2001)] ERROR org.apache.zookeeper.ClientCnxn - from main-SendThread(localhost:2001)
java.lang.OutOfMemoryError: Java heap space
2731157 [Thread-8] ERROR org.apache.zookeeper.server.NIOServerCnxn - Thread Thread[Thread-8,5,main] died
java.lang.OutOfMemoryError: Java heap space
2800693 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2001] ERROR org.apache.zookeeper.server.NIOServerCnxn - Thread Thread[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2001,5,main] died
