OutofMemory error for spark streaming job

0
votes

I have a spark streaming job running in Hortonworks cluster . I am running it in cluster mode through yarn , the job is shown as running in UI , but it is having the below exception in driver logs

Exception in thread "JobGenerator" java.lang.OutOfMemoryError: Java heap space
1
apache-sparkapache-kafkaspark-streaming

1 Answers

1
votes

I fixed the issue by specifying driver-memory in spark-submit command.because the memory issue was in driver