Spark Kinesis Streaming example

Question

I experiencing issues running https://github.com/apache/spark/blob/master/extras/kinesis-asl/src/main/java/org/apache/spark/examples/streaming/JavaKinesisWordCountASL.java

I modified this example and used my own values for "app-name", "stream-name" and "endpoint-url".

I have placed various print lines within my code. When running the job using the cmd "spark-submit" I fail to see any print lines in the stdout logs.

Can someone please explain to me where I can find the system out print lines. Why don't I see the stdout log?

Is anyone able to run that the Java example successfully?

Answer 1

Are you running with local master? if so, have you checked out this post? Apache Spark Kinesis Sample not working

Basically, need two cores per shard.