Cannot run flink job in EMR using EMR steps

Question

I am trying to submit a job in yarn session via EMR step. When I do so I face the following error

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/flink/lib/slf4j-log4j12-1.7.7.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Command exiting with ret '1'

Can you pease help me in resolving this issue

Step Configuration
Name= Custom jar
Jar location = command-runner.jar
Arguments = flink run -c com.comcast.dhome.xh.timeline.events.ProcessKafkaStream s3://aws-logs-518000940535-us-east-1/dh-xh-event-subscriber.jar --region us-east-1
Action on failure = continue

EMR configuration

EMR = 5.15.0
FLINK 1.4.2

wind wind · Accepted Answer · 2019-07-04T09:12:19

You need to download our application JAR locally, i.e. aws s3 cp s3://aws-logs-518000940535-us-east-1/dh-xh-event-subscriber.jar . and then point to the local jar flink run -c com.comcast.dhome.xh.timeline.events.ProcessKafkaStream dh-xh-event-subscriber.jar --region us-east-1. You cannot directly use files placed at the s3.

Cannot run flink job in EMR using EMR steps

1 Answers