Can't connect to resource manager from IDEA

1
votes

I encounter a connection issue when submiting job in yarn-client mode from IDEA Intellij.

I did set the env variables and double checked it by printing them out:

System.setProperty("YARN_CONF_DIR",  "D:\\HadoopDev\\UserClick\\src\\main\\resources\\hadoop-vm");

System.setProperty("HADOOP_CONF_DIR", "D:\\HadoopDev\\UserClick\\src\\main\\resources\\hadoop-vm");

But I still got error message telling me：

INFO - Connecting to ResourceManager at /0.0.0.0:8032 INFO - Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 0 >time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, >sleepTime=1 SECONDS) INFO - Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 1 》>time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, >sleepTime=1 SECONDS) INFO - Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 2 >time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, >sleepTime=1 SECONDS)

All config files hadoop related are in the folder.And I have tried to upload the jar and submited it in yarn-client mode in the cluster which turned out it worked.

Any help?Thx~

1
hadoopintellij-ideaapache-sparkyarn

1 Answers

0
votes
set .config("spark.hadoop.yarn.resourcemanager.address", "hadoop:8032")

does the override trick.