WebUI access for workers in spark

3
votes

We have a cluster which is built by docker swarm Cluster consists of 1 Manager 3 Worker nodes.

it can be seen as follow: enter image description here

and we have run Apache Spark on the cluster. It consists of a master and four workers. It is seen as follow on master web ui

enter image description here

The problem is that I can not access the details of worker node. It wants to connect to an ip(10.0.0.5:8081). But I can not access the link from my local machine.

enter image description here

1
apache-sparkdockercluster-computingdocker-swarm
which tool do you use to see your cluster? – seufagner
Docker swarm visualizer – ugur

1 Answers

0
votes

you need to bind the port of the spark webui service and access the webui using localhost:8081 (if you are binding the localport 8081) example in docker-compose.yml file in the spark webui service put something like this https://docs.docker.com/compose/compose-file/#ports

the ip you specified(10.0.0.5) is the subnet created by docker you cannot access using that ip from your machine