Will mesos & zookeeper uses random ports to communicate with peers

Question

• I'm new to mesos, I've configured mesos cluster setup with 3 masters and 2 slaves in rhel6.7 machines

• I've used available rpm packages to install mesos. I've downloaded zookeeper.tar.gz and using binary's of zookeeper and its configuration is

• Starting mesos-master with below arguments
  

• added few of the rules to accept incoming and outgoing for ports(5050, 8080, 2181, 2888, 3888). I've used below steps to add rules.
• Started three masters and I've seen logs, not able to communicate with each-other and three masters servers are getting crashed, then I flushed(iptales -F) all rules and started three servers and its able to communicate and working properly and I did netstat on leading mesos master to know what ports are using for communication. I'm thinking its using few more ports other than 2181, 5050, 2888, 3888 and 8080 I saw more ports are using for communication. I've attached screen shot.
• We have firewall in production environment, Its not possible to allow all ports in proudction environment
• Will mesos and zookeepr will use random ports to communication (in
  screen shot its using 39958,38574,40097,etc..)?
• If its using random ports, can we have forceport option to avoid usage of random ports from zookeeper or mesos? - Is there any solution for this kind of problem? - Can some one give suggestion to solve this?

Answer 1

Since the Zookeeper servers act as both client (trying to connect to other servers) and server (listening for connections from other ZK servers) it must choose an ephemeral port when trying to connect to another server (as a client).

This ephemeral port is taken from the range defined by the output of the following command.

sysctl net.ipv4.ip_local_port_range

Ie: for my machine the port range is

net.ipv4.ip_local_port_range = 32768 60999

We can reduce the ephemeral port by changing the port range using below command.

sudo sysctl -w net.ipv4.ip_local_port_range="1024 1050"