JMX : cant connect from Windows box to monitor JMX port on a Linux system

Question

I have a lot of experience with jconsole.exe and JVisualVM.exe , in the JDK1.6 and have connected thousands of times from a Windows JVM over to a Windows JVM on another machine via JMX.remote but it fails when I try to monitor a java instance that is running on a Linux host (from a Windows host and I tried JMX listener on redhat and SUSE at Amazon EC2). I also tried using jconsole.exe and get a similar error.

Is there any reason anyone can think of , why this kind of JMX connection would have a problem. Any ideas I can try? Has anyone "actually" done this and can say it will work if I persevere?

The error I get from JVisualVM (on the remote connect attempt) is something like this:

"Cannot connect using service:jmx:rmi:///jndi/rmi://<jmx service ip>:8001/jmxrmi"

My remote JMX service config is like this:

 -Dcom.sun.management.jmxremote
 -Dcom.sun.management.jmxremote.port=8001
 -Dcom.sun.management.jmxremote.authenticate=false
 -Dcom.sun.management.jmxremote.hostname=<jmx service ip>

Found a similar question at this link but it didn't answer my question.

I verified that "iptables" is not enabled as a service and is not on and so I don't imaging there is any sort of firewall blockage. Also, both the windows and linux machine are on a 10.0.0.0 private internal subnet together. I am able to telnet into the port 8001 to verify its there , and its listening (netstat -ap) on the linux machine.

Answer 1

See the system property java.rmi.server.hostname. On your Linux server JVM, set this system property to be the public IP of the host. Then use the public IP in you client JMX URL.

Answer 2

I think your problem is that the RMI implementation is kind of hard to work with through firewalls and such as it requires more than the port you specify to work. Things you typically wouldn't notice if you connect to your own machine or a machine on the same network. This post describes the problems you will run into quite well

If I were you, I would try to setup jmxmp as an alternative protocol. To do so you need to add the jmxremote_optional.jar (free from Oracle, download "JMX Remote API 1.0.1_04 Reference Implementation" from here) to both the server and the jvisualvm classpath but it is worth it.

If you google for jmxmp you will find quite a few examples on how to set things up, one of my first hits are http://pub.admc.com/howtos/jmx/distributed-chapt.html#jmxmp-sect which may be a bit too code oriented but I add it here anyway because it explains the things like like most with jmxmp in a few good sentences.

How to define the server side endpoint is depending on what you are running. Most app servers will let you type in a jmxmp enabled jmx service url but if the server is written from scratch you might have to set it up yourself in code instead of using the -D switches to java you are used to.

Give it a try and return with more specific questions about it if you run into problems.

Edit: After you have added the jar to the classpath, the only thing you have to do in your code (assuming you are not using a server app that already handles it for you) is the following (omitting declarations, exception handling and such as you will figure it out anyway):

url=new JMXServiceURL(jmxurl);
this.server = JMXConnectorServerFactory.newJMXConnectorServer(url, null, ManagementFactory.getPlatformMBeanServer());
this.server.start();

Answer 3

Have you tried creating an SSH tunnel to the Linux box from your Windows machine? http://oldsite.precedence.co.uk/nc/putty.html

Or if you have cygwin, just try ssh -f [email protected] -L 7777:remote-server.com:123 -N where 7777 is the port on YOUR windows machine, and 123 is the port on the remote Linux box that listens to JMX commands.

With either of the above, you can use jconsole or visualvm on your Windows box and connect to localhost:7777.

I know iptables is disabled, but just confirm that JMX is working fine at that port by SSHing into the Linux box and trying to use commandline JMX mode of jconsole for localhost (on the Linux box).

Answer 4

I have actually just tackled this problem myself and figured it out.

I would wager that the problem is the RMI connections - you can't predict which ports it will use and so you can't get it to work with a firewall.

The workaround is to use an SSH proxy:

1. SSH to the box where your application is running but use the -D option like this:

   ssh user@remoteHost -D 9999

   As you are on Windows, you can use Cygwin to run the command above, or you can do the same thing with Putty through the GUI (guide here: http://blog.ashurex.com/2012/03/15/creating-ssh-proxy-tunnel-putty/)

   This will start a socks proxy on your local machine on port 9999.

2. Open JVisualVM and in the preferences, under 'network' configure it to use a socks proxy at localhost, on port 9999.

If you do the above, you should then be able to connect to the remote machine as normal and since all the RMI traffic is now going over the SSH proxy, it is punched through the firewall and works nicely.

Good luck :-)