secondary ajp worker not working between apache and tomcat

Question

I've had this working for months but I had a power cycle today and something broke. Sorry, this is a bit detailed and specific, but I'm desperate for help.

I have apache-2.2 and two tomcat-6 servers (simply running from two separate folders). I don't have any http connectors but I have one ajp connector running on each server on ports 8009 and 8010. Upon startup both tomcats report the connector is running and there doesn't seem to be any problem:

INFO: JK: ajp13 listening on /0.0.0.0:8009
...
INFO: JK: ajp13 listening on /0.0.0.0:8010

I'm reasonably sure I have my workers and apache configuration setup correctly. I can reach the connector on 8009 fine, no problems at all, but when I try 8010 apache gives me a 503. Checking the log (mod_jk.log) it says:

jk_open_socket::jk_connect.c (594): connect to 127.0.0.1:8010 failed (errno=13)
ajp_connect_to_endpoint::jk_ajp_common.c (922): Failed opening socket to (127.0.0.1:8010) (errno=13)
ajp_send_request::jk_ajp_common.c (1507): (eis) connecting to backend failed. Tomcat is probably not started or is listening on the wrong port (errno=13)

But what I can't understand is if I do a simple telnet to that port and send a GET:

# telnet 127.0.0.1 8010
Trying 127.0.0.1...
Connected to localhost.localdomain (127.0.0.1).
Escape character is '^]'.
GET
Connection closed by foreign host.

It hits the correct tomcat (the error is expected):

Oct 25, 2011 6:17:10 PM org.apache.jk.common.MsgAjp processHeader
SEVERE: BAD packet signature 18245
Oct 25, 2011 6:17:10 PM org.apache.jk.common.ChannelSocket processConnection
SEVERE: Error, processing connection

So why can't apache reach it? The telnet works whether I'm a super user or not. What could be going on? I'm completely stumped.

And I've tried restarting both apache and tomcat and nothing changes. Thanks for taking a look.

EDIT-1: Quick update to this ... I added an http connector and it works fine but the ajp connector is still failing.

EDIT-2: Here are the config details per request ...

workers.properties:

worker.list=service1,service2

worker.service1.type=ajp13
worker.service1.host=127.0.0.1
worker.service1.port=8009

worker.service2.type=ajp13
worker.service2.host=127.0.0.1
worker.service2.port=8010

httpd.conf (I see both jkmount and JkMount ... does caps matter?):

JkMount /s1 service1
JkMount /s1/* service1

JkMount /s2 service2
JkMount /s2/* service2

server.xml for service1:

<Connector port="8009" protocol="AJP/1.3" />

server.xml for service2:

<Connector port="8010" protocol="AJP/1.3" />

I think that's it.

You first code block contains the two "ajp13 listening" from the two tomcats. Is that in the same logfile? Do both tomcats log in the same logfile? — cherouvim
Can you post the relevant configuration sections from 2 tomcats server.xml and apache's mod_jk configuration section? — cherouvim
cherouvim: There are two completely independent tomcats and those are from two log files. I could post the config sections but I think it complicates things. I'm more interested in how it could be possible that I can telnet to the port but apache can't find it. Still, I'll tack on the config if you think it'll help. Thanks! — rjcarr
At the point you get the connection error in mod_jk.log, do you get anything in tomcat's catalina log? — cherouvim
Also, what is your JkLogLevel? Lastly, if you shut down both tomcats, is anything listening on 8010? — cherouvim

Marek Schmidt Marek Schmidt · Accepted Answer · 2011-12-21T19:40:34

Do you have SELinux on the system? Can you try:

setenforce 0

It could be preventing the apache process to connect. Also, check your firewall rules...

secondary ajp worker not working between apache and tomcat

2 Answers