hive reach max worker and cannot connect to hiveserver2

2
votes

when i use beeline connect to hiveserver2 ,the err msg is like this. I have connect to the hiveserver2 before. This error is show after i connect to hiveserver2 some times. and i can connect use jdbc:hive2://

beeline> !connect jdbc:hive2://master:10000

SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] Connecting to jdbc:hive2://master:10000 Enter username for jdbc:hive2://master:10000: Enter password for jdbc:hive2://master:10000: 17/11/14 22:09:36 [main]: WARN jdbc.HiveConnection: Failed to connect to master:10000 Unexpected end of file when reading from HS2 server. The root cause might be too many concurrent connections. Please ask the administrator to check the number of active connections, and adjust hive.server2.thrift.max.worker.threads if applicable. Error: Could not open client transport with JDBC Uri: jdbc:hive2://master:10000: null (state=08S01,code=0)

hive-site.xml is set like this

<property>
 39         <name>hive.server2.thrift.min.worker.threads</name>
 40         <value>5</value>
 41     </property>
 42     <property>
 43         <name>hive.server2.thrift.max.worker.threads</name>
 44         <value>500</value>
 45     </property>
 46     <property>
 47         <name>hive.server2.thrift.bind.host</name>
 48         <value>master</value>
 49     </property>
 50     <property>
 51         <name>hive.server2.thrift.port</name>
 52         <value>10000</value>
 53     </property>

i have check the netstat that 10000 port is listening. even if i restart the hadoop server and hiveserver2,the problem is not solved. how can i know how many connection is active or can i clear the thread pool? The active will not be closed when i shutdown the hiveserver and hadoop and linux system?

1
javahadoopapache-sparkhive
anyone help me? i have tried two days – Oukafu
Same problem here. anyone plz – reizist

1 Answers

0
votes

It works fine on my environment:

$ netstat -an | grep 10000
tcp4       0      0  *.10000                *.*                    LISTEN
  • hive-site.xml

set nosasl for not using SASLTransport

<property>
    <name>hive.server2.authentication</name>
    <value>NOSASL</value> <!-- default NONE is for SASLTranspor
</property>

 <property>
    <name>hive.server2.enable.doAs</name>
    <value>false</value> <!-- Execute query as hiveserver2 proc
</property>

and 

!connect jdbc:hive2://localhost:10000/default;auth=noSasl hive password org.apache.hive.jdbc.HiveDriver
  • REFERENCES

This is Japanese blog but it would be helpful: http://tagomoris.hatenablog.com/entry/20131226/1388044783