gRPC socket closed by go-side Server

1
votes

I'm trying to connect my Erlang code and Go code with gRPC, during which I found that if I connect to gRPC too many times at the same time my socket would be closed by Go server, further stops my Erlang client (a gen_server). No error info was given to me, just a simple code in Erlang showing that http2_client socket closed by peer #Port<some port info>.

I'm sure that the limit of concurrent streams gRPC allowed is not reached (as my debug log did not show that we reached that place), and after a carefully look I found the problem occurred in google.golang.org\grpc\server.goserveStreams (line 830), and my logs showed that all the streams successfully reached var wg sync.WaitGroup, but could not finish remaining parts to come back to defer st.Close().

Could someone please kindly help me with this strange error, or at least give me some directions I should look into?

1
gotcpgrpcgrpc-go
Are you talking about this function? github.com/grpc/grpc-go/blob/master/server.go#L838 At a first glance it looks to me like Go is not closing the stream -- if your information is correct, that it doesn't come back to st.Close() – TehSphinX
Really sorry about not giving the source code, that is a very dumb mistake. I've been stuck in this for like two days and my mind is like... so... Anyway, I have updated the question. @TehSphinX – Chien

1 Answers

1
votes

I have managed to solve this problem now, and thus come to answer it.

It turned out that I was using a different log system for debugging, therefore the default Golang error messages are not collected and presented to me... Golang keeps screaming and shouting at me, but I was sitting in another room and cannot hear it.

To the socket problem, this is because the whole connection is closed, due to the lack of long connection support in my 3rd party library http2_client.erl. After manually add a heartbeat to each connection, all problems are solved and the gRPC works just fine.