grpc status code UNAVAILABLE and details as OS error

0
votes

Getting intermittent error, only when the container is brought up using docker stack deploy, working fine using docker-compose ` grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with: status = StatusCode.UNAVAILABLE details = "OS Error" debug_error_string = "{"created":"@1565004909.399609656", "description":"Error received from peer", "file":"src/core/lib/surface/call.cc", "file_line":1036, "grpc_message":"OS Error", "grpc_status":14}" 

channel = implementations.insecure_channel(host, int(port))
stub = prediction_service_pb2.beta_create_PredictionService_stub(channel)
request = predict_pb2.PredictRequest()
result = stub.Predict(request, 100.0)`
1
dockergrpctensorflow-serving

1 Answers

1
votes

On solution what is working is adding the retry logic, catch the grpc error and call stub.Predict again

try:
        result = stub.Predict(request, 100.0)  # 100 seconds
    except grpc.RpcError as e:
        result = stub.Predict(request, 100.0)  # 100 seconds