Unable to use google/apis/annotations

Question

I try to use REST over gRPC using google api annotations. Unfortunately, I'm facing a protoc issue telling me annotations.proto is not exists or had errors. I tried several fixes in vain.

I even tried to reinstall a complete stack in case I did something wrong. I will detail you as much as possible the full lines and files I have set from my fresh install.

From a fresh go install VM I type these shell lines :

$ mkdir sources/golang

$ echo 'export GOPATH=$HOME/sources/golang' >> $HOME/.zshrc

$ source ~/.zshrc

$ cd sources/golang

$ mkdir src

$ cd src

$ export PATH=$PATH:$GOPATH/bin

$ go get -u google.golang.org/grpc

$ go get -u github.com/golang/protobuf/protoc-gen-go

$ go get -u github.com/grpc-ecosystem/grpc-gateway/protoc-gen-grpc-gateway

$ go get -u github.com/grpc-ecosystem/grpc-gateway/protoc-gen-swagger

$ mkdir -p test/proto/test

$ cd test/proto/test

$ vim test.proto

In My test.proto file, I wrote very basic lines :

syntax = "proto3";

package main;

import "google/api/annotations.proto";

service Tester {
    rpc Test (Request) returns (Reply) {
        option (google.api.http) = { get: "/v1/test" };
    }
}

message Request {
    string Message = 1;
}

message Reply {
    string Message = 1;
}

Then

$ cd $GOPATH/src

$ vim main.go

In my main.go, very basic too :

package main

import (
        tc "test/proto/test"
        "context"
        "fmt"
        "log"
        "net"

        grpc "google.golang.org/grpc"
        codes "google.golang.org/grpc/codes"
)

func main() {
        err := StartServer("tcp", "127.0.0.1:50051")
        if err != nil {
                fmt.Printf("Error!! %s", err)
        }
}

type Server struct {
        tc.UnimplementedTesterServer
}

func (s *Server) Test(ctx context.Context, in *tc.Request) (*tc.Reply, error) {
        return &tc.Reply{Message: ""}, nil
}

func StartServer(protocol string, port string) error {
        lis, err := net.Listen(protocol, port)
        if err != nil {
                fmt.Printf("failed to listen: %v", err)
        }
        s := grpc.NewServer()
        tc.RegisterTesterServer(s, &Server{})

        error := s.Serve(lis)

        if error != nil {
                fmt.Printf("failed to serve: %v", error)
                return error
        }

        return nil
}

Finally, I try to compile my proto files :

$ protoc --proto_path=.:$GOPATH/src --go_out=plugins=grpc:. proto/*/*.proto

And I systematically have the following error :

proto/test/test.proto:5:1: Import "github.com/grpc-ecosystem/grpc-gateway/third_party/googleapis/google/api/annotations.proto" was not found or had errors.

When I see files get by go get ..., they are placed under $GOPATH/pkg/mod/

For example the googleapis' annotations.proto file is under : $GOPATH/pkg/mod/github.com/grpc-ecosystem/[email protected]/third_party/googleapis/google/api/annotations.proto

Maybe is it the cause?

Answer 1

Mixing up your protobuf commands and your go commands is making this more complicated than it needs to be. Just focus on the protobuf to handle the error.

You are importing "google/api/annotations.proto"

Your proto_path is --proto_path=.:$GOPATH/src

So that means that when you execute protoc you should have the file located at ./google/api/annotations.proto or $GOPATH/src/google/api/annotations.proto

Answer 2

I faced the same problem and I was trying to automate a solution that would work on any Linux with as little dependencies as possible.

The official docs for grpc_gateway states:

You will need to provide the required third party protobuf files to the protoc compiler. They are included in this repo under the third_party/googleapis folder, and we recommend copying them into your protoc generation file structure. If you've structured your proto files according to something like the Buf style guide, you could copy the files into a top-level ./google folder.

In other words: download it somehow and make it work.

Also, since this quote is also a quote in the original source, it is not very visible which made me skip it the first 2 times I read it looking for a solution, which is bad.

So, as I argued on the comments of @SeanF answer, using go get is not a maintainable option since it saves the project in a folder whose name contains the latest version, which would cause headaches to maintain when the version changes.

So the best option is actually to clone the grpc-gateway project:

git clone https://github.com/grpc-ecosystem/grpc-gateway
protoc -I grpc-gateway/ \
    -I grpc-gateway/third_party/googleapis \
    --go_out=plugins=grpc:. proto/*/*.proto

I also wrote a gist with my solution which depends only on docker and bash and, thus, should be stable:

https://gist.github.com/VinGarcia/43dfa71b412c16b9365c224d3760af5e

There is a problem with it, where I am using go.mod but discarding it after every execution, which is not good. But it works well enough for now and it might be helpful to some people.

Regards

Answer 3

The gRPC-Gateway v2.3.0 has removed the third_party folder, see v2.3.0 Release Note.

Below are some solutions that may work:

Solution 1:

Switch to buf from protoc.

If you are using Buf, then you need to add these lines to your buf.yaml file, directly underneath the version tag:

name: buf.build/yourorg/myprotos
deps:
  - buf.build/beta/googleapis

Additionally, you must always run buf beta mod update after adding a dependency to your buf.yaml.

The issue is that you are using too many dashes. So you need --go-grpc_out. But this sort of thing is a bit easier with the Buf.

Solution 2:

Use an older version. gRPC-Gateway <= v2.2.0 in your go.mod.

Solution 3:

Copy google/api/annotations.proto and google/api/http.proto to your project. Checkout previous version v2.2.0 you can get these files.

You should check out the tutorial series on gRPC-Gateway, i.e., https://grpc-ecosystem.github.io/grpc-gateway/docs/tutorials/. Also, you can refer to my simple hello world program, which uses gRPC-Gateway, i.e., https://github.com/iamrajiv/helloworld-grpc-gateway.