protoc generated grpc files and gateway files end up in different paths?

Question

I'm a bit baffled about this. File structure:

/Project
generate.sh
  gateway/
      cmd/
          main.go
      pkg/
          gatewaypb/
      proto/ 
         service.proto

My generate.sh appears as such:


set -e

module="github.com/my-org/grpc-project"
// Find all dirs with .proto files in them
for proto in */proto; do

    dir=$(echo "$proto"|cut -d"/" -f1)
    echo "regenerating generated protobuf code for ${proto}"
    protoc --proto_path .\
           --go-grpc_out=. --go-grpc_opt=module=${module}\
           --go_out=. --go_opt=module=${module}\
           "${proto}"/*.proto
    echo "creating reverse proxy protobuf code for ${proto}"

    protoc -I . --grpc-gateway_out=.\
     --grpc-gateway_opt logtostderr=true \
     --grpc-gateway_opt paths=source_relative \
     "${proto}"/*.proto
done

Correctly enough, the first protoc does as intended, while the second one does not. They output files in different paths when they are supposed to be in the same.

After running generate.sh, I am left with:

/Project
generate.sh
  Service/
      cmd/
          main.go
      pkg/
          gatewaypb/
              *gateway.pb.go
              *gateway._grpc.pb.go
      proto/ 
         *gateway.pb.gw.go
         service.proto

Why does the generated gateway file end up in the proto/ directory?

What do I need to give as a value to grpc-gateway-out for it to end up in /pkg/gatewaypb as the other generated grpc files?

My service.proto file:

syntax = "proto3";
package gateway.api.v1;
option go_package = "github.com/my-org/grpc-project/gateway/pkg/gatewaypb";

import "google/api/annotations.proto";

message StringMessage {
  string value = 1;
}

service YourService {
      rpc Echo(StringMessage) returns (StringMessage) {
          option (google.api.http) = {
              post: "/v1/example/echo"
              body: "*"
            };
      }
}

Try --grpc-gateway_opt module=${module} (replacing --grpc-gateway_opt paths=source_relative). Doing this should mean the same logic is used as the other plugins (more info in this issue). — Brits
That worked, thank you so much! Feel welcome to put in an answer with a reasoning and I'll make sure to accept it! — cbll

Brits Brits · Accepted Answer · 2021-02-15T19:33:23

As per the comments you were using the module option (--go_opt=module=${module} & --go-grpc_opt=module=${module}) when generating the protobuf & gRPC files. The option you were using when generating the gateway files (--grpc-gateway_opt paths=source_relative) uses a different algorithm to work out where to store the files; the fix being to use the module option - --grpc-gateway_opt module=${module}.

Here is what the docs have to say about the three options (these specific options are for protobuf generation; the other generators have the same functionality but replace go_ with go-grpc_/grpc-gateway_):

By default, the output file is placed in a directory named after the Go package's import path. For example, a file protos/foo.proto with the above go_package option results in a file named example.com/foo/bar/foo.pb.go.
If the --go_opt=module=$PREFIX flag is given to protoc, the specified directory prefix is removed from the output filename. For example, a file protos/foo.proto with the go_package option "example.com/foo/bar" and the flag --go_opt=module=example.com/foo results in a file named bar/foo.pb.go.
If the --go_opt=paths=source_relative flag is given to protoc, the output file is placed in the same relative directory as the input file. For example, the file protos/foo.proto results in a file named protos/foo.pb.go.

protoc generated grpc files and gateway files end up in different paths?

1 Answers