Protobuf - java package name and sharing proto file across projects

Question

I have two java projects that produce and consume messages from Kafka, one project produces a message of type Ticker and another project consumes this message.

So I've created this file

syntax = "proto3";

message Ticker {
    string symbol = 1;
    float ask = 2;
    float bid = 3;
}

I know that in order to create Java objects for this message type, I use protoc

However, I have two problems.

1. I need a way of sharing the proto file across the project (the 2 projects that I have right now and any future project that would need access to Ticker). What is the best way to do it? I've heard something about git subtrees or submodules, is it a good/reliable way?

2. I need a way for protoc to generate the Java objects under some package, for a example, I have two projects, one is com.myorg.TickerProducer and another one is com.myorg.TickerConsumer I need protoc to generate the Java objects in the packages/namespaces com.myorg.TickerProducer.Entities and com.myorg.TickerConsumer.Entities respectively. I know that I can use option java_package inside the proto file, but I have different projects that use the same files and each want the Java objects to be generated under their namespaces. Is there a way to do it?

Thank you.

Answer 1

The sanest way to deal with this might be to have a separate package for these entities, which both the consumer and producer use. As long as you don't want the consumer to depend on the producer or vice-versa.

If you're using Gradle, there's a protobuf-plugin which integrates the building of the protobufs into the consumer project's code.

---

In general, there is a slight impedance mismatch between how protobuf's are meant to be used, due to the fact that Google has a single repository where every build unit / package can technically depend on every other build unit / package, and how they are used in the rest of the world, where projects have many repositories, and those act as a hard boundary for visibility and dependence between build units and packages.

If you can add them up to a package containing other common definitions etc. that would be best, but lacking that, a separate package, ugly as it is, might be your safest bet.

Answer 2

A single .proto file will always generate code in the same package/namespace, so if you want to have two different layouts in the two projects, you'll need two .proto files. Frankly, though, I'd argue that it is correct to have it in the same package/namespace in both: it still is the same thing, ultimately.

Note that as long as you aren't using the "any" feature, protobuf doesn't care much about the type-name / package / namespace at runtime; so an X.Y.Z will be interchangeable with an A.B.C as long as they are actually the same fundamental shape. Even field-names are irrelevant as long as you aren't using json.