How to use typescript protobuf generated files in a gRPC angular application

Question

Summarize the problem

I'm learning how to use gRPC, and wanted to try to do a client-server connection. The server (in Elixir) works, though I had a few problems. But as I am mainly a back end developer, I have way more troubles with implementing it in Angular, and would appreciate to have some help.

I am using Angular 8.2.9, Angular CLI 8.3.8 and Node 10.16.0 for this project.

What have I already done?

1. Created a new angular cli project with ng new test-grpc
2. Generated a module and a component with ng g..., and modify the base routing to have working pages and urls.
3. Installed Protobuf and gRPC libraries npm install @improbable-eng/grpc-web @types/google-protobuf google-protobuf grpc-web-client protoc ts-protoc-gen --save
4. Tooked the .proto file from my Elixir code and copied it in a folder src/proto
5. Added a protoccommand in the package.json scripts : "protoc": "protoc --plugin=protoc-gen-ts=./node_modules/.bin/protoc-gen-ts.cmd --js_out=import_style=commonjs,binary:src/app/proto-gen --ts_out=service=true:src/app/proto-ts -I ./src/proto/ ./src/proto/*.proto"
6. Called the command to generate the js and ts files found in src/app/proto-ts and src/app/proto-js
7. Replaced any _ with - in the names of the generated files.
8. Then I tried to add the following code in a ngOnInit: https://github.com/improbable-eng/grpc-web/tree/master/client/grpc-web#usage-overview such as:

const getBookRequest = new GetBookRequest();
getBookRequest.setIsbn(60929871);
grpc.unary(BookService.GetBook, {
  request: getBookRequest,
  host: host,
  onEnd: res => {
    const { status, statusMessage, headers, message, trailers } = res;
    if (status === grpc.Code.OK && message) {
      console.log("all ok. got book: ", message.toObject());
    }
  }
});

I tried:

• To create the Request in the constructor arguments.
• To find how to add typescript files (I only find ways to add javascript files in a angular project, and the generated javascript are way heavier than the typescript, and don't have any explanation).
• To understand what the jspb.Message is (I still don't, but I think it is linked to this file: https://github.com/protocolbuffers/protobuf/blob/master/js/message.js)
• Tried to find a tutorial on how to implement rGPD in an angular app (404 none found)

The .proto file, the two typescript generated files and the component trying to use them

The .proto file ():

// src/proto/user.proto
syntax = "proto3";

service UserService {
  rpc ListUsers (ListUsersRequest) returns (ListUsersReply);
}

message ListUsersRequest {
  string message = 1;
}


message ListUsersReply {
  repeated User users = 1;
}

message User {
  int32 id = 1;
  string firstname = 2;
}

The typescript generated code (2 files):

// src/app/proto-ts/user-pb.d.ts

import * as jspb from "google-protobuf";

export class ListUsersRequest extends jspb.Message {
  getMessage(): string;
  setMessage(value: string): void;

  serializeBinary(): Uint8Array;
  toObject(includeInstance?: boolean): ListUsersRequest.AsObject;
  static toObject(includeInstance: boolean, msg: ListUsersRequest): ListUsersRequest.AsObject;
  static extensions: {[key: number]: jspb.ExtensionFieldInfo<jspb.Message>};
  static extensionsBinary: {[key: number]: jspb.ExtensionFieldBinaryInfo<jspb.Message>};
  static serializeBinaryToWriter(message: ListUsersRequest, writer: jspb.BinaryWriter): void;
  static deserializeBinary(bytes: Uint8Array): ListUsersRequest;
  static deserializeBinaryFromReader(message: ListUsersRequest, reader: jspb.BinaryReader): ListUsersRequest;
}

export namespace ListUsersRequest {
  export type AsObject = {
    message: string,
  }
}

export class ListUsersReply extends jspb.Message {
  clearUsersList(): void;
  getUsersList(): Array<User>;
  setUsersList(value: Array<User>): void;
  addUsers(value?: User, index?: number): User;

  serializeBinary(): Uint8Array;
  toObject(includeInstance?: boolean): ListUsersReply.AsObject;
  static toObject(includeInstance: boolean, msg: ListUsersReply): ListUsersReply.AsObject;
  static extensions: {[key: number]: jspb.ExtensionFieldInfo<jspb.Message>};
  static extensionsBinary: {[key: number]: jspb.ExtensionFieldBinaryInfo<jspb.Message>};
  static serializeBinaryToWriter(message: ListUsersReply, writer: jspb.BinaryWriter): void;
  static deserializeBinary(bytes: Uint8Array): ListUsersReply;
  static deserializeBinaryFromReader(message: ListUsersReply, reader: jspb.BinaryReader): ListUsersReply;
}

export namespace ListUsersReply {
  export type AsObject = {
    usersList: Array<User.AsObject>,
  }
}

export namespace User {
  export type AsObject = {
    id: number,
    firstname: string,
  }
}

// src/app/proto-ts/user-pb-service.d.ts
import * as user_pb from "./user-pb";
import {grpc} from "@improbable-eng/grpc-web";

type UserServiceListUsers = {
  readonly methodName: string;
  readonly service: typeof UserService;
  readonly requestStream: false;
  readonly responseStream: false;
  readonly requestType: typeof user_pb.ListUsersRequest;
  readonly responseType: typeof user_pb.ListUsersReply;
};

export class UserService {
  static readonly serviceName: string;
  static readonly ListUsers: UserServiceListUsers;
}

export type ServiceError = { message: string, code: number; metadata: grpc.Metadata }
export type Status = { details: string, code: number; metadata: grpc.Metadata }
interface UnaryResponse {
  cancel(): void;
}
interface ResponseStream<T> {
  cancel(): void;
  on(type: 'data', handler: (message: T) => void): ResponseStream<T>;
  on(type: 'end', handler: (status?: Status) => void): ResponseStream<T>;
  on(type: 'status', handler: (status: Status) => void): ResponseStream<T>;
}
interface RequestStream<T> {
  write(message: T): RequestStream<T>;
  end(): void;
  cancel(): void;
  on(type: 'end', handler: (status?: Status) => void): RequestStream<T>;
  on(type: 'status', handler: (status: Status) => void): RequestStream<T>;
}
interface BidirectionalStream<ReqT, ResT> {
  write(message: ReqT): BidirectionalStream<ReqT, ResT>;
  end(): void;
  cancel(): void;
  on(type: 'data', handler: (message: ResT) => void): BidirectionalStream<ReqT, ResT>;
  on(type: 'end', handler: (status?: Status) => void): BidirectionalStream<ReqT, ResT>;
  on(type: 'status', handler: (status: Status) => void): BidirectionalStream<ReqT, ResT>;
}

export class UserServiceClient {
  readonly serviceHost: string;

  constructor(serviceHost: string, options?: grpc.RpcOptions);
  listUsers(
    requestMessage: user_pb.ListUsersRequest,
    metadata: grpc.Metadata,
    callback: (error: ServiceError|null, responseMessage: user_pb.ListUsersReply|null) => void
  ): UnaryResponse;
  listUsers(
    requestMessage: user_pb.ListUsersRequest,
    callback: (error: ServiceError|null, responseMessage: user_pb.ListUsersReply|null) => void
  ): UnaryResponse;
}

The attempt to use them in a component file:

// src/app/modules/user/pages/list/list.component.ts
import { Component, OnInit } from '@angular/core';
import { grpc } from "@improbable-eng/grpc-web";

import { UserService } from '../../../../proto-ts/user-pb-service.d'
import { ListUsersRequest } from '../../../../proto-ts/user-pb.d'

@Component({
  selector: 'app-list',
  templateUrl: './list.component.html',
  styleUrls: ['./list.component.scss']
})
export class ListComponent implements OnInit {

  constructor() { }

  ngOnInit() {
    const listUsersRequest = new ListUsersRequest();
    listUsersRequest.setMessage("Hello world");

    grpc.unary(UserService.ListUsers, {
      request: listUsersRequest,
      host: "0.0.0.0:50051",
      onEnd: res => {
        const { status, statusMessage, headers, message, trailers } = res;
        if (status === grpc.Code.OK && message) {
          console.log("all ok. got the user list: ", message.toObject());
        } else {
          console.log("error");
        }
      }
    });
  }
}

Expected result and actual results

I expect to be able to use non angular typescript code in a component (or a service). As the code above (but the component file) is generated, it shouldn't be modified, because any changes on the .proto file will overwrite any modification done in these two generated files.

For now, I am blocked by these two error message, which appear depending of which file I saved last: TypeError: _proto_ts_user_pb_d__WEBPACK_IMPORTED_MODULE_4__.ListUsersRequest is not a constructor (if I save a protobuf generated file last, no error in the console) or src\app\proto-ts\user-pb-service.d.ts and src\app\proto-ts\user-pb.d.ts is missing from the TypeScript compilation. Please make sure it is in your tsconfig via the 'files' or 'include' property.(if I saved a angular file last, same error message in the console).

Adding the 'files' or 'include' in the tsconfig.json (which is the generated by angular CLI without any modifications) create another error: Refused to load the image 'http://localhost:4200/favicon.ico' because it violates the following Content Security Policy directive: "default-src 'none'". Note that 'img-src' was not explicitly set, so 'default-src' is used as a fallback.

Answer 1

This is not the best way to do it. More of a work around (if your project is not big already big).

I came back recently to this problematic. Didn't want to try new things, so I just created a new angular project following these steps:

1. Put gRPC first
2. Check if everything worked
3. Add a module from the initial project
4. Repeat 2 and 3 until there is no more module to add

Now it works for me.

As I say, it isn't the best solution as the initial problem remains in the initial project. But this is better than nothing.

Answer 2

I realize this question is old and you have kind of solved the problem already.

But I would like to add two points:

1. The grpc-web stack is overly complicated. The protobuf JavaScript API is sparsely documented, grpc-web is no better, and generating TypeScript declarations on top only makes matters worse.

2. There are alternatives to the official stack! ts-proto for example generates pure TypeScript.

I was having very similar experiences with grpc web and Angular and started protobuf-ts out of frustration. It is a protoc-plugin that generates TypeScript and supports grpc-web. It is written from scratch and might be a viable alternative...

Answer 3

Error here:

import { UserService } from '../../../../proto-ts/user-pb-service.d'
import { ListUsersRequest } from '../../../../proto-ts/user-pb.d'

correct import: (remove ".d" at the end of Module name)

Note that *.js and *.d.ts must be the same folder.

import { UserService } from '../../../../proto-ts/user-pb-service'
import { ListUsersRequest } from '../../../../proto-ts/user-pb'

package.json: option compile ts_out=service=grpc-web:

  "scripts": {
    "grpc:all": "protoc --plugin=protoc-gen-ts=node_modules\\.bin\\protoc-gen-ts.cmd --js_out=import_style=commonjs,binary:./src/app/proto/generated --ts_out=service=grpc-web:./src/app/proto/generated -I ./src/app/proto ./src/app/proto/*.proto"
  },

The above compile option is for grpc with lib "@improbable-eng/grpc-web" see the guide: ts-protoc-gen

Your code should be like:

// src/app/modules/user/pages/list/list.component.ts
import { Component, OnInit } from '@angular/core';
import { grpc } from "@improbable-eng/grpc-web";

import { UserService } from '../../../../proto-ts/user-pb-service'
import { ListUsersRequest } from '../../../../proto-ts/user-pb'

@Component({
  selector: 'app-list',
  templateUrl: './list.component.html',
  styleUrls: ['./list.component.scss']
})
export class ListComponent implements OnInit {

  constructor() { }

  ngOnInit() {
    // build ProtoBuf Message to send
    const listUsersRequest = new ListUsersRequest();
    listUsersRequest.setMessage("Hello world");

    // ClientName = ServiceName + "Client"
    const service = new UserServiceClient('http://localhost:8080');
    
    // build headers of httpRequest
    // gRPC base on http2 protocol
    let metadata = new grpc.Metadata();

    metadata.set('header1', 'headerValue1');
    metadata.set('header2', 'headerValue2');

    // call gRPC service method
    service.ListUsers(listUsersRequest , metadata, (error, reply) => {
      if (error) {
        console.log(error)
        return;
      }

      console.log(reply));
    });
  }
}