Passing a base64 image string to a Firebase callable cloud function that calls the Cloud Vision API

Question

I'm implementing a web app the uses the Cloud Vision API to detect text on user-generated images.

I'm using React and Firebase cloud functions.

The flow is the following:

User gets image from Gallery or Camera, crops it using react-image-crop package
This package uses a <canvas> element to generate the cropped images
I'm converting the image to base64 using canvas.toDataURL("image/jpeg",1.0);
I'm passing the imageBase64 string to a cloud function
The cloud function needs to read the image and call the Cloud Vision API

CLIENT CODE

const canvasBase64 = canvas.toDataURL("image/jpeg", 1.0);

const result = await firebase.functions().httpsCallable("readTextFromImage")({
  image: canvasBase64
});

setTextareaValue(result.data.text);

CLOUD FUNCTION CODE

const Vision = require('@google-cloud/vision');
const vision = new Vision.ImageAnnotatorClient();

async function readTextFromImage(data)  {

  const imgBase64 = data.image;
  const [textDetections] = await vision.textDetection(imgBase64);

  // OTHER THINGS I'VE TRIED HERE
  
  const buffer = new Buffer(imgBase64, 'base64');
  const arraybuffer  = Uint8Array.from(buffer).buffer;
  const [textDetections] = await vision.textDetection(buffer);
  const [textDetections] = await vision.textDetection(arraybuffer);
}

NOTE:

The base64 image seems to be generated correctly.

From Google docs on the Node.js Vision API:

https://googleapis.dev/nodejs/vision/latest/v1.ImageAnnotatorClient.html#textDetection

We get that:

When trying with the buffers, I was getting no image present and passing the base64 string I'm getting code: ENAMETOOLONG

QUESTION

How should I convert the base64 string into something the the Cloud Vision API will accept?

cbdeveloper cbdeveloper · Accepted Answer · 2020-07-21T11:30:51

This is how I've solved it.

I opened a Github issue where I got into more details about this.

https://github.com/googleapis/nodejs-vision/issues/808

CLIENT CODE

I had to remove the first part of the base64 string.

This part: "data:image/jpeg;base64,"

const result = await firebase.functions().httpsCallable("readTextFromImage")({
  image: canvasBase64.split("base64,")[1]  // <--- REMOVES THE FIRST PART OF THE STRING
});

CLOUD FUNCTION CODE*

Also I had to call textDetection with an object with { image: { content: base64String } }

const [textDetections] = await vision.textDetection(
  {
    image: {
      content: imgBase64
    }
  }
);

Even though this goes against the documentation on:

https://googleapis.dev/nodejs/vision/latest/v1.ImageAnnotatorClient.html#textDetection

But I got the idea from this other piece of doc:

https://cloud.google.com/vision/docs/base64

Passing a base64 image string to a Firebase callable cloud function that calls the Cloud Vision API

1 Answers