How to get the same output from Buffer.from as fs.readFile?

Question

I am working on a feature for my slackbot to send a post request of binary data from an image to a ticketing system called servicenow was struggling to get the correct output from Nodejs for a little bit. I am getting the image from a url using axios, then sending the data to Buffer.from and using binary, this does not give me the correct output I need. However using fs.readFile also seems to return a Buffer object and this is the output I need.

The docs for fs.readFile says that encoding is set to null for readFile. The docs for Buffer.from says the default encoding is utf8

const fs = require("mz/fs");
const axios = require("axios");

const main = async () => {
  try {
    const attachImage = await axios.get(
      "https://d17fnq9dkz9hgj.cloudfront.net/uploads/2013/09/cat-black-superstitious-fcs-cat-myths-162286659.jpg"
    );

    const { data } = attachImage;

    // console.log(attachImage);

    const imgBuffer = Buffer.from(data, "binary");

    console.log(imgBuffer);

    const testImg = await fs.readFile("./cat1.jpg");

    console.log(testImg);


  } catch (err) {
    console.error(err);
  }
};

main();

The second console.log output(the fs.readFile) is the desired output how can I make buffer output of the blob with this exacted output from a GET request

$ node index.js
<Buffer fd fd fd fd 0c fd 45 78 69 66 00 00 4d 4d 00 2a 00 00 00 08 00 0e 01 00 00 03 00 00 00 01 0f 30 00 00 01 01 00 03 00 00 00 01 0a 20 00 00 01 02 00 03 ... 108862 more bytes>
<Buffer ff d8 ff e1 0c 9f 45 78 69 66 00 00 4d 4d 00 2a 00 00 00 08 00 0e 01 00 00 03 00 00 00 01 0f 30 00 00 01 01 00 03 00 00 00 01 0a 20 00 00 01 02 00 03 ... 114344 more bytes>

jakemingolla jakemingolla · Accepted Answer · 2019-08-06T17:42:09

I believe you're running to an encoding issue from the axios.get() method. Rather than create a buffer from the response, axios can provide one for you. Try changing your request to:

const fs = require("mz/fs");
const axios = require("axios");

const main = async () => {
  try {
    const attachImage = await axios.get(
      "https://d17fnq9dkz9hgj.cloudfront.net/uploads/2013/09/cat-black-superstitious-fcs-cat-myths-162286659.jpg",
      {
        responseType: 'arraybuffer',
        responseEncoding: 'binary'
      }
    );

    const { data } = attachImage;
    console.log(data);

    const testImg = await fs.readFile("./cat1.jpg", { encoding: null });
    console.log(testImg);


  } catch (err) {
    console.error(err);
  }
};

main();

They were identical for me when I ran this locally.

$ node test.js
<Buffer ff d8 ff e1 0c 9f 45 78 69 66 00 00 4d 4d 00 2a 00 00 00 08 00 0e 01 00 00 03 00 00 00 01 0f 30 00 00 01 01 00 03 00 00 00 01 0a 20 00 00 01 02 00 03 ... >
<Buffer ff d8 ff e1 0c 9f 45 78 69 66 00 00 4d 4d 00 2a 00 00 00 08 00 0e 01 00 00 03 00 00 00 01 0f 30 00 00 01 01 00 03 00 00 00 01 0a 20 00 00 01 02 00 03 ... >

See the axios.get() options documentation for more detail.

How to get the same output from Buffer.from as fs.readFile?

1 Answers