Fetch blob from URL and write to file

Question

I'm trying to fetch some binary data (MP3) from a server and then store it in a file:

var fs = require ('fs');
var fetch = require ('node-fetch');

fetch (
   audioUrl,
   {
      method: 'GET',
      headers: { 'Accept': '*/*' }
   }
)
.then ((res) => res.blob())
.then ((blob) => {
    console.log ('Blob size: ', blob.size);

    fs.writeFile (
        `filename.mp3`,
        blob,  // BUG HERE
        (err) => {
            if (err) {
                console.log (`Error writing audio ${audioIdx}`, err);
            }
        }
    );
})

The problem is marked at BUG HERE. I'm passing a blob to fs, which doesn't understand blobs, and simply writes [object Blob]. I also tried the following:

blob                                // writes [object Blob]
new Uint8Array (blob)               // writes empty file
Buffer.from (new Uint8Array (blob)) // writes empty file
new Buffer (blob, 'binary')   // Error: argument is not a Buffer

None of the above work. How to do this correctly?

Note that I'm logging blob.size before calling writeFile. blob.size shows the correct size, so fetch seems to be successful.

Answer 1

Ok, so from the MDN Docs a Blob is not a buffer, which is what you need for writing to the file system.

I think you might be better off just getting the buffer from the response, like:

.then(res => res.buffer())
.then(/* write your file here */)

though depending on how big the files are, your best option might be to skip the .writeFile function altogether and pipe streams instead.

Answer 2

You can stream the response to a file using fs.createWriteStream.

fetch(
    audioUrl,
    {
        method: 'GET',
        headers: { 'Accept': '*/*' }
    }
).then(res => {
    new Promise((resolve, reject) => {
        const dest = fs.createWriteStream(`filename.mp3`);
        res.body.pipe(dest);
        res.body.on('end', () => resolve());
        dest.on('error', reject);
    })
});

Answer 3

With the use Buffer API in nodejs

const fetch = require('node-fetch');
const Buffer = require('buffer').Buffer 
const fs = require('fs');
fetch("https://example.com/file.mp3", {
  "headers": {},
  "method": "GET"
}).then(function(resp) {
    return resp.blob();
}).then(async function(blob) {
    var buffer = await blob.arrayBuffer();
    buffer = Buffer.from(buffer)
    fs.createWriteStream('file.mp3').write(buffer);
})