0
votes

Evolving from Javascript, spliced FileReader for large files with Promises, how?, which showed me how a Promise can also resolve a function, now I am stuck with the same but inside an Array.reduce function.

The goal is that I want to upload a file (which already does) within an array, where each array item (a file) is uploaded sequentially (i.e. controlled through promises).

Then, I understand that the answer is somehow in http://www.html5rocks.com/en/tutorials/es6/promises/?redirect_from_locale=es , but I cannot understand how to apply that to here. My array is not an array of promises, is an array of files. Well, the whole thing is still obfuscated to me.

This is my code, which would work if I could see the ein console.log message:

return myArray.reduce(function(previous, current) {
    var BYTES_PER_CHUNK = 100000;
    var start = 0;
    var temp_end = start + BYTES_PER_CHUNK;
    var end = parseInt(current.size);
    if (temp_end > end) temp_end = end;
    var content = ''; // to be filled by the content of the file
    var uploading_file = current;
    Promise.resolve().then(function() {
        return upload();
    })
    .then(function(content){
        // do stuff with the content
        Promise.resolve();
    });
},0)  // I add the 0 in case myArray has only 1 item
//},Promise.resolve()) goes here?

.then(function(){
    console.log('ein') // this I never see
});

function upload() {
  if (start < end) {
    return new Promise(function(resolve){
      var chunk = uploading_file.slice(start, temp_end);
      var reader = new FileReader();
      reader.readAsArrayBuffer(chunk);
      reader.onload = function(e) {
        if (e.target.readyState == 2) {
          content += new TextDecoder("utf-8").decode(e.target.result);
          start = temp_end;
          temp_end = start + BYTES_PER_CHUNK;
          if (temp_end > end) temp_end = end;
          resolve(upload());
        }
      }
    });
  } else {
    uploading_file = null;
    return Promise.resolve(content);
  }
}
  • updated after several comments, it seems that now it works ... not sure yet

    var uploading_file, start, temp_end, end, content; var BYTES_PER_CHUNK = 100000;

    myArray.reduce(function(previous, current) { return previous .then(function() { BYTES_PER_CHUNK = 100000; start = 0; temp_end = start + BYTES_PER_CHUNK; end = parseInt(current.size); if (temp_end > end) temp_end = end; content = ''; uploading_file = current;

    upload()
    .then(function(content){
        // do stuff with "content"
        console.log('here')
        return Promise.resolve();
    });
    

    }); },Promise.resolve()) .then(function(){ console.log('ein'); });

    function upload() { if (start < end) { return new Promise(function(resolve){ var chunk = uploading_file.slice(start, temp_end); var reader = new FileReader(); reader.readAsArrayBuffer(chunk); reader.onload = function(e) { if (e.target.readyState == 2) { content += new TextDecoder("utf-8").decode(e.target.result); start = temp_end; temp_end = start + BYTES_PER_CHUNK; if (temp_end > end) temp_end = end; resolve(upload()); } } }); } else { uploading_file = null; return Promise.resolve(content); } }

  • improved code, seems to work, perhaps easier to read?

        var start, temp_end, end;
        var BYTES_PER_CHUNK = 100000;
    
        myArray.reduce(function(previous, current) {
            return previous
            .then(function() {
                start = 0;
                temp_end = start + BYTES_PER_CHUNK;
                end = parseInt(current.size);
                if (temp_end > end) temp_end = end;
                current.data = '';
    
                return upload(current)
                .then(function(){
                    // do other stuff
                    return Promise.resolve();
                });
            });
        },Promise.resolve())
        .then(function(){
          // do other stuff
        });
    
        function upload(current) {
            if (start < end) {
                return new Promise(function(resolve){
                    var chunk = current.slice(start, temp_end);
                    var reader = new FileReader();
                    reader.readAsText(chunk);
                    reader.onload = function(e) {
                        if (e.target.readyState == 2) {
                            current.data += e.target.result;
                            start = temp_end;
                            temp_end = start + BYTES_PER_CHUNK;
                            if (temp_end > end) temp_end = end;
                            resolve(upload(current));
                        }
                    }
                });
            } else {
                return Promise.resolve();
            }
        }
    
1
What is the point of using Array reduce if you never even refer to the previous parameter - you're basically doing a forEach with bonus overhead ... - Jaromanda X
You slay me @T.J.Crowder - now I need to know what is the current best practice when chaining asynchronous tasks together based on an array ... no need for details, just a hint will do me :p - Jaromanda X
@T.J.Crowder, no offenses taken, but could you expand a little bit more? Apart from the Array.reduce, the rest of the code is wrong? I cannot find another way to do sequential file uploading when these have to be spliced in chunks. How would you do this? - GWorking
@JaromandaX, you're right, I've got the impression that I had to use Array.reduce for the sequential uploading, maybe I do have to use it but I am doing it wrong here - GWorking
@Gerard: A couple of things jump out: 1. Promise.resolve().then(function() { return upload(); }) is basically just upload(), as upload returns a promise. 2. Promise.resolve(); when you don't use the return value is a no-op. - T.J. Crowder

1 Answers

3
votes

You're very close! You need to use the previous value; it should be a promise. Set the initial value of the reduce to be Promise.resolve(). Then inside the reduce function, instead of Promise.resolve().then(...). you should have something like:

return previous
  .then(function() { return upload(current); })
  .then(function() { /* do stuff */ });

It's important that you return here. This will become previous next time the reduce function is called.


You have quite a few issues with the upload function. The biggest issue is that the way you are passing it variables makes it very hard to read :) (and error-prone!)

If you're only reading a text file, use readAsText instead. Note I've renamed it to readFile, because that's a more accurate name.

// returns a promise with the file contents
function readFile(file) {
    return new Promise(function (resolve) {
        var reader = new FileReader();
        reader.onload = function(e) {
            resolve(e.target.result);
        };
        reader.readAsText(file);
    };
}

Then your reduce is simply:

files.reduce(function(previous, file) {
    return previous
      .then(function() { return readFile(file); })
      .then(function(contents) {
          // do stuff
      });
}, Promise.resolve());

You have a big bug with the upload_file variable though. The variable is local to the scope of the reduce function, so it will but undefined inside upload. Pass that in as an argument instead:

function upload(upload_file) { ... }

Side note on var. This is why even though you set upload_file with var inside the reduce function, it would be whatever it was before that function was called for upload:

var a = 3;

function foo() {
  var a = 4;
  console.log(a); // 4
}

foo();
console.log(a); // 3