Writing a JSON file after a lot of async requests

Question

i need to write the final object formed after a lot of async requests to a JSON file.

Here is the layout of my file

var finalObj = {};
for(i=0;i<100;i++){
  request.get("http://example.com/get/" +i,function(err,res,body){
    var tObj = json.parse(body);
    finalObj[tObj.name] = tObj.value;
  })
}

after all the response object is recieved, i want to save the finalObj to a json file. How do i do this?

Answer 1

Note that it should be JSON.parse(body) to parse the body correctly.

The easiest thing to do would be to use a counter which gets incremented upon completion, and detect whether you're handled all requests;

var finalObj = {};
var counter = 0;

for(i=0;i<100;i++){
  request.get("http://example.com/get/" +i,function(err,res,body){
    var tObj = JSON.parse(body);
    finalObj[tObj.name] = tObj.value;

    if (++counter === i) {
        require('fs').writeFile('foo.txt', JSON.stringify(finalObj), function (err) { 
            if (!err) {
                // celebrate
            }
        }); 
    }
  });
}

See the File System module for details on the file writing side of things; thankfully they've got a writeFile function which eases the process of writing to a file.

Answer 2

That will get data into that object, provided your JSON is correct, but as you said the requests are asynchronous. You will need to add code to monitor when all requests are completed and the object is ready to be written. Then, just write the Javascript object to JSON with JSON.stringify().

Answer 3

How about adding a counter to increment each time a request is received? When the counter matches the total number of requests, it's time to write.

var finalObj = {},
    requestCount = 100,
    ticks = 0;

for(i=0; i<requestCount; i++){
  request.get("http://example.com/get/" + i, function (err,res,body) {
    var tObj = json.parse(body);
    finalObj[tObj.name] = tObj.value;
    ticks++;

    // check if all 100 have been received
    if (ticks == requestCount) {
      // write file
    }
  })
}