How to avoid async.waterfall "TypeError: Cannot read property of undefined"?

Question

I am working on an async.waterfall that I am unsure of how to avoid: "TypeError: Cannot read property 'MediaUrl' of undefined". This TypeError doesn't occur each time the script is run.

The flow is as follows:

getName (randomly chooses a name from a list of names
searchImage (uses the Bing search API to search photographs associated with that name
processBotdata (takes the results from Bing and randomly chooses one of the search results)

Step 3 is where the issue occurs:

function processBotdata (searchData, callback) {

    var photographer = searchData.photographer // the search name
    var array        = searchData.array; // search results from bing
    var randomIndex  = Math.floor(Math.random() * array.length);
    var mediaUrl     = array[randomIndex].MediaUrl; // TypeError here!
    var sourceUrl       = array[randomIndex].SourceUrl;
    var searchData         = {
                           photographer,
                           mediaUrl,
                           sourceUrl
                          };
    fs.readFile('results.json', function (err, data) {
    var json = JSON.parse(data);
    json.push(['search results for ' + photographer + ': ',
                              'mediaUrl: ' + searchData.mediaUrl,
                              'sourceUrl: ' + searchData.sourceUrl]);

    fs.writeFile("results.json", JSON.stringify(json, null, "\t"));
    console.log(' ==========> searchData appended to results.json file...')
    });
    console.log(' ==========> searchData has now been processed for upcoming tweet...');
    setTimeout(function() {
      callback(null, searchData);
      console.log(searchData);
    }, 5000);
}

I implemented a setTimeout for this function hoping that would resolve the issue. My thinking was that the Bing results searchData.array was not yet available to be processed, i.e. randomized and selected in this function. As I am new to Node.js and JavaScript I am unsure of my programming error here. I saw this post and I am wondering if it is related to the Bing Search Array which returns the top 50 results.

Update: here is how the async.waterfall is called:

async.waterfall([
    getName,
    async.retryable([opts = {times: 5, interval: 500}], searchImage),
    async.retryable([opts = {times: 3, interval: 1000}], processBotdata),
    async.retryable([opts = {times: 3, interval: 500}], getImage)
],
function(error, result) {
    if (error) {
      console.error(error);
      return;
      }
});

How are you calling this function processBotdata? Maybe you are calling a callback before the result can be acquired. — Ricardo Souza
I updated my original post to help clarify. I am using the required pattern here github.com/caolan/async#waterfall — mmryspace
What does a console.log() on searchData returns inside the processBotdata() function? — Ricardo Souza
I forgot to state that this TypeError doesn't occur each time. I will update my post. console.log() on searchData returns the expected results when the typeError is not thrown, i.e. the photographers name, the mediaUrl, and the sourceUrl from one of the 50 Bing search results returned. — mmryspace

Paul Paul · Accepted Answer · 2016-06-01T00:52:41

The most obvious possibility is that there is no object at the array index that you're getting. Since you're going after a random index, it makes sense that some random numbers are hitting populated indices, others are not. Is there any chance that the array is sparse or that the random number you're generating is out of range despite your attempts with Math.floor?

I'd check first in any case, something like:

var obj = array[randomIndex];
if(obj){
 // do your stuff
}

/*** EDIT FOR NEW INFO ****/

Based on the comment, there's a chance that the Bing results will be an empty array, so the first thing you want to do is to test if that's the case and 'fail fast' to let async.retryable know it should retry:

function processBotData(searchData, callback){
    if(!searchData || !searchData.array || !searchData.array.length)
        return callback(new Error('No search results'));
    // continue as before from here
}

Note that some folks would prefer my last condition to be something like searchData.array.length > 0, but 0 evals as false so I do it this way.

How to avoid async.waterfall "TypeError: Cannot read property of undefined"?

1 Answers