how do I use set timeout inside forEach with promise/dynamoDB delete?

Question

Since I need to delete lots of item from dynamoDB, I write forEach and call delete item function inside like below, but it would hit the capacity limit(it means I need to slow down to send the query/delete request) of my dynamoDB, and i want to let it delay some seconds between each call.

arrayOfitems.forEach( item => {
   return docClient.delete(params).promise();                     
});

I tried to use npm delay https://www.npmjs.com/package/delay as below, but it would execute after 2 seconds, then still execute all the delete(foreach loop) instead of wait 2 seconds and call the delete every time.

arrayOfitems.forEach( item => {
    delay(2000)
    .then(() => {
         return docClient.delete(params).promise(); 
    });
});

I heard some people suggest to use Promise.Map or P.Map, but after I google it, I saw the example as below, and I'm not sure if this is fit in my case. As I don't use Promise.all or have array of promise, I don't really know how to use this in my case. Could anyone has better idea to solve this problem?

const pMap = require('p-map');
const got = require('got');

const sites = [
	getWebsiteFromUsername('sindresorhus'), //=> Promise
	'ava.li',
	'todomvc.com',
	'github.com'
];

const mapper = el => got.head(el).then(res => res.requestUrl);

pMap(sites, mapper, {concurrency: 2000}).then(result => {
	console.log(result);
	//=> ['http://sindresorhus.com/', 'http://ava.li/', 'http://todomvc.com/', 'http://github.com/']
});

Matt Way Matt Way · Accepted Answer · 2018-05-10T22:00:24

What is sounds like you need to do is build a promise chain of all your deletes spliced with your desired delays. This will keep your sequence synchronous. You could do something like this:

const delay = time => {
  return new Promise((resolve, reject) => {
    setTimeout(() => { resolve() }, time)
  }
}

// create the start of the promise chain
var chain = Promise.resolve()
// go through each item and append a delay and the work (not sure where you get params from)
arrayOfItems.forEach(item => {
  chain = chain.then(() => docClient.delete(params).promise())
               .then(() => delay(2000))
})

Now this is just an untested example, and has issues like adds a two second delay after the final deletion. But you should get the idea. The key is recursively using the last promise as the starting point of the next promise.

how do I use set timeout inside forEach with promise/dynamoDB delete?

2 Answers