unable to exit my process after completion. NodeJS

Question

I'm working on a Node JS program to connect to a remote SFTP server to copy log files. I'm using this NPM package. ssh2-sftp-client

const main = () => {
  const servers = ['server list'];
  servers.forEach((s) => {
    const sftp = new Client();
    // Connect to the server
    sftp.connect({
      host: s,
      port: '22',
      username: '',
      password: '',
    }).then(() => logger.log(`Connected to ${s}`))
      // get list of directories
      .then(() => sftp.list(rootLogPath))
      .then((dirs) => {
        dirs.forEach((d) => {
          const target = createTargetDirectory(d.name);
          // list all files in the directory
          sftp.list(`${rootLogPath}/${d.name}`)
            .then((files) => {
              // filter only today's files
              const todayFiles = files.filter((f) => f.type === '-' && moment().format('MMDDYYYY') === moment(f.modifyTime).format('MMDDYYYY'));
              // copy today's files into target
              todayFiles.forEach((f) => {
                sftp.get(`${rootLogPath}/${d.name}/${f.name}`, `${target}/${f.name}`)
                  .then(() => logger.log(`Copied ${f.name} from ${d.name} located on ${s}`));
              });
            });
        });
        return sftp.end();
      })
      .catch(() => logger.log(`Connection to ${s} failed`));
  });
};

main();

The code is working as expected but the problem is I'm not able to terminate the session. The program just waits after its completed the copy files operation. The sftp.end() call is prematurely closing out connection before copying is complete. If I remove that program just waits after copy is completed.

I'm not sure where to write the sftp.end() line to terminate the program.

EDIT 1: Updated code using Promise.all and making use of async/await now.

Split my code into pieces and now making use of async/await for better readability. Program now ends fine. no more waiting or hanging but the problem is files are not copied. I see a console message "Copied files from source directory".

const copyTodayFiles = async (src) => {
  try {
    let fileList = await sftp.list(`${rootLogPath}/${src}`);
    fileList = fileList.filter(
      (f) => f.type === '-' && moment().format('MMDDYYYY') === moment(f.modifyTime).format('MMDDYYYY'),
    );
    const target = createTargetDirectory(src);

    if (target) {
      fileList.forEach(async (f) => {
        try {
          await sftp.get(`${rootLogPath}/${src}/${f.name}`, `${target}/${f.name}`);
          logger.log(`Copied ${f.name}`);
        } catch (error) {
          logger.log(`Failed to copy ${f.name}`);
        }
      });
      console.log(`Copied files from ${src}`);
    }
  } catch (error) {
    logger.log(`Failed to read files from ${src}`);
    logger.log(error);
  }
};

const workOn = async (server) => {
  sftp = new Client();

  const config = {
    host: server,
    port: '22',
    username: '',
    password: '',
  };

  try {
    await sftp.connect(config);
    logger.log(`Connection to ${server} is successful.`);
    const logDir = await sftp.list(rootLogPath);
    Promise.all(logDir.map((d) => copyTodayFiles(d.name))).then(() => sftp.end());
    // logDir.forEach(async (d) => copyTodayFiles(d.name));
  } catch (error) {
    logger.log(`Connection to ${server} failed.`);
    logger.log(error);
  }
};

const main = () => {
  const servers = ['server list'];
  Promise.all(servers.map((s) => workOn(s)));
};

main();

you are using a forEach loop which executes synchronously and doesn't save function return values, and immediately call sftp.end() when forEach returns. Look up questions on the use of forEach in asynchronous code before proceeding. :-) — traktor

machineghost machineghost · Accepted Answer · 2020-03-17T22:59:06

In short, when you have a "chain" of promises (a.then(() => b.then(() =>...) and you want to do something at the end of them, you simply add that thing to the last .then callback.

However, if you're making multiple promises, you need to wait until all of them are finished, which you can do using Promise.all (https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/all):

Promise.all([promise1, promise2]).then(...

To do that, you'll need to convert all of your forEach calls to instead be map calls, which return the promises they create. Once you do that, the map operation will return an array of those promises, which you can use with Promise.all.

unable to exit my process after completion. NodeJS

1 Answers