Download file from FTP server to client using Node js

Question

System architecture

System consist of 3 components

1: FTP server: Used to store files. Can only be accessed by Node.js app. No direct access.

2: Node.js: Provides an API to interact with FTP server. Only way to interact with FTP server. Node.js app have no collection of files stored on FTP. Only provides method to work with FTP. Node.js app should not store any file uploaded from it and download from it.

3: Client: A user how will upload or download a file to FTP server by using Node.js app.

What I have done:

I am able to download the file stored on FTP by using basic-ftp package. Here is the code for download file function.

async function downloadFile(folderPath, fileName, writeStream) {
    console.log(folderPath);
    console.log(fileName);
    const client = new ftp.Client()
    // client.ftp.verbose = true
    try {
        await client.access({
            'host': process.env.FTP_HOST,
            'user': process.env.FTP_USER,
            'password': process.env.FTP_PASSWORD,
        });
        await client.ensureDir(folderPath);
        await client.downloadTo(writeStream, fileName);
    }
    catch(err) {
        console.log(err)
    }
    client.close()
}

The file is download to the directory named /downloads on Node.js server. What I want to do actually is download the file directly to client computer. To download the file directly to client, I have tried streaming the writeStream object from download method. Here is the code for that

app.post("/download/file", urlencodedParser, (req, res, next) => {
    var writeStream = fs.createWriteStream('./downloads/'+req.body.fileName);
    writeStream.on("data", (data) => {
      res.write(data);
    })
    writeStream.on("close", () => {
      res.end()
    })

    res.setHeader('Transfer-Encoding', 'chunked');
    downloadFile(req.body.folderName, req.body.fileName, writeStream);
})

This does not work. It always ends in error with downloading the file completely.

Another approach I tried is to generate a URL for the file and client will click the URL to download the file. Problem with this approach is the file is not complete by the time I start downloading which results in incomplete file download. For example, If the size of file is 10MB and only 2 MB was download by the time client clicked the link, it will download only 2MB file not 10MB.

Goal:

Download the file to client(browser) from a FTP server through Node js.

Requirement:

Download the file stored on FTP server directly to client through Node js.

Constraints:

Client does not have access to FTP server.

The only way to access the server is through Node js app.

Hello, so basically you want to 'bridge' FROM server (USING NodeJS) TO client, So that the Client never sees the MAIN server, just the NodeJS server, is that it? — JosEvora
I haven't tested this, but i think this can be done with the 'net' module from nodejs, you could just 'pipe' the connection from the server, to the client, and the server would only see the nodejs server, and the client would only see the nodejs server as well! try it... — JosEvora
you just have to include manually the FTP credentials on connection, before the pipe... — JosEvora

Anatoly Anatoly · Accepted Answer · 2020-11-08T10:14:11

You can try to indicate res as an output stream directly. That way you will simply redirect a stream from an ftp to a client:

async function downloadFile(fileName, writeStream) {
    console.log(fileName);
    const client = new ftp.Client()
    // client.ftp.verbose = true
    try {
        await client.access({
            'host': process.env.FTP_HOST,
            'user': process.env.FTP_USER,
            'password': process.env.FTP_PASSWORD,
        });
        await client.downloadTo(writeStream, fileName);
    }
    catch(err) {
        console.log(err)
    }
    client.close()
}

app.post("/download/file", urlencodedParser, (req, res, next) => {
    downloadFile(req.body.fileName, res);
})

Download file from FTP server to client using Node js

1 Answers