I am trying to use multer-gridfs-storage middleware in nestjs explicitly. I wanted to store a file and retrieve it using gridfs-stream package. But nothing works for me. I cannot get the request populated with the file itself. Whenever I try to print the request object it doesn't have anything related to req.file or buffer element in it. And I am unsure how to process the file once I get it using the multer.
Here is the sample controller method
@Post('/uploadFiles')
public async postFileToMongo(@Req() req, @Res() res) {
//logger.info('this is the mutler upload details',req.body);
//logger.info('coming inside upload files', req);
logger.info('this is the sample value set');
logger.info('this is the request body', req);
logger.info('this is the request files', req.file);
await this.fileStorageService.createFilesIntoGridFsUsingGridFSStream();
res.send('Done');
}
Here is the FileStorageService. I have not implemented multerStorage yet, since I do not get anything from the controller (req.file is empty)
@Injectable()
export class FileStorageService {
mongooseConnection: mongoose.Connection;
gridfsConnection: GridFsStream.Grid;
gridfsStorage: GridFSStorage;
uploadStorage;
constructor(
@Inject(MONGO_DB_PROVIDER_TOKEN_TEST) private readonly mongoCon: Connection,
private readonly configService : ConfigProviderService,
) {
this.mongooseConnection = this.mongoCon as mongoose.Connection;
this.gridfsConnection = GridFsStream(
this.mongooseConnection.db,
mongoose.mongo,
);
this.gridfsConnection.collection('default_collection');
// now intialize the storage engine
this.gridfsStorage = new GridFSStorage({
db: this.mongooseConnection,
cache: true,
file: (req, file) => {
// instead of just returning fileName or bucket Name as below
// we could return an object
// let bucket;
// if (file.mimetype === 'multipart/*' || file.mimetype === 'img/jpeg') {
// bucket = 'first_collection';
// } else {
// bucket = 'second_collection';
// }
return {
filename: file.originalname + Date.now(),
bucketName: 'default_collection',
}
},
});
this.uploadStorage = multer({ storage: this.gridfsStorage });
logger.info('gridfsConnection obj', isNullOrUndefined(this.gridfsConnection));
//logger.info('upload storage vale', this.uploadStorage);
}
async createFilesIntoGridFsUsingGridFSStream() {
//console.log(this.uploadStorage);
const writeStream = this.gridfsConnection.createWriteStream({
filename: 'doodle_test.png',
mode: 'w',
root: 'first_collect',
chunkSize: 1024,
metadata: {
file_desc: 'This is a sample metadata added',
},
});
const filestring = path.resolve('./doodle.png');
await fs.createReadStream(filestring).pipe(writeStream).on('error', err => {
logger.info('err ob', err);
});
writeStream.on('close', (file) => {
logger.info('file name is', file.filename);
});
}
async createFilesUsingMulter(req, ){
}
}
I even tried creating a seperate middleware but it did not work either
@Injectable()
export class FileStorageMiddlewareMiddleware implements NestMiddleware {
mongooseConnection: Connection;
constructor(@Inject(MONGO_DB_PROVIDER_TOKEN_TEST) private readonly mongoConnection: Connection,
private readonly configService: ConfigProviderService){
this.mongooseConnection = this.mongoConnection as mongoose.Connection;
}
resolve(...args: any[]): MiddlewareFunction {
return (req, res, next) => {
logger.info('inside file storage middleware');
// const fileStorage = new multerGridfsStorage({
// // url: this.configService.getDBCred().$urlString,
// db: this.mongoConnection as Connection,
// cache: true,
// file :(req, file) => {
// //determine the default filename
// const filename = file.filename+'feedback_file_'+Date.now();
// let bucketname = 'default_file_collection';
// if(file.mimetype === 'plain/text') {
// bucketname = 'files.text';
// } else if(file.mimetype === 'plain/html'){
// bucketname = 'files.html';
// } else if(file.mimetype === 'img/png' || file.mimetype === 'img/jpeg' ||
// file.mimetype === 'img.jpg'){
// bucketname = 'files.image';
// }
// return {fileName: filename, bucketName: bucketname};
// // determing the bucketname
// },
// });
// const upload = multer({storage: fileStorage});
// //upload.single('./../../../src/fileUpload');
// //busboyBodyParser({ limit: '1kb' }, { multi: true });
// // req.busboyBodyParser([{limit: '20mb'}, {multi: true}]);
// upload.single('file');
next();
};
}
}
Note: I included the middleware and consuming it in the module. I am able to see the log in the middleware but not the multiform part data
Here are the details from 'postman' tool (used 3002 as port and 'development' as context path
This is the process that I followed using gridfs-stream module, got a sample file from a local folder and stored it in gridfs and it worked.
async createFilesIntoGridFsUsingGridFSStream() {
this.gridfsConnection = GridFsStream(
this.mongooseConnection.db,
mongoose.mongo,
);
//console.log(this.uploadStorage);
const writeStream = this.gridfsConnection.createWriteStream({
filename: 'doodle_test.png',
mode: 'w',
root: 'first_collect',
chunkSize: 1024,
metadata: {
file_desc: 'This is a sample metadata added',
},
});
const filestring = path.resolve('./doodle.png');
await fs.createReadStream(filestring).pipe(writeStream).on('error', err => {
logger.info('err ob', err);
});
writeStream.on('close', (file) => {
logger.info('file name is', file.filename);
});
}
But there is a warning I think gridfs by default uses the GridStore and try to store it in a collection
DeprecationWarning: GridStore is deprecated, and will be removed in a future version. Please use GridFSBucket instead (node:29332) DeprecationWarning: collection.ensureIndex is deprecated. Use createIndexes instead. (node:29332) DeprecationWarning: collection.save is deprecated. Use insertOne, insertMany, updateOne, or updateMany instead.
Hence wanted to make use of multer-gridfs-storage which makes use of GridFSStorage by default
Just in case, here are the version details
"@types/mongoose": "^5.3.20",
"@types/multer": "^1.3.7",
"@types/multer-gridfs-storage": "^3.1.1", "gridfs": "^1.0.0",
"gridfs-stream": "^1.1.1", "mongoose": "^5.4.11",
"multer": "^1.4.1",
"multer-gridfs-storage": "^3.2.3",
EDIT: This is for learning purposes and wanted to have an indepth understanding of how the express middleware are handled in nestjs. I understand that nestjs has something like FileInterceptor for handling files, but wanted to get more info with regards to this
Also I wanted to ask how multer-gridfs-storage uses gridfs-stream to store files, Is there any provisions to provide what sort of stream should be used in multer-gridfs-storage?
