Hello friends!
At this article, I wanna show you how we can download and crop image with the help of Multer and in the same time to use different adapters, for instance, I'm gonna show two adapters FTP, AWS.
In the beginning, adapt Multer to Nest, main.js can look bellow:
import {NestFactory} from '@nestjs/core';
import {AppModule} from './app.module';
import {INestApplication} from '@nestjs/common';
import fs from 'fs';
import {FastifyAdapter, NestFastifyApplication} from '@nestjs/platform-fastify';
import fmp from 'fastify-multipart';
(async function bootstrap() {
const fastifyAdapter = new FastifyAdapter({
http2: true,
logger: true,
https: {
allowHTTP1: true, // fallback support for HTTP1
key: fs.readFileSync(APP.HTTPS_SERVER_KEY),
cert: fs.readFileSync(APP.HTTPS_SERVER_CRT),
},
});
fastifyAdapter.register(fmp, {
limits: {
fieldNameSize: 100, // Max field name size in bytes
fieldSize: 1000000, // Max field value size in bytes
fields: 10, // Max number of non-file fields
fileSize: 100, // For multipart forms, the max file size
files: 1, // Max number of file fields
headerPairs: 2000, // Max number of header key=>value pairs
},
});
const app: INestApplication = await NestFactory.create<NestFastifyApplication>(
AppModule,
fastifyAdapter,
);
app.enableCors();
await app.listen(APP.PORT, APP.HOST);
})();
Now, Nest can parse multipart/form-data.
After add some abstraction for our storage.
Create UploadImageFactory:
export const UploadImageFactory: FactoryProvider = {
provide: 'IUploadImage',
useFactory: () => {
return StorageFactory.createStorageFromType(TYPE_STORAGE);
},
};
IUploadImage looks like:
export interface IUploadImage {
/**
*
* @param value
*/
setFilename(value: string);
/**
*
* @param value
*/
setCroppedPrefix(value: string): IUploadImage;
/**
*
* @param value
*/
setCroppedPayload(value: CropQueryDto): IUploadImage;
getMulter(): any;
}
StorageFactory looks like:
export class StorageFactory {
static createStorageFromType(type: string): IUploadImage {
switch (type) {
case TYPE_STORAGE.FTP:
return new FtpStorageAdapter({
fileFilter(req, file, cb) {
//TODO validate files on mime-type
cb(null, true);
},
},
);
case TYPE_STORAGE.AWS: {
return new AwsStorageAdapter({
fileFilter(req, file, cb) {
//TODO validate files on mime-type
cb(null, true);
},
});
}
default:
return null;
}
}
}
Create the first adapter for FTP:
FtpStorageAdapter
import FtpStorage from 'multer-ftp';
import fs from 'fs';
import path from 'path';
import {Options, StorageEngine} from 'fastify-multer/lib/interfaces';
import multer from 'fastify-multer';
export class FtpStorageAdapter extends StorageAbstract implements StorageEngine {
private readonly storage;
private readonly storageForCropping;
constructor(options: Options | undefined) {
super();
this.setMulter(multer(
{
...options,
storage: this,
},
).single('file'));
this.storage = new FtpStorage({...FTP_STORAGE});
this.storageForCropping = new FtpStorage({...FTP_STORAGE});
}
async _handleFile(req, file, cb) {
const filePath = await this.saveAsTemp(file);
await this.resize(filePath).then((resizedFile) => {
this.storageForCropping.opts.destination = (inReq, inFile, inOpts, inCb) => {
inCb(null, this.croppedPrefix + this.filename + path.extname(inFile.originalname));
};
this.storageForCropping._handleFile(req, {
...file,
stream: fs.createReadStream(resizedFile as string),
}, (err, destination) => {
if (err) {
Promise.reject(err);
}
Promise.resolve(true);
});
});
const storage: any = await new Promise((resolve, reject) => {
this.storage.opts.destination = (inReq, inFile, inOpts, inCb) => {
inCb(null, this.filename + path.extname(inFile.originalname));
};
this.storage._handleFile(req,
{
...file,
stream: fs.createReadStream(filePath as string),
},
(err, destination) => {
resolve(() => cb(err, destination));
});
});
this.reset();
storage();
}
async _removeFile(req, file, cb) {
this.reset();
}
}
Create the second adapter for AWS:
AwsStorageAdapter
import AwsStorage from 'multer-s3';
import fs from 'fs';
import AwsS3 from 'aws-sdk/clients/s3';
import path from 'path';
import {Options, StorageEngine} from 'fastify-multer/lib/interfaces';
import {StorageAbstract} from '../storage.abstract';
import multer from 'fastify-multer';
export class AwsStorageAdapter extends StorageAbstract implements StorageEngine {
private readonly storage;
private readonly storageForCropping;
private readonly AWS_CONFIG = {
s3: new AwsS3({
credentials: {
accessKeyId: AWS_STORAGE.AWS_ACCESS_KEY_ID,
secretAccessKey: AWS_STORAGE.AWS_SECRET_ACCESS_KEY,
},
s3ForcePathStyle: AWS_STORAGE.AWS_S3_FORCE_PATH_STYLE,
s3BucketEndpoint: AWS_STORAGE.AWS_S3_BUCKET_ENDPOINT,
endpoint: AWS_STORAGE.AWS_ENDPOINT,
}),
acl: AWS_STORAGE.AWS_ACL,
bucket: AWS_STORAGE.AWS_BUCKET,
};
constructor(options: Options | undefined) {
super();
this.setMulter(multer(
{
...options,
storage: this,
},
).single('file'));
this.storage = new AwsStorage({
...this.AWS_CONFIG,
});
this.storageForCropping = AwsStorage({
...this.AWS_CONFIG,
});
}
async _handleFile(req, file, cb) {
const filePath = await this.saveAsTemp(file);
await this.resize(filePath).then((resizedFile) => {
this.storageForCropping.getKey = (inReq, inFile, inCb) => {
inCb(null, this.croppedPrefix + this.filename + path.extname(inFile.originalname));
};
this.storageForCropping.getContentType = (inReq, inFile, inCb) => {
inCb(null, inFile.mimetype);
};
this.storageForCropping.getMetadata = (inReq, inFile, inCb) => {
inCb(null, {fieldName: inFile.fieldname});
};
this.storageForCropping._handleFile(req, {
...file,
stream: fs.createReadStream(resizedFile as string),
}, (err, destination) => {
if (err) {
Promise.reject(err);
}
Promise.resolve(true);
});
});
const storage: any = await new Promise((resolve, reject) => {
this.storage.getKey = (inReq, inFile, inCb) => {
inCb(null, this.filename + path.extname(inFile.originalname));
};
this.storage.getContentType = (inReq, inFile, inCb) => {
inCb(null, inFile.mimetype);
};
this.storage.getMetadata = (inReq, inFile, inCb) => {
inCb(null, {fieldName: inFile.fieldname});
};
this.storage._handleFile(req,
{
...file,
stream: fs.createReadStream(filePath as string),
},
(err, destination) => {
resolve(() => cb(err, destination));
});
});
this.reset();
storage();
}
async _removeFile(req, file, cb) {
this.reset();
}
}
StorageAbstract looks like:
import fs from 'fs';
import sharp from 'sharp';
export abstract class StorageAbstract implements IUploadImage {
protected filename: string;
protected croppedPayload: CropQueryDto;
protected croppedPrefix: string;
private multer: any;
protected constructor() {
}
protected setMulter(multer: any) {
this.multer = multer;
}
setFilename(value): IUploadImage {
this.filename = value;
return this;
}
setCroppedPrefix(value: string): IUploadImage {
this.croppedPrefix = value;
return this;
}
setCroppedPayload(value: CropQueryDto): IUploadImage {
this.croppedPayload = value;
return this;
}
getMulter(): any {
return this.multer;
}
protected async saveAsTemp(file): Promise<string> {
return new Promise((resolve, reject) => {
const tmpFile = '/tmp/' + uuid4();
const writeStream = fs.createWriteStream(tmpFile);
file.stream
.pipe(writeStream)
.on('error', error => reject(error))
.on('finish', () => resolve(tmpFile));
});
}
protected async resize(file: string): Promise<string> {
return new Promise((resolve, reject) => {
const tmpFile = '/tmp/' + uuid4();
let readStream = fs.createReadStream(file as string);
const writeStream = fs.createWriteStream(tmpFile);
if (Object.keys(this.croppedPayload).length !== 0) {
const {cw, ch, cl, ct} = this.croppedPayload;
readStream = readStream.pipe(sharp().extract({
left: cl,
top: ct,
width: cw,
height: ch,
}));
}
readStream
.pipe(writeStream)
.on('error', error => reject(error))
.on('finish', () => resolve(tmpFile));
});
}
protected reset() {
this.setFilename(null);
this.setCroppedPrefix(null);
this.setCroppedPayload({
ch: 0,
cl: 0,
cw: 0,
ct: 0,
});
}
}
Add IUploadImage into a contrived controller:
...
import {BadRequestException, Post,Req,Res} from '@nestjs/common';
...
constructor(
@Inject('IUploadImage')
private readonly uploadImage: IUploadImage,
) {
}
@Post('/')
async upload(
@Query() avatarCropDto: CropQueryDto,
@Req() req,
@Res() res,
....
): Promise<void> {
try {
...
return new Promise((resolve, reject) => {
this.uploadImage
.setFilename(uuid)
.setCroppedPrefix(croppedPrefix)
.setCroppedPayload(cropPayload)
.getMulter()(req, res, (err) => {
if (err) {
reject(err);
}
resolve(req.file);
});
});
} catch (e) {
throw new BadRequestException(e.message);
}
}
After those manipulations, We can not only download but also crop images.
Top comments (11)
I was looking something about uploading to S3 AWS... thanks... I'll try following a little about your setup and hope will works. If you have recommendations would be much appreciated. My challenge is to just host files to AWS with some Multer Validations (I think it already do very good)...
Hello, I can put forward using localstack/localstack:latest for trying it out.
Actually I used part of your code and got AWS I integrated with my stack. I am appreciated.
Thank you so much for your guide. I have tested it out and it works great so far. The response from the controller is replied with the original image detail uploaded to S3 AWS. Is it possible to get the detail of both the cropped image and the original image as a response after finish uploading?
Couldn't get it running. How to export the UploadImageFactory to a module?
Hi, You can do it like below:
Thanks for your response. Figured that out earlier. I was somewhat trying to make it a dynamic module to be exported and used elsewhere in the code. I can see that you have hard-coded AWS config in it's adapter itself which is ofc not the best of practices, and now if you want to take it out of there and pass in from the module itself. How would you recommend doing that?!
To be honest, I made this solution in a hurry. You know, I start thinking about making a library
MulterCropper.You are right about hard-coded config, This solution sucks, As for me the best way is to take it out and pass via a constructor.
It really helped me to say the least. Thank you. That's a nice idea for a library though.
great implementation indeed, couldn't think any better on how to implement this, as I created one with the traditional, controller service approach with providers. one question though, have you tried handling multiple images upload on single request for this one? if yes, how many files do you recommend to be uploaded once or how big is the size for example on the s3 storage?
Hello, by default this solution doesn't support multiple files, about size and how big it depends on your aims