If you ever meet a developer who says size doesn't matter then you'd expect them to have one sizable cloud budget to work with! For everyone else ...
For further actions, you may consider blocking this person and/or reporting abuse
If you don't need to store full quality images in S3 another option is to create a custom
Sharpscript to run during the image upload phase using their SDK. This way you could avoid Lambda function costs as well 😉Uploading images during the processing step might slow down your application as a compression is CPU intensive and would likely block your threads leading to less throughput if you had a lot of requests coming in at the same time.
do you have any clue how can we do that
This is a very nice and very detailed guide. I had to create something similar for my company's website a while ago and I wish I had something clear like this as a starting point.
My company also wanted to be able to responsively display different image sizes depending on the device's resolution (via srcset). We also wanted the ability to crop and rotate images while still maintaining the original image.
Originally I was doing something similar to this, except I was creating 4 differently sized images all at once, but that didn't give us enough flexibility as well as made changing the rotation/cropping later on trickier as we didn't want to upload another image to get the lambda trigger to activate again.
So I was inspired by cloudinary and imgix in particular to create something that would apply transformations to images on-the-fly depending on the URL.
It basically boiled down to using Cloudfront CDN as an endpoint that would call a Lambda@Edge Origin Response trigger which would in turn grab the image from S3, modify it with Sharp, and then store the new cropped image back into S3 [optional].
For example:
imgs.mysite.com/test.jpgis the original raw image.imgs.mysite.com/c/0,0,100,100/q/80/r/90/test.jpgwould crop it to 100x100, set the quality to 80%, and rotate it 90 degrees. All we do is store he original image name in our database as well as crop and rotation boundaries we get from an image uploading component and then it's easy to reconstruct the appropriate urls client-side.Thanks Khauri!
That's a really neat process, and absolutely the next logical step for working with a much wider range of images than just simple avatar pics. I'm assuming this is something similar to what Next.js is currently doing in the background with their image optimization process, but it sounds like your setup may offer a lot more flexibility with cropping and rotation options as well. That's pretty awesome!
It's a really nice article. Got me up and running. But after I made changes to the script how do I re-deploy? It shows error that it already exist. So how do I deploy only the updated code?
Hi Sayantan,
Hard to say what the actual problem might be without seeing all the error logs, but typically when redeploying cloud formation assets there could be a couple of issues that trip it up. The most common would be if one of the resources created by the cloud formation stack has been manually modified in the aws console. For example, if you create the s3 buckets using AWS Sam, then edit the bucket in S3, this can potentially break the logical ids used between Sam/Cloudformation and the actual resource. So when you try to update, cloud formation is unsure of how to resolve the changes. If this is the case, and your not yet running production workload, it’s best to delete the modified resource and re-deploy again.
Hope this helps.
Thanks for your response. I was also doing the same. But even in this way everytime I need to delete the Cloudformation, Buckets etc. I need is to re-deploy the lamdba. Bucket and other things are fine already. Could you please point me towards right direction? I'm actually new to lambda. Created my first ever lambda with the help of your article. Thanks for that.
Yeah it definitely shouldn’t require everything to be manually deleted each time. I’d take a look at the aws SAM docs and walk through some of the examples there to see what might be going wrong.
docs.aws.amazon.com/serverless-app...
docs.aws.amazon.com/serverless-app...
Thanks a lot. 👍👍
Thanks for the awesome article! Did your final images drop noticeably in quality using this method? I have a similar pipeline and my images look noticeably worse when the original image is from a high-end mobile device with a powerful camera. I've tried using max quality for sharp but no luck. Any ideas? Thanks!
Thanks Uche,
No significant quality loss in my end unfortunately? The size of the image you are setting may come into play as well as quality settings? E.g compressing an image down to 200px width and height but displaying at 800px will always look like poor quality?
Nice article! I’m assuming the output bucket stores the image under the same key as was provided to the input bucket — yeah? So for example if I uploaded a profile picture, I can save the ouput_bucket/key combo to my profile picture field in the database. Is that how you’re approaching this here from the frontend’s perspective? Obviously the frontend is clueless as to when the transformation is finished, which is why I’m asking.
Thanks! Yep, spot on. The image uses the same key in both buckets. As for the front end, there’s a couple of options you could do. Update your db record with the compressed bucket / key path immediately, but also keep a cached version of the uploaded file in the browser session after the user uploads. This way you can use the original image straight away, and wait for the compression to complete a second or two later. Alternatively, you could have a fallback strategy when loading the image - e.g try loading the compressed bucket key path and if that fails load in the source bucket key path.
Depending on the size of the images you are working with, the actual compression/conversion time is pretty damn quick, and you could bump up your lambda memory allocation to try and process things a little faster. So it’s possible that you could just use the compressed bucket key path directly without skipping a beat.