If you have trouble organizing your photos spread across several folders like me, this article can help you!
I recently experienced a hard drive crash 💥 containing all of my photos 😢. Luckily, I found backups and recovered some photos from friends and family. But I really had a hard time organizing all these photos. After 4 days of repetitive renaming and moving tasks, I decided to create a script to automate this horribly time-consuming task and maybe save someone else.
As you can see it was a bit of a mess 😅
Defining an effective organization
The paragraph below is an opinionated thought. You are free to implement any other organizational system with the code given.
My first try was to move all the photos and videos into a single folder and let the "Last Updated Date" sort it. Simply scroll to the desired date and view photos from that period. But several thousand photos in a single folder took too long to load and scroll through. Moreover, I had too many name conflicts with photos with the same name from different smartphones.
Second try: Divided by year and month. Few photos in folders, few conflicts. After organizing a few years of photos, I realized that it was painful to read. I was constantly changing folders to view my photos. A really not pleasant experience.
Third try: Just divided by year. Not too many photos in folders, a significant number of name conflicts. But easy to read without changing folders to watch them. This looks better but there are still quite a few name conflicts to resolve…
👉 I chose the option: divide by year to easily navigate over time and limit the loading time of each folder
Create a Node script to import once
The code below recursively scan folders and copy photos to an output folder divided by year. Enough for a single run, but will override photos on incremental import.
const { resolve } = require('path');
const { readdir } = require('fs').promises;
const path = require('path');
var fs = require('fs')
const inPathdir = "/Users/alagrede/Desktop/inputPhotos";
const outPathdir = "/Users/alagrede/Desktop/photos/";
// recursively read folders
async function* getFiles(dir) {
const dirents = await readdir(dir, { withFileTypes: true });
for (const dirent of dirents) {
const res = resolve(dir, dirent.name);
if (dirent.isDirectory()) {
yield* getFiles(res);
} else {
yield res;
}
}
}
(async () => {
let totalCount = 0;
for await (const f of getFiles(inPathdir)) {
const name = path.parse(f).name;
const dirname = path.dirname(f);
const extname = path.extname(f);
const stats = fs.statSync(f);
const time = stats.birthtime;
const year = time.getFullYear();
const yearDir = outPathdir + year + "/";
if (!fs.existsSync(yearDir)) {
fs.mkdirSync(yearDir);
}
const dateformatted = time.toISOString().split('T')[0]; // 2012-10-25
const outFilename = `${yearDir}IMG-${dateformatted}_${count}${extname}`;
fs.copyFileSync(f, outFilename);
fs.utimesSync(outFilename, time, time);
totalCount++;
}
console.log(`Successfully imported ${totalCount} files`);
})()
Improve to import incrementally
Now we do the same thing but this time searching for the last photo number in each year folder to no longer overwrite existing photos.
const { resolve } = require('path');
const { readdir } = require('fs').promises;
const path = require('path');
var fs = require('fs')
const inPathdir = "/Users/alagrede/Desktop/inputPhotos";
const outPathdir = "/Users/alagrede/Desktop/photos/";
// recursively read folders
async function* getFiles(dir) {
const dirents = await readdir(dir, { withFileTypes: true });
for (const dirent of dirents) {
const res = resolve(dir, dirent.name);
if (dirent.isDirectory()) {
yield* getFiles(res);
} else {
yield res;
}
}
}
(async () => {
const sortByNameFunc = function(a,b) {
const nameA = path.parse(a).name;
const nameB = path.parse(b).name;
if (nameA < nameB) return 1;
if (nameA > nameB) return -1;
return 0;
}
const sortByBirthtimeFunc = function(a,b) {
return fs.statSync(a).birthtime - fs.statSync(b).birthtime;
}
async function getFilesSortedBy(dir, sortFunction) {
let files = []
for await (const f of getFiles(dir)) {
// avoid considering hidden cache files
if (!path.parse(f).name.startsWith("."))
files.push(f);
}
return files.sort(sortFunction);
}
async function getMaxCounterInDir(dir) {
const files = await getFilesSortedBy(dir, sortByNameFunc);
if (files.length === 0) return 1;
return path.parse(files[0]).name.split(".")[0].split("_").slice(-1);
}
const inFiles = await getFilesSortedBy(inPathdir, sortByBirthtimeFunc);
const currentCounterByYear = new Map();
let totalCount = 0;
// Start to copy files to dest
for await (const f of inFiles) {
const name = path.parse(f).name;
const dirname = path.dirname(f);
const extname = path.extname(f);
const stats = fs.statSync(f);
const time = stats.birthtime;
const year = time.getFullYear();
const yearDir = outPathdir + year + "/";
// find the current max counter for year directory
if (!currentCounterByYear.has(year)) {
if (!fs.existsSync(yearDir)) {
fs.mkdirSync(yearDir);
currentCounterByYear.set(year, 1);
} else {
currentCounterByYear.set(year, await getMaxCounterInDir(yearDir));
}
}
// copy file to dest
const count = currentCounterByYear.get(year);
const dateformatted = time.toISOString().split('T')[0]; // 2012-10-25
const outFilename = `${yearDir}IMG-${dateformatted}_${count}${extname}`;
currentCounterByYear.set(year, ++count);
fs.copyFileSync(f, outFilename);
fs.utimesSync(outFilename, time, time);
totalCount++;
}
console.log(`Successfully imported ${totalCount} files`);
})()
Perfect for importing photos over time! And I’m super happy with this.
Just one more thing
I also want to show you how to avoid photo duplication, allowing you to import any folder with existing photos without worrying about it.
Remove duplicates
To find duplicate photos in bulk, I cross-reference the file size with the updated time. If these 2 values are exactly the same, I assume they are the same files.
const { resolve } = require('path');
const { readdir } = require('fs').promises;
const path = require('path');
var fs = require('fs')
const pathdir = "/Users/alagrede/Desktop/out/";
async function* getFiles(dir) {
const dirents = await readdir(dir, { withFileTypes: true });
for (const dirent of dirents) {
const res = resolve(dir, dirent.name);
if (dirent.isDirectory()) {
yield* getFiles(res);
} else {
yield res;
}
}
}
(async () => {
let m = new Map();
// count files with same updated time and size
for await (const f of getFiles(pathdir)) {
const name = path.parse(f).name;
const dirname = path.dirname(f);
const extname = path.extname(f);
if (!name.startsWith("."))
const stats = fs.statSync(f);
const length = stats.size;
const mtime = stats.mtime;
const timestamp = mtime.getTime();
const key = `${timestamp}-${length}`
if (!m.has(key)) {
m.set(key, []);
}
m.get(key).push(f);
}
}
// remove files appearing once
for (let k of m.keys()) {
if (m.get(k).length === 1)
m.delete(k);
}
let totalCount = 0;
for (let k of m.keys()) {
let files = m.get(k);
files.shift(); // remove the first file
// Delete the others
for (let f of files) {
fs.unlinkSync(f);
totalCount++;
}
}
console.log(`Successfully deleted ${totalCount} duplicate files!`)
})()
Work done for today 👍
This system is enough for me to organize all photos and videos and enjoy viewing them without wasting time. I hope these scripts will help you.
You can find them here directly executable and editable through Znote. (A very simple way to work on this kind of script)
If you struggle with this tutorial, I also package a little app here: photo-organizer.app with advanced features (a bit long to explain in a single article but you will find below the libraries used). Among these features, you will be able to find similar photos (with pixel comparison) and resize images.
Librairies used:
Top comments (1)
Very Insightful 🤩 thanks for sharing