DEV Community

Cover image for How I Backup/Sync my Dotfiles & Apps
Boris Jamot ✊ /
Boris Jamot ✊ /

Posted on • Updated on • Originally published at mamyn0va.github.io

How I Backup/Sync my Dotfiles & Apps

My concern is to automatize the backup of my configuration files (aka dotfiles) and to generate a snapshot of all installed apps, whether it be through my distro's package manager or through third-party package manager like npm or pip. This is the first step before being able to sync my apps automatically on a newly installed computer (still to be done).

Backuping dotfiles

What I did is create a git repo in my $HOME dir as all my dotfiles are located here. But as I don't want to push all the other files (Pictures, Downloads, Desktop...), I put a .gitignore file with just a wildcard (*) in it. This way, when I want to add a file to the repo, I have to force with: git add -f newfile.

cd ~
git init
echo "*" > .gitignore
git add -f .gitignore
git add -f .bashrc
git add -f .zshrc
git add -f .config/fish
git add -f .tmux.conf
git add -f .SpaceVim.d/init.toml
git commit -m "First commit"
git remote add origin git@github.com:username/mydotfiles
git push -u origin master
Enter fullscreen mode Exit fullscreen mode

⚠️ But be aware that it could be dangerous for your sensible data. For example, don't push your ~/.ssh config!

Thus, you have a quick way to retrieve your configuration in case of a computer fresh install or in case of a crash.

The next step is to do the same with your apps.

Generate a snapshot of your apps

Regarding the apps, it's not so easy. It's not unusual to have several package managers to install new apps. Personally, I use 7 of them: npm, cargo, ghc-pkg, composer, gem, pip, and pacman, the manager of my Manjaro linux distribution. Each of them has its own syntax to install, delete or update packages.
But each of them also has its own syntax to output a list of installed packages.

So basically, I just use the following bash script to generate the list of all installed packages, across all the package managers:

#!/bin/bash

echo "Generating the lists of explicitly installed packages in ~/.backup"

pacman -Qe > ~/.backup/pacman_packages || echo "pacman failed"
gem list > ~/.backup/gem_packages || echo "gem failed"
npm list -g --depth=0 > ~/.backup/npm_packages || echo "npm failed"
pip list > ~/.backup/pip_packages || echo "pip failed"
cargo --list | tail -n +2 | tr -d " " > ~/.backup/cargo_packages || echo "cargo failed"
ghc-pkg list > ~/.backup/ghc-pkg_packages || echo "ghc-pkg failed"
composer global show | cut -d ' ' -f1 > ~/.backup/composer_packages || echo "composer failed"

git add -f .backup

exit 0
Enter fullscreen mode Exit fullscreen mode

Note the git add command at the end of the script. I do this to automatically add the ~/.backup folder to the git index so that it gets pushed to the remote. And as I put this script in the pre-commit hook, my packages list gets synced with git each time I commit:

biros on  master [!]
➜ git commit -m "Update conf"
Generating the lists of explicitly installed packages in ~/.backup
[master 27f73e4] Update conf
 1 file changed, 1 insertion(+)

Enter fullscreen mode Exit fullscreen mode

⏭️ Next step: be able to re-install all the apps automatically from the backup.


💡 Hint: you can sync your Atom plugins with the sync-settings plugin.


📚 Other resource on the same topic:

Top comments (17)

Collapse
 
gwllmnn profile image
Grégoire Willmann

What I do to back up my dot files is I create a symbolic link in dropbox.

For example:

mv /home/greg/.bash_profile /home/greg/dropbox/macOS/home/greg/bash_profile 
ln -s /home/greg/dropbox/macOS/home/greg/bash_profile /home/users/greg/.bash_profile

With this setup I don't have to do a git commit & push each time I change one of my dot files :)

Collapse
 
biros profile image
Boris Jamot ✊ /

Smart idea and so simple that I didn't even think of it!

Collapse
 
igormp profile image
Igor Moura

That's a nice approach, and I thought about it when I tried to sync my dotfiles with git, but it sounded weird to me to have my home folder as an entire repo, so I decided do move all my config files into a dotfiles folder, init the repo there, and create symlinks to those into my home folder with stow (if you've never heard about it, you should give it a go).

Collapse
 
vlasales profile image
Vlastimil Pospichal
$ cat .gitignore 
*
!.gitconfig
!.gitignore
!.bashrc
!.zshrc
!.config/fish
!.tmux.conf
!.SpaceVim.d/init.toml

$ git init
$ git add .
$ git commit
Collapse
 
alichtman profile image
Aaron Lichtman • Edited

I wrote a backup tool to automate both back up and reinstallation of dotfiles, packages and config files: github.com/alichtman/shallow-backup

Collapse
 
biros profile image
Boris Jamot ✊ /

Hi, it sounds great. But it seems to be limited to a list of pre-configured apps, right ?

Collapse
 
alichtman profile image
Aaron Lichtman • Edited

It backs up lists of packages installed from most popular package managers, backs up whatever dotfiles you want (since the config file is user editable), and some configs (this is the only place it's limited right now.) I'm working on adding arbitrary path backing up, which would make it possible for you to back up whatever you want. I just finished up git integration, and the next big step for this project is expanding the scope of what can be backed up.

One cool thing is that it's open source, so if it's not backing up something that you think it should be, you can just write the code and open a PR!

Collapse
 
jpiszczala profile image
Jarosław Piszczała • Edited

Idea with backup list of instaled apps is awesome. I doesnt think about it firstly and then after format I have missing many apps :c

You should check my instalators on my github (github.com/westscz/.dotfiles/tree/...). I created instalators for apt, flatpak, pip, snap, git (still under development), maybe you can learn something from them :)

Collapse
 
biros profile image
Boris Jamot ✊ /

I will have a look on it ! Thx

Collapse
 
loirotte profile image
Philippe Dosch • Edited

Creating a git repository in a home is not a good practice, for a lot of reasons (especially as it basically works recursively). Your first commit is a good illustration: you add all your files, then remove those you do not want to consider. And you will have to keep in mind and perpetuate this scheme potentially each time a new critical file will be added.

Why do not consider an appropriate and standard tool like vcsh (github.com/RichiH/vcsh)? You find it in standard Linux distributions, it even allows multiple repos (if needed) in home and do not change the common workflow: you only add the files you need in repos.

Collapse
 
biros profile image
Boris Jamot ✊ /

In fact, it's the opposite of what you describe: it's a whitelist. First you ignore ALL, then you add the files you want to save.
Maybe it's not the best way but it's quite simple and it does the job ☺️
Thanks for your suggestion, I'll have a look on it !

Collapse
 
loirotte profile image
Philippe Dosch • Edited

Yep, you're right, I misspoke :-) I described the technical steps instead of the conceptual ones (as you initially add your files to a blacklist). Nevertheless, the facts are still there with your workflow: if you insert in your home a new file, independent of your project, you will have to add it to the .gitignore. However, as it is independent it shouldn't involve side effect on your project.

vcsh has been developed for this kind of purposes and helps you to keep a clean workflow. And if you have several repos in your home (for my own, I have an emacs repo, a zsh repo, etc.), you can couple it with mr (multiple repositories) in order to manage them more efficiently.

Thread Thread
 
biros profile image
Boris Jamot ✊ /

No, if I insert a new file in my home, it will be automatically ignored by my git config.

Thread Thread
 
loirotte profile image
Philippe Dosch

Oups, it's echo "*" and not echo * :-)

Collapse
 
pavonz profile image
Andrea Pavoni

I do something similar, but I use symlinks and an installer (sort of, it’s a ruby rake task).

You can find it here: github.com/andreapavoni/dotfiles

Collapse
 
sandordargo profile image
Sandor Dargo

Do you also automatically update your "backup" on GitHub with a cron for example?

Collapse
 
biros profile image
Boris Jamot ✊ /

Not yet 😊
Otherwise I could use a hook in my package managers to automatically push to git when installing new stuff.