Bradley Schofield for Appwrite

Posted on Jul 10, 2021 • Updated on Aug 9 • Originally published at appwrite.io

Appwrite In Production: Backups and Restores

#webdev #opensource #devops #tutorial

Backing up and restoring data is an extremely important part of running servers. It's a virtual safety net against most bad things that can happen. Made a bad config change? Restore a backup. Messed up an update? Restore a backup. Corrupted Drives? Restore a backup.

Not only that, backups can also come in handy when migrating data to other systems, for instance migrating a development server into a production environment or vice versa 🔁

To make this entire process as easy as possible we've written this simple and handy guide to explain everything you need to know about backing up and restoring your Appwrite instance.

Appwrite is broken down into multiple sections and most of Appwrite is stateless. This means that Appwrite only actually has two main things you need to backup: one is Appwrite's Database (MariaDB) and the other is the Docker volumes which store functions data and uploads. The rest - Appwrite can automatically handle and regenerate.

Please note that all these commands need to be run within the same directory as Appwrite's docker-compose.yml

With all that said, lets begin! 😊

💾 Backing up the MariaDB Database

Due to the fact that Appwrite uses a Docker image of MariaDB it is extremely easy to dump the entire database with just one command and likewise to restore the dump.

Creating a Database backup is just one command:

docker-compose exec mariadb sh -c 'exec mysqldump --all-databases --add-drop-database -u"$MYSQL_USER" -p"$MYSQL_PASSWORD"' > ./dump.sql

Going into depth this command does a couple things:

Docker-compose launches a temporary shell onto the MariaDB container to start work
It runs mysqldump on the server with two specific options --all-databases and --add-drop-database these are important since they ensure that when the backup is restored old data doesn't get overlapped with new data.
The output of mysqldump is piped into a dump.sql file. This is your backup ✅️

✍️ Restoring the MariaDB Database

Restoring the database is similarly easy and also requires just one command to do so:

docker-compose exec -T mariadb sh -c 'exec mysql -u"$MYSQL_USER" -p"$MYSQL_PASSWORD"' < dump.sql

This command is very simple once you break it down:

Docker-compose launches a temporary shell onto the MariaDB container to start work
Using the mysql command we restore the dump through a pipe

💿 Backing up your Docker Volumes

Appwrite stores various things in Docker volumes. This includes your file uploads and Cloud Function data and docker volumes makes it easy for us to coordinate data between the central Appwrite container and our various Appwrite workers. Uploads are especially important to backup since they include all your app's file uploads, these commands can take a while to run depending on how much data you have to backup.

Before running these commands is it highly recommended to shutdown your Appwrite instance to ensure you get a complete backup.

To backup the functions volume the command is:

mkdir -p backup && docker run --rm --volumes-from "$(docker-compose ps -q appwrite)" -v $PWD/backup:/backup ubuntu bash -c "cd /storage/functions && tar cvf /backup/functions.tar ."

and to backup the uploads volume the command is:

mkdir -p backup && docker run --rm --volumes-from "$(docker-compose ps -q appwrite)" -v $PWD/backup:/backup ubuntu bash -c "cd /storage/uploads && tar cvf /backup/uploads.tar ."

Both these commands do similar things and when you break them down they are pretty simple.

Start a new Docker container. This Docker container has a few special options

--rm will delete the container once it's done running. The reason we want this is because this container is only being used to package up our backup and give it to the host machine.

--volume-from This flag special as it will mount all of the volumes of the container we give it. To get the container ID we want we use a $(docker-compose ps -q appwrite) to get the ID within the command

-v This flag is being used to mount a volume onto our new container which will give us access to a backup folder we created using the mkdir command at the start

ubuntu is the image we are basing our new container on

Finally with this command created we change directories into the normal Appwrite mount point for uploads and create a tarball which will be created in the backup directory where we will be able to access it.

Once these commands are run you should find a new backup folder which containsuploads.tar and functions.tar these are your backups. Keep them safe.

📝 Restoring your Docker Volumes

Restoring your Appwrite volumes is fairly simple as well. Move the backup folder you just created to your destination machine next to the docker-compose.yml file and simply run the following commands to restore the backup.

Please note that the Appwrite instance should be shutdown while running these commands.

Restoring functions volume:

docker run --rm --volumes-from "$(docker-compose ps -q appwrite)" -v $PWD/backup:/restore ubuntu bash -c "cd /storage/functions && tar xvf /restore/functions.tar --strip 1"

Restoring uploads volume:

docker run --rm --volumes-from "$(docker-compose ps -q appwrite)" -v $PWD/backup:/restore ubuntu bash -c "cd /storage/uploads && tar xvf /restore/uploads.tar --strip 1"

This command creates new temporary Docker container like the backup command but instead extracts the tar back into the functions and uploads endpoint which will restore the backup.

⭐️ In Conclusion

To create a full Appwrite backup you will want to backup MariaDB and the two volumes specified here. Once you have done that make sure to keep it safe, the best backup will store the backup in multiple locations including locally and in multiple clouds. As with any Cloud Native Application, make sure that you backup your Appwrite instance regularly to ensure that you're never in a situation where you have to lose data due to a server failure.

This entire process can be easily done to help migrate an Appwrite installation easily by simply copying the backup files onto another server and running the restore steps.

We hope you enjoyed this article! We love contributions and encourage you to take a look at our open isuses and ongoing RFCs.

If you get stuck anywhere, feel free to reach out to us on our friendly support channels run by humans 👩‍💻

Here are some handy links for more information:

Thank you to Taylor Vick for the server image.

Top comments (19)

Fabian Reyes • Jul 21 '21

Hi, I have tried to replicate this guide and everything works fine except the database restore, I get this error: ERROR 1064 (42000) at line 1: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'mysqldump: Got error: 1045: "Access denied for user 'user'@'localhost' (using...' at line 1

Eldad A. Fux • Sep 4 '21

I think you need to make sure your target database has the same user credentials.

Fabian Reyes • Sep 20 '21 • Edited

For now, I managed to restore the files and the database without any problems. Now, there is a problem. The files do not work, apparently they were restored but it is not possible to access or download them.

dev-to-uploads.s3.amazonaws.com/up...
dev-to-uploads.s3.amazonaws.com/up...

Eldad A. Fux • Sep 21 '21

Make sure you use the same encryption key in your env file

Fabian Reyes • Sep 21 '21

You mean, this one? _APP_OPENSSL_KEY_V1

Eldad A. Fux • Sep 21 '21

Yep :)

Steven • Dec 2 '21

I think there is a typo/bug in the user part of the database backup command. I think it should be:

docker-compose exec mariadb sh -c 'exec mysqldump --all-databases --add-drop-database -u"root" -p"$MYSQL_ROOT_PASSWORD"' > ./dump.sql

Vincent Ge • May 4 '22

Thanks for pointing this out ;)

Took a while but we got to it! It should be fixed now <3

Steven • May 8 '22

Thanks! Looks like the same typo is in the mariadb restore command too.

Steven • Dec 6 '21

I think people have also had problems migrating if they don't migrate their appwrite-certificates and maybe appwrite-config volume. This is because the database has data about certificates generated, but the certificates don't actually exist on disk so HTTPS doesn't work.

If you didn't get a chance to backup those, you can try to manually delete the certificates info in the database manually. $ and > characters to indicate whether the command is executed in the shell or mysql CLI, respectively:

# shell into the mariadb container
$ docker-compose exec mariadb sh

# authenticate with mysql
# do not add whitespace between -p and password
$ mysql -u root -p"$MYSQL_ROOT_PASSWORD"

# use the appwrite database
> use appwrite;

# Find the documentID for your domain in the database:
> SELECT * FROM `app_console.database.properties` p where p.key = "domain";

# note the documentUid - that's what we'll use to query/delete them from the database
# We'll need to remove the document from two tables, here's the other one
> SELECT * FROM `app_console.database.documents` d where d.uid = "thedocumentUID";

# Now we can delete these two rows:
> DELETE FROM `app_console.database.properties` where documentUid = "thedocumentUID";
> DELETE FROM `app_console.database.documents` where uid = "thedocumentUID";

Credits to @kodumbeats for the SQL queries.

Brian Mutiso • Jun 10 '22

I was not able to back up maria db all the scripts in this article only produces a 0byte backup file

docker-compose exec mariadb sh -c exec mysqldump --all-databases --add-drop-database -u"root" -p"$MYSQL_ROOT_PASSWORD" > ./dump.sql

subrotoxing • Apr 15 '22 • Edited

I'm trying to backup my database ... but dump.sql seems to be empty... please advise

i'm typing this into cmd where the .env file located

docker-compose exec mariadb sh -c exec mysqldump --all-databases --add-drop-database -u user -p password > ./dump.sql

Matej Bačo • May 4 '22 • Edited

Hey there 👋 Tiny syntax mistake, it should be:

docker-compose exec mariadb sh -c exec mysqldump --all-databases --add-drop-database -u"user" -p"password" > ./dump.sql

Nicolas Klein • Jul 29 '22

Still does not work for me on Appwrite v15

Matej Bačo • Nov 1 '21

You can't imagine how many times have I opened this article 😅 Thanks a lot for this article!

mandolf0 • Mar 21 '23 • Edited

Your restore/backup gist helps tremendously. I just migrated local dev to the ☁. One suggestion I'd like to make is to delete the cloud function deployments and use only the latest one. The backup I moved was 800MB because of those deployments. Other than that, 100% confirmed it works to a tee.

As of now with Appwrite 1.2.0, I deleted some rows in other tables.

delete from _console_domains;

//show domain permissions
select * from \_console_domains_perms;

//delete domain permissions
delete from_console_domains_perms;

rsync is our friend when moving to a new server.

rsync -avz --progress backups/backup-2023-03-21T02-05-16.tar.gz root@[targetServer]:/root/appwrite/backups/backup.tar.gz

2023-03-21T02-05-16.tar.gz is the source file to copy
backup.tar.gz is the target because that is the name restore.sh expects to find alongside it.

Issue certificates
docker-compose up -d docker compose exec appwrite ssl domain="app.mydomain.com"

FuadEfendi • Oct 3 '22

And how to backup/restore on MacOS?

Brian Mutiso • Jun 10 '22

I don't seem to get it, even after running the above commands i still get an empty dump.sql file

Nicolas Klein • Jul 29 '22

I found this solution on discord:

here , what i did and it works
backup data :
docker-compose exec mariadb sh -c 'exec mysqldump --all-databases --add-drop-database -u"user" -p"password"' > ./dump.sql

restore data :
docker-compose exec -T mariadb sh -c 'exec mysql -u”user”-p"password"' < dump.sql

View full discussion (19 comments)

DEV Community

Appwrite In Production: Backups and Restores

💾 Backing up the MariaDB Database

✍️ Restoring the MariaDB Database

💿 Backing up your Docker Volumes

📝 Restoring your Docker Volumes

⭐️ In Conclusion

Top comments (19)

Read next

Thoroughly experimented with Fine-Tuning / DreamBooth training of Flux-dev-de-distill, PixelWave v03, Verus Vision

Understanding Git Tags and Checkout

Exploring Java's Role in Cloud Computing and AI for 2024

My Experience with the OSS.GG Hackathon: Diving into Open Source with Twenty, Unkey, and Formbricks