MeteorJs and Meteor-Up MongoDB migration to a new major version 🛠️

The problem: you have a Meteor app deployed with MUP and it's running reliable since ages but it's using an old MongoDB version. Now, you can't update Meteor to newer releases, because it requires a newer (and incompatible) version of MongoDB (related issue here).

In my case it was an old MongoDB version 3.4.1, which was still running on the servers. The Meteor release was 2.7.3 and required MongoDB 5+. Simply increasing the MongoDB version in the mup.js config will not work out (see the upcoming section).

🤝 This tutorial will guide you through the whole upgrade process. The main focus is on keeping your data safe, so you will have a nice and smooth upgrade experience.

☝️ Disclaimers: There is always a chance of data-loss. I am not responsible for any problems, related to any of the steps, involved by this tutorial. Furthermore, this article contains affiliate links, marked with a *.

What happens if you try to deploy anyway 💥

I already knew that it won't work, since I regularly read the Meteor changelog, as well as their announcements in the Meteor forums* and in the Meteor blog*. However, I wanted to test with a staging instance, whether it would work anyway.

Of course, it didn't work:

Started TaskList: Start Meteor
[xxx.xxx.xxx.xxx] - Start Meteor
[xxx.xxx.xxx.xxx] - Start Meteor: SUCCESS
[xxx.xxx.xxx.xxx] - Verifying Deployment
[xxx.xxx.xxx.xxx] x Verifying Deployment: FAILED

          ------------------------------------STDERR------------------------------------
(7) Failed to connect to 172.17.0.2 port 3000: Connection refused
=> Logs:
=> Setting node version
NODE_VERSION=14.18.3
v14.18.3 is already installed.
Now using node v14.18.3 (npm v6.14.15)
default -> 14.18.3 (-> v14.18.3 *)
=> Starting meteor app on port 3000
/built_app/programs/server/node_modules/fibers/future.js:313                        throw(ex);
                            ^
    MongoServerSelectionError: Server at mongodb:27017 reports maximum wire version 5, but this version of the Node.js Driver requires at least 6 (MongoDB 3.6)

🤓 save yourself the trouble and skip this step!

Best is you even don't try this and rather get yourself ready for migration.

Alternatives to a manual migration ☁️

If you are already in trouble migrating the MongoDB version on your own server, then you might want to try a managed MongoDB hosting service and Meteor cloud* as they work perfectly together!

Migration overview ✔️

For a fully working migration to a new major version of MongoDB we need to execute the following steps:

• prepare your systems for the migration
• backup your current data
• remove all Docker containers
• remove all MongoDb data
• update mup and mup.js
• setup new environment with mup
• ssh into server
• add ssh2 settings (support rsa) https://github.com/zodern/meteor-up/issues/1318
• restore MongoDb data into new DB
• deploy the app

Preparations for migration

In order to make the migration as fluent as possible you should be well prepared. Make sure, that all of the next steps are checked in your end!

Step 1 - install the latest mup

$ meteor npm install -g mup

🤓 Note, that mup should be installed globally!

Step 2 - ssh and scp

Make sure you have ssh access to your server. You can check this by changing into the deployment directory (where your mup.js file is located) and run meteor mup ssh.

You will also need scp, so make sure it's installed on your system.

If you are using an RSA key for ssh and your server runs on Ubuntu Server at version 22.04 or newer then mup will not be able to do requests via ssh ("All configured authentication methods failed")..

You will have to apply a workaround to make mup work with ssh or (better) replace your ssh keys with keys using the ed25519 methods instead of rsa.

If you need to use the RSA keys then you need to ssh into the server and do the following:

user@target-server:~# sudo echo "PubkeyAcceptedKeyTypes=+ssh-rsa" >> /etc/ssh/sshd_config
user@target-server:~# sudo systemctl restart sshd.service

🤓 I assume you are not ssh into the server as root, however, if you do so, please ignore the sudo commands

Step 3- have your mup.js file ready

It's important to check, that the docker image is compatible with the new Meteor version you will deploy. A good choice are the zodern docker images.

Also check the MongoDB version in the config, as it should be at least 5.0.0.

If have compiled a minimal mup.js config that may give orientation, in case you come from a much older mup version:

module.exports = {
  servers: {
    one: {
      host: 'XXX.XXX.XXX.XXX',
      username: 'www-user'
    }
  },

  app: {
    name: 'myApp',
    path: '../',

    servers: {
      one: {},
    },

    buildOptions: {
      serverOnly: true,
    },

    env: {
      // If you are using ssl, it needs to start with https://./de
      ROOT_URL: 'https://myapp.tld',
      MONGO_URL: 'mongodb://localhost/meteor',
      MONGO_OPLOG_URL: 'mongodb://mongodb/local',
      HTTP_FORWARDED_COUNT: 1,
      port: 8080,
      // other env flags, like
      // MAIL_URL etc.
    },

    docker: {

      image: 'zodern/meteor:latest',

      // if you need run build instructions as ROOT, see
      // https://github.com/zodern/meteor-docker/issues/20
      // buildInstructions: [],


      // (optional) It is set to true when using a docker image
      // that is known to support it. Builds a new docker image containing the
      // app's bundle and npm dependencies to start the app faster and
      // make deploys more reliable and easier to troubleshoot
      prepareBundle: true,

      // (optional, default is false) Uses the new docker image builder
      // during Prepare bundle. When enabled,
      // Prepare Bundle is much faster
      useBuildKit: true,
    },

    // Show progress bar while uploading bundle to server
    // You might need to disable it on CI servers
    enableUploadProgressBar: true
  },

  mongo: {
    version: '5.0.5',
    servers: {
      one: {}
    }
  },

  // (Optional)
  // Use the proxy to setup ssl or to route requests to the correct
  // app when there are several apps

  proxy: {
    domains: 'myapp.tld',
    ssl: {
      forceSSL: true,
      // Enable let's encrypt to create free certificates.
      // The email is used by Let's Encrypt to notify you when the
      // certificates are close to expiring.
      letsEncryptEmail: 'webmaster@myapp.tld'
    },
    clientUploadLimit: '500M'
  }
}

Step 4 - open a new Terminal window

Open a new, clean Terminal window and keep it open for the whole tutorial! We will use a few local shell variables during the tutorial and you will loose them, once the window / tab is closed!

🤓 Of course, I could provide the whole tutorial in a single script but then you wouldn't learn that much. Additionally, such a script would still require lots of configuration or interaction, exceeding the scope of this tutorial dramatically.

Migration 🚧 🏗️ 🚧

Now it's time to get this done with. Grab a hot beverage of your choice, mute the phone and make sure you can 100% focus the next 30 to 60 minutes.

Step 1 - backup your current data ⬇️

At first you should create a local directory for your backup file. Make sure this directory is either outside of your project or part of your .gitignore, otherwise you might check-in production data into your git repository!

$ mkdir -p ~/backups/myapp/

Now you can use a combination of ssh, scp and Docker commands to create the backup and "download" it to the local folder.

To make things more understandable you can prepare the command using a few local variables:

$ BAK_PATH=~/backups/myapp/ # don't use quotes here!
$ BAK_SSH_PATH="user@xxx.xxx.xxx.xxx" # your ssh login, basically
$ BAK_DB_NAME="myApp" # usually the "app" field in mup.js, beware of case-sensitivity!
$ BAK_FILE="$BAK_DB_NAME-mongodump-$(date +%Y-%m-%d-%H-%M).gz"
$ BAK_TEMP_ARCHIVE_DOCKER="/root/mongodump.gz" # backup archive path within docker container
$ BAK_TEMP_ARCHIVE="~/$BAK_FILE" # backup archive path on server os

Now construct the ssh command to generate the DB dump on the server:

$ ssh -t $BAK_SSH_PATH "sudo docker exec -it mongodb mongodump --archive=$BAK_TEMP_ARCHIVE_DOCKER --gzip --quiet && sudo docker cp mongodb:$BAK_TEMP_ARCHIVE_DOCKER $BAK_TEMP_ARCHIVE"
# be patient, don't cancel...

🤓 what happens here?

The command tells your MongoDb container to run a mongodump to the container-owned path /root/mongodump.gz and immediately afterwards copy this file to the server's filesystem (plus renaming it to something like myapp-mongodump-2023-02-01-10-13.gz).

Finally, you can download this file to your local filesystem using scp. Alternatively, you could also upload it some external server or S3 bucket or whatever location you use to store backups.

$ scp $BAK_SSH_PATH:$BAK_TEMP_ARCHIVE $BAK_PATH
myapp-mongodump-2023-02-01-10-18.gz       100%  150MB   6.7MB/s   00:22    
$ ls -la $BAK_PATH # verify file has been downloaded
total 153420
drwxrwxr-x 2 user user      4096 Feb  1 10:23 .
drwxrwxr-x 3 user user      4096 Feb  1 10:17 ..
-rw-r--r-- 1 user user 157086575 Feb  1 10:24 myapp-mongodump-2023-02-01-10-18.gz

From here you could import the backup into your local Meteor development setup and look for errors. I will soon add an own tutorial for this.

Step 2 - wipe data and containers from server 🧹

Warning! From here you will delete things, that might break your setup without recovery. Make sure your DB backup integrity is sufficient.

Go to your .deploy folder (or the folder that contains the mup.js file) and stop / destroy everything:

$ cd .deploy
$ meteor mup stop
Started TaskList: Stop Meteor
[xxx.xxx.xxx.xxx] - Stop Meteor
[xxx.xxx.xxx.xxx] - Stop Meteor: SUCCESS
$ meteor mup mongo stop
Started TaskList: Stop Mongo
[xxx.xxx.xxx.xxx] - Stop Mongo
[xxx.xxx.xxx.xxx] - Stop Mongo: SUCCESS
$ meteor mup meteor destroy --force
The app will be completely removed from the server.
Waiting 5 seconds in case you want to cancel by pressing ctr + c

Started TaskList: Destroy App
[xxx.xxx.xxx.xxx] - Stop App
[xxx.xxx.xxx.xxx] - Stop App: SUCCESS
[xxx.xxx.xxx.xxx] - Destroy App
[xxx.xxx.xxx.xxx] - Destroy App: SUCCESS

Now, ssh into the server and remove the MongoDb data, which is stored outside the container:

🤓 Warning! The next step removes all MongoDb data!

$ ssh $BAK_SSH_PATH
user@target-server:~# rm -rf /opt/mongodb
user@target-server:~# rm -rf /var/lib/mongodb
user@target-server:~# exit

Now we have a "clean" state to setup the new environment.

...

Step 3 - setup new environment with mup 1.5+ 🌱

Stay in your deployment folder and make sure it contains the updated mup.js file (see the preparations section with the example file). Then run the following command:

$ meteor mup setup

Started TaskList: Setup Docker
[xxx.xxx.xxx.xxx] - Setup Docker
[xxx.xxx.xxx.xxx] - Setup Docker: SUCCESS

Started TaskList: Setup Meteor
[xxx.xxx.xxx.xxx] - Setup Environment
[xxx.xxx.xxx.xxx] - Setup Environment: SUCCESS

Started TaskList: Setup Mongo
[xxx.xxx.xxx.xxx] - Setup Environment
[xxx.xxx.xxx.xxx] - Setup Environment: SUCCESS
[xxx.xxx.xxx.xxx] - Copying Mongo Config
[xxx.xxx.xxx.xxx] - Copying Mongo Config: SUCCESS

Started TaskList: Start Mongo
[xxx.xxx.xxx.xxx] - Start Mongo
[xxx.xxx.xxx.xxx] - Start Mongo: SUCCESS

Started TaskList: Setup proxy
[xxx.xxx.xxx.xxx] - Setup Environment
[xxx.xxx.xxx.xxx] - Setup Environment: SUCCESS
[xxx.xxx.xxx.xxx] - Pushing the Startup Script
[xxx.xxx.xxx.xxx] - Pushing the Startup Script: SUCCESS
[xxx.xxx.xxx.xxx] - Pushing Nginx Config Template
[xxx.xxx.xxx.xxx] - Pushing Nginx Config Template: SUCCESS
[xxx.xxx.xxx.xxx] - Pushing Nginx Config
[xxx.xxx.xxx.xxx] - Pushing Nginx Config: SUCCESS
[xxx.xxx.xxx.xxx] - Cleaning Up SSL Certificates
[xxx.xxx.xxx.xxx] - Cleaning Up SSL Certificates: SUCCESS
[xxx.xxx.xxx.xxx] - Configure Nginx Upstream
[xxx.xxx.xxx.xxx] - Configure Nginx Upstream: SUCCESS

Started TaskList: Start proxy
[xxx.xxx.xxx.xxx] - Start proxy
[xxx.xxx.xxx.xxx] - Start proxy: SUCCESS

You can verify the installation using meteor mup status or directly use the MongoDB interactive shell via meteor mup mongo shell. If this looks all good then you can continue to the next step and restore your backup into the new DB.

Step 4 - restore MongoDb data into new DB ⬆️

First, make sure the backup still exists on your server. If not, upload it via scp:

$ scp $BAK_PATH $BAK_SSH_PATH:$BAK_TEMP_ARCHIVE

Then ssh into the server and move the file into the docker container and run mongorestore:

$ ssh $BAK_SSH_PATH
user@target-server:~# docker cp ./myapp-mongodump-2023-02-01-10-18.gz mongodb:/root/mongodump.gz

The file is copied, now open an interactive shell of the mongodb container:

user@target-server:~#  docker exec -it mongodb /bin/bash
root@mongodb:/#

You are not within the container and can now execute the mongorestore command:

root@mongodb:/# mongorestore --archive="/root/mongodump.gz" --gzip --verbose --convertLegacyIndexes --nsInclude="myapp.*"
# ... long verbose output
2023-02-01T10:14:00.937+0000    2425 document(s) restored successfully. 0 document(s) failed to restore.

The restore includes the archive (which we copied into the container under /root/mongodump.gz), the gzip flag and the namespace to include, which is in our case myapp.* (include all collections from the myapp database).
Note, that --convertLegacyIndexes is required to ensure your indexes are still working with the new DB.

If all your documents were restored successfully, you can exit the container shell and the ssh and move on to finally deploy your new app.

Step 5 - deploy the app 🚀

Finally you can deploy your app using

$ meteor mup deploy

From here everything should work the way it was before but now running an updated Meteor and MongoDB.

Troubleshooting 🧐

MongoServerError: E11000 duplicate key error collection: myapp.roles index: name_1 dup key: { name: null }

Open the mongo shell and remove the indexes:

$ meteor mup mongo shell
meteor:PRIMARY> use myapp
meteor:PRIMARY> db.roles.dropIndexes()
exit
$ meteor mup meteor restart

Outlook: automate this tutorial

I highly suggest to wrap these steps into code for further automation. You can try to write it as bash scripts or NodeJs cli tool. Also note, that many of the steps involved to manually ssh into the server or to open a shell for the docker container.

These steps are usually not required to be handled manually.

Sources

• xpressabhi/mup-data-backup
• https://meteor-up.com/
• Radgrad update mongo mup guide, thanks to Philip Johnson

About me

I regularly publish articles here on dev.to about Meteor and JavaScript.

You can also find (and contact) me on GitHub, Twitter and LinkedIn.

If you like what you are reading and want to support me, you can sponsor me on GitHub or send me a tip via PayPal.

Keep up with the latest development on Meteor by visiting their blog* and if you are the same into Meteor like I am and want to show it to the world, you should check out the Meteor merch store*.