Mongodb backup restore common commands

Keywords: Database MongoDB JSON snapshot

Mongodb's export mongodump is used in combination with mongorestore import tool. Can be used for database backup and restore.

mongodump is a utility for creating binary exports of database content.
mongodump can export data from mongod or mongos instances; that is, it can export data from independent, replica set and shard cluster deployment.
In order to avoid affecting the online business, when we use mongodup tool, we should try our best to perform operations at the slave node or delay node of the replica set.

Mongodump and mongorestore can't be part of the 4.2 partition cluster backup strategy, because the backups created with mongodump can't keep the atomicity of cross partition transactions.

The website also emphasizes:
For a 4.2 sharding cluster with sharding transactions in progress, use one of the following coordinated backup and restore processes, which does maintain the atomicity guarantee of cross sharding transactions:
MongoDB Atlas,
MongoDB Cloud Manager, or
MongoDB Ops Manager.

1. You can use mongodump --help to see how to use help documents.

As long as we have the mongodb service installed, we can use the mongodump command,

1.1. Backup the entire instance:

Mongodump - H IP: Port - O export to directory

[root@VM_6_17_centos ~]# /root/mongodb-4.2.1/bin/mongodump -h 192.168.6.17:27020 -o dump/
2020-05-27T10:49:21.610+0800    writing admin.system.version to
2020-05-27T10:49:21.610+0800    done dumping admin.system.version (1 document)
2020-05-27T10:49:21.611+0800    writing test_haijiao.user_file to
2020-05-27T10:49:21.611+0800    writing test_jia.user_info to
2020-05-27T10:49:21.611+0800    done dumping test_jia.user_info (2 documents)
2020-05-27T10:49:21.611+0800    done dumping test_haijiao.user_file (1 document)

1.2. Backup a single database:

Mongodump - H IP: Port - D database name - o export to directory

[root@VM_6_17_centos conf]# /root/mongodb-4.2.1/bin/mongodump -h 192.168.6.17:27020  -d test_jia -o dump/
2020-05-27T10:53:11.404+0800    writing test_jia.user_info to
2020-05-27T10:53:11.405+0800    done dumping test_jia.user_info (2 documents)

1.3. A table of backup database:

Mongodump - H IP: Port - D database name - c collection name - o export to directory

[root@VM_6_17_centos conf]# /root/mongodb-4.2.1/bin/mongodump -h 192.168.6.17:27020  -d test_jia -c user_info  -o dump/
2020-05-27T10:53:49.162+0800    writing test_jia.user_info to
2020-05-27T10:53:49.162+0800    done dumping test_jia.user_info (2 documents)

1.4. Back up a table of the database and filter it:

Look at the data in the table. We filter the data older than 30

db.user_info.find()
{ "_id" : ObjectId("5ecdcd0bc6a43453c65e484f"), "name" : "jiahaijiao", "age" : 32 }
{ "_id" : ObjectId("5ecdcde7c6a43453c65e4850"), "name" : "jiahaijiao", "age" : 32 }
{ "_id" : ObjectId("5ecdd6f5bf235c4a69f3b672"), "name" : "zhang", "age" : 30 }
{ "_id" : ObjectId("5ecdd6febf235c4a69f3b673"), "name" : "wang", "age" : 30 }
{ "_id" : ObjectId("5ecdd780bf235c4a69f3b674"), "name" : "li", "age" : 28 }

[root@VM_6_17_centos conf]# /root/mongodb-4.2.1/bin/mongodump -h 192.168.6.17:27020  -d test_jia -c user_info -q '{"age":{"$gt": 30}}'  -o dump/
2020-05-27T11:18:19.103+0800    writing test_jia.user_info to
2020-05-27T11:18:19.103+0800    done dumping test_jia.user_info (2 documents)

1.5 export the compressed file of a single library:

[root@VM_6_17_centos conf]# /root/mongodb-4.2.1/bin/mongodump -h 192.168.6.17:27020  -d test_jia -o dump/ --gzip
2020-05-27T11:19:33.748+0800    writing test_jia.user_info to
2020-05-27T11:19:33.749+0800    done dumping test_jia.user_info (5 documents)
[root@VM_6_17_centos conf]# ll dump/
//Total dosage 4
drwxr-xr-x 2 root root 4096 5 November 27:19 test_jia
[root@VM_6_17_centos conf]# ll dump/test_jia/
//Total consumption 16
-rw-r--r-- 1 root root 112 5 November 27:18 user_info.bson
-rw-r--r-- 1 root root 152 5 November 27:19 user_info.bson.gz
-rw-r--r-- 1 root root 166 5 November 27:18 user_info.metadata.json
-rw-r--r-- 1 root root 151 5 November 27:19 user_info.metadata.json.gz

1.6. Set the concurrent number of export sets to speed up the export

-j, --numParallelCollections= number of collections to dump in parallel (4 by default) (default: 4) //4 concurrent default exported documents

[root@VM_6_17_centos conf]# /root/mongodb-4.2.1/bin/mongodump -h 192.168.6.17:27020  -d test_jia -o dump/ --gzip -j 6
2020-05-27T11:23:34.568+0800    writing test_jia.user_info to
2020-05-27T11:23:34.569+0800    done dumping test_jia.user_info (5 documents)
[root@VM_6_17_centos conf]#

1.7 data authentication administrator backup

The user name and password are required after the database is opened for authentication and authorization, and the authentication database parameters are backed up.

[root@VM_6_17_centos ~]# /root/mongodb-4.2.1/bin/mongodump -h 192.168.6.17:27020  -u dba -p12387654  -d test_jia -o dump/   --authenticationDatabase=admin
2020-05-27T15:06:51.049+0800    writing test_jia.user_info to
2020-05-27T15:06:51.050+0800    done dumping test_jia.user_info (5 documents)
[root@VM_6_17_centos ~]#

2,mongorestore

Restore the backup generated by mongodump to the running instance

2.1 restore the whole instance

[root@VM_6_17_centos ~]#  /root/mongodb-4.2.1/bin/mongorestore -h 192.168.6.17:27030  dump/
2020-05-27T15:25:52.968+0800    preparing collections to restore from
2020-05-27T15:25:52.970+0800    reading metadata for test_jia.user_info from dump/test_jia/user_info.metadata.json
2020-05-27T15:25:52.973+0800    reading metadata for test_haijiao.user_file from dump/test_haijiao/user_file.metadata.json
2020-05-27T15:25:52.996+0800    restoring test_jia.user_info from dump/test_jia/user_info.bson
2020-05-27T15:25:52.998+0800    no indexes to restore
2020-05-27T15:25:52.998+0800    finished restoring test_jia.user_info (5 documents, 0 failures)
2020-05-27T15:25:53.007+0800    restoring test_haijiao.user_file from dump/test_haijiao/user_file.bson
2020-05-27T15:25:53.011+0800    no indexes to restore
2020-05-27T15:25:53.011+0800    finished restoring test_haijiao.user_file (1 document, 0 failures)
2020-05-27T15:25:53.011+0800    restoring users from dump/admin/system.users.bson
2020-05-27T15:25:53.078+0800    6 document(s) restored successfully. 0 document(s) failed to restore.

[root@VM_6_17_centos ~]#

2.1 restore a single library

[root@VM_6_17_centos ~]#  /root/mongodb-4.2.1/bin/mongorestore -h 192.168.6.17:27030  --db test_jia  dump/test_jia/
2020-05-27T15:35:06.633+0800    the --db and --collection args should only be used when restoring from a BSON file. Other uses are deprecated and will not exist in the future; use --nsInclude instead
2020-05-27T15:35:06.634+0800    building a list of collections to restore from dump/test_jia dir
2020-05-27T15:35:06.634+0800    reading metadata for test_jia.user_info from dump/test_jia/user_info.metadata.json
2020-05-27T15:35:06.657+0800    restoring test_jia.user_info from dump/test_jia/user_info.bson
2020-05-27T15:35:06.658+0800    no indexes to restore
2020-05-27T15:35:06.658+0800    finished restoring test_jia.user_info (5 documents, 0 failures)
2020-05-27T15:35:06.658+0800    5 document(s) restored successfully. 0 document(s) failed to restore.

2.2 restore a single table of the library

[root@VM_6_17_centos ~]#  /root/mongodb-4.2.1/bin/mongorestore -h 192.168.6.17:27030  --db test_jia  -c user_info  dump/test_jia/user_hobby.bson
2020-05-27T15:47:52.552+0800    checking for collection data in dump/test_jia/user_hobby.bson
2020-05-27T15:47:52.552+0800    restoring to existing collection test_jia.user_info without dropping
2020-05-27T15:47:52.552+0800    reading metadata for test_jia.user_info from dump/test_jia/user_hobby.metadata.json
2020-05-27T15:47:52.552+0800    restoring test_jia.user_info from dump/test_jia/user_hobby.bson
2020-05-27T15:47:52.613+0800    no indexes to restore
2020-05-27T15:47:52.613+0800    finished restoring test_jia.user_info (1 document, 0 failures)
2020-05-27T15:47:52.613+0800    1 document(s) restored successfully. 0 document(s) failed to restore.

2.3 compress backup files and restore

[root@VM_6_17_centos ~]#  /root/mongodb-4.2.1/bin/mongorestore -h 192.168.6.17:27030 --gzip --db test_jia   dump/test_jia/
2020-05-27T15:54:05.013+0800    the --db and --collection args should only be used when restoring from a BSON file. Other uses are deprecated and will not exist in the future; use --nsInclude instead
2020-05-27T15:54:05.013+0800    building a list of collections to restore from dump/test_jia dir
2020-05-27T15:54:05.014+0800    reading metadata for test_jia.user_info from dump/test_jia/user_info.metadata.json.gz
2020-05-27T15:54:05.014+0800    reading metadata for test_jia.user_hobby from dump/test_jia/user_hobby.metadata.json.gz
2020-05-27T15:54:05.038+0800    restoring test_jia.user_info from dump/test_jia/user_info.bson.gz
2020-05-27T15:54:05.041+0800    no indexes to restore
2020-05-27T15:54:05.041+0800    finished restoring test_jia.user_info (5 documents, 0 failures)
2020-05-27T15:54:05.063+0800    restoring test_jia.user_hobby from dump/test_jia/user_hobby.bson.gz
2020-05-27T15:54:05.065+0800    no indexes to restore
2020-05-27T15:54:05.065+0800    finished restoring test_jia.user_hobby (1 document, 0 failures)
2020-05-27T15:54:05.065+0800    6 document(s) restored successfully. 0 document(s) failed to restore.

2.4 delete the existing set during restore

[root@VM_6_17_centos ~]#  /root/mongodb-4.2.1/bin/mongorestore -h 192.168.6.17:27030 --gzip  --drop --db test_jia  -c user_info  dump/test_jia/user_info.bson.gz
2020-05-27T15:57:18.065+0800    checking for collection data in dump/test_jia/user_info.bson.gz
2020-05-27T15:57:18.070+0800    reading metadata for test_jia.user_info from dump/test_jia/user_info.metadata.json.gz
2020-05-27T15:57:18.085+0800    restoring test_jia.user_info from dump/test_jia/user_info.bson.gz
2020-05-27T15:57:18.146+0800    no indexes to restore
2020-05-27T15:57:18.146+0800    finished restoring test_jia.user_info (5 documents, 0 failures)
2020-05-27T15:57:18.146+0800    5 document(s) restored successfully. 0 document(s) failed to restore.

2.5 restore the number of parallel sets

The default number of parallel sets is 4. We can set this parameter

[root@VM_6_17_centos ~]#  /root/mongodb-4.2.1/bin/mongorestore -h 192.168.6.17:27030 --gzip  --drop  dump/  -j 6
2020-05-27T16:15:17.370+0800    preparing collections to restore from
2020-05-27T16:15:17.375+0800    reading metadata for test_jia.user_info from dump/test_jia/user_info.metadata.json.gz
2020-05-27T16:15:17.379+0800    reading metadata for test_jia.user_hobby from dump/test_jia/user_hobby.metadata.json.gz
2020-05-27T16:15:17.404+0800    restoring test_jia.user_info from dump/test_jia/user_info.bson.gz
2020-05-27T16:15:17.407+0800    no indexes to restore
2020-05-27T16:15:17.407+0800    finished restoring test_jia.user_info (5 documents, 0 failures)
2020-05-27T16:15:17.412+0800    restoring test_jia.user_hobby from dump/test_jia/user_hobby.bson.gz
2020-05-27T16:15:17.413+0800    no indexes to restore
2020-05-27T16:15:17.413+0800    finished restoring test_jia.user_hobby (1 document, 0 failures)
2020-05-27T16:15:17.413+0800    6 document(s) restored successfully. 0 document(s) failed to restore.
[root@VM_6_17_centos ~]#

2.5 enable the recovery of authentication administrator

[root@VM_6_17_centos ~]#  /root/mongodb-4.2.1/bin/mongorestore -h 192.168.6.17:27030 --gzip  --drop  --dir  dump/  -j 6    -u dba -p12387654  --authenticationDatabase=admin
2020-05-27T16:22:23.883+0800    preparing collections to restore from
2020-05-27T16:22:23.899+0800    reading metadata for test_jia.user_hobby from dump/test_jia/user_hobby.metadata.json.gz
2020-05-27T16:22:23.940+0800    restoring test_jia.user_hobby from dump/test_jia/user_hobby.bson.gz
2020-05-27T16:22:23.941+0800    reading metadata for test_jia.user_info from dump/test_jia/user_info.metadata.json.gz
2020-05-27T16:22:23.942+0800    no indexes to restore
2020-05-27T16:22:23.942+0800    finished restoring test_jia.user_hobby (1 document, 0 failures)
2020-05-27T16:22:23.964+0800    restoring test_jia.user_info from dump/test_jia/user_info.bson.gz
2020-05-27T16:22:23.965+0800    no indexes to restore
2020-05-27T16:22:23.966+0800    finished restoring test_jia.user_info (5 documents, 0 failures)
2020-05-27T16:22:23.966+0800    6 document(s) restored successfully. 0 document(s) failed to restore.

In the case of small amount of data, mongodump can be used for backup. In the case of large amount of data, delayed backup of replication set and disk snapshot of cloud server can be used for backup.

For incremental oplog backup, please click the following link.

Mongodb recovery to any point in time

Posted by rmbarnes82 on Wed, 27 May 2020 06:10:35 -0700