Backup and Recovery of Cassandra Management

Keywords: Database snapshot Java

Cassandra's backup is to take a snapshot of existing data, while recovery is to restore the snapshot. Not only can you take a snapshot of the whole library, but you can also backup tables and keyspace s separately. It also supports incremental backup.

1. Full Backup

[root@db03 ~]# nodetool snapshot
Requested creating snapshot(s) for [all keyspaces] with snapshot name [1570691336948] and options {skipFlush=false}
Snapshot directory: 1570691336948

After the operation is completed, it can be verified by listsnapshots, as shown in the following figure:

After the backup is completed, the system generates a 1570691336948 folder under the directory where each table is located, which stores the backup data of each table. The rhnchecksum table is used for illustration, as shown in the following figure:

2. Incremental backup

By default, its incremental backup is disabled and can be verified by the following command:

[root@db03 ~]# nodetool statusbackup
not running

Enable incremental backup with the following commands:

[root@db03 ~]# nodetool enablebackup 
[root@db03 ~]# nodetool statusbackup 
running

The following backups are performed:

[root@db03 spacewalk]# nodetool snapshot
Requested creating snapshot(s) for [all keyspaces] with snapshot name [1570695931097] and options {skipFlush=false}
Snapshot directory: 1570695931097

3. Data recovery

The recovery process of cassandra is as follows:

  • Empty list
cqlsh:spacewalk> select count(*) from rhnchecksum;           

 count
---------
 8404215

(1 rows)

Warnings :
Aggregation query used without partition key

cqlsh:spacewalk> truncate rhnchecksum;
cqlsh:spacewalk> select count(*) from rhnchecksum;

 count
-------
     0

(1 rows)

  • Restore the data in the snapshot directory
    Here, the snapshot directory is copied to the data directory and refresh is executed as follows:
    [root@db03 rhnchecksum-dbbb8330dde111e99dfb45c418aaa2bc]# pwd
    /var/lib/cassandra/data/spacewalk/rhnchecksum-dbbb8330dde111e99dfb45c418aaa2bc
    [root@db03 rhnchecksum-dbbb8330dde111e99dfb45c418aaa2bc]# cp -ra snapshots/1570695931097/* .
    --Copy if you have indexed data
    [root@db03 rhnchecksum-dbbb8330dde111e99dfb45c418aaa2bc]# cp -ra snapshots/1570695931097/.rhnchecksum_idx/* .rhnchecksum_idx/

    After copying the snapshot file to the directory of the corresponding table, running the refresh command to load the new SSTables does not require restarting the machine node.

[root@db03 ~]# nodetool refresh spacewalk rhnchecksum
--After completion, cassandra The following prompts appear in the log:
[root@db03 ~]# tail -f /var/log/cassandra/system.log 
INFO  [RMI TCP Connection(68)-127.0.0.1] 2019-10-10 16:41:22,513 ColumnFamilyStore.java:734 - Loading new SSTables for spacewalk/rhnchecksum...
INFO  [RMI TCP Connection(68)-127.0.0.1] 2019-10-10 16:41:22,517 ColumnFamilyStore.java:782 - Renaming new SSTable /var/lib/cassandra/data/spacewalk/rhnchecksum-dbbb8330dde111e99dfb45c418aaa2bc/md-47-big to /var/lib/cassandra/data/spacewalk/rhnchecksum-dbbb8330dde111e99dfb45c418aaa2bc/md-50-big
INFO  [RMI TCP Connection(68)-127.0.0.1] 2019-10-10 16:41:22,548 ColumnFamilyStore.java:782 - Renaming new SSTable /var/lib/cassandra/data/spacewalk/rhnchecksum-dbbb8330dde111e99dfb45c418aaa2bc/md-49-big to /var/lib/cassandra/data/spacewalk/rhnchecksum-dbbb8330dde111e99dfb45c418aaa2bc/md-51-big
INFO  [RMI TCP Connection(68)-127.0.0.1] 2019-10-10 16:41:22,567 ColumnFamilyStore.java:782 - Renaming new SSTable /var/lib/cassandra/data/spacewalk/rhnchecksum-dbbb8330dde111e99dfb45c418aaa2bc/md-48-big to /var/lib/cassandra/data/spacewalk/rhnchecksum-dbbb8330dde111e99dfb45c418aaa2bc/md-52-big
INFO  [RMI TCP Connection(68)-127.0.0.1] 2019-10-10 16:41:22,589 ColumnFamilyStore.java:782 - Renaming new SSTable /var/lib/cassandra/data/spacewalk/rhnchecksum-dbbb8330dde111e99dfb45c418aaa2bc/md-46-big to /var/lib/cassandra/data/spacewalk/rhnchecksum-dbbb8330dde111e99dfb45c418aaa2bc/md-53-big
INFO  [RMI TCP Connection(68)-127.0.0.1] 2019-10-10 16:41:22,610 ColumnFamilyStore.java:817 - Loading new SSTables and building secondary indexes for spacewalk/rhnchecksum: [BigTableReader(path='/var/lib/cassandra/data/spacewalk/rhnchecksum-dbbb8330dde111e99dfb45c418aaa2bc/md-52-big-Data.db'), BigTableReader(path='/var/lib/cassandra/data/spacewalk/rhnchecksum-dbbb8330dde111e99dfb45c418aaa2bc/md-51-big-Data.db'), BigTableReader(path='/var/lib/cassandra/data/spacewalk/rhnchecksum-dbbb8330dde111e99dfb45c418aaa2bc/md-53-big-Data.db'), BigTableReader(path='/var/lib/cassandra/data/spacewalk/rhnchecksum-dbbb8330dde111e99dfb45c418aaa2bc/md-50-big-Data.db')]
INFO  [RMI TCP Connection(68)-127.0.0.1] 2019-10-10 16:41:22,614 SecondaryIndexManager.java:366 - Submitting index build of rhnchecksum_idx for data in BigTableReader(path='/var/lib/cassandra/data/spacewalk/rhnchecksum-dbbb8330dde111e99dfb45c418aaa2bc/md-52-big-Data.db'),BigTableReader(path='/var/lib/cassandra/data/spacewalk/rhnchecksum-dbbb8330dde111e99dfb45c418aaa2bc/md-51-big-Data.db'),BigTableReader(path='/var/lib/cassandra/data/spacewalk/rhnchecksum-dbbb8330dde111e99dfb45c418aaa2bc/md-53-big-Data.db'),BigTableReader(path='/var/lib/cassandra/data/spacewalk/rhnchecksum-dbbb8330dde111e99dfb45c418aaa2bc/md-50-big-Data.db')
INFO  [Service Thread] 2019-10-10 16:41:34,134 GCInspector.java:284 - ConcurrentMarkSweep GC in 467ms.  CMS Old Gen: 1227906720 -> 508200320; Code Cache: 35251328 -> 35276800; Metaspace: 45251080 -> 45251832; Par Eden Space: 6575264 -> 139631912; Par Survivor Space: 41943040 -> 14759936
INFO  [Service Thread] 2019-10-10 16:41:43,679 GCInspector.java:284 - ParNew GC in 236ms.  CMS Old Gen: 844136912 -> 967814552; Par Eden Space: 335544320 -> 0; Par Survivor Space: 35314704 -> 41943040
INFO  [RMI TCP Connection(68)-127.0.0.1] 2019-10-10 16:47:00,987 SecondaryIndexManager.java:386 - Index build of rhnchecksum_idx complete
INFO  [RMI TCP Connection(68)-127.0.0.1] 2019-10-10 16:47:00,987 ColumnFamilyStore.java:825 - Done loading load new SSTables for spacewalk/rhnchecksum

After recovery, the validation data is shown as follows:

The options of nodetool command are explained in Chinese. Cassandra nodetool common operations

Posted by mgrphp on Fri, 11 Oct 2019 13:44:56 -0700