MongoDB cluster and security

Replica set

It is a group of Mongod services that maintain the same data set. Replica set can provide redundancy and high availability. It is the basis of all production deployments.

At the same time, it is also similar to the master-slave cluster with automatic fault recovery function. Multiple machines are used for asynchronous and synchronous of the same data, so that multiple machines have multiple copies of the same data. In addition, when the primary database goes down, users do not need to dare to automatically switch other backup servers as the primary database. You can also use the replica server as a read-only server to realize the separation of read and write and improve the load.

(1) Redundancy and data availability

Replication provides redundancy and improves data availability. By providing multiple copies of data on different database servers, replication improves a level of fault tolerance to prevent the loss of a single database server.

In some cases, replication can provide increased read performance because clients can send read operations to different services. Maintaining data copies in different data centers can increase the data location and availability of distributed applications. It can also be used to maintain other copies, disaster recovery, reporting or backup.

(2) Copy

Replica sets are a set of mongod instances that maintain the same dataset. It contains multiple data bearer nodes and an optional arbitration node. Among the nodes carrying data, one and only one becomes the primary node, and other nodes are regarded as secondary nodes.

The master node receives all write operations, and only one copy set can confirm the write date with {w: "most"} write concern; Although in some cases, another mongod instance may temporarily think it is also primary. It mainly records all changes to the dataset in its operation log, that is, oplog.

(3) Differences between master-slave replica sets and replica sets

The biggest difference between the master-slave cluster and the replica set is that the replica set does not have a fixed "master node"; The whole cluster will select a "master node". After it hangs up, select other nodes from the remaining slave nodes as "master nodes". The replica set always has an active point {master and primary} and one or more backup nodes {slave and secondary}

Three roles for replica sets

Replica sets have two data types and three roles

Two types:

• Primary node type: the primary connection point of data operation, which is readable and writable.
• Secondary (auxiliary and slave) node type: data redundant backup node, which can be read or elected.

Three roles:

1. primary: it mainly receives all write operations. Is the master node.

2. Replica member: maintain the same dataset through replication from the master node, that is, data backup. It is not writable, but can be read (but needs to be configured). Is the default slave node type.

3. Arbiter: does not retain any copies of data, but only has the function of voting. Of course, the arbitration server can also be maintained as part of the replica set, that is, the replica members can also be arbitrators. It is also a slave node type.

Build replica set

One master one slave one arbitration.

Master node

Establish a directory for storing data and logs

#---------myrs
#Master node
mkdir -p /Users/didi/xzy/replica_sets/myrs_27017/log
mkdir -p /Users/didi/xzy/replica_sets/myrs_27017/data/db

To create or modify a profile:

vim /Users/didi/xzy/replica_sets/myrs_27017/mongod.conf

myrs_27017:

systemLog:
  #The destination of all log output sent by MongoDB is specified as a file
  destination: file
  #The path of the log file to which mongod or mongos should send all diagnostic logging information
  path: "/Users/didi/xzy/replica_sets/myrs_27017/log/mongod.log"
  #When the mongos or mongod instance restarts, mongos or mongod appends a new entry to the end of the existing log file.
  logAppend: true
storage:
  #The directory where mongod instance stores its data. The storage.dbPath setting is only applicable to mongod.
  dbPath: "/Users/didi/xzy/replica_sets/myrs_27017/data/db"
  journal:
    #Enable or disable persistent logging to ensure that data files remain valid and recoverable.
    enabled: true
processManagement:
  #Enable daemon mode for running mongos or mongod processes in the background
  fork: true
  #Specifies the file location where the process ID of the mongos or mongod process is saved, where mongos or mongod will write its PID
  pidFilePath: "/Users/didi/xzy/replica_sets/myrs_27017/log/mongod.pid"
net:
  #The service instance binds all IPS, which has side effects. When the replica set is initialized, the node name will be automatically set to the local domain name instead of IP
  #bindIpAll:true
  #IP address bound by the service instance
  bindIp: localhost
  #bindIp
  #Bound port
  port: 27017
replication:
  #The name of the replica set
  replSetName: myrs

Start node service:

mongod -f /Users/didi/xzy/replica_sets/myrs_27017/mongod.conf

mongod --dbpath /Users/didi/xzy/replica_sets/myrs_27017/data/db --logpath /Users/didi/xzy/replica_sets/myrs_27017/log/mongod.log --fork

Slave node

Establish a directory for storing data and logs

#---------myrs
#Slave node
mkdir -p /Users/didi/xzy/replica_sets/myrs_27018/log
mkdir -p /Users/didi/xzy/replica_sets/myrs_27018/data/db

To create or modify a profile:

vim /Users/didi/xzy/replica_sets/myrs_27018/mongod.conf

myrs_27018:

systemLog:
  #The destination of all log output sent by MongoDB is specified as a file
  destination: file
  #The path of the log file to which mongod or mongos should send all diagnostic logging information
  path: "/Users/didi/xzy/replica_sets/myrs_27018/log/mongod.log"
  #When the mongos or mongod instance restarts, mongos or mongod appends a new entry to the end of the existing log file.
  logAppend: true
storage:
  #The directory where mongod instance stores its data. The storage.dbPath setting is only applicable to mongod.
  dbPath: "/Users/didi/xzy/replica_sets/myrs_27018/data/db"
  journal:
    #Enable or disable persistent logging to ensure that data files remain valid and recoverable.
    enabled: true
processManagement:
  #Enable daemon mode for running mongos or mongod processes in the background
  fork: true
  #Specifies the file location where the process ID of the mongos or mongod process is saved, where mongos or mongod will write its PID
  pidFilePath: "/Users/didi/xzy/replica_sets/myrs_27018/log/mongod.pid"
net:
  #The service instance binds all IPS, which has side effects. When the replica set is initialized, the node name will be automatically set to the local domain name instead of IP
  #bindIpAll:true
  #IP address bound by the service instance
  bindIp: localhost
  #bindIp
  #Bound port
  port: 27018
replication:
  #The name of the replica set
  replSetName: myrs

Start service

mongod -f /Users/didi/xzy/replica_sets/myrs_27018/mongod.conf

mongod --dbpath /Users/didi/xzy/replica_sets/myrs_27018/data/db --logpath /Users/didi/xzy/replica_sets/myrs_27018/log/mongod.log --fork

Arbitration node

Establish a directory for storing data and logs

#---------myrs
#Slave node
mkdir -p /Users/didi/xzy/replica_sets/myrs_27019/log
mkdir -p /Users/didi/xzy/replica_sets/myrs_27019/data/db

To create or modify a profile:

vim /Users/didi/xzy/replica_sets/myrs_27019/mongod.conf

myrs_27019:

systemLog:
  #The destination of all log output sent by MongoDB is specified as a file
  destination: file
  #The path of the log file to which mongod or mongos should send all diagnostic logging information
  path: "/Users/didi/xzy/replica_sets/myrs_27019/log/mongod.log"
  #When the mongos or mongod instance restarts, mongos or mongod appends a new entry to the end of the existing log file.
  logAppend: true
storage:
  #The directory where mongod instance stores its data. The storage.dbPath setting is only applicable to mongod.
  dbPath: "/Users/didi/xzy/replica_sets/myrs_27019/data/db"
  journal:
    #Enable or disable persistent logging to ensure that data files remain valid and recoverable.
    enabled: true
processManagement:
  #Enable daemon mode for running mongos or mongod processes in the background
  fork: true
  #Specifies the file location where the process ID of the mongos or mongod process is saved, where mongos or mongod will write its PID
  pidFilePath: "/Users/didi/xzy/replica_sets/myrs_27019/log/mongod.pid"
net:
  #The service instance binds all IPS, which has side effects. When the replica set is initialized, the node name will be automatically set to the local domain name instead of IP
  #bindIpAll:true
  #IP address bound by the service instance
  bindIp: localhost
  #bindIp
  #Bound port
  port: 27019
replication:
  #The name of the replica set
  replSetName: myrs

Start service

mongod -f /Users/didi/xzy/replica_sets/myrs_27019/mongod.conf

mongod --dbpath /Users/didi/xzy/replica_sets/myrs_27019/data/db --logpath /Users/didi/xzy/replica_sets/myrs_27019/log/mongod.log --fork

Connection node

Use the client command to connect any node, but try to connect the master node (27017 node) to make it the master node:

mongo --host=localhost --port=27017

The replica must be initialized after connection

rs.initiate()  #Additive parameter configuration

After initialization, click enter to change from secondary to primary

You can use it later

rs.conf() and rs.status() to view the corresponding information

Add replica slave node

Add a slave node to the master node and add other members to the replica set

Syntax:

rs.add(host,arbiterOnly)

Parameter	Type	Description
host	string or document	A new member to add to the replica set. Specify as string or configuration document: 1) if it is a string, specify the host name and optional port number of the new member; 2) If it is a document, specify the replica set member configuration document found in the members array. You must specify the host field in the member configuration document. For the description of the document configuration field, see the following document: "configuration document of host members"
arbiterOnly	boolean	Optional. Applicable only if the value is a string. If true, the added host is the arbiter.

Configuration documents for host members:

{
  _id:<int>,
  host:<string>,
  arbiterOnly:<boolean>,
  buildIndexes:<boolean>,
  hidden:<boolean>,
  priority:<number>,
  tags:<document>,
  slaveDelay:<int>,
  votes:<number>
}

Example:

Add the replica node of 27018 to the replica set summary:

rs.add("localhost:27018")

Add arbiter node

rs.add(host,arbiterOnly)
or
rs.addArb(host)

example:
rs.addArb("localhost:27019")

Replica set read and write operations

Log in to the master node 27017 to write and read data:

mongo --host localhost --port 27017
use test
db.comment.insert({"articleid":"100000","content":"It's a nice day and sunny today","userid":"1001","nickname":"Aoi","createdatetime":new Date()})

Login slave node:

mongo --host localhost --port 27018
#At this time, it will be found that no data can be read when entering. First change the current node to a slave node
rs.slaveOk()
or
rs.slaveOk(true)

To cancel the slave node
rs.slaveOk(false)

Arbiter node

This node does not store any data information, but is only used to view configuration information

Election principle of master node

MongoDB will automatically elect the master node in the replica set. The trigger conditions for the master node election are as follows:

1. Master node failure
2. The primary node network is unreachable (the default heartbeat information is 10 seconds)
3. Manual intervention (rs.stepDown(600))

Once the election is triggered, the master node must be selected according to certain rules

The election rule is to decide who wins according to the number of votes:

• The node with the highest number of votes and the voting support of the "majority" members wins

The definition of "majority" is: assuming that the number of voting members in the replica set is N, the majority is N/2+1. For example, if there are 3 voting members, the majority value is 2. When the number of surviving members in the replica set is less than the majority, the entire replica set will not be able to elect a Primary, and the replica set will not be able to provide write services and will be in a read-only state.

• If the number of votes is the same and they are supported by the "majority" members, the new data node wins.

The old and new data are compared through the operation log oplog.

SpringDataMongoDB connection replica set

Syntax:

mongodb://Host1, host2, host3 /? Connect = replicaset & slaveok = true & replicaset = replica set name

Of which:

• slaveOk=true: enable the read function of replica node to realize read-write separation.
• connect=replicaSet: automatically select the read-write host in the replica set. If slaveOk is on, read-write separation is realized.

Example:

Connect three replica set servers (port 270172701827019) and directly connect the first server, whether it is part of replica set or the master server or the slave server. Write operations are applied to the master server and distributed queries to the slave server.

spring: 
	#Data source configuration
	data:
		mongodb: 
			#Host address
			#host: localhost
			#database
			#database: test
			#The default port number is 27017
			#port: 27017
			#You can also use uri connections
			uri: mongodb://localhost:27017,localhost:27018,localhost:27019/test?connect=replicaSet&slaveOk=true&replicaSet=myrs