I. Preparations
1. Download and install vmware, the steps are omitted.
2. Download the ios package for the CentOS system: http://isoredirect.centos.org/centos/7/isos/x86_64/CentOS-7-x86_64-Everything-1611.iso
3. Download and install Xshell 5, step omitted.
4. Download and install git, step omitted.
5.zookeeper official website: http://zookeeper.apache.org/
6.zookeeper api for golang: https://github.com/samuel/go-zookeeper
7. In vmware, click "Create a New Virtual Machine" - > "Typical" - > "Installer CD-ROM Image File" to select the ios file downloaded above, and then quickly install the CentOS system in the next step along the way.
As for why CentOS is installed, other linux versions are also available. The installed CentOS system is as follows:
II. Installation and configuration of zookeeper
Click Applications in the upper left corner of the screen, open Terminal, and download zookeeper to the / usr/local directory, which can be understood as C:/Progrem Files/directory under windows.
First switch to root permission:
Enter su, return to the train, see Password: Enter the password, note that the password will not be displayed, just enter directly after the completion of the train.
Next, execute the command to enter the / usr/local directory:
cd /usr/local
Download zookeeper-3.5.3-beta.tar.gz:
wget https://mirrors.tuna.tsinghua.edu.cn/apache/zookeeper/zookeeper-3.5.3-beta/zookeeper-3.5.3-beta.tar.gz
There will be a download progress bar in terminal. After reading the progress bar, the file will be downloaded. We will decompress it first.
tar -xvf zookeeper-3.5.3-beta.tar.gz
Next we go to the conf directory to set up:
cd zookeeper-3.5.3-beta/conf/
Copy the zoo_sample.cfg file and name it zoo.cfg:
cp zoo_sample.cfg zoo.cfg
We edit zoo.cfg to set up the relevant settings:
vi zoo.cfg
At this time, if you want to modify the content of this file, press the "i" key on the keyboard, the bottom will become INSERT, you can modify it.
Move the cursor to
dataDir=/tmp/zookeeper
Change to
dataDir=/usr/local/zookeeper-3.5.3-beta
Press Esc to exit editing mode, then enter: wq Enter to save and close the file.
Let's take a look at the LAN ip address of the virtual machine:
ifconfig
You can see that the IP address of my virtual machine is 192.168.40.129.
We need to use a software xshell 5 here. Actually, it runs under the terminal of CentOS. But if zookeeper is installed on the server side instead of on the local virtual machine, it will be more convenient to use xshell 5, so we use xshell 5 here.
Open xshell 5 and log in to our virtual machine:
The user name is root, and the password is the password of the virtual machine.
Enter zookeeper's runtime directory:
cd /usr/local/zookeeper-3.5.3-beta/bin
Let's start zookeeper's service:
./zkServer.sh start
The appearance of STARTED indicates that our zookeeper has been started. We can do some simple operations directly from the command line through the client side provided by STARTED, and input:
./zkCli.sh
You can see that we have entered the client side of zookeeper.
Enter help to view commands from the command line.
3. Leader Election of Distributed System by Goang
Next, we use zookeeper and golang to implement Leader election in distributed system.
Installing and configuring the golang environment is not the focus of this article. It is omitted here.
We can use git to get it. Git can download and install it by itself. Here is the GIT command.
Look at the location of golang:
which go
Set GOROOT:
export GOROOT=/c/Go
I put the golang program under the golang folder of the d disk, then set GOPATH:
export GOPATH=/d/golang/
Get the api of zookeeper for go:
go get github.com/samuel/go-zookeeper/zk
Next, create a go project under the golang folder of the d disk, and write the following code to simulate the Leader election process:
package main
import (
"github.com/samuel/go-zookeeper/zk"
"errors"
"time"
"fmt"
)
type ZookeeperConfig struct {
Servers []string
RootPath string
MasterPath string
}
type ElectionManager struct {
ZKClientConn *zk.Conn
ZKConfig *ZookeeperConfig
IsMasterQ chan bool
}
func NewElectionManager(zkConfig *ZookeeperConfig, isMasterQ chan bool) *ElectionManager {
electionManager := &ElectionManager{
nil,
zkConfig,
isMasterQ,
}
electionManager.initConnection()
return electionManager
}
func (electionManager *ElectionManager) Run() {
err := electionManager.electMaster()
if err != nil {
fmt.Println("elect master error, ", err)
}
electionManager.watchMaster()
}
// Determine whether the connection to zookeeper was successful
func (electionManager *ElectionManager) isConnected() bool {
if electionManager.ZKClientConn == nil {
return false
} else if electionManager.ZKClientConn.State() != zk.StateConnected {
return false
}
return true
}
// Initialize zookeeper connection
func (electionManager *ElectionManager) initConnection() error {
// Get a connection to the zookeeper server if the connection is empty or unsuccessful
if !electionManager.isConnected() {
conn, connChan, err := zk.Connect(electionManager.ZKConfig.Servers, time.Second)
if err != nil {
return err
}
// Waiting for the connection to succeed
for {
isConnected := false
select {
case connEvent := <-connChan:
if connEvent.State == zk.StateConnected {
isConnected = true
fmt.Println("connect to zookeeper server success!")
}
case _ = <-time.After(time.Second * 3): // Return connection timeout after 3 seconds of unsuccessful connection
return errors.New("connect to zookeeper server timeout!")
}
if isConnected {
break
}
}
electionManager.ZKClientConn = conn
}
return nil
}
// Election master
func (electionManager *ElectionManager) electMaster() error {
err := electionManager.initConnection()
if err != nil {
return err
}
// Determine whether the root directory exists in zookeeper and create it if it does not exist
isExist, _, err := electionManager.ZKClientConn.Exists(electionManager.ZKConfig.RootPath)
if err != nil {
return err
}
if !isExist {
path, err := electionManager.ZKClientConn.Create(electionManager.ZKConfig.RootPath, nil, 0, zk.WorldACL(zk.PermAll))
if err != nil {
return err
}
if electionManager.ZKConfig.RootPath != path {
return errors.New("Create returned different path " + electionManager.ZKConfig.RootPath + " != " + path)
}
}
// Create a ZNode for electing master, which is Ephemeral type, indicating that when the client disconnects, the created node will be destroyed
masterPath := electionManager.ZKConfig.RootPath + electionManager.ZKConfig.MasterPath
path, err := electionManager.ZKClientConn.Create(masterPath, nil, zk.FlagEphemeral, zk.WorldACL(zk.PermAll))
if err == nil { // Create success to indicate election master success
if path == masterPath {
fmt.Println("elect master success!")
electionManager.IsMasterQ <- true
} else {
return errors.New("Create returned different path " + masterPath + " != " + path)
}
} else { // Creation failure indicates election master failure
fmt.Printf("elect master failure, ", err)
electionManager.IsMasterQ <- false
}
return nil
}
// Monitor master znode in zookeeper, if deleted, indicating master failure or network delay, re-election
func (electionManager *ElectionManager) watchMaster() {
for {
// The sub-znode under watch zk root znode is deleted when a connection is disconnected and re-elected after triggering an event
children, state, childCh, err := electionManager.ZKClientConn.ChildrenW(electionManager.ZKConfig.RootPath + electionManager.ZKConfig.MasterPath)
if err != nil {
fmt.Println("watch children error, ", err)
}
fmt.Println("watch children result, ", children, state)
select {
case childEvent := <-childCh:
if childEvent.Type == zk.EventNodeDeleted {
fmt.Println("receive znode delete event, ", childEvent)
// Reelection
fmt.Println("start elect new master ...")
err = electionManager.electMaster()
if err != nil {
fmt.Println("elect new master error, ", err)
}
}
}
}
}
func main() {
// zookeeper configuration
zkConfig := &ZookeeperConfig{
Servers: []string{"192.168.40.129"},
RootPath: "/ElectMasterDemo",
MasterPath: "/master",
}
// The channel of communication between the main goroutine and the election goroutine is the same as returning the result of the election.
isMasterChan := make(chan bool)
var isMaster bool
// Election
electionManager := NewElectionManager(zkConfig, isMasterChan)
go electionManager.Run()
for {
select {
case isMaster = <-isMasterChan:
if isMaster {
// do some job on master
fmt.Println("do some job on master")
}
}
}
}
Among them, the ip of main function is rewritten to the virtual machine ip found above.
Enter in Xshell 5:
ls /
We can see that there is only one zookeeper in the current root directory.
To successfully call zookeeper using golang program, we need to turn off the firewall under our CentOS system.
Enter in Xshell 5:
systemctl | grep fire
You can see that the firewall is running.
Let's stop the firewall first:
stop firewalld
Then disable firewalls:
disable firewalld
Save and modify:
iptables-save
After successfully shutting down the firewall and running the go program mentioned above, we can see that the first process succeeded as leader:
GOROOT=C:\Go
GOPATH=D:/golang
C:\Go\bin\go.exe build -i -o C:\Users\renjiashuo\AppData\Local\Temp\Build_main_go_and_rungo D:/golang/src/leader/main.go
"C:\Program Files\JetBrains\IntelliJ IDEA 2017.1.5\bin\runnerw.exe" C:\Users\renjiashuo\AppData\Local\Temp\Build_main_go_and_rungo
connect to zookeeper server success!
2017/07/27 21:02:38 Connected to 192.168.40.129:2181
2017/07/27 21:02:38 Authenticated: id=72057981844586499, timeout=4000
2017/07/27 21:02:38 Re-submitting `0` credentials after reconnect
elect master success!
do some job on master
watch children result, [] &{9 9 1501160558009 1501160558009 0 0 0 72057981844586499 0 0 9}
Do not close this program and execute another go program. We can see that because the first process has become the leader, the second process can only wait:
GOROOT=C:\Go
GOPATH=D:/golang
C:\Go\bin\go.exe build -i -o C:\Users\renjiashuo\AppData\Local\Temp\Build_main_go_and_run1go D:/golang/src/leader/main.go
"C:\Program Files\JetBrains\IntelliJ IDEA 2017.1.5\bin\runnerw.exe" C:\Users\renjiashuo\AppData\Local\Temp\Build_main_go_and_run1go
connect to zookeeper server success!
2017/07/27 21:05:29 Connected to 192.168.40.129:2181
2017/07/27 21:05:29 Authenticated: id=72057981844586500, timeout=4000
2017/07/27 21:05:29 Re-submitting `0` credentials after reconnect
elect master failure, %!(EXTRA *errors.errorString=zk: node already exists)watch children result, [] &{9 9 1501160558009 1501160558009 0 0 0 72057981844586499 0 0 9}
When we close the first process, we can see the change of the second process.
GOROOT=C:\Go
GOPATH=D:/golang
C:\Go\bin\go.exe build -i -o C:\Users\renjiashuo\AppData\Local\Temp\Build_main_go_and_run1go D:/golang/src/leader/main.go
"C:\Program Files\JetBrains\IntelliJ IDEA 2017.1.5\bin\runnerw.exe" C:\Users\renjiashuo\AppData\Local\Temp\Build_main_go_and_run1go
connect to zookeeper server success!
2017/07/27 21:05:29 Connected to 192.168.40.129:2181
2017/07/27 21:05:29 Authenticated: id=72057981844586500, timeout=4000
2017/07/27 21:05:29 Re-submitting `0` credentials after reconnect
elect master failure, %!(EXTRA *errors.errorString=zk: node already exists)watch children result, [] &{9 9 1501160558009 1501160558009 0 0 0 72057981844586499 0 0 9}
receive znode delete event, {EventNodeDeleted Unknown /ElectMasterDemo/master <nil> }
start elect new master ...
connect to zookeeper server success!
2017/07/27 21:06:28 Connected to 192.168.40.129:2181
2017/07/27 21:06:28 Authenticated: id=72057981844586501, timeout=4000
2017/07/27 21:06:28 Re-submitting `0` credentials after reconnect
elect master success!
do some job on master
watch children result, [] &{14 14 1501160787685 1501160787685 0 0 0 72057981844586501 0 0 14}
The second process succeeded as leader.
The annotations of the program code are more detailed, but not detailed here. All api interfaces invoked can be accessed to view their go source code.
Since then, the centrOS environment has installed zookeeper service and implemented Leader election of distributed system with golang.