Big data environment building series - zookeeper cluster building

Keywords: Zookeeper vim log4j Apache

Summary

This blog builds a cluster environment based on zookeeper-3.4.9.

1. Download zookeeper

Edition: http://zookeeper.apache.org/releases.html#download

Download version: zookeeper-3.4.9.tar.gz

2. Host

192.168.8.21, 192.168.8.22, 192.168.8.23, 192.168.8.24 and 192.168.8.25.

3. Decompression

tar -zxvf zookeeper-3.4.9.tar.gz

4. Establish symbolic link or move location

mv /bigdata/source/zookeeper-3.4.9 /bigdata/zookeeper

5. Modify environment variables

vim ~/.bash_profile

#zookeeper
export ZK_HOME=/bigdata/zookeeper
export PATH=$PATH:$ZK_HOME/bin

6. Make the configuration file effective

source ~/.bash_profile

7. Distribute configuration files to each host

#Copy environment variables to hosts
scp -r ~/.bash_profile bigdata@cdh01:~/
scp -r ~/.bash_profile bigdata@cdh02:~/
scp -r ~/.bash_profile bigdata@cdh03:~/
scp -r ~/.bash_profile bigdata@cdh04:~/
scp -r ~/.bash_profile bigdata@cdh05:~/

8. Modify the configuration file of zookeeper

#Copy configuration template
cp zoo_sample.cfg zoo.cfg
vim /bidata/zookeeper/conf/zoo.cfg

#Modify data storage location
dataDir=/home/bigdata/zookeeper

#Configure server.x, where x is a number, consistent with the id in the myid file
server.1=cdh01:2888:3888
server.2=cdh02:2888:3888
server.3=cdh03:2888:3888
server.4=cdh04:2888:3888
server.5=cdh05:2888:3888
vim bin/zkEnv.sh
//take
ZOO_LOG_DIR="."  
//Revised to:
ZOO_LOG_DIR="${ZOOKEEPER_PREFIX}/logs"

//take
ZOO_LOG4J_PROP="INFO,CONSOLE"
//Revised to:
ZOO_LOG4J_PROP="INFO,ROLLINGFILE"
vim bin/zkServer.sh
//take
ZOO_DAEMON_OUT="$ZOO_LOG_DIR/zookeeper.out"
//Revised to:
_ZOO_DAEMON_OUT="$ZOO_LOG_DIR/zookeeper.log"
vim conf/log4j.properties
## Change it to INFO, ROLLINGFILE, which should be consistent with the modified part of zkEnv.sh;
zookeeper.root.logger=INFO,ROLLINGFILE
## Set the maximum value of the log file to rotate if it exceeds
log4j.appender.ROLLINGFILE.MaxFileSize=5MB
## Set the maximum number of log files
log4j.appender.ROLLINGFILE.MaxBackupIndex=1

Note: the configuration here is mainly to optimize the log file of zookeeper

On each host/home/bigdata/zookeeper Add in myid,The contents are 1,2,3,4,5
[cdh01]
$>echo 1 > /home/bigdata/zookeeper/myid
[cdh02]
$>echo 2 > /home/bigdata/zookeeper/myid
[cdh03]
$>echo 3 > /home/bigdata/zookeeper/myid
[cdh04]
$>echo 4 > /home/bigdata/zookeeper/myid
[cdh05]
$>echo 5 > /home/bigdata/zookeeper/myid

9, distribution

xsync.sh /bigdata/zookeeper

10. Start

When starting, it must be started one by one, otherwise an error will be reported

#start-up
zkServer.sh start 
#Stop it 
zkServer.sh stop   
#Check the status to confirm whether the leader\follower is elected
zkServer.sh status 

 

 

 

Posted by andreea115 on Tue, 26 Nov 2019 07:28:59 -0800