Security settings when building a cluster on Baidu cloud server

Keywords: firewall ssh Hadoop Spark

After moving the hadoop cluster on the local virtual machine to Baidu cloud server, I found that there are always many unknown ip addresses logging in to my server, because the firewall is closed locally, but in the actual deployment, this is too unsafe. So I spent two hours setting up the firewall of the cluster and forbidding ssh password login

Firewall open and port open

Firewall on

I use centos7, the specific method can be Baidu

#centos7 start firewall
systemctl start firewalld.service
#centos7 stop / close firewall
systemctl stop firewalld.service
#centos7 restart firewall
systemctl restart firewalld.service
 
 
#Set up power on to enable firewall
systemctl enable firewalld.service
#Set the firewall not to be started after power on
systemctl disable firewalld.service

About the commonly used ports of hadoop and spark systems

Use the command to open the default port. I won't explain it here. Please read this article: CentOS 7 firewall open port / delete port / view port

firewall-cmd --zone=public --add-port=80/tcp --permanent
firewall-cmd --zone=public --add-port=80/tcp --permanent
firewall-cmd --zone=public --add-port=80/tcp --permanent
firewall-cmd --zone=public --add-port=80/tcp --permanent
firewall-cmd --zone=public --add-port=80/tcp --permanent
firewall-cmd --zone=public --add-port=80/tcp --permanent
firewall-cmd --zone=public --add-port=80/tcp --permanent
firewall-cmd --zone=public --add-port=9000/tcp --permanent
firewall-cmd --zone=public --add-port=50010/tcp --permanent
firewall-cmd --zone=public --add-port=50075/tcp --permanent
firewall-cmd --zone=public --add-port=50475/tcp --permanent
firewall-cmd --zone=public --add-port=50020/tcp --permanent
firewall-cmd --zone=public --add-port=50020/tcp --permanent
firewall-cmd --zone=public --add-port=50070/tcp --permanent
firewall-cmd --zone=public --add-port=50470/tcp --permanent
firewall-cmd --zone=public --add-port=8020/tcp --permanent
firewall-cmd --zone=public --add-port=8485/tcp --permanent
firewall-cmd --zone=public --add-port=8480/tcp --permanent
firewall-cmd --zone=public --add-port=8019/tcp --permanent
firewall-cmd --zone=public --add-port=8032/tcp --permanent
firewall-cmd --zone=public --add-port=8030/tcp --permanent
firewall-cmd --zone=public --add-port=8031/tcp --permanent
firewall-cmd --zone=public --add-port=8088/tcp --permanent
firewall-cmd --zone=public --add-port=8040/tcp --permanent
firewall-cmd --zone=public --add-port=8042/tcp --permanent
firewall-cmd --zone=public --add-port=8041/tcp --permanent
firewall-cmd --zone=public --add-port=10020/tcp --permanent
firewall-cmd --zone=public --add-port=19888/tcp --permanent
firewall-cmd --zone=public --add-port=60000/tcp --permanent
firewall-cmd --zone=public --add-port=60010/tcp --permanent
firewall-cmd --zone=public --add-port=60020/tcp --permanent
firewall-cmd --zone=public --add-port=60030/tcp --permanent
firewall-cmd --zone=public --add-port=2181/tcp --permanent
firewall-cmd --zone=public --add-port=2888/tcp --permanent
firewall-cmd --zone=public --add-port=3888/tcp --permanent
firewall-cmd --zone=public --add-port=9083/tcp --permanent
firewall-cmd --zone=public --add-port=10000/tcp --permanent
#spark default port
firewall-cmd --zone=public --add-port=8020/tcp --permanent
firewall-cmd --zone=public --add-port=8088/tcp --permanent
firewall-cmd --zone=public --add-port=8080/tcp --permanent
firewall-cmd --zone=public --add-port=7077/tcp --permanent
firewall-cmd --zone=public --add-port=8081/tcp --permanent
firewall-cmd --zone=public --add-port=9083/tcp --permanent
firewall-cmd --zone=public --add-port=18080/tcp --permanent
firewall-cmd --zone=public --add-port=4040/tcp --permanent
firewall-cmd --zone=public --add-port=9083/tcp --permanent
firewall-cmd --zone=public --add-port=60010/tcp --permanent
firewall-cmd --zone=public --add-port=6379/tcp --permanent
firewall-cmd --zone=public --add-port=8080/tcp --permanent 
firewall-cmd --zone=public --add-port=9092/tcp --permanent 

For the sorting of ports, please move to the original: Application list of Hadoop default port
Spark part: summary of several important ports

Configure public key and disable password login

  1. windows terminal
    Open cmd, input SSH keygen to generate public key and private key, as shown below

    Just press enter all the way, and then find the path to save the public key according to the prompts in the figure, and upload the public key to the server. Pictured

    The operation of windows is over.

  2. linux server
    Append the public key to the / root/.ssh/authorized_keys file

cat id_rsa.pub >> .ssh/authorized_keys

Finally, set ssh to disable password login:

About ssh settings, please move to this article, very clear! ssh login security settings

Finally, the ssh service was restarted and the work was completed

 systemctl restart sshd.service

Try again and you will find that you can't log in with password, but only with public key!

***Note: when you restart the service, make sure that you can log in with the public key!

Published 1 original article, praised 0 and visited 5
Private letter follow

Posted by wonderman on Sun, 15 Mar 2020 02:23:32 -0700