Kolla is a project hatched from openstack. kolla project can produce mirrors including openstack, ceph and other container images.

ansible is an automated deployment tool that performs tasks in playbook.

Kolla-ansible is a container deployment tool, deploying openstack and ceph; the container image deployed by kolla-ansible can be built by kolla or downloaded from docker register (this deployment uses kolla-ansible to deploy CEPH by downloading the image from docker register).

I. Node Planning

host name	ip	role
localhost	172.16.134.43	master node, install kolla-ansible
node58	172.16.134.58	ceph node, with at least one disk used by osd
node59	172.16.134.59	ceph node, with at least one disk used by osd
node61	172.16.134.61	ceph node, with at least one disk used by osd

2. Building master Node

1. Installing docker

yum install -y yum-utils device-mapper-persistent-data lvm2
yum install docker-ce -y

2. Resolving mutual trust between master and ceph nodes

ssh-keygen

ssh-copy-id root@172.16.134.58

ssh-copy-id root@172.16.134.59

ssh-copy-id root@172.16.134.61

3. Installing kolla-ansible dependency packages

yum -y install epel-release
yum install -y python-pip ansible
yum install -y python-devel libffi-devel openssl-devel gcc python-setuptools git

4. Modify the pip source:
mkdir -p ~/.pip
tee ~/.pip/pip.conf <<-'EOF'
[global]
trusted-host=mirrors.aliyun.com
index-url=http://mirrors.aliyun.com/pypi/simple/
EOF
5. Upgrade pip:
pip install -U pip

6. Download kolla-ansible source code and install it

git clone https://github.com/openstack/kolla-ansible.git -b stable/queens

cd kolla-ansilbe

pip install -r requirements.txt -r test-requirements.txt

pip install . -i http://mirrors.aliyun.com/pypi/simple/

7. Copy relevant documents
cp -r etc/kolla /etc/kolla/
cp ansible/inventory/* /home

8. Generating Password

 kolla-genpwd

9. Setting up docker

mkdir /etc/systemd/system/docker.service.d
//Edit kolla.conf file
vim /etc/systemd/system/docker.service.d/kolla.conf 
[Service]

MountFlags=shared
//Edit the daemon.json file
vi /etc/docker/daemon.json
{
"registry-mirrors": ["https://ebu037tr.mirror.aliyuncs.com"],
"insecure-registries": ["docker-registries"]
}

Note: docker-registries is the docker image server. During deployment, kolla-ansible pulls the required image from the docker server, which has the image of each component of ceph.

On the ceph node, docker login {docker-registries} is also used to login to the docker server, otherwise authentication errors will occur during deployment.

10. Restart docker service

systemctl daemon-reload
systemctl restart docker

11. Modify the / etc/hosts file and fill in the ceph node

3. Configuration of the ceph node environment (performing the same operation on three ceph nodes)

1. Disabling node firewalls, security policies, etc.

[root@node58 ~]vim ~/init.sh
#!/bin/sh
sed -i 's/SELINUX=.*/SELINUX=Disabled/g' /etc/selinux/config
echo '' > /etc/resolv.conf
echo nameserver 114.114.114.114 >> /etc/resolv.conf
echo search novalocal >> /etc/resolv.conf
echo " net.ipv4.ip_forward = 1 ">> /etc/sysctl.conf&&sysctl -p
yum install vim wget -y

systemctl stop firewalld

systemctl disable firewalld
-----------------------------------------------------------
[root@node58 ~]# sh init.sh

2. Node Configuration Time Synchronization

[root@node58 ~]# yum install -y chrony

[root@node58 ~]# vi /etc/chrony.conf
server 0.cn.pool.ntp.org iburst
server 1.cn.pool.ntp.org iburst
server 2.cn.pool.ntp.org iburst
server 3.cn.pool.ntp.org iburst

3. Label the disks of the ceph node

[root@node58 ~]# parted /dev/sdb -s -- mklabel gpt mkpart KOLLA_CEPH_OSD_BOOTSTRAP 1 -1

4. Deploy the ceph container service (executed on the master node)

1. Modify kolla-ansible configuration file

[root@node58 ~]# cat /etc/kolla/globals.yml|grep -v '^#'|grep -v '^$'
---
kolla_install_type: "binary"
openstack_release: "queens"
kolla_internal_vip_address: "ip of master"
docker_registry: "{docker-registries}"
docker_namespace: "queens/kolla"
docker_registry_username: "admin"
docker_registry_password: "Harbor12345"
network_interface: "ens33"
enable_ceph: "yes"
enable_haproxy: "no"
enable_keystone: "no"
enable_glance: "no"
enable_neutron: "no"
enable_heat: "no"
enable_nova: "no"
enable_horizon: "no"
ceph_pool_type: "replicated"

Note: The / etc/kolla/globals.yml file will overload the / usr/share/kolla-ansible/ansible/group_vars/all.yml file, and the services that do not need to be installed will be changed to "no" in all.yml.

2. Modify the inventory file of ansible

Fill in the host name of the ceph node under [storage] and empty the remaining section s

6. Deploying the ceph node environment

kolla-ansible bootstrap-servers -i /home/multinode

7. Inspection and deployment

kolla-ansible  prechecks  -i /home/multinode

kolla-ansible deploy -i /home/multinode

8. Testing (performed on the ceph node)

docker exec ceph_mon ceph -s