1, Overview

The deployment flow chart of maven project is as follows:

 

 

 

Environment introduction

 

 

 

 

 

 

 

Due to the shortage of resources, Jenkins+harbor was merged into one. In fact, it should be deployed separately.

For k8s installation, please refer to the link:

https://www.cnblogs.com/xiao987334176/p/11317844.html

 

For gitlab installation, please refer to the link:

https://www.cnblogs.com/xiao987334176/p/11329018.html

 

For harbor installation, please refer to the link:

https://www.cnblogs.com/xiao987334176/p/11326467.html

 

For Jenkins installation, please refer to the link:

https://www.cnblogs.com/xiao987334176/p/11323795.html

 

Note: after Jenkins is installed, the following configuration is required. Otherwise, the function of this article cannot be realized!!!

For the k8s configuration of Jenkins based on https, please refer to the link:

https://www.cnblogs.com/xiao987334176/p/11338827.html

 

GitLab+Jenkins continuous integration, please refer to the link:

https://www.cnblogs.com/xiao987334176/p/11425560.html

 

Jenkins configures maven, please refer to the link:

https://www.cnblogs.com/xiao987334176/p/11433636.html

 

2, Creating maven project with gitlab

  First, find a maven based project from github. The address is as follows:

https://github.com/solochen84/SpringBootDemo

Use the git client to download the project code

 

Create project

Log in to gitlab and create a project called spring boot demo

 

 

Click Set -- > member

 

 

Add two members, one is the jenkins user, which is used to pull the code. One is for me to submit code.

Note: my account has higher permissions and can be submitted directly to the master branch.

 

 

Submit the code downloaded by github to the project just created. The effects are as follows:

 

 

3, harbor create users and projects

Create jenkins user

Log in to the harbor background and click create user

 

 

Note that each item should be filled in. In particular, passwords must meet the requirements of password complexity.

 

Create project

Click new item

 

 

Enter the name, java. This is a private project.

 

  Click java

 

 

Click member -- > User

 

 

Enter jenkins and you will be prompted. Select the developer for the role.

 

4, jenkins modify default user

Insufficient permissions may occur when using jenkins to execute shell commands.

View jenkins default users

cat /etc/default/jenkins

The contents are as follows:

# pulled in from the init script; makes things easier.
NAME=jenkins

# arguments to pass to java

# Allow graphs etc. to work even when an X server is present
JAVA_ARGS="-Djava.awt.headless=true"

#JAVA_ARGS="-Xmx256m"

# make jenkins listen on IPv4 address
#JAVA_ARGS="-Djava.net.preferIPv4Stack=true"

PIDFILE=/var/run/$NAME/$NAME.pid

# user and group to be invoked as (default to jenkins)
JENKINS_USER=$NAME
JENKINS_GROUP=$NAME
...

jenkins can be found_ User and jenkins_ The value of the group variable is jenkins

 

Modify the default user to root

Change the following two variables to root

JENKINS_USER=root
JENKINS_GROUP=root

 

Restart jenkins service

service jenkins restart

 

5, jenkins and k8s master are ssh free

Why do you want to ssh with k8s master? Because jenkins needs to log in to k8s master to do some pod operations.

Generate secret key

Log in to the jenkins server and generate the secret key

ssh-keygen -t rsa -P "" -f ~/.ssh/id_rsa

 

copy key

ssh-copy-id 192.168.10.130

 

Test root password free

ssh 192.168.10.130

If no password is prompted, it means success!

 

6, jenkins logs in to harbor

Modify the docker configuration file and add a harbor Address

vim /etc/docker/daemon.json

The contents are as follows:

{"insecure-registries": ["192.168.10.122"]}

 

Reload docker configuration

/etc/init.d/docker reload

 

Test login

root@ubuntu:~/docker_dir# docker login 192.168.10.122 -u jenkins -p Jenkins@1234
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded

 

If present

Error response from daemon: Get https://192.168.10.122/v2/: dial tcp 192.168.10.122:443: connect: connection refused

express  / Incorrect modification of etc/docker/daemon.json file

 

7, jenkins creates maven project

New task

Log in to jenkins background and click   New Item

Enter a name and choose to build a maven project. Note: the task name should preferably be the project name, because the following deploy_ The docker.sh script will call this variable.

 

 

 

 

Set up parametric build

Discard the old build and keep it for 2 days

 

 

Set source code

Enter the warehouse download address and select user root

 

 

Set maven build command

Enter the command: clean package

 

The mvn clean package successively executes seven phases: clean, resources, compile, testResources, testCompile, test, jar (packaging).

The package command completes the functions of project compilation, unit testing and packaging, but does not deploy the typed executable jar package (war package or other forms of package) to the local maven warehouse and the remote maven private server warehouse

 

The jar package is packaged into a docker image and pushed

Execute shell commands

When build is complete, choose to execute the shell command

 

 

Post the following content

#!/bin/bash

jarName=spring-boot-demo-0.0.1-SNAPSHOT.jar
jarFolder=$JOB_NAME
harborPro=java
projectName=$JOB_NAME

docker_path=${WORKSPACE}

cp ${WORKSPACE}/target/${jarName} ${docker_path}
bash -x /root/docker_dir/deploy_docker.sh ${harborPro} ${projectName} ${docker_path} ${jarName}

 

Content explanation:

Jarname the file name of the jar package, whatever the name is

jarFolder jar package folder

harborPro   For the project in harbor, in the above steps, create a java private project.

projectName project name

docker_ Working directory of the path docker command

 

Finally, a shell script is executed. Let's talk about it!

 

The effects are as follows:

 

 

 

Define shell script

Log in to the jenkins server and create a directory

mkdir /root/docker_dir

 

Edit deploy_docker.sh script

cd /root/docker_dir/
vim deploy_docker.sh

The contents are as follows:

#!/bin/bash
# maven $workspace $jarname
# ${harborPro} ${projectName} ${docker_path} ${jarName}

set -e
harbor_project=$1
projectName=$2
docker_path=$3
appName=$4

# harbor authentication user
user_name=jenkins
password=Jenkins@1234

# harbor Address and tag
tag=$(date +%s)
harbor_server=192.168.10.122
server_path=${harbor_server}
taget_image=${projectName}:${tag}
#${BUILD_NUMBER}
echo ${taget_image}

# Log in to docker
cd ${docker_path}
sudo docker login ${harbor_server} -u ${user_name} -p ${password}

# Generate an image and push it to harbor, and finally delete the local image
sudo docker build --build-arg app=${appName} -t ${taget_image} .
sudo docker tag ${taget_image} ${server_path}/${harbor_project}/${projectName}
echo "The name of image is ${server_path}/${harbor_project}/${projectName}"
sudo docker push ${server_path}/${harbor_project}/${projectName}:latest
sudo docker rmi -f $(docker images|grep ${projectName}|grep ${tag}|awk '{print $3}'|head -n 1)

 

Add execution permission

chmod 755 /root/docker_dir/deploy_docker.sh

 

yaml files are copied to k8s master and applied

k8s master operation

Log in to k8s master and create a directory

mkdir kube-conf

 

Delete kube.yaml in gitlab springbootdemo project and add two yaml files.

ph-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: ph-service
spec:
  type: NodePort
  ports:
  - name: ph
    port: 8080
    nodePort: 31002
    targetPort: 8080
    protocol: TCP
  selector:
    app: ph

 

ph-rc.yaml

apiVersion: v1
kind: ReplicationController
metadata:
  name: ph-rc
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: ph
    spec:
      imagePullSecrets:
        - name: harborsecret
      containers:
        - name: ph
          image: 192.168.10.122/java/ph:latest
          imagePullPolicy: Always
          ports:
          - containerPort: 8080
          env:
            - name: key
              value: "value"

 

Submit 2 yaml files to gitlab

 

k8s generate imagePullSecrets, please refer to the following link

https://www.cnblogs.com/xiao987334176/p/11434326.html

Note: specify jenkins when logging in with docker

docker login 192.168.10.122 -u jenkins -p Jenkins@1234

 

jenkins operation

Log in to the jenkins server and add a restart application script

cd /root/docker_dir
vim reboot_app.sh

The contents are as follows:

#!/bin/bash
MASTER="192.168.10.130"
CONF_DIR="/root/kube-conf"
ProJ=${JOB_NAME}

scp ${WORKSPACE}/*.yaml ${MASTER}:${CONF_DIR}
# Pod running process
RUN=$(ssh $MASTER kubectl get po|grep -w ${ProJ}|wc -l)
if [ $RUN -eq 0 ];then
        ssh ${MASTER} kubectl apply -f ${CONF_DIR}/${ProJ}-rc.yaml
        ssh ${MASTER} kubectl apply -f ${CONF_DIR}/${ProJ}-service.yaml
else
        # Delete pod
        podname=$(ssh ${MASTER} kubectl get po|grep -w ${ProJ}|awk '{print $1}')
        ssh ${MASTER} kubectl delete po $podname --grace-period=0 --force
        ssh ${MASTER} kubectl get po|grep ${ProJ}
fi

 

Add execution permission

chmod 755 /root/docker_dir/reboot_app.sh

 

Add execute shell command

 

 

  Post the following content

#!/bin/bash

bash -x /root/docker_dir/reboot_app.sh

 

The effects are as follows:

 

 

 

Build applications manually

Click Build Now

 

Click #1

 

Click console output

 

It will download some components by itself and need to wait for some time.

 

The spring icon appears, indicating that the application starts to build.

 

An error occurred at the end

 

Why? View the Dockerfile file of spring bootdemo and the first line

FROM registry-scu.cloudtogo.cn/ubuntu:jdk

Need to   registry-scu.cloudtogo.cn added to  / In / etc/docker/daemon.json

 

Log in to jenkins server and modify  / etc/docker/daemon.json

vim /etc/docker/daemon.json

The contents are as follows:

{"insecure-registries": ["192.168.10.122","registry-scu.cloudtogo.cn"]}

 

Reload docker

/etc/init.d/docker reload

 

Manually build it again and click Build Now

View the build process

The following prompt appears indicating that the image is being downloaded   registry-scu.cloudtogo.cn/ubuntu:jdk  

 

Finally, prompt for execution completion

 

 

View harbor warehouse

You will find an additional image, which was submitted when you just built it.

 

8, Access k8s applications

View Pod status

root@k8s-master:~# kubectl get pods -o wide
NAME                                READY   STATUS             RESTARTS   AGE     IP              NODE        NOMINATED NODE   READINESS GATES
ph-rc-nj7j2   0/1     ImagePullBackOff   0          5m19s   192.168.36.68   k8s-node1   <none>           <none>

 

It is found that pod is running on k8s-node1 this server and the status is ImagePullBackOff

 

View pod details

kubectl describe po maven-deployment-7bd87867f8-6tlgp

Output:

Events:
  Type     Reason     Age                     From                Message
  ----     ------     ----                    ----                -------
  Normal   Scheduled  7m15s                   default-scheduler   Successfully assigned default/maven-deployment-7bd87867f8-6tlgp to k8s-node1
  Normal   Pulling    5m49s (x4 over 7m14s)   kubelet, k8s-node1  Pulling image "192.168.10.122/maven:latest"
  Warning  Failed     5m49s (x4 over 7m14s)   kubelet, k8s-node1  Failed to pull image "192.168.10.122/maven:latest": rpc error: code = Unknown desc = Error response from daemon: Get https://192.168.10.122/v2/: dial tcp 192.168.10.122:443: connect: connection refused
  Warning  Failed     5m49s (x4 over 7m14s)   kubelet, k8s-node1  Error: ErrImagePull
  Normal   BackOff    5m36s (x6 over 7m13s)   kubelet, k8s-node1  Back-off pulling image "192.168.10.122/maven:latest"
  Warning  Failed     2m11s (x20 over 7m13s)  kubelet, k8s-node1  Error: ImagePullBackOff

 

appear  

Get https://192.168.10.122/v2/: dial tcp 192.168.10.122:443: connect: connection refused

Description, k8s-node1   The / etc/docker/daemon.json of this server has not been changed

 

Log in to k8s-node1   This server, modify the file

vim /etc/docker/daemon.json

The contents are as follows:

{"insecure-registries": ["192.168.10.122"]}

 

Reload docker

/etc/init.d/docker reload

 

Reload pod

Log in to k8s master server, delete pod and reapply

root@k8s-master:~# cd /root/kube-conf/
root@k8s-master:~/kube-conf# kubectl delete -f ph-rc.yaml
service "maven-service" deleted
deployment.extensions "maven-deployment" deleted
root@k8s-master:~/kube-conf# kubectl apply -f ph-rc.yaml
service/maven-service created
deployment.extensions/maven-deployment created

 

View status again

root@k8s-master:~/kube-conf# kubectl get pods -o wide
NAME                                READY   STATUS    RESTARTS   AGE   IP              NODE        NOMINATED NODE   READINESS GATES
ph-rc-7bd7475544-q9b6j   1/1     Running   0          27s   192.168.36.77   k8s-node1   <none>           <none>

Found in Running status

 

Access the application page

View svc exposed ports

root@k8s-master:~/kube-conf# kubectl get svc|grep maven
ph-service   NodePort    10.98.152.31   <none>        8080:31002/TCP   4m

It can be found that the exposed port is 31002

 

Access pages using Google

http://192.168.10.130:31002/

 

The effects are as follows:

 

ip using k8s-node1

http://192.168.10.131:31002/

The effect is the same as above!

 

9, Update application code

Open the local project springbootdemo and modify the file IndexController.java

The path is as follows:

springbootdemo\src\main\java\com\example\demo\web\controller

 

Modify the following 2 lines and add 1 to the number

jsonObject.put("welcome2", "2");
jsonObject.put("welcome7", "7");

 

Resubmit to gitlab, and then build it again. Click Build Now

After the build is successful, refresh the page

 

  Found content has been updated!

 

Text reference link:

https://linux265.com/news/3465.html

https://www.cnblogs.com/aguncn/p/9789320.html

https://juejin.im/post/5c07b1126fb9a049e82b4cfe

 

Classification: python operation and maintenance development

label: Jenkins