- xxong personal verification

At present, flannel, weave, Pipework, Open vSwitch and so on are the main solutions in the market.

Open vSwitch is relatively simple, mature and powerful, so it is very suitable as a tool to solve the interconnection and interoperability of docker underlying network.
The topology is as follows:
Emphasis I have two host systems here: Master: centos7.2 node: centos6.10
Master 10.16.170.10 docker 192.168.18.0/24
Node 172.16.17.2 docker 192.168.42.0/24
In view of the limitations of centos6 system, we choose the following version of openvswitch-2.3.0 for the problems encountered in installing centos6
The specific implementation steps are as follows:

1. install docker, bridge-utils and openvswitch
[root@master ~]# yum install docker bridge-utils -y 
[root@master ~]# yum install wget openssl-devel -y 
[root@master ~]# yum groupinstall "Development Tools"
[root@master ~]# adduser ovswitch
[root@master ~]# su - ovswitch
[ovswitch@master ~]$ wget http://openvswitch.org/releases/openvswitch-2.3.0.tar.gz
[ovswitch@master ~]$ tar -zxvpf openvswitch-2.3.0.tar.gz
[ovswitch@master ~]$ mkdir -p ~/rpmbuild/SOURCES
[ovswitch@master ~]$ sed 's/openvswitch-kmod, //g' openvswitch-2.3.0/rhel/openvswitch.spec > openvswitch-2.3.0/rhel/openvswitch_no_kmod.spec
[ovswitch@master ~]$ cp openvswitch-2.3.0.tar.gz rpmbuild/SOURCES/

[ovswitch@master ~]$ rpmbuild -bb --without check ~/openvswitch-2.3.0/rhel/openvswitch_no_kmod.spec  
  //If an error is reported here, No build ID found in.... You need to add% define__debug_install_post to the openvswitch_no_kmod.spec file.\
   %{_rpmconfigdir}/find-debuginfo.sh %{?_find_debuginfo_opts} "%{_builddir}/%{?buildsubdir}"\
%{nil}
[ovswitch@master ~]$ exit
[root@master ~]# yum localinstall /home/ovswitch/rpmbuild/RPMS/x86_64/openvswitch-2.3.0-1.x86_64.rpm -y      
//Note: The corresponding location is only this rpm package, remember: if you install version 1.9.3, there are two packages here, but the online documents basically show only one rpm package, that is to say, rpmbuild an openvswitch.spec is needed in this step, otherwise, the lack of dependency will be reported in this step; 
//Also, here may be a hint that you need glibc 2.14, which means that the glibc version is too low and needs to be upgraded. If I download glibc directly from rpm for upgrade, here's a link: https://www.cnblogs.com/dpf-learning/p/8763696.html

[root@master ~]# mkdir /etc/openvswitch
[root@master ~]# setenforce 0

2. docker master The end configuration is as follows
[root@master ~]# systemctl start openvswitch.service  && systemctl enable openvswitch.service
[root@master ~]# ovs-vsctl add-br br0
[root@master ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq state UP qlen 1000
    link/ether 00:0c:29:97:92:e8 brd ff:ff:ff:ff:ff:ff
    inet 10.16.170.10/24 brd 255.255.255.0 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:fe97:92e8/64 scope link
       valid_lft forever preferred_lft forever
3: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
    link/ether 02:42:45:b7:c2:fd brd ff:ff:ff:ff:ff:ff
    inet 192.168.18.1/24 scope global docker0
       valid_lft forever preferred_lft forever
    inet6 fe80::42:45ff:feb7:c2fd/64 scope link
       valid_lft forever preferred_lft forever
5: vethcff8026@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP
    link/ether 32:4a:f5:b7:33:f7 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::304a:f5ff:feb7:33f7/64 scope link
       valid_lft forever preferred_lft forever
6: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether 8a:ac:8e:a1:68:2b brd ff:ff:ff:ff:ff:ff
7: br0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether 82:ae:47:8e:30:4d brd ff:ff:ff:ff:ff:ff
[root@master ~]# ovs-vsctl add-port br0 gre0 -- set interface gre0 type=gre option:remote_ip=172.16.17.2
[root@master ~]# ovs-vsctl show
4fe9a5b3-46ec-432c-a990-bb8e8fee96fe
    Bridge "br0"
        Port "gre0"
            Interface "gre0"
                type: gre
                options: {remote_ip="172.16.17.2"}
        Port "br0"
            Interface "br0"
                type: internal
    ovs_version: "2.3.2"
[root@master ~]# brctl addif docker0 br0
[root@master ~]# brctl show
bridge name bridge id       STP enabled interfaces
docker0     8000.024245b7c2fd   no      br0
[root@master ~]# ip link set dev br0 up
[root@master ~]# ip link set dev docker0 up
[root@master ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq state UP qlen 1000
    link/ether 00:0c:29:97:92:e8 brd ff:ff:ff:ff:ff:ff
    inet 10.16.170.10/24 brd 255.255.255.0 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:fe97:92e8/64 scope link
       valid_lft forever preferred_lft forever
3: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
    link/ether 02:42:45:b7:c2:fd brd ff:ff:ff:ff:ff:ff
    inet 192.168.18.1/24 scope global docker0
       valid_lft forever preferred_lft forever
    inet6 fe80::42:45ff:feb7:c2fd/64 scope link
       valid_lft forever preferred_lft forever
5: vethcff8026@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP
    link/ether 32:4a:f5:b7:33:f7 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::304a:f5ff:feb7:33f7/64 scope link
       valid_lft forever preferred_lft forever
6: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether 8a:ac:8e:a1:68:2b brd ff:ff:ff:ff:ff:ff
7: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UNKNOWN qlen 1000
    link/ether 82:ae:47:8e:30:4d brd ff:ff:ff:ff:ff:ff
    inet6 fe80::80ae:47ff:fe8e:304d/64 scope link
       valid_lft forever preferred_lft forever
8: gre0@NONE: <NOARP> mtu 1476 qdisc noop state DOWN qlen 1000
    link/gre 0.0.0.0 brd 0.0.0.0
9: gretap0@NONE: <BROADCAST,MULTICAST> mtu 1462 qdisc noop state DOWN qlen 1000
    link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
10: gre_sys@NONE: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65490 qdisc fq master ovs-system state UNKNOWN qlen 1000
    link/ether aa:3a:19:78:48:89 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::a83a:19ff:fe78:4889/64 scope link
       valid_lft forever preferred_lft forever
[root@master ~]#  ip route add 192.168.42.0/24 dev docker0
[root@master ~]# docker run -itd --name c1 docker.io/centos '/bin/bash'
WARNING: IPv4 forwarding is disabled. Networking will not work.
a326fb2eae1ecf1c0b1a26b4b947f20eb44864fc5232e253b582c8c7bb50522a
[root@master ~]# vim /etc/sysctl.conf
[root@master ~]# sysctl -p
net.core.default_qdisc = fq
net.ipv4.tcp_congestion_control = bbr
net.ipv4.ip_forward = 1
[root@master ~]#
(3) docker node The end configuration is as follows
[root@node1 ~]# service openvswitch start  && chkconfig openvswitch on
[root@node1 ~]# brctl show
bridge name bridge id       STP enabled interfaces
docker0     8000.02429f5f947d   no
[root@node1 ~]#  ovs-vsctl add-br br0
[root@node1 ~]#  ovs-vsctl add-port br0 gre0 -- set interface gre0 type=gre option:remote_ip=10.16.170.10
//If the ip modification is wrong, the bridge needs to be deleted: ovs-vsctl del-br BR0

[root@node1 ~]# brctl addif docker0 br0
[root@node1 ~]# brctl show
bridge name bridge id       STP enabled interfaces
docker0     8000.02429f5f947d   no      br0
[root@node1 ~]#
[root@node1 ~]# ip link set dev br0 up
[root@node1 ~]# ip link set dev docker0 up
[root@node1 ~]# ip route add 192.168.18.0/24 dev docker0
[root@node1 ~]# brctl show
bridge name bridge id       STP enabled interfaces
docker0     8000.02429f5f947d   no      br0
[root@node1 ~]# ovs-vsctl show
f0be12f7-1aa7-4b93-8d4f-5511b56efec7
    Bridge "br0"
        Port "gre0"
            Interface "gre0"
                type: gre
                options: {remote_ip="10.16.170.10"}
        Port "br0"
            Interface "br0"
                type: internal
    ovs_version: "2.3.2"
[root@node1 ~]# echo 'net.ipv4.ip_forward = 1' >> /etc/sysctl.conf
[root@node1 ~]# sysctl -p
[root@node1 ~]# docker run -itd --name c2 docker.io/centos '/bin/bash'
c9414017f86e6c362b9481ceffc658275b3557cf0991e84853066d4eccb37b0f
[root@node1 ~]#
(4) test
[root@node1 ~]# docker attach c941
[root@c9414017f86e /]# ping -c1 192.168.18.1
PING 192.168.18.1 (192.168.18.1) 56(84) bytes of data.
64 bytes from 192.168.18.1: icmp_seq=1 ttl=64 time=5.19 ms

--- 192.168.18.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 5.194/5.194/5.194/0.000 ms
[root@c9414017f86e /]# ping -c1 192.168.18.2
PING 192.168.18.2 (192.168.18.2) 56(84) bytes of data.
64 bytes from 192.168.1.2: icmp_seq=1 ttl=63 time=2.74 ms

--- 192.168.18.2 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 2.742/2.742/2.742/0.000 ms
[root@master ~]#Docker exec-it 34fb bash chooses a container to enter 
[root@34fb4017f86 /]# ping -c1 192.168.42.1
PING 192.168.42.1 (192.168.42.1) 56(84) bytes of data.
64 bytes from 192.168.42.1: icmp_seq=1 ttl=64 time=0.051 ms

--- 192.168.42.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.051/0.051/0.051/0.000 ms

ok fix