Network Topology
Topological description
The experimental environment is a 16G memory host. The above three virtual machines are run using vmware, and the running system is ubuntu-19.04. The three virtual machines are connected in host-only mode.
- spine, leaf1 and leaf2 are all ubuntu-19.04. The FRR program is running on them.
- host1, host2, host3, host4 are the network namespaces.
- underlay network adopts a two-tier model (limited to experimental conditions)
The whole experiment is a scaled down data center spine-leaf model. leaf2 also needs to act as a border gateway, sending traffic to the public network using default routes, and as firewall (nat only here).
leaf as a functional refinement of border and vtep
Explanation of experimental functions
- The entire data center is a tenant, using vni:100 as the tenant's l3vni
- The tenant uses three subnets. 1.1.1.0/24 subnet has two virtual machines distributed under two vtep s, using 10 as l2vni. 2.2.2.0/24 and 3.3.3.0/24 have only one virtual machine. 5.5.5.0/24 is used as relay subnet to connect default-vrf and evpn-vrf.
- The whole experiment needs to realize the interconnection of all hosts in the tenant, and at the same time the host can access the public network. (temporarily unable to achieve virtual access to the public network, you need to apply for floating-ip before you can, after applying for public network IP, you can do 1:1 nat in default-vrf to achieve mutual access)
spine configuration
bgp evpn configuration
router bgp 7677 bgp router-id 192.168.59.130 bgp bestpath as-path multipath-relax neighbor fabric peer-group neighbor fabric remote-as external neighbor 192.168.59.128 peer-group fabric neighbor 192.168.59.129 peer-group fabric ! address-family l2vpn evpn neighbor fabric activate exit-address-family !
leaf1 configuration
Interface Configuration
#Open Forwarding sudo sysctl -w net.ipv4.ip_forward=1 sudo sysctl -p #Add host1 sudo ip netns add host1 sudo ip link add veth1 type veth peer name eth0 netns host1 sudo ip netns exec host1 ip link set lo up sudo ip netns exec host1 ip link set eth0 up sudo ip netns exec host1 ip addr add 1.1.1.1/24 dev eth0 sudo ip netns exec host1 ip route add default via 1.1.1.254 dev eth0 sudo ip link add br10 type bridge sudo ip link add vxlan10 type vxlan id 10 local 192.168.59.128 dstport 4789 nolearning sudo ip link set br10 up sudo ip link set veth1 up sudo ip link set vxlan10 up sudo ip link set veth1 master br10 sudo ip link set vxlan10 master br10 sudo ip addr add 1.1.1.254/24 dev br10 sudo ip link set dev br10 address 00:00:01:02:03:10 #Distributed two-tier gateway, mac needs consistency #Add host2 sudo ip netns add host2 sudo ip link add veth2 type veth peer name eth0 netns host2 sudo ip netns exec host2 ip link set lo up sudo ip netns exec host2 ip link set eth0 up sudo ip netns exec host2 ip addr add 2.2.2.2/24 dev eth0 sudo ip netns exec host2 ip route add default via 2.2.2.254 dev eth0 sudo ip link add br20 type bridge sudo ip link set br20 up sudo ip link set veth2 up sudo ip link set veth2 master br20 sudo ip addr add 2.2.2.254/24 dev br20 #Add vni 100 as l3vni sudo ip link add br100 type bridge sudo ip link add vxlan100 type vxlan id 100 local 192.168.59.128 dstport 4789 nolearning sudo ip link set br100 up sudo ip link set vxlan100 up sudo ip link set vxlan100 master br100 #Suo IP addr add 5.5.5.254/24 dev BR100 Keep in mind that as the svi interface of l3vni, IP cannot be configured, otherwise the received type-5 routing will not be installed. sudo ip link set dev br100 address 00:00:01:02:03:04 #This is routing mac #Add vrf sudo ip link add evpn-vrf type vrf table 100 sudo ip link set evpn-vrf up sudo ip link set br100 master evpn-vrf sudo ip link set br10 master evpn-vrf sudo ip link set br20 master evpn-vrf
bgp evpn configuration
vrf evpn-vrf vni 100 exit-vrf ! router bgp 7675 bgp router-id 192.168.59.128 bgp bestpath as-path multipath-relax neighbor fabric peer-group neighbor fabric remote-as external neighbor 192.168.59.130 peer-group fabric ! address-family l2vpn evpn neighbor fabric activate advertise-all-vni exit-address-family ! router bgp 7675 vrf evpn-vrf ! address-family ipv4 unicast network 2.2.2.0/24 exit-address-family ! address-family l2vpn evpn advertise ipv4 unicast exit-address-family !
Note:
vrf evpn-vrf vni 100 exit-vrf
This instruction indicates that an l3vni is specified
router bgp 7675 vrf evpn-vrf ! address-family l2vpn evpn advertise ipv4 unicast exit-address-family !
This instruction advertise ipv4 unicast denotes the announcement of RT-5 routing.
Keep in mind: you must not add IP addresses to the svi corresponding to l3vni, otherwise the type 5 routing will not work properly in the kernel.
leaf2 configuration
Interface Configuration
#Open Forwarding sudo sysctl -w net.ipv4.ip_forward=1 sudo sysctl -p #Add host3 sudo ip netns add host3 sudo ip link add veth3 type veth peer name eth0 netns host3 sudo ip netns exec host3 ip link set lo up sudo ip netns exec host3 ip link set eth0 up sudo ip netns exec host3 ip addr add 3.3.3.3/24 dev eth0 sudo ip netns exec host3 ip route add default via 3.3.3.254 dev eth0 # Add bridges, add veth3 to bridges sudo ip link add br30 type bridge sudo ip link set br30 up sudo ip link set veth3 up sudo ip link set veth3 master br30 sudo ip addr add 3.3.3.254/24 dev br30 #Add host4 sudo ip netns add host4 sudo ip link add veth4 type veth peer name eth0 netns host4 sudo ip netns exec host4 ip link set lo up sudo ip netns exec host4 ip link set eth0 up sudo ip netns exec host4 ip addr add 1.1.1.2/24 dev eth0 sudo ip netns exec host4 ip route add default via 1.1.1.254 dev eth0 sudo ip link add br40 type bridge sudo ip link add vxlan10 type vxlan id 10 local 192.168.59.129 dstport 4789 nolearning sudo ip link set vxlan10 up sudo ip link set vxlan10 master br40 sudo ip link set br40 up sudo ip link set veth4 up sudo ip link set veth4 master br40 sudo ip addr add 1.1.1.254/24 dev br40 sudo ip link set dev br40 address 00:00:01:02:03:10 #Distributed two-tier gateway, mac needs consistency #Add vni 100 as l3vni sudo ip link add br100 type bridge sudo ip link add vxlan100 type vxlan id 100 local 192.168.59.129 dstport 4789 nolearning sudo ip link set br100 up sudo ip link set vxlan100 up sudo ip link set vxlan100 master br100 #Suo IP addr add 5.5.5.253/24 dev BR100 Keep in mind that you must not add IP addresses, otherwise type5 routing will not work properly under the kernel sudo ip link set dev br100 address 00:00:01:02:03:05 #This is rmac, routing mac #Add vrf sudo ip link add evpn-vrf type vrf table 100 sudo ip link set evpn-vrf up sudo ip link set br100 master evpn-vrf sudo ip link set br30 master evpn-vrf sudo ip link set br40 master evpn-vrf #Access to Extranet #Add vtep interface connecting evpn-vrf to default VRF sudo ip link add ext1 type veth peer name ext sudo ip link set ext1 up sudo ip link set ext up #Where ext1 is in evpn-vrf and ext is in default sudo ip link set ext1 master evpn-vrf #Use segment 5.5.5.0/24 as relay segment sudo ip addr add 5.5.5.253/24 dev ext1 sudo ip addr add 5.5.5.254/24 dev ext #Add default routing in evpn and allow traffic to access the public network by default. This segment is shared by all tenants and allocated by administrators, so it can not conflict. sudo ip route add default via 5.5.5.254 dev ext1 table 100 #Configure snat to change private network traffic to smac sudo nft add table nat sudo nft add chain nat prerouting { type nat hook prerouting priority 0 \; } sudo nft add chain nat postrouting { type nat hook postrouting priority 100 \; } sudo nft add rule nat postrouting oifname ext1 counter masquerade sudo nft add rule nat postrouting oifname ens33 counter masquerade
bgp evpn configuration
vrf evpn-vrf vni 100 exit-vrf ! router bgp 7676 bgp router-id 192.168.59.129 bgp bestpath as-path multipath-relax neighbor fabric peer-group neighbor fabric remote-as external neighbor 192.168.59.130 peer-group fabric ! address-family l2vpn evpn neighbor fabric activate advertise-all-vni exit-address-family ! router bgp 7676 vrf evpn-vrf ! address-family ipv4 unicast network 3.3.3.0/24 network 0.0.0.0/0 exit-address-family ! address-family l2vpn evpn advertise ipv4 unicast exit-address-family !
View bgp information
leaf1
- View routing information
ubuntu# show bgp l2vpn evpn BGP table version is 7, local router ID is 192.168.59.128 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal Origin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight Path Route Distinguisher: ip 2.2.2.254:2 *> [5]:[0]:[24]:[2.2.2.0] 192.168.59.128 0 32768 i Route Distinguisher: ip 5.5.5.253:2 *> [5]:[0]:[0]:[0.0.0.0] 192.168.59.129 0 7677 7676 i *> [5]:[0]:[24]:[3.3.3.0] 192.168.59.129 0 7677 7676 i Route Distinguisher: ip 192.168.59.128:3 *> [2]:[0]:[48]:[46:48:a2:5e:e2:2f] 192.168.59.128 32768 i *> [2]:[0]:[48]:[46:48:a2:5e:e2:2f]:[32]:[1.1.1.1] 192.168.59.128 32768 i *> [3]:[0]:[32]:[192.168.59.128] 192.168.59.128 32768 i Route Distinguisher: ip 192.168.59.129:3 *> [3]:[0]:[32]:[192.168.59.129] 192.168.59.129 0 7677 7676 i Displayed 7 out of 7 total prefixes ubuntu#
leaf2
- View routing information
ubuntu# show bgp l2vpn evpn BGP table version is 9, local router ID is 192.168.59.129 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal Origin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight Path Route Distinguisher: ip 2.2.2.254:2 *> [5]:[0]:[24]:[2.2.2.0] 192.168.59.128 0 7677 7675 i Route Distinguisher: ip 5.5.5.253:2 *> [5]:[0]:[0]:[0.0.0.0] 192.168.59.129 0 32768 i *> [5]:[0]:[24]:[3.3.3.0] 192.168.59.129 0 32768 i Route Distinguisher: ip 192.168.59.128:3 *> [2]:[0]:[48]:[46:48:a2:5e:e2:2f] 192.168.59.128 0 7677 7675 i *> [2]:[0]:[48]:[46:48:a2:5e:e2:2f]:[32]:[1.1.1.1] 192.168.59.128 0 7677 7675 i *> [3]:[0]:[32]:[192.168.59.128] 192.168.59.128 0 7677 7675 i Route Distinguisher: ip 192.168.59.129:3 *> [3]:[0]:[32]:[192.168.59.129] 192.168.59.129 32768 i Displayed 7 out of 7 total prefixes ubuntu#