Experimental topology
Topological specification
The experimental environment is a 16G memory host. The above three virtual machines are run using vmware, and the running system is ubuntu-19.04. The three virtual machines are connected in host-only mode.
- spine, leaf1 and leaf2 are all ubuntu-19.04. The FRR program is running on them.
- host1, host2, host3, host4 are the network namespaces.
- underlay network adopts a two-tier model (limited to experimental conditions)
In the experiment, host1 and host3 communicate in one vrf, using l3vni 100, host2 and host4 communicate in another vrf, using l3vni 200. In order to make the host in evpn-vrf and evpn-vrf1 communicate with each other, we need to use the routing leak function of bgp to complete the vpc-peering function.
spine configuration
bgp evpn configuration
router bgp 7677 bgp router-id 192.168.59.130 bgp bestpath as-path multipath-relax neighbor fabric peer-group neighbor fabric remote-as external neighbor 192.168.59.128 peer-group fabric neighbor 192.168.59.129 peer-group fabric ! address-family l2vpn evpn neighbor fabric activate exit-address-family !
leaf1 configuration
Interface configuration
ubuntu@ubuntu:~$ cat work/frr-frr-7.1/vpc-peering.sh #!/bin/bash #Open and forward sudo sysctl -w net.ipv4.ip_forward=1 sudo sysctl -p #Add host1 sudo ip netns add host1 sudo ip link add veth1 type veth peer name eth0 netns host1 sudo ip netns exec host1 ip link set lo up sudo ip netns exec host1 ip link set eth0 up sudo ip netns exec host1 ip addr add 1.1.1.1/24 dev eth0 sudo ip netns exec host1 ip route add default via 1.1.1.254 dev eth0 sudo ip link add br10 type bridge sudo ip link set br10 up sudo ip link set veth1 up sudo ip link set veth1 master br10 sudo ip addr add 1.1.1.254/24 dev br10 #Add host2 sudo ip netns add host2 sudo ip link add veth2 type veth peer name eth0 netns host2 sudo ip netns exec host2 ip link set lo up sudo ip netns exec host2 ip link set eth0 up sudo ip netns exec host2 ip addr add 2.2.2.2/24 dev eth0 sudo ip netns exec host2 ip route add default via 2.2.2.254 dev eth0 sudo ip link add br20 type bridge sudo ip link set br20 up sudo ip link set veth2 up sudo ip link set veth2 master br20 sudo ip addr add 2.2.2.254/24 dev br20 #Add vni 100 as l3vni sudo ip link add br100 type bridge sudo ip link add vxlan100 type vxlan id 100 local 192.168.59.128 dstport 4789 nolearning sudo ip link set br100 up sudo ip link set vxlan100 up sudo ip link set vxlan100 master br100 #Suo IP addr add 5.5.5.254/24 dev BR100 Keep in mind that as the svi interface of l3vni, IP cannot be configured, otherwise the received type-5 routing will not be installed. sudo ip link set dev br100 address 00:00:01:02:03:04 #This is routing mac #Add vni 200 as evpn-vrf1 l3vni sudo ip link add br200 type bridge sudo ip link add vxlan200 type vxlan id 200 local 192.168.59.128 dstport 4789 nolearning sudo ip link set br200 up sudo ip link set vxlan200 up sudo ip link set vxlan200 master br200 #Suo IP addr add 5.5.5.254/24 dev BR100 Keep in mind that as the svi interface of l3vni, IP cannot be configured, otherwise the received type-5 routing will not be installed. sudo ip link set dev br200 address 00:00:01:02:03:05 #This is routing mac #Add vrf sudo ip link add evpn-vrf type vrf table 100 sudo ip link set evpn-vrf up sudo ip link set br100 master evpn-vrf sudo ip link set br10 master evpn-vrf sudo ip link add evpn-vrf1 type vrf table 200 sudo ip link set evpn-vrf1 up sudo ip link set br200 master evpn-vrf1 sudo ip link set br20 master evpn-vrf1 #close rp filter sudo sysctl -w net.ipv4.conf.all.rp_filter=0 sudo sysctl -w net.ipv4.conf.default.rp_filter=0 sudo sysctl -w net.ipv4.conf.br100.rp_filter=0 sudo sysctl -w net.ipv4.conf.br200.rp_filter=0 #startup bgp sudo zebra/zebra -d sudo staticd/staticd -d sudo bgpd/bgpd -d sudo vtysh/vtysh
bgp evpn configuration
vrf evpn-vrf vni 100 exit-vrf ! vrf evpn-vrf1 vni 200 exit-vrf ! router bgp 7675 bgp router-id 192.168.59.128 bgp bestpath as-path multipath-relax neighbor fabric peer-group neighbor fabric remote-as external neighbor 192.168.59.130 peer-group fabric ! address-family l2vpn evpn neighbor fabric activate advertise-all-vni exit-address-family ! router bgp 7675 vrf evpn-vrf1 ! address-family ipv4 unicast network 2.2.2.0/24 import vrf evpn-vrf exit-address-family ! address-family l2vpn evpn advertise ipv4 unicast exit-address-family ! router bgp 7675 vrf evpn-vrf ! address-family ipv4 unicast network 1.1.1.0/24 network 5.1.1.0/24 import vrf evpn-vrf1 exit-address-family ! address-family l2vpn evpn advertise ipv4 unicast exit-address-family ! line vty ! end ubuntu#
Note:
import vrf evpn-vrf1
This instruction represents the introduction of routing from evpn-vrf1.
sudo sysctl -w net.ipv4.conf.all.rp_filter=0 sudo sysctl -w net.ipv4.conf.default.rp_filter=0 sudo sysctl -w net.ipv4.conf.br100.rp_filter=0 sudo sysctl -w net.ipv4.conf.br200.rp_filter=0
This instruction is used to turn off the reverse path checking of linux kernel. Because br100 and br200 do not configure ip, they belong to unnumber interface and need to prohibit the reverse path checking.
leaf2 configuration
Interface configuration
#!/bin/bash #Open and forward sudo sysctl -w net.ipv4.ip_forward=1 sudo sysctl -p #Add host3 sudo ip netns add host3 sudo ip link add veth3 type veth peer name eth0 netns host3 sudo ip netns exec host3 ip link set lo up sudo ip netns exec host3 ip link set eth0 up sudo ip netns exec host3 ip addr add 3.3.3.3/24 dev eth0 sudo ip netns exec host3 ip route add default via 3.3.3.254 dev eth0 # Add bridges, add veth3 to bridges sudo ip link add br30 type bridge sudo ip link set br30 up sudo ip link set veth3 up sudo ip link set veth3 master br30 sudo ip addr add 3.3.3.254/24 dev br30 #Add host4 sudo ip netns add host4 sudo ip link add veth4 type veth peer name eth0 netns host4 sudo ip netns exec host4 ip link set lo up sudo ip netns exec host4 ip link set eth0 up sudo ip netns exec host4 ip addr add 4.4.4.4/24 dev eth0 sudo ip netns exec host4 ip route add default via 4.4.4.254 dev eth0 sudo ip link add br40 type bridge sudo ip link set br40 up sudo ip link set veth4 up sudo ip link set veth4 master br40 sudo ip addr add 4.4.4.254/24 dev br40 #Add vni 100 as ievpn-vrf l3vni sudo ip link add br100 type bridge sudo ip link add vxlan100 type vxlan id 100 local 192.168.59.129 dstport 4789 nolearning sudo ip link set br100 up sudo ip link set vxlan100 up sudo ip link set vxlan100 master br100 #Suo IP addr add 5.5.5.253/24 dev BR100 Keep in mind that you must not add IP addresses, otherwise type5 routing will not work properly under the kernel sudo ip link set dev br100 address 00:00:01:02:03:06 #This is rmac, routing mac sudo ip link add br200 type bridge sudo ip link add vxlan200 type vxlan id 200 local 192.168.59.129 dstport 4789 nolearning sudo ip link set br200 up sudo ip link set vxlan200 up sudo ip link set vxlan200 master br200 #Suo IP addr add 5.5.5.253/24 dev BR100 Keep in mind that you must not add IP addresses, otherwise type5 routing will not work properly under the kernel sudo ip link set dev br200 address 00:00:01:02:03:07 #This is rmac, routing mac #Add vrf sudo ip link add evpn-vrf type vrf table 100 sudo ip link set evpn-vrf up sudo ip link set br100 master evpn-vrf sudo ip link set br30 master evpn-vrf sudo ip link add evpn-vrf1 type vrf table 200 sudo ip link set evpn-vrf1 up sudo ip link set br200 master evpn-vrf1 sudo ip link set br40 master evpn-vrf1 #Add static routing across vrf to get through host3 and host4 sudo ip route add 4.4.4.4 dev br40 vrf evpn-vrf sudo ip route add 3.3.3.3 dev br40 vrf evpn-vrf1 #Access external network #Add vtep interface connecting evpn-vrf to default VRF sudo ip link add ext1 type veth peer name ext sudo ip link set ext1 up sudo ip link set ext up #Add vtep interface connecting evpn-vrf1 to default VRF sudo ip link add ext2 type veth peer name ext3 sudo ip link set ext2 up sudo ip link set ext3 up #Where ext1 is in evpn-vrf and ext is in default sudo ip link set ext1 master evpn-vrf #Use segment 5.5.5.0/24 as relay segment sudo ip addr add 5.5.5.253/24 dev ext1 sudo ip addr add 5.5.5.254/24 dev ext sudo ip addr add 5.5.6.253/24 dev ext2 sudo ip addr add 5.5.6.254/24 dev ext3 #Add default routing in evpn to allow traffic to access the public network by default sudo ip route add default via 5.5.5.254 dev ext1 vrf evpn-vrf sudo ip route add default via 5.5.6.254 dev ext2 vrf evpn-vrf1 #Configure snat to change private network traffic to smac sudo nft add table nat sudo nft add chain nat prerouting { type nat hook prerouting priority 0 \; } sudo nft add chain nat postrouting { type nat hook postrouting priority 100 \; } sudo nft add rule nat postrouting oifname ext1 counter masquerade sudo nft add rule nat postrouting oifname ens33 counter masquerade sudo nft add rule nat postrouting oifname ext2 counter masquerade sudo sysctl -w net.ipv4.conf.all.rp_filter=0 sudo sysctl -w net.ipv4.conf.default.rp_filter=0 sudo sysctl -w net.ipv4.conf.br100.rp_filter=0 sudo sysctl -w net.ipv4.conf.br200.rp_filter=0 sudo chmod 777 /var/run/ sudo zebra -d sudo staticd -d sudo bgpd -d sudo vtysh
bgp evpn configuration
vrf evpn-vrf vni 100 exit-vrf ! vrf evpn-vrf1 vni 200 exit-vrf ! router bgp 7676 bgp router-id 192.168.59.129 bgp bestpath as-path multipath-relax neighbor fabric peer-group neighbor fabric remote-as external neighbor 192.168.59.130 peer-group fabric ! address-family l2vpn evpn neighbor fabric activate advertise-all-vni exit-address-family ! router bgp 7676 vrf evpn-vrf1 ! address-family ipv4 unicast network 4.4.4.0/24 import vrf evpn-vrf exit-address-family ! address-family l2vpn evpn advertise ipv4 unicast exit-address-family ! router bgp 7676 vrf evpn-vrf ! address-family ipv4 unicast network 0.0.0.0/0 network 3.3.3.0/24 import vrf evpn-vrf1 exit-address-family ! address-family l2vpn evpn advertise ipv4 unicast exit-address-family ! line vty ! end
View bgp information
leaf1
- View routing information
ubuntu# show bgp l2vpn evpn BGP table version is 6, local router ID is 192.168.59.128 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal Origin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight Path Route Distinguisher: ip 1.1.1.254:3 *> [5]:[0]:[24]:[1.1.1.0] 192.168.59.128 0 32768 i *> [5]:[0]:[24]:[4.4.4.0] 192.168.59.128 0 7677 7676 i *> [5]:[0]:[24]:[5.1.1.0] 192.168.59.128 0 32768 i Route Distinguisher: ip 2.2.2.254:2 *> [5]:[0]:[0]:[0.0.0.0] 192.168.59.128 0 7677 7676 i *> [5]:[0]:[24]:[2.2.2.0] 192.168.59.128 0 32768 i *> [5]:[0]:[24]:[3.3.3.0] 192.168.59.128 0 7677 7676 i Route Distinguisher: ip 5.5.5.253:3 *> [5]:[0]:[0]:[0.0.0.0] 192.168.59.129 0 7677 7676 i *> [5]:[0]:[24]:[3.3.3.0] 192.168.59.129 0 7677 7676 i Route Distinguisher: ip 5.5.6.253:2 *> [5]:[0]:[24]:[4.4.4.0] 192.168.59.129 0 7677 7676 i Displayed 9 out of 9 total prefixes ubuntu#
leaf2
- View routing information
ubuntu# show bgp l2vpn evpn BGP table version is 6, local router ID is 192.168.59.129 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal Origin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight Path Route Distinguisher: ip 1.1.1.254:3 *> [5]:[0]:[24]:[1.1.1.0] 192.168.59.128 0 7677 7675 i *> [5]:[0]:[24]:[5.1.1.0] 192.168.59.128 0 7677 7675 i Route Distinguisher: ip 2.2.2.254:2 *> [5]:[0]:[24]:[2.2.2.0] 192.168.59.128 0 7677 7675 i Route Distinguisher: ip 5.5.5.253:3 *> [5]:[0]:[0]:[0.0.0.0] 192.168.59.129 0 32768 i *> [5]:[0]:[24]:[2.2.2.0] 192.168.59.129 0 7677 7675 i *> [5]:[0]:[24]:[3.3.3.0] 192.168.59.129 0 32768 i Route Distinguisher: ip 5.5.6.253:2 *> [5]:[0]:[24]:[1.1.1.0] 192.168.59.129 0 7677 7675 i *> [5]:[0]:[24]:[4.4.4.0] 192.168.59.129 0 32768 i *> [5]:[0]:[24]:[5.1.1.0] 192.168.59.129 0 7677 7675 i Displayed 9 out of 9 total prefixes ubuntu#
summary
The key points of this experiment are:
- bgp route leak.
- linux kernel routing supports that the next hop interface is not in the same vrf.
- Reverse path checking needs to be turned off when no ip interface is configured to forward messages.