Implementing VPC-PEERING with Route Leakage

Keywords: Linux sudo network Ubuntu Mac

Experimental topology

Topological specification

The experimental environment is a 16G memory host. The above three virtual machines are run using vmware, and the running system is ubuntu-19.04. The three virtual machines are connected in host-only mode.

  • spine, leaf1 and leaf2 are all ubuntu-19.04. The FRR program is running on them.
  • host1, host2, host3, host4 are the network namespaces.
  • underlay network adopts a two-tier model (limited to experimental conditions)

In the experiment, host1 and host3 communicate in one vrf, using l3vni 100, host2 and host4 communicate in another vrf, using l3vni 200. In order to make the host in evpn-vrf and evpn-vrf1 communicate with each other, we need to use the routing leak function of bgp to complete the vpc-peering function.

spine configuration

bgp evpn configuration

router bgp 7677
 bgp router-id 192.168.59.130
 bgp bestpath as-path multipath-relax
 neighbor fabric peer-group
 neighbor fabric remote-as external
 neighbor 192.168.59.128 peer-group fabric
 neighbor 192.168.59.129 peer-group fabric
 !
 address-family l2vpn evpn
  neighbor fabric activate
 exit-address-family
!

leaf1 configuration

Interface configuration

ubuntu@ubuntu:~$ cat work/frr-frr-7.1/vpc-peering.sh    
#!/bin/bash

#Open and forward
sudo sysctl -w net.ipv4.ip_forward=1  
sudo sysctl -p

#Add host1
sudo ip netns add host1
sudo ip link add veth1 type veth peer name eth0 netns host1
sudo ip netns exec host1 ip link set lo up
sudo ip netns exec host1 ip link set eth0 up
sudo ip netns exec host1 ip addr add 1.1.1.1/24 dev eth0
sudo ip netns exec host1 ip route add default via 1.1.1.254 dev eth0

sudo ip link add br10 type bridge
sudo ip link set br10 up
sudo ip link set veth1 up
sudo ip link set veth1 master br10
sudo ip addr add 1.1.1.254/24 dev br10

#Add host2
sudo ip netns add host2
sudo ip link add veth2 type veth peer name eth0 netns host2
sudo ip netns exec host2 ip link set lo up
sudo ip netns exec host2 ip link set eth0 up
sudo ip netns exec host2 ip addr add 2.2.2.2/24 dev eth0
sudo ip netns exec host2 ip route add default via 2.2.2.254 dev eth0

sudo ip link add br20 type bridge
sudo ip link set br20 up
sudo ip link set veth2 up
sudo ip link set veth2 master br20
sudo ip addr add 2.2.2.254/24 dev br20

#Add vni 100 as l3vni
sudo ip link add br100 type bridge
sudo ip link add vxlan100 type vxlan id 100 local 192.168.59.128 dstport 4789 nolearning
sudo ip link set br100 up
sudo ip link set vxlan100 up
sudo ip link set vxlan100 master br100  
#Suo IP addr add 5.5.5.254/24 dev BR100 Keep in mind that as the svi interface of l3vni, IP cannot be configured, otherwise the received type-5 routing will not be installed.
sudo ip link set dev br100 address 00:00:01:02:03:04 #This is routing mac

#Add vni 200 as evpn-vrf1 l3vni
sudo ip link add br200 type bridge
sudo ip link add vxlan200 type vxlan id 200 local 192.168.59.128 dstport 4789 nolearning
sudo ip link set br200 up
sudo ip link set vxlan200 up
sudo ip link set vxlan200 master br200  
#Suo IP addr add 5.5.5.254/24 dev BR100 Keep in mind that as the svi interface of l3vni, IP cannot be configured, otherwise the received type-5 routing will not be installed.
sudo ip link set dev br200 address 00:00:01:02:03:05 #This is routing mac

#Add vrf
sudo ip link add evpn-vrf type vrf table 100
sudo ip link set evpn-vrf up
sudo ip link set br100 master evpn-vrf  
sudo ip link set br10 master evpn-vrf 

sudo ip link add evpn-vrf1 type vrf table 200
sudo ip link set evpn-vrf1 up
sudo ip link set br200 master evpn-vrf1  
sudo ip link set br20 master evpn-vrf1 

#close rp filter
sudo sysctl -w net.ipv4.conf.all.rp_filter=0
sudo sysctl -w net.ipv4.conf.default.rp_filter=0
sudo sysctl -w net.ipv4.conf.br100.rp_filter=0
sudo sysctl -w net.ipv4.conf.br200.rp_filter=0

#startup bgp
sudo zebra/zebra -d
sudo staticd/staticd -d
sudo bgpd/bgpd -d
sudo vtysh/vtysh

bgp evpn configuration

vrf evpn-vrf
 vni 100
 exit-vrf
!
vrf evpn-vrf1
 vni 200
 exit-vrf
!
router bgp 7675
 bgp router-id 192.168.59.128
 bgp bestpath as-path multipath-relax
 neighbor fabric peer-group
 neighbor fabric remote-as external
 neighbor 192.168.59.130 peer-group fabric
 !
 address-family l2vpn evpn
  neighbor fabric activate
  advertise-all-vni
 exit-address-family
!
router bgp 7675 vrf evpn-vrf1
 !
 address-family ipv4 unicast
  network 2.2.2.0/24
  import vrf evpn-vrf
 exit-address-family
 !
 address-family l2vpn evpn
  advertise ipv4 unicast
 exit-address-family
!
router bgp 7675 vrf evpn-vrf
 !
 address-family ipv4 unicast
  network 1.1.1.0/24
  network 5.1.1.0/24
  import vrf evpn-vrf1
 exit-address-family
 !
 address-family l2vpn evpn
  advertise ipv4 unicast
 exit-address-family
!
line vty
!
end
ubuntu# 

Note:

import vrf evpn-vrf1

This instruction represents the introduction of routing from evpn-vrf1.

sudo sysctl -w net.ipv4.conf.all.rp_filter=0
sudo sysctl -w net.ipv4.conf.default.rp_filter=0
sudo sysctl -w net.ipv4.conf.br100.rp_filter=0
sudo sysctl -w net.ipv4.conf.br200.rp_filter=0

This instruction is used to turn off the reverse path checking of linux kernel. Because br100 and br200 do not configure ip, they belong to unnumber interface and need to prohibit the reverse path checking.

leaf2 configuration

Interface configuration

#!/bin/bash
#Open and forward
sudo sysctl -w net.ipv4.ip_forward=1  
sudo sysctl -p

#Add host3
sudo ip netns add host3
sudo ip link add veth3 type veth peer name eth0 netns host3
sudo ip netns exec host3 ip link set lo up
sudo ip netns exec host3 ip link set eth0 up
sudo ip netns exec host3 ip addr add 3.3.3.3/24 dev eth0
sudo ip netns exec host3 ip route add default via 3.3.3.254 dev eth0 

# Add bridges, add veth3 to bridges
sudo ip link add br30 type bridge
sudo ip link set br30 up
sudo ip link set veth3 up
sudo ip link set veth3 master br30
sudo ip addr add 3.3.3.254/24 dev br30

#Add host4
sudo ip netns add host4
sudo ip link add veth4 type veth peer name eth0 netns host4
sudo ip netns exec host4 ip link set lo up
sudo ip netns exec host4 ip link set eth0 up
sudo ip netns exec host4 ip addr add 4.4.4.4/24 dev eth0
sudo ip netns exec host4 ip route add default via 4.4.4.254 dev eth0

sudo ip link add br40 type bridge
sudo ip link set br40 up
sudo ip link set veth4 up
sudo ip link set veth4 master br40
sudo ip addr add 4.4.4.254/24 dev br40

#Add vni 100 as ievpn-vrf l3vni
sudo ip link add br100 type bridge
sudo ip link add vxlan100 type vxlan id 100 local 192.168.59.129 dstport 4789 nolearning
sudo ip link set br100 up
sudo ip link set vxlan100 up
sudo ip link set vxlan100 master br100  
#Suo IP addr add 5.5.5.253/24 dev BR100 Keep in mind that you must not add IP addresses, otherwise type5 routing will not work properly under the kernel
sudo ip link set dev br100 address 00:00:01:02:03:06  #This is rmac, routing mac

sudo ip link add br200 type bridge
sudo ip link add vxlan200 type vxlan id 200 local 192.168.59.129 dstport 4789 nolearning
sudo ip link set br200 up
sudo ip link set vxlan200 up
sudo ip link set vxlan200 master br200  
#Suo IP addr add 5.5.5.253/24 dev BR100 Keep in mind that you must not add IP addresses, otherwise type5 routing will not work properly under the kernel
sudo ip link set dev br200 address 00:00:01:02:03:07  #This is rmac, routing mac

#Add vrf
sudo ip link add evpn-vrf type vrf table 100
sudo ip link set evpn-vrf up
sudo ip link set br100 master evpn-vrf  
sudo ip link set br30 master evpn-vrf 

sudo ip link add evpn-vrf1 type vrf table 200
sudo ip link set evpn-vrf1 up
sudo ip link set br200 master evpn-vrf1  
sudo ip link set br40 master evpn-vrf1 

#Add static routing across vrf to get through host3 and host4
sudo ip route add 4.4.4.4 dev br40 vrf evpn-vrf
sudo ip route add 3.3.3.3 dev br40 vrf evpn-vrf1

#Access external network

#Add vtep interface connecting evpn-vrf to default VRF
sudo ip link add ext1 type veth peer name ext
sudo ip link set ext1 up
sudo ip link set ext up

#Add vtep interface connecting evpn-vrf1 to default VRF
sudo ip link add ext2 type veth peer name ext3
sudo ip link set ext2 up
sudo ip link set ext3 up

#Where ext1 is in evpn-vrf and ext is in default
sudo ip link set ext1 master evpn-vrf

#Use segment 5.5.5.0/24 as relay segment
sudo ip addr add 5.5.5.253/24 dev ext1
sudo ip addr add 5.5.5.254/24 dev ext

sudo ip addr add 5.5.6.253/24 dev ext2
sudo ip addr add 5.5.6.254/24 dev ext3

#Add default routing in evpn to allow traffic to access the public network by default
sudo ip route add default via 5.5.5.254 dev ext1 vrf evpn-vrf
sudo ip route add default via 5.5.6.254 dev ext2 vrf evpn-vrf1

#Configure snat to change private network traffic to smac
sudo nft add table nat
sudo nft add chain nat prerouting { type nat hook prerouting priority 0 \; }
sudo nft add chain nat postrouting { type nat hook postrouting priority 100 \; }
sudo nft add rule nat postrouting oifname ext1  counter masquerade
sudo nft add rule nat postrouting oifname ens33  counter masquerade
sudo nft add rule nat postrouting oifname ext2  counter masquerade

sudo sysctl -w net.ipv4.conf.all.rp_filter=0
sudo sysctl -w net.ipv4.conf.default.rp_filter=0
sudo sysctl -w net.ipv4.conf.br100.rp_filter=0
sudo sysctl -w net.ipv4.conf.br200.rp_filter=0

sudo chmod 777 /var/run/
sudo zebra -d
sudo staticd -d
sudo bgpd -d
sudo vtysh 

bgp evpn configuration

vrf evpn-vrf
 vni 100
 exit-vrf
!
vrf evpn-vrf1
 vni 200
 exit-vrf
!
router bgp 7676
 bgp router-id 192.168.59.129
 bgp bestpath as-path multipath-relax
 neighbor fabric peer-group
 neighbor fabric remote-as external
 neighbor 192.168.59.130 peer-group fabric
 !
 address-family l2vpn evpn
  neighbor fabric activate
  advertise-all-vni
 exit-address-family
!
router bgp 7676 vrf evpn-vrf1
 !
 address-family ipv4 unicast
  network 4.4.4.0/24
  import vrf evpn-vrf
 exit-address-family
 !
 address-family l2vpn evpn
  advertise ipv4 unicast
 exit-address-family
!
router bgp 7676 vrf evpn-vrf
 !
 address-family ipv4 unicast
  network 0.0.0.0/0
  network 3.3.3.0/24
  import vrf evpn-vrf1
 exit-address-family
 !
 address-family l2vpn evpn
  advertise ipv4 unicast
 exit-address-family
!
line vty
!
end

View bgp information

leaf1

  • View routing information
ubuntu# show bgp l2vpn evpn 
BGP table version is 6, local router ID is 192.168.59.128
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: ip 1.1.1.254:3

*> [5]:[0]:[24]:[1.1.1.0]
                    192.168.59.128           0         32768 i
*> [5]:[0]:[24]:[4.4.4.0]
                    192.168.59.128                         0 7677 7676 i
*> [5]:[0]:[24]:[5.1.1.0]
                    192.168.59.128           0         32768 i
Route Distinguisher: ip 2.2.2.254:2

*> [5]:[0]:[0]:[0.0.0.0]
                    192.168.59.128                         0 7677 7676 i
*> [5]:[0]:[24]:[2.2.2.0]
                    192.168.59.128           0         32768 i
*> [5]:[0]:[24]:[3.3.3.0]
                    192.168.59.128                         0 7677 7676 i
Route Distinguisher: ip 5.5.5.253:3

*> [5]:[0]:[0]:[0.0.0.0]
                    192.168.59.129                         0 7677 7676 i
*> [5]:[0]:[24]:[3.3.3.0]
                    192.168.59.129                         0 7677 7676 i
Route Distinguisher: ip 5.5.6.253:2

*> [5]:[0]:[24]:[4.4.4.0]
                    192.168.59.129                         0 7677 7676 i

Displayed 9 out of 9 total prefixes
ubuntu# 

leaf2

  • View routing information
ubuntu# show bgp l2vpn evpn 
BGP table version is 6, local router ID is 192.168.59.129
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: ip 1.1.1.254:3

*> [5]:[0]:[24]:[1.1.1.0]
                    192.168.59.128                         0 7677 7675 i
*> [5]:[0]:[24]:[5.1.1.0]
                    192.168.59.128                         0 7677 7675 i
Route Distinguisher: ip 2.2.2.254:2

*> [5]:[0]:[24]:[2.2.2.0]
                    192.168.59.128                         0 7677 7675 i
Route Distinguisher: ip 5.5.5.253:3

*> [5]:[0]:[0]:[0.0.0.0]
                    192.168.59.129           0         32768 i
*> [5]:[0]:[24]:[2.2.2.0]
                    192.168.59.129                         0 7677 7675 i
*> [5]:[0]:[24]:[3.3.3.0]
                    192.168.59.129           0         32768 i
Route Distinguisher: ip 5.5.6.253:2

*> [5]:[0]:[24]:[1.1.1.0]
                    192.168.59.129                         0 7677 7675 i
*> [5]:[0]:[24]:[4.4.4.0]
                    192.168.59.129           0         32768 i
*> [5]:[0]:[24]:[5.1.1.0]
                    192.168.59.129                         0 7677 7675 i

Displayed 9 out of 9 total prefixes
ubuntu# 

summary

The key points of this experiment are:

  • bgp route leak.
  • linux kernel routing supports that the next hop interface is not in the same vrf.
  • Reverse path checking needs to be turned off when no ip interface is configured to forward messages.

Posted by Dodon on Tue, 01 Oct 2019 19:53:33 -0700