Here’s the scenario. A customer has a datacenter currently outfitted with a single Extreme Networks S4 Core switch. All servers in the datacenter flow through this switch for all Layer 3 communications, which is then handed off to miscellaneous load balancers and firewalls. The customer needs more capacity and redundancy as they cannot tolerate any downtime. So they purchase 2 new S4 Cores for a dual-core design. So how do we shift all Layer 3 from the old core to the 2 new cores without incurring downtime? VRRP.
VRRP (Virtual Router Redundancy Protocol) is a standard that allows you to use multiple routers to advertise a single VIP that you use as your routing endpoint. In this example, I’m going to show you how to shift 3 network’s (vlan 10, 20, and 30) Layer 3 interfaces from 1 Old Core to a new dual-core design. So to start with, I’ve connected the 2 new Cores together and then over to the Old Core. I’ve also added a new vlan (100) that will be used to manage the New Cores and as an uplink for routing between the Cores.
As you can see, we have our 2 New Cores connected over a 20gig LAG, and then another 20gig LAG between New-Core1 and the Old Core. Each of the LAGs are passing all 4 of the VLANs in question.
So first, we stage the 3 vlans’ Layer 3 configuration on the 2 New Cores. VRRP decides which Router is the *master* router, the router that answers the arp for your VIP, using priorities. Higher priorities win the election process. If you do not define a priority on an Extreme switch, it defaults to 100, max is 255. So we’ll leave the default on New-Core1, and set New-Core2 to 90.
New Core 1
configure interface vlan 10 shutdown ip address 192.168.10.3 255.255.255.0 vrrp create 1 v3-IPv4 vrrp address 1 192.168.10.1 vrrp accept-mode 1 vrrp enable 1 exit interface vlan 20 shutdown ip address 192.168.20.3 255.255.255.0 vrrp create 1 v3-IPv4 vrrp address 1 192.168.20.1 vrrp accept-mode 1 vrrp enable 1 exit interface vlan 30 shutdown ip address 192.168.30.3 255.255.255.0 vrrp create 1 v3-IPv4 vrrp address 1 192.168.30.1 vrrp accept-mode 1 vrrp enable 1 exit
New Core 2
configure interface vlan 10 shutdown ip address 192.168.10.2 255.255.255.0 vrrp create 1 v3-IPv4 vrrp address 1 192.168.10.1 vrrp priority 1 90 vrrp accept-mode 1 vrrp enable 1 exit interface vlan 20 shutdown ip address 192.168.20.2 255.255.255.0 vrrp create 1 v3-IPv4 vrrp address 1 192.168.20.1 vrrp priority 1 90 vrrp accept-mode 1 vrrp enable 1 exit interface vlan 30 shutdown ip address 192.168.30.2 255.255.255.0 vrrp create 1 v3-IPv4 vrrp address 1 192.168.30.1 vrrp priority 1 90 vrrp accept-mode 1 vrrp enable 1 exit
In addition to these, we’re going to enable VRRP on the vlan 100 interface on the New Cores so we have a VIP to route to.
New Core 1
configure interface vlan 100 vrrp create 1 v3-IPv4 vrrp address 1 10.10.100.1 vrrp accept-mode 1 vrrp enable 1 exit
New Core 2
configure interface vlan 100 vrrp create 1 v3-IPv4 vrrp address 1 10.10.100.1 vrrp priority 1 90 vrrp accept-mode 1 vrrp enable 1 exit
The Old Core already has layer 3 addresses configured for vlans 10,20, and 30 already, so we just have to add the VRRP portion. We’ll set the priority here to 110 so that no Layer 3 actually moves yet. The end result looks like this:
Old Core
configure interface vlan 10 vrrp create 1 v3-IPv4 vrrp address 1 192.168.10.1 vrrp priority 1 110 vrrp accept-mode 1 vrrp enable 1 exit interface vlan 20 vrrp create 1 v3-IPv4 vrrp address 1 192.168.20.1 vrrp priority 1 110 vrrp accept-mode 1 vrrp enable 1 exit interface vlan 30 vrrp create 1 v3-IPv4 vrrp address 1 192.168.30.1 vrrp priority 1 110 vrrp accept-mode 1 vrrp enable 1 exit
Keep in mind that we haven’t “no shut” the interfaces on the New Cores, so they still aren’t in play just yet. Turning on VRRP does however change change the MAC of your Layer 3 interfaces to the MAC that will be used from now on by the VIP. So your devices will need to re-Arp, but hopefully this isn’t a big deal. This is also a good time to talk about static ARP entries. If you have any on the Old Core, you need to make sure and set those on both New Cores as well, or they won’t work once we transfer the VIP. On the Old Core we can verify that VRRP is up.
new-core1(su)->show ip vrrp
Codes: Pri = Operational Priority
V = Version of the protocol
T = Type ( M-Master IP Address, A-Associate IP Address )
A = Admin status of Associate address ( E-enabled, D-disabled )
O = Owner status of Associate address ( Y-yes, N-no )
Interface Vrid State Pri V T A O IP Address
----------- ---- ------------ --- - - - - -------------------------------------
vlan.0.10 1 master 110 3 M - - 192.168.10.1
A E N 192.168.10.1
vlan.0.20 1 master 110 3 M - - 192.168.20.1
A E N 192.168.20.1
vlan.0.30 1 master 110 3 M - - 192.168.30.1
A E N 192.168.30.1
We’re almost ready to bring up our new Layer 3 interfaces on the New Cores. Except for one major detail that I almost missed when I was doing this.
Imagine this, you bring up vlan 10 on both New Cores, then you shut down vlan 10 on the Old Core. The VRRP Election process would elect New-Core1 as the master and start responding on 192.168.10.1. Everything that uses that as the default gateway would just have to re-arp and everything would be great, right? Sure, until they wanted to talk to anything outside vlan 10. The Old Core would no longer know how to get to vlan 10 since it used to be a Directly Connected route. Since Directly Connected routes are preferred to Static Routes, we can prestage them without issue. We’re also going to prestage a new Default Gateway at this time since we’re going to remove vlan 10’s interface soon. We’ll set a higher metric on it so that it doesn’t take effect until vlan 10 goes down.
Old Core
configure ip route 192.168.10.0/24 10.10.100.1 interface vlan.0.100 1 ip route 192.168.20.0/24 10.10.100.1 interface vlan.0.100 1 ip route 192.168.30.0/24 10.10.100.1 interface vlan.0.100 1 ip route 0.0.0.0/0 10.10.100.1 interface vlan.0.100 10
Almost ready, one more important thing to take care of, our default route out of this network. The default route in this case is handled by vlan 10. So we need to add that to our pre-stages routes on the New Cores, again with a higher metric.
New Core 1
configure ip route 0.0.0.0/0 192.168.10.254 interface vlan.0.10 1
New Core 2
configure ip route 0.0.0.0/0 192.168.10.254 interface vlan.0.10 1
Now comes the fun part, actually shifting everything. With everything we’ve done, this is done is 3 simple steps for each vlan.
- Enable the vlan interface on New-Core1
- Enable the vlan interface on New-Core2
- Disable the vlan interface on Old-Core
It’s important that we save vlan 10 for last since it’s our default gateway out of the system. So here’s vlan 30.
New Core 1
configure interface vlan 30 no shutdown
New Core 2
configure interface vlan 30 no shutdown
Old Core
configure interface vlan 30 shutdown
As soon as we bring up the interfaces on the New Cores, the directly connected routes override the static route for 0.0.0.0/0. Vice versa on the Old Core. Then just rinse and repeat for vlans 20 and 10. You will have a very brief moment in time when you have Directly Connected routes on the New Cores and the Old Cores for each vlan. That will create an asymmetric route while you’re doing the switch. Most IP communications won’t really care, but you want to be quick just in case you have some that do. I lost a total of a single ping on each vlan when I did my changes.
All in all, this turned out to be an extremely effective strategy. My client never noticed any issues from users or their monitoring systems. I will definitely be using the same procedure for future deployments.
