NetApp | Zero-Downtime Migration – Interface-Group Singlemode to Multimode-LACP

By | 2018-04-08

Our first setup with NetApp Clustered Data ONTAP starting with 8.0.1 in late 2010 was build with two single network switches, each connected with a single 10 GbE link. Because you cannot span a link aggregation over two single switches we decided to use an interface group with mode “singlemode”, which means that simply only one of the two links is active at a time. If one switch fails the interface group switches over to the other switch and there is no outage. But at anytime only with the maximum bandwidth of one single link:

In times with massive increasing needs of bandwidth, we have to redesign our network. One options was to increase the count of links to both switches and create a link aggregation to switch 1 and a second one to switch 2, but that means we need to use four 10 GbE ports on each NetApp controller. While using 4x 10 GbE links, we can only consume a maximum of 20 Gbit/s bandwidth.
The second option was so create a logical switch out of our two single switches – using our Juniper Switches this feature is called “Virtual Chassis” (VC). With VC you have a big logical switch, that means you can build a link aggregation over two links each 10 GbE and use both links as active-active. In this scenario you can also consume a maximum of 20 Gbit/s bandwidth, but only need 2x 10 GbE links at your NetApp controller:

Because our NetApp clusters growed massively over the time, we had several hundred VLANs and logical interfaces for our customers. Moving from singlenode to multimode_lacp means creating a new interface group, recreating all VLANs on this group, adding all new VLAN-ports to the broadcast-domains, migration all logical interfaces to the new ports and removing all old configuration. Having this number of VLANs in your cluster this configuration will end up in serveral thousand lines of CLI code or maybe about 10 years clicking around in the GUI 😉 . Impossible to do without scripting or the WFA. I’ve choosen the scripting way and used BASH (mostly ‘AWK’ and ‘SED’) to build this code and Cut&Paste it into our NetApp CLI.

Here we go: starting with singlemode interface group using e0e and e0f:

cluster::> network port ifgrp show -fields mode,ports node
ifgrp mode       ports
-------- ----- ---------- -----
na1a a0a   singlemode e0e, e0f

 

Step 1: Create a new interface group and move one physical port to it:

cluster::> network port ifgrp create -node na1a -ifgrp a0b -distr-func ip -mode multimode_lacp
cluster::> network port ifgrp remove-port -node na1a -ifgrp a0a -port e0e
cluster::> network port ifgrp add-port -node na1a -ifgrp a0b -port e0e

Now we’ve got a singlemode and another multimode_lacp interface group:

cluster::> network port ifgrp show -fields mode,ports 
node   ifgrp mode             ports 
------ ----- ----------       ----- 
na1a   a0a   singlemode       e0f   
na1a   a0b   multimode_lacp   e0e

 

Step 2: Create all VLANs on the new ifgrp:

ssh admin@cluster "network port vlan show -fields vlan-id" | awk '{ print $2 }' | sort | uniq >vlans_cluster1
awk '{ print "network port vlan create -node na1a -vlan-name " $1 }' vlans_cluster1 | sed "s/a0a/a0b/g"

Which results in many commands like:

network port vlan create -node na1a -vlan-name a0b-514
network port vlan create -node na1a -vlan-name a0b-515
(...)

 

Step 3: Add the new VLAN-Ports to the new ifgrp:

ssh admin@cluster "network port broadcast-domain show" | grep -E '(bcast|na1a)'  >bcast_all
awk '{ print $1 }' bcast_all | tr '\n' ' ' | sed -e $'s/bcast_/\\\nbcast_/g' >bcast_without_newline
awk '{ print "network port broadcast-domain add-ports -broadcast-domain " $1 " -ports " $2 }' bcast_without_newline

Which results in many commands like:

 network port broadcast-domain add-ports -broadcast-domain bcast_customer1 -ports na1a:a0b-514 network port broadcast-domain add-ports -broadcast-domain bcast_customer2 -ports na1a:a0b-515 (...) 

 

Step 4: Modify all logical interfaces’ home-port

 ssh admin@cluster "network interface show -fields home-port" | sed "s/a0a/a0b/g" >lifs awk '{ print "network interface modify -vserver " $1 " -lif " $2 " -home-port " $3 }' lifs 

Which results in many commands like:

network interface modify -vserver vserver_customer1 -lif lif_customer1 -home-port a0b-514
network interface modify -vserver vserver_customer1 -lif lif_customer2 -home-port a0b-515
(...)

 

Step 5: Migrate all logical interfaces to the interface group

cluster::> network interface revert -vserver vserver_customer1 -lif lif_customer1

 

Step 6: Remove old interface group from broadcast domains

cluster::> network port broadcast-domain remove-ports -broadcast-domain bcast_customer1 -ports na1a:a0a-514
cluster::> network port broadcast-domain remove-ports -broadcast-domain bcast_customer1 -ports na1a:a0a-515
(...)

 

Step 7: Remove old VLAN ports from interface group

awk '{ print "network port vlan delete -node na1a -vlan-name " $1 }' vlans_cluster1

Which results in many commands like:

network port vlan delete -node na1a -vlan-name a0a-514
network port vlan delete -node na1a -vlan-name a0a-515
(...)

 

Step 8: Delete old interface group

cluster::> network port ifgrp delete -node na1a -ifgrp a0a

 

End result looks like:

 cluster::> network port ifgrp show -fields activeports,ports,mode node          ifgrp mode           activeports ports    ------------- ----- -------------- ----------- -------  na1a           a0b   multimode_lacp full        e0e,e0f 

 

Because there is no “networt port ifgrp rename” command, you need to repeat all these steps if you want your only interface group to be named “a0a” again ;-(

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.