Our first setup with NetApp Clustered Data ONTAP starting with 8.0.1 in late 2010 was build with two single network switches, each connected with a single 10 GbE link. Because you cannot span a link aggregation over two single switches we decided to use an interface group with mode “singlemode”, which means that simply only one of the two links is active at a time. If one switch fails the interface group switches over to the other switch and there is no outage. But at anytime only with the maximum bandwidth of one single link:
In times with massive increasing needs of bandwidth, we have to redesign our network. One options was to increase the count of links to both switches and create a link aggregation to switch 1 and a second one to switch 2, but that means we need to use four 10 GbE ports on each NetApp controller. While using 4x 10 GbE links, we can only consume a maximum of 20 Gbit/s bandwidth.
The second option was so create a logical switch out of our two single switches – using our Juniper Switches this feature is called “Virtual Chassis” (VC). With VC you have a big logical switch, that means you can build a link aggregation over two links each 10 GbE and use both links as active-active. In this scenario you can also consume a maximum of 20 Gbit/s bandwidth, but only need 2x 10 GbE links at your NetApp controller:
Because our NetApp clusters growed massively over the time, we had several hundred VLANs and logical interfaces for our customers. Moving from singlenode to multimode_lacp means creating a new interface group, recreating all VLANs on this group, adding all new VLAN-ports to the broadcast-domains, migration all logical interfaces to the new ports and removing all old configuration. Having this number of VLANs in your cluster this configuration will end up in serveral thousand lines of CLI code or maybe about 10 years clicking around in the GUI 😉 . Impossible to do without scripting or the WFA. I’ve choosen the scripting way and used BASH (mostly ‘AWK’ and ‘SED’) to build this code and Cut&Paste it into our NetApp CLI.
Here we go: starting with singlemode interface group using e0e and e0f:
cluster::> network port ifgrp show -fields mode,ports node ifgrp mode ports -------- ----- ---------- ----- na1a a0a singlemode e0e, e0f
Step 1: Create a new interface group and move one physical port to it:
cluster::> network port ifgrp create -node na1a -ifgrp a0b -distr-func ip -mode multimode_lacp cluster::> network port ifgrp remove-port -node na1a -ifgrp a0a -port e0e cluster::> network port ifgrp add-port -node na1a -ifgrp a0b -port e0e
Now we’ve got a singlemode and another multimode_lacp interface group:
cluster::> network port ifgrp show -fields mode,ports node ifgrp mode ports ------ ----- ---------- ----- na1a a0a singlemode e0f na1a a0b multimode_lacp e0e
Step 2: Create all VLANs on the new ifgrp:
ssh admin@cluster "network port vlan show -fields vlan-id" | awk '{ print $2 }' | sort | uniq >vlans_cluster1
awk '{ print "network port vlan create -node na1a -vlan-name " $1 }' vlans_cluster1 | sed "s/a0a/a0b/g"
Which results in many commands like:
network port vlan create -node na1a -vlan-name a0b-514 network port vlan create -node na1a -vlan-name a0b-515 (...)
Step 3: Add the new VLAN-Ports to the new ifgrp:
ssh admin@cluster "network port broadcast-domain show" | grep -E '(bcast|na1a)' >bcast_all
awk '{ print $1 }' bcast_all | tr '\n' ' ' | sed -e $'s/bcast_/\\\nbcast_/g' >bcast_without_newline
awk '{ print "network port broadcast-domain add-ports -broadcast-domain " $1 " -ports " $2 }' bcast_without_newline
Which results in many commands like:
network port broadcast-domain add-ports -broadcast-domain bcast_customer1 -ports na1a:a0b-514 network port broadcast-domain add-ports -broadcast-domain bcast_customer2 -ports na1a:a0b-515 (...)
Step 4: Modify all logical interfaces’ home-port
ssh admin@cluster "network interface show -fields home-port" | sed "s/a0a/a0b/g" >lifs awk '{ print "network interface modify -vserver " $1 " -lif " $2 " -home-port " $3 }' lifs
Which results in many commands like:
network interface modify -vserver vserver_customer1 -lif lif_customer1 -home-port a0b-514 network interface modify -vserver vserver_customer1 -lif lif_customer2 -home-port a0b-515 (...)
Step 5: Migrate all logical interfaces to the interface group
cluster::> network interface revert -vserver vserver_customer1 -lif lif_customer1
Step 6: Remove old interface group from broadcast domains
cluster::> network port broadcast-domain remove-ports -broadcast-domain bcast_customer1 -ports na1a:a0a-514 cluster::> network port broadcast-domain remove-ports -broadcast-domain bcast_customer1 -ports na1a:a0a-515 (...)
Step 7: Remove old VLAN ports from interface group
awk '{ print "network port vlan delete -node na1a -vlan-name " $1 }' vlans_cluster1
Which results in many commands like:
network port vlan delete -node na1a -vlan-name a0a-514 network port vlan delete -node na1a -vlan-name a0a-515 (...)
Step 8: Delete old interface group
cluster::> network port ifgrp delete -node na1a -ifgrp a0a
End result looks like:
cluster::> network port ifgrp show -fields activeports,ports,mode node ifgrp mode activeports ports ------------- ----- -------------- ----------- ------- na1a a0b multimode_lacp full e0e,e0f
Because there is no “networt port ifgrp rename” command, you need to repeat all these steps if you want your only interface group to be named “a0a” again ;-(