Within this article I will try to give you a clear vision on the HP Virtual Connect FlexFabric Mappings that HP uses to facilitate their blades with NIC’s and HBA’s. If you are looking for the Flex-10 mappings click here.
In our example we use the HP BL460 Gen8 blade with a FlexibleLOM (LAN on Motherboard) Adapter and no additional mezzanine cards. HP’s quote about the FlexibleLOM:
In past server generations, the adapter is embedded on the system motherboard as a LOM (LAN on motherboard). If a customer desired a different kind of networking adapter, they would need to purchase a standup adapter and install it in a slot in the server.
Now with many ProLiant Gen8 servers, our customers can now choose the type of network adapter they want. This new Flexible LOM provides our customer the ability to customize their server’s networking of today and the ability to change to meet future needs without overhauling server infrastructure.
NIC to Interconnect
First we start of looking at the “NIC to Interconnect“-mappings. These are pretty straight forward and should be known to all HP c-Class Administrators. Please note that the connections that are drawn below are hardwired connections on the Backplane of the HP c7000 Enclosure.
FlexNics / FlexHBA
Our ESXi Host is physically equipped with one FlexibleLOM Adapter which consists of two 10 Gb ports. Each 10 Gb port is connected to a dedicated HP VirtualConnect FlexFabric 10 Gb/24 module and can be divided into FlexNics and a FlexHBA. A FlexNic will present itself to ESXi as a vmnic while a FlexHBA will present itself as a vmhba.
The possible FlexNics and FlexHBA scenarios are listed below.
Scenario 1: Ethernet and FC over Ethernet
- 10 Gb port 1: Up to 3x Ethernet (FlexNics) + 1x Fibre Channel (over Ethernet) (FlexHBA)
- 10 Gb port 2: Up to 3x Ethernet (FlexNics) + 1x Fibre Channel (over Ethernet) (FlexHBA)
Scenario 2: Ethernet and Accelerated iSCSI
- 10 Gb port 1: Up to 3x Ethernet (FlexNics) + 1x Accelerated iSCSI (FlexHBA)
- 10 Gb port 2: Up to 3x Ethernet (FlexNics) + 1x Accelerated iSCSI (FlexHBA)
Scenario 3: Ethernet Only
- 10 Gb port 1: Up to 4x Ethernet (FlexNics)
- 10 Gb port 2: Up to 4x Ethernet (FlexNics)
vmnic / vmhba mappings
Now we know which FlexNic and FlexHBA scenarios we can facilitate it’s time to see how this looks like from an ESXi Host perspective. Please note that Scenario 1 “Ethernet and FC over Ethernet” is shown in the diagram below:
Also note that the second physical location (indicated in green) can be a FlexNic or FlexHBA. The other locations (indicated in blue) can only be FlexNics. This is a “hard limit” and applicable to every scenario!
Speeds
Both the FlexNic and the FlexHBA have an adjustable speed in 100Mb increments starting from either 100Mb or 1Gb up to 10Gb depending on the type of hardware that is specifically used. If you for instance want to use a 10Gb vmnic within ESXi it makes sense that no other functions can be added since we only got 10Gb in total to divide. In this case the ESXi Host will end up with only two usable vmnics as shown in the example below:
While you could also design the connectivity as shown in the diagram below:
Key message here is ensure that you have got a good ESXi design and hopefully this article helps you in getting some more insights in the Virtual Connect FlexFabric mappings. Also be sure to check out the HP Virtual Connect FlexFabric Cookbook which is full of technical details and scenario’s that could be of use to you.
Andy
/ March 3, 2013Thank U for excellent explanation.
Can U pls clarify for me what does in means when HP VC manual for c-Class blades ver 3.30 through 3.60 states that “The current maximum number of FC/FCoE connections is four per I/O bay.”?
f.e. if I have fabrics named “san_a” and “san_b” which uses uplinks b1:X3 and b2:X3 as FC ports, and I have associated 4 fcoe connections with 4 server profiles like follows:
add fcoe-connection Profile-1:1 Fabric=san_a
add fcoe-connection Profile-1:2 Fabric=san_b
…
add fcoe-connection Profile-4:1 Fabric=san_a
add fcoe-connection Profile-4:2 Fabric=san_b
I will not be able to create connection like
add fcoe-connection Profile-5:1 Fabric=san_a
add fcoe-connection Profile-5:2 Fabric=san_b
?
Kenneth van Ditmarsch
/ March 18, 2013Not sure in which context that remark is placed?
Steve
/ April 8, 2013Kenneth, thanks for both articles on the Flex-10 mappings. I’m a network guy trying to understand these virtual mappings and they helped a lot! Can you explain the next layer, how the vmnics (which I believe map to separate vswitches) are mapped to the physical x1 – x6 ports of the Flex-10 module?
Kenneth van Ditmarsch
/ April 9, 2013The Flex-10 module has Uplinks (in terms of X1-X6) and has Downlinks which are hardwired via the Enclosure backplane. Basically the vmnics are conneccted to these Downlinks which are hardwired and thus are static towards the blade. My overview displays these mappings as well.
Gaurav
/ December 4, 2013Hello Kenneth,
Thanks for the wonderful article.I am pretty new to this technology and your articles have been very helpful.
Can you please answer my following query.?
If we create two SUS corresponding to each flexfabric.(SUS1 on vc1 and SUS2 on VC2)
Now if we choose 1-a,1-b,1-c or 1-d in edit profile wizard then why do we get an option to select a SUS when you already know that the traffic via this flexNIC can only go through VC1 (as port 1 is hardwired to VC1) hence should’nt the only option be SUS1 which is created on VC1.Similarly for 2-a,b,c,d the SUS should be SUS2.
Can you please tell me where am I getting it wrong ?
Kenneth van Ditmarsch
/ December 6, 2013Hi Gaurav,
Not sure from the top of my head. I would suggest to check out this document: http://h20628.www2.hp.com/km-ext/kmcsdirect/emr_na-c02616817-5.pdf which possibly gives you the answer?
Jim
/ January 22, 2014I agree with Gaurav. In summary, Virtual Connect allows you to define connections which are not physically possible due to the hardwired NIC / VC mappings explained so clearly in Kenneth’s article. VC should give you at least a warning that you are defining something nonsensical, but it doesn’t.
You can define a blade profile and choose to connect any of the blade’s NICs to *any* VC bay. For example, you could map the first NIC (Lom1 or Lom1-a) to a physical uplink port in VC bay 7. It will look fine in the GUI but can never work because VC7 is not available to that NIC. Did I miss something ?
Freaky
/ January 30, 2014Nice article, do have a question though. Everywhere I read ‘up to’ X NICs. I have a blade with only 2 NICs configured, the other 4 + the 2 FCoE connections have been removed in VCM. Unfortunately, the machine still sees 8 NICs (or 6 NICs and 2 FCoE adapters). What’s that about? I don’t want to confuse my collegues and I like neat and tidy – what’s the point of NICs that can’t be used anyways (all bandwidth is assigned to the first 2).
Anyways, can I make it so the OS doesn’t see them?
Thanks :).
Kenneth van Ditmarsch
/ January 30, 2014yeah, well known issue. afaik you
cant “clean” the OS to not see those Nics unfortunately.
CoolHand
/ July 4, 2014Freaky. If you look at the checkboxes when you edit a server profile you should see one that says “Hide Unused FlexNICs”. Check that box and they should all go away!
Kenneth van Ditmarsch
/ July 4, 2014sounds like a cool new feature!
Pramod Nair
/ July 28, 2014Thanks. How can I use all four FlexNICs of the physical port. I don’t want FlexHBA in my scenario. I am not getting that option while configuring. Can you suggest me the steps?
Kenneth van Ditmarsch
/ July 29, 2014dont believe thats possible
Jason Main
/ April 6, 2015Our vm guy wants to do the following. He has vswitch0 on the LOM but he wants a 2nd vswitch0 on a mez card say on the flex10 module in slot 4.
vm guy does not want to put the 2nd vswitch0 on the other LOM because he believe that if the LOM lost connectivity, the entire LOM will lose network communication. Is he correct?
So i started thinking would the flex10 module in slot 1 and slot 2 go out at the same time? Is there a single backplane?
Kenneth van Ditmarsch
/ April 7, 2015First of all, I guess you mean 1x vSwitch0 and 1x vSwitch1 (or whatever number..)
Secondly, the LOM is Lan On Motherboard, and thus from that point of view a single point of failure. If you also got Mezzanine Cards it could be a good idea to split traffic across these “physical” network cards.
There is a single motherboard in the blade, so if the motherboard would stop functioning you obviously loose the whole blade.
Muthukumar
/ June 19, 2015Thanks for the article.
I have a query.
In our c7000, we have 4FC(FC1,FC2,FC3,FC4) ports in our fabric module connected with a Brocade DCX director. We want to use 2(FC6,FC7) of the remaining FC ports in the module to be connected with Cisco MDS switch and add those 2 ports as uplink to SAN A.
Will it work ? And when we add the new uplinks whether the cisco MDS switch will see the existing blade server vHBA wwns ?
Kenneth van Ditmarsch
/ June 19, 2015well…im unsure of that configuration. i dont believe so to be honest but im not the guy to ask obviously 😉
Jason Brown
/ June 30, 2015Excellent help. We cannot enumerate interface 1-b using he same method to create the network and assign to server profile. Do we need to do something different for this interface?
Yassen
/ June 3, 2016Folks, I have the following setup:
36 blades BL460cG9 each with
1LOM and 2 Mezz 650 cards
a pair of 10/10D + a pair of 20/40 F8 ICs.
Only Mezz 1 is utilized in my setup.
The second Mezz is for future expansion, so it doesn’t go to any IC. Hence, Ports 5-8 are empty on the C7000
One would expect that vmnics in vCenter will be consistently identically mapped! Right? Well, i have at least 4 different patterns.
I called HP for explanation, they dodged the question in VMWare’s yard. VMWare support sent me some useless document, and i am about to deploy another 176 blades. I hoped a script will help me assign vmnics to port groups, however, the outcome is unpredictable.
Please, share your thoughts.
Very much appreciated any help.
Cheers,
Yassen
Cahit Bollu
/ November 15, 2016Hi guys,
Is it possible to use VC in Bay1 slot and Passthrough in Bay2 slot on interconnect bays?
Thanks
Cahit
Kenneth van Ditmarsch
/ January 4, 2017Hi Rene,
Honestly I wouldn’t know as I haven’t worked with a similar setup you are working with.
I would recommend to check out HP’s Cookbooks to see if there is a similar setup in there.