Cisco UCS — Notes From the Floor (Not a Tutorial)

This post sits between two other topics on the blog:

vCenter 8 / ESXi 8 preflight — Cisco ESXi images and host patching
Physical hardware — cabling, SAN boot, and rack work

Full pages: VMware · Physical Hardware

This is not a UCS Manager walkthrough. These are the things I keep running into on real UCS builds in VMware shops.

What UCS does in a VMware environment

UCS is more than managing a blade servers. Most of the important work lives in UCS Manager:

Service profiles
Pools
VLAN groups
Firmware bundles

All of that has to match what ESXi and the SAN expect on the other end of the cable.

For initial set up, I spend time on service profile build, Fabric Interconnect (FI) cluster setup, firmware alignment with ESXi, and fixing issues when vMotion or SAN zoning does not work after a change.

Service profiles: keep them the same

Two ESXi hosts in the same cluster should not be built manually. Slight difference in vNIC order, BIOS settings, or one-off WWPN mappings cause problems later.

What I do:

Build one template profile for the cluster (management, vMotion, VM traffic, and SAN).
Copy it for each host. Only change what must change, like the name or pool assignment.
Write the profile name on the build sheet next to switch ports and WWPNs.

Saving five minutes with a custom one-off profile usually costs hours later.

Fabric Interconnect clustering: match firmware first

When you add a second FI to a cluster, the setup checks local vs peer firmware. If they do not match, the join stops.

Sometimes the console offers to update the local FI to match the peer.

Make sure the peer FI is already stable on the target version.
Do not interrupt the FI firmware update with Ctrl+C. The warning is there for a reason.

After both FIs are up, UCS Manager is not fully useful until both sides of the cluster are configured: cluster VIP, fabric A/B management IPs, and DNS/domain if you use them. One FI alone is only half the control plane.

A few setup mistakes I have learned the hard way:

Strong password policy on the FI admin account — a typo during setup can mean starting over.
Fabric A vs Fabric B — picking the wrong one is painful to fix later.
The cluster IP is not the same as each FI’s mgmt0 IP. Write both down before you clean up cables.

Firmware bundles must match everything else

UCS firmware does not match all on its own. The bundle we pick in UCS Manager must match:

Your FI, IOM, and blade or Chassis hardware
The ESXi version on the hosts (and the Cisco custom ESXi image if you use it)
Any SAN or NIC features you rely on, like SR-IOV or boot policy

Updating one blade with a random BIOS or adapter change may look fine until that one host fails vMotion or behaves differently under load. I update at the bundle level per cluster, the same way I patch ESXi in waves.

For ESXi 8 on UCS, I use the Cisco UCS add-on depot profile, not the generic VMware zip. Same lesson as the ESXi upgrade post.

Uplinks, optics, and link errors

When uplinks flap or performance is odd, I check the optics before I blame ESXi:

show interface ethernet <uplink-range> transceiver details
show interface ethernet <uplink-range> counters errors

Good Rx/Tx power and zero FCS/CRC errors do not prove the full path is clean end to end. But bad light levels or rising error counters on FI uplinks are worth fixing before you open a VMware support case.

Label fiber both ends (FI port → patch panel → core switch port). Write down the full physical path — not just the VLAN name in UCS Manager.

VLAN work and a small tool I built

Adding VLANs to the right VLAN group in UCS Manager is repetitive and easy to get wrong. I built a small UCS Infra Console (PowerShell + Cisco.UCSManager, optional Docker on Alma/RHEL 9 when time permits will try to add a seperate post on it) that:

Reads VLAN and VLAN-group membership from UCS Manager
Applies bulk adds from a CSV (VlanName, VlanGroup, VlanId when creating new VLANs)
Commits the change in a UCS transaction when you check Apply

It does not replace UCS Manager. It just saves time on work I do often.

SAN handoff: FI is only one step

Before I call an ESXi host ready, the build sheet should include:

WWPNs from the service profile (both fabrics)
FI → MDS/core port mapping (which uplink goes where)
Storage team confirmation on both fabrics and all paths

The most common SAN-boot failure I still see is a wrong WWPN on the ticket compared to what is on the HBA. UCS was configured correctly. The spreadsheet was not.

If the host runs Veritas or other cluster software on shared LUNs, coordinate storage work with ESXi maintenance. See Veritas SAN migration.

Intersight vs UCSM

I Work mostly on UCS Manager but I watch Intersight compatibility for planning, but day-to-day work stays in UCSM, the FI CLI, and PowerShell against the UCS Manager API.

Quick reference

Problem	Check first
Second FI will not join the cluster	Local vs peer FI firmware / UCS Manager version
One blade behaves differently	Service profile differences, pool assignment, BIOS policy
vMotion fails after a UCS change	vNIC order, VLAN group, MTU on FI uplinks
SAN path missing	WWPN on ticket vs profile vs `multipath -ll`
New VLAN needed in production	VLAN group membership — typos often show up only at deploy time