Update Manager - Cannot Execute Upgrade Script On Host - unequal values of the 'payloads' attribute - ESXi 6.5 to 6.7u1 Upgrade.

Having just upgraded a VMware VCSA from 6.5 to 6.7u1 I set about uplifting the ESXi hosts only to stumble at the first hurdle.

Using Update Manager within vCenter to deploy the custom HP 6.7u1 Build the following error was shown...

The error thrown was pretty generic and did not give much away in the first instance, I then switched to the console on the host and attempted the upgrade by mounting the ESXi 6.7 iso through the HP iLO and booting from it. This gave further detail into the problem...

The issue blocking the upgrade appeared to be with an incompatible VIB installed on the current 6.5 build.

Having checked the problem VIB was nothing to do with any drivers used by the management NIC(s) I set about removing it...

If the VIB blocking the upgrade is linked to a critical network driver it can still be removed and the upgrade performed over the console \ iLO as you need not let host boot again after removal, the steps I took were...

  1. Ensure Host to be upgraded is in Maintenance Mode
  2. Load iLO to the Host and attach remote ISO
  3. Enable SSH on the Host and use Putty to connect
  4. List all installed VIBs to locate the name of the the problem VIB using the command - esxcli software vib list  
  5. Uninstall the problem VIB using the following command - esxcli software vib remove --vibname="name of vib"
  6. Once removed enter "reboot" on the ssh session and hit enter, at this point switch to the iLO and do not allow the host to boot normally, hit F9 on the HP BIOS screen to select the ESXi iso mounted through the iLO or alternatively select the relevant physical media inserted in to the Host, this time having removed the incompatible VIB the upgrade then proceeded as normal.

*If the problem VIB does not affect or impact the drivers used by the management NIC(s) then the alternative method of solving this would be to remove the VIB as stated, reboot the host, and then proceed to upgrade as normal through vCenter with update manager, rather than using an iso direct to each host.

Thanks for reading

Adam