Funtoo:User Services/VMware Overview

Funtoo Linux is starting to use some VMware products for internal services. This page is here to document various general things related to VMware products.

ESXi vs vSphere vs vCenter

You have probably heard of VMware ESXi -- the free (as in beer) version of VMware's hypervisor. VMware has started to transition to a new name which is VMware vSphere Hypervisor. Sometimes you may see this listed as "VMware vSphere Hypervisor (ESXi)".

VMware's product naming can be confusing, so this change is probably a good thing. You may have also heard of something called "vCenter". Let's look at how ESXi, vSphere and vCenter relate to each other. Think of "vSphere" as an umbrella, which contains the following components:

• vSphere
  • Hypervisor (aka ESXi, free license available)
  • vCenter (Commercial, more powerful WebUI, and requirement for higher-end VMware functionality)

User Interfaces

VMware vSphere Hypervisor (aka ESXi) provides its own web user interface, which has basic functionality for managing virtual machines on that particular host. In contrast, vCenter provides a more feature-rich web user interface and can manage multiple ESXi hosts and configure them as highly-available clusters. The catch is that vCenter requires a paid license, whereas the ESXi web interface can be used with a 'free' license. vCenter provides many conveniences for managing a virtual environment, and also serves as a building block for VMware's other commercial offerings.

ESXi Deployment

When deploying ESXi (aka 'vSphere Hypervisor'), it's strongly recommended to secure the management port using a VPN such as WireGuard VPN. It's also highly recommended to enable the NTP service, and deploy a DNS server for the management interface, for both resolution of local VMs that may be deployed, as well as allowing ESXi to resolve the NTP server. For vCenter, an internal DNS server is required, whereas for ESXi it is only highly recommended.

ESXi Upgrades

   Note

The method listed in this and the next section require SSH access to ESXi, which must be manually enabled via the Web UI first. Go to Manage -> Services -> TSM-SSH -> Actions -> Start, and then you can also set policy to start with host if desired. You can then ssh using the root account and the root ESXi password.

The easiest way to perform ESXi upgrades is via the ssh console. This command can be run to browse for the latest updates to ESXi:

user $ ssh root@esxi-host.mgmt
[root@localhost:~] # esxcli software sources profile list --depot=https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml
Name                              Vendor        Acceptance Level  Creation Time        Modification Time
--------------------------------  ------------  ----------------  -------------------  -----------------
...
ESXi-7.0U2c-18426014-standard     VMware, Inc.  PartnerSupported  2021-08-24T00:00:00  2021-08-24T00:00:00
ESXi-7.0U2c-18426014-no-tools     VMware, Inc.  PartnerSupported  2021-08-24T00:00:00  2021-08-04T11:40:25
ESXi-7.0U2d-18538813-standard     VMware, Inc.  PartnerSupported  2021-09-14T00:00:00  2021-09-14T00:00:00
ESXi-7.0U2d-18538813-no-tools     VMware, Inc.  PartnerSupported  2021-09-14T00:00:00  2021-08-27T10:33:50
ESXi-7.0U2e-19290878-standard     VMware, Inc.  PartnerSupported  2022-02-15T00:00:00  2022-02-15T00:00:00
ESXi-7.0U2e-19290878-no-tools     VMware, Inc.  PartnerSupported  2022-02-15T00:00:00  2022-01-31T07:40:31
ESXi-7.0U3c-19193900-standard     VMware, Inc.  PartnerSupported  2022-01-18T00:00:00  2022-01-18T00:00:00
ESXi-7.0U3c-19193900-no-tools     VMware, Inc.  PartnerSupported  2022-01-18T00:00:00  2022-01-12T00:03:42

This command can take up to a minute or so to produce output, but will show an exhaustive list of updates that can be installed. Some of these updates require that the ESXi server be placed into "maintenance mode" (via web UI) prior to the update being applied.

To apply a specific update, a command like the following can be run:

   Note

Name resolution and outbound Internet access from the management network must be functioning for this to work. The "standard" upgrade is recommended.

[root@localhost:~] # esxcli software profile update -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml -p ESXi-7.0U2d-18538813-standard

Maintenance Mode -- Enter and Exit

For most updates, you will be prompted to put the ESXi host into maintenance mode, which can be done through the UI and will require stopping all virtual machines, or if you have another host, migrating them to another host. This is done in the UI via Navigator -> Host -> Actions -> Enter Maintenance Mode. Once this is done, you can re-run the above command and it will work.

Once complete, you will see a bunch of output, and if you scroll up you will likely see a message indicating that the upgrade was successful, but is not yet active and requires the ESXi host to be restarted. This can be easily done via the command-line:

[root@localhost:~] # reboot

Once ESXi is restarted and running the new version, you will need to exit maintenance mode before you will be allowed to start any VM's. Do this via Navigator -> Host -> Actions -> Exit Maintenance Mode. Congratulations! Your ESXi server is back up and running.

ESXi Upgrade Using Offline Bundle

   Note

This method does not require Internet access to be enabled for your ESXi host.

Sometimes, it is inconvenient or impossible to use the online software depot -- or it might be very slow -- and copying an offline bundle to the ESXi host and using it locally is much easier. Here's how that works. First, you will want to download an ESXi offline bundle from customerconnect.vmware.com. This is offered on the standard ESXi download page, next to the ISO. You will then want to use the scp command to copy it to an ESXi datastore, or use the ESXi Web UI to upload the bundle to a datastore (note that copying it to root's home directory generally won't work due to lack of available space.)

Once this is done, you can use this offline bundle just like a mini online software depot:

[root@localhost:~] # esxcli software sources profile list -d /vmfs/volumes/633f4349-c484103e-6df7-000af77a6554/VMware-ESXi-8.0-20513097-depot.zip 
Name                          Vendor        Acceptance Level  Creation Time        Modification Time
----------------------------  ------------  ----------------  -------------------  -----------------
ESXi-8.0.0-20513097-standard  VMware, Inc.  PartnerSupported  2022-09-23T18:59:28  2022-09-23T18:59:28
ESXi-8.0.0-20513097-no-tools  VMware, Inc.  PartnerSupported  2022-09-23T18:59:28  2022-09-23T18:59:28

...and...

[root@localhost:~] # esxcli software profile update -p ESXi-8.0.0-20513097-standard -d /vmfs/volumes/633f4349-c484103e-6df7-000af77a6554/VMware-ESXi-8.0-20513097-depot.zip

Deploying vCenter via VCSA

As of version 7, vCenter is deployed using the vCenter Server Applicance, also called "VCSA". This is a downloadable ISO that is used to install a new VM which will house the appliance. You will need to have at least one vSphere Hypervisor (aka ESXi) host set up to deploy the VCSA. VCSA deployment notes can be found here: https://docs.vmware.com/en/VMware-vSphere/7.0/vsphere-vcenter-server-70-installation-guide.pdf

VMware Licensing: The "Processor"

Commericial VMware products require licenses. These licenses center around the definition of "processor". In VMware terminology, a "processor" is a physical CPU (one per socket), capped at 16 physical cores.

Let's look at an example of how this plays out in reality. Assume you have a vSphere/vCenter "32 Processor" license, and you have one server. This server has two of the following physical CPUs in it:

Intel(R) Xeon(R) Gold 6230 CPU @ 2.10GHz

Each of these processors has 20 physical cores, and 40 hyper-threaded cores. So -- is the "32 Processor" license sufficient for your purposes?

It turns out that the answer is "yes, it is more than enough." Let's do the math and see how this licensing works. We have two physical processors in our server, but they are more than 16 physical cores each. A license for Intel CPU socket 1 is covered by a single VMware processor license, but leaving 4 remaining cores since each VMware license only covers up to 16 per license. Another license for Intel CPU socket 2 is covered by a single VMware processor license, leaving another remaining 4 cores uncovered. We can then cover these leftover "extra" cores from each Intel Processor -- 8 physical cores in total -- with a third single processor license. This means that our single 40-core/80-thread server just used up 3 of our 32 Processor licenses for vSphere/vCenter.

LXD Compatibility

If you are planning to deploy LXD on a virtual machine, then you will likely be using Linux bridging for your containers. If you are mapping this bridge into a VMware virtual switch, then you will need to enable "promiscuous mode" on the VMware virtual switch in order for the bridging on the Linux VM side to work properly. Once this is done, any LXD containers bridged into the vSwitch will be able to communicate with other devices outside the VM itself as expected.