Tag: vmware

open-vm-tools on FreeBSD under VMware ESXi ARM Fling

vince4 Comments

Over the past week or so, I’ve been working with the VMware team to solve compatibility issues with open-vm-tools running under ARM. As of right now, I’m excited to report that initial tests are showing that open-vm-tools is now working great! However, these changes have not been pushed upstream yet either to the FreeBSD Ports collection or into the upstream open-vm-tools repository. There are active requests for this that are under review, but if you’d like to get up and running today, please follow along.

There are two different options to get open-vm-tools on FreeBSD for ARM right now. I’ve provided pre-compiled packages for a few key FreeBSD versions. Alternatively, you can compile them yourself from the custom ports that I’ve provided.

Pre-compiled Packages

Download the appropriate package for your FreeBSD version. Both open-vm-tools and open-vm-tools-nox11. The former is if you’re intending to have a full X11 graphical desktop, the latter for command line only systems such as servers.

Use one of the following commands for your version of FreeBSD.

# open-vm-tools-nox11 dependencies
pkg install -y libmspack libdnet fusefs-libs gettext-runtime glib


# FreeBSD 12.1-RELEASE (desktop)
pkg add https://vincerants.com/public/freebsd-arm/12.1-RELEASE/emulators/open-vm-tools-11.1.5.3%2C2.txz

# FreeBSD 12.1-RELEASE (no-X11)
pkg add https://vincerants.com/public/freebsd-arm/12.1-RELEASE/emulators/open-vm-tools-nox11-11.1.5.3%2C2.txz


#FreBSD 12.2-RC3 (desktop)
pkg add https://vincerants.com/public/freebsd-arm/12.2-RC3/emulators/open-vm-tools-11.1.5.3%2C2.txz

#FreBSD 12.2-RC3 (no-X11)
pkg add https://vincerants.com/public/freebsd-arm/12.2-RC3/emulators/open-vm-tools-nox11-11.1.5.3%2C2.txz


# FreeBSD 13.0-CURRENT 20201015 (desktop)
pkg add https://vincerants.com/public/freebsd-arm/13.0-20201015/emulators/open-vm-tools-11.1.5.3%2C2.txz

# FreeBSD 13.0-CURRENT 20201015 (no-X11)
pkg add https://vincerants.com/public/freebsd-arm/13.0-20201015/emulators/open-vm-tools-nox11-11.1.5.3%2C2.txz

Manually Compiling open-vm-tools

I’ve created a supplementary ports collection which has the needed patches to get open-vm-tools running on FreeBSD under VMware ESXi ARM Fling. Before running the following commands, ensure you have the latest version of pkg built and installed. The version bundled with FreeBSD is out of date.

# Update the version of pkg installed
cd /usr/ports/ports-mgmt/pkg
make
make reinstall

# Build and install open-vm-tools
mkdir /code
cd /code
git clone https://github.com/darkain/ports.git
cd /code/ports/emulators/open-vm-tools
make
make install

Enabling open-vm-tools

After installing open-vm-tools, the associated services need to be enabled. Add the following to your /etc/rc.conf file and reboot your virtual machine.

vmware_guest_vmblock_enable="YES"
vmware_guest_vmhgfs_enable="YES"
vmware_guest_vmmemctl_enable="YES"
vmware_guest_vmxnet_enable="YES"
vmware_guestd_enable="YES"

What Code Changes Were Needed

https://github.com/vmware/open-vm-tools/pull/474

This pull request documents the files that have changed. The custom ports collection listed above simply references these changes. Please feel free to review these changes and comment on the github pull request if you have more to add to make FreeBSD ARM support better!

FreeBSD on ESXi ARM Fling: Fixing Virtual Hardware

vince2 Comments

With the current state of FreeBSD on ARM in general, a number of hardware drivers are either set to not auto-load on boot, or are entirely missing altogether. This page is to document my findings with various bits of hardware, and if possible, list fixes.

USB 2.0 Controller (uhci)

https://www.freebsd.org/cgi/man.cgi?query=uhci&sektion=4

UPDATE 2020-10-22: Today’s FreeBSD snapshot ISO image now includes the uhci driver, meaning you no longer need to worry about which USB controller is selected in VMware (you still need at least one though, as the virtual keyboard is USB based)

If you switch the USB Controller for the FreeBSD virtual machine from the 3.1 controller to the 2.0 controller, you’ll lose all USB support. This also means that virtual USB devices like keyboard and mouse will stop functioning as well. To get the virtual USB 2.0 controller to function, add the following to your /boot/loader.conf file and reboot your FreeBSD VM.

uhci_load="YES"

NOTE: If you create the virtual machine through a remote connection on VMware Workstation, it will default to using the USB 2.0 controller rather than the USB 3.1 controller. Because of this, FreeBSD without the uhci driver will not have a functional keyboard. Change the controller type from USB 2.0 to USB 3.1 and reboot the virtual machine to get keyboard functionality again.

USB Virtual Mouse (ums)

https://www.freebsd.org/cgi/man.cgi?query=ums&sektion=4

The vmmouse driver, or virtual machine mouse driver, will not work by default due to missing the USB Mouse driver. To load the USB Mouse driver, add the following to your /boot/loader.conf file and reboot your FreeBSD VM.

ums_load="YES"

Virtual Network Card (vmxnet)

https://www.freebsd.org/cgi/man.cgi?query=vmx&sektion=4

UPDATE 2020-10-22: Today’s FreeBSD snapshot ISO image now includes the vmxnet driver pre-compiled. However, it is not yet loaded by default. You’ll still need to update your /boot/loader.conf

The precompiled vmxnet driver is not included with FreeBSD on ARM, however compiling it manually works without any modification to source code to make it work on ARM. To do this, you must ensure you either install the system source tree when installing FreeBSD, or download the source matching your FreeBSD version after install.

Next, compile and install the vmxnet driver.

cd /usr/src/sys/modules/vmware/vmxnet3
make
make install

After building and installing the vmxnet driver, add the following to your /boot/loader.conf file and reboot your FreeBSD VM to load the driver.

if_vmx_load="YES"

Alternatively, you can download the pre-compiled kernel module for any of the following FreeBSD versions. Place the if_vmx.ko driver file in /boot/modules/ and then follow the instructions above to for modifying /boot/loader.conf to load the driver at boot time.

Paravirtual SCSI Controller (pvscsi)

UPDATE 2020-10-22: Today’s FreeBSD snapshot ISO image now includes the pvscsi driver pre-compiled. However, it is not yet loaded by default. You’ll still need to update your /boot/loader.conf

The pvscsi driver is new to FreeBSD 13.0-CURRENT, so new in fact it doesn’t have a man page to link to yet. The driver, however, compiles and runs fine on ARM64, however I’ve yet to get it to work as the boot device controller. Currently the ARM64 UEFI inside of the virtual machine doesn’t appear to query the pvscsi controller on boot for potential boot devices.

The instructions are virtually the same as for the vmxnet driver above. Ensure that you have the system source tree option enabled when installing FreeBSD, and then do the following.

cd /usr/src/sys/modules/vmware/pvscsi
make
make install

After building and installing the pvscsi driver, add the following to your /boot/loader.conf file and reboot your FreeBSD VM to load the driver.

pvscsi_load="YES"

Alternatively, you can download the pre-compiled kernel module for any of the following FreeBSD versions. Place the if_vmx.ko driver file in /boot/modules/ and then follow the instructions above to for modifying /boot/loader.conf to load the driver at boot time.

Virtual Machine Communication Interface (vmci)

This driver currently contains i386/AMD64 specific assembly macros that I’ve yet to convert to ARM/aarch64. I also don’t have any way to test working vmci afterwards yet. So for the time being, this driver isn’t working.

CD-ROM (cd)

https://www.freebsd.org/cgi/man.cgi?query=cd&sektion=4

UPDATE 2020-10-16: Yesterday’s snapshot ISO now contains the cd-rom driver. With this, the ISO now boots and installs perfectly as expected. You can get the 2020-10-15 or newer ISO from the following URL. This driver, however, is missing from 12.2-RELEASE. It should be included in future 12-SNAPSHOT releases though!

CD-ROM support was added to the FreeBSD ARM64 GENERIC kernel a few days ago, however, that was after the most recent FreeBSD 13.0-CURRENT snapshot as of this blog post. The next snapshot should have CD-ROM support, meaning the next snapshot’s ISO installer should be bootable under ESXi ARM Fling.

https://download.freebsd.org/ftp/snapshots/arm64/aarch64/ISO-IMAGES/13.0/

FreeBSD under VMware ESXi on Arm Fling

vinceLeave a comment

UPDATE 2020-10-17

As of the FreeBSD 13.0-CURRENT snapshot from 2020-10-15, the installation ISO is now bootable and fully works to install FreeBSD on ARM just as you would do under VMware on i386 or AMD64.

Download the latest disk1.iso image from the following URL, load it into the virtual CD-ROM drive, and boot your new VM like normal. It should boot off of the ISO image and enter the FreeBSD installer as expected.

https://download.freebsd.org/ftp/snapshots/arm64/aarch64/ISO-IMAGES/13.0/

Additional information on getting individual drivers to work under FreeBSD on ARM can be found at the following link.

https://vincerants.com/freebsd-on-esxi-arm-fling-fixing-virtual-hardware/

Earlier this week, VMware released ESXi on Arm Fling, their hypervisor for the ARM platform. Here are instructions to get a FreeBSD virtual machine up and running under VMWare ESXi on Arm Fling.

These instruction assume you’ve followed VMware’s documentation on setting up the hypervisor on your ARM platform, and are familiar with the basics of how ESXi/vSphere functions.

For my hardware configuration, I’m using a Raspberry Pi 4 (8GB) with whatever old MicroSD card I could find for the UEFI firmware, and an equally old USB drive to install the hypervisor on to. The ESXi install is less than 200MiB. For actual VM storage, I am using an existing x86-64 iSCSI VMFS from my NAS.

These instructions follow the process of downloading the official ARM VMDK files from FreeBSD. I’ve as of yet to get the installer ISO to boot properly, so for the time being, this is how we’ll work. (see update above)

Both FreeBSD 12.1-RELEASE and 13.0-CURRENT will work for this process, though I personally recommend 13.0 as it has a more complete collection of pre-compiled 3rd part ports software.

Downloading and preparing the VMDK

Download the compressed VMDK file from one of the following URLs, and then extract it locally.

https://download.freebsd.org/ftp/releases/VM-IMAGES/12.1-RELEASE/aarch64/Latest/

https://download.freebsd.org/ftp/snapshots/VM-IMAGES/13.0-CURRENT/aarch64/Latest/

Use the Datastore browser to create a location to store the FreeBSD VMDK file.
Click on the Upload button and select the extracted VMDK file from your system to begin the upload process.
Take note of the upload progress in the top-right of the window. Due to the size of the VMDK file, it may take a few minutes to upload.
The VMDK file provided by FreeBSD is designed for VMware Workstation. We’ll need to convert it for use on ESXi. This is also a good time to expand the size of the VMDK, so we have room to install additional applications.
  1. Navigate to the folder where the VMDK file was uploaded. In my case, I have it on my tank volume inside of the arm-bsd-base-13 folder.
    • cd /vmfs/volumes/tank/arm-bsd-base-13
  2. Convert the VMDK file from Workstation to ESXi (optionally making it thin provisioned)
    • vmkfstools -i FreeBSD-13.0-CURRENT-arm64-aarch64.vmdk -d thin arm-freebsd.vmdk
  3. Expand the size of the newly created VMDK file (in my case, I’m making it 50GiB)
    • vmkfstools -X 50g arm-freebsd.vmdk

Creating a FreeBSD ARM virtual machine

Now we can start creating our new virtual machine like normal.
Enter the name for your new virtual machine. Select Other as the Guset OS family and then FreeBSD 12 or later version (64-bit) as the Guest OS version.
Select where you want to store this new virtual machine.
NOTE: As of this writing, anything higher than 1 for the CPU setting will prevent FreeBSD from booting.
Remove Hard disk 1, as we’ll be using our custom VMDK instead.
Select Add hard disk and then Existing hard disk.
Navigate the file system, and select our recently created VMDK file. Don’t worry about the reported disk size on this screen, as that doesn’t reflect the expanded size of the disk here.
Our virtual machine configuration is complete, we’re ready to click Finish!
Our virtual machine is now ready. We can interact with it in just about any way that you would expect from a standard ESXi install.
For the sake of safety, this would be a good time to take a virtual machine snapshot. This can be done by selection the Action menu, highlighting Snapshots and then select Take snapshot.
Enter a name and description for this snapshot. I chose “pre-boot” to let me know that this snapshot was taken before the first time I booted the guest FreeBSD operating system.
We can now power on our FreeBSD virtual machine. During the first boot process, FreeBSD will autodetect our new VMDK size and expand the UFS file system automatically.
Once fully booted, simply log in as the root user, no password needed.
NOTE: it would be wise to add a password at this point.