VMware ESX 4 can even virtualize itself

NEW: VMware vSphere 5 makes this even easier and supports nested 64-bit guests.

Running VMware ESX inside a virtual machine is a great way to experiment with different configurations and features without building out a whole lab full of hardware and storage. It is pretty common to do this on VMware Workstation nowadays — the first public documentation of this process that I know of was published by Xtravirt a couple of years ago.

But what if you prefer to run ESX on ESX instead of Workstation?

You may be pleased to know that the GA build of ESX 4 allows installing ESX 4 as a virtual machine as well as powering on nested virtual machines — VMs running on the virtual ESX host.  You can even VMotion a running virtual machine from the physical ESX to a virtual ESX — on the same physical server!

VMware vSphere 4.1 UPDATE: VMware ESXi 4.1 has a keyboard issue when virtualized on an ESX 4.0 host.  In order to virtualize ESXi 4.1, the underlying host must be 4.1.  However, ESX 4.1 classic will work on ESX 4.0.

The extra tweaks to make it all work are minimal, and I will show you how without even opening up a text editor.

After installing ESX 4 onto your real hardware, configure as desired and enable promiscuous mode on a vSwitch:

Create a new VM with the following guidance (choose “Custom”):

• Virtual Machine Version 7
• Guest OS: Linux / Red Hat Enterprise Linux 5 (64-bit)
• 2 VCPUs, 2GB RAM
• 2 NICs – e1000
• LSI Logic Parallel
• New disk – reasonable size greater than 10GB for ESX classic

After you have the VM ready, simply attach a VMware ESX 4 ISO image, power on, and install ESX as a guest OS.

After installation, add the new virtual ESX to vCenter 4 and create a new VM.

If you do not need to run VMs on your virtual ESX, you can stop there.  However, if you try to power on that nested VM, you will see the following error:

To prevent this, just one tweak is needed:

• Shut down the virtual ESX VM
• Click Edit Settings
• Click the Options tab
• Click Advanced / General / Configuration Parameters…
• Click Add Row
• For the Name/Value enter: monitor_control.restrict_backdoor / TRUE

The above procedure is just an alternative to hand-editing the .vmx file — if you prefer to do it that way, feel free.

Now you are ready to power your virtual ESX VM back on, as well as the nested VMs.  This capability should come in handy as you start investigating the new features of vSphere 4.

UPDATE:  In order to take snapshots of a VM running ESX, a quick configuration change is necessary on the physical host .

You may be interested in this related post where a VM is migrated between the physical and virtual ESX hosts.

Related posts:

1. VMware vSphere can virtualize itself + 64-bit nested guests
2. VMotion from physical ESX 4 to virtual ESX 4
3. Taking snapshots of VMware ESX 4 running in a VM
4. Pentacore CPUs in VMware ESX 4