I admit it, I’m a VMware guy. Nevertheless, OpenStack intrigues me. It also makes me crazy, but that’s part of the interest. I decided to learn OpenStack in more detail, so that meant installation time! I could have used dev-stack or VMware’s VOVA, but that’s just boring and too easy. Unfortunately I don’t have a bunch of physical server laying around, in fact I run a vSphere lab on top of a single rack server. I usually virtualize everything I want to test, and so far it has worked perfectly. So, I needed to find a way to deploy full scale OpenStack installation running on top of ESXi. I found a couple of things that the OpenStack installation manual for Ubuntu does not tell you (this was the part the made me crazy, among other things). KVM/QEMU and other hypervisors are not designed to be installed on top of another hypervisor, so that’s why there are issues.
If you haven’t installed OpenStack before, prepare for LOTS of typing and fat fingering. Obviously you should use automation tools like Puppet to do the dirty work for you, but everybody needs to know how to do this manually in case something goes south. I’m not going to go through the details of OpenStack architecture, but you can find some great resources online. The install guide is a good start.
Prep your environment. There are a couple of things to do before we can start building the servers. First of all, you need two networks for OpenStack, one for management and one for VMs (or Instances as they call them in OpenStack). You could have more, for example one additional network for storage traffic, but I’ve used only two.
For the network you are going to use for VM traffic, there’s an additional step. Remember to set the vSwitch or vDS Port Group to Promiscuous mode! I spent a couple of days redoing all the settings I had done to troubleshoot my networking issues. I was certain that the bridge inside Compute node caused all the issues, but it was in fact the vSwitch security settings. If promiscuous mode is disabled, the vSwitch will drop packets that do not belong to a particular port that is receiving them. This caused networking issues between the Controller node and Compute node (VMs). Drove me crazy, again.
Now we can start to build the servers. Before actually installing anything, you need to enable hardware virtualization for the empty VMware VMs that act as physical OpenStack nodes. Without it, you cannot run nested VMs. If you’re running ESXi 5.1 or later, just enable HW virtualization from VM settings with Web Client. C# client does not work here. On older ESXi versions it’s a bit more complicated, but still easy. Check the instructions out here.
Building a template for OpenStack base machine is not a bad idea. I’ve used Ubuntu 14.04, but there are plenty of other options. Get the 14.04 Server ISO from Ubuntu, Create and empty VM and do the steps above and install your server. 1 vCPU, 2 GB of RAM and about 100GB of (thin) storage is enough for our testing. During the Ubuntu installation, click the OpenSSH software option, it helps later on.
After installation, login and enable root:
sudo passwd root
It’s recommended to update the server to latest packages and install Ubuntu Desktop. Having a GUI makes trivial things like copy-paste a bit more enjoyable:
apt-get -y install ubuntu-desktop
There are some mandatory software and a bunch of useful software that you can use on the nodes, especially when you need to troubleshoot. Here’s what you need:
apt-get install kvm qemu-kvm
apt-get install ntp python-pip
Install Ubuntu Cloud Archive for icehouse:
apt-get install python-software-properties
Let’s update the server once more to be certain:
Additionally, I would recommend network troubleshooting software, i.e. wireshark, to be installed:
apt-get install wireshark
Done! Now shut down the VM and convert to a template. Next step, Nova!