OpenStack Icehouse installation on top of ESXi – Part 1: Prep

I admit it, I’m a VMware guy. Nevertheless, OpenStack intrigues me. It also makes me crazy, but that’s part of the interest. I decided to learn OpenStack in more detail, so that meant installation time! I could have used dev-stack or VMware’s VOVA, but that’s just boring and too easy. Unfortunately I don’t have a bunch of physical server laying around, in fact I run a vSphere lab on top of a single rack server. I usually virtualize everything I want to test, and so far it has worked perfectly. So, I needed to find a way to deploy full scale OpenStack installation running on top of ESXi. I found a couple of things that the OpenStack installation manual for Ubuntu does not tell you (this was the part the made me crazy, among other things). KVM/QEMU and other hypervisors are not designed to be installed on top of another hypervisor, so that’s why there are issues.

If you haven’t installed OpenStack before, prepare for LOTS of typing and fat fingering. Obviously you should use automation tools like Puppet to do the dirty work for you, but everybody needs to know how to do this manually in case something goes south. I’m not going to go through the details of OpenStack architecture, but you can find some great resources online. The install guide is a good start.

Prep your environment. There are a couple of things to do before we can start building the servers. First of all, you need two networks for OpenStack, one for management and one for VMs (or Instances as they call them in OpenStack). You could have more, for example one additional network for storage traffic, but I’ve used only two.
vm-networksFor the network you are going to use for VM traffic, there’s an additional step. Remember to set the vSwitch or vDS Port Group to Promiscuous mode! I spent a couple of days redoing all the settings I had done to troubleshoot my networking issues. I was certain that the bridge inside Compute node caused all the issues, but it was in fact the vSwitch security settings. If promiscuous mode is disabled, the vSwitch will drop packets that do not belong to a particular port that is receiving them. This caused networking issues between the Controller node and Compute node (VMs). Drove me crazy, again.
promiscuous-modeNow we can start to build the servers. Before actually installing anything, you need to enable hardware virtualization for the empty VMware VMs that act as physical OpenStack nodes. Without it, you cannot run nested VMs. If you’re running ESXi 5.1 or later, just enable HW virtualization from VM settings with Web Client. C# client does not work here. On older ESXi versions it’s a bit more complicated, but still easy. Check the instructions out here.
hw-virtualizationBuilding a template for OpenStack base machine is not a bad idea. I’ve used Ubuntu 14.04, but there are plenty of other options. Get the 14.04 Server ISO from Ubuntu, Create and empty VM and do the steps above and install your server. 1 vCPU, 2 GB of RAM and about 100GB of (thin) storage is enough for our testing. During the Ubuntu installation, click the OpenSSH software option, it helps later on.
openstack-vmsAfter installation, login and enable root:

sudo passwd root
su -

It’s recommended to update the server to latest packages and install Ubuntu Desktop. Having a GUI makes trivial things like copy-paste a bit more enjoyable:

apt-get update
apt-get -y install ubuntu-desktop

There are some mandatory software and a bunch of useful software that you can use on the nodes, especially when you need to troubleshoot. Here’s what you need: 

apt-get install kvm qemu-kvm
apt-get install ntp python-pip

Install Ubuntu Cloud Archive for icehouse:

apt-get install python-software-properties
add-apt-repository cloud-archive:icehouse

Let’s update the server once more to be certain:

apt-get update
apt-get dist-upgrade

Additionally, I would recommend network troubleshooting software, i.e. wireshark, to be installed:

apt-get install wireshark

Done! Now shut down the VM and convert to a template. Next step, Nova!


One thought on “OpenStack Icehouse installation on top of ESXi – Part 1: Prep

  1. Pingback: OpenStack Juno Lab installation on top of VMware Workstation – Prep + Nova + Compute1 | Virtual YoYo

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s