The latest vROps major version 6.6 introduced an interesting feature called Workload Balancing. The clusters in vSphere tend to get unbalanced as workloads mature, and this feature makes it possible to automate the movement of VMs between vSphere clusters and datastores using vROps metrics.
vRealize Automation 7.3 also has a new feature called Workload Placement, where it uses vROps to get metrics about the target environment and to make better decisions where to place a particular machine. Without vROps, vRA places the workloads to suitable reservations that can fulfil the machine request requirements. In case there are more than one options, vRA uses the priorities set in Reservations and datastores for placement. It doesn’t take the current CPU/memory load in to consideration. The new Workload Placement leverages the vROps feature mentioned earlier for more intelligent machine placement, but there are some major limitations. If you are using Storage Reservation Policies, things get tricky. Setting this up is relatively easy, but the documentation fails to mention everything. Let’s have a look at how the feature is configured and what the limitations are.
Start with getting vROps in order. The workload placement only works with vROps 6.6, so make sure you got the correct version before doing anything else. Log in to vROps and navigate to Environment -> Custom Groups. We want to make a custom group containing all the vRA managed ESXi hosts so we can attach a special policy to them. This is not absolutely necessary, but recommended.
Next step is to create the workload balancing policy. Go to Administration -> Policies -> Policy Library and create a new one. You can base that on the default policy. Change the Workload Balance settings as you see fit in your environment and attach the policy to the custom group created before. You could also use the default policy if you wanted to.
You will also need a vROps Management Pack for vRealize Automation v3.0. Install the pack from here. Note that the download package actually says v.2.3, but when you import it to vROps it shows as 3.0. Go figure.
vRA needs a user account with sufficient privileges to connect to vROps. It’s recommended to create a new account in AD and use it and not any default admin account. For guidance on the needed user rights, follow this doc. After the account is up and running, vROps is done for Workload Balancing. You may notice that the Rebalance and Scheduling buttons are greyed out in Workload Balance tab. That’s one of the limitations when working with vRA. Let’s look at those later on after the configuration is done.
Now that vROps is sorted, vRA needs a couple of steps as well. The Workload Placement needs to be enabled in Infrastructure -> Reservations -> Placement Policy.
The one thing the docs fail to mention is that you also need to configure a new vROps Endpoint in vRA. This is an easy step, but this is where you need the new AD account. Navigate to Infrastructure -> Endpoints -> Endpoints and the vROps Endpoint under Monitoring. One thing to note with vROps authentication is that the user accounts should be inserted in “user@domain@source” format where the source is the name of the AD server in vROps. While you are in there, you could also enable the Metrics collection in Administration -> Reclamation -> Metrics Provider. This step is not needed for the workload placement, but it’s a neat thing to have since you have everything configured anyway. More info here.
That’s it. To get any use out of Workload Placement, you need to have more than one reservation and more than one cluster and/or datastore. vROps will make the decision on which Reservation and datastore to deploy the VM to, and vSphere DRS will make the decision within a cluster. When you deploy a machine and you have more than one eligible Reservation, vRA asks vROps for metrics where to deploy the VM. If you have only one eligible Reservation with one datastore, vROps is not really needed. vRA defaults back to its original decision making method if vROps fails for some reason. This could happen if the recommended datastore does not have anymore free reserved capacity in the Reservation for instance.
To test this out, create two Reservations for a single Business Group with different vSphere Clusters. Then deploy a VM without specifying any datastore or network details (to make sure both Reservations are eligible for the deployment). Then go to the Request information and click Execution Information on the right hand corner. If vROps and vRA are functioning correctly, you should see a “Show Placement Details” link in the information table. Click on that and you will see how vROps made the decision call. If you can see Reservations with an exclamation mark next to them, those Reservations were not eligible for the deployment.
Limitations, there are a few. First of all, if you are running vSAN, game over. vSAN is not currently supported with this feature and same goes with Resource Pools. Also when using vRA, vROps won’t let you use the Rebalance function of clusters. vROps itself can balance the workloads between clusters automatically, but since we are using vRA, those clusters are tagged in vROps as vRA managed. This prevents the VM movement after they are created. This is good since vRA doesn’t like when machines are moved between Reservations. DRS needs to be in Fully Automated mode as well to make this work.
Now for the big one. Storage Reservation Policies do not work with the Workload Placement feature! I couldn’t find any reference for this in the documentation, but if you select an SRP during provisioning, it will be omitted completely. You can even select an SRP that is not part of the eligible reservations, and the machine gets still deployed without errors to vROps recommended datastore. It seems that vRA doesn’t check SRPs when it is using vROps for placement recommendations. vROps obviously has no idea what SRP is, so it will do its recommendations purely on capacity and load basis. vROps then passes those recommendations to vRA. In this particular case, vRA doesn’t care about SRPs and just follows the recommendations. Most customers seem to rely on SRPs to distinguish different storage tiers in their environment, and this bug/feature makes it quite difficult to use vROps for recommendations at this point. You could do a workaround by using just one type of storage in a single Reservation, but that is not how customers usually configure their vRA environment. Especially if you are an EHC customer, SRPs are used heavily. In case you have a single type of storage (i.e. AFA or FAST-VP enabled array) for the clusters, then you should be ok. Keep this in mind if you are planning to use the vROps integration with the current versions.