Install and Configure vRealize Operations Manager 8.2 Part 7 Configure Custom Certificate

In the previous six post’s we went through installing and configuring the vROps virtual appliance, connecting to vCenter server, configuring Window Active directory as an identity source, create custom alerts and notifications, creating dashboards and upgrading the appliance to the latest version.

Part 1: Install and Configure vRealize Operations Manager 8.2 Part 1 – TheSleepyAdmins

Part 2: Install and Configure vRealize Operations Manager 8.2 Part 2 Connect to vCenter – TheSleepyAdmins

Part 3: Install and Configure vRealize Operations Manager 8.2 Part 3 AD Authentication – TheSleepyAdmins

Part 4: Install and Configure vRealize Operations Manager 8.2 Part 4 Create Alerts and Notifications – TheSleepyAdmins

Part 5: Install and Configure vRealize Operations Manager 8.2 Part 5 Create a Dashboard – TheSleepyAdmins

Part 6: Install and Configure vRealize Operations Manager 8.2 Part 6 Upgrading vROps – TheSleepyAdmins

In this post we will be going through requesting and applying a custom certificate. Configuring a custom cert is good practice from security standpoint and also will stop the security warning when access the vRealize web client.

Adding a certificate requires that there is a internal certificate authority that can be used to issue the certificate or we could use a public CA but there would be a cost to that, in this example we will be using a Windows Server CA.

I used the below VMware kb as reference when creating the cert.

Enabling TLS on Localhost Connections (vmware.com)

Configure a Custom Certificate (vmware.com)

First step is to connect to the vROps appliance using ssh connection and to generate the key file and cert request that will be used to generate the cert.

To enabled ssh go to the admin page and enable the ssh status.

If you have not updated the root password on the appliance already then this require to connect by ssh. To do this open a VM console for the appliance and go to login. The default root password is blank so just hit enter and it will prompt for a new password to be set.

Once the above has been completed, ssh to the vrops server I use putty but any ssh client will work.

After connecting I usually create a folder to keep the key file and cert request to they are simpler to find later if I need them again.

Next we need to generate a key file

openssl genrsa -out key_filename.key 2048

Next run the below command to create the certificate request

openssl req -new -key key_filename.key -out certificate_request.csr

Enter in the details for the cert. These can also be pre creating using a .config file but I just typed them in to the ssh console.

There should now be a key file and cert request in the folder.

Copy the .csr file as this will be used to generate the cert from the internal CA.

To generate the certificate logon on to the Microsoft CA web enrollment page.

Click submit and advanced certificate request.

Click submit a certificate request

Open the .csr file in a text editor and copy the content to certificate request box and select the certificate template to be used.

Click submit and the certificate should be generated. The cert needs to be downloaded as base 64.

Save the cert. The root CA cert also needs to be downloaded

Once all the cert files and key file are created, they now need to be combined to a .PEM format as that is the required format for vrops.

To combine the cert using Windows using the type command. The order the of the cert needs to be server cert, then key file, intermediate cert (if there are any in my case I only have the root cert) , root cert and then the PEM output.

type server_cert.cer key_filename.key cacerts.cer > vrops.pem

The .PEM file should now be created and is ready to be applied to vROps.

The last step is to apply the certificate, logon to the vROps admin page and go to the certificate icon in the top right.

Click Install new certificate.

Click browse and select the pem file we created. If there are no issue with verifying the pem file it should show as ready to install.

Click install to complete. The page should now reload and when we check the cert it should now be using the custom cert.

In the next post we will go through installing the Windows vROps agent and configuring the Windows management pack.

Install and Configure vRealize Operations Manager 8.2 Part 6 Upgrading vROps

In the previous five post’s we went through installing and configuring the vROps virtual appliance, connecting to vCenter server, configuring Window Active directory as an identity source, create custom alerts and notifications and creating dashboards.

Part 1: Install and Configure vRealize Operations Manager 8.2 Part 1 – TheSleepyAdmins

Part 2: Install and Configure vRealize Operations Manager 8.2 Part 2 Connect to vCenter – TheSleepyAdmins

Part 3: Install and Configure vRealize Operations Manager 8.2 Part 3 AD Authentication – TheSleepyAdmins

Part 4: Install and Configure vRealize Operations Manager 8.2 Part 4 Create Alerts and Notifications – TheSleepyAdmins

Part 5: Install and Configure vRealize Operations Manager 8.2 Part 5 Create a Dashboard – TheSleepyAdmins

In this post we will be going through upgrading the vROps appliance to the latest version 8.3. Keeping appliances up to date is good from a security standpoint and also allows for new features and fixes to be applied.

To confirm if the upgraded version of vROps is supported by the existing vCenter version use the VMware interoperability matrix site.

Product Interoperability Matrix (vmware.com)

Once this has been confirmed as supported we will need to download the .pak file that will be used to update the appliance.

To download go the following URL

https://my.vmware.com/en/group/vmware/downloads/info/slug/infrastructure_operations_management/vmware_vrealize_operations/8_3

Select the version and edition required in my case it Enterprise

Select upgrade type we will be upgrading from 8.2 so we will use 8.x, there is also a upgrade assessment tool that can be run to do a pre check.

Once the .pak files has been downloaded we are ready to start the upgrade. We will run the assessment first and view the report then process with the full upgrade.

To access the admin page use /admin at the end of the vROps URl. We can check the current version on the system status page.

https://vropsserver/admin

Go to software update and click install a software update.

Click browse and select the .pak file and click upload

Once the file is uploaded click next

Accept the end user agreement.

Next you can review the update info and the last screen is to install.

The installation should now start, it took about 10 minute to complete.

This image has an empty alt attribute; its file name is image-89.png

Once the assessment is completed, we need to go to the support tab and select support bundles. Click on the download icon to download the zip file.

To view the report click the download and extract the zip file go to \apuat-data\report and click on the index.html.

This image has an empty alt attribute; its file name is image-91.png

If there are no issue reported in the pre assessment we can go ahead with the upgrade.

It is recommend to take a backup of the appliance before upgrading in case there are any issue during the install.

First part of the upgrade is to take the cluster offline, go to the system status page

Once the cluster is offline we can uploaded and apply the .pak file similar to the assessment tool.

Go to software update > Install a software update, select the upgrade pak file and upload.

Follow the wizard to start the upgrade.

The upgrade took 30 minutes to complete and the cluster to come back online. Once fully back online we can confirm that the version is now upgraded to 8.3.

In the next post we will go thorough configuring a custom SSL cert to replace the self singed cert to give better security.

Install and Configure vRealize Operations Manager 8.2 Part 5 Create a Dashboard

In the previous four post’s we went through installing and configuring the vROps virtual appliance, connecting to vCenter server, configuring Window Active directory as an identity source and create custom alerts and notifications.

Part 1: Install and Configure vRealize Operations Manager 8.2 Part 1 – TheSleepyAdmins

Part 2: Install and Configure vRealize Operations Manager 8.2 Part 2 Connect to vCenter – TheSleepyAdmins

Part 3: Install and Configure vRealize Operations Manager 8.2 Part 3 AD Authentication – TheSleepyAdmins

Part 4: Install and Configure vRealize Operations Manager 8.2 Part 4 Create Alerts and Notifications – TheSleepyAdmins

In this post we will be creating a custom dashboard. Dashboard can be used to visual your VMware infrastructure, view performance issue and capacity planning / right sizing. Dashboards are made up of views and widgets.

We can create custom dashboards or clone and existing dashboard and modify that to add addtional view or widgets.

To create a new dashboard logon to the vROps web client > Dashboards > Dashboards > Create Dashboard

Give the dashboard a name and we can toggle between view and widgets using the below button.

Start adding the required views / widgets for the dashboard in this dashboard we are looking for performance related.

First we will add a view list that we will use to create an relationship with the other widgets.

Give the view a name and select if the content should be refreshed or not. Since I want to specify the cluster I will be setting self provider to On. If this is not set to on input data object will be greyed out

Next we need to specify the object under the inventory tree. We will be using vSphere Host and Clusters and the object will be vSphere World.

Under output data we will use cluster utilization.

Click save and output should look like the below.

Once we have the view we can add the addtional widgets and start creating the interactions.

No data will be showing till the Interactions are in place.

After adding in the required widgets and click on show interactions.

Now we just connect the LAB_Cluster view to the other widgets we just added.

Now if we select the cluster view we will have alert volume, health data, scoreboard and object relationship information returned.

If we want to share the dashboard with other users we can select share icon

Click on groups and select the group to be included.

Now share icon should show beside the name of the dashboard.

If there are pre existing dashboard that we want to customize we can clone these by going to Dashboards >> Manage Dashboards, select the dashboard you want to clone, click on the three dots and select clone.

Give the dashboard a name and we can then start to modify.

In the next post we will go through scheduling reports and updating vROps appliance

Install and Configure vRealize Operations Manager 8.2 Part 4 Create Alerts and Notifications

In the previous three post we went through installing and configuring the vROPs virtual appliance, connecting to vCenter server and configuring Window Active directory as an identity source.

Part 1: Install and Configure vRealize Operations Manager 8.2 Part 1 – TheSleepyAdmins

Part 2: Install and Configure vRealize Operations Manager 8.2 Part 2 Connect to vCenter – TheSleepyAdmins

Part 3: Install and Configure vRealize Operations Manager 8.2 Part 3 AD Authentication – TheSleepyAdmins

In this post we will be going through the different alert types and configuring actions bases on certain alerts.

Alerts:

Below are the three types of alert in vROPs:
Health Alerts:
The health alert list is all the generated alerts that are configured to affect the health of your
environment and require immediate attention. You use the health alert list to evaluate,
prioritize, and immediately begin resolving the problems.


Risk Alerts:
The risk alerts list is all the generated alerts that are configured to indicate risk in your
environment. Address risk alerts in the near future, before the triggering symptoms that
generated the alert negatively affect the health of your environment.


Efficiency Alerts:
The efficiency alerts list is all the generated alerts that are configured to indicate problems
with the efficient use of your monitored objects in your environment. Address efficiency
alerts to reclaim wasted space or to improve the performance of objects in your environment.

Each alert type has four different severity types, info, warning, immediate and critical. The can all be configured

To create a custom alert logon to vROPs web client > alerts > Alert Definitions

Click Add, give the alert a name

We will be using virtual machine so we will

select base object type and select vCenter Adapter > virtual Machine

We want to alert on Capacity so click on advanced.

Impact = Health

Criticality = Symptom based

Alert Type & Subtype = Virtualization/Hypervisor : Capacity

Next we need to add the a symptom that will be used to trigger the alert. If there is no symptom that matches what we want to alert on we can create a new symptom. For snapshot there is only greater than 2 days so we will create a new symptom.

We will be using Virtual Machine: Disk space > Snapshot > Age (Days) and set the value to greater than 5 days.

We can set a recommendation that already exists to not keep snapshots over 72 hours or create a custom recommendation.

We can also apply a policy we will use the default policy.

Complete the wizard to create the new alert.

Next we will configure an email notification instance to allow alerts to be emailed.

Go to Administration > Outbound Settings > Add

Added in the email servers settings.

Click test to validate the mail flow is working.

Click save to to complete.

Once we have the outbound email instance configured, we can setup alerts to send emails notifications.

I created a new alert for Powered off VM so it would be easier to get a alert to trigger to test the email notification.

Go to Alerts > Notification

Next add in the details and select the email instance we setup earlier. I will be alerting for when LAB-Linux01 is powered off so will use object and specify the VM name and alert definition.

Click save to create the notification.

Now once the VM is powered off we will get an notification.

In the next post we will be going through creating a dashboard.

Install and Configure vRealize Operations Manager 8.2 Part 3 AD Authentication

In the previous post in this series we went through installing vROps virtual appliance and connecting to vCenter. In this post we will go through adding an AD authentication source and configuring access groups.

Part 1: Install and Configure vRealize Operations Manager 8.2 Part 1 – TheSleepyAdmins

Part 2: Install and Configure vRealize Operations Manager 8.2 Part 2 Connect to vCenter – TheSleepyAdmins

There are 5 different authentication sources that can be added to vROps.

  • SSO SAML: An XML-based standard for a web browser single sign-on that enables users to perform single sign-on to multiple applications.
  • VMware Identity Manager: A platform where you can manage users and groups, manage resources and user authentication, and access policies and entitle users to resources.
  • Open LDAP: A platform-independent protocol that provides access to an LDAP database on another machine to import user accounts.
  • Active Directory: Specifies the use on Active directory to be used to import users accounts or groups.
  • Other: Specifies any other LDAP-based directory services, such as Novel or OpenDJ, used to import user accounts from an LDAP database on a Linux Mac machine.

First we need to logon to the vROps web client > Administration > Authentication Sources

Click Add and select the source type required. We use Microsoft AD so we will be using Active Directory.

Give the identity source a display name I usually use the domain name as this make it simpler when view settings. Use basic as this auto-discovers the DC and DN (Distinguished Name).

Add the user account that will be used to for the LDAP connections to the domain. This account should only need to have domain users rights.

I also always create a specific service account to be uses for each application AD integration. I would also recommend using SSL/TLS where possible as this will encrypt the LDAP requests between the appliance and the domain controller.

Click on details to view the auto discovered host and

Click test verify all settings are correct, if set to use SSL there will be a prompt to accept the certificate.

Once the test is successful we can complete adding the authentication source.

This image has an empty alt attribute; its file name is image-54.png

Once completed the AD source should show.

This image has an empty alt attribute; its file name is image-55.png

Next we will configure the groups in AD that will be used to assign access roles in vROps.

To add the groups they need to be imported from AD and then assign the required role.

Go to Administrator > Access Control > import

Use the search string to check for the groups.

Select the role that will be assigned to the group

Assign the other required roles and select the object that are required for the group.

To test we can open a new session and select the AD authentication source instead of local user.

We can check the domain controller security event logs to confirm the authentication.

Based on the roles assinged the user will only have limited access.

In the next post we will go through configure alerting and create some capacity planning reports that can be used to plan for future compute requirements.

Install and Configure vRealize Operations Manager 8.2 Part 2 Connect to vCenter

In part one of the blog series on installing and configuring vROps we deployed the virtual appliance. In this post we will be adding our vCenter server to vRops.

Part 1: Install and Configure vRealize Operations Manager 8.2 Part 1 – TheSleepyAdmins

There are a few different types of accounts that can be added.

  • vCenter
  • VMC
  • AWS
  • Microsoft Azure

To add vCenter we need to logon to the vROps web client and go to Administration > Cloud Accoutns > Add Account

Select the account type for vCenter

Give the cloud Account a name, description and the vCenter DNS address and a logon credentials.

Click the validate connect to confirm the details are correct. If the certificate is not trusted you will be asked to review and confirm the certificate.

Once successfully completed we can then added vCenter.

The connection will now be setup and once completed will show under cloud accounts.

To view if information on the vCenter server is being collected we can go to Environment > vSphere Hosts and Clusters > vSphere World.

vROps can take a little time before metrics and alert start to show.

In the next post we will go through configuring AD Authentication and configuring group based access control.

Install and Configure vRealize Operations Manager 8.2 Part 1

In the next set of post’s we will be going through installing and configure vRealize Operations manager (vROps). I haven’t had to install or configure vROps in a few years so want to go back over it before we replaced our existing deployment.

vROps is a application from VMware that can be used to monitor, optimize and manage VMware management tools like vCentre, ESXi..

There are 3 different editions of vROps.

Standard: Allows management of vSphere only.

Advanced: Adding VMware cloud (AWS / Azure), Operating system monitoring and dashboards.

Enterprise: Give all the advanced features but also allows for application / database monitoring and third party management packs.

vROps Editions: Series Overview – VMware Cloud Management

We will be using Enterpirse edition.

vRops can be used for performance monitoring, over or under provisioned VM’s, capacity planning and trend analysis.

In this post we will be going through the initial virtual appliance deployment.

First step is to check what size appliance is going to be required.

We can use the sizing guidelines to select the right appliance size for the environment.

vRealize Operations 8.2 Sizing Guidelines (80893) (vmware.com)

Or use the VMware sizing tool

vRealize Sizing Tool (vmware.com)

Select the version you are installing and then add in the number of vCenter, host, datastores and VM that will have data collected and this will then give you the recommended sizing for your vROps deployment.

In my case it was extra small deployment.

What I always do before deploying any VMware appliance is create a static DNS record. This makes it easier to connect to the appliance after it’s deployed and for some appliance (like vCenter server its a requirement or the deployment will fail.)

This image has an empty alt attribute; its file name is image-24.png

To download the required vROps appliance go to my VMware and select the required version.

Download VMware vSphere – My VMware

To deploy the OVA create a new VM in VMware and select deploy VM from OVF or OVA file.

Give the appliance a Name and either drag and drop the OVA file or browse to the location and select.

Select a datastore

Agree to the end user license agreement.

Select a network, deployment type size, disk provisioning (thin or thick) and if VMware will be powered on automatically. Since this is only a single vCenter setup we will be using a small deployment type.

Set the timezone and network IP, gateway, netmask and domain name

Review the settings and complete.

The VM will start to deploy.

Once the deployment is completed, connect to either the IP or FQDN of the appliance to start the setup.

Select either express or new installation. We will be using the express installation as we only have one vCenter.

Set the admin password.

Complete the install

When the deployment completed the vROps logon page should show.

Logon and completed the installation.

Accept the End user agreement.

Enter your product key or use the evaluation.

You can join customer experience or untick to not take part.

Click finish to complete.

vROps is now installed.

In the next post we will go through connecting to vCenter Server, configure Active directory integration and build out some dashboards.

VMware PowerCLI Integrated Authentication Issues

Recently we have been having an issue with VMware PowerCLI not passing through the users credentials when running Connect-viserver to connect to our vCenter servers.

This has been causing problems when trying to use scheduled task to automated reports and run remediation task like removing old snapshots or reporting on VMware Tools versions as it is prompting for credentials.

For integrated authentication to work, the vCenter servers needs to be setup to allow single sign on for the domain that you will be connecting from, so confirm that your Active Directory Identity source is added and that SSO works from the web client. If not, complete this first before trying to use PowerCLI with integrated authentication.

I have SSO configured and tested so this wasn’t my issue.

When using the older version of PowerCli version 6.5 and below we had no issue with integrated authentication and would connecting to vCenter server without prompting me for credentials.

We could continue to use the old version of PowerCLI but we would be missing out on improvement and new commandlets so I wanted to try and get the newer version working for automated task.

First we install the VMware.PowerCLI module using Install-Module VMware.PowerCLI, the current version is 12.1

When I tried to connect using this version of PowerCLI I get prompted for a user name and password and this is what is stopping my automated task from running.

There are a few ways to workaround the prompt and this can also be used to confirm if SSO is working correctly. One way would be to add credential store item that can be used for connecting.

To add this use the VICredentialstore commandlet.

New-VICredentialStoreItem -User domain\username -Password Password -Host vc.domain.local -File C:\Temp\vicreds.xml

This outputs the credential to an xml file that can then be imported and called using Connect-VIserver.

$logon = Get-ViCredentialStoreItem -File C:\Temp\vicreds.xml
Connect-VIServer -Server $logon.Host -User $logon.User -Password $logon.Passwor

This works but I don’t really want to have a xml file that has information saved to it and someone might remove the folder or file by mistake.

The other way would be similar but using a txt file with the converted to a secure string password but again this relies on a file which is not ideal and not really all that secure.

The last option and one that I wouldn’t recommend at all is to hard code a username and password in the script.

So now that we have gone through some work arounds I decided to have a look at the actually problem .

The above proves that authenticating against AD is working so I knew it wasn’t an account or SSO issue so it had to be an issue with PowerCLI itself.

I connect using a my user name and passwords and it connect without issue.

I then check the VPX log under /storage/log/vmware/vpxd to see if there are an issue but I didn’t see any issues.

Next I tried to use the -verbose parameter to return more information on what exactly connect-viserver was doing,

This then returned an error for TLS.

I next checked the settings on PowerCLI configuration settings.

To check the PowerCLI configuration use

Get-PowerCLIConfiguration

When checking the configuration the Invalidcerificaeaction was set to unset.

I changed this setting to warn instead of unset

Once this setting was changed I can now connect to PowerCLI with integrated authentication, I do get a long warning message though I could set this to ignore and this returns no warning or error.

There are two ways around this instead of changing the above setting. One is to install the the certificate as a trusted root certificate so that the cert is trusted.

Or if you have an internal certificate authority you to replace the default VMware cert with an internal cert.

Once this is done the connection work without requiring manual intervention and my automated scripts can be run using scheduled tasks again with the latest version of PowerCLI. Hopefully this will be helpful to anyone else having this issue.

Upgrading ESXI Host From 6.7 to 7.0 Using vSphere Lifecycle Manager

In this post we will go through using vSphere Lifecycle Manager VLM to upgrade an ESXI host from 6.7 to 7.0. Lifecycle Manager replaces vSphere Update Manager in vCenter server 7.0 the process is pretty much the same as in VUM.

Before upgrading to a new version of ESXI first step should be to check VMware compatibility, this can be done by either checking the VMware compatibility matrix.

VMware Compatibility Guide – System Search

Or in VLM we can now use Hardware compatibility, This syncs a list of compatible hardware and this can then be checked directly on the host to verify if the ESXI host hardware is supported.

To check using VLM, Open the vSphere web client > Menu > Lifecycle Manager

We then need to sync the hardware compatibility list. Click Actions > Sync HCL

Once synced we can run a compatibility check from the Host > Updates > Hardware Compatibility

After the compatibility is all confirmed and no issues are found, we can go ahead with the upgrade.

Next we need to download the ISO image for ESXI 7.0 that will be imported to VLM and used in the upgrade baseline.

Download VMware vSphere – My VMware

To import the ISO go back to VLM > Imported ISOs > Import ISO

Click browse and select the ESXI ISO that was downloaded earlier.

The ISO will then start to import.

The ISO should now show under Imported ISOs

Next we need to create a new upgrade baseline, this can either be done under baseline or by selecting the image and clicking on New Baseline.

Give the baseline a name

Select the ISO to be used

Click next and finsh off the baseline

Check baselines to confirm the creation has completed.

The baseline can be assigned to the host individually or the cluster. I am going to apply to the cluster. Go to the Cluster and select Updates > Attach.

Select the upgrade baseline.

We can check the compliance for the baseline for all host in the cluster.

To remediate you can either do this from the cluster or on the individual host. I will remediate from the host itself.

Go to the host, Select Updates > Baseline > Upgrade Baseline > Remediate

Accept the end user agreement.

The upgrade will do a remediation pre check before allowing the upgrade.

Once you click Remediate, the upgrade task will start

The host should now reboot and start the upgrade.

When we check the baseline the host should now show as compliant and running ESXI 7.0.

Upgrading a host using VLM is a straight forward process and makes it easy to keep your ESXI host at the latest release version.

How to Update vCenter 7.0 Virtual Appliance

The process of patching vCenter server appliance has become a lot easier in recent years. Keeping vCenter fully up to date is important for stability and security.

In this post we will go through the process of patching for vCenter 7.0 to the latest version using the GUI connecting to the internet. You can also update using command line or by downloading and mounting the ISO image.

First we need to logon to the admin management console.

https://vcenter.domain.local:5480

Use the root logon that was configure when setting up the appliance.

First steps is to confirm there is a valid backup of the appliance.

Click backup now.

There is an issue with vCenter 6.7U2 and above where it fails on SMB with SMB location is invalid if SMBv1 is disabled. So if you get that error you can just enabled SMBv1 temporarily or enabled OpenSSH on Windows to allow SSH connection which is what I would do in production.

Once completed the backup should kick off.

I also usually take a snapshot as that is the quickest recover option.

Once we have a back up, we can now continue with the updating the appliance. The current version of the appliance is 7.0.0.10100.

Go to Update and click check updates

Once the check is completed select the latest patch. Select either stage only or stage and install if you want the update to be installed straight away. The version we will be updating to is 7.0.1.00200.

Accept the end user agreement.

This will run a pre-check on vCenter before the upgrade will continue. Once no issue are found put in the administrator’s password.

Tick the box to confirm that a backup has been completed.

The install will now start and can take a hour or so to complete.

During the upgrade there will be outages to vCenter while services restart.

a

vCenter should now be update to the latest version.