Happy Holidays from VMsources!

Wednesday, December 5, 2018

Howdy, Partners! Happy Holidays from VMsources!

Lots of news and action items in this letter, please check out the sections that are important to you, and let us know if VMsources may be of any assistance.

  • Pending Veeam issue with self-signed Certificate
  • Do you need to get Certificates from a “Trusted” CA at all?
  • Encryption-at-rest
  • VMware Security Advisories (VMSA) Advisories affecting ESXi and vCenter
  • VMsources goes 10Gb for LAN network in the datacenter

Pending Veeam issue with self-signed Certificate

Let’s tackle the problem first: Users who have installed Veeam Backup and Replication Update 3 using a self-signed certificate (that’s most of us!) will see the certificate expire one-year from the date of creation, which is either one year from installation or one year after you updated to “Update 3.”

Fortunately, the fix is simple: just create a new self-signed certificate! There’s a Veeam KB covering the topic, and it’s very easy to implement.

For all VMsources hosted and MSP Veeam customers, we’ll take care of this for youyou do not need to do anything! For any other Veeam users, VMsources is available to generate new self-signed certificates or import CA certificates if desired. Just give us a call at: 866-644-7764

Do you need to get Certificates from a “Trusted” CA?

Because of the Veeam issue, I thought I would touch on the subject of Certificate Authority (CA) certificates and whether or not you actually need them? There is a common misconception that security is improved through installing certificates issued by “trusted” CAs.

External “Trusted” CAs

The truth is that certificates issued by external CAs like Thawte, Verisign and GoDaddy are no more secure than those you create yourself! In fact, by going to an external source in the first place, and trusting them with your Certificate Signing Request (CSR) and Privacy Key (PK) at all, you are placing the security of your organization in their hands! External CAs are generally security-aware, but they are also massive targets for hackers. The risk, if your data got exposed by an external CA, is that hackers could masquerade as you and potentially gain access to critical systems!

The use case for external “trusted” CAs is in creating a secure relationship with an unknown and unauthenticated target, like a visitor to your website, or a client accessing a fileshare server. Because you trust a specific CA, and the client also trusts that CA, you are able to establish secure public communications. VMsources uses GoDaddy for all public-facing certificates, like the one you can see on https://vmsources.com , but uses an internal CA for other certificates.

Self-Signed Certificates are Fine!

Your self-signed certificates are legitimate certificates! Not only are they just fine, they are inherently more secure for internal use by known and authenticated users and services than would be a certificate generated by an external CA. Moreover, self-signed certificates can/are usually generated with expiration dates longer than the one-year commonly purchased through external CAs. Self-signed certificates can be generated with dates of 10 years or longer (as is the default on vCenter/VCSA).

The Best Way

While self-signed certificates are just fine, they are generally created individually on a per-server and per-device basis. That means that the certificate used by Veeam is not the same as the one used by vCenter, is not the same as the one used by IPMI, etc. The disadvantage is that every time you connect with a new device, you will be presented with a warning “untrusted” and you should really compare that certificate with the original server certificate before logging in, to confirm authenticity and avoid spoofing.

The absolute best way to use certificates and avoid external exposure is to create your own CA and generate your own certificates for internal use on servers and services. This is remarkably easy, though very time-consuming, and by doing so it is possible to install the certificate(s) you create on servers and devices to confirm their authenticity organizationally and not on a per-server basis!

If you are interested in having VMsources install external “trusted” CA Certificates, or in creating your own CA and installing internal certificates on your vSphere (vCenter, ESXi, SSO, vRealize, Horizon View, etc.), SANs and other devices; give us a call at: 866-644-7764


Various companies, especially those in the Payment Card Industry (PCI) are requiring encryption-at-rest these days. Encryption-at-rest provides security for physical and logical “disks” (LUNs, Volumes, etc.) which become separated from their systems.

  • Think laptop left in a taxi; maybe the computer is password-protected but if the disk is not encrypted-at-rest, all one need do is remove the HDD to exploit all the data.
  • Likewise for a Virtual Machine; should the disk become detached from a VM and attached to another, the data could potentially be readable

The good news for VMware vSphere users is: VMs which are powered-on are not “at rest.” Also, it’s generally not possible for one disk to be used by two VMs simultaneously (specific exemptions exist for cluster and quorum disks), so VMs in a secure Datacenter may not require encryption-at-rest.

Should additional encryption-at-rest be desirable or required; there are numerous solutions available from hardware-key based storage encryption to VMware vSphere OEM and other 3rd. Party solutions.

For physical systems and individual servers

All modern Linux systems offer native encryption at rest, which VMsources utilizes internally to provide fileshare, network security, and web services. Linux encryption at rest is secure, simple, effective and reliable!

Microsoft BitLocker is also respected and reliable technology, included with Windows, but slightly difficult (not impossible) to deploy on VMs. We use BitLocker on all workstations, laptops and physical Windows systems - to guard against the possibility data access in the event of device loss or theft.

There is also application encryption available with platforms such as MS Office. Merely by protecting your documents with a password, MS Office 2016 or later provides AES-256 protection (Office 2007 and later uses AES-128), which meets most compliance standards!

For vSphere Environments

VMware vSphere Native VM encryption

Beginning with VMware vSphere 6.5, VM encryption is natively possible with the vSphere platform. The primary purpose of VM encryption is to secure the data stored in virtual disks (VMDKs) and prevent unauthorized access. VM encryption with vSphere requires an external Key Management Server (KMS), which is licensed separately at additional expense from a 3rd party vendor. vSphere VM encryption is highly effective, but has some distinct limitations:

  • VM encryption is CPU intensive for all encrypted VMs, potentially affecting performance
  • Cannot manually edit VMX (VM configuration) in case of emergency such as snapshot failure
  • Deduplication and compression (both storage and backup) may not function since data is encrypted at hypervisor level
  • VMs cannot be started if KMS is offline – such as in full Datacenter outage event

Our favorite: HDD/SAN encryption with a hardware-key

At the moment, we strongly favor HDD/SAN encryption with a hardware-key, as the most reliable and least-failure-prone means of encryption. It is completely transparent to VMware and has little impact on overall performance. Your existing SANs may be capable of encryption, but you will have to check with the OEM for cost and compatibility.

Our main SAN vendor has all-SSD solutions available for as low as $1500/TB including encryption.

If your Datacenter is secure, you probably don’t require VM encryption at all; but if security or compliance mandates it, or you would just like to upgrade storage to SSD, call us at: 866-644-7764

VMware Security Advisories

There have been a number of VMware Security Advisories (VMSAs) addressing Common Vulnerabilities and Exposures (CVEs) since I last wrote, necessitating the patching of both ESXi and vCenter/VCSA. Fortunately, none of these has the performance implication of previous VMSAs such as the  L1 Terminal Fault vulnerability. Here are the VMSA announcements affecting ESXi and/or vCenter since October 9, 2018:

  • VMSA-2018-0027 is “CRITICAL” and address uninitialized stack memory usage with the vmxnet3 network interface
  • VMSA-2018-0026 is “CRITICAL” and address an out-of-bounds read vulnerability
  • VMSA-2018-0025 is “important” and address a denial-of-service vulnerability

Please run VMware Update Manager and/or use the CLI to remediate your ESXi Hosts with both Critical and Non-Critical patch baselines. In addition, use the VAMI interface to your vCenter (https://<VCSA-URL>:5480) to update your vCenter/VCSA instance.

Please call or email to schedule VMsources if you would like us to patch your ESXi Hosts and update your vCenter servers: 866-644-7764

VMsources goes 10Gb LAN

While we’ve had a 10Gb storage network since 2013, we have relied on an ever-expanding number of 1Gb interfaces and networks to serve our client networks. Though to date, we’ve never experienced a saturated team of 1Gb adapters, we’ve gone ahead made the investment in physical 10Gb NICs and Switches for all client networks and VLANs! That means that both fully-hosted and failed-over VMs can communicate with one-and-other on 10Gb LAN (Private Networks – Private Cloud Model), using a 1Gb uplink to your internet and multiple 10Gb Host Bus Adapters (HBA’s) to our SANs!

Happy Holidays!

Until next time, have a great holiday!


John Borhek, Lead Solutions Architect

VMsources Group, Inc.