How to Replace Your Default ESXi SSL Certificate With a Self-Signed Certificate: a 101 Introduction

Introduction

I think it’s safe to suggest that any admin who has ever been working with ESXi hosts have had problems with untrusted SSL certificates. Some browsers simply won’t allow connecting a host under such circumstances, and some will just provide you with the HTTP protocol version that doesn’t support secured connections.

As you know, an issue with SSL certificates isn’t something you can just postpone. Without configuring it to fit your infrastructure, in particular, the whole encryption process is gonna be disabled, which compromises the very idea of network protection for your servers. So, if you’re interested in certificate requirements for different solution paths within the VMware product line, don’t wait up and look for it here.

The situation isn’t pretty at all, and it would take a lot of time to fix it. Of course, if you don’t care for the network security (whatever the reasons) and you aren’t annoyed by constant notifications on unsecured connections… Well, then you can skip the rest.

Now, “the rest” is a combination of different suggestions and solutions I have found while surfing the web, and naturally, my personal experience.

The results were fine enough for me to decide to share it!

Mind, though, that the following algorithm is a solution for smaller infrastructures with separate ESXi hosts. If we’re talking about large enterprises, it could work only as a temporary measure until commercial certificates are generated again.

Oh, and also, this method is perfect for various test environments.

So, to fix our problem, we would follow these steps:

1. Configure OpenSSL on your ESXi.
2. Create a key, certificate request file, and certificate itself.
3. Add it to your certificate store on a server or a workstation from which you need access.
4. Check what you got!

So, let’s move on with it.

Configuring OpenSSl on Your ESXi

What OpenSSL is and why do we want it you probably know already. If not, look it up here. It pretty much covers everything you need to know. It should be present by default on your server, but it does require some tinkering. The purpose is to create a configuration file with “right” parameters for generating the “right” certificates:

In further work, you’ll need two more tools:

• Putty – free ssh and telnet client;
• WinSCP – free SFTP client and FTP client for Microsoft Windows.

If you got them all, now you can connect to the host for commands execution and file transfer (in both directions).

Before getting on with it, you should enable certain options in your server console in the “Troubleshooting Option” tab.

Just select what you’re gonna need for work, and click on Enter:

• “Enable ESXi Shell”;
• “Enable SSH”.

This will allow you to connect the host for management. When all the work is done, return all settings to their defaults! Otherwise, your computer won’t stay safe in any public network for long.

Furthermore, in “Configure Management Network”\“DNS Configuration,” change the hostname from “localhost” to any other than that (mine is “ESXi-01”).

Creating a Certificate

Connect to the host with the Putty utility, under an account with the root access rights.

First, you’ll need to create copies of the current certificate and its key so that they could have been restored later. Use the following commands:

Create a directory.

Store the current certificate copy in the new directory;

Store the current certificate creation request in the directory (you could get an error notification if the request weren’t performed before).

 Store the current certificate key in the directory;

Check whether the files are in the directory.

These copies can come in handy should you ever need to return to default settings (if the process goes south somehow).

Let’s connect now to the host with WinSCP, create (Ctrl+F4) a new file.

Let’s name it “/etc/vmware/ssl/webclient.cnf” (no quotation marks!).

Insert the following:

You need to replace values in the quotation marks with the values you require (again, no quotation marks). The values “countryName” and “stateOrProvinceName” should be in the international alpha-2 code. Overall, be very careful with the values related to your host address and domain name because you are risking your certificate’s work.

When you’re done, save the changes.

Generate a new encryption key. It will be stored on the host and keep your certificate up and running:

Generate a certificate signing request for your certificate. This very request uses data from webclient.cnf, the data you will be adding to the certificate as values:

Generate certificate itself for further use, with a validity period of 365 days, in x509 standard format:

Restart the ESXi host services in Putty, that way you can use your brand new certificate without restarting the host itself:

All you have to do now is copy the certificate file to whatever servers and workstations need access to this host.

In WinSCP, update (Ctrl+R) its contents and copy the certificate file (F5) to the local disk, which in our case is C:\Temp directory with a current name rui.crt.

Don’t forget to return all the settings from the “Troubleshooting Option” tab to their defaults!

Adding a Certificate to The Certificate Store

Start PowerShell with admin rights. Import the certificate to the store with the following command:

Let’s make sure that you did have import and find it in the store certificate list with the following command:

In Conclusion

Finally, it’s time for you to start or restart your browser to use the certificate, enter the host address, and enjoy smooth working.

That’s all, folks! I hope this will make a life for you a little bit easier as it did for me.