[lunar.lab] Deploy TKG Management Cluster on vSphere

Now all the preparation completed, I finally able to deploy TKG management cluster. The recommended (and easiest) way to do this for the first time is using the installer interface. From the bootstrap machine prepared earlier (https://dy.si/TAg1M72), I type this:

tanzu management-cluster create --ui --browser none --bind 192.168.110.101:8081

• 192.168.110.101 is my bootstrap machine IP Address
• I set the installer interface to be accessible on port 8081
• --browser none to tell the installer not to open browser locally as my bootstrap machine is without GUI
• Open http://192.168.110.101:8081 on a browser with network access to bootstrap machine
• As always, this is the official documentation that I refer to: https://docs.vmware.com/en/VMware-Tanzu-Kubernetes-Grid/1.5/vmware-tanzu-kubernetes-grid-15/GUID-mgmt-clusters-deploy-ui.html

Step 1 - IaaS Provider

[lunar.lab] Prepare to Deploy TKG Management Clusters to vSphere

• Created a base image template that matches the management cluster’s Kubernetes version.
• Official documentation: https://docs.vmware.com/en/VMware-Tanzu-Kubernetes-Grid/1.5/vmware-tanzu-kubernetes-grid-15/GUID-mgmt-clusters-vsphere.html#import-a-base-image-template-into-vsphere-4
• Import latest available version and -1 to test version upgrade
• In my case for TKGm 1.5.1, I import version Kubernetes v1.21.8 and v1.22.5
• I found out that Ubuntu template is not recognized when trying to deploy workload cluster from TMC, so I also import Photon image.

[lunar.lab] Install Harbor Container Registry as Docker Containers

This is an installation note of Harbor container registry for lunar.lab. Keeping in mind to minimize footprint due to resource constraint, I decided to install Harbor service in bootstrap machine VM which I already deployed (See here for the article: https://dy.si/TAg1M72).

• Official documentation followed for this purpose can be found here: https://goharbor.io/docs/2.5.0/install-config/
• List of software version:
• Harbor 2.5.0
• Docker Engine 20.10.16
• Docker Compose 2.5.0
• For a more proper way to deploy Harbor in TKG, you may want to check this official documentation:
• TKGm: https://docs.vmware.com/en/VMware-Tanzu-Kubernetes-Grid/1.5/vmware-tanzu-kubernetes-grid-15/GUID-packages-harbor-registry.html
• TKGs: https://docs.vmware.com/en/VMware-vSphere/7.0/vmware-vsphere-with-tanzu/GUID-2B3F2498-7BE3-4179-9F92-C83902061D42.html  

[lunar.lab] Prepare Bootstrap Machine for TKGm Deployment

Having a bootstrap machine is one of the step required for deploying TKGm to vSphere as stated here:

https://docs.vmware.com/en/VMware-Tanzu-Kubernetes-Grid/1.5/vmware-tanzu-kubernetes-grid-15/GUID-mgmt-clusters-vsphere.html

This article will run through the step I took to configure that bootstrap machine which involves these following 4 steps.

Step 1 - Starting Point: Ubuntu VM

I create Ubuntu VM from scratch with the following configuration:

• Virtual Hardware specification
• 8 vCPU
• 8GB RAM
• 40GB disk
• Ubuntu 20.04
• Minimum install + OpenSSH
• Configure static IP
• Configure Internet access (using proxy)

Once VM created on vSphere, boot from Ubuntu 20.04 installation image, then all the above settings can be configured easily through the installation wizard. Pretty straightforward.

Help! Where is my tanzu cluster Plugin?

So you have lost your tanzu cluster Plugin after upgrading Tanzu CLI?

I tried to upgrade my TKG from version 1.4.2 to 1.5.1. One of the first step is to upgrade Tanzu CLI. Once upgraded, I realize that tanzu cluster plugin is missing! 😱 How can I manage my TKG cluster then? This did not happen when I did the upgrade from 1.3.1 to 1.4.2! 

Help, I Cannot Pass IaaS Provider Step when Deploying TKG 1.4.2 Management Cluster to vSphere!

In TKG 1.4, if you deploy management clusters to vSphere with the installer interface, as a first step of configuring vCenter Server as IaaS Provider, you’ll need to fill in your vCenter Server IP Address or FQDN, username/password then hit Connect. If your vCenter use the default certificate, you’ll found this error:

Failed to connect to the specified vCenter Server. Post "https://IP_or_FQDN/sdk": x509: cannot validate certificate for IP_or_FQDN because it doesn't contain any IP SANs

[lunar.lab] Configuring NSX Advanced Load Balancer for Tanzu Kubernetes Grid (TKG) on VMware vSphere

Disclaimer:

• This is for lunar.lab only and not intended as a guide for production environment. 
• All deployed on vSphere environment. 
• I use nested environment.
• I use vSphere 7, NSX ALB 20.1.6, TKGm 1.5.1 
• The official documentation is here: https://docs.vmware.com/en/VMware-Tanzu-Kubernetes-Grid/1.5/vmware-tanzu-kubernetes-grid-15/GUID-mgmt-clusters-install-nsx-adv-lb.html

To support my learning as part of VMware SEAK (South East Asia & Korea) Tanzu Take-12 Program, I try to build my own lab. I don't have the luxury of physical lab on my own, so I build on nested environment my company provided. Here is the first of my documentation.

I started with a base pod which provide me a working vSphere cluster. The networking setup is one distributed switch with following portgroup:

• ESXi: management network vmkernel - 192.168.110.0/24
• vMotion: vmotion vmkernel - 10.10.30.0/24
• storage: storage vmkernel - 10.10.20.0/24
• VM: VM management network - 192.168.110.0/24
• tkg-vip-network: load balancer/ingress virtual IP - 192.168.120.0/24
• tkg-network: management/workload cluster nodes - 192.168.100.0/24
• avi-internal: placeholder network for ALB Service Engine

All subnet are routable.