[How To] Enhance Online Boutique App to Use Persistent Volume

Online Boutique (https://github.com/GoogleCloudPlatform/microservices-demo) is a web-based e-commerce microservices demo app built by folks at Google. I use this as demo app to deploy on top of Tanzu Kubernetes platform. One of the demo scenario I do is how to consume vSphere datastore as persistent storage for Kubernetes app, in easy, on-demand, fully automated, and scalable fashion. This can be done by a feature called Cloud Native Storage (CNS). Read more about CNS here:

https://blogs.vmware.com/virtualblocks/2019/08/14/introducing-cloud-native-storage-for-vsphere/

One of Online Boutique service is redis-cart. This is the service in charge for Shopping Cart. If any item added to Shopping Cart, the record will be handled by this service. With default configuration, the data volume used by redis-cart do not use persistent volume. If redis-cart is failed, Shopping Cart data will be lost. This article explains how to alter this and use vSphere datastore to provide persistent storage for redis-cart service.

Step 1 - Create VM Storage Policy on vSphere

Check the official documentation here: https://docs.vmware.com/en/VMware-Tanzu-Kubernetes-Grid/1.5/vmware-tanzu-kubernetes-grid-15/GUID-tanzu-k8s-clusters-storage.html#set-up-cns-and-create-a-storage-policy-vsphere-3

• I follow the step by step for Local VMFS Storage.
• Create tag tkg-storage-dsl under tag category tkg-storage.

• Assign tag tkg-storage-dsl to datastore which will become persistent storage.

• Create VM Storage Policy with name tkg-storage. This is the name that will be used to consume the VM Storage Policy on Kubernetes. Create a rule set to select datastore based on tag tkg-storage-dsl.

Step 2 - Create StorageClass

Official documentation: https://docs.vmware.com/en/VMware-Tanzu-Kubernetes-Grid/1.5/vmware-tanzu-kubernetes-grid-15/GUID-tanzu-k8s-clusters-storage.html#default-storage-classes-2

• Switch to your Kubernetes cluster, create storage-class.yaml with content below.

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: default
  annotations:
    storageclass.kubernetes.io/is-default-class: "true"
provisioner: csi.vsphere.vmware.com
parameters:
  storagePolicyName: tkg-storage

• Note that storagePolicyName is pointing to VM Storage Policy name configured in Step 1.
• Apply the manifest.

kubectl apply -f storage-class.yaml

Step 3 - Create PersistentVolumeClaim

Official documentation: https://kubernetes.io/docs/concepts/storage/dynamic-provisioning/#enabling-dynamic-provisioning.

• Create redis-pvc.yaml with content below.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: redis-cart
  labels: 
    app : redis-cart
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

• Apply the manifest.

kubectl apply -f redis-pvc.yaml

At this point, vSphere CNS will catch the command and create a virtual disk to be used as container volume. At the vSphere Cluster containing the nodes of Kubernetes cluster, go to Monitor>Cloud Native Storage>Container Volume. A new container volume created.

 Browse the datastore, open folder fcd and see the newly created virtual disk.

Step 4 - Modify redis-cart to use the PVC

• This is the original redis.yaml manifest: 

https://raw.githubusercontent.com/GoogleCloudPlatform/microservices-demo/main/kubernetes-manifests/redis.yaml

• Modify the file and save as redis-v2.yaml. Add the highlighted line below.

apiVersion: apps/v1

kind: Deployment

metadata:

  ....

spec:

  ...

  template:

    ...

    spec:

      containers:

      - ...

      volumes:

      - name: redis-data

        persistentVolumeClaim:

          claimName: redis-cart

• Apply the manifest, this will update redis-cart deployment.

kubectl apply -f redis-v2.yaml

At this point, vSphere CNS will catch the command and mount the virtual disk created at Step 3 to Kubernetes worker node where redis-cart pod hosted.

Once this is done, I try to add several items to Shopping Cart and then delete redis-cart pod. The pod will be recreated by Kubernetes, and the items I added to Shopping Cart is still persist. Awesome!