VAST with Kubernetes

Kubernetes is a popular and successful orchestration framework. By using Kubernetes one can deploy and manage large numbers of containers within an enterprise. These containers of course contain applications that execute whatever is needed. Kubernetes does a great job of managing those containers but does not provide for persistence of data within those containers.

When a container runs in Kubernetes it can ask the Kubernetes framework for volumes or persistent volumes. In the simplest case those volumes are stored on the local host. The problem with that is if the container is moved to another host, the stored data is lost. Additionally, two containers running on different hosts cannot see data that is stored locally to one host. To enable shared and reliable persistent storage, Kubernetes allows for customizable persistent storage. Kubernetes applications need a persistent store that can scale well, is very reliable, is very fast, and can be accessed remotely by any container. VAST is one such store.

In this article we'll explain how to use VAST as a persistent store for applications running in Kubernetes. We show two techniques. In the first part we show using VAST simply as an NFS file server which Kubernetes natively supports. Then in the second part we show how to leverage the VAST CSI plugin for deeper Kubernetes integration.

You may be interested in our webinar that covers our Kubernetes work.

In Kubernetes Volumes allow a container (via its YAML specification) to state what persistent storage it wishes to use. Kubernetes supports a variety of volumes types, including an NFS volume. Conveniently VAST supports NFS and thus one can easily reference an existing VAST cluster using standard Kubernetes constructs.

Here's an example of a trivial pod that uses an existing VAST cluster as persistent storage. In advance we did the following:

The application prints a date to a file within VAST, demonstrating proper connectivity. This is the complete YAML file for that application.

# Create a pod that reads and writes to the
# VAST NFS server via an NFS volume.

kind: Pod
apiVersion: v1
metadata:
  name: pod-using-nfs
spec:
  # Add the server as an NFS volume for the pod
  volumes:
    - name: nfs-volume
      nfs:
        # URL for the NFS server. The DNS name must be defined at the OS level, not within K8S DNS.
        server: vastvips
        path: /k8s

  # In this container, we'll mount the NFS volume
  # and write the date to a file inside it.
  containers:
    - name: app
      image: centos

      # Mount the NFS volume in the container
      volumeMounts:
        - name: nfs-volume
          mountPath: /var/nfs

      # Write to a file inside our NFS
      command: ["/bin/sh"]
      args: ["-c", "while true; do date >> /var/nfs/nfsdirect-dates.txt; sleep 5; done"]

To deploy this using kubectl, create the file above and then run:

kubectl apply -f pod-using-nfs.yaml

That's more than sufficient for many applications, but in some Kubernetes environments, the administrators prefer to better abstract storage from the applications by using Persistent Volumes and Persistent Volume Claims. As you likely noticed the YAML file above for the pod included very specific environmental details - the DNS name and mount point for VAST. It is advantageous for applications to simply state "I need storage" and Kubernetes can then provide it as needed. That is done via the higher level abstraction of Persistent Volumes and Persistent Volume Claims. Essentially an administrator creates a Persistent Volume in advance (the information looks a lot like the information above) and then applications express their need by using a Persistent Volume Claim. We aren't going to cover that as it's a rather trivial extension of the previous example.

A much more interesting enhancement is to move from statically creating Persistent Volumes to Dynamic Provisioning. This frees the administrator from creating all of those Persistent Volumes which are container specific. Keep in mind that if your goal is to expose existing storage on VAST to both Kubernetes containerized applications and non-Kubernetes applications, dynamic provisioning is not a good approach. Dynamic provisioning is optimized for easily creating storage dynamically that is accessible only from the Kubernetes containerized environment.

With dynamic provisioning, applications still specify Persistent Volume Claims, but instead of the Kubernetes administrator creating the volumes, the administrator deploys in advance the elements needed for Dynamic Provisioning, specifically these key components:

The linkage between the StorageClass and the provisioner is what enables dynamic provisioning. When a Persistent Storage Claim is made, the Kubernetes framework will use the provisioner to dynamically create a Persistent Volume. Later when a container (via its YAML) references a Persistent Storage Claim Kubernetes will provide the volume to the container. This is a bit of a simplification and Persistent Volumes have lifecycle rules - Persistent Volume lifecycle behaviors are controlled by the Kubernetes framework and outside the scope of this article.

Kubernetes has a few built in provisioners but also provides for CSI based provisioning which is the focus of the remainder of this article.

With the introduction of CSI, containers can obtain dynamically provisioned storage exposed by VAST systems. VAST's CSI allows CO (container orchestrators) such as Kubernetes to provision storage volumes from a VAST system. The VAST CSI Driver 2.1.1 Administrator's Guide explains how to install the plugin on a Kubernetes cluster.

You've seen how VAST can be used easily with Kubernetes to provide dynamically provisioned Persistent Volumes. Since VAST provides a highly scalable, fast, and remotely accessible data store, it is a great choice for use with Kubernetes.

We are very interested in feedback on our Kubernetes support. Please contact your VAST representative to let us know how well this works for you and what enhancements would be useful for a productized version.

VAST CSI Source - https://github.com/vast-data/vast-csi