Performing and Monitoring Storage

You, as a developer, have been tasked with creating a database that will store data in persistent storage. You want to explore how storage works in Kubernetes and how to use it in your applications.

When running stateless containers on Kubernetes, all on-disk files are temporary. This means that if a container crashes, is deleted, or gets scaled up, the data is lost. This is because each new container is created with a clean state.

In Kubernetes, managing cloud storage is different compared to cloud computing. However, Kubernetes provides a set of storage primitives that can be used to manage storage in a Kubernetes cluster and to attach persistent storage to containers:

• Volumes: Volumes are mounted into Pods at a specific path.
• PersistentVolumes (PVs): Represents physical storage and provide an interface to the storage.
• PersistentVolumeClaims (PVCs): Storage request from a volume to get storage from a PersistentVolume.
• StorageClasses: Provides a way for administrators to describe the "classes" of storage they offer.

Kubernetes itself is not a storage system. Physical storage has to be provided by the underlying infrastructure and has to be plugged into Kubernetes by the cluster administrator. This physical storage can be a local disk on a node, NFS-based storage, or a cloud provider storage system (for example, AWS EBS, Azure Disks).

The cluster administrator provisions PersistentVolumes to the cluster, which abstracts the physical storage and makes it available to the cluster. Note that PersistentVolumes are not namespaced. They are available to all namespaces in the cluster. PersistentVolumes (PV) can also be dynamically provisioned by a StorageClass.

On the other hand, Kubernetes users deploy applications requiring persisting state among reboots. To do so, they have to request storage from the cluster. This is done by creating a PersistentVolumeClaim for a Pod. This claim is a request for storage that is fulfilled by a PersistentVolume. The PersistentVolume claim is bound to a PersistentVolume. The Pod mounts the PersistentVolume claim as a volume at a specific path so that the application can read or write data to the PersistentVolume.

SAP is provisioning the underlying Kubernetes cluster through project "Gardener" on the hyperscaler of your choice (AWS, Azure, GCP). The hyperscaler is also providing physical storage.

SAP BTP, Kyma runtime comes with a set of StorageClasses, which are pre-defined by SAP:

As you can see in the labels, they are managed by Gardener. In this example, an SAP BTP, Kyma runtime, on Microsoft Azure is provisioned and uses Azure Disks as storage back end. After creating a PersistentVolume, multiple PersistentVolumeClaims can be created, which are bound to the PersistentVolume, until its capacity is utilized.

The following image depicts an example relationship between PersistentVolumes and PersistentVolumeClaims on SAP BTP, Kyma rurntime running on Microsoft Azure:

You typically want to avoid persisting data in the virtual file system of the respective container. Instead, we mount a volume into the container. For this, the Pod definition has to specify the volume and the mount point inside the container.

123456789101112131415
apiVersion: v1
kind: Pod
metadata:
  name: my-pod
spec:
  containers:
    - name: my-frontend
      image: nginx
      volumeMounts:
      - mountPath: "/var/www/html"
        name: my-volume
  volumes:
  - name: my-volume
    persistentVolumeClaim:
      claimName: my-claim

So the Pod mounts the volume my-volume at the path /var/www/html in the container. The Pod requests the volume through the PersistentVolumeClaim my-claim. The PersistentVolumeClaim looks up the PersistentVolume in the cluster.

A PersistentVolumeClaim can be defined in the Pod definition, but it can also be created separately. This is useful if you want to reuse the same PersistentVolumeClaim for multiple Pods.

123456789101112
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-claim
spec:
  storageClassName: manual
  volumeMode: Filesystem
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

If a binding between the PersistentVolumeClaim and the PersistentVolume is found, the Pod can be created. However, the PersistentVolumeClaim is only bound to a PersistentVolume on first use. Once a Pod uses the PersistentVolumeClaim, the PersistentVolumeClaim is bound to the PersistentVolume. A deletion of the Pod won't cause a deletion of the PersistentVolumeClaim. The PersistentVolumeClaim is still bound to the PersistentVolume.

A PersistentVolume, which stores data on a local disk on a node, can be created as follows:

1234567891011121314
apiVersion: v1
kind: PersistentVolume
metadata:
  name: my-persistent-volume
  labels:
    type: local
spec:
  storageClassName: manual
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: "/mnt/data"

You can also mount a volume and define a PersistentVolumeClaim in a deployment or StatefulSet, which is more typical, since you don't want to create a Pod manually.

The Kyma dashboard provides a view for Persistent Volume Claims at the namespace level.

By clicking on the Persistent Volume Claim, you can see the details of the claim and also the Pods that are using the claim.

Persistent Volumes are visible in the Kyma dashboard on cluster level: