Using and Monitoring Storage

You, as a developer, have been tasked with creating a database that stores data in persistent storage. You want to explore how storage works in Kubernetes and how to use it in your applications.

When running stateless containers in Kubernetes, all on-disk files are temporary. This means that if a container crashes, is deleted, or scaled up, the data is lost. This is because each new container is created with a clean state.

In Kubernetes, managing cloud storage is different from cloud computing. However, Kubernetes provides a set of storage primitives that you can use to manage storage within a Kubernetes cluster and attach persistent storage to containers.

• Volumes: Volumes are mounted into Pods on a specific path.
• PersistentVolumes (PVs): Represent physical storage and provide an interface to the storage.
• PersistentVolumeClaims (PVCs): Storage requests made by a volume to obtain storage from a PersistentVolume.
• StorageClasses: Provide a way for administrators to describe the classes of storage they offer.

Kubernetes itself is not a storage system. The underlying infrastructure must provide physical storage, and the cluster administrator must plug it into Kubernetes. This physical storage can be a local disk on a node, NFS-based storage, or a cloud provider storage system, for example, AWS EBS or Azure Disks.

The cluster administrator provisions PersistentVolumes to the cluster, which abstract the physical storage and make it available to the cluster. Note that PersistentVolumes are not namespaced. They are available to all namespaces in the cluster. StorageClass can also dynamically provision PersistentVolumes.

On the other hand, Kubernetes users deploy applications requiring persistent state among reboots. To do so, they have to request storage from the cluster. This is done by creating a PersistentVolumeClaim for a Pod. This claim is a request for storage that is fulfilled by a PersistentVolume. The PersistentVolumeClaim is bound to a PersistentVolume. The Pod mounts the PersistentVolumeClaim as a volume at a specific path so that the application can read or write data to the PersistentVolume.

SAP is provisioning the underlying Kubernetes cluster using the project "Gardener" on the hyperscaler of your choice (such as, AWS, Azure, or GCP). The hyperscaler provides physical storage.

SAP BTP, Kyma runtime comes with a pre-defined set of StorageClasses.

When you go to Kyma dashboard and choose the default storage class, you can see that the labels indicate that this storage class is managed by Gardener.

In the following example, SAP BTP, Kyma runtime is provisioned on Microsoft Azure and uses Azure Disks as the storage back end. After creating a PersistentVolume, you can create multiple PersistentVolumeClaims, which are bound to the PersistentVolume, until its capacity is utilized. The image depicts an example relationship between PersistentVolumes and PersistentVolumeClaims on SAP BTP, Kyma runtime running on Microsoft Azure:

Typically, it's recommended to avoid persisting data in the virtual file system of the respective container. Instead, it is advised that you mount a volume into the container. For this, the Pod definition must specify the volume and the mount point inside the container.

123456789101112131415
apiVersion: v1
kind: Pod
metadata:
  name: my-pod
spec:
  containers:
    - name: my-frontend
      image: nginx
      volumeMounts:
      - mountPath: "/var/www/html"
        name: my-volume
  volumes:
  - name: my-volume
    persistentVolumeClaim:
      claimName: my-claim

In this example, the Pod mounts the volume my-volume at the path /var/www/html in the container. The Pod requests the volume through the PersistentVolumeClaim my-claim, and the PersistentVolumeClaim looks for the PersistentVolume in the cluster.

You can define a PersistentVolumeClaim in the Pod definition or create it separately. Creating a separate PersistentVolumeClaim is useful if you want to reuse the same PersistentVolumeClaim for multiple Pods.

123456789101112
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-claim
spec:
  storageClassName: manual
  volumeMode: Filesystem
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

If a binding between the PersistentVolumeClaim and the PersistentVolume is found, the Pod can be created. However, once a Pod uses the PersistentVolumeClaim, the PersistentVolumeClaim is bound to the PersistentVolume. Deleting a Pod doesn't cause a deletion of the PersistentVolumeClaim, and the PersistentVolumeClaim is still bound to the PersistentVolume.

See an example of a PersistentVolume that stores data on a node in a local disk:

1234567891011121314
apiVersion: v1
kind: PersistentVolume
metadata:
  name: my-persistent-volume
  labels:
    type: local
spec:
  storageClassName: manual
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: "/mnt/data"

You can also mount a volume and define a PersistentVolumeClaim in a Deployment or in a StatefulSet. This approach is more typical, since usually you don't create a Pod manually but define it in a parent resource.

Kyma dashboard provides a view for Persistent Volume Claims at the namespace level.

By clicking on the Persistent Volume Claim, you can see the details of the claim and the Pods that are using the claim.

Persistent Volumes are visible in Kyma dashboard on the cluster level: