Explaining jobs in Kubernetes

You've already deployed different kinds of workloads to your Kubernetes cluster. However, for some use cases, you don't need a workload that is always on. Instead, you want to run a workload only once and then terminate it or schedule it to run periodically. For this, you want to explore the capabilities of Jobs in Kubernetes.

Jobs are generally used to run a workload only once. For example, you can use a Job to run a database migration or to perform a batch job. Compared to the other workloads in Kubernetes, Jobs are not supposed to be long-running. Instead, they are supposed to run to completion and then terminate. However, if a Job fails, it will be restarted until it succeeds.

Click on the buttons below to find out more about each job type.

A simple Job manifest to declare a one-off job could look like this:

apiVersion: batch/v1
kind: Job
metadata:
  name: my-job
spec:
  backoffLimit: 4
  template:
     spec:
       containers:
       - name: my-job
         image: busybox:1.35.0 # busybox is a small image that contains a few useful utilities
         command: ['sh', '-c', 'echo Hello Kubernetes! && sleep 10 && echo Job done!']
       restartPolicy: OnFailure

The above manifest defines a Job that runs a single pod with a container based on the busybox image. The container runs the command echo Hello Kubernetes! && sleep 10 && echo Job done!. The sleep command simulates some work that takes some time to complete. The restartPolicy is set to OnFailure, which means that the Job will be restarted if it fails. Alternatively, you can set the restartPolicy to Never, which means that the Job will not be restarted if it fails. The backoffLimit is set to 4, meaning the Job will be restarted at most four times. If the Job fails 4 times, it will be marked as failed.

When defining a parallel Job, you can specify the number of parallel pods that should be created in total (.spec.parallelism) and the maximum number of parallel pods that should be running simultaneously (.spec.completions). The following manifest defines a parallel Job that creates six pods in total and runs three pods in parallel:

apiVersion: batch/v1
kind: Job
metadata:
   name: my-parallel-job
spec:
   parallelism: 3
   completions: 6
   backoffLimit: 4
   template:
     metadata:
       labels:
         sidecar.istio.io/inject: "false"
     spec:
       containers:
       - name: my-parallel-job
         image: busybox:1.35.0 # busybox is a small image that contains a few useful utilities
         command: ['sh', '-c', 'echo Hello Kubernetes! && sleep 10 && echo Job done!']
       restartPolicy: Never

Kubernetes also waits until all three pods have been completed successfully before it starts the next three pods.

You can also observe this in the Kyma Dashboard. The following screenshot shows the Job in the Kyma Dashboard. The Job has been created six times and currently runs three times in parallel.

Once a job is triggered, it will create a pod based on the specified pod template. After all containers in this pod are completed successfully, the Job is considered completed. This also applies to sidecar containers. If your current workload is completed, but the sidecarcontainer is still running, the Job is not considered completed. Especially in Kyma, there is the istio-proxy as sidecar container running, if not specified differently. This container is responsible for the service mesh and is always running. Although you will cover Istio and Service Mesh in a later lesson, here is a workaround for this issue. You can use the sidecar.istio.io/inject: "false" label in the pod template to prevent the istio-proxy container from being injected.

For scheduling workload to run periodically, you can use a CronJob. A CronJob is similar to a Job but is scheduled to run regularly at a fixed time, date, or specific time interval. The following manifest defines a CronJob that runs every night at 2:00 AM:

apiVersion: batch/v1beta1
 kind: CronJob
 metadata:
   name: my-cronjob
 spec:
   schedule: "0 2 * * *"
   jobTemplate:
     spec:
       template:
         spec:
           containers:
           - name: my-cronjob
           image: busybox:1.35.0 # busybox is a small image that contains a few useful utilities
           command: ['sh', '-c', 'echo Hello Kubernetes! && sleep 10 && echo Job done!']
         restartPolicy: OnFailure
      backoffLimit: 4

In the editor it looks like this:

The schedule field defines the schedule in the Cron format. In the above example, the CronJob is scheduled to run every night at 2:00 AM. The jobTemplate field defines the Job template that should be used for the CronJob. The CronJob will create a Job based on this template and then schedule it to run periodically.

In this lesson, you learned about the differences between ReplicaSets and DaemonSets. You also learned how to use DaemonSets in Kubernetes to ensure that a given pod is running on a subset of nodes in your cluster.