Skip to main content

Kubernetes jobs

Overview

Kubernetes has the concept of jobs. Kubernetes Jobs are created to run pods for a short period of time, to complete a task; as opposed to other objects in Kubernetes like Replicasets, Replication Controllers, and Daemonesets, which run pods continuously.

Usage

You can use a Kubernetes Job to run batch processes, ETL jobs, sending emails, scanning database keys, etc. Follow the guide on writing a job spec.

This example Job will start a busybox container that executes a bunch of shell commands.

Example:

apiVersion: batch/v1
kind: Job
metadata:
  name: job1
spec:
  template:
    spec:
      containers:
        - name: job
          image: busybox
          args:
            - /bin/sh
            - -c
            - date; echo sleeping....; sleep 90s; echo exiting...; date
      restartPolicy: Never
      ttlSecondsAfterFinished: 100

Deploying a job to your Namespace

You need a yaml file to define your job. You can either create your own, or copy and tweak the above example, then deploy it as usual:

  ```
  kubectl apply -f [file-name].yaml --namespace [your namespace]
  ```

Verify the job is created:

  ```
  kubectl get job --namespace [your namespace]
  ```

Clean up finished jobs

When a Job completes Pods created by the Job are not deleted. Finished Jobs are usually no longer needed in the system. Use the Time-to-live (TTL mechanism) for Jobs by settting ttlSecondsAfterFinished, so that a Job can be cleaned up automatically some time after it finishes.

The Cloud Platform has this delete-completed-jobs concourse pipleline which will clean up any completed jobs which do not have ttlSecondsAfterFinished define. Deleting a completed Job will clean up any Pods it created. You can still see the logs of the deleted Pods in kibana

Note: Kubernetes uses UTC exclusively. Make sure you take that into account when you’re creating your schedule or setting up ttlSecondsAfterFinished.

This page was last reviewed on 6 December 2021. It needs to be reviewed again on 6 March 2022 .
This page was set to be reviewed before 6 March 2022. This might mean the content is out of date.