Skip to main content

Kubernetes jobs

Overview

Kubernetes has the concept of jobs. Kubernetes Jobs are created to run pods for a short period of time, to complete a task; as opposed to other objects in Kubernetes like Replicasets, Replication Controllers, and Daemonesets, which run pods continuously.

Usage

You can use a Kubernetes Job to run batch processes, ETL jobs, sending emails, scanning database keys, etc. Follow the guide on writing a job spec.

This example Job will start a busybox container that executes a bunch of shell commands.

Example:

apiVersion: batch/v1
kind: Job
metadata:
  name: job1
spec:
  template:
    spec:
      containers:
        - name: job
          image: busybox
          args:
            - /bin/sh
            - -c
            - date; echo sleeping....; sleep 90s; echo exiting...; date
      restartPolicy: Never
      ttlSecondsAfterFinished: 100

Deploying a job to your Namespace

You need a yaml file to define your job. You can either create your own, or copy and tweak the above example, then deploy it as usual:

  ```
  kubectl apply -f [file-name].yaml --namespace [your namespace]
  ```

Verify the job is created:

  ```
  kubectl get job --namespace [your namespace]
  ```

Clean up finished jobs

When a Job completes Pods created by the Job are not deleted. Finished Jobs are usually no longer needed in the system. Use the TTL mechanism for Jobs by settting ttlSecondsAfterFinished, so that a Job can be cleaned up automatically some time after it finishes.

The Cloud Platform has this delete-completed-jobs concourse pipleline which will clean up any completed jobs which do not have ttlSecondsAfterFinished define. Deleting a completed Job will clean up any Pods it created. You can still see the logs of the deleted Pods in kibana

Note: Kubernetes uses UTC exclusively. Make sure you take that into account when you’re creating your schedule or setting up ttlSecondsAfterFinished.

This page was last reviewed on 7 June 2021. It needs to be reviewed again on 7 September 2021 .
This page was set to be reviewed before 7 September 2021. This might mean the content is out of date.