Skip to main content

Kubernetes jobs

Overview

Kubernetes has the concept of jobs. Kubernetes Jobs are created to run pods for a short period of time, to complete a task; as opposed to other objects in Kubernetes like Replicasets, Replication Controllers, and Daemonesets, which run pods continuously.

Usage

You can use a Kubernetes Job to run batch processes, ETL jobs, sending emails, scanning database keys, etc. Follow the guide on writing a job spec.

This example Job will start a busybox container that executes a bunch of shell commands.

Example:

apiVersion: batch/v1
kind: Job
metadata:
  name: job1
spec:
  ttlSecondsAfterFinished: 100
  template:
    spec:
      containers:
        - name: job
          image: busybox
          args:
            - /bin/sh
            - -c
            - date; echo sleeping....; sleep 90s; echo exiting...; date
      restartPolicy: Never

Deploying a job to your Namespace

You need a yaml file to define your job. You can either create your own or copy and tweak the above example, then deploy it as usual:

  ```
  kubectl apply -f [file-name].yaml --namespace [your namespace]
  ```

Verify the job is created:

  ```
  kubectl get job --namespace [your namespace]
  ```

Clean up finished jobs

When a Job completes, Pods created by the Job are not deleted. Finished Jobs are usually no longer needed in the system. Use the Time-to-live (TTL mechanism) for Jobs by setting ttlSecondsAfterFinished, so that a Job can be cleaned up automatically some time after it finishes.

Note: Kubernetes uses UTC exclusively. Make sure you take that into account when you’re creating your schedule or setting up ttlSecondsAfterFinished.

We have a delete-completed-jobs concourse job which will clean up all completed jobs which do not have ttlSecondsAfterFinished defined.

This page was last reviewed on 17 October 2024. It needs to be reviewed again on 17 January 2025 by the page owner #cloud-platform .
This page was set to be reviewed before 17 January 2025 by the page owner #cloud-platform. This might mean the content is out of date.