Skip to content
Lucas Mauro

Type to search…

↑↓ to navigate
Esc

What is a Kubernetes Job?

Let's have a look at this one-off kind of workflow.

kubernetes 2 min read

A job is a Kubernetes object that runs Pods to completion. Unlike a Deployment, which keeps pods running indefinitely, a job creates pods with a specific task in mind: once the task finishes successfully, the job is done.

This is useful for batch workloads, one-off tasks, or anything that has a clear start and end (data processing, database migrations, running a script, and so on).

apiVersion: batch/v1
kind: Job
metadata:
  name: data-import
spec:
  completions: 1
  parallelism: 1
  template:
    spec:
      containers:
        - name: importer
          image: my-app:1.0
          command: ['python', 'import.py']
      restartPolicy: Never

Let us break these attributes down:

  • completions tells Kubernetes how many pods need to complete successfully before the job is considered done (1 in this case);
  • parallelism controls how many pods run at the same time (1 means sequential execution);
  • template is the pod template, just like in a Deployment;
  • restartPolicy: Never means that if a pod fails, it is not restarted (the job creates a new pod instead). The alternative is OnFailure, which restarts the same pod.

These two fields give us flexibility over how the job runs:

  • completions: 1, parallelism: 1: a single pod runs and must succeed (the default for simple tasks);
  • completions: 5, parallelism: 2: Kubernetes runs up to 2 pods at a time until 5 have completed successfully (useful for splitting work across multiple pods);
  • completions: 10, parallelism: 10: all 10 pods run at once (fully parallel batch processing).

By default, a job pod is not replaced if it fails (with restartPolicy: Never). Kubernetes marks the job as incomplete and waits. We can control this behaviour with:

  • backoffLimit: the number of times Kubernetes will retry a failed pod before giving up (default is 6);
  • activeDeadlineSeconds: the maximum time the job can run before it is terminated, regardless of completion.

When to use each? Well, the key difference is intent. A deployment is designed for long-running services that should never stop, such as REST API application. A job, however, is designed for tasks that are expected to finish. If we need a recurring schedule (e.g. “run this every night at 2am”), we would use a CronJob instead, which creates jobs on a timer.

Comments