loading...

Kubernetes – Jobs

Deployments and ReplicationControllers are a great way to ensure long-running applications are always up and able to tolerate a wide array of infrastructure failures. However, there are some use cases this does not address, specifically short-running, run once tasks, as well as regularly scheduled tasks. In both cases, we need the tasks to run until completion, but then terminate and start again at the next scheduled interval.

To address this type of workload, Kubernetes has added a batch API, which includes the Job type. This type will create 1 to n pods and ensure that they all run to completion with a successful exit. Based on restartPolicy, we can either allow pods to simply fail without retry (restartPolicy: Never) or retry when a pods exits without successful completion (restartPolicy: OnFailure). In this example, we will use the latter technique as shown in the listing longtask.yaml:

apiVersion: batch/v1
kind: Job
metadata:
  name: long-task
spec:
  template:
    metadata:
      name: long-task
    spec:
      containers:
      - name: long-task
        image: docker/whalesay
        command: ["cowsay", "Finishing that task in a jiffy"]
      restartPolicy: OnFailure

Let’s go ahead and run this with the following command:

$ kubectl create -f longtask.yaml

If all goes well, you’ll see job "long-task" created printed on the screen.

This tells us the job was created, but doesn’t tell us if it completed successfully. To check that, we need to query the job status with the following command:

$ kubectl describe jobs/long-task
Job status

You should see that we had 1 task that succeeded, and in the Events logs, we have a SuccessfulCreate message. If we use the kubectl get pods command, we won’t see our long-task pods in the list, but we may notice the message at the bottom in the listing states that there are completed jobs that are not shown. We will need to run the command again with the -a or --show-all flag to see the long-task pod and the completed job status.

Let’s dig a little deeper to prove to ourselves the work was completed successfully. We could use the logs command to look at the pod logs. However, we can also use the UI for this task. Open a browser and go to the following UI URL: https://<your master ip>/ui/.

Click on  Jobs and then long-task from the list, so we can see the details. Then, in the Pods section, click on the pod listed there. This will give us the Pod details page. At the bottom of the details, click on  View Logs and we will see the log output:

Job log

As you can see in the preceding screenshot, the whalesay container is complete with the ASCII art and our custom message from the runtime parameters in the example.

Other types of jobs

While this example provides a basic introduction to short-running jobs, it only addresses the use case of once and done tasks. In reality, batch work is often done in parallel or as part of a regularly occurring task.

Parallel jobs

Using parallel jobs, we may be grabbing tasks from an ongoing queue or simply running a set number of tasks that are not dependent on each other. In the case of jobs pulling from a queue, our application must be aware of the dependencies and have the logic to decide how tasks are processed and what to work on next. Kubernetes is simply scheduling the jobs.

You can learn more about parallel jobs from the Kubernetes documentation and batch API reference.

Scheduled jobs

For tasks that need to run periodically, Kubernetes has also released a CronJob type in alpha. As we might expect, this type of job uses the underlying cron formatting to specify a schedule for the task we wish to run. By default, our cluster will not have the alpha batch features enabled, but we can look at an example CronJob listing to learn how these types of workloads will work going forward. Save the following code in longtask-cron.yaml file:

apiVersion: batch/v2alpha1
kind: CronJob
metadata:
  name: long-task-cron
spec:
  schedule: "15 10 * * 6"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: long-task-cron
            image: docker/whalesay
            command: ["cowsay", "Developers! Developers! Developers!
          \n\n Saturday task    
            complete!"]
          restartPolicy: OnFailure

As you can see, the schedule portion reflects a crontab with the following format: minute hour day-of-month month day-of-week.  In this example, 15 10 * * 6 creates a task that will run every Saturday at 10:15 am.

Comments are closed.

loading...