Cron Jobs in Kubernet - connect to an existing Pod, execute a script

Question

I am sure that I am missing something obvious. I looked at the documentation for ScheduledJobs / CronJobs on Kubernetes, but I cannot find a way to do the following on schedule:

• Connect to an Existing Pod
• Execute script
• Disable

I have alternative ways to do this, but they do not feel good.

• Cron task schedule for: kubectl exec -it $ (kubectl get pods --selector = some-selector | head -1) / path / to / script

• Create one deployment with the “Cron Pod”, which also houses the application, and many “Non Cron Pods,” which are just the application. The cron pod will use a different image (one with the scheduled cron jobs).

I would prefer to use Kubernetes ScheduledJobs, if possible, to prevent the same task from running at the same time, and also because it makes me a more appropriate way.

Is there a way to do this with ScheduledJobs / CronJobs?

http://kubernetes.io/docs/user-guide/cron-jobs/

Answer 1

As far as I know, there is no “official” way to do it the way you want, and I consider this by design. It is assumed that the Pods will be ephemeral and horizontally scalable, and Jobs is intended to exit. The presence of a cron job for an existing container does not match this module. The scheduler had no idea if the work was completed.

Instead, Job can call an instance of your application specifically to run the job and then take it off after the job is completed. To do this, you can use the same image for the job as for the deployment, but use a different “entry point” by setting command:

If they need access to the data created by your application, then this data must be stored outside the application / Pod, you could do this in several ways, but the obvious way would be a database or a persistent volume. For example, using a database would look something like this:

 apiVersion: extensions/v1beta1 kind: Deployment metadata: name: APP spec: template: metadata: labels: name: THIS app: THAT spec: containers: - image: APP:IMAGE name: APP command: - app-start env: - name: DB_HOST value: "127.0.0.1" - name: DB_DATABASE value: "app_db" 

And a job that connects to one database, but with a different "entry point":

 apiVersion: batch/v1 kind: Job metadata: name: APP-JOB spec: template: metadata: name: APP-JOB labels: app: THAT spec: containers: - image: APP:IMAGE name: APP-JOB command: - app-job env: - name: DB_HOST value: "127.0.0.1" - name: DB_DATABASE value: "app_db" 

Or a constant-volume approach would look something like this:

 apiVersion: extensions/v1beta1 kind: Deployment metadata: name: APP spec: template: metadata: labels: name: THIS app: THAT spec: containers: - image: APP:IMAGE name: APP command: - app-start volumeMounts: - mountPath: "/var/www/html" name: APP-VOLUME volumes: - name: APP-VOLUME persistentVolumeClaim: claimName: APP-CLAIM --- apiVersion: v1 kind: PersistentVolume metadata: name: APP-VOLUME spec: capacity: storage: 10Gi accessModes: - ReadWriteMany persistentVolumeReclaimPolicy: Retain nfs: path: /app --- apiVersion: v1 kind: PersistentVolumeClaim metadata: name: APP-CLAIM spec: accessModes: - ReadWriteMany resources: requests: storage: 10Gi selector: matchLabels: service: app 

With this task attached to the same thing:

 apiVersion: batch/v1 kind: Job metadata: name: APP-JOB spec: template: metadata: name: APP-JOB labels: app: THAT spec: containers: - image: APP:IMAGE name: APP-JOB command: - app-job volumeMounts: - mountPath: "/var/www/html" name: APP-VOLUME volumes: - name: APP-VOLUME persistentVolumeClaim: claimName: APP-CLAIM 

Answer 2

It looks like an anti-pattern. Why can't you just start your work module as a work module?

Regardless, you seem confident that you need to do this. Here is what I would do.

Take your work module and wrap the shell execution in a simple web service, this is 10 minutes of work with almost any language. Expose the port and put the service in front of this worker. Then your working modules can simply collapse .. svc.cluster.local: / (if you are not fixated on dns).

Answer 3

This should help.

 apiVersion: batch/v1beta1 kind: CronJob metadata: name: hello spec: schedule: "*/30 * * * *" jobTemplate: spec: template: spec: containers: - name: hello image: busybox args: - /bin/sh - -c kubectl exec -it <podname> "sh script.sh "; restartPolicy: OnFailure 

Answer 4

Create a scheduled module that uses the Kubernetes API to run the desired command on the target modules through the exec function. The module image must contain client libraries for accessing the API - many of them are available, or you can create your own.

For example, here is a solution using the Python client that runs for each ZooKeeper module and runs the database maintenance command:

 import time from kubernetes import config from kubernetes.client import Configuration from kubernetes.client.apis import core_v1_api from kubernetes.client.rest import ApiException from kubernetes.stream import stream import urllib3 config.load_incluster_config() configuration = Configuration() configuration.verify_ssl = False configuration.assert_hostname = False urllib3.disable_warnings() Configuration.set_default(configuration) api = core_v1_api.CoreV1Api() label_selector = 'app=zk,tier=backend' namespace = 'default' resp = api.list_namespaced_pod(namespace=namespace, label_selector=label_selector) for x in resp.items: name = x.spec.hostname resp = api.read_namespaced_pod(name=name, namespace=namespace) exec_command = [ '/bin/sh', '-c', 'opt/zookeeper/bin/zkCleanup.sh -n 10' ] resp = stream(api.connect_get_namespaced_pod_exec, name, namespace, command=exec_command, stderr=True, stdin=False, stdout=True, tty=False) print("============================ Cleanup %s: ============================\n%s\n" % (name, resp if resp else "<no output>")) 

and related Dockerfile:

 FROM ubuntu:18.04 LABEL maintainer "reDock Inc." ADD ./cleanupZk.py / RUN apt-get update \ && apt-get install -y python-pip \ && pip install kubernetes \ && chmod +x /cleanupZk.py CMD /cleanupZk.py 

Note that if you have an RBAC-enabled cluster, you may need to create a service account and associated roles to make this API call possible. To enumerate the modules and run exec, a role is sufficient as required in the above script example:

 apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: pod-list-exec namespace: default rules: - apiGroups: [""] # "" indicates the core API group resources: ["pods"] verbs: ["get", "list"] - apiGroups: [""] # "" indicates the core API group resources: ["pods/exec"] verbs: ["create", "get"] 

An example of a linked cron job:

 apiVersion: v1 kind: ServiceAccount metadata: name: zk-maint namespace: default --- apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: zk-maint-pod-list-exec namespace: default subjects: - kind: ServiceAccount name: zk-maint namespace: default roleRef: kind: Role name: pod-list-exec apiGroup: rbac.authorization.k8s.io --- apiVersion: batch/v1beta1 kind: CronJob metadata: name: zk-maint namespace: default labels: app: zk-maint tier: jobs spec: schedule: "45 3 * * *" successfulJobsHistoryLimit: 3 failedJobsHistoryLimit: 1 concurrencyPolicy: Forbid jobTemplate: spec: template: spec: containers: - name: zk-maint image: myorg/zkmaint:latest serviceAccountName: zk-maint restartPolicy: OnFailure imagePullSecrets: - name: azure-container-registry