Geek Answers Handbook

Orchestration work

Currently, I am redoing all of our tasks for database orchestration (ETL, backups, daily tasks, report compilation, etc.).

To do this, I will need a task / task manager where I can determine the dependencies of tasks, tasks based on time, asynchronous tasks, etc.

I looked at celery and streaming programming technologies, but I'm not sure if they are good for my use case.

I look more at a framework that will support all these things out of the box. Any suggestions?

+6
source share
1 answer

Maybe late, but I wanted to mention the Job runner to the possible other people who came to this question.

From your GitHub README.rst repository:

Job-Runner is a crontab-like tool with a good web interface for administration and (in real time) for monitoring the current state.

Features:

  • Schedule recurring tasks
  • Job chain
  • Workers with load balancing by placing them in a pool
  • Task schedule for all workers in the pool
  • Integrated control panel (with the ability to start and run individual schedules)
  • Multiple projects and permission management for each project
+3
source

More articles:


All Articles