Process planning

Question

Let's say I have 10 scripts that I want to run regularly as cron jobs. However, I do not want them all to work simultaneously. I want only 2 of them to be executed at a time.

One solution that I think of is to create two scripts, put 5 statements for each of them and them as separate records in crontab. However, the solution seems very complicated.

Is there an existing unix tool to accomplish the above task?

Answer 1

The built-in jobs can tell you how many child processes are running. Some simple shell scripts can accomplish this task:

 MAX_JOBS=2 launch_when_not_busy() { while [ $(jobs | wc -l) -ge $MAX_JOBS ] do # at least $MAX_JOBS are still running. sleep 1 done " $@ " & } launch_when_not_busy bash job1.sh --args launch_when_not_busy bash jobTwo.sh launch_when_not_busy bash job_three.sh ... wait 

Answer 2

NOTE. As stated in mobrule, my original answer will not work, because a wait configured without arguments expects all children to complete. Therefore, the following "parallelexec" script, which avoids polling due to more child processes:

 #!/bin/bash N="$1" I=0 { if [[ "$#" -le 1 ]]; then cat else while [[ "$#" -gt 1 ]]; do echo "$2" set -- "$1" "${@:3}" done fi } | { d=$(mktemp -d /tmp/fifo.XXXXXXXX) mkfifo "$d"/fifo exec 3<>"$d"/fifo rm -rf "$d" while [[ "$I" -lt "$N" ]] && read C; do ($C; echo >&3) & let I++ done while read C; do read -u 3 ($C; echo >&3) & done } 

The first argument is the number of parallel jobs. If there are more of them, each of them is launched as a task, otherwise all the commands to start are read from stdin line by line.

I use a named pipe (which goes into oblivion as soon as the shell opens it) as a synchronization method. Since only single bytes are recorded, problems with race conditions cannot complicate.

Answer 3

GNU Parallel is designed for the following tasks:

 sem -j2 do_stuff sem -j2 do_other_stuff sem -j2 do_third_stuff 

do_third_stuff will only execute when do_stuff or do_other_stuff .

Watch the videos to learn more:

http://www.youtube.com/playlist?list=PL284C9FF2488BC6D1