My short answer is: --max-load is useful if you are willing to invest the time needed to use it effectively. With its current implementation, there is no simple formula for choosing good values or a preliminary tool to detect them.
The line that I support is quite large. Before I started supporting it, the assembly was 6 hours. With -j64 on ramdisk, now it ends in 5 minutes (30 on an NFS mount with -j12 ). My goal was to find reasonable restrictions for -j and -l , which allows our developers to build quickly, but does not make the server (build server or NFS server) unsuitable for everyone else.
To start:
- If you choose a reasonable
-jN value (on your computer) and find a reasonable upper limit for the average load (on your computer), they work great together to balance things. - If you use a very large value of
-jN (or unspecified, for example, -j without a number) and limit the average load, gmake will:- continue spawning processes (gmake 3.81 added a throttling mechanism, but this only slightly alleviates the problem) until the maximum number of jobs is reached or until the average load value exceeds your threshold
- while the average load exceeds your threshold:
- do nothing until all subprocesses are complete
- run one task at a time
- do it all over again
On Linux, at least (and possibly in other * nix variants), the average load level is the exponential moving average (UNIX Load Average Reweighed, Neil J. Gunther), which is the average number of processes waiting for processor time (can be caused by too many processes, waiting for I / O, page errors, etc.). Since this is an exponential moving average, it is weighed so that newer samples have a stronger effect on the current value than older samples.
If you can identify a good sweet spot for the right maximum load and the number of parallel tasks (using a combination of educated guesswork and empirical testing), if you have a long job: your 1 min. (will not fluctuate much). However, if your -jN number -jN too high for the given maximum load value, it will change slightly.
Finding that the sweet spot is essentially equivalent to finding the optimal parameters for the differential equation. Since it will be subjected to initial conditions, the main focus will be on finding parameters that will cause the system to remain in equilibrium, and not when the average value of the “target” load is reached. By "in equilibrium" I mean: 1m load avg does not fluctuate much.
Assuming you're not limited by gmake restrictions: When you find the -jN -lM combination that gives the minimum build time: this combination will push your machine to its limits. If the device must be used for other purposes ...
... you can reduce it a bit when you finish the optimization.
Excluding avg loading, the improvements that I saw during the build with the -jN increase -jN out to be [roughly] logarithmic. That is, I saw a greater difference between -j8 and -j12 than between -j12 and -j16 .
Things peaked for me somewhere between -j48 and -j64 (on Solaris, this was around -j56 ), because the initial gmake process is single-threaded; at some point, a thread cannot start new tasks faster than they end.
My tests were conducted on:
- Non-recursive assembly
- recursive assemblies can see different results; they won't face the bottleneck i made around
-j64 - I did my best to minimize the number of make-isms (extension variables, macros, etc.) in recipes, because recipes are parsed in the same thread that spawns parallel jobs. The more complicated the recipes, the more time he spends in the parser instead of creating or collecting tasks. For example:
- Macros
$(shell ...) are used in recipes; they run during the first parsing session and are cached. - Most variables are assigned with
:= to avoid recursive expansion
- Solaris 10 / sparc
- 256 cores
- no virtualization / logical domains
- assembly works on ramdisk
- x86_64 linux
- 32-core (4x hyper-threading)
- no virtualization
- assembly was performed on a fast local disk