Geek Answers Handbook

Is GNU end-to-end possible to create a job server environment in a subroutine filed through a third-party (non-make)

When running gnu-make rules with -jN make creates a jobserver to control the number of jobs between profiles. In addition, you can "pass the job server environment" to the make recipe by pre-specifying it + - for example:

 target : +./some/complex/call/to/another/make target

Now, instead of sub-make, I have a (python) script that performs some complex packaging actions (too complicated for make). One of the actions that it may run into may actually issue the make .

 package.stamp : $(DEPS) +./packaging.py $(ARGS) touch $@

Now when this make command is called inside package.py

 make[1]: warning: jobserver unavailable: using -j1. Add `+' to parent make rule.

This makes sense because any environment configured by make cannot be executed or passed through python.

Is it possible to pass links through jobserver through a python program to sub-make - if so, how?

+5
source share
2 answers

There are two aspects to the job server that need to be saved: the first is the actual environment variable, which makes use of sub-brands for sending parameters. This value is stored properly, otherwise make will not know that it should even search for the jobs server, and you will not see this warning message.

The second aspect is two open file descriptors that are passed to the child make elements. Your script MUST keep these two descriptors and leave them open when it calls sub-make.

You are not showing us which Python code is used to call the sub-layout. By default, the subprocess module will not close file descriptors, but you can provide the close_fds=True parameter to do this ... you should not use this parameter if you want parallel make invocations to work correctly with the jobs server.

If you are not using subprocess , then you will need to show us what you are doing.

You should probably tag this with the python tag, since this is basically a Python issue.

+5
source

To summarize and clarify the answer - for the job server to work in your subprocesses, you need to save:

  • Environment Variables
  • Fds workstation server

One of the passed environment variables looks (for me) as follows:

 MAKEFLAGS= --jobserver-fds=3,4 -j -- NAME=VALUE

jobserver-fds tells which fds make has opened to communicate with the jobs server. To use the ability to use the job server, you must therefore maintain or organize access to these specific fds (or overwrite the environment variable accordingly to indicate them depending on which fd they end with).

NAME=VALUE are the arguments passed by me to the original make.

0
source

More articles:


All Articles