Info: (make) Job Slots

Info Catalog
make: Integrating make
make: Terminal Output
make: Job Slots

 
 13.1 Sharing Job Slots with GNU 'make'
 ======================================
 
 GNU 'make' has the ability to run multiple recipes in parallel (
 Parallel Execution Parallel.) and to cap the total number of parallel
 jobs even across recursive invocations of 'make' (Communicating
 Options to a Sub-'make' Options/Recursion.).  Tools that 'make' invokes
 which are also able to run multiple operations in parallel, either using
 multiple threads or multiple processes, can be enhanced to participate
 in GNU 'make''s job management facility to ensure that the total number
 of active threads/processes running on the system does not exceed the
 maximum number of slots provided to GNU 'make'.
 
    GNU 'make' uses a method called the "jobserver" to control the number
 of active jobs across recursive invocations.  The actual implementation
 of the jobserver varies across different operating systems, but some
 fundamental aspects are always true.
 
    First, only command lines that 'make' understands to be recursive
 invocations of 'make' (How the 'MAKE' Variable Works MAKE
 Variable.) will have access to the jobserver.  When writing makefiles
 you must be sure to mark the command as recursive (most commonly by
 prefixing the command line with the '+' indicator (Recursive Use
 of 'make' Recursion.).
 
    Second, 'make' will provide information necessary for accessing the
 jobserver through the environment to its children, in the 'MAKEFLAGS'
 environment variable.  Tools which want to participate in the jobserver
 protocol will need to parse this environment variable, as described in
 subsequent sections.
 
    Third, every command 'make' starts has one implicit job slot reserved
 for it before it starts.  Any tool which wants to participate in the
 jobserver protocol should assume it can always run one job without
 having to contact the jobserver at all.
 
    Finally, it's critical that tools that participate in the jobserver
 protocol return the exact number of slots they obtained from the
 jobserver back to the jobserver before they exit, even under error
 conditions.  Remember that the implicit job slot should *not* be
 returned to the jobserver!  Returning too few slots means that those
 slots will be lost for the rest of the build process; returning too many
 slots means that extra slots will be available.  The top-level 'make'
 command will print an error message at the end of the build if it
 detects an incorrect number of slots available in the jobserver.
 
    As an example, suppose you are implementing a linker which provides
 for multithreaded operation.  You would like to enhance the linker so
 that if it is invoked by GNU 'make' it can participate in the jobserver
 protocol to control how many threads are used during link.  First you
 will need to modify the linker to determine if the 'MAKEFLAGS'
 environment variable is set.  Next you will need to parse the value of
 that variable to determine if the jobserver is available, and how to
 access it.  If it is available then you can access it to obtain job
 slots controlling how much parallelism your tool can use.  Once done
 your tool must return those job slots back to the jobserver.
make: Job Slots

Menu
