I have a problem with the generation stage of cmake make files, which is slow, which seems like this unanswered question:
CMake slowly generates makefiles
My project consists of a top-level CMakeLists.txt that uses add_subdirectory () to add various subprojects for a separate library and executable components.
For this component, CMakeLists.txt contains something like:
add_library(mylib SHARED sourceFile1.cpp sourceFile2.cpp ... )
I can only create the contents of this directory using:
make mylib
If I change CMakeLists.txt in a subdirectory (which I did a lot as part of moving from pure Makefiles to cmake), then run make so that it correctly restarts cmake to update the configuration, as if I ran make rebuild_cache. However, I notice that he is actually reconfiguring the entire project. I really want cmake to be smart enough to know that it only needs to restore the Makefile in the current directory and subdirectories.
Is there a better way to structure a cmake project to achieve this? I see that some people use project () for every CMakeLists.txt in every subproject. In general, is this a good idea?
Alternatively / Additionally is there a way to speed up the cmake generation step? (I currently have 60 s +)
Bonus points if you want to discuss why cmake itself should or shouldn't work in parallel (imagine cmake -j)
I added the meson-build tag as a modest generosity, until I got enough attention to guarantee an answer. This is such a problem that can cause people to switch to building systems on meson-build (provided that it does not have similar problems) or something like that.
Perhaps the correct answer is that it cannot be executed without changing the source to cmake. To earn generosity, although I need an explanation as to how cmake works and / or where it is wrong.
Clarification: In my case, a slow generation step. The setup itself is fast enough, but cmake hangs for quite some time between the "Configuration Complete" and "Generation Complete" outputs.
To fully restore the cache, I run:
make -n rebuild_cache
Running CMake to regenerate build system ...
using makefile generator
- FOOBAR_VERSION: 02/01/03
- Configuring done
- Generating done
- Build files have been written to: / home / brucea / work / depot / emma / main / cmake
real 74.87
user 1.74
sys 1.02
under the hood this is done:
cmake -H<source dir> -B<build dir>
I assume that -B is a synonym for --build none of them are described correctly in the documentation (@todo report error) -H is the root of the source directory (not the same as --help as they would have documented to believe you)
Its quick to get to the exit "Configuring done", but slowly from there: For example.
15:44:14 execve ("/ usr / local / bin / cmake",
> grep Generating cmake_strace.log
> grep "Configuring" cmake_strace.log
15:44:15 write (1, "- Configuring done \ n", 20-- Configuring done
15:45:01 write (1, "- Generating done \ n", 19-- Generating done
> grep "Build files" cmake_strace.log
15:45:22 write (1, "- Build files have been written" ..., 77-- Build files have been written to:
If you are editing a single CMakeLists.txt file in a subdirectory then run make -n:
cd /home/project/cmake && /usr/local/bin/cmake -H/home/project/cmake -B/home/project/cmake --check-build-system CMakeFiles/Makefile.cmake 0
- check-build-system - another undocumented option.
The effect is the same - regenerate the entire assembly system, not only the current subtree. There is no difference in behavior between the original and original versions.
If I run a trace, for example:
strace -r cmake --trace -H/home/project/cmake -B/home/project/cmake 2>&1 | tee cmake_rebuild_cache.log sort -r cmake_rebuild_cache.log | uniq
Most of the time spent seems to be spent on (or between) open, available, and canceled calls. The length of each task is quite variable, but there are a huge number of them. I have no idea what the Labels.json and Labels.txt files are (something internal for cmake)
1 run:
49.363537 open ("/ home / projectbar / main / test / foo2bar / CMakeFiles / test2.foo2bar.testViewingSource1.dir / build.make", O_RDONLY) = 5
1.324777 access ("/ home / projectbar / main / test / performance / CMakeFiles / performancetest.chvcthulhu.testChvcthulhuPerformance2.dir", R_OK) = 0
0.907807 access ("/ home / projectbar / main / test / foo2bar / CMakeFiles / test2.foo2bar.testPeripheralConnection2.dir", R_OK) = 0
0.670272 unlink ("/ home / projectbar / main / src / foo2bar / Foo2Bar / CMakeFiles / foo2bar_lib.dir / progress.make.tmp") = -1 ENOENT (No such file or directory)
0.600272 access ("/ home / projectbar / main / test / foo2bar / testFilesModel2.ok", R_OK) = 0
0.599010 access ("/ home / projectbar / main / test / hve2snafu / testInvalidByte2c.ok", R_OK) = 0
0.582466 read (5, "openjdk version \" 1.8.0_71 \ "\ nOpenJ" ..., 1024) = 130
0.570540 writev (3, [{"# CMAKE generated file: DO NOT E" ..., 8190}, {"M", 1}], 2) = 8191
0.553576 close (4) = 0
0.448811 unlink ("/ home / projectbar / main / test / snafu2hve / CMakeFiles / test2.snafu2hve.testNoProbes2.dir / progress.make.tmp") = -1 ENOENT (No such file or directory)
0.431559 access ("/ home / projectbar / main / src / foo2bar / Foo2Bar / CMakeFiles / foo2bar_lib.dir", R_OK) = 0
0.408003 unlink ("/ home / projectbar / main / test / lachesis / CMakeFiles / test2.lachesis.testBadSequenceNumber1.dir / progress.make.tmp") = -1 ENOENT (No such file or directory)
0.407120 write (4, "# The set of languages for which" ..., 566) = 566
0.406674 write (3, "# CMAKE generated file: DO NOT E" ..., 675) = 675
0.383892 read (3, "ewingPeriod.cpp.o -c / home / bruce" ..., 8191) = 8191
0.358490 unlink ("/ home / projectbar / main / cmake / CMakeFiles / mklinks.chvdiff.dir / progress.make.tmp") = -1 ENOENT (No such file or directory)
another run of the same command:
2.009451 unlink ("/ home / projectbar / main / cmake / CMakeFiles / mklinks.lachesis.dir / Labels.json") = -1 ENOENT (No such file or directory)
) = 20
) = 19
1.300387 access ("/ home / projectbar / main / test / chvedit / CMakeFiles / test2.chvedit.tefooultiMatchFactoringEdit2.dir", R_OK) = 0
1.067957 access ("/ home / projectbar / main / test / chvedit / CMakeFiles / test2.chvedit.tefooultiMatchFactoringEdit2.dir", R_OK) = 0
) = 1
0.885854 unlink ("/ home / projectbar / main / src / gorkyorks2bar / CMakeFiles / doxygen.correct.gorkyorks2bar.dir / Labels.json") = -1 ENOENT (No such file or directory)
0.854539 access ("/ home / projectbar / main / test / reportImpressions / ReportImpressions / CMakeFiles / testsuite1_reportImpressions.dir", R_OK) = 0
0.791741 unlink ("/ home / projectbar / main / cmake / CMakeFiles / mklinks.bar_models.dir / progress.make.tmp") = -1 ENOENT (No such file or directory)
0.659506 unlink ("/ home / projectbar / main / cmake / CMakeFiles / mklinks.dir / progress.make.tmp") = -1 ENOENT (No such file or directory)
0.647838 unlink ("/ home / projectbar / main / test / libyar / YarModels / CMakeFiles / testsuite1_yarmodels.dir / Labels.txt") = -1 ENOENT (No such file or directory)
0.620511 unlink ("/ home / projectbar / main / test / libyar / YarModels / CMakeFiles / testsuite1_yarmodels.dir / Labels.json") = -1 ENOENT (No such file or directory)
0.601942 unlink ("/ home / projectbar / main / cmake / CMakeFiles / mklinks.lachesis.dir / Labels.txt") = -1 ENOENT (No such file or directory)
0.591871 access ("/ home / projectbar / main / src / runbardemo / simple_demo / CMakeFiles", R_OK) = 0
0.582448 write (3, "CMAKE_PROGRESS_1 = \ n \ n", 21) = 21
0.536947 write (3, "CMAKE_PROGRESS_1 = \ n \ n", 21) = 21
0.499758 unlink ("/ home / projectbar / main / test / foo2bar / CMakeFiles / test2.foo2bar.testInputDirectory1.dir / progress.make.tmp") = -1 ENOENT (No such file or directory)
0.458120 unlink ("/ home / projectbar / main / test / yak2dcs / CMakeFiles / test2.yak2dcs.testsuite2.dir / progress.make.tmp") = -1 ENOENT (No such file or directory)
0.448104 unlink ("/ home / projectbar / main / test / reportImpressions / CMakeFiles / test2.reportImpressions.dir / progress.make.tmp") = -1 ENOENT (No such file or directory)
0.444344 access ("/ home / projectbar / main / src / bananas / CMakeFiles / bin.bananas.dir", R_OK) = 0
0.442685 unlink ("/ home / projectbar / main / test / rvedit / CMakeFiles / test2.rvedit.tefooissingOptionValue.dir / progress.make.tmp") = -1 ENOENT (No such file or directory)
0.425604 unlink ("/ home / projectbar / main / test / listdcs / CMakeFiles / test2.listdcs.testListCalls5.dir / progress.make.tmp") = -1 ENOENT (No such file or directory)
0.391163 access ("/ home / projectbar / main / src / siedit / CMakeFiles / siedit.dir", R_OK) = 0
0.362171 access ("/ home / projectbar / main / test / foo2bar / CMakeFiles / test2.foo2emma.testHowResults6.dir", R_OK) = 0
Note that the ninja generator is much faster (although not brilliant) For example.
/usr/bin/time -p ninja rebuild_cache
ninja: warning: multiple rules generate ../src/ams2yar/ams2yar. builds involving this target will not be correct; continuing anyway [-w dupbuild = warn]
ninja: warning: multiple rules generate ../src/vox/vox. builds involving this target will not be correct; continuing anyway [-w dupbuild = warn]
ninja: warning: multiple rules generate ../src/bananas/bananas. builds involving this target will not be correct; continuing anyway [-w dupbuild = warn]
ninja: warning: multiple rules generate ../src/fidlertypes2fidlerinfo/fidlertypes2fidlerinfo. builds involving this target will not be correct; continuing anyway [-w dupbuild = warn]
ninja: warning: multiple rules generate ../src/mkrundir/mkrundir. builds involving this target will not be correct; continuing anyway [-w dupbuild = warn]
ninja: warning: multiple rules generate ../src/runyar/runyar. builds involving this target will not be correct; continuing anyway [-w dupbuild = warn]
ninja: warning: multiple rules generate ../src/runyardemo/runyardemo. builds involving this target will not be correct; continuing anyway [-w dupbuild = warn]
[1/1] Running CMake to regenerate build system ...
Generator = Ninja
- FOO_VERSION: 02/01/03
- Configuring done
- Generating done
- Build files have been written to: / home / project / cmake / build
real 12.67
user 1.01
sys 0.31
Please note that the project is not yet completely ready for the ninja, as there are errors such as:
ninja: warning: multiple rules generate ../src/runfoobardemo/runfoobardemo. builds involving this target will not be correct; continuing anyway [-w dupbuild=warn]
and
ninja: error: dependency cycle: ../src/foobar -> ../src/foobar/CMakeFiles/foobar -> ../src/ams2emma/foobar
. This question is really about why the Makefile generator is slow. I'm not sure the ninja problems show useful clues here or red herrings.
Building a cmake with a lot of optimizations doesn't help. Based on my trace, it and time out are unlikely. User time and therefore time spent inside cmake code itself is pretty low. (see, for example, What does “real”, “user”, and “sys” mean in time output (1)? ) Here is what I tried for completeness:
export CXX_FLAGS="-O3 -ftree-vectorise -msse2" cmake -DCMAKE_BUILD_TYPE=RELEASE
In fact, using a more optimized cmake does an accelerated setup, but in my case this is the slower part of generate . From the moment this step is somehow related to I / O, it will be seen.
I decided to explore Florian's idea that using memory to stream files for temporary files can make a difference. I decided to try a simple route and hacked cmake to write .tmp files in ramdisk. Then I went off the whole pig and tried to create a build system on ramdisk
sudo mkdir /mnt/ramdisk sudo mount -t tmpfs -o size=512m tmpfs /mnt/ramdisk /usr/bin/time -p cmake -H/<source> -B/mnt/ramdisk/build
I was very surprised to find that it does not matter for a wall clock:
real 59.61 user 1.55 sys 0.62 >du -sh /mnt/ramdisk/build/ 4.4M /mnt/ramdisk/build/
Similar to ramfs:
real 51.09 user 1.58 sys 0.50
What could be here? I guessed the subprocesses, but I can't figure out which subprocesses the wall clock consumes, if any. They look very short.
for completeness, there is some output from perf (cmake built with -fno-omit-frame-pointer):
perf record -g --call-graph dwarf cmake -H<source> -B<build> perf report -g graph
Samples: 17K of event 'cycles', Event count (approx.): 14363392067
Children Self Command Shared Object Symbol
+ 65.23% 0.00% cmake cmake [.] Do_cmake
+ 65.02% 0.00% cmake cmake [.] Cmake :: Run
+ 60.32% 0.00% cmake cmake [.] Main
+ 59.82% 0.00% cmake libc-2.17.so [.] __Libc_start_main
+ 57.78% 0.00% cmake cmake [.] _Start
+ 55.04% 0.00% cmake cmake [.] CmGlobalUnixMakefileGenerator3 :: Generate
+ 54.56% 0.00% cmake cmake [.] Cmake :: Generate
+ 49.90% 0.00% cmake cmake [.] CmGlobalGenerator :: Generate
+ 38.87% 0.02% cmake cmake [.] CmLocalUnixMakefileGenerator3 :: Generate
+ 18.65% 0.01% cmake cmake [.] CmMakefileTargetGenerator :: WriteTargetBuildRules
+ 17.05% 0.02% cmake cmake [.] CmMakefile :: ExecuteCommand
+ 16.99% 0.01% cmake cmake [.] CmMakefile :: ReadListFile
+ 16.84% 0.01% cmake cmake [.] CmCommand :: InvokeInitialPass
+ 16.79% 0.00% cmake cmake [.] CmMakefile :: Configure
+ 14.71% 0.00% cmake cmake [.] CmMakefile :: ConfigureSubDirectory
+ 14.67% 0.05% cmake cmake [.] CmMacroHelperCommand :: InvokeInitialPass
+ 14.27% 0.02% cmake cmake [.] CmMakefileUtilityTargetGenerator :: WriteRuleFiles
+ 13.91% 0.00% cmake cmake [.] CmGlobalGenerator :: Configure
+ 13.50% 0.05% cmake cmake [.] CmOutputConverter :: Convert
+ 13.48% 0.00% cmake cmake [.] CmAddSubDirectoryCommand :: InitialPass
+ 13.46% 0.00% cmake cmake [.] CmMakefile :: AddSubDirectory
+ 12.91% 0.00% cmake cmake [.] CmGlobalUnixMakefileGenerator3 :: Configure
+ 12.82% 0.00% cmake cmake [.] Cmake :: ActualConfigure
+ 10.90% 0.00% cmake cmake [.] Cmake :: Configure
+ 10.55% 0.02% cmake cmake [.] CmMakefileTargetGenerator :: WriteObjectRuleFiles
+ 10.35% 0.09% cmake cmake [.] CmLocalUnixMakefileGenerator3 :: WriteMakeRule
+ 9.76% 0.03% cmake cmake [.] CmMakefileTargetGenerator :: WriteObjectBuildFile
+ 7.97% 0.00% cmake cmake [.] CmMakefileLibraryTargetGenerator :: WriteRuleFiles
+ 7.93% 0.00% cmake cmake [.] CmMakefileExecutableTargetGenerator :: WriteRuleFiles
+ 7.88% 0.00% cmake cmake [.] CmLocalUnixMakefileGenerator3 :: WriteLocalMakefile
+ 7.68% 0.02% cmake [kernel.kallsyms] [k] sysret_audit
+ 7.60% 0.05% cmake [kernel.kallsyms] [k] __audit_syscall_exit
+ 7.40% 0.08% cmake cmake [.] Cmsys :: SystemTools :: CollapseFullPath
And the perforated report -g graph -no-children:
+ 2.86% cmake libc-2.17.so [.] _Int_malloc
+ 2.15% cmake libc-2.17.so [.] __Memcpy_ssse3_back
+ 2.11% cmake [kernel.kallsyms] [k] find_next_bit
+ 1.84% cmake libc-2.17.so [.] __Memcmp_sse4_1
+ 1.83% cmake libc-2.17.so [.] _Int_free
+ 1.71% cmake libstdc ++. So.6.0.20 [.] Std :: __ ostream_insert>
+ 1.18% cmake libstdc ++. So.6.0.20 [.] Std :: basic_string, std :: allocator> :: ~ basic_string
+ 1.13% cmake libc-2.17.so [.] Malloc
+ 1.12% cmake cmake [.] CmOutputConverter :: Shell__ArgumentNeedsQuotes
+ 1.11% cmake libstdc ++. So.6.0.20 [.] Std :: string :: compare
+ 1.08% cmake libc-2.17.so [.] __Strlen_sse2_pminub
+ 1.05% cmake cmake [.] Std :: string :: _ S_construct
+ 1.04% cmake cmake [.] Cmsys :: SystemTools :: ConvertToUnixSlashes
+ 0.97% cmake cmake [.] Yy_get_previous_state
+ 0.87% cmake cmake [.] CmOutputConverter :: Shell__GetArgument
+ 0.76% cmake libstdc ++. So.6.0.20 [.] Std :: basic_filebuf> :: xsputn
+ 0.75% cmake libstdc ++. So.6.0.20 [.] Std :: string :: size
+ 0.75% cmake cmake [.] CmOutputConverter :: Shell__SkipMakeVariables
+ 0.74% cmake cmake [.] CmOutputConverter :: Shell__CharNeedsQuotesOnUnix
+ 0.73% cmake [kernel.kallsyms] [k] mls_sid_to_context
+ 0.72% cmake libstdc ++. So.6.0.20 [.] Std :: basic_string, std :: allocator> :: basic_string
+ 0.71% cmake cmake [.] CmOutputConverter :: Shell__GetArgumentSize
+ 0.65% cmake libc-2.17.so [.] Malloc_consolidate
+ 0.65% cmake [kernel.kallsyms] [k] mls_compute_context_len
+ 0.65% cmake cmake [.] CmOutputConverter :: Shell__CharNeedsQuotes
+ 0.64% cmake cmake [.] CmSourceFileLocation :: Matches
+ 0.58% cmake cmake [.] CmMakefile :: ExpandVariablesInStringNew
+ 0.57% cmake cmake [.] Std :: __ deque_buf_size
+ 0.56% cmake cmake [.] CmCommandArgument_yylex
+ 0.55% cmake cmake [.] Std :: vector> :: size
+ 0.54% cmake cmake [.] Cmsys :: SystemTools :: SplitPath
+ 0.51% cmake libstdc ++. So.6.0.20 [.] Std :: basic_streambuf> :: xsputn