Using CMake, how do I get verbose output from CTest?

Question

I use CMake to build my project. I added a binary unit test using the Boost unit testing framework. This binary contains all unit tests. I added this binary to run CTest:

ADD_EXECUTABLE( tftest test-main.cpp ) ENABLE_TESTING() ADD_TEST( UnitTests tftest) 

But the build output in Visual Studio shows only the result of running CTest:

  Start 1: UnitTests 1/1 Test #1: UnitTests ................***Failed 0.05 sec 0% tests passed, 1 tests failed out of 1 

This is not very useful because I cannot see which test failed. If I run ctest manually from the command line with --verbose , I get output from the Boost unit test, which reports that it actually failed:

 1: Test command: tftest.exe 1: Test timeout computed to be: 9.99988e+006 1: Running 4 test cases... 1: test-main.cpp(20): error in "sanity_check3": check 1 == 2 failed 1: 1: *** 1 failure detected in test suite "Master Test Suite" 1/1 Test #1: UnitTests ................***Failed 0.00 sec 

So what do I need to change in CMakeLists.txt so that CTest always starts with --verbose ? Is there a better way to use Boost unit tests with CMake / CTest?

Answer 1

You can set the environment variable CTEST_OUTPUT_ON_FAILURE , which will display any output from the test program if the test fails. One way to do this using Makefiles and the command line is as follows:

 env CTEST_OUTPUT_ON_FAILURE=1 make check 

This question and answer of Stack shows how to set environment variables in Visual Studio.

Answer 2

You can call ctest directly after clumping and creating your project.

 ctest --verbose 

Answer 3

• You can check the Testing/Temporary subfolder. It is automatically created after running the make-test. This folder contains two files: LastTest.log and LastTestsFailed.log . LastTest.log contains the desired result for launch tests. LastTestFailed.log contains the names of the failed tests. Therefore, you can check them manually after doing make test .

• The second way is to force ctest to display the contents of the log files after running the tests:

  • put in the build dir (from which you run make test ) the CTestCustom.ctest file (you can do this with the configure file command, for example) with the following contents

    CTEST_CUSTOM_POST_TEST ("cat Testing / Temporary / LastTest.log")

Instead of cat, you can use any Windows cmd command that does similar things.

2. run make test again and make a profit!

You can find more information on configuring ctest here . Just go to the "Configuring cmake" section. Good luck

Answer 4

I had to add a “control” goal myself. doing tests for some reason does nothing. So what I did (as suggested somewhere in stackoverflow) - I added this target manually. To get detailed output, I just wrote it as:

 add_custom_target(check COMMAND ${CMAKE_CTEST_COMMAND} --verbose) 

Answer 5

make check CTEST_OUTPUT_ON_FAILURE=TRUE

Answer 6

There is a very simple solution (which for some reason is difficult to find using a Google search):

 ctest --output-on-failure 

If you are using CMake with the Visual Studio folder open function, you can add

 "ctestCommandArgs": "--output-on-failure" 

customizing your build configuration.

Answer 7

My approach is a combination of answers from ony , from zbyszek and from tarc . I use the variable ${CMAKE_COMMAND} (which is set to the absolute path to the cmake executable executable) with the -E env CTEST_OUTPUT_ON_FAILURE=1 argument -E env CTEST_OUTPUT_ON_FAILURE=1 to invoke the actual ctest command with ${CMAKE_CTEST_COMMAND} -C $<CONFIG> . To clarify what is happening, I start with three cmake -E echo to display the current working directory and the ctest command to invoke. This is what I call add_custom_target .

 add_custom_target(check ${CMAKE_COMMAND} -E echo CWD=${CMAKE_BINARY_DIR} COMMAND ${CMAKE_COMMAND} -E echo CMD=${CMAKE_CTEST_COMMAND} -C $<CONFIG> COMMAND ${CMAKE_COMMAND} -E echo ---------------------------------- COMMAND ${CMAKE_COMMAND} -E env CTEST_OUTPUT_ON_FAILURE=1 ${CMAKE_CTEST_COMMAND} -C $<CONFIG> WORKING_DIRECTORY ${CMAKE_BINARY_DIR} DEPENDS ALL_BUILD ) 

This works well with the MSVC IDE, where any test errors appear as clickable compilation errors. See cmake -E env for documentation of cmake -E command-line portable mode. I also add an ALL_BUILD dependency so that all projects are created before calling the check target. (In Linux builds, you may need to replace ALL_BUILD with ALL ; I have not tested this on Linux yet.)

Answer 8

This makes the test output more verbose:

 make test ARGS="-V" 

Answer 9

For people using Visual Studio, here is another variation (hacking) on ​​the topic:

 cmake -E env CTEST_OUTPUT_ON_FAILURE=1 cmake --build . --target RUN_TESTS 

Answer 10

To show the result with an XML file, you must run the test with the following command

 ~$ ctest -T Test 

and we found the result in Testing / 1234123432 / test.xml and other files are also generated in the testing folder