Libtooling cannot find stddef.h and other headers

Question

I am writing a tool for analyzing C-family source code projects, mainly following these two tutorials 1 2 on clang 3.4 (trunk 192426) on ubuntu 12.04.

Based on the official guide , he says that I could pass compile_commands.json to -p , however, if I only type $ ./main -p [path of compile_commands.json] , he will complain about the lack of positional arguments. It seems like I still need to pass all the file names as arguments, which is impractical if the project is really huge. I prefer it to be able to simply parse all the files specified in compile_commands.json without a request, but cannot find out how to enable this.

Since I cannot find a tutorial for CommonOptionsParser to do any setup, I use CompilationDatabase . There is a dummy visitor returning true for VisitStmt , VisitDecl and VisitType , so I will skip this. The main function is pretty simple:

 int main(int argc, const char **argv) { string errorMsg = ""; CompilationDatabase *cd = CompilationDatabase::autoDetectFromDirectory (argv[1], errorMsg); ClangTool Tool(*cd, cd->getAllFiles()); int result = Tool.run(newFrontendActionFactory<ExampleFrontendAction>()); return result; } 

I choose opencv for parsing since using cmake gaurantee compile_commands.json is correct (right?). However, many errors appear (attached at the end). LibTooling complains that it cannot find stdarg.h , stddef.h and emmintrin.h . This is a FAQ for clang, but it says why this will happen, but did not say how to solve it when using libtooling. Pass all arguments to clang -### , since clang can solve this, but how do I pass these arguments when using libtooling?

 # include <stdarg.h> ^ 1 error generated. Error while processing /home/jcwu/repos/opencv/3rdparty/openexr/IlmImf/ImfCompressionAttribute.cpp. In file included from /home/jcwu/repos/opencv/3rdparty/libjpeg/jmemansi.c:16: /home/jcwu/repos/opencv/3rdparty/libjpeg/jinclude.h:35:10: fatal error: 'stddef.h' file not found #include <stddef.h> ^ 1 error generated. Error while processing /home/jcwu/repos/opencv/3rdparty/libjpeg/jmemansi.c. error: no suitable precompiled header file found in directory '/home/jcwu/repos/opencv/modules/legacy/precomp.hpp.gch' 1 error generated. Error while processing /home/jcwu/repos/opencv/modules/legacy/src/hmmobs.cpp. In file included from /home/jcwu/repos/opencv/3rdparty/libwebp/enc/quant.c:17: In file included from /home/jcwu/repos/opencv/3rdparty/libwebp/enc/../dsp/../enc/vp8enci.h:17: /usr/include/string.h:34:10: fatal error: 'stddef.h' file not found #include <stddef.h> ^ 1 error generated. Error while processing /home/jcwu/repos/opencv/3rdparty/libwebp/enc/quant.c. In file included from /home/jcwu/repos/opencv/modules/imgproc/opencv_test_imgproc_pch_dephelp.cxx:1: In file included from /home/jcwu/repos/opencv/modules/imgproc/test/test_precomp.hpp:12: In file included from /usr/lib/gcc/x86_64-linux-gnu/4.6/../../../../include/c++/4.6/iostream:40: In file included from /usr/lib/gcc/x86_64-linux-gnu/4.6/../../../../include/c++/4.6/ostream:40: In file included from /usr/lib/gcc/x86_64-linux-gnu/4.6/../../../../include/c++/4.6/ios:39: In file included from /usr/lib/gcc/x86_64-linux-gnu/4.6/../../../../include/c++/4.6/iosfwd:42: In file included from /usr/lib/gcc/x86_64-linux-gnu/4.6/../../../../include/c++/4.6/bits/postypes.h:42: In file included from /usr/lib/gcc/x86_64-linux-gnu/4.6/../../../../include/c++/4.6/cwchar:46: /usr/include/wchar.h:40:11: fatal error: 'stdarg.h' file not found # include <stdarg.h> 

==== Update ====

Read the source code of CommonOptionsParser.cpp. It uses FixedCompilationDatabase to guess CompilationDatabase by arguments after - and then pass arguments to - for custom (-p only in CommonOptionParser) options. In my case, compile_commands.json is required, so I can skip using CommonOptionsParser.

So my problem boils down to how to pass these parameters from "clang - ###" to LibTooling when I have compile_commands.json? Should I call a shell command for every file I want to parse?

==== Update ====

I think changing compile_commands.json might be easier. I'm not sure why the compile_commands.json generated by CMake will not correctly contain the system header file folder, since the Makefile generated by this CMakeList.txt can compile correctly, why compile_commands.json skips a lot of things.

Answer 1

I ran into a similar problem using python bindings.

[<Diagnostic severity 4, location <SourceLocation file '/usr/include/stdio.h', line 33, column 11>, spelling "'stddef.h' file not found">]

In the "Tips" section in

http://clang.llvm.org/docs/LibTooling.html

they mention that includes path by default

 $(dirname /path/to/tool)/../lib/clang/3.3/include 

The idea here seems to be that it is expected that your tool will be executed from the bin directory, which also contains the clang executable itself. Typically, this will be a system directory, so one of them will have a lib directory that contains the clang/3.4/includ e clang/3.4/includ . So I manually included $(which clang)../lib/clang/3.4/include in the parser. In python, it looks something like this:

 translation_unit = index.parse("test.cc",["-I/home/archgoon/usr/local/lib/clang/3.4/include"]) 

As a result, translation_unit.diagnostics was empty.

Answer 2

In my case, I was getting an error by installing clang-tidy but not clang also on Ubuntu.

Answer 3

Someone answered me that the compilation database should be autonomous. First I need to make sure that compile_commands.json is generated using clang, and I could use clang to create opencv.

I set these environment variables

 export CC=/home/jcwu/repos/llvm-release/Release/bin/clang export CXX=/home/jcwu/repos/llvm-release/Release/bin/clang++ export C_INCLUDE_PATH=/usr/local/include:/home/jcwu/repos/llvm-release/Release/lib/clang/3.4/include:/usr/include/x86_64-linux-gnu:/usr/include # these are from clang -v -c files.cpp export CPLUS_INCLUDE_PATH=/usr/local/include:/home/jcwu/repos/llvm-release/Release/lib/clang/3.4/include:/usr/include/x86_64-linux-gnu:/usr/include 

then regenerate compile_commands.json, it can find stddef.h, but a new problem appears

 [ 31%] Building CXX object modules/ts/CMakeFiles/opencv_ts.dir/src/ts.cpp.o In file included from /home/jcwu/repos/opencv/modules/ts/src/ts.cpp:116: /usr/include/setjmp.h:60:12: error: conflicting types for '__sigsetjmp' extern int __sigsetjmp (struct __jmp_buf_tag __env[1], int __savemask) __THROWNL; ^ /usr/include/pthread.h:727:12: note: previous declaration is here extern int __sigsetjmp (struct __jmp_buf_tag *__env, int __savemask) __THROW; ^ 1 error generated. make[2]: *** [modules/ts/CMakeFiles/opencv_ts.dir/src/ts.cpp.o] Error 1 make[1]: *** [modules/ts/CMakeFiles/opencv_ts.dir/all] Error 2 make: *** [all] Error 2 

I cannot use clang to create opencv due to a type conflict or two system header files. Havn did not understand how to solve this.

Answer 4

I ran into a similar problem that the stddef.h panic did not detect when I call clang-tidy against compile_commands.json. In my case, the reason is that clang-tidy was based on llvm version 7 , and clang++ was based on llvm version 4.0 , after changing clang-tidy to the correct version, the problems disappeared.

Answer 5

Please use the appropriate clang version for your tools and the first argument to the compile command (and use the correct path).

For example:

  std::vector<std::string> Args {"/home/theuser/.local/llvm-8.0.0/bin/clang-8", "-c", "a.cpp"}; llvm::IntrusiveRefCntPtr<clang::FileManager> fileManager(new clang::FileManager(clang::FileSystemOptions())); clang::tooling::ToolInvocation invoker(Args, new BlankAction(), fileManager.get()); invoker.run(); 

If you use the clang 8.0 library to build this tool, it will succeed. If you use other versions (clang 3.6, etc.), this may lead to this error.