How to use compile_commands.json with python bindings for clang?

Question

I have the following script that tries to print all AST nodes in a given C ++ file. This works great when used in a simple file with a trivial inclusion (header file in the same directory, etc.).

#!/usr/bin/env python from argparse import ArgumentParser, FileType from clang import cindex def node_info(node): return {'kind': node.kind, 'usr': node.get_usr(), 'spelling': node.spelling, 'location': node.location, 'file': node.location.file.name, 'extent.start': node.extent.start, 'extent.end': node.extent.end, 'is_definition': node.is_definition() } def get_nodes_in_file(node, filename, ls=None): ls = ls if ls is not None else [] for n in node.get_children(): if n.location.file is not None and n.location.file.name == filename: ls.append(n) get_nodes_in_file(n, filename, ls) return ls def main(): arg_parser = ArgumentParser() arg_parser.add_argument('source_file', type=FileType('r+'), help='C++ source file to parse.') arg_parser.add_argument('compilation_database', type=FileType('r+'), help='The compile_commands.json to use to parse the source file.') args = arg_parser.parse_args() compilation_database_path = args.compilation_database.name source_file_path = args.source_file.name clang_args = ['-x', 'c++', '-std=c++11', '-p', compilation_database_path] index = cindex.Index.create() translation_unit = index.parse(source_file_path, clang_args) file_nodes = get_nodes_in_file(translation_unit.cursor, source_file_path) print [p.spelling for p in file_nodes] if __name__ == '__main__': main() 

However, when I run the script, I get clang.cindex.TranslationUnitLoadError: Error parsing translation unit. and provide a valid C ++ file with the compile_commands.json file in its parent directory. This code works and builds fine using CMake with clang, but I can't figure out how to pass an argument to correctly point to compile_commands.json.

It is also difficult for me to find this option in the clang documentation and was unable to get -ast-dump to work. However, the clang check works just fine by simply passing the file path!

Answer 1

Your own accepted answer is incorrect. libclang maintains compilation databases and so does cindex.py , the python libclang binding.

The main source of confusion may be that the compilation flags that libclang knows / uses are just a subset of all the arguments that can be passed to the clang interface. The compilation database is supported, but does not work automatically: it must be loaded and requested manually. Something like this should work:

 #!/usr/bin/env python from argparse import ArgumentParser, FileType from clang import cindex compilation_database_path = args.compilation_database.name source_file_path = args.source_file.name index = cindex.Index.create() # Step 1: load the compilation database compdb = cindex.CompilationDatabase.fromDirectory(compilation_database_path) # Step 2: query compilation flags try: file_args = compdb.getCompileCommands(source_file_path) translation_unit = index.parse(source_file_path, file_args) file_nodes = get_nodes_in_file(translation_unit.cursor, source_file_path) print [p.spelling for p in file_nodes] except CompilationDatabaseError: print 'Could not load compilation flags for', source_file_path 

Answer 2

From what I can tell, Libclang does not support compilation database, but Libtooling does. To get around this, I took the path to compile_commands.json as an argument and eventually parsed it myself to find the file of interest and the corresponding ones ( -I and -isystem ).