Makefile w. Protocol Buffer and Auto Dependency

Question

I have a Makefile something like this:

.SECONDARY: NVCC = nvcc NVCCFLAGS = --gpu-architecture compute_20 CXX = g++ CXXFLAGS = -O3 -std=c++0x -Wall CXXLINT = python cpplint.py CXXLINTFLAGS = --filter=-build/include,-readability/streams,-runtime/sizeof,-whitespace/parens PROTOC = protoc PROTOCFLAGS = --cpp_out=. BINS = my_binary LIBS = -lcublas -lcusparse PROTOS = $(wildcard *.proto) SOURCES = $(wildcard *.cu) HEADERS = $(wildcard *.cuh) PBS = $(PROTOS:%.proto=%.pb) DEPS = $(SOURCES:%.cu=%.d) TESTS = my_test all: lint protos all: deps all: bins protos: ${PBS} deps: ${DEPS} bins: ${BINS} clean: rm -rf *.o *.d *.pb.cc *.pb.h ${BINS} ${TESTS} lint: ${CXXLINT} ${CXXLINTFLAGS} ${SOURCES} ${CXXLINT} ${CXXLINTFLAGS} ${HEADERS} tests: lint protos tests: deps tests: ${TESTS} tests: tests-run tests-run: ${TESTS} for f in $^; do eval "/usr/bin/time -f \"$$f runtime: %E\" ./$$f"; done %: %.o ${NVCC} ${NVCCFLAGS} -o $@ $^ ${LIBS} %.d: %.cu # ${CXXLINT} ${CXXLINTFLAGS} $*.cu ${NVCC} -M -o $*.d $*.cu %.o: %.cu ${NVCC} ${NVCCFLAGS} -c -o $@ $*.cu rm $*.d %.pb: %.proto ${PROTOC} ${PROTOCFLAGS} $*.proto ${CXX} ${CXXFLAGS} -c -o $*.pb.o $*.pb.cc ifneq ($(MAKECMDGOALS),clean) -include ${DEPS} endif 

The problem arises because I cannot generate my DEPS until my proto-target is created. Since the construction of protocol buffers will add a new header file to the tree if it is not done before DEPS, nvcc -M (make dependency) will fail, because it will not be able to find the created * .pb.h, Any ideas how to fix this?

Answer 1

Another solution is to make your dependency files depend on the results of the protocol buffer generation. The following snippet contains all the steps for this, since it is difficult to explain them one at a time, explaining some of the elements below:

 CXX_FLAGS := $(shell pkg-config --cflags protobuf) -xc++ LD_FLAGS := $(shell pkg-config --libs protobuf) PROTOS := $(wildcard *.proto) PROTO_OBJS := $(PROTOS:.proto=.pb.o) BINS := my_binary SRCS := $(wildcard *.cu) OBJS := $(SRCS:.cu=.o) DEPS := $(SRCS:.cu=.d) PBSRCS := $(wildcard *.proto) PBOBJS := $(PROTOS:.proto=.pb.o) PBGENS := $(PROTOS:.proto=.pb.cc) $(PROTOS:.proto=.pb.h) all: $(BINS) clean: rm -f $(BINS) $(OBJS) $(DEPS) $(PBOBJS) $(PBGENS) $(BINS): $(OBJS) $(OBJS): $(DEPS) $(DEPS): $(PBOBJS) .PRECIOUS: $(PBGENS) %.d: %.cu $(CXX) -M $(CXX_FLAGS) $< > $@ %.pb.cc: %.proto protoc --cpp_out=. $< %.pb.o : %.pb.cc $(CXX) $(CXX_FLAGS) -c -o $@ $< %.o: %.cu $(CXX) $(CXX_FLAGS) -c -o $@ $< $(BINS): %: %.o $(CXX) $(LD_FLAGS) -o $@ $(PROTO_OBJS) $^ ifneq ($(MAKECMDGOALS),clean) -include $(DEPS) endif 

The pkg-config command is not required, but is convenient if you want to automatically get compilation and link flags corresponding to protobuf files. Of course, you must add your own flags to this variable.

-xc++ is probably useless for you, but it is used here to be able to work with .cu and interpret them as C ++; even for compilers other than nvcc .

The line $(DEPS): $(PBOBJS) is an indication that protobuf files must be created and compiled before dependencies are created. There are several ways to achieve this, so this is just an example of how to do this.

The .PRECIOUS line tells make to save the generated protobuf files. In this example fragment these files are considered intermediate and as such will be deleted without this line.

I posted this as a separate answer because the previous one and this one do not have much in common.

Answer 2

When creating dependencies, you can continue, even if the header files are missing. Using cpp or gcc , this can be achieved using the -MM -MG options. I have no experience with NVCC, but I have not seen support for these flags in the manual. If so, you can try switching to cpp only to generate dependencies. On the cpp man page:

-MG assumes that missing header files are generated and added to the dependency list without raising the error.

Your template rule %.pb: %.proto , it seems to me, does not seem to be correct. These are %.pb.h and %.pb.cc , which depend on %.proto , and with the current approach make does not know how to resolve any dependencies on %.pb.h because it does not know how to create the .pb.h file .pb.h .