Extending a function-like macro creates a separate token

Question

I just found out that gcc seems to treat the result of expanding a functionally similar macro as a separate token. Here is a simple example showing the behavior of gcc:

#define f() foo void f()_bar(void); void f()bar(void); void f()-bar(void); 

When I execute gcc -E -P test.c (only the preprocessor works), I get the following output:

 void foo _bar(void); void foo bar(void); void foo-bar(void); 

It seems that in the first two definitions, gcc inserts space after the extended macro to make sure it is a separate token. Is this really what is going on here?

Is this mandatory for any standard (I could not find documentation on this topic)?

I want to make a _bar part of the same token. Is there any way to do this? I could use the ## token concatenation operator, but this will require several macro levels (since f () is more complex in real code). I was wondering if there is a simple (and probably more readable) solution.

Answer 1

It seems that in the first two definitions, gcc inserts space after the extended macro to make sure it is a separate token. Is this really what is going on here?

Yes.

Is this mandatory for any standard (I could not find documentation on this topic)?

Yes, although a few more spaces will be allowed to implement tokens.

f()_bar

here you have 4 tokens after lexical analysis (they are actually tokens in front of the processor at this stage, but they can be called tokens): f , ( , ) and _bar .

A functionally selected macro replacement semantics (as defined in C11, 6.10.3) should replace 3 tokens f , ( , ) with the new foo . It is not allowed to work with other tokens and change the last _bar token. For this, the implementation must insert at least one space in order to preserve the _bar token. Otherwise, the result would be foo_bar , which is the only token.

gcc preprocessor has several docs here:

Once the input file is split into tokens, the boundaries of the tokens never change, unless the ## preprocessing operator is used to combine tokens. See Concatenation. For example,

 #define foo() bar foo()baz ==> bar baz not ==> barbaz 

In another case, for example f()-bar , there are 5 tokens: f , ( , ) , - and bar . ( - is the accent token in C, while _ in _bar is just the symbol of the identifier token). The implementation should not insert a marker delimiter (like a space) here, because after replacing the macro, -bar are still considered two separate tokens from the C syntax.

gcc preprocessor ( cpp ) does not insert spaces here simply because it does not need it. In the cpp documentation , on the token interval it is written (on another problem):

However, we would like to keep the spatial insertion to a minimum, both for aesthetic reasons and because it creates problems for people who are still trying to abuse the preprocessor for things like the Fortran source and Makefiles.

I did not consider the solution to your problem in this answer, but I think that you should use the operator explicitly specified for combining tokens: ## marker gluing operator.

Answer 2

The only way I can think (if you cannot use the token concatenation operator ##) is using traditional C preprocessing:

 gcc -E -P -traditional-cpp test.c 

Output:

 void foo_bar(void); void foobar(void); void foo-bar(void); 

Additional Information