Most compilers are divided into parts: the compiler interface is called a lexical analyzer or scanner. This part of the compiler reads the actual characters and creates tokens. This one has a state machine that decides upon seeing the escape character whether it is genuine (for example, when it appears inside a string) or changes the next character. The token is displayed accordingly as an escape character or some other token (for example, a tab or a new line) in the next part of the compiler ( analyzer ). A state machine can group multiple characters into a token.
source
share