Geek Answers Handbook

Lexical analysis of the Python programming language

Does anyone know where the FLEX or LEX specification file for Python exists? For example, this is the lex specification for the ANSI C programming language: http://www.quut.com/c/ANSI-C-grammar-l-1998.html

FYI, I'm trying to write code highlighting in a Cocoa application. Regex will not do this because I also want the parsing to collapse the code and recognize the blocks.

+7
python lex syntax-highlighting lexical-analysis
source share
3 answers

Lex is usually used only for tokenization, and not for full analysis. Projects that use flex / lex for tokenization typically use yacc / bison for the actual parsing.

You can take a look at ANTLR , a more “modern” alternative to lexx and yacc.

Github's ANTLR repository project containing many ANTLR 4 grammars , including one for Python 3 .

+6
source share

grammar.txt is an official, complete Python grammar - not directly lex compatible, but you should be able to massage it into a suitable form.

+3
source share

Have you considered using one of the existing code markers, such as Pygments ?

0
source share

More articles:


All Articles