How to profile Antlr grammar

Question

I have an Antlr grammar, which currently is around 1200 lines. It parses the language I want, but for at least one construct it is too slow, even for small input files. The runtime appears to grow exponentially for each added structural item.

I want to know if there are good recommendations for debugging / profiling such performance issues.

I already tried with VisualVM and gave a name to the two methods closCheckingStopState and close_, but this does not come close to understanding what is wrong with the grammar.

Answer 1

I rely on two main elements to analyze and improve grammar performance.

• The latest version of ANTLRWorks 2 includes advanced profiling capabilities. Current restrictions include the following:

  • The profiler does not support languages ​​that require a custom CharStream or TokenStream (for example, for preprocessing input).
  • The profiler does not perform custom embedded actions in the lexer or parser, so your grammar should be able to create a parse tree without relying on these operations. Standard lexer commands, such as -> skip or -> channel(HIDDEN) , are not a problem.
  • The profiler’s output is a table of numbers that is not always understood by most ANTLR users, especially in terms of understanding what you should do in response to numbers.

• I use fork of the main version , which includes a number of optimizations that are not available in the reference version of ANTLR 4. Note. that these features are “restrained” documented, as their only goal today is to support the internal development of ANTLRWorks and GoWorks. For most grammars, this fork performs an approximately equivalent reference version. However, for some well-known grammars, an “optimized” release performs more than 200 times faster than a reference release.

If you could provide grammar and input, in particular, I could do an analysis and try to interpret key pieces of the results.

---

The latest version of ANTLRWorks is distributed through the official NetBeans Update Center. Just run Tools → Plugins , go to Available Plugins and find ANTLRWorks Editor .

To start the profiler, use Run -> Interpret Parser .... The result window is available after the parsing operation by choosing Window → Parser Debugger Controller .

Answer 2

JetBrains IDEA plugin has Profiler option

see: https://github.com/antlr/intellij-plugin-v4/blob/master/README.md

Right click on any rule to check the rule and you will get tabs for

• Parsing tree
• Hierarchy
• Profiles

See sample screenshots below.

The ambiguity lines on the profiler tab help you find ambiguous parsing rules. If you click on such a red line, the rule will be highlighted.

profile tab

Analysis Tree Tab

Answer 3

As Wolfgang Fahl said, IDEA has a great plugin, but that of course just displays the information that your parser has collected.

Therefore, if you cannot use IDEA or, for example, want to perform live profiling, you can do this programmatically, for example:

 public void parseAndProfile(GmmlSaneParser parser) { parser.setProfile(true); // do the actual parsing ParseInfo parseInfo = parser.getParseInfo(); ATN atn = parser.getATN(); for (DecisionInfo di : parseInfo.getDecisionInfo()) { DecisionState ds = atn.decisionToState.get(di.decision); String ruleName = GmmlParser.ruleNames[ds.ruleIndex]; System.out.println(ruleName +" -> " + di.toString()); } } 

Answer 4

If you already have an Android studio, you can use the built-in Antlr V4 plugin to use the Antlr profiler.

The link tutorial works for me http://blog.dgunia.de/2017/10/26/creating-and-testing-an-antlr-parser-with-intellij-idea-or-android-studio/

Android Studion used for testing: 2.3.1