Compiling ANTLR ISO-SQL-2003 Grammar

Question

I am trying to compile an ISO-SQL 2003 grammar here http://www.antlr3.org/grammar/1304304798093/SQL2003_Grammar.zip . All three versions can be found here http://www.antlr3.org/grammar/list.html .

These are the steps that I followed

• java -jar antlr-3.3-complete.jar -Xmx8G -Xwatchconversion sql2003Lexer.g
  
• java -jar antlr-3.3-complete.jar -Xmx8G -Xwatchconversion sql2003Parser.g
• javac ANTLRDemo.java

ANTLRDemo.java file:

import org.antlr.runtime.*; import java.io.IOException; import java.nio.file.Files; import java.nio.file.Path; import java.nio.file.Paths; public class ANTLRDemo { static String readFile(String path) throws IOException { byte[] encoded = Files.readAllBytes(Paths.get(path)); return new String(encoded, "UTF-8"); } public static void main(String[] args) throws Exception { ANTLRStringStream in = new ANTLRStringStream( readFile(args[0]) ); sql2003Lexer lexer = new sql2003Lexer(in); CommonTokenStream tokens = new CommonTokenStream(lexer); sql2003Parser parser = new sql2003Parser(tokens); parser.eval(); } } 

The first two steps work fine, then when compiling my main class I get a lot of errors related to Java syntax, like these:

./sql2003Parser.java:96985: error: not approval $ UnsignedInteger.text == '1'. / sql 2003Parser.java:96985: error: ';' expected $ UnsignedInteger.text == '1'. / sql 2003Parser.java:102659: error: open character literal if (! ((((Unsigned_Integer3887! = null? Unsigned_Integer3887.getText (): null) == '01'))) {

Please let me know if I am doing something wrong in setting up the parser.
It would be helpful if someone could show me exactly how to tune this grammar with ANTLR.

Edit: after a bit more trivial, I think these errors are caused by the actions contained in the lexer and parser rules. Is there a safe way to overcome this?

Answer 1

You are not doing anything wrong, ANTLR could never generate a working Java parser from these grammar files.

According to Douglas Godfrey, antlr-interest in October 2011:

I created a parser and lexer. they generate and compile successfully on my machine with a bunch of 8 GB allocated for Antlr.

...

I do not believe that one day it will be possible to get a working parser in Java. On the other hand, a C parser is quite possible.

Answer 2

Yes, basically you're right. Grammar is broken. But there is also an error in your ANTLRDemo.java as the no eval() method in the Parser class. You must call a method with the name of any parser grammar rule, for example. query_specification() . In the grammar itself, there were some errors that looked like a typo, some calls to the undefined method Java error() , skip() calls in the parser, which only fit in lexer. You see all the fixes in this commit . Ive posted my research on this GitHub repository .

I started to fix obvious grammar errors, which led to compilation errors in the generated Java code. I had the same mistakes as you. In the end, I fixed all the Java syntax errors, but ran into another that cannot be installed directly because it is due to a JVM limitation, compilation error: code too large . Reading the ANTLR mailing list contained a hint to extract some static members of huge classes into separate interfaces and to "implement" them in order to have a kind of multiple inheritance. With trial and error, I ended up with 6 interfaces, numbered by the parser in sql2003Parser.java .

But there are still two problems:

• Rule of incorrect start. Douglas Godfrey wrote a grammar that begins with the sql2003Parser rule. Unfortunately, if you invoke the parser using this run rule, it will not correctly parse even the simplest select a from b . Therefore, I query_specification parser using the query_specification rule to parse only SELECT .
• Some other grammar errors. I didn't go too deep into grammar, but query_specification unable to query_specification some random complex SQL queries.