Location in syntax trees

Question

When writing a parser, I want to remember the location of the tokens found so that I can tell programmers useful error messages, like in "if-less else on line 23" or "unexpected character on line 45", character 6 "or" variable not defined " or something similar, but as soon as I build the syntax tree, I’ll convert it in several ways, optimizing or expanding some macros. Conversions produce or reorder tokens that do not have a significant location.

Therefore, it seems that the type representing the syntax tree should have two aromas, an aroma with places decorating tokens, and an aroma without tokens. Ideally, we would like to work with a purely abstract syntax tree, as defined in the OCaml book :

# type unr_op = UMINUS | NOT  ;;
# type bin_op = PLUS | MINUS | MULT | DIV | MOD  
             | EQUAL | LESS | LESSEQ | GREAT | GREATEQ | DIFF 
             | AND | OR  ;;
# type expression = 
     ExpInt of int 
   | ExpVar of string
   | ExpStr of string 
   | ExpUnr of unr_op * expression
   | ExpBin of expression * bin_op * expression  ;;
# type command = 
     Rem of string
   | Goto of int 
   | Print of expression
   | Input of string 
   | If of expression * int 
   | Let of string * expression  ;;
# type line = { num : int ; cmd : command }  ;;
# type program = line list  ;;

We should be allowed to completely forget about the places when working on this tree and have special functions to map expressionback to its location (for example), which we could use in case of emergency.

What is the best way to determine this type in OCaml or to handle lexeme positions?

Answer 1

- AST, . :

type expression = {
  expr_desc : expr_desc;
  expr_loc : Lexing.position * Lexing.position; (* start and end *)
}

and expr_desc =
     ExpInt of int 
   | ExpVar of string
   | ExpStr of string 
   | ExpUnr of unr_op * expression
   | ExpBin of expression * bin_op * expression

, , . AST - , .

, OCaml- parser.mly, AST .