Design an abstract syntax tree

Question

I surfed the internet looking for information about newcomers to the development of abstract C # syntax trees, but I can only find information for people who already “know”. I am a developer of applications for business, so such topics are a little over my head, but this is for my own education, so I am ready to spend time and find out what concepts are needed.

As a rule, I would like to learn about methods for developing an abstract representation of code from a line of code. More specifically, I would like to be able to use this AST to highlight C # syntax. (I understand that AST does not need syntax highlighting, but it seems like a good opportunity to learn some of the "compiler" methods.)

I apologize if this question is a bit broad, but I'm not sure how else to ask.

Thanks!

Answer 1

First you need to understand what parsing is and what abstract syntax trees are. To do this, you can go to Wikipedia about abstract syntax trees for a quick glance.

You really need to spend some time on the compiler to understand how abstract syntax trees are related to parsing and can be created on parsing; the classic reference is Aho / Ullman / Sethi's book Compilers (easy to find on the Internet). You can find the SO answer to Are there any "funny" words? ways to learn about languages, grammars, analyzes and compilers? Instructive.

Once you understand how to build an AST for simple grammar, you can turn your attention to something like C #. The problem here is large-scale; It's one thing to play with a toy language with 20 grammar rules. Another thing is to work with grammar in a few hundred or thousand rules. The experience will be small, it will greatly facilitate the understanding of how large are united, and how to live with them.

You probably don't want to create your own C # grammar (or implement it according to the C # standard); its quite a lot of work. You can get affordable tools that will give you the C # AST (Roslyn already mentioned, ANTLR has a C # parser, there are many more).

It is true that you can use AST to highlight syntax (although this probably kills the mosquito with a sledgehammer). What most people think little (but the compiler books emphasize) is what happens after you have AST; basically they are not useful on their own. You really need a lot more technique to do something interesting. Instead of repeating it over and over (I constantly see the same questions), you can see my discussion of Life After Parsing for more details.

Answer 2

You should probably take a look at this Phil Trelford talk:

Write your own compiler in 24 hours

This person is a genius, and you will keep you updated to learn about compilers. He explains it literally easy enough to understand him for five years. This five-year-old son is a son, so he probably has an unfair advantage, but five to five.

Answer 3

Take a look at Roslyn. I think this may be what you are looking for. This gives you access to AST compilers, among many other amazing things!

http://blogs.msdn.com/b/visualstudio/archive/2011/10/19/introducing-the-microsoft-roslyn-ctp.aspx

In addition to this, I offer a compiler tutorial.