Nothing Special   »   [go: up one dir, main page]
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Layout processor should use token IDs, not their String rendering #352

Closed
2 of 4 tasks
andreasabel opened this issue Apr 7, 2021 · 1 comment
Closed
2 of 4 tasks

Layout processor should use token IDs, not their String rendering #352

andreasabel opened this issue Apr 7, 2021 · 1 comment
Assignees
@andreasabel
Labels
enhancement layout Concerning layout (keywords)
Milestone
2.9.2

Comments

@andreasabel
Copy link
Member
andreasabel commented Apr 7, 2021 •
edited
Loading

The current layout preprocessor renders tokens as String to check whether they match layout keywords or { } ;. However, each keyword/symbol token comes with an ID that could be checked faster.

Steps to rewrite layout processor (generator) to use IDs instead of String:

  • LBNF: check whether { } ; occur in grammar when layout is used.
  • Give each symbol a name, like ( is LPAREN.
  • Define token constants in generated lexer for all the keywords and symbols.
  • Reference these constants in the layouter, rather than strings.
@andreasabel andreasabel added enhancement layout Concerning layout (keywords) labels Apr 7, 2021
@andreasabel andreasabel self-assigned this Apr 8, 2021
@andreasabel andreasabel added this to the 2.9.2 milestone Apr 8, 2021
andreasabel added a commit that referenced this issue Apr 9, 2021
@andreasabel
[ #352 ] layout: check for symbols { } ; in grammar
d45f9b0
andreasabel added a commit that referenced this issue Apr 9, 2021
@andreasabel
[ re #352 ] C/C++: readable token names in lexer/parser
ee637e9 
Instead of _SYMB_nnn, we have _LPAREN, _KW_while, and T_Identifier.
andreasabel added a commit that referenced this issue Apr 9, 2021
@andreasabel
[ #352 ] Alex: efficient Tok equality, using ID for symbols/keywords
9ad549f
@andreasabel
Copy link
Member Author

I chose a different implementation path: Simply use TokSymbol (with the token ID) instead of String in the generated Layout.hs. We need not define constants for each token ID in the lexer. Could be still nice, but this is another issue.

andreasabel added a commit that referenced this issue Apr 19, 2021
@andreasabel
[ #352 ] allow toplevel layout as soon as we have semicolons
157f9eb 
Braces are not needed for toplevel layout, so we should not require them.
andreasabel added a commit that referenced this issue Apr 27, 2021
@andreasabel
[ #352 ] TokenText flag to cf2Layout is obsolete
793e85d
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@andreasabel andreasabel

Labels
enhancement layout Concerning layout (keywords)
Projects
None yet
Milestone
2.9.2
Development

No branches or pull requests
1 participant
@andreasabel