Commit graph

50 commits

Author SHA1 Message Date
2656a1d328 [parser] Remove bad LALR implementation, start cleanup 2024-10-10 07:58:16 -07:00
eef1db72da [parser] Pager's algorithm. Faster.
As good as LALR but the implementation isn't embarassing. (Still
pretty bad though.)

Honestly the next thing to do is to delete LALR and just use Pager's
and also rebuild ConfigSet et al to be ItemSet so that Pager's alg
can go even faster. I think I want to keep LR1 just for completeness
so I might as well not delete SLR and LR0, although I *could* I
suppose.
2024-10-05 16:00:41 -07:00
8a17cfd586 [wadler] Prettier handling of trivia
Split the rules for pre- and post- trivia, understand when we want to
do either, handle multi-line-break (in an unsatisfying way, I guess)
but otherwise lay the groundwork for thinking about it better.

Also now we don't generate lazy "Text" nodes because I thought I might
want to actually look at the newlines in the source but I don't yet.
I *can* now, though. (I can also detect EOF so there's that.)
2024-09-19 16:39:32 -07:00
c31d527077 [wadler] Trivia escapes groups
This means that forced breaks from comments don't screw up the
following single-line things. But this still isn't right; we need to
fine tune how we represent trivia.
2024-09-15 08:51:18 -07:00
d5ccd5b147 Really messing around with trivia, it's not good yet
It's really not clear how to track it and how to compose it with
groups yet. Really very difficult.
2024-09-14 17:14:07 -07:00
71b59302fa Mess with wrapping and whatnot 2024-09-14 07:28:24 -07:00
265f07fd5a Add document rendering to harness
To debug pretty-printing. Noticing some weirdness!
2024-09-13 16:45:54 -07:00
d7a6891519 Finish annotating test grammar, forced breaks, fixes
Forced breaks force a newline in a spot, which is sometimes what we
want. (Like, this syntax should *never* be on a single line.)
2024-09-13 11:57:16 -07:00
677e60bd27 Dumb tweaks 2024-09-07 07:39:13 -07:00
00b4cd4702 Starting to look at pretty-printing with the idea of auto-indentation
I wonder if it will work?
2024-09-06 16:23:14 -07:00
d7dfd556ec Emit an emacs major mode
With coloring! Next up: formatting but that might be hard.
2024-09-06 11:51:09 -07:00
0243b0bf77 Refactor grammar to make "nicer" trees (I guess) 2024-09-06 10:21:00 -07:00
51c4f14c26 Emit highlight queries for tree-sitter
Now we're starting to get somewhere!
2024-09-05 14:52:35 -07:00
5e12af9f31 Extra marks, fields, whatnot 2024-09-05 06:32:28 -07:00
591da0c971 Rework highlighting metadata GOD 2024-09-02 08:50:36 -07:00
a99b3ecb70 Interpret precedence the way tree-sitter does, kinda
This allows most of our precedence to be re-used. There are some cases
still where tree-sitter gets confused (and we don't), see the
corresponding change to grammar.py. I wish I knew how to fix this but
I don't. :(
2024-09-01 07:38:46 -07:00
0354fbf4a4 More ways of writing
Sometimes prettier
2024-09-01 06:52:13 -07:00
3012df4ac6 Precedence but it doesn't work
Tree sitter doesn't let me do token-based precedence? I don't like
tree-sitter's "make it inline but give it a number" system- seems like
a bug farm to me.
2024-08-31 07:22:49 -07:00
98c4bb950f Fix bugs but still doesn't work for Fine 2024-08-30 09:14:01 -07:00
e07f2be3fa Something wrong with this, need to understand more 2024-08-28 08:29:09 -07:00
02c1aa507e Muck around with usability 2024-08-28 08:27:46 -07:00
cd62b65789 Remove the old hand-lexer
The machine lexer is working now
2024-08-27 16:49:03 -07:00
d62076f3c4 Fix a bug in terminal declaration
whoops. now it parses correctly
2024-08-27 16:47:58 -07:00
49ad7fdb52 Associate metadata with terminals
This is a half-assed attempt at doing syntax coloring which I think
will almost certainly turn out to be insufficient. I'm committing it
just to record some of the work I've done but. BUT.

Probably trying to match tree-sitter is a better way of doing
this. (But, like, emitting tree-sitter grammars? Really? Wow, dude.
Way to give up.)
2024-08-27 15:43:07 -07:00
7a5f17f74b Specify and honor trivia tokens
e.g. "this is how machine-generated parsers know to skip blanks and
comments"

The run time implementation could be better; we don't really want to
just discard trivia because it's useful for e.g. doc comments and the
like. BUT for now this is fine.
2024-08-24 10:01:40 -07:00
8e22c59aa8 This is a nicer number format (e.g. 1e3) 2024-08-24 10:01:05 -07:00
30f7798719 Actual strings and floats
Using the new regex features
2024-08-24 08:36:28 -07:00
72052645d6 Generated lexers actually kinda work
But regular expressions are underpowered and verbose
2024-08-23 15:32:35 -07:00
58c3004702 Move terminals into grammar definition
Starting to work on machine-generated lexers too
2024-08-23 07:24:30 -07:00
e04aa1966e Start moving the examples into tests 2024-06-15 07:52:16 -07:00
a439e1456a Random tweaks 2024-06-08 17:31:33 -07:00
bd70315935 Improve performance by not de-duping before closing
You close *after* you've determined that a successor is new, not
before.
2024-06-06 05:31:15 -07:00
cabc091ee4 Start extracting better expressions 2024-06-02 08:10:18 -07:00
f1507a36f1 Oh, that was a BUG in the grammar! 2024-06-01 05:59:58 -07:00
03937e62e6 Allow the generator to be a little more declarative 2024-06-01 05:57:16 -07:00
e203e27407 Allow the grammar to specify a preference for a generator
Overridable, like start production
2024-06-01 05:42:40 -07:00
55c4675fe5 Use precedence and not a lot of different rules 2024-05-31 08:32:05 -07:00
a7b4a3ce8b Grammars have implicit start symbols 2024-05-31 06:29:57 -07:00
57de8a5b85 More grammar shaping 2024-05-31 05:22:00 -07:00
561dcd87ff Allow nonterminals to be renamed 2024-05-30 19:15:20 -07:00
dfef449c33 Better tree output 2024-05-30 08:24:36 -07:00
56d24c5fb9 Chaos: split tables, interactions, Terminal
- Tables are split into `actions` and `goto` now to make formatting
  nicer
- Token is renamed Terminal
- Likes are now Florps
- Lexer now loaded dynamically (badly)
2024-05-30 08:02:47 -07:00
71078f76b4 Interactive grammar editing 2024-05-29 19:28:42 -07:00
45a9303a27 Transparent rules
Better parsing/action types
Good grief
2024-05-29 09:07:19 -07:00
0fc04cf11e Fix grammar bugs, work on debugging harness. 2024-05-27 19:02:10 -07:00
7c1d9b5f2b A cleaner API
I've been hacking on this in a different repository, so I thought I'd
bring it over here.
2024-05-05 08:45:45 -07:00
f656dbd8f3 faster: Significant performance boost on large grammar
15s to 5s
2024-04-17 23:25:15 -07:00
6fa89a9757 faster: Finish the big grammar 2024-04-17 17:23:14 -07:00
de0a76818e faster: Correct precedence but don't use it
Precedence now works for, like, resolving expressions in a single
big table but uh... it takes the time from 700ms to like 7s so I've
reverted it for now.
2024-04-17 17:13:18 -07:00
7147557e2b faster: Snapshot a big grammar I'm playing with 2024-04-17 11:07:56 -07:00