49ad7fdb52
Associate metadata with terminals
...
This is a half-assed attempt at doing syntax coloring which I think
will almost certainly turn out to be insufficient. I'm committing it
just to record some of the work I've done but. BUT.
Probably trying to match tree-sitter is a better way of doing
this. (But, like, emitting tree-sitter grammars? Really? Wow, dude.
Way to give up.)
2024-08-27 15:43:07 -07:00
76ef85483e
Accept is single-valued, the multi-value thing didn't ever make sense
...
I mean, it did when we thought we were going to weave NFA states as we
were building them but we ended up not doing that and instead just
using the fancy EdgeList splitting magic when building DFAs from the
NFA. It has the same power and is simpler code, and also means that
we'll *never* be asked to have multiple Terminals be accepted from a
single NFA state.
2024-08-27 15:43:01 -07:00
208491d56e
This was out of date
2024-08-26 08:05:01 -07:00
2473ae713d
Trivia tests
2024-08-24 15:09:08 -07:00
7a5f17f74b
Specify and honor trivia tokens
...
e.g. "this is how machine-generated parsers know to skip blanks and
comments"
The run time implementation could be better; we don't really want to
just discard trivia because it's useful for e.g. doc comments and the
like. BUT for now this is fine.
2024-08-24 10:01:40 -07:00
8e22c59aa8
This is a nicer number format (e.g. 1e3)
2024-08-24 10:01:05 -07:00
f29ec5072f
Augment number pattern, tests
...
More robust testing. Error messages would be nice but.
2024-08-24 09:38:21 -07:00
0c952e4905
Correct NFA construction
...
There was a bug in the way that I was converting regular expressions
to NFAs. I'm still not entirely sure what was going on, but I
re-visited the construction and made it follow the literature more
closely and it fixed the problem.
2024-08-24 09:24:29 -07:00
30f7798719
Actual strings and floats
...
Using the new regex features
2024-08-24 08:36:28 -07:00
c0b623bd6d
Remove unused imports
2024-08-24 08:36:20 -07:00
454e6fd6fd
Regex API "improvements"
...
I mean, is it better than a regex parser? No, probably not.
2024-08-24 08:35:45 -07:00
6d6aabdeb3
Terminal name must be explicit on construction
2024-08-24 08:35:10 -07:00
72052645d6
Generated lexers actually kinda work
...
But regular expressions are underpowered and verbose
2024-08-23 15:32:35 -07:00
58c3004702
Move terminals into grammar definition
...
Starting to work on machine-generated lexers too
2024-08-23 07:24:30 -07:00
f6bc2ccea8
Move the examples into tests
2024-06-15 12:23:36 -07:00
e04aa1966e
Start moving the examples into tests
2024-06-15 07:52:16 -07:00
d3b8d0e836
Configure pyright
2024-06-15 06:20:37 -07:00
fe7e67cce6
Ignore more stuff
2024-06-15 06:18:14 -07:00
c82f53c346
Document the decision
2024-06-15 06:18:04 -07:00
fb2dff51df
Tests. Well, test.
2024-06-15 06:14:37 -07:00
3483d99a7d
It's been some time coming
...
The actual library should not require a venv or a setup or anything
but it does make testing and whatnot easier.
2024-06-14 06:29:15 -07:00
7b3c94c469
Move things into more modules
...
It will help with testing and profiling
It breaks pyright (it's probably time to abandon pyright)
2024-06-10 05:50:09 -07:00
c3098aa435
Various bug fixes related to the end of file
2024-06-09 17:18:44 -07:00
b843dc84f4
Comment
2024-06-09 08:53:06 -07:00
a3786c62ba
Proper positions for synthetic tokens
2024-06-09 07:24:19 -07:00
0c3e6b211c
Lots of logging tweaks
2024-06-09 07:12:35 -07:00
38837f5c4d
Don't crash with empty lists
2024-06-09 07:12:14 -07:00
c74a36463e
Fix: handle zero-length children correctly
2024-06-09 07:11:39 -07:00
bea7903686
Line numbers
...
Getting flickery :D
2024-06-09 06:22:31 -07:00
bc5a018ce9
Logging, scrolling bad
2024-06-08 18:35:39 -07:00
1dcc86e9fe
Prettier harness, start logging properly
2024-06-08 18:12:16 -07:00
a439e1456a
Random tweaks
2024-06-08 17:31:33 -07:00
a837d662dd
Error recovery: lame version of CPCT+
2024-06-08 17:30:59 -07:00
3c8ab86a90
Type things in the table so we can do exhaustive action matches
2024-06-08 17:30:39 -07:00
973445a8d8
Scored results
2024-06-06 17:09:14 -07:00
07164d51a0
Fix end of file error reporting
2024-06-06 17:05:41 -07:00
22029edbb2
Even more threaded parsing
2024-06-06 17:02:07 -07:00
f904f7eea3
Stuff to do
2024-06-06 08:19:54 -07:00
68a8e9298b
Simplify token handling in parsing
2024-06-06 08:16:38 -07:00
b60b38d78e
Start making parsing thread-based
2024-06-06 08:05:40 -07:00
bd70315935
Improve performance by not de-duping before closing
...
You close *after* you've determined that a successor is new, not
before.
2024-06-06 05:31:15 -07:00
8c3b1b784c
Performance
2024-06-05 09:55:12 -07:00
cabc091ee4
Start extracting better expressions
2024-06-02 08:10:18 -07:00
e57a827e5c
Better formatting, dump full graph, disallow conflicts
2024-06-02 08:09:47 -07:00
d6131ad53e
Stop deleting blank space in errors
2024-06-02 08:09:32 -07:00
f1507a36f1
Oh, that was a BUG in the grammar!
2024-06-01 05:59:58 -07:00
03937e62e6
Allow the generator to be a little more declarative
2024-06-01 05:57:16 -07:00
e203e27407
Allow the grammar to specify a preference for a generator
...
Overridable, like start production
2024-06-01 05:42:40 -07:00
55c4675fe5
Use precedence and not a lot of different rules
2024-05-31 08:32:05 -07:00
fee1c68dea
Wrap errors
...
Maybe people want to see them
2024-05-31 08:31:47 -07:00
