aboutsummaryrefslogtreecommitdiffstats
path: root/src/tokenizer.rs (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Refactor to improve docs of each functionLibravatar Titus Wormer2022-08-121-5/+6
|
* Refactor attempts to remove unneeded state nameLibravatar Titus Wormer2022-08-111-12/+7
|
* Refactor to handle definitions when parsingLibravatar Titus Wormer2022-08-111-5/+9
|
* Remove unneeded `balance` info on label startsLibravatar Titus Wormer2022-08-111-21/+20
|
* Remove boxes around resolversLibravatar Titus Wormer2022-08-111-56/+42
|
* Refactor to move some code to `event.rs`Libravatar Titus Wormer2022-08-111-96/+38
|
* Refactor to move some code to `state.rs`Libravatar Titus Wormer2022-08-111-640/+7
|
* Refactor internal docs, code style of tokenizerLibravatar Titus Wormer2022-08-111-262/+256
|
* Add improved container exit injectionLibravatar Titus Wormer2022-08-111-13/+10
|
* Refactor some code for document parsingLibravatar Titus Wormer2022-08-101-12/+4
|
* Remove boxes when calling statesLibravatar Titus Wormer2022-08-101-354/+328
| | | | Interestingly, this cuts performance by 20%.
* Add `State::Retry`Libravatar Titus Wormer2022-08-101-11/+37
|
* Rename `State::Fn` to `State::Next`Libravatar Titus Wormer2022-08-101-23/+23
|
* Refactor to reuse some tokenizer codeLibravatar Titus Wormer2022-08-091-69/+58
|
* Add peeking to unindented flow linesLibravatar Titus Wormer2022-08-091-4/+2
|
* Add support for passing `ok`, `nok` as separate states to attemptsLibravatar Titus Wormer2022-08-091-123/+77
|
* Rewrite algorithm to not pass around boxed functionsLibravatar Titus Wormer2022-08-091-227/+785
| | | | | | * Pass state names from an enum around instead of boxed functions * Refactor to simplify attempts a lot * Use a subtokenizer for the the `document` content type
* Refactor to remove most closuresLibravatar Titus Wormer2022-08-021-3/+145
|
* Add missing docs, refactor some codeLibravatar Titus Wormer2022-08-011-57/+43
|
* Refactor to work on bytes (`u8`)Libravatar Titus Wormer2022-07-291-65/+61
|
* Refactor to use `debug_assert`Libravatar Titus Wormer2022-07-281-10/+10
|
* Refactor to work on `char`sLibravatar Titus Wormer2022-07-281-88/+192
| | | | | | | | | | | | | | | Previously, a custom char implementation was used. This was easier to work with, as sometimes “virtual” characters are injected, or characters are ignored. This replaces that with working on actual `char`s. In the hope of in the future working on `u8`s, even. This simplifies the state machine somewhat, as only `\n` is fed, regardless of whether it was a CRLF, CR, or LF. It also feeds `' '` instead of virtual spaces. The BOM, if present, is now available as a `ByteOrderMark` event.
* Refactor to simplify tokenizerLibravatar Titus Wormer2022-07-261-123/+77
|
* Remove unneeded parameterLibravatar Titus Wormer2022-07-261-20/+16
|
* Refactor to not pass codes aroundLibravatar Titus Wormer2022-07-251-32/+26
|
* Refactor to store less dataLibravatar Titus Wormer2022-07-251-47/+43
|
* Remove no longer needed field in `State::Ok`Libravatar Titus Wormer2022-07-251-9/+9
|
* Refactor to remove need for cloning codesLibravatar Titus Wormer2022-07-251-33/+21
|
* Improve performance w/ a single feed loopLibravatar Titus Wormer2022-07-251-97/+90
|
* Refactor to remove unneeded `Option`Libravatar Titus Wormer2022-07-221-14/+18
|
* Refactor to use a single shared edit mapLibravatar Titus Wormer2022-07-221-16/+7
|
* Refactor to reserve some more vecsLibravatar Titus Wormer2022-07-221-5/+9
|
* Refactor to pass more references aroundLibravatar Titus Wormer2022-07-221-26/+39
|
* Refactor to remove unneeded tuples in every statesLibravatar Titus Wormer2022-07-221-60/+50
|
* Refactor to pass ints instead of vecs aroundLibravatar Titus Wormer2022-07-221-74/+47
|
* Refactor to move `index` field to `point`Libravatar Titus Wormer2022-07-211-19/+10
|
* Refactor to move some event fields to `link`Libravatar Titus Wormer2022-07-211-11/+24
|
* Refactor to share edit mapLibravatar Titus Wormer2022-07-201-2/+17
|
* Refactor to use less vecs for eventsLibravatar Titus Wormer2022-07-201-7/+5
|
* Remove a couple of clonesLibravatar Titus Wormer2022-07-191-3/+2
|
* Refactor to remove cloning in `edit_map`Libravatar Titus Wormer2022-07-191-0/+1
|
* Use `edit_map` in `subtokenize`Libravatar Titus Wormer2022-07-191-2/+0
|
* Remove an unneeded `HashMap`Libravatar Titus Wormer2022-07-191-15/+43
|
* Fix token that should be voidLibravatar Titus Wormer2022-07-181-1/+10
|
* Add docs and refactor listLibravatar Titus Wormer2022-07-151-1/+1
|
* Add docs and refactor document content typeLibravatar Titus Wormer2022-07-151-1/+10
|
* Add missing docs to tokenizerLibravatar Titus Wormer2022-07-151-14/+16
|
* Fix annoying bug around virtual spaces in containersLibravatar Titus Wormer2022-07-151-18/+22
|
* Add support for container stateLibravatar Titus Wormer2022-07-141-0/+10
| | | | | | * Fix to parse list item continuation based on how big the initial list item prefix was * Fix list items that start with blank lines
* Add support for lazy linesLibravatar Titus Wormer2022-07-111-1/+0
|
generated by cgit (git 2.34.1) at 2024-11-25 15:29:07 +0000