aboutsummaryrefslogtreecommitdiffstats
path: root/src/content/text.rs (unfollow)
Commit message (Collapse)AuthorFilesLines
2022-08-12Refactor to improve docs of each functionLibravatar Titus Wormer1-3/+33
2022-08-11Refactor attempts to remove unneeded state nameLibravatar Titus Wormer1-49/+62
2022-08-11Remove boxes around resolversLibravatar Titus Wormer1-1/+2
2022-08-11Refactor to move some code to `event.rs`Libravatar Titus Wormer1-34/+38
2022-08-11Refactor to move some code to `state.rs`Libravatar Titus Wormer1-38/+35
2022-08-11Refactor internal docs, code style of tokenizerLibravatar Titus Wormer1-7/+3
2022-08-10Add `State::Retry`Libravatar Titus Wormer1-2/+2
2022-08-10Rename `State::Fn` to `State::Next`Libravatar Titus Wormer1-21/+21
2022-08-09Add support for passing `ok`, `nok` as separate states to attemptsLibravatar Titus Wormer1-21/+64
2022-08-09Rewrite algorithm to not pass around boxed functionsLibravatar Titus Wormer1-23/+22
* Pass state names from an enum around instead of boxed functions * Refactor to simplify attempts a lot * Use a subtokenizer for the the `document` content type
2022-08-02Refactor to remove most closuresLibravatar Titus Wormer1-9/+13
2022-07-29Refactor to work on bytes (`u8`)Libravatar Titus Wormer1-10/+10
2022-07-28Refactor to work on `char`sLibravatar Titus Wormer1-12/+12
Previously, a custom char implementation was used. This was easier to work with, as sometimes “virtual” characters are injected, or characters are ignored. This replaces that with working on actual `char`s. In the hope of in the future working on `u8`s, even. This simplifies the state machine somewhat, as only `\n` is fed, regardless of whether it was a CRLF, CR, or LF. It also feeds `' '` instead of virtual spaces. The BOM, if present, is now available as a `ByteOrderMark` event.
2022-07-26Refactor to drastically improve perf around whitespaceLibravatar Titus Wormer1-26/+31
2022-07-25Refactor to not pass codes aroundLibravatar Titus Wormer1-5/+5
2022-07-25Remove no longer needed field in `State::Ok`Libravatar Titus Wormer1-1/+1
2022-07-22Refactor to remove unneeded tuples in every statesLibravatar Titus Wormer1-4/+4
2022-07-22Refactor to pass ints instead of vecs aroundLibravatar Titus Wormer1-1/+1
2022-07-21Refactor to improve performance by passing markers aroundLibravatar Titus Wormer1-46/+19
2022-07-18Refactor to improve performance when constructs are turned offLibravatar Titus Wormer1-27/+48
2022-07-18Add support for turning off constructsLibravatar Titus Wormer1-0/+1
2022-07-05Refactor code styleLibravatar Titus Wormer1-4/+4
2022-07-04Add support for attention (emphasis, strong)Libravatar Titus Wormer1-7/+11
2022-06-30Add support for trimming whitespace around string, textLibravatar Titus Wormer1-9/+13
This commit introduces trimming initial and final whitespace around the whole string or text, or around line endings inside that string or text. * Add `register_resolver_before`, to run resolvers earlier than others, used for labels * Add resolver to merge `data` events, which are the most frequent token that occurs, and can happen adjacently. In `micromark-js` this sped up parsing a lot * Fix a bug where a virtual space was not seen as an okay event * Refactor to enable all turned off whitespace tests
2022-06-30Refactor some docs, fix some grammarLibravatar Titus Wormer1-2/+3
2022-06-24Add link, images (resource)Libravatar Titus Wormer1-5/+12
This is still some messy code that needs cleaning up, but it adds support for links and images, of the resource kind (`[a](b)`). References (`[a][b]`) are parsed and will soon be supported, but need matching. * Fix bug to pad percent-encoded bytes when normalizing urls * Fix bug with escapes counting as balancing in destination * Add `space_or_tab_one_line_ending`, to parse whitespace including up to one line ending (but not a blank line) * Add `ParserState` to share codes, definitions, etc
2022-06-22Refactor to improve tokenizer, add docsLibravatar Titus Wormer1-8/+10
2022-06-21Make data a constructLibravatar Titus Wormer1-41/+12
2022-06-16Refactor to reorder thing alphabeticallyLibravatar Titus Wormer1-1/+0
2022-06-16Add support for hard break (trailing)Libravatar Titus Wormer1-6/+9
2022-06-16Add support for hard break escapeLibravatar Titus Wormer1-3/+4
2022-06-15Add code (text)Libravatar Titus Wormer1-4/+6
2022-06-14Fix support for deep subtokenizationLibravatar Titus Wormer1-7/+14
* Fix a couple of forgotten line ending handling in html (text) * Fix missing initial case for html (text) not having a `<` 😬 * Add line ending handling to `text` construct
2022-06-14Add docs for html (text)Libravatar Titus Wormer1-1/+1
2022-06-13Add basic html (text)Libravatar Titus Wormer1-5/+9
* Add all states for html (text) * Fix to link paragraph tokens together * Add note about uncovered bug where linking paragraph tokens together doesn’t work 😅
2022-06-13Add autolinksLibravatar Titus Wormer1-4/+5
2022-06-10Add text content typeLibravatar Titus Wormer1-14/+16
* Add character reference and character escapes in text * Add recursive subtokenization
2022-06-10Add proper support for subtokenizationLibravatar Titus Wormer1-36/+6
- Add “content” content type - Add paragraph - Add skips - Add linked tokens
2022-06-09Add basic subtokenization, string content in fenced codeLibravatar Titus Wormer1-16/+4
2022-06-09Refactor to pass more slices aroundLibravatar Titus Wormer1-1/+1
generated by cgit (git 2.34.1) at 2024-11-22 21:46:25 +0000