aboutsummaryrefslogtreecommitdiffstats
path: root/src/construct/text.rs (unfollow)
Commit message (Collapse)AuthorFilesLines
2022-09-14Fix to prefer flow over definitions, setext headingsLibravatar Titus Wormer1-1/+5
An undocumented part of CommonMark is how to deal with things in definition labels or definition titles (which both can span multiple lines). Can flow (or containers?) interrupt them? They can according to the `cmark` reference parser, so this was implemented here. This adds a new `Content` content type, which houses zero or more definitions, and then zero-or-one paragraphs. Content can be followed by a setext heading underline, which either turns into a setext heading when the content ends in a paragraph, or turns into the start of the following paragraph when it is followed by content that starts with a paragraph, or turns into a stray paragraph.
2022-09-09Add mdx expression (flow, text)Libravatar Titus Wormer1-1/+10
2022-09-06Add initial states for MDX JSX (text)Libravatar Titus Wormer1-3/+20
2022-09-05Fix to implement GFM autolink literals exactlyLibravatar Titus Wormer1-1/+19
2022-08-26Add support for math (text)Libravatar Titus Wormer1-16/+19
2022-08-25Add support for GFM footnotesLibravatar Titus Wormer1-3/+21
2022-08-22Add support for GFM task list itemLibravatar Titus Wormer1-1/+9
2022-08-22Add support for GFM strikethroughLibravatar Titus Wormer1-4/+10
2022-08-18Add support for GFM autolink literalsLibravatar Titus Wormer1-2/+10
2022-08-15Refactor to move `content` to `construct`Libravatar Titus Wormer1-0/+0
2022-08-12Refactor to improve docs of each functionLibravatar Titus Wormer1-3/+33
2022-08-11Refactor attempts to remove unneeded state nameLibravatar Titus Wormer1-49/+62
2022-08-11Remove boxes around resolversLibravatar Titus Wormer1-1/+2
2022-08-11Refactor to move some code to `event.rs`Libravatar Titus Wormer1-34/+38
2022-08-11Refactor to move some code to `state.rs`Libravatar Titus Wormer1-38/+35
2022-08-11Refactor internal docs, code style of tokenizerLibravatar Titus Wormer1-7/+3
2022-08-10Add `State::Retry`Libravatar Titus Wormer1-2/+2
2022-08-10Rename `State::Fn` to `State::Next`Libravatar Titus Wormer1-21/+21
2022-08-09Add support for passing `ok`, `nok` as separate states to attemptsLibravatar Titus Wormer1-21/+64
2022-08-09Rewrite algorithm to not pass around boxed functionsLibravatar Titus Wormer1-23/+22
* Pass state names from an enum around instead of boxed functions * Refactor to simplify attempts a lot * Use a subtokenizer for the the `document` content type
2022-08-02Refactor to remove most closuresLibravatar Titus Wormer1-9/+13
2022-07-29Refactor to work on bytes (`u8`)Libravatar Titus Wormer1-10/+10
2022-07-28Refactor to work on `char`sLibravatar Titus Wormer1-12/+12
Previously, a custom char implementation was used. This was easier to work with, as sometimes “virtual” characters are injected, or characters are ignored. This replaces that with working on actual `char`s. In the hope of in the future working on `u8`s, even. This simplifies the state machine somewhat, as only `\n` is fed, regardless of whether it was a CRLF, CR, or LF. It also feeds `' '` instead of virtual spaces. The BOM, if present, is now available as a `ByteOrderMark` event.
2022-07-26Refactor to drastically improve perf around whitespaceLibravatar Titus Wormer1-26/+31
2022-07-25Refactor to not pass codes aroundLibravatar Titus Wormer1-5/+5
2022-07-25Remove no longer needed field in `State::Ok`Libravatar Titus Wormer1-1/+1
2022-07-22Refactor to remove unneeded tuples in every statesLibravatar Titus Wormer1-4/+4
2022-07-22Refactor to pass ints instead of vecs aroundLibravatar Titus Wormer1-1/+1
2022-07-21Refactor to improve performance by passing markers aroundLibravatar Titus Wormer1-46/+19
2022-07-18Refactor to improve performance when constructs are turned offLibravatar Titus Wormer1-27/+48
2022-07-18Add support for turning off constructsLibravatar Titus Wormer1-0/+1
2022-07-05Refactor code styleLibravatar Titus Wormer1-4/+4
2022-07-04Add support for attention (emphasis, strong)Libravatar Titus Wormer1-7/+11
2022-06-30Add support for trimming whitespace around string, textLibravatar Titus Wormer1-9/+13
This commit introduces trimming initial and final whitespace around the whole string or text, or around line endings inside that string or text. * Add `register_resolver_before`, to run resolvers earlier than others, used for labels * Add resolver to merge `data` events, which are the most frequent token that occurs, and can happen adjacently. In `micromark-js` this sped up parsing a lot * Fix a bug where a virtual space was not seen as an okay event * Refactor to enable all turned off whitespace tests
2022-06-30Refactor some docs, fix some grammarLibravatar Titus Wormer1-2/+3
2022-06-24Add link, images (resource)Libravatar Titus Wormer1-5/+12
This is still some messy code that needs cleaning up, but it adds support for links and images, of the resource kind (`[a](b)`). References (`[a][b]`) are parsed and will soon be supported, but need matching. * Fix bug to pad percent-encoded bytes when normalizing urls * Fix bug with escapes counting as balancing in destination * Add `space_or_tab_one_line_ending`, to parse whitespace including up to one line ending (but not a blank line) * Add `ParserState` to share codes, definitions, etc
2022-06-22Refactor to improve tokenizer, add docsLibravatar Titus Wormer1-8/+10
2022-06-21Make data a constructLibravatar Titus Wormer1-41/+12
2022-06-16Refactor to reorder thing alphabeticallyLibravatar Titus Wormer1-1/+0
2022-06-16Add support for hard break (trailing)Libravatar Titus Wormer1-6/+9
2022-06-16Add support for hard break escapeLibravatar Titus Wormer1-3/+4
2022-06-15Add code (text)Libravatar Titus Wormer1-4/+6
2022-06-14Fix support for deep subtokenizationLibravatar Titus Wormer1-7/+14
* Fix a couple of forgotten line ending handling in html (text) * Fix missing initial case for html (text) not having a `<` 😬 * Add line ending handling to `text` construct
2022-06-14Add docs for html (text)Libravatar Titus Wormer1-1/+1
2022-06-13Add basic html (text)Libravatar Titus Wormer1-5/+9
* Add all states for html (text) * Fix to link paragraph tokens together * Add note about uncovered bug where linking paragraph tokens together doesn’t work 😅
2022-06-13Add autolinksLibravatar Titus Wormer1-4/+5
2022-06-10Add text content typeLibravatar Titus Wormer1-14/+16
* Add character reference and character escapes in text * Add recursive subtokenization
2022-06-10Add proper support for subtokenizationLibravatar Titus Wormer1-36/+6
- Add “content” content type - Add paragraph - Add skips - Add linked tokens
2022-06-09Add basic subtokenization, string content in fenced codeLibravatar Titus Wormer1-16/+4
2022-06-09Refactor to pass more slices aroundLibravatar Titus Wormer1-1/+1
generated by cgit (git 2.34.1) at 2024-11-22 18:32:42 +0000