aboutsummaryrefslogtreecommitdiffstats
path: root/src/construct/flow.rs (unfollow)
Commit message (Collapse)AuthorFilesLines
2022-09-14Fix to prefer flow over definitions, setext headingsLibravatar Titus Wormer1-24/+9
An undocumented part of CommonMark is how to deal with things in definition labels or definition titles (which both can span multiple lines). Can flow (or containers?) interrupt them? They can according to the `cmark` reference parser, so this was implemented here. This adds a new `Content` content type, which houses zero or more definitions, and then zero-or-one paragraphs. Content can be followed by a setext heading underline, which either turns into a setext heading when the content ends in a paragraph, or turns into the start of the following paragraph when it is followed by content that starts with a paragraph, or turns into a stray paragraph.
2022-09-09Add mdx expression (flow, text)Libravatar Titus Wormer1-1/+23
2022-09-08Add support for mdx jsx (flow)Libravatar Titus Wormer1-2/+17
2022-08-31Add support for GFM tablesLibravatar Titus Wormer1-23/+18
2022-08-26Add support for math (flow)Libravatar Titus Wormer1-12/+14
2022-08-16Add support for frontmatterLibravatar Titus Wormer1-12/+12
2022-08-15Refactor to move `content` to `construct`Libravatar Titus Wormer1-0/+0
2022-08-12Refactor to improve docs of each functionLibravatar Titus Wormer1-35/+70
2022-08-11Refactor attempts to remove unneeded state nameLibravatar Titus Wormer1-56/+64
2022-08-11Refactor to move some code to `event.rs`Libravatar Titus Wormer1-55/+55
2022-08-11Refactor to move some code to `state.rs`Libravatar Titus Wormer1-50/+51
2022-08-10Add `State::Retry`Libravatar Titus Wormer1-2/+2
2022-08-10Rename `State::Fn` to `State::Next`Libravatar Titus Wormer1-32/+32
2022-08-09Add peeking to unindented flow linesLibravatar Titus Wormer1-11/+47
2022-08-09Add support for passing `ok`, `nok` as separate states to attemptsLibravatar Titus Wormer1-29/+68
2022-08-09Rewrite algorithm to not pass around boxed functionsLibravatar Titus Wormer1-27/+30
* Pass state names from an enum around instead of boxed functions * Refactor to simplify attempts a lot * Use a subtokenizer for the the `document` content type
2022-07-29Refactor to work on bytes (`u8`)Libravatar Titus Wormer1-2/+2
2022-07-28Refactor to work on `char`sLibravatar Titus Wormer1-7/+7
Previously, a custom char implementation was used. This was easier to work with, as sometimes “virtual” characters are injected, or characters are ignored. This replaces that with working on actual `char`s. In the hope of in the future working on `u8`s, even. This simplifies the state machine somewhat, as only `\n` is fed, regardless of whether it was a CRLF, CR, or LF. It also feeds `' '` instead of virtual spaces. The BOM, if present, is now available as a `ByteOrderMark` event.
2022-07-25Refactor to not pass codes aroundLibravatar Titus Wormer1-14/+14
2022-07-25Remove no longer needed field in `State::Ok`Libravatar Titus Wormer1-4/+4
2022-07-22Refactor to remove unneeded tuples in every statesLibravatar Titus Wormer1-12/+12
2022-07-22Refactor to pass ints instead of vecs aroundLibravatar Titus Wormer1-6/+6
2022-07-07Refactor to move token types to `token`Libravatar Titus Wormer1-5/+6
2022-07-07Add basic support for block quotesLibravatar Titus Wormer1-47/+2
2022-07-05Refactor code styleLibravatar Titus Wormer1-2/+2
2022-07-04Add support for unicode punctuationLibravatar Titus Wormer1-1/+1
2022-07-04Update list of todosLibravatar Titus Wormer1-1/+1
2022-07-01Make paragraphs really fastLibravatar Titus Wormer1-22/+24
The approach that `micromark-js` takes is as follows: to parse a paragraph, check whether each line starts with something else. If it does, exit, otherwise continue. That is slow, because our actual flow parser does similar things: the work was being done twice. To fix this, this commit introduces parsing each line of a paragraph separately. And finally, when done with flow, combining adjacent paragraphs. This same mechanism is reused for setext headings. Additionally, this commit adds support for interrupting things (or not). E.g., HTML (flow, complete) cannot interrupt paragraphs. Definitions cannot interrupt paragraphs, and connect be interrupted either, but they can follow each other.
2022-06-29Add support for sharing identifiers, references before definitionsLibravatar Titus Wormer1-5/+7
2022-06-24Add link, images (resource)Libravatar Titus Wormer1-6/+14
This is still some messy code that needs cleaning up, but it adds support for links and images, of the resource kind (`[a](b)`). References (`[a][b]`) are parsed and will soon be supported, but need matching. * Fix bug to pad percent-encoded bytes when normalizing urls * Fix bug with escapes counting as balancing in destination * Add `space_or_tab_one_line_ending`, to parse whitespace including up to one line ending (but not a blank line) * Add `ParserState` to share codes, definitions, etc
2022-06-22Add support for normalizing identifiersLibravatar Titus Wormer1-1/+24
2022-06-22Refactor to improve tokenizer, add docsLibravatar Titus Wormer1-8/+10
2022-06-20Add some more enabled testsLibravatar Titus Wormer1-1/+0
2022-06-20Add improved whitespace handlingLibravatar Titus Wormer1-34/+11
* add several helpers for parsing betwen x and y `space_or_tab`s * use those helpers in a bunch of places * move initial indent parsing to flow constructs themselves
2022-06-20Add paragraphLibravatar Titus Wormer1-130/+9
2022-06-20Remove unneeded `content` content typeLibravatar Titus Wormer1-27/+27
2022-06-17Add support for definitionsLibravatar Titus Wormer1-9/+15
* Add definitions * Add partials for label, destination, title * Add `go`, to attempt something, and do something else on `ok`
2022-06-16Add heading (setext)Libravatar Titus Wormer1-11/+8
2022-06-14Reorganize to split utilLibravatar Titus Wormer1-2/+2
2022-06-10Add text content typeLibravatar Titus Wormer1-5/+9
* Add character reference and character escapes in text * Add recursive subtokenization
2022-06-10Add proper support for subtokenizationLibravatar Titus Wormer1-19/+26
- Add “content” content type - Add paragraph - Add skips - Add linked tokens
2022-06-09Add basic subtokenization, string content in fenced codeLibravatar Titus Wormer1-17/+6
2022-06-09Refactor to pass more slices aroundLibravatar Titus Wormer1-1/+1
2022-06-09Add support for indented lines in paragraphsLibravatar Titus Wormer1-23/+20
2022-06-09Add basic support for interrupting contentLibravatar Titus Wormer1-69/+95