Commit message (Collapse) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | Refactor internal docs, code style of tokenizer | Titus Wormer | 2022-08-11 | 1 | -5/+1 |
| | |||||
* | Add improved container exit injection | Titus Wormer | 2022-08-11 | 1 | -5/+8 |
| | |||||
* | Rename `State::Fn` to `State::Next` | Titus Wormer | 2022-08-10 | 1 | -2/+2 |
| | |||||
* | Refactor to share some code | Titus Wormer | 2022-08-09 | 1 | -56/+86 |
| | |||||
* | Rewrite algorithm to not pass around boxed functions | Titus Wormer | 2022-08-09 | 1 | -7/+6 |
| | | | | | | * Pass state names from an enum around instead of boxed functions * Refactor to simplify attempts a lot * Use a subtokenizer for the the `document` content type | ||||
* | Refactor to use `debug_assert` | Titus Wormer | 2022-07-28 | 1 | -7/+7 |
| | |||||
* | Refactor to drastically improve perf around whitespace | Titus Wormer | 2022-07-26 | 1 | -6/+8 |
| | |||||
* | Refactor to simplify tokenizer | Titus Wormer | 2022-07-26 | 1 | -6/+3 |
| | |||||
* | Refactor to remove need for cloning codes | Titus Wormer | 2022-07-25 | 1 | -10/+4 |
| | |||||
* | Improve performance w/ a single feed loop | Titus Wormer | 2022-07-25 | 1 | -2/+6 |
| | |||||
* | Refactor to remove unneeded tuples in every states | Titus Wormer | 2022-07-22 | 1 | -13/+9 |
| | |||||
* | Refactor to pass ints instead of vecs around | Titus Wormer | 2022-07-22 | 1 | -4/+6 |
| | |||||
* | Refactor to move `index` field to `point` | Titus Wormer | 2022-07-21 | 1 | -5/+5 |
| | |||||
* | Refactor to move some event fields to `link` | Titus Wormer | 2022-07-21 | 1 | -35/+36 |
| | |||||
* | Refactor to share edit map | Titus Wormer | 2022-07-20 | 1 | -3/+3 |
| | |||||
* | Refactor to use less vecs for events | Titus Wormer | 2022-07-20 | 1 | -2/+4 |
| | |||||
* | Refactor to remove cloning in `edit_map` | Titus Wormer | 2022-07-19 | 1 | -2/+2 |
| | |||||
* | Use `edit_map` in `subtokenize` | Titus Wormer | 2022-07-19 | 1 | -67/+40 |
| | |||||
* | Remove an unneeded `HashMap` | Titus Wormer | 2022-07-19 | 1 | -1/+1 |
| | |||||
* | Fix annoying bug around virtual spaces in containers | Titus Wormer | 2022-07-15 | 1 | -1/+1 |
| | |||||
* | Add support for `Flow` content type | Titus Wormer | 2022-07-07 | 1 | -2/+4 |
| | |||||
* | Refactor to do some to dos | Titus Wormer | 2022-07-05 | 1 | -3/+2 |
| | |||||
* | Add support for unicode punctuation | Titus Wormer | 2022-07-04 | 1 | -1/+1 |
| | |||||
* | Update list of todos | Titus Wormer | 2022-07-04 | 1 | -2/+0 |
| | |||||
* | Fix jumps in `edit_map` | Titus Wormer | 2022-06-28 | 1 | -101/+99 |
| | | | | | | | | | * Use resolve more often (e.g., heading (atx, setext)) * Fix to link whole phrasing (e.g., one big chunk of text in heading (atx, setext), titles, labels) * Replace `ChunkText`, `ChunkString`, with `event.content_type: Option<ContentType>` * Refactor to externalize `edit_map` from `label` | ||||
* | Add link, images (resource) | Titus Wormer | 2022-06-24 | 1 | -12/+26 |
| | | | | | | | | | | | | This is still some messy code that needs cleaning up, but it adds support for links and images, of the resource kind (`[a](b)`). References (`[a][b]`) are parsed and will soon be supported, but need matching. * Fix bug to pad percent-encoded bytes when normalizing urls * Fix bug with escapes counting as balancing in destination * Add `space_or_tab_one_line_ending`, to parse whitespace including up to one line ending (but not a blank line) * Add `ParserState` to share codes, definitions, etc | ||||
* | Refactor some unneeded assignments | Titus Wormer | 2022-06-22 | 1 | -2/+1 |
| | |||||
* | Add docs for token types | Titus Wormer | 2022-06-22 | 1 | -1/+3 |
| | |||||
* | Add docs for `subtokenize` | Titus Wormer | 2022-06-21 | 1 | -2/+51 |
| | |||||
* | Update todo list | Titus Wormer | 2022-06-21 | 1 | -8/+1 |
| | |||||
* | Add support for BOM | Titus Wormer | 2022-06-20 | 1 | -0/+4 |
| | |||||
* | Remove unneeded `content` content type | Titus Wormer | 2022-06-20 | 1 | -6/+3 |
| | |||||
* | Fix support for deep subtokenization | Titus Wormer | 2022-06-14 | 1 | -9/+19 |
| | | | | | | * Fix a couple of forgotten line ending handling in html (text) * Fix missing initial case for html (text) not having a `<` 😬 * Add line ending handling to `text` construct | ||||
* | Reorganize to split util | Titus Wormer | 2022-06-14 | 1 | -6/+4 |
| | |||||
* | Add docs for html (text) | Titus Wormer | 2022-06-14 | 1 | -0/+1 |
| | |||||
* | Add basic html (text) | Titus Wormer | 2022-06-13 | 1 | -3/+9 |
| | | | | | | | * Add all states for html (text) * Fix to link paragraph tokens together * Add note about uncovered bug where linking paragraph tokens together doesn’t work 😅 | ||||
* | Add text content type | Titus Wormer | 2022-06-10 | 1 | -4/+10 |
| | | | | | * Add character reference and character escapes in text * Add recursive subtokenization | ||||
* | Add proper support for subtokenization | Titus Wormer | 2022-06-10 | 1 | -50/+116 |
| | | | | | | | - Add “content” content type - Add paragraph - Add skips - Add linked tokens | ||||
* | Add basic subtokenization, string content in fenced code | Titus Wormer | 2022-06-09 | 1 | -0/+67 |