Commit message (Collapse) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | Update todo list | Titus Wormer | 2022-06-21 | 1 | -8/+1 |
| | |||||
* | Add support for BOM | Titus Wormer | 2022-06-20 | 1 | -0/+4 |
| | |||||
* | Remove unneeded `content` content type | Titus Wormer | 2022-06-20 | 1 | -6/+3 |
| | |||||
* | Fix support for deep subtokenization | Titus Wormer | 2022-06-14 | 1 | -9/+19 |
| | | | | | | * Fix a couple of forgotten line ending handling in html (text) * Fix missing initial case for html (text) not having a `<` 😬 * Add line ending handling to `text` construct | ||||
* | Reorganize to split util | Titus Wormer | 2022-06-14 | 1 | -6/+4 |
| | |||||
* | Add docs for html (text) | Titus Wormer | 2022-06-14 | 1 | -0/+1 |
| | |||||
* | Add basic html (text) | Titus Wormer | 2022-06-13 | 1 | -3/+9 |
| | | | | | | | * Add all states for html (text) * Fix to link paragraph tokens together * Add note about uncovered bug where linking paragraph tokens together doesn’t work 😅 | ||||
* | Add text content type | Titus Wormer | 2022-06-10 | 1 | -4/+10 |
| | | | | | * Add character reference and character escapes in text * Add recursive subtokenization | ||||
* | Add proper support for subtokenization | Titus Wormer | 2022-06-10 | 1 | -50/+116 |
| | | | | | | | - Add “content” content type - Add paragraph - Add skips - Add linked tokens | ||||
* | Add basic subtokenization, string content in fenced code | Titus Wormer | 2022-06-09 | 1 | -0/+67 |