Commit message (Expand) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | Add `attempt_opt` to tokenizer | Titus Wormer | 2022-06-22 | 1 | -2/+2 |
* | Add docs for token types | Titus Wormer | 2022-06-22 | 1 | -0/+5 |
* | Refactor to improve a bunch of states | Titus Wormer | 2022-06-21 | 1 | -12/+8 |
* | Add improved whitespace handling | Titus Wormer | 2022-06-20 | 1 | -5/+4 |
* | Remove unneeded `content` content type | Titus Wormer | 2022-06-20 | 1 | -3/+3 |
* | Remove unneeded `pub` keywords | Titus Wormer | 2022-06-20 | 1 | -32/+29 |
* | Fix support for deep subtokenization | Titus Wormer | 2022-06-14 | 1 | -10/+14 |
* | Add docs for html (text) | Titus Wormer | 2022-06-14 | 1 | -55/+272 |
* | Add basic html (text) | Titus Wormer | 2022-06-13 | 1 | -0/+480 |