Commit message (Expand) | Author | Files | Lines | |
---|---|---|---|---|
2022-06-22 | Add docs for token types | 1 | -1/+3 | |
2022-06-21 | Add docs for `subtokenize` | 1 | -2/+51 | |
2022-06-21 | Update todo list | 1 | -8/+1 | |
2022-06-20 | Add support for BOM | 1 | -0/+4 | |
2022-06-20 | Remove unneeded `content` content type | 1 | -6/+3 | |
2022-06-14 | Fix support for deep subtokenization | 1 | -9/+19 | |
2022-06-14 | Reorganize to split util | 1 | -6/+4 | |
2022-06-14 | Add docs for html (text) | 1 | -0/+1 | |
2022-06-13 | Add basic html (text) | 1 | -3/+9 | |
2022-06-10 | Add text content type | 1 | -4/+10 | |
2022-06-10 | Add proper support for subtokenization | 1 | -50/+116 | |
2022-06-09 | Add basic subtokenization, string content in fenced code | 1 | -0/+67 |