aboutsummaryrefslogtreecommitdiffstats
path: root/src/tokenizer.rs (unfollow)
Commit message (Expand)AuthorFilesLines
2022-06-22Add `attempt_opt` to tokenizerLibravatar Titus Wormer1-35/+30
2022-06-22Rename `Whitespace` token to `SpaceOrTab`Libravatar Titus Wormer1-15/+15
2022-06-22Refactor to improve tokenizer, add docsLibravatar Titus Wormer1-144/+58
2022-06-22Add docs for token typesLibravatar Titus Wormer1-6/+1102
2022-06-21Add support for passing token types to destination, label, titleLibravatar Titus Wormer1-1/+1
2022-06-21Refactor to improve a bunch of statesLibravatar Titus Wormer1-0/+1
2022-06-21Update todo listLibravatar Titus Wormer1-16/+23
2022-06-21Add support for inferring line ending, configurableLibravatar Titus Wormer1-2/+4
2022-06-20Add support for BOMLibravatar Titus Wormer1-0/+10
2022-06-20Fix bug with tabsLibravatar Titus Wormer1-1/+6
2022-06-20Add improved whitespace handlingLibravatar Titus Wormer1-27/+5
2022-06-20Remove unneeded `content` content typeLibravatar Titus Wormer1-2/+0
2022-06-17Add support for definitionsLibravatar Titus Wormer1-15/+67
2022-06-16Add heading (setext)Libravatar Titus Wormer1-4/+7
2022-06-16Refactor to reorder thing alphabeticallyLibravatar Titus Wormer1-21/+7
2022-06-16Add support for hard break (trailing)Libravatar Titus Wormer1-2/+11
2022-06-16Add support for hard break escapeLibravatar Titus Wormer1-3/+11
2022-06-15Add code (text)Libravatar Titus Wormer1-5/+22
2022-06-14Reorganize to split utilLibravatar Titus Wormer1-0/+2
2022-06-13Add basic html (text)Libravatar Titus Wormer1-2/+32
2022-06-13Add autolinksLibravatar Titus Wormer1-0/+5
2022-06-10Add text content typeLibravatar Titus Wormer1-1/+1
2022-06-10Add proper support for subtokenizationLibravatar Titus Wormer1-3/+37
2022-06-09Add basic subtokenization, string content in fenced codeLibravatar Titus Wormer1-7/+8
2022-06-09Refactor to pass more slices aroundLibravatar Titus Wormer1-5/+5
2022-06-09Add basic support for interrupting contentLibravatar Titus Wormer1-1/+57