diff options
| author |  Titus Wormer <tituswormer@gmail.com> | 2022-06-10 16:47:43 +0200 |
|---|---|---|
| committer | Titus Wormer <tituswormer@gmail.com> | 2022-06-10 16:47:43 +0200 |
| commit | 17f4eec55ad0a5f74aedbcff6c2f0119ad52e584 (patch) | |
| tree | 1839c796de977421456d1b9006f2f2c1e23cf809 /src/tokenizer.rs | |
| parent | 5133042973f31a3992f216e591d840bb491bfd45 (diff) | |
| download | markdown-rs-17f4eec55ad0a5f74aedbcff6c2f0119ad52e584.tar.gz markdown-rs-17f4eec55ad0a5f74aedbcff6c2f0119ad52e584.tar.bz2 markdown-rs-17f4eec55ad0a5f74aedbcff6c2f0119ad52e584.zip | |
Add text content type
* Add character reference and character escapes in text
* Add recursive subtokenization
Diffstat (limited to '')
| -rw-r--r-- | src/tokenizer.rs | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/src/tokenizer.rs b/src/tokenizer.rs index 1746a19..4d235ed 100644 --- a/src/tokenizer.rs +++ b/src/tokenizer.rs @@ -63,10 +63,10 @@ pub enum TokenType { BlankLineWhitespace, Content, - ContentChunk, Paragraph, + ChunkContent, ChunkString, ChunkText, } |