diff options
| author |  Titus Wormer <tituswormer@gmail.com> | 2022-07-04 15:21:11 +0200 |
|---|---|---|
| committer | Titus Wormer <tituswormer@gmail.com> | 2022-07-04 15:21:11 +0200 |
| commit | 0450e7c2b12bd3ef53e0cffb60a3dd860325b478 (patch) | |
| tree | cc73bb48ae6102b27b7b864f13585eb77ef86c2c /src/tokenizer.rs | |
| parent | 8eb4631bd7c4345ec2a0c9b2ca2e05bdb1d79dd7 (diff) | |
| download | markdown-rs-0450e7c2b12bd3ef53e0cffb60a3dd860325b478.tar.gz markdown-rs-0450e7c2b12bd3ef53e0cffb60a3dd860325b478.tar.bz2 markdown-rs-0450e7c2b12bd3ef53e0cffb60a3dd860325b478.zip | |
Add support for unicode punctuation
Diffstat (limited to '')
| -rw-r--r-- | src/tokenizer.rs | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/src/tokenizer.rs b/src/tokenizer.rs index 9d870c9..1fa94d7 100644 --- a/src/tokenizer.rs +++ b/src/tokenizer.rs @@ -11,9 +11,9 @@ //! [`attempt`]: Tokenizer::attempt //! [`check`]: Tokenizer::check -use std::collections::HashMap; use crate::constant::TAB_SIZE; use crate::parser::ParseState; +use std::collections::HashMap; /// Semantic label of a span. // To do: figure out how to share this so extensions can add their own stuff, |