Enable wide Unicode support for names #24
Conversation
github-actions
bot
added
👋 phase/new
Codecov ReportAll modified and coverable lines are covered by tests ✅
❗ Your organization needs to install the Codecov GitHub app to enable full functionality. Additional details and impacted files@@ Coverage Diff @@
## main #24 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 9 9
Lines 1416 1439 +23
=========================================
+ Hits 1416 1439 +23 ☔ View full report in Codecov by Sentry. |
| return self.previous === codes.dash || self.previous === codes.underscore | ||
| ? nok(code) | ||
| : ok(code) | ||
| return allowedEdgeCharacter(self.previous) ? ok(code) : nok(code) |
There was a problem hiding this comment.
shouldn’t dashes also be edge characters?
There was a problem hiding this comment.
If you mean allowed edge characters, it was forbidden by the spec previously. I kept it but I don't mind changing.
Currently, the name cannot either start or end with any punctuation or underscore.
Is this something you suggest to change?
There was a problem hiding this comment.
To refresh my memory: So, in name, this last stuff is about what is possible to exit after.
That behavior at the end is very different from whether the first character is allowed to start a name.
Before, there was a very different check compared to the check in start: - and _ were allowed in names but not at the end.
Now they’re the same. I’m not sure if that’s useful? Perhaps the last line should just be return ok(code)?
There was a problem hiding this comment.
I have tried return ok(code) at the end. Allowing the name to end with an underscore interferes with emphasis notation. I reverted the commit.
I think, there is no point to allow punctuation in the end but don't allow at the start. If we are going to allow punctuation, it should be (almost) equal.
Possible options:
- Leave as is
- Allow any punctuation at the start but forbid markdown special chars at the end. Those should include
=,~(special in some flavours),_,*, parentheses and perhaps some more — basically anything in the ASCII (in comparison to option 3, blacklist under 128). - Allow any punctuation at the start and end if it's beyond ASCII (in comparison to option 2, whitelist above 127)
|
You could maybe test the math symbol for pi |
Interferes with emphasis This reverts commit f4ec634.
| assert.equal(micromark('::𝜋∈ℝ', options()), '') // Italic | ||
| assert.equal(micromark('::𝛑≈3.14', options()), '') // Bold | ||
| assert.equal(micromark('::𝝅∉ℚ', options()), '') // Bold italic | ||
| assert.equal(micromark('::𝞹≠3.14', options()), '') // Sans bold italic |
|
Thanks for your continued work. I’ve been wondering the last week what to do about punctuation. And about the implications to the semver version of this package. In the ASCII range we allow ascii alphanumerics and Some examples of symbols/punctuation outside of the ascii range are |
Initial checklist
Description of changes
Enables almost full Unicode support for directive names. This is tricky to test, I've added only Latin, Greek and Cyrillyc characters. Also, I have tested combining accent modifiers at the middle and at the end of directive name.
Attribute names are worth to look too but maybe in a separate PR.
The PR should be ready to review. Thank you!
Closes #23