Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Heading token with mutiple end-of-line characters does not tokenized into Space token #3513

Open
Bistard opened this issue Oct 30, 2024 · 3 comments
Labels

Comments

@Bistard
Copy link

Bistard commented Oct 30, 2024

Marked version: 14.1.2

Describe the bug

Heading Token (buggy)

Heading token with mutiple end-of-line characters does not tokenized into Space token:

// '# Heading\n\n'
{type:"heading", raw:"# heading\n\n", depth:1, text:"heading", tokens:[
  {type:"text", raw:"heading", text:"heading"}
]}

But for other block-level tokens, two \n will be tokenized into a Space token:

BlockQuote Token

// '> hello\n\n'
{type:"blockquote", raw:"> hello", tokens:[
  {type:"paragraph", raw:"hello", text:"hello", tokens:[
    {type:"text", raw:"hello", text:"hello"}
]}
], text:"hello"}
{type:"space", raw:"\n\n"}

CodeBlock Token

// '```\nhello\n```\n\n'
{type:"code", raw:"```\nhello\n```", lang:"", text:"hello"}
{type:"space", raw:"\n\n"}

Paragraph Token

// 'paragraph here\n\n'
{type:"paragraph", raw:"paragraph here", text:"paragraph here", tokens:[
  {type:"text", raw:"paragraph here", text:"paragraph here"}
]}
{type:"space", raw:"\n\n"}

List Token

// '* list item\n\n'
{type:"list", raw:"* list item", ordered:false, start:"", loose:false, items:[
  {type:"list_item", raw:"* list item", task:false, checked:undefined, loose:false, text:"list item", tokens:[
    {type:"text", raw:"list item", text:"list item", tokens:[
      {type:"text", raw:"list item", text:"list item"}
]}
]}
]}
{type:"space", raw:"\n\n"}

Expected behavior

Why is the Heading token ending with two (or more) \n not tokenized into a space token? Is this expected?

Thank you for your response.

@UziTech
Copy link
Member

UziTech commented Oct 30, 2024

I don't think there is a reason for it. If you would like to create a PR to make it consistent we could get it into the next major version. 😁👍

@Bistard
Copy link
Author

Bistard commented Oct 30, 2024

OK. In the next few days or weeks, I will look up the source code and try to make it consistent through a PR.

@markedjs markedjs deleted a comment Oct 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

5 participants
@UziTech @Bistard and others