feat: marked extensions #230

nobkd · 2024-03-14T01:54:49Z

resolve Allow adding Marked Extensions in NueJS #182

WIP/POC implementation for marked extension loading.

Description

The default file to load from is marked.config.js but this can be changed using the marked_config key in site.yaml.
The provided file path currently has to be relative to the root folder of the project / the root passed to nue command

The config js file has to be of format:

// e.g. imports

export default [ // has to be an array
  // initialize marked extensions with functions / own objects here
]

If no file is found, the import() function silently returns an empty object which, on unpacking, results in an empty list for marked_extensions variable

TODO

exclude marked config file from output folder
add tests

Notes

https://marked.js.org/using_pro#use

Maybe allow additional renderers with the following?

marked.use({
 extensions: [
   //...
 ]
})

tipiirai · 2024-03-15T06:59:19Z

Seems legit. This should only cover extensions that are not possible to implement as Nuemark extensions/tags. Footnote is probably the only one. Anything else?

nobkd · 2024-03-24T03:22:35Z

Well, you can also add all the other marked extensions.
It is sometimes easier to just use an existing package instead of re-building stuff yourself.

e.g.

extended tables syntax
footnotes, smartypants (lite) (as it was removed from the base package)
katex could also be interesting for some use cases
etc.

https://marked.js.org/using_advanced#extensions

Still WIP, as I have to do the todos, and want to use joinRootPath (or changed name) to import the file.

BTW, I think, this config isn't needed, as the options are deprecated (smartypants, headerIds, mangle) or nonexistent (smartLists at least it's not in this table):
https://github.com/nuejs/nue/pull/230/files#diff-9aaf075b4e5c776d59c27ef6c62e9c1079673f706482643126f348d7c6550cdfR9-R14

nobkd · 2024-03-24T21:05:17Z

Rebased on #232 and removed deprecated options. Still have the todo's open...

Edit:
Notes:

Seems like the path for import has to be absolute, as it probably tries to load relative to the current js file (just an assumption) / packages from node_modules folder
I'm not really sure, how to append the marked config file to the list of files to be excluded from dist, because
1. I don't quite know the location in code where this has to be added
2. I have an absolute file path for importing, which could theoretically be absolute from the start

nobkd · 2024-04-03T00:30:35Z

I've reached the conclusion, that for now, I'll just define the config file name to marked.config.js, as I am not too sure, how to add a dynamic relative or absolute path to the IGNORE field properly.
If you are against my current concept, please let me know.

(An alternative I thought of was to create a nue.config.js or similar, where multiple config keys are included, that might need some js import. But that's just a thought experiment)

The current loading variant for marked extensions also does not allow changing the marked extensions at runtime, but I don't think that's a big problem.

BTW, I think, this config isn't needed, as the options are deprecated (smartypants, headerIds, mangle) or nonexistent (smartLists at least it's not in this table):

I have deleted that part from the code

I've also added package-lock.json and yarn.lock to the IGNORE field, even though it's unrelated to this PR. If you want to, I can also create a separate PR for this.

The only thing, that's left (for now), would be tests.

(Also haven't looked into the Notes part of my PR description, but the most relevant part is generally loading marked extensions)

nobkd · 2024-04-07T23:52:01Z

Fixed the test, forgot some quotation marks... and for some reason the rendered md, is escaped, when run with nuekit.

Currently, the test is not run, because process.exit() which will be fixed with #254
But I tested it locally with the line commented out.

This is now ready for review.

tipiirai · 2024-04-08T06:35:10Z

packages/nuekit/src/site.js

@@ -81,6 +81,9 @@ export async function createSite(args) {
 self.is_empty = true
 }

+ const markedConfig = joinRootPath(root, 'marked.config.js', true)
+ const { default: marked_extensions=[] } = await import(markedConfig).catch(() => ({}))


Oh. Didn't even know about this default: .. syntax. Nice one!

Yeah. Renames 'default' to 'marked_extensions' while unwrapping the variable from an object

tipiirai · 2024-04-08T06:49:36Z

Looks solid. I did some changes to how header id's are rendered on the dev branch. Can you check if this causes any conflicts when merging to dev?

Thanks!

nobkd · 2024-04-08T07:50:12Z

I'm relatively sure, it will result in merge conflicts.
I moved the variable renderer and marked.use so the Nue markdown renderer is loaded before other extensions.
But it shouldn't be that much of a problem, since I only moved it inside the file without changing it.

nobkd · 2024-04-18T00:48:20Z

When I use the workspace packages (using changes from #261), the tests with node+jest (on Linux) still fail, somehow connected to the dynamic import with absolute path not being found in the test, even though the file exists (in package checks checking for the file fail too though (jest)...)

Bun in contrast resolves the absolute path correctly without problems. The same goes for npm if I link the nuekit package to some test project. I think, it just fails in the jest environment...

I haven't checked it with bun+jest yet. If it still occurs there, I can be sure that jest is the culprit. It might also be an --experimental-vm-modules issue...
Not sure, what to do here and how to check where this issue comes from (other than being related to dynamic import())

nobkd mentioned this pull request Mar 24, 2024

fix: use cwd + nue root as base for relative file output path #232

Merged

3 tasks

nobkd force-pushed the feat/marked-ext branch from 44aea5d to d96afb4 Compare March 24, 2024 21:00

nobkd added 3 commits March 25, 2024 08:34

feat: (WIP) load marked extensions from js file

dcefc4a

chore: rebase

beaa27d

chore: drop deprecated options

403536c

nobkd force-pushed the feat/marked-ext branch from 8026cdf to 403536c Compare March 25, 2024 07:41

chore: update IGNORE, static marked config file path

57cd73e

nobkd force-pushed the feat/marked-ext branch from 825c726 to 57cd73e Compare April 3, 2024 00:24

nobkd marked this pull request as ready for review April 3, 2024 00:35

test: basic marked extension test

81f343b

nobkd force-pushed the feat/marked-ext branch from f7e338b to 81f343b Compare April 7, 2024 23:49

nobkd requested a review from tipiirai April 7, 2024 23:53

tipiirai reviewed Apr 8, 2024

View reviewed changes

Merge branch 'master' into feat/marked-ext

a87f042

This comment was marked as outdated.

Sign in to view

test: fix test by not ignoring other inlineText tokenizers

3f116d0

nobkd force-pushed the feat/marked-ext branch from d3948a3 to 3f116d0 Compare April 11, 2024 12:16

Merge branch 'nuejs:master' into feat/marked-ext

a4f59db

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: marked extensions #230

feat: marked extensions #230

nobkd commented Mar 14, 2024 •

edited

tipiirai commented Mar 15, 2024

nobkd commented Mar 24, 2024 •

edited

nobkd commented Mar 24, 2024 •

edited

nobkd commented Apr 3, 2024 •

edited

nobkd commented Apr 7, 2024 •

edited

tipiirai Apr 8, 2024

nobkd Apr 8, 2024 •

edited

tipiirai commented Apr 8, 2024

nobkd commented Apr 8, 2024 •

edited

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

nobkd commented Apr 18, 2024 •

edited

feat: marked extensions #230

Are you sure you want to change the base?

feat: marked extensions #230

Conversation

nobkd commented Mar 14, 2024 • edited

Description

TODO

Notes

tipiirai commented Mar 15, 2024

nobkd commented Mar 24, 2024 • edited

nobkd commented Mar 24, 2024 • edited

nobkd commented Apr 3, 2024 • edited

nobkd commented Apr 7, 2024 • edited

tipiirai Apr 8, 2024

Choose a reason for hiding this comment

nobkd Apr 8, 2024 • edited

Choose a reason for hiding this comment

tipiirai commented Apr 8, 2024

nobkd commented Apr 8, 2024 • edited

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

nobkd commented Apr 18, 2024 • edited

nobkd commented Mar 14, 2024 •

edited

nobkd commented Mar 24, 2024 •

edited

nobkd commented Mar 24, 2024 •

edited

nobkd commented Apr 3, 2024 •

edited

nobkd commented Apr 7, 2024 •

edited

nobkd Apr 8, 2024 •

edited

nobkd commented Apr 8, 2024 •

edited

nobkd commented Apr 18, 2024 •

edited