Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parallel Processing / Speed Up Generation? #509

Open
reedsutliff opened this issue Feb 27, 2023 · 1 comment
Open

Parallel Processing / Speed Up Generation? #509

reedsutliff opened this issue Feb 27, 2023 · 1 comment
Labels
enhancement New feature or request

Comments

@reedsutliff
Copy link

I am using Marp as part of a static site generation, in which i generate a slide cover image using --image jpg

I am running this against 40 presentations and it takes 1.5 minutes to process on github actions. I am doing the whole directory at one, so it seems like it would be trivial to parallelize the generation. Any ideas on how this could be done?

Thanks

@yhatt
Copy link
Member

yhatt commented Feb 28, 2023

Interesting. A one of possible ways is running Converter.prototype.convertFile() in parallel within Converter.prototype.convertFiles(), powered by worker_threads module.

for (const file of files) await this.convertFile(file, opts)

I feel there are a lot of barriers to using multi processes than you thought, e.g. a shared Converter class, the process handling of a singleton Chromium launched by Puppeteer, and compatibility with post-processing such as --server, --watch, and --preview.

@yhatt yhatt added the enhancement New feature or request label Feb 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: Todo
Development

No branches or pull requests

2 participants