Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: implement multithreading for rendering and local search indexing, use JSDOM for better section split performance and reliability. #3386

Open
wants to merge 52 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
d0b34f6
feat(build): log duration for each task, use humanized format
zhangyx1998 Dec 28, 2023
0b37eff
feat(build): log duration for each task, use humanized format
zhangyx1998 Dec 28, 2023
e568de3
feat(build): add progress handler for task callback, show progress fo…
zhangyx1998 Dec 28, 2023
8849d85
fix: remove debugger statement for local test
zhangyx1998 Dec 28, 2023
9281648
move callback function outside try catch block, refactor logic
zhangyx1998 Dec 29, 2023
f9f9684
config: use local machine's cpu count for default concurrency value
zhangyx1998 Dec 29, 2023
5c06a08
render: code cleanup (html composition)
zhangyx1998 Dec 29, 2023
e5ed19d
feat/multithread-render: it works!
zhangyx1998 Dec 29, 2023
b796f21
rollup.config: clarify comment
zhangyx1998 Dec 29, 2023
2149f6b
rollup.config: remove debug console logging
zhangyx1998 Dec 29, 2023
1b47c8e
perf: improve rpc efficiency
zhangyx1998 Dec 29, 2023
541ad0a
cleanup logic, improve perf
zhangyx1998 Dec 30, 2023
f196fb6
Merge branch 'main' into feat/multithread-render
zhangyx1998 Dec 30, 2023
8c429b4
revert rollup.config: manualChunks not helpful
zhangyx1998 Dec 30, 2023
e31a3e5
Merge branch 'main' into feat/log-duration-for-each-task
zhangyx1998 Dec 30, 2023
ace515b
use picocolors instead of terminal control sequence for color marks
zhangyx1998 Dec 31, 2023
9195369
expose updateHandle for subtasks
zhangyx1998 Dec 31, 2023
007fa59
build/bundle: use task return value passthrough
zhangyx1998 Dec 31, 2023
25ca443
infrastructure: worker context and dispatch manager
zhangyx1998 Jan 3, 2024
e7c0a82
feat: report subtask progress without a spinner
zhangyx1998 Jan 3, 2024
37c94cc
Merge branch 'main' into feat/multithread-render
zhangyx1998 Jan 3, 2024
50093fd
Merge branch 'feat/log-duration-for-each-task' into feat/multithread-…
zhangyx1998 Jan 3, 2024
cc101a7
fix outdated pnpm lock file
zhangyx1998 Jan 3, 2024
2449ffc
fix merge errors
zhangyx1998 Jan 3, 2024
9d5aedc
perf(localSearch): Use JSDOM for section split, adapt to worker API
zhangyx1998 Jan 3, 2024
046b653
Merge branch 'perf/local-search' into feat/multithread-render
zhangyx1998 Jan 3, 2024
3a720eb
refactor localSearch: clean up code
zhangyx1998 Jan 3, 2024
1426208
update pnpm lock file
zhangyx1998 Jan 3, 2024
d86be7a
render and mini search splitter both working under the new API
zhangyx1998 Jan 3, 2024
304a9e5
build: remove debug log
zhangyx1998 Jan 3, 2024
ba0c3f5
Merge branch 'main' into feat/multithread-render
zhangyx1998 Jan 3, 2024
6f3d80e
remove unneeded file
zhangyx1998 Jan 3, 2024
a45bae9
upgrade rpc-magic-proxy to 1.0.3
zhangyx1998 Jan 4, 2024
3479538
upgrade rpc-magic-proxy to 1.0.4
zhangyx1998 Jan 4, 2024
d79465a
everything is working !!!
zhangyx1998 Jan 4, 2024
9307bbb
fix: make contentLoader happy
zhangyx1998 Jan 4, 2024
8b8fe14
dirty fix: bypass side effect of md.render()
zhangyx1998 Jan 4, 2024
c3df64a
cleanup worker code, solve server hanging issue (only happens in tests)
zhangyx1998 Jan 4, 2024
8edb076
test script: use run-s instead of run-p for prettier console output, …
zhangyx1998 Jan 4, 2024
346e54d
siteConfig: provide better control over parallelism, disable parallel…
zhangyx1998 Jan 5, 2024
77f708a
cleanup code
zhangyx1998 Jan 5, 2024
93aff9f
Merge branch 'main' into feat/multithread-render
zhangyx1998 Jan 5, 2024
e75adf9
revert irrelevant change
zhangyx1998 Jan 5, 2024
754a8cb
Merge branch 'main' into feat/multithread-render
zhangyx1998 Jan 8, 2024
e98da5b
resolve pnpm lock error
zhangyx1998 Jan 8, 2024
5862747
local search: avoid indexing 2 times when bundling for server and client
zhangyx1998 Jan 8, 2024
1c880a8
package upgrade
zhangyx1998 Jan 8, 2024
5493202
revert changes for parallel bundling
zhangyx1998 Jan 8, 2024
1dd35b2
strip changes from #3385
zhangyx1998 Jan 8, 2024
e5a1c75
fix broken imports due to stripped feature
zhangyx1998 Jan 8, 2024
25ed7dc
Merge branch 'main' into feat/multithread-render
zhangyx1998 Jan 13, 2024
671087c
fix pnpm lock file after merge
zhangyx1998 Jan 13, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
6 changes: 5 additions & 1 deletion package.json
Expand Up @@ -64,7 +64,7 @@
"build:prepare": "rimraf dist && node scripts/copyShared",
"build:client": "vue-tsc --noEmit -p src/client && tsc -p src/client && node scripts/copyClient",
"build:node": "tsc -p src/node --noEmit && rollup --config rollup.config.ts --configPlugin esbuild",
"test": "run-p --aggregate-output test:unit test:e2e test:init",
"test": "run-s test:unit test:e2e test:init",
"test:unit": "vitest run -r __tests__/unit",
"test:unit:watch": "vitest -r __tests__/unit",
"test:e2e": "run-s test:e2e-dev test:e2e-build",
Expand Down Expand Up @@ -100,6 +100,7 @@
"@vueuse/core": "^10.7.1",
"@vueuse/integrations": "^10.7.1",
"focus-trap": "^7.5.4",
"jsdom": "^23.0.1",
"mark.js": "8.11.1",
"minisearch": "^6.3.0",
"shikiji": "^0.9.19",
Expand Down Expand Up @@ -139,6 +140,7 @@
"@types/debug": "^4.1.12",
"@types/escape-html": "^1.0.4",
"@types/fs-extra": "^11.0.4",
"@types/jsdom": "^21.1.6",
"@types/lodash.template": "^4.5.3",
"@types/mark.js": "^8.11.12",
"@types/markdown-it-attrs": "^4.1.3",
Expand All @@ -155,6 +157,7 @@
"conventional-changelog-cli": "^4.1.0",
"cross-spawn": "^7.0.3",
"debug": "^4.3.4",
"dom-traverse": "^0.0.1",
"esbuild": "^0.19.11",
"escape-html": "^1.0.3",
"execa": "^8.0.1",
Expand Down Expand Up @@ -190,6 +193,7 @@
"rollup": "^4.9.5",
"rollup-plugin-dts": "^6.1.0",
"rollup-plugin-esbuild": "^6.1.0",
"rpc-magic-proxy": "^2.0.6",
"semver": "^7.5.4",
"simple-git-hooks": "^2.9.0",
"sirv": "^2.0.4",
Expand Down