typescript bindings maintenance

[iimez]

Describe your changes

• Changed the build scripts and docs a bit so its easier to build the bindings from source.
• Deleted some outdated comments from the gyp config, and removed the deprecated nCtx on completions.
• Update the build_unix script to produce so files with the correct build type.
• Changed CI to call cmake with Release build type.
• Added support for streamed token decoding, following the python bindings implementation
• Updated the bindings to expose additional LLModel:load and LLModel:initGpu methods so its easier to copy the loading and backend-picking logic of the CUDA Backend PR. I also renamed a few things for consistency.
• Include nThreads in the LoadModelOptions, previously it had to be configured via setThreadCount.
• Added nGpuLayers as alias for ngl to the LoadModelOptions.
• Updated types and examples accordingly.
• I added a test suite that does actual inference and loads a real model on both cpu and gpu with minimal resource usage. And moved the existing tests into a separate test:ci script. Can switch CI config back to yarn test if inference in CI is possible.
• Updated deps and verified that the fallback node-addon build via node-gyp-build is still working. Changed the build setup a bit to include a assert-backend-sources.js script that facilitates easier testing of the fallback during development. This also allows us to get rid of the second bindings.gyp config.
• Added support for linux-arm64 runtimes+bindings builds. Tested with ubuntu 22 x86_64 (build-time) and debian bullseye arm64 docker image (runtime). Currently defaults to cpu (and no gpu support).

Breaking changes

• The CompletionOptions onReponseToken(tokenId, text) callback has been changed to onResponseTokens({ tokenIds, text }) (Note the plural.). This is related to the streamed emoji decoding / multi token character issue.

Issue ticket number and link

#2326

[cebtenzzre]

@jacoobes Please use shorter merge commit messages in the future - even a blank merge description is better than adding two pages of text to the top of the git log. Refined GitHub even clears the description by default. Thanks.

[jacoobes]

Oop, thanks!

…

On Mon, Jun 3, 2024 at 2:10 PM Jared Van Bortel ***@***.***> wrote: @jacoobes <https://github.com/jacoobes> Please use shorter merge commit messages in the future - even a blank merge description is better than adding two pages of text to the top of the git log. Refined GitHub <https://github.com/refined-github/refined-github> even clears the description by default. Thanks. — Reply to this email directly, view it on GitHub <#2363 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ASJS6O3HHAKRZL5LEUF5KQ3ZFS5TDAVCNFSM6AAAAABH5EJ5QCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBVHEZTEMBQG4> . You are receiving this because you were mentioned.Message ID: ***@***.***>