ts-loader project references "build" support

[johnnyreilly]

ts-loader has partial support for project references in that it will load dependent composite projects that are already built, but will not currently build/rebuild those upstream projects. We don’t want to keep it that way; this issue has been opened to track ts-loader triggering the building/rebuilding of upstream projects.

Copied from #815 (comment) by @andrewbranch:

Small status update: I spent an hour or so this morning exploring the current state of the tsbuild API to see if it contains what we need for ts-loader to build/watch upstream projects. It mostly does, but currently the API isn't finalized and the whole thing is marked internal: https://github.com/Microsoft/TypeScript/blob/85a3475df87b9c10240570f32090970262c99b54/src/compiler/tsbuild.ts#L1-L2

I think ts-loader’s full support for project references will have to wait until that’s finalized, lest we get broken by minor or patch TypeScript releases.

@sheetalkamat, two things:

1. Would you be able to share with us whether that API is imminent, or more of a longer term goal?
2. The thing that ts-loader needs to be able to do, eventually, is build every dependency of a project, but not that project itself. After a little exploring this morning, it looks like the way to do that (in the hypothetical world where today’s internal APIs are final) is currently something like:

const project = path.resolve('./tsconfig.json'); // “Root” project, has dependencies
const builder = ts.createSolutionBuilder(host, [project], {});
const { buildQueue } = builder.getBuildGraph([project]);

// buildQueue is an array of paths to tsconfig files, in the order that they need to be built.
// That means our “root” project is the last element in the array. We want to build every
// project _except_ that one.
const projectsIWantToBuild = buildQueue.slice(0, buildQueue.length - 1);
const dependencyBuilder = ts.createSolutionBuilder(host, projectsIWantToBuild, {});
dependencyBuilder.buildAllProjects();

I realize all this is still in flux, so I just wanted to give you an idea of the functionality we’ll need here, since it’s not the typical case of running a solution build programmatically. At the moment, it feels a little kludgy to have to create two SolutionBuilders. Might there be a way in the future to build all projects with a filter, or simply to build all dependencies of a project? Thanks for all your work on this!

cc @RyanCavanaugh @rbuckton

[sheetalkamat]

@johnnyreilly yes we aim to make api for solutionBuilder but it would take a while since we are still working on fixing nitty gritties of project references.
But once that is done I think api would look different from what you are seeing now in that it might have a little different host than compilerHost and provide you api such that you take a custom action on each project or something similar and use solution builder for different answers about the status. (note that even today after getting projectsIWantToBuild you don't need to create solution builder here. you want to just createProgram with your custom set of emit/error actions just like you do for the project)

[johnnyreilly]

Thanks so much for that @sheetalkamat!

cc @andrewbranch

[andrewbranch]

Hey @sheetalkamat, any update on this? I realize you probably can't promise a timeline, but I just wanted to make sure we still have hope to keep this issue alive. 🙂 Thanks!

[sheetalkamat]

microsoft/TypeScript#28640 tracks this and it would be a while before API is public as we work through the perf issues and shape it accordingly.

[Bnaya]

TypeScript 3.3 supports incremental build with composite projects
https://github.com/Microsoft/TypeScript/wiki/Roadmap#33-january-2019

[kirill-onufrienko-db]

Is there any work in progress on this or the issue was put on hold?

[johnnyreilly]

I'm not working on it... It's possible @andrewbranch may be but I don't know

[andrewbranch]

This is still the current status: #851 (comment)

It should be something that's possible in the future, just not quite yet. Thanks for your patience, everyone 🌟

[styfle]

It looks like TS just added support

• Issue: Create tsbuild public API and use incremental build in tsbuild --watch mode microsoft/TypeScript#28640
• PR: Api for tsc --build and --incremental microsoft/TypeScript#31432

[kukjevov]

Anything new here ?

[paulvanbrenk]

I kinda need this. If nobody is looking at implementing i'll take a stab at it.

[kirill-konshin]

YES PLEASE! It's a long awaited feature )))

TypeScript 3.6 has API support for this: https://devblogs.microsoft.com/typescript/announcing-typescript-3-6/#apis-to-support-build-and-incremental

#913 this is a similar issue

[andrewbranch]

#935 should fix this.