Start-up / shut-down tasks?

[mindplay-dk]

Is there any way to perform some tasks before the test-suite starts, and after it ends?

(I don't mean before/after each test, but before/after the whole test-suite.)

I'm launching an express server for some functional tests - I'd like to keep this running for the duration of the entire test-suite, and then need to dispose of the server at the end.

I managed to "emulate" a start-up task this way:

import express from "express";
import type { Server } from "http";

let testServer: Promise<Server>; // 👈 singleton

export async function getTestServer(): Promise<Server> {
  return testServer || (testServer = createTestServer()); // 👈 initialize singleton
}

async function createTestServer(): Promise<Server> {
  return new Promise(resolve => {
    const app = express();

    // ... configure express...

    const server = app.listen(options.port, () => resolve(server));
  });
}

But how do I shut down?

I need to somehow wait for zora's test-runner finish, so I know when to shut down the test-server - for as long as it's up, the application doesn't end, which means the test hangs after it finishes.

I could start it up and shut it down for every individual test, but that will slow things down a lot - and it's a stateless server, so there's no practical reason to do that. I'd like to reuse it across all tests.

(incidentally, this is a good example of why I argued here that being required to explicitly kick off the test-runner would be "a good thing" - an obvious guess would have been something await run() after queing up your tests...)

Is there still a way to manually create your test-harness? That bit appears to have vanished from the README with version 5. The code of course is still there, but doesn't seem to have anything for this.

I could add a decorator around the reporter, I guess - but that seems a bit hacky and probably not really what that's for?

[mindplay-dk]

My solution for now is, I take control of the test-harness myself - I added a test.ts:

import { createHarness } from "zora";

export const harness = createHarness({});

export const { test } = harness;

Calling createHarness disables the default, global harness.

In all of my tests, I import { test } from "./harness" instead of from "zora".

In the test entry-point script, I can then manually await the harness:

import { harness } from "./harness";
import { createJSONReporter } from "zora";
import { getTestServer } from "./test-server"; // 👆 see above

(async () => {

  await harness.report({
    reporter: createJSONReporter(),
  });

  const server = await getTestServer();

  await new Promise(resolve => server.close(resolve));
  
})();

Gets the job done. 👍

As discussed here) though, this honestly still leaves me wishing that the default harness didn't even exist - global state, and things executing without your starting them, is pretty confusing, and it seems unnecessary. Creating the harness and starting the reporter saves you literally two lines of code.

Just my opinion, but I'd really like to see zora be a pure library - the automatic starting and reporting makes it feel more like a framework, and it seems like we already have pta filling that role.

[lorenzofox3]

I finally got the test-suite running (duh, rtfm) and my changes don't seem to break anything, so something isn't covered by tests - this should definitely be a breaking change in a couple of places...

Yes: I had not finished to migrate some of the tests (the ones with the extension .todo). The ones dealing with only mode should break in your case.

I added exports to the root package.json, and maybe that's wrong? I don't fully understand how this workspaces feature works - the package.json in the root, and the one the zora folder both have "name": "zora", so that's pretty confusing.

The root package is not a package per se (not published). The name does not really matter and I should simply remove it. It simply helps with the scaffolding of various projects within the same git repository (cf npm workspaces).

Which one is actually the package? I'm guessing the one in the zora folder, so I probably did it wrong. I don't understand yet what the package.json in the root is for then - is that just for the "development project" and not for any of the packages that actually get published?

Yes you are correct. You can google for npm workspaces for more info.

The pacakges published are: zora, pta and zora-reporters

[lorenzofox3]

Okay, I get it now - the root package isn't really a "package" per se. It never gets published - it's just a project that provides a structure (workspace) for the actual packages. (I learned from this video and reading the documentation.)

correct.

So I definitely got it wrong: the root package shouldn't have exports at all - it never gets published, so it can't export anything.

correct again. Only there for development

With that in mind, I corrected both package.json files, and configured rollup to produce separate modules: mindplay-dk@fe65288

That is the idea. Although I would create a core file which exports not only the harness but also the Assert prototype and eventually some other functions (to create protocol messages for example)

This resulted in chunking (modules with ugly hashes in the names) for two reasons:

1. Both `index` and `harness` import from `test`, and rollup doesn't like to duplicate code.

2. Both `index` and `harness` import from `zora-reporters`, which gets bundled into the `zora` package.

I went ahead and configured rollup to simply emit all the input modules one to one: mindplay-dk@e3a8ef8

That is the idea: you can create multiple entry points (core and index in our case) and rollup should then be able to factorize common chunks into "utility files" which avoid code duplication. (you can have a look at rollup doc if you want more info)

That also means zora-reporters is now treated as external, meaning it no longer gets bundled - otherwise, because it's a dependency in two places, we would get chunks.

That's problematic because zora needs to be bundled together with the two default reporters (TAP and JSON) which are platform agnostic.

I don't know if this was done intentionally? It isn't defined as either a dependency or a dev-dependency, so it doesn't get installed for the zora package, and I don't fully understand how you can load it in the first place? Maybe this is part of how workspaces work - enabling you to load packages you haven't actually listed as dependencies? In the tutorial video (above) they were adding their dependencies.

Yes everything is intentional. You don't need to (and should not) list zora-reporters as a dependency: node is able to resolve it at development time thanks to the workspaces set up.
When Rollup builds the artefacts for zora, it is also able to resolve the code it needs from zora-reporters (and trim all what is not relevant). So goes the same with the assertion workspace which is defined as an independent project (but do not get published)
So then, I only need to publish the built artefacts to npm and that's it. For the consumer, it is transparent: it gets the artefacts defined by zora's package.json and does not have to know the development set up and internal dependency tree. Once published zora is dependency free and fully bundled in one file (or two because I need the CJS version for legacy project and TS, enventually)

zora-reporters get published too on its side to get the bin file and all the reporters (not only the ones zora needs) but technically has nothing to do with zora itself.

I'm unsure, but probably the dependency should be added, now that zora-reporters is treated as external: mindplay-dk@055462f

I think this makes sense? Since zora-reporters gets published as a separate module, it shouldn't also be bundled internally with zora, sine that would make it impossible to update one or the other independently. It also creates a weird situation, if you actually install zora-reporters as a dependency in your project, since the default harness would reference the bundled version of zora-reporters, while a custom harness would reference your installed version, right?

You should not see zora-reporters as a dependency of zora. The bundled version of zora happens to have the code related to the TAP and the JSON reporter but the fact it comes from the zora-reporters workspace is just a coincidence and an organisational trick.

The zora-reporters pacakge comes as an independent package with a lot more (the diff reporter, the cli and any new reporter). If I put the TAP and the JSON reporter into the zora-reporters workspace, it is just a matter of organization.

I put that last change on a separate branch, in case I'm wrong.

I'm kind of fumbling around here. 😅

Footnote: one thing that threw me off was, both the root package and the actual zora package have "name": "zora" in package.json - that happens to work because the root package never gets installed as a dependency, but it's maybe a little confusing? I would suggest maybe zora-workspace for the root package. I don't think it strictly needs a name at all? Since it never gets published. But it's not the zora package you get on npm, so it might be nice to disambiguate?

yes the name in the root package is not useful at all and I should simply remove it

[mindplay-dk]

you can create multiple entry points (core and index in our case) and rollup should then be able to factorize common chunks into "utility files" which avoid code duplication

I wasn't sure if you would be okay with that - some developers are opposed to publishing bundler artifacts for node packages. I'm not really for or against myself - but if you're going to do that, it should probably ship with source-maps, which it also isn't set up for right now?

That also means zora-reporters is now treated as external, meaning it no longer gets bundled [...]

That's problematic because zora needs to be bundled together with the two default reporters (TAP and JSON) which are platform agnostic.
[...]
The zora-reporters pacakge comes as an independent package with a lot more (the diff reporter, the cli and any new reporter). If I put the TAP and the JSON reporter into the zora-reporters workspace, it is just a matter of organization.

I would suggest you move the TAP and JSON reporters to the core, then?

Since they have to be bundled anyhow, they're not really an optional dependency - they are a part of the core functionality. It might feel good to put them in zora-reporters because they're reporters, but as you said, it's just a matter of organization, and arguably this is core functionality, since the core doesn't function without it.

That also avoids having the bundler produce artifacts - probably don't need source-maps either then.

And it removes any concerns about zora-reporters having more dependencies.

[lorenzofox3]

I wasn't sure if you would be okay with that - some developers are opposed to publishing bundler artifacts for node packages. I'm not really for or against myself - but if you're going to do that, it should probably ship with source-maps, which it also isn't set up for right now?

You don't need source map here as there is no "source" in the published package, see the file served by the CDN for example. The bundler simply concatenates files, there is no compilation, obfuscation etc.
Again, the rest are details of the development flow. From the consumer perspective, zora is juste one file of source code, that's it.

I would suggest you move the TAP and JSON reporters to the core, then?

Since they have to be bundled anyhow, they're not really an optional dependency - they are a part of the core functionality. It might feel good to put them in zora-reporters because they're reporters, but as you said, it's just a matter of organization, and arguably this is core functionality, since the core doesn't function without it.

No thanks you, from a development perspective, I prefer to find all the reporters in the reporters directory

[mindplay-dk]

Just for the record, you understand the bundler is currently copying from zora-reporters to the published files, right?

It wouldn't matter if reporters was just an internal package/repository (implementation detail) - the only reason I bring it up is because zora-reporters is also being published as a separate package, which means you end up with two copies (two different versions) of the two bundled reporters at run-time. It could get weird, that's all. Since the dependency isn't declared anywhere, it isn't managed by npm, so there's technically no guarantee the bundled reporters will be compatible with an installed reporters package.

Just FYI, if this wasn't clear. If you don't care, that's your call. 🙂