Towards v1.0.0

[tommy-mitchell]

tsd is in need of a refresh. I think it can be improved and a 1.0 version released.

Table of Contents

• Improvements

  • Remove index.d.ts Requirement
  • CLI Overhaul
  • API Updates
  • Assertions Improvements
  • Configuration Schema
  • Documentation Rewrite
  • Internal Improvements
  • Support Additional Extensions

• Ideas

  • Test Multiple TypeScript Versions
  • eslint-plugin-tsd
  • Test Runner Integration
  • Type Quality Checks
  • @tsdjs / @tsdts

Improvements

Addressing issues with tsd.

Remove index.d.ts Requirement

One of tsd’s biggest pain points is its typings file requirement. Here's just some of the issues expressing confusion or frustration with it:

• How to write tests for source code in .ts files? #32
• The index.d.ts does not exist #92
• The type definition index.d.ts does not exist. Create one and try again #134
• The type definition index.d.ts does not exist. Create one and try again. #135
• TSD throws error about not finding typing definitions #151
• Why does a typingsFile need to be specified? #195

tsd should be updated to work without one - whether that be on pure TypeScript projects, or on plain JavaScript inputs. As part of this, options relating to the typings file should be removed - more on that in CLI Overhaul and API Updates.

CLI Overhaul

tsd's CLI is bound to projects with a package.json and an index.d.ts file. Many other tools run on glob input, and search for configuration files in the current directory if none are provided. I think tsd should do the same, and have a new --package option for testing a package's entry-point(s):

Usage
  $ tsd <patterns> [options]

Info
  --help            Display help text
  --version         Display version info
  --print-config    Dispaly resolved tsconfig.json

Options
  --package, --pkg  Test a package's entry-point(s)
  --show-diff       Show type error diffs            [Default: don't show]

Examples
  $ tsd test-d/**/*.ts --show-diff

  $ tsd --pkg

    index.test-d.ts
    ✖  10:20  Argument of type string is not assignable to parameter of type number.

This would make the --typings and --files flags obsolete.

API Updates

Besides updates related to adding/removing options mentioned above, the API's return value should be updated to report more information. Currently, it only returns an array of Diagnostics. Returning an object allows adding more information in the future without breaking changes (#75 (comment)). For example, it could report the number of tests run:

type Result = {
	diagnostics: Diagnostic[];
	numberOfTests: number;
};

Programmatic usage of tsd should also be able to work without a package.json. Current behavior could be maintained with a usePackageJson: true option, like the CLI's --package. Maybe an input: string[] or files: string | string[] option could be added to test string/file sources (which would resolve #86).

Assertions Improvements

Assertions can currently fail in unexpected ways (#141, #190). Possible improvements:

• const type parameters, introduced in TypeScript 5.0

• Using tsd-lite’s assertion type definitions:

   export declare function expectType<T>(expression: unknown): void;

I'd also like to extend expectError to support specific errors:

expectError<2304>(undeclared = one('foo', 'bar'));

This can be an escape hatch as an alternative to the warning added in #178.

Configuration Schema

Currently, tsd’s CLI can be configured via flags or a tsd field in a project’s package.json. We should export a JSON schema so users can get autocomplete/ errors / etc. when using the latter. This should be built from a TypeScript definition so we don’t have to duplicate anything.

In the future, if we decide to support more configuration locations (e.g. a .tsd.config.json), having an exported schema would be beneficial.

The package.json config should also support all of the available options (#198).

Documentation Rewrite

The readme is convoluted in places. I think adding a docs/, recipes/, and an examples/ folder (similar to AVA) where we could separate concerns would help organize things. The readme then could document installation and basic CLI/API usage.

Internal Improvements

Some ideas for maintaining tsd:

• Move to ESM (Move to ESM #183, Move to ESM, remove index.d.ts requirement #186)
  • Allows for updating some dependencies
• Transpile source code for tests (e.g ts-node, tsx)
  • Improves DX
• Explore using ts-api-utils to simplify interacting with the TS compiler

Support Additional Extensions

We should add .mts and .cts files to the default input path (#197).

Ideas

Things that can complement/improve tsd.

These are all "nice-to-haves", and the other improvements above have priority.

Test Multiple TypeScript Versions

Currently, tsd comes with a specific patched version of TypeScript as a direct dependency, and its version only changes when tsd itself is updated. While some work was done to allow testing multiple versions in #47, some other potential solutions are:

• BYOTS
• ts-expose-externals
• ts-patch
• Setting @tsd/typescript as a peer dependency - like in tsd-lite
• unleashed-typescript - as mentioned in Allow testing another TypeScript version #47 (comment)

eslint-plugin-tsd

An ESLint plugin that checks tsd usage and reports assertion failures, similar to eslint-plugin-expect-type. This could then suppress errors from the TypeScript compiler from being reported when using expectError. An alternative to #44, this could integrate with ESLint editor plugins and save us from writing one.

Test Runner Integrations

jest-runner-tsd is an extension for Jest that uses tsd-lite to report tsd type tests as Jest tests. Perhaps we could have similar integrations for other test runners, like AVA. Another nice feature would be using tsd in actual tests:

import ava from 'ava';
import {expectType} from 'tsd';

const sum = (a: number, b: number) => a + b;

test('tsd works', t => {
	t.is(sum(1, 1), 2);
	expectType<number>(sum(1, 1));
});

Type Quality Checks

We could optionally report type coverage (#63) and type memory usage (#34), among others. Perhaps this could be integrated with the potential eslint-plugin-tsd.

@tsdjs / @tsdts

I think it’s time that tsd and @tsd/typescript are moved to their own organization on GitHub, especially if some of these other ideas are implemented (plugins, test runner integrations, etc.)

---

Thoughts? I’d love to hear any other improvements or ideas anyone might have, or any concerns with my suggestions.

cc: @sindresorhus @BendingBender @skarab42 @mrazauskas

[mrazauskas]

Thanks for tagging me and thanks for mentioning tsd-lite as an example.

One of the details I was thinking to add, but didn’t had enough time was primitive type matchers like: expectString, expectNumber, etc. At the moment tsd has only expectNever.

[mrazauskas]

By the way, integration with Jest wasn’t very good idea. jest-runner-tsd needs Jest to select test files, but there is no way to disable haste map which Jest creates by crawling files. That’s an overhead. Also the selected file names are passed to the runner one by one, but TS compiler could take could take a list of files. Another problem: no way to pass CLI options to the runner. Of course, integration with AVA (or another runner) is possibly a different story.

---

Semi related: tsd-lite reads compiler options from the nearest tsconfig.json. (This is because there was no easy way to pass options to jest-runner-tsd as mentioned above.) That was intuitive decision, but I think it was very good idea. The same tsconfig.json is used by IDEs or tools like typescript-eslint. Worth to think about, if wider integration with various tools is considered.

[sindresorhus]

Many good ideas here 👍

---

Remove index.d.ts Requirement

👍

tsd's CLI is bound to projects with a package.json and an index.d.ts file. Many other tools run on glob input, and search for configuration files in the current directory if none are provided. I think tsd should do the same, and have a new --package option for testing a package's entry-point(s):

I agree, it should accept globs patterns. I would still prefer (for selfish reasons) it to default to running on the entry point when no input is specified though. But I could be convinced otherwise.

Besides updates related to adding/removing options mentioned above, the API's return value should be updated to report more information. Currently, it only returns an array of Diagnostics. Returning an object allows adding more information in the future without breaking changes (#75 (comment)). For example, it could report the number of tests run:

👍

Programmatic usage of tsd should also be able to work without a package.json. Current behavior could be maintained with a usePackageJson: true option

👍

Assertions Improvements

👍

Currently, tsd’s CLI can be configured via flags or a tsd field in a project’s package.json. We should export a JSON schema so users can get autocomplete/ errors / etc. when using the latter.

👍

The readme is convoluted in places. I think adding a docs/, recipes/, and an examples/ folder (similar to AVA) where we could separate concerns would help organize things. The readme then could document installation and basic CLI/API usage.

👍

Test Multiple TypeScript Versions

It's a "nice to have", but there's a huge amount of other work that is more important than this.

eslint-plugin-tsd

Also a "nice to have", but the core tsd improvements are more important.

I think it’s time that tsd and @tsd/typescript are moved to their own organization on GitHub, especially if some of these other ideas are implemented (plugins, test runner integrations, etc.)

👍

[tommy-mitchell]

It's a "nice to have"

Agreed, meant to clarify that the Ideas section are all nice to haves. I'll update the description.

I think it’s time that tsd and @tsd/typescript are moved to their own organization on GitHub, especially if some of these other ideas are implemented (plugins, test runner integrations, etc.)

Would you want to set this up, or get in touch with Sam to do so?

[sindresorhus]

Would you want to set this up, or get in touch with Sam to do so?

It's not up to me, but I will ask Sam.

[sindresorhus]

// @SamVerschueren

[SamVerschueren]

I'm definitely open to moving everything tsd related to an organisation instead.

I just checked, the name tsd is already taken. Any suggestions?

[tommy-mitchell]

tsdjs is available, similar to xojs or avajs.

[sindresorhus]

tsdjs is available

👍

[tommy-mitchell]

@SamVerschueren friendly bump :)

[SamVerschueren]

Thanks for the ping :). I totally missed the responses. I now moved tsd and tsd-typescript to the tsdjs organization.

[mrazauskas]

Quick update, since tsd-lite was mentioned here. I am about to deprecate it, see mrazauskas/tsd-lite#364

It was interesting to play with it. I learned a lot, but some limitations were inherited from the architecture of tsd. As I mentioned, another portion of limitations came from my idea of integration with Jest.

---

This brought me to TSTyche. It took two years to develop the code. (That is a hobby project, so writing a line or two in the evening was good enough speed.)

Repo: https://github.com/tstyche/tstyche
Documentation: https://tstyche.org/

Give it a try, please. TSTyche is published as beta to collect feedback. Stable release is coming out in a month or two.

PS Some people might be allergic to expect style assertions. I am open to implement assert style assertions as well. If some is interested, please open an issue. Let’s talk.