Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is it possible to add snapshots in type testing? #5857

Open
4 tasks done
MWhite-22 opened this issue Jun 7, 2024 · 12 comments · May be fixed by #6963
Open
4 tasks done

Is it possible to add snapshots in type testing? #5857

MWhite-22 opened this issue Jun 7, 2024 · 12 comments · May be fixed by #6963
Labels
p2-to-be-discussed Enhancement under consideration (priority)

Comments

@MWhite-22
Copy link

Clear and concise description of the problem

Here's my scenario:

  • Developing components in a component library
  • Some of the prop signatures for the more complicated components are getting huge as they union with other prop types
  • We want an easy to parse and visually identifiable way of tracking those prop signatures and see what may have changed in a PR

Suggested solution

So I was thinking, some type of type signature snapshot. Similar to the normal .toMatchSnapshot (or inline), but essentially we could pass in MyComponentProps and get out the whole:

type MyComponentProps = {
  variant: 'A' | 'B' | 'C';
  label: string;
  someOtherProp: any;
/// ...etc
}

It's essentially what IDE's are doing on hover when I want to inspect a type signature, but we inject it as a string snapshot for tracking and validating against unwanted changes.

Alternative

Open to any and all suggestions on how this might be possible. I have not found the right set of words to put in to google to find something that does this exact thing.

Additional context

No response

Validations

@MWhite-22
Copy link
Author

I've been looking through these tsc compiler API docs and I feel like this should be possible, though likely will require a lot of effort to be performant.

https://github.com/Microsoft/TypeScript/wiki/Using-the-Compiler-API

@mrazauskas
Copy link

Can I ask how do you write type tests for components? I am simply curious to see real world example, but that could also be helpful for someone else who is interested to implement this feature.

@sheremet-va
Copy link
Member

sheremet-va commented Jun 23, 2024

I would also like to know the usecase so we can discuss this with the team. While it sounds useful, I am not sure it is useful.

Is it just to not bother writing the names yourself and let Vitest generate the string? (Which is a completely valid use case, current runtime snapshots are basically used because of this)

@tmm
Copy link

tmm commented Jun 25, 2024

Maybe unrelated, but snapshotting compiler-related information (e.g. completions, instantiations, compiler errors) would be useful for libraries that perform a lot of type-level programming. For example, snapshotting compiler errors allows for more granular test safety than @ts-expect-error, which silences errors you might not want to.

import { expectTypeOf, test } from 'vitest'
import { mount } from './mount.js'

test('error', () => {
  // @ts-expect-error 
  expectTypeOf(mount({ name: 42 })).toHaveErrorMatchingInlineSnapshot(`Type 'number' is not assignable to type 'string'.`)
})

Also, think letting Vitest generate the strings is useful to save time for complex types!

@mrazauskas
Copy link

@tmm Looking at the example you provided, I think a matcher like .not.toBeCallable() or .not.toBeAssignable() would be better idea. It would make it more clear what is the intent of this test.

Similar for JSX components. I think a dedicated .toAcceptProps() matcher is a be better idea than a snapshot.

@sheremet-va
Copy link
Member

I think this is more of a philosophical question. Any snapshot could've been an assertion (and probably should've), but it's easier to have an inline snapshot generated and forget about this.

@sheremet-va sheremet-va added p2-to-be-discussed Enhancement under consideration (priority) and removed enhancement: pending triage labels Jul 1, 2024
@sheremet-va sheremet-va moved this to P2 - 2 in Team Board Jul 1, 2024
@MWhite-22
Copy link
Author

@sheremet-va Our use case is two-fold:

  1. Component Library

We have a component library that has a lot of base components that then all build on top of each other into aggregated molecule components.

For example, a simple dialog, from which an interactive dropdown is built (extending its props), which is then used in the interactiveNavDropdown (again, inheriting sub-component props). The final interactiveNavDropdown props combine all the sub-component props, a few of the sub-component props with a few of its own, so its prop signature is very small from a code standpoint, but pretty big from an actual size standpoint.

type NavDropdownProps = {
  //...custom navDropdown props in here
} & DropdownProps;

// Dropdown.tsx

export type DropdownProps = {
  //... the dropdown's props
} & GenericDialogProps;

// Dialog.tsx
export type GenericDialogProps = {
  //... the dialog's props
} & React.ComponentPropsWithoutRef<'dialog'> // <- all HTML valid props

Having the inline type snapshot means that I can see when NavDropdownProps changes due to any changes in the sub-component props leading in to it. It also means I don't have to write out EVERY HTML PROP in some giant object to keep track.

  1. Zod Schemas

Imagine the same as above but we have a giant multi-part form for our application. Each part, and many of the sub-parts, have their own zod schemas for validation that all get rolled together for the final API shape via z.infer. It would be really nice to pass the big shape to a type snapshot and not have to write out the whole thing.

@MWhite-22
Copy link
Author

You're probably right these could all be tested by actually writing out the individual type shapes were attempting to test against. However, in the interest of updateability, and frankly convenience too, being able to snapshot quickly would be a godsend.

@sheremet-va sheremet-va moved this from P2 - 2 to Discussing in Team Board Jul 18, 2024
@remorses
Copy link

Attest already has support for type snapshots, it would be cool to integrate it directly into Vitest

https://github.com/arktypeio/arktype/tree/main/ark/attest#readme

@sheremet-va
Copy link
Member

I wonder if it's possible to just provide attest in Vitest out of the box 🤔

@ssalbdivad
Copy link

@sheremet-va It would definitely be possible!

Based on some feedback from @danielroe, I made another pass at attest's type serialization format + integration a while back that I think should accommodate a lot of external integration quite easily.

Would be happy to meet some time and discuss what could be done on both ends to enable this.

@sheremet-va
Copy link
Member

@ssalbdivad hey! We've been thinking about switching to runtime TypeScript checks instead of doing static analysis with expectTypeOf - although it also has its uses, it can be used with a single tsx command and doesn't require the whole test runner. Using attest in regular tests will also make internals much smaller and easier to maintain.

You are welcome to join the Discord. Message me directly, and I will assign you the ecosystem role so we can start a thread discussing this 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
p2-to-be-discussed Enhancement under consideration (priority)
Projects
Status: Discussing
6 participants