Visual testing

[Newbie012]

A third-party jest matcher called toMatchImageSnapshot performs image comparison. It's primarily useful when you do visual tests. It would be pretty cool if vitest could support this kind of testing as one of its built-in features.

The lack of visual testing prevents me from migrating from jest to vitest. I believe many other developers who perform visual testing will be interested in this feature.

[sheremet-va]

This library should work with Vitest, no?

[Newbie012]

I haven't really tried. After inspecting the source code, it looks like it really should. I'll keep you posted :)

[hcampos-professional]

Agreed 👍. It would be great to support something similar to what Playwright has built in: https://playwright.dev/docs/test-snapshots

With Vitest we can take the screenshots but don't have a built in mechanism for comparing and asserting them. And even if we implement our own comparison (e.g. with pixelmatch or phash) there is no way (that I could find, at least) to display results in the Vitest UI. At the moment we're forced to build a custom reporter that produces a separate report that developers need to check separately.

It would be awesome if we could pass some metadata along with errors to let Vitest display Before, After and Diff, like Playwright does:

Vitest already has something similar, in the form of meta.failScreenshotPath:

vitest/packages/ui/client/components/views/ViewReport.vue

Lines 117 to 123 in 68e6c63

	const currentScreenshotUrl = computed(() => {
	const file = currentTask.value?.meta.failScreenshotPath
	// force refresh
	const t = timestamp.value
	// browser plugin using /, change this if base can be modified
	return file ? `/__screenshot-error?file=${encodeURIComponent(file)}&t=${t}` : undefined
	})

Alternatively (or in addition), Vitest could add support for rendering HTML error messages. That would let apps render before/after/diff directly in the UI. Probably a way of opting out of escaping HTML here:

vitest/packages/ui/client/components/views/ViewReport.vue

Lines 33 to 61 in 68e6c63

	function createHtmlError(filter: Convert, error: ErrorWithDiff) {
	let htmlError = ''
	if (error.message?.includes('\x1B')) {
	htmlError = `<b>${error.nameStr || error.name}</b>: ${filter.toHtml(
	escapeHtml(error.message),
	)}`
	}
	const startStrWithX1B = error.stackStr?.includes('\x1B')
	if (startStrWithX1B || error.stack?.includes('\x1B')) {
	if (htmlError.length > 0) {
	htmlError += filter.toHtml(
	escapeHtml((startStrWithX1B ? error.stackStr : error.stack) as string),
	)
	}
	else {
	htmlError = `<b>${error.nameStr || error.name}</b>: ${
	error.message
	}${filter.toHtml(
	escapeHtml((startStrWithX1B ? error.stackStr : error.stack) as string),
	)}`
	}
	}
	if (htmlError.length > 0) {
	return htmlError
	}
	return null
	}

[Shiwa]

I agree it would be pretty cool to do visual regression testing just using vitest built-ins.

I’m working on a project with lots of home made components and design that can be technically usable but visually broken (popovers, sticky toolbars, responsive layouts, etc.).
We already use chromatic for our design system and it’s awesome, and I’d really like to have the same kind of tooling in the app, without setting up extra libs.