case_cache.md

Case Cache

The WebGPU CTS contains many tests that check that the results of an operation fall within limits defined by the WebGPU and WGSL specifications. The computation of these allowed limits can be very expensive to calculate, however the values do not vary by platform or device, and can be precomputed and reused for multiple CTS runs.

File cache

To speed up execution of the CTS, the CTS git repo holds holds pre-computed test cases, generated from *.cache.ts files and serialized in a set of binary files under src/resources/cache.

These files are regenerated by src/common/tools/gen_cache.ts which can be run with npx grunt run:generate-cache. This tool is automatically run by the various Grunt build commands.

As generating the cache is expensive (hence why we build it ahead of time!) the cache generation tool will only re-build the cache files it believes may be out of date. To determine which files it needs to rebuild, the tool calculates a hash of all the transitive source TypeScript files that are used to build the output, and compares this hash to the hash stored in src/resources/cache/hashes.json. Only those cache files with differing hashes are rebuilt.

Transitive imports easily grow, and these can cause unnecessary rebuilds of the cache. To help avoid unnecessary rebuilds, files that are known to not be used by the cache can be annotated with a MUST_NOT_BE_IMPORTED_BY_DATA_CACHE comment anywhere in the file. If a file with this comment is transitively imported by a .cache.ts file, then the cache generation tool will error with a trace of the imports from the .cache.ts file to the file with this comment.

The cache files are copied from src/resources/cache to the resources/cache subdirectory of the out and out-node build directories, so the runner can load these cache files.

The GitHub presubmit checks will error if the cache files or hashes.json need updating.

In memory cache

If a cache file cannot be found, then the CaseCache will build the cases during CTS execution and store the results in an in-memory LRU cache.

Using the cache

To add test cases to the cache:

1. Create a new my_file.cache.ts file.

2. In that file, import makeCaseCache from 'case_cache.js';

import { makeCaseCache } from '../case_cache.js'; // your relative path may vary

3. Declare an exported global variable with the name d, assigned with the return value of makeCaseCache():

export const d = makeCaseCache('unique/path/of/your/cache/file', {
  // Declare any number of fields that build the test cases
  name_of_your_case: () => {
    return fullI32Range().map(e => { // example case builder
      return { input: i32(e), expected: i32(-e) };
    });
  },
});

4. To use the cached cases in a my_file.spec.ts file, import d from my_file.cache.js, and use d.get();

import { d } from './my_file.cache.js';

const cases = await d.get('name_of_your_case');
// cases will either be loaded from the cache file, loaded from the in-memory
// LRU, or built on the fly.

5. Run npx grunt run generate-cache to generate the new cache file.