Skip to content

Minutes 2020 12 21

Jump to bottom
Corentin Wallez edited this page Jan 4, 2021 · 1 revision

GPU Web 2020-12-21

Chair: Corentin

Scribe: Ken

Location: Google Meet

Tentative agenda

Attendance

(seems small, did some people not get noted?)

  • Apple
    • Dean Jackson
  • Google
    • Corentin Wallez
    • Kai Ninomiya
    • Ken Russell
  • Mozilla
    • Dzmitry Malyshau
  • François Daoust

WebGPU/WGSL feature prioritization

  • Canvas Interop:
    • DM: Can be done with ImageBitmap so not super necessary.
    • DJ: Would be nice to have as a simple thing to do.
    • KR: Have been convinced by colleagues that making everything going through ImageBitmap makes it really hard to eliminate copies.
    • KN: ImageBitmap makes more sense for canvas than other things because it the concepts are closer. Less issues than video. There's concern that ImageBitmap isn't widely implemented yet too.
      • KR: Safari's ImageBitmap implementation is complete but not efficient yet. Addresses some of the concerns. Firefox’s still needs support for some of the createImageBitmap optional arguments.
    • CW: one use case is rendering 2D APIs to a canvas every frame and then uploading to WebGPU.
    • DM: I’m not advocating for using createImageBitmap for the texture upload path - realize it won’t be efficient. I’m just suggesting that users already have a workaround for this, so is this really P0 for V0 of the API? Concerned about putting a lot of work on Firefox’s plate.
      • CW: in that sense, I recognize this might be less important than other features we can’t do without. Just an optimization, not a capability of the API.
      • KN: that assumes FF keeps / implements the path through ImageBitmap. If FF didn’t implement that, it’d be easier to just implement it for Canvas.
      • DM: that part I’m not aware of. Would be great to have JG here for that part. If we have an API here, make sure it’s easy to polyfill on top of ImageBitmap if we want to?
      • KR: createImageBitmap is async, so not clear the synchronous API can be polyfilled on top of it.
      • CW: let’s make this P1? and look into it more later.
  • WebCodecs
    • KN: Since this is mainly being prototyped by Chromium folks, we can leave this as is.
    • CW: think this can be “other” since this isn’t any other browser’s focus. Semantically, more a part of WebCodecs than WebGPU.
    • KN: makes sense.
  • Interop with HTMLVideoElement
    • DJ: falls into same bucket as HTMLCanvasElement, but harder to do in some way. 3 main things (canvas, image, video) that are parameters in WebGL. Consider them all the same type of feature?
    • CW: probably different. Experience with WebGL - you need specific things for each to optimize perf. Esp. video, since it’s a copy you incur every frame. Sort of similar with Canvas, too. Generic API probably doesn’t work to have best perf for video.
    • KR: I’d advocate for doing Video of any of those three. That’s the one that WebKit, for example, has GPU acceleration for.
    • KN: if we have ImageBitmap we probably don’t need direct support for HTMLImageElement. It’s less general - can’t be used on workers - and I think the main use case will be loading resources on workers.
    • CW: also Image doesn’t need as much optimization, only need to upload it once.
    • CW: agreement that Video is the most important of all of them. Worry it’s a lot of work. But the HTMLVideoElement seems probably hard to specify. Initial investigations, probably need some sort of VideoFrameSampler object to optimize things.
    • KN: yes, for 0-copy.
    • CW: scary to make it V0 for that reason.
    • KN: copyFromVideoElementToGPUTexture could be V0. 0-copy can come a little later. I think that’s not as important as 1-copy.
    • KR: Compared to the ImageBitmap path, it's going to be very difficult to get that down from 2 copy to 1 copy. If we want we could specify it in a way that would allow a 0 copy would work, (lock / unlock), but internally it could be 1 copy.
    • KN: could be hard to spec everything you’ll need to achieve 0-copy in the future.
    • CW: how about separating this into 1-copy and 0-copy use cases? Do 1-copy more urgently.
  • Interpolation declarators / qualifiers
    • DM: why is this P1? Seems needed.
    • CW: out of all the things to do in WGSL, this seems least important, but seems necessary for V1 as well.
    • KN: I think this is a P0. So many graphics applications use there. They should be fairly straightforward - just pass through some metadata.
    • CW: I think there’s an issue with provoking vertex, or “flat” stuff.
    • KN: those are less critical I think.
    • DM: we have to support flat anyway. Might as well as have the keyword and say that “one of the vertices will support it”. Will reach most of the use cases anyway.
    • CW: OK, happy to have as a V0.
    • KN: there might be some we can push out, but this one needs to be V0.
  • Refined multi-worker semantics
    • CW: need to figure out aspects like timing attacks / high-precision timers.
    • DM: we put P1 there because we think it’s fine to ship V1 even if it doesn’t have multi-worker support. Can still do a lot with this API. Need to make sure we’re ready for the API to go into that territory.
    • KN: think we’ve had a lot of discussion. We know more or less that what we have now will be OK. Probably OK to downgrade this.
    • CW: would be nice to have in a V1 - one of WebGPU’s promises is that you can record in multiple threads. But can be a stretch goal for V1.
  • Device creation API tweaks
    • CW: KN opened ~12 PRs. P0, we’ll discuss soon.
  • Adapter API changes
    • KN: we’re in the middle of discussing is just the adapter stuff. Device stuff will come soon. Imp’t that we make some very small changes to it now.
    • CW: in the spreadsheet there are some tasks as opposed to API features.
  • FP64
    • DM: why is this so much needed?
    • CW: easy extension of the API that’s useful for scientific computations. Not crucial, but easy addition.
    • KN: good point, sci is something we haven’t thought about. ML is hot, but sci is an imp’t use case.
    • DM: when considering core features, there’s a cost to adding it later, because people will have to add divergent paths. But for optional features, there’s no point in rushing them in ASAP. Not losing anything by us shipping it later.
    • CW: for people interested in WebGPU would be good to tell them what optional features we intend to add soon. Targeting 1 yr after release, etc.
    • DM: clearly P2, then, at least. Here’s our roadmap.
    • CW: I think that’s right.
  • Exotic texture methods
    • DM: there’s a difference between gather and some of the other methods.
    • KN: we need texture size queries for storage textures anyway.
    • CW: don’t think we need to - all APIs guarantee that out-of-bound accesses of storage textures are well-behaved.
    • DM: this is storing to and loading from image.
    • DM: gather instructions are popular, seem good for P1. Others should be done separately.
    • KN: sounds good.
  • “Compatibility mode” for GLES31 / D3D11
    • CW: “Other”, but soon
    • DM: what do you mean by this? Do you suggest that the WG develops the spec for a compatibility mode? Or, when are we going to ship for various browsers? It’s out of scope for the WG.
    • KN: it’s not out of scope for the WG. It’d be a subset of the working group, but needs to be standardized.
    • DM: so would this be a subgroup of the WG?
    • CW: similar to what we’re doing with WGSL.
    • DM: WGSL, host API and CTS are main deliverables of the W3C WG.
    • CW: concern is, it’s not in the charter.
    • CW: let’s call it “other” and discuss admin aspects another time.
  • FP accuracy of builtins
    • CW: want to state for WGSL V1 the guarantees we can have. Memory / execution models, builtin accuracy.
    • DM: we expect the graphics programmer to write shaders, they use other shading languages without considering accuracy guarantees. We can specify this later. Not like we’ll change the sin function’s behavior.
    • CW: more for the spec to be complete.
    • KR: in WebGL for example we didn’t make an attempt to do this. The previous conformance tests used a Taylor series approximation to all of these primitives - got much better precision on all platforms by computing the reference image in JavaScript (and then another precision improvement from Olli Etuaho by improving the interpolation computation). Suggest defining this by the conformance tests.
    • KN: think we should aim to define things in terms of ulps of accuracy.
    • CW: OK, this sounds controversial, let’s add a question mark and discuss in a WGSL meeting.
  • Texture format reinterpretation
    • KN: think we have almost all the data we need for this. Potentially very useful with texture storage atomics, but that’s an optional feature. D3D12 also carves out a whole set of things that can be treated as R32 for this reason.
    • CW: OK, so V1.
  • Float literal hex notation
    • DJ: need input from Myles.
  • Implicit types for variables and constants
    • CW: quality of life improvement.
    • DM: JG argued for P0, but we can leave this as V1 for now.
  • CW: long tail of stuff. Which would you not want as “other”?
    • DM: maybe multi-queue?
    • CW: we can treat this as “contested” in a sense. Basically, comes after all the V0 / V1s because it’s contested.
    • DM: sounds reasonable. V1.5?
  • CW: I’ll put everything else as “Other” and if you disagree you can put it back to TBD.
  • DM: “Other” means we don’t think about this now?
  • CW: yes, and it’s not part of the list of things we track for V1. If we have time we’ll add more stuff to that list. Sure there’s some stuff here that some people want for V1. Can move those back to TBD.

RenderPipeline restructuring

TAG review

  • CW: we’ll have to make a spec that’s a W3C Recommendation; step toward this is TAG review. Looked at this with Kai, talked with former TAG reps at Google. Would help the TAG if we could produce some explainers in addition to the spec, which is really dry. Also doesn’t explain things in rational / contingent ways. Need to talk about certain aspects with the TAG. Not sure how explainers, etc. work for the TAG in other groups. Apple was very interested in TAG review multiple times in the past.
  • CW: how do we go about TAG review for a new language of the web? Only been one?
    • DJ: right, Wasm. Hope they admit their expertise in their response, or go out and get experts.
  • CW: earlier rather than later: we can iterate on it with them.
  • DJ: would expect they’d come back with comment “don’t do it this way” because they didn’t have the 2 years of discussion background. Educate them on our design decisions.
  • CW: everyone OK with writing explainers and filing for TAG review in coming months?
  • DM: yes. Earlier we start, the better.
  • KR: will the explainers be in the spec? Alongside?
  • KN: restate things in the spec, just in more digestible way.
  • CW: let’s file issues and split off the explainers. I can take buffer mapping, for example. If not enough progress, can discuss in the meeting. Also, for TAG, there are various questionnaires, like privacy. Can have on the wiki, or in the repo.
  • KR: security should be front and center.
  • KN: yes, that’s in the spec.
  • FD: also privacy group. Security and privacy. Good practice to start ASAP.
  • CW: what other reviews?
  • FD: accessibility. Don’t expect much from them. Internationalization too.

GPURenderPipeline restructure (#1307)

  • Not discussed

Agenda for next meeting

  • WGSL meeting not happening - no WGSL topics.
  • Adapter+device stuff?
  • RenderPipeline restructuring?
  • New PRs, etc.
Clone this wiki locally