Minutes 2021 04 19

GPU Web 2021-04-19

Chair: Corentin

Scribe: Ken, Corentin

Location: Google Meet

Tentative agenda

• Need a proposal for specialization constants on the API side. #1608
• Specify what happens to mappings when the buffer is GCed. #1544
• Removing the need "non-filtering" samplers #1562
• GPUCompilationMessage: Specify linePos units, consider span #1597
• (stretch) Make lost devices appear to function as much as possible #1629
• (stretch) Should "mark adapters stale" be called more quickly? #1630
• PR burndown
  • copyExternalImageToTexture for VideoElement and canvases #1581
  • Specify WGSL as UTF-16-encoded text. #1626
  • Add GPURequestAdapterOptions.allowSoftware #1634
• Agenda for next meeting

Attendance

• Apple
  • Myles C. Maxfield
• Google
  • Alan Baker
  • Brandon Jones
  • Corentin Wallez
  • James Darpinian
  • Ken Russell
  • Shrek Shao
• Intel
  • Yunchao He
• Kings Distributed Systems
  • Hamada Gasmallah
• Microsoft
  • Rafael Cintron
• Mozilla
  • Dzmitry Malyshau
  • Jeff Gilbert
• Dominic Cerisano
• Francois Guthmann
• Michael Shannon
• Mehmet Oguz Derin
• Rick Battagline
• Timo de Kort

Administrative stuff

• CW: Agreed FPWD, so we will have to send an email to the list to offer an opportunity for feedback there, not just in-meeting.

Need a proposal for specialization constants on the API side. #1608

• CW: Thanks Alan for putting the specialization constant proposal.
• DM: What do you think is the downside of not specifying the type?
• AB: You're forced to always use reflection to know the size of what you need to provide in native.
• DM: To do it at pipeline validation, then you need to validate anyway.
• AB: Wouldn't want this to be the only thing that needs to give reflection.
• MM: Unifying float, sint, uint makes sense for a JS API. Otherwise just repeated information.
• <discussion about why Dawn would need to run without validation on, not a design constraint for WebGPU>
• MM: Is there a reason why sequence<ShaderConstant> and not a map? So there is no issue with duplicate bindings.
• AB: think it's fine to change that
• MM: counter-argument: maybe keys need to be extensible in the future? Dictionary full of stuff rather than string?
• KN: keys here are ints, not strings.
• AB: should be keyed on ID of constant.
• MM: variable names have to be distinct because they're globals. Wish we could use variable name rather than assigning numbers.
• DM: you can have different shader modules and they can have different IDs.
• CW: this is on programmable shader stage; only single modules.
• AB: right now WGSL defines constant ID. Probably what should be used.
• MM: we identify variables by number in other places. Only slight preference in this case. Identifying by name makes sense.
• CW: other vars we identify by numbers are those in limited spaces - textures, render targets, etc. This is pure compilation, so unbounded.
• DM: are there limits on spec constants in Metal or Vulkan?
• MM: not aware of any
• AB: don't think so
• CW: we could have it - decoration, this is a spec constant, you can have a string here. Spec constant, by default uses name of variable, and if you want to pass an argument as a string in there you could. But no precedent for string decorations in WGSL. Questions about how to encode special chars, etc.
• AB: also if you're writing other shader modules, you might name that variable in a different way. Always using the spec constant for those purposes?
• JG: think that's a narrow use case.
• DM: if we use names, any problems generating SPIR-V early? Would impl assign IDs internally, have to remap at pipeline creation time?
• CW: yes, would remap strings to IDs.
• DM: how to avoid collisions?
• AB: your spec constants are per-entry-point. In each module, can decide what the spec constants are.
• DM: no concerns about SPIR-V generation then?
• AB: not until we want to start allowing linking modules together.
• JG: can I set a spec constant, have that setting apply to more than one module?
• AB: not under this proposal, unless you have a default value in which case you don't need to specify it via the API.
• JG: as long as we don't have to align between shader modules, we can randomly assign IDs to them.
• KN: one alternative: on topic of linking, if we want to have some APIs for linking: we could do linking/specialization as a step. Here are shader modules, here are spec constants, etc., link them together, give me a new GPUShaderModule.
• MM: yes, that's like one of the proposals I made about supplying PipelineLayout. No strong opinions here, but if we do allow authors to specify PipelineLayouts in this flow, then we should use the same mechanism for both.
• AB: if you want to allow it early then we should decide where you want the statically-accessed rules to apply. Mostly applies to Vulkan spec constants.
• MM: in Vk you have to supply all of them even if they're not statically-used?
• AB: the rules apply earlier. Even if they'd be optimized away they have to be bound earlier in the pipeline.
• CW: 3 questions:
  • Do we need the type on GPUShaderConstant?
  • Is it a sequence? A map?
  • How do we index / reference / identify a spec constant?
• DM: idea was since it's not a limited resource, could just expose by names? Seems OK to me.
• MM: if we add namespaces and these can be in namespaces then we'd need some idea of fully-qualified name. I think this is a good general direction, but adding this feature is a good thing.
• CW: should we have it as a map of DOMString to GPUShaderConstant? Still TBD whether GPUShaderConstant has a type or not?
  • KN: record<USVString, GPUShaderConstant>
• KN: think we can agree to not have the type.
• MM: OK, sounds like way forward.
• KN: we'd define conversions from double to all types. Share conversions with e.g. clearColor which need them.
• AB: just need WGSL change as well.
• KN: yes, to identify by name.
• MM: let's preliminarily go in this direction.
• AI: JG: put this on WGSL agenda.

Specify what happens to mappings when the buffer is GCed. #1544

• CW: discussed in the past. Nothing in the spec. What happens with the ArrayBuffer when the buffer is GC'd? Can the GPUBuffer be collected when the mapping is active?
• JG: understanding. Would be much more limited if we were in the world 3 years ago. But now facilities exist to tell you when GC happens.
• KN: this is for limited use cases.
• KR: Run into this repeatedly in WebGL. Bunch of objects and bunch of objects bound into context. Had to say bindings are all strong refs, add spec and tests, ensure they’re not GCed. Not sure what the reachability rules should be for WebGPU.
• KN: Mainly a problem in WebGL because you could query those objects back, right?
• BJ: If you get a pipeline layout out of a pipeline is it compatible or same object?
• CW: New object every time but compatible.
• KR: If you have a bindgroup and have the layout GCed as well, then you can have issue.
• KN: The layout is referenced internally but never exposed so no worries about expando properties.
• MM: This is how it currently works in WebKit: ownership goes from the leaves to the root. So the mapping refs the GPUBuffer because it is a leaf object.
• CW: So are we ok with the mapping keeping the GPUBuffer alive even if it is non-observable?
• KR: In WebGL, different object model, we had to say that the object graph was rooted at the context level. Need to be sure about the semantic.
• BJ: If you map a buffer and lose access to the buffer handle. Then there is no way to unmap it. So is there sense to keep the buffer alive then?
• MM: The buffer can get unmapped in a non-observable way.
• BJ: In some implementations the ArrayBuffer could stay alive even if the buffer is deleted.
• KN: The idea is that GCing the buffer doesn't change the ArrayBuffer state.
• MM: Makes sense.
• KR: In some implementations you might need a secret field to keep the GPUBuffer alive. Turned out to be a massive perf issue in Chrome's WebGL implementation, had to reimplement, need to be careful with it.
• CW: let's investigate more. AI: chromium team: look into what it'll take.
• KN: just need to make sure the ArrayBuffer's backing store doesn't go away.

Removing the need "non-filtering" samplers #1562

• CW: idea: right now have 3 types of samples. Filtering, non-filtering, comparison. At pipeline creation, validation says based on pipeline layout, you can't have unfilterable float sampled texture used with filtering sampler. Complicated in compiler infrastructure. Define static use of textures with samplers.
• CW: strict subset of what spec currently allows. In WGSL, compiler would say: this texture's used with sampler or not. Unfilterable textures can't be used with samplers. Removes some flexibility from API, where sampler can be used to choose addressing mode for unfilterable float texture, but think it's probably OK.
• DM: we already implement this; it's OK. Generally, approach our group takes: expose things HW allows, safely, unless requires heavy runtime workaround. This isn't heavy, it's runtime pipeline creation. Go through texture/sampler pairs. Little bit of impl concern for Chrome & Apple.
• MM: if we restricted API in this way could we add back non-filtering samplers if we need them?
• CW: yes. essentially this proposal says there are no unfiltering samplers. If we add them back, spec becomes what it is now.
• MM: sounds fine either way then. Cost/benefit analysis.
• DM: could you take AI to see how much work it'll be for you?
• CW: yes, will take that AI and give it to Ben. :)
• MM: because textures/samplers can't be passed by pointer, each function has to name the texture it receives? True for both samplers and textures?
• DM: it's true today.
• CW: do we see a future where textures will be passed by pointer?
• KR: bindless?
• MM: we'd like to get there but there's a long way from here to there
• DM: then you can't bind 32-bit float textures as sampled textures.
• MM: unless people ask for it, then we add it back.
• KR: concern about this functionality already being in WebGL and people using it.
• KN: if float32 became filterable, then we wouldn't be using it with non-filtering sampler, but filtering samper. Wouldn't need to add this to give that optional functionality. Textures bound as non-filterable float, can't use them with samplers at all. If 32-bit float textures become samplable, then you bind them as samplable float (?). Still don't need non-filtering sampler type.
• DM: not eager to make this change because we already have this working as specified. One thing to cut out features we don't want to spec/implement; this is cutting out one that's already spec'd/implemented.
• CW: I'll ask for clarity for what the impl cost is. Tentatively, we keep it? Unless it implies something major on our impl.
• CW: maybe we'll just investigate, and close out this issue.
• KN: think the reason this isn't in WebGL is that texelFetch came later. But for compatibility, think we should probably keep it in WebGPU.

GPUCompilationMessage: Specify linePos units, consider span #1597

• BJ: maybe addressed this in this group previously?
• BJ: concern: should be able to identify spans of shader code corresponding to messages that come back from the compiler. Non-controversial, I think. Most of tooling can identify spans.
• BJ: more interesting q: how to denote? Kai gave interesting options. Two (line,position) pairs? Actual offsets into string? In all cases, what are the units used? Code points? If lines, then do \n \r count? (Windows / Linux line breaks) etc.
• BJ: even if we don't want to make these spans we should talk about what these offsets mean.
• JG: we want people to be able to do source.slice() and get back that segment.
• KN: for lines/columns: do we want people to be able to do source.slice()? then should be positions instead of lines/positions.
• JG: mixed feelings. Most useful thing in any warning is the line number and not the position.
• KN: can display line numbers somewhere developer-visible, like the console.
• JG: part of idea was for tooling.
• KN: think tooling should be responsible for telling people the line number.
• BJ: for tooling, would be nice in Shadertoy to have list of errors, jump to line in editor space. Not impossible to reverse engineer line out of offset, but a pain. Having line number, even if isn't absolute value, has benefit. Also benefit to doing slices.
• JG: fair distinction between what we print to developers, former wants line number, latter wants to synthesize.
• MM: do both?
• BJ: not opposed.
• JG / CW: let's do both.
• JG: source.slice(), whatever those units are?
• MM: yes, but I have some thoughts about that. Let's discuss in #1626. (UTF-16 vs. UTF-8)
• KN: useful background in this issue in #1597. Iterating string: you get code points. slice / substring / etc: you get UTF-16 code units. Iterating is newer, code points make more sense than code units. Previously said didn't know at JS level that things are encoded as UTF-16: this is where you can tell the difference.
• JG: this is what everyone does.
• MM: definitely shouldn't use code points because they aren't random access. Have to iterate up to code point #300 to be able to use it.
• CW: can we have a simple resolution?
• JG: we'll have both line numbers and start position. end position too? Units are things that operate on slice(), so code units, not code points.
• MM: can we say there's also an end point too? Want to identify particular point, use a single one.
• CW: inclusive or exclusive end?
• KN: slice uses exclusive.
• MM: match substring?
• CW: ok, start and length?
• KN: has benefit - if you have line & start pos, would need end line/end position, or length. length is simpler.

PR burndown

• copyExternalImageToTexture for VideoElement and canvases #1581
  • CW: Kai / Dzmitry, there are lots of discussions here. What portions do we need this group's input on? Lots of concerns around the name.
  • KN: can discuss the name. Discussion with Shaobo, don't think we need to discuss here. What about the verb? Argument is that unlike our other copies it's not a byte-by-byte copy.
  • DM: there's a requirement for render attachment. Once we have that it isn't a
  • KR: Don’t want to make optional features. But not sure that requiring the destination texture to have RENDER_ATTACHMENT is ideal.
  • JG: Effectively true in WebGL, since we use it as a framebuffer object attachment.
  • KR: Not all paths in WebGL do this. Decode-then-copy, or even paths that use CPU upload.
  • CW: based on DM's and KR's feedback think it's OK to have either copy or render attachment capability.
  • DM: why do we want that? Adding another usage isn't a big deal. Won't impose additional performance pitfalls.
  • CW: render attachment has some cost I think. Allocates more planes, has bigger alignment constraints.
  • DM: not a big cost.
  • MM: extra cost will be on the website, right? Not in the browser? Better for the browser to do it.
  • Discussion.
  • DM: there isn't a runtime cost for render attachment. Maybe alignment, etc. Order of magnitude less than original texture allocation. No performance cost.
  • MM: concerned about asking devs to add render attachment flag to all textures.
  • CW: would like to maybe second that. Could see how app would have render attachment on every texture because they initialize it from HTMLImageElement.
  • DM: maybe they shouldn't init from HTMLImageElements. Not the practice we want to encourage.
  • BJ: note that easy path for generating mipmaps would also make them add render attachment usage to their textures.
  • KR: if there isn't a runtime perf cost - like render attachment textures are all linear layout and can't be sampled effectively - then no worries, let's add the validation requirement.
  • CW: OK, current plan - add render attachment requirement & validation rule for now.
• Add GPURequestAdapterOptions.allowSoftware #1634
  • MM: ship it
  • CW: OK, great.
  • JG: what we agreed on last week?
  • CW: yes, it is.

Agenda for next meeting

• Discussion on F2F?
  • JG: schedule, then cancel if we don't want it?
  • CW: verdict was - ask the WGSL group. API think is mostly under control, so don't need F2F - but if there is one we could tag along. Mostly WGSL needs intensive discussion.
  • JG: we'll see what we have left to do.
  • CW: can discuss, maybe have some stretch issues / PRs to discuss.
• CW: feel good - discussions are converging!