Minutes 2022 05 04

GPU Web meeting 2022-05-04/05 APAC-timed

Chair: KG

Scribe: KR

Location: Google Meet

Tentative agenda

• Administrivia
• CTS Update
• “Tacit Resolution” Queue
• GPUExternalTexture active_imports only works if there are RAFs #2785
• Does compositingAlphaMode affect using the canvas as an image source? #1847
• Specify that texture_external sampling functions clamp coords to [0, 1] #2766
• All the canvas issues!
  • Rely on canvas size instead of allowing size to be configured #2416
  • rgba8unorm canvas textures and IOSurface #2535
  • WebGPURenderingContext should support preserveDrawingBuffer: true #2743
    • related: Does compositingAlphaMode affect using the canvas as an image source? #1847 (comment)
  • Handling out-of-memory when resizing canvas backing store #2520
• Agenda for next meeting

Attendance

• Apple
  • Dan Glastonbury
  • Myles C. Maxfield
• Google
  • Gregg Tavares
  • Kai Ninomiya
  • Ken Russell
• Intel
  • Hao Li
  • Jiawei Shao
  • Jie Chen
  • Shaobo Yan
  • Yang Gu
  • Zhaoming Jiang
• Mozilla
  • Jim Blandy
  • Kelsey Gilbert
• Michael Shannon

Administrivia

• KR: WebGPU/Compat progress on behalf of Stephen White (senorblanco@)
  • Significant D3D11 progress recently (Dawn on ES3.1-ANGLE on D3D11)
  • Stephen's addressed about 70% of the WebGPU CTS failures on this backend
  • Hoping Stephen will join these calls going forward and offer progress and feedback w.r.t. what features are implementable/not implementable, to extend WebGPU's reach to cover the entire graphics + compute ecosystem

CTS Update

• KN: Not a lot to relay. DS stubbed out a bunch of WGSL tests that we’ll need, even if they’re not fleshed out right now. Should be easy to pick up if anyone can write the remainder of them!
• KN: Austin working on some refactors for perf (and me on some other refactors). Austin got up to 30% speedup by parallelizing subcases! - not waiting for popErrorScope between every subcase

“Tacit Resolution” Queue

• https://github.com/gpuweb/gpuweb/pull/2820
  • Declare 8? Or more?
  • KN: I agreed with 8 for a long time - that's why I was in favor of keeping it out of the limits. Perspective changing; I think we should add it as a limit now. Mark specific limits as constant values in the spec? e.g. impls must not expose a higher value for these limits.
  • Still want to do that - happy to do so for this limit - but think it's a good thing for us to expose. Have other limits equally justified to not change. Have some alignments which we could lift but don't seem important to.
  • Think we should put it in here, and mark some things as being constants.
  • KG: differences from uniform issue - don't expect this to ever change. Won't fight too hard about adding things to the spec, but think it's materially different from others. It's been 8 for ~20 years.
  • KN: I'm not familiar with the ecosystem - thought some APIs allowed this to be >8.
  • KG: I'm fine with this.
  • MM: no opinion.
  • KG: will merge.
  • KN: on Vulkan no device goes over 8, fwiw.
  • MM: are there devices that return under 8?
  • KN: a small number.
  • KG: probably won't ship on those.
  • KN: most are weird configs like SwiftShader or D3D12 (?).
  • KG: we'll merge this. Like having the idea written down that these are constant and won't change.
  • KN: group would have to agree for them to change before making the change.
  • KN: I'll make a note on the issue about 8 being the real-life limit.
  • MS: Raspberry Pi only reports 4, fwiw.
• https://github.com/gpuweb/gpuweb/pull/2750
  • KN: need for marshaling in/out
  • Gets rid of polymorphism. Single ErrorType enum. Can subclass later if we want.
  • KG: I don't mind but the TAG might. Moving away from the way the web works, where things are instanceof.
  • KN: I think we have a really strong reason for exposing an ErrorType, even if we have an inheritance hierarchy.
  • KG: diff between this and serializing based on Object.constructor.name?
  • KN: it's more straightforward. You can do that, feels more like a hack - maybe not really.
  • MM: idea's your not serializing when trying to turn JS object into C++ object.
  • KN: think it works in both cases. Marshaling into C - take ErrorType string. Put into C heap. Could also expose name of constructor function.
  • MM: if calling code has to switch on a type, or switch on Object.constructor.name, those seem the same to me. Don't see benefit of one vs. the other. One more JavaScript-y does hold for me, think it breaks the tie.
  • KN: not super clear cut. Error types are related. Both come out of the API.
  • KN: DOMException is old, but this is closer to the DOMExecption API. Type attached to each error.
  • KG: timebox. Can revisit as more formal discussion another time.
• https://github.com/gpuweb/gpuweb/issues/244
  • KG: anyone opposed?
  • MM: Domenic appears to be.
  • KG: take out of tacit resolution then.

GPUExternalTexture active_imports only works if there are RAFs #2785

• SY: Kai had inputs that this might not be a problem.
• SY: current spec fully covers background tabs.
• KN: not an expert - tried to read it. My understanding - to expire ExternalTextures we insert a step in the event loop, in "update the rendering". From reading the spec - that step's executed regardless of whether tab's in foreground/background. Steps that e.g. trigger rAF callbacks - individually responsible for throttling BG tabs.
• KN: for this step we're adding - think it should execute regardless of the state of the tab.
• KN: if BG tabs are idle, can pause this loop entirely. Think that's OK - if page not running JS, page can't see whether we expired the texture. No problem if we do it earlier/later.
• KN: spec as written I think is OK. No issue with ExternalTextures staying alive when tab's in BG.
• KG: any problem with reclaiming VideoFrames for decoders?
• KN: my conclusion - we will be able to take them away promptly. Nothing preventing us from doing that. Either event loop will be running, and our spec text will normatively expire the texture - or page is idle, event loop not running, and people can't tell the difference between expiring the video frame now or later.
• KG: think it might be observable - we throttle BG setTimeouts more than rAF.
• KN: I think if anything's happening on the page, the event loop will keep rendering, and "update the rendering".
• Discussion about how the algorithm works.
• KN: can point to the part of the HTML spec. Complicated to read.
• KG: ours says this applies regardless of document.
• KN: think browsers have leeway to free these imported textures as they need to.
• KG: so we agree on what we want it to do?
• MM: can you state what we want?
• KN: we want - find well-defined point, portable across browsers, where the frames will get expired. Want the browser to always be in control to expire textures within a few frames of being decoded. Pool of 4 decoded frames - expire them before running out of space in that pool.
• MM: I can get on board with that. Next step - if Kai's reading is correct, spec currently supports that goal. So is next step - Kai asks someone whether his claim is true?
• KG: who to ask?
• KN: Domenic? I'll reach out.

Does compositingAlphaMode affect using the canvas as an image source? #1847

• SY: this implies we have two buffers - getCurrentTexture, write something into it - but compositingMode's applied to drawing buffer on the way to the screen.
• SY: need to get a snapshot from it. We tend to get the drawing buffer. Heavier weight than 2D canvas. Have statements in the spec?
• SY: for example, compositingAlphaMode=opaque, drawImage into 2D canvas. Might get alpha channel when they didn't expect it.
• KN: reason we called it compositingAlphaMode was to make it clearer. Maybe compositing isn't the right word. Make it clear it's not a property of the canvas/rendering/backing store. Just how it's used later.
• KN: for me, question is - what performs well, what's compatible with implementations? HTML spec not helpful. Only has 2D canvas, and 2D canvas is "funny" when it comes to rendering - all immediate mode, pretends everything happens on the CPU - partly because the spec's old.
• KN: now you have a backbuffer. WebGL has a backbuffer/front buffer. Long, slow frame rendering while OS display compositor's running, don't' want to show intermediate results. Need front+back buffer. In 2D canvas, too. Spec doesn't describe it that way. Hard to understand how things should work. In practice, need front+back buffer for 2D canvas too.
• KN: think we should say this is different than what's displayed. Readback APIs should contain the alpha channel and not the "opaque" one. JS APIs don't access what's displayed on the screen.
• SY: also my points. Just think the spec needs to make this clearer.
• KG: isn't this a problem? compositingAlphaMode, is this something that happens post-process? You can change it at the end of the frame? Or would we patch pipelines?
• KN: could conceptually be done at the end, yes. Shouldn't have to patch anything. Want to avoid doing alpha clear on some systems that would require oen.
• KG: that's where it ends up mattering
• KN: we were designing things to avoid the alpha clear. SHould get to those at some point.
• KG: if not getting to those - say compositingAlphaMode doesn't apply to anything, display purposes only.
• MM: makes sense to me - dont' think drawImage should consult this value.
• KN: good point. makes sense for drawImage to get same result as copyTextureToTexture inside WebGPU.
• KG: think we should cut off here.
• KN: makes sense.

Specify that texture_external sampling functions clamp coords to [0, 1] #2766

• KG: I still don't understand how this is different from what should happen by default.
• KG: if it'd be clearer to say this is clamped to [0,1] - that's fine - but think we only need a non-normative node.
• KG: or is this about repeat?
• KN: problem is - we're getting a sampler config from user-provided GPUSampler object. Want to use that sampler to sample backing textures. If we do that - we'll get incorrect results for wrap modes. This avoids that problem - there's no difference between wrapping mode.
• MM: external textures can be cropped though. Decoder could be decoding in 8x8 blocks. Could have extra black pixels - don't think you can use the sampler in the first place.
• KN: can't use sampler's wrap modes - that's the problem here. BUt can use its filter modes. Maybe.
• KG: can we rephrase - external sampling functions always use Clamp mode?
• KN: yes.
• KG: I'm fine with that.
• KN: I do prefer to phrase it in the way this issue does, but they should have the same effect.
• KG: I'd prefer - it behaves as <spec text we've already written>.
• KN: don't think it's simpler. Have to say - take currently bound sampler object, modify in this way, then use it as the sampling. Both fine though. No strong feeling.
• KG: let's let the note stand.
• KN: option (3) in Corenin's comment - stop taking user's sampler, and put filter modes in the texture. Have it be combined texture/sampler.
• KG: let's discuss more offline.
• MM: we'd like to talk about it more offline too.

All the canvas issues!

Rely on canvas size instead of allowing size to be configured #2416

• SY: concern about size mismatches causing performance problems.
• KN: don't think it'll affect performance. Perf-wise, this scaling will be the same as in CSS - don't think we're adding another blit that needs to happen.
• SY: I see.
• KN: if that's not true it's concerning though.
• SY: if this isn't a concern I don't think there are more concerns from our side.

rgba8unorm canvas textures and IOSurface #2535

• KN: main thing I'd like to agree on - framebuffer-only option on Metal doesn't work for us. If you use framebuffer-only, and can't readback - no way to implement printing, Save Image As… etc. Shouldn't concern ourselves with it.
• MM: think that's almost right. Can't snapshot a WebGLcanvas - think that's a reasonable option. There's other evidence that supports your conclusion - toDataUrl doesn't work - that's a part of the web platform, breaking that is scarier. We have snapshotting code in our browser that doesn't work on a bunch of different elements.
• KN: giving the page an option that breaks it - saying it will work if you don't use this option and vice versa - seems fine to me. No compat issues.
• KN: concerning if we give the page an option to take agency away from the user, prevent them from copying/printing.
• MM: Under no situation will File>Print be grayed out. Might get a gray box instead of WebGPU canvas, on printed page.
• KG: prefer that not happen. Maps applications in particular. Fastest way for them to have smooth animation is to flip switches like this - would break printing. Like to not shepherd folks who care about performance into a path that breaks things like printing.
• KN: think it's a UA decision. If we want to leave things open, could pursue an option like that in the future. But not in line with how UAs work. Was hoping we could say, this doesn't work for us.
• MM: I don't disagree with this - maybe some of the reasoning, but not the conclusion. Also think that's a tangent on this issue.
• KN: it is, but it's a compounding factor. "framebuffer-only" makes this more complicated.
• MM: was suggesting that as a value-add, but not the bulk of my argument. Important that we can use CADrawables rather than IOSurfaces into contents of layers.
• KN: we agreed on that.
• MM: hard argument to make - can't tell you the technical reason why.
• KN: can take it as a platform constraint, don't need to know why.
• MM: other thing - the default probably should be that printing works. In the future, can add something that can only be displayed to user, make it fast - as an opt-in.
• KN: think that works.
• MM: also don't know perf benefit of framebuffer-only, yet at least.
• KG: want to offer users the choice - when I load/reload a page, will I be able to print everything on the page? Or do I just want it to be fast?
• MM: AAA games probably don't care about printing.
• KG: want the UA to be able to offer. Don't trust developers to preserve ability to print things.
• MM: even if we disagree on the reasoning, if we agree on the conclusion - no framebuffer-only - let's go forward.
• KG: we started with - rgba8unorm canvases/IOSurfaces - then convo morphed into framebuffer-only.
• MM: think this topic is a request to not require / allow using a BGRA texture to render into - instead, have mac/iOS code paths render into RGBA textures, because that'd eliminate diversity in the platforms targeting WebGPU, because Android requires RGBA. It is possible to write a progam that uses Metal to write into RGBA texture and have it show up on screen. So can we say all browsers must / may use RGBA textures to render into? Get rid of all complexity of BGRA/RGBA textures.
• KN: that's the most ideal. Think we've come to the conclusion that we're not going to be able to do that. Talked last week about preferred formats in more detail. Can't hide the RGBA/BGRA detail entirely.
• KG: stopping here, out of time.
• KN: think we're all in agreement on that part.

WebGPURenderingContext should support preserveDrawingBuffer: true #2743

• related: Does compositingAlphaMode affect using the canvas as an image source? #1847 (comment)

Handling out-of-memory when resizing canvas backing store #2520

Agenda for next meeting

• Non-APAC time slot next week.
• WGSL next week will be APAC-timed.
• Any new feedback on GPUAdapterInfo?