Minutes 2023 04 19

GPU Web 2023-04-19 Atlantic

Note that unless stated otherwise this is a GPU for the Web community group meeting and not a working group meeting.

Chair: Corentin

Scribe: Ken

Location: Google Meet

Tentative agenda

• Administrivia
• CTS Update
• “Tacit Resolution” Queue
• requestAdapterInfo() should be required to prompt (in at least some circumstances) #3962
• Add rgb10a2uint texture format #3841
• Investigation: Import VideoFrame from WebCodec to WebGPU #1380
• support blending 32 bit float textures #3556
• depthWriteEnabled and depthCompare are required even for stencil-only formats #3905
• Agenda for next meeting

Attendance

• Apple
  • Mike Wyrzykowski
  • Myles C. Maxfield
• Google
  • Ben Clayton
  • Brandon Jones
  • Corentin Wallez
  • Kai Ninomiya
  • Ken Russell
  • Loko Kung
  • Stephen White
• Microsoft
  • Rafael Cintron
• Mozilla
  • Jim Blandy
  • Kelsey Gilbert
• Nvidia
  • Anders Leino

Administrivia

• KG: regarding Milestone 2 - on Moz's side we have a lot of work to do. Rather than talking about Milestone 2, we'd rather cancel meetings.
• CW: could work for us. One thing we'd like to champion for Milestone 2 is WebCodecs. Safari 16.4 has WebCodecs support and we think this is an important integration point. Want to get this out as an important web platform integration point.
  • KG: OK.
• CW: otherwise happy to defer Milestone 2 stuff a bit.

CTS Update

• KN: Chrome team’s work on CTS will continue: Ryan + Tint team on WGSL, Dawn team on API
• KN: Please contribute :)

“Tacit Resolution” Queue

• None

requestAdapterInfo() should be required to prompt (in at least some circumstances) #3962

• MM: Found this when thinking about what we should prompt for. This group discussed this issue last week?
• BJ: Mike suggested we revisit it this week to have MM's opinion. But after discussion, Apple's opinion that API as it exists is probably OK, given that we're taking out the hints. Accurate?
• MM: accurate. Were worried that if we didn't have resolution now, that no browser'd ever be able to prompt. But CHrome's throwing an exception from this API today if hints are passed anyway. Nobody likes prompts. Focusing mostly on promptless mitigations. If we really do need to prompt, bad if that option's taken away.
• BJ: clarification. Right now, you can call requestAdapterInfo in Chrome, and get info back. But if you try to pass unmasking hints, that's when you throw the exception. And we'll be taking that codepath out, to avoid setting developer expectations / behaviors. Chrome wants to remove unmaskHints entirely - don't have time to implement it properly at this time, and look at reintroducing down the line.
• MM: flexibility is the reason we opened this issue. Want to have the same flexibility you do.
• MM: is the expectation - if someone calls requestAdapterInfo and don't provide unmaskHints, are you going to give back an object with no strings inside?
• BJ: our intent - unmaskHints won't be an option. Want to remove them from the spec. If you call this API, it will give back an object and it will be populated with what the browser's comfortable returning. It can be nothing.
• MM: and not commenting on what you plan to return?
• CW: we can describe what's planned to ship. Right now - returns non-empty strings for vendor and architecture.
• BJ: yes. There are other fields we leave there empty, unless you've set a developer flag in about:flags. Including up to driver version, which isn't in the spec, just for developers, don't intend to expose to the web at large.
• KG: dream of how this works in Firefox - we'd never prompt for this API, ignore / remove unmaskHints from spec, return you sanitized description of device. In future, to add more info, if valuable - don't think it's strictly needed - we'd come back to this group and propose a new entry point that would prompt. So, reserve the ability to add something that's expected to prompt by adding a new API entry point.
• BJ: seems reasonable and in line with what Chromium wants. Returning sanitized data's perfectly fine. General flexibility of revisiting this later - in line with what Chromium wants to do, too.
• CW: hearing:
  • Remove unmaskHints
  • No prompting
  • Revisit later if need be
• BJ: agreement between Google/Moz. Slight mismatch between what MM thought Chrome was doing and what we are doing - addressed / OK?
• MM: don't think we should resolve today to remove unmaskHints. Not sure that was an option. Privacy team told us it's important for websites to ask for what they want. Like to defer that at least a week.
• MM: clarification about Chrome's shipping behavior - concerning because people will be calling this API and it will never prompt. Since you're only returning vendor/arch - that's helpful enough to avoid the privacy problems that I think that's OK.
• KG: personally don't want to implement vendor/arch, but have sanitized device. Lets me tune buckets more easily. Arch maybe too narrow, but sometimes too broad for what I want to tell you. That's why I tell you my aspiration. Always give you sanitized thing with no prompt. Vendor's easy, Arch not so much.
• BJ: data's not necessarily tiered. Viable to say, I'll fill in vendor/device. Maybe not your actual device, but something representing the device class. Skip arch entirely.
• MM: "AMD RTX 4090"
• BJ: yes. You could also plug in the architecture since the card has one associated. But you're not compelled to fill in the arch.
• KG: doesn't feel that way.
• BJ: not the intent of the spec.
• KN: dev expectation is that it'd be there. Chromium won't even fill in the device field.
• CW: though there's expectation of the formatting of the architecture, there's no specification of it.
• KG: in practice we really don't want to do that. Arguably, dev should know the range of things. Strings are fine to start. Agreed I don't have to impl full arch lookup thing, so no concerns there. Vendor's trivial.
• BJ: hope there wasn't an impression given that when you call this API that the unmaskHints would be required at any point. Haven't been considering that.
• MM: if someone calls this API and doesn't pass an input, they're asking for all the info they could get?
• BJ: they're asking for all the info they can get without a prompt. And spec text says, if you give unmaskHints, you may be subject to a prompt.
• KG: opinion - when you call this API in Firefox - we'll give you back 1 of 32 or fewer options. Paraphrasing - that's the one piece of info you're asking for when you call this API. Satisfying the requirement for getting back what you need. But you can't ask for just the vendor and get back one of 8 options.
• CW: in Chromium we'd like to consider giving less info to pages we trust less, too. (Third-party iframe for example.)
• BJ: supporting what Kelsey said - Chromium still falls under the 32-bucket limit.
• MM: nitpick - goal of bucketing was for all APIs together.
• KG: if that's not true in our browser - please file a bug.
• BJ: though this API might return various combinations, and limits another set of combinations - the union will still fall under that limit.
• KN: note, this is an interface. Handing back data - we haven't done that until they've accessed the attribute. Still an opportunity to see what info they use. In terms of privacy/tracking, there's more nuance there, and the chance to make late-binding decision of what you return for each attribute.

Add rgb10a2uint texture format #3841

• KG: seems we could literally just add this. Call it Post-V1-Polish instead of Milestone 2?
• CW: is this supported on all WebGPU V1 platforms?
• KG: think it is.
• CW: would like to see the Vulkan Info report for this, and double-check on D3D12. Issue doesn't mention these. Would like to have the fact that it is supported recorded in the issue.
• CW: if we can determine it's available / supported, then we add the format?
  • MM: thumbs up
  • KG: sure.
• KN: I volunteer to do the GPUInfo investigation.
• CW: we'll do this for next week, then tacit resolution.

Investigation: Import VideoFrame from WebCodec to WebGPU #1380

• CW: summary:
• CW: 1) Source of importExternalTexture can be VideoFrame, too. Takes a union type.
• CW: 2) Lifetime of external texture's tied to lifetime of VideoFrame. If you close the VideoFrame, it expires the external texture. External texture stays alive until you close the VideoFrame.
• CW: we've implemented this in Chrome, it works, and is easier than HTMLVideoElement.
• KR: Is there any autoexpiry of VideoFrame at the Web platform level. If you get the VideoFrame from a stream and drop it on the floor, does it expire?
• KG: The decoders will deadlock if it doesn't expire. Some OSes video decoder have a ring buffer of frames and block if you don't stop using one (they always go in order).
• KN: You could let GC take care of them, it might work, but you should close them.
• BJ: Question is, does letting the GC close the object? If yes, does that destroy the externalTexture?
• KN: The ExternalTexture would ref the VideoFrame.
• KR: That works, and avoids the ExternalTexture transitioning unexpectedly to destroyed.
• KN: Yes, that would be exposing the GC.
• MM: The interesting about videos is not the sequence of frames, it is that frames have a timestamp associated with them. This is an important thing to let the window server handle the progression to avoid jank, and to let the display adapt to the refresh rate of the video.
• MM: So GPUExternalTexture doesn't get these benefits. The swapchain is presented each frame immediately. So for WebGPU we'd like a way to integrate something like that, either have a timestamp on the WebGPU canvas, or have a way to put the result of WebGPU back in WebCodecs with a timestamp.
• KG: you can do that. You can create a stream from your canvas.
• MM: yes, but that loses the presentation time.
• KG: do you need that now?
• MM: no, but should come eventually.
• KN: pretty sure you can already do this. Can create VideoFrame from canvas, turn into MediaStream, and the videoframe has a timestamp associated. Looks like you can specify a timestamp upon construction.
• RC: not only are timestamps important for MM's reason - imp't for battery life too. Want to wake up at video framerate, not display refresh rate and go to sleep for most frames.
• CW: think we all agree on the use cases/optimizations described. The integration of WebGPU with WebCodecs doesn't need to worry about this.
• KG: the ingestion - not integration - with WebGPU. Timestamps, just pulling in WebCodecs - is more limited.
• CW: WebGPU producing WebCodecs frames already works because of the Canvas API. Have prototyped that flow - it works. WebGPU's just a computation block in this model.
• KG: main point - this issue is only about pulling in VideoFrames. That's something we can more easily talk about, rather than use cases for e.g. timestamps, which are important but which should be discussed in other issues.
• KG: the proposal above makes sense.
• KG: when the VideoFrame's closed, it invalidates the external texture?
• CW: yes.
• KG: seems reasonable.
• CW: suggest that Chromium folks can close this issue, open a new one with just the description of what we discussed today. Either issue or PR form. Discuss more there.

support blending 32 bit float textures #3556

• KR: fyi this is oft-requested support for WebGL. Only support gap is on some mobile hardware which only supports blending 16-bit floats.
• CW: Theo was keen on this one as well.
• KG: we're not pushing this too hard. Not having this represents a regression compared to WebGL.
• KG: use cases presented in the bug so far are folks asking us for a feature that they'll rely on exclusively in their app, rather than being portable - worrisome to us.
• KN: think we have the data we need in the issue. Metal doesn't have RGBA32F blending on mobile. Has to be a separate capability. Don't know the OpenGL story for that format.
• KG: people do use 1-channel textures.
• KN: one person wanted R32F. Could add RGBA32F, but significantly less interesting since it misses iOS.
• KG: wish we could ask for a feature optionally. Warn on desktop - you asked for this, and it's commonly unavailable on mobile.
• MM: you can do that - request 2 devices.
• KG: we don't know the requests are related.
• MM: don't we have other optional features that don't have widespread support?
• KG: yes - not saying no to this proposal, but would be nice to have the app's intent - that they have a fallback path in place.
• MM: could imagine an API structure for this.
• KG: for this concretely - Moz's internal consensus is, that since WebGL has this, we should have it.
• MM: totally fine if there are extensions that don't run on iOS. Bar for optional features is - are there 2 or more impls they run on.
• KG: nobody's objecting, so if anyone wants this, they can make a proposal.
• CW: on our side, need to discuss a bit more exactly what the use cases are. RGBA32F not being blendable might be the customers' key concern.

depthWriteEnabled and depthCompare are required even for stencil-only formats #3905

Agenda for next meeting

• requestAdapterInfo
• WebCodecs
• Let's keep the agenda light.
• Can “internal” errors be surfaced in createShaderModule or are they always deferred to the pipeline? #4058