Minutes 2021 04 12

GPU Web 2021-04-12

Chair: Corentin

Scribe: Ken

Location: Google Meet

Tentative agenda

• Administrivia
• What is the toDataURL behavior for GPUCanvasContext? #1548
• GPUAdapter.gpuApiPreferenceRating #1439
• (stretch) Specify what happens to mappings when the buffer is GCed. #1544
• (stretch) Removing the need "non-filtering" samplers #1562
• (stretch) GPUCompilationMessage: Specify linePos units, consider span #1597
• PR burndown:
  • Add adapter.current #1523
  • copyExternalImageToTexture for VideoElement and canvases #1581
• Agenda for next meeting

Attendance

• Apple
  • Myles C. Maxfield
• Google
  • Austin Eng
  • Brandon Jones
  • Corentin Wallez
  • James Darpinian
  • Kai Ninomiya
  • Ken Russell
  • Shrek Shao
• Microsoft
  • Rafael Cintron
• Mozilla
  • Dzmitry Malyshau
  • Jeff Gilbert
• Francois Daoust
• Michael Shannon
• Rick Battagline
• Timo de Kort

Administrivia

• Chair change for WG, chair for CG
  • Dean stepping down from both CG and WG chair, after ~4 years of service
  • Jeff Gilbert taking on working group chairing - congrats Jeff!
  • Have single chair for CG right now. Think it's best to have same setup as before.
  • Any concern with Jeff being chair for CG as well, so CG and WG have same chairpersonship?
    • WG chairing comes by appointment from W3C, but in CG, we decide our leadership in more direct way
  • No other concerns raised
  • Congrats Jeff on doing double chairperson duty!
• Next VF2F
  • Around June?
  • VF2F can be easily canceled - point in favor
  • DM: Any contentious topics trying to resolve at VF2F?
  • CW: Couple on the API side - shader module hint, adapter & device limits, bucketing, secure contexts.
  • JG: VF2F could be effort to burn down blockers to shipping. Finish writing stuff, addressing TAG feedback, etc.
  • CW: maybe the API doesn't need F2F? Just WGSL? Should we ask WGSL group first?
  • JG: ideas: did we finalize off-main-thread rendering? WebGPU in workers? Multithreaded WebGPU? How do we feel about initially shipping, or punting, those? Maybe punting for V1.
    • CW: think we're punting on them.
    • MM: does anyone's impl support multithreaded WebGPU now?
    • CW: think you can use it in 2 workers at the same time, just not on the same device.
    • DM: did anyone put P0 in the table for priorities?
    • MM: we probably did for this topic. We've heard from multiple independent parties that multithreading's a core value proposition for WebGPU.
    • KN: think we could do it for 1.0. Have a pretty good idea of how it should work. But don't think I'll have time to prioritize before F2F in 6-10 weeks.
    • JG: even if we didn't have it working, is it worth talking about how it's expected to work?
    • KN: don't think I'd have time to write down solid proposal.
    • CW: think we'd like to do some prototyping. But also start discussions with TC39 about instance-sharing.
  • CW: maybe on API side, not too many topics, so maybe WGSL subgroup should decide whether they want a VF2F.
    • Let's ask that subgroup; if a desire, could be end of May, beginning of June.
• First public working draft
  • FD expressed desire for this. We all agreed to do the first stage of patent exclusion stuff.
  • CW: any concern with going forward with this, just a snapshot of the spec from today/tomorrow/next week?
  • JG: think we should just do it.
  • FD: will probably start expectations from other groups that you'll initiate horizontal reviews of the spec. Know you're preparing explainers, etc.
  • CW: other groups we should reach out to proactively?
  • FD: privacy, security, and the TAG.
  • MM: what's the name of the security group?
  • FD: webapps-sec I think.
  • FD: I can work with the editors. Depends on what group wants to resolve. Need resolution in the minutes that I can point to.
  • FD: need document to be published as FPWD. I can take care of the rest.
  • CW: we have a resolution to publish FPWD for WebGPU (no concerns raised).
  • CW: what about WGSL?
    • JG: I assume it's the same thing. They're separate documents but go together.
    • MM: WGSL's useless without WebGPU - WebGPU pretty useless without WGSL too. They go together.
  • CW: OK, resolved then. Could take snapshot as early as today. Let's work with the editors.
• Explainers -> TAG review
  • CW: Kai's written a lot of explainer text for WebGPU. Hoping to use that for TAG review. Few items needing to be filled out.
  • CW: we'll file issues on the TAG repo:
    • Explainer
    • Draft spec
    • Privacy self-questionnaire
  • OK with people if some of us fill out that questionnaire and put it up as a PR on the repo, then file TAG review?
    • MM: sounds fine to me.
• Triage issues more aggressively, proposals->listed on wiki page.
  • CW: we have a lot of issues filed. Makes it hard to see what needs to be done. Have a lot of outstanding proposals open. Some of these are opinions, not really actionable.
  • CW: propose to close out some of these which are mainly opinions.
  • CW: idea to put links on Wiki page for topics.
  • DM: we do have a label already - they're listed concisely.
  • CW: can we close them so we free up the issue tracker?
  • JG: we could alternatively add a different label, but I do like using the issue tracker for this. Agree it's hard to get through some of the stalled things.
  • MM: isn't this a use case for the "Project" with columns, cards, etc.?
  • JG: yes.
  • BJ: WebXR - we used milestones to achieve some of this. Current, next milestones, and "sometime in the future" one. Bucketed things into those. Worked well for us. Didn't have the volume of issues here though.
  • DM: think it's fine to close the investigations.
  • KN: agree. We do have milestones, but haven't been tracking them closely.
  • CW: OK. We can work offline on this - more on the editors' side. Will figure out how to triage more aggressively.
  • MM: if we're going to close the investigation issues, want to make sure they don't get lost. Only close the investigations for topics already in the spec, marked "resolved done"? Or close the investigations and link to them from an issue?
  • CW: makes sense. Can have milestone "for the future", put investigations for things like bindless, indirect commands, etc.
  • MM: makes sense.
  • DM: query for all investigations:
    • https://github.com/gpuweb/gpuweb/issues?q=is%3Aissue+label%3Ainvestigation+

What is the toDataURL behavior for GPUCanvasContext? #1548

• KR: The resolution is that on Windows, the needed swap chain behavior means that we can basically always get Windows' preserveDrawingBuffer:true behavior.
• KN: we can really only use swap chains in desynchronized:true or OffscreenCanvas mode. So don't think the DXGI restrictions are a concern.
• RC: agree. Remaining things here: see what other platforms provide when we use native swap chains there.
• CW: blocked on investigation of other platforms?
• MM: on Metal, you get a metal drawable, lets you get texture from that. Once you present, the texture's gone. So the semantics are what's described.
• KR: But that means that once you send the texture to the browser compositor it is gone, this is why WebGL has preserveDrawingBuffer. It lets you get the data with toDataUrl and friends.
• KN: preserveDrawingBuffer in WebGL does 2 things: 1) lets you get the data back from the canvas. 2) lets you start from the last frame's rendered results. In WebGPU we want (1) but not (2).
• JG: pDB doesn't mediate (1). Think I described it in the first comment. Even with pDB:false, until you present the thing, you can get the data back from the back buffer.
• KR: If you return from the RAF callback and rendered WebGL with preserveDrawingBuffer:false the content is handed off to the compositor and you can't get it anymore. Drawing the WebGL canvas to something else will get you black.
• JG: Firefox used to return black and changed to match CHromium's behavior of giving the handed off frame.
• KR: We had customer issues about this from WebGL.
• JG+KR: We need to figure out in WebGL land.
• KR: The behavior you get depends on the compositor behavior and not necessarily at the end of RAF.
• JG: AI to figure out what WebGL does.
• MM: Would have thought that if I have a canvas on the page drew to it, and then called toDataUrl() would be surprised if data I got was black.
• KN: Would like to have the non-surprising behavior in WebGPU.
• MM: Two ways to do it on Metal: wrap IOSurfaces in MTLTexture and implement things themselves. And CAMTLDrawable. Would be nice that WebGPU could be implemented in both but understandable that we might need IOSurface (?)
• KR: All WebGL browsers use IOSurfaces so it should be fine to use.
• KR: Also you need to make sure you are synchronized rendering with the rest of the page, otherwise that's desynchronized:true.
• MM: It should be possible to use synchronized rendering between Metal and other things - this is done frequently in native apps.
• Resolved to postpone the remainder of this discussion. Can hopefully assume that we will manage buffer flipping manually in the common case in WebGPU, so we don't need WebGL's preserveDrawingBuffer concept.

GPUAdapter.gpuApiPreferenceRating #1439

• KN: didn't like the preference rating as written, but interested in something that lets you say "here are my backends, tell me which I should use based on what's available in software/hardware".
• MM: conceivable there's a workload doing lot of uploads/downloads that'd be faster in software?
• KN: theoretically yes.
• CW: they'd use WebAssembly then.
• KN: also if we give them control to choose whatever they want, solves that problem. They don't have to use this API - they can make their own choice. But also don't think we should design around that possibility. Don't think it's a very good use case.
• MM: think allowSoftware makes sense to me. Would prefer that as opposed to offering two adapters.
• JG: I understand but would prefer the opposite.
• KN: somewhat orthogonal conversation to this one.
• JG: one concern / question I run into with how WebGL works: failIfMajorPerformanceCaveat. People use it without understanding what it means. If you got software, you'd get null back from the context. On many users' machines, people don't get content and don't see things. They'd like WebGL to function & be interactive, even if it isn't high frame rate. Want them to really choose that they want software.
• CW: sounds like you're (JG) happy with allowSoftware defaulting to true?
• JG: would prefer to have developers write more code. Fetch the adapter, have a flag on the adapter saying whether it's software.
• MM: developers would specify failIfMajorPerformanceCaveat, would make content not work on some machines? So don't have failIfMajorPerformanceCaveat, and you preferentially get first hardware WebGPU impl, then software impl? Would get rid of this flag.
• CW: root of discussion - what about applications that'd rather run on hardware WebGL than software WebGPU? Chrome will support WebGPU everywhere, in software.
• JG: my ideal software - give indication of device you're running on. You'd know that it's hardware of some kind. Made suggestion to Google Maps team. Their response: we really want to know if we'll have bad performance. Instead, they create WebGL context, read the renderer string, and if it has stuff in it that says it's software (like Microsoft's Basic Rasterizer), they do a fallback e.g. to using 2D canvas.
  • Want them to think harder about whether to do all this extra work just to switch rendering modes.
• MS: other direction too: some options let you specify you want software when others don't.
• MM: that's the approach we generally prefer.
• MS: not saying allowSoftware, but forceSoftware.
• MM: allowSoftware, forceSoftware, failIfMajorPerfCaveat - all fine with us as long as it's not a boolean on the adapter.
• CW: Michael, think that doesn't solve the allowSoftware case.
• CW: can we just roll with allowSoftware defaulting to true
• KN: thing about allowSoftware is that it doesn't default to true. Lets me choose between my WebGPU and WebGL impl.
• MM: app doesn't care, will get WebGPU either way. App that does care, would say allowSoftware:false, fail to get device, then try WebGL.
• KN: yes, with failIfMajorPerfCaveat. If that fails, fall back to WebAssembly, 2D canvas, etc.
• CW: in that case it solves everything except Michael's case, forceSoftware:true.
• MS: can probably tell our customers how to get what they want. We're moving to web tech from desktop rendering. Have been driver bugs, we have customers that have to have it working right then, no matter how bad it is. In our current app we have a checkbox to turn on software rendering (currently Mesa). That "fixes" those issues until we can get a patch out in a few weeks.
• CW: in Chromium we'll have such a flag. There'll be at least one browser that can force software rendering for WebGPU. Same for Firefox, a pref.
• MM: Safari won't have software rendering.
• CW: "if you need it working right now" - you have a couple of browser options by turning on a pref.
• MS: that does solve our use case.
• KN: more general form of this problem: many users, piece of problematic hardware, want to force those users onto software. Can't instruct those users on setting a flag.
• MM: maybe can make progress with uncontroversial solution, and if someone comes back needing more we can do more. Think allowSoftware and forceSoftware together wouldn't be too bad.
• CW: OK, so allowSoftware?
• KN / JG: sounds fine.

(stretch) Specify what happens to mappings when the buffer is GCed. #1544

(stretch) Removing the need "non-filtering" samplers #1562

(stretch) GPUCompilationMessage: Specify linePos units, consider span #1597

PR burndown:

• Add adapter.current #1523
  • RC: think I'm fine with the PR's latest wording. If we have something like "current", has to say - an adapter can become non-current at any time; here are some examples of when that happens - the browser can do anything it wants. Weird to have something in V1 that's similar to having it be lost.
  • KN: anyone have opinions on this?
  • MM: not caught up.
  • KN: internal boolean on adapter. (1) requestAdapter will always give you a new wrapper object. (2) GPUAdapter objects have internal flag "current". Initially true. Tells you whether adapter can vend new devices. This flag can become false at any time, whenever browser feels like adapter selection might be outdated for any reason. Timer based; power state of system; external GPU plugged in; etc.
  • KN: reason to do this: encodes a way to implement "there's been a system state change". Many different situations look the same. Plug in eGPU; old adapter will stop being able to make new devices. Have to do requestAdapters again.
  • KN: for portability's sake - browsers should mark adapters as stale after some amount of time, so devs calling requestDevice again on same adapter after 5 minutes should do adapter selection logic again.
  • MM: I like this - sounds great. Like that it doesn't tell you which event happened. Also if you plugged in a device doesn't give you the name.
  • MM: to clarify: I call requestAdapter, get an adapter, browser-internal timer expires, then set this boolean on that adapter? Web app calls requestAdapter again, gets distinct new object? Represents same physical GPU, but one has the bit set and the other doesn't?
  • KN: correct.
  • MM: sounds great.
  • CW: one concern: if app's presenting an adapter selection box to the user, some users might get bit by this.
  • JD: that's potentially a problem. GPU could become unavailable between letting user choose a GPU and trying to create it.
  • MM: well, you'd run the logic before and after presenting that dialog.
  • JD: what about defining it more tightly? Like invalidating adapters after every frame? "This is the list of adapters you can have this frame."
  • KN: concern - would prevent you from doing any async work at all between requestAdapter & requestDevice. Thinking we would mark stale after 5-10 seconds.
  • JD: think it'll make things flaky. Something takes a little longer, stops working.
  • BJ: since requestAdapter's async, you'll likely pass the result to another async thing. If we invalidate quickly, when I add another layer of abstraction, things will stop working. Generally think invalidating these things is a good idea, but think it's a little aggressive.
  • CW: think we have general consensus, but have details to work on about adapter invalidating logic. Let's do that offline.
• copyExternalImageToTexture for VideoElement and canvases #1581

Agenda for next meeting

• GPU canvas context toDataURL / preserveDrawingBuffer behavior
• CW: if someone made a specialization constant proposal, could work on that. It's in WGSL but not the API yet.
• copyExternalImageToTexture for VideoElement and canvases #1581
  • CW: on the issue we should make specific points we want to talk about in the meeting.
• Had some smaller stretch issues; can discuss them, and/or make progress offline. Let's try to do those next meeting.
  • Let's do larger topics in a following meeting.