Skip to content

Minutes 2021 03 15

Jump to bottom
Corentin Wallez edited this page Mar 19, 2021 · 1 revision

GPU Web 2021-03-15

Chair: Corentin

Scribe: Austin

Location: Google Meet

Tentative agenda

  • Proposal for a GPUExternalTexture object. #1415
  • Proposal: GPUAdapter.gpuApiPreferenceRating #1439
    • Add GPURequestAdapterOptions.majorPerformanceCaveat #1302
  • Require storeOp when resolveTarget is specified? #1376
  • PR burndown
  • Method of ensuring GPUShaderModules can contain MTLLibraries #1064
  • Agenda for next meeting

Attendance

  • AMD
    • Chas Boyd
  • Apple
    • Dean Jackson
    • Myles C. Maxfield
    • Robin Morisset
  • Google
    • Austin Eng
    • Brandon Jones
    • Corentin Wallez
    • James Darpinian
    • Kai Ninomiya
    • Ken Russell
    • Shrek Shao
  • Kings Distributed Systems
    • Daniel Desjardins
    • Hamada Gasmallah
  • Microsoft
    • Rafael Cintron
  • Mozilla
    • Dzmitry Malyshau
    • Jeff Gilbert
  • Francois Daoust
  • Francois Guthmann
  • Michael Shannon
  • Mehmet Oguz Derin
  • Timo de Kort

Proposal for a GPUExternalTexture object. #1415

  • CW: No longer video texture - may also be HMTLCanvasElement or VideoFrame. Also WGSL changes for the texture type.
  • JG: What does sampling from luminance mean? What would the correct impl give?
  • CW: It would return Y if you have YUV data. If you have RGB data, it would do the color transform to Y and give you that. In terms of memory traffic, if you only care about luminance, you’re no worse if you had RGB, but better if you do have YUV.
  • JG: I think the luminance might be a little different depending on what format you get it in, but I guess that’s fine. Same for filtering - but maybe close enough.
  • CW: there will be some slight differences there as well. Otherwise for first version we can also drop that if you think it’s better, and figure out luminance aspects there later.
  • JG: It feels weird to me, but I think having it at least makes it a better proposal, or more palatable. Vague concerns about the details, but we can hash that out. Like: when you fetch from it, do you get reliably the same result, or does the incoming format have something to do with it. But.. I think everyone uses the same white point, so it should be the same.
  • CW: no idea about color space / management - should talk with media folks.
  • MM: we’ll say - when you sample from this image, this interpolation will be done in the original source’s color space, rather than in RGB? if you wanted to do interpolation in RGB, you’d have to do 4 loads of original texture, 4 times as costly.
  • JG: need to write that down but yet.
  • MM: so not undefined - in colorspace of original media.
  • JG: not necessarily. Depending on code path in Firefox, occasionally decode into RGB not YUV. If you get that behavior, same media / same FF / same device / different driver might get different filtering results.
  • MM: this is desirable?
  • JG: not desirable - acceptable sacrifice
  • CW: can be left unspecified - up to user agent.
  • JG: not even guarantees that sRGB encoded data is filtered in color space you expect. May be filtered in sRGB color space naively, or properly, in linear space. Our APIs don’t give us that guarantee. Metal might do better because of smaller IHV space.
  • CB: in DX there is a flag, app can request filtering with gamma correction applied. Only works for certain pixel formats, 8-bit per channel. Something you need to be crisp on or you’ll get different results.
  • JG: as of newest version of GLES there’s no way to guarantee filtering.
  • CB / MM: what about Vulkan?
  • JG: not sure.
  • MM: if D3D and Metal define this well then it’s a Vulkan issue.
  • JG: filtering in general tends to be inexactly specified in Khronos APIs. Not optimistic.
  • CW: That’s why the proposal as it is now can’t be implemented with vulkan YCBR samplers and must be done with shader code injection.
  • DM: Maybe we should talk about immutable samplers then. THese you need to know at pipeline layout creation time. I assume this is what you meant about Vulkan restrictions.
  • CW: Yes, and you need to know the incoming format, etc… and these things can change mid-video. I don’t see how we can use immutable samplers for this.
  • DM: so users don’t know the format throughout the execution, because it can change?
  • CW: videos can change from YUV 420, 422, etc. at any time.
  • MM: The reason for this is that in a movie, you might want a particular pixel format for one scene, and another for a different scene.
  • CB: heard about this but never seen it actually happen.
  • KN: one place expected to happen: video streams, switch quality levels. Keep using same stream, request server use different quality level. Server keeps stream, switches track.
  • DM: Does the user know what formats will be using, ever?
  • MM: for HLS, no. For YouTube’s, yes. By user, I think you mean author
  • JG: I usually use “app” talking about clients of our API.
  • MM: user is the person who doesn’t know how to program.
  • DM: So there is this optimized path in Vulkan, and we’re just discarding it without doing proper consideration.
  • CW: we did give it proper investigation. There was a very old investigation. In #1415 there is no discussion of this.
  • DM: let’s at least link to it.
  • CW: We have a design doc-ish about how video stuff can work in Chromium, and there we have a discussion about this.
  • KR: In Internal discussions with CHrome’s media team, they pointed out that doing bilinear interp in YUV space is better than not doing anything. They demonstrated this on real users in real web apps. So that’s what they’re doing.
  • JG: That’s my intuition here as well. It’s in the class of filtering where it works on image data. It gives you an approximation that’s still useful even though it’s not the strictest mathematical calculation.
  • KR: I think we can test this in the conformance tests with some reasonable tolerance.
  • CW: Brings me back to: right now the only way to create external textures from HTMLCanvasElement, etc.. do we want a way for users to create external textures from a bunch of GPUTextureViews? So people can inject their own.
  • MM: If it’s for canvas, then people can test their code with Canvas.
  • CW: Yes, though it doesn’t test it with YUV.
  • MM: That would be a browser bug though, not a bug in the app. If they wanted to create one of these textures from texture views, that would also hit the browser bug.
  • KN: Less worried about browser bug, and more about people who aren’t happy with the browser’s interpolation behavior.
  • MM: Can an app load instead of sample from one of these external textures?
  • CW: Yes.
  • MM: Okay, so they can do their own interpolation if they don’t like it.
  • CW: Ok. I’m hearing it looks good overall. We need to bikeshed the builtins a bit.
  • MM: we’ll see if DM comes back after reading more about it.
  • DM: not seeing description of where format’s required just yet.
  • CW: in comment thread with Chad Versace.
  • MM: let’s tentatively move forward, might have to revisit.

#1439 GPUApiPreferenceRating

  • Add GPURequestAdapterOptions.majorPerformanceCaveat #1302
  • JG: not sure exactly what to do here. Think I’m trying to get around the fact that sometimes apps are written poorly. Maybe I should give up on that. Don’t know how much want to discuss this topic right now since not sure this is the right answer.
  • JG: related to failIfMajorPerformanceCaveat, which == is_software. Idea: want to empower users to still have web pages work, when e.g. Google Maps says I don't want to render with software. Don’t tell anyone I have a perf caveat. But app like Three.js that has multiple backends at parity (feature / perf), rather use HW WebGL2 than SW WebGPU, but not a good way to surface that to the app right now. Maybe all we can do is put is_software on all of our rendering APIs.
  • MM: Another option is expose this by the behavior of the APIs and not by additional bits. If they’re sensitive to performance, they can run shaders on both, and it should be immediately obvious if one is software and one isn’t.
  • JG: People have tried this in the past, and we’ve recommended doing so. They end up just guessing. Not sure if it helps fingerprinting.
  • MM: this isn’t a fingerprinting concern. Partly for simplicity, partly for precedence on the web platform. If you want to call a function, you just see if it exists, don’t look in some other place.
  • JD: Want to point out that software raster isn’t the only reason why failIfMajorPerformancCaveat exists. It’s also for software readback for compositing.
  • JG: in FF we stopped exposing that. WebGL + software readback is faster than 2D for Google Maps.
  • JD: For Google maps in particular?
  • JG: Yes. keep in mind Canvas 2D is not HW accelerated everywhere. Per-frame extra hit -- nothing the user can do about it. Not that much useful information to website. Not perf killer. Minor caveat. Can’t play Doom clone in WebGL on SW on 1080p screen. SW readback, tolerable.
  • JD: depends on machine. SW readback would freeze mouse pointer, etc.
  • JG: Part of it is I don’t like the name. We knew that failIfMajorPerformanceCaveat is an awful name. If we can just call it isSoftware, it would be better.
  • CW: think that would work?
  • KN: not sure I’m the one to judge it.
  • JD: isSoftware flag sounds good to me.
  • MM: Which browsers plan to ship software versions of WebGPU?
  • JG: FF, Chrome,
  • RC: Probably would not fall back to Warp software.
  • CW: Would Edge use isSoftware?
  • RC: At first, we would fall back the same way Chromium does.
  • MM: and Chrome plans to fall back to SW Vulkan?
  • CW: SwiftShader, yes.
  • KN: GPU API performance rating - think we don’t want something looking exactly like this. There is value in something that lets people make decision between SW WebGPU and HW WebGL. Could have API, here’s the list of backends I can support, HW / SW WebGPU/WebGL[2,1] - which is HW, give me list of ones that have which support. More abstracted than preference rating thing, but let people make same kind of decision.
  • MM: Do we have any indication anyone would actually call this?
  • JG: if this is the only way to determine SW WebGPU / HW WebGL2, Three.js would probably use this.
  • KN: If we had a failIfMajorPerfCaveat, they could try calling WebGPU with failIIf, and then try calling WebGL 2 and WebGL 1 with that as well… and try all the APIs with software.
  • JG: that’s one of things I’d like to talk about - think failIfMajorPerfCaveat shouldn’t be context creation flag that fails. Would be improvement where WebGL had adapters, see if it’s HW/SW before you create it. WebGL ecosystem - people don’t want failIfMajorPerfCaveat, set it to true, don’t have a fallback. If they had to write code to handle SW adapters, would be better.
  • KN: In that case, we would basically be saying you always get all the adapters, and if you don’t want software, you have to check for it.
  • MM: our thought is pretty much opposite of Jeff’s. It’s more of a fingerprinting vector too.
  • JG: Not a strong argument since it’s two-call vs one-call. And you can easily tell if it’s software.
  • CW: so no resolution?
  • JG: direction: I’ll close my own api preference rating - not the direction we want - can talk more about what to do with majorPerfCaveat. Need to discuss more to reach consensus about how to expose. See if we can address concern Myles raised.
  • MM: We would prefer it to be the style of failIfMajorPerfCaveat, rather than an additiontional bit.
  • KN: I think this entangles with the requestAdapters proposal.
  • JD: Will we expose isSoftware even if you have hardware?
  • JG: I would like that.
  • MS: We would like that well. For testing on software on computers that do have hardware. And sometimes we have customers that have weird GPU issues and we’d like to force software.
  • MM: first argument, for testing - don’t need to expose web API because it’s internal testing. Second, if machine’s GPU doesn’t satisfy WebGPU requirements, adapter shouldn’t be exposed at all, and should just expose software WebGPU.
  • JG: think that throws baby out with the bath water - expects us to not write any bugs. If we accidentally let a bug slip to release, past QA, broken for months/even days is too long for people depending on our stuff. At least, it’ll work in software, and you’ll see something.
  • MM: For applications that are going to be able to detect this problem - they could fall back to software, or also fallback to WebGL.
  • JG: Well, it’s absolutely trivial to just select a different adapter. We’ve heard ISVs say they have no intent of having a WebGL backend, so this would be a problem for them. It would be better to offer them a software adapter.
  • JD: I wonder if there are situations where people prefer a software device. Like it’s better than WASM SIMD for example.
  • MM: that argument would be compelling from our side. Think it’s worth trying to come up with such an example. Computing something nontrivial where computing was faster on software than on HW.
  • MS: We are the people Jeff is talking about. The reason we can’t fallback is because Z buffer support, and WebGL is not the same as WebGPU. We just can’t render the scene unless we fallback to a crazy logarithmic Z buffer.
  • CW: makes sense that applications written starting from now, some might just target WebGPU. At that point, want to be able to fall back to software.
  • KN: MM is there a concern with always exposing software?
  • MM: concern: failIfMajorPerfCaveat would be better design, and it doesn’t allow exposing both.
  • KN: we could have a flag “please give me software”.
  • CW: let’s time box this discussion & move to something else.
  • MM: if someone could provide example, would be compelling.
  • CW: SwiftShader only does SSE4.
  • JG: probably better than SIMD Wasm. Theoretically possible, if it could handle this would be neat.
  • DM: our impl not as affected by this - can reply on different backends. But if people develop on Chrome I see.
  • JG: on native APIs we say trying / testing on different backends is a good idea all the time.
  • MM: I think we would object to a flag that says useD3D/useVulkan/useMetal.
  • DM: not proposing that. Bigger picture with those ideas in mind. Talking about testing, need to handle these both, not just isSoftware.

Require storeOp when resolveTarget is specified? #1376

  • DM: no final approval from MM?
  • MM: making it always required is unfortunate for new developers. When want to render something, they probably want it to be rendered.
  • KN: new developers are going to be looking at tutorials. Small amount of friction.
  • JG: possibly good opportunity - teach developers what this is.
  • KN: Three options:
    • never required - today
    • always required when you have a resolveTarget
    • always require it
  • KN: If someone is using a resolveTarget anyway, then adding the requirement to say storeOp is not an issue.
  • MM: sometimes requiring it would be fine. If someone specifies resolve target, think it’s reasonable to say, what do you want us to do with non-resolve target? please make sure. 2 options, both make sense, you need to clarify to us.
  • DM: concern - having resolve target isn’t only place where people should think about store ops. Almost everybody uses depth one way or another. Doesn’t need to be stored. Required == people recognize it, instantly get perf benefit from not storing it.
  • KN: for context: DepthStoreOp is already required.
  • JG: if we make them do the depth, why not the color? Think people should make that decision.
  • MM: if question is - why not make them specify them the color - isn’t rhetorical, then I reiterate - when people draw, they expect the results to show up.
  • JG: I just don’t think it’s unreasonable for them to have to tell us that they do want it to be there when they’re done drawing.
  • MM: I think if you're writing a deferred renderer, you’re not in the class of developers I’m talking about here.
  • JG: if you’re in developer category you’re worried about this, they’d like to have to specify it because then they learn about it.
  • MM: If we take this as a principle, what’s stop us from applying that argument to literally everything in the API.
  • JG: slippery slope arguments aren’t helpful - I’m comfortable making the judgment call in this case. Not too onerous.
  • KN: my argument in support of JG - we don’t want people to think everything they render shows up somewhere. Ther’es a cost to that. Don’t want people to think it just happens, it’s free. It’s not free, it has a cost, and it’s good for them to understand that they need to do something to pay that cost.
  • MM: The fact that there’s a cost means there’s an option they can change. It doesn’t affect the argument at it’s a good default.
  • JG: The argument is that it’s not a good default. How much do you care? I think we have rough consensus.
  • CW: Would it be fine to go with the more restrictive thing now, and then we can always make it optional later if users are upset?
  • MM: This is not going to be the straw that breaks the camel’s back. Developers instead will say “writing hello world requires 500 lines”.
  • MS / JG: I’m fine with that.
  • DM: If I understand, render pass objects are some of the more complex bits, so being a little hideous is ok.
  • JG: might not be clear path toward consensus here, but would like to decide.
  • MM: from our perspective, this is less imp’t than other issues, so if yielding here will give us benefit on other issues, we’ll yield here.
  • CW: In the spirit of progress, we should just make it required. General consensus says it should be required. We shouldn’t spend so much time on a small issue.
  • DM: Should we start collecting a question list of ISVs for early adopters? And we can send it to people who try the origin trials?
  • CW: Yes, and I was thinking we could do more - and ask them if particular flexibility would be helpful for them. like rowPitch alignment, and other paper cuts. Will you start making such a list?
  • DM: Sure.

Add GPU.adapters_active #1477

  • KN: The goal is to make there be fewer different situations that applications should have to respond to. Make requestDevice behave more similarly across scenarios so devs don’t use it unportably by accident. THere are some less extreme things like explicit calls to device.destroy(), and more extreme things like unplugging an eGPU.
  • KN: proposal: add global flag. if false, can’t create adapters at all. Gets set to false aggressively. If any device lost: gets set to false. If adapter config of system changes, or power config changes - say, system prefers integrated GPU on battery, then unplug laptop: should set this to false. Forces app to always check current adapter state, see what browser would like it to use, rerun adapter selection logic, at any point where it wants new device - instead of using adapter it’s already holding on to.
  • MM: when is it set to true?
  • KN: When requestAdapters returns - resolves the Promise. It gets set to true and then the app returns the list of adapters.
  • MM: so if it’s false you have to call requestAdapters again?
  • KN: yes. Anytime you want a device, you should call requestAdapters. (?? I didn’t get the beginning of this answer)
  • CW: this is great socialization, to give people an idea. Should continue discussion on Github issue.

Agenda for next meeting

  • Talk about pipeline / shader module warmup thing
    • MM: yes, I’ll do more homework by next time.
  • GPUTextureObject thing?
    • CW: think only thing was whether to use Vk immutable samplers.
    • DM: OK, if we resolve on Github by then sounds good. Otherwise, on the agenda.
  • CW: with spec editors lets make list of small outstanding items we can use as filler.
  • JG: alphaMode stuff?
  • CW: as top level issue - yes, sounds good.
  • DM: on WGSL call - q about max number of builtins. Confusing point: builtins need to be accounted for.
  • MM: something to discuss?
  • JG: maybe WGSL meeting thing? Reflection for it in host API if it’s configurable.
  • DM: ok.
  • MM: question about mrshannon’s point - depth buffers working differently in webgl / webgpu - any background on this?
  • MS: webgl is stuck in the opengl -1.0 .. 1.0 non-reversed range of depth buffer, while webgpu can use reversed range 1..-1 or 1..0. very useful for large scenes. If didn’t have that would need logarithmic z-buffers and multiple depth ranges, render in sections, just not worth the hassle.
  • JG: we’re discussing something related - changing color buffer formats for default back buffer (drawing buffer) - maybe can consider.
    • KR: would depend on various GLES extensions - not sure if any actually support reversed depth buffers - need to see. Otherwise probably infeasible.
Clone this wiki locally