Skip to content

Minutes 2019 02 04

Jump to bottom
Corentin Wallez edited this page Jan 4, 2022 · 1 revision

GPU Web 2019-02-04

Chair: Corentin

Scribe: Ken and Dean

Location: Google Meet

TL;DR

  • Discussions about error handling postponed.
  • Naming convention for vertex and texture formats
    • Agreement to use separate enums.
    • Apple to contribute naming similar to Metal’s enums.
  • #182 Swapchain present
    • Low-latency rendering (vsync off) not solved on the Web yet. WebGPU should solve it at a later time.
    • Consensus to use a simple model for now: no present() call, getTexture always returns a valid texture that will be presented at the next animation task.
  • #188 binding model change for WHLSL debate
    • One side: WHLSL is blessed, API should make it smooth to use WHLSL that uses a triple identifying variables.
    • Other side: WHLSL is a flavor of HLSL, different flavors have different register annotations. WHLSL should match WebGPU’s modern binding model.
    • To be continued.

Tentative agenda

  • Error handling updates from Kai #196 #197 #198
  • Naming convention for vertex and texture formats.
  • #182 Swapchain present
  • #188 binding model change for WHLSL
  • #186 and #151, vertex input / attribute changes
  • Agenda for next meeting

Attendance

  • Apple
    • Dean Jackson
    • Justin Fan
    • Myles C. Maxfield
  • Google
    • Corentin Wallez
    • James Darpinian
    • Kai Ninomiya
    • Ken Russell
  • Intel
    • Yunchao He
  • Microsoft
    • Chas Boyd
    • Jungkee Song
    • Rafael Cintron
  • Mozilla
    • Dzmitry Malyshau
    • Jeff Gilbert
  • Yandex
    • Kirill Dmitrenko
  • Elviss Strazdiņš
  • Joshua Groves
  • Mehmet Oguz Derin

Error handling updates from Kai #196 #197 #198

  • CW: Have people done homework?
  • RC: I’ve looked at 198.
  • CW: That’s the most complex.
  • KN: This tries to handle fatal errors. The pull request has IDL, sample code and a bunch of case studies. The case studies try to describe a situation and make sure we send the right thing to app. I’ve tried to make the API as simple as possible.
  • KN: <describes WebGL error handling> … where a context restored event says the same context can be used after errors. This pull request requires the developer to get a new device. e.g. if the device is lost, and the app is in a bg tab, the promise for the new device can wait until the tab becomes foreground.
  • KN: I think the API is relatively simple, and gives the least complicated signals to the applications.
  • KN: Any feedback or missing case studies?
  • RC: I reviewed 197 not 198. Q however: is the device returned in the callback the new device you’re meant to use?
    • KN: not in the device lost callback. That one gives you the old device. Doesn’t need to. Could be taken out. What you’re supposed to do: request a new device, gives you a Promise, and when that resolves, gives you a new device.
    • RC: what’s the rationale for giving the Device in the callback?
    • KN: don’t know. App might use it. Just using it to say “this is the device that was lost”.
    • CW: could be captured in the JS scope.
    • RC: if we’re going to keep it, name it “lostDevice” or something. Like requestDevice as a Promise in general.
  • CW: AIs for everyone: read and comment on these error handling pull requests and discuss next week.
  • DM: what can user do with a device that’s lost? Is there anything but getDebugLabel that user can do?
  • KN: no operations can be done, but it is a JS wrapper, so people could have hung properties off it or used it as a key in a table. Mainly there as a JS wrapper.
  • CW: that was Rafael’s point too: could be captured in the scope anyway.

Naming convention for vertex and texture formats.

  • CW: also should they be a single enum?
  • CW: Issue 164
  • CW: first question is whether they should be the same.
  • JG: not very useful for them to be separate. Could just have a GpuFormats enum. Similar to what D3D does. Various APIs declare which ones are legal.
  • MM: using the type system to enforce things rather than the spec makes it easier for programmers to write correct code.
  • JG: that’s only true if the spec doesn’t change.
  • CW: vertex formats can’t be compressed and can’t be too expensive, while texture formats can be both, so I don’t think they’ll ever merge. Our thought was to have separate enums.
  • JG: I think it doesn’t add anything and duplicates some enums. But ultimately don’t care.
  • RC: fine with having them be merged. Also fine with having them separate. Don’t feel strongly.
  • KD: fine either way.
  • JG: 2 to 1. Let’s split them.
  • CW: ok let’s split them.
  • CW: naming convention for vertex formats? #164 . Someone suggested vec4_float32, vec4_uint8_norm.
  • DJ: kind of like that.
  • JG: me too. I hate typing R8_G8_B8_A8, so as long as we don’t write that…
  • CW: we can abbreviate.
  • DM: the vec approach doesn’t follow any of the native APIs. Seems weird as opposed to what Vulkan and friends do.
  • CW: what does Metal do?
  • CW: we could be closer to what Metal does.
  • JG: does Metal handle BGRA?
    • MTLPixelFormatBGRA8Unorm, etc.
  • CW: not in the vertex formats.
  • AE: metal has UChar4Normalized_BGRA.
  • CW: Metal’s convention sounds nice.
  • CW: I can take the AI to make a PR with vertex formats. We can look at the naming convention there.
  • CW: what about texture formats?
  • JG: would be silly for them to be different.
  • RC: doesn’t the endianness matter? For example, if you’re going to load, cast, etc.
  • CW: think it’s all little endian.
  • MM: when you upload buffers you don’t tell it the data type. So it can’t byte swap. CPU and GPU have to be same endianness.
  • DM: Vulkan actually has differing rules. “Packed” data. Would behave different on big and little endian machines. (Point that maybe there are no big endian Vulkan systems). Matters, for example if you copy from BGRA to RGBA.
  • CW: we can reference Khronos’ data formats spec, which defines compressed texture formats as well as e.g. what it means to be a 16-bit float.
  • CW: someone needs to make a pull request.
  • DJ: if we’re going to pick metal names then I think Apple should put together the pull request. I’ll do it.
  • CW: Intel’s also interested in investigating vertex and texture formats, but that’s a bit orthogonal.

#182 Swapchain present

  • CW: right now SwapChain has getNextTexture and present. What is the use of the present call given that we’re inside rAF? RC suggested you could present early, but then getNextTexture returns null after you’ve presented.
  • MM: one possibly-natural way to do it: getCurrentTexture. When you call between presents you get the same thing. Only way to get next one is to present.
  • RC: disagree with that. Shouldn’t allow while(true) and let them appear to present correctly. Don’t want to allow that on the main thread. Have getNext/getCurrent return same one.
  • JG: Worried that when imposing constraints on getNextTexture/present inside raf, we prevent ourselves from doing low-latency rendering. This is really important in video-games where you want to render many frames and let the presentation engine decide what is shown.
  • MM: is that the opposite of what RC suggests?
  • RC: I’m OK with doing this from a worker. From foreground thread, if you just call present/present/present, events back up.
  • JG: you need to be able to submit more new frames.
  • MM: in description of what I said earlier, only last present in rAF would show up on screen.
  • JG: that may be too late to submit to screen on vsync.
  • CW: are you thinking about games where you turn vsync off and run at 200 FPS?
  • JG: yes.
  • CW: can we make that require offscreenCanvas?
  • JG: it’s not a busy loop. Can say if you haven’t called present from within rAF you don’t get anything. You should not be allowed to call …
  • KN: what about rAF being too late?
  • JG: pick up most recently completed frame.
  • KN: that task will be running when the browser wants to present. Prevents it from happening.
  • CW: sounds like this is still an open problem for how to do this in new-WebGL land.
  • MM: it’s bad if you call too early because you wasted time before vsync. Could have been drawn later. RC’s problem with drawing too many times is maybe worse. Burning battery life is probably worse. Probably easier to make that mistake and produce too many frames.
  • KN: spin loop will prevent web page from working at all. People will realize the mistake quickly.
  • MM: situation I was describing: app draws as many frames as possible in 16 ms. I think that’s bad.
  • KN: I don’t think we should prevent that use case. Maybe make it more complex. 1 frame per rAF should be primary use case.
  • KR: Thoughts on low-latency and web specs, the spec for Web GPU has to obey the HTML spec: if you update Web GPU and other stuff in a rAF then everything has to show up at the sametime. Can do offscreen canvas to decouple the rendering from presenting. Hitting frame deadlines is hard in the browser. Lots of pipelining and scheduling madness. Fairly simple GPU content would be badly scheduled and skip frames -> jittering. We shouldn’t be too clever: if we go explicit presentation model, then we should get the same texture throughout the rAF callback.
  • JG: Disagree, allows someone to “get one more frame in” and misses the vsync window.
  • CW: If application uses the fact that it can render to the same(?) texture multiple times, and uses it to emulate vsync=off, think Ken is saying it should be done through OffscreenCanvas.
  • KR: You can, even on the main thread.
  • JG: Why restrict canvas if we don’t restrict OffscreenCanvas?
  • KR: HTML5 compositing model requires it.
  • JG: OffscreenCanvas lets you subvert it?
  • KR: No, because commit was postponed. There is no current explicit present.
  • JG: None of them have e.g. context loss either. We can invent things we need.
  • KR: Also don’t want applications to burn power needlessly.
  • CW: Seems we’re covered from the IDL side if getTexture returns nullable, keep present which might be noop in rAF. Exact behavior TBD.
  • JG: Present not required for minimal API.
  • MM: Present has no observable behavior, only timing.
  • JG: IDL for this?
  • CW: Current IDL: swapchain
  • JG: How do we get the texture back?
  • KD: What if app holds ref to texture beyond end of rAF?
  • CW: Texture is invalidated implicitly.
  • RC: Are we saying if you do rAF, and you get texture but don’t do anything, do we draw the old thing or blank screen?
  • KN: you’re saying if you’re in a rAF and you do get a texture?
  • RC: what if you don’t get a texture? And what if you do get a texture and don’t do anything with it?
  • CW: for sure if you don’t get a texture, nothing happens. If you do get a texture and do nothing with it, depending, you’ll get black on the screen. You’ll get a lazily-cleared texture from the swapchain and because of the implicit clear you’ll get black.
  • KN: depends on whether present is required. If you don’t call it you’ll get the old frame. Call it: you’ll get black frame. (Discussion of present() semantics.)
  • JG: status quo would be that it’d be presented to the compositor at that time, at the rAF boundary. Think we can remove present() here.
  • CW: ok, so getTexture returns a nullable texture, so null outside rAF. Present is required inside rAF.
  • KN: there is sort of something like “inside a rAF” but not really. There’s an animation frame event every so often. rAF says, do your work now for the next animation frame. In callback, you render your stuff. After, the whole page will get composited.
  • CW: my only worry is, you call setTimeout(1), and you getTexture and stuff with the texture, you’re able to see when the animation task runs.
  • JG: this is current behavior in WebGL.
  • KN: it’s the same in DOM in general. Computed DOM element sizes change at that boundary.
  • MM: also, if you call timeout(1), it won’t get called multiple times inside rAF callback anyway.
  • CW: OK. We removed present(). Can call getTexture any time you want. Changes to that texture will happen during next animation task.
  • DM: think we should do it before the animation task. present should invalidate it. What you get at rAF boundary: you will get a new texture.
  • JG: would like to push back on that until we have more time to think about it. Think Corentin’s suggestion is what we should do next.
  • DM: browser has to hold frame in this model until end of rAF.
  • JG: think we can do better but not in this pull request.
  • DM: like the idea that present functions as specific destroy.
  • JG: me too but let’s discuss this more after the snapshot.
  • KN: think it’s easier to settle on a behavior that matches everything now, then expand it.
  • KR: we were just trying to come up with one solution in the WebGL arena (runNestedEventLoop) which would avoid the need for explicit commit. Can we please finish that investigation first?
  • KN: yes, let’s.
  • RC: what if user holds on to texture after the current frame?
  • CW: validation error.
  • RC: think this is a good idea. Like the textures being transient.
  • CW: we all agree on the first version of this. RC, since you’re there on the screen, can you remove present() and make the changes we agreed upon at the F2F?
  • RC: can do. What are the other changes?
  • CW: the IDL is in the F2F minutes, second day. Would really appreciate it.

#188 binding model change for WHLSL

  • MM: the function of a binding model is to match variables from a shading language to a set of API objects. In WebGPU we’ve used the Vulkan set of API objects, and the officially supported shading language is WHLSL. There’s a problem because HLSL uses a triple when identifying a variable: binding number, type, and space. But in the Vulkan binding model you only get two of those: binding number and descriptor set it’s in. (BindGroup.) Question is, how are we going to merge these together? Lots of options.
    • Have Vulkan binding model win. Adjust shading language.
    • Shading language wins. Adjust API objects to fit.
    • Have a munging step. API objects stay the way they are, shading language too, and add something in between. This thing could be described in the spec, by the API author with another set of API objects, or in the shading language (like a root signature).
  • DM: the string-based approach I described serves both the Vulkan and HLSL model.
  • CW: Dzmitry’s comment says, bind group takes strings as keys, bindings as values. Sampler 0 has visibility “blah”.
  • CW: our position: WHLSL is the language we agreed would be blessed, but it’s not part fo the API. There are lots of variants of HLSL. Each slightly incompatible in the binding model. HLSL/SPIR-V has multiple modes. Can require register/space number. HLSL for D3D12 keeps old binding model but adds root signature - kind of a bandaid. Because of D3D11, and HLSL for D3D11 which is what you’re trying to be compatible with, have to change the binding model for more dimensions.
  • MM: as far as which HLSL flavor, we have no philosophy - trying to make existing shaders in the wild work. If shaders use a particular mechanism to identify variables, it’s either a constraint of the language - we’ll try to match the existing ones - or it isn’t a constraint, we will break compatibility in WHLSL, and will change how we evolve the language.
  • CW: we have examples of GLSL -> Vulkan ports. Replace all their bindings in the shader code with #ifdefs. GLSL: do this. Vulkan: do DescriptorSet# and index. Don’t see a way for us to not require that. The binding model is so much part of the API that we can’t ask people to not change the binding model in source.
  • RC: would DM’s solution help one or the other? (Using the dictionary?)
  • CW: problem with DM’s dictionary is it encodes semantics in bind group layout level instead of pipeline layout.
    • This is unfortunate because single bindgroup is locked into single pipeline index. Removes a lot of flexibility.
    • KN: unless there’s a fixed mapping, but don’t think that’s possible because of limitations on size of indices.
  • MM: I don’t see what the argument is for not changing the binding model. Have made fairly strong arguments for why shader source code shouldn’t change.
  • CW: we have the binding model that’s proven multiple times; used in Vulkan, and it works. Used in Dawn, not as successful as Vulkan, but works with multiple backends. Adding a third dimension with registers makes the binding model even more complicated. Also encourages people to omit the spaceN declaration and put everything in bind group 0, which is not what we want for WebGPU.
  • MM: as far as this is implemented in Vulkan and successful, D3D is also a successful graphics API.
  • JG: one of the points there is there was not a successful transition between D3D11 and 12 without changing shader source.
  • CW: D3D12 have to spec root shader signature source either in shader or programmatically.
  • MM: in regard to making this more complex, I don’t think it is. Single line pull request. HLSL programmers are already doing this.
  • CW: what you have on Github is D3D11 code. You don’t see the root signatures people are using in D3D12 code
  • MM: you could do it either in API or shader code.
  • CW: people are using root signature to bandaid their code from D3D11 to 12. Should not do that in WebGPU. Require people to use modern binding model.
  • DM: RootSignature and Vulkan’s bindgroup are basically identical.
  • CW: let’s talk about this more next time.

#186 and #151, vertex input / attribute changes

  • NOTREACHED()

Agenda for next meeting

  • Please review trivial pull requests so they can be merged!
  • #188, and binding model
  • Vertex input and attribute changes
  • Error handling
  • Pull request burndown
Clone this wiki locally