GPU Web 2020 05 11

Chair: Corentin

Scribe: Ken / Austin

Location: Google Meet

Tentative agenda

• Error conventions for extensions #701
• Add pipeline shader module error enumeration. #646
• Add GPUShaderModuleDescriptor/sourceMap. #645
• Depth-stencil stuff #306 #652 #755
  • Should we have a core "stencil8" format? #306
  • Copying of depth24plus formats #652
    • Provided the data extraction from vulkan.gpuinfo.org is done
  • Add "depth32float_stencil8" extension for GPUTextureFormat #755
• 0-sized buffer #546 (we didn't resolve last time)
• Add minimumBufferSize to BGL entry #678
  • Do we need the performance investigation to make progress?
• Add Queue/writeTexture method #761
• Agenda for next meeting

Attendance

• Apple
  • Dean Jackson
  • Justin Fan
  • Myles C. Maxfield
• Google
  • Austin Eng
  • Brandon Jones
  • Corentin Wallez
  • James Darpinian
  • Kai Ninomiya
  • Ken Russell
• Microsoft
  • Damyan Pepper
• Mozilla
  • Dzmitry Malyshau
  • Jeff Gilbert
• Matijs Toonen
• Timo de Kort

Administrative item

• CW: Anything we do at F2Fs that you'd like to do in this group?
• MM: 14 hours of meetings at CSSWG, split across 4 days. Worked pretty well. Effectively like a regular telecon, just went for 3 hours. Break every hour. 3 50-minute chunks. Format, etc - just like regular telecon. One was scheduled at 6 AM, unfortunate, but we could do better.
  • DJ: I did not attend the one that started at midnight.
• MM: at first had only scheduled the first two. At end of first two, realized bunch of other topics, so extended with 2 extra meetings the following week. Could start small.
• KN: think we could easily spread this across multiple weeks. If we had extra meetings for couple of weeks, could get us through things that we would do at a F2F.
• CW: what about good enough lead time for people? End of June-ish?
• DJ: I could start 1 hour earlier than this time.
• MM: would help if we made a Doodle for this.
• CW: let's go for 3 hour chunks, will make Doodle poll.

Error conventions for extensions #701

• KN: Previously, I made a proposal for guidelines on how we would spec how IDL changes for extensions and optional features interact. The context here is that adding members to dictionaries in WebIDL is technically a breaking change. It’s unlikely to actually break content, but members that were previously ignored are now type-checked. Extensions are weird here. Dictionaries are expected to grow over time via extensions. We want code to behave the same way regardless of whether the browser supports the extension, or it’s enabled, or the hardware supports it, etc. Right now, we say: any key in the dictionary that is not known or part of an enabled extension is an error. This is not implementable in IDL; we’d have to do that ourself somehow. We could: \

1. Say all extension dicts have type "any" and we check all the things ourselves. \
2. Or, we take on the backward-incompatibility the rest of the platform already has. It’s a little worse for us because extensions are not expected to be on all platforms. The backward-incompatibility should be relatively minor. When we add a field for an extension it’s optional, and if you were using it before with the wrong type, you get an error. Want to get confirmation from everyone whether this sounds okay. The editors decided that this was okay.
   • BJ: this is similar to issue in WebXR; problem was existing browsers don't have built-in way to enumerate all values you can have in a dictionary. Figured out a way to do it but it's slow and fragile. We came to conclusion that's an anti-pattern. Didn't want to do anything in that API which relied on knowledge of non-specified dictionary elements. Wasn't clear we could do it in a safe, fast, consistent manner.
   • KN: For us, I think we’d have to pull out the v8::Object and call against the standard v8 API. Don’t want to do that. Rather have an attribute in IDL that is “strict”. Not allowed to have extra members.
   • MM: there are APIs that take "any" type and require run-time type checking. This is the same thing as that. You'd get the JS object and look at its fields. It is implementable.
   • KN: we'd make all the new fields "any" type. That's option (1).
   • MM: 2 points:
     • 1. we will be adding things to core in the future. Not all new functionality will be added just in extensions.
       • KN: extensions aren't solving this problem, they're causing it. Similar problem for core, though.
     • 2. JS APIs evolved for dozens of years. Standard procedure for this. Add new members to dictionaries. Easy for us, dictionaries are descriptor types. Wouldn't expect much JS to add unexpected fields to them. They're temp objects JS passes into construction methods. If I understand the options, option (2) makes the most sense.
   • KN: that's the one I want to go with too.
   • BJ: If the intent is to have extensions in the future which will extend the dictionaries, that’s hard to do in WebIDL as well. There’s no concept of a partial dictionary.. okay maybe there is but it seems highly discouraged to do that of any sort. It’s been awkward for us to split things into modules.
   • KN: The intent for webGPU is to update the entire spec to include the extension, and every time we use the field, we say that “if X is enabled, we use this field”.

• CW; everyone OK with option (2)?
• (yes)
• Solved.

Add pipeline shader module error enumeration. #646

• CW: Getting compilation messages back from shader modules. I think we agreed
• JG: idea was to split it up so this didn't implicitly depend on source maps.
• CW: should we talk about async shader reflection being on there as well? MM pointed this out in #624. Would probably be a good place to put reflection information.
• JG: in a Promise could be a good place for it. Imagine that you'd have GpuCompilationStatus/Result, some sequence of messages, sequence of binding points, or similar.
• MM: makes sense. Probably also want opt-in for reflection. Only create if authors want it.
• CW: So for reflection information, would it make sense to have CompilationResult which is a dictionary which has messages as well as reflection information and stuff if we need more in the future?
• MM: is Result the right word? Result is the Pipeline.
• CW: yes.
• MM: think you're right, just don't know what the right name would be.
• CW: ...
• JG: CompilationOutput? Should we land that first or this first?
• MM: Don’t think it needs to be blocked.
• JG: Okay, will remove the source maps and we’ll land this.

Add GPUShaderModuleDescriptor/sourceMap. #645

• CW: DM or JG pointed out this could be an extension if there are concerns about tying this to a particular source map format.
  • DevTools folks said don't rely too much on the format.
• MM: specific forma'ts a good start. If you want lldb / web inspector tools, this isn't the right format.
• JG: can you say more about that?
• MM: in most debuggers you can print values of values, etc.
• JG: I don't expect us to have native support for printing from shaders.
• MM: I think developer tools are an area of competition for different browsers.
• JG: not sure how those relate.
• MM: I guess the point I’m saying is that this is an active area of research at least for WebAssembly and other places too. We might want to update whatever we do to match what the WebASsembly community creates.
• JG: OK, but source maps are shipping already.
• MM: I’m not saying that this is a bad format, it’s that there may be new formats in the future.
• JG: sure. It's versioned. I don't see how to resolve the issue that we might change the format in the future.
• CW: I think the difference is that source maps are only in dev tools, so user agents can do whatever they want. They can extend stuff with DWARF or etc. without incurring a compatibility / breaking change. An extension is an extension can be turned off at any time, but there’s a slightly higher compatibility risk. Programmable makes it a but more difficult to innovate.
• JG: had trouble imagining where we'd turn off this extension. Does it make it worse than saying we don't support v4 in core source maps any more? Either content that's backwards-incompatibly affected by removing v4 sourcemaps, or not. If there is, we can basically never remove it. If there isn't, we can remove it from core.
• MM: Wasn’t arguing for or against extensions. I’m agnostic about extensions specifically.
• JG: seems there's most consensus for making extension. Like in WebGL, though, we have a bunch of extensions which are effectively "core", and should be folded into core.
• KN: I think that if we want to expose data back to the application, like mapped line numbers, we should consider making that an extension, but the unobservable part of passing a source map in has no effect to the application. The changes only appear in dev tools. It should be just like JS where you provide the source map and devtools might do something with it. We should stick with that way of specifying so we don’t worry about breaking changes or extensions, etc.
• MM: might be good user experience to return mapped line numbers. Worth considering.
• KN: I do think it’s worth considering, but that functionality alone could be behind an extension. You’d still be allowed to pass a source map without an extension.
• DM: that source map would just be an opaque object? Not tied to version of SourceMap spec?
• KN: pretty much.
• DM: sounds fine to me.
• MM: So the spec wouldn’t have version numbers, you pass in something and cross your fingers.
• KN: I think that's what JS does. If you open DevTools & sourcemaps aren't there, you look at warnings in the console. If only affects DevTools, not sure why this should exist in spec aside from having dictionary members to put sourcemaps into.
• JG: Generally the reason is so we have the portability concerns we care about. and test it.
• KN: we can't test it though.
• JG: In order to test without having some specification would be to load it up in every browser and manually check. That’s a little bit worrisome.
• KN: I suppose so. Don't think we need to innovate relative to JS though.
• JG: was strong desire to be able to have feedback from shader compilation that wasn't the actual line numbers from the source. This was a compromise for that. Can't have #line directives in WGSL if we don't emit them.
• KN: you can use client-side source map library to do the mapping.
• JG: yes, but I'm not going to counter my own argument.
• DP: is this about being able to surface error messages with line information or passing debug info into compiler?
• JG: not necessarily into compiler. It's if you had HLSL-like source, you compiled into WGSL, and wanted to surface info like you had duplicate binding name, incompatible binding type. Gives you line number into HLSL.
• DP: I think that would be useful and could be done in user space. The part that’s important to me is to not create side channels into the WebGPU backend in order for the WGSL compiler to pass debug information through.
• MM: what sort of things are you envisioning? When WGSL compiler passes debug info through?
• DP: So on Windows, when we compile a shader we generate a .pdb file. I want us to be able to build source-level debugging from the original source language.
• MM: you want web developer to be able to supply .pdb into WebGPU API?
• DP: that's what the spec says.
• MM: that's not what the spec says right now.
• JG: You’d compile the .pdb into a source map.
• MM: Is that sufficient for you or do you want to pass the .pdb directly?
• DP: source map's sufficient for that.
• JG: you'd have WGSL targeting compiler, DXIL -> WGSL + source map. Source language + target language => one true format for both source maps and shaders. Sounds like there's consensus on way to provide a source map.
• MM: When we discussed this previously, I had some concerns about the method of delivery (like extra member / argument). I talked with our JS team and it’s fine, no problem.
• CW: seems everyone's OK with source map being v3 feature / extension. Does it give back mapped lines / position, file, etc. in compilation message, or not?
• MM: Could we not specify that?
• JG: if you want it to not be super portable.
• MM: error messages in native compilers are not specified.
• JG: but ... ok.
• KN: are you talking about WGSL line numbers, or the mapped ones?
• MM: talking about having fields but not specifying what's in them.
• KN: No we can’t do that. If we’re going to pass back WGSL line numbers, the application has to know that it can use it to do source mapping in user space.
• MM: that's true.
• JG: for this proposal, sounds like general consensus: you should be able to provide a SourceMap object. If it's defined, it's v3 format. If it's defined, it might be interpreted as such. It may do something else.
• CW: I think if it’s an extension, we could just call the member “sourceMapV3”
• KN: my argument: it doesn't need to be extension if it isn't observable. No objection to sourceMapV3.
• CW: It is observable if we have the mapped line, and I think people were reasonably in favor of that.
• KN: I’m not convinced because it can be easily done in user space so few apps will need it in production.
• MM: if app gives you a sourcemap you might as well use it.
• KN: can use it in DevTools but doesn't make sense to me to expose it. At that point we do bake it into the API and it should be an extension because we should consider whether it should stay there forever.
• MM: We could also not make an extension and say every implementation needs to understand v3.
• KN: I don’t see the value in locking ourselves into supporting this format forever in the API. I don’t think Web apps need it if they can do it in user space.
• JG: you see this only as a way for apps to communicate into DevTools?
• KN: That’s what I’d like.
• MM: maybe we could have two fields, one for mapped and one for unmapped.
• KN: That’s what the original PR had, but we’d have to make it so the mapped one is optional. You might not get it back?
• MM: could be.
• KN: don't want to make API surface that we can't unship, that adds lot of complexity, when it can be done easily in user space.
• DP: MM you were proposing #include files in WGSL? Could do concatenation in user space. Every dev env wanting to map errors back to source files will have to mapping themselves. Valid use for having browser-understood mapping.
• JG: Doing the mapping back isn’t that hard, but sort of the idea here is that browsers are going to be trying really hard to do this anyway. I think we should think about this more and move on. I’ll put up a consensus proposal to move us forward.
• CW: Sounds good, thank you.

Depth-stencil stuff #306 #652 #755

Should we have a core "stencil8" format? #306

• CW: biggest concern, D3D doesn't have this. D3D has S8 + 24 bpp of nothing. Users would think S8, but get 32 bits per pixel.
• KN: D3D would always do this, many Vulkan devices probably (haven’t looked deeply). D3D doesn’t worry me all that much because it is mostly a desktop API. Additional memory usage is easier to take? There’s always the possibility that as D3D starts supporting mobile architectures better it will gain a stencil8 format. Apps can use depth24plus-stencil8, but they wouldn’t be able to use a smaller format if supported.
• DM: this may end up being D32F+S8, 64 bpp, not great.
• JG: we should expect that to happen generally. D32F+S8 is strided to 8 bytes.
• KN: yes. Don't think there are lot of situations where we'd use 8 bytes. Most likely 4. Most platforms have S8 or D24S8 or X24S8 available.
• MM: sounds good to me, think we should have it.
• RC: I’d challenge that D3D is a Desktop API. We have things like Hololens coming up. Increased memory usage would be bad in those cases.
• KN: that's basically my point - the fundamental limitation in D3D right now is because of desktop GPU architectures. Since D3D12 is evolving to run well on mobile archs, where it can be done with true S8, think we'd just get rid of that memory overhead on those devices.
• MM: Maybe we can ask RC directly how he would feel about adding stencil8 directly.
• RC: no plans I'm aware of - of having S8 as true 8-bit. Have misgivings about emulating something that's a small thing and it turns out to be a big thing. E.g., in WebGL, uncompressing compressed texture formats behind the scenes. We shouldn't do this by default unless we communicate this to web developers.
• KN: that's fair. I think it's a little different. If all you need is stencil buffer, there's no other alternative than D24S8 and ignore depth buffer. Or do algo not involving stencil at all. Would like signal of what's going on in the backend. Portability hazard.
• MM: What if we added a stencil8 format and a bit somewhere that says whether it’s emulated or not.
• RC: could do that. When you say "a bit somewhere", would you query this?
• MM: It could be on the GPUDevice, for example.
• RC: maybe good compromise. Not having it at all, or extension, is less good.
• DM: would we do the same for the plus formats as well?
• KN: we could consider that. Mem overhead on those can be bad too (2x).
• JG: Another way would be to make true stencil8 be an extension. There’s stencil8plus which is an extension, but then true stencil8 lets you do other things now that you know it’s fixed.
• MM: philosophically, adding modes to APIs in general is difficult for authors. Increases coding + testing paths. Have to determine what warrants a mode or not. Think this doesn't. Still implementable everywhere, just have memory cost. Don't think this is worth an additional mode.
• CW: On D3D, applications will probably use stencil8 which in Dawn would be D32S8 which is worse than directly using D24S8. So I think it’d be better to do the emulation plus a bit.
• KN: if I understand correctly, idea wasn't to add a mode, but to have a "X0plusS8" format and an extension which exposes S8. Similar to D24plusS8. And perhaps an extension for D24(concrete)S8.
• RC: does + mean more than 24?
• KN: yes, 24 or 32, in depth.
• RC: >= then.
• KN: yes.
• RC: why emulated in Dawn with D32S8?
• CW: think there's some confusion.
• KN: that's why I'm suggesting extension. Apps OK with X0+S8 can use it, but those needing S8 can only do it when available.
• MM: Sounds like making authors’ lives more difficult. Choices are no longer give my S8 or not S8. The two sound too close that people may get it wrong and not understand it.
• JG: alternatives: not give them tools to choose precisely, or have portability hazards in core, where some formats aren't supported on some platforms.
• KN: Just a rephrasing of the bit, and I think it’s a more extensible one.
• JG: extension would let you do other cool things, like we can't support aliasing unless we have strict encoding for the bits, which we don't have for these "plus" formats. Can't read S8+ buffer as R8 buffer, for example.
• MM: haven't designed aliasing feature. If we were to do so, would like something safer. Reuse memory, but don't reinterpret the bits.
• DM: how do you feel about query that would tell you size of texel or texel block, per format? User would ask about 24+, we'd say 32 bits, and they can decide whether to use it.
• CW: Think it’s a bit more complicated, because we’d just give what we know as an implementation, but hardware might have… 35bits on average because of compression planes.
• DM: fine to not take compression planes into account.
• CW: Basically give our best guess, and the application can make choices in consequence.
• KN: thinking about it, doesn't that mean D3D12 could probably implement X24S8 format without any actual depth storage? Since you can't read that data anyway unless it's aliased? Which you can't do unless you allocate it as TYPELESS?
• JG: I think the TYPELESS things are the main limitations there. THat’s why I think it’s better to have an extension to leave the door open to TYPELESS. There’s definite prior art that reinterpretation is useful.
• CW: Progress. Let's follow up on the issue.
• KN: I think I know what I want to propose now [(JG’s idea)].

Copying of depth24plus formats #652

Add "depth32float_stencil8" extension for GPUTextureFormat #755

0-sized buffer #546 (we didn't resolve last time)

Add minimumBufferSize to BGL entry #678

• Do we need the performance investigation to make progress?

Add Queue/writeTexture method #761

Agenda for next meeting

• Texture reinterpret_cast issues
  • MM: implementing this in WebKit, tried every combination, will make PR about what's allowed and not.
• writeTexture.
• Please volunteer topics by putting them on the NeedsDiscussion in the main project on Github. Will make an agenda for this on Thursday.
• Again minBufferSize? Don't have data yet on perf cost of CPU-side checks.
  • MM: me neither.
  • CW: OK, that'd be at the end of the agenda then.