Skip to content

Minutes 2019 08 12

Jump to bottom
Corentin Wallez edited this page Jan 4, 2022 · 1 revision

GPU Web 2019-08-12

Chair: Dean (Corentin on vacation)

Scribe: Ken et al

Location: Google Meet

TL;DR

  • Coordinate systems and winding #379
    • YH did an investigation (just published).
    • 3 coordinate systems: NDC, viewport, texture coords
    • Can we add switches for these systems to ease portability?
    • No one seems to have very strong feelings, some want NDC and viewport to match (Vulkan), others want to match WebGL or D3D.
    • To review and discuss again later.
  • Bindless resources #380
    • Can’t be implemented everywhere. Not available on Android.
    • Requires residency checks since we don’t know what’s bound at API level.
    • Resolution: Punt until users want it.
  • Generate Mipmaps #386
    • WebGL, OpenGL, and Metal have generateMipmaps functions (GL, at least, doesn’t specify results). D3D has it in a helper library. Vulkan has nothing.
    • Makes it easier for authors to generate mipmaps. But can also be provided as a small library for authors to copy in.
    • Lots of corner cases (desired quality, srgb aware, etc.)
    • A helper function for this would be really easy if we had blit commands.
    • But blit can be complicated.
    • JG to provide pseudocode for a helper function.
  • Investigation: Copy With Compressed Texture Format #363
    • Compressed textures (mip levels in particular) aren’t always multiple of block size.
    • Vulkan spec seems to disallow non-block-sized T2T copies (unless copying whole image).
    • Not really clear what compressed T2T subimage copies are useful for.
    • Resolution: For MVP, disallow all compressed T2T copies until someone needs it. T2B2T as a workaround.

Attendance

  • Apple
    • Dean Jackson
    • Justin Fan
    • Myles C. Maxfield
  • Google
    • Austin Eng
    • James Darpinian
    • Kai Ninomiya
    • Ken Russell
    • Shrek Shao
  • Intel
    • Yunchao He
  • Microsoft
    • Chas Boyd
    • Damyan Pepper
    • Rafael Cintron
  • Mozilla
    • Dzmitry Malyshau
    • Jeff Gilbert
  • Joshua Groves
  • Mehmet Oguz Derin

Administration

Coordinate systems and winding #379

https://github.com/gpuweb/gpuweb/issues/379

  • YH: did an investigation.
    • Affects things like texture uploading. NDC / FB coordinates might be different than texcoords, for example.
    • Impl perspective:
  • DM: This would be the Vulkan convention and opposite of D3D12 / Metal?
  • YH: Yes for NDC
  • RC: Probably should go with what most of the APIs do.
  • MM: From the Github conversation, lots of people disagree with at least NDC. Can we configure APIs separately? Can we add a switch?
  • JG: for NDC Vulkan is the odd one out. Didn’t they add a way to select the normal coordinates?
  • KR: Thought they did add an extension to flip this.
  • YH: there’s a way to select -y for the viewport coords.
  • MM: We could also modify the shaders. All the implementations will at least validate the shaders and probably change parts of it as well. Adding two math ops around every vert shaders’ output, it doesn’t seem too difficult.
  • YH: agree
  • DM: find the proposal appealing because it’s consistent between NDC and texture coords.
  • YH: if NDC coords != framebuffer coords, when you develop some apps, you have to keep in mind that you prepare something for geometry but it’s not what you draw to screen.
  • JG: Only thing worse about that is that neither is particularly better than the other, and most of the APIs have gone with NDC Y up. Vulkan is the odd one out but it looks like they added an extension to fix it. It would be a big change for us to make NDC Y Down.
  • MM: Yea, I think that if Vulkan added that option, it would be appealing to make it nicer for WebGL developers.
  • RC: agree
  • DM: do they need that appeal? Isn’t it a matter of us giving a matrix that would do the conversion for most cases? Aside from uses of gl_FragPosition
  • JG: want to nip in the bud as much fragmentation as possible. Would like to not have this fragmentation between us and the native APIs. Should do the same as what everyone’s doing.
  • JG: more interesting to make better choices when APIs don’t agree. In this case almost everyone agrees, maybe today everyone does agree on NDC (Vulkan 1.0 being the odd one out).
  • DM: can someone clarify where Jasper’s position fits? He suggests Y-up, Y+ down coordinates. Says it’s what D3D12 and Metal use. Confusing.
  • JG: want more time to go over the document.
  • DJ: just linked to investigation from github.
  • MM: say we’re consistent with WebGL, D3D12, Metal and OpenGL on NDC. Then question: is it more valuable to be consistent with that decision for the other types of coordinates, or try to make more sane decisions? If imp’t that all coordinates agree, don’t decide just on NDC now, but all of them.
  • KN: don’t think it’s that critical they all match given that none of the native APIs have them all match.
  • MM: think that the fact the native APIs disagree with themselves means we can too.
  • JG: agree. Think we’ll likely have NDC up and everything else down. Want to look at tables more offline.
  • DJ: Mr. Doob gives conflicting information too. Let’s give ourselves another week on this. Meantime, continue commenting on Github issue.

Bindless Resources #380

https://github.com/gpuweb/gpuweb/issues/380

  • MM: not lot to discuss. Imp’t to get something down for the minutes on this topic. Metal and D3D have natural ways to express bindless. Vulkan has 2 extensions that are similar, but not exactly the same. On Android, one of those extensions has 0% popularity and other has 1%, so can’t do it in core. Might want to do it as extension, but probably don’t have to do before the MVP. Bindless is desirable, worth at least talking about, see if we can do it, but maybe not right now.
  • DM: I agree. Still didn’t get answer: how to implement on Metal? If can’t use UseResource in advance.
  • MM: if extension, we could enable it on some machines and not others.
  • DM: you don’t know what bindless resource will be used.
  • MM: Both D3D and Metal have 2 pieces of API. Facilities for passing around descriptors, and for app authors to say what might be used in the shader. Metal: UseResource, D3D: MakeResident. Would have to expose same thing in the API.
  • DM: thanks.
  • DJ: could you add that as a comment on Github?
  • MM: sure.
  • DJ: thoughts on this as an extension? Esp. on Android? 0/1% accurate?
  • KN: think it’s accurate. Probably NVIDIA Tegra supports this. Hard to specify. Won’t reach Android for some time. Not priority for us.
  • DJ: are you open to the group developing the extension in parallel?
  • KN: if no specific motivation for it now, we should be able to put it off till post-MVP. Not that critical for developers. Won’t get on Android devices anyway.
  • MM: advantage: you can access lots of resources in shaders. 4 bind groups. What’s limit on resources in bindgroups? If high, need for bindless goes down.
  • AE: either 16 or 32 but haven’t investigated what the limits should be.
  • KN: 31. (?) Not huge.
  • DJ: is that big enough?
  • MM: it’s OK for a start. Related issue: to make bindless safe, each access to bindless resource has to be gated with “exists && is_resident”. Every access will have to effectively do a binary search through list of resources passed in out-of-band. Tight array accesses in loop will each have to do the same thing. Maybe perf is so bad it’s not worth it.
  • KN: maybe can be done differently but still massive amount of work.
  • DM: also portability issue. Depending on runtime, may have validation fail at shader invocation time, which isn’t the case with current bindgroups.
  • MM: semantics would probably be the same as out-of-bounds array access.
  • DJ: what are next steps? Hold off on this until after 1.0? Leave the issue and proposal open? Not have as work item for a while?
  • MM: would like to know if Android people think we should never do this.
  • KN: think we should base it on developer requests. If people want it on desktop regardless of mobile support, we should enable that.
  • MM: OK.

Generate Mipmaps #386

https://github.com/gpuweb/gpuweb/issues/386

  • MM: WebGL and OpenGL and Metal have built-in ways to generate mipmaps from the largest mip level. D3D has a library by MSFT that does it, but not part of Windows SDK. Vulkan doesn’t have anything. I think having something like this would be valuable for WebGPU, mainly because it doesn’t restrict what author / 3p library could do (can do your own still), but built-in way will be easier for smaller developers to get things working. Our demos don’t look good because there’s no easy way to get mipmaps for a texture. Something we’ve run into, heard explicitly that other 3p devs are running into. On Github the WebGL version is called 46,000 times so clearly it’s useful.
  • JG: would they not just include a little JS module that does it for them? Seems the cleanest answer to me. How do you want to do your filtering? Fastest possible, or Lanczos filtering? Linear / sRGB aware? Lots of questions. Easier to leave it to a module. Not particularly complicated.
  • MM: don’t think we should let perfect be enemy of good. When people
  • KR: Should implement it as a module. Easy to use. Bugs can be turned around quickly - don’t have to be fixed in browser. Uses the low level primitives we provide in the API.
  • MM: lower-level primitives in the API are hard to use.
  • KR: If this is a library, it won’t be hard to use. It’ll be super easy. It’ll be just as easy as if it were built into the browser.
  • DM: you would still need to initialize the context of the library. Initialize the shader.. etc.
  • JG: this is standard JS injection stuff.
  • DM: I think a library would be great. Would also be nice for users to generate mipmaps in their own code. BlitImage primitive?
  • JG: stretch-capable copy?
  • DM: yes. We’d document: target must be renderable, source shader-readable, filterable, and would run shader for you. Wrapping mode, etc.
  • JG: also filterable.
  • KN: yes. Has been on my mind too. If we had blit then GenerateMipmaps would be a really small function.
  • JD: impl could be very complex. Combinations of formats / etc.
  • JG: this is where composability would be essential. Sometimes easier to impl things as polyfills rather than in-browser. However, if a prototype wanted to expose a feature-detectable function for it, this is an easy function to feature detect.
  • KN: Could even be an extension / Layered API. Probably not extension.
  • MM: if extension, would have to have fallback path. Doesn’t help.
  • KN: right. Layered API would require them to have some impl.
  • KR: Do you think this type of stuff should bake inside a user-level library for a while? Chrome for example has CopyTextureChromium which is like a stretch rect. It has been a huge task to maintain and support this for all the various formats. It’s a lot of code and as soon as a user runs into a problem, they have to do a workaround anyway. It’s a lot faster to fix things that are not shipped in the browser.
  • RC: tend to agree. Need to separate things between “core” and “extensions”. “Extensions” are something not available in hardware. CSS has module concept. Big chunk of functionality all becoming available at once. If not in V1, we take heat, we add to WebGPU 2. Prefer for MVP that we leave this as helper function, and stick to things web devs cannot do.
  • DJ: MM you seem to be the only one arguing for it to be part of the API?
  • DM: how would it be a single function?
  • MM: use the Metal model, making it a Blit command (on command buffer itself, in WebGPU), taking resource that only has its biggest mip level full, and populates the others.
  • DM: understand. How to expose as single function outside the API as a user library? What would it take?
  • JG: can take in all args you have to, including device. Implicitly, can polyfill and attach as method on the WebGPU prototype.
  • KN: would expect it to be standalone: takes device, command encoder, and resource you need bound / mips generated for. Wouldn’t have standard API so you’d have to pass in everything you need.
  • MM: in your idea of blit function would that be more generally useful, or only for generating mips?
  • DM: I would hope it would be more generally useful. I think people use Blits to convert between formats quickly. In GL and Vulkan it is built-in (though we probably can’t use the built-in impl in all cases).
  • MM: maybe that’s a good solution to this problem. If it more or less solves mipmap use case and also handles format conversions maybe that’s a good compromise.
  • JG: if it provides extra perf that’s interesting. Worry though, longest sets of validation in WebGL is BlitFramebuffer. Hundreds of validation lines of code. Not trivial. Some of this is because WebGL’s blit is for a bunch of uses, so if this is special-purpose that’s better.
  • MM: About performance, the fact that if it were built on top of blitting, in Metal it could operate in parallel to a compute command encoder, so it could potentially be faster on some devices. Because it’s not a compute / graphics shader.
  • MM: maybe way forward is to come up with blit proposal.
  • DM: I would like someone to suggest a mipmapping piece in JS for reference.
  • DJ: version 1 that’s a standard library function.
  • KN: with shaders, generating mipmaps.
  • JG: shaders are a pain point right now.
  • MM: right.
  • JG: I helped the Moz Hubs team with their custom damage-rect GenerateMipmaps function. Can write a skeleton, “here’s what shader would do”, would that help?
  • JG: I can write something in WebGPU pseudocode.
  • DJ: why don’t you just write the API code and a shader in GLSL, and we can look into it. Thanks. Let’s assume this is an AI for you, and MM / DM will propose a blit function that we’ll discuss separately.

Investigation: Copy With Compressed Texture Format #363

  • JG: still needs work. Anyone want to look at this? Think most people don’t care. Maybe people want to see a proposal instead of reading the investigation?
  • DJ: in nearly case think it’ll be more actionable.
  • KN: there is a small proposal here in the last section. Think usable as-is.
  • JG: but there’s a problem with the proposal: main difference is, tex-tex copies and how they’re defined for block-based texture formats. Core issue: alignment of coords for copies. D3D / Metal have to align to multiples of block size even if they’re at edge / off edge of texture. Second mip level, of 60x60 texture, is 15x15 logical texture. In D3D / Metal, way you copy in would be: copy in to 16x16 group of blocks. In Vulkan, you have to align end size to either alignment of blocks, or size of texture. Incoming copy can’t be bigger than target size. Source would be 15x15 image. Can’t copy from 16x16 into 15x15 level 2, it sounds like.
  • AE: you can only copy multiple of 4 sizes?
  • JG: how do you handle that with non-multiple-of-4 sized images?
  • KN: does Vk really not allow this?
  • JG: there’s a link to an issue in the Vk spec.
  • KN: #999.
  • JG: sounds like this kind of copy is disallowed. Assertion that this isn’t bug in spec but design decision. Since we have this constraint, seems we can impl Vk-style restrictions in copies on D3D12 and Metal, which use physical size validation, but not other way around.
  • KN: does anyone know how useful it is to do texture-texture copies of compressed textures?
  • JG: probably uncommon.
  • KN: what if you want to make another copy of a texture you already have? Compressed, but not multiple of 4? Want to go do level 1 of other texture? Can’t think of reason you want to do that.
  • AE: whole texture or part of it?
  • KN: part of it.
  • AE: if it’s the entire mip it’s OK. Only sub-region has to be aligned to 4.
  • JG: bulk intervals.
  • KN: don’t know what compressed-to-compressed texture copies are good for.
  • JG: atlases maybe? Not sure. Couldn’t copy from 3x3 into 124x124.
  • KN: copy from image into atlas or vice versa?
  • KN: find it weird to expose something so limited when we don’t know how useful the whole feature will be.
  • MM: this doesn’t let you encode / decode compressed textures?
  • JG: no.
  • MM: doesn’t let you copy between textures of different compression types?
  • JG: no.
  • MM: wonder why this is useful.
  • KN: texture -> buffer -> texture if you want. Actually, can’t, because texture source has this limitation.
  • AE: think you can do that. Had pull request into Dawn. Not sure it’s correct.
  • KN: still can’t copy out of 15x15 mip level.
  • AE: this example, copying from 16x16 region, going into 15x15 region. Maybe that’s OK?
  • KN: not sure.
  • JG: one possibility, forbid compresssed texture -> compressed texture copies?
  • CB: you can, but just don’t let them cross a tile boundary / block boundary. Just tile boundary, should be oK.
  • KN: is it useful?
  • CB: not useful. Most uses of compressed textures are different from those modified at runtime.
  • KN: we could punt this, not allow for now.
  • CB: sounds OK for the MVP.
  • JG: sounds OK to me.
  • DM: are you forbidding texture -> texture copies?
  • KN: no, only compressed.
  • DM: still buffer -> texture?
  • KN: yes.
  • MM: unless new ways of uploading not including buffers.
  • KN: if we added a way to do that would have all the same problems, but maybe could add it.
  • AE: can we say, both mips have to be multiples of 4?
  • JG: think you have a complete mip chain that way. Should be able to force people onto texture -> buffer -> texture if we want to do this.
  • KN: then have to make sure that works.
  • JG: seems easy to me. Main problem: collision between logical & virtual sizes. When doing copies, don’t allow stretching, so width/height have to be same. That’s the problem. texture -> buffer -> texture, you have 2 different rects.
  • KN: that does work on Vulkan yes?
  • JG: think it has to.
  • KN: otherwise you couldn’t upload compressed texture mip levels.
  • DM: don’t you still have the issues of alignment, or using the full mip plane? Think we should have those restrictions.
  • JG: that’s the Vk way, could do it that way and enforce the restrictions.
  • DJ: punt from here. Think we should start with PR burndown next week, and come back to this issue afterward.

Agenda for next meeting

  • JG: isn’t the 19th our pre-shader day?
  • DJ: I’m open to sticking with pre-shaders.
  • KN: think we have to.
  • JG: would be nice to have 15 min for PR burndown still.
  • DJ: good idea. Next week, 12:15 will be start of shader discussion.
Clone this wiki locally