Skip to content

WGSL 2020 07 21

Jump to bottom
François Daoust edited this page Dec 2, 2020 · 1 revision
Dan sinclair

:

 Chair
Mehmet Oguz Derin

:𐰆𐰍𐰔 :

 ⌨️ Scribe
Google Meet

:

 Location
https://webgpu.dev/wgsl

:

 Specification
WGSL Issues

:

 Open Issues
Marked Issues

:

 Meeting Issues

Tentative Agenda

  • Add stride array decoration. (#910)
  • NumWorkgroup support (#920)
  • No explicit grammar for decorations (#689)
  • C-like declaration order (#677)
  • Vertex and Instance ID and base ID builtins. (#901)
  • Use brackets for array types (#854)
  • Validation rules for WGSL modules: binding collisions (#889)
  • Constant propagation (#905)
  • Interface matching rules (#644)
  • Extern declarations (#883)
  • Forbid --x (#801)
  • Invariant qualifier (#893)
  • Support #line directive (#606)
  • Function overloading (#876)
  • Shader portability (and performance portability). (#895)

📋 Attendance

WIP, the list of all the people invited to the meeting. In bold, the people that have been seen in the meeting:

  • Apple
    • Dean Jackson
    • Fil Pizlo
    • Myles C. Maxfield
    • Robin Morisset
  • Google
    • Dan Sinclair
    • David Neto
    • Kai Ninomiya
    • Ryan Harrison
    • Sarah Mashayekhi
  • Intel
    • Yunchao He
    • Narifumi Iwamoto
  • Microsoft
    • Damyan Pepper
    • Rafael Cintron
    • Greg Roth
    • Michael Dougherty
    • Tex Riddell
  • Mozilla
    • Dzmitry Malyshau
    • Jeff Gilbert
  • Joshua Groves
  • Mehmet Oguz Derin
  • Timo de Kort
  • Lukasz Pasek
  • Tyler Larson
  • Lukasz Pasek
  • Pelle Johnsen
  • Matijs Toonen

⚖️ Discussion

Add stride array decoration. (#910)

  • DS: any more input on array stride decorations?
  • MM: One thing we should consider is multi-dimensional arrays
  • DN: They're arrays of arrays so compose just fine
  • JG: Would need to compose composite types?
  • DN: The element of the outer array is another array. If you want strides on that you can have type defs.
  • JG: Would it look like stride 64 array<stride 16>
  • DS: No: type foo = stride 16 array<f32>; type bar = stride 64 array<foo>;
  • JG: That's what I meant by composite types
  • MM: So we're alreayd putting the stride 64 on the right of the equals in a type def so that indicates the stride 64 is part of a generic type decl. So, it should be able to be anywhere else in the program
  • DS: Would depend on the grammar
  • JG: Is stride a proprty of type or variable
  • DN: type
  • MM: Can I make a function where the return type has a stride.
  • DS: No
  • MM: But i can put it on the right side of a type def, why are these different
  • JG: You coudl do it as part of the type
  • DS: No, it's a decoration on the type def
  • MM: So how does multi dimension array work
  • JG: Multiple lines
  • MM: One typedef per dimension, sounds like a lot of typedefs
  • JG: Not super common
  • MM: So why not do the code I mentioned
  • JG: Depends where stride goes in the grammar. If stride is part of the type then it's an optional preamble to defining the type. As long as you define the type you can return with these inline decorations. Then you coudl do it without a typedef. If that was true, you could do stride 64 array<stride 16 array<>>.
  • MM: Makes sense to me, then you couldmake a typedef where that big thing is on the right side if you dont' wnat to type the type
  • JG: Big question is if that breaks anything else. Nice from a purity standpoint to be able to say where the type is you specify the type
  • DN: Similar to structs. Where before we had type name = struct and we changed it to struct name. Would be in the same spirit as that change. On Myles' comment (on the PR) woudl be the 2nd bullet point.
  • DS: We could move it in the grammar, but need to think if there is anything weird in the grammar
  • MM: Works out well that we have the return on the right so you know what hte stride attaches too
  • DS: Will update the PR to move the stride into the type_decl.
  • DN: Has anyone progressed on checking the grammar for non-ambiguity.
  • MM: Something we'll have to do before MVP. Apple has some experience doing this as we did it for WSL. We can take an action item to do that
  • JG: We stalled out on doing it in pure markdown to do it in the CI and never finished up that work.
  • MM: Another thing we can do is use on matrices for row and column major
  • DN: SPIR-V is funny and choses the opposite. Those are part of the struct member not hte type. Want's numeric types including matrices to be unique so when you load it, it's a matrix without a stride
  • MM: In SPIRV arrays are value types
  • DN: Yes
  • MM: So why the difference
  • DN: Arrays are not numeric. Design choice. Having stride on array gives type safety on expressions when doing address arithmetic. Convenient for the opposite reason to not have a buncho f strides on vector and matrix types. If we choose to make the layout an attribute of the matrix type we'd have to do value conversions at boundries. It's a cost, but debatable.
  • MM: That explanation is quite un-intuitive. Seems what we pick should be the same for arrays and matrices. Intuitively they're very similar types.
  • DN: Will elaborate what SPIR-V does in the issue. If we got your route, would need a value conversion that makes the pure type out of the layed out type.
  • MM: If someone gave you a SPIR-V program to turn it in to WGSL, you have to figure out which strides refer to each place the type appears in the program. The other direction, WGSL to SPIR-V get rid of the strides and uniquify the types.
  • DN: Will let you know. I think it's just a value conversion from the memory to value domain. Should be convenient enough. Already have sign vs unsigned distinction and not that bad.
  • MM: So direction is David will try to implement strides on matrices the same as on arrays and see how it goes and bring back to the group?
    DN: Yes,will try it out with paper and pencil to see what happens
  • MM: What are the matrices decorations. Majorness and stride, is that it.
  • DN: Yes.
  • MM: If you want both stride 2, row_major ?
  • DN: Sure. Did an exercise looking at the CTS GLSL shaders, both row and column major are used.
  • MM: Sounds like a good direction
  • DM: What strides are in MSL and HLSL, are matrix strides in there?
  • MM: Dont' know the answer to that. Will take an AI to look into that. The answer maybe no.
  • GR: In HLSL there are some restrictions but not sure what they exactly are.
  • DN: The question is what are the constraints on the strides of matrices in HLSL and DXIL. The number of bytes between vectors if they are column and row
  • MM: So can I have a mat2x2 where the number of bytes between the two vectors is 40.
  • DN: Yes
  • DM: Is this clear for arrays? What stride do we support for arrays
  • DN: Rules oncompluting the base alignment. Array stride must be a multiple of the base alignment.
  • MM: In SPIR-V is there a difference between size and stride
  • DN: Yes. In LLVM there is the store size and allocation size which are different. The store size of a 3 element vector maybe the 3 elements. The stride would be a vec4. Store size is how many elements for a memory element, the stride depends on the number of things in there. Different on the end.
  • DM: Suppose array<T> with base alignment of 5, why put a stride of 10 instead of just T*2.
  • MM: I like this direction
  • DN: Confused, if I have ?
  • DM: Is there a value you can use in the multiples which you can use instead of the exact values in teh stride?
  • DN: Typically uniform buffers have bad array support where every array element requires multiple of 16. Hardware maps to vec4. Would have to map other types to that constraints which is why we got explicit strides. Even with vec2, base alignment is 8, but requires stride of 16.
  • MM: But what about a stride of 32?
  • DN: Viable, but possibly not a good idea. Maybe trying to avoid cache lines? Not necessarily in a GPU context, but multi-CPU context.
  • DS: So I will update PR. DN will try row and column in SPIR-V. MM will look for matrix requirements in MSL. GR will look at similar in HLSL/DXIL and Apple will look into validating grammar on CI.

NumWorkgroup support (#920)

  • MM: We should have it
  • DS: Talk to HLSL
  • DN: Is there a thing to tell you how big the entire grid is in HLSL
  • DS: SPIR-V has NumWorkgroups, MSL has grid_size, HLSL doesn't appear to have the equivalent.
  • MM: Just to clarify this isn't setting the number of threads, but knowing the number
  • DS: Yes
  • MM: Useful in MSL, should be emulated in HLSL by passing in extra values. If the shader never reads the extra values you'll know at complication so don't have to pass them in.
  • DN: As a general principle can extend to other things
  • MM: Case by case.
  • DS: Does that have API ramifications
  • DN: If you take a descriptor or binding slot for the datum.
  • DM: If we have auxilary data with unbound array sizes we can add to that buffer but not exactly sure what we'll have. Maybe not have this builtin for now and consider for later when it's more clear the cost. User can do it themselves anyway.
  • DN: Depends. You can write a shader where you don't know the max workgroup size. The system picks the workgroups size in which case, done by the system after the submit and can't touch the command. So, a little iffy.
  • DM: Woudl be easier with push constants.
  • DN: We could emulate those … Maybe in the push constants domains. Nice to have, maybe not MVP
  • GR: Confirmed there is no builtin for HLSL along those lines. Could be passed in a constant buffer or something, not free though.
  • MM: In MSL you see this all the time. Do you not see it all the time in HLSL. The reason MSL has the builtin is that it's super common. Is it not common in HLSL
  • GR: Shrug.
  • DM: Can you bring some common examples to give intuition?
  • MM: Sure, particle system where they didn't do the sort optimization. Just a bunch of interacting particles and you have to iterate over the whole grid.
  • DN: Are you skipping over the number of elements in the grid because you want to know which work is yours?
  • MM: Was thinking grid_size
  • KN: Like an n-body simulation
  • MM: Yes that's what I was thinking about
  • DN: Dont' have this in my head, example link would be useful.
  • MM: So, there are a bunch of stars, represented as points with mass/velocity. There are few enough of them the author doesn't optimize the shader. The forces interact between each pair. Updating a star has to iterate over all stars, calculate force and add. Each shader invocation has to iterate over all other stars. To know stars needs to know grid size
  • DN: That's data size not compute parallelism size
  • MM: Yea, simple implementation where those are the same
  • DM: The boids example iterates in that manner. Does it use num workgroups?
  • KN: I'm pretty sure the size of the array is hard coded
  • DM: I'd expect a constant saying your data size to allow early exit treads outside of data size. So, grid_size doesn't help in this case.
  • DS: So, any objection to reserving the word and punting to after MVP?
  • GR: Sounds fine to me
  • DS: Will take that as an AI to make a PR.
  • JG: Super convenient for GL and WebGL to have whole namespaces reserved so don't have to worry about this. Anything starting with gl_ is reserved
  • MM: Did we talk about this when talking about namespaces?
  • DN: We have a proposed forward direction #600. Talked about various options, in favour but hasn't been added yet.
  • DM: Slightly confused, reserving the word from used anywhere? What about naming as a variable?
  • JG: Builtins are not in the grammar now.
  • MM: If we can agree there will never be user defined declarations we don't need to reserve anything. Looking at #600 there are 7 options and no consensus.
  • DN: Probably why we haven't moved forward yet. Not cemented.
  • DS: Makes easier. Remove for now, file bug to determine what to do post mvp.

No explicit grammar for decorations (#689)

  • MM: Don't think this effects anything. Don'tthink this effects written shaders, maybe error messages. No prior valid shaders will be invalid and no prior invalid shaders will be valid.
  • JG: That's right.
  • MM: Seems editorial
  • JG: Disagree. What point are we making spec and what point it becomes editors. If the WG is describing what they want and editors figure out details of grammar then yes. This does change error messages and feels more then editoral.
  • RM: Don't specify exact error messages so don't see anything that would prevent implentaions from making the shader. Some are type rules, some are grammar rules seems more like a presentation thing. An implementation can do it in however many passes it wants with what messages they want
  • JG: See what you mean but generally speaking it's a bad idea to hand parse your grammers. We'll probably have a grammar parser to parse for us. Trying to inject detailed error messages is trickly. Yes can do it but then keeping diffs between grammar and ours. If there is no good reason to put this in the grammar, shouldn't. Simpler grammars are better. Currently validating twice, once implicitly as partof the grammar and then after the fact. Woudl rather do single validation pass and not say it's in the grammar. If you really wanted to tie this down int eh grammar could do that. But, making the grammar more restrictive as opposed to artifically relaxing to make more error messages.
  • RM: Fine with this change, was trying to explain whether it is done or not won't affect implementations as they can go either way.
  • MM: No making argument as for or against.
  • JG: Same spirt, this doesn't mandate a change, pushes for a change in the implementation. Would implement differently depending on how this went.
  • MM: Agreeing it doesn't change programs. Can give summary as to why this is better?
    JG: Main thing, simplified grammar. Had to understand for humans. Have to do validation anyway at validation phase not parse phase. In the grammar, any time we add we modify the grammar which we'd like to avoid. When in the grammar we're also validating at parse time along with validation time. Why not skip parse validation? Also compelling as it's easier for a simple yacc based parser to have simple error messages if it isn't' part of the grammar. Would pull grammar directly from the spec. Arguments for the counter-factual? Where it isnt' part of the grammar and you want to make it part of the grammar? If not, should take the minor improvement.
  • DM: Is there a cost to changing the grammar?
  • JG: Have to re-evaluate the grammar and thinking about the grammar can be difficult. Adding a new decoration grammar thing isn't difficult but it's easier if we don't have too. If it stays in the grammar and we want to add to it, we have extra busy work to add to the grammar and validation. If it isnt' in the grammar we just have to change our validation.
  • DM: So, the validation logic will happen regardless. Either done by teh grammar or parser or done by the later logic.
  • JG: Done by both
  • DM: But if the grammar validates in this spot i can use array stride don't have to do it later
  • JG: These are lists with multiple valid options. Even though builtin is part of the grammar, even though frag_coord is there we have to validate it's there in that situation.
  • DN: Or that you don't se same binding value twice or inconsistently
  • JG: Multiple location values.
  • DM: Don't have an objection. Don't see a big difference. Defer to other peoples decisions.
  • DN: Neutral. Could go either way, hear what JG is saying and agree the validation has to occur in any case.
  • RM: No strong opinion. I'm fine with it.
  • DS: Will need validation that you can't have multiple builtins. We'll probably still create the types in Tint, but doesn't matter if they're explicitly in the grammar.
  • JG: Will fix merge conflicts.
  • DM: Naga also has AST knowledge of the builtins as they're context specific
  • JG: Does it get harder if they aren't there?
  • DM: Would do what we do today. Won't' change behaviour if the builtins are in the grammar or not.
  • DS: I think it's fine to take.
  • JG: Will rebase.

C-like declaration order (#677)

  • DN: Im' against it. We have nice, composable, regular system.
  • DM: Just discussed it would be good for array strides. So good reason to close.
  • DS: Myles, as initial filer, any comments?
  • MM: A couple. One, we can't do this later. If we're going to do it, must do it now. Deferring is not possible. Either do it early, or we don't do it ever. That's non-controversial. The benefit is it makes it easier for folks with existing source code in a c-like language and don't want to use a compiler for simple find and replace. This happens, people do it. The downsides are, 1- would be easier to introduce parsing ambiguities, like mentioned earlier. 2- WGSL would not be in line with modern languages (swift, rust, js). So, reasons on both sides. Professional developers will use whole compile. Non professional it will be more difficult for them to introduce a new compiler into their tool chain. Synthesis is we've already got a lot of momentum in the group. Making good progress. Some implementations are close to shipping. Given there isn't a clear winner, and good arguments on both sides. Should try to continue momentum and not redesign grammar.
  • DN: Thank you Myles for good summary and good debate to push forward.

AIs

  • AI(DN): Will try row and column in SPIR-V.
  • AI(MM) Will look for matrix requirements in MSL.
  • AI(GR) Will look at similar in HLSL/DXIL
  • AI(Apple) Will look into validating grammar on CI.
  • AI(DS): Move stride decoration into type_decl
  • AI(DS): Remove NumWorkgroups from grammar, update bug to handle post MVP.
  • AI(JG): Will update builtin PR to fixup merge conflicts.

Next meeting

  • Use brackets for array types (#854)
  • Matrix row/column stuff as blocking CTS.
  • Vertex and Instance ID and base ID builtins. (#901)
  • Constant propagation (#905)
Clone this wiki locally