-
Notifications
You must be signed in to change notification settings - Fork 110
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fused RMSNorm incompatible with PP tracing (dynamic stride) #217
Comments
short term is the stride check can be removed to explore tracing (this check is rarely needed, confirmed on llama_7b). Longer term this will either need a refactor to support dynamic strides (harder) or given the rarity, just a simple assert that we don't support non-contiguous. |
I did not look into this closely, but could we rely on |
The incompatibility is that during backwards, fused_rmsnorm does dynamic control flow over strides, which isn't safe for export tracing used by PP.
Which leads to a stacktrace ending in
Would it be possible to refactor this in a more export friendly way, or is that difficult?
cc @lessw2020, @kwen2501
The text was updated successfully, but these errors were encountered: