Running demo code on V100

[mikeleatila]

Hi,

Congrats on the great work!

I only have V100 GPUs available to me.

Is there a way to run your inference/demo code and how (e.g. with no flash attention)?

Many thanks in advance!

[lxtGH]

@zhang-tao-whu Check the issues. It seems like you need to modify the code to remove the flash attention.

[mikeleatila]

@lxtGH @zhang-tao-whu Thanks a lot. Do you have some hints how to remove flash attention? Can it be done by "passing-a-parameter-kind-of-thing"? Been looking around and trying out but it still does not work.