Is that we should fully use all 70b parameter? I mean can we use some skill like discrimination to make it smaller Or use another some small model to help its inference Or separate the whole big model into some small part and use Moe to help it move faster?

[Kevin-shihello-world]

No description provided.

[ASC-Competition]

@Kevin-shihello-world
Yes, the challenger need use all 70B parameters and is not allowed to reduce it or change the model architecture.

[Kevin-shihello-world]

Can we change it a little bit? I mean it hasn't written into the rules of this competition. I mean we can for sure that we won't change the model architecture too much and we would not reduce the total parameter.
I mean some structural Changes like Moe sounds very Close to the meaning of this competition and by change the architecture of it a little bit we can Know more about how to deal with the changed model Like how to assign memory for flexible architecture like moe.
And by the way, I also want to ask something about the fourth question in the competition. I mean it said we can not change any code related to the parameters of the software. So does that means we can't change any code of that software or we just cannot change the those codes related to the parameter?
I would still appreciate it if you could answer the questions properly To the committees of this competition and I think it means a lot to we competitors
And also in the notification of this competition, it wrote that: ' we encourage participants to begin with the baseline code and build a tailored'？ If any code related to architecture of that llm Can't be changed why it said it would like us to build a tailored code？ If we do things like this, we can not even use windows sliding skills to accelerate the model a little bit. We participants were learned a lot for this competition and tried hard for the development on the llm inference and other Super computer Challenges and if we were restrained Not to use any structural code to take this Challenge. It would highly depress us .So we'd be appreciated it if you could ask The organizers of this competition detailedly. Thank you for your time.

[Kevin-shihello-world]

And I apologize it that last time I didn't review it carefully for the question and some reply After I transform those spoken language to words

[ASC-Competition]

Can we change it a little bit? I mean it hasn't written into the rules of this competition. I mean we can for sure that we won't change the model architecture too much and we would not reduce the total parameter. I mean some structural Changes like Moe sounds very Close to the meaning of this competition and by change the architecture of it a little bit we can Know more about how to deal with the changed model Like how to assign memory for flexible architecture like moe.

I am sorry but the answer is no. The point of the challenge is inference optimizations and strategies, MoE is not included.

And by the way, I also want to ask something about the fourth question in the competition. I mean it said we can not change any code related to the parameters of the software. So does that means we can't change any code of that software or we just cannot change the those codes related to the parameter?
I would still appreciate it if you could answer the questions properly To the committees of this competition and I think it means a lot to we competitors

It's afraid that the fourth question is not about LLM inference and I cannot anwer your question. You can contact the concerned person by email [email protected]

And also in the notification of this competition, it wrote that: ' we encourage participants to begin with the baseline code and build a tailored'？ If any code related to architecture of that llm Can't be changed why it said it would like us to build a tailored code？ If we do things like this, we can not even use windows sliding skills to accelerate the model a little bit. We participants were learned a lot for this competition and tried hard for the development on the llm inference and other Super computer Challenges and if we were restrained Not to use any structural code to take this Challenge. It would highly depress us .So we'd be appreciated it if you could ask The organizers of this competition detailedly. Thank you for your time.

You can change the baseline code or just start from strach to build a inference engine which proper to your machine system. The ASC24 committee does not set too many limits and kinds of methods can be used as long as it is clearly presented in the proposal.

[Kevin-shihello-world]

Okay, thanks all organized for this competition！And I also wants to know Can we add a little bit permit to it? I mean like sometimes I may thought we can use an auto encoder to encode Some data and make them smaller. So as we can deliver it with cheaper bus price in distributed system when inference .
Thanks you for your time. And wish you all a precede merry Christmas!

[ASC-Competition]

@Kevin-shihello-world if you mean to truncate or shorten the samples, the answer is it's not permitted for fairness.

BTY, if you have any question you can contact the email [email protected]. It will be replied as soon as possible.