Enhancement: Using koboldcpp to compare 2 model outputs isn't very efficient. Is there an easy way to load two models at the same time?

[Laince1212]

Sorry I'm a bit of an amateur here, I want to load 2 models at the same time to compare giving them the same exact input and see their outputs are. Currently the only way to do that with Koboldcpp is to load the two models using two instances, use a different port for the second instance, open two browser tabs and copy the input between each after changing the settings for each one to match. It's not very efficient.

I feel like a perfect example would be LMSYS chatbot arena where you can compare two models directly with the same input and same settings to see which outputs better. I feel like if Koboldcpp supported this it would give beginners like me a way to compare models directly and also help anyone that wants to use two models at the same time for any reason not just to compare the two. My request is: A lmsys chatbot arena like UI with the ability to load two models at the same time. Both models should get the same input then either generate outputs one after the other or both generate (probably slower) outputs at the same time.

Finally, Thanks for working on this amazing program, it's the best portable program to run GGUF files and literally the simplest to use. I switched to Koboldcpp after using many other apps and have been using Koboldcpp for months due to its extreme ease of use and portability. I know my request requires alot of work but hopefully it's going to be in a future update for everyone to try out. Having a local chatbot arena koboldcpp feature would be wonderful.

[LostRuins]

Hmm not sure how I can implement that easily though. If you are running one model after the other, swapping them out and clicking Retry might work. Otherwise, the dual window thing seems to be the best bet for now

[Laince1212]

Yeah it's definitely not an easy ask, but anyways thanks for this amazing app. I'm glad that ease of use and portability is the top priority here and it definitely shows. Used it for months now and have had a great experience. Hopefully after the more important issues here, there can be a way to compare models directly. Absolutely marvelous work though!