Supporting open source models via Anyscale Endpoints

[richardliaw]

Hey! This is a super cool project.

I noticed your comment on https://news.ycombinator.com/item?id=38285191 about supporting open source models. I'm from the Anyscale team and we offer an OpenAI compatible API for both fine-tuning and serving for the most popular Open Source models (Llama-2, Codellama, Mistral, Zephyr).

Perhaps this would be a fast way to support open source models for monkeypath.py?

[JackHopkins]

Hey Richard, This is cool, I’m definitely aware of your project. Let me chat with the guys and figure our how we can make this work. It would be great to put out a case-study blog together if we can get it to work too! Cheers, Jack

…

On 16 Nov 2023, at 15:30, Richard Liaw ***@***.***> wrote: Hey! This is a super cool project. I noticed your comment on https://news.ycombinator.com/item?id=38285191 <https://news.ycombinator.com/item?id=38285191> about supporting open source models. I'm from the Anyscale team and we offer an OpenAI compatible API for both fine-tuning <https://docs.endpoints.anyscale.com/fine_tune/files_api#using-the-openai-sdk> and serving <https://docs.endpoints.anyscale.com/guides/migrate-from-openai> for the most popular Open Source models (Llama-2, Codellama, Mistral, Zephyr). Perhaps this would be a fast way to support open source models for monkeypath.py? — Reply to this email directly, view it on GitHub <#74>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA5T6OWV6RA24AXB5J6EZDLYEWXJ5AVCNFSM6AAAAAA7NUAFLCVHI2DSMVQWIX3LMV43ASLTON2WKOZRHE4TMMJVGY3DSMQ>. You are receiving this because you are subscribed to this thread.

[richardliaw]

For the joint case study, certainly can do! On Thu, Nov 16, 2023 at 4:51 AM Jack Hopkins ***@***.***> wrote:

…

Hey Richard, This is cool, I’m definitely aware of your project. Let me chat with the guys and figure our how we can make this work. It would be great to put out a case-study blog together if we can get it to work too! Cheers, Jack > On 16 Nov 2023, at 15:30, Richard Liaw ***@***.***> wrote: > > > Hey! This is a super cool project. > > I noticed your comment on https://news.ycombinator.com/item?id=38285191 <https://news.ycombinator.com/item?id=38285191> about supporting open source models. I'm from the Anyscale team and we offer an OpenAI compatible API for both fine-tuning < https://docs.endpoints.anyscale.com/fine_tune/files_api#using-the-openai-sdk> and serving < https://docs.endpoints.anyscale.com/guides/migrate-from-openai> for the most popular Open Source models (Llama-2, Codellama, Mistral, Zephyr). > > Perhaps this would be a fast way to support open source models for monkeypath.py? > > — > Reply to this email directly, view it on GitHub < #74>, or unsubscribe < https://github.com/notifications/unsubscribe-auth/AA5T6OWV6RA24AXB5J6EZDLYEWXJ5AVCNFSM6AAAAAA7NUAFLCVHI2DSMVQWIX3LMV43ASLTON2WKOZRHE4TMMJVGY3DSMQ>. > You are receiving this because you are subscribed to this thread. > — Reply to this email directly, view it on GitHub <#74 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABCRZZJDFWOH5YWYRL6FTF3YEYD3RAVCNFSM6AAAAAA7NUAFLCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMJUGM3TSMJUGA> . You are receiving this because you authored the thread.Message ID: ***@***.***>