Integrating Arch with Microsoft Word through a local Word Add-in

[GPTLocalhost]

Is there any document or sample code for the following? I'd like to create a demo on how to integrate Arch with Microsoft Word for personalized agent through my local Word Add-in (GPTLocalhost). The demo may attract users to test out both Arch and GPTLocalhost, whether the agent is in a hosted environment or runs locally.

Today, the function calling LLM (Arch-Function) designed for the agentic and RAG scenarios is hosted free of charge in the US-central region. To offer consistent latencies and throughput, and to manage our expenses, we will enable access to the hosted version via developers keys soon, and give you the option to run that LLM locally. Pricing for the hosted version of Arch-Function will be ~ $0.10/M output token (100x cheaper that GPT-4o for function calling scenarios).

[cotran2]

Hi @GPTLocalhost, thanks for creating the issue. We have the model card for the Arch-Function model at
Arch-Function model card, you can choose the size appropriates with your local hardware and use vllm/ollama to host it locally.

[GPTLocalhost]

Thank you for the pointer. Is there any example of "arch_config.yaml" for using "vllm/ollama"?

[cotran2]

Right now we are working on an update to let the user automatically hosts the model locally with vllm/ollama unfortunately :(. However, the gateway uses the public endpoint defined at

archgw/model_server/app/commons/constants.py

Line 14 in 970db68

arch_function_endpoint = "https://api.fc.archgw.com/v1"

and the model server is at the port defined at the yml file

archgw/demos/hr_agent/arch_config.yaml

Line 21 in 970db68

endpoint: host.docker.internal:18080

. You can try it out as is right now or modify

archgw/model_server/app/commons/constants.py

Line 14 in 970db68

arch_function_endpoint = "https://api.fc.archgw.com/v1"

with localhost url. http://localhost:{VLLM_PORT}