Separate Core Agent and Server functionalities..Make agent embeddable

[raolak]

Is your feature request related to a problem? Please describe.
For some of the use cases, we need to execute tools which has local resource dependency...example Code generation, build and deploy etc... At present, we need to deploy entire agent and server code to get this working... Is is possible to isolate [agent+tools] and server code?

Describe the solution you'd like
-Keep agent light weight with ability deploy tools along side agents .

• One should able to embed agents into their apps.. Example can extend agent base image and add more features as required
• Agent can talk to letta server to fetch memory, documents/sources , settings etc

Describe alternatives you've considered
I am prototyping building workflow which uses a specific agent ..agent needs to execute tools on local resource, i end setting up letta per workflow, which i think little overkill considering letta already has tenancy/ruser based isolation that I can work with.

[sarahwooders]

Is the use-cases here that you want to have an ephemeral agent embedding in your application? The reason why we have a lot of stuff related to persistence (and requiring tools to be persistable outside of the application) is so that agents can be fully saved in a DB and reloaded later by the server. Or is it that you want to have persisted agents (agent memories are saved) but tools only exist locally?

The challenge with having the server executing remotely but the tools executing locally is that then there needs to be a way for the server to communicate back to the local code to request that it execute a tool. This could be done by having a listener on the client that allows it to receive requests from the server, or to have internal client code manage multiple requests to the server until the server has completed execution:

[client] call agent 
[server] run agent step(), call tool_A -> send tool_A(args) to client 
[client] execute tool_A(args) -> send response to server 
[server] continue... 

An alternative would be for you to use the LocalClient (if you are using Python) so that the server code is running on your machine, but we have Tool objects actually not be persisted, and the agent has access to a reference to the code you want to run -- and this is used instead of loading the tool code form the DB (what happens now). So

def my_local_function(self, args)
    ... 

# keep a reference to the code, to call from the locally running server
tool = Tool(func=my_local_function, ...) 

agent = client.create_agent(tools=[tool], ...) 

This would require fewer changes to Letta, but would require that the code be written in Python and is a bit hacky (and you also wouldn't be able to interact with a remote server).

Let me know what you think of either of these options, and if I'm understanding your issue correctly.

[raolak]

Hi Sarah, Apologies for the delayed response; I’ve been a bit tied up with work. Please find my comments below: *Is the use case here that you want an ephemeral agent embedded in your application? Or do you want persisted agents (with saved memories) but tools that only exist locally?* Yes, my use case involves creating a workflow within my application for specific tasks, where one or more agents are attached to automate specific aspects of the app being developed. These agents are configured for specific personas and execute task-specific workflows. Here’s how I envision it: - A workflow consists of agent steps that take inputs, produce outputs, and optionally read/write to agent memory. - Agents are composable within each step, meaning they can be bootstrapped in a container. Once a task is completed, the agent running in the container is terminated and the agent state is persisted in server. - The agent performs tasks requiring tool execution in the local runtime environment within the workflow—for example, building, packaging, or deploying. If errors occur, the steps can be retried. The workflow can be represented as a DAG of agent steps. *Specific Requirements:* 1. *State Persistence:* The agent’s state should be saved on the server, which aligns with my needs. However, I don’t want the agent step to run entirely on the server. Instead, the agent should interact with the server to manage memory, handle conversations, etc. Tools requiring execution should run locally. 2. *Execution Flow:* - Configure the agent using a Docker image with necessary environment inputs. - When the Docker container is created, it pulls tools and installs dependent packages. - The application initiates a chat pointing to the agent running in the container. - The agent interacts with the server for responses. If a response requires tool execution, it happens locally in the container. 3. *Ephemeral Chats:* Conversations with the agent are ephemeral, but the entire chat history is stored on the server. If the same agent is attached to a new workflow, it can resume from its previous state. If a clean state is required, a new agent must be provisioned. 4. *Short-Term Memory:* It’s unclear whether short-term memory should persist locally or on the server. Open to suggestions here. I’d be happy to demonstrate the integration on the platform I’m building or discuss this further over a call to clarify any questions. Thanks again for your support! regards _lnr

…

On Tue, Nov 12, 2024 at 12:14 AM Sarah Wooders ***@***.***> wrote: Is the use-cases here that you want to have an ephemeral agent embedding in your application? The reason why we have a lot of stuff related to persistence (and requiring tools to be persistable outside of the application) is so that agents can be fully saved in a DB and reloaded later by the server. Or is it that you want to have persisted agents (agent memories are saved) but tools only exist locally? The challenge with having the server executing remotely but the tools executing locally is that then there needs to be a way for the server to communicate back to the local code to request that it execute a tool. This could be done by having a listener on the client that allows it to receive requests from the server, or to have internal client code manage multiple requests to the server until the server has completed execution: [client] call agent [server] run agent step(), call tool_A -> send tool_A(args) to client [client] execute tool_A(args) -> send response to server [server] continue... An alternative would be for you to use the LocalClient (if you are using Python) so that the server code is running on your machine, but we have Tool objects actually *not* be persisted, and the agent has access to a reference to the code you want to run -- and this is used instead of loading the tool code form the DB (what happens now). So def my_local_function(self, args) ... # keep a reference to the code, to call from the locally running servertool = Tool(func=my_local_function, ...) agent = client.create_agent(tools=[tool], ...) This would require fewer changes to Letta, but would require that the code be written in Python and is a bit hacky (and you also wouldn't be able to interact with a remote server). Let me know what you think of either of these options, and if I'm understanding your issue correctly. — Reply to this email directly, view it on GitHub <#2021 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGPZOLTUQ43BXHLZ4MX2B32AD3J3AVCNFSM6AAAAABRRFLTROVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINRYHAYTONZVGM> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[github-actions]

This issue is stale because it has been open for 30 days with no activity.