[Issue]: BadRequestError - 400 When Using MessageHistoryLimiter in GroupChat Workflow

[fjmoronreyes]

Describe the issue

Error

I am encountering a BadRequestError while using MessageHistoryLimiter for context management in a multi-agent GroupChat setup. Below is the error message:

BadRequestError: Error code: 400 - {'error': {'message': "Invalid parameter: messages with role 'tool' must be a response to a preceeding message with 'tool_calls'.", 'type': 'invalid_request_error', 'param': 'messages.[1].role', 'code': None}}

Context

Context:
I am using a GroupChat of multiple agents where each agent performs specific tasks, including scraping content from a URL and processing it into structured data. The workflow involves passing tasks between agents and calling registered functions (tools). I implemented the MessageHistoryLimiter to manage context within this chat environment, but the error consistently occurs during the workflow.

The error seems to occur when an agent sends a message with a tool role, but the system expects a tool_calls entry to precede it.

The issue likely stems from the context management changes introduced with MessageHistoryLimiter, which may have affected the sequence of tool-related messages.

I have registered the tools with the register_function:

autogen.register_function(
    func,
    caller=reader,
    executor=coder,
    description="",
)

And i create assistant functions like this:

reader = autogen.AssistantAgent( 
    name="reader",
    system_message="""message
    """,
    human_input_mode="NEVER",
    llm_config=CONFIG,
)

And there is one agent that executes the tools and others that call that agent.

I have created a custom_speaker_selection_funct that i use like this:

def custom_speaker_selection_func(last_speaker: Agent, groupchat: GroupChat):
    messages = groupchat.messages

    if len(messages) == 0:
        return agent1
    if last_speaker == agent1:
        return agen2
    elif last_speaker == agen5:
        return agen3
    return "auto"


groupchat = GroupChat(
    agents=agents,
    speaker_selection_method=custom_speaker_selection_func,
    messages=[],
    max_round=30,
)

manager = autogen.GroupChatManager(groupchat=groupchat, llm_config=CONFIG)

And then i start with initiate_chat

chat_result = agen1.initiate_chat(
    manager, message=prompt, summary_method="reflection_with_llm", silent=True
)

Example

I leave here an example of my chat_history but without the content. I can ensure that every tool is preceeded by a valid tool call.

Role: user
Name: collector
Content:

---

---

Role: assistant
Name: finder
Content:

Tool Calls:
Tool ID: CALL_ID
Tool Name: TOOL_NAME
Tool Arguments: ARGS

---

Role: tool
Name: coder
Content:

---

---

Role: assistant
Name: reader
Content:

Tool Calls:
Tool ID: CALL_ID
Tool Name: TOOL_NAME
Tool Arguments: ARGS

---

Role: tool
Name: coder
Content:

Steps to reproduce

1. Setup a GroupChat with multiple agents using MessageHistoryLimiter for context management.
2. Initiate a workflow where agents interact with tools to scrape and process content.
3. Observe the error after one or more tool invocations.

Screenshots and logs

No response

Additional Information

autogen 0.3.0
pyautogen 0.2.31

[ekzhu]

@WaelKarkoub is this something you can comment on?

BTW we are using autogen-agentchat package now, and the latest version is 0.2.36. See FAQ: https://github.com/microsoft/autogen?tab=readme-ov-file#faqs

[WaelKarkoub]

hmm currently message history limiter doesn't account for that behavior.

I think the fix here is to add a check if the transform violates this rule, e.g. 'tools' must be preceded by tool calls. I suspect this error can also show up in other transforms. One solution could be to create a new transform _DropBadMesages or _DropViolatingMessages that the other transforms could use internally. lmk what you guys think.

[WaelKarkoub]

@ekzhu I opened a small pr, didn't think generalizing the solution is required just yet #3875