Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Transform to add an agent's name into the message content #3334

Merged
merged 10 commits into from
Aug 26, 2024

Conversation

marklysze
Copy link
Collaborator

@marklysze marklysze commented Aug 9, 2024

The recent addition of non-OpenAI client classes has expanded the utility of AutoGen to a lot of different models through other cloud providers as well as the ability to run it with local LLMs.

However, the name field on each message is removed as only OpenAI can accept and utilise that field.

This means that functionality, like the GroupChat's select speaker, isn't as effective as the LLM may not be able to determine which 'agent' belongs to each message.

This PR introduces a transform that can add an agent's name to the start or end of message's text content. As this is a transform, it isn't permanent and doesn't change the original message.

How it works:

from autogen.agentchat.contrib.capabilities import transform_messages, transforms

# Create a transform
name_transform = TextMessageContentName(position="start", format_string="'{name}' said:\n")

# Create the TransformMessages
context_handling = transform_messages.TransformMessages(
            transforms=[
                name_transform
            ]
        )

# Add it to an agent so when they run inference it will apply to the messages
context_handling.add_to_agent(my_agent)

# Example
# Messages before
[
        {"role": "system", "content": "I am the system."},
        {"role": "user", "name": "charlie", "content": "I think the sky is blue."},
        {"role": "user", "name": "mary", "content": "The sky is red."},
        {"role": "user", "name": "bob", "content": "The sky is crimson."},
    ]

# Messages going into LLM (where name is removed, non-OpenAI clients)
[
        {"role": "system", "content": "I am the system."},
        {"role": "user", "content": "'charlie' said:\nI think the sky is blue."},
        {"role": "user", "content": "'mary' said:\nThe sky is red."},
        {"role": "user", "content": "'bob' said:\nThe sky is crimson."},
    ]

Documentation to be done.

Dependency

For the GroupChat select speaker to be able to utilise this, PR #2719 needs to be merged first.

Why are these changes needed?

To provide the necessary name context for non-OpenAI inference.

Related issue number

PR #2719

Checks

@marklysze marklysze added the models Pertains to using alternate, non-GPT, models (e.g., local models, llama, etc.) label Aug 9, 2024
@marklysze marklysze self-assigned this Aug 9, 2024
@marklysze marklysze requested a review from Hk669 August 16, 2024 00:12
Copy link
Collaborator

@thinkall thinkall left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @marklysze for the PR! Would like to add an example of using this new transform? What about updating an existing notebook and show the differences between with and without this transform?

@codecov-commenter
Copy link

codecov-commenter commented Aug 26, 2024

Codecov Report

Attention: Patch coverage is 10.00000% with 36 lines in your changes missing coverage. Please review.

Project coverage is 21.12%. Comparing base (6279247) to head (8e07d59).
Report is 108 commits behind head on main.

Files Patch % Lines
...togen/agentchat/contrib/capabilities/transforms.py 10.00% 36 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##             main    #3334       +/-   ##
===========================================
- Coverage   32.90%   21.12%   -11.78%     
===========================================
  Files          94       95        +1     
  Lines       10235    10729      +494     
  Branches     2193     2469      +276     
===========================================
- Hits         3368     2267     -1101     
- Misses       6580     8313     +1733     
+ Partials      287      149      -138     
Flag Coverage Δ
unittest 12.09% <10.00%> (?)
unittests 21.06% <10.00%> (-11.85%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Collaborator

@thinkall thinkall left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @marklysze . LGTM!

@thinkall thinkall added this pull request to the merge queue Aug 26, 2024
Merged via the queue into main with commit ed47ec9 Aug 26, 2024
134 of 146 checks passed
@thinkall thinkall deleted the transformwithname branch August 26, 2024 07:41
victordibia pushed a commit that referenced this pull request Aug 28, 2024
* Initial commit with ability to add name into content with a transform

* Transforms documentation

* Fix transform links in documentation

---------

Co-authored-by: Li Jiang <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
models Pertains to using alternate, non-GPT, models (e.g., local models, llama, etc.)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants