-
-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
deciduus' development pulls ~ IDE terminal compatibility, docker quality of life, prompt optimizations, execution handling & style encoding, NVIDIA CUDA (GPU support) ~ #19
base: main
Are you sure you want to change the base?
Conversation
added pycharm ide running compatibility
slight edit of prompt
slight revision of prompt
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
changes look good.
refined prompts for clarity and conciseness adding some new steps for logic and reducing chance of subordinate looping increased main prompt timeout to 2 min so it won't run so quick for so little value
…jects. made edits to the docker.py to include a health check (threaded), it does not impact framework flow. also added a check and reconnect mechanic for the docker due to some issues where it would disconnect, try terminal commands, and get no response (wasting api responses). the health check thread also follows up to close the docker container upon force close in IDE, which helps save computer resources. knowledge tool was updated to include a top 3~ hits for wikipedia content to build the contextual awareness. timed input was adjusted upward for testing.
I think the latest commit from a few minutes ago has several useful features that could benefit the main project. Docker Enhancements:
Knowledge Tool Update:
Timed Input Adjustment:
|
…d errors (lack of gpu resources), to prevent the error in tool. docker wait time was decreased as the connection doesnt take that long print style added encoding utf-8 to prevent read errors, this resulted in many terminal tasks that were previously "erroring" due to simply taking too long or having a bad symbol to actually function better.
|
edited docker file and readme to allow for nvidia cuda support, giving access to gpu resources. added readme steps to enable the GPU compatibility setup
undid the code execution tool edit from before edited docker setup to change to nvidia cuda, allowing gpu resource access for docker added readme steps at bottom elaborating on the cuda setup (ubuntu and docker)
|
…d perms for container. added commented out code in dockerfile for trying to import torch and numpy at start, but currently having issues with it initiating (but pushing so others have it incase they want to try to fix)
Updated README with comprehensive instructions for setting up NVIDIA CUDA, Ubuntu, permissions, and Docker configurations. Note: Dockerfile includes commented-out code for the initial install of Numpy and PyTorch, but I haven't had it work yet. |
looks interesting, I'm curious does this mean : i) agent-zero can run in Ubuntu with a GUI ii) does agent-zero itself run using cuda or iii) is it so one can work on nvidia cuda related code projects? I started installing Linux Mint to see if I can have Agent-Zero use a GUI environment, like controlling Chrome with Puppeteer... instead of Ubuntu can your fork also use Linux Mint? |
my unprompted reply: gpt 4o helping me objectify the description above: Yes, the setup aims to enable GPU access within the container using NVIDIA's CUDA, allowing for the utilization of GPU resources. This configuration enables Agent-Zero to leverage GPU acceleration for tasks such as running PyTorch models, significantly enhancing performance for machine learning workloads. Key Points: Usage of CUDA: Linux Mint Compatibility: Additional Note: |
- build.txt same ^ - code execution tool got elaborate updates for prompt-handling pass-back, idle timeout, comparing old and new outputs to prevent spamming user with identitical terminal commands, and a summarized output feature to prevent slamming API with too many tokens on big terminal tasks, also the terminal will retain control and the agent will not check in unless idle proc (about 30 seconds) -docker.py sleeps slightly longer anticipating some may take longer to spin up the new container. -docker compose has been added for the user to create the nvidia image - please see readme for detailed instructions -dockerfile has been adjusted, new container (old is commended out), timezone check for automatic selection on image compose. organized a bit. -main.py adjusted the commented out docker names just for clarity -adjusted rate_limiter.py to fix a bug when the agent was doing long tasks and lots of calls -added intensive README section for ubuntu and nvidia cuda runtime setup and troubleshooting. -timeout_input was reverted to original value
This update introduces significant enhancements, focusing on enabling NVIDIA CUDA support for GPU access, which now allows for running GPU-intensive tasks such as deep learning and other computational workloads. This includes setting up the necessary infrastructure for efficient GPU utilization. Alongside this, comprehensive improvements have been made to streamline the user experience and optimize system performance. The code_execution_tool.py has been significantly upgraded to manage extensive terminal tasks more effectively, with new features like prompt handling, idle timeout detection, and summarized output for efficient API communication. Additionally, new setup instructions and configuration changes have been implemented across various files to support these updates. See README for in-depth setup instructions. These changes collectively aim to provide a more robust and user-friendly environment, ensuring better resource management and smoother operations. Summary of Changes agent.py build.txt code_execution_tool.py docker.py docker-compose.yml Dockerfile main.py rate_limiter.py README.md timeout_input.py |
KNOWN ISSUE - sometimes on an incomplete terminal output the agent will no check in as quickly as it would on the original - however, the idle count/score will kick in and pass it back to user after approximately 30 seconds. If however it gets stuck and you want to manually pass back, simply user intervention, prompt, and or pass it details about how it was stuck, or looping, etc. TLDR if you get stuck, wait, or E and relaunch 'python main.py' |
…plified) extensive dockerfile changes to remove venv and add custom package installs when composing docker image. (machine learning packages) adjusted the image reference comment on main for my fork readme is a bit of a mess (not gracefully formatted additions) but i added some troubleshooting info for docker/ubuntu composing, images etc.
A lot of these changes are specific to my fork, but we should be able to add optionality. I haven't focused on this aspect as much, as I've been primarily building new functionality.
These changes streamline the setup process for nvidia cuda and packages (custom image) and improve the overall functionality of the application within a Docker environment. |
I've resolved conflicts - |
except NotFound: | ||
try: | ||
print(f"Creating new container: {self.name}") | ||
self.container = self.client.containers.run( | ||
self.image, | ||
detach=True, | ||
ports=self.ports, | ||
name=self.name, | ||
volumes=self.volumes, | ||
) | ||
print(f"Started new container with ID: {self.container.id}") | ||
except docker.errors.ImageNotFound as inf: | ||
print(f"Image not found: {inf}") | ||
except docker.errors.APIError as api_error: | ||
print(f"Docker API error: {api_error}") | ||
except Exception as e: | ||
print(f"Unexpected error when starting container: {e}") | ||
finally: | ||
atexit.register(self.cleanup_container) | ||
time.sleep(5) | ||
self.start_health_check_thread() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd opt to break out to variable instead for readability, just like in the original
It runs cuda within the terminal (not ubuntu) but the setup and compose of
the custom cuda image (within which agent Zero runs) is composed in Ubuntu.
…On Thu, Aug 8, 2024, 11:29 AM shellkjell ***@***.***> wrote:
***@***.**** approved this pull request.
—
Reply to this email directly, view it on GitHub
<#19 (review)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/BEA7AQ55TZNY5XEAS6YQBYTZQOMFLAVCNFSM6AAAAABLWPMCDKVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDEMRYGM3DCOJRGE>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
This code causes a error loop, I am not sure if it is the renaming of the docker file I will have to test, but it sends any ai into a loop with:
|
If you had the container running from composing it up you need to close it
prior to launch in this scenario.
…On Fri, Aug 16, 2024, 9:14 AM Jason Schoeman ***@***.***> wrote:
This code causes a crash, I am not sure if it is the renaming of the
docker file but I will have to test, but it sends any ai into a loop with:
Code: touch test.txt
Creating new container: docker-agent-zero-exe
Image not found: 404 Client Error for http+docker://localhost/v1.46/images/create?tag=latest&fromImage=docker-agent-zero-exe: Not Found ("pull access denied for docker-agent-zero-exe, repository does not exist or may require 'docker login': denied: requested access to the resource is denied")
SSH Connection attempt 1...
SSH Connection attempt 2...
{
"system_error": "Traceback (most recent call last):
File "/Users/jasonschoeman/Documents/Projects/agent-zero-2/python/helpers/shell_ssh.py", line 27, in connect
self.client.connect(self.hostname, self.port, self.username, self.password)
File "/Users/jasonschoeman/.pyenv/versions/3.12.5/lib/python3.12/site-packages/paramiko/client.py", line 409, in connect
raise NoValidConnectionsError(errors)
paramiko.ssh_exception.NoValidConnectionsError: [Errno None] Unable to connect to port 50022 on 127.0.0.1 or ::1
"
}
—
Reply to this email directly, view it on GitHub
<#19 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/BEA7AQ7E3XJB4USYO5PVDOLZRYCMDAVCNFSM6AAAAABLWPMCDKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOJTGU4TAMJXGY>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
…ide (for redundant reference) code execution tool summarizing extended for more context on longer files.
updated the nvidia cuda setup steps to address errors and prevent the docker desktop overwriting the default runtime. |
I tested this as a clean install. And it seemed to work nicel y for me (Win11, CUDA 12.4, GeForce GTX 980Ti ). Thanks, deciduus. |
Incorporate changes from deciduus agent-zero fork [~ IDE terminal compatibility, docker quality of life, prompt optimizations, execution handling & style encoding, NVIDIA CUDA (GPU support) ~ frdel#19]
❤️❤️
…On Mon, Aug 19, 2024, 4:31 AM Caspar Addyman ***@***.***> wrote:
I tested this as a clean install. And it seemed to work nicel y for me
(Win11, CUDA 12.4, GeForce GTX 980Ti ). Thanks, deciduus.
—
Reply to this email directly, view it on GitHub
<#19 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/BEA7AQ627PYHCMZHPD7UVCLZSG3P5AVCNFSM6AAAAABLWPMCDKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOJWGEYDQNBZHE>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
Deciduus agent zero fork changes (Incorporate changes from deciduus agent-zero fork [~ IDE terminal compatibility, docker quality of life, prompt optimizations, execution handling & style encoding, NVIDIA CUDA (GPU support) ~ frdel#19]
I deleted the container completely, it is on a MacOs, still get the same issue: I dont get it on the main branch. |
You may need to purge all the old images/containers in ubuntu, recompose
the image, and try again. I've also updated the README and setup
instructions since your original post, it may be worth trying again from
step 1.
On Mon, Aug 19, 2024 at 10:24 AM Jason Schoeman ***@***.***>
wrote:
… If you had the container running from composing it up you need to close it
prior to launch in this scenario.
… <#m_8730107855366028201_>
On Fri, Aug 16, 2024, 9:14 AM Jason Schoeman *@*.*> wrote: This code
causes a crash, I am not sure if it is the renaming of the docker file but
I will have to test, but it sends any ai into a loop with: Code: touch
test.txt Creating new container: docker-agent-zero-exe Image not found: 404
Client Error for
http+docker://localhost/v1.46/images/create?tag=latest&fromImage=docker-agent-zero-exe:
Not Found ("pull access denied for docker-agent-zero-exe, repository does
not exist or may require 'docker login': denied: requested access to the
resource is denied") SSH Connection attempt 1... SSH Connection attempt
2... { "system_error": "Traceback (most recent call last): File
"/Users/jasonschoeman/Documents/Projects/agent-zero-2/python/helpers/shell_ssh.py",
line 27, in connect self.client.connect(self.hostname, self.port,
self.username, self.password) File
"/Users/jasonschoeman/.pyenv/versions/3.12.5/lib/python3.12/site-packages/paramiko/client.py",
line 409, in connect raise NoValidConnectionsError(errors)
paramiko.ssh_exception.NoValidConnectionsError: [Errno None] Unable to
connect to port 50022 on 127.0.0.1 or ::1 " } — Reply to this email
directly, view it on GitHub <#19 (comment)
<#19 (comment)>>, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/BEA7AQ7E3XJB4USYO5PVDOLZRYCMDAVCNFSM6AAAAABLWPMCDKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOJTGU4TAMJXGY
<https://github.com/notifications/unsubscribe-auth/BEA7AQ7E3XJB4USYO5PVDOLZRYCMDAVCNFSM6AAAAABLWPMCDKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOJTGU4TAMJXGY>
. You are receiving this because you authored the thread.Message ID: @.*>
I deleted the container completely, it is on a MacOs, still get the same
issue:
Image not found: 404 Client Error for
http+docker://localhost/v1.46/images/create?tag=latest&fromImage=docker-agent-zero-exe:
Not Found ("pull access denied for docker-agent-zero-exe, repository does
not exist or may require 'docker login': denied: requested access to the
resource is denied")
I dont get it on the main branch.
—
Reply to this email directly, view it on GitHub
<#19 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/BEA7AQ5KM7TJ2UODJCYPJ3DZSIE4HAVCNFSM6AAAAABLWPMCDKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOJWHA2DKOBZGE>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
Thanks for your response, I got it working by overriding these in the main.py: code_exec_docker_enabled = True,
code_exec_docker_name = "agent-zero-exe",
code_exec_docker_image = "frdel/agent-zero-exe:latest",
code_exec_docker_ports = {"22/tcp": 50022},
code_exec_docker_volumes = {files.get_abs_path("work_dir"): {"bind": "/root", "mode": "rw"}},
code_exec_ssh_enabled = True,
code_exec_ssh_addr = "localhost",
code_exec_ssh_port = 50022,
code_exec_ssh_user = "root",
code_exec_ssh_pass = "toor", |
I'm new here, but continuing to play around and tweak but I'm submitting a pull request as these seem like good QoL for everyone.