Refactor #864

ATheorell · 2023-11-23T10:59:26Z

And here it comes! The request to merge the refactored gpt-engineer into main.

Main changes:

Agent classes working as main interface to applications
Most operations now in-memory (though executions are still on disk)
No longer a unified Step interface
New file structure
Much better test coverage
and probably much more that I fail to think of now!

Reviews highly appreciated!

Merging to refactor branch so that we can work async on the missing functionality

Refactor

Add standardised benchmarking capabilities, changes to lean_agent

ErikBjare · 2023-12-01T10:50:39Z

.langchain.db

Was this added intentionally?

Edit: I assume not, given it's in the .gitignore, change made in #882

ErikBjare · 2023-12-01T10:52:50Z

gpt_engineer/applications/cli/__init__.py

-    - collect: Collect send learning data for analysis and improvement.
+    - collect: Collect send learning tools for analysis and improvement.
    - file_selector: Selecting files using GUI and terminal-based file explorer.
-    - learning: Tools and data structures for data collection.
+    - learning: Tools and tools structures for tools collection.


Looks like a search-replace gone wrong?

ErikBjare · 2023-12-01T10:55:46Z

gpt_engineer/applications/cli/collect.py


    Note:
-    This function is only called if consent is given to share data.
+    This function is only called if consent is given to share tools.


Lots of these s/data/tools that seems to be a mistake, might want to grep the diff for tools and go through them all.

Edit: I did so in #882

ErikBjare · 2023-12-01T11:32:44Z

gpt_engineer/applications/cli/cli_agent.py

+    Attributes:
+        path (str): The file path where the `Agent` will operate, used for version management and
+                    file operations.
+        version_manager (BaseVersionManager): An object that adheres to the VersionManagerInterface,
+                        responsible for version control of the generated code. Defaults to `VersionManager`
+                        if not provided. PROBABLY GIT SHOULD BE USED IN THE DEFAULT
+        step_bundle (StepBundleInterface): Workflows of code generation steps that define the behavior of gen_code and
+        improve.
+        ai (AI): Manages calls to the LLM.
+
+    Methods:
+        __init__(self, path: str, version_manager: VersionManagerInterface = None,
+                 step_bundle: StepBundleInterface = None, ai: AI = None):
+            Initializes a new instance of the Agent class with the provided path, version manager,
+            step bundle, and AI. It falls back to default instances if specific components are not provided.
+
+        init(self, prompt: str) -> Code:
+            Generates a new piece of code using the AI and step bundle based on the provided prompt.
+            It also snapshots the generated code using the version manager.
+
+            Parameters:
+                prompt (str): A string prompt that guides the code generation process.
+
+            Returns:
+                Code: An instance of the `Code` class containing the generated code.
+
+        improve(self, prompt: str) -> Code:
+            Improves an existing piece of code using the AI and step bundle based on the provided prompt.
+            It also snapshots the improved code using the version manager.
+
+            Parameters:
+                prompt (str): A string prompt that guides the code improvement process.
+
+            Returns:
+                Code: An instance of the `Code` class containing the improved code.


I'm curious: why aren't these generated with things like autodoc?

The project has a lot of docstrings which seem a pain to manually maintain and keep up-to-date, which autodoc would simplify massively (at the cost of not getting it exactly how you want it, which imo is a good thing as it prevents bikeshedding).

Reminds me of this video

Edit: I realize now that you do use autodoc, but it isn't really working (and would still duplicate a lot of work):

If of interest, I could take a stab at cleaning this up (after this PR is merged).

Would be amazing if you could have a shot at this once we merge.

Thanks

ErikBjare · 2023-12-01T16:00:25Z

gpt_engineer/applications/cli/cli_agent.py

+        self.preprompts_holder = preprompts_holder or PrepromptsHolder(PREPROMPTS_PATH)
+
+    @classmethod
+    def with_default_config(


This doesn't seem to do anything different than __init__, can be removed.

* build(deps): removed unused click dependency (now using typer) * chore: remove redundant default ruff ignore config * chore: remove redundant default black exclude config * docs: added docstring to cli main, shown for `gpt-engineer --help`

…ersion requirement (">=3.8.1,<3.12")

ATheorell · 2023-12-04T10:18:54Z

Closing this in favor of the super PR #886

ATheorell and others added 30 commits November 6, 2023 14:09

Moved cli to applications

e171b32

First skeleton of agent indicated

ebf75e0

WIP before setting up execution path

4591400

First lean implementation of agent and subclasses

7879797

Before separating step bundle in cli and simple

af539d9

core now free from inputs from applications

9101b45

Initial success of new pipeline

be986c4

code generation now working

da08777

documenting agent class

2eeaa45

formatting

383a512

Neater Code class

c94f16f

Before removing StepBundle interface

faad845

Most basics implemented, but cli still not working

cc04338

Fixed first execution problem

732c32f

Passing ExecutionEnv instead of path in new signature

5054f78

formatting

e9346a9

Renamed data module to tools and moved repository file to core/default

75a06f0

Implemented the to file operation in the on_disk_execution_env

40cad8b

main.py now using the cli_agent, which includes human review

547608b

formatting

f2aa1fa

Merge pull request #840 from ATheorell/refactor

82d2bb1

Merging to refactor branch so that we can work async on the missing functionality

New improve implemented but not tested

7146dbb

cleaning up chat_to_files

74a1f7e

Structure of tests dir now mimics project structure

8496fc6

fixed broken import

e0213af

Merge pull request #846 from ATheorell/refactor

132f428

Refactor

Removed all disk IO from chat_to_files and made much better tests

6c36257

Chat_to_files tests WIP

9cf41d6

Testing of chat_to_files complete

ac4b361

Merge pull request #847 from ATheorell/refactor

6b6cbc7

Refactor

AntonOsika and others added 4 commits November 30, 2023 19:52

change order of args

2dbd33b

imports

1eb3208

Merge pull request #878 from AntonOsika/ao/benchmark

a5e03e5

Add standardised benchmarking capabilities, changes to lean_agent

tests passing again

9b92009

ErikBjare reviewed Dec 1, 2023

View reviewed changes

fix: removed mistakenly committed .langchain.db

276f064

ErikBjare mentioned this pull request Dec 1, 2023

Refactor additions #882

Merged

fix: reverted mistaken s/tools/data

ae98066

ErikBjare reviewed Dec 1, 2023

View reviewed changes

ATheorell and others added 4 commits December 1, 2023 12:55

formatting

f11a680

Merge pull request #882 from ErikBjare/dev/refactor-additions

bc4bc74

Thanks

Improve exec env

3e48098

fixed test

0c8463b

ErikBjare reviewed Dec 1, 2023

View reviewed changes

ErikBjare mentioned this pull request Dec 2, 2023

Misc minor nits #885

Merged

ErikBjare and others added 11 commits December 3, 2023 13:36

Misc minor nits (#885)

579c716

* build(deps): removed unused click dependency (now using typer) * chore: remove redundant default ruff ignore config * chore: remove redundant default black exclude config * docs: added docstring to cli main, shown for `gpt-engineer --help`

feat: poetry - updating pyproject.toml

9ca45d3

feat: poetry - updating pyproject.toml

e28751f

feat: poetry - updating pyproject.toml

987552d

feat: poetry - updating pyproject.toml to manage llama-index python v…

42e3f15

…ersion requirement (">=3.8.1,<3.12")

test: poetry - updating test_install for poetry

a871a02

test: poetry - updating test_install for poetry

bce57ed

doc: poetry - updating docs

a3c9fb3

Reverting previous poetry commits and moving to refactor-poetry branch.

91debfc

revert: poetry - revising build-system

5e592be

chore: fix pre-commit complaint about .gitignore

d086f5a

ATheorell closed this Dec 4, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor #864

Refactor #864

ATheorell commented Nov 23, 2023

ErikBjare Dec 1, 2023 •

edited

Loading

ErikBjare Dec 1, 2023

ATheorell Dec 1, 2023

ErikBjare Dec 1, 2023 •

edited

Loading

ErikBjare Dec 1, 2023 •

edited

Loading

ATheorell Dec 1, 2023

ErikBjare Dec 1, 2023

ATheorell commented Dec 4, 2023

Refactor #864

Refactor #864

Conversation

ATheorell commented Nov 23, 2023

ErikBjare Dec 1, 2023 • edited Loading

Choose a reason for hiding this comment

ErikBjare Dec 1, 2023

Choose a reason for hiding this comment

ATheorell Dec 1, 2023

Choose a reason for hiding this comment

ErikBjare Dec 1, 2023 • edited Loading

Choose a reason for hiding this comment

ErikBjare Dec 1, 2023 • edited Loading

Choose a reason for hiding this comment

ATheorell Dec 1, 2023

Choose a reason for hiding this comment

ErikBjare Dec 1, 2023

Choose a reason for hiding this comment

ATheorell commented Dec 4, 2023

ErikBjare Dec 1, 2023 •

edited

Loading

ErikBjare Dec 1, 2023 •

edited

Loading

ErikBjare Dec 1, 2023 •

edited

Loading