-
Notifications
You must be signed in to change notification settings - Fork 39
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Propagate DB schema changes: Function <-> CompiledRouteSpec #65
Propagate DB schema changes: Function <-> CompiledRouteSpec #65
Conversation
…r-generated-function' of github.com:Significant-Gravitas/codex into zamilmajdy/agpt-433-argument-and-return-type-parsing-for-generated-function
…-return-type-parsing-for-generated-function' into zamilmajdy/agpt-433-argument-and-return-type-parsing-for-generated-function # Conflicts: # codex/develop/agent.py
@@ -124,15 +125,24 @@ class GameStateResponse(BaseModel): | |||
turn: str | |||
state: str | |||
board: str | |||
|
|||
class Board(BaseModel): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Testing new class being introduced by function generation
return 'Draw' | ||
|
||
return 'In Progress' | ||
|
||
def make_turn(request: TurnRequest) -> GameStateResponse: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated because now request is being 1-level flatten
) | ||
|
||
if not generated_response.function_id: | ||
raise AssertionError("Function ID is required to update") | ||
|
||
func: Function | None = await Function.prisma().update( | ||
where={"id": generated_response.function_id}, data=update_obj |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you can update and fetch in a single go
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, but I couldn't map the ApiRouteSpec to child objects in a single call
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you should be able to, but since we no longer have apiroutespec here, I don't think it's relevant now.
user_id: str | ||
cloud_services_id: str | ||
app_id: str | ||
user_id: str | None = None |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is causing a ton of error checking around the app to be needed. Why this change?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Testing codes not setting some of the ids, so I'm setting it as optional for all of them just for the consistency.
|
||
|
||
|
||
async def create_object_type( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ntindle linking nested classes is slightly more complicated than it seems.
if you have ObjectType1 (with fields X,Y,Z) and ObjectType2 (with fields A,B,C)
X should be able to connect to ObjectType2, or B should be able to connect to ObjectType1, combined with the complexity of list/dict and tuple.
I'm going to resolve everything in this function.
so if you have list of object that you need to create, you can simply call this:
objects = [ObjectDef1, ObjectDef2]
available_objects = {} # we can start from empty dict
for obj in objects:
available_objects = await create_object_type(obj, available_objects)
# objects are now persisted in the DB as ObjectType
S
@@ -34,7 +34,8 @@ async def develop_application(ids: Identifiers, spec: Specification) -> Complete | |||
CompletedApp: The completed application. | |||
|
|||
""" | |||
compiled_routes = [] | |||
app = await create_app(ids, spec, []) | |||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Create the CompletedApp first, then append the compiled routes, then generate the function on each.
As discussed with @Swiftyos
"RootFunction": { | ||
"include": {"FunctionArgs": True, "FunctionReturn": True} | ||
} | ||
}, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Create the compiledRoute first before generating the function
) | ||
compiled_route = await CompiledRoute.prisma().create(data) | ||
compiled_route = await CompiledRoute.prisma().update( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It will only need to update the compiled code because everything has been eagerly created.
@@ -237,11 +240,13 @@ def validate( | |||
# Validate the requested_func.args and requested_func.returns to the invoke_params | |||
expected_args = invoke_params["function_args"] | |||
expected_rets = invoke_params["function_rets"] | |||
if expected_args != requested_func.arg_types: | |||
|
|||
if any([x[0] != y[0] or not is_type_equal(x[1], y[1]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Comparison on different types are becoming more complex.
we need to allow list[String]
and [String]
or List[str]
to be all equal.
@@ -9,8 +9,68 @@ | |||
from codex.develop.model import FunctionDef | |||
|
|||
|
|||
def get_type_children(type: str) -> (str, [str]): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is used for matching two different types
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All working hardcoded requirements are passing 🚀
Nice work!
Successfully created 6/9 applications.
✗ Failed: TimeSync
✗ Failed: InvenTrack
✗ Failed: FinFlow
✓ Created: Availability Checker
✓ Created: Invoice Generator
✓ Created: Appointment Scheduler
✓ Created: Distance Calculator
✓ Created: Profile Management System
✓ Created: TicTacToe Game
) | ||
|
||
if not generated_response.function_id: | ||
raise AssertionError("Function ID is required to update") | ||
|
||
func: Function | None = await Function.prisma().update( | ||
where={"id": generated_response.function_id}, data=update_obj |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, but I couldn't map the ApiRouteSpec to child objects in a single call
* Route schema changes * Delete ruff.config.json * save current changes * style fixes by ruff * Add class parsing capability * Fix bugs * Address comments * Fix benchmarks * Fix benchmarks * Fix benchmarks * Add comments * fixing types * Add error handling and status messages for application creation * Refactor fetch_deliverable function and update application creation status * Refactor benchmark output formatting * Remove unnecessary print statement in fetch_deliverable function * Propagate DB schema changes: Function <-> CompiledRouteSpec (#65) Propagate DB schema changes: Function <-> CompiledRouteSpec (#65) --------- Co-authored-by: majdyz <[email protected]> Co-authored-by: Swifty <[email protected]> Co-authored-by: majdyz <[email protected]> * emoved broken hardcoded requirements * Add benchmarking functionality * Update branch restrictions in GitHub workflows * Add pytest-cov to dependencies * Update ruff formatter command to include --check option * Update workflow names * Refactor code formatting and fix indentation issues * Update Python dependency caching and installation process * Add Prisma client generation step to CI workflow * Refactor test_process_object_type to use async/await * Add pytest and ValidationError imports, update arg_types assertion, and add test for ValidationError * Remove unused imports and commented code * Add logger to OpenAIChatClient class * Add async context manager for database connection and disconnection * Add blank line for readability * Refactor function_def.arg_types assignment in ast_test.py * mark integration tests * Update test workflow name * Add OpenAI API key configuration * Better Types for Requirements Request/Response (#76) Better Types for Requirements Request/Response --------- Co-authored-by: majdyz <[email protected]> * [AGPT-474] End-to-end Benchmark (#77) * Remove unnecessary fetch_deliverable calls and update invoice_generator_requirements * Fix typo in hardcoded.py * Add missing app IDs and interview IDs for calendar booking, inventory management, and invoicing systems * Add options to benchmark and example commands * Refactor benchmarking code and add new functions * Add task description method to ExampleTask enum * Added end-to-end running * Add hardcoded flag to filter examples in benchmark.py * Refactor run_benchmark_example function to handle exceptions * Fix error handling in specification routes * Fix Function generation bug on nested object type (#79) Co-authored-by: majdyz <[email protected]> * Speeding up requirements system --------- Co-authored-by: Zamil Majdy <[email protected]> Co-authored-by: majdyz <[email protected]> * Refactor code: Remove unused imports and clean up formatting * [AGPT-478] Add Support for Composite Type with multiple Pydantic/nont-primitive classes (#78) * [AGPT-478] Add support for multiple composite Pydantic objects * Ruff Ruff! * Fix failing test * formatting * fix populate-db command * fix run all benchmarks --------- Co-authored-by: majdyz <[email protected]> Co-authored-by: Swifty <[email protected]> * Refactor package handling in packager.py (#81) * [Fix] User provided app name being replaced (#82) * Only make databases when required (#83) * feat: make the name match * feat: database becomes optional * Update agent.py * Integrate related types into the requirements system (#84) * feat: make the name match * feat: database becomes optional * feat: initial model changes * feat: add prompts * feat: more prompts * fix: nits * fix: wrong type 😢 * feat: improved prompts * Cleanup for field type (#85) Co-authored-by: majdyz <[email protected]> * [AGPT-489] Improve type composite-type matching & extracting algorithm (#86) Co-authored-by: majdyz <[email protected]> * [AGPT-489] Add Autogeneration of Typing import (#87) Co-authored-by: majdyz <[email protected]> * [AGPT-489] Fix broken class compiling logic (#88) Co-authored-by: majdyz <[email protected]> * [AGPT-489] Fix function fetching to always include RelatedTypes (#89) [AGPT-489] Fix function fetching to always include RelatedTypes --------- Co-authored-by: majdyz <[email protected]> * fix lock file * feat: endpoint updates (#95) * feat: endpoint updates * fix: ambiguous param * [AGPT-486] Fix codegen generate code with missing pydantic object (#92) Co-authored-by: majdyz <[email protected]> * Fix retry prompt (#96) Co-authored-by: majdyz <[email protected]> * Allow benchmark on $PORT (#98) * Ntindle/agpt 506 add a bit of debug tooling for requirements gen (#100) * [AGPT-485] Fix missing pydantic object during requirement-step; leading to LLM to guess wrong signature (#97) [AGPT-485] Fix missing pydantic object during requirement-step; leading to LLM to guess wrong signature --------- Co-authored-by: majdyz <[email protected]> * [AGPT-157] Add Database Integration Generation (#90) * Added prisma client to server code * Add DatabaseSchema to INCLUDE_FUNC * Update functions to be async and add database schema documentation * Add DatabaseSchema and DatabaseTables to ApiRouteSpec * Updated dev prompts for db * Update profile management descriptions * Update route function definition to be async * Remove logging of database schema * Add Prisma schema file generation * Refactor benchmark and agent modules * Refactor code and comment out debug statements * formatting * Add available_functions to develop_route * Update prisma import and method calls * loading all related tables * Refactor create_prisma_scheama_file function to handle cases with no database schema * Fix async function call in packager.py * Fix database table query in packager.py * Fix database access and unnecessary imports * review comments * [AGPT-508] Enum support for Codex - Generation Logic Layer (#102) Co-authored-by: majdyz <[email protected]> * Add Enum Generation to Requriements (#101) * Added prisma client to server code * Add DatabaseSchema to INCLUDE_FUNC * Update functions to be async and add database schema documentation * Add DatabaseSchema and DatabaseTables to ApiRouteSpec * Updated dev prompts for db * Update profile management descriptions * Update route function definition to be async * Remove logging of database schema * Add Prisma schema file generation * Refactor benchmark and agent modules * Refactor code and comment out debug statements * formatting * Add available_functions to develop_route * Update prisma import and method calls * loading all related tables * Refactor create_prisma_scheama_file function to handle cases with no database schema * Fix async function call in packager.py * Fix database table query in packager.py * Fix database access and unnecessary imports * feat: add enums to the database gen * fix: formatting * Updated to new schema * update hardcoded requirements * fix: objectType generation without type bool * Add database enum import statement * removed unused var as failing lint check --------- Co-authored-by: SwiftyOS <[email protected]> * More Linting Issues Fixes (#99) * fix: more linting issues * Remove error log for async functions * Fix return_type default value in FunctionDef class * Added another validation check --------- Co-authored-by: Swifty <[email protected]> * [AGPT-463] Add code generation validation using Ruff (#103) --------- Co-authored-by: majdyz <[email protected]> --------- Co-authored-by: majdyz <[email protected]> Co-authored-by: majdyz <[email protected]> Co-authored-by: Zamil Majdy <[email protected]> Co-authored-by: Nicholas Tindle <[email protected]>
* Route schema changes * Delete ruff.config.json * save current changes * style fixes by ruff * Add class parsing capability * Fix bugs * Address comments * Fix benchmarks * Fix benchmarks * Fix benchmarks * Add comments * fixing types * Add error handling and status messages for application creation * Refactor fetch_deliverable function and update application creation status * Refactor benchmark output formatting * Remove unnecessary print statement in fetch_deliverable function * Propagate DB schema changes: Function <-> CompiledRouteSpec (#65) Propagate DB schema changes: Function <-> CompiledRouteSpec (#65) --------- Co-authored-by: majdyz <[email protected]> Co-authored-by: Swifty <[email protected]> Co-authored-by: majdyz <[email protected]> * emoved broken hardcoded requirements * Add benchmarking functionality * Update branch restrictions in GitHub workflows * Add pytest-cov to dependencies * Update ruff formatter command to include --check option * Update workflow names * Refactor code formatting and fix indentation issues * Update Python dependency caching and installation process * Add Prisma client generation step to CI workflow * Refactor test_process_object_type to use async/await * Add pytest and ValidationError imports, update arg_types assertion, and add test for ValidationError * Remove unused imports and commented code * Add logger to OpenAIChatClient class * Add async context manager for database connection and disconnection * Add blank line for readability * Refactor function_def.arg_types assignment in ast_test.py * mark integration tests * Update test workflow name * Add OpenAI API key configuration * Better Types for Requirements Request/Response (#76) Better Types for Requirements Request/Response --------- Co-authored-by: majdyz <[email protected]> * [AGPT-474] End-to-end Benchmark (#77) * Remove unnecessary fetch_deliverable calls and update invoice_generator_requirements * Fix typo in hardcoded.py * Add missing app IDs and interview IDs for calendar booking, inventory management, and invoicing systems * Add options to benchmark and example commands * Refactor benchmarking code and add new functions * Add task description method to ExampleTask enum * Added end-to-end running * Add hardcoded flag to filter examples in benchmark.py * Refactor run_benchmark_example function to handle exceptions * Fix error handling in specification routes * Fix Function generation bug on nested object type (#79) Co-authored-by: majdyz <[email protected]> * Speeding up requirements system --------- Co-authored-by: Zamil Majdy <[email protected]> Co-authored-by: majdyz <[email protected]> * Refactor code: Remove unused imports and clean up formatting * [AGPT-478] Add Support for Composite Type with multiple Pydantic/nont-primitive classes (#78) * [AGPT-478] Add support for multiple composite Pydantic objects * Ruff Ruff! * Fix failing test * formatting * fix populate-db command * fix run all benchmarks --------- Co-authored-by: majdyz <[email protected]> Co-authored-by: Swifty <[email protected]> * Refactor package handling in packager.py (#81) * [Fix] User provided app name being replaced (#82) * Only make databases when required (#83) * feat: make the name match * feat: database becomes optional * Update agent.py * Integrate related types into the requirements system (#84) * feat: make the name match * feat: database becomes optional * feat: initial model changes * feat: add prompts * feat: more prompts * fix: nits * fix: wrong type 😢 * feat: improved prompts * Cleanup for field type (#85) Co-authored-by: majdyz <[email protected]> * [AGPT-489] Improve type composite-type matching & extracting algorithm (#86) Co-authored-by: majdyz <[email protected]> * [AGPT-489] Add Autogeneration of Typing import (#87) Co-authored-by: majdyz <[email protected]> * [AGPT-489] Fix broken class compiling logic (#88) Co-authored-by: majdyz <[email protected]> * [AGPT-489] Fix function fetching to always include RelatedTypes (#89) [AGPT-489] Fix function fetching to always include RelatedTypes --------- Co-authored-by: majdyz <[email protected]> * fix lock file * feat: endpoint updates (#95) * feat: endpoint updates * fix: ambiguous param * [AGPT-486] Fix codegen generate code with missing pydantic object (#92) Co-authored-by: majdyz <[email protected]> * Fix retry prompt (#96) Co-authored-by: majdyz <[email protected]> * Allow benchmark on $PORT (#98) * Ntindle/agpt 506 add a bit of debug tooling for requirements gen (#100) * [AGPT-485] Fix missing pydantic object during requirement-step; leading to LLM to guess wrong signature (#97) [AGPT-485] Fix missing pydantic object during requirement-step; leading to LLM to guess wrong signature --------- Co-authored-by: majdyz <[email protected]> * [AGPT-157] Add Database Integration Generation (#90) * Added prisma client to server code * Add DatabaseSchema to INCLUDE_FUNC * Update functions to be async and add database schema documentation * Add DatabaseSchema and DatabaseTables to ApiRouteSpec * Updated dev prompts for db * Update profile management descriptions * Update route function definition to be async * Remove logging of database schema * Add Prisma schema file generation * Refactor benchmark and agent modules * Refactor code and comment out debug statements * formatting * Add available_functions to develop_route * Update prisma import and method calls * loading all related tables * Refactor create_prisma_scheama_file function to handle cases with no database schema * Fix async function call in packager.py * Fix database table query in packager.py * Fix database access and unnecessary imports * review comments * [AGPT-508] Enum support for Codex - Generation Logic Layer (#102) Co-authored-by: majdyz <[email protected]> * Add Enum Generation to Requriements (#101) * Added prisma client to server code * Add DatabaseSchema to INCLUDE_FUNC * Update functions to be async and add database schema documentation * Add DatabaseSchema and DatabaseTables to ApiRouteSpec * Updated dev prompts for db * Update profile management descriptions * Update route function definition to be async * Remove logging of database schema * Add Prisma schema file generation * Refactor benchmark and agent modules * Refactor code and comment out debug statements * formatting * Add available_functions to develop_route * Update prisma import and method calls * loading all related tables * Refactor create_prisma_scheama_file function to handle cases with no database schema * Fix async function call in packager.py * Fix database table query in packager.py * Fix database access and unnecessary imports * feat: add enums to the database gen * fix: formatting * Updated to new schema * update hardcoded requirements * fix: objectType generation without type bool * Add database enum import statement * removed unused var as failing lint check --------- Co-authored-by: SwiftyOS <[email protected]> * More Linting Issues Fixes (#99) * fix: more linting issues * Remove error log for async functions * Fix return_type default value in FunctionDef class * Added another validation check --------- Co-authored-by: Swifty <[email protected]> * [AGPT-463] Add code generation validation using Ruff (#103) --------- Co-authored-by: majdyz <[email protected]> * feat: extract external command on in memory code to a function with tests * feat: prisma model parser * feat: database test updates now requires prisma to run * feat: test for prisma accessible * feat: update to use format and validate Also saves the formatted and validated models in place * fix: lint * fix: remove unused variable * feat: allow multiple generators * .gitignore and .env.example (#107) * feat: .env.example * feat: gitignore * fix: replace all bad chars with NOTHING also fix the format string * Add Database Docker Compose File (#108) * feat: .env.example * feat: gitignore * fix: replace all bad chars with NOTHING also fix the format string * feat: add docker compose with only db --------- Co-authored-by: Swifty <[email protected]> * feat: remove vector requirement (#106) * Fix lifecycle and prisma models and enums missing (#105) * fix: add prisma model imports fro root level * fix: reorder start code and add lifecycle * Validate Database Generation (#104) * feat: extract external command on in memory code to a function with tests * feat: prisma model parser * feat: database test updates now requires prisma to run * feat: test for prisma accessible * feat: update to use format and validate Also saves the formatted and validated models in place * fix: lint * fix: remove unused variable * feat: allow multiple generators * formatting * Change logger.info to logger.debug in exec_external_on_contents function * Update error logging in exec_external_on_contents function * Update response model and route annotation in compile.py * Force * Ruff * Fix Api route spec * formatting * Dont generate envexample passwords with symbols in them (#117) * fix: don't use special chars in db password * fix: linting * [AGPT-540] Add missing initialization of available object types (#113) * [AGPT-540] Add missing initialization of available object types * [AGPT-540] Add missing initialization of available object types * [AGPT-540] Add missing initialization of available object types * Ruff * allow non-annotated type checking * Include all available function for ruff * Fix tests * Make DB work * Make stubbing work * Make papa-lint happy * Ruffs * Add prisma model validation layer * Improve prisma querying by avoiding conflict * Move on during failure formatting --------- Co-authored-by: majdyz <[email protected]> * main into dev (#118) * Update setup guide. * [AGPT-544] Vague 500 errors in interview stage (#115) * Add error handling for user not found in database during interview step. * Add error handling for app not found in database during interview step. * Add error handling for interview not found in database during interview step. * Add error handling for nonexistent questions in interview step. * Add error handling for non-"ask" tool in user answers in interview step. * Clarify comment describing interview next_step. --------- Co-authored-by: Toran Bruce Richards <[email protected]> * [AGPT-511] Resume running from any point (#114) * Refactor populate_db function and add get_resume_points command * Fix app ID reference in benchmark.py * Add ResumePoint model * Added runner funcitons to running codex from the cli * Add application description field to ApplicationBase model and update create_app method in CodexClient * Refactor code to run tasks concurrently * Add include parameter to find_many() method in get_resume_points() * Add disconnect calls to asyncio event loop * Update resume_point name to display only the first 30 characters * Remove commented out code and unused imports * Remove unused code * formatting * Fix validation error handling in develop.py * formatting * fixes * Add validation error for unimplemented main function * Validate and try-fix generated endpoint db and model usage (#116) * feat: log status * feat: trivial implementation of copy works for request -> response and response-> request for unfilled object but not list[unfilled_object] * fix: i cant rememember * feat: more complex * feat: mostly working * fix: fixes from tests * feat: tests * fix: linting problems and test name * feat: add more supported types * fix: remove unused data * fix: stop using print * fix: arrays of arrays * fix: lint * fix: reduce mess * feat: more complex type that I hope we never actually use * fix: lint --------- Co-authored-by: majdyz <[email protected]> Co-authored-by: majdyz <[email protected]> Co-authored-by: Zamil Majdy <[email protected]> Co-authored-by: Nicholas Tindle <[email protected]> Co-authored-by: Nicholas Tindle <[email protected]> Co-authored-by: Toran Bruce Richards <[email protected]>
Changes: