Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Global variables implementation #1548

Merged
merged 91 commits into from Mar 27, 2024
Merged
Show file tree
Hide file tree
Changes from 58 commits
Commits
Show all changes
91 commits
Select commit Hold shift + click to select a range
86ccae8
Feat: make text input capable of support dropdown inputs also
igorrCarvalho Feb 5, 2024
ccb7947
Merge branch 'dropdownInput' into globalVariables
igorrCarvalho Feb 5, 2024
5eef277
Remove blank space
igorrCarvalho Feb 5, 2024
1be52e2
Fix: arrow icon dont appear in dropdown
igorrCarvalho Feb 5, 2024
0a0f35c
Add global variable API functions
anovazzi1 Feb 5, 2024
c1d9e12
Add global variables store using Zustand
anovazzi1 Feb 5, 2024
f30c1df
Feat: make input dropdown filter with text
igorrCarvalho Feb 5, 2024
e4e5b06
fix(App.tsx): import getGlobalVariables function from API controller …
anovazzi1 Feb 5, 2024
911d681
feat(addNewVariableButton.tsx): implement form logic to handle saving…
anovazzi1 Feb 5, 2024
4a1c9f8
Add global variable functionality and display on Global Variables page
anovazzi1 Feb 5, 2024
d0dcb32
Refactor global variables page and add delete functionality
anovazzi1 Feb 5, 2024
476a670
Add global variables support to InputComponent
anovazzi1 Feb 5, 2024
1558934
Add delete_credential endpoint to API
ogabrielluiz Feb 6, 2024
84f4c32
Refactor Credential model to use Optional for provider field
ogabrielluiz Feb 6, 2024
937a504
Add load_from_db_fields attribute to Vertex class
ogabrielluiz Feb 7, 2024
108d0b2
Refactor class instantiation and update params with load_from_db_fields
ogabrielluiz Feb 7, 2024
ec1fe03
Update TemplateField class in base.py
ogabrielluiz Feb 7, 2024
45e2405
feat(parameterComponent): add support for global variables in paramet…
anovazzi1 Feb 7, 2024
a91668b
fix(parameterComponent): add options prop to ParameterComponent to di…
anovazzi1 Feb 8, 2024
b76eb65
Refactor API functions and update global variables types
anovazzi1 Feb 8, 2024
9c24aff
Refactor globalVariablesStore to use optional provider
anovazzi1 Feb 8, 2024
326a0f9
Refactor getGlobalVariables function to store data in globalVariables…
anovazzi1 Feb 8, 2024
021612f
Add provider field to AddNewVariableButton component
anovazzi1 Feb 8, 2024
c724d11
Refactor registerGlobalVariable function signature
anovazzi1 Feb 8, 2024
928ba60
Refactor credential creation logic
ogabrielluiz Feb 8, 2024
3d2a764
fix(parameterComponent): fix logic for setting load_from_db property …
anovazzi1 Feb 8, 2024
6b73ad2
Fix loading.py to handle custom component keys properly
ogabrielluiz Feb 9, 2024
56133bc
Update update_params_with_load_from_db_fields signature to include ty…
ogabrielluiz Feb 9, 2024
b57173d
Add docs to CustomComponent
ogabrielluiz Feb 9, 2024
06f0e0e
Added dropdown and global page
lucaseduoli Mar 18, 2024
c57aadb
Fix merge conflicts in backend code
ogabrielluiz Mar 18, 2024
9bcd74c
Refactor code formatting and style
ogabrielluiz Mar 18, 2024
6c9a87b
Merge branch 'zustand/io/migration' into globalVariables
lucaseduoli Mar 18, 2024
e38cf0d
Changed component.py to fix bug
lucaseduoli Mar 18, 2024
735d5ed
Add imports and update log level in run_langflow function
ogabrielluiz Mar 19, 2024
50d27ee
Remove duplicate import statements and unused imports
ogabrielluiz Mar 19, 2024
d003be4
Updated global variables to ShadCN and updated logic to show variable…
lucaseduoli Mar 20, 2024
1b61025
Fixed logic when options are present but its not global variables.
lucaseduoli Mar 20, 2024
6dc4599
Changed icons for other options that are not global variable
lucaseduoli Mar 20, 2024
41cbd91
Deleted variables page
lucaseduoli Mar 20, 2024
ea21106
Implemented Delete function
lucaseduoli Mar 21, 2024
ca46044
Fixed position on command when filtering
lucaseduoli Mar 21, 2024
ba9124c
Formatting
lucaseduoli Mar 21, 2024
99561d0
Merge remote-tracking branch 'origin/zustand/io/migration' into globa…
lucaseduoli Mar 21, 2024
1604680
Formatting
lucaseduoli Mar 21, 2024
178db4b
Fix merge bug
lucaseduoli Mar 21, 2024
3b7cae8
Update delete_credential function to return status code 204
ogabrielluiz Mar 21, 2024
69735b0
Refactor addNewVariableButtonComponent and API registerGlobalVariable
ogabrielluiz Mar 21, 2024
7f34516
Change key to name
ogabrielluiz Mar 21, 2024
fbe346c
Refactor globalVariables key names in GlobalVariablesStore
ogabrielluiz Mar 21, 2024
df62170
Implemented error alerts and function to clean deleted global variables
lucaseduoli Mar 22, 2024
ad8e016
Fixed error message appearing when deleting field
lucaseduoli Mar 22, 2024
0f867dc
Merge remote-tracking branch 'origin/zustand/io/migration' into globa…
lucaseduoli Mar 22, 2024
cded4aa
Refactor build_template_config method to use eval_custom_component_code
ogabrielluiz Mar 22, 2024
002da17
Remove unecessary method
ogabrielluiz Mar 22, 2024
479d6ec
Refactor template_config generation in Component class
ogabrielluiz Mar 22, 2024
b121a03
Update description for AddNewVariableButton component
ogabrielluiz Mar 22, 2024
fe2f30d
Added global input component to modularize
lucaseduoli Mar 22, 2024
7aa3c5f
Added type into new Global input component
lucaseduoli Mar 22, 2024
2ba305b
Add Variable model and remove Credential model
ogabrielluiz Mar 25, 2024
6d258a7
Add variables_router to APIRouter
ogabrielluiz Mar 25, 2024
8292cb8
Update global variable type in API and store
ogabrielluiz Mar 25, 2024
f8bd698
Remove UTC import and fix datetime function call
anovazzi1 Mar 25, 2024
dc531de
fix datetime utc for python<3.11
anovazzi1 Mar 25, 2024
bb3461c
Refactor Variable model in database
ogabrielluiz Mar 25, 2024
0175fc4
Add support for loading template field from the database
ogabrielluiz Mar 26, 2024
f741b97
Fix capitalization in error messages
ogabrielluiz Mar 26, 2024
5db74fb
Changed provider to type and hid field
lucaseduoli Mar 26, 2024
42739a2
Added types
lucaseduoli Mar 26, 2024
ae7d827
Fixed type to always exist
lucaseduoli Mar 26, 2024
1315137
put optional label on type
lucaseduoli Mar 26, 2024
d123d5b
Added editNode variable on global variables for inputs
lucaseduoli Mar 26, 2024
ff8c3d7
Fixed input of global variables
lucaseduoli Mar 26, 2024
95f5c44
Merge remote-tracking branch 'origin/zustand/io/migration' into globa…
ogabrielluiz Mar 26, 2024
0385cfb
Add setup_devcontainer target to Makefile
ogabrielluiz Mar 26, 2024
00322ed
Refactor Makefile to include conditional statements for environment v…
ogabrielluiz Mar 26, 2024
5326155
Add VariableService and related files
ogabrielluiz Mar 26, 2024
16e91a0
Update mypy configuration to include namespace packages
ogabrielluiz Mar 27, 2024
a92dcaf
Update agent.py and experimental components
ogabrielluiz Mar 27, 2024
257f802
Update function signature and import statements
ogabrielluiz Mar 27, 2024
d54e450
Refactor code and fix bugs
ogabrielluiz Mar 27, 2024
827d6be
Add new langflow helper functions and remove base model component
ogabrielluiz Mar 27, 2024
73a23ca
Add new files and modify existing files
ogabrielluiz Mar 27, 2024
b603d73
Add version module and update imports
ogabrielluiz Mar 27, 2024
56a485e
Update packages include path in pyproject.toml
ogabrielluiz Mar 27, 2024
2bc6d99
Update Poetry version to 1.8.2
ogabrielluiz Mar 27, 2024
eaf2479
Update function signature, refactor code, add new langflow helper fun…
ogabrielluiz Mar 27, 2024
fc432f2
Update poetry version to 1.8.2 in Dockerfiles
ogabrielluiz Mar 27, 2024
a08b84c
Merge branch 'fix_lint_problems' into globalVariables
ogabrielluiz Mar 27, 2024
1434081
Merge branch 'zustand/io/migration' into globalVariables
ogabrielluiz Mar 27, 2024
75d1c83
Merge branch 'zustand/io/migration' into globalVariables
ogabrielluiz Mar 27, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
13 changes: 6 additions & 7 deletions src/backend/langflow/__main__.py
Expand Up @@ -6,6 +6,11 @@

import typer
from dotenv import load_dotenv
from langflow.main import setup_app
from langflow.services.database.utils import session_getter
from langflow.services.deps import get_db_service, get_settings_service
from langflow.services.utils import initialize_services, initialize_settings_service
from langflow.utils.logger import configure, logger
from multiprocess import cpu_count # type: ignore
from rich import box
from rich import print as rprint
Expand All @@ -14,12 +19,6 @@
from rich.table import Table
from sqlmodel import select

from langflow.main import setup_app
from langflow.services.database.utils import session_getter
from langflow.services.deps import get_db_service, get_settings_service
from langflow.services.utils import initialize_services, initialize_settings_service
from langflow.utils.logger import configure, logger

console = Console()

app = typer.Typer(no_args_is_help=True)
Expand Down Expand Up @@ -304,7 +303,7 @@ def run_langflow(host, port, log_level, options, app):
app,
host=host,
port=port,
log_level=log_level,
log_level=log_level.lower(),
)
else:
from langflow.server import LangflowApplication
Expand Down
15 changes: 11 additions & 4 deletions src/backend/langflow/api/v1/credential.py
Expand Up @@ -6,7 +6,12 @@

from langflow.services.auth import utils as auth_utils
from langflow.services.auth.utils import get_current_active_user
from langflow.services.database.models.credential import Credential, CredentialCreate, CredentialRead, CredentialUpdate
from langflow.services.database.models.credential import (
Credential,
CredentialCreate,
CredentialRead,
CredentialUpdate,
)
from langflow.services.database.models.user.model import User
from langflow.services.deps import get_session, get_settings_service

Expand All @@ -25,7 +30,10 @@ def create_credential(
try:
# check if credential name already exists
credential_exists = session.exec(
select(Credential).where(Credential.name == credential.name, Credential.user_id == current_user.id)
select(Credential).where(
Credential.name == credential.name,
Credential.user_id == current_user.id,
)
).first()
if credential_exists:
raise HTTPException(status_code=400, detail="Credential name already exists")
Expand Down Expand Up @@ -90,7 +98,7 @@ def update_credential(
raise HTTPException(status_code=500, detail=str(e)) from e


@router.delete("/{credential_id}", response_model=CredentialRead, status_code=200)
@router.delete("/{credential_id}", status_code=204)
def delete_credential(
*,
session: Session = Depends(get_session),
Expand All @@ -106,6 +114,5 @@ def delete_credential(
raise HTTPException(status_code=404, detail="Credential not found")
session.delete(db_credential)
session.commit()
return db_credential
except Exception as e:
raise HTTPException(status_code=500, detail=str(e)) from e
70 changes: 40 additions & 30 deletions src/backend/langflow/graph/vertex/base.py
Expand Up @@ -14,8 +14,6 @@
Optional,
)

from loguru import logger

from langflow.graph.schema import (
INPUT_COMPONENTS,
INPUT_FIELD_NAME,
Expand All @@ -31,6 +29,7 @@
from langflow.utils.constants import DIRECT_TYPES
from langflow.utils.schemas import ChatOutputResponse
from langflow.utils.util import sync_to_async, unescape_string
from loguru import logger

if TYPE_CHECKING:
from langflow.graph.edge.base import ContractEdge
Expand Down Expand Up @@ -81,6 +80,7 @@ def __init__(
self.is_task = is_task
self.params = params or {}
self.parent_node_id: Optional[str] = self._data.get("parent_node_id")
self.load_from_db_fields: List[str] = []
self.parent_is_top_level = False
self.layer = None
self.should_run = True
Expand Down Expand Up @@ -167,6 +167,7 @@ def __getstate__(self):
"_built": False,
"parent_node_id": self.parent_node_id,
"parent_is_top_level": self.parent_is_top_level,
"load_from_db_fields": self.load_from_db_fields,
"is_input": self.is_input,
"is_output": self.is_output,
}
Expand Down Expand Up @@ -196,6 +197,7 @@ def __setstate__(self, state):
self.task_id: Optional[str] = None
self.parent_node_id = state["parent_node_id"]
self.parent_is_top_level = state["parent_is_top_level"]
self.load_from_db_fields = state["load_from_db_fields"]
self.layer = state.get("layer")
self.steps = state.get("steps", [self._build])

Expand Down Expand Up @@ -294,68 +296,76 @@ def _build_params(self):
else:
params[param_key] = self.graph.get_vertex(edge.source_id)

for key, value in template_dict.items():
if key in params:
load_from_db_fields = []
for field_name, field in template_dict.items():
if field_name in params:
continue
# Skip _type and any value that has show == False and is not code
# If we don't want to show code but we want to use it
if key == "_type" or (not value.get("show") and key != "code"):
if field_name == "_type" or (not field.get("show") and field_name != "code"):
continue
# If the type is not transformable to a python base class
# then we need to get the edge that connects to this node
if value.get("type") == "file":
if field.get("type") == "file":
# Load the type in value.get('fileTypes') using
# what is inside value.get('content')
# value.get('value') is the file name
if file_path := value.get("file_path"):
if file_path := field.get("file_path"):
storage_service = get_storage_service()
flow_id, file_name = file_path.split("/")
full_path = storage_service.build_full_path(flow_id, file_name)
params[key] = full_path
params[field_name] = full_path
else:
raise ValueError(f"File path not found for {self.display_name}")
elif value.get("type") in DIRECT_TYPES and params.get(key) is None:
val = value.get("value")
if value.get("type") == "code":
elif field.get("type") in DIRECT_TYPES and params.get(field_name) is None:
val = field.get("value")
if field.get("type") == "code":
try:
params[key] = ast.literal_eval(val) if val else None
params[field_name] = ast.literal_eval(val) if val else None
except Exception:
params[key] = val
elif value.get("type") in ["dict", "NestedDict"]:
params[field_name] = val
elif field.get("type") in ["dict", "NestedDict"]:
# When dict comes from the frontend it comes as a
# list of dicts, so we need to convert it to a dict
# before passing it to the build method
if isinstance(val, list):
params[key] = {k: v for item in value.get("value", []) for k, v in item.items()}
params[field_name] = {k: v for item in field.get("value", []) for k, v in item.items()}
elif isinstance(val, dict):
params[key] = val
elif value.get("type") == "int" and val is not None:
params[field_name] = val
elif field.get("type") == "int" and val is not None:
try:
params[key] = int(val)
params[field_name] = int(val)
except ValueError:
params[key] = val
elif value.get("type") == "float" and val is not None:
params[field_name] = val
elif field.get("type") == "float" and val is not None:
try:
params[key] = float(val)
params[field_name] = float(val)
except ValueError:
params[key] = val
elif value.get("type") == "str" and val is not None:
params[field_name] = val
params[field_name] = val
elif field.get("type") == "str" and val is not None:
# val may contain escaped \n, \t, etc.
# so we need to unescape it
if isinstance(val, list):
params[key] = [unescape_string(v) for v in val]
params[field_name] = [unescape_string(v) for v in val]
elif isinstance(val, str):
params[key] = unescape_string(val)
params[field_name] = unescape_string(val)
elif val is not None and val != "":
params[field_name] = val

elif val is not None and val != "":
params[key] = val
params[field_name] = val
if field.get("load_from_db"):
load_from_db_fields.append(field_name)

if not value.get("required") and params.get(key) is None:
if value.get("default"):
params[key] = value.get("default")
if not field.get("required") and params.get(field_name) is None:
if field.get("default"):
params[field_name] = field.get("default")
else:
params.pop(key, None)
params.pop(field_name, None)
# Add _type to params
self.params = params
self.load_from_db_fields = load_from_db_fields
self._raw_params = params.copy()

def update_raw_params(self, new_params: Dict[str, str], overwrite: bool = False):
Expand Down
Expand Up @@ -39,8 +39,7 @@ def __init__(self, **data):
def __setattr__(self, key, value):
if key == "_user_id" and hasattr(self, "_user_id"):
warnings.warn("user_id is immutable and cannot be changed.")
else:
super().__setattr__(key, value)
super().__setattr__(key, value)

@cachedmethod(cache=operator.attrgetter("cache"))
def get_code_tree(self, code: str):
Expand All @@ -66,6 +65,12 @@ def get_function(self):
return validate.create_function(self.code, self._function_entrypoint_name)

def build_template_config(self) -> dict:
"""
Builds the template configuration for the custom component.

Returns:
A dictionary representing the template configuration.
"""
if not self.code:
return {}

Expand Down