[Feature]: Decouple database(Postgres) migration from proxy server initialization

[nihal1294]

Current Approach

I noticed that the current implementation uses the prisma db push --accept-data-loss command as part of the setup process every time the proxy server starts. While this approach works, it carries significant risks:

1. Potential data loss due to the --accept-data-loss flag
2. Lack of version control for database changes
3. Possible race conditions if multiple instances start simultaneously
4. Unnecessary database operations on every server start

Suggested Implementation:

1. Create a separate script for database migrations using Prisma Migrate
2. Update proxy server deployment documentation to include running migrations as a separate step
3. Remove the Prisma db push command from the proxy server initialization process
4. Add a check in the proxy server startup to verify the database schema version without modifying it, or only verify if the necessary tables exist

Questions:

1. Are there any specific reasons for the current approach that need to be considered?
2. Would this change impact any other parts of the project setup or deployment process?

Motivation

The official Prisma documentation explicitly advises against using Prisma db push in production environments:
Prototyping your schema

The documentation states that db push is meant for quick schema prototyping, not for managing production database schemas. Using it in a production setting goes against best practices and puts data at risk.

Furthermore, it is mentioned in the documentation that prisma migrate dev is also a development-only command and should never be used in production environments due to its reliance on a shadow database and potential database resets.
Development and production

Alternative Solution:

Implement a separate, decoupled database migration process that:

1. Runs as a distinct step in the deployment pipeline
2. Uses Prisma Migrate for version-controlled schema changes
3. Separates the concerns of database schema management from server initialization

Replace the current Prisma db push approach with Prisma Migrate:

1. For development:
   prisma migrate dev
   This command creates a new migration, applies it to the database, and updates the Prisma client. The generated migration files should be committed to the repo.

2. For production:
   prisma migrate deploy
   This command applies all pending migrations, ensuring the database schema is up-to-date without risking data loss.

These commands should be run during the deployment process, not within the proxy server initialization.

Implementing this change would enhance the robustness and safety of the setup process, making it more suitable for production use. I believe this improvement aligns with the project's goal of providing a reliable and scalable solution for interacting with various LLM APIs.

Benefits:

1. Improved data safety by removing the risk of unintended data loss
2. Better control and tracking of database schema changes
3. Reduced risk of database-related issues during proxy server startup
4. Alignment with database management best practices

Note

For whatever reason if this proposed solution is not preferred or not suitable for the proxy server, at the very least, there should be an option to disable running of the prisma db push --accept-data-loss command when running the proxy server, via an argument variable or the config file, to make it more suitable to use in a production deployment reliably.

Twitter / LinkedIn details

Twitter: @nihal1294

[krrishdholakia]

This seems more appropriate as a discussiong, moving there.

[krrishdholakia]

Hi @nihal1294,

Update proxy server deployment documentation to include running migrations as a separate step

• Add a check in the proxy server startup to verify the database schema version without modifying it, or only verify if the necessary tables exist

Could you share how you'd expect this to work?

Follow-up q:

• How would schema be updated?
• How would the initial set-up for litellm w/ db be impacted?

[nihal1294]

Hi @krrishdholakia

I assume that the reason to run the prisma db push --accept-data-loss command from proxy_cli.py is to ensure that the necessary schema and tables exist for the proxy server to work with the database features.

Here's how the proposed approach would work:

1. Schema updates:

   • Developer creates migrations using prisma migrate dev during development. (In this case the maintainers and collaborators of liteLLM)
   • These migrations are committed to the repository.
   • During deployment, prisma migrate deploy is run to apply pending migrations.

2. Initial setup:

   • Add a separate setup script that runs prisma migrate deploy. (so that this command does not run every time the proxy server starts up)
   • Update documentation to instruct users to run this script before first use.

3. LiteLLM startup:

   • Remove prisma db push from the proxy server code.
   • Add a simple check to verify if the required tables exist, and the schema matches.

This approach separates schema management from the LiteLLM server, improving safety and maintainability. It requires minimal changes to the current workflow while aligning with database best practices.

[nihal1294]

Add a simple check to verify if the required tables exist, and the schema matches.

To do this, I would suggest versioning the schema and maintaining this version in one of the tables so that you can verify this version on proxy server startup, and show an appropriate log message, and/or stop the server.

Here's a code snippet to check for a schema mismatch between what LiteLLM expects and what's actually in the database.

from prisma import Prisma
from prisma.errors import PrismaError
import sys

EXPECTED_SCHEMA_VERSION = "1.2.3"  # This should match the current LiteLLM schema version

def check_schema_version():
    db = Prisma()
    db.connect()

    try:
        # Assuming you have a 'SchemaVersion' table to store the current schema version
        schema_version = db.schemaversion.find_first()
        
        if not schema_version or schema_version.version != EXPECTED_SCHEMA_VERSION:
            print(f"Schema version mismatch. Expected {EXPECTED_SCHEMA_VERSION}, but found {schema_version.version if schema_version else 'no version'}")
            print("Please run 'prisma migrate deploy' to update your database schema.")
            sys.exit(1)
        
        print("Schema version is up to date.")
    
    except PrismaError as e:
        if "Table `SchemaVersion` does not exist" in str(e):
            print("Schema version table not found. Database may not be initialized.")
            print("Please run 'prisma migrate deploy' to set up your database schema.")
            sys.exit(1)
        else:
            print(f"An error occurred while checking schema version: {e}")
            sys.exit(1)
    
    finally:
        db.disconnect()

If using another table for schema versioning is not desirable, we can also use a consistent checksum, say sha256 for the schema versioning, and just compare the new schema hash to check if that matches with the deployed schema hash. This can be done in a similar way to the above code snippet.

Let me know if you need help with this.