Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support SQLite database #345

Closed
2 tasks done
mitchnielsen opened this issue Jun 5, 2024 · 3 comments · Fixed by #418
Closed
2 tasks done

Support SQLite database #345

mitchnielsen opened this issue Jun 5, 2024 · 3 comments · Fixed by #418
Labels
feature A new feature

Comments

@mitchnielsen
Copy link
Contributor

mitchnielsen commented Jun 5, 2024

Summary

In #336, we found that the database configuration was centralized within PostgreSQL - meaning database configuration couldn't be provided outside of the postgresql key, which matters if someone wants to use MySQL or even SQLite.

Let's consider providing a new top-level key for configuring the database that's considered if postgresql.enabled=false.

Notes

The Prefect docs mention:

Prefect supports the following databases:

  1. SQLite (default in Prefect): Recommended for lightweight, single-server deployments. SQLite requires essentially no setup.
  2. PostgreSQL: Best for connecting to external databases, but requires additional setup (such as Docker). Prefect uses the pg_trgm extension, so it must be installed and enabled.

Elsewhere, it says:

Use SQLite database for testing only
SQLite does not support multiprocessing. For high orchestration volume, use PostgreSQL.

So realisticically, most users should be on PostgreSQL. But the documentation shows you how to use SQLite, so I think it's reasonable to support that configuration in the charts. As mentioned in #365, users are already using SQLite with the charts with some workarounds.

Acceptance criteria

  • Database configuration can be provided outside of the top-level postgresql key if PostgreSQL is not in use.
  • SQLite specifically is tested to confirm nothing is hard-coded to work with PostgreSQL
@mitchnielsen
Copy link
Contributor Author

I bet this was indirectly addressed by https://github.com/PrefectHQ/prefect-helm/pull/365, but let's try it to be sure and add an SQLite test case to be sure.

@mitchnielsen
Copy link
Contributor Author

@tmyhu - wanted to tag you since you mentioned you were using SQLite. We added direct support for it in https://github.com/PrefectHQ/prefect-helm/releases/tag/2024.12.5195120. If you have any suggestions feel free to create an issue/PR 🤝

@tmyhu
Copy link
Contributor

tmyhu commented Dec 5, 2024

Awesome, that looks great. Thanks for letting me know @mitchnielsen

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature A new feature
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants