-
Notifications
You must be signed in to change notification settings - Fork 54
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chore: move deployment docs to separate documents #869
Open
marcklingen
wants to merge
3
commits into
main
Choose a base branch
from
move-deployment-docs-to-separate-documents
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
<Callout type="info"> | ||
Platform-specific deployment guides rely on community contributions, the | ||
Langfuse team maintains the [self-host](/docs/deployment/self-host) guide. | ||
Please let us know if this guide does not work anymore or if you have a better | ||
solution via [Discord](/discord) or by raising an [issue](/issue) on GitHub. | ||
</Callout> | ||
|
||
<Callout type="warning"> | ||
Before following this guide, please make sure that you are familiar with the | ||
[self-hosting documentation](/docs/deployment/self-host). | ||
</Callout> | ||
|
||
[Langfuse](/docs) is open source (MIT licensed core product) and you can deploy it on any platform to add observability, evaluation and prompt management to your LLM applications. Langfuse is configurable via environment variables, you can find the full list of available configuration options in the [self-host guide](/docs/deployment/self-host#configuring-environment-variables). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
export default { | ||
aws: "AWS", | ||
azure: "Azure", | ||
"google-cloud-platform": "Google Cloud Platform (GCP)", | ||
heroku: "Heroku", | ||
railway: "Railway", | ||
porter: "Porter.run", | ||
kubernetes: { | ||
title: "Kubernetes", | ||
href: "/docs/deployment/self-host#kubernetes-deployments", | ||
}, | ||
}; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
--- | ||
description: "Deploy Langfuse (Open Source Observability, Evaluation and Prompt Management for LLM Applications) on AWS" | ||
--- | ||
|
||
# Deploy Langfuse on AWS | ||
|
||
import DeploymentGuideCallout from "@/components-mdx/deployment-guide-callout.mdx"; | ||
|
||
<DeploymentGuideCallout /> | ||
|
||
We recommend deploying Langfuse on AWS using the Elastic Container Service (ECS) and Fargate for a scalable and low-maintenance container deployment. Note: you can use AWS Cognito for SSO. | ||
|
||
## Template | ||
|
||
Have a look at this configuration template: [aws-samples/deploy-langfuse-on-ecs-with-fargate](https://github.com/aws-samples/deploy-langfuse-on-ecs-with-fargate) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
--- | ||
description: "Deploy Langfuse (Open Source Observability, Evaluation and Prompt Management for LLM Applications) on Azure" | ||
--- | ||
|
||
# Deploy Langfuse on Azure | ||
|
||
import DeploymentGuideCallout from "@/components-mdx/deployment-guide-callout.mdx"; | ||
|
||
<DeploymentGuideCallout /> | ||
|
||
Deploy Langfuse to Azure using the Azure Container Instances service for a flexible and low-maintenance container deployment. Note: you can use Azure AD for SSO. | ||
|
||
You can deploy Langfuse to Azure via the Azure Developer CLI using this template: [Azure-Samples/langfuse-on-azure](https://github.com/Azure-Samples/langfuse-on-azure). |
137 changes: 137 additions & 0 deletions
137
pages/docs/deployment/platform-guides/google-cloud-platform.mdx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,137 @@ | ||
--- | ||
description: "Deploy Langfuse (Open Source Observability, Evaluation and Prompt Management for LLM Applications) on Google Cloud Platform (Cloud Run & Cloud SQL)" | ||
--- | ||
|
||
# Deploy Langfuse on Google Cloud Platform (Cloud Run & Cloud SQL) | ||
|
||
import DeploymentGuideCallout from "@/components-mdx/deployment-guide-callout.mdx"; | ||
|
||
<DeploymentGuideCallout /> | ||
|
||
The simplest way to deploy Langfuse on Google Cloud Platform is to use Cloud Run for the containerized application and Cloud SQL for the database. | ||
|
||
## Option 1: UI Deployment | ||
|
||
**Create Cloud SQL Instance:** | ||
|
||
1. Open Google Cloud SQL. | ||
2. Click on Create Instance. | ||
3. Choose PostgreSQL and configure the instance according to your requirements. | ||
4. You'll need the following details: | ||
- default > user: postgres | ||
- default > database schema: public | ||
- setup > password: `<password>` | ||
- connection > connection name: `<google-cloud-project-id>:<region-id>:<sql-instance-id>` | ||
|
||
**Optionally: Create OAuth Credentials for sign-in with Google** | ||
|
||
1. Open [API Credentials](https://console.cloud.google.com/apis/credentials) | ||
2. Click "Create Credentials" and then "OAuth Client ID" | ||
3. Choose "Web Application" and then give it an appropriate name | ||
4. Click Create | ||
|
||
**Create Secrets:** | ||
|
||
1. Open [Secret Manager](https://console.cloud.google.com/security/secret-manager) | ||
2. For each secret needed (at least `AUTH_GOOGLE_CLIENT_ID, AUTH_GOOGLE_CLIENT_SECRET, DATABASE_URL, DIRECT_URL, NEXTAUTH_SECRET, NEXTAUTH_URL,` and `SALT`), click "Create Secret" and fill in the name and value. | ||
|
||
Notes: | ||
|
||
- `DATABASE_URL` is the connection string to the Cloud SQL instance. `postgresql://<user-name>:<password>@localhost/<db-name>/?host=/cloudsql/<google-cloud-project-id>:<region-id>:<sql-instance-id>&sslmode=none&pgbouncer=true` | ||
- `DIRECT_URL` is for database migrations, without `&pgbouncer=true`, the value should look like this: `postgresql://<user-name>:<password>@localhost/<db-name>/?host=/cloudsql/<google-cloud-project-id>:<region-id>:<sql-instance-id>&sslmode=none` | ||
- Set `NEXTAUTH_URL` to `http://localhost:3000`. This is a placeholder, we'll update it later. | ||
|
||
**Deploy on Cloud Run:** | ||
|
||
1. Open Google Cloud Run. | ||
2. Click on Create Service. | ||
3. Enter the following container image URL: `docker.io/langfuse/langfuse:2`. We use tag `2` to pin the major version. | ||
4. Configure the service name and region according to your requirements. | ||
5. Select authentication as 'Allow unauthenticated invocations', as Langfuse will have its own built-in Authentication that you can use. | ||
6. Choose 'CPU Allocation and Pricing' as "CPU is only allocated during request processing" to scale down the instance to 0 when there are no requests. | ||
7. Configure ingress control according to your needs. For most cases, 'All' should suffice. | ||
8. "Container(s), Volumes, Networking & Security": | ||
|
||
- Specify container port as `3000`. | ||
- On "Variables & Secrets" tab, add the required environment variables (see table above): `SALT`, `NEXTAUTH_URL`, `NEXTAUTH_SECRET`, and `DATABASE_URL`, etc. | ||
|
||
9. Scroll all the way down to enable the Cloud SQL connections. Select the created Cloud SQL instance in the dropdown. Context: Your Cloud Run service won't be assigned a static IP, so you can't whitelist the ingress IP in Cloud SQL or any other hosted databases. Instead, we use the Google Cloud SQL Proxy. | ||
10. Finally, you can finish deploying the application. | ||
11. While the application is deployed for the first time, you can see how the database migrations are applied in the logs. | ||
12. Once the application is up and running, you can find the Cloud Run service URL on top of the page. Now, choose "Edit and deploy new revision" to update the `NEXTAUTH_URL` environment variable to the Cloud Run service URL ending in `.run.app`. | ||
13. Optionally, configure a custom domain for the Cloud Run service. | ||
|
||
**Troubleshooting: Cloud SQL Connection Issues** | ||
|
||
If you encounter an error like "Error 403: boss::NOT_AUTHORIZED: Not authorized to access resource" or "Possibly missing permission cloudsql.instances.connect" when deploying the Langfuse container, you may need to grant 'Cloud SQL Client' permissions to the relevant service accounts. Here's how to resolve this: | ||
|
||
1. In the Google Cloud search box, search for and select "Service Accounts". | ||
2. Find the service accounts with names ending in `@appspot.gserviceaccount.com` and `[email protected]`. | ||
3. In the Google Cloud search box, search for and select "IAM & Admin". | ||
4. Click "Grant Access", then "Add Principals". | ||
5. Enter the name of the first service account you found. | ||
6. Select the "Cloud SQL Client" role and save. | ||
7. Repeat steps 4-6 for the second service account. | ||
|
||
After granting these permissions, try redeploying your Cloud Run service. This should resolve any authorization issues related to connecting to your Cloud SQL instance. | ||
|
||
## Option 2: Cloud Build | ||
|
||
Google Cloud Build is GCP's continuous integration and continuous deployment (CI/CD) service that automates the building, testing, and deployment of your applications. To deploy Langfuse, you can specify your workflow in a cloudbuild.yaml file. Additionally, GCP's Secret Manager can be used to securely handle sensitive information like DATABASE_URL and NEXTAUTH_SECRET. Below is an example of how to set up a Cloud Build configuration: | ||
|
||
```yaml | ||
# Deployment configuration for Langfuse on Google Cloud Run | ||
substitutions: | ||
_SERVICE_NAME: langfuse | ||
_REGION: europe-west1 # Change to your desired region | ||
_PROJECT_ID: your-project-id # Change to your Google Cloud project ID | ||
_SQL_INSTANCE_ID: my-cool-db # the name of the cloud sql database you create | ||
|
||
tags: ["${_PROJECT_ID}", "${_SERVICE_NAME}"] | ||
|
||
steps: | ||
# Step to deploy the Docker image to Google Cloud Run | ||
- name: "gcr.io/cloud-builders/gcloud" | ||
id: deploy-cloud-run | ||
entrypoint: bash | ||
args: | ||
- "-c" | ||
- | | ||
gcloud run deploy ${_SERVICE_NAME} --image docker.io/langfuse/langfuse:2 \ | ||
--region ${_REGION} \ | ||
--project ${_PROJECT_ID} \ | ||
--platform managed \ | ||
--port 3000 \ | ||
--allow-unauthenticated \ | ||
--memory 2Gi \ | ||
--cpu 1 \ | ||
--min-instances 0 \ | ||
--max-instances 3 \ | ||
--set-env-vars HOSTNAME=0.0.0.0 \ | ||
--add-cloudsql-instances=_PROJECT_ID:_REGION:_SQL_INSTANCE_ID \ | ||
--update-secrets AUTH_GOOGLE_CLIENT_ID=AUTH_GOOGLE_CLIENT_ID:latest,AUTH_GOOGLE_CLIENT_SECRET=AUTH_GOOGLE_CLIENT_SECRET:latest,SALT=SALT:latest,NEXTAUTH_URL=NEXTAUTH_URL:latest,NEXTAUTH_SECRET=NEXTAUTH_SECRET:latest,DATABASE_URL=DATABASE_URL:latest,DIRECT_URL=DIRECT_URL:latest | ||
``` | ||
|
||
You can submit this build using [`gcloud build submit`](https://cloud.google.com/sdk/gcloud/reference/builds/submit) in your local console by issuing the below in the same folder as the `cloudbuild.yaml` file. | ||
|
||
To submit this build, use the following command in your local console, in the directory containing the `cloudbuild.yaml` file: | ||
|
||
``` | ||
gcloud builds submit . | ||
``` | ||
|
||
For automatic rebuilds upon new commits, set up a [Cloud Build Trigger](https://cloud.google.com/build/docs/automating-builds/create-manage-triggers) linked to your repository holding the `cloudbuild.yaml` file. This will redeploy Langfuse whenever changes are pushed to the repository. | ||
|
||
## Note on AlloyDB | ||
|
||
[AlloyDB](https://cloud.google.com/alloydb) is a fully-managed postgres compatible database offered by Google Cloud Platform that is tuned for better performance for tasks such as analytical queries and in-database embeddings. It is recommend you use it within a [Shared VPC](https://cloud.google.com/vpc/docs/shared-vpc) with your Cloud Run runtime, which will expose AlloyDB's private ip address to your application. If you are using it the DB connection string changes slightly: | ||
|
||
``` | ||
# ALLOYDB_CONNECTION_STRING | ||
postgresql://<USER>:<PASSWORD>@<ALLOY_DB_PRIVATE_IP>:5432/<ALLOY_DB_DATABASE>/?sslmode=none&pgbouncer=true | ||
``` | ||
|
||
``` | ||
# ALLOYDB_DIRECT_URL | ||
postgresql://<USER>:<PASSWORD>@<ALLOY_DB_PRIVATE_IP>:5432/<ALLOY_DB_DATABASE>/?sslmode=none | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,96 @@ | ||
--- | ||
description: "Deploy Langfuse (Open Source Observability, Evaluation and Prompt Management for LLM Applications) on Heroku" | ||
--- | ||
|
||
# Deploy Langfuse on Heroku | ||
|
||
import DeploymentGuideCallout from "@/components-mdx/deployment-guide-callout.mdx"; | ||
|
||
<DeploymentGuideCallout /> | ||
|
||
To deploy this image on heroku you have to run through the steps in the following deployment guide: | ||
|
||
1. **Pull the docker image. This can be achieved by running the following command in your terminal:** | ||
|
||
``` | ||
docker pull langfuse/langfuse:2 | ||
``` | ||
|
||
2. **Get the ID of the pulled image** | ||
|
||
_Linux / MacOS_: | ||
|
||
Running the following command should result in directly printing the image ID | ||
|
||
``` | ||
docker images | grep langfuse/langfuse | awk '[print $3]' | ||
``` | ||
|
||
Following this tutorial, you will always have to insert this image ID when [IMAGE_ID] is written. | ||
|
||
_Windows_: | ||
|
||
On windows you can print the full information of the pulled image using: | ||
|
||
``` | ||
docker images | findstr /S "langfuse/langfuse" | ||
``` | ||
|
||
This will result in something like: | ||
|
||
``` | ||
langfuse/langfuse 2 cec90c920468 28 hours ago 595MB | ||
``` | ||
|
||
Here you have to manually retrieve the image ID which in this case is `cec90c920468`. It should be located between the tag `2` and the created `28 hours ago` in this example. | ||
|
||
3. **Prepare your terminal and docker image** | ||
|
||
First of all, you will have to be logged in to heroku using | ||
|
||
``` | ||
heroku login | ||
``` | ||
|
||
If this is not working, please visit the [heroku CLI setup](https://devcenter.heroku.com/articles/heroku-cli). | ||
|
||
If you succeeded in logging in to heroku via the CLI, you can continue by following the next steps: | ||
|
||
Tag the docker image (Insert your image ID into the command). You will also have to insert the name of your heroku app/dyno into [HEROKU_APP_NAME]: | ||
|
||
``` | ||
docker tag [IMAGE_ID] registry.heroku.com/[HEROKU_APP_NAME]/web | ||
``` | ||
|
||
4. **Setup a database for your heroku app** | ||
|
||
In the dashboard of your heroku app, add the `Heroku Postgres`-AddOn. This will add a PostgreSQL database to your application. | ||
|
||
5. **Set the environment variables** | ||
|
||
For the minimum deployment in heroku, you will have to set the following environment variables (see table above). The `DATABASE_URL` is your database connection string starting with `postgres://` in the configuration of your added PostgreSQL database. | ||
|
||
``` | ||
DATABASE_URL= | ||
NEXTAUTH_SECRET= | ||
NEXTAUTH_URL= | ||
SALT= | ||
``` | ||
|
||
Have a look at the other optional environment variables in the table above and set them if needed to configure your deployment. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. logic: Again, reference to 'table above' is incorrect |
||
|
||
6. **Push to heroku container registry** | ||
|
||
In this step you will push the docker image to the heroku container registry: (Insert the name of your heroku app/dyno) | ||
|
||
``` | ||
docker push registry.heroku.com/[HEROKU_APP_NAME]/web | ||
``` | ||
|
||
7. **Deploy the docker image from the heroku registry** | ||
|
||
In the last step you will have to execute the following command to finally deploy the image. Again insert the name of your heroku app: | ||
|
||
``` | ||
heroku container:release web --app=[HEROKU_APP_NAME] | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
--- | ||
description: "Deploy Langfuse (Open Source Observability, Evaluation and Prompt Management for LLM Applications) on Porter.run" | ||
--- | ||
|
||
# Deploy Langfuse on Porter.run | ||
|
||
import DeploymentGuideCallout from "@/components-mdx/deployment-guide-callout.mdx"; | ||
|
||
<DeploymentGuideCallout /> | ||
|
||
If you use [Porter](https://porter.run) to deploy your application, you can easily add a Langfuse instance to your cluster via the "Add-ons". The add-on will automatically configure the necessary environment variables, inject your database credentials, handle HTTPS and custom domains, and deploy and autoscale the Langfuse container. | ||
|
||
![Porter.run](/images/changelog/2024-08-15-deployment-as-porter-add-on.jpg) | ||
|
||
Thanks to the Porter team for this great integration! |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
--- | ||
description: "Deploy Langfuse (Open Source Observability, Evaluation and Prompt Management for LLM Applications) on Railway" | ||
--- | ||
|
||
# Deploy Langfuse on Railway | ||
|
||
import DeploymentGuideCallout from "@/components-mdx/deployment-guide-callout.mdx"; | ||
|
||
<DeploymentGuideCallout /> | ||
|
||
Railway is a cloud platform that provides a simple and easy way to deploy and manage applications. With the template provided by the Langfuse team, you can easily deploy a Langfuse instance to Railway. | ||
|
||
## Get started | ||
|
||
<Steps> | ||
|
||
### Clone template | ||
|
||
[![Deploy on Railway](https://railway.app/button.svg)](https://railway.app/template/gmbqa_) | ||
|
||
### Configure environment variables | ||
|
||
While deploying to Railway, you can configure the Langfuse instance by setting the environment variables. Have a look [here](/docs/deployment/self-host#configuring-environment-variables) for available options. | ||
|
||
</Steps> | ||
|
||
## Upgrade | ||
|
||
Updating Langfuse on Railway is as easy as deploying a new version, just stop the current instance and start a new one. The Railway app will automatically recognize that a new version is available and will run the latest version. Changes are automatically applied to the database. |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
logic: Reference to 'table above' is incorrect as there is no table in this document