-
-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add supavisor to supabase #2712
Conversation
Can you post the directions here? Discord link does not lead anywhere. |
Sure. Here it is and also need to run this in supavisor
there might be a way to set this up without running this, like with an init script tho and my url looks like this
|
- PROXY_PORT_TRANSACTION=6543 | ||
- DATABASE_URL=ecto://postgres:${SERVICE_PASSWORD_POSTGRES}@${POSTGRES_HOST:-supabase-db}:${POSTGRES_PORT:-5432}/${POSTGRES_DB:-postgres} | ||
- CLUSTER_POSTGRES=true | ||
- SECRET_KEY_BASE=12345678901234567890121234567890123456789012345678903212345678901234567890123456789032123456789012345678901234567890323456789032 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These probably shouldn't be hard coded
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What do you recommend? They need to specifically be this length
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ideally there's a way to generate a random string either through docker-compose or coolify, I couldn't find one though.
Considering this should be unique to every user, and I don't imagine many people are changing the defaults while using these, I'd either wait for the functionality to be added into coolify, or force the user to enter the information.
You can get docker-compose to error if an environment variable is missing using ${VAR:?error}
so that would mean changing it to: SECRET_KEY_BASE=${SECRET_KEY_BASE:?error}
.
Supabase is a fairly popular template, so people would need to be told that they need to put in a variable that contains a string that is X character long, but I don't know where a good place for the instruction to go would be.
Failing all that, maybe setting it to SECRET_KEY_BASE=${SECRET_KEY_BASE}
and finding where the default values for Dashboard User and Dashboard Password are generated and ensure a new one is generated for the 2 variables that need them.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Appears there is some functionality to do this:
coolify/bootstrap/helpers/shared.php
Lines 1969 to 2046 in e9158b7
case 'PASSWORD': | |
$generatedValue = Str::password(symbols: false); | |
break; | |
case 'PASSWORD_64': | |
$generatedValue = Str::password(length: 64, symbols: false); | |
break; | |
// This is not base64, it's just a random string | |
case 'BASE64_64': | |
$generatedValue = Str::random(64); | |
break; | |
case 'BASE64_128': | |
$generatedValue = Str::random(128); | |
break; | |
case 'BASE64': | |
case 'BASE64_32': | |
$generatedValue = Str::random(32); | |
break; | |
// This is base64, | |
case 'REALBASE64_64': | |
$generatedValue = base64_encode(Str::random(64)); | |
break; | |
case 'REALBASE64_128': | |
$generatedValue = base64_encode(Str::random(128)); | |
break; | |
case 'REALBASE64': | |
case 'REALBASE64_32': | |
$generatedValue = base64_encode(Str::random(32)); | |
break; | |
case 'USER': | |
$generatedValue = Str::random(16); | |
break; | |
case 'SUPABASEANON': | |
$signingKey = $service->environment_variables()->where('key', 'SERVICE_PASSWORD_JWT')->first(); | |
if (is_null($signingKey)) { | |
return; | |
} else { | |
$signingKey = $signingKey->value; | |
} | |
$key = InMemory::plainText($signingKey); | |
$algorithm = new Sha256(); | |
$tokenBuilder = (new Builder(new JoseEncoder(), ChainedFormatter::default())); | |
$now = new DateTimeImmutable(); | |
$now = $now->setTime($now->format('H'), $now->format('i')); | |
$token = $tokenBuilder | |
->issuedBy('supabase') | |
->issuedAt($now) | |
->expiresAt($now->modify('+100 year')) | |
->withClaim('role', 'anon') | |
->getToken($algorithm, $key); | |
$generatedValue = $token->toString(); | |
break; | |
case 'SUPABASESERVICE': | |
$signingKey = $service->environment_variables()->where('key', 'SERVICE_PASSWORD_JWT')->first(); | |
if (is_null($signingKey)) { | |
return; | |
} else { | |
$signingKey = $signingKey->value; | |
} | |
$key = InMemory::plainText($signingKey); | |
$algorithm = new Sha256(); | |
$tokenBuilder = (new Builder(new JoseEncoder(), ChainedFormatter::default())); | |
$now = new DateTimeImmutable(); | |
$now = $now->setTime($now->format('H'), $now->format('i')); | |
$token = $tokenBuilder | |
->issuedBy('supabase') | |
->issuedAt($now) | |
->expiresAt($now->modify('+100 year')) | |
->withClaim('role', 'service_role') | |
->getToken($algorithm, $key); | |
$generatedValue = $token->toString(); | |
break; | |
default: | |
$generatedValue = Str::random(16); | |
break; | |
} | |
return $generatedValue; | |
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
And the docs for how to use them: https://coolify.io/docs/knowledge-base/add-a-service#password
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe this will work.
- SECRET_KEY_BASE=${SERVICE_BASE64_128_SUPAVISOR}
- VAULT_ENC_KEY=${SERVICE_PASSWORD_64_SUPAVISOR}
Doesn't actually need to be 128 characters I don't think, fly deployment docs suggest some secure value: https://supabase.github.io/supavisor/deployment/fly/
Might be worth setting them both to PASSWORD_64 if only for better naming.
Thinking on the issue of running the
Maybe this could also be a run once container like the minio create-bucket, using a curl image and overriding the entrypoint. |
the vault secret has to be 32 characters:
and this tenant business might actually be even easier than that. i think we can just add this to the sql file where i create the _supavisor schema. but we have to encrypt the password on the fly -- which we probably do need a cron image for, unless sql can do it for us? config :supavisor, Supavisor.Vault,
ciphers: [
default: {
Cloak.Ciphers.AES.GCM,
tag: "AES.GCM.V1", key: System.get_env("VAULT_ENC_KEY")
}
] SET statement_timeout = 0;
SET lock_timeout = 0;
SET idle_in_transaction_session_timeout = 0;
SET client_encoding = 'UTF8';
SET standard_conforming_strings = ON;
SELECT pg_catalog.Set_config('search_path', '', false);
SET check_function_bodies = false;
SET xmloption = content;
SET client_min_messages = warning;
SET row_security = OFF;
--
-- Data for Name: tenants; Type: TABLE DATA; Schema: _supavisor; Owner: postgres
--
INSERT INTO _supavisor.tenants
(id,
external_id,
db_host,
db_port,
db_database,
inserted_at,
updated_at,
default_parameter_status,
ip_version,
upstream_ssl,
upstream_verify,
upstream_tls_ca,
enforce_ssl,
require_user,
auth_query,
default_pool_size,
sni_hostname,
default_max_clients,
client_idle_timeout,
default_pool_strategy,
client_heartbeat_interval,
allow_list)
VALUES ('7453afc5-d04c-45f7-b0ed-11edd51a3279',
'dev_tenant',
'supabase-db',
5432,
'postgres',
'2024-06-28 20:14:29',
'2024-06-28 20:14:29',
'{"server_version": "15.1 (Ubuntu 15.1-1.pgdg20.04+1)"}',
'auto',
false,
NULL,
NULL,
false,
false,
'SELECT rolname, rolpassword FROM pg_authid WHERE rolname=$1;',
15,
NULL,
1000,
0,
'fifo',
60,
'{0.0.0.0/0,::/0}');
--
-- Data for Name: users; Type: TABLE DATA; Schema: _supavisor; Owner: postgres
--
INSERT INTO _supavisor.users
(id,
db_user_alias,
db_user,
db_pass_encrypted,
pool_size,
mode_type,
is_manager,
tenant_external_id,
inserted_at,
updated_at,
pool_checkout_timeout,
max_clients)
VALUES ('524e2da5-b4cb-4917-88ed-ad6a304649e8',
'postgres',
'postgres',
'-the-encrypted-password-here-',
20,
'transaction',
true,
'dev_tenant',
'2024-06-28 20:14:29',
'2024-06-28 20:14:29',
60000,
NULL); the tricky part is to get the encrypted password there |
Chatgpt gave me a confidently incorrect answer, but it doesn't look like pgcrypto can be used with AES-GCM, it also suggested some code using the v8 extension to run js, but honestly at this point just sending the curl is probably best option. Shame |
so something like this could work? services:
init-curl:
image: curlimages/curl:8.8.0
command: >
sh -c "curl -X PUT 'http://supabase-supavisor:4000/api/tenants/dev_tenant' \
--header 'Accept: */*' \
--header 'User-Agent: Thunder Client (https://www.thunderclient.com)' \
--header 'Authorization: Bearer ${SERVICE_SUPABASEANON_KEY}' \
--header 'Content-Type: application/json' \
--data-raw '{
\"tenant\": {
\"db_host\": \"${POSTGRES_HOST:-supabase-db}\",
\"db_port\": ${POSTGRES_PORT:-5432},
\"db_database\": \"${POSTGRES_DB:-postgres}\",
\"ip_version\": \"auto\",
\"enforce_ssl\": false,
\"require_user\": false,
\"auth_query\": \"SELECT rolname, rolpassword FROM pg_authid WHERE rolname=$1;\",
\"users\": [
{
\"db_user\": \"postgres\",
\"db_password\": \"${SERVICE_PASSWORD_POSTGRES}\",
\"pool_size\": 20,
\"mode_type\": \"transaction\",
\"is_manager\": true
}
]
}
}'"
depends_on:
- supabase-supavisor
entrypoint: [ "sh", "-c" ]
supabase-supavisor:
...
depends_on:
supabase-db:
condition: service_healthy
init-curl:
condition: service_completed_successfully |
I think the only thing potentially missing is: One last thing, I wonder if those string replacements will work. If not you may need to link the environment variables to the curl container and pass them through using bash. https://github.com/coollabsio/coolify/blob/main/templates/compose/supabase.yaml#L1053-L1071 The minio gives an example of how this is done with a docker-compose defined file. |
i updated the pr, but needs testing |
you can check with
|
entrypoint: [ "sh", "-c" ] | ||
command: > | ||
#!/bin/sh | ||
|
||
while ! curl -sSfL --head -o /dev/null -H "Authorization: Bearer $SETUP_SERVICE_SUPABASEANON_KEY" http://supabase-supavisor:4000/api/health; do | ||
sleep 2; | ||
done; | ||
|
||
curl -X PUT 'http://supabase-supavisor:4000/api/tenants/dev_tenant' \ | ||
--header 'Accept: */*' \ | ||
--header 'User-Agent: Thunder Client (https://www.thunderclient.com)' \ | ||
--header 'Authorization: Bearer $SETUP_SERVICE_SUPABASEANON_KEY' \ | ||
--header 'Content-Type: application/json' \ | ||
--data-raw '{ | ||
"tenant": { | ||
"db_host": "${SETUP_POSTGRES_HOST}", | ||
"db_port": ${SETUP_POSTGRES_PORT}, | ||
"db_database": "${SETUP_POSTGRES_DB}", | ||
"ip_version": "auto", | ||
"enforce_ssl": false, | ||
"require_user": false, | ||
"auth_query": "SELECT rolname, rolpassword FROM pg_authid WHERE rolname=$1;", | ||
"users": [ | ||
{ | ||
"db_user": "postgres", | ||
"db_password": "${SETUP_SERVICE_PASSWORD_POSTGRES}", | ||
"pool_size": 20, | ||
"mode_type": "transaction", | ||
"is_manager": true | ||
} | ||
] | ||
} | ||
}' | ||
exit 0 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
entrypoint: sh
command: /docker-entrypoint-initdb.d/supavisor.sh
volumes:
-
type: bind
source: ./volumes/supavisor.sh
target: /docker-entrypoint-initdb.d/supavisor.sh
content: |
while ! curl -v -H "Authorization: Bearer ${SETUP_SERVICE_SUPABASEANON_KEY}" http://supabase-supavisor:4000/api/health;
do sleep 2;
done;
curl -X PUT -v \
'http://supabase-supavisor:4000/api/tenants/dev_tenant' \
--header 'Accept: */*' \
--header 'User-Agent: Thunder Client (https://www.thunderclient.com)' \
--header "Authorization: Bearer ${SETUP_SERVICE_SUPABASEANON_KEY}" \
--header 'Content-Type: application/json' \
--data '
{
"tenant": {
"db_host": "'"${SETUP_POSTGRES_HOST}"'",
"db_port": '$SETUP_POSTGRES_PORT',
"db_database": "postgres",
"ip_version": "auto",
"enforce_ssl": false,
"require_user": false,
"auth_query": "SELECT rolname, rolpassword FROM pg_authid WHERE rolname=$1;",
"users": [
{
"db_user": "postgres",
"db_password": "'"$SETUP_SERVICE_PASSWORD_POSTGRES"'",
"pool_size": 20,
"mode_type": "transaction",
"is_manager": true
},
]
}
}'
exit 0
fixes the formatting issues in the compose and vars in single quotes
also makes it much easier to add extra users, since you prob dont want to be using postgres user account you can just edit storage file
I have a much simpler way working: supabase-supavisor:
image: 'supabase/supavisor:1.1.56'
healthcheck:
test:
- CMD
- curl
- '-sSfL'
- '-o'
- /dev/null
- 'http://127.0.0.1:4000/api/health'
timeout: 5s
interval: 5s
retries: 10
restart: unless-stopped
ports:
- "5432:5432"
- "6543:6543"
depends_on:
supabase-db:
condition: service_healthy
environment:
- PORT=4000
- PROXY_PORT_SESSION=5432
- PROXY_PORT_TRANSACTION=6543
- 'DATABASE_URL=ecto://supabase_admin:${SERVICE_PASSWORD_POSTGRES}@${POSTGRES_HOSTNAME:-supabase-db}:${POSTGRES_PORT:-5432}/${POSTGRES_DB:-postgres}'
- CLUSTER_POSTGRES=true
- 'SECRET_KEY_BASE=${SERVICE_PASSWORD_SUPAVISORSECRET}'
- 'VAULT_ENC_KEY=${SERVICE_PASSWORD_VAULTENC}'
- 'API_JWT_SECRET=${SERVICE_PASSWORD_JWT}'
- 'METRICS_JWT_SECRET=${SERVICE_PASSWORD_JWT}'
- REGION=local
- 'ERL_AFLAGS=-proto_dist inet_tcp'
command:
- sh
- '-c'
- "/app/bin/migrate && /app/bin/supavisor eval '{:ok, _} = Application.ensure_all_started(:supavisor)\n{:ok, version} =\n case Supavisor.Repo.query!(\"select version()\") do\n %{rows: [[ver]]} -> Supavisor.Helpers.parse_pg_version(ver)\n _ -> nil\n end\nparams = %{\n \"external_id\" => \"${SUPAVISOR_TENANT:-dev_tenant}\",\n \"db_host\" => \"${POSTGRES_HOSTNAME}\",\n \"db_port\" => ${POSTGRES_PORT},\n \"db_database\" => \"${POSTGRES_DB}\",\n \"require_user\" => false,\n \"auth_query\" => \"SELECT * FROM pgbouncer.get_auth($1)\",\n \"default_max_clients\" => 100,\n \"default_pool_size\" => 20,\n \"default_parameter_status\" => %{\"server_version\" => version},\n \"users\" => [%{\n \"db_user\" => \"pgbouncer\",\n \"db_password\" => \"${SERVICE_PASSWORD_POSTGRES}\",\n \"mode_type\" => \"transaction\",\n \"pool_size\" => 20,\n \"is_manager\" => true\n }]\n}\nif !Supavisor.Tenants.get_tenant_by_external_id(params[\"external_id\"]) do\n {:ok, _} = Supavisor.Tenants.create_tenant(params)\nend' && /app/bin/server" This is based off what it does within the cli when running it locally. Requires the change made to the db volumes: - type: bind
source: ./volumes/db/supavisor.sql
target: /docker-entrypoint-initdb.d/migrations/99-supavisor.sql
content: |
\set pguser `echo "supabase_admin"`
create schema if not exists _supavisor;
alter schema _supavisor owner to :pguser; |
@Mortalife that's great, can you modify it so that the script is in a file and use a volume bind, just so its much easier to read and edit i guess you could also add a psql line to create if not exists the schema as well, since it wont run if the database already exists. |
supavisorhas anyone seen this in the supavisor container? i get hundreds of these ClientHandler: User requested SSL connection but no downstream cert/key found minioand for this container i get hundreds of these every day: minio/minio#19622 for minio, but idk how coolify comes into play here. i have over 200gb free space |
|
GitGuardian id | GitGuardian status | Secret | Commit | Filename | |
---|---|---|---|---|---|
- | - | GitHub App Keys | ccbbfd8 | database/seeders/GithubAppSeeder.php | View secret |
- | - | Generic Password | e1bcae7 | templates/compose/resend.yaml | View secret |
🛠 Guidelines to remediate hardcoded secrets
- Understand the implications of revoking this secret by investigating where it is used in your code.
- Replace and store your secrets safely. Learn here the best practices.
- Revoke and rotate these secrets.
- If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.
To avoid such incidents in the future consider
- following these best practices for managing and storing secrets including API keys and other credentials
- install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.
🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.
@Geczy Thanks for all the efforts towards this! Do we have a list of all things blocking this from getting merged? |
this pr can be closed |
Thank you for your pull request. This has been addressed differently in our codebase. |
@peaklabs-dev can you link references to how this has been addressed? |
more necessary setup instructions here https://discord.com/channels/459365938081431553/1217542291225448611/1256342603318956042