Update supabase add supavisor #3810
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
peaklabs-dev
added
the
⚙️ Service
|
Thanks for the PR 💜. And don't worry about the breaking change, service templates are not synced with existing deployments, so only new templates will be affected by your change. |
github-actions
bot
removed
the
⚙️ Service
|
amazing! ty |
|
@peaklabs-dev This is great work, thank you! I have a self-hosted supabase already running in prod, could you please elaborate on these instructions you mentioned to incorporate the new docker-compose file in the existing stack: Can you explicitly point out which 2 new and 1 altered sql file need to be run? In my case, if I update the my current docker compose file with this new one, and then run the sql commands, and then restart th containers, should it just work? |
|
If you use the new updated template and start it, everything should just work without manual intervention (I am not too familiar with Supabase itself though). |
|
@rajaiswal the updated sql files refer to creating the database _supabase and the 2 schemas inside (supavisor and analytics) if you do not want this you can contine to just use your existing compose and just paste in the supavisor section, but you will need to alter it to point to your current postgres database not the _supabase database to store the tenants. Please note you would still need to create a schema for supavisor in order for it to work |
|
@djsisson Thank you for breaking it down. My question was meant for you but idk how I ended up tagging peaklabs-dev. But thanks so much for all your efforts here! |
|
@djsisson As per your recommendation, I replaced '_supabase' with 'postgres' and created a schema for supavisor and copied the supavisor section, and all services are now up and running green. I can even access the supabase dashboard. But I am unable to connect to the db directly or supavisor via the postgres protocol. I tried: or am I missing something in my caddy config? |
|
@rajaiswal assuming you are in same docker network then it would be something like this: postgresql://postgres.dev_tenant:postgrespassword@supabase-supavisor:6543/postgres?pgbouncer=true if you are currently connecting via public port mapping, then you would need to add one to compose for supavisor too the .dev_tenant refers to the tenant name (its default is dev_tenant in compose) you can also check in your supabase dashboard, switch to _supavisor schema and view tables |
|
@djsisson Perfect, that worked. Thanks so much! |
|
Am I doing Should I instead be passing it through the username as well? I can try The error I'm noticing is in Sentry, |
|
Also, I don't see the supavisor schema. Could this be why it's acting as if it's not working?
|
|
@Geczy your connection string is also valid method, if you are using the new template, the supavisor schema is in the other database with the logs. but the error you are getting would be to do with no supavisor if it wasnt working, (you can also check supavisor logs) is this prod mode or in dev? |
|
The error is in prod, and my supabase instance is "prod" too, not sure if that's a selection I needed to make? this does show up in the logs page
but returns no results
|
|
This is a completely new coolify supabase setup, using the template from a few days ago on next branch https://github.com/coollabsio/coolify/blob/next/templates/compose/supabase.yaml I copy pasted this template into a coolify docker compose project since I wanted to modify it to add some exposed ports, and pg_audit logging in the db startup config |
|
in the logs in coolify what do u see in supavisor |
|
here's the diff of my template UPDATED LINK for better diff: Geczy@7e143d5 2024-10-16T13:06:55.298004126Z 13:06:55.295 request_id=F_7wwDZ6hoCVN_MABjvh region=local [info] GET /api/health
2024-10-16T13:06:55.298064886Z 13:06:55.295 request_id=F_7wwDZ6hoCVN_MABjvh region=local [info] Sent 204 in 166µs
2024-10-16T13:07:00.382003923Z 13:07:00.381 request_id=F_7wwWWmcrlvEXYABjwB region=local [info] GET /api/health
2024-10-16T13:07:00.382386205Z 13:07:00.382 request_id=F_7wwWWmcrlvEXYABjwB region=local [info] Sent 204 in 392µs
2024-10-16T13:07:05.450954967Z 13:07:05.449 request_id=F_7wwpO1qoY4vSAABjwh region=local [info] GET /api/health
2024-10-16T13:07:05.451013567Z 13:07:05.449 request_id=F_7wwpO1qoY4vSAABjwh region=local [info] Sent 204 in 144µs
2024-10-16T13:07:10.558281774Z 13:07:10.556 request_id=F_7ww8Qgp4UyXKIABAOC region=local [info] GET /api/health
2024-10-16T13:07:10.558891617Z 13:07:10.557 request_id=F_7ww8Qgp4UyXKIABAOC region=local [info] Sent 204 in 873µs
2024-10-16T13:07:15.626867944Z 13:07:15.625 request_id=F_7wxPJE66qyPHwABAOi region=local [info] GET /api/health
2024-10-16T13:07:15.628472272Z 13:07:15.626 request_id=F_7wxPJE66qyPHwABAOi region=local [info] Sent 204 in 338µs
2024-10-16T13:07:20.712309606Z 13:07:20.711 request_id=F_7wxiFcvEbZ-_IABAPC region=local [info] GET /api/health
2024-10-16T13:07:20.712903009Z 13:07:20.711 request_id=F_7wxiFcvEbZ-_IABAPC region=local [info] Sent 204 in 556µs
2024-10-16T13:07:25.793724811Z 13:07:25.793 request_id=F_7wx1BO_vHHFnwAAthj region=local [info] GET /api/health
2024-10-16T13:07:25.794546855Z 13:07:25.793 request_id=F_7wx1BO_vHHFnwAAthj region=local [info] Sent 204 in 272µs
2024-10-16T13:07:30.871238159Z 13:07:30.870 request_id=F_7wyH7tHVuZ2-AABjxB region=local [info] GET /api/health
2024-10-16T13:07:30.871380080Z 13:07:30.870 request_id=F_7wyH7tHVuZ2-AABjxB region=local [info] Sent 204 in 340µs
2024-10-16T13:07:35.956335871Z 13:07:35.955 request_id=F_7wya4H8bzpdPQABjxh region=local [info] GET /api/health
2024-10-16T13:07:35.957941200Z 13:07:35.956 request_id=F_7wya4H8bzpdPQABjxh region=local [info] Sent 204 in 299µs
2024-10-16T13:07:41.053389090Z 13:07:41.052 request_id=F_7wyt3PSzRaKCMABjyB region=local [info] GET /api/health
2024-10-16T13:07:41.054331255Z 13:07:41.053 request_id=F_7wyt3PSzRaKCMABjyB region=local [info] Sent 204 in 1ms
2024-10-16T13:07:46.153552768Z 13:07:46.150 request_id=F_7wzA2zubp6BvwABjyh region=local [info] GET /api/health
2024-10-16T13:07:46.153611769Z 13:07:46.153 request_id=F_7wzA2zubp6BvwABjyh region=local [info] Sent 204 in 2ms
2024-10-16T13:07:51.255725061Z 13:07:51.254 request_id=F_7wzT3h4DzR01MABjzB region=local [info] GET /api/health
2024-10-16T13:07:51.255777101Z 13:07:51.254 request_id=F_7wzT3h4DzR01MABjzB region=local [info] Sent 204 in 144µs
2024-10-16T13:07:51.512465316Z 13:07:51.509 project=dev_tenant user=postgres region=local mode=transaction type=single [warning] ClientHandler: socket closed with reason {:shutdown, :socket_closed}
2024-10-16T13:07:51.512526477Z 13:07:51.509 project=dev_tenant user=postgres region=local mode=transaction type=single [warning] ClientHandler: socket closed with reason {:shutdown, :socket_closed}
2024-10-16T13:07:51.617627383Z 13:07:51.612 project=dev_tenant user=postgres region=local mode=transaction type=single [warning] ClientHandler: socket closed with reason {:shutdown, :socket_closed}
2024-10-16T13:07:56.342900320Z 13:07:56.342 request_id=F_7wzm0jMEpAYvwABjzh region=local [info] GET /api/health
2024-10-16T13:07:56.342956960Z 13:07:56.342 request_id=F_7wzm0jMEpAYvwABjzh region=local [info] Sent 204 in 165µs
2024-10-16T13:08:01.444650418Z 13:08:01.442 request_id=F_7wz50l-ObGhMwABj0B region=local [info] GET /api/health
2024-10-16T13:08:01.444712099Z 13:08:01.442 request_id=F_7wz50l-ObGhMwABj0B region=local [info] Sent 204 in 157µs
2024-10-16T13:08:06.535372583Z 13:08:06.530 request_id=F_7w0MxvqQ7TtH8AAtiD region=local [info] GET /api/health
2024-10-16T13:08:06.535422263Z 13:08:06.531 request_id=F_7w0MxvqQ7TtH8AAtiD region=local [info] Sent 204 in 553µs
2024-10-16T13:08:11.642122275Z 13:08:11.641 request_id=F_7w0f0R2RiZEEAAAtij region=local [info] GET /api/health
2024-10-16T13:08:11.642283996Z 13:08:11.641 request_id=F_7w0f0R2RiZEEAAAtij region=local [info] Sent 204 in 139µs
2024-10-16T13:08:16.726108093Z 13:08:16.725 request_id=F_7w0ywW_MG0jmIABj0h region=local [info] GET /api/health
2024-10-16T13:08:16.726169773Z 13:08:16.725 request_id=F_7w0ywW_MG0jmIABj0h region=local [info] Sent 204 in 136µs
2024-10-16T13:08:21.837448496Z 13:08:21.836 request_id=F_7w1Fy_V5QE6rIABj1B region=local [info] GET /api/health
2024-10-16T13:08:21.837507336Z 13:08:21.836 request_id=F_7w1Fy_V5QE6rIABj1B region=local [info] Sent 204 in 141µs
2024-10-16T13:08:26.910824986Z 13:08:26.907 request_id=F_7w1Yr33q41h_AABj1h region=local [info] GET /api/health
2024-10-16T13:08:26.910886466Z 13:08:26.907 request_id=F_7w1Yr33q41h_AABj1h region=local [info] Sent 204 in 181µs
2024-10-16T13:08:31.994902134Z 13:08:31.994 request_id=F_7w1rohk9ZEgqgABj2B region=local [info] GET /api/health
2024-10-16T13:08:31.996199101Z 13:08:31.994 request_id=F_7w1rohk9ZEgqgABj2B region=local [info] Sent 204 in 1ms
2024-10-16T13:08:37.079817411Z 13:08:37.079 request_id=F_7w1-lFa5jyUx8ABj2h region=local [info] GET /api/health
2024-10-16T13:08:37.079858331Z 13:08:37.079 request_id=F_7w1-lFa5jyUx8ABj2h region=local [info] Sent 204 in 186µs
2024-10-16T13:08:42.134604295Z 13:08:42.134 request_id=F_7w2RaQcDtTthYABj3B region=local [info] GET /api/health
2024-10-16T13:08:42.134823296Z 13:08:42.134 request_id=F_7w2RaQcDtTthYABj3B region=local [info] Sent 204 in 142µs
2024-10-16T13:08:47.215362757Z 13:08:47.214 request_id=F_7w2kVlrFEJRvwABj3h region=local [info] GET /api/health
2024-10-16T13:08:47.215422757Z 13:08:47.214 request_id=F_7w2kVlrFEJRvwABj3h region=local [info] Sent 204 in 142µs
2024-10-16T13:08:52.291694919Z 13:08:52.290 request_id=F_7w23PyQbpRFlYABj4B region=local [info] GET /api/health
2024-10-16T13:08:52.291743600Z 13:08:52.290 request_id=F_7w23PyQbpRFlYABj4B region=local [info] Sent 204 in 159µs
2024-10-16T13:08:57.369174531Z 13:08:57.368 request_id=F_7w3KKX7xZPLeYABj4h region=local [info] GET /api/health
2024-10-16T13:08:57.369234812Z 13:08:57.368 request_id=F_7w3KKX7xZPLeYABj4h region=local [info] Sent 204 in 163µs
2024-10-16T13:09:02.490257174Z 13:09:02.488 request_id=F_7w3dPJXZ0zCZwAAtjD region=local [info] GET /api/health
2024-10-16T13:09:02.490309214Z 13:09:02.488 request_id=F_7w3dPJXZ0zCZwAAtjD region=local [info] Sent 204 in 141µs
2024-10-16T13:09:07.602589574Z 13:09:07.601 request_id=F_7w3wSMC4ENgQIABj5B region=local [info] GET /api/health
2024-10-16T13:09:07.602629974Z 13:09:07.601 request_id=F_7w3wSMC4ENgQIABj5B region=local [info] Sent 204 in 142µs
2024-10-16T13:09:12.716163905Z 13:09:12.715 request_id=F_7w4DVeQ2WWNaYABj5h region=local [info] GET /api/health
2024-10-16T13:09:12.716216745Z 13:09:12.715 request_id=F_7w4DVeQ2WWNaYABj5h region=local [info] Sent 204 in 161µs
2024-10-16T13:09:17.800684527Z 13:09:17.798 request_id=F_7w4WRXrzSwTSMABj6B region=local [info] GET /api/health
2024-10-16T13:09:17.800727607Z 13:09:17.798 request_id=F_7w4WRXrzSwTSMABj6B region=local [info] Sent 204 in 136µs
2024-10-16T13:09:22.878478477Z 13:09:22.877 request_id=F_7w4pMSRSjx5wEABj6h region=local [info] GET /api/health
2024-10-16T13:09:22.878510957Z 13:09:22.877 request_id=F_7w4pMSRSjx5wEABj6h region=local [info] Sent 204 in 146µs
2024-10-16T13:09:27.945266413Z 13:09:27.944 request_id=F_7w48EYz6J7B34ABj7B region=local [info] GET /api/health
2024-10-16T13:09:27.945319454Z 13:09:27.944 request_id=F_7w48EYz6J7B34ABj7B region=local [info] Sent 204 in 148µs
2024-10-16T13:09:33.026841790Z 13:09:33.026 request_id=F_7w5O_6uzMTv8IABj7h region=local [info] GET /api/health
2024-10-16T13:09:33.028896521Z 13:09:33.026 request_id=F_7w5O_6uzMTv8IABj7h region=local [info] Sent 204 in 218µs
2024-10-16T13:09:38.092764368Z 13:09:38.092 request_id=F_7w5h3tpFx15AMABj8B region=local [info] GET /api/health
2024-10-16T13:09:38.095296461Z 13:09:38.092 request_id=F_7w5h3tpFx15AMABj8B region=local [info] Sent 204 in 188µs
2024-10-16T13:09:43.170986213Z 13:09:43.170 request_id=F_7w50ycLsHBP74ABj8h region=local [info] GET /api/health
2024-10-16T13:09:43.173506626Z 13:09:43.170 request_id=F_7w50ycLsHBP74ABj8h region=local [info] Sent 204 in 148µs
2024-10-16T13:09:48.263360536Z 13:09:48.261 request_id=F_7w6HwOP6Sp_ywABj9B region=local [info] GET /api/health
2024-10-16T13:09:48.267148955Z 13:09:48.262 request_id=F_7w6HwOP6Sp_ywABj9B region=local [info] Sent 204 in 1ms
2024-10-16T13:09:53.370270817Z 13:09:53.369 request_id=F_7w6ax8KYmf4mAABj9h region=local [info] GET /api/health
2024-10-16T13:09:53.374560599Z 13:09:53.369 request_id=F_7w6ax8KYmf4mAABj9h region=local [info] Sent 204 in 681µs
2024-10-16T13:09:58.454685544Z 13:09:58.453 request_id=F_7w6tuFfc7LXsoABj-B region=local [info] GET /api/health
2024-10-16T13:09:58.459354128Z 13:09:58.454 request_id=F_7w6tuFfc7LXsoABj-B region=local [info] Sent 204 in 1ms
2024-10-16T13:10:03.564934328Z 13:10:03.563 request_id=F_7w7AwkbImmcCkABj-h region=local [info] GET /api/health
2024-10-16T13:10:03.565959054Z 13:10:03.564 request_id=F_7w7AwkbImmcCkABj-h region=local [info] Sent 204 in 758µs
2024-10-16T13:10:08.654893490Z 13:10:08.653 request_id=F_7w7TuBCAHsyBkABj_B region=local [info] GET /api/health
2024-10-16T13:10:08.654942491Z 13:10:08.653 request_id=F_7w7TuBCAHsyBkABj_B region=local [info] Sent 204 in 514µs
2024-10-16T13:10:13.739499067Z 13:10:13.737 request_id=F_7w7mp_vlQIh-sABj_h region=local [info] GET /api/health
2024-10-16T13:10:13.741150396Z 13:10:13.737 request_id=F_7w7mp_vlQIh-sABj_h region=local [info] Sent 204 in 1ms
2024-10-16T13:10:18.813729193Z 13:10:18.813 request_id=F_7w75kPaVMOyWkABkAB region=local [info] GET /api/health
2024-10-16T13:10:18.813770313Z 13:10:18.813 request_id=F_7w75kPaVMOyWkABkAB region=local [info] Sent 204 in 147µs
2024-10-16T13:10:23.892502905Z 13:10:23.891 request_id=F_7w8MfMrxWzkTYABkAh region=local [info] GET /api/health
2024-10-16T13:10:23.894838637Z 13:10:23.892 request_id=F_7w8MfMrxWzkTYABkAh region=local [info] Sent 204 in 349µs
2024-10-16T13:10:28.981319113Z 13:10:28.979 request_id=F_7w8fb7fqG51k4AAtjj region=local [info] GET /api/health
2024-10-16T13:10:28.986599940Z 13:10:28.982 request_id=F_7w8fb7fqG51k4AAtjj region=local [info] Sent 204 in 4ms
2024-10-16T13:10:34.075488471Z 13:10:34.072 request_id=F_7w8yaguYDHvvwAAtkD region=local [info] GET /api/health
2024-10-16T13:10:34.075553232Z 13:10:34.073 request_id=F_7w8yaguYDHvvwAAtkD region=local [info] Sent 204 in 591µs
2024-10-16T13:10:39.169114230Z 13:10:39.159 request_id=F_7w9FXMfEld5c0ABkBB region=local [info] GET /api/health
2024-10-16T13:10:39.169174510Z 13:10:39.160 request_id=F_7w9FXMfEld5c0ABkBB region=local [info] Sent 204 in 1ms
2024-10-16T13:10:44.228138290Z 13:10:44.226 request_id=F_7w9YPRZjcACIwABkBh region=local [info] GET /api/health
2024-10-16T13:10:44.228218411Z 13:10:44.226 request_id=F_7w9YPRZjcACIwABkBh region=local [info] Sent 204 in 662µs
2024-10-16T13:10:49.300851705Z 13:10:49.298 request_id=F_7w9rIhq4JfficABkCB region=local [info] GET /api/health
2024-10-16T13:10:49.301710510Z 13:10:49.298 request_id=F_7w9rIhq4JfficABkCB region=local [info] Sent 204 in 725µs
2024-10-16T13:10:54.395391077Z 13:10:54.393 request_id=F_7w9-HXuGKY5YkABkCh region=local [info] GET /api/health
2024-10-16T13:10:54.395444957Z 13:10:54.394 request_id=F_7w9-HXuGKY5YkABkCh region=local [info] Sent 204 in 592µs
2024-10-16T13:10:57.260107522Z 13:10:57.259 project=dev_tenant user=postgres region=local mode=transaction type=single [warning] ClientHandler: socket closed with reason {:shutdown, :terminate_received} |
|
Supabase rest logs show this, but only 2 instances of this error 2024-10-16T12:00:28.925153638Z 16/Oct/2024:12:00:28 +0000: {"code":"PGRST003","details":null,"hint":null,"message":"Timed out acquiring connection from connection pool."}See a bunch of these in supabase vector logs 2024-10-16T11:22:14.316452011Z 2024-10-16T11:22:14.316351Z ERROR transform{component_kind="transform" component_id=db_logs component_type=remap component_name=db_logs}: vector::internal_events::remap: Internal log [Mapping failed with event.] has been rate limited 5 times.
2024-10-16T11:22:14.316802733Z 2024-10-16T11:22:14.316427Z ERROR transform{component_kind="transform" component_id=db_logs component_type=remap component_name=db_logs}: vector::internal_events::remap: Mapping failed with event. error="function call error for \"upcase\" at (475:515): expected string, got null" error_type="conversion_failed" stage="processing" internal_log_rate_limit=true
2024-10-16T11:22:14.321603438Z 2024-10-16T11:22:14.320955Z ERROR transform{component_kind="transform" component_id=db_logs component_type=remap component_name=db_logs}: vector::internal_events::remap: Internal log [Mapping failed with event.] is being rate limited.
2024-10-16T11:28:42.707150074Z 2024-10-16T11:28:42.706789Z ERROR transform{component_kind="transform" component_id=db_logs component_type=remap component_name=db_logs}: vector::internal_events::remap: Internal log [Mapping failed with event.] has been rate limited 1 times.
2024-10-16T11:28:42.707629876Z 2024-10-16T11:28:42.706980Z ERROR transform{component_kind="transform" component_id=db_logs component_type=remap component_name=db_logs}: vector::internal_events::remap: Mapping failed with event. error="function call error for \"upcase\" at (475:515): expected string, got null" error_type="conversion_failed" stage="processing" internal_log_rate_limit=true
2024-10-16T11:28:42.707651716Z 2024-10-16T11:28:42.707503Z ERROR transform{component_kind="transform" component_id=db_logs component_type=remap component_name=db_logs}: vector::internal_events::remap: Internal log [Mapping failed with event.] is being rate limited.
2024-10-16T11:43:29.967137110Z 2024-10-16T11:43:29.966391Z ERROR transform{component_kind="transform" component_id=db_logs component_type=remap component_name=db_logs}: vector::internal_events::remap: Internal log [Mapping failed with event.] has been rate limited 1 times.
2024-10-16T11:43:29.967438992Z 2024-10-16T11:43:29.966412Z ERROR transform{component_kind="transform" component_id=db_logs component_type=remap component_name=db_logs}: vector::internal_events::remap: Mapping failed with event. error="function call error for \"upcase\" at (475:515): expected string, got null" error_type="conversion_failed" stage="processing" internal_log_rate_limit=true
2024-10-16T11:43:29.967451992Z 2024-10-16T11:43:29.966843Z ERROR transform{component_kind="transform" component_id=db_logs component_type=remap component_name=db_logs}: vector::internal_events::remap: Internal log [Mapping failed with event.] is being rate limited.
2024-10-16T11:46:04.926451510Z 2024-10-16T11:46:04.926135Z ERROR transform{component_kind="transform" component_id=db_logs component_type=remap component_name=db_logs}: vector::internal_events::remap: Internal log [Mapping failed with event.] has been rate limited 3 times.
2024-10-16T11:46:04.927564556Z 2024-10-16T11:46:04.926402Z ERROR transform{component_kind="transform" component_id=db_logs component_type=remap component_name=db_logs}: vector::internal_events::remap: Mapping failed with event. error="function call error for \"upcase\" at (475:515): expected string, got null" error_type="conversion_failed" stage="processing" internal_log_rate_limit=true
2024-10-16T11:46:04.927707196Z 2024-10-16T11:46:04.927668Z ERROR transform{component_kind="transform" component_id=db_logs component_type=remap component_name=db_logs}: vector::internal_events::remap: Internal log [Mapping failed with event.] is being rate limited.
2024-10-16T12:00:02.006962008Z 2024-10-16T12:00:02.000787Z INFO source{component_kind="source" component_id=docker_host component_type=docker_logs component_name=docker_host}: vector::internal_events::docker_logs: Stopped watching for container logs. container_id=b411e8f5915e15bbc7c6d1abb5c99ac1f30270a64713dee411cd78da8adf3fe0
2024-10-16T12:00:02.007032688Z 2024-10-16T12:00:02.004285Z INFO source{component_kind="source" component_id=docker_host component_type=docker_logs component_name=docker_host}: vector::internal_events::docker_logs: Started watching for container logs. container_id=b411e8f5915e15bbc7c6d1abb5c99ac1f30270a64713dee411cd78da8adf3fe0
2024-10-16T12:00:02.398299137Z 2024-10-16T12:00:02.371140Z ERROR source{component_kind="source" component_id=docker_host component_type=docker_logs component_name=docker_host}: vector::internal_events::docker_logs: Internal log [Received out of order log message.] has been rate limited 52 times.
2024-10-16T12:00:02.399444303Z 2024-10-16T12:00:02.371165Z ERROR source{component_kind="source" component_id=docker_host component_type=docker_logs component_name=docker_host}: vector::internal_events::docker_logs: Received out of order log message. error_type="condition_failed" stage="receiving" container_id="b411e8f5915e15bbc7c6d1abb5c99ac1f30270a64713dee411cd78da8adf3fe0" timestamp="2024-10-16T12:00:01.812259168Z" internal_log_rate_limit=true
2024-10-16T12:00:02.400202747Z 2024-10-16T12:00:02.371815Z ERROR source{component_kind="source" component_id=docker_host component_type=docker_logs component_name=docker_host}: vector::internal_events::docker_logs: Internal log [Received out of order log message.] is being rate limited.
2024-10-16T12:00:02.401514154Z 2024-10-16T12:00:02.371967Z INFO source{component_kind="source" component_id=docker_host component_type=docker_logs component_name=docker_host}: vector::internal_events::docker_logs: Stopped watching for container logs. container_id=b411e8f5915e15bbc7c6d1abb5c99ac1f30270a64713dee411cd78da8adf3fe0
2024-10-16T12:00:03.087983460Z 2024-10-16T12:00:03.086891Z INFO source{component_kind="source" component_id=docker_host component_type=docker_logs component_name=docker_host}: vector::internal_events::docker_logs: Started watching for container logs. container_id=b411e8f5915e15bbc7c6d1abb5c99ac1f30270a64713dee411cd78da8adf3fe0
2024-10-16T12:00:12.148823598Z 2024-10-16T12:00:12.148272Z INFO source{component_kind="source" component_id=docker_host component_type=docker_logs component_name=docker_host}: vector::internal_events::docker_logs: Stopped watching for container logs. container_id=770635c191551373913981d9098f0c05eb8349ed2972e41406e7d3ff8eaa6861
2024-10-16T12:00:12.149066079Z 2024-10-16T12:00:12.148380Z INFO source{component_kind="source" component_id=docker_host component_type=docker_logs component_name=docker_host}: vector::internal_events::docker_logs: Started watching for container logs. container_id=770635c191551373913981d9098f0c05eb8349ed2972e41406e7d3ff8eaa6861
2024-10-16T12:00:12.890198276Z 2024-10-16T12:00:12.890057Z ERROR source{component_kind="source" component_id=docker_host component_type=docker_logs component_name=docker_host}: vector::internal_events::docker_logs: Internal log [Received out of order log message.] has been rate limited 6 times.
2024-10-16T12:00:12.890384797Z 2024-10-16T12:00:12.890080Z ERROR source{component_kind="source" component_id=docker_host component_type=docker_logs component_name=docker_host}: vector::internal_events::docker_logs: Received out of order log message. error_type="condition_failed" stage="receiving" container_id="770635c191551373913981d9098f0c05eb8349ed2972e41406e7d3ff8eaa6861" timestamp="2024-10-16T12:00:10.269489804Z" internal_log_rate_limit=true
2024-10-16T12:00:12.890392557Z 2024-10-16T12:00:12.890112Z ERROR source{component_kind="source" component_id=docker_host component_type=docker_logs component_name=docker_host}: vector::internal_events::docker_logs: Internal log [Received out of order log message.] is being rate limited.
2024-10-16T12:00:12.890913040Z 2024-10-16T12:00:12.890729Z INFO source{component_kind="source" component_id=docker_host component_type=docker_logs component_name=docker_host}: vector::internal_events::docker_logs: Stopped watching for container logs. container_id=770635c191551373913981d9098f0c05eb8349ed2972e41406e7d3ff8eaa6861
2024-10-16T12:00:12.925321103Z 2024-10-16T12:00:12.924823Z INFO source{component_kind="source" component_id=docker_host component_type=docker_logs component_name=docker_host}: vector::internal_events::docker_logs: Started watching for container logs. container_id=770635c191551373913981d9098f0c05eb8349ed2972e41406e7d3ff8eaa6861
2024-10-16T12:00:37.854231740Z 2024-10-16T12:00:37.853923Z ERROR transform{component_kind="transform" component_id=db_logs component_type=remap component_name=db_logs}: vector::internal_events::remap: Internal log [Mapping failed with event.] has been rate limited 3 times.
2024-10-16T12:00:37.854263381Z 2024-10-16T12:00:37.853943Z ERROR transform{component_kind="transform" component_id=db_logs component_type=remap component_name=db_logs}: vector::internal_events::remap: Mapping failed with event. error="function call error for \"upcase\" at (475:515): expected string, got null" error_type="conversion_failed" stage="processing" internal_log_rate_limit=true
2024-10-16T12:00:37.854269101Z 2024-10-16T12:00:37.854068Z ERROR transform{component_kind="transform" component_id=db_logs component_type=remap component_name=db_logs}: vector::internal_events::remap: Internal log [Mapping failed with event.] is being rate limited. |
|
none of that is relevant here, that all looks normal can you run this in sql editor select count(*) from pg_stat_activity; |
84 |
|
are all connections through 6543 or do you have any direct connections? also can you run this SELECT |
|
yeah i have direct connections in another service. i use prisma 6543 on my nextjs app count | usename | application_name
-------+----------------+-------------------------------------
25 | supabase_admin | supavisor_meta
2 | supabase_admin | cluster_node_supavisor@c944d3d8c699
11 | postgres | Supavisor
(3 rows) |
|
in your other app add a connection timeout so the connections do not stay open, i think the error is from the pooler being set to 20, but its hitting a connection limit and cant open anymore, presumable because of your other app, thats all i can think of |
|
or increase supabase max limit from 100 |
|
can i add a connection timeout on the rest url?
|
|
I meant to say I don't have any direct connections lol, I only use REST or 6543 Supavisor |
|
look through here, SELECT |
|
what to look for? [
{
"pid": 35,
"datname": null,
"usename": null,
"application_name": "",
"client_hostname": null,
"client_port": null,
"backend_start": "2024-10-16 13:39:55.205298+00",
"query_start": null,
"query": "",
"state": null
},
{
"pid": 37,
"datname": "postgres",
"usename": "postgres",
"application_name": "",
"client_hostname": null,
"client_port": null,
"backend_start": "2024-10-16 13:39:55.20819+00",
"query_start": null,
"query": "",
"state": null
},
{
"pid": 38,
"datname": "postgres",
"usename": "postgres",
"application_name": "pg_cron scheduler",
"client_hostname": null,
"client_port": null,
"backend_start": "2024-10-16 13:39:55.209659+00",
"query_start": null,
"query": "",
"state": null
},
{
"pid": 39,
"datname": null,
"usename": "postgres",
"application_name": "",
"client_hostname": null,
"client_port": null,
"backend_start": "2024-10-16 13:39:55.220827+00",
"query_start": null,
"query": "",
"state": null
},
{
"pid": 36,
"datname": null,
"usename": "postgres",
"application_name": "TimescaleDB Background Worker Launcher",
"client_hostname": null,
"client_port": null,
"backend_start": "2024-10-16 13:39:55.223952+00",
"query_start": null,
"query": "",
"state": null
},
{
"pid": 59,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "",
"client_hostname": null,
"client_port": 49844,
"backend_start": "2024-10-16 13:40:08.503073+00",
"query_start": "2024-10-16 13:55:59.094981+00",
"query": "SELECT s0.\"id\", s0.\"all_logs_logged\", s0.\"node\", s0.\"inserted_at\", s0.\"updated_at\" FROM \"system_metrics\" AS s0 WHERE (s0.\"node\" = $1)",
"state": "idle"
},
{
"pid": 60,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "",
"client_hostname": null,
"client_port": 49850,
"backend_start": "2024-10-16 13:40:08.505638+00",
"query_start": "2024-10-16 13:55:59.614193+00",
"query": "SELECT p0.\"id\", p0.\"name\", p0.\"stripe_id\", p0.\"price\", p0.\"period\", p0.\"limit_sources\", p0.\"limit_rate_limit\", p0.\"limit_source_rate_limit\", p0.\"limit_alert_freq\", p0.\"limit_saved_search_limit\", p0.\"limit_team_users_limit\", p0.\"limit_source_fields_limit\", p0.\"limit_source_ttl\", p0.\"type\", p0.\"inserted_at\", p0.\"updated_at\" FROM \"plans\" AS p0",
"state": "idle"
},
{
"pid": 69,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "",
"client_hostname": null,
"client_port": 49918,
"backend_start": "2024-10-16 13:40:08.518344+00",
"query_start": "2024-10-16 13:40:08.597561+00",
"query": "START_REPLICATION SLOT cainophile_5vkwzrsp LOGICAL 0/0 (proto_version '1', publication_names '\"logflare_pub\"')",
"state": "active"
},
{
"pid": 61,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "",
"client_hostname": null,
"client_port": 49852,
"backend_start": "2024-10-16 13:40:08.506315+00",
"query_start": "2024-10-16 13:55:58.943889+00",
"query": "SELECT s0.\"id\", s0.\"all_logs_logged\", s0.\"node\", s0.\"inserted_at\", s0.\"updated_at\" FROM \"system_metrics\" AS s0",
"state": "idle"
},
{
"pid": 62,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "",
"client_hostname": null,
"client_port": 49868,
"backend_start": "2024-10-16 13:40:08.508225+00",
"query_start": "2024-10-16 13:56:00.948303+00",
"query": "SELECT s0.\"id\", s0.\"all_logs_logged\", s0.\"node\", s0.\"inserted_at\", s0.\"updated_at\" FROM \"system_metrics\" AS s0",
"state": "idle"
},
{
"pid": 64,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "",
"client_hostname": null,
"client_port": 49878,
"backend_start": "2024-10-16 13:40:08.51336+00",
"query_start": "2024-10-16 13:55:59.945884+00",
"query": "SELECT s0.\"id\", s0.\"all_logs_logged\", s0.\"node\", s0.\"inserted_at\", s0.\"updated_at\" FROM \"system_metrics\" AS s0",
"state": "idle"
},
{
"pid": 63,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "",
"client_hostname": null,
"client_port": 49874,
"backend_start": "2024-10-16 13:40:08.51829+00",
"query_start": "2024-10-16 13:55:59.616585+00",
"query": "SELECT s0.\"id\", s0.\"name\", s0.\"token\", s0.\"public_token\", s0.\"favorite\", s0.\"bigquery_table_ttl\", s0.\"api_quota\", s0.\"webhook_notification_url\", s0.\"slack_hook_url\", s0.\"bq_table_partition_type\", s0.\"custom_event_message_keys\", s0.\"log_events_updated_at\", s0.\"notifications_every\", s0.\"lock_schema\", s0.\"validate_schema\", s0.\"drop_lql_filters\", s0.\"drop_lql_string\", s0.\"v2_pipeline\", s0.\"suggested_keys\", s0.\"user_id\", s0.\"notifications\", s0.\"inserted_at\", s0.\"updated_at\" FROM \"sources\" AS s0 WHERE (s0.\"user_id\" = $1)",
"state": "idle"
},
{
"pid": 65,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "",
"client_hostname": null,
"client_port": 49888,
"backend_start": "2024-10-16 13:40:08.514104+00",
"query_start": "2024-10-16 13:56:02.952061+00",
"query": "SELECT s0.\"id\", s0.\"all_logs_logged\", s0.\"node\", s0.\"inserted_at\", s0.\"updated_at\" FROM \"system_metrics\" AS s0",
"state": "idle"
},
{
"pid": 66,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "",
"client_hostname": null,
"client_port": 49894,
"backend_start": "2024-10-16 13:40:08.515786+00",
"query_start": "2024-10-16 13:55:59.617761+00",
"query": "SELECT e0.\"id\", e0.\"token\", e0.\"name\", e0.\"query\", e0.\"language\", e0.\"source_mapping\", e0.\"sandboxable\", e0.\"cache_duration_seconds\", e0.\"proactive_requerying_seconds\", e0.\"max_limit\", e0.\"enable_auth\", e0.\"user_id\", e0.\"sandbox_query_id\", e0.\"inserted_at\", e0.\"updated_at\" FROM \"endpoint_queries\" AS e0 WHERE (e0.\"user_id\" = $1)",
"state": "idle"
},
{
"pid": 67,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "",
"client_hostname": null,
"client_port": 49902,
"backend_start": "2024-10-16 13:40:08.516563+00",
"query_start": "2024-10-16 13:55:57.941843+00",
"query": "SELECT s0.\"id\", s0.\"all_logs_logged\", s0.\"node\", s0.\"inserted_at\", s0.\"updated_at\" FROM \"system_metrics\" AS s0",
"state": "idle"
},
{
"pid": 68,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "",
"client_hostname": null,
"client_port": 49906,
"backend_start": "2024-10-16 13:40:08.517372+00",
"query_start": "2024-10-16 13:56:01.950072+00",
"query": "SELECT s0.\"id\", s0.\"all_logs_logged\", s0.\"node\", s0.\"inserted_at\", s0.\"updated_at\" FROM \"system_metrics\" AS s0",
"state": "idle"
},
{
"pid": 77,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "",
"client_hostname": null,
"client_port": 49920,
"backend_start": "2024-10-16 13:40:08.946557+00",
"query_start": "2024-10-16 13:56:03.386576+00",
"query": "INSERT INTO \"_analytics\".\"log_events_e7fd5c0b_c1ea_4826_b0bd_209b23e15cc9\" (\"body\",\"event_message\",\"id\",\"timestamp\") VALUES ($1,$2,$3,$4)",
"state": "idle"
},
{
"pid": 78,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "",
"client_hostname": null,
"client_port": 49936,
"backend_start": "2024-10-16 13:40:08.947372+00",
"query_start": "2024-10-16 13:56:03.390227+00",
"query": "INSERT INTO \"_analytics\".\"log_events_e7fd5c0b_c1ea_4826_b0bd_209b23e15cc9\" (\"body\",\"event_message\",\"id\",\"timestamp\") VALUES ($1,$2,$3,$4)",
"state": "idle"
},
{
"pid": 79,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "",
"client_hostname": null,
"client_port": 49950,
"backend_start": "2024-10-16 13:40:08.948122+00",
"query_start": "2024-10-16 13:56:03.388472+00",
"query": "INSERT INTO \"_analytics\".\"log_events_e7fd5c0b_c1ea_4826_b0bd_209b23e15cc9\" (\"body\",\"event_message\",\"id\",\"timestamp\") VALUES ($1,$2,$3,$4)",
"state": "idle"
},
{
"pid": 80,
"datname": "postgres",
"usename": "authenticator",
"application_name": "PostgREST 12.2.0 (ec89f6b)",
"client_hostname": null,
"client_port": 34292,
"backend_start": "2024-10-16 13:40:11.326188+00",
"query_start": "2024-10-16 13:55:58.926495+00",
"query": "COMMIT",
"state": "idle"
},
{
"pid": 81,
"datname": "postgres",
"usename": "authenticator",
"application_name": "PostgREST 12.2.0 (ec89f6b)",
"client_hostname": null,
"client_port": 34304,
"backend_start": "2024-10-16 13:40:11.486148+00",
"query_start": "2024-10-16 13:40:11.52721+00",
"query": "LISTEN \"pgrst\"",
"state": "idle"
},
{
"pid": 108,
"datname": "postgres",
"usename": "supabase_storage_admin",
"application_name": "",
"client_hostname": null,
"client_port": 47922,
"backend_start": "2024-10-16 13:40:21.060634+00",
"query_start": "2024-10-16 13:55:51.079821+00",
"query": "SELECT pg_backend_pid()",
"state": "idle"
},
{
"pid": 151,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "supavisor_meta",
"client_hostname": null,
"client_port": 36422,
"backend_start": "2024-10-16 13:40:28.842521+00",
"query_start": "2024-10-16 13:40:29.200375+00",
"query": "SELECT u0.\"id\", u0.\"db_user_alias\", u0.\"db_user\", u0.\"db_pass_encrypted\", u0.\"is_manager\", u0.\"mode_type\", u0.\"pool_size\", u0.\"pool_checkout_timeout\", u0.\"max_clients\", u0.\"tenant_external_id\", u0.\"inserted_at\", u0.\"updated_at\", u0.\"tenant_external_id\" FROM \"_supavisor\".\"users\" AS u0 WHERE (u0.\"db_user_alias\" = $1) AND (u0.\"tenant_external_id\" = $2) ORDER BY u0.\"tenant_external_id\"",
"state": "idle"
},
{
"pid": 149,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "supavisor_meta",
"client_hostname": null,
"client_port": 36402,
"backend_start": "2024-10-16 13:40:28.840667+00",
"query_start": null,
"query": "",
"state": "idle"
},
{
"pid": 147,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "supavisor_meta",
"client_hostname": null,
"client_port": 36386,
"backend_start": "2024-10-16 13:40:28.839415+00",
"query_start": "2024-10-16 13:40:28.976891+00",
"query": "SELECT u0.\"id\", u0.\"db_user_alias\", u0.\"db_user\", u0.\"db_pass_encrypted\", u0.\"is_manager\", u0.\"mode_type\", u0.\"pool_size\", u0.\"pool_checkout_timeout\", u0.\"max_clients\", u0.\"tenant_external_id\", u0.\"inserted_at\", u0.\"updated_at\", t1.\"id\", t1.\"db_host\", t1.\"db_port\", t1.\"db_database\", t1.\"external_id\", t1.\"default_parameter_status\", t1.\"ip_version\", t1.\"upstream_ssl\", t1.\"upstream_verify\", t1.\"upstream_tls_ca\", t1.\"enforce_ssl\", t1.\"require_user\", t1.\"auth_query\", t1.\"default_pool_size\", t1.\"sni_hostname\", t1.\"default_max_clients\", t1.\"client_idle_timeout\", t1.\"client_heartbeat_interval\", t1.\"allow_list\", t1.\"inserted_at\", t1.\"updated_at\" FROM \"_supavisor\".\"users\" AS u0 INNER JOIN \"_supavisor\".\"tenants\" AS t1 ON u0.\"tenant_external_id\" = t1.\"external_id\" WHERE (((u0.\"db_user_alias\" = $1) AND (t1.\"require_user\" = TRUE)) OR ((t1.\"require_user\" = FALSE) AND (u0.\"is_manager\" = TRUE))) AND (t1.\"external_id\" = $2)",
"state": "idle"
},
{
"pid": 148,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "supavisor_meta",
"client_hostname": null,
"client_port": 36388,
"backend_start": "2024-10-16 13:40:28.839963+00",
"query_start": null,
"query": "",
"state": "idle"
},
{
"pid": 150,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "supavisor_meta",
"client_hostname": null,
"client_port": 36408,
"backend_start": "2024-10-16 13:40:28.841409+00",
"query_start": "2024-10-16 13:40:29.194796+00",
"query": "SELECT t0.\"id\", t0.\"db_host\", t0.\"db_port\", t0.\"db_database\", t0.\"external_id\", t0.\"default_parameter_status\", t0.\"ip_version\", t0.\"upstream_ssl\", t0.\"upstream_verify\", t0.\"upstream_tls_ca\", t0.\"enforce_ssl\", t0.\"require_user\", t0.\"auth_query\", t0.\"default_pool_size\", t0.\"sni_hostname\", t0.\"default_max_clients\", t0.\"client_idle_timeout\", t0.\"client_heartbeat_interval\", t0.\"allow_list\", t0.\"inserted_at\", t0.\"updated_at\" FROM \"_supavisor\".\"tenants\" AS t0 WHERE (t0.\"external_id\" = $1)",
"state": "idle"
},
{
"pid": 153,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "supavisor_meta",
"client_hostname": null,
"client_port": 36442,
"backend_start": "2024-10-16 13:40:28.850435+00",
"query_start": null,
"query": "",
"state": "idle"
},
{
"pid": 154,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "supavisor_meta",
"client_hostname": null,
"client_port": 36444,
"backend_start": "2024-10-16 13:40:28.851218+00",
"query_start": null,
"query": "",
"state": "idle"
},
{
"pid": 156,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "supavisor_meta",
"client_hostname": null,
"client_port": 36456,
"backend_start": "2024-10-16 13:40:28.852837+00",
"query_start": null,
"query": "",
"state": "idle"
},
{
"pid": 155,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "supavisor_meta",
"client_hostname": null,
"client_port": 36454,
"backend_start": "2024-10-16 13:40:28.854041+00",
"query_start": null,
"query": "",
"state": "idle"
},
{
"pid": 157,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "supavisor_meta",
"client_hostname": null,
"client_port": 36460,
"backend_start": "2024-10-16 13:40:28.868478+00",
"query_start": null,
"query": "",
"state": "idle"
},
{
"pid": 158,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "supavisor_meta",
"client_hostname": null,
"client_port": 36472,
"backend_start": "2024-10-16 13:40:28.86922+00",
"query_start": null,
"query": "",
"state": "idle"
},
{
"pid": 159,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "supavisor_meta",
"client_hostname": null,
"client_port": 36486,
"backend_start": "2024-10-16 13:40:28.880166+00",
"query_start": null,
"query": "",
"state": "idle"
},
{
"pid": 152,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "supavisor_meta",
"client_hostname": null,
"client_port": 36432,
"backend_start": "2024-10-16 13:40:28.892777+00",
"query_start": null,
"query": "",
"state": "idle"
},
{
"pid": 163,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "supavisor_meta",
"client_hostname": null,
"client_port": 36530,
"backend_start": "2024-10-16 13:40:28.905468+00",
"query_start": null,
"query": "",
"state": "idle"
},
{
"pid": 162,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "supavisor_meta",
"client_hostname": null,
"client_port": 36526,
"backend_start": "2024-10-16 13:40:28.904348+00",
"query_start": null,
"query": "",
"state": "idle"
},
{
"pid": 165,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "supavisor_meta",
"client_hostname": null,
"client_port": 36548,
"backend_start": "2024-10-16 13:40:28.906749+00",
"query_start": null,
"query": "",
"state": "idle"
},
{
"pid": 161,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "supavisor_meta",
"client_hostname": null,
"client_port": 36512,
"backend_start": "2024-10-16 13:40:28.912278+00",
"query_start": null,
"query": "",
"state": "idle"
},
{
"pid": 160,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "cluster_node_supavisor@36763b71dafa",
"client_hostname": null,
"client_port": 36496,
"backend_start": "2024-10-16 13:40:28.910563+00",
"query_start": "2024-10-16 13:55:59.568915+00",
"query": "NOTIFY supavisor_partisan_local_1_1, 'REDACTED/YQBhAGEBZAAEcG9ydGIAAE6EamREADACTED'",
"state": "idle"
},
{
"pid": 164,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "supavisor_meta",
"client_hostname": null,
"client_port": 36534,
"backend_start": "2024-10-16 13:40:28.914007+00",
"query_start": null,
"query": "",
"state": "idle"
},
{
"pid": 167,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "supavisor_meta",
"client_hostname": null,
"client_port": 36580,
"backend_start": "2024-10-16 13:40:28.917445+00",
"query_start": null,
"query": "",
"state": "idle"
},
{
"pid": 168,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "supavisor_meta",
"client_hostname": null,
"client_port": 36584,
"backend_start": "2024-10-16 13:40:28.932549+00",
"query_start": null,
"query": "",
"state": "idle"
},
{
"pid": 166,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "supavisor_meta",
"client_hostname": null,
"client_port": 36564,
"backend_start": "2024-10-16 13:40:28.937022+00",
"query_start": null,
"query": "",
"state": "idle"
},
{
"pid": 169,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "supavisor_meta",
"client_hostname": null,
"client_port": 36600,
"backend_start": "2024-10-16 13:40:28.944507+00",
"query_start": null,
"query": "",
"state": "idle"
},
{
"pid": 171,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "supavisor_meta",
"client_hostname": null,
"client_port": 36618,
"backend_start": "2024-10-16 13:40:28.94551+00",
"query_start": null,
"query": "",
"state": "idle"
},
{
"pid": 170,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "supavisor_meta",
"client_hostname": null,
"client_port": 36614,
"backend_start": "2024-10-16 13:40:28.946126+00",
"query_start": null,
"query": "",
"state": "idle"
},
{
"pid": 173,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "supavisor_meta",
"client_hostname": null,
"client_port": 36632,
"backend_start": "2024-10-16 13:40:28.949276+00",
"query_start": null,
"query": "",
"state": "idle"
},
{
"pid": 172,
"datname": "_supabase",
"usename": "supabase_admin",
"application_name": "cluster_node_supavisor@36763b71dafa",
"client_hostname": null,
"client_port": 36622,
"backend_start": "2024-10-16 13:40:28.953122+00",
"query_start": "2024-10-16 13:40:28.981178+00",
"query": "LISTEN \"supavisor_partisan_local_1_1\"",
"state": "idle"
},
{
"pid": 175,
"datname": "postgres",
"usename": "postgres",
"application_name": "Supavisor",
"client_hostname": null,
"client_port": 36648,
"backend_start": "2024-10-16 13:40:29.211433+00",
"query_start": "2024-10-16 13:54:00.944735+00",
"query": "COMMIT",
"state": "idle"
},
{
"pid": 184,
"datname": "postgres",
"usename": "authenticator",
"application_name": "PostgREST 12.2.0 (ec89f6b)",
"client_hostname": null,
"client_port": 54168,
"backend_start": "2024-10-16 13:40:29.804391+00",
"query_start": "2024-10-16 13:56:00.550649+00",
"query": "COMMIT",
"state": "idle"
},
{
"pid": 185,
"datname": "postgres",
"usename": "postgres",
"application_name": "Supavisor",
"client_hostname": null,
"client_port": 36650,
"backend_start": "2024-10-16 13:40:29.952849+00",
"query_start": "2024-10-16 13:54:00.965682+00",
"query": "COMMIT",
"state": "idle"
},
{
"pid": 186,
"datname": "postgres",
"usename": "postgres",
"application_name": "Supavisor",
"client_hostname": null,
"client_port": 36660,
"backend_start": "2024-10-16 13:40:29.997049+00",
"query_start": "2024-10-16 13:54:35.871509+00",
"query": "COMMIT",
"state": "idle"
},
{
"pid": 190,
"datname": "postgres",
"usename": "supabase_admin",
"application_name": "supabase_mt_realtime",
"client_hostname": null,
"client_port": 57040,
"backend_start": "2024-10-16 13:40:30.914223+00",
"query_start": "2024-10-16 13:54:41.699939+00",
"query": "SELECT e0.\"id\", e0.\"type\", e0.\"settings\", e0.\"tenant_external_id\", e0.\"inserted_at\", e0.\"updated_at\", e0.\"tenant_external_id\" FROM \"extensions\" AS e0 WHERE (e0.\"tenant_external_id\" = $1) ORDER BY e0.\"tenant_external_id\"",
"state": "idle"
},
{
"pid": 187,
"datname": "postgres",
"usename": "supabase_admin",
"application_name": "supabase_mt_realtime",
"client_hostname": null,
"client_port": 57006,
"backend_start": "2024-10-16 13:40:30.911906+00",
"query_start": "2024-10-16 13:55:11.723276+00",
"query": "SELECT t0.\"id\", t0.\"name\", t0.\"external_id\", t0.\"jwt_secret\", t0.\"jwt_jwks\", t0.\"postgres_cdc_default\", t0.\"max_concurrent_users\", t0.\"max_events_per_second\", t0.\"max_bytes_per_second\", t0.\"max_channels_per_client\", t0.\"max_joins_per_second\", t0.\"suspend\", t0.\"notify_private_alpha\", t0.\"inserted_at\", t0.\"updated_at\" FROM \"tenants\" AS t0 WHERE (t0.\"external_id\" = $1)",
"state": "idle"
},
{
"pid": 189,
"datname": "postgres",
"usename": "supabase_admin",
"application_name": "supabase_mt_realtime",
"client_hostname": null,
"client_port": 57032,
"backend_start": "2024-10-16 13:40:30.913536+00",
"query_start": "2024-10-16 13:55:41.744803+00",
"query": "SELECT e0.\"id\", e0.\"type\", e0.\"settings\", e0.\"tenant_external_id\", e0.\"inserted_at\", e0.\"updated_at\", e0.\"tenant_external_id\" FROM \"extensions\" AS e0 WHERE (e0.\"tenant_external_id\" = $1) ORDER BY e0.\"tenant_external_id\"",
"state": "idle"
},
{
"pid": 188,
"datname": "postgres",
"usename": "supabase_admin",
"application_name": "supabase_mt_realtime",
"client_hostname": null,
"client_port": 57020,
"backend_start": "2024-10-16 13:40:30.913086+00",
"query_start": "2024-10-16 13:53:41.633348+00",
"query": "SELECT t0.\"id\", t0.\"name\", t0.\"external_id\", t0.\"jwt_secret\", t0.\"jwt_jwks\", t0.\"postgres_cdc_default\", t0.\"max_concurrent_users\", t0.\"max_events_per_second\", t0.\"max_bytes_per_second\", t0.\"max_channels_per_client\", t0.\"max_joins_per_second\", t0.\"suspend\", t0.\"notify_private_alpha\", t0.\"inserted_at\", t0.\"updated_at\" FROM \"tenants\" AS t0 WHERE (t0.\"external_id\" = $1)",
"state": "idle"
},
{
"pid": 191,
"datname": "postgres",
"usename": "supabase_admin",
"application_name": "supabase_mt_realtime",
"client_hostname": null,
"client_port": 57052,
"backend_start": "2024-10-16 13:40:30.915136+00",
"query_start": "2024-10-16 13:55:41.743025+00",
"query": "SELECT t0.\"id\", t0.\"name\", t0.\"external_id\", t0.\"jwt_secret\", t0.\"jwt_jwks\", t0.\"postgres_cdc_default\", t0.\"max_concurrent_users\", t0.\"max_events_per_second\", t0.\"max_bytes_per_second\", t0.\"max_channels_per_client\", t0.\"max_joins_per_second\", t0.\"suspend\", t0.\"notify_private_alpha\", t0.\"inserted_at\", t0.\"updated_at\" FROM \"tenants\" AS t0 WHERE (t0.\"external_id\" = $1)",
"state": "idle"
},
{
"pid": 200,
"datname": "postgres",
"usename": "supabase_admin",
"application_name": "realtime_connect",
"client_hostname": null,
"client_port": 60104,
"backend_start": "2024-10-16 13:40:37.732768+00",
"query_start": "2024-10-16 13:56:03.222466+00",
"query": "COMMIT",
"state": "idle"
},
{
"pid": 205,
"datname": "postgres",
"usename": "supabase_admin",
"application_name": "realtime_listen",
"client_hostname": null,
"client_port": 60140,
"backend_start": "2024-10-16 13:40:37.803433+00",
"query_start": "2024-10-16 13:40:37.811144+00",
"query": "LISTEN \"realtime:broadcast\"",
"state": "idle"
},
{
"pid": 224,
"datname": "postgres",
"usename": "supabase_admin",
"application_name": "realtime_rls",
"client_hostname": null,
"client_port": 34460,
"backend_start": "2024-10-16 13:40:46.659494+00",
"query_start": "2024-10-16 13:56:03.672074+00",
"query": "select * from realtime.list_changes($1, $2, $3, $4)",
"state": "idle"
},
{
"pid": 225,
"datname": "postgres",
"usename": "supabase_admin",
"application_name": "realtime_subscription_manager",
"client_hostname": null,
"client_port": 34470,
"backend_start": "2024-10-16 13:40:46.675005+00",
"query_start": "2024-10-16 13:49:46.79762+00",
"query": "select\n schemaname, tablename, format('%I.%I', schemaname, tablename)::regclass as oid\n from pg_publication_tables where pubname = $1",
"state": "idle"
},
{
"pid": 227,
"datname": "postgres",
"usename": "supabase_admin",
"application_name": "realtime_subscription_manager",
"client_hostname": null,
"client_port": 34494,
"backend_start": "2024-10-16 13:40:46.677982+00",
"query_start": "2024-10-16 13:54:46.813531+00",
"query": "select\n schemaname, tablename, format('%I.%I', schemaname, tablename)::regclass as oid\n from pg_publication_tables where pubname = $1",
"state": "idle"
},
{
"pid": 226,
"datname": "postgres",
"usename": "supabase_admin",
"application_name": "realtime_subscription_manager",
"client_hostname": null,
"client_port": 34486,
"backend_start": "2024-10-16 13:40:46.679466+00",
"query_start": "2024-10-16 13:53:46.810077+00",
"query": "select\n schemaname, tablename, format('%I.%I', schemaname, tablename)::regclass as oid\n from pg_publication_tables where pubname = $1",
"state": "idle"
},
{
"pid": 228,
"datname": "postgres",
"usename": "supabase_admin",
"application_name": "realtime_subscription_manager",
"client_hostname": null,
"client_port": 34502,
"backend_start": "2024-10-16 13:40:46.679695+00",
"query_start": "2024-10-16 13:55:46.816033+00",
"query": "select\n schemaname, tablename, format('%I.%I', schemaname, tablename)::regclass as oid\n from pg_publication_tables where pubname = $1",
"state": "idle"
},
{
"pid": 231,
"datname": "postgres",
"usename": "supabase_admin",
"application_name": "realtime_subscription_manager_pub",
"client_hostname": null,
"client_port": 34524,
"backend_start": "2024-10-16 13:40:46.688456+00",
"query_start": null,
"query": "",
"state": "idle"
},
{
"pid": 232,
"datname": "postgres",
"usename": "supabase_admin",
"application_name": "realtime_subscription_manager_pub",
"client_hostname": null,
"client_port": 34532,
"backend_start": "2024-10-16 13:40:46.688757+00",
"query_start": null,
"query": "",
"state": "idle"
},
{
"pid": 230,
"datname": "postgres",
"usename": "supabase_admin",
"application_name": "realtime_subscription_manager_pub",
"client_hostname": null,
"client_port": 34516,
"backend_start": "2024-10-16 13:40:46.692246+00",
"query_start": "2024-10-16 13:40:50.560351+00",
"query": "COMMIT",
"state": "idle"
},
{
"pid": 229,
"datname": "postgres",
"usename": "supabase_admin",
"application_name": "realtime_subscription_manager_pub",
"client_hostname": null,
"client_port": 34514,
"backend_start": "2024-10-16 13:40:46.693993+00",
"query_start": null,
"query": "",
"state": "idle"
},
{
"pid": 233,
"datname": "postgres",
"usename": "supabase_admin",
"application_name": "realtime_subscription_checker",
"client_hostname": null,
"client_port": 34540,
"backend_start": "2024-10-16 13:40:46.76379+00",
"query_start": null,
"query": "",
"state": "idle"
},
{
"pid": 235,
"datname": "postgres",
"usename": "supabase_admin",
"application_name": "realtime_subscription_checker",
"client_hostname": null,
"client_port": 34568,
"backend_start": "2024-10-16 13:40:46.76627+00",
"query_start": null,
"query": "",
"state": "idle"
},
{
"pid": 234,
"datname": "postgres",
"usename": "supabase_admin",
"application_name": "realtime_subscription_checker",
"client_hostname": null,
"client_port": 34554,
"backend_start": "2024-10-16 13:40:46.766898+00",
"query_start": null,
"query": "",
"state": "idle"
},
{
"pid": 236,
"datname": "postgres",
"usename": "supabase_admin",
"application_name": "realtime_subscription_checker",
"client_hostname": null,
"client_port": 34570,
"backend_start": "2024-10-16 13:40:46.7674+00",
"query_start": null,
"query": "",
"state": "idle"
},
{
"pid": 401,
"datname": "postgres",
"usename": "postgres",
"application_name": "Supavisor",
"client_hostname": null,
"client_port": 54860,
"backend_start": "2024-10-16 13:42:31.99678+00",
"query_start": "2024-10-16 13:55:51.562957+00",
"query": "SELECT 1",
"state": "idle"
},
{
"pid": 422,
"datname": "postgres",
"usename": "postgres",
"application_name": "Supavisor",
"client_hostname": null,
"client_port": 46002,
"backend_start": "2024-10-16 13:42:38.50249+00",
"query_start": "2024-10-16 13:54:00.978253+00",
"query": "COMMIT",
"state": "idle"
},
{
"pid": 487,
"datname": "postgres",
"usename": "postgres",
"application_name": "Supavisor",
"client_hostname": null,
"client_port": 54318,
"backend_start": "2024-10-16 13:43:21.097672+00",
"query_start": "2024-10-16 13:56:00.275799+00",
"query": "COMMIT",
"state": "idle"
},
{
"pid": 988,
"datname": "postgres",
"usename": "postgres",
"application_name": "Supavisor",
"client_hostname": null,
"client_port": 36120,
"backend_start": "2024-10-16 13:48:25.175413+00",
"query_start": "2024-10-16 13:56:03.644918+00",
"query": "DEALLOCATE ALL",
"state": "idle in transaction"
},
{
"pid": 1149,
"datname": "postgres",
"usename": "postgres",
"application_name": "Supavisor",
"client_hostname": null,
"client_port": 51440,
"backend_start": "2024-10-16 13:49:59.097429+00",
"query_start": "2024-10-16 13:56:00.105589+00",
"query": "COMMIT",
"state": "idle"
},
{
"pid": 1380,
"datname": "postgres",
"usename": "postgres",
"application_name": "Supavisor",
"client_hostname": null,
"client_port": 43326,
"backend_start": "2024-10-16 13:52:19.813086+00",
"query_start": "2024-10-16 13:55:52.888797+00",
"query": "COMMIT",
"state": "idle"
},
{
"pid": 1506,
"datname": "postgres",
"usename": "postgres",
"application_name": "Supavisor",
"client_hostname": null,
"client_port": 44386,
"backend_start": "2024-10-16 13:53:33.248615+00",
"query_start": "2024-10-16 13:53:33.261937+00",
"query": "SELECT 1",
"state": "idle"
},
{
"pid": 1760,
"datname": "postgres",
"usename": "supabase_admin",
"application_name": "",
"client_hostname": null,
"client_port": 46736,
"backend_start": "2024-10-16 13:56:03.692324+00",
"query_start": "2024-10-16 13:56:03.701268+00",
"query": "\n-- source: dashboard\n-- user: self host\n-- date: 2024-10-16T13:56:03.685Z\n\nSELECT\npid\n,datname\n,usename\n,application_name\n,client_hostname\n,client_port\n,backend_start\n,query_start\n,query\n,state\nFROM pg_stat_activity limit 100;\n",
"state": "active"
},
{
"pid": 32,
"datname": null,
"usename": null,
"application_name": "",
"client_hostname": null,
"client_port": null,
"backend_start": "2024-10-16 13:39:53.822902+00",
"query_start": null,
"query": "",
"state": null
},
{
"pid": 31,
"datname": null,
"usename": null,
"application_name": "",
"client_hostname": null,
"client_port": null,
"backend_start": "2024-10-16 13:39:53.822041+00",
"query_start": null,
"query": "",
"state": null
},
{
"pid": 34,
"datname": null,
"usename": null,
"application_name": "",
"client_hostname": null,
"client_port": null,
"backend_start": "2024-10-16 13:39:55.234574+00",
"query_start": null,
"query": "",
"state": null
}
] |
|
nothing looks unusual, and you only had 84/100 total connection with 11 from pooler so why youre hitting connection limits i really dont know |
|
i recently did a server reboot so probably closed a bunch of connections. i can check again later in the day |
|
something's not right about this implementation, but i didn't have max connection issues with my old supavisor setup at #2712 and do with the new coolify template this time it's 89/100 connections. why so many idle connections from supavisor_meta? basically all of these are idle except for like 2? supabase_default_Untitled query.csv got some help on discord, added |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Please Note this is a breaking change for existing templates, due to moving logs out of main database.
This template now has the supavisor builtin and matches the current version found for self hosted supabase
due to init scripts not running for existing databases, you can simply run the 2 new and 1 altered sql file before implementing:
create the _supavisor db, create 2 schemas for logs and supavisor
the template will then work as expected
Changes
updated versions for most images
added supavisor
move logs out of main database
Issues
Add supavisor to supabase #2712