AFFiNE selfhost

[Brooooooklyn]

Ask your questions about selfhost AFFiNE here

[phankhanhpvk]

hi how i can config google oauth

[Brooooooklyn]

Set these environment variables to the Docker image:

• OAUTH_GOOGLE_ENABLED=true
• OAUTH_GOOGLE_CLIENT_ID=YOUR_GOOGLE_CLIENT_ID
• OAUTH_GOOGLE_CLIENT_SECRET=YOUR_GOOGLE_CLIENT_SECRET

[ylbeethoven]

Did you remove the Google log in option in 0.13.0?

OAUTH_GOOGLE_ENABLED has no effect either

~$ docker exec -it affine sh
# printenv | grep OAUTH
OAUTH_GOOGLE_ENABLED=true

[darkskygit]

we have refactor the oauth logic as a plugins in 0.13
now as long as the OAUTH_GOOGLE_CLIENT_ID and OAUTH_GOOGLE_CLIENT_SECRET are configured, the relevant modules will be automatically enabled, therefore OAUTH_GOOGLE_ENABLED has been removed

[ylbeethoven]

Hi @darkskygit

I tried these 2 environment variable and I still don't see the google log in button.

~$ docker exec -it affine sh
# printenv | grep OAUTH
OAUTH_GOOGLE_CLIENT_ID=xxx
OAUTH_GOOGLE_CLIENT_SECRET=xxx

Version
AFFiNE
0.13.0
AFFiNE
Editor Version
0.13.0-canary-202403140735-2367cd5

[RandomLegend]

Do i understand correct i am limited to 10GB storage when selfhosting??? And still the 10MB upload max?

[forehalo]

That's not supposed to be. We will fix it.

[RandomLegend]

Awesome!

Is there an ETA ?

[glego]

Reference issue: #6641

[gymnae]

Hi there, thanks for putting in the effort to provide a ready-to-go Docker compose file, getting up and running was a breeze.
Couple of questions before I consider onboarding:

1. What are the licensing plans for self-hosting?
2. Why is a self-hosted installation connecting to affine.pro both in the back- and frontend?
3. Why does it state that I am on a free plan with the limitations of a free plan? I have not selected any plan
4. Will the limit to invite 3 other users to a workspace remain for the self-hosted variant?
5. Will you integrate custom OICD providers?

[RandomLegend]

I'm currently running the 0.13.0-canary-202403050653-934469c version and do still have the 10GB / 10MB Upload / 3 Users limitation.

I assume you will fix these in the 13.0 stable release?

Btw: Thanks for the quick responses. Awesome to see you open the project up for all us homelabbies

[gymnae]

I concurr with @RandomLegend, thank you for supporting us self-hosters, even if we fall outside of your revenue stream (for now - maybe some premium features will change that)

[RandomLegend]

I'd be happy to pay for a license. But i will never do anything subscription based i'm sorry.

If you'd offer some Lifetime License where we would get like 1 year of updates from the moment of purchase and then we can decide to stay on the last update or renew the license i'd love to support that.

But subscriptions are getting way too much. Too many things want to be subscription based and i really despise that.

[gymnae]

Hi there, with 0.13.0 the UI still shows the same limitations and "free" account note. Was it supposed to be "fixed", or is there a change of plan?

[darkskygit]

free plan for self-hosted flavor is an impl mistake, please pay attention to the progress of #6156

[RandomLegend]

Are there feature plans for:

1. Importing multiple markdown documents at once? (i have a big library of seperate markdown docs from Obsidian i want to mass import)
2. Make either a hotkey to add a Tag to a page or make it so tabbing after entering the title will jump to tags? I want to tag my documents without having to use the mouse
3. Save all the pages as markdown documents on the server. I'd love to be able to just go into the storage path and find a bunch of markdown documents for me to freely copy over and edit in any other software.

[EYHN]

The affine file is used in the desktop app, but since selfhost currently cannot use the desktop app, you cannot export the selfhost workspace as an affine file. But we expect to support selfhost using desktop app in the summer, so you'll be able to do this.

[dshiS]

That's awesome! Look forward to it!!

[dshiS]

I have deployed it on my machine. For some reason I cannot restart the docker container now after I upgrade my docker engine. Here is the error I got.

affine_selfhosted  | yarn run v1.22.19
affine_selfhosted  | $ yarn prisma migrate deploy && node --import ./scripts/register.js ./dist/data/index.js run
affine_selfhosted  | $ /app/node_modules/.bin/prisma migrate deploy
affine_selfhosted  | Error: request to https://binaries.prisma.sh/all_commits/473ed3124229e22d881cb7addf559799debae1ab/debian-openssl-3.0.x/schema-engine.sha256 failed, reason:
affine_selfhosted  | error Command failed with exit code 1.
affine_selfhosted  | info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
affine_selfhosted  | error Command failed with exit code 1.
affine_selfhosted  | info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
affine_selfhosted  | node:internal/errors:984
affine_selfhosted  |   const err = new Error(message);
affine_selfhosted  |               ^
affine_selfhosted  |
affine_selfhosted  | Error: Command failed: yarn predeploy
affine_selfhosted  |     at genericNodeError (node:internal/errors:984:15)
affine_selfhosted  |     at wrappedFn (node:internal/errors:538:14)
affine_selfhosted  |     at checkExecSyncError (node:child_process:890:11)
affine_selfhosted  |     at execSync (node:child_process:962:15)
affine_selfhosted  |     at runPredeployScript (file:///app/scripts/self-host-predeploy.js:67:3)
affine_selfhosted  |     at file:///app/scripts/self-host-predeploy.js:75:1
affine_selfhosted  |     at ModuleJob.run (node:internal/modules/esm/module_job:222:25)
affine_selfhosted  |     at async ModuleLoader.import (node:internal/modules/esm/loader:323:24)
affine_selfhosted  |     at async loadESM (node:internal/process/esm_loader:28:7)
affine_selfhosted  |     at async handleMainPromise (node:internal/modules/run_main:113:12) {
affine_selfhosted  |   status: 1,
affine_selfhosted  |   signal: null,
affine_selfhosted  |   output: [ null, null, null ],
affine_selfhosted  |   pid: 20,
affine_selfhosted  |   stdout: null,
affine_selfhosted  |   stderr: null
affine_selfhosted  | }
affine_selfhosted  |
affine_selfhosted  | Node.js v20.12.2
affine_selfhosted exited with code 0```

Could I get any help with this or recovering my previous data/notes?

Thanks so much! 

[darkskygit]

please check your network, when the container start first time, the container need to download some necessary dependencies on the Internet

[zkwolf]

add environment to docker-compose.yml PRISMA_ENGINES_MIRROR=https://registry.npmmirror.com/-/binary/prisma

[bomberstudios]

I'm trying to change my user avatar in a self-hosted instance (using stable) and while it shows me a 'success' message when uploading the new avatar, it does not change in the web app.

What am I doing wrong?

Thanks in advance, and kudos for the self-host effort. I want to use AFFiNE for my family, and not having to depend on an external service is really awesome.

[bomberstudios]

Nice! Will do and report back.

[bomberstudios]

I updated the custom options, but for some reason the app is ignoring the port:

[EYHN]

How did you configure it? The port should be another variable

[bomberstudios]

I am configuring the port everywhere I've seen it 😅

This is the command I use to launch Docker:

// This is on a git checkout of the `canary` branch:
AFFINE_SERVER_HOST=3020.local AFFINE_SERVER_PORT=3010 [email protected] AFFINE_ADMIN_PASSWORD="affine" docker compose -f ./.github/deployment/self-host/compose.yaml up --detach

This is the content of the files on my $HOME/.affine/selt-host/config/ folder:

//.env
AFFINE_SERVER_PORT=3010
AFFINE_SERVER_HOST=3020.local
AFFINE_SERVER_HTTPS=false
# DATABASE_URL="postgres://affine@localhost:5432/affine"

// affine.js
AFFiNE.https = false;
AFFiNE.host = '3020.local';
AFFiNE.port = 3010;

I know I only need to specify this stuff in one place, not all, but I did not want to take any chances 😂

In any case: image uploads don't seem to work (as another user mentioned in other comment) and I can't change the avatar for my user or the icon for a workspace. Feels like all of these might be related...

[chaoticgood1]

I confirm changing the AFFINE_SERVER_PORT=3001 and AFFINE_SERVER_HTTPS=true is not working on my end too. Is there something more we have to do to make this work? Thanks.

[zkwolf]

1. I think blocksuite should provide a way to change the default imageProxy middleware in import-doc.
   Now AFFINE use the blocksuite func with https://affine-worker.toeverything.workers.dev to proxy image，the self-hosted version should be banned by CORS.
2. About imageProxy and linkPreview worker
   AFFINE use /api/worker prefix to fetch workers but in self-hosted version there is no implement. However reverse proxy is a way to get this work. I think AFFINE can open source these workers and users can deploy this on their Cloudflare account.

[EYHN]

@darkskygit @forehalo take a look

[darkskygit]

1. we will support customization of image proxy and link preview endpoint in the next version
2. we have open sourced it here: https://github.com/toeverything/affine-workers

[zkwolf]

It seems that this repo only contains link-preview worker.

[darkskygit]

we have update the worker repos

[patrick-men]

I deployed AFFiNE with your docker compose image, and it works fine with currently one main issue: I can't upload images. Pasting screenshots doesn't do anything, and importing images with /image results in an error stating either failed to upload image! crypto.subtle is undefined or just a Image loading failed. This happens on every browser I tested so far, and I haven't managed to get any pictures into my notes. I've tried both .png and .jpg/.jpeg

Other than that, awesome app you guys made! Most everything has worked pretty well so far

[darkskygit]

same as #6049

[YourDaniel]

Is there a method to add new users without configuring mailer service? For example, via adding users directly to the database?

[EYHN]

#5957

[YourDaniel]

Thanks. But now how do you add users to your workspace? I added a bunch of users, they can login no problem. But only option to add them to workspace for shared work is via option "Invite members". But that requires configured mailer service. Is there a method to add a user to specific workspace?

[alissa-tung]

seems that

#5946 (comment)

AFFiNE/packages/backend/server/src/core/workspaces/resolvers/workspace.ts

Line 322 in a721b38

async invite(

[YourDaniel]

Thank you, that helped!

[aliuq]

Hi，Whether the self-host version is subject to free plan restrictions, such as size and members

[darkskygit]

this is an impl mistake, please pay attention to the progress of #6156

[ylbeethoven]

Hi, thanks for providing the self-hosted option. I was wondering if there is an option to disable /onboarding page and Demo Workspace. I want to lock down my instance with just a log in page.

Thank you.

[darkskygit]

please follow #6141

[Dayonixe]

Hello,
When I follow the procedure to install AFFiNE locally, I get the following problem:

affine_selfhosted  | [Nest] 135  - 03/15/2024, 6:39:50 PM     LOG [RunCommand] Running SelfHostAdmin1605053000403...
affine_selfhosted  | [Nest] 135  - 03/15/2024, 6:39:50 PM   ERROR [RunCommand] Failed to run data migration
affine_selfhosted  | [Nest] 135  - 03/15/2024, 6:39:50 PM   ERROR [RunCommand] PrismaClientKnownRequestError:
affine_selfhosted  | Invalid `prisma.user.create()` invocation:
affine_selfhosted  |
affine_selfhosted  |
affine_selfhosted  | An operation failed because it depends on one or more records that were required but not found. No 'Features' record(s) (needed to inline the relation on 'UserFeatures' record(s)) was found for a nested connect on one-to-many relation 'FeaturesToUserFeatures'.
affine_selfhosted  | error Command failed with exit code 1.
affine_selfhosted  | info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
affine_selfhosted  | node:internal/errors:932
affine_selfhosted  |   const err = new Error(message);
affine_selfhosted  |               ^
affine_selfhosted  |
affine_selfhosted  | Error: Command failed: yarn predeploy
affine_selfhosted  |     at checkExecSyncError (node:child_process:890:11)
affine_selfhosted  |     at execSync (node:child_process:962:15)
affine_selfhosted  |     at runPredeployScript (file:///app/scripts/self-host-predeploy.js:44:3)
affine_selfhosted  |     at file:///app/scripts/self-host-predeploy.js:52:1
affine_selfhosted  |     at ModuleJob.run (node:internal/modules/esm/module_job:218:25)
affine_selfhosted  |     at async ModuleLoader.import (node:internal/modules/esm/loader:329:24)
affine_selfhosted  |     at async loadESM (node:internal/process/esm_loader:28:7)
affine_selfhosted  |     at async handleMainPromise (node:internal/modules/run_main:113:12) {
affine_selfhosted  |   status: 1,
affine_selfhosted  |   signal: null,
affine_selfhosted  |   output: [ null, null, null ],
affine_selfhosted  |   pid: 20,
affine_selfhosted  |   stdout: null,
affine_selfhosted  |   stderr: null
affine_selfhosted  | }
affine_selfhosted  |
affine_selfhosted  | Node.js v20.11.1
affine_selfhosted exited with code 0

Full logs: logs.log

[EYHN]

please use the image tag with stable-cccea3e , until we fix this

[Dayonixe]

It works, thank you so much 🙏

[ylbeethoven]

Another question is how do I manage tags? Let's say I want to modify existing tags, I can't seem to find any settings to delete/edit tags.

[EYHN]

Make sure you are in 0.13.0. On the all doc page, there is a tags button at the top.
Click the button, and you can manage all tags, including deletion and modification.

[ylbeethoven]

@EYHN Thank you. 0.13.0 solves this issue.

[RandomLegend]

It appears as if the latest stable image still doesn't let you login; We still have to use stable-cccea3e ?

Also, why would you have a cloudflare check on my selfhosted install?

[RandomLegend]

What exactly do you mean with the default key should always pass? Is this in regards to my statement that i can't login?

If i use the ADMIN mail and ADMIN Password i just get a red border around the password field when trying to log in; This is after that cloudflare check.

I'm on arch and just updated everything; I use Thorium browser which is up to date.

[darkskygit]

default key means the secret key of cloudflare captcha, self hosted version now using this key by default -- the relevant modules will be disabled in the next major version of self-hosted
if there has a red border around the password field, it means that the password you entered does not comply with the default password security rules -- this may be a bug, I will check it tomorrow

[RandomLegend]

What exactly are the default password security rules?

I can't find them anywhere

[darkskygit]

length >= 8

[RandomLegend]

yeah then this is a bug.

i have 1 capital, 1 lowercase, 1 number, 1 special and more than 8 but not more than 20

[ovizii]

Is there a summary/status about self-hosting affine behind a reverse proxy?
###edit###
I tested it and it just works.

[Vinetos]

Hey!
Can you share details about how you done that ? I am trying to deploy affine behind a reverse proxy to enable HTTPS and custom domain but I did not make it work for now.
Thanks

[ovizii]

Any particular reason why the password is capped at 20 characters for the self-hosted version?

Also, I noticed that when the onboarding pages are displayed, I can click the titles, I can go to the next one, but I cannot scroll with the mouse wheel.

[darkskygit]

we use argon2 as the password hashing algorithm, length = 20 is enough to prevent your password from being cracked in a meaningful time: https://datatracker.ietf.org/doc/html/rfc9106

[ovizii]

Thx, I know, I was simply wondering why put a cap on a password length at all. Anyway, no big deal. I only noticed this issue as I had changed my password manager from using passwords to passphrases and set it to 6 words, so I went over 20 characters.

[EYHN]

I think we should review this limit, 20 characters seems too short, github is 72 characters, 1password is able to create up to 64 characters

[forehalo]

it will be able to customize the min/max length of password in selfhost server soon

[ylbeethoven]

Does anyone know what below error means?

I pulled the stable image tonight and after creating a new container with this new image, I can not start affine with below error in log.

[Nest] 81  - 03/18/2024, 2:21:24 PM     LOG [RunCommand] Done 0 migrations
Done in 5.79s.
node:internal/modules/esm/resolve:264
    throw new ERR_MODULE_NOT_FOUND(
          ^

Error [ERR_MODULE_NOT_FOUND]: Cannot find module '/app/dist/prelude' imported from /app/dist/index.js
    at finalizeResolution (node:internal/modules/esm/resolve:264:11)
    at moduleResolve (node:internal/modules/esm/resolve:917:10)
    at defaultResolve (node:internal/modules/esm/resolve:1130:11)
    at ModuleLoader.defaultResolve (node:internal/modules/esm/loader:396:12)
    at ModuleLoader.resolve (node:internal/modules/esm/loader:365:25)
    at ModuleLoader.getModuleJob (node:internal/modules/esm/loader:240:38)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/module_job:85:39)
    at link (node:internal/modules/esm/module_job:84:36) {
  code: 'ERR_MODULE_NOT_FOUND',
  url: 'file:///app/dist/prelude'
}

Node.js v20.11.1

Now I can't start affine at all even after tagging the old image to stable. Does anyone know how to fix this?

Also can we provide a specific version for ghcr.io/toeverything/affine-graphql instead of using stable?

[ylbeethoven]

sha256:c7df7f0f865d745c5f835901b91d040d3ef201a090edafdf3fa6e056b5156600

This is the image caused the issue.

[forehalo]

you may run docker pull ghcr.io/toeverything/affine-graphql:stable to get the latest image, docker compose won't auto pull the latest image for you

[ylbeethoven]

That's the exact command I used to pull the new image... and then started a new container...

Anyway, it seems there is a new image ghcr.io/toeverything/affine-graphql@sha256:dd5b1d29b0a5970dbafb0d1584ad0bc803fa0e8b3abc8d5cfa1cce034bc015ad".

It gives me the same error

[Nest] 81  - 03/19/2024, 3:12:44 AM     LOG [RunCommand] Done 0 migrations
Done in 4.89s.
node:internal/modules/esm/resolve:264
    throw new ERR_MODULE_NOT_FOUND(
          ^

Error [ERR_MODULE_NOT_FOUND]: Cannot find module '/app/dist/prelude' imported from /app/dist/index.js
    at finalizeResolution (node:internal/modules/esm/resolve:264:11)
    at moduleResolve (node:internal/modules/esm/resolve:917:10)
    at defaultResolve (node:internal/modules/esm/resolve:1130:11)
    at ModuleLoader.defaultResolve (node:internal/modules/esm/loader:396:12)
    at ModuleLoader.resolve (node:internal/modules/esm/loader:365:25)
    at ModuleLoader.getModuleJob (node:internal/modules/esm/loader:240:38)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/module_job:85:39)
    at link (node:internal/modules/esm/module_job:84:36) {
  code: 'ERR_MODULE_NOT_FOUND',
  url: 'file:///app/dist/prelude'
}

Node.js v20.11.1

[ylbeethoven]

problem solved after moving compose.yaml to a different folder.

[forehalo]

docker compose won't use the latest image automatically, you need to use --force-recreate option when restart your deploy

[matheusriios]

I'm encountering an issue while using the Affine self-hosted instance, which I've configured on Kubernetes. Specifically, I'm receiving the error message "User xxxx-yyyy-zzzz has no quota." This error is impeding my workflow, and I'm seeking assistance to resolve it.

As the error seems to be related to user quotas, I'm wondering if there are specific configurations within setup Affine itself that I may need to adjust. Could you please provide guidance on what might be causing this error in the Kubernetes environment and how I can rectify it?

[forehalo]

it might caused by a bad stable tag. if you can access your DB by someway, you may find a table named _data_migrations and truncate all records in it.
after that, ensure you are going to run node --import=./scripts/register.js ./scripts/self-host-predeploy.js before starting the server

[matheusriios]

Thank you for your suggestions. Yes, I do have access to the database, and I've already executed the command you mentioned in my entrypoint: command: ["sh", "-c", "node ./scripts/self-host-predeploy && node ./dist/index.js"].

Regarding the Kubernetes image, I am using the image ghcr.io/toeverything/affine-graphql:stable-cccea3e. As far as I'm aware, I'm using the correct image. If there are any additional steps or considerations regarding the image usage, please let me know.

[matheusriios]

@forehalo @darkskygit can you help me?

[matheusriios]

CLOSED!

[alissa-tung]

Hello, I am curious about how can I add user to workspace, since the invite user seems will not work until email service is configured.

and there is a /graphql playground, will dangerous operations needs authentication?
if everyone can view the graphql history, that is dangerous

#5946 (comment)

AFFiNE/packages/backend/server/src/core/workspaces/resolvers/workspace.ts

Line 322 in a721b38

async invite(

[forehalo]

all GraphQL APIs are protected with authentication & permission.
there is nothing called GraphQL history

[alissa-tung]

thank you, so seems that graph web playground only stores query history in browser storage

[YourDaniel]

I was not logged in as any user. Then I proceed to /graphql and was able to create a new account with SignUp mutation. I didn't provide any auth information in my API call. Then I was able to log in as a new user I created and make a new workspace. So, it seems that anyone who has URL of your Affine self-host can create any number of users, log in and start to upload a data on your server. Is creating new users without authentication a bug?

[darkskygit]

the corresponding fix has not entered the stable branch, we expect to fix this problem in the next version

[YourDaniel]

How do you safely update self-hosted running service to a new version with keeping all of your data?

[Brooooooklyn]

Just upgrade the image tag

[EricWvi]

Hi there, is object storage like S3-compatible, say Minio, support on the plan? for images and files
By the way, excellent work for ideas of Pageless 👍

[jzmini]

Has the bug that limited self-hosting to 10GB storage been resolved? Stable-4ebe8f5 doesn't seem to work.

[dasrocky]

I dont think so coz i am still facing the same issue. 10 GB hosting limit. There is work around for it tho, by manipulating the database directly