Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Searching with Postgres #11803

Merged
merged 129 commits into from
Oct 23, 2024
Merged

Searching with Postgres #11803

merged 129 commits into from
Oct 23, 2024

Conversation

LoayGhreeb
Copy link
Member

@LoayGhreeb LoayGhreeb commented Sep 22, 2024

Closes #11823
Closes #11798
Closes #10490

Resolves https://discourse.jabref.org/t/citations-missing-in-action-in-jabref-5-16/4525/11

Mandatory checks

  • Change in CHANGELOG.md described in a way that is understandable for the average user (if applicable)
  • Tests created for changes (if applicable)
  • Manually tested changed features in running JabRef (always required)
  • Screenshots added in PR description (for UI changes)
  • Checked developer's documentation: Is the information available and up to date? If not, I outlined it in this pull request.
  • Checked documentation: Is the information available and up to date? If not, I created an issue at https://github.com/JabRef/user-documentation/issues or, even better, I submitted a pull request to the documentation repository.

github-actions[bot]

This comment was marked as resolved.

@InAnYan
Copy link
Collaborator

InAnYan commented Sep 22, 2024

I have couple of questions:

  1. Is the Postgres server one for every library?
  2. One library - one DB?
  3. What is the schema of the DB? If you can explain it shortly.
  4. Where the Postgres data is stored? (Somwhere local to user, alongside BIB)

I was thinking if I can use this Postgres service as a vector database for my AI features

external-libraries.md Outdated Show resolved Hide resolved
Co-authored-by: Loay Ghreeb <[email protected]>
Co-authored-by: Carl Christian Snethlage <[email protected]>
github-actions[bot]

This comment was marked as resolved.

github-actions[bot]

This comment was marked as resolved.

github-actions[bot]

This comment was marked as resolved.

github-actions[bot]

This comment was marked as resolved.

@koppor
Copy link
Member

koppor commented Sep 23, 2024

I have couple of questions:

1. Is the Postgres **server** one for every library?

1 for running JabRef

2. One library - one **DB**?

One table per library. We kept it very simple for the search.

3. What is the schema of the DB? If you can explain it shortly.

Very simple - just one table

                CREATE TABLE IF NOT EXISTS "%s" (
                    %s TEXT NOT NULL,
                    %s TEXT NOT NULL,
                    %s TEXT,
                    PRIMARY KEY (%s, %s)
                )
                """.formatted(tableName,
                PostgreConstants.ENTRY_ID,
                PostgreConstants.FIELD_NAME,
                PostgreConstants.FIELD_VALUE,
                PostgreConstants.ENTRY_ID,
                PostgreConstants.FIELD_NAME));

Feel free to create other tables :)

4. Where the Postgres data is stored? (Somwhere local to user, alongside BIB)

In the AppData directory. Even "temp". Thus, very fragile ^^. Not sure if one can change it.

Reason for the fragile setting: The solution we are using is intended for testing. The java-german channel said: WTF, please use DuckDB. But we could not manage to get DuckDB up and running that fast. Moreover, Postgres has more plugins for speed. And third, we also support Postgres for remote storage.

You could also think of hooking into the remote postgres. Meaning: Advanced features only work with a remote postgres. (Although I like using the local postgres database. Keeps complexity for the user to a minimum. Only 500 MB additional storage is needed...)

@calixtus calixtus added this to the 6.0-alpha milestone Sep 23, 2024
@LoayGhreeb LoayGhreeb marked this pull request as ready for review October 22, 2024 22:46
@LoayGhreeb LoayGhreeb changed the title [WIP] Searching with Postgres Searching with Postgres Oct 22, 2024
@LoayGhreeb LoayGhreeb added the status: ready-for-review Pull Requests that are ready to be reviewed by the maintainers label Oct 23, 2024
Copy link
Contributor

github-actions bot commented Oct 23, 2024

The build for this PR is no longer available. Please visit https://builds.jabref.org/main/ for the latest build.

@koppor koppor added this pull request to the merge queue Oct 23, 2024
Merged via the queue into main with commit bd7219f Oct 23, 2024
30 of 31 checks passed
@koppor koppor deleted the postgresql branch October 23, 2024 11:15
@Siedlerchr
Copy link
Member

Yeah finally!

@calixtus
Copy link
Member

Yay!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component: search [outdated] type: feature status: ready-for-review Pull Requests that are ready to be reviewed by the maintainers
Projects
None yet
7 participants