[WIP] Fix database sync #11879

koppor · 2024-10-03T21:42:47Z

Improves the NOTIFY and LISTEN for PostgreSQL database (to get content)

Removes MySQL/MariaDB and Oracle support (due to high maintenance effort).

Open points:

finish ^^
Include idea of Draft for fixing sync with groups and shared db #11314

Mandatory checks

Change in CHANGELOG.md described in a way that is understandable for the average user (if applicable)
Tests created for changes (if applicable)
Manually tested changed features in running JabRef (always required)
Screenshots added in PR description (for UI changes)
Checked developer's documentation: Is the information available and up to date? If not, I outlined it in this pull request.
Checked documentation: Is the information available and up to date? If not, I created an issue at https://github.com/JabRef/user-documentation/issues or, even better, I submitted a pull request to the documentation repository.

Co-authored-by: Christoph <[email protected]>

github-actions

Your code currently does not meet JabRef's code guidelines.
We use Checkstyle to identify issues.
The tool reviewdog already placed comments on GitHub to indicate the places. See the tab "Files" in you PR.
Please carefully follow the setup guide for the codestyle.
Afterwards, please run checkstyle locally and fix the issues.

You can check review dog's comments at the tab "Files changed" of your pull request.

- try to simplify SQL statements - create indexes

Siedlerchr · 2024-10-04T22:06:16Z

src/main/java/org/jabref/logic/shared/PostgreSQLProcessor.java

-                        + "\"KEY\" VARCHAR,"
-                        + "\"VALUE\" TEXT)");
+        // TODO: Before a release, fix the names (and migrate data to the new names)
+        //       Think of using Flyway or Liquibase instead of manual migration


I have experience with liquibase, we use it at work.they offer a gralde plugin https://contribute.liquibase.com/extensions-integrations/directory/integration-docs/gradle/

https://forum.liquibase.org/t/adding-liquibase-to-an-existing-project/6076

github-actions

You modified Markdown (*.md) files and did not meet JabRef's rules for consistently formatted Markdown files.
To ensure consistent styling, we have markdown-lint in place.
Markdown lint's rules help to keep our Markdown files consistent within this repository and consistent with the Markdown files outside here.

You can check the detailed error output by navigating to your pull request, selecting the tab "Checks", section "Tests" (on the left), subsection "Markdown".

github-actions

Your code currently does not meet JabRef's code guidelines.
We use Checkstyle to identify issues.
The tool reviewdog already placed comments on GitHub to indicate the places. See the tab "Files" in you PR.
Please carefully follow the setup guide for the codestyle.
Afterwards, please run checkstyle locally and fix the issues.

You can check review dog's comments at the tab "Files changed" of your pull request.

github-actions

Your code currently does not meet JabRef's code guidelines.
We use Checkstyle to identify issues.
The tool reviewdog already placed comments on GitHub to indicate the places. See the tab "Files" in you PR.
Please carefully follow the setup guide for the codestyle.
Afterwards, please run checkstyle locally and fix the issues.

You can check review dog's comments at the tab "Files changed" of your pull request.

* upstream/main: Bump luceneVersion from 9.11.1 to 9.12.0 (#11888) Bump buildres/abbrv.jabref.org from `234cce8` to `6ec4cd0` (#11899)

github-actions

You modified Markdown (*.md) files and did not meet JabRef's rules for consistently formatted Markdown files.
To ensure consistent styling, we have markdown-lint in place.
Markdown lint's rules help to keep our Markdown files consistent within this repository and consistent with the Markdown files outside here.

You can check the detailed error output by navigating to your pull request, selecting the tab "Checks", section "Tests" (on the left), subsection "Markdown".

github-actions

You modified Markdown (*.md) files and did not meet JabRef's rules for consistently formatted Markdown files.
To ensure consistent styling, we have markdown-lint in place.
Markdown lint's rules help to keep our Markdown files consistent within this repository and consistent with the Markdown files outside here.

You can check the detailed error output by navigating to your pull request, selecting the tab "Checks", section "Tests" (on the left), subsection "Markdown".

github-actions · 2024-10-09T06:04:36Z

src/main/java/org/jabref/logic/shared/notifications/FieldChange.java

+        String bibEntryId,
+        String field,
+        String oldValue,
+        String newValue) {}


🚫 [checkstyle] <com.puppycrawl.tools.checkstyle.checks.whitespace.WhitespaceAroundCheck> _{reported by reviewdog 🐶}
'{' is not followed by whitespace.

github-actions · 2024-10-09T06:04:36Z

src/main/java/org/jabref/logic/shared/notifications/Notifier.java

+import com.google.gson.Gson;
+import com.google.gson.GsonBuilder;
+
+import org.jabref.model.entry.event.FieldChangedEvent;


🚫 [checkstyle] <com.puppycrawl.tools.checkstyle.checks.imports.ImportOrderCheck> _{reported by reviewdog 🐶}
Wrong order for 'org.jabref.model.entry.event.FieldChangedEvent' import.

github-actions · 2024-10-09T06:04:36Z

src/main/java/org/jabref/logic/shared/notifications/Notifier.java

+import com.google.gson.GsonBuilder;
+
+import org.jabref.model.entry.event.FieldChangedEvent;
+import org.postgresql.PGConnection;


🚫 [checkstyle] <com.puppycrawl.tools.checkstyle.checks.imports.ImportOrderCheck> _{reported by reviewdog 🐶}
'org.postgresql.PGConnection' should be separated from previous imports.

github-actions · 2024-10-09T06:04:36Z

src/main/java/org/jabref/logic/shared/notifications/Notifier.java

+        this.pgConnection = pgConnection;
+    }
+
+    public void notifyAboutChangedField(FieldChangedEvent event) {}


🚫 [checkstyle] <com.puppycrawl.tools.checkstyle.checks.whitespace.WhitespaceAroundCheck> _{reported by reviewdog 🐶}
'{' is not followed by whitespace.

github-actions

Your code currently does not meet JabRef's code guidelines.
We use Checkstyle to identify issues.
The tool reviewdog already placed comments on GitHub to indicate the places. See the tab "Files" in you PR.
Please carefully follow the setup guide for the codestyle.
Afterwards, please run checkstyle locally and fix the issues.

You can check review dog's comments at the tab "Files changed" of your pull request.

github-actions

Your code currently does not meet JabRef's code guidelines.
We use Checkstyle to identify issues.
The tool reviewdog already placed comments on GitHub to indicate the places. See the tab "Files" in you PR.
Please carefully follow the setup guide for the codestyle.
Afterwards, please run checkstyle locally and fix the issues.

You can check review dog's comments at the tab "Files changed" of your pull request.

# Conflicts: # src/main/java/org/jabref/gui/shared/SharedDatabaseUIManager.java

github-actions · 2024-10-10T13:43:18Z

src/main/java/org/jabref/logic/shared/DBMSProcessor.java

@@ -506,38 +527,26 @@ public List<BibEntry> partitionAndGetSharedEntries(List<Integer> sharedIDs) {
    /**
     * Queries the database for shared entries. Optionally, they are filtered by the given list of sharedIds
     *
-     * @param sharedIDs the list of Ids to filter. If list is empty, then no filter is applied
+     * @param sharedIds the list of Ids to filter. If list is empty, then no filter is applied


🚫 [checkstyle] <com.puppycrawl.tools.checkstyle.checks.javadoc.JavadocMethodCheck> _{reported by reviewdog 🐶}
Unused @param tag for 'sharedIds'.

github-actions

Your code currently does not meet JabRef's code guidelines.
We use Checkstyle to identify issues.
The tool reviewdog already placed comments on GitHub to indicate the places. See the tab "Files" in you PR.
Please carefully follow the setup guide for the codestyle.
Afterwards, please run checkstyle locally and fix the issues.

You can check review dog's comments at the tab "Files changed" of your pull request.

# Conflicts: # build.gradle # src/main/java/module-info.java # src/main/java/org/jabref/logic/shared/notifications/NotificationListener.java

github-actions · 2024-11-12T08:58:42Z

The build of this PR is available at https://builds.jabref.org/pull/11879/merge.

koppor and others added 10 commits October 3, 2024 22:22

Refactor method signature

443765f

Co-authored-by: Christoph <[email protected]>

Rename maps

f3921c2

Co-authored-by: Christoph <[email protected]>

Cleanup EntriesAddedEvent

92a2287

Co-authored-by: Christoph <[email protected]>

Clean up EntriesRemovedEvent

f0465a6

Co-authored-by: Christoph <[email protected]>

Disable save actions for SQL

95fe7f6

Co-authored-by: Christoph <[email protected]>

Fix logger string

279da5b

Add TODO

e19bc1a

Co-authored-by: Christoph <[email protected]>

Remove MySQL and Oracle dependencies

90e067d

Co-authored-by: Christoph <[email protected]>

Remove PullChangesFromSharedAction

35d6411

Co-authored-by: Christoph <[email protected]>

Improve doc and code comments

f4f40c6

Co-authored-by: Christoph <[email protected]>

koppor added this to the 6.0-beta milestone Oct 3, 2024

github-actions bot reviewed Oct 3, 2024

View reviewed changes

koppor and others added 15 commits October 4, 2024 14:03

Mark variable to be removed

7e459be

Fix class name

f901151

Introduce BibEntry.ENTRY_LINK_SEPARATOR

f6a2fe5

Cleanup code in CitationKeyListener

8a11cb9

More Java 8

fd958b6

Minor code updates

fd7697b

Begin: separate metadata sync with all-entry-sync

770cef0

Use "ON CONFLICT" of PostgreSQL

2c8faa7

Remove non-used get

dcf605f

Fix language

55bb387

Fix module-info.java

fd3c184

Switch from "external" Postgres to embedded Postgres

5c863dc

Remove escape and escape_Table (and add indexes)

7e0ba3e

- try to simplify SQL statements - create indexes

Fix method name "isFiltered" and also link reasoning

407b307

Merge branch 'main' into fix-sync

f19101b

Siedlerchr reviewed Oct 4, 2024

View reviewed changes

github-actions bot reviewed Oct 4, 2024

View reviewed changes

Integrate PostgreSQLProcessor in DBMSProcessor

5a36849

koppor mentioned this pull request Oct 5, 2024

Add dockerunit JabRef/jabref-koppor#438

Closed

koppor added 2 commits October 7, 2024 12:19

Merge branch 'fix-sync' of github.com:JabRef/jabref into fix-sync

2e66e10

Merge branch 'main' into fix-sync

383c665

github-actions bot reviewed Oct 7, 2024

View reviewed changes

koppor and others added 2 commits October 7, 2024 12:26

Some minor code comments and SQL optimization

5bc7abd

Merge branch 'main' into fix-sync

f919bbd

github-actions bot reviewed Oct 7, 2024

View reviewed changes

Siedlerchr added 2 commits October 8, 2024 19:38

fix compile errors

c901b84

Merge remote-tracking branch 'upstream/main' into fix-sync

d2afcb4

* upstream/main: Bump luceneVersion from 9.11.1 to 9.12.0 (#11888) Bump buildres/abbrv.jabref.org from `234cce8` to `6ec4cd0` (#11899)

github-actions bot reviewed Oct 8, 2024

View reviewed changes

Merge branch 'fix-sync' of github.com:JabRef/jabref into fix-sync

cedafb1

github-actions bot reviewed Oct 8, 2024

View reviewed changes

WIP: notificatoin

d1fd3f0

github-actions bot reviewed Oct 9, 2024

View reviewed changes

koppor added 2 commits October 9, 2024 22:58

Merge branch 'main' into fix-sync

e4c7130

Fix compile error (and typo)

598d83e

github-actions bot reviewed Oct 9, 2024

View reviewed changes

koppor added 5 commits October 10, 2024 08:30

Add databaseName as parameter

a78afc9

Compilefix

7eb4b07

Merge branch 'fix-sync' of github.com:JabRef/jabref into fix-sync

a9a2708

# Conflicts: # src/main/java/org/jabref/gui/shared/SharedDatabaseUIManager.java

Remove unnecessary JavaDoc

bc68c19

Fix tests

dd64830

github-actions bot reviewed Oct 10, 2024

View reviewed changes

koppor added 2 commits November 7, 2024 12:50

Merge branch 'main' into fix-sync

801acdf

# Conflicts: # build.gradle # src/main/java/module-info.java # src/main/java/org/jabref/logic/shared/notifications/NotificationListener.java

Compile fix

1b896d0

koppor self-assigned this Nov 11, 2024

Merge branch 'main' into fix-sync

2dd2beb

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] Fix database sync #11879

[WIP] Fix database sync #11879

koppor commented Oct 3, 2024 •

edited

Loading

github-actions bot left a comment

Siedlerchr Oct 4, 2024

Siedlerchr Oct 4, 2024

github-actions bot left a comment

github-actions bot left a comment

github-actions bot left a comment

github-actions bot left a comment

github-actions bot left a comment

github-actions bot Oct 9, 2024

github-actions bot Oct 9, 2024

github-actions bot Oct 9, 2024

github-actions bot Oct 9, 2024

github-actions bot left a comment

github-actions bot left a comment

github-actions bot Oct 10, 2024

github-actions bot left a comment

github-actions bot commented Nov 12, 2024

[WIP] Fix database sync #11879

Are you sure you want to change the base?

[WIP] Fix database sync #11879

Conversation

koppor commented Oct 3, 2024 • edited Loading

Mandatory checks

github-actions bot left a comment

Choose a reason for hiding this comment

Siedlerchr Oct 4, 2024

Choose a reason for hiding this comment

Siedlerchr Oct 4, 2024

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot Oct 9, 2024

Choose a reason for hiding this comment

github-actions bot Oct 9, 2024

Choose a reason for hiding this comment

github-actions bot Oct 9, 2024

Choose a reason for hiding this comment

github-actions bot Oct 9, 2024

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot Oct 10, 2024

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot commented Nov 12, 2024

koppor commented Oct 3, 2024 •

edited

Loading