maintain import tables; document clearing; fix import bugs #284
+359
−136
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cloverich
force-pushed
the
cloverich/keep_import_tables
branch
from
8c776a0 to
43dc774
Compare
cloverich
commented
Dec 25, 2024
| @@ -38,7 +38,7 @@ export class FilesImportResolver { | |||
| ): Promise<string | undefined> => { | |||
| // check db for chronicles id matching name, if any | |||
| const result = await this.knex("import_files") | |||
| .where({ filename: name }) | |||
| .where({ filename: name, importerId: this.importerId }) | |||
There was a problem hiding this comment.
If no importerId, may grab files from different imports, specifically from different import directories. Since the current wikilinks do not handle duplicate file names, this could (did) result in e.g. a different image or video than in the original note being displayed (if they had the same name).
- re-work import process to maintain data in import table by default - fix staging error tracking so staging errors are kept but cleared on re-run - expose clear import tables button on settings, document it - fix bug where tags or wikilinks in document would cause import to fail at staging phase - fix bug where same filename from different import would be preferentially used - fix bug where empty document would fail to import (potentially leaving valid title / frontmatter behind) - add success message when clear import button succeeds
cloverich
force-pushed
the
cloverich/keep_import_tables
branch
from
43dc774 to
a29957f
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Import errors can now be found in the table and I used this process to fix my own notes, rather than have Chronicles attempt to repair everything on import (scope was blowing up). Exposed a button to clear import tables, to allow re-importing.
Note a great UX, but these two paths are possible (and I used them myself):
Staging errors are cleared automatically on each import run; everything else is persisted so re-running the importer will not create duplicate files. One major limitation is if imports are done in batches (while fixing errors), links between correct vs broken notes won't be setup properly. Chronicles could fix this, and / or could track links to surface it, but that's yet more scope. Can re-visit in future if app see's usage.
NOTE: This is the final (planned) PR in #247; at this point I can run an import from:
And all three show up in Chronicles in a functional state. There's still edge cases I don't handle (noted in code); performance of sync is bad; the IDs dont sort the way I'd hoped. But at this point the original intent of v0.8 is complete.
Closes #276