Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add database support #43

Open
haijeploeg opened this issue Jan 6, 2022 · 2 comments
Open

Add database support #43

haijeploeg opened this issue Jan 6, 2022 · 2 comments
Labels
enhancement New feature or request
Milestone

Comments

@haijeploeg
Copy link
Owner

Is your feature request related to a problem? Please describe.
Right now Excludarr is pulling all the data from JustWatch and doing multiple searches at JustWatch to gather the justwatch ID. This is causing an overload of API calls at JustWatch, when you only need to pull this data once in a while and not everytime you run the script. Big libraries take a lot of time to process right now. Also the re-add function only works with not-monitored media. If we keep a local database in sync with Radarr and Sonarr, we can also use the re-add functionality on deleted media.

Describe the solution you'd like
Sync all records of Sonarr and Radarr to a database with the JustWatch ID and the streaming providers for the configured locale. Using this principle we can fetch all the information from the database and speed up the process. Syncing the database once or twice a week will be sufficient. Also add a flag to force a sync before exclude or re-add the movies/series.

Describe alternatives you've considered
A web application will be too big for the project to maintain. Another solution is to store a file based database like json files over the system. But this will be too hard to maintain for big libraries and will possible result in high IO.

Additional context
None

@haijeploeg haijeploeg added the enhancement New feature or request label Jan 6, 2022
@haijeploeg haijeploeg added this to the v1.1.0 milestone Jan 6, 2022
@haijeploeg
Copy link
Owner Author

Tried to do this with a database, but it is really hard to keep that database up to date. The database needs to be always in sync with Radarr/Sonarr. I tried a PoC, but in the end it was too complex, too hard to maintain and it tooks longer.

Another solution is to do this with labels on the Radarr/Sonarr entries. Once the Justwatch ID is found, place a label with the JustWatchID on the Radarr/Sonarr entry. Everytime a search is done, look for this label, otherwise find it and create the label. This will speed up the next few iterations as the JustWatchID is already found.

@faulander
Copy link

i will fork excludarr and implement a json file with just the sonarr id and the justwatch id. Let's see how much i/o this really is.
Whenever excludarr is run, first it checks if all justwatch ids are still in sonarr, if not clean the json file. then get the missing justwatch id's.
I am not sure what you mean by it's hard to keep them synced, you don't need to. It's enough and should speed up the process considerably when running excludarr.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants