You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
By @rashadg1030 calculations it takes ~6 hours to sync only all repos. And I can't even imagine how much time it will take to sync all issues as well... For testing purposes, we can have a configurable function that allows us to specify caching in a more flexible way (see issue #126 for more details). But we need some long-term strategy as well...
The text was updated successfully, but these errors were encountered:
@chshersh Maybe we should only go back to like 2015 or something. That will save time. I think right now we sync repos going all the way back to 2011 or so. I doubt users will want to go that far back, but on the other hand, there are old repos that might be popular.
A repository can be created a long time ago, but it's still can be maintained. So we need all repositories eventually. I propose that we will have 3-stage syncing for the proper implementation eventually:
First initial fat sync that fetches all information. It's supposed to be launched only once when the application is started. But it would be good to implement it as a separate task if we want to perform total sync once per year.
A daily job that will sync for the last 1-2-3 (can be configurable) years of data.
When Admin is implemented, we should be able to force sync specific repo.
By @rashadg1030 calculations it takes ~6 hours to sync only all repos. And I can't even imagine how much time it will take to sync all issues as well... For testing purposes, we can have a configurable function that allows us to specify caching in a more flexible way (see issue #126 for more details). But we need some long-term strategy as well...
The text was updated successfully, but these errors were encountered: