Skip to content
This repository has been archived by the owner on Aug 28, 2023. It is now read-only.

Fix memory issue in run_jobs #817

Open
CloudNiner opened this issue Jun 25, 2018 · 1 comment
Open

Fix memory issue in run_jobs #817

CloudNiner opened this issue Jun 25, 2018 · 1 comment

Comments

@CloudNiner
Copy link
Contributor

One of the changes in #804 appears to have re-created the issue where each dataset import consumes a large amount of memory (up to 8gb), preventing the regular ECS management task from being used. As a workaround, we've created a custom ECS management task that allots 8gb memory instead of the usual 2gb.

The goal of this task would be to find and fix the memory regression that would allow us to import using the regular management task again.

The original memory improvement was added in #754

@CloudNiner
Copy link
Contributor Author

This issue bit @flibbertigibbet again when testing #854

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

2 participants