Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP Fix for LOCALLY parallelism #3417

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

WIP Fix for LOCALLY parallelism #3417

wants to merge 2 commits into from

Conversation

vladaionescu
Copy link
Member

@vladaionescu vladaionescu commented Oct 21, 2023

LOCALLY targets should not execute in parallel. This PR fixes that by introducing a global lock.

This is WIP - it seems that this is introducing a deadlock for now. Needs more work.

@idelvall
Copy link
Member

LOCALLY targets should not execute in parallel

Why is that? :)

@vladaionescu
Copy link
Member Author

It's a design principle - that things should be as repeatable as possible. We figured that kinda like how you rarely run make with the -j argument because it's hard to get parallelism right when the environment is shared, we should avoid parallelizing in LOCALLY cases in Earthly too.

It's probably not an important guardrail now that there is also WAIT, which allows the user to explicitly control parallelism where necessary. Plus nobody has asked for this feature, so we could also abandon it. CC @alexcb

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: In Progress
Development

Successfully merging this pull request may close these issues.

None yet

2 participants