New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Failed Workflow memoized resubmit does not respect DAG dependencies #12936
Labels
Milestone
Comments
shuangkun
added
area/templates/dag
area/memoization
area/server
and removed
area/memoization
labels
Apr 15, 2024
Reproduced it and will have a look. |
agilgur5
changed the title
Failed Workflow resubmit does not respect DAG dependencies
Failed Workflow memoized resubmit does not respect DAG dependencies
Apr 15, 2024
shuangkun
added a commit
to shuangkun/argo-workflows
that referenced
this issue
Apr 15, 2024
Signed-off-by: shuangkun <[email protected]>
shuangkun
added a commit
to shuangkun/argo-workflows
that referenced
this issue
Apr 15, 2024
Signed-off-by: shuangkun <[email protected]>
Hi @shuangkun. Any update on this? When could we expect this PR to be merged? Many thanks 🙏 |
Oh, I will ask an approver to check it. |
terrytangyuan
pushed a commit
that referenced
this issue
May 17, 2024
…12940) Signed-off-by: shuangkun <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Pre-requisites
:latest
What happened/what did you expect to happen?
I have a data pipeline DAG composed of a series of dependent transformations. When one particular tasks fails, I would expect a memoized resubmit to restart at the failed task and to respect dependencies.
What actually happens is that all subsequent tasks are executed at the same time, without respecting their dependencies. This happens whether
failFast
is set to true or false. This happens using both the UI and the CLI.Here is the original workflow view:
Now when I run
argo resubmit --memoized resubmit-bug-dag
, the resubmit workflow ends up in the following state:Thus I cannot use the resubmit
memoized
feature for the DAG as it breaks my pipelines. For the moment, I think I will try to implement step-level memoization as a workaround, though it is not optimal as I now have to set an arbitrarymaxAge
for future pipelines to properly work.Thank you for your help!
Version
v3.5.5
Paste a small workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.
Logs from the workflow controller
Logs from in your workflow's wait container
The text was updated successfully, but these errors were encountered: