Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update nextflow version to 22.10.7 #196

Open
jaserud opened this issue Jan 19, 2022 · 8 comments · Fixed by #216
Open

Update nextflow version to 22.10.7 #196

jaserud opened this issue Jan 19, 2022 · 8 comments · Fixed by #216
Assignees
Labels
blocker This issue is blocking another major High Priority

Comments

@jaserud
Copy link
Contributor

jaserud commented Jan 19, 2022

Detailed Description

Possible Implementation

  • update the version in pom file

Current Challenge

Nextflow new version has a bug, if we upgrade, our system will break. The older version affects running any WF.

Edmund is not blocked, he's on germline VC pipeline. Needs to run benchmarking test before needing the version upgrade.

Bug raised in nextflow: nextflow-io/nextflow#4019

Updates

Give them our PR, and see if they respond?
Check if they have any contribution guide.

@jaserud jaserud added the new-feature Request is a new feature label Jan 19, 2022
@jaserud jaserud changed the title Update nextflow version to update nested log4j versions Update nextflow version Jan 19, 2022
@jaserud jaserud changed the title Update nextflow version Update nextflow version to 21.10.6 Jan 19, 2022
@lindaxiang lindaxiang changed the title Update nextflow version to 21.10.6 Update nextflow version to 22.10.7 Mar 22, 2023
@lindaxiang
Copy link

We need to upgrade nextflow version to support running workflows which are written according to nf-core framework.
@Buwujiu @puneet-oicr Can you please prioritize the ticket. Thanks!

@UmmulkiramR
Copy link
Contributor

works with the new version on QA now.

@UmmulkiramR
Copy link
Contributor

The workflows triggered with the new version execute successfully the first time. For the subsequent runs, however, the library keeps throwing the IllegalArgumentException("Plugin system was already setup") exception.

After reaching out to the nextflow developers they requested for a github issue to be raised for this.
Here's the github ticket I created - nextflow-io/nextflow#4018

@Buwujiu Buwujiu added blocker This issue is blocking another and removed new-feature Request is a new feature labels Jun 15, 2023
@UmmulkiramR
Copy link
Contributor

UmmulkiramR commented Jun 28, 2023

The Nextflow team converted the 4018 bug report to a discussion. Here's the link -> nextflow-io/nextflow#4019
Requested an update again last week on this but no revert from them yet.

@Buwujiu
Copy link
Contributor

Buwujiu commented Aug 15, 2023

can we make our own nextflow version and use it until they release a new version?
We will fork the code, make the change, see if nextflow would approve.

@UmmulkiramR UmmulkiramR linked a pull request Oct 2, 2023 that will close this issue
@UmmulkiramR
Copy link
Contributor

UmmulkiramR commented Oct 2, 2023

Management in dev and QA now uses the forked version of the nextflow library that contains the bug fix.
PR created-> #216

Works for both prealnqc and dnaalnqc workflows as tested by @lindaxiang here -> icgc-argo/workflow-roadmap#388

@UmmulkiramR
Copy link
Contributor

UmmulkiramR commented Nov 16, 2023

PR created on the original nextflow repo -> nextflow-io/nextflow#4355
No revert on it yet.

@UmmulkiramR
Copy link
Contributor

In the meantime, have forked the Nextflow library into our icgc-argo repo and added the proposed fix in there to unblock workflows in RDPC. We need to plan a release of WF management to prod that uses this forked version of the Nextflow library

UmmulkiramR added a commit that referenced this issue Dec 14, 2023
* upgrade springboot 2.6.6 and misc dependencies (#200)

* cannot start runs on prod #204 --  added config params to enable rabbitmq consumers to reconnect when the connection breaks.

* Develop with loggers (#208)

* Added logger statements

* Added logger statements

* research and investigation

Co-authored-by: UmmulkiramR <[email protected]>

* removed some logging (#211)

Co-authored-by: UmmulkiramR <[email protected]>

* updated nextflow version for #196 (#212)

Co-authored-by: UmmulkiramR <[email protected]>

* Nextflow 22 10 7 plugin fix (#213)

* changed nextflow library version to use the forked copy that contains the plugins fix

* changed repository name

* add github token for maven packages

---------

Co-authored-by: UmmulkiramR <[email protected]>
Co-authored-by: Anders Richardsson <[email protected]>

* readme update (#216)

* updated readme with instructions on how to use the forked version of the nextflow library

* Update README.md

Co-authored-by: Anders Richardsson <[email protected]>

* Update README.md

Co-authored-by: Anders Richardsson <[email protected]>

* Update README.md

Co-authored-by: Anders Richardsson <[email protected]>

* Update README.md

Co-authored-by: Anders Richardsson <[email protected]>

* Update README.md

Co-authored-by: Anders Richardsson <[email protected]>

* Update README.md

Co-authored-by: Anders Richardsson <[email protected]>

* added review suggestions.

---------

Co-authored-by: UmmulkiramR <[email protected]>
Co-authored-by: Anders Richardsson <[email protected]>

* Jenkins migration (#217)

* Update Jenkinsfile

* docker file update for adding server.xml for maven

---------

Co-authored-by: Yelizar Alturmessov <[email protected]>
Co-authored-by: UmmulkiramR <[email protected]>

* test command edit (#218)

Co-authored-by: UmmulkiramR <[email protected]>

* update to 3.6.0
contains:
- nextflow version change with the forked library
- Jenkins file update for cumulus build pipelines.

---------

Co-authored-by: jaserud <[email protected]>
Co-authored-by: UmmulkiramR <[email protected]>
Co-authored-by: Anders Richardsson <[email protected]>
Co-authored-by: Anders Richardsson <[email protected]>
Co-authored-by: Yelizar Alturmessov <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
blocker This issue is blocking another major High Priority
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants