-
Notifications
You must be signed in to change notification settings - Fork 595
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
publishDir compatibility with Azure Blob Storage #4683
Labels
Comments
Don't think it makes sense to support http-based URLs when you are specifying an upload destination, maybe for downloading only |
Agreed! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Bug report
Expected behavior and actual behavior
Expected behavior is that
publishDir
directive should work with Azure links, using different formats.Actual behavior is that
publishDir
fails for:https://
az://<storage-acccount>.<bucket>
Steps to reproduce the problem
Working example:
Failing example1 - storage account in the path:
Failing example 2 - https path used:
Root cause of failures is:
FileHelper.groovy
paths get transformed into canonicalPath (for example into/<storage-acccount>.<bucket>
)Files.createDirectories(this.path)
fails with the given error messageEnvironment
Additional context
Reasoning for path with storage account name included support:
Azure bucket/container names are not unique, they are only unique in a storage account. So to be able to identify them correctly, in Seqera Platform the following path format is used
az://<storage-acccount>.<bucket>
. Because Nextflow has knowledge of the storage account name - it has to be set up in the config - this part could be easily removed from the path, fixing the issue.Reasoning for path with https support:
Azure docs about referencing blobs suggest using an URL like this:
https://<storage-acccount>.blob.core.windows.net/<bucket>
.The text was updated successfully, but these errors were encountered: