How to configure conda-store allowed channels?

[alimanfoo]

Hi there,

I'd like to be able to create environments using channels other than conda-forge (e.g., bioconda). With default configuration, using nebari 2022.10.1, only the conda-store channel is allowed. I believe this is the default but could be changed via the CondaStore.conda_allowed_channels configuration parameter.

What would be the best way to set this configuration parameter for my nebari deployment?

Thanks ❤️

[alimanfoo]

Just to add a little information, I see that a file gets generated at 07-kubernetes-services/modules/kubernetes/services/conda-store/config/conda_store_config.py and so it looks like potentially this could be modified.

However, I'm not clear if it is safe for me to edit this file myself after nebari has initially rendered it, or whether I should avoid editing these files.

[alimanfoo]

It looks like any change to the 07-kubernetes-services/modules/kubernetes/services/conda-store/config/conda_store_config.py is overwritten by nebari's rendering.

However, there is a workaround where you first call nebari render, then edit 07-kubernetes-services/modules/kubernetes/services/conda-store/config/conda_store_config.py, then call nebary deploy --disable-render. I can confirm that editing the file to add this code:

c.CondaStore.conda_allowed_channels = []

...does then allow any channels to be used in environments.

Here is a a git patch for that change:

diff --git a/stages/07-kubernetes-services/modules/kubernetes/services/conda-store/config/conda_store_config.py b/stages/07-kubernetes-services/modules/kubernetes/services/conda-store/config/conda_store_config.py
index dfee8f5..dc3b815 100644
--- a/stages/07-kubernetes-services/modules/kubernetes/services/conda-store/config/conda_store_config.py
+++ b/stages/07-kubernetes-services/modules/kubernetes/services/conda-store/config/conda_store_config.py
@@ -44,6 +44,9 @@ c.S3Storage.bucket_name = "conda-store"
 c.CondaStore.default_namespace = "global"
 c.CondaStore.filesystem_namespace = config["default-namespace"]
 
+# customisation: allow all channels
+c.CondaStore.conda_allowed_channels = []
+
 # ==================================
 #        server settings
 # ==================================

[iameskild]

Hi @alimanfoo, thank you for starting this discussion! This is definitely an item that needs to be configurable by the end-user. This is being tracked, albeit without much movement, in this issue: conda-incubator/conda-store#357

@costrouc do you think adding a section to the nebari-config.yaml which can be used to override or modify the conda_store_config.py is the best way to handle this (and other conda-store changes)?

[alimanfoo]

Thanks @iameskild.

FWIW I would suggest conda-store change the default to allow all channels. I expect this is what most nebari users will expect, it's certainly what we want. This would mean most nebari users wouldn't have to worry about making any conda-store configuration changes.

Then having some mechanism for overriding this (and any other conda-store configuration parameters) when deploying nebari would make sense for the minority of situations where the conda-store defaults need to be overridden.

[dharhas]

definitely the default should be allow all channels.

[iameskild]

@alimanfoo @dharhas I agree that we should set the default to allow "all channels". We might still want to provide a mechanism for Nebari administrators to limit the channels their users can use.

[dharhas]

There are also a few other conda.atore config options that need to be exposed. So it probably needs its own section.

[iameskild]

@alimanfoo it appears that making changes to the conda_store_config.py is possible from within the nebari-config.yaml:

conda_store:
  extra-settings:
     # general case
     Class:
        key: value
     # specific example
     CondaStore:
       conda_allowed_channels = []

We will need to update our documentation to reflect this capability. Until then, let us know how this goes :)

cc @costrouc

[alimanfoo]

Ah amazing, let me give that a try!

[alimanfoo]

Hi @iameskild, adding this config fails validation, it looks like the schema says it should be "extra_settings" with an underscore rather than "extra-settings" with a dash.

Also I just tested a deployment using:

conda_store:
  extra_settings:
     CondaStore:
       conda_allowed_channels = []

And the deployment succeeded but the conda-store service is unavailable. That may be bad luck but just thought I'd flag things maybe not behaving as expected here.

[dharhas]

@alimanfoo can you file an issue on the above so we can track it as a bug.

[alimanfoo]

Done - #1594 - hope that's useful.