Merge pull request #57 from KatherLab/dev_upgrade_SL_2.1.0

Dev upgrade sl 2.1.0

.github/swam-learning-issue-reporting-template.md

@@ -0,0 +1,36 @@
+---
+name: Swam Learning Issue Reporting Template
+about: 'Content Suggestion for reporting issues '
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+# Issue description
+- issue description:
+- occurrence - consistent or rare: 
+- error messages: 
+- commands used for starting containers: 
+- docker logs [APLS, SPIRE, SN, SL, SWCI]: 
+
+# Swarm Learning Version:
+- Find the docker tag of the Swarm images ( $ docker images | grep hub.myenterpriselicense.hpe.com/hpe_eval/swarm-learning )
+
+# OS and ML Platform
+- details of host OS: 
+- details of ML platform used: 
+- details of Swarm learning Cluster (Number of machines, SL nodes, SN nodes): 
+
+# Quick Checklist: Respond [Yes/No]
+- APLS server web GUI shows available Licenses?
+- If Multiple systems are used, can each system access every other system?
+- Is Password-less SSH configuration setup for all the systems?
+- If GPU or other protected resources are used, does the account have sufficient privileges to access and use them?
+- Is the user id a member of the docker group?
+
+# Additional notes
+- Are you running documented example without any modification?
+- Add any additional information about use case or any notes which supports for issue investigation: 
+
+## NOTE: Create an archive with supporting artifacts and attach to issue, whenever applicable.

.gitignore

@@ -154,3 +154,8 @@ dmypy.json
 
 # test files
 **/test_*
+
+# vpn credentials
+assets/openvpn_configs/good_access/*.ovpn
+/assets/Odelia_credentials.txt
+/workspace/odelia-breast-mri/results/

README.md

@@ -2,7 +2,7 @@
 
 [![standard-readme compliant](https://img.shields.io/badge/readme%20style-standard-brightgreen.svg?style=flat-square)](https://github.com/RichardLitt/standard-readme)
 
-Swarm learning based on HPE platform, experiments performed based on HPE Swarm Learning version number 1.2.0
+Swarm learning based on HPE platform, experiments performed based on HPE Swarm Learning version number 2.1.0
 
 This repository contains:
 
@@ -65,6 +65,13 @@ This is the Swarm Learning framework:
  * Any experimental release of Ubuntu greater than LTS 20.04 MAY result in unsuccessful swop node running.
  * It also works on WSL2(Ubuntu 20.04.2 LTS) on Windows systems. WSL1 may have some issues with the docker service.
 
+### Upgrade the Swarm Learning Environment
+1. Run the following command to upgrade the Swarm Learning Environment from 1.x.x to 2.x.x
+```sh
+$ sh workspace/automate_scripts/server_setup/cleanup_old_sl.sh
+```
+Then proceed to 1. `Prerequisite` in [Setting up the Swarm Learning Environment](#setting-up-the-swarm-learning-environment)
+
 ### Setting up the user and repository
 1. Create a user named "swarm" and add it to the sudoers group.
 Login with user "swarm".
@@ -145,7 +152,6 @@ All the processes are automated, so you can just run the above command and wait
 If any problem occurs, please first try to figure out which step is going wrong, try to google for solutions and find solution in [Troubleshooting.md](Troubleshooting.md). Then contact the maintainer of the Swarm Learning Environment and document the error in the Troubleshooting.md file.
 
 ## Usage
-
 ### Data Preparation
 1. Make sure you have downloaded Duke data.
 
@@ -155,20 +161,20 @@ mkdir workspace/<workspace-name>/user/data-and-scratch/data/WP1
 mkdir workspace/<workspace-name>/user/data-and-scratch/data/WP1/{test,train_val}
 ```
 3. Search for your institution in the [Node list](#nodelist) and note the data series in the column "Data"
- 
-4. Copy the clinic table and slide table into WP1
+
+4. Prepare the clinical tables
 ```sh
-cp workspace/<workspace-name>/user/data-and-scratch/data/{clinical_table,slide_table}.csv workspace/<workspace-name>/user/data-and-scratch/data/WP1
+cp workspace/<workspace-name>/user/data-and-scratch/data/*.xlsx workspace/<workspace-name>/user/data-and-scratch/data/WP1
 ```
- 
-5. Copy the features from feature folder into `WP1/test` from 801 to 922
+
+5. Copy the nifty files from feature folder into `WP1/test` from 801 to 922
 ```sh
-cp workspace/<workspace-name>/user/data-and-scratch/data/features_odelia_sub_imagenet/Breast_MRI_{801..922}_{right,left}.h5 workspace/<workspace-name>/user/data-and-scratch/data/WP1/test
+cp -r workspace/<workspace-name>/user/data-and-scratch/data/odelia_dataset_only_sub/{801..922}_{right,left} workspace/<workspace-name>/user/data-and-scratch/data/WP1/test
 ```
 
-6. Copy the features from feature folder with the order you noted into `WP1/train_val` from xxx to yyy
+6. Copy the nifty files from feature folder with the order you noted into `WP1/train_val` from xxx to yyy
 ```sh
-cp workspace/<workspace-name>/user/data-and-scratch/data/features_odelia_sub_imagenet/Breast_MRI_{<first_number>..<second_number>}_{right,left}.h5 workspace/<workspace-name>/user/data-and-scratch/data/WP1/train_val
+cp -r workspace/<workspace-name>/user/data-and-scratch/data/odelia_dataset_only_sub/{<first_number>..<second_number>} workspace/<workspace-name>/user/data-and-scratch/data/WP1/train_val
 ```
 
 ### Running Swarm Learning Nodes

assets/openvpn_configs/good_access/README.md

@@ -0,0 +1,2 @@
+Please place your VPN credentials(.ovpn file) under this folder
+Please check in the .ovpn file if /etc/openvpn/credentials is after auth-user-pass