chore: Merge pull request #3 from brickmanlab/docs/reorg

docs: more stuff

docs/0_setup.md

@@ -0,0 +1,52 @@
+# First time on danserver
+
+For starting on the server make sure to read:
+
+- [DanServer manual](https://sgn102.pages.ku.dk/a-not-long-tour-of-dangpu/)
+- [Genomics Platform wiki](https://sundgenomics.github.io)
+- **platforms**: [JupyterHub](http://dangpu01fl:8989) and [RStudio](http://dangpu01fl:8787)
+
+## First time on server checklist
+
+1. Login to danhead: `ssh [email protected]`
+2. Run `nano ~/.bash_profile`
+
+```text
+if [ -f ~/.bashrc ]; then
+ . ~/.bashrc
+fi
+```
+
+3. Run `nano ~/.bashrc`
+
+```text
+# .bashrc
+
+# Source global definitions
+if [ -f /etc/bashrc ]; then
+ . /etc/bashrc
+fi
+
+# User specific environment
+if ! [[ "$PATH" =~ "$HOME/.local/bin:$HOME/bin:" ]]
+then
+    PATH="$HOME/.local/bin:$HOME/bin:$PATH"
+fi
+export PATH
+
+# Uncomment the following line if you don't like systemctl's auto-paging feature:
+# export SYSTEMD_PAGER=
+
+# User specific aliases and functions
+### Source DanGPU definitions
+if [ -f /maps/projects/dan1/apps/etc/bashrc ]; then
+ . /maps/projects/dan1/apps/etc/bashrc
+fi
+
+### Source Brickman definitions
+if [ -f /maps/projects/dan1/data/Brickman/config/brickman.bashrc ]; then
+ . /maps/projects/dan1/data/Brickman/config/brickman.bashrc
+fi
+```
+
+4. Logout and login again, you should have see now `Brickman` folder

docs/starting-assay-project.md → docs/2_starting-assay-project.md

@@ -7,25 +7,13 @@ us with tracking of all sequencing done in our lab.
 
 ## Assay
 
-```bash
-module load miniconda/latest
-source activate brickman
-
-cd /home/$USER/Brickman/assays
-cruft create https://github.com/brickmanlab/ngs-template --directory="assay"
-```
-
-Next, please copy all `fastq` files from CPR share to folder `data/raw/fastq`.
-All processed data from a pipeline should be copied to `data/processed` folder.
+When you sequence an experiment, we create an **Assay** out of it, so we can use
+it in a project afterwards.
 
-```bash
-rsync -avzh --progress --chmod=2775 ~/ucph/ndir/SUN-CPR-genomics_data/... <ASSAY_ID>/raw/fastq
-```
-
-After copying all the files, please run the last command.
+Login to danhead and run command:
 
 ```bash
-chmod -R 775 /maps/projects/dan1/data/Brickman/assays/<ASSAY_ID>
+create_assay
 ```
 
 ## Project
@@ -37,11 +25,7 @@ project and make your analysis easier.
 Please use the following naming convention: `surname-<YOUR_CODENAME>`
 
 ```bash
-module load miniconda/latest
-source activate brickman
-
-cd /home/$USER/Brickman/projects
-cruft create https://github.com/brickmanlab/ngs-template --directory="project"
+create_project
 ```
 
 Link required assays to your project.
@@ -50,7 +34,7 @@ Link required assays to your project.
 ln -s /maps/projects/dan1/data/Brickman/assays/<ASSAY_ID> /maps/projects/dan1/data/Brickman/projects/<PROJECT_ID>/data/assays/
 ```
 
-Link external data if needed
+Link **external data** if needed
 
 ```bash
 ln -s /maps/projects/dan1/data/Brickman/shared /maps/projects/dan1/data/Brickman/projects/<PROJECT_ID>/data/external/

docs/3_pipelines.md

@@ -0,0 +1,31 @@
+# Running pipelines
+
+By default, we run [nf-core](https://nf-co.re) pipelines. To run a pipeline, read
+the official [documentation with an example](https://nf-co.re/configs/ku_sund_danhead).
+
+## Monitoring runs with Nextflow Tower
+
+This is a guide on how to use [Nextflow Tower](https://help.tower.nf/23.1/) to
+monitor nf-core pipeline runs.
+
+We have created an API token for our GitHub account (brickmanlab) and restricted
+it to run only pipelines, nothing else. The `TOWER_WORKSPACE_ID` and `TOWER_ACCESS_TOKEN`
+are stored in `Brickman/config/brickman.bashrc`.
+
+To do more advance stuff, you have to create your own [personal access token](https://tower.nf/tokens).
+
+### Tower CLI installation
+
+The tower cli[^1] is required to be installed only once to connect the server as
+a computing resource. Afterward, it's not required any more[^2].
+
+``` bash
+# Download the latest version of Tower CLI:
+wget https://github.com/seqeralabs/tower-cli/releases/download/v0.7.3/tw-0.7.3-linux-x86_64
+
+# Make the file executable and move to directory accessible by $PATH variable:
+mkdir ~/.local/bin && mv tw-* tw && chmod +x ~/.local/bin/tw
+```
+
+[^1]: [Tower CLI configuration](https://github.com/seqeralabs/tower-cli/#2-configuration)
+[^2]: [Tower Agent](https://help.tower.nf/22.3/agent/)

docs/4_conda.md

@@ -0,0 +1,73 @@
+# Conda & modules
+
+## Conda
+
+If you work with `conda` you can use `mamba` instead, which is **faster** tool
+to install packages.
+
+We created shared `conda` environments to simplify your life.
+
+- To list all available envs: `conda env list`
+- To activate env: `source activate brickman`
+
+### Creating own shared environment
+
+Here is an example how we created shared environment called `brickman`.
+
+```bash
+module load miniconda/latest
+
+conda create --prefix /maps/projects/dan1/data/Brickman/conda/envs/brickman python=3.10
+source activate brickman
+pip install cruft cookiecutter
+
+chmod -R 755 /maps/projects/dan1/data/Brickman/conda/envs/brickman
+```
+
+To install shared `conda` environment for the lab, follow the steps below.
+
+1. Create a configuration file which contains the list of packages you want
+  to use. Use prefix `brickman-<NGS>.yml`
+2. Populate the configuration file
+3. Create the conda environment
+
+```bash
+mamba env create -p /projects/dan1/data/Brickman/conda/envs/brickman-<NGS>.yml -f brickman-<NGS>.yml
+```
+
+### Example conda environment
+
+Configuration for `brickman-chipseq` environment.
+
+```yaml
+name: brickman-chipseq
+channels:
+  - conda-forge
+  - bioconda
+  - anaconda
+  - defaults
+dependencies:
+  - bioconda::bedtools==2.31.0
+  - bioconda::deeptools==2.31.0
+  - bioconda::homer==4.11
+  - bioconda::intervene==0.6.4
+  - bioconda::macs2==2.2.9.1
+  - bioconda::pygenometracks==3.8
+  - bioconda::seacr==1.3
+  - bioconda::samtools==1.17
+prefix: /projects/dan1/data/Brickman/conda/envs/brickman-chipseq
+```
+
+To install the environment, run
+
+```bash
+mamba env create -p /projects/dan1/data/Brickman/conda/envs/brickman-chipseq -f brickman-chipseq.yml
+```
+
+## Modules
+
+```bash
+module avail
+
+module load miniconda/latest
+```

docs/dangpu/R_vscode.md → docs/5_vscode.md

@@ -3,22 +3,22 @@
 This setup guides you through setting up `R` in [VSCode](https://code.visualstudio.com)
 so you can use it on `dancmpn01fl` and `dancmpn02fl` computing nodes.
 
-> [!IMPORTANT]
-> The original RStudio server is using 4.0.5 version. If you want to stick
-> this version, make sure to specify it when loading modules.
+!!! info
 
-and
+    The original RStudio server is using 4.0.5 version. If you want to stick
+    this version, make sure to specify it when loading modules.
 
-> [!NOTE]
-> Why do you need this?
-> Because RStudio server sucks when you don't have a license and our place, so
-> alternative it is. Also, VSCode has a bunch of plugins.
+!!! question "Why do you need this?"
+
+    Because RStudio server sucks when you don't have a license and our place, so
+    alternative it is. Also, VSCode has a bunch of plugins.
 
 ## Setting up Remote Tunnels
 
-> [!CAUTION]
-> In this example we use version **R/4.3.1**. If you want to use a different one
-> change the R version!
+!!! warning
+
+    In this example we use version **R/4.3.1**. If you want to use a different one
+    change the R version!
 
 ### Requirements
 
@@ -32,15 +32,15 @@ and
 3. Start a new job: `srun -c 2 --mem=30gb --time=0-4:00:00 --pty bash`
 4. Load modules: `module load vscode_cli gcc/11.2.0 R/4.3.1 miniconda/latest`
 5. Run command: `code tunnel`
-   1. Choose `Microsoft account` when asked how you would like to log in to VScode
-   2. Open the link from terminal and paste the **CODE**
-   3. Login with your KU credentials
-   4. Go back to terminal and wait
-   5. Hit <kbd>ENTER</kbd> on questions about tunnel name
-   6. If everything went well you should see some random messages about port forwarding
+    1. Choose `Microsoft account` when asked how you would like to log in to VScode
+    2. Open the link from terminal and paste the **CODE**
+    3. Login with your KU credentials
+    4. Go back to terminal and wait
+    5. Hit <kbd>ENTER</kbd> on questions about tunnel name
+    6. If everything went well you should see some random messages about port forwarding
 6. Go to your VSCode and on the left panel search for `Remote Explorer`
 7. Click `Sign in to the tunnels registered with Microsoft`
-   1. You should see `dancmpn01flunicphdom` or `dancmpn02flunicphdom`
+    1. You should see `dancmpn01flunicphdom` or `dancmpn02flunicphdom`
 8. Move cursor on it and click the `->`
 
 To use `R`, install additional packages by clicking `Extensions` in the left panel.
@@ -51,13 +51,13 @@ Search for packages:
 
 1. Next, top panel lick `View` -> `Terminal` -> Write `R` and hit <kbd>ENTER</kbd>
 2. `install.packages("languageserver")`
-   1. If it asks to install to stuff to other directory hit <kbd>ENTER</kbd>
-   2. When asking about mirror type `30` (Denmark servers to download packages)
+    1. If it asks to install to stuff to other directory type `YES` then hit <kbd>ENTER</kbd>
+    2. When asking about mirror type `30` (Denmark servers to download packages)
 3. `install.packages("httpgd")`
-4. `q()` to get our
-5. Top left pannel: `Code` -> `Settings` -> `Settings`
-   1. Type in search `r.plot.useHttpgd`
-   2. Tick the box if it is not checked
+4. `q()` to get out
+5. Top left panel: `Code` -> `Settings` -> `Settings`
+    1. Type in search `r.plot.useHttpgd`
+    2. Tick the box if it is not checked
 
 If everything went well, you should be able to do this. If not, you know what
 to do.
@@ -89,7 +89,11 @@ VSCode can be installed as a server `code-server`, however it is not possible
 to listen on the port when on computing node. This works only in the case of
 `dangpu01fl`.
 
-Error when trying to do reverse ssh: `error listen EADDRINUSE: address already in use 127.0.0.1:8080`
+Error when trying to do reverse ssh:
+
+```console
+error listen EADDRINUSE: address already in use 127.0.0.1:8080
+```
 
 VSCode `code-server` is an alternative to `code tunnel` that consists of running
 [code-server](https://coder.com/docs/code-server/install) on a compute node and