docs/instructions.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Example of the commands need it to execute the rules\n",
    "\n",
    "Not completly necessary to follow this instructions, in some cases would be better to do some samples separated of each other."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Clone the repository"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "git clone https://github.com/Juankkar/TFM.git"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Install all Conda enviroments, choose to use conda or mamba:\n",
    "\n",
    "* Command ```mamba env install -f /path/to/env.yml```\n",
    "\n",
    "    * [Gereference_tools](../code/environments/Greference_tools.yml)\n",
    "    * [vep](../code/environments/vep.yml)\n",
    "    * [biostatistcsR](../code/environments/biostatisticsR.yml)\n",
    "    * [igv_viz](../code/environments/igv_viz.yml)\n",
    "\n",
    "* Activate the first environment ```conda activate Greference_tools```."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### To execute the snakemake rules remember to stay in the TFM directory\n",
    "\n",
    "* ```cd /your/path/to/TFM```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## First time running the workflow (ie. chr3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is how your config file should look like:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* config.yaml:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "#########################\n",
    "##  SNAKEFILE SAMPLES  ##\n",
    "#########################\n",
    "\n",
    "samples:\n",
    "  ERR696683_chr3: data/original_bam/filtering/ERR696683_chr3.bam \n",
    "  ERR753368_chr3: data/original_bam/filtering/ERR753368_chr3.bam \n",
    "  ERR753369_chr3: data/original_bam/filtering/ERR753369_chr3.bam \n",
    "  ERR753370_chr3: data/original_bam/filtering/ERR753370_chr3.bam \n",
    "  ERR753371_chr3: data/original_bam/filtering/ERR753371_chr3.bam \n",
    "  ERR753372_chr3: data/original_bam/filtering/ERR753372_chr3.bam  \n",
    "  ERR753373_chr3: data/original_bam/filtering/ERR753373_chr3.bam  \n",
    "  ERR753374_chr3: data/original_bam/filtering/ERR753374_chr3.bam \n",
    "  ERR753375_chr3: data/original_bam/filtering/ERR753375_chr3.bam \n",
    "  ERR753376_chr3: data/original_bam/filtering/ERR753376_chr3.bam \n",
    "  ERR753377_chr3: data/original_bam/filtering/ERR753377_chr3.bam \n",
    "  ERR753378_chr3: data/original_bam/filtering/ERR753378_chr3.bam  \n",
    "\n",
    "########################\n",
    "##  SNAKEFILE PARAMS  ##\n",
    "########################\n",
    "\n",
    "samples_names:\n",
    "  ERR696683_chr3\n",
    "  ERR753368_chr3\n",
    "  ERR753369_chr3\n",
    "  ERR753370_chr3\n",
    "  ERR753371_chr3\n",
    "  ERR753372_chr3\n",
    "  ERR753373_chr3\n",
    "  ERR753374_chr3\n",
    "  ERR753375_chr3\n",
    "  ERR753376_chr3\n",
    "  ERR753377_chr3\n",
    "  ERR753378_chr3\n",
    "\n",
    "\n",
    "## Choose your chromosome\n",
    "chromosome: chr3 \n",
    "# Chromosome 3, 5, 7, 12, 17 ENSEMBL:\n",
    "# https://ftp.ensembl.org/pub/release-109/fasta/homo_sapiens/dna/Homo_sapiens.GRCh38.dna.chromosome.3.fa.gz \n",
    "# https://ftp.ensembl.org/pub/release-109/fasta/homo_sapiens/dna/Homo_sapiens.GRCh38.dna.chromosome.5.fa.gz\n",
    "# https://ftp.ensembl.org/pub/release-109/fasta/homo_sapiens/dna/Homo_sapiens.GRCh38.dna.chromosome.7.fa.gz\n",
    "# https://ftp.ensembl.org/pub/release-109/fasta/homo_sapiens/dna/Homo_sapiens.GRCh38.dna.chromosome.12.fa.gz\n",
    "# https://ftp.ensembl.org/pub/release-109/fasta/homo_sapiens/dna/Homo_sapiens.GRCh38.dna.chromosome.17.fa.gz\n",
    "url_reference_genome: https://ftp.ensembl.org/pub/release-109/fasta/homo_sapiens/dna/Homo_sapiens.GRCh38.dna.chromosome.3.fa.gz\n",
    "\n",
    "## Fixing the fastq files, FASTP parameters:\n",
    "fastp_cuttail: 30\n",
    "fastp_cutfront: 30\n",
    "fastp_cutmeanq: 30\n",
    "fastp_length: 75\n",
    "\n",
    "## Extracting variants\n",
    "ref_genome_name_file: genome.fa\n",
    "# Min reads that has to be mapping to considerate a variant\n",
    "min_reads_variant: 5\n",
    "\n",
    "## Doing VEP for variant annotation\n",
    "vep_species: homo_sapiens\n",
    "vep_assembly: GRCh38\n",
    "\n",
    "## Gene That you want to filter in the VCF using R\n",
    "## PIK3CA (chromosome 3); APC (chromosome 5); BRAF (chromosome 7); KRAS (chromosome 12) ;TP53 (chromosome 17)\n",
    "gene_to_filterR: PIK3CA   \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Copy each command in the CLI, one by one\n",
    "\n",
    "On the other hand, you could use the notebook [automate_snakemake](../auto_snakemake.ipynb) for a more automate version. It is true that it is not perfect, and if there are errors (problems downloading the data from the servers for example), could be a problem detecting them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "# 1 rule download data\n",
    "snakemake --cores 1 download_data\n",
    "# 2 rule pre_processing\n",
    "snakemake --cores 1 data/original_bam/filtering/ERR696683_chr3_sorted.bam\n",
    "# 3 rule reference_genome\n",
    "snakemake --cores 1 data/reference/genome.fa\n",
    "# 4 rule fastqc\n",
    "snakemake --cores 1 results/fastqc_result/ERR696683_chr3_1_fastqc.html results/fastqc_result/ERR7533{68..78}_chr3_1_fastqc.html  \n",
    "# 5 rule fastp\n",
    "snakemake --cores 1 data/processed/ERR696683_chr3_1_fastp.fastq.gz data/processed/ERR7533{68..78}_chr3_1_fastp.fastq.gz  \n",
    "# 6 rule fastqc_trimmed\n",
    "snakemake --cores 1 results/fastqc_result/trimmed/ERR696683_chr3_1_fastp_fastqc.html results/fastqc_result/trimmed/ERR7533{68..78}_chr3_1_fastp_fastqc.html  \n",
    "# 7 rule bwa_mapping\n",
    "snakemake --cores 1 results/mapped_reads/ERR696683_chr3_sorted.sam results/mapped_reads/ERR7533{68..78}_chr3_sorted.sam  \n",
    "# 8 rule sam_to_bam\n",
    "snakemake --cores 1 results/mapped_reads/bam_files/ERR696683_chr3_sorted.bam results/mapped_reads/bam_files/ERR7533{68..78}_chr3_sorted.bam  \n",
    "# 9 rule delete_duplicates\n",
    "snakemake --cores 1 results/mapped_reads/bam_files/ERR696683_chr3_dedup.bam results/mapped_reads/bam_files/ERR7533{68..78}_chr3_dedup.bam  \n",
    "# 10 rule extracting_variants\n",
    "snakemake --cores 1 results/variants/ERR696683_chr3.vcf results/variants/ERR7533{68..78}_chr3.vcf \n",
    "# 11 rule vep_install_db: GRCh38 = 109 will ask you if you want new version, press n\n",
    "snakemake --cores 1 vep_install_db --use-conda\n",
    "# 12 rule vep_cli\n",
    "snakemake --cores 1 results/variants/vep/ERR696683_chr3.txt results/variants/vep/ERR7533{68..78}_chr3.txt --use-conda\n",
    "# 13 rule parsing_dataR\n",
    "snakemake --cores 1 parsing_dataR --use-conda"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Second Time (ie. chrx)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Change the values of the config.yaml.\n",
    "    * Chromosmes \n",
    "    * Reference genome \n",
    "    * And the gene\n",
    "\n",
    "* Example of the config.file with x where you have to change one of this params"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "#########################\n",
    "##  SNAKEFILE SAMPLES  ##\n",
    "#########################\n",
    "\n",
    "samples:\n",
    "  ERR696683_chrx: data/original_bam/filtering/ERR696683_chrx.bam \n",
    "  ERR753368_chrx: data/original_bam/filtering/ERR753368_chrx.bam \n",
    "  ERR753369_chrx: data/original_bam/filtering/ERR753369_chrx.bam \n",
    "  ERR753370_chrx: data/original_bam/filtering/ERR753370_chrx.bam \n",
    "  ERR753371_chrx: data/original_bam/filtering/ERR753371_chrx.bam \n",
    "  ERR753372_chrx: data/original_bam/filtering/ERR753372_chrx.bam  \n",
    "  ERR753373_chrx: data/original_bam/filtering/ERR753373_chrx.bam  \n",
    "  ERR753374_chrx: data/original_bam/filtering/ERR753374_chrx.bam \n",
    "  ERR753375_chrx: data/original_bam/filtering/ERR753375_chrx.bam \n",
    "  ERR753376_chrx: data/original_bam/filtering/ERR753376_chrx.bam \n",
    "  ERR753377_chrx: data/original_bam/filtering/ERR753377_chrx.bam \n",
    "  ERR753378_chrx: data/original_bam/filtering/ERR753378_chrx.bam  \n",
    "\n",
    "########################\n",
    "##  SNAKEFILE PARAMS  ##\n",
    "########################\n",
    "\n",
    "samples_names:\n",
    "  ERR696683_chrx \n",
    "  ERR753368_chrx \n",
    "  ERR753369_chrx\n",
    "  ERR753370_chrx\n",
    "  ERR753371_chrx\n",
    "  ERR753372_chrx\n",
    "  ERR753373_chrx\n",
    "  ERR753374_chrx\n",
    "  ERR753375_chrx\n",
    "  ERR753376_chrx\n",
    "  ERR753377_chrx\n",
    "  ERR753378_chrx\n",
    "\n",
    "\n",
    "## Choose your chromosome\n",
    "chromosome: chrx \n",
    "# Chromosome 3, 5, 7, 12, 17 ENSEMBL:\n",
    "# https://ftp.ensembl.org/pub/release-109/fasta/homo_sapiens/dna/Homo_sapiens.GRCh38.dna.chromosome.3.fa.gz \n",
    "# https://ftp.ensembl.org/pub/release-109/fasta/homo_sapiens/dna/Homo_sapiens.GRCh38.dna.chromosome.5.fa.gz\n",
    "# https://ftp.ensembl.org/pub/release-109/fasta/homo_sapiens/dna/Homo_sapiens.GRCh38.dna.chromosome.7.fa.gz\n",
    "# https://ftp.ensembl.org/pub/release-109/fasta/homo_sapiens/dna/Homo_sapiens.GRCh38.dna.chromosome.12.fa.gz\n",
    "# https://ftp.ensembl.org/pub/release-109/fasta/homo_sapiens/dna/Homo_sapiens.GRCh38.dna.chromosome.17.fa.gz\n",
    "url_reference_genome: https://ftp.ensembl.org/pub/release-109/fasta/homo_sapiens/dna/Homo_sapiens.GRCh38.dna.chromosome.x.fa.gz\n",
    "\n",
    "## Fixing the fastq files, FASTP parameters:\n",
    "fastp_cuttail: 30\n",
    "fastp_cutfront: 30\n",
    "fastp_cutmeanq: 30\n",
    "fastp_length: 75\n",
    "\n",
    "## Extracting variants\n",
    "ref_genome_name_file: genome.fa\n",
    "# Min reads that has to be mapping to considerate a variant\n",
    "min_reads_variant: 5\n",
    "\n",
    "## Doing VEP for variant annotation\n",
    "vep_species: homo_sapiens\n",
    "vep_assembly: GRCh38\n",
    "\n",
    "## Gene That you want to filter in the VCF using R\n",
    "## PIK3CA (chromosome 3); APC (chromosome 5); BRAF (chromosome 7); KRAS (chromosome 12) ;TP53 (chromosome 17)\n",
    "gene_to_filterR: x   \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Execute this commands in the CLI for the rest of chromosomes "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "# 2 rule pre_processing\n",
    "snakemake --cores 1 data/original_bam/filtering/ERR696683_chrx_sorted.bam \n",
    "# 3 rule reference_genome\n",
    "snakemake --cores 1 data/reference/genome.fa --force ## if you haven't delete the contente of data/reference \n",
    "# 4 rule fastqc\n",
    "snakemake --cores 1 results/fastqc_result/ERR696683_chrx_1_fastqc.html results/fastqc_result/ERR7533{68..78}_chrx_1_fastqc.html  \n",
    "# 5 rule fastp\n",
    "snakemake --cores 1 data/processed/ERR696683_chrx_1_fastp.fastq.gz data/processed/ERR7533{68..78}_chrx_1_fastp.fastq.gz  \n",
    "# 6 rule fastqc_trimmed\n",
    "snakemake --cores 1 results/fastqc_result/trimmed/ERR696683_chrx_1_fastp_fastqc.html results/fastqc_result/trimmed/ERR7533{68..78}_chrx_1_fastp_fastqc.html  \n",
    "# 7 rule bwa_mapping\n",
    "snakemake --cores 1 results/mapped_reads/ERR696683_chrx_sorted.sam results/mapped_reads/ERR7533{68..78}_chrx_sorted.sam  \n",
    "# 8 rule sam_to_bam\n",
    "snakemake --cores 1 results/mapped_reads/bam_files/ERR696683_chrx_sorted.bam results/mapped_reads/bam_files/ERR7533{68..78}_chrx_sorted.bam  \n",
    "# 9 rule delete_duplicates\n",
    "snakemake --cores 1 results/mapped_reads/bam_files/ERR696683_chrx_dedup.bam results/mapped_reads/bam_files/ERR7533{68..78}_chrx_dedup.bam  \n",
    "# 10 rule extracting_variants\n",
    "snakemake --cores 1 results/variants/ERR696683_chrx.vcf results/variants/ERR7533{68..78}_chrx.vcf \n",
    "# 12 rule vep_cli\n",
    "snakemake --cores 1 results/variants/vep/ERR696683_chrx.txt results/variants/vep/ERR7533{68..78}_chrx.txt --use-conda\n",
    "# 13 rule parsing_dataR\n",
    "snakemake --cores 1 parsing_dataR --use-conda"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## After all gene tables are created\n",
    "\n",
    "* In this case you can execute the last rule, for plotting your data in ggplot2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "snakemake --cores 1 R_plotting --use-conda"
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}