Version 1 - 116 improve output data formatting (#118)

* update to V1. see #118 for details.

.github/workflows/build-and-test.yml

@@ -0,0 +1,55 @@
+name: Build and test
+
+on: [pull_request]
+
+jobs:
+ Windows_pip:
+ runs-on: windows-latest
+ timeout-minutes: 60
+
+ steps:
+ - name: Check out code
+ uses: actions/checkout@v2
+
+ - name: Install Python
+ uses: actions/setup-python@v4
+ with:
+ python-version: '3.10'
+
+ - name: Install pyopia
+ run: pip install ./
+
+ - name: Test
+ run: python -m pytest -v
+ Ubuntu_poetry:
+ runs-on: ubuntu-latest
+ timeout-minutes: 60
+
+ steps:
+ - uses: actions/checkout@v3
+ - name: Install Python
+ uses: actions/setup-python@v4
+ with:
+ python-version: '3.10' 
+ - name: Install poetry
+ uses: abatilo/actions-poetry@v2
+ - name: Install dependencies
+ run: poetry install
+ - name: Run the automated tests
+ run: poetry run pytest -v
+ MacOS_poetry:
+ runs-on: macos-latest
+ timeout-minutes: 60
+
+ steps:
+ - uses: actions/checkout@v3
+ - name: Install Python
+ uses: actions/setup-python@v4
+ with:
+ python-version: '3.10' 
+ - name: Install poetry
+ uses: abatilo/actions-poetry@v2
+ - name: Install dependencies
+ run: poetry install
+ - name: Run the automated tests
+ run: poetry run pytest -v

.gitignore

@@ -259,3 +259,7 @@ test-report/*
 docs/.DS_Store
 .DS_Store
 docs/notebooks/header.tfl.txt
+docs/jupyter_execute/
+*.nc
+
+poetry.lock

README.md

@@ -81,37 +81,36 @@ cd pyopia-main
 
 For the next steps, you need to be located in the directory that contains the file 'environment.yml'.
 
-2. Create a virtual environment using the environment.yml (will create an environment called pyopia):
+1. (optional, but recommended) Create a virtual environment using the environment.yml. This will create an environment called pyopia, but with no dependencies installed. Dependencies are managed by poetry (in step 2):
 
 ```bash
 conda env create -f environment.yml
 ```
-
-3. To update, we recommend a forced re-install:
+and activate the environment:
 
 ```bash
-conda env create -f environment.yml --force
+conda activate pyopia
 ```
 
-(but you could also try this, which might be quicker but a less reliable form of updating):
+2. Install dependencies using poetry:
 
 ```bash
-conda env update --file environment.yml --prune
+poetry install
 ```
 
-4. To activate:
+3. (optional) Run local tests:
 
 ```bash
-conda activate pyopia
+poetry run pytest
 ```
 
-5. Test that it works with:
+## Build docs locally
 
-```bash
-python setup.py develop
 ```
+sphinx-apidoc -f -o docs/source docs/build --separate
 
-Note that `pip install` or `python setup.py install` will probably cause you problems if you want to develop the code, and should only be used for deployment purposes.
+sphinx-build -b html ./docs/ ./docs/build
+```
 
 ----
 # License

docs/_toc.yml

@@ -8,7 +8,10 @@ parts:
 - caption: Examples
  chapters:
  - file: examples
+ - file: notebooks/toml_config.ipynb
+ - file: notebooks/cli
  - file: notebooks/stats
+ - file: notebooks/STATSnc
 - caption: Code docs
  chapters:
  - file: api

docs/api.rst

@@ -11,6 +11,7 @@ API documentation
 
 **Content**
 
+* `CLI`_
 * `Pipeline`_
 * `Background`_
 * `Process`_
@@ -33,6 +34,11 @@ API documentation
 
 ----
 
+CLI
+--------------
+.. automodule:: pyopia.cli
+ :members:
+
 Pipeline
 --------------
 .. automodule:: pyopia.pipeline

docs/examples.md

@@ -3,4 +3,4 @@ PyOPIA by example
 
 More will come here soon. In the meantime, have a look at the [notebooks](https://github.com/SINTEF/pyopia/tree/main/notebooks)
 
-Some good examples of how to use {class}`pyopia.pipeline.Pipeline` for silcam can be found [here](https://github.com/SINTEF/pyopia/blob/main/notebooks/pipeline.ipynb) and holographic analysis [here](https://github.com/SINTEF/pyopia/blob/main/notebooks/pipeline.ipynb).
+Some good examples of how to use {class}`pyopia.pipeline.Pipeline` for silcam can be found for SilCam [here](https://github.com/SINTEF/pyopia/blob/main/notebooks/single-image-stats.ipynb) and holographic analysis [here](https://github.com/SINTEF/pyopia/blob/main/notebooks/pipeline-holo.ipynb).

docs/intro.md

@@ -1,24 +1,24 @@
 Welcome!
 ==================================
 
-This is documentation for PyOpia: Python Ocean Particle Image Analysis, hosted by [SINTEF Ocean](https://www.sintef.no/en/ocean/), in collaboration with University of Plymouth, NTNU, The MBA, and National Oceanography Center (Southampton). We hope to encourage wider collaboration on analysis tools for ocean particles and welcome new contributors!
+This is documentation for PyOPIA: Python Ocean Particle Image Analysis, hosted by [SINTEF Ocean](https://www.sintef.no/en/ocean/), in collaboration with University of Plymouth, NTNU, The MBA, George Washington University, and National Oceanography Center (Southampton). We hope to encourage wider collaboration on analysis tools for ocean particles and welcome new contributors!
 
-PyOpia started in Feb. 2022 as a 'spin-off' from elements of [PySilCam](https://github.com/SINTEF/PySilCam/wiki) that could be relevant for other hardware and image types, including holography.
+PyOPIA started in Feb. 2022 as a 'spin-off' from elements of [PySilCam](https://github.com/SINTEF/PySilCam/wiki) that could be relevant for other hardware and image types, including holography.
 
-The code repository for PyOpia can be found [here](https://github.com/SINTEF/PyOpia/).
+The code repository for PyOPIA can be found [here](https://github.com/SINTEF/pyopia/).
 
 Pipelines
 ==================================
-PyOpia aims to provide a pipeline-based workflow as a standard for analysis of particle images in the ocean.
+PyOPIA aims to provide a pipeline-based workflow as a standard for analysis of particle images in the ocean.
 
 This pipeline should be consistent across different instruments (hardware), and therefore has flexibility to adapt analysis steps to meet instrument-specific processing needs (i.e. holographic reconstruction), while maintaining a traceable workflow that is attached as metadata to a standard output file (HDF5) that helps users follow FAIR data principles.
 
-See {class}`pyopia.pipeline.Pipeline` for more details and examples of how to process images with PyOpia.
+See {class}`pyopia.pipeline.Pipeline` for more details and examples of how to process images with PyOPIA.
 
 A function-based toolbox
 ==================================
 
-PyOpia tools are organised into the following modules:
+PyOPIA tools are organised into the following modules:
 
 * background correction {mod}`pyopia.background`.
 * processing {mod}`pyopia.process`.
@@ -27,7 +27,7 @@ PyOpia tools are organised into the following modules:
 * metadata & datafile formatting {mod}`pyopia.io`.
 
 You can combine these tools for exploring different analysis approaches (i.e. in notebooks).
-We hope this can help more exploratory development and contributions to the PyOpia code.
+We hope this can help more exploratory development and contributions to the PyOPIA code.
 
 If you are analysing data for publication, we recommend using the {class}`pyopia.pipeline.Pipeline` standard so that your analysis steps are documented and the output format is more easily shareable.
 
@@ -36,10 +36,19 @@ Full documentation for the code is [here](api)
 Installing
 ==================================
 
-Users are expected to be familiar with Python, and have [Python](https://github.com/conda-forge/miniforge/#download), [pip](https://pypi.org/project/pip/) and [git](https://github.com/git-guides/install-git) installed. You can then install Pyopia like this:
+Users are expected to be familiar with Python, and have [Python](https://github.com/conda-forge/miniforge/#download), [pip](https://pypi.org/project/pip/) and [git](https://github.com/git-guides/install-git) installed. You can then install PyOPIA like this:
 
 ```
-pip install git+https://github.com/SINTEF/pyopia.git@main
+pip install git+https://github.com/SINTEF/PyOPIA.git@main
 ```
 
-We would usually recommend installing within a virtual python environment, which you can read more about [here](https://jni.github.io/using-python-for-science/intro-to-environments.html).
+We would usually recommend installing within a virtual python environment, which you can read more about [here](https://jni.github.io/using-python-for-science/intro-to-environments.html).
+
+Links to libraries PyOPIA uses
+==================================
+
+PyOPIA is a high-level tool that makes use of several open source libraries. Please see the list of libraries listed in the pyproject.toml file if you are interested in what is used.
+
+Processing and plotting modeuls makes routine use of several functions provided by libraries including: [scikit-image](https://scikit-image.org/),
+[numpy](https://numpy.org/), [pandas](https://pandas.pydata.org/), [xarray](https://docs.xarray.dev), [matplotlib](https://matplotlib.org/),
+[cmocean](https://matplotlib.org/cmocean/), [tensorflow](https://www.tensorflow.org/), and [keras](https://keras.io/).