Merge pull request #601 from yzhao062/development

v2.0.2

CHANGES.txt

@@ -191,4 +191,9 @@ v<2.0.0>, <05/21/2024> -- Moving from TF to Torch -- implement dl base with more
 v<2.0.1>, <06/16/2024> -- Moving from TF to Torch -- reimplement DeepSVDD.
 v<2.0.1>, <06/17/2024> -- Moving from TF to Torch -- reimplement dl_base.
 v<2.0.1>, <06/21/2024> -- Moving from TF to Torch -- reimplement MO_GAAL.
-v<2.0.1>, <06/21/2024> -- Moving from TF to Torch -- reimplement AE and VAE.
+v<2.0.1>, <06/21/2024> -- Moving from TF to Torch -- reimplement AE and VAE.
+v<2.0.2>, <07/01/2024> -- Add AE1SVM.
+v<2.0.2>, <07/04/2024> -- Moving from TF to Torch -- reimplement ALAD.
+v<2.0.2>, <07/04/2024> -- Moving from TF to Torch -- reimplement anogan.
+v<2.0.2>, <07/06/2024> -- Complete of removing all Tensorflow and Keras code.
+v<2.0.2>, <07/21/2024> -- Add DevNet.

docs/about.rst

@@ -71,7 +71,6 @@ Lorenzo Perini (Ph.D. Researcher, KU Leuven):
 - Joined 2023 (implemented prediction confidence and more)
 - `Homepage (Lorenzo Perini) <https://people.cs.kuleuven.be/~lorenzo.perini/>`_
 
-
 Tiankai Yang (PhD student @ USC):
 
 - Joined 2024 (implemented Pytorch version of algorithms)
@@ -91,3 +90,8 @@ Zhuo Xiao (MS student @ USC):
 
 - Joined 2024 (implemented Pytorch version of algorithms)
 - `Homepage (Zhuo Xiao) <https://www.linkedin.com/in/zhuox5/>`_
+
+Bourne Li (MS student @ USC):
+
+- Joined 2024 (implemented Pytorch version of algorithms)
+- `Homepage (Bourne Li) <https://www.linkedin.com/in/bourne-li-a9a899231/>`_

docs/benchmark.rst

@@ -4,14 +4,15 @@ Benchmarks
 Latest ADBench (2022)
 ---------------------
 
-We just released a 36-page, the most comprehensive `anomaly detection benchmark paper <https://arxiv.org/abs/2206.09426>`_ :cite:`a-han2022adbench`.
-The fully `open-sourced ADBench <https://github.com/Minqi824/ADBench>`_ compares 30 anomaly detection algorithms on 55 benchmark datasets.
+
+We just released a 45-page, the most comprehensive `ADBench: Anomaly Detection Benchmark <https://arxiv.org/abs/2206.09426>`_ [#Han2022ADBench]_.
+The fully `open-sourced ADBench <https://github.com/Minqi824/ADBench>`_ compares 30 anomaly detection algorithms on 57 benchmark datasets.
 
 The organization of **ADBench** is provided below:
 
 .. image:: https://github.com/Minqi824/ADBench/blob/main/figs/ADBench.png?raw=true
    :target: https://github.com/Minqi824/ADBench/blob/main/figs/ADBench.png?raw=true
-   :alt: benchmark
+   :alt: benchmark-fig
 
 
 For a simpler visualization, we make **the comparison of selected models** via
@@ -21,6 +22,7 @@ For a simpler visualization, we make **the comparison of selected models** via
    :target: https://github.com/yzhao062/pyod/blob/development/examples/ALL.png?raw=true
    :alt: Comparison_of_All
 
+
 Old Results (2019)
 ------------------
 

docs/example.rst

@@ -17,8 +17,6 @@ PyOD has been well acknowledged by the machine learning community with a few fea
 
 **Towards Data Science**: `Anomaly Detection for Dummies <https://towardsdatascience.com/anomaly-detection-for-dummies-15f148e559c1>`_
 
-**Computer Vision News (March 2019)**: `Python Open Source Toolbox for Outlier Detection <https://rsipvision.com/ComputerVisionNews-2019March/18/>`_
-
 **awesome-machine-learning**: `General-Purpose Machine Learning <https://github.com/josephmisiti/awesome-machine-learning#python-general-purpose>`_
 
 

docs/fast_train.rst

@@ -3,7 +3,7 @@ Fast Train with SUOD
 
 **Fast training and prediction**: it is possible to train and predict with
 a large number of detection models in PyOD by leveraging SUOD framework.
-See  `SUOD Paper <https://www.andrew.cmu.edu/user/yuezhao2/papers/21-mlsys-suod.pdf>`_
+See  `SUOD Paper <https://proceedings.mlsys.org/paper_files/paper/2021/file/37385144cac01dff38247ab11c119e3c-Paper.pdf>`_
 and  `SUOD example <https://github.com/yzhao062/pyod/blob/master/examples/suod_example.py>`_.
 
 

docs/index.rst

@@ -40,7 +40,7 @@ Welcome to PyOD documentation!
 
 .. image:: https://github.com/yzhao062/pyod/actions/workflows/testing.yml/badge.svg
    :target: https://github.com/yzhao062/pyod/actions/workflows/testing.yml
-   :alt: testing
+   :alt: Testing
 
 
 .. image:: https://coveralls.io/repos/github/yzhao062/pyod/badge.svg
@@ -67,71 +67,56 @@ Welcome to PyOD documentation!
 Read Me First
 ^^^^^^^^^^^^^
 
-Welcome to PyOD, a versatile Python library for detecting anomalies in multivariate data. Whether you're tackling a small-scale project or large datasets, PyOD offers a range of algorithms to suit your needs.
+Welcome to PyOD, a comprehensive but easy-to-use Python library for detecting anomalies in multivariate data. Whether you're tackling a small-scale project or large datasets, PyOD offers a range of algorithms to suit your needs.
 
 * **For time-series outlier detection**, please use `TODS <https://github.com/datamllab/tods>`_.
 
 * **For graph outlier detection**, please use `PyGOD <https://pygod.org/>`_.
 
-* **Performance Comparison \& Datasets**: We have a 45-page, the most comprehensive `anomaly detection benchmark paper <https://arxiv.org/pdf/2206.09426>`_. The fully `open-sourced ADBench <https://github.com/Minqi824/ADBench>`_ compares 30 anomaly detection algorithms on 57 benchmark datasets.
+* **Performance Comparison & Datasets**: We have a 45-page, comprehensive `anomaly detection benchmark paper <https://openreview.net/forum?id=foA_SFQ9zo0>`_. The fully `open-sourced ADBench <https://github.com/Minqi824/ADBench>`_ compares 30 anomaly detection algorithms on 57 benchmark datasets.
 
-* **Learn more about anomaly detection** \@ `Anomaly Detection Resources <https://github.com/yzhao062/anomaly-detection-resources>`_
+* **Learn more about anomaly detection** at `Anomaly Detection Resources <https://github.com/yzhao062/anomaly-detection-resources>`_
 
-* **PyOD on Distributed Systems**: you could also run `PyOD on databricks <https://www.databricks.com/blog/2023/03/13/unsupervised-outlier-detection-databricks.html>`_.
+* **PyOD on Distributed Systems**: you can also run `PyOD on databricks <https://www.databricks.com/blog/2023/03/13/unsupervised-outlier-detection-databricks.html>`_.
 
 ----
 
 About PyOD
 ^^^^^^^^^^
 
-PyOD, established in 2017, has become a go-to **Python library** for **detecting anomalous/outlying objects** in
-multivariate data. This exciting yet challenging field is commonly referred as
-`Outlier Detection <https://en.wikipedia.org/wiki/Anomaly_detection>`_
-or `Anomaly Detection <https://en.wikipedia.org/wiki/Anomaly_detection>`_.
-
-PyOD includes more than 50 detection algorithms, from classical LOF (SIGMOD 2000) to
-the cutting-edge ECOD and DIF (TKDE 2022 and 2023). Since 2017, PyOD has been successfully used in numerous academic researches and
-commercial products with more than `17 million downloads <https://pepy.tech/project/pyod>`_.
-It is also well acknowledged by the machine learning community with various dedicated posts/tutorials, including
-`Analytics Vidhya <https://www.analyticsvidhya.com/blog/2019/02/outlier-detection-python-pyod/>`_,
-`KDnuggets <https://www.kdnuggets.com/2019/02/outlier-detection-methods-cheat-sheet.html>`_, and
-`Towards Data Science <https://towardsdatascience.com/anomaly-detection-for-dummies-15f148e559c1>`_.
+PyOD, established in 2017, has become a go-to **Python library** for **detecting anomalous/outlying objects** in multivariate data. This exciting yet challenging field is commonly referred to as `Outlier Detection <https://en.wikipedia.org/wiki/Anomaly_detection>`_ or `Anomaly Detection <https://en.wikipedia.org/wiki/Anomaly_detection>`_.
 
+PyOD includes more than 50 detection algorithms, from classical LOF (SIGMOD 2000) to the cutting-edge ECOD and DIF (TKDE 2022 and 2023). Since 2017, PyOD has been successfully used in numerous academic research projects and commercial products with more than `22 million downloads <https://pepy.tech/project/pyod>`_. It is also well acknowledged by the machine learning community with various dedicated posts/tutorials, including `Analytics Vidhya <https://www.analyticsvidhya.com/blog/2019/02/outlier-detection-python-pyod/>`_, `KDnuggets <https://www.kdnuggets.com/2019/02/outlier-detection-methods-cheat-sheet.html>`_, and `Towards Data Science <https://towardsdatascience.com/anomaly-detection-for-dummies-15f148e559c1>`_.
 
 **PyOD is featured for**:
 
 * **Unified, User-Friendly Interface** across various algorithms.
-* **Wide Range of Models**\, from classic techniques to the latest deep learning methods.
+* **Wide Range of Models**, from classic techniques to the latest deep learning methods in **PyTorch**.
 * **High Performance & Efficiency**, leveraging `numba <https://github.com/numba/numba>`_ and `joblib <https://github.com/joblib/joblib>`_ for JIT compilation and parallel processing.
 * **Fast Training & Prediction**, achieved through the SUOD framework :cite:`a-zhao2021suod`.
 
-
-**Outlier Detection with 5 Lines of Code**\ :
-
+**Outlier Detection with 5 Lines of Code**:
 
 .. code-block:: python
 
-
     # Example: Training an ECOD detector
     from pyod.models.ecod import ECOD
     clf = ECOD()
     clf.fit(X_train)
     y_train_scores = clf.decision_scores_  # Outlier scores for training data
     y_test_scores = clf.decision_function(X_test)  # Outlier scores for test data
 
-**Selecting the Right Algorithm:**. Unsure where to start? Consider these robust and interpretable options:
+
+**Selecting the Right Algorithm:** Unsure where to start? Consider these robust and interpretable options:
 
 - `ECOD <https://github.com/yzhao062/pyod/blob/master/examples/ecod_example.py>`_: Example of using ECOD for outlier detection
 - `Isolation Forest <https://github.com/yzhao062/pyod/blob/master/examples/iforest_example.py>`_: Example of using Isolation Forest for outlier detection
 
 Alternatively, explore `MetaOD <https://github.com/yzhao062/MetaOD>`_ for a data-driven approach.
 
-**Citing PyOD**\ :
+**Citing PyOD**:
 
-`PyOD paper <http://www.jmlr.org/papers/volume20/19-011/19-011.pdf>`_ is published in
-`Journal of Machine Learning Research (JMLR) <http://www.jmlr.org/>`_ (MLOSS track).
-If you use PyOD in a scientific publication, we would appreciate
-citations to the following paper::
+`PyOD paper <http://www.jmlr.org/papers/volume20/19-011/19-011.pdf>`_ is published in `Journal of Machine Learning Research (JMLR) <http://www.jmlr.org/>`_ (MLOSS track). If you use PyOD in a scientific publication, we would appreciate citations to the following paper::
 
     @article{zhao2019pyod,
         author  = {Zhao, Yue and Nasrullah, Zain and Li, Zheng},
@@ -148,8 +133,7 @@ or::
 
     Zhao, Y., Nasrullah, Z. and Li, Z., 2019. PyOD: A Python Toolbox for Scalable Outlier Detection. Journal of machine learning research (JMLR), 20(96), pp.1-7.
 
-For a broader perspective on anomaly detection, see our NeurIPS papers
-`ADBench: Anomaly Detection Benchmark <https://viterbi-web.usc.edu/~yzhao010/papers/22-neurips-adbench.pdf>`_ \& `ADGym: Design Choices for Deep Anomaly Detection <https://viterbi-web.usc.edu/~yzhao010/papers/23-neurips-adgym.pdf>`_::
+For a broader perspective on anomaly detection, see our NeurIPS papers `ADBench: Anomaly Detection Benchmark Paper <https://arxiv.org/abs/2206.09426>`_ and `ADGym: Design Choices for Deep Anomaly Detection <https://arxiv.org/abs/2309.15376>`_::
 
     @article{han2022adbench,
         title={Adbench: Anomaly detection benchmark},
@@ -169,13 +153,12 @@ For a broader perspective on anomaly detection, see our NeurIPS papers
     }
 
 
-
 ----
 
 ADBench Benchmark and Datasets
-==============================
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-We just released a 45-page, the most comprehensive `ADBench: Anomaly Detection Benchmark <https://arxiv.org/abs/2206.09426>`_ :cite:`a-han2022adbench`.
+We just released a 45-page, the most comprehensive `ADBench: Anomaly Detection Benchmark <https://arxiv.org/abs/2206.09426>`_ [#Han2022ADBench]_.
 The fully `open-sourced ADBench <https://github.com/Minqi824/ADBench>`_ compares 30 anomaly detection algorithms on 57 benchmark datasets.
 
 The organization of **ADBench** is provided below:
@@ -245,8 +228,10 @@ Neural Networks      SO_GAAL           Single-Objective Generative Adversarial A
 Neural Networks      MO_GAAL           Multiple-Objective Generative Adversarial Active Learning                                               2019   :class:`pyod.models.mo_gaal.MO_GAAL`                 :cite:`a-liu2019generative`
 Neural Networks      DeepSVDD          Deep One-Class Classification                                                                           2018   :class:`pyod.models.deep_svdd.DeepSVDD`              :cite:`a-ruff2018deepsvdd`
 Neural Networks      AnoGAN            Anomaly Detection with Generative Adversarial Networks                                                  2017   :class:`pyod.models.anogan.AnoGAN`                   :cite:`a-schlegl2017unsupervised`
-Neural Networks      ALAD                Adversarially learned anomaly detection                                                               2018   :class:`pyod.models.alad.ALAD`                       :cite:`a-zenati2018adversarially`
-Graph-based          R-Graph           Outlier detection by R-graph                                                                            2017   :class:`pyod.models.rgraph.RGraph`                   :cite:`you2017provable`
+Neural Networks      ALAD              Adversarially learned anomaly detection                                                                 2018   :class:`pyod.models.alad.ALAD`                       :cite:`a-zenati2018adversarially`
+Neural Networks      DevNet            Deep Anomaly Detection with Deviation Networks                                                          2019   :class:`pyod.models.devnet.DevNet`                   :cite:`a-pang2019deep`
+Neural Networks      AE1SVM            Autoencoder-based One-class Support Vector Machine                                                      2019   :class:`pyod.models.ae1svm.AE1SVM`                   :cite:`a-nguyen2019scalable`
+Graph-based          R-Graph           Outlier detection by R-graph                                                                            2017   :class:`pyod.models.rgraph.RGraph`                   :cite:`a-you2017provable`
 Graph-based          LUNAR             LUNAR: Unifying Local Outlier Detection Methods via Graph Neural Networks                               2022   :class:`pyod.models.lunar.LUNAR`                     :cite:`a-goodge2022lunar`
 ===================  ================  ======================================================================================================  =====  ===================================================  ======================================================
 
@@ -284,18 +269,6 @@ Utility              :func:`pyod.utils.utility.get_label_n`          Turn raw ou
 Utility              :func:`pyod.utils.utility.precision_n_scores`   calculate precision @ rank n
 ===================  ==============================================  =====================================================================================================================================================
 
-**The comparison among of implemented models** is made available below
-(\ `Figure <https://raw.githubusercontent.com/yzhao062/pyod/master/examples/ALL.png>`_\ ,
-`compare_all_models.py <https://github.com/yzhao062/pyod/blob/master/examples/compare_all_models.py>`_\ ,
-`Interactive Jupyter Notebooks <https://mybinder.org/v2/gh/yzhao062/pyod/master>`_\ ).
-For Jupyter Notebooks, please navigate to **"/notebooks/Compare All Models.ipynb"**.
-
-
-.. figure:: figs/ALL.png
-    :alt: Comparison of selected models
-
-Check the latest `benchmark <https://pyod.readthedocs.io/en/latest/benchmark.html>`_. You could replicate this process by running
-`benchmark.py <https://github.com/yzhao062/pyod/blob/master/notebooks/benchmark.py>`_.
 
 
 API Cheatsheet & Reference

docs/install.rst

@@ -1,8 +1,7 @@
 Installation
-============
+^^^^^^^^^^^^
 
-PyOD is designed for easy installation using either **pip** or **conda**.
-We recommend using the latest version of PyOD due to frequent updates and enhancements:
+PyOD is designed for easy installation using either **pip** or **conda**. We recommend using the latest version of PyOD due to frequent updates and enhancements:
 
 .. code-block:: bash
 
@@ -13,17 +12,15 @@ We recommend using the latest version of PyOD due to frequent updates and enhanc
 
    conda install -c conda-forge pyod
 
-Alternatively, you could clone and run setup.py file:
+Alternatively, you can clone and run the setup.py file:
 
 .. code-block:: bash
 
    git clone https://github.com/yzhao062/pyod.git
    cd pyod
    pip install .
 
-
-**Required Dependencies**\ :
-
+**Required Dependencies**:
 
 * Python 3.8 or higher
 * joblib
@@ -33,15 +30,14 @@ Alternatively, you could clone and run setup.py file:
 * scipy>=1.5.1
 * scikit_learn>=0.22.0
 
-
 **Optional Dependencies (see details below)**:
 
 * combo (optional, required for models/combination.py and FeatureBagging)
-* keras/tensorflow (optional, required for AutoEncoder, and other deep learning models)
+* pytorch (optional, required for deep learning models)
 * suod (optional, required for running SUOD model)
 * xgboost (optional, required for XGBOD)
 * pythresh (optional, required for thresholding)
 
 .. warning::
 
-    PyOD includes several neural network-based models, such as AutoEncoders, implemented in Tensorflow and PyTorch. These deep learning libraries are not automatically installed by PyOD to avoid conflicts with existing installations. If you plan to use neural-net based models, please ensure these libraries are installed. See the `neural-net FAQ <https://github.com/yzhao062/pyod/wiki/Setting-up-Keras-and-Tensorflow-for-Neural-net-Based-models>`_ for guidance. Additionally, xgboost is not installed by default but is required for models like XGBOD.
+    PyOD includes several neural network-based models, such as AutoEncoders, implemented in PyTorch. These deep learning libraries are not automatically installed by PyOD to avoid conflicts with existing installations. If you plan to use neural-net based models, please ensure these libraries are installed. See the `neural-net FAQ <https://github.com/yzhao062/pyod/wiki/Setting-up-Keras-and-Tensorflow-for-Neural-net-Based-models>`_ for guidance. Additionally, xgboost is not installed by default but is required for models like XGBOD.