v0.5.0rc3

This is the third release candidate for version 0.5.0!

• sy.load is now automatic
• apache arrow torch serde
• python 3.9 and pydp support
• updated dev tooling with tox
• new docker containers for grid
• added docker-compose example
• unified versioning system for 0.5.0rc3
• grid supports PostgreSQL
• lots of bug fixes

v0.5.0 - Release Candidate 2

0.5.0rc2

This is the second release candidate for version 0.5.0!

• Recursive Serializable Plans with ResNet 18 example
• PyGrid Integration fixes
• New Mono Repo structure
• Renamed master branch to main
• Added torch==1.8.1 support
• Many more bug fixes and minor improvements

v0.5.0 - Release Candidate 1

0.5.0rc1

This is the first release candidate for version 0.5.0!

Happy April Fools Day - sorry, this release is NOT a joke. 😄

We have added support for the following:

• AST
  • New API for support libraries
  • static methods
  • global variables
  • dynamic methods
  • enums
• Core
  • Storeable refactored
  • Basic Plans
  • Model Centric Federated Learning
    • KotlinSyft
    • PySyft (as a worker lib)
    • SwiftSyft is still in development
    • syft.js is still in development
  • CLI / Repl Support
  • Remote Dataloader
    • Torch Tensor
    • CSV
• *Library Support
  • Python
    • OrderedDict
    • Slice
    • Iterator
    • Tuple
  • TenSEAL
  • SyMPC
  • PyDP
  • PSI
  • opacus
  • pandas
  • petlib
  • PIL
  • PyTorch Lightning (requires our branch)
  • zsks
  • numpy arrays
• Grid
  • Grid Client
  • FL Client
• PyTorch Support
  • 1.6.x
  • 1.7.x
  • 1.8.x
• Python
  • 3.6
  • 3.7
  • 3.8
  • 3.9
• OS
  • Mac
  • Windows
  • Linux
• Updated Duet Logo
• New Logging System
• Integration Tests
• Duet Notebook Tests
• Benchmark Tests
• Examples
  • DCGAN
  • MNIST
  • MNIST Lightning
  • SuperResolution
  • VAE
  • Model Centric Federated Learning
  • Homomorphic Encryption
  • Vertical Learning
  • Private Set Intersection
  • Secure Multi Party Compute
  • Privacy AI Series (Course 2)

* Please know that library support is still incomplete. This release is intended to support the basic, minimum requirements for the below libraries.

Acknowledgements

While this isn't an exhaustive list, we'd like to thank the following people for contributing to this release!

Adam J Hall
Animesh Gupta
Anshuman Singh
Anubhav Raj Singh
Ariann Farias
ArtistBanda
Arturo Marquez Flores
Avinash Swaminathan
Ayoub Benaissa
Ben Fielding
Bogdan Cebere
Chinmay Shah
Chirag Gomber
Chitresh Goel
Daniele Gadler
George-Cristian Muraru
Gokkulnath TS
H4LL
Harkirat Singh
Hideaki Takahashi (Syumei)
Ionesio Junior
Irina Bejan
JMLourier
Jasopaum
Jirka Borovec
Joel Lee
José Benardi de Souza Nunes
Koen van der Veen
Lee Yi Jie Joel
Madhava Jay
Nabanita Dash
Nahua
Nicolas Remerscheid
Param Mirani
Patrick Cason
Plamen Hristov
Praful932
Raghav Prabhakar
Rima
Robert Perrotta
Syumei
Théo Ryffel
Tudor Cebere
Vaibhav Vardhan
Vova Manannikov
Wansoo Kim
Zarreen Naowal Reza
akshay.verma
animesh-007
avinsit123
curtismitchell
dnabanita7
gleec14
jmaunon
koukyosyumei
lazy
marcalph
nahuakang
rahul
rajatrc1705
root
shubhendra
syumeikoukyo
tchaton
uid42
xutongye

v0.3.0

PySyft 0.3.0 Release Notes

Welcome to the next evolution of PySyft. Today we're releasing PySyft 0.3.0, which is a complete rewrite over the 0.2.x product line. Since this is a complete rewrite of the codebase, we strongly suggest that you use 0.3.0 if you're just getting started with PySyft. If you're an existing user of PySyft 0.2.x, there will still be some support for that product line but we encourage you to upgrade. As always, we are happy to assist in this process over Slack.

For those interested in trying out PySyft 0.3.0, we suggest you start with our "Duet" series of demos. All future releases will include a detailed breakdown of breaking changes, fixes, bug patches, and improvements.

v0.2.9

Patched

v0.2.9

TL;DR: Improve Function Secret Sharing for faster execution (5x faster than SecureNN) ; Add support for the BFV scheme ; Add benchmarks for monitoring ; Initialize FALCON implementation.

PySyft 0.2.9 Release Notes

New functionalities:

• Replicated Sharing Tensor (RST): secret sharing, reconstruction, + - linear operations (#3856 by @abogaziah)
• Replicated Sharing Tensor: Multiplication, Matrix multiplication, and Convolution (#4015 by @abogaziah)
• Add Falcon module structure & design (#4484 by @abogaziah)
• Added reversed operations to RST (#4508 by @abogaziah)
• Falcon - Select share + evaluating (#4474 by @abogaziah @gmuraru)
• RNS variant multiplication operation for FV scheme. (#3893 by @IamRavikantSingh)
• Update FV scheme to use context chain instead of single context object (#3962 by @IamRavikantSingh)
• Implement Relinearization operation of BFV Scheme (#3977 by @IamRavikantSingh)
• Refactor bfv tests for faster results (#4012 by @IamRavikantSingh)
• Add black2b hash to generate param_id for bfv scheme (#4509 by @IamRavikantSingh)
• Add option to launch PyGrid network and nodes upon creation of a cluster (#3896 by @rimijoker)
• Add argmin and min method in AST (#3961 by @marload)
• Add RNG for workers (#3973 by @gmuraru and @abogaziah)
• Add Benchmark Graph, Scripts for Sigmoid Approx. Methods (#4031 by @aanurraj)
• Add benchmark graph and script for tanh function approximation methods (#4465 by @arturomf94)
• Add MPC Benchmark Abstraction (sigmoid, tanh) (#4494 by @marload)
• Added scripts, graphs for benchmarking AST operations (#4064 by @aanurraj)

Bug fixes

• Fix CrypTen tutorials (#3951 by @gmuraru)
• Fix imports to avoid having imports in code called by __del__ (#3932 by @LaRiffle)
• Overload mod operator for the fixed precision tensor (#3960 by @arturomf94)
• Increase key-size to 2048-bit keys in test_websocket_worker (#3788 by @arturomf94)
• Grid clients serialisation correction (#4020 by @Nilanshrajput)
• Fix Return invalid dtype when MPC is applied (#4035 by @aanurraj)
• Fix syft-proto version (#4469 by @gmuraru)

Refactoring:

• Improve the FSS crypto protocol to use Numpy + Rust for AST (#3892 by @LaRiffle)
• Refactor SPDZ to use the crypto store (#3909 by @LaRiffle)
• Optimize max and argmax for AdditiveSharingTensor (#3895 by @LaRiffle)
• Code refactor using list comprehension + cleaning (#3920 #3921 #3929 #3949 #3938 #3955 #3947 by @marload)
• Remove unused ss_scale var in PATE (#3956 by @marload)
• Add has_child method to AbstractTensor (#3948 by @marload)
• Make EmptyCryptoPrimitiveStoreError serializable (#3944 by @LaRiffle)
• Refactor various little things (#3943 by @LaRiffle)
• Update PyGrid URLs and cleanup Model-Centric notebooks a bit (#3959 by @vvmnnnkv)
• Refactor AdditiveSharingTensor (#3924 by @LaRiffle)
• Raise RuntimeError when negative integers is entered in the pow (#3987 by @marload)
• Make Sandbox Tutorial more Intuitive (#3996 by @himalayanZephyr)
• Improve Fast FixedPrecisonTensor Reciprocal Method (150x faster) (#4021 by @marload)
• Use division for sigmoid (#4044 by @gmuraru)
• Update simplify and detail in datacentclient (#4056 by @Nilanshrajput)
• Add support for negative numbers in reciprocal method (#4065 by @addy369)
• Error message if non integer value used with BFV scheme (#4042 by @IamRavikantSingh)
• Add reformat and pin black version (#4500 by @gmuraru)
• Updated startup scripts used in the Auto-scale API (#4007 by @rimijoker)

Docs & tutorials:

• Add a new tutorial on encrypted inference on resnet18 (#4539 by @LaRiffle)
• Make tutorials about encrypted DNN faster using FSS (#3933 by @LaRiffle)
• Documentation plan and protocol (#3974 by @nahuakang)
• Update Part 2 to just print the loss tensor (#3964 by @himalayanZephyr)
• Update Docs (#4475 by @IamRavikantSingh)

Tests & CI

v0.2.9

Patched

v0.2.8

TL;DR: An incremental release for the benefit of mobile workers and pygrid. A new backend for encrypted computation.

PySyft 0.2.8 Release Notes

Breaking changes since v0.2.7:

• Python 3.6 is not supported anymore (#3894)
• Static and Dynamic FL concepts are resp. replaced with Model and Data centric (#3907 by @cereallarceny)

New functionality:

• Add support for CrypTen as a framework (#3894 by @youben11 @gmuraru & @LaRiffle)
• Add BatchNorm to AdditiveSharingTensor (#3889 by @LaRiffle)
• Improves the FV scheme with Parameters validation (#3870 by @IamRavikantSingh)

Bug fixes

• Fix inplace operations += and -= for AutogradTensor (#3872 by @juharris)
• Fix nn.Module copy() (#3904 by @omershlo)

Refactoring:

• Improve remote garbage collection so support batched GC (#3805 by @LaRiffle)
• Optimize communication in AvgPool, MaxPool and AdaptivePooling (#3836 by @LaRiffle)
• Namespace dynamic FL client routes. Added url prefix dynamic to the Dynamic FL client url (#3853 by @sachin-101)
• Improve public mul and div for AdditiveSharingTensor (#3835 by @LaRiffle)

Docs & tutorials:

• New Demo Tutorial on Training Encrypted NN using CrypTen on Grid (#3717 by @youben11 @gmuraru)
• New tutorial on static FL: the Bandit Demo by @cereallarceny
• Add notebook to show trained static FL model retrieval (#3861 by @vvmnnnkv)
• Update install instructions in notebooks (#3830 by @mapmeld)
• Update README.md by @cereallarceny

Tests

• Add Plan example using a RNN in tests (#3860 by @juharris and @vvmnnnkv)

v0.2.7

PySyft 0.2.7 Release Notes

New functionality:

• Implement msgpack serde for RoleAssignments (#3595 by @Jasopaum)
• Create AbstractSendable class to provide comms methods generically (#3611 by @karlhigley)
• Support serialization of lists in Arg (#3617 by @vvmnnnkv and @karlhigley)
• Add FV homomorphic encryption scheme (#3276 @IamRavikantSingh)
• Proto messaging support (#3638 by @tudorcebere)
• BaseDataset proto support (#3641 by @tudorcebere & @karlhigley)
• Add iterable behavior for PointerTensor (#3659 by @gmuraru)
• Add connection speed test for workers (#3609 by @vkkhare)
• Add mean and var methods for AST (#3682 by @LaRiffle)
• Add new gradient functions to Autograd (#3681 by @LaRiffle)
• Implemented Subtraction operation of FV Scheme (#3775 by @IamRavikantSingh)
• Add a mechanism for arbitrary remote exec on Python workers (#3689 by @LaRiffle)
• Add Multiplication operation of FV scheme. (#3711 by @IamRavikantSingh)

Bug fixes

• Fix federated_avg side effect (#3614 @joaolcaas)
• serde, torch: fix typo in numpy tensor serializer. (#3644 by @shubham3121)
• Fixed precision tensor support (#3650 by @tudorcebere & @karlhigley)
• Fix: Copy issue with ciphertext in FV scheme. (#3672 by @IamRavikantSingh)
• Fix dtype issues in AST (#3679 by @LaRiffle)
• Fix decorator issues for FSS in AST (#3680 by @LaRiffle)
• Fix bug in set_model_params() in Create Plan (#3781 @sachin-101)
• Fix grid model hosting for large models (#3719 by @IonesioJunior)
• Fix Plan trace for inplace ops, add Role actions pruning (#3798 by @vvmnnnkv and @LaRiffle)
• Fix issues in handle_func_command for syft tensors (#3807 by @LaRiffle)

Refactoring:

• Use serde for tags serialization in PointerTensor (#3566 by @gmuraru)
• Move torch arg hooking into the relevant classes to reduce circular imports (#3612 by @karlhigley)
• Remove protocol parameter in decrypt (#3599 by @youben11)
• Add utilities for hooking and chaining methods calls along tensor chain (#3636 by @karlhigley)
• Add an __init__.py file to syft.generic.abstract (#3642 by @karlhigley)
• Move permissions from TorchTensor.send() to PrivateTensor._before_send hook (#3643 by @karlhigley)
• Remove owner attribute from FrameworkWrapper (#3652 by @Jasopaum)
• Move tests for generic functionality out of test/torch/tensors dir (#3645 by @karlhigley)
• Clean up AbstractSendable to remove extraneous methods (#3660 by @karlhigley)
• Improve @hookable decorator to allow modifying return values after (#3655 by @karlhigley)
• Adding support in msgpack for external libs. (#3653 by @tudorcebere)
• Extract MessageHandler classes from BaseWorker (#3661 by @karlhigley)
• Change whitelist to allow_list (#3675 by @karlhigley)
• Update Operations: For non same size polynomials (#3684 @IamRavikantSingh )
• Removed wrap() on non-tensor pointers in respond_to_search() (#3685 by @AlanAboudib)
• feat: add plan translation to tfjs (#3371 @Nolski)
• Update hook_args list (#3677 by @LaRiffle)
• Customizable timeout in workers (#3760 by @youben11)
• Serialize Plan's roles and base_framework fields (#3801 by @vvmnnnkv and @gmuraru)
• Grid module architecture refactoring (#3818 by @hericlesme)

Docs & tutorials:

• Update CODEOWNERS file to reflect new team names (#3620 by @karlhigley)
• Update Part 11 - Secure Deep Learning Classification.ipynb (#3444 by @znreza)
• Simplify encrypted aggregation in the tutorials (#3666 by @bobsonlin26 )

Tests & CI

• Delete pull_request_template.md by @cereallarceny
• Register the translation mark with PyTest to suppress warnings (#3618 by @karlhigley)
• Update requirements.txt with tornado compatible notebook version (#3606 by @rajathpatel23)
• Update the label used to mark stale issues/PRs (#3630 by @karlhigley)
• Added functionality to create a cluster using autoscale API (#3637 @rimijoker)

v0.2.6

PySyft 0.2.6 Release Notes

Breaking changes since v0.2.5:

• Message.contents field has now been fully removed after previous deprecation (#3477 by @karlhigley)
• TrainConfig has been fully removed (#3532 by @karlhigley)

New functionality:

• Add initial support for auto-scaling PyGrid workers and notebooks on Google Cloud Platform (#3437 and #3483 by @rimijoker)
• Add asynchronous send_command method to WebsocketClient (#3454 by @LaRiffle and @Syzygianinfern0)
• Build Protocol Roles from @func2protocol decorator (#3488 by @Jasopaum)
• Trace CommunicationActions into Protocol Roles (#3448 by @Prtfw)
• Add a reset() method on Role (#3495 by @Jasopaum)
• Add Plan argument type-checking along with support for lists and dicts as Plan arguments (#3386 by @tudorcebere)
• Support additional ML/DL frameworks during Plan tracing (#3461 by @gmuraru)
• Provide a way to trace module-level methods like torch.rand() into Plan/Protocol Roles (#3522 by @karlhigley)
• Allow Protocol Roles to have State for initial values (#3499 by @Jasopaum)
• Add ability to seed the Syft IdProvider and safely toggle verbose mode (#3457 by @Syzygianinfern0)
• Add support for WebRTC connections between peer Workers (#3548 and several more by @IonesioJunior)
• Add Protobuf serialization support for torch.dtype, syft.generic.String, and PointerDataset (#3584, #3594, #3600 by @tudorcebere)
• Print useful information on syft.grid.register() (#3589 by @IonesioJunior)
• Add RoleAssignments to Protocols (#3544 by @Jasopaum)
• Add ways to fetch Grid network peers to Network and WebRTCManager #3582 by @joaolcaas)
• Host private datasets locally with Grid (#3596 by @IonesioJunior)
• Add a method to return the supported ML/DL frameworks in BaseWorker (#3535 by @shubham3121)

Bug fixes:

• Fix the type of the allowed users parameter to PrivateTensors (#3446 by @gmuraru)
• Fix PointerTensor.move() when the source and destination workers are the same (#3442 by @Syzygianinfern0)
• Remove stray objects from FSS Worker/Plan initialization (#3525 by @karlhigley)
• Fix Batchnorm for Federated Learning (#3472 by @LaRiffle)
• Add scaffolding for Federated Learning client worker API (#3426 by @vvmnnnkv)
• Avoid registering Grid peer Workers multiple times (#3591 by @jefersonf)
• Remove test_spinup_time which has been flaky (#3603 by @robert-wagner)

Refactoring:

• Clean up Syft implementation of RNNs (#3441 by @dcato98)
• Store logged message in BaseWorker in an unserialized format for readability (#3473 by @karlhigley)
• Update Role.register_action() to allow registering CommunicationActions (#3465 by @karlhigley)
• Refactor BaseWorker.send_command() to accept explicit arguments (#3487 by @shubham3121)
• Add a SyftSerializable base class for Syft classes that need to be serialized to msgpack or Protobuf (#3386 by @tudorcebere)
• Convert ObjectStore to a stand-alone class instead of a base class for Workers (#3484 by @karlhigley)
• Extract @abstractmethods from BaseWorker to AbstractWorker (#3519 by @karlhigley)
• Create a VirtualWorker for each Plan/Protocol Role and use it for tracing (#3521 by @karlhigley)
• Improve the serde interface with lazy data loaders (#3530 by @tudorcebere)
• Split FrameworkHook into multiple parts (#3561 by @karlhigley)
• Clean up the code style of PySyft (#3535 by @gmuraru)

Examples and Tutorials:

• Add an experimental notebook for how to create a Plan that trains a NN on MNIST using Syft autograd (#3567 by @vvmnnnkv)

Translations:

• Updates to Bengali translation of Parts 4-5, 12-13 (#3415 by @jabertuhin, #3445, #3449, #3466, and #3500 by @znreza)

Documentation:

• Update the Pull Request templates (#3506 by @tkrsh, #3520 by @karlhigley, additional work by @cereallarceny)

Tests and Builds:

• Add a check for docstring coverage to the Github builds (#3462 and #3505 by @karlhigley)
• Remove verbose flag from code format check to highlight necessary changes (#3475 by @karlhigley)