Merge branch 'dev' into aziz/job_no_consumer_err

OpenMined · May 20, 2024 · 09c6a6d · 09c6a6d
2 parents 1ed333a + c10b0e4
commit 09c6a6d
Show file tree

Hide file tree

Showing 88 changed files with 1,722 additions and 2,885 deletions.
diff --git a/.dockerignore b/.dockerignore
diff --git a/.github/workflows/pr-tests-syft.yml b/.github/workflows/pr-tests-syft.yml
@@ -202,7 +202,7 @@ jobs:
       matrix:
         os: [ubuntu-latest]
         python-version: ["3.10", "3.11", "3.12"]
-        deployment-type: ["k8s"]
+        deployment-type: ["remote"]
         notebook-paths: ["api/0.8"]
       fail-fast: false
 

diff --git a/README.md b/README.md
@@ -108,18 +108,11 @@ For Google GKE we need the [`gce` annotation](https://cloud.google.com/kubernete
 helm install ... --set ingress.class="gce"
 ```
 
-## Deploy to a Container Engine or Cloud
+## Note:
 
-1. Install our handy 🛵 cli tool which makes deploying a Domain or Gateway server to Docker or VM a one-liner:  
-   `pip install -U hagrid`
+🚨 Our deployment tool `Hagrid` has been `Deprecated`. For the updated deployment options kindly refer to
 
-2. Then run our interactive jupyter Install 🧙🏽‍♂️ Wizard<sup>BETA</sup>:  
-   `hagrid quickstart`
-
-3. In the tutorial you will learn how to install and deploy:  
-   `PySyft` = our `numpy`-like 🐍 Python library for computing on `private data` in someone else's `Domain`
-
-   `PyGrid` = our 🐳 `docker` / 🐧 `vm` `Domain` & `Gateway` Servers where `private data` lives
+- 📚 <a href="https://github.com/OpenMined/PySyft/tree/dev/notebooks/tutorials/deployments">Deployments</a>
 
 ## Docs and Support
 
@@ -128,10 +121,8 @@ helm install ... --set ingress.class="gce"
 
 # Install Notes
 
-- HAGrid 0.3 Requires: 🐍 `python` 🐙 `git` - Run: `pip install -U hagrid`
-- Interactive Install 🧙🏽‍♂️ Wizard<sup>BETA</sup> Requires 🛵 `hagrid`: - Run: `hagrid quickstart`
 - PySyft 0.8.1 Requires: 🐍 `python 3.10 - 3.12` - Run: `pip install -U syft`
-- PyGrid Requires: 🐳 `docker`, 🦦 `podman` or ☸️ `kubernetes` - Run: `hagrid launch ...`
+- PyGrid Requires: 🐳 `docker`, 🦦 `podman` or ☸️ `kubernetes`
 
 # Versions
 
@@ -154,13 +145,9 @@ Deprecated:
 
 PySyft and PyGrid use the same `version` and its best to match them up where possible. We release weekly betas which can be used in each context:
 
-PySyft (Stable): `pip install -U syft`  
-PyGrid (Stable) `hagrid launch ... tag=latest`
-
-PySyft (Beta): `pip install -U syft --pre`  
-PyGrid (Beta): `hagrid launch ... tag=beta`
+PySyft (Stable): `pip install -U syft`
 
-HAGrid is a cli / deployment tool so the latest version of `hagrid` is usually the best.
+PySyft (Beta): `pip install -U syft --pre`
 
 # What is Syft?
 

diff --git a/notebooks/admin/Custom API + Custom Worker.ipynb b/notebooks/admin/Custom API + Custom Worker.ipynb
@@ -36,8 +36,8 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "## k8s mode\n",
-    "# os.environ[\"ORCHESTRA_DEPLOYMENT_TYPE\"] = \"k8s\"\n",
+    "## remote mode\n",
+    "# os.environ[\"ORCHESTRA_DEPLOYMENT_TYPE\"] = \"remote\"\n",
     "# os.environ[\"DEV_MODE\"] = \"True\"\n",
     "domain_client = sy.login(\n",
     "    email=\"[email protected]\",\n",

diff --git a/notebooks/api/0.8/01-submit-code.ipynb b/notebooks/api/0.8/01-submit-code.ipynb
@@ -482,7 +482,7 @@
    "outputs": [],
    "source": [
     "# Once we start the project, it will submit the project along with the code request to the Domain Server\n",
-    "project = new_project.start()\n",
+    "project = new_project.send()\n",
     "project"
    ]
   },
@@ -599,7 +599,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.16"
+   "version": "3.10.13"
   },
   "toc": {
    "base_numbering": 1,

diff --git a/notebooks/api/0.8/06-multiple-code-requests.ipynb b/notebooks/api/0.8/06-multiple-code-requests.ipynb
@@ -250,7 +250,7 @@
    },
    "outputs": [],
    "source": [
-    "project = new_project.start()\n",
+    "project = new_project.send()\n",
     "\n",
     "project"
    ]
@@ -578,7 +578,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.5"
+   "version": "3.10.13"
   },
   "toc": {
    "base_numbering": 1,

diff --git a/notebooks/api/0.8/09-blob-storage.ipynb b/notebooks/api/0.8/09-blob-storage.ipynb
@@ -36,7 +36,6 @@
     "node = sy.orchestra.launch(\n",
     "    name=\"test-domain-1\",\n",
     "    dev_mode=True,\n",
-    "    in_memory_workers=True,\n",
     "    reset=True,\n",
     "    create_producer=True,\n",
     ")"

diff --git a/notebooks/api/0.8/10-container-images.ipynb b/notebooks/api/0.8/10-container-images.ipynb
@@ -91,8 +91,7 @@
     "# Disable inmemory worker for container stack\n",
     "running_as_container = os.environ.get(\"ORCHESTRA_DEPLOYMENT_TYPE\") in (\n",
     "    \"container_stack\",\n",
-    ")\n",
-    "in_memory_workers = not running_as_container"
+    ")"
    ]
   },
   {
@@ -106,7 +105,6 @@
     "    name=\"test-domain-1\",\n",
     "    dev_mode=True,\n",
     "    create_producer=True,\n",
-    "    in_memory_workers=in_memory_workers,\n",
     "    reset=True,\n",
     "    port=8081,\n",
     ")"
@@ -1485,7 +1483,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.8"
+   "version": "3.11.7"
   }
  },
  "nbformat": 4,

diff --git a/notebooks/api/0.8/11-container-images-k8s.ipynb b/notebooks/api/0.8/11-container-images-k8s.ipynb
@@ -45,7 +45,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "os.environ[\"ORCHESTRA_DEPLOYMENT_TYPE\"] = \"k8s\"\n",
+    "os.environ[\"ORCHESTRA_DEPLOYMENT_TYPE\"] = \"remote\"\n",
     "os.environ[\"DEV_MODE\"] = \"True\"\n",
     "\n",
     "# Uncomment this to add custom values\n",

diff --git a/notebooks/api/0.8/12-custom-api-endpoint.ipynb b/notebooks/api/0.8/12-custom-api-endpoint.ipynb
@@ -33,7 +33,6 @@
     "    dev_mode=True,\n",
     "    create_producer=True,\n",
     "    n_consumers=3,\n",
-    "    in_memory_workers=True,\n",
     "    reset=True,\n",
     "    port=8081,\n",
     ")\n",