notebooks/api/0.8/02-review-code-and-approve.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Reviewing and Approving Code in Syft as a Data Owner"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "SYFT_VERSION = \">=0.8.2.b0,<0.9\"\n",
    "package_string = f'\"syft{SYFT_VERSION}\"'\n",
    "# %pip install {package_string} -q"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# syft absolute\n",
    "import syft as sy\n",
    "\n",
    "sy.requires(SYFT_VERSION)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Login to Syft Domain Server"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Launch and connect to test-domain-1 server we setup in the previous notebook\n",
    "node = sy.orchestra.launch(name=\"test-domain-1\", port=\"auto\", dev_mode=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Log into the node with default root credentials\n",
    "domain_client = node.login(email=\"info@openmined.org\", password=\"changethis\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Selecting Project in the Syft Domain Server\n",
    "\n",
    "Let's see all the projects that are created by Data Scientists in this Domain Server"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "domain_client.projects"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Select the project you want to work with\n",
    "project = domain_client.projects[0]\n",
    "project"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All code requests submitted by the Data Scientists as a part of this project can be accessed by invoking the following"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "project.requests"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Tests\n",
    "assert len(project.events) == 1\n",
    "assert isinstance(project.events[0], sy.service.project.project.ProjectRequest)\n",
    "assert len(project.requests) == 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Reviewing Code Requests\n",
    "\n",
    "To review a specific request, we can select it and explore its attributes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "request = project.requests[0]\n",
    "request"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# See the code written by the Data Scientist and its metadata in the request\n",
    "func = request.code\n",
    "func"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# To see just the code\n",
    "func.show_code"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Reference to the assets that the function will run on\n",
    "func.assets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Viewing the Asset and it's mock/private variants that the Data Scientist will be running on"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "asset = func.assets[0]\n",
    "asset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "mock_data = asset.mock\n",
    "mock_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Private data. Accessible as we are logged in with Data Owner credentials\n",
    "pvt_data = asset.data\n",
    "pvt_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Tests\n",
    "assert len(asset.data_subjects) == 1\n",
    "assert mock_data.shape == (10, 22)\n",
    "assert pvt_data.shape == (10, 22)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Policies in Syft Function\n",
    "\n",
    "Each Syft Function requires an Input & Output policy attached to the python function against which executions are verified.\n",
    "\n",
    "Syft provides the following default policies:\n",
    "* `sy.ExactMatch()` Input policy ensures that function executes against the exact inputs specified by Data Scientist.\n",
    "* `sy.OutputPolicyExecuteOnce()` Output policy makes sure that the Data Scientist can run the function only once against the input.\n",
    "\n",
    "We can also implement custom policies based on our requirements. (Refer to notebook [05-custom-policy](./05-custom-policy.ipynb) for more information.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "op = func.output_policy_type\n",
    "op"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# See the implementation of the policy\n",
    "print(op.policy_code)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Execute the Data Scientist's code\n",
    "\n",
    "While Syft makes sure that the function is not tampered with, it does not perform any validation on the implementation itself.\n",
    "\n",
    "**It is the Data Owner's responsibility to review the code & verify if it's safe to execute.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Let's grab the actual executable function that was submitted by the user\n",
    "users_function = func.unsafe_function"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If the code looks safe, we can go ahead and execute it on the private dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mock_result = users_function(trade_data=mock_data)\n",
    "mock_result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "real_result = users_function(trade_data=pvt_data)\n",
    "real_result"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Sharing results back to the Data Scientist"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By calling this function we attach the result of the function to the original request\n",
    "\n",
    "`request.accept_by_depositing_result(real_result)`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's say we accidentally submit incorrect results, we can correct it by using override it using the `force=True` flag"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Uploaded wrong result - we shared mock_result instead of the real_result\n",
    "result = request.accept_by_depositing_result(mock_result)\n",
    "result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "assert isinstance(result, sy.SyftSuccess)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# upload correct result\n",
    "result = request.accept_by_depositing_result(real_result, force=True)\n",
    "result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "assert isinstance(result, sy.SyftSuccess)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Denying a request\n",
    "\n",
    "At times you would want to deny a request in cases where the output is violating privacy, or if either of the policy is too lineant, or perhaps the code is confusing!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Deny the request with an appropriate reason\n",
    "result = request.deny(\n",
    "    reason=(\n",
    "        \"The Submitted UserCode does not add differential privacy to the output.\"\n",
    "        \"Kindly add differential privacy and resubmit the code.\"\n",
    "    )\n",
    ")\n",
    "result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "assert isinstance(result, sy.SyftSuccess)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We can verify the status by checking our request list\n",
    "project.requests"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Re-approving requests\n",
    "\n",
    "Let's re-approve the request so that we can work with the results in the later notebooks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "result = request.accept_by_depositing_result(real_result)\n",
    "result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Verify the request status again\n",
    "project.requests"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Cleanup local domain server\n",
    "\n",
    "if node.node_type.value == \"python\":\n",
    "    node.land()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that the code request has been approved, let's go through the [03-data-scientist-download-result](./03-data-scientist-download-result.ipynb) notebook to see how a Data Scientist can access the results."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.2"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}