Chapter_02_Getting_Started/problems.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Problems"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2-1 Insertion sort on small arrays in merge sort\n",
    "\n",
    "> Although merge sort runs in $\\Theta(n \\lg n)$ worst-case time and insertion sort runs in $\\Theta(n^2)$ worst-case time, the constant factors in insertion sort can make it faster in practice for small problem sizes on many machines. Thus, it makes sense to __coarsen__ the leaves of the recursion by using insertion sort within merge sort when subproblems become sufficiently small. Consider a modification to merge sort in which $n=k$ sublists of length $k$ are sorted using insertion sort and then merged using the standard merging mechanism, where $k$ is a value to be determined."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> __*a*__. Show that insertion sort can sort the $n/k$ sublists, each of length $k$, in $\\Theta(nk)$ worst-case time."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$\\Theta(k^2) \\cdot n/k = \\Theta(nk)$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> __*b*__. Show how to merge the sublists in $\\Theta(n\\lg(n/k))$ worst-case time."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Layers: $\\lg(n/k)$\n",
    "* Each: $n$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> __*c*__. Given that the modified algorithm runs in $\\Theta(nk+n\\lg(n/k))$ worst-case time, what is the largest value of k as a function of n for which the modified algorithm has the same running time as standard merge sort, in terms of $\\Theta$-notation?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since $n \\lg (n/k) <= n \\lg n$, thus $nk = n\\lg n$, $k=\\lg n$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> __*d*__. How should we choose $k$ in practice?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Profiling with large data set."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2-2 Correctness of bubblesort\n",
    "\n",
    "> Bubblesort is a popular, but inefficient, sorting algorithm. It works by repeatedly swapping adjacent elements that are out of order.\n",
    "```\n",
    "BUBBLESORT(A)\n",
    "1 for i= 1 to A.length - 1\n",
    "2     for j = A.length downto i + 1\n",
    "3         if A[j] < A[j - 1]\n",
    "4             exchange A[j] with A[j - 1]\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> __*a*__. Let $A'$ denote the output of BUBBLESORT(A). To prove that BUBBLESORT is correct, we need to prove that it terminates and that\n",
    ">\n",
    "> $A'[1] \\le A'[2] \\le \\cdots \\le A'[n]$  (2.3)\n",
    "> \n",
    "> where $n$ = $A$.length. In order to show that BUBBLESORT actually sorts, what else do we need to prove?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$A'$ is a permutation of $A$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> __*b*__. State precisely a loop invariant for the for loop in lines 2–4, and prove that this loop invariant holds. Your proof should use the structure of the loop invariant proof presented in this chapter."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Initialization: $A[1]$ is sorted\n",
    "* Maintenance: Move the smallest element to the left\n",
    "* Termination: $A[1..i]$ is sorted with the next smallest element in $A[i]$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> __*c*__. Using the termination condition of the loop invariant proved in part (b), state a loop invariant for the for loop in lines 1–4 that will allow you to prove inequality (2.3). Your proof should use the structure of the loop invariant proof presented in this chapter."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Initialization: $A[1..i-1]$ is sorted with smallest elements\n",
    "* Maintenance: Move the next smallest element to $A[i]$ and $A[i - 1] <= A[i]$\n",
    "* Termination: (2.3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> __*d*__. What is the worst-case running time of bubblesort? How does it compare to the running time of insertion sort?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$\\Theta(n^2)$\n",
    "\n",
    "For average cases insertion sort is better."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2-3 Correctness of Horner’s rule\n",
    "\n",
    "> The following code fragment implements Horner’s rule for evaluating a polynomial\n",
    ">\n",
    "> $\\begin{matrix}\n",
    "P(x) & = & \\sum_{k=0}^n a_k x^k \\\\ \n",
    " & = & a_0 + x(a_1 + x(a_x + \\cdots + x(a_{n-1} + xa_n)\\cdots))\n",
    "\\end{matrix}$\n",
    ">\n",
    "> given the coefficients $a_0, a_1, \\cdots, a_n$ and a value for $x$:\n",
    ">\n",
    "> ```\n",
    "> 1 y = 0\n",
    "> 2 for i = n downto 0\n",
    "> 3     y = ai + x y\n",
    ">```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> __*a*__. In terms of $\\Theta$-notation, what is the running time of this code fragment for Horner’s rule?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$\\Theta(n)$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> __*b*__. Write pseudocode to implement the naive polynomial-evaluation algorithm that computes each term of the polynomial from scratch. What is the running time of this algorithm? How does it compare to Horner’s rule?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "def polynomial_evaluation(a, x):\n",
    "    sum = 0\n",
    "    for i in range(len(a)):\n",
    "        sum += a[i] * x ** i\n",
    "    return sum"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$\\Theta(n^2)$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> __*c*__. Consider the following loop invariant:\n",
    "> \n",
    "> At the start of each iteration of the for loop of lines 2–3,\n",
    "> \n",
    "> $y=\\sum_{k=0}^{n-(i+1)} a_{k+i+1}x^k$.\n",
    ">\n",
    "> Interpret a summation with no terms as equaling 0. Following the structure of the loop invariant proof presented in this chapter, use this loop invariant to show that, at termination, $y=\\sum_{k=0}^n a_k x^k$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Initialization: $y=0$\n",
    "* Maintenance: $y=a_i+x\\sum_{k=0}^{n-(i+1)} a_{k+i+1}x^k$ $=a_ix^0+\\sum_{k=0}^{n-(i+1)} a_{k+i+1}x^{k+1}$ $=a_ix^0+\\sum_{k=1}^{n-i} a_{k+i}x^{k}$ $=\\sum_{k=0}^{n-i} a_{k+i}x^{k}$\n",
    "* Termination: $y=\\sum_{k=0}^{n} a_{k}x^k$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> __*d*__. Conclude by arguing that the given code fragment correctly evaluates a polynomial characterized by the coefficients $a_0, a_1, \\cdots, a_n$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$\\sum y_i = P(x)$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2-4 Inversions\n",
    "\n",
    "> Let $A[1..n]$ be an array of n distinct numbers. If $i < j$ and A[i] > A[j], then the pair $(i, j)$ is called an __*inversion*__ of A."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> __*a*__. List the five inversions of the array $\\left \\langle 2, 3, 8, 6, 1 \\right \\rangle$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* $(2, 1)$\n",
    "* $(3, 1)$\n",
    "* $(8, 6)$\n",
    "* $(8, 1)$\n",
    "* $(6, 1)$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> __*b*__. What array with elements from the set $\\{1,2,\\cdots,n\\}$ has the most inversions? How many does it have?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Most: $\\{n,n-1,\\cdots,1\\}$\n",
    "* How many: $\\frac{n(n-1)}{2}$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> __*c*__. What is the relationship between the running time of insertion sort and the number of inversions in the input array? Justify your answer."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Equal."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> __*d*__. Give an algorithm that determines the number of inversions in any permutation on $n$ elements in $\\Theta(n \\lg n)$ worst-case time. (Hint: Modify merge sort.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def count_inversion_sub(arr, l, r):\n",
    "    if l >= r:\n",
    "        return 0\n",
    "    mid = (l + r) // 2\n",
    "    cnt = count_inversion_sub(arr, l, mid) + count_inversion_sub(arr, mid+1, r)\n",
    "    arr_l = [arr[i] for i in range(l, mid+1)]\n",
    "    arr_l.append(1e100)\n",
    "    arr_r = [arr[j] for j in range(mid+1, r+1)]\n",
    "    arr_r.append(1e100)\n",
    "    i, j = 0, 0\n",
    "    for k in range(l, r+1):\n",
    "        if arr_l[i] <= arr_r[j]:\n",
    "            arr[k] = arr_l[i]\n",
    "            i += 1\n",
    "        else:\n",
    "            arr[k] = arr_r[j]\n",
    "            j += 1\n",
    "            cnt += len(arr_l) - i - 1\n",
    "    return cnt\n",
    "\n",
    "\n",
    "def count_inversion(arr):\n",
    "    return count_inversion_sub(arr, 0, len(arr) - 1)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}