{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Disparity by block matching\n",
    "\n",
    "EECE 5554 Robotics Sensing and Navigation, Lecture 13. Put `block_matching.py` in the same folder as this notebook, then run the cells in order."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Each dark box below is a code cell. Edit it, then press **Run** or **Shift+Enter**. Cells share variables from top to bottom, like Jupyter."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from block_matching import make_pair, block_matching, evaluate, cost_volume\n",
    "\n",
    "left, right, truth = make_pair()\n",
    "print(left.shape, 'true disparities:', np.unique(truth))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. Look at the pair"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "fig, ax = plt.subplots(1, 3, figsize=(12, 3))\n",
    "ax[0].imshow(left, cmap='gray'); ax[0].set_title('left')\n",
    "ax[1].imshow(right, cmap='gray'); ax[1].set_title('right')\n",
    "ax[2].imshow(truth, cmap='turbo'); ax[2].set_title('true disparity')\n",
    "for a in ax: a.axis('off')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. The cost of one pixel\n",
    "One pixel on the near box (true d = 20). SAD cost for every shift: the lowest one wins."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "cost = cost_volume(left, right, window=7, max_disp=32)\n",
    "y, x = 60, 120\n",
    "plt.plot(range(32), cost[:, y, x], 'o-')\n",
    "plt.xlabel('shift d (px)'); plt.ylabel('SAD cost')\n",
    "print('best d =', cost[:, y, x].argmin(), ' true d =', truth[y, x])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now try a pixel on the plain wall, such as `y, x = 60, 40`, or on the fence, `y, x = 60, 178`. Why is the minimum hard to trust there?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Window size"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "for w in (3, 7, 15, 21):\n",
    "    s = evaluate(block_matching(left, right, w), truth)\n",
    "    print(f\"window {w:2d}: all {s['all']['correct']:.0%}, plain wall {s['plain wall']['correct']:.0%}, fence {s['fence']['correct']:.0%}, box edges {s['box edges']['correct']:.0%}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. Your turn\n",
    "Turn on the checks. How many pixels stay valid, and how many of those are right?"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "d = block_matching(left, right, window=9, uniqueness=0.15, lr_check=True)\n",
    "s = evaluate(d, truth)\n",
    "print(f\"valid {s['all']['valid']:.0%}, right among valid {s['all']['correct']:.0%}\")\n",
    "plt.imshow(np.where(d < 0, np.nan, d), cmap='turbo'); plt.colorbar(label='disparity (px)')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}