Improve demo notebook narrative

"You should notice" callouts

"In the JupyterGIS GUI..." callouts

More prompts for UI actions

Better basemap

TODO: Visualizing correlation

... and more.

Co-authored-by: Maryam Hosseini <[email protected]>

Author: mfisher87

modules/06-geojupyter/.gitignore

@@ -0,0 +1 @@
+*.jGIS

modules/06-geojupyter/demo.ipynb

@@ -25,7 +25,12 @@
     "\n",
     "You can [🔗try it right now in JupyterLite](https://jupytergis.readthedocs.io/en/latest/lite/lab/)!\n",
     "\n",
-    "Let's explore some functionality together (based on [🔗Carl Boettiger](https://ourenvironment.berkeley.edu/people/carl-boettiger)'s [🔗ESPM-288 course](https://espm-288.carlboettiger.info/)). We'll explore whether neighborhoods that were highly-rated (A) under the disciminatory 1930s practice of [🔗redlining](https://en.wikipedia.org/wiki/Redlining) are greener today than neighborhoods graded D?"
+    "Let's explore some functionality together (based on [🔗Carl Boettiger](https://ourenvironment.berkeley.edu/people/carl-boettiger)'s [🔗ESPM-288 course](https://espm-288.carlboettiger.info/)). We'll explore the question: are neighborhoods that were highly-rated (A) under the disciminatory 1930s practice of [🔗redlining](https://en.wikipedia.org/wiki/Redlining) are greener today than neighborhoods graded D?\n",
+    "\n",
+    ":::{warning}\n",
+    "Please be aware that JupyterGIS is a young project and it's likely you'll run in to bugs.\n",
+    "We encourage you to try to break it and [report issues you find on GitHub](https://github.com/geojupyter/jupytergis/issues)!\n",
+    ":::"
    ]
   },
   {
@@ -72,15 +77,26 @@
    "source": [
     "from jupytergis import GISDocument\n",
     "\n",
-    "jgis_project = GISDocument()\n",
+    "jgis_project = GISDocument(\"new_haven_redlining_analysis.jGIS\")\n",
     "jgis_project.add_raster_layer(\n",
-    "    url=\"https://tile.openstreetmap.org/{z}/{x}/{y}.png\",\n",
+    "    url=\"https://a.basemaps.cartocdn.com/dark_all/{z}/{x}/{y}@2x.png\",\n",
     "    name=\"Basemap\",\n",
     ")\n",
     "\n",
+    "# Open this in a new panel on the right\n",
     "jgis_project.sidecar()"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "40c7da62-6405-4bb7-8c81-ff950db59ff3",
+   "metadata": {},
+   "source": [
+    "You should see a map on the right with a simple dark basemap.\n",
+    "\n",
+    "Try closing or shrinking the file browser on the left to make more room!"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "b2133145-ef92-4d5e-84b8-a53546ef1fe1",
@@ -108,14 +124,30 @@
     "    con\n",
     "    .read_geo(\"/vsicurl/http://dsl.richmond.edu/panorama/redlining/static/mappinginequality.gpkg\")\n",
     "    .filter(_.city == \"New Haven\", _.residential)\n",
+    "    # Add a numeric grade to provide a scalar value for JupyterGIS symbology\n",
+    "    # \"A\" is ascii value 65, so we subtract 64 to start with 1.\n",
+    "    # TODO: Do this in Pandas not DuckDB! (.astype('category').cat.codes)\n",
+    "    # Or: https://pandas.pydata.org/docs/reference/api/pandas.Series.map.html#pandas.Series.map\n",
+    "    .mutate(numeric_grade=_.grade.ascii_str() - 64)\n",
     ")\n",
     "\n",
     "new_haven_redlining =  redlines.execute().set_crs(\"EPSG:4326\")\n",
-    "# TODO: Set a numeric grade for graduated symbology\n",
+    "new_haven_bbox = new_haven_redlining.total_bounds\n",
     "\n",
     "new_haven_redlining.to_file(INEQUALITY_GEOJSON_FILE, engine=\"fiona\")\n",
+    "new_haven_redlining.head()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "d5b8285d-b3c3-4dc8-b6dd-6c2c654ea806",
+   "metadata": {},
+   "source": [
+    "You should notice that 5 rows of data are shown above. They're all residential neighborhoods with `grade` \"A\", meaning the neighborhoods were mostly occupied by white citizens. This grade was used to discriminate against non-white people seeking home loans.\n",
+    "\n",
+    "There is a `fill` column containing a hexadecimal color code, and a `geom` column containing polygon shapes.\n",
     "\n",
-    "new_haven_bbox = new_haven_redlining.total_bounds"
+    "There's an additional `numeric_grade` column which contains a number-encoded version of the `grade` column."
    ]
   },
   {
@@ -125,7 +157,7 @@
    "source": [
     "#### Explore the data\n",
     "\n",
-    "Let's explore the data a little bit. After running the cell below, **right-click the \"New Haven neighborhood redlining\" layer** in the JupyterGIS interface, and **select \"Zoom to layer\"**. "
+    "Let's explore the data a little bit in JupyterGIS. First, add the data to the map:"
    ]
   },
   {
@@ -141,16 +173,49 @@
     ");"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "d10bc86d-5c57-4b81-b52b-731bcd153059",
+   "metadata": {},
+   "source": [
+    "You should notice that the layers panel on the left of the JupyterGIS UI contains a new layer: \"New Haven neighborhood redlining\"."
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "7663ab54-2d69-479b-b8ce-cbdc0bc13143",
    "metadata": {},
    "source": [
+    ":::{important} In the JupyterGIS UI...\n",
+    "After running the cell above, **right-click the \"New Haven neighborhood redlining\" layer** in the JupyterGIS interface, and **select \"Zoom to layer\"**. \n",
+    "\n",
     "Now, with the \"New Haven neighborhood redlining\" layer selected, **click the `i` (identify) icon in the toolbar** at the top of the JupyterGIS interface.\n",
     "\n",
     "Select some neighborhoods and view their \"Grade\" and \"Category\" attributes.\n",
+    ":::"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "37e363b6-8fbc-4cf0-aa77-1d099872fca7",
+   "metadata": {},
+   "source": [
+    "#### Configure symbology\n",
+    "\n",
+    "This dataset has a \"fill\" attribute which determines which color each polygon should be rendered with based on its grade.\n",
+    "\n",
+    "::::::{important} In the JupyterGIS UI...\n",
     "\n",
-    "TODO: Symbologize on numeric grade"
+    ":::{warning} Beware of bugs!\n",
+    "The symbology menu is a bit fragile. Stick to the instructions and you'll be OK! If anything goes wrong, try removing the layer (right-click) and re-adding it.\n",
+    ":::\n",
+    "\n",
+    "Right-click the layer \"New Haven neighborhood redlining\" and select **Edit Symbology**. This symbology mode renders each feature based on a data attribute containing a color.\n",
+    "\n",
+    "**Select \"Canonical\"** from the \"Render Type\" menu. The \"Value\" field should contain the \"fill\" attribute already, so **click OK**.\n",
+    "\n",
+    "You should notice that the neighborhoods ranked \"A\" are green, \"B\" are blue, \"C\" are yellow, and \"D\" are red.\n",
+    "::::::"
    ]
   },
   {
@@ -198,11 +263,19 @@
     "    bbox=new_haven_bbox,\n",
     "    resolution=10,\n",
     "    groupby=\"solar_day\",\n",
-    "    chunks = {},  # this tells odc to use dask\n",
+    "    chunks = {},\n",
     ")\n",
     "data"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "bb7cfddd-8a32-447c-97f2-823a86b5531f",
+   "metadata": {},
+   "source": [
+    "You should notice that the two requested data variables (near-infrared and red) are present in the xarray DataSet."
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "81a2b33a-117d-411b-8f73-4dd873252738",
@@ -252,7 +325,7 @@
     "    \"EPSG:4326\",\n",
     ").rio.to_raster(\n",
     "    raster_path=NDVI_GEOTIFF_FILE, \n",
-    "    driver=\"COG\",\n",
+    "    driver=\"COG\",  # Cloud-Optimized GeoTIFF\n",
     ")"
    ]
   },
@@ -264,7 +337,13 @@
     "#### Explore the data\n",
     "\n",
     "Let's explore the data in JupyterGIS again.\n",
-    "This time, we'll add the layer with the GUI.\n",
+    "This time, we'll add the NDVI data as a layer with the GUI.\n",
+    "\n",
+    "::::::{important} In the JupyterGIS UI...\n",
+    "\n",
+    ":::{warning} Beware of bugs!\n",
+    "The symbology menu is a bit fragile. You may see some things that look a little weird, but stick to the instructions and you'll be OK! If anything goes wrong, try removing the layer (right-click) and re-adding it.\n",
+    ":::\n",
     "\n",
     "If the \"identify\" tool is still active, click the `i` icon in the toolbar again to disable it.\n",
     "\n",
@@ -281,21 +360,30 @@
     "\n",
     "Scroll down to **input the layer name as \"NDVI\"**.\n",
     "\n",
+    "**Click \"OK\"** to add the layer to the map.\n",
+    "\n",
+    "You should notice that the data doesn't look right -- it's entirely black-and-white.\n",
+    "\n",
+    "We need to set up the symbology now.\n",
+    "\n",
     "Finally, **right-click the \"NDVI\" layer** and **select \"Edit Symbology\"**. The symbology menu may take a moment to load. Be patient! **Select \"Classify\" then click \"OK\".**\n",
     "\n",
     "The brighter areas have a higher NDVI value, and the darker areas have a lower one.\n",
     "\n",
-    "We can use the identify tool (`i` icon in the toolbar) to explore the raw values."
+    "We can use the identify tool (`i` icon in the toolbar) to explore the raw values.\n",
+    "\n",
+    "You might notice that the colored polygons show through in areas where the NDVI is `NaN`. You can hide that layer with the 👁️ icon in the JupyterGIS layers panel.\n",
+    "::::::"
    ]
   },
   {
    "cell_type": "markdown",
    "id": "41a3f550-04da-4170-9e69-b4131d428fdd",
    "metadata": {},
    "source": [
-    "### Calculating mean NDVI for each New Haven neighborhood\n",
+    "### Zonal Statistics: Calculating mean NDVI for each New Haven neighborhood\n",
     "\n",
-    "To find out whether neighborhoods graded \"A\" are greener than neighborhoods graded \"D\", we'll calculate the mean NDVI for each neighborhood using [🔗exactextract](https://isciences.github.io/exactextract/background.html), which is known for its capability to include fractional grid cells in its calculation (as opposed to other tools, where a cell is binary, either in or out)."
+    "To find out whether neighborhoods graded \"A\" are greener than neighborhoods graded \"D\", we'll calculate the mean NDVI for each neighborhood using [🔗exactextract](https://isciences.github.io/exactextract/background.html), which is known for its capability to include fractional grid cells in its calculation (as opposed to other tools, where a cell is binary, either inside or outside the polygon)."
    ]
   },
   {
@@ -310,15 +398,25 @@
     "new_haven_redlining_and_ndvi = exact_extract(\n",
     "    NDVI_GEOTIFF_FILE,\n",
     "    new_haven_redlining,\n",
-    "    \"mean_ndvi=mean\",\n",
+    "    \"mean_ndvi=mean\",  # Give the column a human-readable name!\n",
     "    include_geom = True,\n",
     "    include_cols=[\"label\", \"grade\", \"city\", \"fill\"],\n",
     "    output=\"pandas\",\n",
     ")\n",
     "\n",
     "new_haven_redlining_and_ndvi.set_crs(\n",
     "    \"EPSG:4326\"\n",
-    ").to_file(INEQUALITY_NDVI_GEOJSON_FILE, engine=\"fiona\")"
+    ").to_file(INEQUALITY_NDVI_GEOJSON_FILE, engine=\"fiona\")\n",
+    "\n",
+    "new_haven_redlining_and_ndvi.head()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "2d9542f4-131a-4fb4-9328-7b7a2981510b",
+   "metadata": {},
+   "source": [
+    "You should notice that the dataset now includes a `mean_ndvi` column with the results of our zonal statistics calculation! 🎉"
    ]
   },
   {
@@ -328,10 +426,44 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# jgis_project.add_geojson_layer(\n",
-    "#     path=INEQUALITY_NDVI_GEOJSON_FILE,\n",
-    "#     name=\"New Haven neighborhood redlining w/ NDVI\",\n",
-    "# )  # TODO"
+    "jgis_project.add_geojson_layer(\n",
+    "    path=INEQUALITY_NDVI_GEOJSON_FILE,\n",
+    "    name=\"New Haven neighborhood redlining w/ NDVI\",\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "7a1d4e12-9363-4625-9001-bc6aef655a7f",
+   "metadata": {},
+   "source": [
+    "Notice that there are now two copies of the New Haven neighborhoods polygon layer. The new one is titled the same as the old one with \"w/ NDVI\" appended.\n",
+    "\n",
+    "::::::{important} In the JupyterGIS UI...\n",
+    "\n",
+    ":::{warning} Beware of bugs!\n",
+    "If the new redlining w/ NDVI layer disappears, remove it from the map by right-clicking it in the JupyterGIS layers panel and selecting \"Remove Layer\". Then re-run the cell above.\n",
+    ":::\n",
+    "\n",
+    "Temporarily hide the \"NDVI\" raster layer with the 👁️ icon in the JupyterGIS layers panel so you can more clearly see the new layer.\n",
+    "\n",
+    "Hide the \"old\" copy of the redlining layer (\"New Haven neighborhood redlining\") with the 👁️ icon in the JupyterGIS layers panel.\n",
+    ":::::::\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "0e66de07-3861-414e-b2e0-d3d320d4673c",
+   "metadata": {},
+   "source": [
+    "### Visualizing correlation\n",
+    "\n",
+    "Now that we have neighborhood grades and mean NDVI for each neighborhood, how do we visualize any potential correlation?\n",
+    "\n",
+    "TODO: Bivariate choropleth?\n",
+    "\n",
+    "TODO: ?"
    ]
   },
   {