{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction\n",
    "\n",
    "This notebook contains all of the code from the corresponding post on the [One Codex Blog](https://www.onecodex.com/blog/2019/05/01/onecodex-library-for-data-viz/). These snippets are exactly what are in the blog post, and let you perfectly reproduce those figures.\n",
    "\n",
    "This is meant to be a starting off point for you to get started analyzing your own samples. You can copy this notebook straight into your account using the button in the header. To \"run\" or execute a cell, just hit `Shift + Enter`. A few other resources you may find useful include: notes on getting started with [our One Codex library](https://github.com/onecodex/onecodex); the [full documentation on our API](https://docs.onecodex.com) (more technical); a cheat sheet on [getting started with Pandas](http://nbviewer.jupyter.org/github/pybokeh/ipython_notebooks/blob/master/pandas/PandasCheatSheet.ipynb), a Python library for data manipulation; and [reading a few of our blog posts](https://www.onecodex.com/blog/) (where we plan to have nice demos with these notebooks). As always, also feel free to send us questions or suggestions by clicking the chat icon in the bottom right!\n",
    "\n",
    "Now we're going to dive right in and start crunching some numbers!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Fetching data\n",
    "\n",
    "To get started, we create an instance of our API, grab the DIABIMMUNE project, and download 500 samples from the cohort."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from onecodex import Api\n",
    "\n",
    "ocx = Api()\n",
    "\n",
    "project = ocx.Projects.get(\"d53ad03b010542e3\")  # get DIABIMMUNE project by ID\n",
    "samples = ocx.Samples.where(project=project.id, public=True, limit=50)\n",
    "\n",
    "samples.metadata[[\n",
    "    \"gender\",\n",
    "    \"host_age\",\n",
    "    \"geo_loc_name\",\n",
    "    \"totalige\",\n",
    "    \"eggs\",\n",
    "    \"vegetables\",\n",
    "    \"milk\",\n",
    "    \"wheat\",\n",
    "    \"rice\",\n",
    "]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Question #1: How does alpha diversity vary by sample group?\n",
    "\n",
    "Here, we display observed taxa, Simpson’s Index, and Shannon diversity side-by-side, grouped by the region of birth. Each group includes samples taken across the entire three-year longitudinal study.\n",
    "\n",
    "**Note:** Shannon diversity is calculated using log base 2 instead of base ``e`` (natural log)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "observed_taxa = samples.plot_metadata(vaxis=\"observed_taxa\", haxis=\"geo_loc_name\", return_chart=True)\n",
    "simpson = samples.plot_metadata(vaxis=\"simpson\", haxis=\"geo_loc_name\", return_chart=True)\n",
    "shannon = samples.plot_metadata(vaxis=\"shannon\", haxis=\"geo_loc_name\", return_chart=True)\n",
    "\n",
    "observed_taxa | simpson | shannon"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from onecodex.notebooks.report import *\n",
    "\n",
    "ref_text = 'Roo, et al. \"How to Python.\" Nature, 2019.'\n",
    "legend(f\"Alpha diversity by location of birth{reference(text=ref_text, label='roo1')}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Question #2: How does the microbiome change over time?\n",
    "\n",
    "The `plot_metadata` function can\n",
    "search through all taxa in your samples and pull out read counts or relative abundances."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "samples.plot_metadata(haxis=\"host_age\", vaxis=\"Bacteroides\", plot_type=\"scatter\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Question #3: How does an individual subject's gut change over time?\n",
    "\n",
    "Here, we're going to drop into a dataframe, slice it to fetch all the data points from a single subject of the study, and generate a stacked bar plot. It's nice to see the expected high abundance of Bifidobacterium early in life, giving way to Bacteroides near age three!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# generate a dataframe containing relative abundances\n",
    "df_rel = samples.to_df(rank=\"genus\")\n",
    "\n",
    "# fetch all samples for subject P014839\n",
    "subject_metadata = samples.metadata.loc[samples.metadata[\"host_subject_id\"] == \"P014839\"]\n",
    "subject_df = df_rel.loc[subject_metadata.index]\n",
    "\n",
    "# put them in order of sample date\n",
    "subject_df = subject_df.loc[subject_metadata[\"host_age\"].sort_values().index]\n",
    "\n",
    "# you can access our library using the ocx accessor on pandas dataframes!\n",
    "subject_df.ocx.plot_bargraph(\n",
    "    rank=\"genus\",\n",
    "    label=lambda metadata: str(metadata[\"host_age\"]),\n",
    "    title=\"Subject P014839 Over Time\",\n",
    "    xlabel=\"Host Age at Sampling Time (days)\",\n",
    "    ylabel=\"Relative Abundance\",\n",
    "    legend=\"Genus\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Question #4: Heatmaps?!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_rel[:30].ocx.plot_heatmap(legend=\"Relative Abundance\", tooltip=\"geo_loc_name\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Question #5: How do samples cluster?\n",
    "\n",
    "First up, we'll plot a heatmap of weighted UniFrac distance between the first 30 samples in the dataset. This requires unnormalized read counts, so we'll generate a new, unnormalized dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# generate a dataframe containing read counts\n",
    "df_abs = samples.to_df()\n",
    "\n",
    "df_abs[:30].ocx.plot_distance(metric=\"weighted_unifrac\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Question #6: Can I do PCA?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "samples.plot_pca(color=\"geo_loc_name\", size=\"Bifidobacterium\", title=\"My PCoA Plot\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Question #6: Can I do something _better_ than PCA?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "samples.plot_mds(\n",
    "    metric=\"weighted_unifrac\", method=\"pcoa\", color=\"geo_loc_name\", title=\"My PCoA Plot\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "page_break()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bibliography()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  },
  "onecodex_notebook": true
 },
 "nbformat": 4,
 "nbformat_minor": 1
}