{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tools for Modeling: The Brain Modeling Toolkit and SONATA data format\n",
    "\n",
    "## https://github.com/AllenInstitute/bmtk/tree/develop/docs/tutorial/modeling_tut_2021\n",
    "\n",
    "This tutorial will show how to build and simulate a toy model of the mouse V1 cortical area network using the Brain Modeling Toolkit (BMTK) [\\[Dai et al., PLOS Comp. Bio., 2020\\]](https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1008386). Including how to modify and simulate larger more realistic visual cortical models released by the Allen Institute [\\[Billeh et al., Neuron, 2020\\]](https://www.cell.com/neuron/pdf/S0896-6273(20)30067-2.pdf). The tutorial will also briefly explain the SONATA data format [\\[Dai et al., PLOS Comp. Bio., 2020\\]](https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1007696) and how the BMTK uses the format to describe network models, simulations and results.\n",
    "\n",
    "In this tutorial we will show how to build and simulate a a modistly sized toy network based on the Mouse Primary Visual Cortex (V1) for Layer 4 (which is the primary layer of the L4 for receiving thamalic stimulus). The L4 model [\\[Arkhipov et al., PLOS Comp. Bio., 2018\\]](https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1006535), further described [here](https://portal.brain-map.org/explore/models/l4-mv1), will have a full model that contains 45,000+ cells and millions of connections: \n",
    "\n",
    "<img src=\"images/layer4_side_view.png\" alt=\"Drawing\" style=\"width: 300px; float: left\"/>\n",
    "<img src=\"images/layer4_angled_view.png\" alt=\"Drawing\" style=\"width: 300px; float: left\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Building and running the actual L4 model can take many hours and will require access to HPC resources. The toy model we will build and simulate will have only 300 cells to allow us to build and simulate the model in a few minutes for most desktop and laptop machines.\n",
    "\n",
    "For more information about cortical modeling and simulation at the Allen Institute:\n",
    "https://portal.brain-map.org/explore/models\n",
    "\n",
    "For more information and tutorials on using the BMTK:\n",
    "https://alleninstitute.github.io/bmtk"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Installing BMTK\n",
    "[\\[https://alleninstitute.github.io/bmtk/installation.html\\]](https://alleninstitute.github.io/bmtk/installation.html)\n",
    "\n",
    "BMTK will require python 2.7 or 3.6+. You can clone the latest developement branch from https://github.com/AllenInstitute/bmtk.git. \n",
    "\n",
    "```bash\n",
    " $ https://github.com/AllenInstitute/bmtk\n",
    " $ cd bmtk\n",
    " $ python setup.py install\n",
    "```\n",
    "\n",
    "\n",
    "Alternatively one can install using python's pip utility:\n",
    "\n",
    "```bash\n",
    " $ pip install bmtk\n",
    "```\n",
    "\n",
    "Or if you're using a CONDA environment:\n",
    "\n",
    "```bash\n",
    " $ conda install -c kaeldai bmtk\n",
    "```\n",
    "\n",
    "For simulation [NEURON](https://www.neuron.yale.edu/neuron/) will also be required to be installed (including python bindings). To quickly install NEURON on your machine:\n",
    "```bash\n",
    " $ pip install neuron\n",
    "```\n",
    "\n",
    "or \n",
    "```\n",
    " $ conda install -c conda-forge neuron\n",
    "```\n",
    "\n",
    "\n",
    "#### The BMTK Docker Image\n",
    "\n",
    "We have included a docker image that will come with BMTK, neuron, and a ready-to-use jupyter notebook server with this tutorial and it's content already installed. To run the docker image\n",
    "\n",
    "```bash\n",
    " ## Linux or OSX\n",
    " $ docker run -v $(pwd):/home/shared/workspace -p 8888:8888 alleninstitute/bmtk jupyter\n",
    " \n",
    " ## Windows PowerShell\n",
    " $ docker run -v ${PWD}:/home/shared/workspace -p 8888:8888 alleninstitute/bmtk jupyter\n",
    "```\n",
    "\n",
    "Then open a web browser to __127.0.0.1:8888/__. This current tutorial can be found under the folder *tutorials/modeling_tut_2021/Mouse_L4.ipynb*."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Building the Network\n",
    "\n",
    "<img src=\"images/bmtk_workflow_builder.png\" style=\"width: 700px; float: left\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before we can start simulation we must first build (or receive from a third party) the network model files. To create our toy network of 300 cells in BMTK we will use the *builder* component, which allows us to build heterogeneous, multi-graph networks and save them in the SONATA format.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Building the cells\n",
    "<img src=\"images/builder_add_nodes.png\" style=\"width: 350px; float: left\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first step is figuring out different cell types and parameters we want to use in our network. When we know that we can begin building our network model first by initializing the network using _bmtk.builder.networks.NetworkBuilder_ plus calls to _add_nodes()_ method"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "from bmtk.builder.networks import NetworkBuilder\n",
    "\n",
    "# intialize a new population of cells called \"V1\"\n",
    "v1 = NetworkBuilder('V1')\n",
    "\n",
    "# Add a population of 80 \"Scnn1a\" type nodes\n",
    "v1.add_nodes(\n",
    "    N=80,\n",
    "    \n",
    "    # Reserved SONATA keywords used during simulation\n",
    "    model_type='biophysical',\n",
    "    model_template='ctdb:Biophys1.hoc',\n",
    "    dynamics_params='472363762_fit.json',\n",
    "    morphology='Scnn1a_473845048_m.swc',\n",
    "    model_processing='aibs_perisomatic',\n",
    "    \n",
    "    # The x, y, z locations of each cell in a column\n",
    "    x=np.random.normal(0.0, 20.0, size=80),\n",
    "    y=np.random.uniform(400.0, 500.0, size=80),\n",
    "    z=np.random.normal(0.0, 20.0, size=80),\n",
    "    \n",
    "    # Euler rotations of the cells\n",
    "    rotation_angle_xaxis=np.random.uniform(0.0, 2*np.pi, size=80),\n",
    "    rotation_angle_yaxis=np.random.uniform(0.0, 2*np.pi, size=80),\n",
    "    rotation_angle_zaxis=-3.646878266,\n",
    "    \n",
    "    # Optional parameters\n",
    "    tuning_angle=np.linspace(start=0.0, stop=360.0, num=80, endpoint=False),\n",
    "    pop_name='Scnn1a',\n",
    "    location='L4',\n",
    "    ei='e',\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The above call to add_nodes() will add __N=80__ different individual cells to our network. \n",
    "\n",
    "The attributes __model_type__, __model_template__, __dynamics_params__, __model_processing__ and __morphology__ are SONATA reserved keywords that will instruct BMTK (or other SONATA compliant tools) how to build cell models during the simulation. In this model we are using biophysically realistic models, with dynamical parameters (.json) and morphology (.swc) that we can download from the [Allen Cell Types Database](https://celltypes.brain-map.org/data). We have already downloaded the required json and swc files and have saved them under *components/biophysical_neuron_templates* and \n",
    "*components/morphologies/*, respectively.\n",
    "\n",
    "Attributes __x__, __y__, __z__ are used as the euclidian locations of the cells, while the __rotation_angle__ parameters are used to orientat to rotations of cells. Note that for the x and y rotations we are passing in an array of size N, but the z rotation is defined by a scalar. SONATA and the BMTK Network Builder allows us to differentiate between attributes that are unique to each cell versus shared attributes. Each of the 80 cells will have unique x and y rotations, but share the same pre-calculated z rotatation\n",
    "\n",
    "The final remaining attributes are not reserved SONATA keywords, but rather optional attributes that will be helpful during analysis. The __tuning_angle__ property (how well a cell is tuned for different stimulus grating) is an attribute shown is excitatory cells of the visual cortex and thus wouldn't be applicable to other models or even non-exciatory cells-types\n",
    "\n",
    "We now can ad other cell-types to our model, including inhibitory PV (parvalbumin) cells, with future calls to __add_nodes__:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "v1.add_nodes(\n",
    "    # Rorb excitatory cells\n",
    "    N=80, pop_name='Rorb', location='L4', ei='e',\n",
    "    model_type='biophysical',\n",
    "    model_template='ctdb:Biophys1.hoc',\n",
    "    dynamics_params='473863510_fit.json',\n",
    "    morphology='Rorb_325404214_m.swc',\n",
    "    model_processing='aibs_perisomatic',\n",
    "    x=np.random.normal(0.0, 20.0, size=80),\n",
    "    y=np.random.uniform(400.0, 500.0, size=80),\n",
    "    z=np.random.normal(0.0, 20.0, size=80),   \n",
    "    rotation_angle_xaxis=np.random.uniform(0.0, 2*np.pi, size=80),\n",
    "    rotation_angle_yaxis=np.random.uniform(0.0, 2*np.pi, size=80),\n",
    "    rotation_angle_zaxis=-4.159763785,\n",
    "    tuning_angle=np.linspace(start=0.0, stop=360.0, num=80, endpoint=False),\n",
    ")\n",
    "\n",
    "\n",
    "v1.add_nodes(\n",
    "    # Nr5a1 excitatory cells\n",
    "    N=80, pop_name='Nr5a1', location='L4', ei='e',\n",
    "    model_type='biophysical',\n",
    "    model_template='ctdb:Biophys1.hoc',\n",
    "    dynamics_params='473863035_fit.json',\n",
    "    morphology='Nr5a1_471087815_m.swc',\n",
    "    model_processing='aibs_perisomatic',\n",
    "    x=np.random.normal(0.0, 20.0, size=80),\n",
    "    y=np.random.uniform(400.0, 500.0, size=80),\n",
    "    z=np.random.normal(0.0, 20.0, size=80),   \n",
    "    rotation_angle_xaxis=np.random.uniform(0.0, 2*np.pi, size=80),\n",
    "    rotation_angle_yaxis=np.random.uniform(0.0, 2*np.pi, size=80),\n",
    "    rotation_angle_zaxis=-4.159763785,\n",
    "    tuning_angle=np.linspace(start=0.0, stop=360.0, num=80, endpoint=False),\n",
    ")\n",
    "\n",
    "v1.add_nodes(\n",
    "    # Parvalbuim inhibitory cells, note these don't have a tuning angle and ei=i\n",
    "    N=60, pop_name='PV1', location='L4', ei='i',\n",
    "    model_type='biophysical',\n",
    "    model_template='ctdb:Biophys1.hoc',\n",
    "    dynamics_params='472912177_fit.json',\n",
    "    morphology='Pvalb_470522102_m.swc',\n",
    "    model_processing='aibs_perisomatic',\n",
    "    x=np.random.normal(0.0, 20.0, size=60),\n",
    "    y=np.random.uniform(400.0, 500.0, size=60),\n",
    "    z=np.random.normal(0.0, 20.0, size=60),   \n",
    "    rotation_angle_xaxis=np.random.uniform(0.0, 2*np.pi, size=60),\n",
    "    rotation_angle_yaxis=np.random.uniform(0.0, 2*np.pi, size=60),\n",
    "    rotation_angle_zaxis=-2.539551891\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Creating the connections \n",
    "<img src=\"images/builder_add_edges.png\" style=\"width: 350px; float: left\"/>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have our cells/nodes, we can go ahead and create the synapse/connectivity matrix using the __add_edges__ method. We will want to call this method multiple times to add different connection types with different properties and rules. For each connection type we must select what subpopulation of cells to use for the source (pre-synaptic) and target (post-synaptic) cells. We can select our cells using any combination of attributes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<bmtk.builder.connection_map.ConnectionMap at 0x7fee6d903c88>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "v1.add_edges(\n",
    "    ## Exc --> Inh type connections.\n",
    "    source={'ei': 'e'}, \n",
    "    target={'ei': 'i'},\n",
    "    \n",
    "    # Each source (ei='e') cell will connect once to every target (e) cell\n",
    "    connection_rule=1,\n",
    "              \n",
    "    # SONATA keywords\n",
    "    syn_weight=0.0006,\n",
    "    delay=2.0,\n",
    "    dynamics_params='ExcToInh.json',\n",
    "    model_template='exp2syn',\n",
    "    \n",
    "    # Special BMTK attributes specifying where to set target synape location\n",
    "    target_sections=['somatic', 'basal'],\n",
    "    distance_range=[0.0, 1.0e+20]\n",
    ")\n",
    "\n",
    "v1.add_edges(\n",
    "    ## Inh --> Exc type connections.\n",
    "    source={'ei': 'i'}, \n",
    "    target={'ei': 'e'},\n",
    "    \n",
    "    connection_rule=1,\n",
    "    \n",
    "    syn_weight=0.0002,\n",
    "    delay=2.0,\n",
    "    dynamics_params='InhToExc.json',\n",
    "    model_template='exp2syn',   \n",
    "    target_sections=['somatic', 'basal', 'apical'],\n",
    "    distance_range=[0.0, 50.0]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The __connection_rule__ parameter is used to determine the number of connections/synapses between each source and target cell. In the simplist case like above we can pass in a scalar value. You may also pass in a matrix of shape T x S (T=# or targets, S=# of sources), or use in a custom function.\n",
    "\n",
    "With a user defined function it will pass in a source and target, which can be used as dictionaries to access cell parameters, and expects to return the number of connections\n",
    "```python\n",
    "def <my_connection_rule>(source, target, **opt_params):\n",
    "    return N\n",
    "```\n",
    "\n",
    "For example for Inh --> Inh connection we want there to be no connections if the cells have the same id, otherwise randomly select the number of synapses"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<bmtk.builder.connection_map.ConnectionMap at 0x7fee6d903be0>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def ignore_autapses(source, target):\n",
    "    # No synapses if source == target, otherwise randomize\n",
    "    if source['node_id'] == target['node_id']:\n",
    "        return 0\n",
    "    else:\n",
    "        return np.random.randint(1, 5)\n",
    "\n",
    "v1.add_edges(\n",
    "    ## Inh --> Inh type connections.\n",
    "    source={'ei': 'i'}, \n",
    "    target={'ei': 'i'},\n",
    "    \n",
    "    connection_rule=ignore_autapses,\n",
    "              \n",
    "    syn_weight=0.00015,\n",
    "    delay=2.0,\n",
    "    dynamics_params='InhToInh.json',\n",
    "    model_template='exp2syn',\n",
    "    target_sections=['somatic', 'basal'],\n",
    "    distance_range=[0.0, 1.0e+20]\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For Exc --> Exc we want to test an theory that cells with closer __tuning_angle__ are more likely to be connected with each other:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<bmtk.builder.connection_map.ConnectionMap at 0x7fee6d90e0f0>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def tuning_angle(source, target, max_syns):\n",
    "    # ignore autoapses\n",
    "    if source['node_id'] == target['node_id']:\n",
    "        return 0\n",
    "    \n",
    "    # num of synapses is higher the closer the tuning_angles\n",
    "    src_tuning = source['tuning_angle']\n",
    "    trg_tuning = target['tuning_angle']\n",
    "    dist = np.abs((src_tuning - trg_tuning + 180) % 360 - 180)\n",
    "    p_dist = 1.0 - (np.max((dist, 10.0)) / 180.0)\n",
    "    return np.random.binomial(n=max_syns, p=p_dist)\n",
    "\n",
    "v1.add_edges(\n",
    "    source={'ei': 'e'}, \n",
    "    target={'ei': 'e'},\n",
    "    \n",
    "    connection_rule=tuning_angle,\n",
    "    connection_params={'max_syns': 5}, # pass in options to tuning_angle function\n",
    "    \n",
    "    syn_weight=3.0e-05,\n",
    "    delay=2.0,\n",
    "    dynamics_params='ExcToExc.json',\n",
    "    model_template='exp2syn',\n",
    "    target_sections=['basal', 'apical'],\n",
    "    distance_range=[30.0, 150.0],\n",
    "    \n",
    "    # Will allow us to use our own custom functions to set syn_weight during simulation\n",
    "    weight_function='set_syn_weight'\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Other connection parameters, particularly __syn_weight__ and __delay__, are scalar values and will be the same for each connection. There are multiple ways to set such values so it is customized for each connection. We will show one way of doing so later during the simulation stage, using the special BMTK __weight_function__ keyword."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Building the network\n",
    "<img src=\"images/builder_complete_network.png\" style=\"width: 300px; float: left\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally we want to build our network and save the network files in SONATA format into the _network/_ directory. BMTK will intuit the names of the files."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "v1.build()\n",
    "v1.save(output_dir='network')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### External Network\n",
    "\n",
    "BMTK and SONATA gives modelers the options to build different parts of a network piecewise. For example we may want to build models of thalamacortical and higher cortical areas that we can connect to our V1 model.\n",
    "\n",
    "Next we create a separate network of the LGN, an flat area of the Thalamus with exciatory cells that project onto the V1. As before we define our cells, connect them to the v1 network, then build and save them as SONATA files in the _network/_ directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "lgn = NetworkBuilder('LGN')  # Initialize network called 'lgn'\n",
    "\n",
    "lgn.add_nodes(\n",
    "    N=50, \n",
    "    model_type='virtual',\n",
    "    model_template='lgnmodel:sOFF_TF8',\n",
    "    dynamics_params='sOFF_TF8.json',\n",
    "    \n",
    "    # Special keyword used to determine the location and size of unit in \n",
    "    # the receptive field\n",
    "    x=np.random.uniform(0.0, 240.0, 50),\n",
    "    y=np.random.uniform(0.0, 120.0, 50),\n",
    "    spatial_size=1.0\n",
    ")\n",
    "\n",
    "lgn.add_nodes(\n",
    "    N=50, \n",
    "    model_type='virtual',\n",
    "    model_template='lgnmodel:sON_TF8',\n",
    "    dynamics_params='sON_TF8.json',\n",
    "    x=np.random.uniform(0.0, 240.0, 50),\n",
    "    y=np.random.uniform(0.0, 120.0, 50),\n",
    "    spatial_size=1.0\n",
    ")\n",
    "\n",
    "lgn.add_edges(\n",
    "    source=lgn.nodes(), # Select all LGN cells as the sources\n",
    "    target=v1.nodes(ei='e'), # select V1 exc cells are the target\n",
    "              \n",
    "    connection_rule=lambda *_: np.random.randint(0, 5),\n",
    "    syn_weight=0.0003,\n",
    "    delay=2.0,\n",
    "    dynamics_params='LGN_ExcToExc.json',\n",
    "    model_template='exp2syn',\n",
    "    target_sections=['basal', 'apical', 'somatic'],\n",
    "    distance_range=[0.0, 50.0]\n",
    ")\n",
    "\n",
    "lgn.add_edges(\n",
    "    source=lgn.nodes(), # Select all LGN cells as the sources\n",
    "    target=v1.nodes(ei='i'), # select V1 exc cells are the target\n",
    "              \n",
    "    connection_rule=lambda *_: np.random.randint(0, 5),\n",
    "    syn_weight=0.002,\n",
    "    delay=2.0,    \n",
    "    dynamics_params='LGN_ExcToInh.json',\n",
    "    model_template='exp2syn',\n",
    "    target_sections=['basal', 'apical'],\n",
    "    distance_range=[0.0, 1e+20]\n",
    ")\n",
    "\n",
    "lgn.build()\n",
    "lgn.save(output_dir='network')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Unlike the V1 cells, the LGN cells are __model_type=virtual__. In SONATA _virtual_ cells are special type of cell models that can't be recorded from, but can be used to synapse onto and stimulate other cells. There are essentially place-holders for spike trains (that will be set during the simulation).\n",
    "\n",
    "we have given the lgn special dynamics_params and model_template values, however these won't be used during the V1 simulation. There parameter values are used by FilterNet module."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Simulating the Network\n",
    "\n",
    "\n",
    "<img src=\"images/bmtk_workflow_simulator.png\" style=\"width: 700px; float: left\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally we want to build our network and save the network files in SONATA format into the _network/_ directory. BMTK will intuit the names of the files.\n",
    "\n",
    "Now that we have built the SONATA network models files (or recieved the files from another source), we can go ahead and setup a simulation to run. To simulate the multi-compartment cell models we will use the BioNet component of the BMTK - which uses the [NEURON](https://www.neuron.yale.edu/neuron/) simulator libraries for simulating our network. \n",
    "\n",
    "Before we can do that there are still a few things that need to be taken care of, including setting up run-time parameters, choosing the stimulus, and deciding what to record."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Simulation Directory Structure\n",
    "\n",
    "Almost all of the parameters required to initialize and run a full V1 simulation will be stored in a [SONATA base json configuration file](https://github.com/AllenInstitute/sonata/blob/master/docs/SONATA_DEVELOPER_GUIDE.md#tying-it-all-together---the-networkcircuit-config-file), in this case named _config.bionet.json_. If you need to (re)create the file the following command will create a template:\n",
    "\n",
    "```python\n",
    "  python -m bmtk.utils.sim_setup  \\\n",
    "   --network network              \\\n",
    "   --dt 0.1                       \\\n",
    "   --tstop 2000.0                 \\  \n",
    "   --compile-mechanisms           \\\n",
    "   --include-examples             \\\n",
    "   --config=config.bionet.json    \\\n",
    "   bionet .\n",
    "```\n",
    "\n",
    "Opening up the json with a text editor we can see the configuration is divided into different sections\n",
    "\n",
    "\n",
    "#### \"networks\"\n",
    "\n",
    "This section contains paths to the sonata network files we built above. We can use this to add or remove node populations and edges before running another simulation.\n",
    "\n",
    "\n",
    "#### \"components\"\n",
    "\n",
    "Contains the paths to the different external files and parameters required to run a simulation; morphologies, electricophysicological parameters (eg. NeuroML files), synaptic model parameters, etc. Files that are not directly defined as a part of the SONATA files but used by specific simulator software (eg NEURON, NEST).\n",
    "\n",
    "The required files for this tutorial has already be created. In other cases one would likely have to spend a signficant amount of time creating the optimized parameter files.\n",
    "\n",
    "\n",
    "#### \"run\"\n",
    "\n",
    "Run-time parameters and conditions, including simulation time and the time-step (all time values are in miliseconds).\n",
    "\n",
    "\n",
    "#### \"output\"\n",
    "\n",
    "Paths and parameters used for simulation output and logging.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### \"reports\"\n",
    "\n",
    "By default BMTK will record the V1 cell spikes into a file specified under **outputs/spikes_file**. For these types of models we can also record other parameters like membrane potential, calcium fluxuation, or extracellular electrical fields by adding to the \"reports\" section of the configuration file. \n",
    "\n",
    "For example the following can be used to record the soma membrane potential traces for our Scnn1a cells and save them in *output/membrane_potential.h5*\n",
    "\n",
    "```json\n",
    "    \"membrane_potential\": {\n",
    "      \"cells\": {\n",
    "          \"population\": \"V1\", \n",
    "          \"pop_name\": \"Scnn1a\"\n",
    "      },\n",
    "      \"variable_name\": \"v\",\n",
    "      \"module\": \"membrane_report\",\n",
    "      \"sections\": \"soma\"\n",
    "    }\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### \"inputs\"\n",
    "\n",
    "This section can be used to specify one or more stimulus we want to use on our V1 model. Depending on the type of cell models being used, we can use spike trains, current clamps, voltage clamps, or extracellular electrodes.  \n",
    "\n",
    "For our V1 network we created a population of LGN \"virtual\" cells that synapse onto our V1 cells and drive them with spike trains. All that is missing is a \"spike-trains\" file, which can be either a CSV, NWB, or SONATA HDF5 file. BMTK includes way to create spike-train files.\n",
    "\n",
    "The following will create a series of spike trains for the LGN cells using a Poisson distrubtion of 8 Hz:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "from bmtk.utils.reports.spike_trains import PoissonSpikeGenerator\n",
    "\n",
    "psg = PoissonSpikeGenerator(population='LGN')\n",
    "psg.add(\n",
    "    node_ids=range(0, 100),\n",
    "    firing_rate=8.0,    # We can also pass in a nonhomoegenous function/array\n",
    "    times=(0.0, 2.0)    # Firing starts at 0 s up to 2 seconds\n",
    ")\n",
    "\n",
    "psg.to_sonata('inputs/lgn_spikes.poisson.h5')\n",
    "psg.to_csv('inputs/lgn_spikes.poisson.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and update the \"inputs\" section \n",
    "\n",
    "```json\n",
    "    \"LGN_spikes\": {\n",
    "        \"input_type\": \"spikes\",\n",
    "        \"module\": \"h5\",\n",
    "        \"input_file\": \"./inputs/lgn_spikes.poisson.h5\",\n",
    "        \"node_set\": \"LGN\"\n",
    "    }\n",
    "```\n",
    "\n",
    "BMTK also includes a tool FilterNet that allows us to turn movies and images into firing rates using optimized filter models, which we will do in future sections."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Running the simulation\n",
    "\n",
    "Now that we have our configuration files as we need, we can go ahead and run a simulation. We can execute the *run_bionet.py* script using the command line option:\n",
    "\n",
    "```bash\n",
    " $ python run_bionet.py config.bionet.json\n",
    "```\n",
    "\n",
    "Although this toy network is small enough to run most modern computers and laptops, for larger networks we may need to run the program on a cluster with <N> cores. To do so \n",
    "\n",
    "```bash\n",
    " $ mpirun -np <N> nrniv -mpi -python run_bionet.py config.bionet.json\n",
    "```\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2021-06-24 11:23:10,647 [INFO] Created log file\n",
      "2021-06-24 11:23:10,780 [INFO] Building cells.\n",
      "2021-06-24 11:23:35,286 [INFO] Building recurrent connections\n",
      "2021-06-24 11:25:53,734 [INFO] Building virtual cell stimulations for LGN_spikes\n",
      "2021-06-24 11:26:18,614 [INFO] Running simulation for 2000.000 ms with the time step 0.100 ms\n",
      "2021-06-24 11:26:18,615 [INFO] Starting timestep: 0 at t_sim: 0.000 ms\n",
      "2021-06-24 11:26:18,616 [INFO] Block save every 5000 steps\n",
      "2021-06-24 11:27:47,544 [INFO]     step:5000 t_sim:500.00 ms\n",
      "2021-06-24 11:29:18,175 [INFO]     step:10000 t_sim:1000.00 ms\n",
      "2021-06-24 11:30:49,458 [INFO]     step:15000 t_sim:1500.00 ms\n",
      "2021-06-24 11:32:21,530 [INFO]     step:20000 t_sim:2000.00 ms\n",
      "2021-06-24 11:32:21,593 [INFO] Simulation completed in 6.0 minutes, 2.979 seconds \n"
     ]
    }
   ],
   "source": [
    "from bmtk.simulator import bionet\n",
    "\n",
    "conf = bionet.Config.from_json('config.bionet.json')\n",
    "conf.build_env()\n",
    "net = bionet.BioNetwork.from_config(conf)\n",
    "sim = bionet.BioSimulator.from_config(conf, network=net)\n",
    "sim.run()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As specified in the \"output\" and \"reports\" section, the results of the simulation will be saved in different files in the _output_ directory. The spike-trains files, *spikes.h5*, and membrane potentials, *membrane_potential.h5* are [SONATA output data formatted file](https://github.com/AllenInstitute/sonata/blob/master/docs/SONATA_DEVELOPER_GUIDE.md#output-file-formats) and can be read and analyized by any tools that can read SONATA. \n",
    "\n",
    "Using BMTK to display the results:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 64368 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from bmtk.analyzer.spike_trains import plot_raster\n",
    "\n",
    "# Plot the spike trains raster\n",
    "_ = plot_raster(config_file='config.bionet.json', group_by='pop_name')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 35480 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from bmtk.analyzer.compartment import plot_traces\n",
    "\n",
    "# Plot the membrane potential trace\n",
    "_ = plot_traces(config_file='config.bionet.json', group_by='pop_name', report_name='membrane_potential')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Cell perturbations\n",
    "\n",
    "A usefull tool in network neuroscience is the ability to stimuluate or depress a one or more neurons in a network, either using optogenetic tools or current clamps. To simulate this effect in BMTK we can use a current clamp on a multiple neurons by adding the following to the \"inputs\" section of the configuration file:\n",
    "\n",
    "\n",
    "```json\n",
    "  \"inputs\": {\n",
    "    ...\n",
    "     \n",
    "    \"inh_perturbation\": {\n",
    "      \"input_type\": \"current_clamp\",\n",
    "      \"module\": \"IClamp\",\n",
    "      \"node_set\": {\n",
    "        \"population\": \"V1\",\n",
    "        \"pop_name\": \"Nr5a1\", \n",
    "        \"location\": \"L4\"\n",
    "      },\n",
    "      \"amp\": -1.0,\n",
    "      \"delay\": 500.0,\n",
    "      \"duration\": 1500.0\n",
    "    }\n",
    "  }\n",
    "```\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Will reset BioNet and the NEURON simulator. Required if we want\n",
    "# to run the simulator multiple times in the same notebook.\n",
    "bionet.reset()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2021-06-24 11:33:07,574 [INFO] Created log file\n",
      "Mechanisms already loaded from path: ./components/mechanisms.  Aborting.\n",
      "2021-06-24 11:33:07,675 [INFO] Building cells.\n",
      "2021-06-24 11:34:58,720 [INFO] Building recurrent connections\n",
      "2021-06-24 11:37:23,018 [INFO] Building virtual cell stimulations for LGN_spikes\n",
      "2021-06-24 11:37:49,440 [INFO] Running simulation for 2000.000 ms with the time step 0.100 ms\n",
      "2021-06-24 11:37:49,441 [INFO] Starting timestep: 0 at t_sim: 0.000 ms\n",
      "2021-06-24 11:37:49,442 [INFO] Block save every 5000 steps\n",
      "2021-06-24 11:40:57,854 [INFO]     step:5000 t_sim:500.00 ms\n",
      "2021-06-24 11:44:23,932 [INFO]     step:10000 t_sim:1000.00 ms\n",
      "2021-06-24 11:47:32,163 [INFO]     step:15000 t_sim:1500.00 ms\n",
      "2021-06-24 11:50:43,726 [INFO]     step:20000 t_sim:2000.00 ms\n",
      "2021-06-24 11:50:43,783 [INFO] Simulation completed in 12.0 minutes, 54.34 seconds \n"
     ]
    }
   ],
   "source": [
    "conf = bionet.Config.from_json('config.bionet.json')\n",
    "conf.build_env()\n",
    "net = bionet.BioNetwork.from_config(conf)\n",
    "sim = bionet.BioSimulator.from_config(conf, network=net)\n",
    "sim.run()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 58120 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "_ = plot_raster(config_file='config.bionet.json', group_by='pop_name')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Recording ECP\n",
    "\n",
    "The configuration has been set-up to record spike-trains and the membrane potentials of a subset of cells. BioNet also includes the ability to record the extracelluar potential using a simulated electrode, which will allow us to find the ECP and even the local field potential.\n",
    "\n",
    "First we must add the following to the \"inputs\" section\n",
    "\n",
    "```json\n",
    "\"reports\": {\n",
    "  ...\n",
    "   \n",
    "  \"ecp\": {\n",
    "    \"module\": \"extracellular\",\n",
    "    \"variable_name\": \"v\",\n",
    "    \"cells\": {\n",
    "        \"population\": \"V1\",\n",
    "        \"model_type\": \"biophysical\"\n",
    "    },\n",
    "    \"electrode_positions\": \"./components/devices/linear_electrode.csv\",\n",
    "    \"file_name\": \"ecp.h5\",\n",
    "    \"electrode_channels\": \"all\"\n",
    "  }\n",
    "}\n",
    "```\n",
    "\n",
    "The *devices/linear_electrode.csv* file is what we use to specify the electrode and all the recording channels. for this simulation the electrode will be placed through the center of the V1 column with recording channels at every 100 microns (the **x**, **y**, **z** coordinates of the electrode should be in the same space as the cells).\n",
    "\n",
    "<img src=\"images/v1_neuropixel_probe.png\" style=\"width: 350px; float: left\"/> <!-- style=\"width: 5px; float: left\"/> -->\n",
    "\n",
    "| channel | x_pos | y_pos | z_pos |\n",
    "| :--- | :--- | :--- | :--- |\n",
    "|0 | 0.0 | 0 | 0.0 |\n",
    "|1 | 0.0 | 100 | 0.0 |\n",
    "|2 | 0.0 | 200 | 0.0 |\n",
    "|3 | 0.0 | 300 | 0.0 |\n",
    "|4 | 0.0 | 400 | 0.0 |\n",
    "|5 | 0.0 | 500 | 0.0 |\n",
    "|6 | 0.0 | 600 | 0.0 |\n",
    "|7 | 0.0 | 700 | 0.0 |\n",
    "|8 | 0.0 | 800 | 0.0 |\n",
    "|9 | 0.0 | 900 | 0.0\n",
    "\n",
    "\n",
    "\n",
    "Now we can run the simulation as before:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Will reset BioNet and the NEURON simulator. Required if we want\n",
    "# to run the simulator multiple times in the same notebook.\n",
    "bionet.reset()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2021-06-24 12:04:20,572 [INFO] Created log file\n",
      "2021-06-24 12:04:20,722 [INFO] Building cells.\n",
      "2021-06-24 12:04:49,375 [INFO] Building recurrent connections\n",
      "2021-06-24 12:07:39,111 [INFO] Building virtual cell stimulations for LGN_spikes\n",
      "2021-06-24 12:08:06,819 [INFO] Running simulation for 2000.000 ms with the time step 0.100 ms\n",
      "2021-06-24 12:08:06,820 [INFO] Starting timestep: 0 at t_sim: 0.000 ms\n",
      "2021-06-24 12:08:06,820 [INFO] Block save every 5000 steps\n",
      "2021-06-24 12:26:10,664 [INFO]     step:5000 t_sim:500.00 ms\n",
      "2021-06-24 12:43:55,769 [INFO]     step:10000 t_sim:1000.00 ms\n",
      "2021-06-24 13:01:19,182 [INFO]     step:15000 t_sim:1500.00 ms\n",
      "2021-06-24 13:19:58,561 [INFO]     step:20000 t_sim:2000.00 ms\n",
      "2021-06-24 13:19:58,629 [INFO] Simulation completed in 71.0 minutes, 51.81 seconds \n"
     ]
    }
   ],
   "source": [
    "conf = bionet.Config.from_json('config.bionet.json')\n",
    "conf.build_env()\n",
    "net = bionet.BioNetwork.from_config(conf)\n",
    "sim = bionet.BioSimulator.from_config(conf, network=net)\n",
    "sim.run()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 69672 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 10 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from bmtk.analyzer.ecp import plot_ecp\n",
    "\n",
    "_ = plot_ecp(config_file='config.bionet.json', report_name='ecp')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Adjusting the synaptic  weights\n",
    "\n",
    "Currently the synaptic weights, eg __syn_weight__ property, are set to the same for each edge-type. There are multiple ways to make it so the synaptic weights are nonhomogenous. We can calculate and store nonhomogeneous synaptic weights inside the SONATA edges files. Or we can adjust the weights before each simulation - which is a bit less efficent but will give us greater flexiblity as we try to optimize the weights.\n",
    "\n",
    "for Exc --> Exc edges we had a BMTK special keyword __weight_function__='set_syn_weight', which allows us to create user-defined functions to adjust the synaptic weights during the initialization of the simulation. BMTK will look for a function with with signature \n",
    "```python\n",
    "def <function_name>(edge_props, source, target)\n",
    "```\n",
    "\n",
    "Where **edge_props**, **source**, **target** are dictionary-like objects that can be used to fetch edges-properties, source cell properties, and target-cell properties.\n",
    "\n",
    "For our function we want to rewrite it so the __syn_weight__ value is readjusted based on the distance.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Will reset BioNet and the NEURON simulator. Required if we want\n",
    "# to run the simulator multiple times in the same notebook.\n",
    "bionet.reset()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "2021-06-24 14:04:25,042 [INFO] Created log file  \n",
      "2021-06-24 14:04:25,147 [INFO] Building cells.\n",
      "2021-06-24 14:04:49,026 [INFO] Building recurrent connections\n",
      "2021-06-24 14:07:13,852 [INFO] Building virtual cell stimulations for LGN_spikes\n",
      "2021-06-24 14:07:38,980 [INFO] Running simulation for 2000.000 ms with the time step 0.100 ms\n",
      "2021-06-24 14:07:38,980 [INFO] Starting timestep: 0 at t_sim: 0.000 ms\n",
      "2021-06-24 14:07:38,980 [INFO] Block save every 5000 steps\n",
      "2021-06-24 14:09:05,559 [INFO]     step:5000 t_sim:500.00 ms\n",
      "2021-06-24 14:10:31,855 [INFO]     step:10000 t_sim:1000.00 ms\n",
      "2021-06-24 14:11:59,001 [INFO]     step:15000 t_sim:1500.00 ms\n",
      "2021-06-24 14:13:24,835 [INFO]     step:20000 t_sim:2000.00 ms\n",
      "2021-06-24 14:13:24,894 [INFO] Simulation completed in 5.0 minutes, 45.91 seconds \n",
      "\n"
     ]
    }
   ],
   "source": [
    "def set_syn_weight(edge_props, source, target):\n",
    "    # fetch base-line syn_weight value stored in edge_types properties\n",
    "    syn_weight = edge_props[\"syn_weight\"]\n",
    "    \n",
    "    # get the coords for the pre/post-synaptic cell and calculate distance\n",
    "    src_coords = np.array([source['x'], source['y'], source['z']])\n",
    "    trg_coords = np.array([target['x'], target['y'], target['z']])\n",
    "    dist = np.linalg.norm(src_coords - trg_coords)\n",
    "    \n",
    "    if dist > 60: \n",
    "        return syn_weight\n",
    "    else:\n",
    "        return syn_weight*(-0.165*dist + 10.0)\n",
    "\n",
    "\n",
    "# This will register our function so BMTK can find it\n",
    "bionet.add_weight_function(set_syn_weight)\n",
    "    \n",
    "conf = bionet.Config.from_json('config.bionet.json')\n",
    "conf.build_env()\n",
    "net = bionet.BioNetwork.from_config(conf)\n",
    "sim = bionet.BioSimulator.from_config(conf, network=net)\n",
    "sim.run()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 57480 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from bmtk.analyzer.spike_trains import plot_raster\n",
    "\n",
    "_ = plot_raster(config_file='config.bionet.json', group_by='pop_name')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Generating realistic stimulus\n",
    "\n",
    "To drive our network model we have been using external \"LGN\" virtual cells that fire at a constant rate. This can be useful for testing some fundamental properties of the network. However to better optimize our model it will be useful to simulate it against more realistic stimulus.\n",
    "\n",
    "FilterNet is a component of the BMTK that allows us to project an image or movie onto one or more neurons to produce spike trains. Using FilterNet with the \"LGN\" network we can create spike-train files using various types of visual stimulus, which can then be used as input for simulating the V1 model.\n",
    "\n",
    "<img src=\"images/filternet_lnp.png\" alt=\"Drawing\" style=\"width: 700px; float: left\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### FilterNet Directory Structure\n",
    "\n",
    "FilterNet simulation directory is structured the same as BioNet, using a json configuration file to specify the paths and parameters required to run a full simulation. The configuration and model files are already set-up, but when building a simulation from scratch one can create directory template using the command line tool:\n",
    "\n",
    "```bash\n",
    " $ python -m bmtk.utils.sim_setup    \\\n",
    "   --network network                 \\\n",
    "   --tstop 2000.0                    \\  \n",
    "   --include-examples                \\\n",
    "   --config=config.filternet.json    \\\n",
    "   filternet .\n",
    "```\n",
    "\n",
    "Controlling the FilterNet simulation is done through *config.filternet.json*, which has the same format used by _config.bionet.json_, except with some noticeable changes.\n",
    "\n",
    "##### \"networks\"\n",
    "\n",
    "We only want to run FilterNet against the LGN nodes (and FilterNet currently doesn't support recurrent networks), which is reflected by the \"network\" sections in the configuration:\n",
    "\n",
    "```json\n",
    "  \"networks\": {\n",
    "    \"nodes\": [\n",
    "      {\n",
    "        \"nodes_file\": \"$NETWORK_DIR/LGN_nodes.h5\",\n",
    "        \"node_types_file\": \"$NETWORK_DIR/LGN_node_types.csv\"\n",
    "      }\n",
    "    ]\n",
    "  }\n",
    "```\n",
    "\n",
    "#### \"output\"\n",
    "Like in BioNet, FilterNet will produce a SONATA formated spike-trains file, but for our \"LGN\" nodes only. The location of the output is controlled by the \"output\" section of the configuration file:\n",
    "\n",
    "``` json\n",
    "  \"output\": { \n",
    "    \"log_file\": \"log.txt\",\n",
    "    \"output_dir\": \"./inputs\",\n",
    "    \"spikes_file\": \"lgn_spikes.flash.h5\"\n",
    "  },\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### \"inputs\"\n",
    "\n",
    "The \"inputs\" section of the configuration file determines which type of stimulus we want to project onto LGN cells. Although we can use any user-created movie (described below), FilterNet also comes with options for easily creating special stimulus types that are common in experiments testing the visual system.\n",
    "\n",
    "One of the simplest types of stimuli is a full-field flash, a temporary bright (or dark) flash on the receptive field, which can be used to test simple ON/OFF cells found in the LGN. The following will create a bright flash between 500 to 1500 ms seconds\n",
    "\n",
    "```json\n",
    "  \"inputs\": {\n",
    "    \"full_field_flash\": {\n",
    "        \"input_type\": \"movie\",\n",
    "        \"module\": \"full_field_flash\",\n",
    "        \"row_size\": 120,\n",
    "        \"col_size\": 240,\n",
    "        \"t_on\": 500.0,\n",
    "        \"t_off\": 1500.0,\n",
    "        \"max_intensity\": -1.0,\n",
    "        \"frame_rate\": 1000.0\n",
    "    } \n",
    "  }\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Running the Simulation\n",
    "\n",
    "Now that we have our configuration file setup we can run filternet. In the command line this can be done by:\n",
    "    \n",
    "```bash\n",
    " $ python run_filternet.py config.filernet.json\n",
    "```\n",
    "\n",
    "For every large networks that would require multiple processors to complete in a reasonable amount of time we can use the command\n",
    "```bash\n",
    " $ mpirun -n <N> python run_filternet.py config.filernet.json\n",
    "```\n",
    "\n",
    "Or through the following code:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2021-06-24 14:16:06,482 [WARNING] Directory ./inputs already exists.\n",
      "2021-06-24 14:16:06,483 [INFO] Created log file\n",
      "2021-06-24 14:16:07,970 [INFO] Building cells.\n",
      "2021-06-24 14:16:11,337 [INFO] Evaluating rates.\n",
      "2021-06-24 14:16:33,327 [INFO] Done.\n"
     ]
    }
   ],
   "source": [
    "from bmtk.simulator import filternet\n",
    "\n",
    "config = filternet.Config.from_json('config.filternet.json')\n",
    "config.build_env()\n",
    "net = filternet.FilterNetwork.from_config(config)\n",
    "sim = filternet.FilterSimulator.from_config(config, net)\n",
    "sim.run()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once completed it will create the spikes-file lgn_results/lgn_spikes.flash.h5. This file follows the same format as the spike-trains file generated by BioNet. To see how our LGN ON/OFF cells responded to the full-field flash:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 20536 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from bmtk.analyzer.spike_trains import plot_raster\n",
    "\n",
    "_ = plot_raster(config_file='config.filternet.json', group_by='model_template')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Using the results for the V1 model\n",
    "\n",
    "To use the results to test our V1 model with more realistic stimulus, we just need to open _config.bionet.json_ and edit the \"inputs\" section so the __input_file__ is changed from *./inputs/lgn_spikes.h5* to *./lgn_results/lgn_spikes.flash.h5*\n",
    "\n",
    "```json\n",
    "  \"inputs\": {\n",
    "    ...,\n",
    "      \n",
    "    \"LGN_spikes\": {\n",
    "        \"input_type\": \"spikes\",\n",
    "        \"module\": \"h5\",\n",
    "        \"input_file\": \"./inputs/spikes.flash.h5\",\n",
    "        \"node_set\": \"LGN\"\n",
    "    }\n",
    "  }\n",
    "```\n",
    "\n",
    "Then just rereun the BioNet code as above and see what happens when we use the new stimulus."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 23768 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from bmtk.analyzer.spike_trains import plot_raster\n",
    "\n",
    "_ = plot_raster(config_file='config.bionet.flash.json', group_by='pop_name')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### More Stimulus Types\n",
    "\n",
    "Besides Full-Field Flashes, FilterNet also can generate other types of pre-set visual stimuli. However for more advanced stimuli one will have to create their own movies.\n",
    "\n",
    "To do so first you will need to create a [numpy npy or npz file](https://numpy.org/doc/stable/reference/generated/numpy.save.html) - eg a 3-dimensional matrix consisting of the frames x rows x columns of each pixel. FilterNet will gray-scale the images automatically. Once the movie has been saved, open up the FilerNet json configuration file and modify the \"inputs\" section accordingly (make sure frame_rate is correct).\n",
    "\n",
    "```json\n",
    "  \"inputs\": {\n",
    "    \"full_field_flash\": {\n",
    "      \"input_type\": \"movie\",\n",
    "      \"module\": \"movie\",\n",
    "      \"data_file\": \"/path/to/my/movie.npy\",\n",
    "      \"frame_rate\": 1000.0\n",
    "    } \n",
    "  }\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Simulating across different levels of Resultion\n",
    "\n",
    "So far we've been running a simulation using multi-compartment models of cells using the BioNet component of the BMTK. Often in modeling we want to test different types of models, for both computational and scientific reasons. \n",
    "\n",
    "#### Point-neuron models\n",
    "With PointNet, we can run our network using point-neuron based models using the [NEST](https://www.nest-simulator.org/) simulator libraries. PointNet will also use the same SONATA network files and configuration format for describing a network and initializing a simulation, so we can compare the same network across different levels of resolution. However certain properties, like __syn_weight__, __model_template__, and __dynamics_params__ will need to be adjusted. For more information see the [PointNet tutorial](https://alleninstitute.github.io/bmtk/tutorial_pointnet_modeling.html).\n",
    "\n",
    "\n",
    "#### Population based firing rate models\n",
    "With PopNet we can simulate the different cell-types as singular populations using the [DipDE](http://alleninstitute.github.io/dipde/) simulator. Just like BioNet and PointNet it utilizes the SONATA formats. However instead of recording spike-trains for individual cells, PopNet will record the firing-rate dynamics over the course of the simulation. For more information see the [PopNet tutorial](https://alleninstitute.github.io/bmtk/tutorial_population_modeling.html)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Additional Resources\n",
    "\n",
    "https://alleninstitute.github.io/bmtk\n",
    "\n",
    "\n",
    "https://github.com/AllenInstitute/bmtk/tree/develop/docs/tutorial/modeling_tut_2021\n",
    "\n",
    "\n",
    "### Additional Tutorials\n",
    "\n",
    "* **Network Builder** - https://alleninstitute.github.io/bmtk/tutorial_NetworkBuilder_Intro.html\n",
    "\n",
    "* **BioNet** - https://alleninstitute.github.io/bmtk/tutorial_single_pop.html\n",
    "\n",
    "* **FilterNet** - https://alleninstitute.github.io/bmtk/tutorial_filter_models.html\n",
    "\n",
    "* **PointNet** - https://alleninstitute.github.io/bmtk/tutorial_pointnet_modeling.html\n",
    "\n",
    "* **PopNet** - https://alleninstitute.github.io/bmtk/tutorial_population_modeling.html\n",
    "\n",
    "\n",
    "### Examples\n",
    "\n",
    "* https://github.com/AllenInstitute/bmtk/tree/develop/docs/examples\n",
    "\n",
    "\n",
    "\n",
    "### Mouse Layer 4 Model\n",
    "\n",
    "* http://portal.brain-map.org/explore/models/l4-mv1\n",
    "\n",
    "\n",
    "### Mouse Full V1 Model\n",
    "\n",
    "* http://portal.brain-map.org/explore/models/mv1-all-layers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
