Posts

Gaussians as a LogLinear Family

Short Paper on Square Roots and Critical Points
In the next section, we define an analogous algorithm for finding critical points. That is, we again try to solve a rootfinding problem with NewtonRaphson, but this introduces a division, which we reformulate as an optimization problem.
Today a short paper I wrote posted to the arXiV. It’s on a cute connection between the algorithm I use to find the critical points of neural network losses and the algorithm used to compute square roots to high accuracy.
Check out this Twitter thread for a laymanfriendly explanation.

Tails You Win, One Tail You Lose
I presented the math for this at the #cosyne19 diversity lunch today.
— Megan Carey (@meganinlisbon) March 2, 2019
Success rates for first authors with known gender:
Female: 83/264 accepted = 31.4%
Male: 255/677 accepted = 37.7%
37.7/31.4 = a 20% higher success rate for men https://t.co/u2sF5WHHmyControversy over hypothesis testing methodology encountered in the wild a second time! At this year’s Computational and Systems Neuroscience conference, CoSyNE 2019, there was disagreement over whether the acceptance rates indicated bias against women authors. As it turns out, part of the disupute turned over which statistical test to run!

Multiplication Made Convoluted, Part II: Python
import numpy as np class DecimalSequence(): def __init__(self, iterable): arr = np.atleast_1d(np.squeeze(np.asarray(iterable, dtype=np.int))) self.arr = arr def multiply(self, other): return DecimalSequence(np.convolve(self.arr, other.arr))

Multiplication Made Convoluted, Part I: Math
Well, actually, this is more right than you think:
— Yann LeCun (@ylecun) October 20, 2018
A multiplication *is* a convolution of one multidigit number by another one over the digit dimension.
Think about it. 
Fréchet Derivatives 4: The Determinant

Fréchet Derivatives 3: Deep Linear Networks

Google Colab on Neural Networks
The core ideas that make up neural networks are deceptively simple. The emphasis here is on deceptive.
For a recent talk to a group of undergraduates interested in machine learning, I wrote a short tutorial on what I think are the core concepts needed to understand neural networks in such a way that they can be understood by someone with no more than high school mathematics and a passing familiarity with programming.
Thanks to the power of the cloud, you can check out the tutorial and run the examples. Just check out this link. This time, I chose to use Google’s “Colaboratory”, which is like Google Drive for Jupyter notebooks.

Functors and Film Strips

Use You Jupyter Notebook For Great Good
As part of this year’s Data Science Workshop at Berkeley, I put on a tutorial on using Jupyter Notebooks: a quick sprint over the basics and then examples for inline animations and videos, embedded iframes, and interactive plotting!
Click the badge above to launch the tutorial on binder. You’ll want to check out the
JupyterNotebookForGreatGood
folder.Check out the repo it works off of here, where you can find local installation instructions.

Hypothesis Testing
$$2+2=5$$ $$2+2\neq5$$ Tails $$0$$ $$0.5$$ Heads $$0$$ $$0.5$$ To celebrate the latest stable version of Applied Statistics for Neuroscience, here’s a tutorial on hypothesis testing, based on the lecture notes for the course. Make sure to check out the whole course if you liked this snippet!

Fréchet Derivatives 2: Linear Least Squares

Fréchet Derivatives 1: Introduction

How Long is a Matrix?

Hypothesis Testing in the Wild
Apart from being an interesting exercise in the reallife uses of probability, this example, with its massive gap between the true negative rate and the negative predictive value, highlights the importance of thinking critically (and Bayesianly) about statistical evidence.

A Differential Equations View of the Gaussian Family

The Surprise Game

Mixture Models and Neurotransmitter Release
import numpy as np def generate_number_releases(size=1): return np.random.poisson(lam=2.25, size=size) def generate_measured_potentials(size=1): release_counts = generate_number_releases(size=size) measured_potentials = [generate_measured_potential(release_count) for release_count in release_counts] return np.asarray(measured_potentials) def generate_measured_potential(release_count): measured_potential = np.sum(0.4 + 0.065*np.random.standard_normal(size=release_count)) return measured_potential

Tutorial on Linear Generative Models
Inspired by the discussion on linear factor models in Chapter 13 of Deep Learning by Courville, Goodfellow, and Bengio, I wrote a tutorial notebook on linear generative models, including probabilistic PCA, Factor Analysis, and Sparse Coding, with an emphasis on visualizing the data that is generated by each model.
You can download the notebook yourself from GitHub or you can click the badge below to interact with it in your browser without needing a compatible Python computational environment on your machine.

Graphical Model View of Discrete Channels

Linear Algebra for Neuroscientists

Tutorial Notebooks on Machine Learning in Python
Head to this GitHub link to check out a collection of educational Jupyter notebooks that I cowrote as part of a workshop on data science.

Convolution Tutorials Redux
Previously, I posted a link to some Jupyterbased tutorials on convolution that I wrote. In order to use them, you needed to install an appropriate computing environment.
Now, thanks to the folks at binder and the magic of the cloud, you can just click this link and use them with nothing more than a web browser.
Neat!

Statistics in One Sentence
Statistics is the study of pushforward probability measures from a probability space of datasets to a measurable space of statistics under maps that we call statistical procedures.

Paper in Print at Journal of Neurophysiology
To evaluate the developmental ontogeny of spontaneous circuit activity, we compared two different areas of sensory cortex that are also differentiated by sensory inputs that follow different developmental timelines. We imaged neuronal populations in acute coronal slices of mouse neocortex taken from postnatal days 3 through 15. We observed a consistent developmental trajectory of spontaneous activity, suggesting a consistent pattern for cortical microcircuit development: anatomical modules are wired together by coherent activations into functional circuits.
The final version of my research paper with Jason MacLean on the developmental time course of spontaneous activity in mouse cortex is now available through the Journal of Neurophysiology.
Check it out! You can also read a layman’s summary here.

Guest post at BecauseScience
…nature adopts a strategy straight out of Saw II: motor neurons are, from the moment they are born, searching frantically for the antidote to a poison that will kill them when a timer runs out. They are, like Biggie Smalls, born ready to die.
Head to BecauseScience to check out a fun little guest blog post I wrote explaining the process by which neurons and muscles find each other!

What is information theory? What does entropy measure? Mutual information?

Convolution Tutorial IPython Notebooks
I recently gave a tutorial on convolutions. You can check out the IPython Notebooks at the GitHub repo for Berkeley’s Neuro Data Mining Group.
For more information about the group, check out our website. Come join us if you’re interested!

What is Bayes' Rule?
subscribe via RSS