My Research

Neural Network Loss Surfaces

At the Redwood Center for Theoretical Neuroscience, I studied the loss surfaces of artificial neural networks, which underlie many recent advances in machine learning and AI.

My goal is to understand how the geometric properties of these surfaces, like their curvature, help give rise to the unreasonable effectiveness of neural networks. Check out this arXiV paper for an in-depth comparison of numerical algorithms for computing these quantities and this tutorial for a “derivation” of the best-performing algorithm as a slight upgrade to a method for computing square roots.

The end result of this research was the discovery of an interesting new class of points on neural network loss surfaces, gradient-flat points, which cause trouble for second-order methods. Check out this arXiV paper for the details.

The Development of Spontaneous Activity

In the laboratory of Jason MacLean, I studied the development of spontaneous activity patterns in brain slices taken from mice during the first two weeks of life.

I published my research in this paper. In short, we discovered that spontaneous activity starts out only recruiting very small clusters of cells, only a hundred microns across. The activations then suddenly blossom nine days after birth into a complicated mixture of small clusters of cells, giant waves, and medium-sized groups of cells spread over about a millimeter. After the mouse is two weeks old, the activity becomes more stereotyped again, but now all of the activations are medium-sized and spread out.

This research catalogued these spontaneous patterns at a novel scale and in brain areas that hadn’t been previously studied. Understanding the properties of these patterns is important for understanding how the brain develops its wiring and therefore all of its functions, including sensation, learning, and action. Our findings corroborated other research that suggests that much of this wiring process is innate and independent of sensory input, and further showed that the process proceeds at the same pace in multiple areas of the brain.

MDMA and Social Cognition in Human Subjects

As an undergraduate, I worked on an honors thesis in the laboratory of Harriet de Wit. There, I studied the effects of MDMA, also known as “Ecstasy” or “Molly”, on the cognitive responses of human beings to simulated social rejection and acceptance.

The thesis was published in this paper. We showed that research participants reported increased negative emotions, including reduced self-esteem, when exposed to a social rejection simulation called “Cyberball”, pictured above. When the participants were given MDMA, however, they were less saddened by the social rejection simulation and more accurately reported the magnitude of their rejection.

This research provided useful information about the psychological mechanism behind the effects of MDMA. This is particularly helpful for the growing movement of physicians and scientists who aim to develop MDMA-assisted psychotherapy, particularly for post-traumatic stress disorder (PTSD). Our research indicated that humans exposed to MDMA weren’t simply “getting high” and therefore becoming blunted to negative emotions, but were actually better-equipped to perceive and deal with a minor negative social experience.