My neuroscience work has concentrated on simulating and visualizing the response of ensembles of neurons, using techniques that allow neurobiologically accurate representations to be computed efficiently. I have been collaborating with several neurophysiologists to build real-time visualizations of ensemble auditory responses to sound.

The above diagram shows a highly simplified computational view of the auditory pathway, emphasizing the auditory brainstem. The regions marked in green (Cochlea, MC=Multipolar Cells, SBC=Spherical Bushy Cells, MSO=Medial Superior Olive, AC=Auditory Cortex) indicate regions for which I have produced movies of responses. A few stills are shown below.

Several of my neuroscience collaborators have asked for animations and drawings for use in their introductory lectures.

• Cochlear Traveling Waves (200 kByte GIF). This one shows cochlear traveling waves in response to a sine wave. The top wave is the basilar membrane displacement, and the bottom one is fluid pressure, computed in a 2-D model.
• Multipolar Cell response to Tone Sweep (1.2 MByte QuickTime).
• Multipolar Cell response to Piano Arpeggio (1.2 MByte QuickTime).
• Multipolar Cell response to Speech (1.2 MByte QuickTime).
• All three of the above videos in one..
• My thesis (2.2 MByte PDF) has lots of drawings of cochlear anatomy that may be useful. If you use the figures in a lecture, I would appreciate a verbal acknowledgment.

Permission is granted to use these animations in educational lectures, as long as the title page with the light-grey link to this site is displayed and a verbal acknowledgment is given. Feel free to distribute or send a link to this page to colleagues who might also want to use them. If you want to incorporate them in a book/movie/documentary/(anything but a lecture), please contact me for written permission.

I have many other examples, including spatial localization and familiar music such as Star Trek, THX movie trailer etc., but the files are very large and there are copyright issues if I post them on the web. Buy me a beer and I'll gladly give you a seminar.

In January, 2000, I built the world's first high-resolution (240-tap 10-octave) real-time implementation of a cochlear model using FPGA technology. The figure below shows the immediate response of a cochlear pressure wave to a piano note.

The cochlear model is running in a PC on Interval's custom supercomputing platform based on Xilinx Field-Programmable Gate Arrays (FPGAs).

In June 2000, I formed Audience, Inc. (originally known as Applied Neurosystems Corporation) to continue and commercialize this work.

So, why am I working on this? Here's a figure from Ray Kurzweil's book, The Age of Spiritual Machines (1999), that is relevant and inspiring. In the next 10 years, computers will be capable of performing computations fast enough, and have enough memory, to be able to perform at the level of a mouse, which is pretty darn good -- mice can see and hear in stereo, navigate unknown environments, find food, interact socially, etc. The hard part for us will be figuring out what algorithms to run on these amazing computers of 2010. That's why I'm working on reverse engineering the brain now. In 10 years, the silicon will be ready to do amazing, brain-like things.

At Caltech, I developed a fast event-driven simulator, called Spike, for simulating large networks of simple spiking neurons. You can enter your neural circuit graphically or textually, and view the simulation output logic-analyzer-style, as shown below.

The above circuit is an Adapting Tonic Burster. That is, it has bursting response based on a tonic input current, and it has an inhibitory feedback path that causes its firing rate during a burst to decrease, or adapt. The circuit demonstrates the use of the summating synapse feature of Spike, which in this case is used to model a calcium-dependent potassium channel, to create the adapting behavior.

The above circuit is a locust walking circuit developed by Sylvie Ryckebusch in 1991. This was the first use of Spike to model a real biological circuit.