GEORGIA INSTITUTE OF TECHNOLOGY SCHOOL OF ELECTRICAL AND COMPUTER ENGINEERING LABORATORY FOR NEUROENGINEERING

Journal Publications

[1]	C.J. Rozell and D.H. Johnson. Resource allocation and control in redundant wireless sensor and actuator networks. In preparation. [ bib ]
[2]	C.J. Rozell, D.H Johnson, R.G. Baraniuk, and B.A. Olshausen. Sparse coding via thresholding and local competition in neural circuits. Neural Computation, 20(10):2526-2563, October 2008. [ bib | .pdf ] While evidence indicates that neural systems may be employing sparse approximations to represent sensed stimuli, the mechanisms underlying this ability are not understood. We describe a local ly competitive algorithm (LCA) that solves a collection of sparse coding principles minimizing a weighted combination of mean-squared error (MSE) and a coefficient cost function. LCAs are designed to be implemented in a dynamical system composed of many neuron-like elements operating in parallel. These algorithms use thresholding functions to induce local (usually one-way) inhibitory competitions between nodes to produce sparse representations. LCAs produce coefficients with sparsity levels comparable to the most popular centralized sparse coding algorithms while being readily suited for neural implementation. Addi- tionally, LCA coefficients for video sequences demonstrate inertial properties that are both qualitatively and quantitatively more regular (i.e., smoother and more predictable) than the coefficients produced by greedy algorithms.
[3]	S.W. Bishnoi, C.J. Rozell, C.S. Levin, M.K. Gheith, B.R. Johnson, D.H. Johnson, and N.J Halas. All-optical nanoscale pH meter. Nano Letters, 6(8):1687-1692, August 2006. [ bib ] We show that an Au nanoshell with a pH sensitive molecular adsorbate functions as a standalone, all-optical nanoscale pH meter that monitors its local environment through the pH-dependent surface enhanced Raman scattering (SERS) spectra of the adsorbate molecules. Moreover, we also show how the performance of such a functional nanodevice can be quantitatively assessed. The complex spectral output is reduced to a simple device characteristic by application of a locally linear manifold approximation algorithm. The average accuracy of the nano-“meter” was found to be ± 0.10 pH units across its operating range.
[4]	C.J. Rozell and D.H. Johnson. Analyzing the robustness of redundant population codes in sensory and feature extraction systems. Neurocomputing, 69(10-12):1215-1218, June 2006. Also appears in Proceedings of the Computational Neuroscience Meeting (CNS), Madison, WI, July 2005. [ bib | .pdf ] Sensory systems often use groups of redundant neurons to represent stimulus information both during transduction and population coding of features. This redundancy makes the system more robust to corruption in the representation. We approximate neural coding as a projection of the stimulus onto a set of vectors, with the result encoded by spike trains. We use the formalism of frame theory to quantify the inherent noise reduction properties of such population codes. Additionally, computing features from the stimulus signal can also be thought of as projecting the coefficients of a sensory representation onto another set of vectors specific to the feature of interest. The conditions under which a combination of different features form a complete representation for the stimulus signal can be found through a recent extension to frame theory called “frames of subspaces.” We extend the frame of subspaces theory to quantify the noise reduction properties of a collection of redundant feature spaces.
[5]	C.J. Rozell and D.H. Johnson. Examining methods for estimating mutual information in spiking neural systems. Neurocomputing, 65-66C:429-434, June 2005. Also appears in Proceedings of the Computational Neuroscience Meeting (CNS), Baltimore, MD, July 2004. [ bib | .pdf ] Mutual information enjoys wide use in the computational neuroscience community for analyzing spiking neural systems. Its direct calculation is difficult because estimating the joint stimulus-response distribution requires a prohibitive amount of data. Consequently, several techniques have appeared for bounding mutual information that rely on less data. We examine two upper bound techniques and find that they are either unreliable or introduce strong assumptions about the neural code. We also examine two lower bounds, showing that they can be very loose and possibly bear little relation to the mutual information's actual value.
[6]	C.J. Rozell, D.H. Johnson, and R.M. Glantz. Measuring information transfer in crayfish sustaining fiber spike generators. Biological Cybernetics, 90(2):89-97, February 2004. [ bib | .pdf ] We present a method based on information-theoretic distances for measuring the information transfer efficiency of voltage to impulse encoders. In response to light pulses, we simultaneously recorded the EPSP and spiking output of crayfish sustaining fibers. To measure the distance between analog EPSP responses, we developed a membrane noise model that accurately captures stimulus-induced nonstationarities. By comparing the EPSP and spike responses, we found encoding efficiencies on the order of 10-4, with interesting dynamics occurring during initial transients. A simple analog to point-process converter predicted the small information transfer efficiencies and dynamic properties we measured. Copyright held by Springer-Verlag. The original publication is available at springerlink.com. DOI:10.1007/s00422-003-0458-y
[7]	C.J. Rozell, D.H. Johnson, and R.M. Glantz. Information processing during transient responses in the crayfish visual system. Neurocomputing, 52-54:53-58, June 2003. Also appears in Proceedings of the Computational Neuroscience Meeting (CNS), Chicago, IL, July 2002. [ bib | .pdf ] We analyzed sustaining fiber responses in the crayfish visual system to light pulses using information processing techniques. The light pulse stimuli elicited a transient and a steady-state component in the EPSP input and in the firing rate of the spike train output. The overall information transfer of the system was very low (10-4), with a sharp increase during the transient portion of the response followed by a steady decrease. The information transfer dynamics are consistent with a simple spike generator model that depends explicitly on stimulus changes. The present analysis also corroborates the observed light reflex behavior.

Book Chapters

[1]	D.H. Johnson, C.J. Rozell, and I.N. Goodman. Information theory and systems neuroscience. In S. Grün and S. Rotter, editors, The analysis of spike data. Springer-Verlag, 2008. In press. [ bib ]

Conference Publications

[1]	C.J. Rozell. Distributed processing in frames for sparse approximation. In Proceedings of the Conference on Information Sciences and Systems (CISS), Princeton, NJ, March 2008. Invited talk. [ bib ]
[2]	R.L. Ortman, C.J. Rozell, and D.H Johnson. Reconstruction of compressively sensed images via neurally plausible local competitive algorithms. In Proceedings of the Conference on Information Sciences and Systems (CISS), Princeton, NJ, March 2008. [ bib ]
[3]	C.J. Rozell, D.H. Johnson, R.G. Baraniuk, and B.A. Olshausen. Locally competitive algorithms for sparse approximation. In Proceedings of the International Conference on Image Processing (ICIP), San Antonio, TX, September 2007. [ bib ]
[4]	P. Casazza, G. Kutyniok, S. Li, and C.J. Rozell. Modeling sensor networks with fusion frames. In Proceedings of SPIE, Wavelets XII at SPIE Optics and Photonics, volume 6701, San Diego, CA, August 2007. [ bib | .pdf ] The new notion of fusion frames will be presented in this article. Fusion frames provide an extensive framework not only to model sensor networks, but also to serve as a means to improve robustness or develop efficient and feasible reconstruction algorithms. Fusion frames can be regarded as sets of redundant subspaces each of which contains a spanning set of local frame vectors, where the subspaces have to satisfy special overlapping properties. Main aspects of the theory of fusion frames will be presented with a particular focus on the design of sensor networks. New results on the construction of Parseval fusion frames will also be discussed.
[5]	C.J. Rozell and D.H. Johnson. Power scheduling for wireless sensor and actuator networks. In Proceedings of the International Conference on Information Processing in Sensor Networks (IPSN), Cambridge, MA, April 2007. [ bib | .pdf ] We previously presented a model for some wireless sensor and actuator network (WSAN) applications based on the vector space tools of frame theory. In this WSAN model there is a weight associated to each sensor-actuator link denoting the importance of that communication link to the actuation fidelity. These weights were shown to be useful in pruning away communication links to reduce the number of active channels. Inspired by recent work in power scheduling for decentralized estimation, we investigate the optimal allocation of system resources for achieving a desired actuation fidelity. In this scheme, each sensor acquires a noisy observation and sends a message to a subset of actuators using an MQAM transmission strategy. The message sent on each sensor-actuator communication link is quantized with a variable number of bits, with the number of bits optimized to minimize the total network power consumption subject to a constraint on the actuation distortion. We show analytically and verify through simulation that performing this optimal power scheduling can yield significant power savings over communication strategies that use a fixed number of bits on each communication link.
[6]	C.J. Rozell, D.H. Johnson, R.G. Baraniuk, and B.A. Olshausen. Neurally plausible sparse coding via competitive algorithms. In Proceedings of the Computational and Systems Neuroscience (Cosyne) meeting, Salt Lake City, UT, February 2007. [ bib ]
[7]	S.W. Bishnoi, C.S. Levin, C.J. Rozell, B.R. Johnson, D.H. Johnson, and N.J Halas. All-optical nanoscale pH meter: a plasmonic nanodevice with quantifiable output. In Proceedings of the Annual Meeting of the IEEE Lasers and Electro-Optics Society (IEEE LEOS), Montreal, Canada, October 2006. Invited paper. [ bib ]
[8]	C.J. Rozell and D.H. Johnson. Evaluating local contributions to global performance in wireless sensor and actuator networks. Lecture Notes in Computer Science, 4026:1-16, 2006. Proceedings of the International Conference on Distributed Computing in Sensor Systems (DCOSS), San Francisco, CA, June 2006. [ bib | .pdf ] Wireless sensor networks are often studied with the goal of removing information from the network as efficiently as possible. However, when the application also includes an actuator network, it is advantageous to determine actions in-network. In such settings, optimizing the sensor node behavior with respect to sensor information fidelity does not necessarily translate into optimum behavior in terms of action fidelity. Inspired by neural systems, we present a model of a sensor and actuator network based on the vector space tools of frame theory that applies to applications analogous to reflex behaviors in biological systems. Our analysis yields bounds on both absolute and average actuation error that point directly to strategies for limiting sensor communication based not only on local measurements but also on a measure of how important each sensor-actuator link is to the fidelity of the total actuation output.
[9]	C.J. Rozell, I.N. Goodman, and D.H. Johnson. Feature-based information processing with selective attention. In Proceedings of the International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Toulouse, France, May 2006. [ bib | .pdf ] We present a simple but general model for feature-based information processing with selective attention. We model feature extraction as projections onto frames of subspaces, which accounts for redundancies in the representations of individual features as well as between features. To manage limited resources, we use feedback attentional signals to dynamically allocate system resources according to the observed events. In our model, attention maximizes the average information retained about all events weighted by their relative priorities. We illustrate the model with a simple system under a total bit constraint and discuss how the organization of the feature extraction affects the optimal bit allocation.
[10]	C.J. Rozell and D.H. Johnson. Analysis of noise reduction in redundant expansions under distributed processing requirements. In Proceedings of the International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Philadelphia, PA, March 2005. [ bib | .pdf ] We considered signal reconstruction with redundant expansions under distributed processing in noisy environments. Redundant expansions have the ability to reduce noise corrupting the coefficients, but distributed processing schemes will not be able to take full advantage of the redundancy present. We apply frame theory and a generalization called “frames of subspaces” to find conditions when distributed reconstruction suffers no loss in noise reduction ability, and we bound performance loss in more general cases.
[11]	M.A. Lexa, C.J. Rozell, S. Sinanović, and D.H. Johnson. To cooperate or not to cooperate: Detection strategies in sensor networks. In Proceedings of the International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Montreal, Canada, May 2004. [ bib | .pdf ] This paper is an initial investigation into the following question: Can cooperation among sensors in a sensor network improve detection performance in a simple hypothesis test? We analyze a simple cooperative system using the Kullback-Leibler (KL) discrimination distance and a quantity known as the information transfer ratio which is a ratio of KL distances. We discover that, asymptotically, gain over a non-cooperative system depends on the conditional KL distance. We conclude with an illustrative example which demonstrates that cooperation not only significantly improves performance but can also degrade it.
[12]	C.J. Rozell and D. Manolakis. Matched filter performance for unequal target and background covariance matrices. In Proceedings of the SPIE Defense and Security Symposium: Algorithms and Technologies for Multispectral, Hyperspectral, and Ultraspectral Imagery X, Orlando, FL, April 2004. [ bib ] Detection of military and civilian targets from airborne platforms using hyperspectral imaging (HSI) sensors is of great interest. Relative to multispectral sensing, hyperspectral sensing can increase the detectability of targets by exploiting finer detail in spectral signatures. A multitude of adaptive detection algorithms have appeared in the literature or have found their way into software packages and end-user systems. The most widely known among them is the linear matched filter. However, despite its popularity, the fact that the matched filter is used under conditions that deviate from the implicit optimality assumptions has not been investigated.
[13]	M. Simoni, B. Broening, C. Rozell, C. Meek, and G. Wakefield. A theoretical framework for electro-acoustic music. In International Computer Music Conference (ICMC), Beijing, China, 1999. [ bib | .pdf ]

Other Reports and Conference Abstracts

[1]	C.J. Rozell. Analog and digital sparse approximation with applications to compressed sensing. In 2008 SIAM Conference on Imaging Science, San Diego, CA, July 2008. Invited talk. [ bib ] In the compressed sensing (CS) framework, the sampling phase is resource constrained, taking a small number of linear samples. However, the price paid for this simplicity is a computationally expensive reconstruction algorithm that forms a bottleneck in using the sensed data. We will present a sparse approximation framework that both unifies many of the recently proposed digital algorithms and introduces novel analog architectures that solve the same problems. We will demonstrate how these analog systems solve CS recovery problems orders of magnitude faster than current digital systems, at speeds limited only by the underlying hardware components.
[2]	C.J. Rozell, D.H. Johnson, R.G. Baraniuk, and B.A. Olshausen. Neural architectures for sparse approximation. In Information Theory and Applications Workshop, La Jolla, CA, January 2008. Invited talk. [ bib ] Modeling naturally occurring signals as being sparsely representable has seen increased utility in many recent signal processing applications. Interestingly, recent evidence indicates that sensory neural systems may be employing sparse approximations to represent stimuli. However, the sparse coding hypothesis remains unconfirmed and the computational mechanisms that would enable neural systems to solve this challenging mathematical problem have been unclear. In this talk I will describe our development of a dynamical system capable of solving a family of sparse approximation problems using a network of simple neurally-plausible analog components. In addition to providing a proposed mechanism that can be used by experimentalists to test the sparse coding hypothesis in neural systems, this architecture is also amenable to analog hardware implementations that are much faster than the digital counterparts. I will describe this dynamical system architecture, discuss its properties, and demonstrate its performance in several relevant sensing and signal processing tasks.
[3]	C.J. Rozell. Distributed redundant representations in man-made and biological sensing systems. PhD thesis, Rice University, Houston, TX, 2007. [ bib | .pdf ] The ability of a man-made or biological system to understand its environment is limited by the methods used to process sensory information. In particular, the data representation is often a critical component of such systems. Neural systems represent sensory information using distributed populations of neurons that are highly redundant. Understanding the role of redundancy in distributed systems is important both to understanding neural systems and to efficiently solving many modern signal processing problems. This thesis makes contributions to understanding redundant representations in distributed processing systems in three specific areas. First, we explore the robustness of redundant representations by generalizing existing results regarding noise-reduction to Poisson process modulation. Additionally, we characterize how the noise-reduction ability of redundant representation is weakened when we enforce a distributed processing constraint on the system. Second, we explore the task of managing redundancy in the context of distributed settings through the specific example of wireless sensor and actuator networks (WSANs). Using a crayfish reflex behavior as a guide, we develop an analytic WSAN model that implements control laws in a completely distributed manner. We also develop an algorithm to optimize the system resource allocation by adjusting the number of bits used to quantize messages on each sensor-actuator communication link. This optimal power scheduling yields several orders of magnitude in power savings over uniform allocation strategies that use a fixed number of bits on each communication link. Finally, we explore the flexibility of redundant representations for sparse approximation. Neuroscience and signal processing both need a sparse approximation algorithm (i.e., representing a signal with few non-zero coefficients) that is physically implementable in a parallel system and produces smooth coefficient time-series for time-varying signals (e.g., video). We present a class of locally competitive algorithms (LCAs) that minimize a weighted combination of mean-squared error and a coefficient cost function. LCAs produce coefficients with sparsity levels comparable to centralized algorithms while being more realistic for physical implementation. The resultant LCA coefficients for video sequences are more regular (i.e., smoother and more predictable) than the coefficients produced by existing algorithms.
[4]	C.J. Rozell. Dynamic systems for sparse coding. Technical report, Walter Karplus Summer Research Grant Final Report, February 2007. [ bib ] Despite evidence that neural systems may be employing sparse stimulus coding, the mechanisms underlying this ability are not understood. I present a class of neurally plausible locally competitive algorithms (LCAs) that correspond to a collection of sparse approximation principles. These systems minimize a combination of MSE and a coefficient cost function. LCAs use a combination of thresholding and inhibitory connections to induce local (possibly one-way) competitions. LCAs demonstrate sparsity levels comparable to existing sparse coding algorithms while being much more realistic for neural implementation. Additionally, LCAs coefficients for video sequences demonstrate inertial properties, making them both qualitatively and quantitatively more regular (i.e., smoother and more predictable) than the coefficients produced by greedy algorithms. In addition to being a first step toward an experimentally testable hypothesis of biological mechanisms, LCAs may provide new methods for hardware implementations of sensing systems (e.g., digital cameras) and new front-end representations for both video coding and computer vision tasks.
[5]	D.H. Johnson, C.J. Rozell, and I.N. Goodman. Information theory and neuroscience: A tutorial. In Gulf Coast Consortium Conference on Theoretical & Computational Neuroscience, Houston, TX, November 2006. Invited talk. [ bib ] When Shannon developed information theory, he envisioned a systematic way to determine how much "information" could be transmitted over an arbitrary communications channel. While this classic work embraces many of the key aspects of neural communication (e.g., stochastic stimuli and communication signals, multiple-neuron populations, etc.), there are difficulties in applying his concepts meaningfully to neuroscience applications. We describe the classic information theoretic quantities-entropy, mutual information, and capacity-and how they can be used to assess the ultimate fidelity of the neural stimulus representation. We also discuss some of the problems that accompany using and interpreting these quantities in a neuroscience context. We also present an overview of post-Shannon research areas that leverage his work in rate-distortion theory that are extremely relevant to neuroscientists looking to understand the neural code. The presentation is meant to be mostly tutorial in nature, setting the stage for succeeding presentations.
[6]	B.A. Olshausen, C.J. Rozell, D.H. Johnson, and R.G. Baraniuk. Sparse coding via thresholding and local competition. In Gordon Research Conference on Sensory Coding and the Natural Environment, Big Sky, MT, August 2006. Contributed poster. [ bib ]
[7]	D.H. Johnson and C.J. Rozell. Information theory and neuroscience. In Computational Neuroscience Meeting Workshop on Methods of Information Theory in Computational Neuroscience, Edinburgh, UK, July 2006. Invited talk. [ bib ] When Shannon developed information theory, he envisioned a systematic way to determine how much "information" could be transmitted over an arbitrary communications channel. While this classic work embraces many of the key aspects of neural communication (e.g., stochastic stimuli and communication signals, multiple-neuron populations, etc.), there are difficulties in applying his concepts meaningfully to neuroscience applications. We describe the classic information theoretic quantities-entropy, mutual information, and capacity-and how they can be used to assess the ultimate fidelity of the neural stimulus representation. We also discuss some of the problems that accompany using and interpreting these quantities in a neuroscience context. We also present an overview of post-Shannon research areas that leverage his work in rate-distortion theory that are extremely relevant to neuroscientists looking to understand the neural code. The presentation is meant to be mostly tutorial in nature, setting the stage for other workshop presentations.
[8]	C.J. Rozell. Analyzing dynamics and stimulus feature dependence in the information processing of crayfish sustaining fibers. Master's thesis, Rice University, Houston, TX, May 2002. [ bib | .pdf ] The sustaining fiber (SF) stage of the crayfish visual system converts analog stimulus representations to spike train signals. A recent theory quantifies a system's information processing capabilities and relates to statistical signal processing. To analyze SF responses to light stimuli, we extend a wavelet-based algorithm for separating analog input signals and spike output waveforms in composite intracellular recordings. We also present a time-varying RC circuit model to capture nonstationary membrane noise spectral characteristics. In our SF analysis, information transfer ratios are generally on the order of 10-4. The SF information processing dynamics show transient peaks followed by decay to steady-state values. A simple theoretical spike generator is analyzed analytically and shows general dynamic and steady-state properties similar to SFs. The information transfer ratios increase with spike rate and dynamic properties are due to direct spike generator dependence on input changes.