CENS Technical Seminar Series

Embedded Imagers: Detecting, Localizing and Recognizing Objects and Events in Natural Habitats

Invited Speaker: Josh Hyman, Teresa Ko
Date: September 25, 2009
Time: 1:00 PM - 2:00 PM
Venue: Boelter Hall 4760

Abstract

Cameras are a natural choice of sensors for monitoring natural
habitats. They are cheap, both in terms of cost and energy, and at the
same time data-rich. They are remote, not requiring contact, and
passive, not requiring to broadcast signals into the environment, and
at the same time they can be tuned to be sensitive to different bands,
most commonly the visible and near-infrared spectra.

However, their use in natural habitats presents significant
challenges, due to the variability that objects or events of interest
can manifest depending on "nuisance factors" such as illumination,
vantage point and occlusion. For instance, the image of a moss is a
function of its CO2 uptake, which is of interest, but also of the
illumination, depending on the time of the day, the day of the season,
the weather, and the presence of cast shadows. It also depends on the
vantage point and the geometric layout in three dimensions, and the
presence of occlusions of line-of-sight. Similarly, events of
interest exhibit complex dynamics, for instance the motion of birds or
the configuration of a swarm of pollinators, but so do nuisances such
as the complex background foliage moving under the elements.

As a result, sensing the environment with images requires modeling the
complex spatio-temporal statistics of the objects and events of
interest, as well as the nuisances, for they often overlap due to
natural adaptation of species to their habitats. Unlike monitoring
indoor or urban environments, where one can assume a static
background, monitoring natural environments requires modeling the
distributional properties of portions of images (natural textures) and
their temporal evolution, and learning the natural statistics from
training data. For instance, detecting the presence of a bird at a
feeder station from an image from an embedded imaging sensor can be
difficult even to a trained expert. However, extended observation
reveals the characteristic variabilities of the object and enable
successful detection, localization and species recognition. Different
species can exhibit different appearance, depending on their pose, and
different patterns of typical motion; these in turn are different from
the characteristic background motion, for instance foliage moving in
wind. We describe algorithms for recognizing objects and events based
on extended observations of spatial and temporal statistics, including
the various phases of nesting from infrared images, the interaction of
various species at a feeder station, and the oxygenation of a moss in
natural illumination.

Finally, in order to aggregate the information abstracted from the raw
data at each imaging sensor, parsimonious rapresentations of these
processes are needed, depending on the task, that includes storage,
transmission, relation to human observers, or performing in-situ
decision such as the triggering of other sensory or communication
assets.

Biography

Josh Hyman is Ph.D. candidate working with CENS under the direction of
Deborah Estrin and Mark Hansen. He previously received his B.S. in
Computer Science and Engineering and M.S. in Computer Science from
UCLA. The focus of his research is building systems that use imagers
to indirectly measure biological phenomena when other sensing
modalities are too cumbesome or too invasive to use. He has worked at
IBM Almaden Research Center and is currently a Senior Software
Engineer at Google in Santa Monica.

Teresa Ko is currently a 3rd year PhD candidate in the Department of
Computer Science at UCLA, advised by Stefano Soatto and Deborah
Estrin. She received her B.S and M.Eng at the Massachusetts
Instittute of Technology in Computer Science and Electrical
Engineering. Prior to arriving at UCLA, she worked at Sandia National
Laboratories in the area of embedded sensing for national security
applications. She is interested in sensing technology for
environmental monitoring applications, specifically on remote
non-contact sensing for habitat monitoring and to monitor global
warming and other societally relevant ecological problems.