Microclimate, Acoustic, and Video Enabled Sensor Networks:
Adaptive Communication in Acoustic Sensor Arrays

1.  Research Objectives

During the past year we received a significant 5-year award from the NSF program in Biocomplexity and the Environment.   This has prompted us to redirect our efforts somewhat, from research on adaptive communication in the abstract toward coordinating with the goals of that grant. The abstract of this research, from the proposal, is as follows: We propose to develop sensor arrays so that they will be useful for observing and analyzing bird diversity and behavior. The focus will be on sensors and their abilities, especially for robustness and adaptability.  We believe birds provide a good test bed for developing and testing the abilities of sensor arrays because: a great deal is know about them; birds are fairly well studied for the properties we wish to explore; and because they are important for biodiversity. 

The work will be performed in laboratories at UCLA and at several field sites:
(1) the UC Riverside James Reserve, near Idyllwild, CA. --- a heavily instrumented field site with full-time specialists on sensor arrays. Here the methods will be initially tried out, then deployed to: (2) the Hastings Reserve, near Monterey CA, where acorn woodpeckers have been marked and studied. We will use the sensor arrays to identify individual woodpeckers, locate them, and to identify patterns in their behavior associated with the 10 or so calls they employ. Or else they will be deployed to (3) the Montes Azules Biodiversity Reserve in Chiapas,

Mexico, which has a rich diversity of bird species in a tropical rainforest.  Here we will develop the ability of the sensors and sensor arrays to identify and locate different bird species in an area. Initially we will focus on ant birds and wrens that are hard to observe except by sound. There is a severe need to assay biodiversity in such ecosystems; the methods we are developing will help to do so.

2.  Approaches, Systems and Experiments and Accomplishments

Based on the development of a recent, successful line of research in learning theory (Angluin, Kanazawa, Vapnik), we are studying distributed sensing, adaptation and communication in two quite different domains.  First, primarily to further our theoretical understanding of how such systems can operate and evolve in simple, controlled settings, we are developing a environment in which distributed computing machines can communicate with each other about network events, with particular attention to irregular activities that could signify intrusions or malfunctions of various kinds. And second, primarily to further our understanding of how such systems work in natural settings, we are beginning the study of a social bird, the Acorn Woodpecker (Melanerpes formicivorus), in its natural habitat.

Network monitoring and communication: In a first study, the ``external events'' being monitored were network communications, which were described in a simple English-like language to allow for human intervention when necessary.  An early version of this system was reported in (Wee et al.2001), and the basic linguistic components of the current system are publicly available from the project web page. In a networked environment now under study, different machines describe their network communications to each other, and since many of these events will be common to all or most of the communicating machines, they can, in certain settings, successfully "ground" their utterances, that is, successfully learn the association between utterances and what they signify about the environment - the key step in adaptive communication. Communication within and between acorn woodpecker family groups: Just recently we have developed plans to deploy sensor arrays for the study of acorn woodpeckers, a variety of woodpecker that typically clusters in family groups that maintain a territory, and share food in a jointly stocked `granary'.  Yuan Yao successfully defended her thesis proposal with a project that involves recording these birds at various sites in the James Reserve, and at the Hastings Reserve in Northern California.  At Hastings Reserve the woodpeckers are all marked, so the goal there will be to identify individuals and associate them with particular recorded calls. We have developed tools for analyzing and comparing these calls with sound spectrum cross correlations. But the more interesting results came from modeling these calls with Hidden Markov Models, so that the dynamics of the bird signals can be captured and compared. This technology was developed primarily for the analysis of human speech, but in the past decade or so it was discovered that the acoustic analysis and classification abilities of non-human animals is surprisingly similar: chinchillas and macaques can recognize particular human speech sounds (Kuhl & Padden 1983), and starlings can be trained to distinguish English and Swedish vowels, generalizing to new sounds and new speakers (Kluender et al. 1998). Very few studies of bird communication have used this technology, but Yao and other members of the project have successfully applied it in our lab, and we will apply it to our new recordings with the goal of identifying and studying communication among these birds in natural settings.

3.  Problems Encountered:

No major, unanticipated problems have been encountered.

We have also analyzed animal calls with both spectrographic correlation methods and with HMM methods, and the results look promising.  The recording of woodpecker calls requires special technology and arrangements with researchers at the field sites: these plans are well underway.

4.  Future Goals and Objectives:

Goals in network monitoring and communication: We have developed a learning model on paper (as reported in the publications cited above), and are now deploying it in a network study of (i) semantically "grounding" new signals to allow adaptive communication, and (ii) the evolution of languages, of signaling systems, in a collection of communicating software agents.  In particular, when the language is adaptive, if some aspect of communication becomes disabled or degraded, we expect these devices will be able to adjust the signals appropriately, resulting in a change in the language.  The infrastructure for this study is now working (as reported in the references above), and so now we are beginning simulation studies of "grounding" in noisy settings, and language evolution as a consequence of signal changes of various kinds. These studies are underway and so we expect results that can be reported in the next year.

Goals in woodpecker monitoring and communication: These birds have been studied before, but no study has applied sophisticated modeling technology to recordings at distributed sites.  We hope to proceed through several of the following steps with new recordings and our HMM models: (i) identifying the calls of the species (ii) distinguishing the different calls of the species (which have already been classified in previous studies, by human listeners); (iii) identifying the different calls of *particular individuals* in different recordings and at different locations; (iv) localize those individuals (done in collaboration with Kung Yao); (v) identifying patterns of communication in the family groups. The preparations for these studies are underway, and so we expect results that can be reported in the next two years.

One piece of infrastructure that will be useful in both of the previously mentioned studies is a data base technology that can handle large amounts of temporal sequence data, for concurrent analysis.

And this work is opening new theoretical questions which we are pursuing as the experimental studies develop. In particular, Lee's dissertation research is developing models of spatial patterns of language adaptation and language "speciation;" Collier's dissertation research is developing models of language change; the work of Stabler, Kobele, and Taylor aims to develop methods for identifying the simplest models compatible with the data.

PEOPLE

Faculty:

Charles Taylor, OBEE
Edward Stabler, Linguistics

Students:

Dept OBEE:
Travis Collier -- CENS fellow
Yoosook Lee
Yuan Yao
Alex Kirschel

Dept Linguistics:
Gregory Kobele
Ying Line

 

Foreign Collaborator:

Edgar Vallego, Dept of Computer Science, Monterrey Institute of Technology, Monterrey, Mexico (arrived August, 2003).

 

 

---

From 2003:

SYSTEMS / EXPERIMENTS

Experiments:

Classification

Digesting sensor data to detect and classify events and patterns is largely in the domain of engineering and real progress is being made by other groups. However, to close contact and interaction with those groups, we are also developing techniques.

• Aural classification of frog calls using spectral signature correlation
• Minimum Description Length (MDL) based regularity discovery for activity classification [figure of frog call classifier (TC to provide)]

Language Acquisiton

We have developed a system that can (proveably) efficiently learn an expressively powerful language from examples and relational information (dependency structures) [ref Stabler]. Current efforts are being focused on deriving the dependency structures from directly observable relationships beween sensed objects. This approach combines grounding and language aquisition in such a way that two independently insoluable problems form a tractable whole. Learner Diagram As a test domain, we are using symbolic information generated by a simple computer intrusion detection system and have desinged a symbolic regime for communicating about the tracking of physical objects.