Lead Investigators:

Mani Srivastava, John Villasenor, Deborah Estrin, Mohammad Rahimi

Overview

Wireless image networks have been attracting increasing research interest due to improvements in imaging microelectronics, processing computation, and wireless networking. In recognition of the importance and richness of the imaging modality, our recent work at Center for Embedded Networked Sensing has studied large-scale distributed image sensing. Our research has not only led to perhaps the most widely used first generation wireless sensor platform capable of image sensing, Cyclops, but through pilot applications and system studies has provided us with a wealth of data and insight into image network challenges and opportunities. Cyclops, which we reported previously, has been used extensively by large body of researchers at CENS and the broader sensor network community to investigate application of computer vision techniques to generate high-quality information from time-series of relatively low-resolution images, study low power computation paradigm, explore vision architectures and system issues, and analyze communication vs. computation constraints. Additionally, Cyclops has also brought together an elite group of interdisciplinary researchers to investigate the application of image network technology in a wide range of environmental biology and ecology application. As reported earlier in this manuscript, there are currently multiple system development and deployment activities underway at CENS that consist of a heterogeneous network of Cyclops and environmental sensors in the context of biology and ecology applications.

Accomplishments

a. Network Deployment and Image Data Collection

We have deployed the Cyclops nodes in James San Jacinto Mountains Reserve for applications that require long-time untethered monitoring of wildlife activities by embedding imagers in their native habitat. These deployments include a network of infrared Cyclops nodes for avian studies by embedding them in their nests during their nesting season and a network of Cyclops for herpetological studies to monitor reptiles and amphibians in pitfall bucket and funnel trap arrays, which are typically used in such studies. Our network architecture and design, which to a great extent inherits from CENS rich set of developed system architecture and available implementation, includes a heterogeneous network consisting of micro-servers and pairs of Mote/Cyclops nodes. Micro-servers have a Mote radio to connect to the network of Mote/Cyclops and a broadband radio to connect to a wireless local-area network and ultimately to the Internet. They facilitate programming and configuration of the sensor nodes and retrieval of the information from them. We exploit SensorBase; CENS data archival, retrieval and search infrastructure, to archive the visual information and the result of intermediate image processing routines. This, in turn, contributes to the SensorBase project by providing a rich set of data that can be exploited in sensor data storage, management and search research and by stress-testing their architectural performance under large volume of image data.

b. Automated Analysis of Time Series of the Images

While the visual and environmental data that has been collected provide valuable insight and intuition about the biological phenomenon, the sheer volume of information requires techniques to automate the processing and analysis of the images. So far, our study focuses on avian application, in which inspired by biological question, we analyze various parameters of the nest during the nesting period including the occupancy, the number of eggs in the nest and the state of the activity of the nest. To determine the pattern of occupancy of the nest, we have studied a toolbox of simple surveillance techniques such as background subtraction and histogram-based scene analysis to more complex techniques such as Much & Low object classifier. Figure-1 illustrates the relative occupancy of the nest for NB19 based on the result of the background subtraction alongside the environment temperature information and state of the nest activity. The data illustrates the difference in pattern of occupancy of the nest based on the day/night time transition and the state of the activity of the nest.

Figure-1- Left column, is a sample of collected infrared images from a nest (NB19) during the nesting season at time of different activity state of the box such as building, living, laying eggs, incubating, hatching and feeding. Right graph is the result of analysis of the images illustrating the relative presence of the bird in the nest alongside the environment temperature. The relative occupancy is the result of smoothness of binary occupancy information. Different state of the nest has been marked with different color and the night time is highlighted with darker shading.

Figure-2- Left graph, is the performance of SIFT classifier in terms of counting the number of eggs in the nest. Right graph, is the temporal response of the SIFT classifier during the nesting period alongside the ground-truth data.

To determine the timing and number of eggs in the nest, we use the Scale-invariant Feature Transform (SIFT) algorithm for extracting distinctive features of the eggs from the images. Figure-2 illustrates the result of an experimentation, in which, we extracted the features of the eggs from a set of training images containing eggs from NB31, learning a SIFT model of the egg via a standard Support Vector Machine (SVM) algorithm, and then applying the model to a new series of nest images from NB21.  Since the performance of SIFT classifier degrades as the number of laid eggs increases, we are looking at combinatory approaches that takes into account the result of SIFT classification as well as other classification approaches to determine the number of eggs in the nest. Although in all our experimentation the state of the activity of the nest has been determined though a labor intensive visual inspection of the collected images, we are now looking at various approaches to model the state of the nest activity based on the extracted time-series occupancy information and the time-series estimate of egg-count to automate estimation of the state of the nest activity throughout the season. In all the above experiments, the data from higher resolution and lower noise wired CCD cameras as well as lower resolution wireless CMOS Cyclops cameras have been used to explore the impact of image quality on the performance of our computation and investigate proper imager characteristic for the target applications.

c. Low Power Embedded Imager as Biological Sensor

To build a large-scale long-lived network of wireless imagers, some of the image processing computations should be performed locally on the sensors. Consider the nesting experimentation, in which, we collect images at 15 minutes interval. This sums up to more than 8000 images for the three month of active nesting season, which if we take CIF (320×280) images, is equivalent to more than 683MB of data per sensor node. Clearly transporting this massive amount of data, which also includes redundant and uninteresting image frames, over a wireless network consumes considerable radio capacity and energy. As we later illustrate numerically, in many cases local computation significantly reduces the power consumption and size of the data leading to longer lifetime of the node and more efficient use of network resources.

Although most of our deployment and data collection efforts have been so far geared towards collecting images from the sensor nodes and post-processing them at the back-end server, we feel that this is a necessary step to create an automated process and evaluate the performance of such process with the ground truth data. However, we have taken this further by analyzing the cost of locally performing those processes on an embedded platform. In our analysis we have not confined ourselves to the Cyclops platform and have looked at a range of computational choices including the Atmega128 and MSP430 Micro-controllers and ADSP-BF533 Blackfin Digital Signal Processors (DSPs). The formers are the microcontrollers used in Cyclops and various incarnations of Motes and the latter is a DSP processor used in many consumer devices that require heavy computation. We have done so by running the computations in the processor-specific simulation environment and evaluating the running time, memory footprint and energy requirement of the algorithm. The interesting observation is that while a generic computer vision approach which is designed for accuracy through thoroughness does not necessarily lend itself to existing on an embedded device, it was great to see that by exploiting 1) application-specific characteristics and 2) architecture-specific optimizations we can reduce the cost of the algorithm to a manageable size. As an example of the former optimization, the comparison of the measured power-consumption of running SIFT classification to determine the number of eggs on the ADSP-BF533 Blackfin processor in the VisualDSP++ simulation environment shows that while the classification of eggs over the entire image scale takes 608mJ, it only takes 45mJ on limited scale space with minimum loss in classification performance. This is due to relatively uniform distance of the Cyclops imagers from the eggs and hence relatively uniform size of the eggs in the image. An example of the latter optimization is the Much & Low object classifier which heavily relies on 2D convolution of the image with Gabor Filter. In this case, when we exploit various architectural optimization of the ADSP-BF533 Blackfin processor such as using the dual Multiple Accumulate Units (MACs), parallelism through Very Long Instruction Word (VLIW) architecture and internal Data Memories the cost of running the computation from 252mJ in non-optimized form reduces to 12.4mJ when optimized for the target architecture. A simple comparison of the above numbers with the total cost of transmission of the original image which is 94mJ (over the CC1000 radio used in many Mote class devices) suggests the significant opportunities of the above optimizations in reducing the cost of local computation. We have done various such analysis for the Much & Low object classifiers, background subtraction and Scale-invariant Feature Transform (SIFT) on Atmega128 and ADSP-BF533 processors.

d. Embedded Image Compression

While our emphasis is on local processing of the images, there are many instances that we are interested in sending the image for calibration and training purposes, recording critical events or performing heavy processing of an image. Although it is intuitive that image communication based on compression followed by transmission is generally more energy efficient than direct transmission of uncompressed images, there has been very little quantitative study of the specific energy tradeoffs characterizing these two approaches in the context of resource-constrained sensor platforms. This is due in part to the challenges involved in implementing compression algorithms on sensor platforms with stringent limitations on memory, processing speed, and energy consumption. We have investigate this tradeoff by mapping JPEG compression algorithms onto resource limited processors using a power aware design environment that makes specific use of native word lengths of the target processor. In traditional compression implementations, the precision used are often far greater than is necessary in light of the approximations inherent in the imaging devices and in later stages the compression computations. Optimizing the precision minimizes the energy consumption, and is thus a critical element in any fair evaluation of the compression/transmission issue. The focus of our JPEG implementation is to ensure that the compression computations utilize the minimum precision needed to ensure that computation approximations do not become the dominant source of error. Our results shows that our optimized JPEG implementation consumes only 105.5µJ to compress the test images on the Cyclops ATmega128 processor while the standard integer implementation from the Independent JPEG Group (IJG) takes 133.15µJ to compress the image with the same amount of Peak Signal to Noise Ratio (PSNR) performance.

Furthermore, we have looked at the trade-offs between sending raw images and compressed images in wireless network. Figure-3 illustrates some of the subtleties in the compression/communication tradeoffs using a precision-optimized version of the JPEG algorithm. Figure-3 gives the total energy for JPEG compression and transmission averaged over a set of test images for three different motes. The upper and lower parts of each bar indicate the energy consumed by compression and transmission respectively. Results are given for different image resolutions (square images of linear sizes 64, 128, and 256), and within each resolution for low (Q=10), medium (Q=50), and high (Q=90) JPEG image quality settings. Energy measurements for direct transmission images with no compression are also provided. While the data in the figure are representative of particular combinations of motes, image sizes, etc., they support the intuitive conclusion the compression followed by transmission is generally more energy-efficient than compression alone, and more importantly, they provide the exact amount of this tradeoff for the systems concerned. This can then be weighed against other factors, such as the computation time spent performing the compression. Interestingly, in some cases (in this example, when high image quality is desired and the MicaZ mote is used), compression followed by transmission can cost more energy than direct transmission without compression. By gathering and analyzing a larger set of data similar to that shown here and extending this to a wider range of wireless transmission technologies, distances, and channel error models, it will be possible to provide a framework allowing us to make the most energy-optimal design choices.

Figure 3: Average energy for compressing and transmitting an image. The upper and lower parts of each bar indicate the portions used by transmission and compression respectively. The two cropped Mica2 bars are 380 mJ and 942.1 mJ.

e. Reconfigurable Image Processing and Acquisition Environment

Various CENS deployments and experimentations have undermined the importance of reconfigurable image acquisition and processing environment to remotely and dynamically compose, instantiate, tune, and reconfigure application-specific embedded image sensor processing algorithms on a resource-constrained platform. To accomplish this, we have leveraged our previous work on SOS which is an embedded operating system that includes a static kernel and dynamically loadable modules. SOS provides multiple levels of reconfiguration and parameter running via mechanisms such as dynamic binary modules, dynamic virtual machines, and dynamic module parameter update. So far we have ported the SOS operating system to Cyclops platform and have created an initial system on top of SOS for tasking image processing computation as arbitrary dataflow graphs that are dynamically reconfigurable via rewiring and parameter tuning from the backend. Additionally, we have built a user interface on top of the Ptolemy system that offers graphical modeling of computation flow graphs (Figure-4). In our implementation, the application consists of a family of image processing modules and a configuration module that contains the wiring organization of the image processing modules. Any change in the application results in dynamic update of only the required image processing modules as well the new configuration file. This minimizes required update information to the newly created modules as well as the new configuration module. While our initial research is promising, we have to further study the performance and efficiency of our approach.

Figure 4: The signal flow graph of simplified background subtraction routine used to track the nest occupancy. The image processing algorithm consists of SOS modules that are configured in a user-friendly environment. Since image processing libraries are dynamically loadable SOS modules, reconfiguration results in only changing the modified or added image processing libraries as well as new configuration module requiring minimum update overhead.

Future Directions

One focus in the coming year will be understanding science problems through interaction with domain scientists and expanding lifetime, geographical span and goal of our deployments. Although we have investigated various image processing techniques to automate analysis of the images at the backend server, they are far from comprehensive and require further investigation. We will do so in close cooperation with domain scientist. A second important research focus will be the design of autonomous embedded image sensors for environmental and biology applications. We extend our simulation of the algorithms to implementation on the embedded platform. A third focus will be revisiting the sensor node architecture based on the rich experiences that we have gained through data collection, analysis and embedded environment simulation. Finally, we try to mature our reconfigurable image processing environment and exploit it in real-life deployment.

People

Faculty:

• Mani Srivastava, Electrical Engineering, UCLA
• John Villasenor, Electrical Engineering, UCLA
• Deborah Estrin, Computer Science, UCLA

Staff:

• Mohammad Rahimi, CENS, UCLA

Postdoc Research:

• Dong-U Lee, UCLA

Graduate Students:

• Hyungjin Kim, UCLA
• Shaun Ahmadian, UCLA
• Teresa Ko, UCLA
• Jeff Mascia, UCLA
• Juo-Yu Lee, UCLA

Undergraduates:

• David Zats, UCLA
• Hootan Nikbakht, UCLA