Overview

A broad class of CENS ENS applications including environmental sampling, public health environment monitoring, precision agriculture, and security require distributed sensing capabilities. This requirement arises from the unpredictable nature of the appearance of events and the characteristic high spatiotemporal frequency associated field variables that must be measured. For example, the objectives of experimental scientists investigating solar radiation and microclimate phenomena in ecosystems may require measurement spatial resolution of centimeters to meters while also requiring that field variables be mapped over spatial scales of greater that 100 m. Now, the use of only static sensors for such sampling often requires an impractically large number of devices to be distributed in large volumes. New methods are now required that permit the combination of sparsely distributed static sensors and autonomously operating mobile devices to provide sampling that meets the high fidelity requirements. Such systems must adapt to environmental variation to acquire sensor data at those optimal times and locations that allow for the reconstruction of field variables with a fidelity that meets a user’s measurement objectives. As will be described, the CENS Multiscale Sensing and Actuation research thrust has developed a new method that adds an additional tier of sensing capability that dramatically expands sensing performance for accurate measurements over large environmental volumes.

Approach

The new Multiscale sensing and actuation is based on a hierarchical system that enables autonomous arrangement of sensors with the objective of optimizing sensing fidelity, spatial coverage, and mobility characteristics. This system of sensors then can be used for efficient high fidelity sampling of high frequency spatiotemporal phenomena.

Consider an example of a two-tier multiscale architecture. In this architecture, a dynamic phenomena of high spatiotemporal frequency associated with its field variables is captured by a first tier sensor. The first tier is represented by a static low-fidelity high spatial coverage sensor providing “global” information about the environment. This information is then used to extract the regions of interest (regions of high phenomenon variability). These regions form a set of sampling tasks for the second-tier sensors to pursue. The second tier is represented by the mobile robots equipped with high-fidelity low spatial coverage (spot measurement) sensors. A set of new tasks is given as an input to the Task Allocation module. The task allocation module prioritizes tasks based on the selected utility and assigns the task with highest utility to the available mobile robot for high-fidelity sampling.

An output of the system is a set of high-fidelity phenomenon measurements in a given region, which then can be used by scientists.

Systems/Experiments

The performance of the multiscale paradigm for sampling light intensity was tested and analyzed through simulations and in physical system. The first tier sensor data (image sensor data sources) were captured from a study area located at the James San Jacinto Mountain Reserve during daylight periods. A down-looking imager captured snapshots (768x480 pixels) of the understory of a forest canopy covering an area approximately 6 meters in length by 4 meters in width. Images captured between 10:00 a.m. and 11:00 a.m. were analyzed in simulation and also were sampled in the laboratory using the NIMS 3D system described in this same Annual Report section.

These images were experimentally verified (by analyzing images during other times of the day) to be representative of the spatial and temporal variations occurring in the transect throughout the day. The images captured constitute the information sensed using a high spatial coverage, low-fidelity sensor (imager) and were processed to extract a set of tasks that represent possible regions that could be sampled using the low spatial coverage, high-fidelity optical sensor carried by the mobile robot.

Figure 1ab displays results comparing actual and reconstructed scenes. Here, the dark and light areas of this image display the presence and absence of solar radiation. The form of this distribution is of interest and is the measurement objective of the user. This result shows an example where the multiscale system performs task allocation with priority assignments based on the measured area of an environmental region. Figure 1c displays the complete results for varying actuated sensor motion speed capability (for actuated sensors moving at speeds of 40 to 500 cm/s). Here, this compares normalized sample area (ratio of total sampled and reconstructed area to actual area) to sampling period (density), s, where s varies from one sample per 4 centimeters to one sample per 20 centimeters. This experiment compares the performance of a multiscale approach to a traditional full raster scan of the environment. The raster scan samples the complete environment with a desired density with an average speed of 40 cm/s for all sampling densities

Figure 1. Experimental Results: Multiscale vs. Raster Scan. Comparison of light intensity distribution captured in the field (a) and reconstructed data at a particular instant of time (b). c) Simulation results for Multiscale and Raster scan sampling comparing normalized sampled area for different densities and speeds.

Figure 1c demonstrates that for corresponding speeds, a multiscale approach performs better than a simple raster scan in terms of the amount of information extracted from the environment. Additionally, multiscale yields greater fidelity as well. In the raster scan, the information extracted by sampling the complete transect area initially results in greater error because of the phenomenon dynamics. This is evident, for example, from the number of images processed in the raster scan. They varied from 1 image for s = 4 to 9 images for s = 20. The total number of images processed (using the same average speed) using a multiscale approach varied from 13 images for s = 4 to 121 images for s = 20. Thus, a multiscale approach captures more up-to-date information from the environment resulting in greater fidelity.

MultiScale Sampling ( MSS) and Adaptive Sampling (AS) methods were also compared in simulations for both static and dynamic phenomena. For simulation purpose, these phenomena were created with a black background having a white colored object. The shape of the object was derived from one of the images captured from the real environment and represents the random regions covered with bright sunlight in the understory of a tree.

The performance of multiscale sampling for dynamic phenomena was simulated by moving the object at the rate of 12 pixels per minute along both the horizontal and the vertical direction. This translates into a motion rate of 12 cm/min. This produces a phenomena distribution characteristic of results observed in the natural environment. The performance of both MSS and AS is analyzed for different phenomena (different object sizes). Estimation error after collecting each sample is calculated by comparing the estimated phenomenon with the current ground truth. Finally, this error is normalized with the total number of pixels in the image (total pixel area of the environment).

Figure 2 shows the performance comparison of MSS and AS. The sampling fidelity for MSS is better than AS. MSS quickly converges to a low estimation error and then stabilizes around this value. For AS, there is a time delay, due to the coarse scan, for achieving a low sampling

fidelity. This fidelity becomes worse as the object moves to a new location, other than where it was sampled during the coarse scan. Overall improvement with MSS can be observed with

Figure 2. Comparison of sampling performance for each sampling algorithm choice for static (left) and dynamic (right) object.

lower running average of the estimation error. Estimation error is shown only till real time of 1400 seconds as it stabilizes around the same value thereafter.

Accomplishments

Multiscale sensing and actuation systems were developed and analyzed during this reporting period. Accomplishments include:

1. Development of experimental systems including
   • Two tier light sampling system deployed in the field at James Reserve with light sampling via imaging and fixed sensor systems.
   • Development of laboratory evaluation systems including mobile NIMS devices and light field projection systems
2. Analysis of multiscale system performance for actual environmental field variables and for both static and dynamic phenomena
3. Development of the new NIMS 3D system as shown below for multiscale actuated sensing.

Future Directions

Multiscale sensing is anticipated to become the primary method for CENS sampling applications in the future. The nature of the first tier sensing resources will vary between applications. In the above example, this is a local imaging asset. In other examples, this first tier sensor may be supplied by remote sensing systems or derived from Internet-accessible standard data sources. In each case, multiscale guidance is expected to dramatically advance performance. While multiscale will be applied generally, there are specific examples of near term applications. This includes a detailed investigation of understory light that combines the development of new multiscale algorithms, software systems, and platforms in collaboration with Terrestrial Ecosystem research. Multiscale methods will also be applied to aquatic system research in both lake and river systems.

People

Graduate Students: Diane Budzik, Victor Chen, Willie Chen, Henrik Borgstrom, Cathy Kong. Amarjeet Singh, Michael Stealey

CENS Staff Members: Dr. Maxim Batalin, Dr. Eric Graham

CENS Faculty: Professor Deborah Estrin, Professor Mark Hansen, Professor William Kaiser, Professor Greg Pottie, Professor Phil Rundel, Professor Gaurav Sukhatme