Skip Header NavigationIntranet 
CENTER FOR EMBEDDED NETWORKED SENSINGContactDirectionsEmploymentEventsNews
HomeAbout UsResearchEducationResourcesPeople

Research Project

Large-Scale Vision Networks

Technology > Systems Area Projects > Large-Scale Vision Networks

On this page: Overview | Accomplishments | Future Directions | People

Overview

Measuring traffic on city roads, evaluating peoples interest in an art gallery, monitoring the perimeter of an oil refinery, or safeguarding the remote water reservoirs of the nation are just a few of the promises of large-scale vision networks. Emergence of CMOS imaging devices with low power consumption and tight integration with CMOS logic on the same chip provides new opportunities in low power vision networks. However, exploiting CMOS images in sensor networks, requires research in design and development of vision platforms that are tailored for the specific requirements of sensor networks such as extended lifetime, in-network processing and simple interface. In addition, we need to redesign low power vision algorithms and approaches that gain their strength from multiple observations across the vision network. This introduces new research avenues in large scale vision networks that are studied in this project.

Accomplishments

  1. Vision platform: Cyclops

    To bridge the gap between existing commercial CMOS devices and the requirements of the sensor network, we designed and built Cyclops in collaboration with Agilent Technology. Cyclops, which is now commercially available, uses a CMOS imaging sensor with reduced-size imaging. The main focus in the design of Cyclops is dedicated image processing, reduced interface complexity while exposing control over the critical imaging parameters (e.g. size, type, zoom, pan) and low-power operation. Cyclops can be coupled with a sensor network node such as Berkeley Mote (Figure-1) and the combination represents a wireless vision network node.

    Figure 1- Cyclops couples with a Berkeley Mote and they represent a wireless vision network node.

    Figure 1- Cyclops couples with a Berkeley Mote and they represent a wireless vision network node.
  2. Vision Network: Deep Vision

    We developed Deep Vision, a distributed image processing and aggregation network. Deep Vision controls image capturing and processing on the vision nodes through queries, which disseminate across the network. A query consists of attributes that represent either a computation on a vision node or the characteristics of a node’s environment. Examples of the computations are object detection, motion flow measurement, or image segmentation (Figure-2), and examples of the attributes of the node’s environment are its location, viewing direction or the name of the objects in its field of view. In our approach, a user level question such as “what is the popularity of a gallery in a museum?” can be answered via a query that invokes proper computation (e.g. image segments per unit of time) on the relevant nodes and returns the aggregated results to the user.

    Figure 2- Cyclops is used as a vision sensor by performing local image processing on the images.

    Figure 2- Cyclops is used as a vision sensor by performing local image processing on the images. Top images illustrate consecutive image frames to calculate the center of mass of the consecutive frames to use Cyclops as a motion flow sensor. The Bottom images illustrate the instantaneous images, a calculated foreground image and the resulting segmentation operation to use Cyclops as an object count sensor.
  3. Vision Network Deployment for Habitat Monitoring

    We are deploying the first generation of Cyclops vision network for habitat monitoring applications to support diverse domains in monitoring birds nesting, reptile populations and plant phenology. Our deployment in James San Jacinto Mountains Reserve is in its initial phase (Figure-3). So far, we have focused on collecting images in large numbers to build a rich archive of image data. The study of the images enables us to investigate proper image processing operations for addressing the domain science problems and migrate some of the image processing operations to the vision nodes. This would reduce the bandwidth requirement and power consumption of the network.

    Figure 3- This figure illustrates a Cyclops vision node that has been deployed in the nest box and a picture that Cyclops has taken from inside the nest box.

    Figure 3- This figure illustrates a Cyclops vision node that has been deployed in the nest box and a picture that Cyclops has taken from inside the nest box.
  4. Vision network simulation and emulation tool: ARGOS

    Designing algorithms on the vision nodes remains as an important challenge due to the lack of visibility into the device operation. This is particularly important, since the performance of image processing algorithms are highly sensitive to environmental conditions such as variations in illumination and color. To expedite development, testing and debugging of large scale vision networks we have developed ARGOS, a simulation and emulation tool in Emstar programming environment. ARGOS enables the users to run native TinyOS code on simulated nodes that are instantiated in Emstar environment. The code on the nodes, can run in simulation mode where the node runs against archived image scenes or in emulation mode where the images are taken from set of Cyclops that are directly connected to emulation machine via a serial multiplexer. ARGOS enables collecting large scale image scenes as data sets to test performance of the algorithms over controlled input data. In addition, it enables testing the system in real time emulation mode before final deployment on the actual nodes.
  5. Reconfigurable image processing using SOS operating system:

    Reconfiguration of image processing libraries is an important asset in a vision network. This provides the ability to
    incrementally update the vision network after initial deployment, add new image processing libraries to a vision node, and remove unused libraries when they are no longer needed. The vision node can dynamically run new image processing algorithms based on the variation in requirements of the applications or in response to dynamics of the environment. To incorporate the growing requirement for such reconfigurable image processing capability, we have started porting SOS operating system to Cyclops platform. This enables the image processing logic to be encapsulated in the form of modules that are supported in SOS operating system. Image processing modules can be uploaded into or removed from the Cyclops node in real time. A user can create variant of image processing modules that are inserted and used based on the requirement of the application.
  6. Actuated Vision Network

    The ability of the vision nodes to actuate in order to influence their field of view, calibrate their coordinate system or in response to stimuli in the environment is an important research extension to the conventional vision networks. To this end, we have developed a pan-tilt platform for Cyclops (Figure-4). The pan-tilt platform is composed of two servomechanisms, and the associated circuitry which are controlled by a Mote. The combination of the Mote, Cyclops and pan-title module is an actuation-enabled wireless vision node. The servomechanism maintains its state when the power is disconnected to minimize the overall power consumption of the node. The drivers for the system are written in SOS operating system and provide a simple interface for positioning the Cyclops.



    Figure 4- Pan-tilt Cyclops enables large scale actuated vision network experiments.
  7. Energy-optimized Function Evaluation on Cyclops Platforms

    The performances of underlying mathematical functions that are used in processing of the images in an embedded platform are of key importance to overall performance of the system. We address this problem, by directly exploiting the native word-length of the processor and utilizing the minimum amount of resources to deliver the required precision in a function. A key element of our method is the determination of the minimal number of bits for each signal in the fixed-point data-path of a software program through the use of multi-word arithmetic. This results in minimal processor execution time, leading to a low energy/performance operation ratio. For the evaluation of the functions, we opt for polynomial approximations including those based on splines (piecewise polynomials), which provide a flexible set of tradeoffs between precision, table size, and computation through adjustments to polynomial degree, segmentation intervals and other parameters. Figures 5 and 6 illustrate the latency and code size comparisons using various function evaluation methods for the approximation of log(x). These methods enable finding an optimal solution in terms of latency and code size suitable for a given application requirement. For instance, degree-2 splines would be the best choice for a precision of 18 bits in terms of both latency and code size, since they are the second fastest and result in the smallest code size.


    Figure 5 (left) and Figure 6 (right)

Future Directions

One focus in the coming year will be on better understanding the needs of domain scientists as the life-cycle of our deployed vision networks progresses. While we deploy system and collect images we can recognize key science problems that can be addressed by vision networks.

A second focus will be development of image processing operations that reveal underlying scientific information from the images. Investigating these image processing operations requires thorough analysis of archived images. In addition, we need to investigate feasibility and power and bandwidth trade-offs in moving some of the image processing operations to the vision node.

A third focus will be on system and architecture development. This includes completing the SOS port into Cyclops to support dynamic reconfigurable systems, enhancing of ARGOS simulation and emulation environment, and completing the pan-tilt application programming interface.

People