Summary

Project Overview

ADSC strives to strengthen the safety and security of Singapore through computation technologies, for example, surveillance video could be analyzed in real time to recognize criminal suspects; critical infrastructure such as the power grid can be better protected by detecting suspicious activities in real-time. These visions can only be realized with the capacity to process massive amounts of challenging data in real time, including video, audio and text.  Such capabilities, however, are far beyond the current state-of-the-art.

This project aims to design and develop Resa, a general framework for performing common real-time analytics tasks on challenging data. Resa will support real-time processing of fundamental analytics tasks commonly used by other research groups at ADSC and I2R collaborators, and expose a friendly user interface. We expect Resa to play a major role in several major projects of ADSC, including detecting suspicious events using audio and video from the SSIPO testbeds, and updating the 4D map of Singapore in real-time.

Research Findings

  1. We have been developing Resa, the main result of this project. So far, we have implemented a basic prototype, and devised an accurate performance model for a simplified version of Resa, based on the theory of queuing networks.
  2. We designed and implemented Chronos, a system for generating testing data for Resa. Chronos takes a small real dataset as the seed, and generates arbitrarily large data that preserves the properties of the seed data.
  3. We developed Pabirs, a data access module to be used in Resa. Pabirs encapsulates low-level access to a distributed file system, and provides several types of efficient indexing schemes.
  4. We extended Abacus, an auction-based resource allocation module for Resa, to support quality-of-service contracts. Abacus was a side project that started before the main project. Initial results on Abacus has been published in our ICDE’13 best paper.
  5. We built OceanRT, a demo prototype for real-time analytics over static temporal data. Early results show that OceanRT can be up to 10x faster than the current state-of-the-art. OceanRT is to be merged with Resa at a later stage, to enable real-time analytics on both streaming and static data.

Demonstrations & prototypes

We have been building a prototype of Resa, our main real-time analytics system for massive streams of challenging data. We have implemented a basic version, which was presented in a SIGMOD’13 poster. We are now improving this prototype by adding sophisticated features, such as accurate performance modelling, run-time elasticity, and fast fault recovery.

We have built a demo system for real-time data analytics on static temporal data, to be shown in SIGMOD’14. This system forms the basis for our new collaboration project with I2R.