The great challenges in peta-scale data management for urban sensing data include the following.
A centralized solution for data storage becomes invalid.
The smarter urban sensing system involves vast amounts of multi-source, multi-modal data from hybrid networks.
Sensing data possess rich semantics, dynamically grow over time, and are in nature high dimensional.
Large scale data storage and processing in datacenters consume significant amount of power.
High-level applications may require sensing data from multiple different sources that are typically dispersed.
Since data are generated over time, there will be an increasingly vast amount of data.
Few existing solutions can be straightly applied for peta-scale urban data management. To address the above coupled challenges, we propose a cloud-based data management software platform that is tailored for urban sensing and sustainable applications. It can handle large-scale streaming data with different formats.
In our design, the whole software stack of the system consists of a cloud data repository module at the bottom layer, a large-scale stream processing and big data analytics at the middle layer, and finally a real-time visual interface atop to provide decision-making, megacity management and real-time support for users.