Mining Massive Time Series for Knowledge Discovery in IoT Systems

Dr. Tan Yan
NEC Labs America


Todayís IoT systems are often equipped with a large network of sensors that generate huge amount of time series reflecting the operational status of the system. How to mine such huge data and discover knowledge to help system operation is a challenge. In this talk, I will focus on a typical IoT system, complex physical system, and discuss its two important issues: system degradation profiling and operation behavior segmentation. In the first part of the talk, I will introduce our novel time series analysis technique published in SIGKDD 2015. It profiles the long-term system degradation by decomposing the sensor time series data into trend and fluctuation components, providing the monitoring software with actionable information about the changes of the systemís behavior over time. We analyze the underlying problem and formulate it to a Quadratic Programming (QP) problem that can be solved with existing QP-solvers. To further speed up the problem solving, we further transform the problem and present a novel QP formulation, Non-negative QP, for the problem and demonstrate a tractable solution that bypasses the use of slow general QP-solvers. In the second part, we propose a solution reported in ICDM 2015 for accurate and automated identification of operation behavior switching in complex systems by inferring the relationship changes among massive time series. Our method first learns a sequence of local relationship models that can best fit the time series data, and then combines the changes of local relationships to identify the system level behavior switching. We formulate the underlying switching identification as a segmentation problem, and propose a sophisticated optimization algorithm to accurately discover different segments in time series. To unveil the system level behavior switching, we present a density estimation and mode search algorithm to effectively aggregate the segmented local relationships so that the global switch points can be captured. We implemented our methods to real physical systems including power plants and chemical production systems, which demonstrate the effeteness of our solution.