Anomaly detection from time-changing environmental sensor data streams

Mamun, Abdullah-Al- (2016) Anomaly detection from time-changing environmental sensor data streams. Masters thesis, Memorial University of Newfoundland.

[img] [English] PDF - Accepted Version
Available under License - The author retains copyright ownership and moral rights in this thesis. Neither the thesis nor substantial extracts from it may be printed or otherwise reproduced without the author's permission.

Download (1MB)

Abstract

This thesis stems from the project with real-time environmental monitoring company EMSAT Corporation. They were looking for methods to automatically ag spikes and other anomalies in their environmental sensor data streams. The problem presents several challenges: near real-time anomaly detection, absence of labeled data and time-changing data streams. Here, we address this problem using both a statistical parametric approach as well as a non-parametric approach like Kernel Density Estimation (KDE). The main contribution of this thesis is extending the KDE to work more effectively for evolving data streams, particularly in presence of concept drift. To address that, we have developed a framework for integrating Adaptive Windowing (ADWIN) change detection algorithm with KDE. We have tested this approach on several real world data sets and received positive feedback from our industry collaborator. Some results appearing in this thesis have been presented at ECML PKDD 2015 Doctoral Consortium.

Item Type: Thesis (Masters)
URI: http://research.library.mun.ca/id/eprint/11750
Item ID: 11750
Additional Information: Includes bibliographical references (pages 43-50).
Keywords: Data Mining, Anomaly Detection, Change Detection, Data Streams, Outlier, Environmental Sensor
Department(s): Science, Faculty of > Computer Science
Date: January 2016
Date Type: Submission
Library of Congress Subject Heading: Environmental monitoring--Data processing; Anomaly detection (Computer security); Data mining; Multisensor data fusion; Sensor networks

Actions (login required)

View Item View Item

Downloads

Downloads per month over the past year

View more statistics