The 2020 Paul Panayotatos Endowed Scholarship in Sustainable Energy was awarded to ECE graduate student Tong Wu. Tong Wu is a PhD student working with Professor Jorge Ortiz in the Department of Electrical and Computer Engineering. Tong is a member of Prof. Jorge Ortiz' lab – the Cyber-Physical Intelligence lab – where researchers focus on designing new machine learning techniques on sensors in smart environments. One of the main application domains is in smart buildings.
Conserving energy and reducing energy consumption has been a major topic during recent years. A variety of energy monitoring systems have been designed and deployed on facilities such as buildings, server clusters and mobile phones to provide users with consumption information. It collects time-series data which shows how much energy they are using and how it is used in a period. Some advanced systems can provide further analysis and insights to identify unexpected excessive usage. Typically, these analyzes are based on pre-defined constant thresholds, regardless of the diversity and non-stationary characteristics of the normal pattern. In the context of big data and Internet-of-Things, there have been much more data collected by smart meters and sensors which makes it possible to develop machine learning algorithms that can differentiate between normal pattern and anomalous behavior.
Tong's current work is on designing interactive anomaly detection algorithms that use feedback from experts to find anomalies in building data. The data consists of sequences of readings from sensors that compromise the building management system (BMS); a supervisory control system that allows a building manager to centrally observe and respond to the operational state of building-related sub-systems – such as the Heating, ventilation, and air conditioning (HVAC) and lighting.
To address these challenges, Tong and Prof. Ortiz introduced a new algorithm called RLAD. It combines reinforcement learning with active learning and label propagation to learn an anomaly detection model that generalizes and can adapt and improve dynamically through getting feedback reward on observing new data. With active learning, our model selects samples with most information and asks an expert to label them as ground truth. Our model is able to learn with historical data using very few labeled examples and directly adapt to the real-time applications without any additional parameter tuning. To the best of our knowledge, we are the first to combine deep reinforcement learning for anomaly detection with active learning. In the early experiments, we outperform all state-of-art methods, both unsupervised and semi-supervised on precision, recall and F1-score.