Taking advantage of correlated information for energy-aware scheduling in the IoT: A deep reinforcement learning approach

Abstract

Millions of battery-powered sensors deployed for monitoring purposes in a multitude of scenarios, e.g., agriculture, smart cities, industry, etc., require energy-efficient solutions to prolong their lifetime. When these sensors observe a phenomenon distributed in space and evolving in time, it is expected that collected observations will be correlated in time and space.

In this thesis, we first outline how data gathered from correlated sensors can be used to improve the timeliness of updates of another sensor in the network. We consider a system of two correlated information sources, i.e. sensors, which periodically send updates to a gateway, regarding the observed physical phenomenon distributed in space and evolving in time. The optimal use of updates in such a system greatly depends on the correlation between the two sources, and to explore this effect we investigate three different models of the covariance between independently obtained observations of the phenomenon of the interest. We extract values for the parameters in the covariance models from data coming from a real sensor network, to provide the reader with a realistic feel for scaling parameters values and the applicability of our analysis in a real scenario. We demonstrate that using correlated information results in a significant increase in device lifetime and compare our approach to others proposed in the literature.

In the second part, we build on the gained insight and propose a Deep Reinforcement Learning (DRL) based scheduling mechanism capable of taking advantage of correlated information. We design our solution using the two DRL algorithms, namely Deep Q Network and Deep Deterministic Policy Gradient. The proposed mechanism is capable of determining the frequency with which sensors should transmit their updates, to ensure accurate collecting of observations, while simultaneously considering the energy available. To evaluate our scheduling mechanism, we use multiple datasets containing multiple environmental observations obtained in a real deployment. We show that it is capable of significantly extending sensors lifetime. We compare the mechanism to an idealized, all-knowing, scheduler to demonstrate that its performance is near-optimal. Additionally, we present the unique feature of our design-energy-awareness by displaying the impact of sensors' energy levels on the set frequency of updates.