Evaluation of Deep Learning-Based Electricity Theft Detection Models Against Adversarial Evasion Attacks

Abstract

The increasing deployment of smart grids and smart meters has significantly improved the monitoring and management of electricity consumption. However, electricity theft remains a major challenge, causing substantial economic losses and affecting the reliability of power systems. Recently, machine learning and deep learning techniques have shown promising results in electricity theft detection. Nevertheless, these models are vulnerable to adversarial evasion attacks that can manipulate consumption data and degrade detection performance. This thesis evaluates the robustness of electricity theft detection models against adversarial attacks. First, electricity consumption data are preprocessed and used to train four baseline detection models: Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), and Multi-Layer Perceptron (MLP). The baseline performance of each model is evaluated using standard classification metrics. Subsequently, five adversarial evasion attacks, namely Fast Gradient Sign Method (FGSM), Basic Iterative Method (BIM), Carlini and Wagner (C&W), Nearest Neighbor Perturbation (NNP), and Nearest Neighbor Distance (NND), are applied to generate adversarial examples and assess the vulnerability of the trained detectors. The performance degradation caused by each attack is analyzed and compared across all models. To explore potential improvements, a two-stage detector is proposed, where consumers are first clustered using K-means, and a fused ensemble of CNN, LSTM, GRU, and MLP models is then trained for each cluster and evaluated under the same attack scenarios. Experimental results show that while the proposed detector offers modest improvements against gradient-based attacks, particularly C&W, significant vulnerabilities remain, especially under dynamic perturbation attacks such as NND, which completely defeats both baseline and proposed models. This study highlights the importance of systematically evaluating robustness and the need for more fundamental advances to ensure secure smart grid operations.