Unsupervised deep learning approach for network intrusion detection combining convolutional autoencoder and one-class SVM

With the rapid advancement in network technologies, the need for cybersecurity has gained increasing momentum in recent years. As a primary defense mechanism, an intrusion detection system (IDS) is expected to adapt and secure the computing infrastructures from the ever-changing sophisticated threat landscape. Many deep learning approaches have recently been proposed; however, these techniques face significant challenges in identifying all types of attacks, especially rare attacks due to network traffic imbalances and the lack of a sufficient number of abnormal traffic samples for model training. To overcome these shortcomings and improve detection performance, this paper presents an unsupervised deep learning approach for intrusion detection. Unlike the existing IDS model that extracts features and trains a classifier in two separate stages, a single-stage IDS approach that integrates a one-dimensional convolutional autoencoder (1D CAE) and a one-class support vector machine (OCSVM) as a classifier into a joint optimization framework is introduced in this paper for the first time. Using only the normal traffic samples, the approach simultaneously optimizes the 1D CAE for compact feature representation and the OCSVM for classification by defining a unified objective function combining reconstruction error with classification error. Thus, the generated compact feature representation has not only reconstruction ability but also discriminative ability for classification. An in-depth ablation analysis validates the design decisions and provides further insight of the proposed approach. An extensive set of experiments on two benchmark intrusion datasets, NSL-KDD and UNSW-NB15, demonstrates the generalization ability of the proposed model for unseen attacks and confirms it as a competitive approach over the recent state-of-the-art intrusion detection baselines. Overall, the obtained results emphasize that the proposed approach has potential to serve as a baseline for building an effective IDS.