Feature Extraction and Malware Detection on Large HTTPS Data Using MapReduce

Abstract

Secure HTTP network traffic represents a challenging immense data source for machine learning tasks. The tasks usually try to learn and identify infected network nodes, given only limited traffic features available for secure HTTP data.

In this paper, we investigate the performance of grid histograms that can be used to aggregate traffic features of network nodes considering just 5-min batches for snapshots. We compare the representation using linear and k-NN classifiers.

We also demonstrate that all presented feature extraction and classification tasks can be implemented in a scalable way using the MapReduce approach.

Keywords

• Hadoop
• MapReduce
• HTTPS data
• Intrusion detection
• Approximate similarity join