Toward intrusion detection using belief decision trees for big data

摘要

Big data refers to datasets that we cannot manage with standard tools and within which lie valuable information previously hidden. New data mining techniques are needed to deal with the increasing size of such data, their complex structure as well as their veracity which is on covering questions of data imperfection and uncertainty. Even though big data veracity is often overlooked, it is very challenging and important for an accurate and reliable mining and knowledge discovery. This paper proposes MapReduce-based belief decision trees for big data as classifiers of uncertain large-scale datasets. The proposed averaging and conjunctive classification approaches are experimented for intrusion detection on KDD’99 massive intrusion dataset. Several granularity attacks’ levels have been considered depending on whether dealing with whole kind of attacks, or grouping them in categories or focusing on distinguishing normal and abnormal connections.