A soft computing approach for benign and malicious web robot detection

作者：

Highlights：

• We propose a method called SMART (Soft computing for MAlicious RoboT detection).

• The method detects benign and malicious robots, and human visitors to a web server.

• SMART selects its features on a particular web server by fuzzy rough set theory.

• A graph-based clustering algorithm classifies sessions into the three agent types.

• Analyses on web logs suggest state-of-the-art results to detect both robot types.

摘要

•We propose a method called SMART (Soft computing for MAlicious RoboT detection).•The method detects benign and malicious robots, and human visitors to a web server.•SMART selects its features on a particular web server by fuzzy rough set theory.•A graph-based clustering algorithm classifies sessions into the three agent types.•Analyses on web logs suggest state-of-the-art results to detect both robot types.

论文关键词：Markov clustering algorithm,Web Robot Detection,Web crawler,Malicious web agents,Fuzzy Rough Set Theory

论文评审过程：Received 12 October 2016, Revised 31 May 2017, Accepted 2 June 2017, Available online 7 June 2017, Version of Record 15 June 2017.

论文官网地址：https://doi.org/10.1016/j.eswa.2017.06.004