The Design and Implementation of Computer Traffic Log Searcher System using Hadoop Map/Reduce Framework

Abstract

Abstract

Computer Crime Act B.E. 2550's section 26 requires Internet Service Provider (ISP) to keep users' traffic log for at
least 90 days since the first log-on. ISP operator must keep essential information that can identify users and time of usage. For this reason each ISP has to store a vast amount of data log which takes quite long to search for needed log data. This paper proposes the design and implementation of computer traffic log searcher system by applying Hadoop Distributed File System (HDFS) to help in storing users’ log and using Map/Reduce paradigm for searching users' log. The user interface is written in Java. Upon testing the system with 10 PCs by searching a log size of 50 Gigabytes, it is found that the search result is about 10 times faster than using a single PC.