基于 Hadoop平台的并行特征匹配算法研究
李宝禄;张伟
【期刊名称】《计算机应用研究》 【年(卷),期】2014(000)011
【摘要】很多大企业采用Hadoop分布式文件系统来存储海量数据，而传统的病毒扫描主要针对单机系统环境。研究如何并行化病毒扫描中的核心特征匹配算法来处理分布式海量数据。在Hadoop平台下，基于MapReduce并行编程模型来实现大数据高效的病毒扫描，特别是针对Hadoop处理海量小文件效率低的问题，通过将小文件合并，再利用索引来提高海量小文件的处理效率。实验结果表明，提出的并行特征匹配算法可以显著降低处理时间，适用于大数据的病毒扫描。%Many enterprises use Hadoop distributed file system to store mass data,but traditional virus scanning mainly face to single machine.This paper studied the way to make the core feature matching algorithm of virus scanning parallel to deal with the mass distributed data.With the frame of MapReduce,on the Hadoop platform,this paper realized efficient virus scanning of big data.Especially for the problem of low efficiency of processing mass small files on Hadoop platform,it incorporated small files,then used index to improve the efficiency of virus scanning of mass small files.The experimental results show that the parallel feature matching algorithm can reduce the processing time significantly,and is applicable to virus scanning of big data. 【总页数】4页(3320-3323)
基于 Hadoop平台的并行特征匹配算法研究
