摘要
互联网的快速发展已经使得网页数据成为目前各种应用与研究的重要数据源之一.网页数据包含各种内容,如广告、导航条、相关链接、正文等,然而对于不同的研究和应用来说,并非所有内容都是必需的,相反地,不相关的内容反而会影响研究和应用的效果和效率,所以网页去噪是一个基础问题,且是目前热点研究的问题.因此很有必要对网页去噪领域进行总结,以便更好地进行深入研究.首先说明了网页去噪的必要性,并对网页去噪进行了定义和分类,概述了多种网页去噪的方法和框架,然后对评估网页去噪算法所使用的数据集和方法进行了总结,最后讨论了该领域存在的问题和今后的研究方向.
The rapid development of the Internet has made a variety of Web applications and Web data, which become the major source of data for lots of research. Web page includes a variety of content, such as advertising, navigation bar, related links, text, etc. However, for different studies and applications, not all content is necessary; oppositely, the unrelated content will affect the effectiveness and efficiency of the research and applications. So Web page cleaning is a highlighted topic of information retrieval with booming search engines. Thus it is necessary to sum up the field on the page de-noise, in order to better carry out in-depth study. Firstly, this paper gives a brief introduction to the necessity of Web page cleaning and its related concepts. The authors present a classification hierarchy of the Web page cleaning methods, including the single-model based Web page cleaning methods and the multi-model based Web page cleaning methods. Then, this paper summarizes all kinds of Web page cleaning techniques and frameworks, including SST, Shingle, Pagelet, DSE, etc. Thirdly, this paper describes the experimental datasets and experimental methods used in all kinds of Web page cleaning techniques. Finally, this paper discusses the existing problems and the future directions in the Web page cleaning field.
出处
《计算机研究与发展》
EI
CSCD
北大核心
2010年第12期2025-2036,共12页
Journal of Computer Research and Development
基金
国家自然科学基金项目(70903008
60933004)
国家"八六三"高技术研究发展计划基金项目(2007AA01Z154
2009AA01Z143)
CNCI搜索引擎项目(CNCI2008-122)
