当前位置: 首页>博士论文>资源详情
Web应用关键性能指标的异常检测与异常定位研究
中文摘要

 近年来,随着Web应用(如搜索引擎、电商网站等)的不断扩展,大型互联网公司服务规模及复杂度都在不断增加,与此同时,用户对Web应用服务体验的要求越来越高,一些问题采用传统的性能管理方法变的效率低下甚至不能满足当前需求,因此,互联网公司对其所提供服务的性能管理面临许多新的挑战。本文针对异常检测与异常定位这两个服务性能管理的关键环节进行分析,对其中的三个具体问题进行了深入研究,首先对面向海量监控指标的自适应异常检测算法进行研究,然后对异常检测中报警信息爆炸式增长的优化策略进行研究,最后对多维度检测指标体系中的异常定位问题进行研究。本文的主要工作及贡献如下: (1)提出并实现了面向海量KPI的自适应异常检测算法。通过对大量KPI的数据特征进行分析研究,采用聚类分析的方法对大量KPI按其特征进行分类,对不同类别的KPI类簇进行异常检测算法的适配选择,基于该模型的建立,对待测KPI只需接入系统即可自动判别所属类簇,并自适应适配该类别的最优算法。最后应用真实的数据进行实验,验证了分类的合理性以及自适应算法的有效性。 (2)针对报警管理系统中KPI数量巨大等原因导致的报警数量过多的问题,提出了相应的优化机制并实现了一系列优化策略。首先通过对过量的报警信息进行分析,明确了若干项具有普遍性的导致该问题的现象及原因。然后对各项原因,有针对性的进行深入研究并制定一系列优化策略,例如报警合并策略,报警等级划分策略,报警接收冗余避免策略,以及报警自动恢复的策略等。最后,应用真实的数据进行了实验对比,并将各项优化策略部署到实际的报警管理系统中,验证了优化机制的有效性。 (3)实现了解决多维度KPI指标体系中异常定位问题的方法——HotSpot。首先,针对多维度KPI指标体系中元素之间复杂的互相影响关系,HotSpot提出了“连锁效应”规则,并基于该规则提出了评判任一指标集合是根因的可能性的评价指标——Potential Score,然后,针对多维度指标体系中搜索空间巨大搜索速度慢等问题,采用了蒙特卡洛树搜索(MCTS)算法和分层剪枝策略相结合的方法,大幅度提升了搜索速度。最后,采用真实数据进行实验,验证了HotSpot的有效性和稳健性。 关键词:Web应用;海量KPI;异常检测;报警聚合;异常定位

英文摘要

 In recent years, with the rapid development of Web applications, such as search engines and e-commerce sites, the scale and complexity of large Internet companies’ services are increasing. At the same time, the users’ demand for the service experience is becoming more and more high. These phenomena bring many new challenges to the application performance management of the Internet companies. Some traditional methods are inefficient or can not meet the current needs to these problems of new background. Anomaly detection and anomaly localization are two key parts of application performance management. In this paper, three specific problems of the two parts are studied. First, we implemented a self-adaptive anomaly detection algorithm for a huge number of KPIs. Then, we studied the optimization strategies of the explosive growth of alarts for anomaly detection. Finally, we studied the anomaly localization problem for additive KPIs with Multi-Dimensional Attributes. The main work and contributions of this paper are as follows: (1)Design and implement a self-adaptive anomaly detection algorithm for huge scale KPIs. By analyzing and studying the data features of a large number of KPIs, we apply cluster analysis algorithm to classify the KPIs. Then the anomaly detection algorithms are selected and adapted for different clusters of KPIs. Based on the model, we can automatically identify the KPI’s cluster and match the most suitable algorithm for a new strange KPI. Finally, using the real-world data, we evaluate this approach and compare it with other classical algorithms. The results show that the classification is reasonable, and the approach is effective. (2)Aiming at the excessive number of alerts in alert management system, several optimization strategies are proposed and implemented. First of all, we propose several general phenomena and reasons that cause the problem by analyzing the excessive alert information. Then we conduct in-depth research on each reasons, and propose several optimization strategies, such as alert grouping strategy, level calibration of the alerts, oncall schedule and escalation strategy, and automatic self-healing of the alerts. Finally, we conducted a comparative experiment, then deployed the strategies to a real-word company. The result proves that these strategies are very effective and efficient. (3)We propose and implement an approach of anomaly localization for additive KPIs with multi-dimensional attributes——HotSpot. First, we conclude a rule that reveals the interaction relationship among elements - Ripple effect. Then based on this rule, we propose potential score to measure the potential of a set to be root cause. To deal with the huge search space, we adopt the MCTS approach (the first time in anomaly localization literature) whose action value is our novel potential score. In addition, we propose a hierarchical pruning approach to further reducc the search space. At last, our experiments based on a real-world search engine show that HotSpot achieves much better accuracy than previous approaches. Key words: Web Application; Huge Number of KPI; Anomaly Detection; Alert Convergence; Anomaly Localization

作者相关
主题相关
看过该书的人还在看哪些书