NETUITIVE BEHAVIOR LEARNING ENGINE
Netuitive's Behavior Learning Engine
What is it?
The Netuitive Behavior Learning Engine™ is the key analytics technology behind Netuitive’s self-learning, predictive analytics. Built on nine patented technologies and techniques resulting from over 20 years of academic and commercial research and development, the Behavior Learning Engine:
- Continuously analyzes, correlates, and normalizes thousands of simultaneous performance variables from dozens of enterprise sub-systems.
- Builds an Adaptive Behavior Profile for each data stream, reflecting the unique rhythms and trends of each performance metric — by the hour, month and season.
- Monitors the contextual relationships between each performance metric and applies real-time analysis and correlation to identify statistically significant anomalies.
- Delivers actionable intelligence in the form of Trusted Alarms® whenever it detects or forecasts changes in service levels. Composite dashboards tell you "at-a-glance" how applications and systems are performing. Diagnostics help isolate root-causes and reports provide current, historical, and trended views of performance.
How is the Netuitive Behavior Learning Engine "predictive"?
Often application or infrastructure outages or degradations start with a "ripple" or some sort of anomalous behavior in one or more performance metrics. These anomalies are often leading indicators of serious problems, so by detecting and evaluating these "anomalies", Netuitive provides early warning of impending problems and their level of severity.
Netuitive's approach is not to "look" for the anomalies (via some sort of pattern detection), but to learn what is "normal" and to alert when behavior becomes "abnormal". This approach has proven itself to be very effective in early detection of impending performance problems, as evidenced by the results of Netuitive customers.
The basic mathematics behind this approach (multivariate correlation and regression analysis) has proved itself in areas like actuarial science (risk assessment for insurance and finance), weather forecasting, retail (sales and demand forecasting), manufacturing (demand forecasting), pharmaceuticals (drug trials), and many other fields - where people try to predict or forecast results based on unbiased statistical analysis.
Netuitive is the only behavior learning technology that has applied the proven mathematics of multivariate correlation and regression to IT systems and applications performance management.
How does the Behavior Learning Engine learn?
Netuitive's patented Behavior Learning Engine uses multivariate correlation and regression analysis algorithms, coupled with self-learning and adaptive capabilities, and a multitude of proprietary and patented statistical heuristics to process data in real-time. This ability is complemented by its rapid learning capability to adapt to sudden changes.
Netuitive's multivariate correlation analysis algorithm automatically learns, identifies and understands how two or more variables, or performance metrics, co-vary in the natural environment. Correlation analysis techniques enable the engine to self-discover and study the relationships between variables. It understands how one metric relates to the variability of another.
In addition, Netuitive uses multivariate regression analysis techniques to describe the numerical relationships between performance metrics. Whereas correlation analysis intuitively surmises the relationship strength between two metrics, regression analysis provides an algebraic equation describing the nature of the relationship. Furthermore, regression analysis provides the variance measures, which enables prediction and contextual analysis with contextual values.
The correlation and regression analysis algorithms are self-learning, meaning there is no initial bias when the system is configured. As a result, correlation coefficients and regression weights are based on the actual system behavior, not a pre-defined model.
Finally, the Behavior Learning Engine is adaptive. In other words, as the engine is continually fed new data it automatically adjusts the statistical model (correlation coefficients and regression weights). Its models and dynamic thresholds are continually adapting.
What is unique about Netuitive’s Behavior Learning Engine?
Out of all the behavior learning tools mentioned by analysts, Netuitive is the only one that is truly self-learning, and the only one that has proven that it scales to enterprise-class deployment. In a nutshell: the technological advantages are around speed, accuracy, automation, low processing and storage requirements.
- Speed — It takes only milliseconds to analyze thousands of simultaneous inputs. The speed of the analytics also accelerates root cause isolation.
- Accuracy – Because Netuitive is analyzing and correlating the behavior of all relevant metrics (and not single metrics in isolation), the technology is able to accurately detect impending performance anomalies hours before service performance is impacted and experienced by end-users.
- Automation — Netuitive automates or eliminates most of the menial tasks associated with performance management. The technology self-learns an environment's normal behavior, and readily adapts to changing IT requirements. It automatically generates dynamic thresholds, and performs all of its computations with no manual configuration required. This capability is particularly important for dynamic virtualized and cloud computing infrastructures.
- Low processing requirements — A highly efficient numerical approach enables Netuitive to perform, on a standard off-the-shelf server, massive calculations that would otherwise require a mainframe computer and databases.
- Minimal storage — Netuitive uses a proprietary algorithms that process data as fast as it is received and doesn't rely on large historical databases to perform its calculation. One derived benefit of this patented capability is that Netuitive does not reprocess previously obtained data as new data is made available, reducing the amount of required storage.
How quickly does the Behavior Learning Engine learn?
It only takes two time intervals (of 1, 5, 10, 15, 30 or 60 minutes each, chosen for the deployment) for a baseline to start being defined. It takes a couple of more intervals for health information to be calculated. After just one day, you have a daily baseline that can be extrapolated to the remaining days of the week. After a week, your initial weekly baseline is completed. The initial “learning” process can be accelerated by taking recently collected data for all the relevant metrics and feeding it to Netuitive at a faster than real-time rate.
If systems perform poorly, won’t Netuitive learn “bad” behavior”?
In theory, yes. However, Netuitive's Behavior Learning Engine has a built-in fast learning algorithm that detects abnormal change and can accelerate the pace at which environmental changes are learned. Bottom line: if detected problems are addressed in a timely way, it can unlearn as fast as it learned a bad behavior. In addition, there are capabilities to reset behavior patterns to historical ones or rapidly initialize new behaviors.
Furthermore, Netuitive has complemented its purely mathematical approach by a set of user-defined policies and domain-specific heuristics and rules that allow users to model specific requirements defined in Service Level Agreements (SLAs). For instance, you can define a policy threshold forcing a deviation if the CPU Utilization exceeds a certain value (e.g. 95%), even if there is no statistical ground for that (e.g. low variability of CPU Utilization near the 100% ceiling). Similarly, you can define a policy filter that cancels any statistical deviation that happens in a considered-safe zone (e.g. below 30% for CPU Utilization), thus reducing the alarming noise.
How does Netuitive's analytics approach differ from other IT analytics technologies?
Other behavior analysis tools use simple single-variable moving average calculations to determine dynamic thresholds for each individual metric, irrespective of the others. Then they typically rely on manual event scripting to create relationships between variables, which are inherently inaccurate since they're based on guesswork and regularly alerting metrics
Netuitive software auto-configures and has a self-learning and adaptive behavior analysis engine. It requires no manual scripting whatsoever. Netuitive's technology is based on multivariate "data" correlation and regression analysis as opposed to "event" correlation like most other techniques. Unlike other correlation products which rely on rules-based "events," Netuitive learns relationships between metrics by processing raw subsystem monitoring data to assure that alerts are based on actual behavior. Without Netuitive, an event correlation console is as good as the already-polluted events it is fed and relies on a human operator to define and script inference rules.
Furthermore, Netuitive products can complement existing rule-based inference engines. Too often, the rule-based systems are fed event streams based on static thresholds or single-variable moving average thresholds containing false-positives. When metrics are outside of single-variable baseline thresholds such as running mean plus or minus two standard deviations, then alarms are generated roughly 5% of the time. As a result, by themselves these systems are not reliable (garbage-in-garbage-out). To the contrary, when Netuitive products are placed in front of an event correlation system, they will feed it a complete set of high-quality real-time and predictive events (Trusted Alarms), with no false-positives and no missed events, thus increasing the accuracy of rules-based systems.
How does the behavior learning technology lead to better alerts in Netuitive?
Trusted Alarms® are one of Netuitive’s most valued features and only made possible through Netuitive’s automated contextual analysis approach, named ASOD – which leverages the Behavior Learning Engine.
Delivered on both a real-time and forecasted basis, these alerts provide an easy-to-understand composite view of impending issues, and can be integrated into existing monitoring consoles and trouble ticketing systems. They are generated using an Accumulated Score Of real-time, contextual and forecasted Deviations (or ASOD) – taking into account the number, frequency, type and severity of each deviation. Each system-verified Trusted Alarm results from analyzing dozens of real-time and forecasted calculations. It accounts for real-time workload level analysis as well as contextual behavior analysis using multivariate correlation and regression analysis, the fundamental technology behind Netuitive Behavior Learning Engine. A Trusted Alarm can be generated for an individual system issue or an end-to-end business service or application. Third-party studies have shown that, when compared to alerts generated from manually set static-thresholds or even single-variable “dynamic” thresholds, Netuitive Trusted Alarms are not only proactive, but reduce false alerts by 90 to 99%.
Visit our Resource Center for White Papers with more information about Netuitive’s behavior learning technology.