Why we need a Dynamic Baseline and Thresholding Tool?

Why we need a Dynamic Baseline and Thresholding Tool?

With the introduction of opCharts v4.2.5 richer and and more meaningful data can be used in decision making. Forewarned is forearmed the poverb goes, a quick google tells me “prior knowledge of possible dangers or problems gives one a tactical advantage”. The reason we want to baseline and threshold our data is so that we can receive alerts forewarning us of issues in our environment, so that we can act to resolve smaller issues before they become bigger. Being proactive increases our Mean Time Between Failure. If you are interested in accessing the Dynamic Baseline and Thresholding Tool, please Contact Us.

Types of Metrics

When analysing time series data you quickly start to identify a common trend in what you are seeing, you will find some metrics you are monitoring will be “stable” that is they will have very repeated patterns and change in a similar way over time, while other metrics will be more chaotic, with a discernible pattern difficult to identify. Take for example two metrics, response time and route number (the number of routes in the routing table), you can see from the charts below that the response time is more chaotic with some pattern but really little stability in the metric, while the route number metric is solid, unwavering.
meatball-responsetime - 750
meatball-routenumber - 750

Comparing Metrics with Themselves

This router meatball is a small office router, with little variation in the routing, however a WAN distribution router would be generally stable, but it would have a little more variability. How could I get an alarm from either of these without configuring some complex static thresholds?

The answer is to baseline the metric as it is and compare your current value against the baseline, this method is very useful for values which are very different on different devices, but you want to know when the metric changes, example are route number, number of users logged in, number of processes running on Linux, response time in general, but especially response time of a service.

The opCharts Dynamic Baseline and Threshold Tool

Overall this is what opTrend does. The sophisticated statistical model it builds is very powerful and helps spots these trends with the baseline tool. We have extended opTrend with some additional functionality so that you can quickly get alerts from metrics which are important to you.

What is really key here is that the baseline tool will detect downward changes as well as upward changes, so if your traffic was reducing outside the baseline you would be alerted.

Establishing a Dynamic Baseline

Current Value

Firstly I want to calculate my current value, I could use the last value collected, but depending on the stability of the metric this might cause false positives, as NMIS has always supported, using a larger threshold period when calculating the current value can result in more relevant results.

For very stable metrics using a small threshold period is no problem, but for wilder values, a longer period is advised. For response time alerting, using a threshold period of 15 minutes or greater would be a good idea. That means that there is some sustained issue and not just a one off internet blip. However with our route number we might be very happy to use the last value and get warned sooner.

Multi-Day Baseline

Currently two types of baselines are supported by the baseline tool, the first is what I would call opTrend Lite, which is based on the work of Igor Trubin’s SEDS and SEDS lite, this methods calculates the average value for a small window of time looking back the configured number of weeks, so if my baseline was 1 hour for the last 4 weeks and the time now is 16:40 on 1 June 2020 it would look back and gather the following:

  • Week 1: 15:40 to 16:40 on 25 May 2020
  • Week 2: 15:40 to 16:40 on 18 May 2020
  • Week 3: 15:40 to 16:40 on 11 May 2020
  • Week 4: 15:40 to 16:40 on 4 May 2020

With the average of each of these windows of time calculated, I can now build my baseline and compare my current value against that baseline’s value.

Same-Day Baseline

Depending on the stability of the metric it might be preferable to use the data from that day. For example if you had a rising and falling value It might be preferable to use just the last 4 to 8 hours of the day for your baseline. Take this interface traffic as an example, the input rate while the output rate is stable with a sudden plateau and is then stable again.

asgard-bits-per-second - 750

If this was a weekly pattern the multi-day baseline would be a better option, but if this happens more randomly, using the same-day would generate an initial event on the increase, then the event would clear as the ~8Mbps became normal, and then when the value dropped again another alert would be generated.

Delta Baseline

The delta baseline is only concerned with the amount of change in the baseline, for example from a sample of data from the last 4 hours we would see that the average of a metric is 100, we then take the current value, for example, the spike of 145 below, and we calculate the change as a percentage, which would be a change of 45% resulting in a Critical event level.

amor-numproc - 750

The delta baseline configuration then allows for defining the level of the event based on the percentage of change, for the defaults, this would result in a Major, you can see the configuration in the example below, this table is how to visualize the configuration.

  • 10 – Warning
  • 20 – Minor
  • 30 – Major
  • 40 – Critical
  • 50 – Fatal

If the change is below 10% the level will be normal, between 10% and 20% Minor, and so up to over 50% it will be considered fatal.

In practicality this spike was brief and using the 15 minute threshold period (current is the average of the last 15 minutes) the value for calculating change would be 136 and the resulting change would be 36% so a Major event. The threshold period is dampening the spikes to remove brief changes and allow you to see changes which last longer.

Installing the Baseline Tool

Copy the file to the server and do the following, upgrading will be the same process.

tar xvf Baseline-X.Y.tgz
cd Baseline/
sudo ./install_baseline.sh

Working with the Dynamic Baseline and Thresholding Tool

The Dynamic Baseline and Threshold Tool includes various configuration options so that you can tune the algorithm to learn differently depending on the metric being used. The tool comes with several metrics already configured. It is a requirement of the system that the stats modeling is completed for the metric you require to be baseline, this is how the NMIS API extracts statistical information from the performance database.

Conclusion

For more information about the installation and configuration steps required to implement opCharts’ Dynamic Baseline and Thresholding tool, it is all detail in our documentation – here.

How To Thrive In A Post-Covid Era: 10 Predictions For Enterprise Network Infrastructures

How To Thrive In A Post-Covid Era: 10 Predictions For Enterprise Network Infrastructures

An enterprise network serves as the foundation for reliably connecting users, devices and applications, providing access to data across local area networks and the cloud, as well as delivering crucial insight into analytics.

But in the wake of a year that was no doubt shaped by COVID-19 and the disruption it brought to the industry, how have enterprise networks been impacted, and what are the requirements moving forward?

What were previously technology nice-to-haves and future infrastructure intentions, are now swiftly becoming business imperatives.

In this blog, we’ll explore our top 10 predictions for network infrastructure in 2021.

 

1.   Cloud Application Delivery

 

The traditional office-based-model has no doubt permanently changed and flexible working arrangements brought forward by the pandemic will continue. A Boston Consulting study from last year found that 63% of employees want a hybrid model whereby they continue to work from home part of the time.

Organizations will further turn to the cloud for application delivery, placing an investment in remote connectivity and new security functionality.

 

2.   Businesses Turn to Big Data and Analytics

 

The requirement for businesses to be agile, change and adapt is more prevalent than ever, and decision-makers need to identify trends and ultimately stay ahead of the curve through outcomes-based strategies.

Big data is becoming an imperative tool in every organization’s arsenal, though its presence is superfluous without the appropriate means to disseminate and analyse it.

We predict this will drive the recruitment of data professionals and further, the simplification in data management through self-service means, accessible to non-data-professionals.

“It’s really about democratizing analytics. It is really about getting insight in a fraction of the time with less skill than is possible today.” – Rita Sallam, vice president and analyst at Gartner.

 

3.   The Year of Mass Adoption for Cognitive / Artificial Intelligence

 

With big data, comes big responsibility and moreover – big processing requirements, which is where AI will be heavily recruited.

2021 will be the year of mass adoption for AI, as businesses of all levels have experienced a paradigm shift into a digital-first model. Corporate networks have been tested through remote working arrangements, uncovering major reliability issues and security threats. IT leaders are looking for a set and forget solutions that automatically provide optimization and security, which is where software such as Opmantek’s NMIS, opEvents, opConfig and Open-AudIT can assist.

Opmantek software is a key system used by IT operations teams across all industries — it acts as the dashboard of a car and tells them how fast everything is going and lets them know when something is faulty. It even predicts future faults, and that’s a big part of the AI. The longer you run our software, the smarter it gets — it learns about your IT Infrastructure and starts to automatically manage it better and deliver better information to the IT operations team.” said Danny Maher, Chairman of Opmantek .

 

4.   Hybrid Clouds in High Demand

 

Agility, speed, security, scalability and compliance are all considerations for IT decision-makers.

Though, there’s never a blanket / one size fits all solution for every business use case, and so the demand for hybrid cloud environments will continue to grow. The traditional model of cloud providers is that of a one-stop-shop. However, we predict as demand grows; cloud market leaders will introduce greater interoperability and further allow users to introduce cloud tools across their existing on-campus networks. Collaboration between cloud providers may even be on the cards as users demand greater flexibility.

 

5.   Networking Virtualization

 

Network virtualization offers many benefits by automating and simplifying processes, including network configuration flexibility, improved control over-segmentation, speed, increased security and cost savings.

According to research by Spiceworks, 30% of businesses currently use network virtualization technology — and an additional 14% plan to use it within the next 2 years.

 

6.   Unified Communication And Collaboration Tools Are Here To Stay

 

End-user adoption is often one of the greatest barriers for IT professionals looking to implement new software. However, seemingly overnight, employees were catapulted into a reality where unified communications as a service (UCaaS) was no longer just an occasional collaboration tool, but rather a necessity of the employment.

We have changed our habits and the way in which we do business. Even as the workforce begin to transition back to office or hybrid office/work from home environments, there’s no doubt that UCaaS is here to stay. Providers will introduce new functionality and continue to diversify their offering to accommodate hybrid working in 2021.

 

7.   WiFi Gets an Upgrade

 

Businesses and consumers alike want things faster, easier and more efficient, and WiFi is no exception. Enter WiFi 6e.

6e not only offers new airwaves for routers to use, but it doesn’t require overlapping signals.

One of the major benefits of 6e is a reduction in network congestion, specifically in areas where users are closely spaced.  As the pandemic continues to unfold, rush hour and crowded spaces are less of an issue, so it may be a waiting game as to when in 2021 we realise 6e’s true potential.

 

8.   IoT (Internet of Things) – More than just Alexa

 

As digital transformation is on the rise, so is IoT and its use cases. A SecurityToday article forecasted that by 2021 there would be 35 billion IoT devices installed worldwide.

IoT is already revolutionizing the way key industries do business, however, healthcare will double down in 2021. Reduced access to face-to-face medical contact has accelerated the need for remote care, and according to Allied Market research – the global internet of things in the healthcare market is expected to reach $332.672 billion by 2027.

 

9.   A Focus on Cybersecurity

 

In light of recent high profile cybersecurity attacks which infiltrated private companies, state and federal organizations by inserting malicious code into trusted software; cybersecurity and secure network monitoring will be paramount.

If you have data or services of value, you need to protect it properly. Keith Sinclair – CTO & Co-founder of Opmantek says, “It is critical to business continuity and data security that you have security controls in your environment to mitigate risk.”

 

10.    Infrastructure Management Software Leveraged

 

Application demands are continuing to grow and networks must respond. Network professionals must find means of simplifying these increasingly complex systems and environments. Here’s where automated network management software will be leveraged.

Opmantek Software serves to augment a network engineering or system administration role. As well as emulating actions that network engineers take within a network management system, it can also perform advanced maintenance tasks, assist in the interpretation of network data and communicate effectively with other digital systems in order to categorise, resolve and escalate potential network issues.

 

 

For more information about Opmantek and the services we provide, get in touch. Our network engineers are available to chat through specific issues you may be facing within your own network environment.

Book a Demo

[USER STORY] Children’s Organization Achieves a New Level of Network Visibility & Control

[USER STORY] Children’s Organization Achieves a New Level of Network Visibility & Control

Learn how this IT department now has a cost-effective, easy to use IT Audit solution that tells them exactly what is connected to their network, how it is configured and when the configuration changes.

 

If you have an interesting story to tell about how your company is using Opmantek software and would like to participate in a case study, please send us a note via the contact us page.

 

Book a Demo

[White Paper] Optimizing and Automating Event Management to Support Incident Management

[White Paper] Optimizing and Automating Event Management to Support Incident Management

This guide is designed for those who’d like to find a better way to filter out and optimize the amount and type of events showing up in their operations team’s Inbox; providing the information needed to troubleshoot and resolve incidents on their network, before the customers and the business sees the issue. 

 

Download this White Paper by Network Operations Expert Rob Pavone for strategies and solutions on reducing the organization’s mean time to resolution (MTTR), eliminate the unnecessary “noise” seen from the event notifications & more. 

Key Points:

    • What the difference is between Events & Incidents
    • How to Filter, correlate and suppress Event Notifications
    • Learn proactive ways to automate, detect, diagnose and action events
    • Enriching Event Information to support incident management
    • What are the musts for integrated management processes & tracking tools
    • How to use Opmantek’s tools to specifically support your solutions
    • Case study solutions on how to reduce your teams MTTR (the Opie Challenge)

Get the E-Book

The Concept of Inventory in opCharts

The Concept of Inventory in opCharts

Join Nick Day, Senior APAC Engineer, to discuss the real life business problems that brought this opCharts Inventory feature to life, how it is best applied and what this means for your organization.

Nick explains how one of our MSP’s saved an enormous amount of time using our Live Inventory Data and how you can get on top of your warranty states in seconds for every device and part on your network.

Dive deep into config related information in terms of BGP peer tables, OSPF neighbours and Mac addresses plus how to find all your end-of-life equipment easier and quicker than ever.

Key Takeaways:

  • Find any interface among tens of thousands
  • How to find devices by their serial numbers
  • How to find config related information in terms of inventory
  • Find your Virtual Machines and where they are
  • What this means for your business!

Key Timestamps:

  • 0:34 – The power of Inventory Search
  • 2:14 – How to use Inventory & the large MSP that go this feature started
  • 3:43 – How to find Types of equipment instantly eg end of life hardware, warranties, device locations, BGP peer tables, assets, discs, vm’s etc
  • 5:52 – How much time does this save?
  • 6:27 – Where does Inventory find the information?
  • 7:03 – How to Free Trial these product
  • 7:57 – Whats new with Metrics, Health and Dash boarding
  • 9:41 – New capabilities with Dashboards

[E-Book] Building a Topological Diagram

[E-Book] Building a Topological Diagram

This E-book will teach you how to build dynamic, live and informational diagrams based on the logical Layer 2 connections devices have.

Key Points Discussed:

    • Where & how to get started creating a new topological diagram
    • Flexible design options to best suit your device discovery needs & requirements
    • How to add nodes, maps, groups, links, interface links, placeholders, neighbours, subnets
    • Using opCharts as a complete dashboarding solution that visualises your infrastructure.

Get the E-Book