Automated pattern analysis and anomaly detection in Amazon CloudWatch Logs

Log data analysis is known as the process of collecting, processing, and analyzing generated data. When using AWS Cloud, this is an essential task for every business but often takes a lot of time and effort if using manual methods. Therefore, AWS has applied advanced AI technology, and brought the feature of automating the log analysis process in Amazon CloudWatch, a comprehensive monitoring service provided by AWS. This service allows businesses to monitor operational status, manage performance, and use AWS resources effectively.

Below, CMC Telecom's MultiCloud expert shares about the new feature in Amazon CloudWatch.

Latest Features Just Updated in Amazon CloudWatch

The "Patterns" tab on the Logs Insights page helps users find recurring patterns in query results. This is useful for analyzing error details or identifying trends in log data.

The "Compare" button on Logs Insights allows you to quickly compare query results across different time periods, such as yesterday, last week, or last month. This helps administrators minimize the time needed to identify changes in usage patterns.

The "Log Anomalies" page in the Logs section of the navigation bar automatically detects and displays log anomalies, helping users quickly identify problems or deviations from normal operation.

New Amazon CloudWatch features provide powerful capabilities for log analysis and troubleshooting. Here are some examples of how these three features can be used in real-world scenarios.

The “Pattern” tab can be used to find recurring errors in the website log. This may indicate that the business website may be inaccessible. This issue leads to a frustrated customer service experience, causing them to leave the site and look for another business product or service.

The "Compare" function compares website traffic between today and yesterday. If traffic spikes at a particular time, this may indicate that the business website is under a distributed denial of service (DDoS) attack.

A business can use the "Log Anomalies" page to detect unusual signs such as spikes in the number of events. This can help administrators react quickly, investigate the cause of the problem to prevent it from causing disruption to the system.

Benefits of new features

New features in Amazon CloudWatch Logs not only provide convenience in log management but also provide the ability to monitor and analyze data intelligently. Automatic detection of recurring patterns and anomalies helps users quickly identify and resolve problems, reduce analysis time and increase accuracy. Integrating AI into log analysis opens up more detailed insights, capture trends and early warnings, thereby improving monitoring efficiency. These improvements transform log monitoring from a manual task to an automated, cost-effective and intelligent process.

Deploying this feature involves two main steps: Enabling anomaly detection for desired metrics from CloudWatch Metrics and creating alerts based on these anomaly detections.

The cost for this feature is based on the number of alerts initiated and the number of metrics each alert monitors. Each anomaly detection alert will be associated with three metrics: a base metric, an upper bound, and a lower bound of expected behavior. Each of these metrics will cost $0.10 per month.

For example, as a system administrator for a company, you choose 10 standard metrics to detect anomalies, including CPU performance, network traffic, etc. You decide to create alerts for 5 of them, each alert monitors 3 metrics. The cost per metric is $0.10/month, so each alert will cost $0.30/month. The total cost for 5 alerts is $1.50/month. This is a cost-effective investment of only $1.50/month to improve system monitoring and rapid response.

CMC Telecom representative shared: “With new improvements from Amazon CloudWatch Logs, data management and monitoring have become smarter and more efficient than ever. AWS continues to affirm its position as a leader in the field of cloud technology with the most optimal solutions. Currently, CMC Telecom is a high-level service partner of AWS in Vietnam. We have been accompanying AWS for 6 years, so businesses can rest assured in the expertise of the CMC Telecom team in designing, deploying and managing solutions on AWS.”

Thuy Nga

Source