01 Observability Scenarios

Advanced application architectures such as distributed systems, microservices, and Cloud-Native have enabled agile development, rapid iteration, and elastic scaling, while simultaneously decomposing monolithic applications into multiple independently deployed, intercommunicating composite applications. The number of applications has grown exponentially, dependencies between business modules have become intricately complex, and establishing real-time, effective mapping relationships across different business tiers and dimensions has become increasingly difficult. Meanwhile, as containers are frequently started and stopped, changes in monitoring targets and their associated metrics have become the norm, making it difficult to preserve fault scenes and effectively pinpoint the root cause of failures.

02 Scenario Pain Points

The observability challenges inherent in Cloud-Native architectures pose significant obstacles to fault analysis, root cause identification, and business continuity and stability in application operations. The key difficulties in application observation can be summarized as follows:

• Complex information dimensions make it difficult to establish multi-dimensional data correlation mappings

Monitoring and measuring Cloud-Native applications involves resource attributes and performance metrics across multiple layers—including application processes, middleware, container orchestration platforms, container processes, and infrastructure resources. Furthermore, application troubleshooting and performance profiling involve complex interactions across multiple services and components, requiring analysis of fault root causes based on request chain dependency relationships.

• Dynamic architectural changes make it difficult to preserve fault scenes and pinpoint issues.

Container deployment architectures are based on declarative, desired-state design principles, where deployed resource instances change frequently and service node drifting becomes routine. A runtime observation and analysis matrix built upon multi-dimensional detailed data and metric data correlation mappings can effectively reconstruct historical fault scenes.

03 Solution

Vertical and Horizontal Fault Tracing

Vertical: Establish drill-down analysis logic for runtime software architecture cascading objects. Construct global dependency topologies for different services based on actual business traffic, enabling panoramic analysis of a single business domain within a selectable time range. Effectively analyze service traffic loads and service health status through topology node size and color differentiation. Support drill-down analysis of service nodes with golden metrics—including service requests, load, errors, and latency—within specified time ranges. Within a service, further drill down to analyze individual interfaces or individual service instances for precise fault tracing. By associating service instances with CMDB-managed resources (hosts, containers), users can drill down to the IaaS layer to analyze how anomalies in IT infrastructure resource metrics impact service traffic.

Horizontal: Construct per-request chain tracing based on Trace correlation. Each business request generates a unique request identifier at the entry service. As traffic flows through multiple downstream services, the unique request identifier, the current node's request identifier, and upstream service information are passed as context, thereby constructing the complete business call chain. Additionally, users can extract business-characteristic data from HTTP request headers, request parameters, cookies, and other sources based on actual business scenarios to complete data instrumentation. During chain analysis, request dependency relationships with specified business characteristics can assist in business anomaly analysis.

Call Chain and Log Detail Correlation for Root Cause Identification

In scenarios where KAPM and KLC are delivered together, call chains and log details can be correlated to achieve efficient root cause identification. Through KAPM's call chain analysis capabilities, users can determine request dependencies, narrow down the troubleshooting scope, and accurately locate faults. However, detailed root cause information cannot be obtained through call chain analysis alone. In such scenarios, correlating call chains with log analysis data seamlessly bridges the last mile of troubleshooting.

04 Product Features

Feature 1: Application Analysis

Call Chain Topology: By instrumenting service processes with probes, the system automatically detects and renders the application runtime topology. Node area size differentiates service traffic volumes, and service nodes are color-coded with Apdex scores based on response latency, request errors, and other data—providing comprehensive application monitoring capabilities.

Resource Relationships: Based on probe-collected data, the system automatically correlates basic monitoring instances for hosts, databases, and message queues, enabling integrated observation between APM and infrastructure monitoring.

Metric Analysis: Golden application metrics (health status, request load, and request error count) are centrally displayed on the page, along with TOP request lists for services and interfaces.

Feature 2: Service Analysis

Service List: Displays the complete list of services belonging to the application, with aggregated summary metric data.

Metric Analysis: Displays service-perspective operational metrics, including Apdex scores, request load, request error count, and request latency.

Call Chain Topology: Displays the local upstream and downstream topology relationships originating from a specified service, precisely identifying the scope of impact on service operations.

Interface List: Centrally displays interfaces belonging to the service, with aggregated summary information.

Request List: Enumerates detailed request data, precisely recording each business operation execution. Supports one-click navigation to interface analysis, service analysis, call chain search, and other scenario pages.

Error List: Centrally displays error invocations, with support for one-click navigation to the call chain search page, where error stack trace information can be viewed directly.

Instance Analysis – Request Overview: An analysis overview page at the service instance (Java process) level, enabling users to view health status, request load, request error count, request latency, and other data.

Instance Analysis – JVM Analysis: JVM analysis for instance-level processes, including metric views for GC analysis, heap memory analysis, thread analysis, and more.

SQL/NoSQL Analysis: Application-level data analysis views for peripheral databases invoked by services, including aggregated analysis views for SQL commands, slow commands, and errors.

Feature 3: Interface Analysis

Drill down to specific service interfaces to view application runtime conditions, including an application runtime data overview, request list, and error list with aggregated analysis. Supports one-click navigation to service analysis and drill-down to call chain search for detailed inspection.

Feature 4: Call Chain Search

Trace/Span Search: A detail search page for call chain Trace and Span segments. Supports precise request detail identification via Trace/Span ID, and supports searching request lists by request latency range.

Request Details: Analyze system runtime conditions through detailed data inspection, including probe raw data and integrated analysis capabilities with log linkage.

Feature 5: Application Configuration

Health Score Configuration: Supports user-defined health score thresholds, enabling application runtime assessment under different conditions for various application scenarios.

SQL/NoSQL Slow Command Configuration: Supports user-defined threshold configuration for "slow commands/slow SQL."

05 Implementation Vision

In today's era of rapid technological advancement, every industry is continuously undergoing transformation. In the field of IT O&M, the Cloud-Native concept has become increasingly prevalent with the large-scale adoption of technologies such as microservices, cloud computing, and containerization. The new operational challenges and technical difficulties brought about by Cloud-Native architectures have also become increasingly acute. Call chain tracing technology, which has emerged in response to these challenges, will help IT O&M and even development personnel more conveniently and efficiently accomplish tasks such as system stabilization, code optimization, and fault troubleshooting. At the same time, comprehensive wide-event data is driving the monitoring domain's transformation from passive alert response to proactive full-stack observability, thereby accelerating product iteration across industries and ultimately propelling the advancement of the entire IT field.