Describing Observability in Kyma

You are now operating some workloads on the SAP BTP, Kyma runtime and want to gain more insights into your running applications. For this purpose, you want to start collecting application logs, traces, and metrics data of your application and start analyzing them. In this lesson, you will learn about the telemetry features of Kyma for collecting the relevant data and how to integrate these data with SAP Cloud Logging to analyze them.

Observability means the ability of a system to provide information about its internal state. This information can be used to monitor the system's behavior and diagnose problems. Observability is a key concept in microservice-based applications, where the number of components is large and the interactions between them are complex. Applications can expose insights about their internal state using logs, metrics, and traces. Collectively, these insights are called "telemetry".

Application logs are essential for operating and troubleshooting applications. Ideally, logs are stored in an external system for several days and can be easily filtered.

If an application logs to stdout/stderr, the Kyma Telemetry module offers an API called LogPipeline to enable the collection of these logs seamlessly. When you specify an SAP Cloud Logging endpoint as destination, you get instant access to the log data over a configurable amount of time. Additionally, if the logs of the application are in JSON format, the log will be parsed and pushed to the target in a way that you can search and filter across all log attributes.

The Telemetry module uses Fluent Bit for the collection and processing of the logs. It supports a protocol based on HTTP that is specific to Fluent Bit, and thus enables integration to all backends that support this protocol.

Distributed traces help you understand the application behavior and relations between the frontend actions and backend implementation. They can record the request chain across several workloads and execution chains inside an application. Traces consist of spans that are emitted by the individual components taking part in a trace. If you collect all the spans and send them to an observability backend, you can analyze the distributed traces going on in your SAP BTP, Kyma runtime.

The Telemetry module comes with a solution for collecting distributed traces based on the OpenTelemetry Collector. Using an API called TracePipeline, you can establish a cluster-central gateway, to which instrumented applications can push span data. Istio and many other Kyma modules are preconfigured to push spans to this gateway. If you specify an SAP Cloud Logging endpoint as destination, you get instant access to the trace data with some visualizations. To see the correlation between logs and traces, enrich your application logs with a trace ID attribute.

To instrument the span data inside your application, use the available OpenTelemetry SDKs. Additionally, a lot of auto-instrumentations are available for most of the languages and tools. For sending spans to the gateway or to the destination by the gateway, only the OpenTelemetry protocol (OTLP) is supported.

While application logs and traces usually provide request-oriented data, metrics are aggregated statistics over time. They reflect the internal state of a component. Typical statistics like the amount of processed requests, or the amount of registered users, can be very useful to monitor the current state and the health of a component. Also, you can define proactive and reactive alerts if metrics are about to reach thresholds, or if they already passed thresholds.

The Kyma Telemetry module provides an API MetricPipeline, which will establish a cluster-central gateway based on the OpenTelemetry Collector, where instrumented workloads can send metric data to. If you specify an SAP Cloud Logging endpoint as destination, you get instant access to the metric data with a basic metric explorer.

Additionally, the MetricPipeline supports different kinds of input scenarios to easily collect metrics:

• from the Kubernetes runtime about your application
• from Istio sidecars
• from any application exposing metrics in a Prometheus format

To instrument the metric data inside your application, use the available OpenTelemetry SDKs. Additionally, a lot of auto-instrumentations are available for most of the languages and tools. For sending spans to the gateway or to the destination by the gateway, only the OpenTelemetry protocol (OTLP) is supported.

The integration of SAP Cloud Logging happens in the following steps:

1. Create an instance of SAP Cloud Logging with an account and have a Kubernetes Secret available with the account credentials. SAP Cloud Logging is a SAP BTP service and the instance is created via the regular service management features.
   • With the SAP BTP Operator module, you can define a ServiceInstance resource, referencing the service plan to use and providing additional parameters for the service configuration.
   • You must also create a ServiceBinding resource, which requests an account in the service. Here, you can configure the regular rotation of the credentials.
   • The ServiceBinding will trigger the generation of a Kubernetes Secret that holds the credentials.
2. Define a LogPipeline with the Telemetry API referencing the Secret that holds the credentials.
   • With the Kyma Telemetry module, you can define a LogPipeline, which will create the relevant agent or gateways in the cluster for collecting the telemetry data.
   • The LogPipeline references the Secret with the service credentials, which will be used by the log agent to ship the data.
   • When the automatic credential rotation of the ServiceBinding updates the Secret, the log agent is reconfigured with the new credentials.

The following diagram illustrates the integration with the LogPipeline as example. The same is supported for the TracePipeline and MetricPipeline features.

In this lesson, you have learned about the different aspects of the Telemetry module in Kyma and how to integrate them with SAP Cloud Logging, in order to get instant access to the telemetry data for analysis and alerting.

In the next exercise, you will practice the integration with SAP Cloud Logging using the Kyma Telemetry API, and start discovering the available data.