Tracing Message Processing in the EIC Cluster

Objective

After completing this lesson, you will be able to trace an integration message through EIC and explain Message Processing Logs, their storage, monitoring, and self-healing mechanisms.

Introduction

Lessons 3.1 through 3.3 established what EIC runs on, which components it consists of, and how they map to Kubernetes objects. This lesson shifts the perspective from static architecture to dynamic behavior.

This lesson traces what happens at runtime- following a single integration message from its arrival at EIC to its delivery to the target system, and then examining what is recorded, where it is stored, and how it becomes visible to operations engineers. This end-to-end trace brings together the architectural concepts from Units 1, 2, and the preceding lessons of this unit into a concrete operational picture.

The Message Journey - Overview

The following diagram traces the path of a single integration message through the EIC Kubernetes cluster - from the sender system to the target system. The four numbered stages form the structure of this lesson.

Message processing and monitoring within EIC architecture

The Policy Engine is depicted here in the context of API proxy traffic. It also participates in authentication and authorization for integration flows, in collaboration with the Edge Local Authentication and Authorization component.

The processing sequence unfolds across four stages:

  1. Message Arrival- at the cluster via the Kubernetes LoadBalancer Service
  2. Processing by the Worker Pod - the iFlow logic executes
  3. Delivery to the Target System
  4. MPL Generation and Observability - the processing record is created and made accessible

Step 1 - Message Arrival at the Cluster

A sender system - for example, SAP S/4HANA - sends a message to EIC's HTTPS integration endpoint. This endpoint is a network address within Global Manufacturing AG's private network; it is not publicly accessible from the internet.

The endpoint is exposed by a Kubernetes LoadBalancer Service. When the message arrives at the LoadBalancer, it is forwarded to the Istio Ingress Gateway - a service mesh component deployed by EIC that operates at the edge of the cluster. Istio terminates the TLS connection, and the request is then authenticated by the local Auth Service in the edge-icell namespace. After authentication, the request is routed to one of the available Worker Pod replicas - in an HA deployment, there may be three Worker Pods, and traffic is distributed across them based on load..

The LoadBalancer:

  • Provides a stable IP address and port for the integration endpoint - even as individual Worker Pods are replaced, the endpoint address never changes
  • Enables horizontal scaling: adding more Worker replicas means the LoadBalancer can distribute traffic across a larger pool
  • Is provisioned automatically on managed Kubernetes platforms (AKS, EKS, GKE) and must be provisioned manually on on-premise clusters

For API proxy traffic, the request additionally passes through the Policy Engine (Envoy-based), which evaluates configured API policies - authentication tokens, rate limits, authorization - before forwarding the request to the Worker or directly to the backend system

Step 2 - Processing by the Worker Pod

Once the message reaches a Worker Pod, the Apache Camel runtime within the Worker begins executing the iFlow logic.

The iFlow execution proceeds through its configured steps:

  1. Adapter receives the message: The sender-side adapter (for example, the HTTPS adapter or the OData adapter) receives the incoming message from the LoadBalancer and prepares it for processing.
  2. Processing steps execute in sequence: The iFlow's step pipeline runs - which may include message mapping (transforming the source format into the target format), content-based routing (directing the message based on its content), filtering, enrichment from external data sources, or any other steps configured in the iFlow.
  3. Result is prepared for delivery: After all processing steps complete, the transformed message is ready for delivery to the target system.

For asynchronous iFlows: If the iFlow uses the JMS message queue, the Worker places the message into the Message Service (JMS queue) at the appropriate point in the flow. The Message Service provides durable storage - if the Worker experiences a temporary issue, the message remains in the queue and is not lost. A separate Worker thread picks up the message from the queue and continues processing. This decoupled pattern is used for reliable, ordered, or deferred delivery scenarios.

For API proxy traffic: If the incoming request is to an API proxy deployed on EIC, it first passes through the Policy Engine (Envoy proxy). The Policy Engine evaluates the configured API policies - verifying authentication tokens, checking rate limits, enforcing authorization - before forwarding the request to the Worker or directly to the backend system. Rejected requests are returned with an appropriate error response without reaching the Worker.

Step 3 - Delivery to the Target System

After processing, the Worker sends the transformed message to the target system - for example, a Quality Management System (QMS) via HTTP/REST, an on-premise system via RFC, or a file system via SFTP.

Key characteristics of this delivery step:

  • Both the sender system (SAP S/4HANA) and the target system (QMS) are typically within the same private network as the EIC cluster. The entire message exchange - from S/4HANA to EIC to QMS - occurs within the customer's private infrastructure. No message content leaves the network boundary.
  • The target adapter within the Worker handles the connection - establishing the outbound connection to the target system, formatting the message as required by the target protocol, and handling the response.
  • The target system sends an acknowledgment or response back to the Worker. This response may be used within the iFlow for further logic (for example, error handling if the target rejected the message) or may simply confirm successful delivery.
  • If the target system is temporarily unavailable, the iFlow's retry configuration governs behavior: the Worker can retry the delivery according to the configured retry interval and maximum retry count, or escalate to an error queue if retries are exhausted.

Step 4 - Message Processing Logs (MPLs): Generated and Stored Locally

After message processing completes - whether successfully, with an error, or after retries - the Worker writes a Message Processing Log (MPL) to the external PostgreSQL or SAP HANA database.

An MPL contains:

FieldDescription
Message IDA unique identifier for this processing instance
iFlow name and versionWhich integration flow processed the message
Processing start and end timeTimestamps for the complete processing duration
StatusSuccess, Failed, Retry, or Escalated
Processing steps logA record of each step executed and its outcome
Error detailIf the status is Failed, the error type, message, and stack trace
Custom log entriesAny custom log points added by the iFlow developer

Important characteristics of MPL storage and access:

  • MPLs are stored locally in the EIC database (PostgreSQL or SAP HANA). They are not automatically pushed to the SAP Business AI Platform cloud control plane in real time.
  • MPLs are accessible via the SAP Integration Suite Monitor - specifically the Message Processing view - but only when the monitor is filtered to the specific EIC runtime instance. As noted in Unit 2, Lesson 2.4, the monitoring experience is unified: the same cockpit is used for all runtimes, but logs are scoped per runtime.
  • The business message payload never leaves the EIC cluster. Only the MPL metadata (message IDs, timestamps, status codes, processing step records, and error messages) is made accessible to the SAP Business AI Platform control plane. The SAP Integration Suite monitoring UI retrieves this metadata from the EIC runtime via APIs through the Cloud Connector tunnel.

Observability

A complete monitoring approach for an EIC deployment covers multiple surfaces:

SAP Integration Suite Cockpit - Message Processing Monitor

The primary tool for integration operations engineers. From the Message Processing view, filtered to the EIC runtime instance, engineers can:

  • View the list of recent message processing instances with their status (Success, Failed, Retry)
  • Access the detailed MPL for any individual processing instance, including step-by-step logs
  • Analyze error details - error type, error message, and processing step where the failure occurred
  • Review processing duration and throughput trends

This view is identical in structure whether the iFlow runs on the Cloud Integration Runtime or on EIC. As introduced in Unit 2 Lesson 2.4, the SAP Integration Suite Cockpit provides unified visibility.

Kubernetes Native Tooling

For real-time troubleshooting at the Pod level, Kubernetes command-line tooling provides direct access:

CommandWhat It Shows
kubectl get pods -n <your-namespace>Current status of EIC Pods in the specified namespace (Running, Pending, CrashLoopBackOff)
kubectl logs <pod-name> -n <your-namespace>Container log output for a specific Pod - useful for identifying the reason for errors
kubectl describe pod <pod-name> -n <your-namespace>Detailed Pod status including events and any startup failures

These commands are particularly useful when a Pod is in an abnormal state such as CrashLoopBackOff, or when an error cannot be fully diagnosed from the SAP Integration Suite Cockpit alone..

ELM Health Dashboard

Edge Lifecycle Management provides an overall health view of each registered EIC instance - connection status, component health indicators, and version information. This is the first check when an EIC instance appears unreachable or when the Cloud Connector's connection to SAP Business AI Platform is interrupted.

Centralized Monitoring (Cloud ALM / Focused Run)

For organizations operating multiple EIC instances - as introduced in Unit 2 Lesson 2.4 - Message Processing Log data can be forwarded to SAP Cloud ALM or SAP Focused Run for cross-instance aggregation. This is an operational choice for larger deployments; it requires additional configuration but enables a unified monitoring view across all EIC instances from a single interface.

Error Handling and Self-Healing

EIC includes several layers of error handling and self-healing, operating at three levels: the integration flow level, the Kubernetes infrastructure level, and the cloud connectivity level:

iFlow-Level Error Handling

  • Camel retry behavior: The Worker's Camel runtime retries message processing according to the retry policy configured in the iFlow. Transient errors (for example, a momentary network timeout to the target system) are handled automatically without operator intervention.
  • JMS queue durability: If the Message Service is used, messages in the queue survive Worker Pod failures. When the Worker recovers, it resumes processing from the queue - no messages are lost.
  • Escalation paths: If retries are exhausted, the iFlow can be configured to route the message to an error queue, send an alert notification, or log a specific error entry in the MPL for manual review.

Kubernetes-Level Self-Healing

  • Pod restart: If a Worker Pod crashes, the Kubernetes Deployment Controller detects the failure and immediately schedules a replacement Pod. For most transient failures, this restart happens within seconds and is transparent to the integration endpoint.
  • Liveness and readiness probes: EIC components expose health probes that Kubernetes uses to determine whether a Pod is healthy and ready to serve traffic. If a Pod's liveness probe fails repeatedly, Kubernetes terminates and restarts it. If a Pod's readiness probe fails, Kubernetes removes it from the Service's load balancing pool until it recovers.
  • Node failure: In an HA deployment with multiple Worker replicas spread across multiple nodes, a single node failure removes one Worker replica from service. The remaining replicas continue processing, and Kubernetes schedules a replacement replica on a healthy node. The LoadBalancer Service automatically routes traffic only to healthy Pods.

Cloud Connectivity Loss

As established in Unit 2 Lesson 2.3, the Cloud Connector maintains an outbound HTTPS connection to SAP Business AI Platform. If this connection is interrupted:

  • Integration flows continue running: Message processing between on-premise systems continues unaffected. The data plane is operationally independent of the control plane for ongoing processing.
  • No new deployments or updates: Without the cloud connection, new iFlow deployments, configuration updates, and lifecycle commands from the control plane cannot reach the EIC instance.
  • MPL visibility is delayed: Monitoring metadata from processing events cannot be transmitted to the SAP Integration Suite Cockpit until connectivity is restored.

The EIC runtime supports an offline mode for up to 4 hours to ensure business continuity during temporary connectivity loss. During this window, iFlows already deployed and configuration already applied remain active and functional.

After this window, iFlows do not simply stop running. However, adequate support for an extended offline phase cannot be ensured - as a hybrid runtime, EIC's combined functions with SAP Integration Suite, particularly monitoring and development, require an active cloud connection.

Scenario Check-In: Global Manufacturing AG

A quality inspection completion event is triggered in SAP S/4HANA at Global Manufacturing AG's Frankfurt facility. S/4HANA sends an HTTP message to EIC's integration endpoint - an HTTPS address on Global Manufacturing AG's private Frankfurt network, exposed by the Kubernetes LoadBalancer Service provisioned as part of the EIC deployment. The LoadBalancer Service distributes the incoming request to one of the three available Worker Pod replicas.

The Worker Pod's Apache Camel runtime receives the message and begins executing the deployed iFlow. The iFlow maps the S/4HANA quality notification structure to the QMS's expected REST payload format, applies routing logic to select the correct QMS endpoint based on the inspection type, and delivers the transformed message to the QMS via an outbound HTTP call - all within Global Manufacturing AG's private Frankfurt data center network. The QMS acknowledges receipt, and the Worker records a successful processing outcome.

The Worker writes a Message Processing Log to the external PostgreSQL database: the message ID, the iFlow name and version, the processing start and end timestamp, the status (Success), and the step-level log. Within seconds, the MPL metadata becomes available to the SAP Business AI Platform control plane via the Cloud Connector, and an operations engineer monitoring the SAP Integration Suite Cockpit. - filtered to the Frankfurt EIC instance - sees the processing entry appear in the Message Processing Monitor. The actual quality inspection data in the message never left Global Manufacturing AG's network.

Key Takeaways

In this lesson, you traced the complete lifecycle of a message through the EIC cluster -from arrival at the LoadBalancer Service, through iFlow execution in the Worker Pod, to delivery and MPL storage in the external database. You learned how the Message Service and Policy Engine support asynchronous and API proxy scenarios, how monitoring works across Cockpit, Kubernetes tooling, and SAP Cloud ALM, and how error handling operates simultaneously at the iFlow, JMS, Kubernetes, and connectivity layers.