Resilience and Caching

Circuit Breaker is a design pattern where the application automatically stops making remote service calls if the remote service call has failed too many times.  The CircuitBreaker is implemented via a finite state machine with 3 normal states.

• CLOSED
• OPEN
• HALF_OPEN

When the number of consecutive remote service call failures exceed a configured threshold, the circuit breaker switches to the OPEN state.  No more remote service calls are made by the application for the duration of the timeout period.  After the timeout expires, the circuit breaker switches to the HALF_OPEN state.  Limited remote service calls are made by the application.  If these remote service calls succeed, the circuit breaker switches to the CLOSED state and normal operations are resumed.  

Bulkhead pattern is used to limit the number of concurrent requests to a remote service.  If the number of concurrent incoming requests exceed the configured threshold, the bulkhead is said to be saturated.  In this case, further requests are automatically stopped until existing requests are completed.

If the response time of a remote service call exceeds the configured timeout duration, the remote service call is considered to have failed.  Resilience4j allows setting custom timeout durations for every remote service call.  

Additionally, the SAP Cloud SDK enables you to provide fallback functions.  For example, if the bulkhead is saturated, or if the circuit breaker switches to the OPEN state, SAP Cloud SDK can check if a fallback method is implemented and execute it automatically.  So even if a remote service is unavailable, you can still provide some meaningful result.  For example, you can return cached data.

The most important aspects of a cache are its size and lifetime of its items.  Determining how long objects are to be cached and the memory to be allocated for caching needs to be carefully considered.  If the application is repetitively reading large amounts of data by making remote service calls, then using caches significantly improves responsiveness and reduces load on the remote server. 

Caching is applicable whenever

• You are willing to spend some memory to improve responsiveness
• You expect that specific data will get queried more than once
• Your cache will not need to store more data than what would fit in RAM

If your application meets these requirements, then it is highly recommended to use the caching features provided by SAP Cloud SDK. 

SAP Cloud SDK makes it easy to cache your requests since it handles most of the complexity under the hood.  This includes handling of tenant-aware requests which is essential in a multi-tenant application.  SAP Cloud SDK uses JCache as the underlying caching technology.