Demonstrating Generative AI Examples and Reference Architecture

The RAG architecture combines the strengths of Foundation Models (FMs) or Large Language Models (LLMs) with external retrieval or search mechanisms to improve the performance of LLMs. It is used for tasks such as question answering and knowledge-intensive Natural Language Processing (NLP). The architecture is an interesting fusion of retrieval-based and generation-based approaches in NLP.

RAG is a neural architecture that combines the strengths of Foundation Models (FMs) or Large Language Models (LLMs) with external retrieval or search mechanisms. The RAG architecture has one clear goal: to improve the capability of LLMs. It does this by allowing them to pull relevant information from a vast corpus, much like how search engines retrieve relevant web pages based on queries. RAG is used for various tasks such as question answering (Q&A) and knowledge-intensive Natural Language Processing (NLP) tasks. The architecture represents an interesting fusion of retrieval-based and generation-based approaches to NLP.

Here is a high-level flow of how Retrieval Augmented Generation works:

• Question Encoding: The user provides a question or prompt, which is then encoded by an embedding model into a dense vector (embedding).

• Document Retrieval: This embedding is used as part of a query to retrieve relevant documents (or chunks) from a large corpus of pre-embedded documents applying SAP HANA Cloud vector engine. The retrieval is typically done using a similarity search like cosine similarity. The top-k most relevant documents or passages are retrieved based on their proximity to the query vector.

• Answer Generation: The retrieved documents and the original question are fed into an LLM to generate an answer. The model is instructed to generate relevant responses based on both the input question and the retrieved passages.

Retrieval Augmented Generation and Generative AI on SAP BTP

GenAI Mail Insights is a comprehensive SAP BTP solution and automation with a range of AI features that analyzes incoming e-mails and provides categorization, sentiment analysis, urgency scoring, and key facts. The development of the app and further details can be found on GitHub and in a blog post.

This multitenant application, crafted by a potential SAP partner or customer, is tailored for SAP Business Technology Platform (SAP BTP). This scenario presents a comprehensive SaaS solution for enhancing customer support within a travel agency, using advanced e-mail insights and automation. The system uses Large Language Models (LLMs) to analyze incoming e-mails and provide core insights such as categorization, sentiment analysis, and urgency assessment. It analyzes e-mails, and extracts key facts and customizable fields like location. You can manage this through a dedicated configuration page.

One innovative feature is the use of e-mail embeddings to identify similar historical e-mails, which aids in understanding how similar requests were handled previously. This ensures consistent and efficient customer service. The code also proves that the system can summarize and translate both e-mail subject and body, ensuring seamless comprehension across languages.

Furthermore, the system takes automation to the next level by generating potential responses for customer inquiries. This response generation is influenced by configurable actions and services, enhancing response accuracy and speed. The flexibility to connect with SAP systems like SAP Concur adds an enterprise dimension, allowing seamless integration of processes and data.

The following are the direct effects on the mail application. The detailed description can be found in the following blog post: GenAI Mail Insights - Leveraging the generative AI hub in SAP AI Core to improve customer support

The development of the complete app is described in detail at: Git Hub Sample: GenAI Mail Insights - Develop a CAP-based (multitenant) application using GenAI and RAG on SAP BTP. There is also a video to help you understand the application.

• Blog Post : GenAI Mail Insights - Leveraging the generative AI hub in SAP AI Core to improve customer support
• Video: Generative AI from user perspective and Reference Application Mail Insights