Analyzing Document Grounding in Generative AI Hub

Facility Solutions Company delivers comprehensive facility management, maintenance, and cleaning services for residential and commercial properties. Their mission is to create safe, efficient, and impeccably maintained environments, allowing clients to focus on core activities. The company serves a diverse market, including luxury residential complexes, individual homes, and commercial clients such as office buildings, retail spaces, industrial facilities, and educational institutions.

The company receives thousands of emails daily, encompassing customer requests, complaints, and general inquiries. Manually processing these emails, which involves transferring data to internal applications, categorizing, and prioritizing tasks, is time-consuming, prone to errors, and frequently causes delays in addressing critical customer needs.

The company previously leveraged generative AI and prompt engineering to improve information extraction from customer emails. This allowed them to query AI models for specific tasks, leading to more accurate responses and enhanced efficiency in categorizing and prioritizing customer requests and complaints.

Despite prior advancements, the company still faces significant hurdles in email management that require deeper integration and intelligence:

1. Structured Grounding of Information: Ensuring extracted email information is grounded in a structured format for internal applications is critical for accurate categorization and prioritization. For example, a maintenance request needs details anchored within the company’s systems. The generative AI hub facilitates this through document grounding techniques, leveraging SAP HANA vector databases to store and retrieve structured information.
2. Contextual Understanding with Embeddings: Making informed decisions requires a deeper understanding of email context and semantics. For instance, recognizing the nuance of customer dissatisfaction is vital for effective resolution. The generative AI hub’s embedding models can enhance categorization and prioritization by capturing this context, enabling more timely and appropriate responses.

By addressing these evolving challenges, the company aims to streamline email categorization and prioritization further, significantly reduce manual effort, and boost its facility management services' overall efficiency and accuracy. We will see how the generative AI hub can implement these solutions in this unit.

In generative AI, a vector is a mathematical representation that encodes an object's features, typically a list of numbers. An embedding is this vector representation, learned from data, that captures the essential attributes, semantic relationships, and contextual meaning of objects such as text, images, or customer records. These embeddings enable machines to analyze, compare, and reason about data in a meaningful way.

Within the SAP HANA vector engine, vector embeddings specifically refer to these numerical representations of various data types like text, images, or audio. A Text Embeddings model is the model responsible for converting text into these numerical embeddings. These representations are then stored and managed efficiently within the SAP HANA Cloud’s vector engine, a key part of its multimodal processing capabilities.

This vector engine enables efficient storage, retrieval, and analysis of high-dimensional vectors. This, in turn, powers advanced applications such as semantic search and Retrieval Augmented Generation (RAG). Integrating vector embeddings with other data types facilitates the development of intelligent data applications and automated decision-making processes.

The SAP HANA Vector Engine, a core feature of SAP HANA Cloud, empowers storing, processing, and analyzing high-dimensional vectors, such as text embeddings, directly alongside your business data. This engine is integral to SAP HANA Cloud’s multimodal processing capabilities, supporting seamless integration with relational, graph, spatial, and document data.

Key functionalities of the SAP HANA Vector Engine include:

• Efficient Vector Management: It stores and manages numerical vector embeddings, which represent various data types, such as text, images, or audio.
• Advanced AI Applications: It enables sophisticated applications, such as semantic search and RAG, which effectively combine LLMs with business data and context.
• Powerful Vector Searches: The engine supports efficient vector searches directly via SQL, utilizing specialized functions such as L2DISTANCE() and COSINE_SIMILARITY().
  • L2DISTANCE(): This function calculates the Euclidean distance between two vectors, commonly used to measure straight-line distance in high-dimensional spaces for tasks like clustering and nearest neighbor searches.
  • COSINE_SIMILARITY(): This function computes the cosine of the angle between two vectors, indicating their directional similarity. It is widely used in text analysis and information retrieval to determine the semantic similarity between documents or embeddings.

These functions facilitate highly efficient vector searches and can be seamlessly integrated with other SQL operations within SAP HANA Cloud. The engine facilitates context-aware responses and automated decision-making by applying the semantic meaning captured in vector representations.

Overall, the SAP HANA vector engine enhances the ability of intelligent data applications to provide detailed, context-aware responses, significantly improving the overall efficiency and scalability of data processing within SAP HANA Cloud.

Document grounding is a sophisticated process that merges LLMs with advanced information retrieval techniques to enhance the quality and accuracy of AI responses. It achieves this without the extensive time, complexity, and expense of training or fine-tuning an LLM on proprietary company data, instead, grounding leverages your organization’s trusted knowledge sources (such as HR policy manuals) to directly supplement the LLM’s internal knowledge base, rendering the model more accurate and reliable.

Within the SAP Generative AI Hub, embedding models play a foundational and distinct role in enabling this grounding process. Unlike generative models, embedding models themselves do not produce text or answers. Instead, they are specialized AI components, like the Text Embedding model, designed to convert raw data (such as document chunks or user queries) into numerical vector representations. These vectors capture the semantic meaning of the data, making it computationally understandable and comparable. This capability is crucial for implementing RAG and enhancing AI responses' contextual relevance and accuracy.

The system is built upon several interconnected key components:

• Document Store: This centralized knowledge base houses various document types (e.g., PDFs and text files).
• Generative AI Hub: Acting as the central coordinator, this hub manages the entire information processing and retrieval process. It orchestrates three primary stages:
  • Data Ingestion: Incoming documents are preprocessed, segmented into smaller chunks, and converted into numerical vector embeddings using embedding models.
  • Orchestration: This stage manages the overall workflow, including the grounding process and interaction with LLMs.
  • Retrieval: Relevant information is retrieved through embedding and federated searches (across multiple data sources). Query embeddings are dynamically generated to facilitate highly efficient similarity searches.
• SAP HANA Cloud Vector Engine: This component provides the fundamental infrastructure for conducting similarity searches based on the generated embeddings, effectively storing user-provided content.
• Embedding & Similarity Search: This critical function connects the retrieval stage with SAP and non-SAP content sources, matching information based on the similarity between query and document embeddings.

Essentially, the system ingests documents, transforms them into a searchable format, and then intelligently uses embedding and federated search techniques to retrieve the most relevant information based on user queries. This finally facilitates a more robust and grounded text generation by LLMs.

This lesson introduced document grounding as an essential technique for enhancing Generative AI responses by connecting LLMs with external, trusted knowledge sources. We explored a facility management scenario to illustrate the business challenges addressed by grounding, then delved into foundational concepts like vector embeddings and the SAP HANA vector engine. You learned how the generative AI hub orchestrates this process, leveraging these components to deliver accurate, contextually relevant, and reliable AI outputs, preparing you for practical implementation.