Understanding Document Grounding in Generative AI Hub

Objective

After completing this lesson, you will be able to define document grounding, SAP HANA vector engine, and embedding models.

Vector Embeddings

Introduction

Let's understand some basic concepts and features before using advanced techniques like document grounding. We will first understand the significance of the SAP HANA vector engine and vector embeddings. We will see how the generative AI hub simplifies the usage of these techniques.

Vector Embeddings

A vector is a list of numerical float values with magnitude and direction.

In generative AI, it represents an object such as a book, car, or customer record by describing the object itself, its attributes, or its characteristics for comparison.

An embedding is a number within the vector that represents data by capturing meaningful information, semantic relationships, or contextual characteristics.

A table showing text-to-embedding pairs. In blue text are: tea [75, 75], coffee [80, 60], milk [130, 80]. In green text are: dog [290, 90], puppy [240, 100], wolf [300, 100]. In orange text is congress [400, 40].

Vector embeddings in the context of the SAP HANA vector engine refer to numerical representations of objects such as text, images, or audio. The model responsible for converting text to embeddings is called the Text Embeddings model. These embeddings are stored and managed within SAP HANA Cloud's vector engine, which is part of its multimodel processing capabilities.

The vector engine allows for efficient storage, retrieval, and analysis of high-dimensional vectors, enabling advanced applications like semantic search and RAG. This integration supports combining vector embeddings with other data types, facilitating intelligent data applications, and automated decision-making.

SAP HANA Vector Engine

The SAP HANA Vector Engine is a feature of SAP HANA Cloud that enables the storage, processing, and analysis of high-dimensional vectors, such as text embeddings, alongside other types of business data. This engine is a part of SAP HANA Cloud's multimodel processing capabilities, which allow for the integration of various data types, including relational, graph, spatial, and document data.

Key functionalities of the SAP HANA Vector Engine include:

  1. Storing and managing vector embeddings, which are numerical representations of objects like text, images, or audio.
  2. Enabling advanced applications such as semantic search and RAG, which combines LLMs with private business data.
  3. Supporting efficient vector searches using SQL, with functions like L2DISTANCE() and COSINE_SIMILARITY(). L2DISTANCE() and COSINE_SIMILARITY() are vector functions available in the SAP HANA Vector Engine.

    L2DISTANCE(): This function calculates the Euclidean distance between two vectors. It is commonly used to measure the straight-line distance between points in a high-dimensional space, which is useful for various applications such as clustering and nearest neighbor search.

    COSINE_SIMILARITY(): This function computes the cosine similarity between two vectors. Cosine similarity measures the cosine of the angle between two vectors, which indicates how similar the vectors are in terms of their direction. It is widely used in text analysis and information retrieval to determine the similarity between documents or text embeddings.

    A scatter plot with points labeled tea, coffee, milk, puppy, dog, wolf, and congress. Displays Euclidean distance and cosine of the angle between vectors puppy, dog, and other points.

    These functions enable efficient vector searches and can be combined with other SQL operations within SAP HANA Cloud.

  4. Facilitating context-aware responses and automated decision-making by applying the semantic meaning captured in vector representations.

The vector engine enhances the ability of intelligent data applications to provide detailed, context-aware responses and improves the overall efficiency and scalability of data processing within SAP HANA Cloud.

Grounding in Generative AI Hub

Document grounding is a process that combines generative LLMs with advanced information retrieval techniques to improve the quality and accuracy of responses. It achieves this without the time, complexity, and expense of training or fine-tuning an LLM with company-specific data. Instead, it uses a customer’s own knowledge sources (such as HR policy manuals) to supplement the LLM's internal representation of information, making the models more accurate and reliable.

Diagram of Generative AI workflow showing data ingestion, orchestration, and retrieval. Integrates SAP HANA Cloud vector engine for search and embedding, processing document store content.

Here is how document grounding in generative AI hub uses embedding models and the SAP HANA Cloud vector engine to enhance the contextual relevance and accuracy of AI responses. The system comprises several key components:

  1. Document Store

    Houses various document types (for example, PDF, text files) which serve as the knowledge base.

  2. Generative AI Hub

    This central hub manages the processing and retrieval of information. It consists of three main stages:

    • Data Ingestion: Incoming documents undergo preprocessing, chunking (breaking into smaller segments), and embedding (converting text into numerical vectors).
    • Orchestration: This stage handles the overall workflow, including the grounding process and access to Large Language Models (LLMs).
    • Retrieval: This stage retrieves relevant information using embedding search and federated search (searching across multiple data sources). Query embeddings are generated to facilitate similarity search.
  3. SAP HANA Cloud Vector Engine

    This component provides the underlying infrastructure for similarity search based on the generated embeddings and stores user-provided content.

  4. Embedding & Similarity Search

    Connects the retrieval stage with both SAP and non-SAP content sources based on similarity between query and document embeddings.

    In essence, the system takes documents, processes them into a searchable format, and uses a combination of embedding and federated search techniques to retrieve relevant information based on user queries, facilitating grounded text generation by LLMs.

Summary

In summary, grounding in the generative AI hub uses the SAP HANA vector engine to enhance the contextual relevance of AI responses. The generative AI hub serves as an abstraction layer to access a wide range of LLMs from various providers. The SAP HANA Cloud vector engine stores "embeddings" of unstructured data, which are numerical representations of objects, such as text, images, or audio, in high-dimensional vectors. These embeddings are used for semantic data retrieval, enabling advanced text search and similarity queries.

This helps organizations to achieve more accurate, contextually relevant AI responses, driving better decision-making and innovation.

Log in to track your progress & complete quizzes

Learning

SAP FacebookSAP YoutubeSAP LinkedIn