Implementing Document Grounding in the Orchestration Service

The grounding capability is integral to the orchestration module of the generative AI hub. This module facilitates specialized data retrieval from vector databases, ensuring the AI’s responses are grounded in external, context-relevant information. In addition, the Pipeline API integrates vector stores like the managed SAP HANA database, which is directly accessible and works within the SAP Generative AI Hub via SAP AI Core.

SAP AI Launchpad also provides the grounding management app lets you manage the lifecycle of your data pipelines.

The generative AI hub supports robust document grounding through several key features and streamlined processes:

• Access to Diverse LLMs: The generative AI hub provides instant access to a wide range of LLMs from various providers, including Azure OpenAI models (gpt-5), Anthropic models(anthropic--claude-4-sonnet), and open-source models like Mistral and Meta models. This broad access enables you to orchestrate multiple LLMs to best suit grounding and content generation needs. To see all the available models, refer to 3437766 - Availability of Generative AI Models.
• Seamless Integration with SAP AI Launchpad: You can execute and monitor grounded prompts directly within the SAP AI Launchpad. This integration shows how generative AI, combined with your business data, can directly assist business processes while leveraging the underlying SAP AI Core infrastructure for secure operations.
• Efficient Document Indexing: Unstructured and semi-structured documents are preprocessed, divided into chunks, and converted to numerical embeddings with embedding models. These embeddings are efficiently stored in the SAP HANA Vector Engine for rapid and precise querying, fundamental to grounding AI responses in real, relevant data.

These integrated features empower you to build generative AI solutions that leverage your organization's trusted document repositories and provide reliable, transparent, and contextually accurate responses.

Using the RAG approach, you will utilize the Document Grounding module within the Orchestration Service to generate content effectively. This module aligns user queries with relevant documents by retrieving them from a knowledge base. This knowledge base can be sourced from various repositories, including SharePoint folders, Elasticsearch engines, or dedicated vector data repositories.

You perform the following steps:

• Create the knowledge base with relevant documents.
• Configure the Document Grounding module in the Orchestration Service.
• Generate content using the RAG approach based on the knowledge base.

• Install the SAP Cloud SDK for AI (Python) - generative using the command:
  Python

  Copy codeSwitch to dark mode

  1
  %pip install "sap-ai-sdk-gen[all]"

• Set the credentials for the SDK.

A detailed setup is also described here to prepare a knowledge base and verify it.

1
orchestration_service_url = <your url code from deployment of the orchestration service>

1234567891011121314151617181920212223
# Set up the Orchestration Service
aicore_client = get_proxy_client().ai_core_client
orchestration_service = OrchestrationService(api_url=orchestration_service_url)
llm = LLM(
    name="gpt-4o",
    parameters={
        'temperature': 0.0,
    }
)
template = Template(
            messages=[
                SystemMessage("""Facility Solutions Company provides services to luxury residential complexes, apartments,
                individual homes, and commercial properties such as office buildings, retail spaces, industrial facilities, and educational institutions.
                Customers are encouraged to reach out with maintenance requests, service deficiencies, follow-ups, or any issues they need by email.
                """),
                UserMessage("""You are a helpful assistant for any queries for answering questions.
                Answer the request by providing relevant answers that fit to the request.
                Request: {{ ?user_query }}
                Context:{{ ?grounding_response }}
                """),
            ]
        )

12345678910111213141516171819
# Set up Document Grounding
filters = [DocumentGroundingFilter(id="vector",
                                   data_repositories=["<add your data repository ID>"],  ,
                                   search_config=GroundingFilterSearch(max_chunk_count=15),
                                   data_repository_type=DataRepositoryType.VECTOR.value
                                   )
]

grounding_config = GroundingModule(
            type="document_grounding_service",
            config=DocumentGrounding(input_params=["user_query"], output_param="grounding_response", filters=filters)
        )

config = OrchestrationConfig(
    template=template,
    llm=llm,
    grounding=grounding_config
)

123456
response = orchestration_service.run(config=config,
                            template_values=[
                                TemplateValue("user_query", "List the issues that are reported by customers."),
                            ])
print(response.orchestration_result.choices[0].message.content)

1
print(response.module_results.grounding.data['grounding_result'])

In this lesson, you have accomplished the following key tasks:

• Established a Vector Knowledge Base: You learned to upload and vectorize documents, forming a robust, searchable knowledge base.
• Configured the Document Grounding Module: You set up the module within the Orchestration Service to intelligently retrieve relevant documents based on user queries.
• Generated Grounded AI Responses: You utilized the Orchestration Service to produce accurate and contextually relevant answers, demonstrating the power of grounding.

Building on your understanding of document grounding’s foundational concepts, vector embeddings, the SAP HANA vector engine, and RAG, this lesson provided a practical deep dive into implementation. You successfully learned to establish a vector knowledge base, configure the Document Grounding module within the Generative AI Hub’s Orchestration Service, and generate accurate, context-relevant AI responses. This end-to-end experience solidified how grounding significantly enhances AI’s reliability and relevance by integrating precise, external data into your solutions.

This course provided comprehensive conceptual and practical coverage of a grounding technique to solve business problems.

Throughout this learning journey, you started with discovering the significance of SAP Business AI and how it addresses business challenges by automating processes, enhancing decision-making, and supporting scalable, responsible innovation.

You then explored the foundations of LLMs, how they function, their remarkable strengths, and their inherent limitations. You saw how LLMs can automate tasks, generate content, and synthesize information at scale, but also why it’s essential to use them thoughtfully, with an awareness of risks like hallucinations, bias, and data privacy. You discovered that practical prompt engineering is the key to guiding LLMs to deliver relevant and reliable results.

You then delved into SAP’s generative AI hub, part of AI Foundation, understanding how it is a secure gateway to enterprise-grade AI. The hub empowers you to access and orchestrate leading LLMs, manage prompts, and confidently integrate AI into business processes. It ensures that your solutions are grounded in a real business context, reliable through robust security and compliance, and responsible by design.

Building on this foundation, you learned practical techniques for solving real business problems with LLMs, from prompt development and refinement to integrating prompts into applications using SDKs. You understood the art of prompt engineering, version control, and evaluation, ensuring your AI solutions are accurate, actionable, and scalable. The scalability is supported by creating and managing prompt templates in generative AI hub. Advanced techniques like few-shot prompting and meta prompting further enhanced the quality and consistency of AI outputs, enabling you to tackle even more complex scenarios.

Finally, you discovered how document grounding connects LLMs to your organization’s trusted knowledge sources, ensuring accurate and context-aware responses. You can transform documents into searchable vectors by leveraging SAP HANA Vector Engine and embedding models, enabling semantic search and RAG. Grounding minimizes hallucinations and aligns AI outputs with real business facts, making your solutions more trustworthy and effective.

In summary, you are now equipped to design, build, and deploy intelligent, reliable, and business-ready AI solutions, combining the power of LLMs, robust prompt engineering, and SAP Business AI to drive real value and innovation.