Implementing Document Grounding in the Orchestration Service

• Prepare your knowledge base before using the Grounding module in the orchestration pipeline.
• The generative AI hub provides several options for users to prepare their knowledge base data:
  • Upload documents to a supported data repository, then run the data pipeline to vectorize the documents. For more details, refer to the Pipeline API.
  • Use the Vector API to directly provide chunks of the document. For additional information, see the Vector API.

Choose one of the following options to use grounding:

• The pipeline collects documents and segments the data into chunks.
• It generates embeddings, which are multidimensional representations of textual information, and stores them efficiently in a vector database.

The process involves the following steps:

1. Perform Initial One-Time Administrative Steps: Create a resource group and a generic secret for grounding. For more information, see:
   • Create a Resource Group for Grounding
   • Grounding Generic Secrets for Microsoft SharePoint
2. Prepare Vector Knowledge Base: Configure the Pipeline API to read unstructured data from data repositories and store it in a vector database. Use the Pipeline API to:
   • Read unstructured documents from various data repositories. Break the data into chunks and create embeddings.
   • Store the multidimensional representations of the textual information in the vector database.
   • Provide a repository ID to access the data.

     For more information, see Preparing Data Using the Pipeline API.

Provide chunks of data directly and store them using the Vector API. The process involves the following steps:

1. Perform Initial One-Time Administrative Steps:Create a Resource Group for Grounding.
2. Prepare Vector Knowledge Base: Provide chunks of information directly and store data in the vector database using the Vector API. Use the Vector API to:
   • Create collections.
   • Create documents by directly using the chunks of data provided by users.
   • Store data in the vector database.
   • Assign repository IDs to access the data.
   • For more information, see Preparing Data Using the Vector API.
3. Configure Grounding Module in the Orchestration: In the orchestration pipeline, you add configuration for the grounding requests:
   • Create a grounding request configuration in the orchestration pipeline using the repository IDs and filters.
   • Run the orchestration pipeline and check that the response refers to the user data. For more information, see Using the Grounding Module.

Now, you must define the configuration for the Document Grounding module, including setting up filters, and specifying the data repositories.

1
orchestration_service_url = <your url code from deployment of the orchestration service>

You must have at least one orchestration-compatible deployment for a generative AI model running. For more information, see and Create a Deployment for Orchestration in SAP AI Core.

Next, you must import all relevant libraries. See the code in the code repository here.

1234567891011121314151617181920212223
# Set up the Orchestration Service
aicore_client = get_proxy_client().ai_core_client
orchestration_service = OrchestrationService(api_url=orchestration_service_url)
llm = LLM(
    name="gpt-4o",
    parameters={
        'temperature': 0.0,
    }
)
template = Template(
            messages=[
                SystemMessage("""Facility Solutions Company provides services to luxury residential complexes, apartments,
                individual homes, and commercial properties such as office buildings, retail spaces, industrial facilities, and educational institutions.
                Customers are encouraged to reach out with maintenance requests, service deficiencies, follow-ups, or any issues they need by email.
                """),
                UserMessage("""You are a helpful assistant for any queries for answering questions.
                Answer the request by providing relevant answers that fit to the request.
                Request: {{ ?user_query }}
                Context:{{ ?grounding_response }}
                """),
            ]
        )

This Python code sets up an orchestration service crucial for handling complex tasks. It initializes an AI core client and configures an orchestration service with a given URL. The code then sets up an LLM with specific parameters to ensure consistent responses. Lastly, it creates a message template to aid in answering customer inquiries efficiently and effectively.

Next, we set up grounding services.

12345678910111213141516171819
# Set up Document Grounding
filters = [DocumentGroundingFilter(id="vector",
                                   data_repositories=["<add your data repository ID>"],  ,
                                   search_config=GroundingFilterSearch(max_chunk_count=15),
                                   data_repository_type=DataRepositoryType.VECTOR.value
                                   )
]

grounding_config = GroundingModule(
            type="document_grounding_service",
            config=DocumentGrounding(input_params=["user_query"], output_param="grounding_response", filters=filters)
        )

config = OrchestrationConfig(
    template=template,
    llm=llm,
    grounding=grounding_config
)

This Python code sets up and configures a document grounding service. It defines filters for document repositories and specifies search parameters. The grounding configuration is then integrated into an orchestration system, which uses templates and a language model to process and respond to user queries. This setup ensures efficient and accurate document retrieval based on user inputs.

Run the orchestration service with the configured settings to generate answers based on user queries.

123456
response = orchestration_service.run(config=config,
                            template_values=[
                                TemplateValue("user_query", "List the issues that are reported by customers."),
                            ])
print(response.orchestration_result.choices[0].message.content)

This Python code sends a request to an orchestration service to run a specific configuration. It includes a template value with a user query asking to list customer-reported issues. After running the service, it prints the response from the orchestration result, specifically the message content of the first choice. This helps automate and fetch data on customer issues efficiently.

You get the following output:

You can see that the list of issues reported by customer, which is grounded in mails that customers provided.

You can see the documents that were retrieved for the context from the data repository.

1
print(response.module_results.grounding.data['grounding_result'])

This code extracts and displays the value of 'grounding_result' from a nested data structure within the 'response' object. This specific piece of data could be critical for understanding the outcome of a grounding module, making the code essential for debugging or analysis.

The output lists all the relevant mails used for response earlier, providing a deep insight into the context of the grounding technique. You can see this output in the repository.