Using SAP Cloud SDK for AI to Interact with Orchestration Services

To interact with the Orchestration Service programmatically, you generally follow these high-level steps:

• Authentication: Obtain an authentication token (Bearer token).
• Deployment: Create a deployment for the Orchestration Service and obtain your unique ORCH_DEPLOYMENT_URL.
• Consumption: Programmatically send requests to your deployed Orchestration Service endpoint. This is the primary focus of this lesson.

The SAP Cloud SDK for AI for Python provides a robust and intuitive way to define, configure, and execute Orchestration Service workflows directly from your Python applications. This allows you to integrate complex Generative AI capabilities into your business logic with ease.

Before proceeding, ensure you have completed the Python SDK installation and configuration steps discussed earlier, including setting up your ~/.aicore/config.json file.

1. Get an Auth Token for Orchestration
2. Create a Deployment for Orchestration
3. Consume Orchestration

1. Initialize the Orchestration Service Client
2. Define the Orchestration Template
3. Define the LLM
4. Create the Orchestration Configuration
5. Run the Orchestration Request

First, you need to import the necessary class and point your SDK client to the ORCH_DEPLOYMENT_URL you obtained when deploying your Orchestration Service.

1234
# Assuming YOUR_API_URL is the ORCH_DEPLOYMENT_URL.
YOUR_API_URL = "https://your-orchestration-deployment-url.ai.sap.com/orchestration-service-endpoint"
from gen_ai_hub.orchestration.service import OrchestrationService

The Template object in the SDK allows you to programmatically define the structure of your prompt, including system, user, and assistant messages. It also lets you define placeholders for dynamic input and set default values.

Let’s consider an example for a translation assistant.

12345678910111213141516
  from gen_ai_hub.orchestration.models.message import SystemMessage, UserMessage
from gen_ai_hub.orchestration.models.template import Template, TemplateValue

template = Template(
    messages=[
        SystemMessage("You are a helpful translation assistant."),
        UserMessage(
            "Translate the following text to {{?to_lang}}: {{?text}}"
        ),
    ],
    defaults=[
        TemplateValue(name="to_lang", value="German"),
    ],
)

This Python code sets up a template for a translation assistant. It defines a SystemMessage to establish the assistant’s role and a UserMessage that includes placeholders ({{?to_lang}} and {{?text}}) for the target language and the text to be translated. It also sets a default target language to "German", making the template ready for immediate use in translating text to German without explicitly specifying the language each time.

Next, you define which LLM the Orchestration Service should use for the prompt_templating module. This involves specifying the model’s name, version, and any specific parameters (like max_tokens or temperature) to control its behavior.

123456

from gen_ai_hub.orchestration.models.llm import LLM

llm = LLM(name="gpt-4o", version="latest", parameters={"max_tokens": 256, "temperature": 0.2})

This code creates an instance of an LLM, for example here it is "gpt-4o". It configures the orchestration service to use the latest version available in generative AI hub.

It also sets parameters like a maximum token limit of 256 for the response and a temperature of 0.2 to control response variability (lower temperature means more deterministic outputs). This setup defines the generative AI engine for your orchestrated task.

The OrchestrationConfig object combines your defined template and LLM (and optionally, other modules like grounding, filtering, masking, translation) into a single, executable configuration for your workflow.

12345678

from gen_ai_hub.orchestration.models.config import OrchestrationConfig

config = OrchestrationConfig(
    template=template,
    llm=llm,
)

This code initializes an instance of OrchestrationConfig using the template and LLM variables defined in the previous steps. This configuration object encapsulates the entire design of your specific orchestrated AI workflow, ensuring the system uses the specified template and language model parameters when executed.

Finally, you execute your defined orchestration workflow by calling the run method of the OrchestrationService client. You pass any dynamic template_values required by your template’s placeholders.

123456789101112

# Instantiate the OrchestrationService with your API URL and defined config
orchestration_service = OrchestrationService(api_url=YOUR_API_URL, config=config)

# Run the orchestration, providing values for the template placeholders
result = orchestration_service.run(template_values=[
    TemplateValue(name="text", value="The Orchestration Service is working!")
])

# Print the content of the LLM's response
print(result.orchestration_result.choices[0].message.content)

This code first initializes the OrchestrationService client with your deployed endpoint URL and the configured workflow. It then executes the orchestrated task by calling run(), supplying the text "The Orchestration Service is working!" for the {{?text}} placeholder in your template. The result’s content (the translated text in German, based on the defaults from the template) is then printed, demonstrating a successful programmatic interaction with the Orchestration Service.

The example above demonstrated a simple translation using prompt_templating and LLM. However, the real power of the Orchestration Service lies in its ability to chain multiple modules. You can further extend your OrchestrationConfig to include:

• grounding: To inject real-time, factual data from your enterprise sources.
• filtering: For input and output content safety checks.
• masking: To protect sensitive data like PII.
• translation: To handle multilingual inputs and outputs.

You can use modules for content filtering and other tasks. Obtain Optional Modules for details and the latest orchestration configurations.

You have successfully learned how to use the SAP Cloud SDK for AI (Python) to programmatically interact with the Orchestration Service. You’ve walked through the steps of initializing the service client, defining prompt templates and LLM parameters, creating a comprehensive orchestration configuration, and executing a workflow. This programmatic control is vital for integrating advanced Generative AI capabilities seamlessly into your enterprise applications, enabling you to build scalable, automated, and feature-rich solutions that leverage the full power of SAP’s generative AI hub.