Identifying the Need for Using Generative-AI-Hub-SDK

An SDK is a collection of tools, libraries, and documentation that developers use to create software applications for specific platforms or frameworks. SDKs typically include APIs, sample code, and other resources to help developers build and integrate their own applications with the platform or framework.

You have seen a way to access generative AI hub using SAP AI Launchpad. Using generative-AI-hub-SDK you can leverage the power of LLMs available in generative AI for creating customized solutions combining the power of LLMs with and buisness context using SAP data.

With this SDK, you can interact with these models and create natural language completions, chat responses, and embeddings.

Here are some key points:

1. Installation: You can install the SDK using the following pip command:

   Python

   Copy codeSwitch to dark mode

   12
   pip install "generative-ai-hub-sdk[all]"
   

   Note

   Here [all] implies installing all extra packages including langchain, which is not available by default.

   .

2. Configuration: Ensure that you have access to generative AI hub and deployed models. Refer to the topic generative AI hub in lesson 1 in unit 1. The SDK reuses configuration settings from the AI-core-SDK. These include client ID, client secret, authentication URL, base URL, and resource group. You can set these values as environment variables or via a config file.

   To utilize LLMs, you must configure the proxy modules.

   We suggest setting these values as environment variables for AI core credentials via a configuration file. The default path for this file is ~/.aicore/config.json for Mac.

3. You can use the following steps for Mac:

   1. Open Notepad and replace the values in below json with your AI core Service keys that you downloaded from BTP and press Ctrl + S (Command + O for mac) to save file. A pop up will appear on the screen where navigate to ~/.aicore/ and location and save the file as config.json

   2. In case you are using mac, you might not be able to create .aicore folder directly. In that case, create the folder using the command

      Code Snippet

      Copy codeSwitch to dark mode

      1
      mkdir ~/.aicore/

   3. Now use the following command to open config.json file in nano.
      Code Snippet

      Copy codeSwitch to dark mode

      1
      nano ~/.aicore/config.json

   4. Now paste the following json script on config.json file.

      JSON

      Copy codeSwitch to dark mode

      123456
      "AICORE_AUTH_URL": "https://* * * .authentication.sap.hana.ondemand.com",
        "AICORE_CLIENT_ID": "* * * ",
        "AICORE_CLIENT_SECRET": "* * * ",
        "AICORE_RESOURCE_GROUP": "* * * ",
        "AICORE_BASE_URL": "https://api.ai.* * *.cfapps.sap.hana.ondemand.com/v2"
      }

4. For windows, use the following steps:

   • Create .aicore folder in C:\Users\<current user>
   • Create a config.json file with the below contents
   • JSON

     Copy codeSwitch to dark mode

     1234567
     {
     "AICORE_AUTH_URL": 
     "AICORE_CLIENT_ID": ,
     "AICORE_CLIENT_SECRET": ,
     "AICORE_RESOURCE_GROUP": 
     "AICORE_BASE_URL":
     }

   You can get these values from AI Core service key in BTP.

5. Alternatively, you can refer to the https://github.com/SAP-samples/ai-core-samples/blob/main/10_Learning_Journeys/README.md file for detailed configuration steps to execute the notebook for this learning journey in your system.

6. Usage Examples:

   • OpenAI-like API Completion:

     Python

     Copy codeSwitch to dark mode

     123456789
     from gen_ai_hub.proxy.native.openai import completions
     response = completions.create(
         model_name="tiiuae--falcon-40b-instruct",
         prompt="The Answer to the Ultimate Question of Life, the Universe, and Everything is",
         max_tokens=7,
         temperature=0
     )
     print(response)
     

     This Python code generates a completion for the given prompt using the "tiiuae--falcon-40b-instruct" model from the OpenAI library. It asks for a short response about the answer to the ultimate question of life and sets parameters for response length and randomness. Finally, it prints the generated completion.

   • Chat Completion:

     Python

     Copy codeSwitch to dark mode

     1234567891011
     from gen_ai_hub.proxy.native.openai import chat
     messages = [
         {"role": "system", "content": "You are a helpful assistant."},
         {"role": "user", "content": "Does Azure OpenAI support customer managed keys?"},
         {"role": "assistant", "content": "Yes, customer managed keys are supported by Azure OpenAI."},
         {"role": "user", "content": "Do other Azure Cognitive Services support this too?"}
     ]
     kwargs = dict(model_name='gpt-35-turbo', messages=messages)
     response = chat.completions.create(**kwargs)
     print(response)
     

     This code interacts with the OpenAI API to simulate a chat conversation. By loading predefined messages into the "messages" list and specifying model details in "kwargs", it sends these inputs to the API to generate a response. This allows seamless integration for querying and getting automated replies within your application using any model, for example, "gpt-35-turbo" model from OpenAI in this code.

   • Embeddings:

     Python

     Copy codeSwitch to dark mode

     1234567
     from gen_ai_hub.proxy.native.openai import embeddings
     response = embeddings.create(
         input="Every decoding is another encoding.",
         model_name="text-embedding-ada-002",
         encoding_format='base64'
     )
     

     This code imports the "embeddings" module from "gen_ai_hub.proxy.native.openai" and generates an encoded representation of the text "Every decoding is another encoding." by calling the "create" function. It specifies the model "text-embedding-ada-002" and uses "base64" for the encoding format. This helps convert text into a machine-readable format for advanced text processing.