Understanding Joule Architecture and Extensibility

Joule provides an embedded co-pilot experience across solution areas, consistent and with memory of interactions across multiple products.

The users start with their queries in the natively integrated Joule client embedded in our respective SAP cloud applications:

• Each incoming request is processed and orchestrated based on what is available. Joule takes three categories into consideration:
  1. Joule analyzes if the user-prompt is relevant for Joule’s Scenario Catalog. This catalog contains metadata of all available scenarios, functions, and skills of SAP cloud applications.
  2. Joule conducts an informational filtering based on the Knowledge Catalog. This contains SAP-knowledge as well as the customer-owned knowledge. This process is based on Retrieval Augmented Generation for enterprise (RAGe) and lets the LLM generate an answer based and grounded on dedicated text that was previously retrieved.
  3. Joule is aware of the user's context and history. This includes information about which SAP cloud application the respective user is using and which additional SAP application the customer has licensed and activated with Joule. Also, Joule is aware of the user's role(s) and permissions. This means a user cannot access information or adjust business processes they couldn't do directly in the SAP cloud application without using Joule. Lastly, Joule understands a user's chat history and context, ensuring that the best possible answers are provided in response to questions asked shortly before.
• Joule takes all of this contextual information and provides a much richer query to the LLM which is responsible for the dialog management. This LLM is based on a rich ecosystem of technology partner LLM offerings also available directly via SAP Business Technology Platform's AI Foundation and the SAP Generative AI Hub. Here we are leveraging the latest LLMs and innovations available in the market under strict contractual agreements with SAP's partners which prohibits them from training any of their AI-models on any customer data.
• Next, the LLM provides a grounded response back to the Joule service or identifies the need to leverage Joule functions which includes the Joule scenarios of our SAP Cloud Applications.
  • In this case, Joule calls the respective SAP backend system to proceed with the requested query, e.g. requesting time off.
  • The response is then filtered as appropriate, and the customer gets a response knowing the conversation and output has been processed all with enterprise security, data privacy, and responsible AI considered.

As positioned at SAP SAPPHIRE 2024, we will support 80% of the most used SAP tasks in Joule by end of the year. All SAP cloud applications are in process to develop a rich user experience for our customers.

SAP plans to make Joule available in all major SAP Cloud applications.

Here’s an overview of the planned availability of Joule.

Joule can be hosted globally with regional coverage in Europe, America, and Asia-Pacific and Japan via the three main hyperscalers: Microsoft Azure, Amazon Web Services, and Google Cloud Platform.

We want to make Joule usable for all our Cloud customers. By end of 2024, users can interact with Joule in multiple language, starting with our focus languages French, German, Spanish, and Portuguese.

Many of SAP’s customers use Microsoft 365 as a collaboration and productivity suite.

Depending on industry and position, their users use both platforms, Microsoft 365 and SAP Cloud applications equally, but are offered 2 digital assistants which strive to accelerate their productivity independently.

Customers need to jump back and forth between applications.

While they are in one application, they cannot use the other one.

By integrating Joule and M365 Copilot, we enable customers to leverage M365 Copilot capabilities within Joule and vice versa, letting customers leverage Joule capabilities within M365 Copilot.

A user will be able to ask questions to M365 Copilot within the Joule chat UI embedded in their respective SAP Cloud application. Joule will understand that the question is relevant for M365 Copilot and forward the prompt. M365 Copilot creates the response and sends it back to Joule and the user.

The answers from M365 Copilot are shown in the same Joule chat UI and are marked stating that they were provided by M365 Copilot. This gives the user full transparency and a seamless UX.

• Users will be able to stay in the respective application (Microsoft 365 or SAP) and still leverage the assistant of the other application. They will be able to do this with full transparency and a seamless UX.
• The boundary to use SAP is lowered for users who spend most of their day in M365 products, by simply using Joule via M365 Copilot.

As a long-term objective (2-year plan), we plan to integrate the two copilots even deeper to enable much smarter responses, combining the intelligence, context, and data of both ecosystems to deliver results which boost the productivity of our joint customers no matter where they are.

We plan to release the first scope of this integration by end of 2024 to a limited number of customers.