Generative Business AI creates new valuable content for SAP Business Processes on cloud platforms like SAP Business Technology Platform SAP BTP, AWS or Azure.
Generative AI harnesses Foundation Models (FM) and Large Language Models (LLM) to transform single or multi modal inputs to new output content.
Foundation models are trained for general purposes and can be integrated into versatile business scenarios, in contrast to LLMs which are often specialized for specific Business AI use-cases. These Generative AI models can be integrated into SAP Business Processes with the SAP Generative AI Hub which is built on top of SAP AI Core.
Generative Business AI models perform downstream tasks in SAP business scenarios. AI foundation models process multi-modal prompts as input to generate generate new business content with various formats like text, audio, speech, code or image.
Based on Generative AI Transformer Architectures, foundation models are trained with transfer and self-supervised learning. Self-supervised training createa labels based on data structures which can be applied to versatile downstream tasks.
Customization and orchestration empowers GenAI models with SAP Business AI capabilities like domain specific context, prompt management and enhanced security features.
Input features and output predictions of Genenerative AI models are represented as vectors. As part of Generative AI data preparation, tokenization splits natural language input into small text pieces and converts these pieces into vectors.
Byte-Pair Encoding (BPE) is widely used to implement Generative AI tokenizers like for instance tiktoken for OpenAI models.
Vectorization converts text tokens into Embeddings which are numerical vector representations of features like texts or images optimized for machine learning processing.
Vector representations and their directions are used for similarity calculations with Euclidian distance, dot product or cosine formulas. Use cases in NLP analysis are comparisons for text mining, sentiment analysis, document clustering or similarity search.
Large Language Models are foundation models trained on large text datasets to perform several tasks. The reusability of these models reduces training effort and costs which enables use-cases with small training datasets or machine learning teams.
NLP use-cases are typically implemented with Large Language Models (LLMs) to process language input and generate text, code or image output. Some Generative AI NLP tasks are classification, sentiment determination, summarization, comparison or text generation for application areas like autonomous AI assistants or different kind of document processing.
Cloud platforms like AWS Bedrock, Hugging Face or SAP GenAI Hub offer huge amounts of Generative AI models and AI engineers or architects have to compare these models to find AI capabilities which help them to implement Business AI scenarios.
Important selection criteria for GenAI foundation models are parameter count or model size, benchmarks, licenses and costs, context window size and latency.
Some important benchmarks, based on prepared datasets and questions, are listed below.
| Name | Description |
|---|---|
| MMLU | general mostly knowledge-driven evaluation of various tasks |
| MMLU-PRO | enhanced, more reasoning-focused multi-choice options |
| GPQA | evaluates expert-level reasoning |
| HumanEval | measures the functional correctness of generated code and programs |
| MATH / MATH-500 / GSM-8K | measures arithmetic reasoning from basic arithmetic to advanced mathematics |
| BFCL / NFCL | evaluate the ability to generate arguments to call functions of external tools |
| MTEB | evaluates the performance of embedding models |
The SAP BTP Generative AI Hub offers foundation models and LLMs with a toolset to engineer LLM prompts and integrate GenAI into SAP business processes. SAP manages Open Source from MistralAI or Meta and GenAI models hosted on other cloud platforms like AWS Bedrock or Azure OpenAI as SAP AI Core scenarios.
Selected SAP Generative AI Hub models are listed below with on-demand costs ($) calculated for 1000 In/Output each or 1000 Embeddings input tokens.
| Model | Capabilities Samples | Size | Context | Costs | MMLU | GPQA | Code | MATH | MMLU-Pro | Speed | Orchestration | Modalities | MMLU-Pro |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| MistralAI | permissive licenses and open weights | ||||||||||||
| 8x7b-instruct-v01 | SOTA model | 46b | 32k | 0.002 | 70.6 | 28.4 | yes | text | |||||
| Large Instruct | multingual text & code generation, reasoning | 123b | 128k | 0.014 | 85 | 47 | 90 | 70 | 38 | yes | text | ||
| Meta | Llama open source and weights | ||||||||||||
| LLama 3.1 70b Instruct | multilingual SOTA capabilities | 70b | 128k | 0.01 | 88 | 51 | 89 | 74 | 73 | yes | text | ||
| AWS Bedrock | Titan & Nova models | ||||||||||||
| Titan Text Express | multilingual text generation, chat, RAG | 8k | 0.008 | yes | text | ||||||||
| Titan Text Lite | English only summarization, copywriting, fine-tuning | 4k | 0.002 | yes | text | ||||||||
| Nova Micro | low latency | 11b | 128k | 0.008 | 76 | 38 | 80 | yes | text | ||||
| Nova Lite | fast NLP, QA, interactions, doc analysis | 20b | 300k | 0.039 | 79 | 43 | 84 | 148 | yes | multi | |||
| Nova Pro | versatile | 90b | 300k | 0.039 | 84 | 48 | 88 | 92 | yes | multi | |||
| Antropic | Claude family models | ||||||||||||
| Claude-3 Haiku | Light & fast | 70b | 200k | 0.0035 | 75 | 33 | 76 | 39 | 65 | yes | multi | ||
| Claude-3.5 Sonnet | Complex reasoning & coding | 175b | 200k | 0.037 | 89 | 59 | 92 | 71 | 72 | yes | multi | ||
| Claude-3 Opus | Premium capabilities | 100b | 200k | 0.185 | 87 | 50 | 85 | 60 | yes | multi | |||
| Azure OpenAI | |||||||||||||
| GPT-3.5 Turbo | Flexible tasks | 16.4k | 0.007 | 70 | 31 | 68 | 43 | yes | multi | ||||
| GPT-4 | Capable versatile | 175b | 128K | 0.163 | 86 | 41 | 87 | 65 | yes | multi | |||
| GPT-4o | versatile reduced latency | 200b | 128k | 0.039 | 89 | 54 | 90 | 77 | 112 | yes | image, text | ||
| GPT-4o mini | focused tasks | 128k | 0.002 | 85 | 54 | 92 | 129 | yes | image, text | ||||
| GCP VertexAI | |||||||||||||
| Gemini 1.5 flash | summarization, chat, document extraction | 32b | 1m | 51 | 74.3 | 77.9 | yes | text, image | 67.3 |
SAP Generative AI Hub Embedding models:
| Model | Context | Dim | Costs | MTEB |
|---|---|---|---|---|
| AWS Bedrock | ||||
| Titan Embed Text | 8k | 256-1024 | 0.0004 | 66 |
| Azure OpenAI | ||||
| Embedding 3 Small | 8191 tokens | 1536 | 0.00006 | 62.3 |
| Embedding 3 Large | 8191 tokens | 3072 | 0.0003 | 64.6 |
State of the art (SOTA) GenAI models mostly implement transformer architectures with neural networks to remember long-range dependencies and to process whole sentences with positional embeddings and self attention.
Transformer architectures are separated into encoder and decoder parts both with attention and feed forward components.
Encoders transform language tokens into coordinates within multidimensional vector spaces of semantic language models. The distance of tokens within these embedding models represent their semantic relationship. Embeddings are used for NLP analysis tasks like summarization, key phrase extraction, sentiment analysis with confidence scores or translation.
Decoder are able to generate new text sequences and enable decoder only Conversational or Generative AI solutions.
Attention layer weights of encoder or decoder components control the choice of prediction results. Encoder attention layer weights try to quantify the meaning of words within text sequences. Decoder attention layers predict the most probable output token in a sequence.
Interaction with Generative AI assistants like Microsoft Copilot or SAP Joule improve digital user experience based on data-driven decisions supported by foundation models or LLMs. Microsoft Copilot combines ChatGPT LLMs with data provided by Microsoft Graph and offers the option to build custom copilots for various business-specific tasks.
Open source Large Language Models, interference servers and GenAI tools can be used to implement solutions for intelligent downstream tasks.
Advantages of open-source GenAI are transparency, cost reduction, local solutions to fulfill advanced data security requirements or flexible model customization opens like fine-tuning. Examples of open source GenAI model families are LLama, Mistral or Falcon.
GenAI platforms like SAP Generative AI Hub or AWS Bedrock support traning and serving Bring Your Own Models (BYOM) as containerized web applications. GenAI model interference server definitions use serving templates with parameters like number of replicas. Generative AI models can be called via inference request endpoints to return predictions for consuming customer services.