SAP Certified Development Associate - SAP Integration Suite

- comment start -

Power CAT

Event Mesh enterprise messaging service - Solace PubSub+ as digital backbone

Advanced Event Mesh Migration Guide for SAP Process Orchestration Optimized local SAP Integration Suite Edge Integration Cell (12/2022 available as Beta version) DevOps implementation with project Piper and tools like Git and Jenkins

- comment end -

Integration architectures for SAP S/4HANA Intelligent Enterprises combine hybrid multi-cloud technologies and business processes:

SAP Business Technology Platform (BTP) Integration Strategy

Intelligent Enterprises are integrated Enterprises. The SAP S/4HANA Intelligent Enterprise implements this strategy of integrated end-to-end business processes with the SAP Business Technology Platform (BTP) Integration Suite.

SAP S/4HANA Intelligent Enterprise Integration

These end-to-end processes manage business aspects for specific capability areas like Hire-to-Retire for workforce, Lead-to-Cash for customer experience and revenue, Design-to-Operate for SAP Digital Supply Chain from design to planning and Source-to-Pay for purchasing processes together with spend management.

The SAP Cloud Integration Suite is the part of the SAP Business Technology Platform (BTP). SAP BTP integration strategies implement the SAP Integration Solution Advisory Methodology (ISA-M) with end-to-end business process roadmaps and out-of-the-box scenarios.

SAP Business Technology Platform (BTP) Integration Suite

The SAP Business Technology Platform (BTP) Integration Suite offers a common launchpad for multiple integration capabilities. 

Capability Short Description
API Management expose APIs, manage API lifecycle management and security with policies, with IdP selection and assignment of role collections to handle security and authorization for your services, API analytics and monetization
SCP Integration Web UI discover predefined integration content, design integration flows with graphical editor and monitor deployed integration flows
Open Connector Cockpit create an unified API layer and standard-based implementation across over 150 API providers
SAP Cloud Platform Integration Advisor
  • crowd based tool using machine learning to accelerate A2A/B2B/B2G integration scenarios
  • components are library of type systems, runtime artifacts and implementation guidelines for messaging (MIG) and Mapping (MAG)
  • MIG describe the structure of a customized interface
  • can also create all necessary runtime artifacts for Cloud Integration and PI/PO along with documentation in various formats
  • the knowledge graph works bidirectional with contributions of created models and proposals created for MIG or MAG

SAP Integration Solution Advisory Methodology (ISA-M)

Enterprise architects can design and document their integration strategy with the help of the Integration Solution Advisory Methodology (ISA-M). ISA-M provides a technology-agnostic framework with common terminology, to definine and execute integration strategies, based on assessments with integration domains, integration styles and use-case pattern.

ISA-M templates accelerate this integration architecture assessment, the selection of relevant integration domains and the documentation of the current integration services / technologies (as-is) and future integration architectures (to-be).

Integration Domain Relevance categories Integration Services / Technologies
OnPremise2OnPremise <relevant, not relevant, under evaluation> description of <as is, to be>
OnPremise2Cloud
Cloud2Cloud
User2OnPremise
User2Cloud
Thing2OnPremise
Thing2Cloud

Decision table example with integration technologies and domains:

Integration Technology Description / Recommendation Integration Domain
CPI for all hybrid and multi-cloud scenarios OnPremise2Cloud, Cloud2Cloud
SAP PO prefer usage of cloud integration runtime version 7.5 OnPremise2OnPremise
SAP AIF Error Handling and Business related monitoring OnPremise2Cloud, Cloud2Cloud, OnPremise2OnPremise

ISA-M differentiates between integration styles (process, data, analytics, user, thing) and their related use-case pattern:

Integration styles are characterized with

Cross-use cases like API-managed, Event-based, Workflow management or RPA integrations, are related to and complement one or more integration styles.

SAP Business Hub

The SAP API Business Hub is fully integrated with the SAP Business Technology Platform Integration Suite, as the central location where you can find information about integration options with SAP. The integration options are divided into integration content for SAP Cloud Integration, SOAP or REST OData APIs, together with API specifications and business documentation.

APIs are grouped by SAP Solutions and integration scenarios for on-premise and cloud environments. Modern SAP APIs are implemented as REST OData interfaces or CDS views, but there are also a lot of legacy SOAP APIs existing. Furthermore, existing SAP IDoc interfaces can be integrated as SOAP XML-HTTP web APIs.

These APIs are solving multiple integration requirements, for example with S/4HANA deployed as workload on a hyperscaler cloud or as S/4HANA Cloud public edition.

SAP Cloud Integration

SAP Cloud Integration is a service of the Business Technology Platform and supports as cloud native based integration middleware, hybrid multi-cloud deployments with integration scenarios. With cloud qualities like horizontal scalability, elasticity, security or multi-tenant customer isolation, hybrid and multi-cloud integration requirements can be implemented.

Prepackaged Integration Content can be copied from the API Business Hub to the workspace design tab and adapted depending on the type (configure-only, editable). Configurable options are naming of the integration content e.g. with an suffix, external parameters and the channels.

SAP Cloud Platform Integration Building Blocks

SAP CPI is based on the lightweight integration framework Apache Camel and the deployed CPI services are based on Apache CXF, an open-source Web services framework.

Component Short Description
OSGi the CPI runtime is based on the OSGi (Open Service Gateway initiative) component model with jar components deployment bundles. OSGi supports software platforms based on Java VM, offering a service registry and deployment environment for modules (bundles, services) with published interfaces
Apache CXF CXF (combinations of Java projects Celtix / XFire) supports developing web services based on APIs with protocols such as JAX-WS (Java API XML - Webservice) for SOAP or JAX-RS (Restful Service) dor RESTful HTTP services
Apache Camel Apache Camel acts as payload agnostic mediation and routing engine under the hood of CPI. Camel provides a fluent API (readable like common language) and implements all Enterprise Integration Pattern (EIP) with an (Enterprise Integration) Domain Specific Language (DSL) to implement routes, containing flow and logic of the integration.The main Camel entry points are CamelContext and RouteBuilder.
Adapter Development use ADK to develop adapters to wrap camel components with OSGI compliant bundles and deploy them to integration flows. XML metadata gets generated from component and bundles jar files via introspection as basis for configuration in web UI. Configuration parameters can be reduced with comments
Message / Exchange

SAP S/4HANA Intelligent Enterprise Integration - Camel Message Exchange
Camel introduces core concepts like Messages for data transferred by a route or Exchange allowing interactions for systems like one-way or request-response messages. Message or Exchange content can be accessed with XPath expressions or the simple expression language (syntax e.g. ${body}).

Messages
  • are uniquely identified by IDs of type java.lang.String
  • containing body (payload as java.lang.Object), headers, attachments
  • headers are pairs of unique names with values of type java.lang.Object e.g. sender, content encodong, authentication information
  • attachments are optional and mainly used for webservice invocations and email

Exchange
  • message container during runtime, for message Processing within integration Flow
  • identified by unique Exchange ID
  • fields: unique ID, MEP, exception
  • containing one (in) or two (inout) messages
  • properties to place data during message Processing, will not be send to receiver in contrast to header values
  • exception: filled in case of errors
  • contains in and out message

Message Exchange Pattern (MEP)
  • InOnly (synchronous) e.g. JMS messaging
  • InOut (asynchronous) request-response message e.g. via HTTP-based transports

Data Types
  • unique identifier have type java.lang.string
  • header key-value pair with values of type java.lang.object
  • body contains payload of type java.lang.object

SAP Cloud Platform Integration Technical Infrastructure

CPI is available on SAP NEO Cloud Platform and as Cloud Foundry solution on hyperscaler platforms, with different features, license models and technical infrastructures. In all of these environments, data is processed at runtime on a tenant, representing the allocated, strictly separated and accessible customer resources like CPU or data storage. Worker container (Cloud Foundry) or nodes (NEO) host the integration runtime and perform the message processing.

The cloud integration runtime is also available as part of SAP Process Orchestration 7.5 and allows customers moving to the cloud with own pace. Downloaded packages can be imported with the SAP Cloud Integration Content Management Cockpit and deployed together with required security artifacts, such as user credentials, known hosts and Oauth 2.0 authentication.

Content developed and configured in CPI can be downloaded to on-premise PI, to run Side-by-side with the static PO/PI pipeline (receiver determination, interface determination and mapping). Beyond that, as of PO release 7.5 SP10 the configuration of externalized properties within the deployment cockpit of PO is enabled. Message Monitoring is available within the Cloud Integration Content Management Cockpit.

SAP Business Technology Platform Enterprise Integration Pattern

Enterprise Integration Pattern describe integration problems and their solutions with common vocabulary as best practises.

EIP Short Description
Request-Reply (Response) Two-way communication pattern e.g. for external OData call, where the response replaces the current payload. The SAP CPI query editor automatically creates EDMX and XSD (for internal work with XML)
Content Enricher Combines or merges the response payload of an external request reply API call
Poll Enricher retrieve an (S)FTP file and merge it into the current message
Content-based router examines the message content and routes the message based on the content to different channels. Implementation with gateway and routing conditions and default route
Splitter splits the message content into a list of elements and publishes one new message for each element
Scatter-Gather broadcasts a message to multiple recipients and re-aggregates the responses into a single message
Message Translator translates one data format into another using mappings. As first step, input/output XSD/wsdl gets converted to/from internal XML. This XML format can be translated with predefined mapping functions of the CPI Graphical Mapping editor for complex transformations or with custom functions implemented as XLST transformations or scripts (Java, Groovy)

SAP Business Technology Platform Enterprise Integration Adapter

Adapters allow to configure technical communication between remote systems and the integration platform with channels as connection from the server component to the integration flow. Sender adapter handle inbound messages with CPI as server and receiver adapter are used to connect to remote systems such as Twitter to post tweets.

Asynchronous communication scenarios can be implemented with SFTP, XI, AS2 or SOAP Adapter (with assigned WSDL file and input message operation).

The listed adapters below are a subset of the complete list of SAP Cloud Platform connectivity adapters.

Receiver Adapter Short Description
AmazonWebServices connects CPI with AWS S3, SQS, SNS, SWF
HTTP(S) supports HTTP 1.1 only, with TLS and the following methods HEAD, TRACE,DELETE, GET, POST, PUT
OData available configuration parameters are address, resource path, EDMX. Queries can be configured dynamically with Simple Expression Language expressions. To be used e.g. for request-reply patterns
ODC connects SCP tenant to SAP Gateway Odata Channel
Facebook, Twitter receiver uses OAuth to receive messages on behalf of Facebook, Twitter user
Mail encrypt outbound e-mails with S/MIME
JDBC connects integration flows with HANA or ASE databases hosted on customers global account

Sender/Receiver Adapter Short Description
AS2 designed for Ecommerce, with MDN (message Distribution notification), to document that the message was received
AS4 B2B webservice Communication
IDoc exchange Idocs via SOAP web services
JMS enables asynchronous communication
SFTP enable secure file transfer over the internet with the following configuration parameters: server, directory, filename, connection and file access parameter
Soap SAP RM simplified communication protocol for asynchronous Web service communication
Process Direct call local integration flows to connect different integration flows on the same tenant, without load balancer routing and address as single parameter for both sender and receiver. Can be used to extend standard content with custom integration flows to enhance e.g. mappings.

Sender Adapter Short Description
Soap asynchronous processing with MEP (Message Exchange Pattern) one-way. With Processing Setting Robust the returned HTTP code reflects successfully (with HTTP code 202) or errors after processing all iFlow steps. Process setting WS Standard sender will receive OK when message completely inside the iFlow, but before processing, regardless of processing errors.
The selected WSDL binding represents the endpoint for the sender. Mapping wsdl namespaces to custom namespaces avoids conflicts when calling multiple services with identical field names. Namespace can be defined on runtime configuration tab e.g. xmlns:p1=http://...webserviceX.net

OEM / Cloud Adapter Short Description
OEM Salesforce (e.g. Advanco), MQTT (e.g. Advanco), Microsoft Dynamics CRM, Amazon Web Services
SAP Cloud Solutions Ariba, Sucessfactors, Hybris, Concur, Fieldglass, Access Control, Health Engagement

SAP Business Technology Platform Integration Development

Integration development shall follow Design Guidelines and divide responsibilities to integration developer, content publisher and reviewer roles, used by the SCI Web UI to discover, deploy, run and monitor integration content.

During the discovery phase existing content gets examined and copied to the own tenant, where the content gets adapted. The deployed iFlows can be monitored and managed within Message Processing, Integration Content and Security sections.

The Web UI editor uses BPMN to model the integration flows. Pools, displayed as rectangles, are used to structure modeling with subprocesses eg. for exception handling.

Integration flows specify how messages are processed in a tenant with

Development Objects Short Description
Local process structuring large iFlows with palette shapes Local Call and Local Integration Process. Header and exchange properties are shared between processes, which are relying on same exchange. Use content modifier before child process to write information e.g. order number to header properties, content to exchange property. Invocation of local process is implemented with Local Call step
Process defines sequence of containers for integration main process, local Integration or exception subprocesses
Message defines processing steps with various types such as events, mapping, transformation, aggregations, calls, routing and persistence. Calls are differentiated by the direction they are coming from into external and local calls.
Events
  • Start Message sequence by incoming message
  • Start Event starts sequence from local call
  • Timer to run sequence once or repeatedly
  • End Message
  • End Event within local integration process
  • Terminate
  • Escalation for synchronous messages, an error message is sent to the sender
Mapping & Transformation
  • Mapping sender to receiver format using the graphical editor for XSD + EDMX, custom mapping functions or XSLT, with simulate button in mapping edit mode
  • Transformation with
    Content Modifier for message header, body or exchange data container.
    Converter to transforms json, csv, EDI/ASC-X12 to xml or vice versa.
    Encoder and Decoder support Base64, GZIP, Zip. MIME Multipart can be used if the protocol (HTTP/SFTP) doesn't support attachments.
    Filter information by extracting specific node using Xpath expressions with nodes, nodelist, int, string, boolean values.
    Message Digest stores calculated digest of palyoad or parts in message header.
    Scripts to execute Java or Groovy for message processing
  • Aggregator configured with correlation expression (XPath) and aggregation strategy, with aggregation algorithm such as Combine or Combine in Sequence (Number in incoming Message) and terminated by Last Message Condition / Completion Timeout
  • Call external with synchronous Request-Reply, where system replaces current payload or Send without reply or
    Content Enricher access external source, merges returned content with original message with limitations regarding adapters (e.g. SuccessFactors, Soap 1.x, OData), format (only XML) and matching request format of external data source and current one in exchange.
    Local call from process to integration sub process or with looping process calls
Routing
  • Splitter splits a single message into individual messages with supported types (general with envelope, iterating without envelope, PKCS#7/CMS, IDoc, EDI) and options (grouping, parallel processing, streaming, stop on exception)
  • Gather combines multiple messages into single message e.g. with XPath aggregation algorithm
  • Router routes message content-based to one or more receivers
  • Multicast forwards same message to more than one receivers, parallel or sequential
  • Join merges the control of messages from multicast branxhes (different routes) into one branch, with single message processing in combination with e.g. Gather or Content Modifier steps (not to be used with Splitter or Router)52
Persistence Data store operations allow to persist message payload on tenant SAP ASE Platform Edition with disk space limit of 32 GB for default 90 days, with retention threshold for alerting and expriration period, for global or specific iFlows, with SELECT, GET, WRITE, DELETE operations and values written to variables during message processing.
Transaction management can be configured on main or sub process level to store data databases or JMS queues. Asynchronous parallel processing of messages cannot be transactional.
Write variables can be consumed with Content Modifiers across multiple integration flows on the same tenant. You can also use Content Modifier to define local properties for storing additional data, during message processing.
Security
  • Encyptor or Decryptor (PGP, PKCS Public-Key Cryptography Standard for Enveloped and Signed Data)
  • Signer or Verifier (PKCS, WS-Security, XML digital signature)
Validator to validate XML against schema
XML namespace can be defined in the runtime configuration of the integration flow to avoid element name conflicts
Customer Extension
  • Content Modifier externalizes properties custom_extension_enabled, which controls further routing to default route or local integration flow implementing the extension, and original_payload (stores original message) to be exchanged
  • Mapping extensions as local integration flows through Process Direct, with optional Pre-Exit point (required if the customer needs to extend the source message) and required Post-Exit to map output of extended source structure to extended target structure
Generating iFlow Integration flows can be created from API with basic authentication and DELETE, GET, POST, PUT, GET_ID operations. Supported APIs and operations are OData (GET, POST, DELETE) and REST (GET, POST, PUT, DELETE)
OData APIs develop OData APIs as CPI OData Service artifacts from scratch with the Java Olingo API or based on existing data sources to be consumed with apps like BTP Mobile Services, Fiori UI
Exception Handling with Error End or Excalation End Event in exception subprocess to trigger Failed status of integration process
Lifecycle Management
  • Monitoring message processing, integration artifacts, value mappings, security related artifacts
  • Settings
    Product profile (CPI, PO) to specify the target runtime (Integration Content, PI runtime)for your content. Product profiles may also be configured for individual integration flows or marked as default for a tenant
    Transport Options: CTS+ to perform transport of a web service MTA from one subaccount to another. CTS has to be activated using the SAP NetWeaver SOA Management Web Tool. Transport Management Service, MTAR download, manual export
  • Versioning with Save as version, revert specific version in history

SAP Cloud Platform Integration Operation and Monitoring

Area Short Description
Operating model SAP is responsible for monthly product updates, resource management and protection, backup and restore (every 10 min on primary storage, every 2 h transferred to secondary storage, full backup every day)
Web-based Monitoring
  • Message Monitor for Processing Log (MPL) with properties Message ID and Status. Accessible under Monitor -> access Logs: Log Text View with details
  • Integration Content Monitor for deployed artifacts (integration flows, value mappings, key store) with properties status, artifact details, log configuration
  • Monitoring APIs for HTTP access of log files to analyze authentication and authorization errors, with filters scope, LogFileType e.g. 'http', NodeScope e.g. 'worker'. Log implementation with
Log Implementation SAP_MessageProcessingLogLevel header variable to set log level, MessageLog setStringProperty, addAttachmentAsString with the need of storage shared between messaging and monitor. Trace contains technical processing information. Message Processing Logs provide status, time, sender, receiver information. Message Store persists encrypted messages on the runtime node for 90 days with property and attachment information.
Log Level
  • None
  • Info as default level, for failed processes additional detailed information about the last steps will be retained
  • Debug
  • Trace enabled 10 min then switch to previous level, by default, trace data will be deleted after 60 min. Log will be written at the highest possible detail level and the message content gets collected and stored
Other Information
  • integration content information about artifacts (integration flow, value mapping,OData service)
  • Security Content for User credentials, Keystore, certificate-to-user mapping
  • Partner Directory in B2B scenarion

SAP Integration Monitoring Tools

SAP offers several integration monitoring solutions with different capabilities and focus.

SAP ALM SaaS on BTP enables cloud-centric monitoring of cloud or hybrid integrations e.g. in the context of RISE Cloud transformations. The SAP Analytics dashboard provides an analytical view for SAP Cloud Integration with information about performance, message status and integration content artifacts. SAP Cloud Integration integrated monitoring provides e.g. Message Processing, Access / Audit Logs, status integ content, lock entries, temporary data stores information.

On-premise or hybrid cloud monitoring solutions are SAP Solution Manager for on-premise centric integration scenarios and SAP Cloud Integration Monitoring with exception metrics like number of erroneous integration flows or response times. SAP Focused Run enables high-volume system and application monitoring in hybrid landscapes and SAP Application Interface Framework (AIF) embedded in S/4HANA on-premise and cloud.

SAP Cloud Platform Integration Security

Security of the multi-tenant cloud environment has to be ensured with data isolation, secure authenticated access and message communication with content encryption and signing. For this, SAP Cloud Platform Integration supports the implementation of multi-level security concepts. Integration scenarios with external apps (e.g. SaaS, B2B) have to use HTTPS (Hypertext Transfer Protocol Secure) and encrypt message content and payload using digital certificates, with applied principle of least privilege.

Security Level Short Description
Transport
  • SSH (Secure Shell) secure and authenticated connection between client and server, with basic or certificate based authentication. Used by SFTP
  • TLS hybrid encryption for HTTPS, SMTPS, Mail Adapter (Inbound STARTTLS, IMAPS, POP3S, Outbound STARTTLS, SMTPS)
Message / Payload
  • en-/description, signing, signature verification
  • Signature: calculate digest and encrypts the digest using a private key
  • PGP Message for encryption/decryption, signing/verification of payloads with format public key, literal data, signature
  • PKCS#7 (public key cryptography standard used in public key infrastructures / PKI) for en-/decryption of message content and signing/verification of payload
  • XML signature for signing/verification of payload with canonical SignedInfo + SignatureValue elements as part of the XML document
  • WS security for signing/verification of SOAP body, signing + encryption/decryption of message content, signing a message (SOAP body) based on the WS-Security is an additional feature with regard to signing/verifying on payload level based on the following standards: PKCS#7, XML DigitalSignature
SAP S/4HANA Intelligent Enterprise Integration - Message Payload Security
Certificate Management
  • X.509 used for level transport (TLS), message (PCKS#7, WS-Security, XML Digital Signature) with elements issuer, subject, Distinguished Name (DN)
  • PGP Keys on message-level
  • known host files for SFTP transport-level security
Roles and groups Role esb.messaging.send for basic authentication, component access with well defined permissions
Persistency
  • Secure storage of key material in database
  • Encrypted data persistency
  • Multitenancy and data isolation is separated schemas
  • Persistence process step with options Step ID (name or number) and encryption (AES with 128 bit key). GUID, timestamp, payload are stored on the runtime node für 90 days. Message access via Odata API
Cryptography
  • Symmetric cryptography one single key has to be distributed
  • Asymmetric, public-key in combination with certificates, which identify user / server. The public key can be extracted from the certificate.
SAP S/4HANA Intelligent Enterprise Integration - Certificate Management
Certifications ISO27001, SOC1, SOC2
Load Balancer for secure inbound HTTP connections, the remote sender system must trust the Business Technology Platform load balancer and store the root certificate of the load balancer in its trust store. The load balancer terminates each inbound Transport Layer Security (TLS) request and reestablishes a new one for the connection to the tenant where the message is processed
Authentication
  • Sender (inbound)
    • Basic authentication
    • Client Certificate (sender's root certificate of trusted CA imported in the keystore of the BTP load balancer)
    • OAuth with client credentials (ID + Secret for) supported for SOAP (SOAP 1.x), SOAP (SAP RM), HTTPS and OData sender adapters
  • Receiver (outbound) with Basic, Client Certificate, OAuth (Twitter, Facebook)
Authorization the authentication options have to be combined with CPI User role based authorization (e.g. with pre-delivered role  ESBMessaging.send)
Firewall the SAP BTP firewall allows per default secure outbound channel (receiver adapter of connected systems) communication over HTTP/HTTPS port 443, SFTP port 22 (SSH data channel between SAP Cloud Integration and the SFTP server has to be open) and SMTP port 25