Ten concepts of interface and integration in SAP-from evolution point of view

The journey began with the classical programming model. I enjoyed the freestyle programming. But as time passed, the business trends started changing and the market began to evolve with cloud and mobile applications significantly in the digital infrastructure. And now when the best-recognized practice for programming is the ABAP Programming Model for SAP Fiori, I look back and see a journey of evolution in several areas in SAP. And it is going to continue as the future programming model is already ready called the ABAP RESTful programming model. From an SAP traveler’s point of view, many tools arrived, several techniques were developed and new concepts emerged in the context of the interface, infrastructure, and integration in SAP. There are several blogs on each of these topics but they all are segregated and written from a purely technical point of view. The idea of this blog-post is to provide an insight from an SAP traveler’s point of view so that it becomes easy to understand even for beginners. Hence without giving much technical info let me share a glimpse of concepts in sequence.

Evolution of techniques, concepts and tools
The pictorial view above doesn’t depict any priority index for using the techniques, it is just a representation of arrival of different techniques in an approximated sequence of chronology.

1. File Interface: Interfaces based on file exchange is one of the oldest techniques to provide the data to the third party system from SAP. Either pushing the text or excel file to a particular directory or extracting from the application server; both ways it works. Without any complex coding and configuration, using FTP/SFTP the file can be transferred from SAP to any third-party system. With the arrival of PI (process integration), the extracted file from SAP is sent first to PI where mapping and conversion happens based on the requirements posed by the third party system and then it is possible to send the file in the desired format, JSON, XML, etc

PI as middleware
So the biggest advantage of introducing NetWeaver PI as middleware for data transfer is that whatever the format or type of data file requested by the business partner is possible to fulfill which is not so flexible for the ERP system to do.
A few important tcodes, statements, Function Modules, and Methods frequently used in this context are:
AL11 (tcode), GUI_UPLOAD, GUI_DOWNLOAD, OPEN DATASET, CLOSE DATASET, CL_GUI_FRONTEND_SERVICES=>GUI_DOWNLOAD, CL_GUI_FRONTEND_SERVICES=>FILE_OPEN_DIALOG, CL_GUI_FRONTEND_SERVICES=>GUI_UPLOAD and CL_GUI_FRONTEND_SERVICES=>DIRECTORY_EXIST

2. RFC: Another approach under the classic model is a remote function call. RFC interface is used to set up the communication between SAP and non-SAP, also between two SAP systems. It runs on the concept of Client and Server. The client seeks for a function to perform and makes a call to the RFC server. The function is executed at the remote system on the server-side and it can be synchronous. 
Under the RFC interface concept, SAP came up with BAPI (Business Application Programming Interface). The idea was to expose the business objects (BO) to the external systems. So a java based system can access the instance of the exposed business object (Customer, Orders, Employee, etc) through exclusive methods provided by the BO. An RFC connection is created in SM59 for setting the connection between two SAP systems.

RFC Interface
A few transaction codes BAPI, SWO1, SE37, SM59, and table SWOTLV are common to use for RFC Interfaces.SAP distinguishes between four different types of RFCs:· Synchronous (sRFC)· Asynchronous (aRFC)· Tranactional (tRFC)· Queued (qRFC)

3. IDOC: Intermediate Document is another old but proven technique to communicate between SAP to SAP or SAP to non-SAP systems. IDOC Interfaces provide a file format for exchanging the data between systems. The format of the data file consists of structure, fields with position and length and it is further divided into header, data segments, and the status of record segment. 
The communication via IDOC interfaces is based on the framework provided by SAP called ALE (Application Link Enabling). However, there is a restriction of using ALE for customers and suppliers who are outside the company. For them, SAP communicates via EDI(Electronic Data Interchange) and for in-house third-party systems like CRM, HR or Finance the communication is based on ALE.
With the introduction of middleware like PI, it has become even more easy and flexible to handle the communication between the in-house SAP system and customers/suppliers outside the company. Non-SAP systems are capable of sending messages on SOAP protocol to the middleware PI which in turn converts format into IDOC file and sends to SAP. SAP sends the IDOC file back to PI and PI returns the response in IDOC file. The sending message by SAP is termed as Outbound Idoc and the receiving is known as Inbound Idoc. Unlike RFC interfaces, generally, IDOC interfaces are asynchronous. 
There are a bunch of tcodes used for IDOC interface development and monitoring.
The important ones are: WE20, WE21, WE80, WE81, WE82, BD64, SM59, WE30, WE31, WE02, WE05, WE18, WE19, BD59, SPRO: SAP Web Application Server -> Application Server > IDOC Interface/ALE -> Communication.

4. SOAP (Simple Object Access Protocol): With SAP version 7.0 and NetWeaver, SAP introduced a new technology called Web Service for communicating and exchanging data between SAP and SAP, SAP and non-SAP systems. So one can say that SOAP is a messaging protocol that is supported by version 7.0 onward with NetWeaver and it is internet-based technology. To call the functions, HTTP protocol is used and the client receives the details of parameters and functions in the form of WSDL(Web Service Description Language) which can be retrieved through a defined URL. The exchange of communication using SOAP is via XML messages. The mode of processing is generally Asynchronous but it depends on the requirement. 
A few important remarks and standards of the webservice based on SOAP:
◉ It provides a technique for communication called reliable messaging (RM). This will make sure that the receiver system has received the message for sure.
◉ The SOAP interface can support any middleware, preferably PI
◉ The delta transfer is possible on a real-time basis
◉ The transfer of payloads can be configured via bgRFC queue. So if the queue is stuck/blocked for some reason then transfer via replication doesn’t take place until the queue is unblocked or re-started. That is one drawback of using bgRFC queue.
◉ In SOAMANAGER, for authentication check, there are several options. Generally used is Basic authentication which means only if the application succeeds login using the same credentials maintained in SOAMANAGER, the payloads can be retrieved from the URL.
Important: ESR(Enterprise Service Repository), SOAMANAGER, SPROXY( for the Proxy Objects), SBGRFCMON

PROXY Objects in ESR

ESR for Modelling

5. REST: REST stands for Representational State Transfer which is an architectural style for designing networked applications. First thing first, what is the meaning of Representational? By nature, REST is stateless which means no session is explicitly created for any communication occurring between two systems. It only represents the resource but doesn’t create any state of its own. Like SOAP it also uses HTTP protocol but it generally works as Synchronous. REST is resource led technology where all of its components act as a resource.
With the introduction of SAP Gateway, it is now possible to provide the RFC interfaces and ABAP functions in the form of REST API. Consider the below scenario where an outside system wants to pull data from ERP. The pictorial view shows, how the request call passes through several layers of an API Gateway and returns the message. Through REST API it is quite possible to implement such PULL functionality even with layered API Gateway. API Key is offered from SAP-ERP which makes sure that the necessary authentication is carried out and only the requested data is sent to the system requesting data. 

API Gateway
A few comparisons between SOAP and REST:
◉ REST is lighter compared to SOAP in terms of design and function.◉ REST is preferably set to synchronous but it depends on the requirement◉ REST can transform data into JSON or XML format◉ REST is faster than SOAP◉ Since REST is stateless, so a lot of resources are saved◉ REST API supports CRUD(Create, Read, Update and Delete) operations
The conclusion is REST Web Service is making HTTP call to a URL and nothing specific more than that. By this HTTP request and response, both are created and it is to the caller, how to deal with it.
A few important tcodes: SE24, SICF, and Postman (as a testing tool for CRUD operations).

6. S/4 HANA: It is the fourth version of the ERP business suite provided by SAP which is based on HANA in-memory database. It is an evolution both in technology and the concept in SAP. After it got released in 2015, since then it has become a constant intent to make the suite easy to use and administer. Over the period, it has proven that no matter how complex are the business requirements are or how large the data is, S/4 HANA is capable to deal with every problem. It is available in the cloud, on-premise, or possible to have it as hybrid deployment models.
S/4 HANA follows the code-to-data paradigm which uses the mechanism of code pushdown which means the complicated data computations happens in the database layer instead of the application layer.To make the data pull easy for everyone (technical or non-technical) from the DB Layer, S/4 has introduced a concept of information models. These are basically: Attribute view, Analytical view, Calculation view, and Decision Table. It has given so much flexibility to join several DB tables in a graphical representation that even a non-technical person can create models on demand and use them for displaying data. By doing this one can avoid writing complex SQL statements with join conditions.Several concepts and techniques emerged during this time:
◉ Core Data Services: As it is an enhancement of SQL standard in ABAP Layer, it has evolved from ABAP CDS views to HANA CDS views
◉ OIA(Open Item Analytics)
◉ AMDP(ABAP Managed Data Procedure)
◉ CDS Table Functions
◉ BOPF( Business Object Processing Framework)

Green Boxes will remain and Rest are fading

7. OData: OData stands for Open Data protocol. As I mentioned in the introduction of the blog that the future programming model is a RESTful programming model and in that context OData is going to play an imperative role. Because OData facilitates the best practices for building and consuming RESTful APIs. So coming to the concept, OData is so open that it can be consumed by any application, program, software, or device. Even with non-SAP systems provided they are connected to SAP via HTTP(s). It can parse and construct the XML file.
A mobile application if requires data from S/4 HANA system, CDS view alone cannot provide data. It is oData that enables the CDS view to expose data to the mobile application. Hence the medium is always oData. That is the reason oData is going to be part of the future-ready intelligent ERP as well.
Important transaction codes: SEGW and /IWFND/MAINT_SERVICE (to register the service)

CRUD Operations in OData

Redefine the methods based on requirement

8. CDS and BOPF: As I mentioned earlier about CDS that, it is a Core Data Service which means in the application layer CDS is defined to extract the data from the DB layer to present to the application. So the function is similar to SQL statements but the data provided is semantically rich as it contains readable annotations. It creates a run-time object in the underlying DB and also a DDIC object. Instead of applying inner joins on the DB tables directly, CDS uses the abstraction using a new concept called association. CDS is possible to create only through ADT(ABAP Development Tool) and once it is created, the basic necessary SQL query is automatically generated at runtime in order to select data from DB Layer. The application communicates to CDS instead of the DB layer.One biggest advantages of having CDS is, it is the single place for any change in selection requirement. For example, the join between several tables is required to change for some specific need. The application program which is just using the CDS views is not needed for any modification. The only change is required to do in the association part, hence a lot of effort can be saved. CDS is much faster than any complex SQL statements with several joins.
Important Tcodes: RSRTS_ODP_DIS, RSRT and ADT tool (Eclipse to develop)
A sample example of CDS view joining two DB tables:

CDS-Data Definition
Let’s discuss now about BOPF:

BOPF in the NetWeaver-ABAP Layer
You would have seen the pictorial view under S/4 HANA where I have explained about the components which are going to remain in future. So BOPF is an additional element of NetWeaver.
It allows persisting intermediate state of an application or the intermediate state of a transactional app. It gives draft functionality which means the temporary version of the business object instance is created until it is permanently stored in the persistent layer as an active version. It can manage LUW in order to keep the database consistent. It can perform validation at run time. Using BOPF one can create a transactional app capable to perform CRUD operations without any additional UI5 design.

Business Object as model category in CDS View
After activation, a link is created for BOPF

The link takes to the details of BOPF
One important remark about BOPF is, with the RESTful programming model the ABAP language has been enhanced to support the implementation of transactional BO behavior(Create, Update and Delete) which means no additional framework is required. Currently in the programming model for SAP Fiori, CDS based BOPF is used for transactional BO but it will be no more in the future programming model.

9. Web API: In the future, with the RESTful programming model (RAP) the concept of Web APIs will be adopted in parallel to oData and Fiori. Because it is an open-source framework for writing HTTP APIs. Like SOAP or REST, using HTTP Protocol Web API can refer to the API on the internet. Since it is just a concept so using any technologies (Java, .NET, etc.) Web APIs can be built. OData can be consumed by Web APIs. 

10. Behavior Definition Language (BDL): The purpose of BDL is to add the behavior to the business objects (BO). So how the entity is going to behave, such definitions will be defined along with the header information. Using BDT, CDS entities are created which are managed in ABAP compiler instead of ABAP dictionary. The syntax of BDL is oriented to DDL(Data Dictionary Language).