SAP Data Services and Object Promotions

SAP Data Services and Object Promotions

Harminder Singh SAP Basis Lead/Architect

1340 Noah Rd, North Brunswick NJ-08902, USA

Abstract SAP Data Services is the repository based ETL tool used for data profiling, SAP job scheduling and to convert the complex data into understandable or easy format. SAP Data Ser- vices allows you to integrate or connect with any SAP system like S4, MDG, and ECC etc. SAP Data Services is connected to all the SAP ERP system using Remote function call, Remote function call is setup in SAP Data Services where you connect to any SAP ERP system by providing the information about hostname, gateway number, user ID and password details. This connection will be bi- directional setup, once remote function call setup done in Data Services then we can create a TCP/IP remote function call in SAP ERP system like S4, MDG and ECC etc. With this bi-directional setup of the connectivity, now the complex data can be transfer to and fro from SAP Data services to SAP ERP systems. SAP Data services basically covers metadata repository, remote function call connections and Data Services management console, metadata re- pository is stored into the database of SAP Data Services and the repository in Data services is created in the central management console. Biggest advantage of using SAP Data Services is it helps to improve the Data Quality, also provide us execute or compute the large amount of data from source to target quickly via SAP Data Services job scheduler. This article contains SAP Data Ser- vices architecture, framework and the code promotion from devel- opment to production landscape for the business purposes. Once a new project kick-started a strong architect has to review the land- scape and provide the developers with this tool to create better code in the start of the project. This is a very strong tool and quite flexible to integrate in complex landscape.

INTRODUCTION

SAP Data Services is a business warehouse tool that is used for the data quality, job scheduling and data profiling. SAP Data Services is a web-based solution which can be accessed using browser. All three-functionality described here can be accessed with different URLs. SAP Data Services is the Extraction, Transformation and Loading technology between any SAP ERP products as source or target. This tool is quite flexible to con- nect with any source or target system. Source or target can be any SAP ERP product like S4, ECC, MDG, CRM or any data- base like MYSQL, ORACLE, and HANA. Important to under- stand here is how we can connect these systems. So, connectiv- ity of these systems can be done using remote function call and by creating SAP Data Store in SAP Designer tool.

SAP BODS Architecture

A. SAP BODS Architecture:

The following diagram will let us understand the architecture of SAP BODS, to understand the architecture better, BODS is basically divided into 3 categories:

Fig. 1. Architecture diagram of SAP BODS

1. User Interface Layer/Data Services management console: Since this is the Java application so we can use this appli- cation via web application layer (http/https) and with this we can login to CMC, data services and information stew- ard.

2. Database server layer: Every operation happens in the re- pository, where each developer develops their code and thats sits into that repo. Now you can have various type of repos (local, central, profiler etc.)

3. Data Services Designer Tool: Each developer is assigned to their individual local repo and they can create (project, job, workflows, etc.) this tool.

   1. HA SETUP IN DATA SERVICES

      Depending on the business needs, you can install data services as single system install or High availability.Since SAP Data Services is very important SAP product in any SAP landscape, it is advisable to install it in HA.As we do data cleansing or data quality mostly in production we go with high availability op- tion.Although we install Devlopement and test systems as a sin- gle system install.Now important to understand is how we can install or achieve high availability, but before we understand HA, we need to understand the load balancing and how it is different from high availability. Load balancing is to balance the load between the application, so that one server in the land- scape if not highly loaded or we can use the computing power of other servers or machine. So now load balancing can be achieved using hardware/software load balancer (Hardware LB: NetScaler, F5, etc. & Software LB: Web Dispatcher). Be- fore we understand the architecture of HA, lets understand the difference in failover and load balancing.

      FAILOVER (High Availability) vs. LOAD BALANCING

      High Availabilty as from the name it is quite understable, 100% system availabity, so that business will not be impacted. Now

      question comes to our mind how we can achieve or design this. But before we reach there, lets understand from the architec- ture of BODS how this can be achieved. To achieve HA in SAP Data Services, we need to plan for HA for database and appli- cation. In Database layer, and we have a concept of primary and secondary database. In case we have issue with the primary node we can fail over to secondary node and vice versa. This can be achieved automatically or manually using different clus- ter management tools. Now we will talk about the application layer failover. In SAP Data Services we can achieve high avail- ability in application also. Important here to understand is while doing the installation of SAP Data Services it creates the SIA services and that is unique to each installation, so in HA the SIA services on each node should be pointing to CMS database. Also, with that we need to have common NFS for input and output FRSs among different servers. Create a common name for the CMC system, so that user can be able to login to any node in case of any issue and they need not to change that man- ually.

      Load balancing as from the name it is quite easy to understand, in HA environment or in case of system availability issues, end users should not get impacted and could be able to login to high available landscape. In application layer or user interface layer, we can install a software based or hardware-based load bal- ancer. SAP provides us with the software-based load balancer which is known as SAP Web dispatcher and the hardware-based load balance is Netscaler or F5 etc. Now how we can use that in our landscape or how we can use it on top of the HA. So, the end users will be given with an https/https URL to login which we created while installing thesoftware or hardware load bal- ancer based on company requirements. In case of any issues in the landscape or any node is down user will not be impacted because the other node will take the load and business or end users will not be impacted. Critical environments where 24/7 system availability is needed, this is the best solution to imple- ment in your landscape. Hope you learnt the differences be- tween HA and LB. Now with this this we will move to very interesting topic or we can say the heart of the SAP Data Ser- vices, which is SAP object promotion. How developers are cre- ating the code in business requirements and how they are pro- moting to production.

   2. SAP OBJECT PROMOTION

      Now here we will learn about how we can move or promote the objects from development to production system in SAP data services, which is very important for a SAP person to under- stand how objects are moved from one system to another. This can be achieved using three different approaches. This tool is quite flexible and userfriendly and help us in many ways for the promotion of the code.

      1. ATL FILE TRANSFER

         ATL (Advanced Transformation language) file transfer is the way with which export of job/project/workflow etc. from de- signer tool can be done. Step by step approach to export the object from designer tool based on the repo, you can export job/project/workflow/dataflows etc. Just an example below, we have created a test job in development and exported that job from from SAP designer tool by logged in to required repo

         where that job is created. Now go to tools and select the export option and drag the job TEST123 into right window. Right click and select export to ATL and save the file with suitable name. Enter the keyphrase for the job name while taking the export. Once the export is done in development system, all the objects will be captured in ATL file, then login to production system into required repo, where you want to promote the changes or the landscape where you want to move the job which is ex- ported. Go to the associated project with the job, import the job. Enter the same keyphrase which was used while taking the ex- port. Goto tool option and right click on the import job option. New enhancement or objects are promoted to production from development as per the business requirements.

         Fig. 2. Example of taking ATL export of objects

      2. CENTRAL REPO CHECKIN/CHECKOUT

         Cental repository check-in and check-out is the best and safest approach in moving or promoting the objects from landscape to another. Go to SAP designer tool and go to Central Repo from tool and go to required job or dataflow or workflow and right click and check out without replacement or if its 1st time you are adding this object to central repo then select add to cen- tral. Diagrammatic view for code promotion using central repo check-in and check-out to undertand this better.

         Fig. 3. Diagramatic view of central checkin/checkout

         Login to SAP designer and go to central repository and check out the central repository with replacement. Promote one of the SAP data services job using checkin/checkout methodology.

         Once check out option is enabled a red tick mark will be ena- bled against the job which is going to be promoted from one landscape to another and now right click on the same dataflow and check in. Add the required comments to what change s you made to the Job or dataflow or workflow.

         Fig. 4. CheckIn of the central object

         Fig. 5. CheckOut of the data service object

      3. ENHANCED CTS PLUS OBJECT PROMOTION

         The Change and Transport System (CTS+) in SAP solution manager or SAP NetWeaver is used to transport SAP Data Ser- vices objects from source system to target system like develop- ment to production. Enhanced CTS+ is the ability to transport non-ABAP objects.

         So now lets understand how to configure CTS + in the land- scape. To configure CTS+ you need solution manager in your landscape. Solution manager is the ABAP system which inte- grates with the Java system like SAP Data Services.

         To move the object from source to target, login to data services. Goto object promotions and login to central repository where the object is been checked out, capture it in the SAP transport. In SAP Solution manager you will define the TMS path of SAP Data service landscape, like development, quality and produc- tion systems. Since SAP Data Services is the non abap system we need to add a non abap system in STMS transaction. With this you can save time and be more efficient by installing the CTS+ plug-in and set your system to transport non-ABAP Data Services objects. You also must install or upgrade to Data Ser- vices 4.2 Support Package 6 patch 1 or later to use CTS+. This is a new feature which has been introduced from the above- mentioned version only. So to move the object from source to target, login to data services. Goto object promotions and login to repository where the object is been checked out, capture it in the SAP transport.

         Fig. 6. Object captured in the SAP transport using CTS Once it is captured in a transport, release the transport.

         Fig. 7. Release of the modifiable object captured in the transport

         Once the transport is released a lock symbol will appear which means no more changes to this object can be done in this transport. So, no modification to object can be done.

   3. CONCULSION

      After going through this document, it will explain us about the overview of BODS (information platform services, data ser- vices, and information steward) and to understand the require- ments of business and how those can be implemented into the landscape to integrate it with different SAP application (ECC, MDG, S4, etc.) and various databases (HANA, Oracle, MySQL, etc.) to make complex job/project/workflows into much easier and understandable format. With this article you will able to understand the compelte architecture of SAP Data Services which cover the user interface layer, database interface layer, application layer. How you can achieve the the 24/7 avail- ability of the landscape in critical business operation platforms is also covered in this documenAlso the key aspect of this doc- ument is to understand the object promotion from development landscape to production. The object promotion tool in the data services administrator enables you to move one or more data services objects from a development environment to a QA en- vironment, or directly to a production environment. To ensure security, these environments typically do not have direct access to one another. Object promotion in data services is accom- plished over secure FTP or over a shared directory that only an administrator or a user with an account assigned to the data ser- vices administrator group can configure for this purpose, main- taining security in both source and target environments.What- ever the best way or optimal way where business does not allow or having budgeting issues, any of the above mentioned ap- proach can be used to configure the object promotion, Archi- tects or Leads can decide the best way optimal way of objects promotion.

   4. REFERENCES

1. https://help.sap.com/viewer/d6dcecbc85bc432792e87692ad5008f2

   /4.2.9/en-US/57913e296d6d1014b3fc9283b0e91070.html

2. https://help.sap.com/viewer/8df1dcdec88b4df2afc3eb1cc954b3dc/ 4.2.8/en-US/577d9ed66d6d1014b3fc9283b0e91070.html

3. https://www.sap.com/documents/2016/11/a607a3b1-947c-0010- 82c7-eda71af511fa.html

4. https://launchpad.support.sap.com/#/notes/2056228