Advertisement

Data Conversion in S/4HANA

  • Sanket Kulkarni
Chapter
  • 860 Downloads

Abstract

Data conversion is an essential part of your SAP S/4HANA project. If you opt for a Greenfield implementation, data conversion means the extraction, transformation, and loading of your legacy data into SAP S/4HANA. If you opt for a Brownfield migration, then the data conversion process is much simpler, as you need to migrate data from the source SAP ERP system to SAP S/4HANA. SAP provides tools to automate the entire data conversion process in this case. Thus, the data conversion process is executed much faster. More details on the data migration process and associated tools can be found in Chapter  7.

Data conversion is an essential part of your SAP S/4HANA project. If you opt for a Greenfield implementation, data conversion means the extraction, transformation, and loading of your legacy data into SAP S/4HANA. If you opt for a Brownfield migration, then the data conversion process is much simpler, as you need to migrate data from the source SAP ERP system to SAP S/4HANA. SAP provides tools to automate the entire data conversion process in this case. Thus, the data conversion process is executed much faster. More details on the data migration process and associated tools can be found in Chapter  7.

Let us now discuss various aspects of data conversion as executed during SAP S/4HANA implementation.

Phases of Data Conversion

Data conversion consists of three main phases: extraction, transformation, and load.
  • Extraction
    • During the extraction phase, data is pulled from one or many of the existing legacy systems. In some cases, it will be manually constructed.

    • This process is done via extraction programs built on the legacy side.

  • Transformation
    • During the transformation phase, the extracted data is altered into the format expected by SAP S/4HANA.

    • Automated mapping rules are developed to adapt extracted data to the SAP S/4HANA format.

    • Additional data quality checks are done as part of the transformation process.

  • Load
    • Data is loaded via automated programs or by individual records manually by an end user into SAP S/4HANA.

    • Manual data load is done where the number of records is minimal, and the development of load tools is too complicated and costly.

Data conversion discussion starts early during an SAP S/4HANA project. During the planning and assessment phase, you start with defining your business data requirements. During the design phase, you detail data design considerations as follows.
  • Data objects in scope.

  • The subset of the data to be converted.

  • Mandatory and optional fields for a data object.

  • Business rules for each field and object.

  • Validation of values for each field.

  • Security classification as per General Data Protection Regulation (GDPR) or similar regulations.

During the build phase, several mapping rules are configured, and load programs are developed. During the test phase, multiple cycles of mock conversions are executed to fine-tune the data conversion process further. Once it is optimized in terms of performance as well as runtimes, final data conversion is performed during the cutover and deployment phase.

Your data conversion strategy for SAP S/4HANA should cover all four of these aspects:
  • Data objects in scope.

  • Tool sets.
    • Should cover mechanisms for data extraction, profiling, transformation, and loading.

    • Automated vs. manual loading.

  • Data conversion processes.
    • Key steps involved, including dependencies among them.

    • Mock conversion and dress rehearsal process.

    • Data validation, before loading data and once it is loaded successfully.

    • Data profiling and cleansing activities.

  • Organization.
    • Who owns the data?

    • Who is responsible for data quality?

    • Who does data cleansing and validation?

Data Objects in Scope

Each data object is categorized as one of the following:
  • Master data
    • It consists of the data objects with longer a life cycle and is used repeatedly in business transactions.

    • For example, customer master describes an entity with which you do business. Once created, it gets updated only when there are changes to the customer’s information (address, phone number). This customer record can be used in a variety of business documents such as sales orders, contracts, and invoices.

  • Transactional data
    • It represents transactions that take place in the SAP S/4HANA system. It is very dynamic and references one or more master data objects.

    • Examples include sales orders and purchase orders.

During the design phase, the inventory of data objects is finalized. This drives all further discussions on transformation and loading processes. Once completed, specific field-level requirements are defined in the detailed design phase.

Another important consideration developing a security classification for data conversion objects based on GDPR or similar regulations. Such classification drives specific data to be scrambled in nonproduction for privacy concerns, often best done at the source during the extraction process. Masking or encrypting data early ensures proper security throughout the conversion process from the source systems to the SAP S/4HANA deployment.

Please note that for data conversion during SAP S/4HANA deployments in Europe, the GDPR must be considered. The GDPR (EU Regulation 2016/679) is a regulation by which the European Parliament, the European Council, and the European Commission intend to strengthen and unify data protection for individuals within the European Union (EU). It also addresses the export of personal data outside the EU. The regulation was adopted on April 27, 2016 and applied from May 25, 2018. More details can be found at https://en.wikipedia.org/wiki/General_Data_Protection_Regulation .

Table 5-1 provides the representative list of data objects used during the SAP S/4HANA implementation.
Table 5-1

Data Objects Used During SAP S/4HANA Implementation

Area

Master Data

Transaction Data

Common

Financial Master

GL Accounts–Global/ Company Code

Profit Center

Cost Center

Supplier Master (Business Partner–Supplier)

Customer Master (Business Partner–Customer)

Material Master

 

Finance

Bank Keys

Exchange Rate

Asset Master

Activity Type

Activity Price

Assets Values

Condition Contract

GL–Balances

GL–Open Items

AP–Open Items

AR–Open Items

Procurement

Purchasing Info Record

Purchasing Category

Quota Arrangement

Purchase Orders–Open Items

Manufacturing

Characteristic

Class

Bill of Material

Work Center–Resources

Recipe

Production Route

Production Version

Inspection Characteristic

Inspection Plan

Inspection Task List

Inspection Lot

Process Order–Open Items

Stock Balance

Sales and Distribution

Customer-Material Info Record

Pricing Condition

Output Condition

Route Determination

Material Determination

Contract for Pricing

Sales Order–Open Items

Data Conversion Tool Sets

SAP provides a variety of tools to extract, transform, and load data into SAP S/4HANA.

SAP Data Services and Rapid Data Migration

SAP recommends SAP Data Services (DS) and Rapid Data Migration (RDM) as the primary data conversion tool for SAP S/4HANA. SAP provides the basic SAP DS with S/4HANA. If you need any additional data quality tools with specific functionalities for data deduplication, data parsing, or geolocation matching, and so on, they can also be added to the package.

RDM provides about 50 prebuilt standard SAP S/4HANA master and transaction data objects. The RDM package covers the entire process of data conversion, including extract, transform, cleanse, map, and upload to SAP S/4HANA.

Once the RDM package is configured, you can execute the following steps to accelerate data conversion for specific objects.
  • Connect to legacy data.

  • Map legacy data to SAP S/4HANA structures.

  • Execute an object processing job and complete the extraction process.

  • Validate extracted data in terms of data quality (consistency, uniqueness, and Completeness).

  • Map input values to SAP S/4HANA data structures.

  • Load data into SAP S/4HANA.

  • Validate data into SAP S/4HANA.

Table 5-2 lists the various prebuilt RDM packages available for SAP S/4HANA data objects. New objects that are not part of RDM can be built directly in SAP DS.
Table 5-2

Prebuilt RDM Packages

Batch

Maintenance Plan

Production Order

Secondary Cost Element

Characteristic

Class

Activity Type

Activity Price

Exchange Rate

Cost Centres

Profit Centres

Bank Master

Customer Master

Vendor Master

Work Centre

Material Master

Material Master Classification

Configuration Profile for Material

Inspection Method

Material Inspection Type

Material Inspection Characteristics

Inspection Plan

Bill of Materials

Equipments

Service Master

Object Dependency

Material Customer Replenishment

Reference Operation Sets

Routing

SD Pricing

Inventory Balances

Functional Location

Cost Center Groups

Source List

Work Breakdown Structure

Fixed Assets

Purchase Info Record

Planned Independent Requirements

Order Reservations

Purchasing Requisition

Contracts

Scheduling Agreements

Sales Order

Purchase Order

Internal Order

Open Deliveries

Activity Type Groups

Profit Centre Groups

Business Partner (Customer and Vendor)

Statistical Key Figures

Supplier Invoice

Credit Memo

Data Migration Data Quality

Customer Invoice Billing

Along with SAP Data Services, you can also consider using SAP Information Steward (IS), a data quality profiling and monitoring tool. With SAP IS, it is possible to build data quality rules based on the business rules provided in the data mapping documents and verify the data quality during the mock conversions or final load.

There are various data quality capabilities useful during data conversion, including the following:
  • Matching scoring.

  • Consolidation.

  • Standardization.

  • Parsing.

  • Cleansing.

  • Geocoding.

SAP IS also provides multiple dashboards to measure data quality through the different data quality dimensions, capture score trends, drill down to a field level.

SAP Legacy System Migration Workbench

Use of SAP Legacy System Migration Workbench (LSMW) is not recommended for SAP S/4HANA. However, it is still used as a backup plan when some objects do not work correctly in SAP DS or other methods.

Migration Cockpit and Migration Object Modeler

Migration Cockpit (MC) is built using SAP Landscape Transformation (SLT) technology. It was originally built to support extract, transform, and Load (ETL) for SAP S/4HANA in the cloud, so their functionalities are simpler and easier to use because the cloud version of SAP S/4HANA is simpler than the on-premise version.

Migration Object Modeler (MOM) is a functionality built into SAP S/4HANA to support modifications and updates to the standard programs and data models prebuilt in MC.

Figure 5-1 provides an overview of the data conversion process and tools to be used during SAP S/4HANA implementations.
Figure 5-1

S/4HANA data conversion process with tools

Data Conversion Methods

There are two ways in which data conversion happens during SAP S/4HANA implementation: automated or manual. The automated method uses programs and tools to extract data from the legacy source systems, transform it, and then load it into SAP S/4HANA. It requires the programs to be designed, coded, and tested with end-to-end automation.

It is useful with the following conditions:
  • A large number of data objects in scope for conversion.

  • Large volumes of records and fields for most data objects.

  • Complex cleansing and transformation rules.

  • Limited cutover window, the time frame to extract, transform, and load all data objects.

  • To mitigate the risk of human error during manual steps.

  • Repeatable data conversion process to execute multiple mocks and cutover.

The manual data conversion process is used when a data object has a small volume of records and fields. Data is manually entered into the SAP S/4HANA system using specific transactions (t-codes). A manual load is also considered when the amount of data manipulation is excessive for the automated approach. Manual conversion requires no development effort and is very flexible. However, this approach should be avoided due to the high chance of human error and time constraints.

Particular objects and source systems will require manual steps and manual loading due to technical limitations. The threshold for manual loads is often based on the volume of records, several columns, and complexity of data, typically capped at 100 records.

Some organizations prefer to develop a hybrid approach, which is a combination of the manual and automated methods. It has an automated process with limited manual intervention, to ensure that the conversion and validation process is successfully executed, repeatedly practiced, and then delivered during cutover. Such a semiautomated approach is used where the data is manually collected or constructed in data load template files, but automatically loaded via a loading tool.

Data Conversion Processes

In this section, we discuss key data conversion processes followed during the SAP S/4HANA implementation.

Data Cleansing

Data cleansing focuses on the quality of data. It involves analyzing data, identifying inaccuracies, and making the necessary corrections to data. Data profiling acts as input to the data cleansing exercise. Data cleansing is run throughout the SAP S/4HANA implementation to ensure data quality improves over time.

The source of the data is the legacy system from which its extracted. Data cleansing is focused on the extracted data for further analysis. It checks current data against (1) standard SAP field rules, and (2) business rules as defined by the process teams. Such an exercise, when automated, is referred to as data profiling. The results of profiling provide insight into the current quality of the data and the amount of effort required to update values as per the requirements of the SAP S/4HANA system. The data profiling exercise assesses extracted data across these parameters:
  • Uniqueness to ensure data records with their attributes are not duplicated.

  • Completeness to ensure all mandatory data is filled.

  • Accuracy to validate appropriate values are populated.

  • Integrity to ensure data is correctly referenced to drop-down values.

  • Conformity to confirm proper formatting of data in terms of field lengths and data types.

The data profiling reports help you discover quality issues and generate recommendations for data cleansing.

Corrective action is taken either by manually correcting data or automated uploading in the source system. Corrections happen in the legacy system before the extraction. The direct manual method is preferred to address obvious data quality issues, like misspelled names, incorrect or incomplete addresses, and so on. It is also used in cases when work cannot be completed programmatically and requires manual intervention.

On the other hand, automated uploading helps you update multiple records at once; for example, status fields. It is used when many documents need to be updated using similar values. If not available, you will need to develop such a mass update tool in the legacy system.

In some cases, you cleanse the data within a tool like SAP Data Services instead of in the legacy system. It is suited to cleansing activities where precise rules can be defined to enable a systematic addition, update, or replacement of values; for example, mandatory fields expected by the SAP S/4HANA data structure. In such a case, source data in the legacy system is still inaccurate, although the load files for each mock and final load contain the correct values.

Data cleansing done in the SAP Data Services tool falls under data transformation activity. There are various ways in which data transformation is achieved during SAP S/4HANA implementation.
  • Relevancy rules determine the relevance of the extracted data by performing cross-checks between master and transactional data. Such rules separate the live data from data that is obsolete. For example, you might decide to deactivate a particular plant. Master and transaction data related to that inactive plant is filtered out. Only current data is then sent to SAP S/4HANA.

  • Data standardization ensures the data is in in the same format and fields as required by SAP S/4HANA. It is performed by checking addresses, postal codes, and country codes against third-party data.

  • Data deduplication is done by identifying and analyzing duplicate master data records.
    • Matching criteria are applied to enable the identification of duplicate records. For example, the same name, address, postal code, and city might help you identify duplicate customer master records.

    • Such records are presented to the user, who identifies unique records and tags the rest of them as duplicates. Such duplicate records are filtered out, and individual records are sent to SAP S/4HANA.

    • You also develop a deliberate mechanism to systematically deactivate duplicate records and delete associated transaction data so that such records do not appear in further iterations.

  • Data enrichment is done by mapping legacy values into target values relevant for SAP S/4HANA. For example, plant 1234 in the legacy system is mapped to PL01 in the SAP S/4HANA system. It also involves filling in the missing (but mandatory) information before data loading.

Data Validation

Data validation is a process of validating the converted data based on specific criteria during the ETL process. There are two ways in which data validation is done.
  • Systematic data validation is done by the data conversion teams and process teams. It is achieved by using conversion reconciliation tools to reconcile the data throughout the conversion process.

  • Business data validation is done by business data object owners who own the converted data. It is achieved by having business resources verifying the SAP S/4HANA screen values with data load files.
    • Because manual verification of data takes quite an effort, you might decide to validate only the sample data size of the total converted records. If its small-sized data consists of up to 100 records, you might check the entire set of records one by one.

    • You can do manual data validation either by count matching, aggregate validation, or spot checking for data in the legacy and SAP S/4HANA system.

Data validation is typically performed at three points during the entire ETL cycle.

Postextraction Data Validation

In this activity, the extracted data is validated to match the selection criteria and relevancy rules defined for the extraction process. This ensures that only the intended set of records is extracted.

Posttransformation Data Validation

In this activity, transformed data is validated once transformation rules are applied before loading into the SAP S/4HANA system. You compare the final load file against the data transformation rules (including data mapping rules) and ensure all the transformation logic is correctly applied to the data.

Postload Data Validation

Once the data load is completed, a comparison is made between the load file and the SAP S/4HANA data to ensure that the data is loaded correctly and ready to be used in business transactions in the new system. Note that once data validation fails at any step, corrective action should be taken. Data should be reloaded and validated again to ensure identified issues are sorted out, and data quality is up to par.

Not all data validations are necessary for all converted data objects. Business data validation is needed to validate data when it is extracted, transformed, and loaded using a tool. Manually constructed data might need only postload validation. Because you manually prepare the data, no transformation logic is applied to it. Therefore, it is assumed that the data is accurate and complete as is. On the other hand, data objects with currency values or calculations might need all three types of validations. Data objects with low complexity and low volume only need posttransformation and postload validation.

Mock Data Conversion

Mock data conversions are the test cycles where the end-to-end data conversion process and tools are tested and continuously refined for all data objects. For each mock conversion, you set up activities and the scope of each data conversion cycle (ETL). Depending on the data scope, you could decide to run as many mocks as needed.

Mock data conversion testing often precedes with test cycles such as system integration testing, performance testing, and user acceptance testing. A portion of the converted data from the mocks is used as test data for these test cycles.
  • Before the start of each mock conversion cycle, the following prerequisites should be in place.
    • A mock conversion plan, including the detailed steps and timing for the conversion of data from the source systems to SAP S/4HANA. This includes extraction, transformation, load, and validation timelines.

    • Mock environments with adequate sizing specifications to prevent impact on the processing time of the data conversion programs.

    • Proper access for the data conversion team as well as the data validation and cleansing team.

    • Completed functional configuration in the SAP S/4HANA system as per the requirements of data objects.

    • Data conversion tools required to execute the mock built and unit tested. By the time the final mock is performed, you should have all data conversion tools in place.

    • Data validation scripts prepared with step-by-step instructions that business data object owners follow during each data validation.

  • The following details are tracked during each mock cycle for every data object.
    • Processing time for extraction, transformation, load, and validation and cleansing.

    • Record counts.

    • Load results.

  • Initial mocks might include the limited set of data, depending on the readiness of data objects in terms of extraction programs, transformation rules, and load programs. They are used to validate dependencies between various data objects and to test the sequencing of data conversion activities.
    • Subsequent mocks are run with a full set of data to be used for various test cycles and help refine the end-to-end data conversion cycle, including sequencing of data objects and runtimes. As execution times are recorded with each data conversion step, incremental data load happens with each iteration until the production-size data load is executed successfully.

    • Data validation, as well as cleansing, is completed to ensure proper data quality.

    • The final cutover and data deployment plan with the necessary tasks, timings, sequence, and resources (both local and remote resources) is confirmed based on the results of the final’ production size load. This mock test cycle (often called dress rehearsal) ensures that 100% of master and transaction data is successfully loaded within the designated time.

Data Conversion Cutover

Data conversion cutover is the set of data conversion activities performed to transition from the legacy system(s) to the new SAP S/4HANA system. The activities are like those executed during a mock conversion cycle, except that data load is directly done into the production system during a small window of time. The final cutover and data deployment plan with the necessary tasks, timings, sequence, and resources is followed closely as each step is executed in sequence by the respective teams.

Data conversion activities at cutover start with an initial load of master data once all the prerequisites for data readiness are met. Transaction and historical data load are followed by systematic and business data validation of the cutover loads. If any errors are encountered, they are fixed either manually or programmatically. Data is reloaded or set directly into the production system. Once business data object owners validate and sign off on the data load, data conversion cutover is complete.

Data Freeze

Before data conversion cutover, data creation and maintenance activities in the legacy systems are halted for a specific period. This process is called the data freeze . The purpose of the data freeze is to extract all relevant data from the legacy systems and load it into SAP S/4HANA without risk of the data being changed. If such changes continue to occur in master and transaction data in the legacy system, it causes issues related to data consistency in SAP S/4HANA. For example, change in the material attributes or open order items causes problems among the business users doing the postload validation and might require extensive analysis to identify and fix discrepant data. Reloading of such data in the production environment might cause additional issues unless tested thoroughly in the lower environments.

In cases where data freeze cannot be planned, dual maintenance forms are maintained. These are the templates populated manually by business users to record data additions, updates, and transactions. These updates are then keyed manually into the SAP S/4HANA system after data conversion cutover is complete. Such dual maintenance adds a lot of effort for tracking and then syncing with SAP S/4HANA, so it’s often done only for a limited period.

Business Partner Data Conversion

With SAP S/4HANA, customer and vendor masters are redundant. Instead, the approach now is of a Business Partner, which is a universal account that could play multiple roles within the organization.

The specific tables for customer data (KNA1) and vendor data (LFA1) remain available and are not affected. Although it is a good idea to keep business partner ID and customer ID and vendor ID the same, it might not be possible in the case of overlapping number ranges. Here are some of the options in case of overlapping number ranges between partner IDs.
  • You can choose to the harmonize vendor and customer master. For example, you can retain the customer number range while the vendor number range is migrated to the customer number range. Then, business partner IDs are generated from the same number range.
    • The problem with this approach is that you lose the history of your vendors, as the vendor data is divided over two numbers.

    • During data conversion, specific mapping rules are to be developed to map old vendor numbers to new business partner IDs.

  • You can choose to keep the same vendor and customer IDs and link them to the business partner with a whole new number range.
    • You do not lose the history of your vendors and customers. Also, you do not have to expend additional data conversion effort to map them to the new number range.

    • It does not affect external integrations with third parties that use current customer and vendor numbers.

  • You can choose a hybrid approach, which is a combination of the preceding strategies, based on the different number ranges allocated to the specific types of vendors and customers.

Additional Reference

Copyright information

© Sanket Kulkarni 2019

Authors and Affiliations

  • Sanket Kulkarni
    • 1
  1. 1.PuneIndia

Personalised recommendations