Data processing is divided into the three steps input, processing, and output.

Processing step Input:

• Initialization of the process
• Opening the data source and obtaining the raw data
• Preparing the raw data for processing

Depending on the data source, the input processing step requires often to run on the platform assigned to the data source, for example, bulk transfer process with direct access to IDMS database. In many cases, this step can also run on another platform, for example, Microsoft SQL Server CDC using SQL driver accesses.

Such dependencies are explained in more detail in the respective process description. When defining a process, the dashboard offers only the agents on the appropriate platforms for the input.

Processing step Processing:

• Merging the individual change instructions into transactions (LUW management)
• Filtering out transactions that are not to be replicated (Commit/Rollback/LoopBack)
• Decoupling input and output
• Preparation of data from input for normalization
• Correlate input data with metadata structures from the repository
• Converting the input data fields into a universal format (normalization)
• Preparation of the normalized data for the defined output

Data processing can be performed on any platform. Normally, however, this step is performed on a platform that is most efficiently equipped to handle the load of normalization (CPU), LUW management (memory), and decoupling (file system). When redefining a process, the dashboard selects the most efficient platform as default whenever possible.

When an agent switches between input and processing, the input part of the process automatically starts a subprocess on the other agent. The process parts connect via TCP/IP and the data is sent from the input part to the processing part at the lowest possible processing level.

Processing step Output:

• Preparation of the data for the output data target(s)
• Apply of the data to the defined output target

Depending on the output target, the output processing step must often run on the platform associated with the output target, for example, ADABAS on z/OS. In many cases, this step can also run on another platform, for example, Connection to Db2 via DRDA Protocol.

Such dependencies are explained in more detail at the respective output target.

When an agent switches between processing and output, the processing part of the process automatically starts a subprocess on the other agent. The process parts connect via TCP/IP and the data is sent from the processing part to the output part at the highest possible processing level.