If you read no other page in this documentation site before starting to use S2PX, read this one! This short page is for people who are just starting on their journey with the Server to Parallel Migration Utility (S2PX) or are returning to it after some absence. The full documentation is extensive and while it is encouraged to take the time to read it all, this page summarises the most important items that all users (and their managers) should know up front.
Basic End-to-End Workflow
These are the steps that must be followed to perform a successful conversion:-
-
Ensure that you have downloaded all of the installation files for the latest MettleCI release ((v2.0) Release History ), which will include the latest S2PX plugins.
-
Get the MettleCI CLI running. Since S2PX uses the DSX format, it is fine to use a Windows machine with a DataStage Windows Client installed.
-
Get MettleCI Workbench and both the Parallel and Server Unit Test Harnesses installed and tested as working on your DataStage Engine host.
-
Perform S2PX Analysis.
-
Determine the subset of Jobs for you’ll be generating a Unit-Test-based regression test suite (prior to conversion).
-
If the Analysis spreadsheet indicates it as necessary, run CCMT (against the source Project(s).
-
Export Jobs to DSX and put them somewhere ready for conversion.
-
Copy the Unit Tests from the relevant DataStage Engine host to somewhere on the same filesystem as the DSX files staged in the previous step.
-
Run S2PX Convert
-
Import the S2PX-generated DSX into a supported DataStage
-
Minimum DataStage Version: 11.7.1.4 SP1. Certain functions like StrCmp() are not available in earlier versions. See S2PX Prerequisites
-
-
Copy to converted Unit Tests to the DataStage Engine host where you just imported the post-conversion DSX
-
Run regression tests
-
Export successfully-tested Jobs as ISX files
-
Deploy into CP4D or other downstream environments
For more details, please visit the Architecture and Workflow here:-
Unexpected Changes During Conversion
The conversion process will produce some unexpected outputs for first-time users. Therefore, it is strongly suggested that users familiarise themselves with these aspects to avoid being surprised by the assets that S2PX generates (and, worse-still, making unnecessary modifications to the converted Jobs). The key outcomes are listed below and links are provided to more detailed documentation for those wishing to understand the factors that inform the solution’s behaviour in each case:
-
Server Jobs are frequently decomposed into multiple Parallel jobs (with a co-ordinating Sequence in place of the original Job). Read more to understand why (link)
-
The resulting Parallel Jobs frequently have more Stages in them than their Server originals.
-
Column definitions frequently will be different to the original Job and often set to Unbounded Varchars. and other Metadata settings you may not expect.
-
Hashed File stages are converted to DRS Connector (i.e. database) stages and the hashed file data will need to be migrated to the chosen database instance before running associated Parallel jobs.
-
Hashed file database tables will have programatically-generated names. varchar types
-
To properly handle nulls, S2PX converts System Variables to certain characters (like “!” and “@” characters) which are injected into the data during processing but are removed again during output.
-
-
Transformer derivations will frequently become more complex than the original (albeit functionally equivalent).
-
The resulting Parallel Jobs might run slower than the original Server jobs. Analyse the batch execution critical path for your Server jobs and plan for the possibility of targeted post-conversion tuning work.
Common Pitfalls and Mistakes
-
Not reading this page and related documentation before using the S2PX software.
-
Not planning for the manual effort required to convert custom functions used by your Server jobs. This can often represent the largest area of manual conversion work in the overall process.
-
Not running S2PX Analysis and reviewing the spreadsheet in detail before starting to convert Jobs.
-
Not running CCMT (if indicated as necessary by S2PX Analysis) before starting to convert Jobs.
-
Not using MettleCI Unit Tests to rapidly build a regression test suite so you can show early progress.
-
Assuming S2PX has made a mistake instead of checking its output against the relevant piece of documentation.
-
Taking converted Jobs straight into DataStage NextGen on CP4D instead of shaking them out first in a DataStage 11.7.1.4 SP1 (or newer) instance using Unit Tests, etc.