Business needs
NCR Corporation, a Fortune 500 global enterprise technology provider wanted to modernize their Teradata workloads to Spark Scala for reducing maintenance costs and enhancing scalability and agility. In addition, they were looking to resolve the mismatch in the metadata definition of their enterprise data warehouse and enterprise data lake. To address this, they wanted to leverage a modern data processing engine that would deliver analytics faster.
Auto-migrated 1860 BTEQ scripts along with ~5000 SQLs and 64 TB data
Solution
The Impetus team partnered with NCR to help them migrate one of the most complex finance applications to a modern data platform. The team leveraged LeapLogic, an Impetus product for automated workload transformation to auto-migrate 1860 BTEQ scripts along with ~5000 SQLs and 64 TB data from Teradata to Spark Scala to run on Hadoop in four steps – assessment and prescription, transformation, validation, and execution.
Achieved ~90% auto-transformation of BTEQ scripts
One of the requirements was to execute the transformed workloads within 6 hours, for which the Impetus team:
- Optimized the workloads by breaking bigger sequential scripts/jobs into parallel executable scripts to meet business SLAs
- Used a single Spark session to execute all the scripts of a specific job to reduce overall execution time
- Recommended target cluster sizing for the production environment
LeapLogic was also used to:
- Migrate code to Spark Scala programs and execute the code on the modern data platform
- Create a mapping sheet to map columns between enterprise data lake and data warehouse for resolving the metadata mismatch
- Automate unit testing for data and logic migration
- Perform cell-level validation to ensure matching data post migration and processing
Impact
- Achieved ~90% auto-transformation of Teradata BTEQ scripts
- Executed the migrated code in 5 hours, beating the targeted 6-hour SLA
- Saved 50% cost compared to manual transformation
- Ensured zero business disruption
LeapLogic helped NCR Corporation rapidly transform workloads from Teradata to Hadoop. The solution simplified and accelerated the transformation process with speed, reliability, and minimal coding, while avoiding business disruption.
