Case Study

Power massive scale, real-time data processing by modernizing legacy ETL frameworks

Reduced the overall release cycle from 8 months to 8 weeks


To power real-time decision making on large data sets, enterprises need an expert team, high-performing hardware systems, and a scalable ETL solution that can accelerate development and deployment of ETL frameworks, while swiftly accommodating changing business needs.

Next-generation ETL tools allow enterprises to eff­ectively design and create an environment to mine and analyze data for making informed decisions. They isolate data from transactional systems, which ensures business-as-usual while data is analyzed in an optimized environment. These frameworks also help users solve business problems without spending cycles perfecting boilerplate code.

Business needs

A communication analytics solutions provider wanted to modernize their existing data applications and was looking for an easy-to-use and scalable solution that could process over 1.5 billion user interactions generated per day from multiple real-time feeds.

Solution

StreamAnalytix enabled the client to implement applications that run on a scalable Spark compute engine as structured streaming data pipelines while providing self-service and analytics capabilities for large-scale data processing.

The ETL solution used StreamAnalytix’s vast library of components for data acquisition, processing, enrichment, and storage. The entire data flow was created and orchestrated using a low-code methodology.

Results

StreamAnalytix enabled end-to-end data ingestion, enrichment, machine learning, action triggers, and visualization to modernize hand-written data applications to Spark structured streaming in weeks. This, in turn, helped the customer realize several strategic benefits:

  • Received roughly ~1 million lines of code in ~3 weeks using StreamAnalytix frameworks
  • Achieved a high throughput of 100000+ transactions/second, enabling processing of 1.5 billion records per day
  • Reduced the overall release cycle from 8 months to 8 weeks

Click here to download the full case study


You may also be interested in…