Business activity monitoring (BAM) provides end-to-end visibility of business processes, enabling enterprises to make better-informed decisions and quickly address problem areas. A BAM framework is used for continuous event correlation, real-time alerts, and monitoring KPI statistics for business activities in real-time.
01.
Business needs
A US-based Fortune 500 mortgage lender wanted a BAM solution that could apply data reconciliation checks, aggregate data anomalies for reporting, and perform rule-based data quality checks in real-time. They were looking for a solution that would:
- Power real-time ingestion, event processing, and KPI computing
- Detect data issues with continuous matching and monitoring
- Ensure high performance, alerts, and customization
02.
Solution
Gathr enabled the client to implement an end-to-end continuous monitoring solution leveraging out-of-the-box drag-and-drop operators and a custom development framework. Using a data-first, code-free approach, Gathr allowed users to effortlessly design and manage complex data flows on a visual canvas.
Highlights:
- Reading data from various souDeployed and managed data pipelines on Apache Spark-based infrastructure
- Created streaming and batch ETL pipelines for real-time alerting and historical data processing to generate SLA reports
- Enabled real-time data summary generation at different levels (including historical data) and provided an aggregated view of these
- Provided 200+ operators for data wrangling and transformation with support for custom extensions and late data arrival
- Enabled rapid, efficient prototyping, and operationalization of ETL frameworks
- Delivered an SLA monitoring solution in 3 modules based on different types of pipelines:
- Data ingestion, quality management, and landing
- Data correlation and SLA monitoring
- End of day and month views of ETL processing data
03.
Results
- Enabled ingestion and processing of 1 TB data on a daily basis
- 10x faster data processing compared to the existing system
- Efficient near real-time KPI tracking with code-free business rule updates at runtime
- Improved visibility through correlation of real-time and historical data
- Better decision-making with instant SLA status checks
Click here to download the full case study
