A Fortune Global 500 insurance brokerage and risk management company wanted to have a single source of truth and a comprehensive data repository to enhance cost efficiency, performance, and security. They also wanted to migrate their use cases as-is without compromising on data accuracy, SLAs, and existing production applications and business users.
The brokerage firm wanted a self-service ingestion platform that business users could directly use. They had specific governance and compliance needs and wanted to integrate with enterprise tools to reduce time-to-market. They wanted a solution that would dynamically handle load fluctuation depending on accounting SLAs, priority, and cost.
Additionally, the firm was looking for a cloud-based data lake that would track project costs, provision user authorization for business data access needs, and help them to:
- Enable robust network policies for on-cloud systems adhering to compliance specifications
- Ensure audit certification and compliance
- Enable automated detection, masking, and encryption of PII, PHI, and SPI data
- Ensure high availability and operational readiness of production systems
- Leverage a self-service web portal for data ingestion, tracking, and debugging
Supported ingestion of 500+ batch and real-time data feeds on the platform with multi-TB data volume
The Impetus team migrated all the existing applications to AWS with zero downtime for the business. A fully secured data lake on the AWS platform with enterprise governance and security compliance was implemented with the following capabilities:
- Data pipelines connecting Teradata, Oracle, and MS SQL to AWS S3
- Consume layer enabled with Tableau, Presto, Jupyter NoteBook, and Hue
- Voltage-enabled encryption and Ranger powered access policies
- Self-service data ingestion
- Automated analysis of data quality profiles and trends
- High availability of all critical components
- Single-click deployment for data pipelines and platform
- Serverless data pipelines leveraging Lambda, Step Functions, EMR, and CloudWatch
- Single Jenkins pipeline to onboard 100+ Spring Boot, Apigee, Angular, Node.js, and Java applications as Docker containers
- High resiliency leveraging HAProxy to route network traffic between containers
- End-to-end DevOps for data platform and use cases
Moving from disparate sources to a unified data lake created a single source of truth for the insurance firm, enabling a unified, clear, and present view of their business.