Enabled 2x faster application deployment with AWS Kubernetes

Digital customer journey experience company reduced bootstrap time significantly

Challenge

A digital customer journey experience company wanted to reduce the rising infrastructure costs of their AWS data processing platform, which was deployed on Amazon Elastic Compute Cloud (EC2) instances. They were looking to reduce bootstrapping time, as this was impacting scalability and slowing down application releases.

Requirements

The client wanted to rearchitect their AWS-based data processing platform and switch to a container-based deployment that could rapidly scale up or down, as needed, during peak and off-time hours. They wanted to automate environment provisioning and application deployment processes to drive faster and better quality releases. They were also looking for 24x7 customer support for their cloud-based infrastructure, data processing platform, and applications.

Solution

Impetus Technologies leveraged Amazon Elastic Kubernetes Service (EKS), a fully-managed Kubernetes service, to implement a scalable container-based microservices architecture. Kubernetes was chosen for its automated scaling capabilities and powerful ability to run multiple containers on shared infrastructure. Containerization was adopted to enable a multi-cloud solution and accelerate release delivery, as containers support deployment of specific services rather than the entire application. We set up pod scaling to quickly scale up/down the number of API containers based on the user load and configured node-level scaling to manage additional worker nodes at runtime. In addition, we used Kubernetes’ Namespaces to run Dev and QA environments in the same cluster, which helped reduce the number of compute instances. The entire transition was completed with near-zero disruption to production.

Solution highlights:

Automated infrastructure set up for Kubernetes cluster via Jenkins pipelines to create an on-demand integration testing environment

Docker images for critical application services hosted on Amazon Elastic Container Registry (ECR)

Continuous delivery pipelines for containerized services on Flux CD to reduce deployment and release time

Horizontal pod scaling, auto-scaling groups at node level and Elastic Load Balancing for quick scalability and high availability

Airtight security leveraging Virtual Private Cloud (VPC), security groups, and Identity and Access Management (IAM) policies. Introduced Ranger for granular control on table and column level access

Unified dashboard for service-level monitoring of AWS infrastructure (as compared to multiple custom dashboards), using Datadog

On-demand transient EMR clusters with automated scripts for faster data processing

Lambda functions linked to Amazon SNS topics for terminating idle EMR clusters

Robust incident management system with 24x7 customer support

Benefits