Apache Spark: The New Enterprise Backbone for ETL, Batch and Real-time Streaming

In spite of investments in modern data lakes, there is wide use of expensive proprietary products for data ingestion, integration, and transformation (ETL) while bringing and processing data on the lake.

Enterprises have now successfully tested Apache Spark for its versatility and strengths as a distributed computing framework that can completely handle all needs for data processing, analytics, and machine learning workloads.

Since public cloud already includes Apache Spark, there is nothing new to be procured. However, the skills required to put Spark to good use are typically unavailable today.

In this webinar, we will discuss how Apache Spark can be an inexpensive enterprise backbone for all types of data processing workloads. We will also demo how a visual framework on top of Apache Spark makes it much more viable.

The following scenarios will be covered:

On-Prem

• Data quality and ETL with Apache Spark using pre-built operators
• Advanced monitoring of Spark pipelines

On Cloud

• Visual interactive development of Apache Spark Structured Streaming pipelines
• IoT use-case with event-time, late-arrival and watermarks
• Python based predictive analytics running on Spark

Speakers:

Anand Venugopal,
Product Head, StreamAnalytix

Punit Shah,
Solution Architect, StreamAnalytix