Realtime Streaming with Apache Flink

Join us in this detailed tutorial as we build an end-to-end data engineering project using Apache Flink, Kafka, Elasticsearch, and Docker. From system architecture to real-time dashboard results, we cover all aspects of creating a scalable data pipeline for real-time data processing and analytics.

Created by

Yusuf Ganiyu

02:26:01 Hours

(0 Reviews)

English

Last updated

Mon, 04-Dec-2023

Course description

In this comprehensive video, we embark on an exciting journey to build a complete data engineering project from scratch. We utilize some of the most influential technologies in the industry, including Apache Flink, Kafka, Elasticsearch, and Docker.

The course begins with an introduction to the system architecture, followed by practical steps in sales analytics data generation, producing data into Kafka Broker, and setting up the Apache Flink project.

We then delve into consuming data from Kafka with Apache Flink, managing Apache Flink on Mac, and writing Kafka streams to a Postgres database. The tutorial further explores aggregating transactions into Postgres by category, day, and month, and writing Kafka streams data into Elasticsearch.

We also cover reindexing data on Elasticsearch with a timestamp and creating a streaming dashboard on Elasticsearch, culminating in showcasing the real-time dashboard results.

This video is a must-watch for anyone interested in real-time data processing, offering a step-by-step guide to mastering these powerful technologies.

What will i learn?

Requirements

Realtime Streaming with Apache Flink | End to End Data Engineering Project
02:26:01

Yusuf Ganiyu

View profile Follow

£14.99

£24.99

Lectures

1

Skill level

Beginner

Expiry period

Lifetime

Certificate

Yes

Remove from cart Add to cart Buy now

Related courses

Beginner

Realtime Data Streaming | End To End Data Engineering Project

(1 Reviews)

Compare

In this video, you will be building a real-time data streaming pipeline, covering each phase from data ingestion to processing and finally storage. We'll utilize a powerful stack of tools and technologies, including Apache Airflow, Python, Apache Kafka, Apache Zookeeper, Apache Spark, and Cassandra—all neatly containerized using Docker.

£14.99

~~£24.99~~

01:27:48 Hours

Beginner

Realtime Streaming with Google Cloud (GCP) | End to End Data Engineering Project

(0 Reviews)

Compare

Build a real-time YouTube Analytics pipeline! Learn to fetch data from YouTube, process it with Kafka and ksqlDB, and get instant Telegram notifications. Dive into advanced Python and Google Cloud configurations.

£14.99

~~£24.99~~

01:28:23 Hours

Beginner

Realtime Socket Streaming | End to End Data Engineering Project

(0 Reviews)

Compare

£14.99

~~£24.99~~

01:42:25 Hours

Beginner

Apache Flink for Sales Analytics — End to End Data Engineering

(0 Reviews)

Compare

Discover the power of Apache Flink in Sales Analytics with our comprehensive guide to end-to-end data engineering. Learn to process and analyze big data in real time, transforming raw data into actionable insights.

£14.99

~~£24.99~~

00:58:06 Hours

Beginner

Realtime Change Data Capture Streaming

(0 Reviews)

Compare

Master real-time data streaming and processing with our comprehensive course on Change Data Capture (CDC) using Postgres, Debezium, Kafka, Apache Spark, and Slack. Transform data management in your organization with cutting-edge technologies.

£14.99

~~£24.99~~

01:05:31 Hours

Beginner

Real-Time Election Voting System | End-to-End Data Engineering Project

(0 Reviews)

Compare

Dive into the world of real-time data processing with our comprehensive course on building a Real-Time Election Voting System. Master the integration of Kafka, Docker, Postgres, and Streamlit in this end-to-end data engineering project.

£14.99

~~£24.99~~

02:18:35 Hours