Reddit Data Pipeline Engineering | AWS End to End Data Engineering

Integrate Reddit, Airflow, Celery, S3, AWS Glue, Athena, and Redshift for a seamless ETL process. Dive deep into data extraction, transformation, storage, and analytics with this comprehensive guide.

Created by

Yusuf Ganiyu

01:35:01 Hours

    • (0 Reviews)

English

Last updated

Thu, 21-Mar-2024

Course description

Embark on a comprehensive journey into the world of modern data engineering.

This tutorial is meticulously crafted to guide you through the integration of platforms like Reddit, Apache Airflow, Celery, Postgres, and Amazon's suite of data services.

Begin by extracting data from Reddit using its API, then set up and orchestrate ETL processes with Apache Airflow and Celery. Dive into efficient data storage with Amazon S3 and leverage the power of AWS Glue for data cataloging and ETL jobs.

The tutorial further delves into querying and transforming data with Amazon Athena, setting up the Redshift Cluster, and understanding the best practices for loading data into Amazon Redshift for insightful analytics.

What will i learn?

Requirements

  • Basic Knowledge of Apache Airflow
  • Basic knowledge of Docker

Yusuf Ganiyu

View profile

£14.99

£24.99

Lectures

1

Skill level

Beginner

Expiry period

Lifetime

Certificate

Yes
Remove from cart Add to cart Buy now

Related courses

Beginner

Football Data Engineering with Azure | End To End Data Engineering Project

0

(0 Reviews)

Compare

In this tutorial, we dive deep into the world of football data engineering. We'll walk through the entire process of extracting football data from Wikipedia using Apache Airflow, storing it in Azure Data Lake, migrating the data with Azure Data Factory, querying with Azure Synapse, and finally visualizing our findings in Tableau.

£14.99

£24.99

02:39:48 Hours

Beginner

End to End Data Engineering On Azure Spark Cluster: Japan Visa Analysis

0

(0 Reviews)

Compare

Master Spark on Azure with Docker! Dive into data processing and visualization of Japan's visa numbers using PySpark and Plotly. Clean, transform, and gain insights into visa trends interactively.

£14.99

£24.99

00:58:57 Hours

Beginner

Modern Data Engineering with DBT (Data Build Tool) and BigQuery

5

(2 Reviews)

Compare

Master the power of DBT & BigQuery in modern data engineering. From setup to advanced transformations and documentation, this tutorial is your comprehensive guide to staying ahead in analytics.

£14.99

£24.99

00:30:13 Hours

Beginner

Apache Airflow on Steriods for Data Engineers

0

(0 Reviews)

Compare

Dive deep into the world of data engineering with the "Apache Airflow on Steroids for Data Engineers" full course. Master the integration of Apache Airflow with Docker, Spark Clusters, and programming in Scala, Python, and Java to build and monitor complex data workflows.

£14.99

£24.99

01:08:42 Hours

Intermediate

AWS EMR (Elastic Map Reduce) For Data Engineers

0

(0 Reviews)

Compare

Unlock the power of AWS for data engineering with our comprehensive guide on Amazon Elastic Map Reduce (EMR). Ideal for both beginners and seasoned professionals, this course offers a deep dive into EMR, teaching you how to leverage its capabilities for efficient data processing and visualization.

£14.99

£24.99

00:39:24 Hours

Beginner

CI/CD For Modern Data Engineering

0

(0 Reviews)

Compare

£14.99

£24.99

01:09:45 Hours

Beginner

Kubernetes for Data Engineering: An end to end guide

0

(0 Reviews)

Compare

Enhance your data engineering prowess with Kubernetes and Apache Airflow

£14.99

£24.99

02:28:17 Hours

Beginner

Building a Smart City: An End-to-End Big Data Engineering Project

0

(0 Reviews)

Compare

Learn to harness the power of IoT devices, Apache Kafka, Apache Spark, AWS, and more to create a real-time data streaming pipeline that brings smart cities to life.

£14.99

£24.99

02:32:16 Hours

Intermediate

Mastering Apache Spark Realtime Streaming for Unstructured Data

0

(0 Reviews)

Compare

This course is designed with precision to provide you with an in-depth understanding, practical skills, and the latest tools required to develop scalable, efficient, and robust streaming data pipelines.

£14.99

£24.99

02:35:23 Hours

Beginner

Real Estate Data Engineering with Artificial Intelligence

0

(0 Reviews)

Compare

Master the use of cutting-edge tools such as Large Language Models (ChatGPT), WebSocket, Chrome DevTools Protocol (CDP), Docker, Apache Kafka, Apache Spark, Apache Zookeeper, Confluent Control Center, and Cassandra

£14.99

£29.99

02:06:04 Hours

Data Mastery Lab is an online learning and teaching platform for data engineering, data science, data analytics, machine learning and artificial intelligence.

Top categories

Useful links

Help

Subscribe to our newsletter