Real Time Spark Project for Beginners: Hadoop, Spark, Docker

Simpliv

Course

Online

£ 30 + VAT

The best Other programming courses

See all

See all

Description

  • Type

    Course

  • Methodology

    Online

  • Start date

    Different dates available

Sub Title: Building Real Time Data Pipeline Using Apache Kafka, Apache Spark, Hadoop, PostgreSQL, Django and Flexmonster on Docker.In many data centers, different type of servers generate large amount of data(events, Event in this case is status of the server in the data center) in real-time
There is always a need to process these data in real-time and generate insights which will be used by the server/data center monitoring people and they have to track these server's status regularly and find the resolution in case of issues occurring, for better server stability
Since the data is huge and coming in real-time, we need to choose the right architecture with scalable storage and computation frameworks/technologies
Hence we want to build the Real Time Data Pipeline Using Apache Kafka, Apache Spark, Hadoop, PostgreSQL, Django and Flexmonster on Docker to generate insights out of this data
The Spark Project/Data Pipeline is built using Apache Spark with Scala and PySpark on Apache Hadoop Cluster which is on top of Docker
Data Visualization is built using Django Web Framework and Flexmonster

Facilities

Location

Start date

Online

Start date

Different dates availableEnrolment now open

About this course

Complete Development of Real Time Streaming Data Pipeline using Hadoop and Spark Cluster on Docker
Setting up Single Node Hadoop and Spark Cluster on Docker
Features of Spark Structured Streaming using Spark with Scala
Features of Spark Structured Streaming using Spark with Python(PySpark)
How to use PostgreSQL with Spark Structured Streaming
Basic understanding of Apache Kafka
How to build Data Visualisation using Django Web Framework and Flexmonster
Fundamentals of Docker and Containerization

Questions & Answers

Add your question

Our advisors and other users will be able to reply to you

Who would you like to address this question to?

Fill in your details to get a reply

We will only publish your name and question

Emagister S.L. (data controller) will process your data to carry out promotional activities (via email and/or phone), publish reviews, or manage incidents. You can learn about your rights and manage your preferences in the privacy policy.

Reviews

This centre's achievements

2021

All courses are up to date

The average rating is higher than 3.7

More than 50 reviews in the last 12 months

This centre has featured on Emagister for 6 years

Subjects

  • Server
  • Apache
  • Database
  • Database training
  • Web
  • Project
  • Install
  • Information Systems
  • Information Systems management
  • IT

Course programme

Introduction 2 lectures 41:08 Introduction to Apache Spark preview Real Time Spark Project Overview | Building End to End Streaming Data Pipeline Introduction 2 lectures 41:08 Introduction to Apache Spark preview Real Time Spark Project Overview | Building End to End Streaming Data Pipeline Introduction to Apache Spark preview Introduction to Apache Spark preview Introduction to Apache Spark preview Introduction to Apache Spark preview Real Time Spark Project Overview | Building End to End Streaming Data Pipeline Real Time Spark Project Overview | Building End to End Streaming Data Pipeline Real Time Spark Project Overview | Building End to End Streaming Data Pipeline Real Time Spark Project Overview | Building End to End Streaming Data Pipeline Environment Setup 6 lectures 01:38:08 Setting up Docker Environment Create Single Node Kafka Cluster on Docker Create Single Node Apache Hadoop and Spark Cluster on Docker Setting up IntelliJ IDEA Community Edition(IDE) Setting up PyCharm Community Edition(IDE) Setting up Django Web Framework Environment Setup 6 lectures 01:38:08 Setting up Docker Environment Create Single Node Kafka Cluster on Docker Create Single Node Apache Hadoop and Spark Cluster on Docker Setting up IntelliJ IDEA Community Edition(IDE) Setting up PyCharm Community Edition(IDE) Setting up Django Web Framework Setting up Docker Environment Setting up Docker Environment Setting up Docker Environment Setting up Docker Environment Create Single Node Kafka Cluster on Docker Create Single Node Kafka Cluster on Docker Create Single Node Kafka Cluster on Docker Create Single Node Kafka Cluster on Docker Create Single Node Apache Hadoop and Spark Cluster on Docker Create Single Node Apache Hadoop and Spark Cluster on Docker Create Single Node Apache Hadoop and Spark Cluster on Docker Create Single Node Apache Hadoop and Spark Cluster on Docker Setting up IntelliJ IDEA Community Edition(IDE) Setting up IntelliJ IDEA Community Edition(IDE) Setting up IntelliJ IDEA Community Edition(IDE) Setting up IntelliJ IDEA Community Edition(IDE) Setting up PyCharm Community Edition(IDE) Setting up PyCharm Community Edition(IDE) Setting up PyCharm Community Edition(IDE) Setting up PyCharm Community Edition(IDE) Setting up Django Web Framework Setting up Django Web Framework Setting up Django Web Framework Setting up Django Web Framework Development | Project Code Walk-through 5 lectures 01:46:23 Event Simulator using Python(Server Status Detail) Building Streaming Data Pipeline using Scala - Spark Structured Streaming Building Streaming Data Pipeline using PySpark | Spark Structured Streaming Setting up PostgreSQL Database(Events Database) Building Dashboard using Django Web Framework and Flexmonster | Visualization Development | Project Code Walk-through 5 lectures 01:46:23 Event Simulator using Python(Server Status Detail) Building Streaming Data Pipeline using Scala - Spark Structured Streaming Building Streaming Data Pipeline using PySpark | Spark Structured Streaming Setting up PostgreSQL Database(Events Database) Building Dashboard using Django Web Framework and Flexmonster | Visualization Event Simulator using Python(Server Status Detail) Event Simulator using Python(Server Status Detail) Event Simulator using Python(Server Status Detail) Event Simulator using Python(Server Status Detail) Building Streaming Data Pipeline using Scala - Spark Structured Streaming Building Streaming Data Pipeline using Scala - Spark Structured Streaming Building Streaming Data Pipeline using Scala - Spark Structured Streaming Building Streaming Data Pipeline using Scala - Spark Structured Streaming Building Streaming Data Pipeline using PySpark | Spark Structured Streaming Building Streaming Data Pipeline using PySpark | Spark Structured Streaming Building Streaming Data Pipeline using PySpark | Spark Structured Streaming Building Streaming Data Pipeline using PySpark | Spark Structured Streaming Setting up PostgreSQL Database(Events Database) Setting up PostgreSQL Database(Events Database) Setting up PostgreSQL Database(Events Database) Setting up PostgreSQL Database(Events Database) Building Dashboard using Django Web Framework and Flexmonster | Visualization Building Dashboard using Django Web Framework and Flexmonster | Visualization Building Dashboard using Django Web Framework and Flexmonster | Visualization Building Dashboard using Django Web Framework and Flexmonster | Visualization Complete Project Demo 2 lectures 24:42 Real Time Spark Project Demo Running Real Time Streaming Data Pipeline using Spark Cluster On Docker Complete Project Demo 2 lectures 24:42 Real Time Spark Project Demo Running Real Time Streaming Data Pipeline using Spark Cluster On Docker Real Time Spark Project Demo Real Time Spark Project Demo Real Time Spark Project Demo Real Time Spark Project Demo Running Real Time Streaming Data Pipeline using Spark Cluster On Docker Running Real Time Streaming Data Pipeline using Spark Cluster On Docker Running Real Time Streaming Data Pipeline using Spark Cluster On Docker Running Real Time Streaming Data Pipeline using Spark Cluster On Docker Bonus Tutorial: Docker Tutorial for Beginners 9 lectures 02:03:59 Introduction to Docker Install Docker on Ubuntu 18.04 Docker Commands - Commonly Used Create First Docker Image and Container Create MySQL Docker Container Cassandra on Docker Container MongoDB on Docker Container Setting up Docker Compose How to create Docker Volume Bonus Tutorial: Docker Tutorial for Beginners. 9 lectures 02:03:59 Introduction to Docker Install Docker on Ubuntu 18.04 Docker Commands - Commonly Used Create First Docker Image and Container Create MySQL Docker Container Cassandra on Docker Container MongoDB on Docker Container Setting up Docker Compose How to create Docker Volume Introduction to Docker Introduction to Docker Introduction to Docker Introduction to Docker Install Docker on Ubuntu 18.04 Install Docker on Ubuntu 18.04 Install Docker on Ubuntu 18.04 Install Docker on Ubuntu 18

Additional information

Basic understanding of Programming Language Basic understanding of Apache Hadoop Basic understanding of Apache Spark

See related categories

Real Time Spark Project for Beginners: Hadoop, Spark, Docker

£ 30 + VAT

The best Other programming courses

See all

See all