Build scalable applications with Apache Kafka

Apache Kafka Architecture 5 lectures 14:44 The Course Overview This video provides an overview of the entire course. Building Applications in the Publish-Subscribe Architecture What is the need for Publish-Subscribe Architecture? • Understand how Kafka tackles these problems • Learn how Kafka is used in the Streaming World Topics and Partitions What are Topics and Partitions? • Learn how to scale Kafka Topics • Understand what is an topic offset • Learn how offsets are stored per consumer group How Kafka Cluster is Built? - Brokers What is a Kafka Broker? • Learn how Topic Partitions are Replicated between Brokers • Understand what is the leader of Partitions Setup of Kafka - Using Docker How to setup Kafka using Docker? • Install Zookeeper • Install Kafka • Use Docker to setup Kafka Locally Apache Kafka Architecture 5 lectures 14:44 The Course Overview This video provides an overview of the entire course. Building Applications in the Publish-Subscribe Architecture What is the need for Publish-Subscribe Architecture? • Understand how Kafka tackles these problems • Learn how Kafka is used in the Streaming World Topics and Partitions What are Topics and Partitions? • Learn how to scale Kafka Topics • Understand what is an topic offset • Learn how offsets are stored per consumer group How Kafka Cluster is Built? - Brokers What is a Kafka Broker? • Learn how Topic Partitions are Replicated between Brokers • Understand what is the leader of Partitions Setup of Kafka - Using Docker How to setup Kafka using Docker? • Install Zookeeper • Install Kafka • Use Docker to setup Kafka Locally The Course Overview This video provides an overview of the entire course. The Course Overview This video provides an overview of the entire course. The Course Overview This video provides an overview of the entire course. The Course Overview This video provides an overview of the entire course. This video provides an overview of the entire course. This video provides an overview of the entire course. Building Applications in the Publish-Subscribe Architecture What is the need for Publish-Subscribe Architecture? • Understand how Kafka tackles these problems • Learn how Kafka is used in the Streaming World Building Applications in the Publish-Subscribe Architecture What is the need for Publish-Subscribe Architecture? • Understand how Kafka tackles these problems • Learn how Kafka is used in the Streaming World Building Applications in the Publish-Subscribe Architecture What is the need for Publish-Subscribe Architecture? • Understand how Kafka tackles these problems • Learn how Kafka is used in the Streaming World Building Applications in the Publish-Subscribe Architecture What is the need for Publish-Subscribe Architecture? • Understand how Kafka tackles these problems • Learn how Kafka is used in the Streaming World What is the need for Publish-Subscribe Architecture? • Understand how Kafka tackles these problems • Learn how Kafka is used in the Streaming World What is the need for Publish-Subscribe Architecture? • Understand how Kafka tackles these problems • Learn how Kafka is used in the Streaming World Topics and Partitions What are Topics and Partitions? • Learn how to scale Kafka Topics • Understand what is an topic offset • Learn how offsets are stored per consumer group Topics and Partitions What are Topics and Partitions? • Learn how to scale Kafka Topics • Understand what is an topic offset • Learn how offsets are stored per consumer group Topics and Partitions What are Topics and Partitions? • Learn how to scale Kafka Topics • Understand what is an topic offset • Learn how offsets are stored per consumer group Topics and Partitions What are Topics and Partitions? • Learn how to scale Kafka Topics • Understand what is an topic offset • Learn how offsets are stored per consumer group What are Topics and Partitions? • Learn how to scale Kafka Topics • Understand what is an topic offset • Learn how offsets are stored per consumer group What are Topics and Partitions? • Learn how to scale Kafka Topics • Understand what is an topic offset • Learn how offsets are stored per consumer group How Kafka Cluster is Built? - Brokers What is a Kafka Broker? • Learn how Topic Partitions are Replicated between Brokers • Understand what is the leader of Partitions How Kafka Cluster is Built? - Brokers What is a Kafka Broker? • Learn how Topic Partitions are Replicated between Brokers • Understand what is the leader of Partitions How Kafka Cluster is Built? - Brokers What is a Kafka Broker? • Learn how Topic Partitions are Replicated between Brokers • Understand what is the leader of Partitions How Kafka Cluster is Built? - Brokers What is a Kafka Broker? • Learn how Topic Partitions are Replicated between Brokers • Understand what is the leader of Partitions What is a Kafka Broker? • Learn how Topic Partitions are Replicated between Brokers • Understand what is the leader of Partitions What is a Kafka Broker? • Learn how Topic Partitions are Replicated between Brokers • Understand what is the leader of Partitions Setup of Kafka - Using Docker How to setup Kafka using Docker? • Install Zookeeper • Install Kafka • Use Docker to setup Kafka Locally Setup of Kafka - Using Docker How to setup Kafka using Docker? • Install Zookeeper • Install Kafka • Use Docker to setup Kafka Locally Setup of Kafka - Using Docker How to setup Kafka using Docker? • Install Zookeeper • Install Kafka • Use Docker to setup Kafka Locally Setup of Kafka - Using Docker How to setup Kafka using Docker? • Install Zookeeper • Install Kafka • Use Docker to setup Kafka Locally How to setup Kafka using Docker? • Install Zookeeper • Install Kafka • Use Docker to setup Kafka Locally How to setup Kafka using Docker? • Install Zookeeper • Install Kafka • Use Docker to setup Kafka Locally Producers and Consumers 6 lectures 38:10 Sending Events to Kafka - Producers API How to send events to Kafka? • Configure the Kafka Producer • Configure the Producer API • Send Events in a Synchronous Way Asynchronous Send What is asynchronous send? • Learn about Producers API – Async Send • Investigate Record Metadata • Write tests for Kafka producer Partitioning of Topics - Implementing Custom Partitioner What is partitioning of topics and how to implement custom partitioner? • Use Key of Record to Partition Data in Kafka • Configure Number of Partitions for specific topic • Implement Custom Partitioner Reading Events from Kafka - Consumer API How to read events from Kafka using Consumer API? • Learn about Consumer API • Configure and Create Kafka Consumer • Implement consuming of messages Consumer Pool Loop - Offset Management How to implement Consumer Pool Loop with committing offset? • Commit offset synchronously • Commit offset async • Test Kafka Consumer Rebalancing of Consumers What is Kafka rebalance? • Implement Rebalance Listener • Handle Offsets properly while rebalance Producers and Consumers eplicas we should have • Understand how the number of replicas affect Kafka Append-Only Distributed Log - Storing Events in Kafka What is append-only Distributed log and what is the structure of storing events in Kafka? • Learn why Scaling Kafka to multiple consumers and producers is so easy • Understand why one partition can be read by only one consumer because Kafka is storing offset per partition • Read topics from multiple consumer groups Compaction Process What is Kafka Compaction? • Learn how Compaction is performed • Understand how Compaction differ from Delete Summary of Apache Kafka Course Summarize what was covered in the entire course. • We learned about Kafka Architecture • We learned about Producer and Consumer API • We learned about Kafka Internals Advanced Kafka - Understanding Internals 5 lectures 09:09 Electing Partition Leaders - Kafka Controller Component How does Kafka elect Leaders for Partition? • Learn what is Controller component • Learn about brokers Advanced Topics Data Replication in Kafka How are Topics and Partitions replicated? • Learn how many replicas we should have • Understand how the number of replicas affect Kafka Append-Only Distributed Log - Storing Events in Kafka What is append-only Distributed log and what is the structure of storing events in Kafka? • Learn why Scaling Kafka to multiple consumers and producers is so easy • Understand why one partition can be read by only one consumer because Kafka is storing offset per partition • Read topics from multiple consumer groups Compaction Process What is Kafka Compaction? • Learn how Compaction is performed • Understand how Compaction differ from Delete Summary of Apache Kafka Course Summarize what was covered in the entire course. • We learned about Kafka Architecture • We learned about Producer and Consumer API • We learned about Kafka Internals Electing Partition Leaders - Kafka Controller Component How does Kafka elect Leaders for Partition? • Learn what is Controller component • Learn about brokers Advanced Topics Electing Partition Leaders - Kafka Controller Component How does Kafka elect Leaders for Partition? • Learn what is Controller component • Learn about brokers Advanced Topics Electing Partition Leaders - Kafka Controller Component How does Kafka elect Leaders for Partition? • Learn what is Controller component • Learn about brokers Advanced Topics Electing Partition Leaders - Kafka Controller Component How does Kafka elect Leaders for Partition? • Learn what is Controller component • Learn about brokers Advanced Topics How does Kafka elect Leaders for Partition? • Learn what is Controller component • Learn about brokers Advanced Topics How does Kafka elect Leaders for Partition? • Learn what is Controller component • Learn about brokers Advanced Topics Data Replication in Kafka How are Topics and Partitions replicated? • Learn how many replicas we should have • Understand how the number of replicas affect Kafka Data Replication in Kafka How are Topics and Partitions replicated? • Learn how many replicas we should have • Understand how the number of replicas affect Kafka Data Replication in Kafka How are Topics and Partitions replicated? • Learn how many replicas we should have • Understand how the number of replicas affect Kafka Data Replication in Kafka How are Topics and Partitions replicated? • Learn how many replicas we should have • Understand how the number of replicas affect Kafka How are Topics and Partitions replicated? • Learn how many replicas we should have • Understand how the number of replicas affect Kafka How are Topics and Partitions replicated? • Learn how many replicas we should have • Understand how the number of replicas affect Kafka Append-Only Distributed Log - Storing Events in Kafka What is append-only Distributed log and what is the structure of storing events in Kafka? • Learn why Scaling Kafka to multiple consumers and producers is so easy • Understand why one partition can be read by only one consumer because Kafka is storing offset per partition • Read topics from multiple consumer groups Append-Only Distributed Log - Storing Events in Kafka What is append-only Distributed log and what is the structure of storing events in Kafka? • Learn why Scaling Kafka to multiple consumers and producers is so easy • Understand why one partition can be read by only one consumer because Kafka is storing offset per partition • Read topics from multiple consumer groups Append-Only Distributed Log - Storing Events in Kafka What is append-only Distributed log and what is the structure of storing events in Kafka? • Learn why Scaling Kafka to multiple consumers and producers is so easy • Understand why one partition can be read by only one consumer because Kafka is storing offset per partition • Read topics from multiple consumer groups Append-Only...