Apache Kafka Series – Learn Apache Kafka for Beginners v2

Apache Kafka Series – Learn Apache Kafka for Beginners v2
Apache Kafka Series – Learn Apache Kafka for Beginners v2

English | MP4 | AVC 1280×720 | AAC 48KHz 2ch | 7.5 Hours | 3.16 GB
eLearning | Skill level: All Levels


START HERE: Learn Apache Kafka 2.0 Ecosystem, Core Concepts, Real World Java Producers/Consumers & Big Data Architecture

If you look at the documentation, you can see that Apache Kafka is not easy to learn…

Thanks to my several years of experience in Kafka and Big Data, I wanted to make learning Kafka accessible to everyone.

We’ll take a step-by-step approach to learn all the fundamentals of Apache Kafka.

At the end of this course, you’ll be productive and you’ll know the following:

  • The Apache Kafka Ecosystem Architecture
  • The Kafka Core Concepts: Topics, Partitions, Brokers, Replicas, Producers, Consumers, and more!
  • Launch your own Kafka cluster in no time using native Kafka binaries – Windows / MacOS X / Linux
  • Learn and Practice using the Kafka Command Line Interface (CLI)
  • Code Producer and Consumers using the Java API
  • Real world project using Twitter as a source of data for a producer and ElasticSearch as a sink for our consumer
  • Overview of Advanced APIs (Kafka Connect, Kafka Streams)
  • Real World Case Studies and Big Use cases
  • Overview of Advanced Kafka for Administrators
  • Advanced Topic Configurations
  • Annexes (starting a Kafka cluster locally, using Docker, etc…)

Note: The hands-on section is based on Java, which is the native Kafka programming language. But, good news! Your learning in Java will be completely applicable to other programming languages, such as Python, C#, Node.js or Scala, and Big Data frameworks such as Spark, NiFi or Akka

What you’ll learn

  • Understand Apache Kafka Ecosystem, Architecture, Core Concepts and Operations
  • Master Concepts such as Topics, Partitions, Brokers, Producers, Consumers
  • Start a personal Kafka development environment
  • Learn major CLIs: kafka-topics, kafka-console-producer, kafka-console-consumer, kafka-consumer-groups, kafka-configs
  • Create your Producers and Consumers in Java to interact with Kafka
  • Program a Real World Twitter Producer & ElasticSearch Consumer
  • Extended APIs Overview (Kafka Connect, Kafka Streams), Case Studies and Big Data Architecture
  • Practice and Understand Log Compa
+ Table of Contents

Kafka Introduction
1 Welcome!
2 Apache Kafka in 5 minutes
3 Course Objectives
4 Important Message

Code Download
5 Code Download

====== Kafka Fundamentals ======
6 Kafka Fundamentals

Kafka Theory
7 Kafka Theory Overview
8 Topics, Partitions and Offsets
9 Brokers and Topics
10 Topic Replication
11 Producers and Message Keys
12 Consumers & Consumer Groups
13 Consumer Offsets & Delivery Semantics
14 Kafka Broker Discovery
15 Zookeeper
16 Kafka Guarantees
17 Theory Roundup

Starting Kafka
18 Important: Starting Kafka
19 FAQ for Setup Problems
20 Mac OS X – Download and Setup Kafka in PATH
21 Mac OS X – Using brew
22 Mac OS X – Start Zookeeper and Kafka
23 Mac OS X – Summary
24 Linux – Download and Setup Kafka in PATH
25 Linux – Start Zookeeper and Kafka
26 Linux – Summary
27 Windows – Download Kafka and PATH Setup
28 Windows – Start Zookeeper & Kafka
29 Windows – Summary

CLI (Command Line Interface) 101
30 CLI Introduction
31 WINDOWS WARNING: PLEASE READ
32 Kafka Topics CLI
33 Kafka Console Producer CLI
34 Kafka Console Consumer CLI
35 Kafka Consumers in Group
36 Kafka Consumer Groups CLI
37 Resetting Offsets
38 CLI Options that are good to know
39 What about UIs?
40 Kafka Tools UI
41 KafkaCat as a replacement for Kafka CLI

Kafka Java Programming 101
42 Intro to Kafka Programming
43 Installing Java 8 & IntelliJ Community Edition
44 Creating Kafka Project
45 Java Producer
46 Java Producer Callbacks
47 Java Producer with Keys
48 Java Consumer
49 Java Consumer inside Consumer Group
50 Java Consumer with Threads
51 Java Consumer Seek and Assign
52 Client Bi-Directional Compatibility
53 Configuring Producers and Consumers

===== Kafka Real World Project =====
54 Real World Project Overview
55 Real World Exercise

Kafka Twitter Producer & Advanced Configurations
56 Producer and Advanced Configurations Overview
57 Twitter Setup
58 Producer Part 1 – Writing Twitter Client
59 Producer Part 2 – Writing the Kafka Producer
60 Producer Configurations Introduction
61 acks & min.insync.replicas
62 retries & max.in.flight.requests.per.connection
63 Idempotent Producer
64 Producer Part 3 – Safe Producer
65 Producer Compression
66 Producer Batching
67 Producer Part 4 – High Throughput Producer
68 Producer Default Partitions and Key Hashing
69 [Advanced] max.block.ms and buffer.memory
70 Refactoring the Project

Kafka ElasticSearch Consumer & Advanced Configurations
71 Consumer and Advanced Configuration Overview
72 Setting up ElasticSearch in the Cloud
73 ElasticSearch 101
74 Consumer Part 1 – Setup Project
75 Consumer Part 2 – Write the Consumer & Send to ElasticSearch
76 Delivery Semantics for Consumers
77 Consumer Part 3 – Idempotence
78 Consumer Offset Commit Strategies
79 Consumer Part 4 – Manual Commit of Offsets
80 Consumer Part 5 – Performance Improvement using Batching
81 Consumer Offsets Reset Behaviour
82 Consumer Part 6 – Replaying Data
83 Consumer Internal Threads

===== Kafka Ecosystem & Real-World Architectures =====
84 Kafka in the Real World

Kafka Extended APIs for Developers
85 Kafka Connect Introduction
86 Kafka Connect Twitter Hands-On
87 Kafka Streams Introduction
88 Kafka Streams Hands-On
89 Kafka Schema Registry Introduction
90 Which Kafka API should I use?

Real World Insights and Case Studies (Big Data / Fast Data)
91 Choosing Partition Count & Replication Factor
92 Kafka Topics Naming Convention
93 Case Study – MovieFlix
94 Case Study – GetTaxi
95 Case Study – MySocialMedia
96 Case Study – MyBank
97 Case Study – Big Data Ingestion
98 Case Study – Logging and Metrics Aggregation

Kafka in the Enterprise for Admins
99 Kafka Cluster Setup High Level Architecture Overview
100 Kafka Monitoring & Operations
101 Kafka Security
102 Kafka Multi Cluster & MirrorMaker

===== Advanced Kafka =====
103 Advanced Kafka

Advanced Topics Configurations
104 Changing a Topic Configuration
105 Segment and Indexes
106 Log Cleanup Policies
107 Log Cleanup Delete
108 Log Compaction Theory
109 Log Compaction Practice
110 min.insync.replicas reminder
111 Unclean Leader Election

===== Annexes =====
112 What are annexes?

Annex – Starting Kafka Differently
113 Annex 1 – Overview
114 Starting Kafka with the Confluent CLI
115 Starting a multi broker Kafka Cluster using Binaries
116 Note on Docker Setup
117 Start Kafka Development environment using Docker
118 Starting a multi broker Kafka Cluster using Docker
119 Kafka Advertised Host Setting
120 Starting Kafka on a Remote Machine

===== Next Steps =====
121 What’s Next?
122 Bonus Lecture: Student Special Coupons for my Other courses