Kafka For Developers - Data Contracts Using Schema Registry

What you'll learn
Introduction to AVRO and its advantages of using them for sharing messages between applications
Publish AVRO records using Kafka Producer
Introduction to Schema Registry
Consume AVRO records using Kafka Producer
Use Schema Registry to register the AVRO Schema
Learn how Kafka Producer and Consumer interacts with the Schema Registry
Enforce Data Contracts using Schema Registry
Evolve AVRO schema using Schema Registry
Build Spring Boot Kafka Producer and Consumer applications that uses AVRO as a serialization format and interacts with Schema Registry
Requirements
Java 11 or Higher is needed (Java 17 recommended)
Docker must be installed
Intellij IDE or similar is needed
Experience build Kafka Producer and Consumer applications
Description
This course is structured to give you a theoretical and coding experience of Building Kafka Applications using AVRO and Schema Registry. If you are looking forward to learning the below-listed things:Techniques that are available to evolve the data between the applications that uses Kafka as a Streaming Platform Use a compact data format like AVRO to exchange data between the applicationsUse Schema Registry and its benefitsEnforcing Data Contracts between applications that uses Kafka as a Streaming PlatformHandle Data evolution gracefully using Schema RegistryThen this is the right course for you. This is a pure hands-on oriented course where you will be learning the concepts through code.By the end of this course, you will have a complete understanding of these concepts:Use AVRO as a data serialization formatEvolution of the data using Schema RegistryGetting Started with KafkaIn this section, I will give you all an introduction to the course and what to expect from this course.Data Contract & Serialization in KafkaLearn "How serialization is connected to Kafka" and how it benefits the overall Kafka architecture.We will look into different Serialization formats and the support for Schema in AVRO, Protobuf and ThriftIntroduction to AVRO - A data serialization systemAn introduction to AVRO and why AVRO is popular to work with Kafka and Schema Registry.Learn to build a simple AVRO schema.Kafka Setup & Demo in Local Using DockerIn this section , we will setup Kafka in local, Produce and Consume messages using Kafka Console Producer and Consumer.Greeting App - Base AVRO Project SetUp - GradleWe will set up the base project for the greeting app which we can use to generate the Java Classes from the Greetings schema using Gradle build tool.Greeting App - Base AVRO Project SetUp - MavenWe will set up the base project for the greeting app which we can use to generate the Java Classes from the Greetings schema using Maven build tool.Build AVRO Producer and Consumer in JavaWe will learn to build a Kafka Producer to publish AVRO records in to the Kafka topic.We will learn to build a Kafka Consumer to consume AVRO records from the Kafka topic.CoffeeShop Order Service Using AVRO - A Real time use CaseWe will build a AVRO schema for a real time use case and build Kafka Producers and Consumers to it.Logical Types in AVROI will cover the different logical types in AVRO and how to use them.TimeStampDecimalUUIDDateAVRO Record- Under the HoodAnatomy of an AVRO record when the data is published and consumed as AVRO recordSchema Changes in AVRODemonstration of how the consumer breaks with changing business requirementsData Evolution using Schema RegistryCover the different techniques of evolving a Schema with the changing business requirements.I will cover the different Compatibility techniques to share data between the producer and consumer applicationsBackward CompatibilityForward CompatibilityFull CompatibilityNone CompatibilitySchema Naming StrategiesI will cover the different naming strategies for Schema and how its impacts the application events.TopicName StrategyRecordName StrategyTopicRecordName StrategyBuild a Coffee Order Service using SpringBoot & Schema RegistryIn this section, we will code and build a Spring Boot Kafka application that exchanges the data in an AVRO format and interacts with Schema Registry for data evolution.Build a RestFul service to publish the events through which we receive events through the rest interface and then publish them to KafkaBy the end of this course, you will have a complete understanding of these concepts:Use AVRO as a data serialization formatEvolution of the data using Schema Registry
Overview
Section 1: Getting Started with the Course
Lecture 1 Introduction
Lecture 2 Pre-requestites
Section 2: Data Contract and Serialization in Kafka
Lecture 3 Data Contract & Serialization in Kafka
Lecture 4 Serialization Formats
Section 3: Introduction to AVRO - A data serialization system
Lecture 5 Introduction to AVRO - What is AVRO and Why AVRO?
Lecture 6 Build a simple AVRO Schema
Section 4: Kafka Setup & Demo in Local Using Docker
Lecture 7 Set up Kafka Broker and Zookeeper using Docker Compose
Lecture 8 Producer and Consumer Messages using CLI
Lecture 9 Produce and Consume using AVRO Console Producer & Consumer
Section 5: Greeting App - Base AVRO Project SetUp - Gradle
Lecture 10 Base Project SetUp for Greeting App
Lecture 11 Generate AVRO Java Records using AVRO Schema Files
Section 6: Greeting App - Base AVRO Project SetUp - Maven
Lecture 12 Base Project SetUp for Greeting App - Maven
Lecture 13 Generate AVRO Java Records using AVRO Schema Files - Maven
Section 7: Build AVRO Producer and Consumer in Java
Lecture 14 Lets build AVRO Kafka Producer
Lecture 15 Lets build AVRO Kafka Consumer
Section 8: CoffeeShop Order Service Using AVRO - A Real time Use Case
Lecture 16 Application Overview
Lecture 17 Project SetUp for CoffeeShop - Gradle
Lecture 18 Project SetUp for CoffeeShop - Maven
Lecture 19 Build a Coffee Order Schema using AVRO
Lecture 20 Generating AVRO classes using Gradle
Lecture 21 Generating AVRO classes using Maven
Lecture 22 Build a CoffeeShop Order Producer
Lecture 23 Build a CoffeeShop Order Consumer
Section 9: Logical Schema Types in AVRO
Lecture 24 Introduction to Logical Types in AVRO
Lecture 25 Add a timestamp, decimal logical type to the CoffeeOrder Schema
Lecture 26 Adding the UUID as Key for CoffeeOrder
Lecture 27 Date Logical Type
Section 10: AVRO Record- Under the Hood
Lecture 28 Whats inside an AVRO Record ?
Section 11: Schema Changes in AVRO - Issues without Schema Registry
Lecture 29 Evolving the Schema - Consumer Fails to Read the new Schema
Section 12: Introduction to Schema Registry
Lecture 30 Introduction to Schema Registry
Lecture 31 Publish and Consumer Record using Schema Registry
Lecture 32 Schema Registry internals & Interacting with Schema Registry using REST Endpoint
Lecture 33 Publish and Consume "Key" as an AVRO record
Section 13: Data Evolution using Schema Registry
Lecture 34 Data Evolution & Schema Evolution
Lecture 35 Update the code to interact with mavenLocal Repository - Gradle
Lecture 36 Update the code to interact with mavenLocal Repository - Maven
Lecture 37 Deleting a Field in Schema - BACKWARD Compatibility
Lecture 38 Adding a new Field in Schema - FORWARD Compatibility
Lecture 39 Add/Delete Optional Fields - FULL Compatibility
Lecture 40 Modify Field Names - NONE Compatibility
Section 14: Schema Naming Strategies
Lecture 41 Different Types of Naming Strategies
Lecture 42 Coffee Update Event AVRO Schema
Lecture 43 Publish and Consume CoffeeOrder UpdateEvent using RecordNameStrategy
Section 15: Build a Coffee Order Service using SpringBoot & Schema Registry
Lecture 44 Overview of the app
Lecture 45 Setting up the base project - Gradle
Lecture 46 Setting up the base project - maven
Lecture 47 Build the DTOs for the CoffeeOrderService
Lecture 48 Build the POST endpoint for the CoffeeOrderService - /coffee_orders
Lecture 49 Build the Service layer to map the DTO to AVRO domain object
Lecture 50 Configure the Kafka Producer properties in Coffee Order Service
Lecture 51 Build KafkaProducer to publish the CoffeeOrder Events
Lecture 52 Build the Coffee Order Consumer
Lecture 53 Build the PUT endpoint for the CoffeeOrderService - PUT /coffee_orders/{id}
Experienced Java Developers,Developers interested in learning AVRO and how to exchange data between applications using AVRO and Kafka,Developers who are interested in learning about Schema Registry and how it fits in to the Kafka,Developers who are interested in learning techniques to evolve the data


Homepage

https://www.udemy.com/course/kafka-for-developers-data-contracts-using-schema-registry/