Event Streaming

Saurabh Sharma

While reading Apache Kafka it was interesting analogy that they described

Event streaming is the digital equivalent of the human body’s central nervous system.

Apache Kafka

It draws a great parallel between how the central nervous system allows for real time event generations and consumption in terms of reflex actions. The examples are well explained and captured here.

The biggest advantage of using such a platform is that besides the obvious

  • Distributed
  • Secure
  • Highly scalable & fault-tolerant

It can be deployed on bare-metal hardware and containers.

Terminologies

The basic terms that you might often hear when talking about Kafka are as under.

Server and Clients

Since it is distributed system it interacts over TCP via server and client system. It uses a binary protocol over TCP. (More details here)

Kafka cluster storage layer is termed as Broker.

Events

These are essentially the information (a record) that is produced and read in the Kafka universe. They generally have a Key, Value and Timestamp. Events are organized and stored in Topics.

In kafka one can define for how long should the events should be retained through a per-topic configuration setting, after which old events will be discarded. 

Topics are partitioned, meaning a topic is spread over a number of “buckets” located on different Kafka brokers. 

Producers

Applications that generates and publishes the information.

Consumers

Applications that consume the information available in Kafka universe

Leave a Reply

Your email address will not be published.