Heroku Kafka

Kafka provides a powerful set of primitives for connecting your distributed application: messages, topics, partitions, producers, consumers, and log compaction.

Messages

Kafka is a message passing system, messages are events and can have keys.

Brokers

A Kafka cluster is made up of brokers that run Kafka processes.

Topics

Topics are streams of messages of a particular category.

Partitions

Partitions are append only, ordered logs of a topic’s messages. Messages have offsets denoting position in the partition. Kafka replicates partitions across the cluster for fault tolerance and message durability.

Producers

Producers are client processes that send messages to a broker on a topic and partition. Producers can use a partitioning function on keys to control message distribution.

Consumers

Consumers read messages from topics’ partitions on brokers, tracking the last offset read to coordinate and recover from failures. Consumers can be deployed in groups for scalability.

Log Compaction

Log compaction keeps the most recent value for every key so clients can restore state.


Original URL: http://feedproxy.google.com/~r/feedsapi/BwPx/~3/MKwCRYoYZ9g/kafka

Original article

Comments are closed.

Proudly powered by WordPress | Theme: Baskerville 2 by Anders Noren.

Up ↑

%d bloggers like this: