Apache Kafka Topic

➡️ Kafka Topic is a core component of the 🐦 Kafka ecosystem, responsible for organizing and storing data streams within the system.

Notes

Apache Kafka Team

A topic in Kafka acts as a category or feed name to which records are stored and published.

A topic is essentially a channel to which data producers write messages, and from which data consumers read those messages. Each record within a topic is assigned to a specific partition, enabling parallel processing and enhancing throughput.

flowchart LR
    Producer --> |writes records to| Broker["fa:fa-twitter Broker"]
    subgraph "Broker"
        direction LR
        TopicA -->|contains| PA1(PartitionA1)
        TopicA -->|contains| PA2(PartitionA2)
        TopicB -->|contains| PB1(PartitionB1)
        TopicB -->|contains| PB2(PartitionB2)
        style TopicA fill:#887766,stroke:#333,stroke-width:2px;
        style TopicB fill:#998877,stroke:#333,stroke-width:2px;
    end
    Broker --> |allows reading from| Consumer

TakeAways

• 📌 Kafka Topics allow for organized data streams
• 💡 Each topic is divided into Partitions, enabling parallel processing.

• 🔍 Multiple consumers can read from the same topic, but each consumer reads from different partitions by default.

Process

• 📄 Data Production : Producers write records to specific topics within a Kafka Broker.
• 🗄️ Topic Management : The Broker manages topics and their respective partitions.
• 🔒 Durability : Topics ensure data durability by storing records in commit logs.
• 🖥 Load Balancing : Topics help balance the load by distributing data across multiple partitions.

Thoughts

• HighAvailability: Topics contribute to data organization, ensuring no single point of failure.
• Scalability: By managing multiple topics and partitions, Kafka supports ↔️ Horizontal scaling and increased throughput.
• FaultTolerance: Data is replicated across partitions within topics, ensuring that messages are not lost in case of failures.

References

2. 🐦 Kafka
3. Apache Kafka Topic Official Documentation