AI Course
    NEW
    Placements

    Progress

    0%

    What is Software Design?High-Level vs Low-Level Design — Drawing the BoundarySoftware Architecture vs Software DesignFunctional vs Non-Functional RequirementsThe Software Design Process and Design ThinkingCharacteristics of Good Software DesignCommon Software Design MistakesHow to Read and Write Design DocumentsThe Framework: How to Approach LLD Problems

    Introduction to OOP and the Java Memory ModelClasses, Objects, and the this KeywordConstructors and Object InitializationEncapsulation and Access ModifiersAbstraction : Interfaces vs Abstract ClassesInheritance and Its TypesPolymorphism : Compile-Time vs RuntimeAssociation, Aggregation, and CompositionDependency RelationshipsGenerics and Type SafetyThe Golden Rule : Composition Over Inheritance

    Introduction to Design PrinciplesSingle Responsibility PrincipleOpen-Closed PrincipleLiskov Substitution PrincipleInterface Segregation PrincipleDependency Inversion PrincipleDRY, KISS, and YAGNI PrinciplesHigh Cohesion and Loose CouplingLaw of DemeterSeparation of Concerns and Information HidingDesigning for Testability : The Ultimate Proof of Loose Coupling

    Why UML Matters in LLDClass DiagramsSequence DiagramsActivity DiagramsState DiagramsComponent and Object DiagramsUsing UML Effectively in InterviewsCommon UML Mistakes to Avoid

    What Are Design Patterns?History and the Gang of FourCategories of Design PatternsHow to Choose a Design PatternAnti-Patterns and Common MisuseDesign Patterns Interview Strategy

    Creational Patterns OverviewSingleton Pattern: Including Thread-Safe and Bill Pugh ImplementationsFactory Method PatternAbstract Factory PatternBuilder and Fluent Builder PatternPrototype PatternFactory vs Builder : When to Use WhichReal-World Usage of Creational Patterns

    Structural Patterns OverviewAdapter PatternBridge PatternComposite PatternDecorator PatternFacade PatternProxy PatternFlyweight PatternAdapter vs Facade vs ProxyReal-World Usage of Structural Design Patterns — How Spring Security Uses Filters and Proxies

    Behavioral Patterns OverviewStrategy PatternObserver PatternCommand PatternChain of ResponsibilityTemplate Method PatternState PatternMediator PatternIterator PatternMemento PatternVisitor PatternStrategy vs State — Key DifferencesReal-World Usage of Behavioral Patterns

    Introduction to Domain ModelingIdentifying Entities and Value ObjectsAggregates and Aggregate RootsDomain ServicesRepositories in Domain DesignBusiness Rules ModelingDomain EventsDesigning Rich vs Anemic Domain ModelsPutting It Together — Full Domain Model Walkthrough

    Introduction to API Design — Principles and GoalsREST Principles and Resource ModelingDTO Design and Mapping Strategies — MapStruct and JacksonRequest and Response DesignInput Validation DesignError Handling and Standardized Error ResponsesPagination, Filtering and SortingAPI Versioning StrategiesIdempotency in APIsAPI Security BasicsDesigning Internal vs External APIs

    Introduction to Persistence DesignThe Repository Pattern — Spring Data JPA Under the HoodDAO vs Repository PatternTransactions and ACID PropertiesUnit of Work PatternOptimistic vs Pessimistic LockingLazy Loading vs Eager Loading and the N+1 ProblemCaching FundamentalsWrite-Through vs Write-Back vs Cache-AsidePersistence Design Best PracticesDesigning for Data Consistency in Concurrent Systems

    Introduction to Concurrency — Threads vs ProcessesThread LifecycleThread Class vs Runnable vs CallableThe Executor Framework and Thread PoolsSynchronization and LocksReentrantLock and ReadWriteLockSemaphores and Countdown LatchesConcurrent Collections — ConcurrentHashMap and FriendsFuture and CompletableFuture for Async DesignThread Communication — Wait, Notify, NotifyAllCommon Concurrency Problems — Deadlock, Starvation, Race ConditionsDesigning Thread-Safe ClassesConcurrency Design Best Practices

    Introduction to Event-Driven ArchitectureThe Publisher-Subscriber ModelObserver Pattern vs Pub-Sub — Where They DifferEvent Modeling — Designing Good EventsMessage Queues — Concepts and Internal DesignEventual Consistency and What It Means for Object DesignEvent Sourcing — Concepts and When to Use ItCQRS — Command Query Responsibility SegregationCommon Pitfalls in Event-Driven Systems

    Logging Fundamentals and Best PracticesAudit LoggingMonitoring and Metrics DesignHealth ChecksRetry Mechanisms and Backoff StrategiesCircuit Breaker PatternRate Limiting AlgorithmsFeature FlagsDesigning for Observability — Putting It All Together

    How to Approach Any LLD Case Study — The 45-Minute FrameworkDesign a Parking LotDesign an Elevator SystemDesign a Vending MachineDesign an ATM MachineDesign a Chess GameDesign Tic-Tac-ToeDesign Snake and LadderDesign a Library Management SystemDesign a Logging FrameworkDesign an Inventory Management SystemDesign a Car Rental SystemDesign a Hotel Booking SystemDesign a URL ShortenerDesign Splitwise

    Advanced LLD — What Changes at the Senior LevelDesign a Movie Ticket Booking System — Handling Concurrency and LockingDesign an E-Commerce Shopping Cart and Order FlowDesign a Notification System — Internal Class OrchestrationDesign a Rate LimiterDesign a Job SchedulerDesign a Real-Time Chat System — WebSocket Handlers and Message Queues at the Class LevelDesign a Pub-Sub System Like Kafka — Internal Data Structures of the BrokerDesign a Payment Processing SystemRequirement Gathering Techniques for InterviewsIdentifying Entities and Relationships Under PressureChoosing Design Patterns in an InterviewDiscussing Trade-Offs ConfidentlyCommon LLD Interview MistakesMock LLD Interview Walkthrough — End to EndWhat to Do After the Interview — Learning from Every Session

    Message Queues: Concepts and Internal Structure

    Learning Objectives#

    • Name the pieces of a log-based queue: partition, offset, and consumer group.
    • Explain the trade a partitioned log makes between per-key ordering and parallelism.
    • State what the offset commit decides and what a consumer owes for at-least-once.

    Introduction#

    A topic needs a home, and the home is a log. This article is the internal layer of the broker, the parts you inherit the moment you pick Kafka or RabbitMQ. The honest framing is a story about a partitioned, keyed log: a fixed number of ordered append logs, an offset that says how far a consumer has read, and a consumer group that hands out parallel slices. Read a queue as a log and the forward-looking pieces of an event-driven system stop being dark magic.

    Problem Statement#

    A catalog service writes hundreds of thousands of events per second, and a consumer draining a single ordered list is far too slow. You want parallel readers, but two parallel readers must not both consume the same order's events out of sequence. Then a broker node dies and a consumer has to know where its reading stands. Ordering, parallelism, and replay are three demands pulling in different directions, and without a clear model the queue is where they collide.

    Core Concept#

    A message queue is not one list. It is a fixed number of ordered, append-only logs called partitions. An event goes to whichever partition its key hashes to, and inside a partition the events are ordered by arrival. Every event gains an offset, its position in that partition. The ordering guarantee is deliberately narrow: it is per partition and per key, never across the whole topic.

    ProducerRecord<String, OrderPlaced> record =
    new ProducerRecord<>("orders", order.getId(), event);
producer.send(record);  // the key picks the partition

    The consumer group is how parallelism is reconciled with order. A group is a set of members that together read the topic. Each partition is assigned to one member at a time, so all the events for one key arrive in one partition and are drained in order by one member. The group sets the ceiling on parallelism: with three partitions you can run three busy members, and a fourth consumer adds nothing.

    consumer.subscribe("orders"); // the group shares the offsets for the topic

    The last piece is the commit. A consumer reads up to offset 5, does its work, then commits that it reached offset 6. If the consumer dies, the group restarts from the last committed offset. Where the commit sits decides the delivery semantics:

    • At-least-once: do the work, then commit. A crash before the commit replays the event, so duplicates can happen and the consumer must be idempotent.
    • At-most-once: commit before the work. A crash can skip an event, no duplicates, best for a miss you can absorb.
    • Exactly-once: runs on transactions and deduplication, it is a hard feature of the product, never the default of a plain queue.

    Diagram: a topic splits into three partitions, and each worker in the consumer group reads exactly one.

    Topic: orders Partition 0 (key) Partition 1 (key) Partition 2 (key) Consumer group Member A Member B Member C offset 0..N committed after the work

    The commit box sits at the bottom on purpose. The offset is the cursor, and the commit is the decision about where the cursor is published. The important companion fact is that the log retains the data for a configured window, so replay is a normal consumer action, not a reconstruction of something lost.

    Real Production Usage#

    Kafka is the log, and the group is an event-handling service. The offset is persisted by the consumer, so the work resumes on a rebalance from the last committed offset, and the commit is placed after the side-effect write, wrapped in a transaction or made idempotent. spring-kafka hides the listener, the group, and the ack behind a template, but the model on this page is the one underneath. The rule of thumb I keep: make the consumer idempotent and treat every redelivery as a no-op, because in the long run you will be redelivered.

    Common Mistakes#

    1. Adding a consumer past the partition count. Extra threads sit idle, because the throughput is set by the partition count and the group size, not by the number of workers you threw at it.
    2. Committing the offset before the side-effect. A crash skips the event and the reply never fires; choose where the commit sits and prove it.
    3. Presuming order across the topic. Two partitions, two members, and the events for two orders can be applied in a different order than they were written. Order on the key, not on the stream.

    Interview Perspective#

    The interviewer wants the tension: how a topic is both parallel and ordered. Weak: "a message is consumed by one reader." Strong: "a topic splits into partitions; order is per partition, the consumer group assigns each partition to one member, and the offset carries the state, so each key is one serial member while several keys run in parallel." Follow-up: "what does a rebalance do" and "how does at-least-once duplicate." Both hinge on the same commit: a rebalance hands a partition away and the new member resumes from the last committed offset, so a crash between the work and the commit replays the event.

    Knowledge Check#

    • A consumer applies an event, updates its read model, then commits. If it crashes between the update and the commit, what is replayed and what delivery mode is that?
    • You need all events for one order consumed in order while the topic stays fast overall. Which of the partition, the producer key, the group size, and the commit make that true?
    • A topic has four partitions and six consumers in a group. What is the ceiling on busy consumers, and what are the two idle ones doing?

    Key Takeaways#

    • A topic is partitioned into order per partition and per key, never per whole topic.
    • The consumer group is the only place parallelism meets order, one member per partition.
    • The commit position is the policy, at-least-once or at-most-once, and the consumer is idempotent for the predictable.
    • The whole log is the record; replay is reading a history, not a special case.

    What's Next#

    The queue creates the lag between the publish and the consumer's view, and the next article is about living with it. Eventual consistency and what it means for object design speaks to the fact that a consumer reads a stale view and rebuilds it, which the offset commit and the replay above make the daily reality.

    Want to Master Spring Boot and Land Your Dream Job?

    Struggling with coding interviews? Learn Data Structures & Algorithms (DSA) with our expert-led course. Build strong problem-solving skills, write optimized code, and crack top tech interviews with ease

    Learn more

    Last updated on Aug 19, 2026

    Was it helpful?

    Subscribe to our newsletter

    Read articles from Coding Shuttle directly inside your inbox. Subscribe to the newsletter, and don't miss out.

    Previous
    Event Modeling — Designing Good Events
    Next
    Eventual Consistency and What It Means for Object Design